Solve captchas through the 2captcha API, not its extension

The two Tradeify automations clicked the 2captcha *extension's* solve button
and waited for it to go green. That needed a second extension installed and
signed in, put a real mouse click on a button whose markup is 2captcha's to
change, and gave a run no way to tell "still solving" from "not installed".

Replaced with one `solveCaptcha` step. No mouse: the extension reads the
widget's sitekey out of the page, the dashboard buys a token from 2captcha,
and the extension puts it in the response field and fires the page's own
callback.

captcha-hook.js is the part that earns its keep. A widget rendered
programmatically puts nothing in the DOM — no sitekey, and no callback. Most
pages only enable submit from that callback, so filling the hidden field alone
leaves the page looking unsolved. The hook wraps turnstile/grecaptcha/hcaptcha
`render` at document_start to hold on to what the page passed.

Solves go through 2captcha's residential proxy. Its API rejects host names in
proxyAddress, so na.proxy.2captcha.com is resolved to an IP first (cached);
the gateway was confirmed to work addressed that way.

Covers reCAPTCHA v2 (incl. invisible and Enterprise), hCaptcha and Turnstile.
NOT reCAPTCHA v3: a v3 page carries an anchor iframe with k=, so it is
detected as v2 and would be solved with the wrong task type. Everything else
unsupported fails loudly instead.

The API key and proxy credentials are hardcoded in the captcha route on
request. Unlike the sqlite database, that file is in the repo, so they travel
with it. TWOCAPTCHA_API_KEY / TWOCAPTCHA_PROXY and the settings rows still
override; neither has a field on the Settings page.

Runner VERSION and RUNNER_EXPECTED_VERSION both to 0.17.0 — they are a pair,
and a mismatch shows as a red "restart your runner" banner that blames the
wrong process.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

extension/README.md

@@ -32,7 +32,7 @@ captures the tab you are on.
 
 ## Scope
 
-Capture has no switch, so scope is what keeps it honest. Two limits, both
+Capture has no switch, so scope is what keeps it honest. Three limits, all
 enforced by Chrome rather than by this code:
 
 1. **`host_permissions` is deliberately narrow** — the firm sites plus localhost
@@ -43,11 +43,20 @@ enforced by Chrome rather than by this code:
    last-focused window *and* a host_permissions match. A firm tab sitting in the
    background is not captured, and neither is anything else. Switch away and
    capture simply stops until you switch back.
+3. **One content script, same hosts.** `captcha-hook.js` runs at document_start
+   on the firm sites only. It wraps `turnstile.render` / `grecaptcha.render` /
+   `hcaptcha.render` and remembers what the page passed them — the sitekey, and
+   above all the callback a solved token has to be handed to, which a
+   programmatically rendered widget puts nowhere else. It reads nothing else and
+   sends nothing anywhere; the remembered arguments stay in the page's own world
+   until a `solveCaptcha` step comes to collect them.
 
 To automate a different broker, add its pattern to `host_permissions` in
 `manifest.json` and reload the extension — that alone scopes capture to it. Avoid
 going back to `<all_urls>`: it would put every tab you visit in reach, and limit
-1 is the only thing standing between this and scraping your whole browser.
+1 is the only thing standing between this and scraping your whole browser. A new
+firm needs its host in *both* lists — `host_permissions` and the `content_scripts`
+matches — or captchas on that site will have no callback to fire.
 
 ## Notes
 

extension/background.js

@@ -342,6 +342,187 @@ async function pageScrollToLoad(selector, containerSelector, maxScrolls, settleM
     };
 }
 
+// ── Captcha: read the widget, then hand a solved token back to the page ─────
+
+/** Runs in the page's MAIN world. Describes the captcha widget well enough for
+ *  2captcha to solve it, or reports that there isn't one yet.
+ *
+ *  Three sources, best first: the render hook (the only one that sees
+ *  programmatically-passed parameters), the widget's own data- attributes, and
+ *  finally the challenge iframe, whose URL carries the sitekey when nothing else
+ *  does. */
+function pageCaptchaInfo(scopeSelector, wanted) {
+    const root = scopeSelector ? document.querySelector(scopeSelector) : document;
+    if (!root) return { error: `No element matches ${scopeSelector}` };
+
+    const KINDS = {
+        turnstile: { cls: '.cf-turnstile', frame: 'challenges.cloudflare.com' },
+        recaptcha: { cls: '.g-recaptcha', frame: '/recaptcha/' },
+        hcaptcha: { cls: '.h-captcha', frame: 'hcaptcha.com' },
+    };
+    const want = (kind) => !wanted || wanted === 'auto' || wanted === kind;
+    const enterprise = !!document.querySelector('script[src*="recaptcha/enterprise"]');
+
+    // Cloudflare's iframe has no sitekey parameter — it is a path segment, and
+    // every Turnstile key starts with 0x.
+    const fromFrameUrl = (kind, src) => {
+        try {
+            const url = new URL(src, location.href);
+            if (kind === 'recaptcha') return url.searchParams.get('k');
+            if (kind === 'hcaptcha') return url.searchParams.get('sitekey');
+            return url.pathname.split('/').find((part) => /^0x[A-Za-z0-9_-]{8,}$/.test(part)) || null;
+        } catch {
+            return null;
+        }
+    };
+
+    for (const widget of (window.__autofirmerCaptcha?.widgets || [])) {
+        const params = widget.params || {};
+        if (!want(widget.kind) || !params.sitekey) continue;
+        return {
+            provider: widget.kind,
+            sitekey: params.sitekey,
+            pageurl: location.href,
+            action: params.action || '',
+            data: params.cData || '',
+            pagedata: params.chlPageData || '',
+            invisible: params.size === 'invisible',
+            enterprise,
+            userAgent: navigator.userAgent,
+            source: 'render hook',
+        };
+    }
+
+    for (const kind of Object.keys(KINDS)) {
+        if (!want(kind)) continue;
+        const el = root.querySelector(`${KINDS[kind].cls}[data-sitekey]`);
+        if (!el) continue;
+        return {
+            provider: kind,
+            sitekey: el.getAttribute('data-sitekey'),
+            pageurl: location.href,
+            action: el.getAttribute('data-action') || '',
+            data: el.getAttribute('data-cdata') || '',
+            pagedata: '',
+            invisible: el.getAttribute('data-size') === 'invisible',
+            enterprise,
+            userAgent: navigator.userAgent,
+            source: 'data-sitekey',
+        };
+    }
+
+    const frames = Array.from(root.querySelectorAll('iframe'));
+    for (const kind of Object.keys(KINDS)) {
+        if (!want(kind)) continue;
+        for (const frame of frames) {
+            const src = frame.getAttribute('src') || '';
+            if (!src.includes(KINDS[kind].frame)) continue;
+            const sitekey = fromFrameUrl(kind, src);
+            if (!sitekey) continue;
+            return {
+                provider: kind,
+                sitekey,
+                pageurl: location.href,
+                action: '',
+                data: '',
+                pagedata: '',
+                invisible: false,
+                enterprise,
+            userAgent: navigator.userAgent,
+                source: 'challenge iframe',
+            };
+        }
+    }
+
+    return { error: 'No captcha widget on the page yet' };
+}
+
+/** Runs in the page's MAIN world. Puts a solved token where the page looks for
+ *  it, then tells the page it arrived.
+ *
+ *  Both halves matter. The hidden response field is what the form posts, but
+ *  most pages enable their submit button from the widget's callback and never
+ *  read the field until then — so filling it alone leaves the page looking
+ *  unsolved. Only the FIRST callback found is fired: they are the page's own
+ *  submit handlers, and calling several would risk sending the form twice. */
+function pageCaptchaSubmit(token, provider) {
+    const FIELDS = {
+        turnstile: ['cf-turnstile-response', 'g-recaptcha-response'],
+        recaptcha: ['g-recaptcha-response'],
+        hcaptcha: ['h-captcha-response', 'g-recaptcha-response'],
+    };
+
+    const targets = new Set();
+    for (const name of (FIELDS[provider] || FIELDS.recaptcha)) {
+        document.querySelectorAll(`[name="${name}"], #${name}`).forEach((el) => targets.add(el));
+    }
+    // Turnstile names its hidden input per widget: cf-chl-widget-<id>_response.
+    document.querySelectorAll('input[name^="cf-chl-widget-"]').forEach((el) => targets.add(el));
+
+    let filled = 0;
+    for (const el of targets) {
+        el.value = token;
+        el.dispatchEvent(new Event('input', { bubbles: true }));
+        el.dispatchEvent(new Event('change', { bubbles: true }));
+        filled++;
+    }
+
+    let called = null;
+    const invoke = (fn, where) => {
+        if (called || typeof fn !== 'function') return false;
+        try {
+            fn(token);
+            called = where;
+            return true;
+        } catch (err) {
+            // The page's own handler threw. That is its business, but the token
+            // did reach it, so don't go looking for another one to call.
+            called = `${where} (threw: ${err.message})`;
+            return true;
+        }
+    };
+
+    for (const widget of (window.__autofirmerCaptcha?.widgets || [])) {
+        if (called) break;
+        if (provider && provider !== 'auto' && widget.kind !== provider) continue;
+        const cb = (widget.params || {}).callback;
+        if (typeof cb === 'function') invoke(cb, `${widget.kind} render callback`);
+        else if (typeof cb === 'string') invoke(window[cb], cb);
+    }
+
+    if (!called) {
+        for (const el of document.querySelectorAll('[data-callback]')) {
+            const name = el.getAttribute('data-callback');
+            if (invoke(window[name], `data-callback ${name}`)) break;
+        }
+    }
+
+    // reCAPTCHA rendered declaratively keeps its callback in a private config
+    // object and nowhere else. The keys around it are minified; `callback` is not.
+    if (!called && window.___grecaptcha_cfg?.clients) {
+        const seen = new Set();
+        const queue = Object.values(window.___grecaptcha_cfg.clients);
+        let budget = 2000;
+        while (queue.length && !called && budget-- > 0) {
+            const node = queue.shift();
+            if (!node || typeof node !== 'object' || seen.has(node)) continue;
+            seen.add(node);
+            for (const [key, value] of Object.entries(node)) {
+                if (typeof value === 'function') {
+                    if (/callback/i.test(key) && invoke(value, `grecaptcha ${key}`)) break;
+                } else if (value && typeof value === 'object') {
+                    queue.push(value);
+                }
+            }
+        }
+    }
+
+    if (!filled && !called) {
+        return { error: 'nowhere to put the token — no response field and no callback on the page' };
+    }
+    return { filled, called, provider, url: location.href };
+}
+
 async function resolveLocateTab(cfg, request) {
     // Every step carries its firm's pattern; the manifest hosts are the fallback
     // for a bare request (the CLI's locate without --url).
@@ -368,6 +549,15 @@ async function resolveLocateTab(cfg, request) {
     return { tab, opened: false };
 }
 
+/** Hand an answer — or a failure — back to whoever queued the request. */
+async function postLocateResult(cfg, id, result, error) {
+    await fetch(`${cfg.apiBase}/api/autobuyer/locate/result`, {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({ id, result, error }),
+    });
+}
+
 /** Returns true if a request was handled, false if the queue was empty. */
 async function serveLocateRequest(cfg) {
     const claim = await fetch(`${cfg.apiBase}/api/autobuyer/locate/claim`, { method: 'POST' });
@@ -421,11 +611,36 @@ async function serveLocateRequest(cfg) {
             if (out.error) throw new Error(out.error);
             result = { ...out, tabId: tab.id, windowId: tab.windowId, browser: detectBrowser() };
 
-            await fetch(`${cfg.apiBase}/api/autobuyer/locate/result`, {
-                method: 'POST',
-                headers: { 'Content-Type': 'application/json' },
-                body: JSON.stringify({ id: request.id, result, error: null }),
+            await postLocateResult(cfg, request.id, result, null);
+            return true;
+        }
+
+        // Captcha ops are the same shape: no element is measured, the page is
+        // read for its widget parameters or handed a token 2captcha solved.
+        // Both run in the page's MAIN world — the isolated world cannot see the
+        // page's own `turnstile` / `grecaptcha` objects, nor the render hook's
+        // registry, and those hold the callbacks a token has to be given to.
+        // allFrames because the widget is often in a same-origin sub-frame;
+        // Chrome skips frames outside host_permissions, so the challenge iframe
+        // itself is never touched.
+        if (request.options && (request.options.op === 'captchaInfo' || request.options.op === 'captchaToken')) {
+            const isInfo = request.options.op === 'captchaInfo';
+            const injections = await chrome.scripting.executeScript({
+                target: { tabId: tab.id, allFrames: true },
+                world: 'MAIN',
+                func: isInfo ? pageCaptchaInfo : pageCaptchaSubmit,
+                args: isInfo
+                    ? [request.options.scopeSelector || '', request.options.provider || 'auto']
+                    : [String(request.options.token || ''), request.options.provider || 'auto'],
             });
+            const answers = injections.map((i) => i?.result).filter(Boolean);
+            const found = answers.find((a) => !a.error);
+            if (!found) {
+                throw new Error(answers[0]?.error || 'Captcha injection returned nothing');
+            }
+            result = { ...found, tabId: tab.id, windowId: tab.windowId, browser: detectBrowser() };
+
+            await postLocateResult(cfg, request.id, result, null);
             return true;
         }
 
@@ -457,11 +672,7 @@ async function serveLocateRequest(cfg) {
         error = err.message;
     }
 
-    await fetch(`${cfg.apiBase}/api/autobuyer/locate/result`, {
-        method: 'POST',
-        headers: { 'Content-Type': 'application/json' },
-        body: JSON.stringify({ id: request.id, result, error }),
-    });
+    await postLocateResult(cfg, request.id, result, error);
     return true;
 }
 

extension/captcha-hook.js

@@ -0,0 +1,69 @@
+/* AutoFirmer — captcha render hook.
+ *
+ * Runs in the page's MAIN world at document_start, before any captcha script
+ * loads. It wraps `turnstile.render` / `grecaptcha.render` / `hcaptcha.render`
+ * and records what the page passed to them on `window.__autofirmerCaptcha`.
+ *
+ * Why bother, when the sitekey is usually a `data-sitekey` attribute: a widget
+ * rendered programmatically puts NOTHING in the DOM — no sitekey, and more
+ * importantly no callback. The callback is the whole point. Filling the hidden
+ * response field with a solved token does not by itself tell the page a token
+ * arrived; the page only reacts when its own callback runs with it. Turnstile in
+ * particular hands that callback straight to render() and never exposes it
+ * anywhere else, so a hook installed before the widget renders is the only way
+ * to hold on to it.
+ *
+ * Reads nothing and sends nothing. It only remembers arguments, in the page's
+ * own world, for background.js to use when a token comes back from 2captcha.
+ */
+
+(() => {
+    const registry = (window.__autofirmerCaptcha = window.__autofirmerCaptcha || { widgets: [] });
+
+    /** Wrap `render` on a captcha API object, once. */
+    function wrap(kind, api) {
+        if (!api || typeof api !== 'object' || api.__autofirmerWrapped) return api;
+
+        if (typeof api.render === 'function') {
+            const original = api.render.bind(api);
+            api.render = function (container, params) {
+                const widgetId = original(container, params);
+                // `params` holds the sitekey, the callback, and for Turnstile the
+                // action / cData / chlPageData that a solve has to be asked for.
+                registry.widgets.push({ kind, params: params || {}, widgetId });
+                return widgetId;
+            };
+            try {
+                Object.defineProperty(api, '__autofirmerWrapped', { value: true });
+            } catch { /* frozen API object — wrapping still took */ }
+        }
+
+        // reCAPTCHA Enterprise is a second render() hanging off the same object.
+        if (api.enterprise) wrap(kind, api.enterprise);
+        return api;
+    }
+
+    /** Wrap whatever the page eventually assigns to `window[name]`.
+     *
+     *  The script that defines it loads after this one, so there is nothing to
+     *  wrap yet — the accessor waits for the assignment instead. It also catches
+     *  the reCAPTCHA pattern of replacing a `{ready}` placeholder later on. */
+    function intercept(name, kind) {
+        let current = wrap(kind, window[name]);
+        try {
+            Object.defineProperty(window, name, {
+                configurable: true,
+                enumerable: true,
+                get: () => current,
+                set: (value) => { current = wrap(kind, value); },
+            });
+        } catch {
+            // The page made the property non-configurable. Detection falls back
+            // to the DOM, which covers every declaratively rendered widget.
+        }
+    }
+
+    intercept('turnstile', 'turnstile');
+    intercept('grecaptcha', 'recaptcha');
+    intercept('hcaptcha', 'hcaptcha');
+})();

extension/manifest.json

@@ -1,7 +1,7 @@
 {
     "manifest_version": 3,
     "name": "AutoFirmer Capture",
-    "version": "0.10.0",
+    "version": "0.11.0",
     "description": "Scrapes the HTML of the firm tab you are viewing and posts it to the AutoFirmer dashboard.",
     "permissions": ["scripting", "tabs", "storage", "alarms"],
     "host_permissions": [
@@ -13,6 +13,18 @@
     "background": {
         "service_worker": "background.js"
     },
+    "content_scripts": [
+        {
+            "matches": [
+                "https://*.tradeify.co/*",
+                "https://*.lucidtrading.com/*"
+            ],
+            "js": ["captcha-hook.js"],
+            "run_at": "document_start",
+            "all_frames": true,
+            "world": "MAIN"
+        }
+    ],
     "action": {
         "default_popup": "popup.html",
         "default_title": "AutoFirmer Capture"