Add autobuyer page capture, browser extension, and desktop clicker

Builds the pipeline the autobuyer needs: see the page, find an element,
click it.

extension/ — MV3 Chromium extension. Polls /api/autobuyer/status and,
while on, scrapes the target tab's HTML and posts it back. Also serves
locate requests: focuses the window, scrolls the element into view, and
reports its position. host_permissions is scoped to tradeify plus
localhost so it cannot read other sites — an empty target pattern would
otherwise capture whatever tab happened to be active, including banking
or mail.

app/api/autobuyer/ — status toggle, capture store, and the locate request
queue. CORS is open because the extension's origin changes every time an
unpacked extension is reloaded.

app/autobuyer/page.tsx — ON switch, source view (default) and a rendered
view. The render uses sandbox="allow-scripts" without allow-same-origin:
the page's own JS is needed because sites ship content at opacity:0 and
fade it in, but the frame must not reach the dashboard's same-origin API
routes, which serve firm credentials.

clicker/ — Python CLI. Asks the extension where a selector is, adds the
element rect to the window's screen position and the browser chrome
height to get desktop coordinates, then clicks with a human motion model
(curved path, eased velocity, occasional overshoot, dwell before press).
Raises the browser application first, since macOS consumes a click on an
unfocused window rather than delivering it.

Refuses to click when the element is covered by an overlay, when the
coordinates fall off-screen, or when the browser cannot be confirmed
frontmost.

Verified: API round-trips, capture pruning, locate claim-once semantics,
motion geometry and timing, and focus activation — the last two against
stubs, since pyautogui and pyobjc are not installed here. NOT verified
end to end: Chrome is still running a stale build of the extension, so a
locate request has never completed against a real page and no real click
has been sent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Brandon Li
2026-08-27 16:15:18 -05:00
co-authored by Claude Opus 5
parent 24a54c6642
commit 54221bbc0c
19 changed files with 1752 additions and 2 deletions
+53
View File
@@ -0,0 +1,53 @@
# AutoFirmer Capture (Chromium extension)
Polls the dashboard for an on/off flag and, while it's on, scrapes the target
tab's HTML and posts it back to the dashboard.
## Install (unpacked)
1. Start the dashboard (`npm run dev`) so `http://localhost:3000` is up.
2. Open `chrome://extensions`, enable **Developer mode** (top right).
3. **Load unpacked** → select this `extension/` folder.
4. Click the extension icon to open the popup and set:
- **Dashboard URL** — default `http://localhost:3000`. Use the VPS IP if the
dashboard runs elsewhere.
- **Poll interval** — seconds between status checks (default 3).
- **Target URL pattern** — a Chrome match pattern like
`https://*.tradovate.com/*`. Leave blank to capture whichever tab is active.
- **Element selector** — optional CSS selector; matching elements get their
on-screen position measured alongside the HTML.
The badge shows `ON` (green) while capturing, `!` (red) if the dashboard is
unreachable, and nothing when the switch is off.
## Flow
```
AutoBuyer page --PATCH /api/autobuyer/status--> SQLite settings
extension --GET /api/autobuyer/status--> { enabled }
extension --POST /api/autobuyer/capture-> html + viewport + element rects
AutoBuyer page --GET /api/autobuyer/capture-> renders the HTML
```
## Scope
`host_permissions` is deliberately narrow — `https://*.tradeify.co/*` plus localhost
for the dashboard. The extension is technically incapable of reading any other site,
so an accidental capture of your bank or mail tab can't happen. The default
**Target URL pattern** matches, so it only ever captures the broker tab regardless
of which tab is focused.
To automate a different broker, add its pattern to `host_permissions` in
`manifest.json`, update the popup's target pattern, and reload the extension. Avoid
going back to `<all_urls>` — that re-enables scraping whatever tab is active.
## Notes
- The extension never captures the dashboard's own pages — otherwise it would
just mirror its own output back.
- `chrome://`, `about:` and Web Store pages cannot be scripted by any extension;
they're skipped.
- MV3 service workers are torn down when idle. Each poll makes an extension API
call, which keeps the worker alive; a 30-second alarm revives it if Chrome
kills it anyway. So worst-case cadence is 30s, normal cadence is the poll
interval.