Brandon LiandClaude Opus 5
b748f95372
Turns the autobuyer from a page scraper into something that acts. A dashboard button queues a run; a desktop process executes it against the real browser. lib/automations.ts — automations are declarative step lists nested inside the firm whose site they drive. Steps are click / type / wait / navigate, and they inherit the firm's tab pattern and URL, so one firm's automation can't act on another's tab. Adding a button means adding an entry here; the page renders buttons from the API and the runner receives steps from the server, so neither needs editing. Runs key on firm:automation — every firm will plausibly have its own "buy-accounts", and a bare id would resolve to the wrong one. clicker/runner.py — the daemon behind the buttons. Claims a queued run, works through the steps, reports each one back for the page's live log. Only one run executes at a time: two processes driving one physical mouse would interleave clicks. Heartbeats on its own thread, because a step can block for tens of seconds and folding the beat into the main loop would show the runner as offline in the middle of the run it was executing. clicker/actions.py — one implementation of the safety checks, shared by the CLI and the runner. Refuses to act when the element is covered by an overlay, when coordinates fall off-screen, when the browser can't be confirmed frontmost, or (for type) when the target isn't an editable field. Typing: uneven human cadence, and the field is read back afterwards and compared against what was typed — a field that never took focus fails silently and looks identical to success otherwise. Non-ASCII is rejected because pyautogui skips those characters without complaint, and newlines because Enter may submit the form. Typos are deliberately not simulated: a mistyped digit in a trading form is a real loss, and the correction is the part that can go wrong. Extension: opens the firm's page when no tab matches, navigates to a specific page for a navigate step (skipped when already there, so page state survives), and retries the locate while a freshly loaded React app mounts — `complete` only means the document loaded. Staleness reporting, after it cost three debugging rounds: Chrome doesn't reload an unpacked extension and Python doesn't reload a running process, so both now report their version. A stale runner gets a red banner naming both versions and the automation buttons are disabled, rather than failing mid-run on a step type it predates. Scale detection is now conservative: a raw OS/browser width ratio is only trusted when it lands on a real scaling factor. On this multi-monitor desktop the previous logic would have silently halved every coordinate. Verified end to end against the live browser: navigate, locate, and a real click (run #12, all three steps). API round-trips, claim-once semantics, run cancellation, the heartbeat online/offline lifecycle, motion geometry and timing, focus activation, and typing verification all pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
194 lines
9.0 KiB
Markdown
194 lines
9.0 KiB
Markdown
# Clicker
|
||
|
||
Moves the real desktop mouse and clicks an element you name with a CSS selector.
|
||
|
||
The extension can see the DOM but can't move the mouse; this script can move the
|
||
mouse but can't see inside Chrome. They meet at the dashboard API:
|
||
|
||
```
|
||
clicker --POST /api/autobuyer/locate------> "where is button.buy?"
|
||
extension --POST /api/autobuyer/locate/claim takes the request
|
||
extension raises the Chrome window, brings
|
||
the tab to the front, scrolls the
|
||
element to centre, measures it
|
||
extension --POST /api/autobuyer/locate/result desktop x,y
|
||
clicker --GET /api/autobuyer/locate?id=---> reads the answer
|
||
clicker moves the mouse, clicks
|
||
```
|
||
|
||
## Two ways to run it
|
||
|
||
**`clicker.py`** — one action at a time, from the shell. Use this to find
|
||
selectors and confirm coordinates before wiring anything up.
|
||
|
||
**`runner.py`** — the daemon behind the dashboard's buttons. Leave it running; it
|
||
polls for queued runs and executes the steps.
|
||
|
||
```bash
|
||
python runner.py # then press a button on the AutoBuyer page
|
||
python runner.py --dry-run # walks the steps, never presses or types
|
||
```
|
||
|
||
Automations are defined in `lib/automations.ts`, and the steps are sent to the
|
||
runner by the server — so adding a button means editing that file, and nothing in
|
||
the runner or the page changes. Each run reports step-by-step progress back to the
|
||
dashboard, and the page's Stop button halts a run between steps.
|
||
|
||
Only one run executes at a time: two processes driving the same physical mouse
|
||
would interleave clicks.
|
||
|
||
## Setup
|
||
|
||
```bash
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
On **macOS** you must grant Accessibility permission to whatever runs the script
|
||
(Terminal, iTerm, VS Code) — System Settings → Privacy & Security → Accessibility.
|
||
Without it `pyautogui` moves nothing and fails silently.
|
||
|
||
## Use
|
||
|
||
The AutoBuyer switch on the dashboard is the master arm: the script refuses to run
|
||
while it's off.
|
||
|
||
```bash
|
||
# Measure only — no mouse movement. Start here.
|
||
python clicker.py locate "button.buy"
|
||
|
||
# Move the cursor to the target but don't press.
|
||
python clicker.py click "button.buy" --dry-run
|
||
|
||
# Actually click.
|
||
python clicker.py click "button.buy"
|
||
|
||
# Type into a field: clicks it to place the caret, then types.
|
||
python clicker.py type 'input[name="quantity"]' --text "5000" --clear
|
||
|
||
# Keep the text out of shell history.
|
||
echo "5000" | python clicker.py type 'input#qty' --stdin --clear
|
||
|
||
# Pin it to a specific tab, and pick the 3rd match.
|
||
python clicker.py click ".trade-btn" --index 2 --url "https://tradeify.co/*"
|
||
```
|
||
|
||
| Flag | Meaning |
|
||
|---|---|
|
||
| `--url` | Chrome match pattern for the tab. Without it, the active tab is used. |
|
||
| `--index` | Which match, when the selector hits several (default 0). |
|
||
| `--api` | Dashboard URL (default `http://localhost:3000`). |
|
||
| `--timeout` | Seconds to wait for the extension (default 20). |
|
||
| `--scale` | CSS-to-desktop pixel ratio. Auto-detected; override if clicks land off. |
|
||
| `--dry-run` | Move the cursor, don't press. |
|
||
| `--force` | Click even when something is covering the element. |
|
||
| `--robotic` | Straight-line move and instant click, skipping the motion model. |
|
||
| `--seed` | Seed the motion RNG so a run replays identically (debugging). |
|
||
| `--no-activate` | Don't raise the browser first. The click may then be swallowed. |
|
||
| `--text` | Text to type (action `type`). |
|
||
| `--stdin` | Read the text from stdin instead, keeping it out of shell history. |
|
||
| `--clear` | Select-all and delete before typing, instead of appending at the caret. |
|
||
| `--allow-enter` | Permit newlines. Each one presses Enter, which may submit the form. |
|
||
| `--no-verify` | Skip reading the field back after typing. |
|
||
|
||
## Typing
|
||
|
||
`type` clicks the field to place the caret, then types with an uneven human
|
||
cadence (45–130ms between keys, a beat after each space, an occasional longer
|
||
pause). Real OS-level keystrokes are also the *correct* way to fill a React form:
|
||
setting `.value` directly is ignored by controlled components, while genuine key
|
||
events are not.
|
||
|
||
Four guards, all of which catch silent failures:
|
||
|
||
- **Non-ASCII is rejected.** `pyautogui.write()` has no keycode for `é` or `£` and
|
||
skips them without complaint, which would leave a quietly truncated value in the
|
||
field. Better to refuse than to submit `caf` where you meant `café`.
|
||
- **Newlines are rejected** unless `--allow-enter`, because Enter may submit the
|
||
form — and on this site that could mean placing an order.
|
||
- **Non-editable targets are refused** — disabled, read-only, or simply not an
|
||
input. The keystrokes would go nowhere.
|
||
- **The field is read back afterwards** and compared against what was typed
|
||
(`--no-verify` skips it). A field that never took focus, or that ignored the
|
||
input, otherwise looks exactly like success. Exit code 4 means the text did not
|
||
land.
|
||
|
||
`--clear` selects-all and deletes first. Without it the text is inserted at
|
||
wherever the caret landed, which for a field with existing contents usually
|
||
produces something like `50005000`.
|
||
|
||
Typos are deliberately *not* simulated. A mistyped digit in a trading form is a
|
||
real loss, and the backspace-and-correct step is exactly the part that can go
|
||
wrong — a field with input masking or autocomplete can swallow the correction and
|
||
leave the wrong number behind.
|
||
|
||
Don't pass credentials via `--text`: it lands in your shell history and in the
|
||
process list. `--stdin` avoids both, but nothing here is built to handle secrets
|
||
safely.
|
||
|
||
## Window focus
|
||
|
||
A click on a window that isn't focused is consumed by the window manager
|
||
*activating* that window — it never reaches the control underneath. That's why an
|
||
automated click against a background Chrome appears to do nothing the first time
|
||
and work the second: the first click only brought Chrome forward.
|
||
|
||
The extension calls `chrome.windows.update({focused: true})`, but that only orders
|
||
windows **within** Chrome. If the frontmost *application* is your terminal — which
|
||
it is, since that's where you launched this — Chrome is still in the background.
|
||
|
||
So `focus.py` raises the browser application itself immediately before the press,
|
||
then confirms it actually came forward before committing to the click. If it can't
|
||
verify, it refuses rather than firing a click that would be eaten (exit code 3).
|
||
|
||
On macOS this uses `NSWorkspace` via pyobjc, which needs no Automation permission —
|
||
activating an app is not scripting it. Without pyobjc it falls back to `osascript`,
|
||
which does prompt for Automation permission the first time.
|
||
|
||
## Cursor motion
|
||
|
||
`humanize.py` moves the pointer the way a hand does rather than teleporting:
|
||
|
||
- a curved (cubic Bézier) path instead of a straight line, bowing to one side
|
||
- eased velocity — accelerate out, coast, decelerate in
|
||
- sub-pixel tremor that decays near the target, so the landing stays exact
|
||
- long throws (>260px) sometimes overshoot slightly and pull back
|
||
- a 60–170ms dwell after arriving, before the press
|
||
- the button held down 55–120ms rather than an instant down/up
|
||
|
||
This is about reliability as much as appearance. Plenty of web controls only arm
|
||
once they have actually been hovered — dropdowns, tooltip-gated buttons, custom
|
||
widgets — and a cursor that arrives and presses in the same tick can outrun the
|
||
page's own `mousemove` handlers. The dwell is what lets those catch up.
|
||
|
||
Note `pyautogui.PAUSE` is set to 0 on import: it otherwise sleeps 0.1s after
|
||
*every* call, which would add tens of seconds across a stepped path.
|
||
|
||
Exit codes: `0` ok, `1` error, `2` capture switch off, `3` refused to click
|
||
(covered element, or coordinates off-screen).
|
||
|
||
## Safety
|
||
|
||
- **Failsafe**: slam the cursor into a screen corner to abort mid-run.
|
||
- **Covered elements**: before reporting, the extension checks
|
||
`document.elementFromPoint()` at the target centre. If a cookie banner or modal
|
||
is on top, the click is refused rather than sent into the overlay — `--force`
|
||
overrides.
|
||
- **Off-screen check**: coordinates outside the display bounds are refused, which
|
||
catches a Chrome window on a second monitor or partly off the edge.
|
||
- The element is scrolled to the centre of the viewport before measuring, so a
|
||
target below the fold is handled rather than mis-clicked.
|
||
|
||
## Known limits
|
||
|
||
- **Latency** is bounded by the extension's poll interval (default 3s), so a click
|
||
takes a few seconds to fire. Drop the interval in the extension popup if that
|
||
matters.
|
||
- **Multi-monitor**: coordinates come from `window.screenX/screenY`, which are
|
||
relative to the primary display's origin. A Chrome window on a secondary monitor
|
||
usually still works, but verify with `locate` before trusting `click`.
|
||
- **Display scaling**: the scale factor is inferred by comparing the screen width
|
||
the OS reports against the one the browser reports. On macOS and unscaled Windows
|
||
this is 1:1. If clicks land at a consistent offset, set `--scale` explicitly.
|
||
- The measurement and the click are separate moments. If the page moves the element
|
||
in between (a re-render, a late-loading banner), the click lands where it *was*.
|