Add automation framework, typing, and runner for the autobuyer

Turns the autobuyer from a page scraper into something that acts. A dashboard
button queues a run; a desktop process executes it against the real browser.

lib/automations.ts — automations are declarative step lists nested inside the
firm whose site they drive. Steps are click / type / wait / navigate, and they
inherit the firm's tab pattern and URL, so one firm's automation can't act on
another's tab. Adding a button means adding an entry here; the page renders
buttons from the API and the runner receives steps from the server, so neither
needs editing. Runs key on firm:automation — every firm will plausibly have its
own "buy-accounts", and a bare id would resolve to the wrong one.

clicker/runner.py — the daemon behind the buttons. Claims a queued run, works
through the steps, reports each one back for the page's live log. Only one run
executes at a time: two processes driving one physical mouse would interleave
clicks. Heartbeats on its own thread, because a step can block for tens of
seconds and folding the beat into the main loop would show the runner as offline
in the middle of the run it was executing.

clicker/actions.py — one implementation of the safety checks, shared by the CLI
and the runner. Refuses to act when the element is covered by an overlay, when
coordinates fall off-screen, when the browser can't be confirmed frontmost, or
(for type) when the target isn't an editable field.

Typing: uneven human cadence, and the field is read back afterwards and compared
against what was typed — a field that never took focus fails silently and looks
identical to success otherwise. Non-ASCII is rejected because pyautogui skips
those characters without complaint, and newlines because Enter may submit the
form. Typos are deliberately not simulated: a mistyped digit in a trading form
is a real loss, and the correction is the part that can go wrong.

Extension: opens the firm's page when no tab matches, navigates to a specific
page for a navigate step (skipped when already there, so page state survives),
and retries the locate while a freshly loaded React app mounts — `complete` only
means the document loaded.

Staleness reporting, after it cost three debugging rounds: Chrome doesn't reload
an unpacked extension and Python doesn't reload a running process, so both now
report their version. A stale runner gets a red banner naming both versions and
the automation buttons are disabled, rather than failing mid-run on a step type
it predates.

Scale detection is now conservative: a raw OS/browser width ratio is only
trusted when it lands on a real scaling factor. On this multi-monitor desktop
the previous logic would have silently halved every coordinate.

Verified end to end against the live browser: navigate, locate, and a real
click (run #12, all three steps). API round-trips, claim-once semantics, run
cancellation, the heartbeat online/offline lifecycle, motion geometry and
timing, focus activation, and typing verification all pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Brandon Li
2026-08-28 14:00:33 -05:00
co-authored by Claude Opus 5
parent 54221bbc0c
commit b748f95372
17 changed files with 1463 additions and 99 deletions
+67
View File
@@ -16,6 +16,27 @@ clicker --GET /api/autobuyer/locate?id=---> reads the answer
clicker moves the mouse, clicks
```
## Two ways to run it
**`clicker.py`** — one action at a time, from the shell. Use this to find
selectors and confirm coordinates before wiring anything up.
**`runner.py`** — the daemon behind the dashboard's buttons. Leave it running; it
polls for queued runs and executes the steps.
```bash
python runner.py # then press a button on the AutoBuyer page
python runner.py --dry-run # walks the steps, never presses or types
```
Automations are defined in `lib/automations.ts`, and the steps are sent to the
runner by the server — so adding a button means editing that file, and nothing in
the runner or the page changes. Each run reports step-by-step progress back to the
dashboard, and the page's Stop button halts a run between steps.
Only one run executes at a time: two processes driving the same physical mouse
would interleave clicks.
## Setup
```bash
@@ -41,6 +62,12 @@ python clicker.py click "button.buy" --dry-run
# Actually click.
python clicker.py click "button.buy"
# Type into a field: clicks it to place the caret, then types.
python clicker.py type 'input[name="quantity"]' --text "5000" --clear
# Keep the text out of shell history.
echo "5000" | python clicker.py type 'input#qty' --stdin --clear
# Pin it to a specific tab, and pick the 3rd match.
python clicker.py click ".trade-btn" --index 2 --url "https://tradeify.co/*"
```
@@ -57,6 +84,46 @@ python clicker.py click ".trade-btn" --index 2 --url "https://tradeify.co/*"
| `--robotic` | Straight-line move and instant click, skipping the motion model. |
| `--seed` | Seed the motion RNG so a run replays identically (debugging). |
| `--no-activate` | Don't raise the browser first. The click may then be swallowed. |
| `--text` | Text to type (action `type`). |
| `--stdin` | Read the text from stdin instead, keeping it out of shell history. |
| `--clear` | Select-all and delete before typing, instead of appending at the caret. |
| `--allow-enter` | Permit newlines. Each one presses Enter, which may submit the form. |
| `--no-verify` | Skip reading the field back after typing. |
## Typing
`type` clicks the field to place the caret, then types with an uneven human
cadence (45130ms between keys, a beat after each space, an occasional longer
pause). Real OS-level keystrokes are also the *correct* way to fill a React form:
setting `.value` directly is ignored by controlled components, while genuine key
events are not.
Four guards, all of which catch silent failures:
- **Non-ASCII is rejected.** `pyautogui.write()` has no keycode for `é` or `£` and
skips them without complaint, which would leave a quietly truncated value in the
field. Better to refuse than to submit `caf` where you meant `café`.
- **Newlines are rejected** unless `--allow-enter`, because Enter may submit the
form — and on this site that could mean placing an order.
- **Non-editable targets are refused** — disabled, read-only, or simply not an
input. The keystrokes would go nowhere.
- **The field is read back afterwards** and compared against what was typed
(`--no-verify` skips it). A field that never took focus, or that ignored the
input, otherwise looks exactly like success. Exit code 4 means the text did not
land.
`--clear` selects-all and deletes first. Without it the text is inserted at
wherever the caret landed, which for a field with existing contents usually
produces something like `50005000`.
Typos are deliberately *not* simulated. A mistyped digit in a trading form is a
real loss, and the backspace-and-correct step is exactly the part that can go
wrong — a field with input masking or autocomplete can swallow the correction and
leave the wrong number behind.
Don't pass credentials via `--text`: it lands in your shell history and in the
process list. `--stdin` avoids both, but nothing here is built to handle secrets
safely.
## Window focus