senofy/autofirmer-expanded
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add automation framework, typing, and runner for the autobuyer

Browse Source 
Turns the autobuyer from a page scraper into something that acts. A dashboard
button queues a run; a desktop process executes it against the real browser.

lib/automations.ts — automations are declarative step lists nested inside the
firm whose site they drive. Steps are click / type / wait / navigate, and they
inherit the firm's tab pattern and URL, so one firm's automation can't act on
another's tab. Adding a button means adding an entry here; the page renders
buttons from the API and the runner receives steps from the server, so neither
needs editing. Runs key on firm:automation — every firm will plausibly have its
own "buy-accounts", and a bare id would resolve to the wrong one.

clicker/runner.py — the daemon behind the buttons. Claims a queued run, works
through the steps, reports each one back for the page's live log. Only one run
executes at a time: two processes driving one physical mouse would interleave
clicks. Heartbeats on its own thread, because a step can block for tens of
seconds and folding the beat into the main loop would show the runner as offline
in the middle of the run it was executing.

clicker/actions.py — one implementation of the safety checks, shared by the CLI
and the runner. Refuses to act when the element is covered by an overlay, when
coordinates fall off-screen, when the browser can't be confirmed frontmost, or
(for type) when the target isn't an editable field.

Typing: uneven human cadence, and the field is read back afterwards and compared
against what was typed — a field that never took focus fails silently and looks
identical to success otherwise. Non-ASCII is rejected because pyautogui skips
those characters without complaint, and newlines because Enter may submit the
form. Typos are deliberately not simulated: a mistyped digit in a trading form
is a real loss, and the correction is the part that can go wrong.

Extension: opens the firm's page when no tab matches, navigates to a specific
page for a navigate step (skipped when already there, so page state survives),
and retries the locate while a freshly loaded React app mounts — `complete` only
means the document loaded.

Staleness reporting, after it cost three debugging rounds: Chrome doesn't reload
an unpacked extension and Python doesn't reload a running process, so both now
report their version. A stale runner gets a red banner naming both versions and
the automation buttons are disabled, rather than failing mid-run on a step type
it predates.

Scale detection is now conservative: a raw OS/browser width ratio is only
trusted when it lands on a real scaling factor. On this multi-monitor desktop
the previous logic would have silently halved every coordinate.

Verified end to end against the live browser: navigate, locate, and a real
click (run #12, all three steps). API round-trips, claim-once semantics, run
cancellation, the heartbeat online/offline lifecycle, motion geometry and
timing, focus activation, and typing verification all pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Brandon Li
2026-08-28 14:00:33 -05:00
co-authored by Claude Opus 5
parent 54221bbc0c
commit b748f95372
17 changed files with 1463 additions and 99 deletions
Download Patch File Download Diff File Expand all files Collapse all files
clicker/actions.py
+174
View File
@@ -0,0 +1,174 @@
"""Execution of a single automation step: locate the element, then click or type.
Shared by the CLI (clicker.py) and the automation runner (runner.py) so there is
one implementation of the safety checks. Callers differ only in how they report
progress and how they surface failures.
"""
import random
import focus
class StepError(Exception):
"""A step refused to run, or ran and could not be verified.
`code` mirrors the CLI's exit codes: 3 = refused before acting,
4 = acted but verification failed.
"""
def __init__(self, message: str, code: int = 3):
super().__init__(message)
self.code = code
# Display scaling factors that actually exist. Anything else means the two sides
# are describing different things (usually a multi-monitor desktop) rather than a
# scaled single display.
KNOWN_SCALES = (1.0, 1.25, 1.5, 1.75, 2.0, 2.5, 3.0)
def scale_factor(found: dict, override: float | None, report=None) -> float:
"""CSS pixels and desktop pixels match on macOS and unscaled Windows, but
Windows display scaling breaks that.
Deliberately conservative: a raw ratio is only trusted when it lands on a real
scaling factor. On a multi-monitor desktop the browser reports the display it
is on while pyautogui reports the primary one, and the resulting ratio is
meaningless — halving every coordinate would put clicks far from the target.
Fall back to 1:1 and say so, rather than silently scaling.
"""
if override is not None:
return override
try:
import pyautogui
except ImportError:
return 1.0
css_width = (found.get("screenSize") or {}).get("width")
if not css_width:
return 1.0
os_width = pyautogui.size().width
ratio = os_width / css_width
for known in KNOWN_SCALES:
if abs(ratio - known) < 0.02:
return known
if report:
report(
f"screen size disagrees (browser {css_width}px, OS {os_width}px, "
f"ratio {ratio:.3f}) — assuming 1:1. If clicks land off, pass --scale."
)
return 1.0
def perform(
dash,
action: str,
selector: str,
*,
index: int = 0,
url: str = "",
open_url: str = "",
text: str | None = None,
clear: bool = False,
scale: float | None = None,
timeout: float = 20.0,
rng: random.Random | None = None,
activate: bool = True,
verify: bool = True,
force: bool = False,
dry_run: bool = False,
robotic: bool = False,
report=None,
on_located=None,
) -> dict:
"""Run one step. Returns the located element's data.
Raises DashboardError if the extension can't find it, StepError if the step
is refused or fails verification.
"""
import pyautogui
import humanize
rng = rng or random.Random()
say = report or (lambda _m: None)
found = dash.locate(selector, index, url, timeout, open_url)
if on_located:
on_located(found)
if found.get("covered") and not force:
raise StepError(
f"{selector} is covered by {found.get('coveredBy')} — the click would hit that instead"
)
if action == "type" and not found.get("editable", True) and not force:
kind = found.get("inputType")
raise StepError(
f"<{found['tag']}{f' type={kind}' if kind else ''}> is not an editable field "
"(disabled, read-only, or not an input)"
)
factor = scale_factor(found, scale, report)
x = found["screen"]["x"] * factor
y = found["screen"]["y"] * factor
screen_w, screen_h = pyautogui.size()
if not (0 <= x < screen_w and 0 <= y < screen_h):
raise StepError(
f"target ({x:.0f}, {y:.0f}) is off-screen ({screen_w}x{screen_h}) — "
"is the Chrome window on another display?"
)
say(f"at {x:.0f},{y:.0f}")
# A click on a background window is consumed activating it and never reaches
# the page, so raise the browser immediately before pressing.
if activate:
focused = focus.ensure_frontmost()
say(focused.detail)
if not focused.ok:
raise StepError(
"the browser is not frontmost — the click would be consumed "
"activating its window instead of pressing the element"
)
pyautogui.FAILSAFE = True
if robotic:
pyautogui.moveTo(x, y, duration=0.25)
if not dry_run:
pyautogui.click()
else:
humanize.click(x, y, rng=rng, press=not dry_run)
if action == "click" or dry_run:
return found
# ── type ────────────────────────────────────────────────────────────────
import time
time.sleep(0.15) # let the field take focus and any JS handlers settle
if clear:
humanize.clear_field(rng)
humanize.type_text(text or "", rng)
if not verify:
return found
# Typing into a field that never took focus, or that ignores the input, fails
# silently and is otherwise indistinguishable from success.
after = dash.locate(selector, index, url, timeout, open_url)
got = after.get("value")
if got is None:
say("element exposes no value to verify against")
return found
if (text or "") in got:
say(f"verified, field contains {got!r}")
return after
raise StepError(f"verification failed — field contains {got!r}", code=4)