autofirmer-expanded/README.md

# AutoFirmer

Automated futures trading dashboard for prop firm accounts via Tradovate and
TopstepX (ProjectX Gateway).

---

## Setup

### Windows

In **PowerShell**, from wherever you want the instance to live:

```powershell
irm https://git.juicerroom.com/senofy/autofirmer-expanded/raw/branch/master/setup-windows.bat -OutFile setup-windows.bat
```

```powershell
.\setup-windows.bat
```

That is the whole install. It clones into an `autofirmer` folder beside itself,
installs anything missing (Git, Node, Python) via winget, installs dependencies,
builds, asks for the master dashboard URL and an instance name, and offers to
set up auto-start and auto-update.

Re-running it updates an existing checkout rather than cloning again, so it
doubles as a repair tool.

Two things it deliberately does, worth knowing:

- **Node LTS, not latest.** `better-sqlite3` ships prebuilt binaries for LTS
  only. On a newer major it falls back to compiling with node-gyp and fails
  without Visual Studio Build Tools, so the script warns before that happens.
- **It looks for tools off `PATH`.** A tool installed in the same session it is
  needed — or installed without its "add to PATH" option — is invisible to a
  running shell, so the script checks the standard install directories too.

### Raspberry Pi 5 / Debian / Ubuntu

```bash
curl -fsSLO https://git.juicerroom.com/senofy/autofirmer-expanded/raw/branch/master/setup-linux.sh
```

```bash
bash setup-linux.sh
```

Same idea, apt instead of winget. It installs Node 22 from NodeSource (Pi OS
ships one too old for Next 16), `xdotool` and `scrot` for the clicker, and puts
the Python dependencies in a virtualenv — Pi OS Bookworm enforces PEP 668, so a
system-wide `pip install` fails outright.

**The clicker needs X11.** Pi OS on a Pi 5 defaults to Wayland, which does not
let one client synthesise input into another or read the focused window, so
neither `xdotool` nor pyautogui can work there. The dashboard is unaffected, but
for the AutoBuyer:

```bash
sudo raspi-config    # Advanced Options -> Wayland -> X11, then reboot
```

`python clicker/diagnose.py` reports the session type, `DISPLAY` and whether
`xdotool` is present — run it before trusting a new machine.

The dashboard runs on port **3000**: `http://localhost:3000`.

---

## First-time setup

On first run, the SQLite database (`autotrader.sqlite`) is created automatically in the project root. No migrations need to be run manually.

Firms and their account types live on the master dashboard, not here — every
machine pulls the same catalog so they cannot drift apart. This machine only
stores the Tradovate login for each firm.

1. On the **master dashboard**, open **Firms**, add the firm, and give it one
   account type per prefix + account size combination (profit target,
   consistency %, min trading days, and so on)
2. Set **Dashboard URL** on this machine's **Settings** page so it can reach the
   master. The catalog is pulled at startup and then every 60 seconds
3. On the main page, each firm from the catalog appears. Firms without stored
   credentials are greyed out — enter the Tradovate username and password in the
   fields beside the firm name and click **Save**. A firm whose platform is
   **TopstepX** on the master asks for an API key instead (generated in the
   TopstepX platform under Settings → API); the username is optional to store
   but the ProjectX login needs it before the firm can connect, so add it when
   you have it. Password is unused on TopstepX
4. Accounts populate once the firm connects. Each is matched to an account type
   by its name prefix *and* its size, so a new account shows as **Unassigned**
   until its opening funding transaction has been fetched. Use the **Account
   Type** dropdown to pin one by hand; a manual pick is never overwritten

---

## The master dashboard

Set **Instance Name** and **Dashboard URL** on the Settings page and this
instance does two things with them: it reports its firm and account counts every
30 seconds, and it holds a WebSocket open to `<url>/ws/instance` so the master
can drive it.

What the master can trigger here:

| | |
|---|---|
| Trade | Buy / Sell / Auto on a symbol, with the same "stop after all" option as the button on this dashboard |
| Stop | Halts the auto-trade scheduler |
| Run an automation | Queues a run for `clicker/runner.py`, exactly as the AutoBuyer page does |

Commands land in [`lib/master-link.ts`](lib/master-link.ts), which calls the same
functions the pages call — `runTrade`, `startScheduler`, `queueRun`. Nothing
reaches the broker by a path the local UI does not also use, and every command
is acked with its result or its error so the master shows what actually happened
rather than assuming.

The link is entirely optional. With no Dashboard URL configured it never starts,
and if the master is unreachable the instance keeps trading on its own and falls
back to the plain HTTP report until the socket comes back (retrying with backoff,
capped at a minute).

**Nothing authenticates this link in either direction.** Whatever answers at the
Dashboard URL can place orders on every account this instance holds, so point it
only at a master you control, and prefer `https` so the address cannot be
redirected in transit. If the socket will not stay up, the `connected` light on
the master is the only symptom — check this instance's logs for `[master-link]`.

## AutoBuyer (browser automation)

The AutoBuyer page drives a real browser to buy and reset prop-firm accounts. It
is three pieces, and all three must be running:

| Piece | What it does |
|---|---|
| the dashboard | Defines automations, queues runs, shows progress |
| `extension/` | A Chromium extension that reads the broker page and measures elements |
| `clicker/runner.py` | A desktop process that moves the real mouse and keyboard |

Automations are declared in `lib/automations.ts` — one entry per firm, with the
steps inside. Adding a button means editing that file; the page and the runner
pick it up from the server.

### 1. Install the extension

The setup scripts do this as their last step: give them the Chrome Enterprise
Core enrolment token from the Admin console (Devices → Chrome → Managed
browsers) and they enrol this machine's Chrome. The extension then arrives from
the console already set to force install, and Chrome keeps it updated. Run on
its own at any time:

```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\enroll-chrome-management.ps1
```

```bash
sudo scripts/install-extension-policy.sh
```

The enrolment token is built into both scripts — a token is reusable across any
number of machines — so there is nothing to type. Pass one explicitly to use a
different org. Chrome itself is installed first if the machine hasn't got it,
from Google's own enterprise MSI on Windows and the official .deb on Linux;
Chromium is not accepted as a substitute, because it has no cloud-management
support and can never be enrolled.

**A Raspberry Pi needs none of it.** Chrome is not published for ARM, but
Chromium is, and on Linux Chromium honours the same forcelist policy — the CRX
is pure JavaScript, so the architecture is irrelevant. `setup-linux.sh` writes
the policy and Chromium installs the extension from the update server on its
next start. Loading `extension/` unpacked still works and still self-reloads,
but it is no longer the only option there.

**Enrolment is a Windows problem only.** Windows refuses to force-install an
off-store extension unless the machine is domain-joined, Entra-joined or
enrolled in Chrome Enterprise Core — it reads the policy and blanks it — and
macOS wants MDM. Linux has no such requirement: a policy file under `/etc` is
enough, which is why a Raspberry Pi running Chromium installs this with no
Google account involved at all. On Linux, run `install-extension-policy.sh` and
skip enrolment entirely. See [deploy/README.md](deploy/README.md). Either way the
extension notices when
the dashboard is running newer code than it is: a policy install asks Chrome to
fetch the new CRX, an unpacked one reloads itself from disk. Clicking the reload
icon on its card still works and is the thing to try when something looks
stuck.

Adding a new firm also means adding its host to `host_permissions` in
`extension/manifest.json` and reloading. Without it the extension cannot read
that site, and every step fails to locate.

### 2. The clicker

`setup-windows.bat` installs its Python dependencies, and re-applies them
whenever `clicker/requirements.txt` changes. To do it by hand:

```powershell
python -m pip install -r clicker\requirements.txt
```

See `clicker/README.md` for the per-platform notes — display scaling and
foreground lock both matter on Windows — and for the verification sequence to
run before letting it click anything on a new machine.

### 3. Start the runner

Auto-start runs it under PM2, so normally there is nothing to do. To run it by
hand instead:

```powershell
python clicker\runner.py
```

It reports in every two seconds, and the dashboard greys out
the automation buttons when it is not there. A running Python process does not
reload when the source changes, so restart it after pulling updates; the
dashboard warns when its version is behind.

### 4. Captchas

A run that hits a captcha solves it through the **2captcha API**, not by clicking
2captcha's own browser extension. The API key is hardcoded in
`app/api/autobuyer/captcha/route.ts` — to bill a different account, set
`TWOCAPTCHA_API_KEY`, or PATCH `twocaptcha_api_key` to `/api/settings`. Either way
the key stays on the dashboard: the runner asks this machine to solve and never
handles it.

A `solveCaptcha` step then does three things, with no mouse involved: the
extension reads the widget's sitekey out of the page, the dashboard buys a token
from 2captcha, and the extension puts that token into the page's response field
and fires the page's own callback. reCAPTCHA v2 (including Enterprise), hCaptcha
and Turnstile are detected automatically.

If the 2captcha *extension* is still installed, disable it or turn off its
auto-solve. Two solvers racing on one widget is a good way to burn balance and
end up with a token the page has already replaced.

Solving costs money per captcha, and a failed solve is still billed. `timeoutSeconds`
on the step caps how long a run waits; 2captcha usually answers in 10–30s.

**The solver proxy.** Solves go through 2captcha's residential proxy, so the
challenge is answered from a residential address rather than a datacentre one.
The credentials are hardcoded in `app/api/autobuyer/captcha/route.ts`; override
them without touching code via `TWOCAPTCHA_PROXY`, or by PATCHing
`twocaptcha_proxy` to `/api/settings`. Either accepts `none` to solve proxyless.
Neither has a field on the Settings page — both are hardcoded, so there is
nothing to fill in. The run log names the address that
solved each captcha — the first thing to check when a token comes back and the
site rejects it anyway.

Two things that proxy does *not* do. It is not the browser's proxy: the token is
solved from the proxy's address but the form is submitted from this machine's,
and nothing makes those match. And the pool is not geo-pinned — despite the
`na.` hostname, test requests exited from Russia and Japan. For reCAPTCHA v2 and
Turnstile that mismatch is usually tolerated; if a site starts rejecting tokens
that 2captcha reports as solved, set the proxy to `none` and compare.

### Running it

Press an automation's button. Progress appears per step, and **Stop** halts a run
between steps.

There is no switch to throw first. Page capture follows whichever firm tab you
are looking at and stops when you switch away, and a run starts because someone
asked for one — from this page or from the master dashboard. The only thing that
gates a run is whether `clicker/runner.py` is up to claim it, which the page
shows as **Autoclicker daemon**.

That does mean a queued run executes as soon as the runner sees it, with no
second confirmation. Stop the runner if you want the machine idle.

The browser must be visible and frontmost while a run is in flight — the clicks
are real OS-level input, so the machine cannot be used for anything else, and a
dialog stealing focus fails the step. That makes an RDP session a poor host:
disconnecting can suspend the desktop and break clicks in ways that are hard to
diagnose.

## Keeping it running

`setup-windows.bat` offers to set this up for you at step 6. To enable it later,
or after declining:

```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\install-autostart.ps1
```

That puts the dashboard, the clicker and an update checker under PM2, adds a
Startup-folder entry so PM2 comes back at logon, and lets PM2's cron restart run
the update check every 5 minutes. It needs no administrator rights, changes
nothing machine-wide, and is safe to re-run.

```powershell
pm2 list                 # what is running
pm2 logs autofirmer      # dashboard output
pm2 logs clicker         # runner output
```

**Why not a Windows service.** The clicker sends real mouse and keyboard input
and has to own a desktop. A service runs in session 0, which has none, so the
clicks would go nowhere. Everything therefore runs in your logged-in session —
which also means an unattended reboot leaves the instance down until somebody
logs in.

To stop it starting at logon, delete the `AutoFirmer.cmd` shortcut from your
Startup folder (`shell:startup` in the Run dialog).

To start everything by hand without waiting for a logon:

```powershell
node scripts\start-all.mjs
```

`start-autofirmer.bat` still runs the dashboard in a visible window without PM2,
which is the easier thing to watch when a build is misbehaving.


## Firewall

To restrict port 3000 to a specific trusted IP only:

```powershell
New-NetFirewallRule -DisplayName "AutoFirmer" -Direction Inbound -Protocol TCP -LocalPort 3000 -RemoteAddress <your-ip> -Action Allow
```

Or open it to all inbound (less secure):

```powershell
New-NetFirewallRule -DisplayName "AutoFirmer" -Direction Inbound -Protocol TCP -LocalPort 3000 -Action Allow
```

---

## Publishing extension updates

The extension is signed and served from the droplet, which is what lets Chrome
update it on its own. Everything else — the dashboard, the clicker — stays on
the trading machine.

```bash
# on the droplet, in /opt/autofirmer
git pull && ./deploy/deploy.sh
```

That packs `extension/` at the version in its `manifest.json` — **bump it or
nothing updates** — signs it, and publishes the CRX and update manifest. See
[deploy/README.md](deploy/README.md).

---

## Updating

Once auto-start is installed, nothing here is manual. Every 5 minutes the update
task fetches `master`, and when it has moved it pulls, reinstalls dependencies if
`package-lock.json` or `clicker/requirements.txt` changed, rebuilds, restarts
AutoFirmer, and restarts the clicker if anything under `clicker/` changed.

**A failed build is never deployed.** The build runs before anything restarts, so
a broken push leaves the previous build serving and logs the failure instead.

### Checking it is alive

`scripts/.last-check` is rewritten on every check, whether or not anything came
down:

```
2026-08-31T00:43:27.155Z  up to date at 27a61fd2
```

This exists because the alternatives mislead. `update.log` only records real
events, so it stays empty for days when nothing is pushed — and PM2 reports a
cron-restart process as `stopped` with a restart count of `0` even while it is
firing on schedule. Neither is evidence of a problem; `.last-check` is the
signal to trust.

Everything that actually happens is appended to `scripts/update.log`. To see
what it would do without touching anything:

```powershell
node scripts\update-check.mjs --dry-run
```

To apply an update immediately rather than waiting for the next check:

```powershell
node scripts\update-check.mjs
```

### What reloads itself, and what does not

- **The extension** updates itself now. `/api/autobuyer/status` reports the
  version the dashboard's own checkout ships, and when the running build is
  older the extension acts on it: a policy install asks Chrome to fetch the new
  CRX from the update server, an unpacked one calls `chrome.runtime.reload()`
  and comes back on the files the update check pulled. The unpacked path can
  only run code already on disk, so if it reloads before the pull lands it says
  so in the popup and waits rather than restarting in a loop.
- **The scheduler is fine now.** It persists to the settings table and resumes
  after a restart, so an update no longer silently stops automated trading.
- **The runner** does not. It is a long-lived Python process; the update check
  restarts it when anything under `clicker/` changes, and the dashboard warns
  when the version it reports is behind what the server expects.

The dashboard reports the version it sees from the extension and the runner, and
warns when either is behind. Most AutoBuyer bugs that look mysterious are the
extension still running the previous code.