autofirmer-expanded/README.md

# AutoFirmer

Automated futures trading dashboard for prop firm accounts via Tradovate.

---

## Setup

### Windows

In **PowerShell**, from wherever you want the instance to live:

```powershell
irm https://git.juicerroom.com/senofy/autofirmer-expanded/raw/branch/master/setup-windows.bat -OutFile setup-windows.bat
```

```powershell
.\setup-windows.bat
```

That is the whole install. It clones into an `autofirmer` folder beside itself,
installs anything missing (Git, Node, Python) via winget, installs dependencies,
builds, asks for the master dashboard URL and an instance name, and offers to
set up auto-start and auto-update.

Re-running it updates an existing checkout rather than cloning again, so it
doubles as a repair tool.

Two things it deliberately does, worth knowing:

- **Node LTS, not latest.** `better-sqlite3` ships prebuilt binaries for LTS
  only. On a newer major it falls back to compiling with node-gyp and fails
  without Visual Studio Build Tools, so the script warns before that happens.
- **It looks for tools off `PATH`.** A tool installed in the same session it is
  needed — or installed without its "add to PATH" option — is invisible to a
  running shell, so the script checks the standard install directories too.

### Raspberry Pi 5 / Debian / Ubuntu

```bash
curl -fsSLO https://git.juicerroom.com/senofy/autofirmer-expanded/raw/branch/master/setup-linux.sh
```

```bash
bash setup-linux.sh
```

Same idea, apt instead of winget. It installs Node 22 from NodeSource (Pi OS
ships one too old for Next 16), `xdotool` and `scrot` for the clicker, and puts
the Python dependencies in a virtualenv — Pi OS Bookworm enforces PEP 668, so a
system-wide `pip install` fails outright.

**The clicker needs X11.** Pi OS on a Pi 5 defaults to Wayland, which does not
let one client synthesise input into another or read the focused window, so
neither `xdotool` nor pyautogui can work there. The dashboard is unaffected, but
for the AutoBuyer:

```bash
sudo raspi-config    # Advanced Options -> Wayland -> X11, then reboot
```

`python clicker/diagnose.py` reports the session type, `DISPLAY` and whether
`xdotool` is present — run it before trusting a new machine.

The dashboard runs on port **3000**: `http://localhost:3000`.

---

## First-time setup

On first run, the SQLite database (`autotrader.sqlite`) is created automatically in the project root. No migrations need to be run manually.

Open the app in your browser and add your firms through the UI:

1. Click **+ Add Firm** on the main page
2. Enter the firm name, Tradovate username, and password
3. Go to the firm's **Settings** page to configure account types (prefix, profit target, consistency %, etc.)
4. Return to the main page — accounts will populate once the firm connects to Tradovate

---

## The master dashboard

Set **Instance Name** and **Dashboard URL** on the Settings page and this
instance does two things with them: it reports its firm and account counts every
30 seconds, and it holds a WebSocket open to `<url>/ws/instance` so the master
can drive it.

What the master can trigger here:

| | |
|---|---|
| Trade | Buy / Sell / Auto on a symbol, with the same "stop after all" option as the button on this dashboard |
| Stop | Halts the auto-trade scheduler |
| Run an automation | Queues a run for `clicker/runner.py`, exactly as the AutoBuyer page does |

Commands land in [`lib/master-link.ts`](lib/master-link.ts), which calls the same
functions the pages call — `runTrade`, `startScheduler`, `queueRun`. Nothing
reaches the broker by a path the local UI does not also use, and every command
is acked with its result or its error so the master shows what actually happened
rather than assuming.

The link is entirely optional. With no Dashboard URL configured it never starts,
and if the master is unreachable the instance keeps trading on its own and falls
back to the plain HTTP report until the socket comes back (retrying with backoff,
capped at a minute).

**Nothing authenticates this link in either direction.** Whatever answers at the
Dashboard URL can place orders on every account this instance holds, so point it
only at a master you control, and prefer `https` so the address cannot be
redirected in transit. If the socket will not stay up, the `connected` light on
the master is the only symptom — check this instance's logs for `[master-link]`.

## AutoBuyer (browser automation)

The AutoBuyer page drives a real browser to buy and reset prop-firm accounts. It
is three pieces, and all three must be running:

| Piece | What it does |
|---|---|
| the dashboard | Defines automations, queues runs, shows progress |
| `extension/` | A Chromium extension that reads the broker page and measures elements |
| `clicker/runner.py` | A desktop process that moves the real mouse and keyboard |

Automations are declared in `lib/automations.ts` — one entry per firm, with the
steps inside. Adding a button means editing that file; the page and the runner
pick it up from the server.

### 1. Load the extension

`chrome://extensions` → enable **Developer mode** → **Load unpacked** →
select the `extension` folder.

Chrome does **not** reload an unpacked extension when its files change, and it
has no update channel for one either. The extension therefore checks the version
the dashboard ships on every poll and reloads *itself* when it is behind, which
re-reads the folder from disk. Clicking the reload icon on its card still works
and is the thing to try when something looks stuck.

Adding a new firm also means adding its host to `host_permissions` in
`extension/manifest.json` and reloading. Without it the extension cannot read
that site, and every step fails to locate.

### 2. The clicker

`setup-windows.bat` installs its Python dependencies, and re-applies them
whenever `clicker/requirements.txt` changes. To do it by hand:

```powershell
python -m pip install -r clicker\requirements.txt
```

See `clicker/README.md` for the per-platform notes — display scaling and
foreground lock both matter on Windows — and for the verification sequence to
run before letting it click anything on a new machine.

### 3. Start the runner

Auto-start runs it under PM2, so normally there is nothing to do. To run it by
hand instead:

```powershell
python clicker\runner.py
```

It reports in every two seconds, and the dashboard greys out
the automation buttons when it is not there. A running Python process does not
reload when the source changes, so restart it after pulling updates; the
dashboard warns when its version is behind.

### 4. Captchas

A run that hits a captcha solves it through the **2captcha API**, not by clicking
2captcha's own browser extension. The API key is hardcoded in
`app/api/autobuyer/captcha/route.ts` — to bill a different account, set
`TWOCAPTCHA_API_KEY`, or PATCH `twocaptcha_api_key` to `/api/settings`. Either way
the key stays on the dashboard: the runner asks this machine to solve and never
handles it.

A `solveCaptcha` step then does three things, with no mouse involved: the
extension reads the widget's sitekey out of the page, the dashboard buys a token
from 2captcha, and the extension puts that token into the page's response field
and fires the page's own callback. reCAPTCHA v2 (including Enterprise), hCaptcha
and Turnstile are detected automatically.

If the 2captcha *extension* is still installed, disable it or turn off its
auto-solve. Two solvers racing on one widget is a good way to burn balance and
end up with a token the page has already replaced.

Solving costs money per captcha, and a failed solve is still billed. `timeoutSeconds`
on the step caps how long a run waits; 2captcha usually answers in 10–30s.

**The solver proxy.** Solves go through 2captcha's residential proxy, so the
challenge is answered from a residential address rather than a datacentre one.
The credentials are hardcoded in `app/api/autobuyer/captcha/route.ts`; override
them without touching code via `TWOCAPTCHA_PROXY`, or by PATCHing
`twocaptcha_proxy` to `/api/settings`. Either accepts `none` to solve proxyless.
Neither has a field on the Settings page — both are hardcoded, so there is
nothing to fill in. The run log names the address that
solved each captcha — the first thing to check when a token comes back and the
site rejects it anyway.

Two things that proxy does *not* do. It is not the browser's proxy: the token is
solved from the proxy's address but the form is submitted from this machine's,
and nothing makes those match. And the pool is not geo-pinned — despite the
`na.` hostname, test requests exited from Russia and Japan. For reCAPTCHA v2 and
Turnstile that mismatch is usually tolerated; if a site starts rejecting tokens
that 2captcha reports as solved, set the proxy to `none` and compare.

### Running it

Press an automation's button. Progress appears per step, and **Stop** halts a run
between steps.

There is no switch to throw first. Page capture follows whichever firm tab you
are looking at and stops when you switch away, and a run starts because someone
asked for one — from this page or from the master dashboard. The only thing that
gates a run is whether `clicker/runner.py` is up to claim it, which the page
shows as **Autoclicker daemon**.

That does mean a queued run executes as soon as the runner sees it, with no
second confirmation. Stop the runner if you want the machine idle.

The browser must be visible and frontmost while a run is in flight — the clicks
are real OS-level input, so the machine cannot be used for anything else, and a
dialog stealing focus fails the step. That makes an RDP session a poor host:
disconnecting can suspend the desktop and break clicks in ways that are hard to
diagnose.

## Keeping it running

`setup-windows.bat` offers to set this up for you at step 6. To enable it later,
or after declining:

```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\install-autostart.ps1
```

That puts the dashboard, the clicker and an update checker under PM2, adds a
Startup-folder entry so PM2 comes back at logon, and lets PM2's cron restart run
the update check every 5 minutes. It needs no administrator rights, changes
nothing machine-wide, and is safe to re-run.

```powershell
pm2 list                 # what is running
pm2 logs autofirmer      # dashboard output
pm2 logs clicker         # runner output
```

**Why not a Windows service.** The clicker sends real mouse and keyboard input
and has to own a desktop. A service runs in session 0, which has none, so the
clicks would go nowhere. Everything therefore runs in your logged-in session —
which also means an unattended reboot leaves the instance down until somebody
logs in.

To stop it starting at logon, delete the `AutoFirmer.cmd` shortcut from your
Startup folder (`shell:startup` in the Run dialog).

To start everything by hand without waiting for a logon:

```powershell
node scripts\start-all.mjs
```

`start-autofirmer.bat` still runs the dashboard in a visible window without PM2,
which is the easier thing to watch when a build is misbehaving.


## Firewall

To restrict port 3000 to a specific trusted IP only:

```powershell
New-NetFirewallRule -DisplayName "AutoFirmer" -Direction Inbound -Protocol TCP -LocalPort 3000 -RemoteAddress <your-ip> -Action Allow
```

Or open it to all inbound (less secure):

```powershell
New-NetFirewallRule -DisplayName "AutoFirmer" -Direction Inbound -Protocol TCP -LocalPort 3000 -Action Allow
```

---

## Hosting the dashboard on a server

The dashboard can run on the droplet instead of on the trading machine, with the
clicker and the extension pointing at it over HTTPS. The clicker and the
extension cannot move — one needs a desktop to send real input to, the other is
your browser.

```bash
cp deploy/.env.example deploy/.env    # set FIRMER_HOST
./deploy/bootstrap-tls.sh             # first run: certificate, then deploy
./deploy/deploy.sh                    # every run after that
```

It shares the droplet's existing nginx and certbot rather than standing up its
own, and the database lives on a Docker volume. There is no login on it. See
[deploy/README.md](deploy/README.md) before opening it to the internet.

---

## Updating

Once auto-start is installed, nothing here is manual. Every 5 minutes the update
task fetches `master`, and when it has moved it pulls, reinstalls dependencies if
`package-lock.json` or `clicker/requirements.txt` changed, rebuilds, restarts
AutoFirmer, and restarts the clicker if anything under `clicker/` changed.

**A failed build is never deployed.** The build runs before anything restarts, so
a broken push leaves the previous build serving and logs the failure instead.

### Checking it is alive

`scripts/.last-check` is rewritten on every check, whether or not anything came
down:

```
2026-08-31T00:43:27.155Z  up to date at 27a61fd2
```

This exists because the alternatives mislead. `update.log` only records real
events, so it stays empty for days when nothing is pushed — and PM2 reports a
cron-restart process as `stopped` with a restart count of `0` even while it is
firing on schedule. Neither is evidence of a problem; `.last-check` is the
signal to trust.

Everything that actually happens is appended to `scripts/update.log`. To see
what it would do without touching anything:

```powershell
node scripts\update-check.mjs --dry-run
```

To apply an update immediately rather than waiting for the next check:

```powershell
node scripts\update-check.mjs
```

### What reloads itself, and what does not

- **The extension** reloads itself now. `/api/autobuyer/status` reports the
  version the dashboard's own checkout ships; when the running build is older,
  the extension calls `chrome.runtime.reload()` and comes back on the files the
  update check pulled. It only ever runs code already on disk, so if it reloads
  before the pull lands it says so in the popup and waits rather than restarting
  in a loop. Clicking reload on its card is still the manual fallback.
- **The scheduler is fine now.** It persists to the settings table and resumes
  after a restart, so an update no longer silently stops automated trading.
- **The runner** does not. It is a long-lived Python process; the update check
  restarts it when anything under `clicker/` changes, and the dashboard warns
  when the version it reports is behind what the server expects.

The dashboard reports the version it sees from the extension and the runner, and
warns when either is behind. Most AutoBuyer bugs that look mysterious are the
extension still running the previous code.