PM2's cron for autofirmer-update dies inside a live daemon (six Windows
machines in October), and nothing on the machine noticed. The app now
watches scripts/.last-check and, past 10 minutes, runs
update-check.mjs --heal: a checker PM2 still lists is restarted (which
re-registers its cron), one it lost is registered again, and one whose
errored entry still holds a live pid is left alone, since PM2 would kill
that pid. The heal scrubs PM2's per-app fields from its environment first.
The master's new 'update-now' command runs --heal --now, skipping a
failed update's retry wait. A failed pull, npm install or build records
one line of reason (or the chown to run when a file isn't ours), which
rides on .last-check to the master. A package-lock.json rewritten by npm
is put back before the pull. The fetch timeout drops under the cron's
minute. scripts/.no-auto-update opts a checkout out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The master saw a scheduler's last run time and nothing of what it did. Each
tick now records, per lane, its outcome, a line on what happened (what
fired, or the spread, contract or hold cap that stopped it) and why the
lane's accounts sat it out, counted by reason with a few names each. It
rides the scheduler status as lastTicks.
Each report also carries the auto-updater's state from the files it keeps
and from git: its last check, a failed update waiting to retry, how far the
checkout is behind origin, and tracked files edited by hand, which stop the
fast-forward pull.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lib/instrument-specs.ts (pure, tested): the CME specs — point value,
tick size, one-tick spread, Yahoo ticker — replacing POINT_VALUES; the
draw's maths; the 12-hour hold cap; the micro for drain and extra-day
legs; reading a firm's list (the master's, or the old rules against an
older master: this machine's enabled set less bans, the symbol map).
- lib/instruments.ts: the one module the trading paths ask. Measures every
future's daily move from Yahoo's hourly candles once per trade date, and
shows entry cost (a tick of spread plus Tradovate's fee parameters or
TopstepX's last measured fill) without ever choosing by it.
- Futures Auto draws one root per batch from the batch firm's list: movement
per dollar of spread early in the session with a 10% floor each, sliding
to the fastest movers by the last entry; nothing expected to hold 12h or
more; micros only if nothing else is listed. Every draw logs its odds.
- The batch is chosen before its instrument, and a group that can't trade
anything yields to the next. Copy trade matches the held contract against
the listed roots and won't copy an instrument it can't identify. Manual
legs, manual orders and drain/extra-day legs follow the lists.
- DXtrade firms rank their own list whatever root was drawn.
- Capabilities, the dashboard's trade bar and contract resolves follow the
lists; the trade bar and scheduler default to Auto. The master link
reconnects when master_dashboard_url changes, and the catalog drops
?since after a switch so two masters' versions can't collide.
- Reports carry each futures firm's instrumentStats for the master's Firms
page; GET /api/instruments shows them (?simulate=N runs the draw, no
orders); the Settings page shows the lists read-only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Why the autotrader sometimes broke after a weekend until `pm2 restart`:
`next start` only logs uncaught errors, so nothing ever crashed, and a few
paths could leave a client dead for the life of the process.
Tradovate: a closed socket scheduled exactly one re-login, with no catch —
one failure during weekend maintenance (or a rate-limit penalty returned as
a 200 without a token) and the client never came back. connectAndAuth is now
a single-flight loop that never throws: jittered backoff capped at 5 min,
penalty tickets waited out, captcha demands backed off. A 15s watchdog
abandons silent, never-authorized or stuck-closing sockets and stale syncs;
renewal falls back to a fresh login; orders fail fast (OrderNotPlacedError)
on a dead socket and time out instead of hanging; a bad sync answer no
longer wipes accounts and positions.
TopstepX: hub start/stop/invoke and every REST call are bounded; a wedged
hub is discarded and rebuilt; a stuck sync is superseded after 3 min;
re-login no longer tears down the sync loop; a watchdog re-logs when syncs
go stale; a no-fill timeout cancels the order.
Scheduler: one unready client used to hold every firm forever. It now holds
the tick for a 10-minute grace, then trades the other firms without it.
Per-firm try/catch in batch collection, one batch at a time, accounts marked
in flight, legs un-counted only when the entry never reached the broker, and
the scheduler starts before the first batch fires.
Boot and supervision: instrumentation.ts boots the process at server start
(it used to wait for the first dashboard request); a supervisor rebuilds a
client that has been unhealthy for 20 min (not a TopstepX one holding a
position); broker HTTP calls default to a 60s timeout; GET /api/health shows
each client's liveness and the scheduler's last decision. Shared state the
eager boot would otherwise split across bundles lives on globalThis. Logs no
longer carry access tokens or whole AxiosErrors (which include the encoded
Tradovate password).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four features:
- Backup: lib/backup-sync.ts pushes the full settings table and firm
credentials (the deliberate exception to "credentials never leave the
machine") to the master's bearer-gated /api/instances/backup — every
5 minutes when the content hash changed, within seconds of a settings
or credential edit, and on the new backup-now command. Enabled by the
master_backup_token setting.
- Restore: a dead VPS's replacement pulls the backup and adopts its
uuid/name, so roster, payouts and card links carry on. Pre-boot via
seed-settings.js --restore-from (setup-linux.sh prompts for it) or
the Settings page's restore picker (lib/backup-restore.ts). Stop the
old machine first; the master 409s a live-name rename.
- Proxy: the proxy_url setting routes BROKER traffic only — TopstepX
REST + SignalR hub, Tradovate REST + websocket, Yahoo quotes — via
lib/outbound-proxy.ts (axios agents, undici dispatcher, ws agent).
The master link, reporter, catalog and backup push stay direct, so a
dead proxy cannot take down the control channel. Restart to apply.
- Multi-instance: AUTOFIRMER_DB moves the SQLite file so one checkout
runs N instances; scripts/instances.json (per-device, gitignored)
drives start-all/update-check, and add-instance.sh / remove-instance.sh
manage the extra PM2 processes. Absent registry = single-instance
behavior, unchanged. Instance 1 keeps port 3000 for the clicker and
extension.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The version is the checkout's HEAD commit (short hash + commit time),
read once at startup — updates arrive as `git pull`, so there is no
release number to bump. It rides on the socket hello, socket reports,
and the HTTP report fallback; the field is optional in the protocol so
older masters ignore it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The welcome now carries a personId; it lands in the instance_id setting
and rides on every hello and HTTP report from then on. The master keys
the person by that uuid, so changing instance_name moves our history to
the new name instead of starting a stranger. Read-only in Settings on
purpose — editing it by hand would detach this machine from its person.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three changes that ride together because they touch the same surface.
A control link to the master. lib/master-link.ts dials out to
<master>/ws/instance and holds it open, so the dashboard can trigger a trade or
queue an AutoBuyer run here. Commands call the same functions the local pages
call — runTrade, startScheduler, queueRun — so nothing reaches the broker by a
path the UI does not also use. Reports prefer the socket and fall back to the
existing POST, which keeps older masters working unchanged.
startMasterLink is idempotent, and that matters more than it looks: getClients()
is lazy and starts the link, so the first command re-enters it from inside a
command the live socket is waiting to ack. Tearing the socket down there loses
the ack and reports a trade that actually executed as a failure — which invites
pressing the button twice.
queueRun is lifted out of the automations route so the socket and HTTP share one
copy of the input clamping that decides how many times a purchase fires.
Page capture loses its switch and gains a scope. It now runs every tick, but
only against the active tab of the last-focused window that matches
host_permissions — so it follows the page in front of you and stops when you
switch away, where before it would fall back to any firm tab left open in the
background.
autobuyer_enabled is gone entirely. The AutoBuyer is ready whenever the runner
is; a run happens because someone asked for one. That does mean a queued run
executes as soon as clicker/runner.py claims it, with no second gate — stopping
the runner is now the way to make a machine idle.
Extension to 0.10.0; it needs a manual reload at chrome://extensions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Brandon Li