Updater: detached worker, built-commit marker, restorable builds

PM2 runs update-check.mjs as a cron *restart*, and PM2 7 stops a process
that is still running at the next minute boundary. Any update that needed
npm install or next build (more than ~55s) was killed half-way: the lock
was left behind, HEAD had already moved at the pull so the next run saw
nothing to do, and next build had already wiped .next under the live
server — a box that broke after a weekend push and stayed broken.

- The cron run is now a quick CHECK: fetch, decide, and hand the work to a
  detached worker (--apply) that PM2's next tick can't reach.
- "Up to date" is judged against scripts/.built-commit (what .next was built
  from), not HEAD, so a pull that was never built finishes on the next check.
  First run on a machine adopts a complete build that is newer than HEAD's
  last move and rebuilds anything else — that rescues boxes the old updater
  already stranded.
- The previous build is set aside as .next-prev and restored if the build
  fails, so a broken push never leaves a box without a build; failures retry
  after 15 min, then hourly (scripts/.update-failed).
- The lock holds a pid and is atomic; build dirs are deleted without ever
  following the links in .next/node_modules into the real node_modules.
- Every instance in scripts/instances.json is restarted, and python/pip
  children stay hidden on Windows now that the worker has no console.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Li
2026-09-23 20:18:33 -05:00
co-authored by Claude Opus 5.5
parent ccad48cf0a
commit 6146031a38
4 changed files with 826 additions and 96 deletions
+37 -10
View File
@@ -355,13 +355,32 @@ nothing updates** — signs it, and publishes the CRX and update manifest. See
## Updating
Once auto-start is installed, nothing here is manual. Every 5 minutes the update
task fetches `master`, and when it has moved it pulls, reinstalls dependencies if
`package-lock.json` or `clicker/requirements.txt` changed, rebuilds, restarts
AutoFirmer, and restarts the clicker if anything under `clicker/` changed.
Once auto-start is installed, nothing here is manual. Every minute (the
installer's default interval) the update check fetches `master`. When there is something to do, it starts a background
worker. The worker pulls, reinstalls dependencies if `package-lock.json` or
`clicker/requirements.txt` changed, rebuilds, restarts every instance, and
restarts the clicker if anything under `clicker/` changed.
**A failed build is never deployed.** The build runs before anything restarts, so
a broken push leaves the previous build serving and logs the failure instead.
**Why a check and a separate worker.** PM2 runs the check as a cron *restart*,
and a restart kills a run that is still going at the next minute boundary. An
npm install or a build takes longer than that, so doing the work in the cron
process itself got it killed half-way through. The check finishes in seconds,
and the worker runs outside PM2's process tree, where the next tick can't
reach it.
**Deployed means built, not pulled.** `scripts/.built-commit` records the commit
`.next` was last built from, and the check compares `master` with that, not with
HEAD. A pull that was never built is therefore finished on the next check, and
so is a missing `.next/BUILD_ID`. The first time this version runs on a machine
it judges the build already on disk: one that is complete, newer than HEAD and
already serving is adopted, and anything else is rebuilt.
**A failed build is never deployed.** The build runs before anything restarts,
and the previous build is set aside as `.next-prev` first. A failed build is
rolled back to it, so the box is left serving the previous build and a later
restart or reboot still has one. A failure is retried 15 minutes later, then
hourly until it succeeds or a new commit lands. `scripts/.update-failed` records
which step failed and when the next try is due.
### Checking it is alive
@@ -378,19 +397,27 @@ cron-restart process as `stopped` with a restart count of `0` even while it is
firing on schedule. Neither is evidence of a problem; `.last-check` is the
signal to trust.
Everything that actually happens is appended to `scripts/update.log`. To see
what it would do without touching anything:
While a worker is running, `.last-check` says `update in progress` with its pid.
After a failure it says `waiting to retry` with the time of the next attempt.
Everything that actually happens is appended to `scripts/update.log`. The full
npm and build output of the latest worker run is in `scripts/update-worker.log`,
which is the first place to look when an update failed. To see what it would do
without touching anything:
```powershell
node scripts\update-check.mjs --dry-run
```
To apply an update immediately rather than waiting for the next check:
To apply an update immediately in the foreground, bypassing the retry wait:
```powershell
node scripts\update-check.mjs
node scripts\update-check.mjs --apply
```
Without `--apply` it does what the schedule does: it checks, starts a worker if
there is anything to do, and returns straight away.
### What reloads itself, and what does not
- **The extension** updates itself now. `/api/autobuyer/status` reports the