Add scrollToLoad and an absent variant of waitFor

scrollToLoad walks a progressively-loading list to the bottom before the steps
that act on its items run. Stopping is two-part: no new matches appeared AND the
container was already pinned to the bottom — counting alone stops early on a slow
fetch. Hitting the scroll cap is reported rather than passed off as done, so a
later step never works quietly on a partial list.

The scrolling element is usually not the window. Lists like this live in a div
with its own overflow, and scrolling the document does nothing at all, so the
step walks up from a matched item to the ancestor that actually scrolls —
overflow allows it and there is more content than fits — with containerSelector
to name one outright when the guess is wrong. Verified against a page whose
document also scrolls, which is the case that tells the two apart: it found the
inner div and pulled 12 items up to 60 in 7 scrolls.

waitFor gains `absent`, for waiting on something to go rather than arrive — a
modal closing after a reset. It only accepts a genuine "selector matched
nothing"; an unreachable extension looks the same from a distance and would
otherwise satisfy the gate for the wrong reason, sending the next iteration into
a page that still has the modal open.

The locate queue carries a free-form options blob now, so a new kind of request
stops meaning a new column each time.

Also fixes a missing comma in the reset flow that broke the build.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Brandon Li
2026-08-30 14:31:36 -05:00
co-authored by Claude Opus 5
parent 3ac9fe060f
commit 3cc7ddcc5c
8 changed files with 190 additions and 23 deletions
+13 -4
View File
@@ -394,6 +394,7 @@ export interface LocateRow {
url_pattern: string;
open_url: string;
navigate_url: string;
options: string; // JSON, per-request extras (scroll parameters, ...)
status: 'pending' | 'claimed' | 'done' | 'error';
result: string | null;
error: string | null;
@@ -403,10 +404,10 @@ export interface LocateRow {
const LOCATE_HISTORY = 20;
export function createLocateRequest(selector: string, matchIndex: number, urlPattern: string, openUrl = '', navigateUrl = ''): LocateRow {
export function createLocateRequest(selector: string, matchIndex: number, urlPattern: string, openUrl = '', navigateUrl = '', options = '{}'): LocateRow {
const res = db.prepare(
'INSERT INTO autobuyer_locate (selector, match_index, url_pattern, open_url, navigate_url, created_at) VALUES (?, ?, ?, ?, ?, ?)'
).run(selector, matchIndex, urlPattern, openUrl, navigateUrl, Date.now());
'INSERT INTO autobuyer_locate (selector, match_index, url_pattern, open_url, navigate_url, options, created_at) VALUES (?, ?, ?, ?, ?, ?, ?)'
).run(selector, matchIndex, urlPattern, openUrl, navigateUrl, options, Date.now());
db.prepare(`
DELETE FROM autobuyer_locate
@@ -459,6 +460,14 @@ db.exec(`
);
`);
// Migration: free-form per-request options, so a new kind of request doesn't
// need a new column each time.
try {
db.exec("ALTER TABLE autobuyer_locate ADD COLUMN options TEXT NOT NULL DEFAULT '{}'");
} catch {
// Column already exists
}
// Migration: the page to send the tab to before locating.
try {
db.exec("ALTER TABLE autobuyer_locate ADD COLUMN navigate_url TEXT NOT NULL DEFAULT ''");
@@ -604,7 +613,7 @@ export const RUNNER_TIMEOUT_MS = 7000;
/** The runner version this server's step vocabulary requires. A running process
* doesn't reload when the source changes, so an older one silently fails on
* steps it predates — the dashboard warns instead. */
export const RUNNER_EXPECTED_VERSION = '0.14.0';
export const RUNNER_EXPECTED_VERSION = '0.16.0';
export interface RunnerHeartbeat {
at: number;