senofy/autofirmer-expanded
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Pre-roll fills stay visible; holds survive restarts and judge fund amounts

Browse Source 
Second review round (e496404, 9b2d677):

- Tradovate: past the 5 PM roll an old-dated cash record counts only what
  it gained since the roll (its realizedPnL as last seen while current is
  kept as the roll base), so tonight's fills booked onto it before
  Tradovate rolls the account stay visible — no re-trading a one-and-done
  account. A record this process never saw before the roll leaves the
  account waiting for the broker's roll (ROLL_PENDING, counted in play for
  stop-after-all). The 60s snapshot carries the roll base and can't undo a
  newer record the socket delivered.
- A wipe's hold is kept in account_meta, so a restart onto a failing Cash
  History still holds the account.
- A Cash History whose pre-request balance the last vouched-for baseline
  can't explain, and which booked nothing, holds the account instead of
  rebasing onto an unbooked payout.
- Holds compare fund transactions by date and amount (a second payout on a
  date that already has one), and an in-flight hold is judged against the
  rows from before its report.
- The saved baseline tracks the day's realized P&L and folds, so a restart
  judges against current figures; held refetches back off to an hour, and
  a balance hold no Cash History can check is released after an hour.
- TopstepX: a fetched account is promoted only by a sync whose
  reconcileFunds actually checked the books (not one skipped for an open
  position), and a direct fetch awaits that sync, so wipe routes report
  the real outcome.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Li
2026-09-24 18:45:45 -05:00
co-authored by Claude Opus 5.5
parent 9b2d677662
commit 088886ed73
4 changed files with 204 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/tradovate-class.ts
+175 -92
View File
@@ -118,15 +118,36 @@ function reconnectBackoffMs(failures: number): number {
}
interface CashBaseline { dayStart: number; realized: number }
interface HistoryHold { reason: 'balance' | 'wipe'; fundCount: number; since: number; checks: number; retryAt: number }
/** fundSig: the fund transactions (date:amount) held when the hold began — a
* Cash History "books" the change when they differ. checks counts Cash
* History reports, attempts every refetch. */
interface HistoryHold { reason: 'balance' | 'wipe'; fundSig: string; since: number; checks: number; attempts: number; retryAt: number }
/** A held account's refetch pace. */
/** A held account's first refetch comes after this, then doubling to an hour. */
const HOLD_RETRY_MS = 5 * 60_000;
const HOLD_RETRY_MAX_MS = 60 * 60_000;
/** A balance hold no Cash History ever explains with a fund transaction — a
* fee, or an adjustment the report files as trading — is let go after this
* many clean reports spanning at least HOLD_RELEASE_MS, with a warning. */
const HOLD_MAX_CHECKS = 3;
const HOLD_RELEASE_MS = 15 * 60_000;
/** One no Cash History can ever check (the account's report fails for good —
* a passed account's 404s) is let go after this long, with a warning. */
const HOLD_UNCHECKED_RELEASE_MS = 60 * 60_000;
/** account_meta keys: the baseline a Cash History vouched for, and a wipe's
* hold (so a restart onto a failing report keeps it). */
const BASELINE_META = 'cash_baseline';
const HOLD_META = 'history_hold';
function holdRetryMs(attempts: number): number {
return Math.min(HOLD_RETRY_MAX_MS, HOLD_RETRY_MS * 2 ** Math.max(0, attempts - 1));
}
/** Fund transactions as one comparable value: a second payout on a date that
* already has one changes the amount, not the row count. */
function fundSignature(txns: { date: string; amount: number }[] | undefined): string {
return (txns ?? []).map((f) => `${f.date}:${f.amount}`).join('|');
}
/** Whether cash is still what the baseline accounts for: the same start-of-day
* balance ('same'), or the daily reset having folded the day's realized P&L
@@ -147,31 +168,55 @@ interface CashBalanceEntity {
archived?: boolean;
}
/** A stored cash record. realizedPnL and pendingRoll are judged when read. */
interface StoredCash {
amount: number;
realizedPnL: number;
tradeDate: string | null;
archived: boolean;
/** The record's realizedPnL as last seen while its trade date was the
* current one — the ended day's final figure once the 5 PM roll passes. */
rollBase: number | undefined;
/** Past the roll with no rollBase seen (the process started after it):
* today's share of the record can't be told apart, so the account waits
* for Tradovate to roll it. */
pendingRoll: boolean;
}
/**
* An account's cash as the plan must read it. Tradovate rolls each account's
* cash record to the new trade date on its own schedule — archiving the old
* record and opening one with realizedPnL 0 — well after the 17:00 CT roll
* (18 minutes on 2026-09-24), or not until the account first trades. Until
* then the latest record still carries the ended day's realizedPnL, which the
* plan would read as today's: legs sized off yesterday, one-and-done accounts
* read as already traded, the entry-commission delta swallowing a whole day's
* P&L. That day is over, and its P&L is already in amount: nothing is
* realized today yet.
* (17-21 minutes on 2026-09-24). Until then the latest record still carries
* the ended day's realizedPnL, which the plan would read as today's: legs
* sized off yesterday, one-and-done accounts read as already traded, the
* entry-commission delta swallowing a whole day's P&L. And tonight's own
* fills before the roll may be booked onto that same old record. So past the
* roll an old-dated record's realized P&L counts only what it gained since
* the roll (recorded - rollBase); its amount already holds the ended day.
*/
function todaysCash(cb: CashBalanceEntity): { amount: number; realizedPnL: number; tradeDate: string | null; archived: boolean } {
function todaysCash(cb: CashBalanceEntity, prev: StoredCash | undefined): StoredCash {
const t = cb.tradeDate;
const tradeDate = t?.year
? `${t.year}-${String(t.month).padStart(2, '0')}-${String(t.day).padStart(2, '0')}`
: null;
const recorded = cb.realizedPnL;
// Seen while current: this is the base. Seen past the roll: keep the base
// the same record carried before it, if this process saw one.
const current = tradeDate === null || tradeDate >= currentTradeDateCT();
const rollBase = current ? recorded : prev && prev.tradeDate === tradeDate ? prev.rollBase : undefined;
const ended = () => tradeDate !== null && tradeDate < currentTradeDateCT();
return {
amount: cb.amount,
tradeDate,
archived: cb.archived === true,
// Judged when read, not when stored: a record stored at 16:59 has
// ended by 17:01, whether or not a sync has come in since.
rollBase,
get realizedPnL(): number {
return tradeDate !== null && tradeDate < currentTradeDateCT() ? 0 : recorded;
if (!ended()) return recorded;
return rollBase === undefined ? 0 : Math.round((recorded - rollBase) * 100) / 100;
},
get pendingRoll(): boolean {
return ended() && rollBase === undefined;
},
};
}
@@ -179,17 +224,21 @@ function todaysCash(cb: CashBalanceEntity): { amount: number; realizedPnL: numbe
/**
* Store a cash record unless the one already held is newer. At the roll the
* snapshot and the socket deliver the ended day's record (archived) and the
* new one together, in either order; the older must never win back.
* new one together, in either order; the older must never win back. carried
* is the entry a fresh snapshot map replaces: it hands on the roll base, and
* a snapshot requested before the socket delivered a newer record can't
* undo it.
*/
function storeCash(
into: { [accountId: number]: { amount: number; realizedPnL: number; tradeDate?: string | null; archived?: boolean } },
cb: CashBalanceEntity,
): void {
const next = todaysCash(cb);
const prev = into[cb.accountId];
if (prev?.tradeDate && next.tradeDate) {
if (next.tradeDate < prev.tradeDate) return;
if (next.tradeDate === prev.tradeDate && next.archived && !prev.archived) return;
function storeCash(into: { [accountId: number]: StoredCash }, cb: CashBalanceEntity, carried?: StoredCash): void {
const held = into[cb.accountId] ?? carried;
const next = todaysCash(cb, held);
if (held?.tradeDate && next.tradeDate) {
const older = next.tradeDate < held.tradeDate
|| (next.tradeDate === held.tradeDate && next.archived && !held.archived);
if (older) {
into[cb.accountId] ??= held;
return;
}
}
into[cb.accountId] = next;
}
@@ -220,17 +269,9 @@ export class TradovateClient {
private deviceId = randomUUIDV4();
public accountList: AccountItem[] = [];
public accountCashBalances: {
[accountId: number]: {
amount: number;
/** Today's realized P&L — see todaysCash. */
realizedPnL: number;
/** The trade date of the Tradovate record it came from (YYYY-MM-DD). */
tradeDate?: string | null;
/** That record was superseded at a roll. */
archived?: boolean;
};
} = {};
/** Per account, its latest cash record — today's realized P&L judged at
* read time (see todaysCash). */
public accountCashBalances: { [accountId: number]: StoredCash } = {};
public positions: {
[accountId: number]: {
contractId: number;
@@ -921,7 +962,9 @@ export class TradovateClient {
});
const balances: TradovateClient['accountCashBalances'] = {};
for (const item of (response.cashBalances ?? []) as CashBalanceEntity[]) storeCash(balances, item);
for (const item of (response.cashBalances ?? []) as CashBalanceEntity[]) {
storeCash(balances, item, this.accountCashBalances[item.accountId]);
}
this.accountCashBalances = balances;
this.positions = (response.positions ?? [])
@@ -1234,6 +1277,7 @@ export class TradovateClient {
// while the report is in flight isn't absorbed unexplained.
const cashNow = this.accountCashBalances[account.id];
const cashBefore = cashNow ? { amount: cashNow.amount, realizedPnL: cashNow.realizedPnL } : null;
const fundSigBefore = fundSignature(this.fundTransactions[account.id]);
// --- Try Cash History first (true after-fee daily P&L) ---
try {
@@ -1319,7 +1363,7 @@ export class TradovateClient {
this.daysTraded[account.id] = active.filter((d) => d.pnl !== 0).length;
saveDailyPnL(account.id, account.name, fresh); // upsert only fresh entries — preserves older cached rows
delete this.lastFetchErrors[account.name];
this.markHistoryLoaded(account.id, true, cashBefore);
this.markHistoryLoaded(account.id, true, cashBefore, fundSigBefore);
continue;
} catch {
// fall through to Fills fallback
@@ -1427,10 +1471,11 @@ export class TradovateClient {
/** Drop the account's history from memory (a wipe): it reads unloaded, and
* stays out of trading, until a Cash History rebuilds it — the wipe took
* its fund transactions, which neither the Fills report nor the cache can
* bring back. */
* bring back. Kept on record too, so a restart doesn't forget it. */
public forgetHistory(accountId: number, accountName: string): void {
this.historyLoadedIds.delete(accountId);
this.historyHolds.set(accountId, { reason: 'wipe', fundCount: 0, since: Date.now(), checks: 0, retryAt: 0 });
this.hold(accountId, 'wipe', '');
saveAccountMeta(accountId, HOLD_META, 'wipe');
delete this.balanceBaseline[accountId];
delete this.dailyPnL[accountId];
delete this.fullDailyPnL[accountId];
@@ -1442,22 +1487,32 @@ export class TradovateClient {
delete this.lastFetchRaw[accountName];
}
private hold(accountId: number, reason: HistoryHold['reason'], fundSig: string, retryAt = 0): void {
this.historyHolds.set(accountId, { reason, fundSig, since: Date.now(), checks: 0, attempts: 0, retryAt });
}
private accountName(accountId: number): string | number {
return this.accountList.find((a) => a.id === accountId)?.name ?? accountId;
}
/**
* A fetch pass reached the account. Cash History is the only source that
* books fund transactions, so only it can lift a hold, and a balance hold
* only once it books a fund transaction the account didn't have when the
* hold began. The Fills fallback and the DB cache still count as loaded.
* books fund transactions, so only it can lift a hold — a balance hold
* only once its fund transactions differ from when the hold began. The
* Fills fallback and the DB cache still count as loaded.
*
* cashBefore is the balance when the Cash History was requested: the
* baseline comes from it, not from the balance when the report lands, so
* a payout posting in between is still caught — and if the balance did
* move in between, this report may not have seen it, so the account is
* held for the next one.
* cashBefore / fundSigBefore are the balance and fund transactions when
* the Cash History was requested. The balance is judged against the
* baseline this history last vouched for: if it moved and this report
* booked nothing, the move is held for a later report rather than taken
* as the new baseline; if it moved while the report was in flight, this
* report may not have seen it, so it is held for the next one.
*/
private markHistoryLoaded(
accountId: number,
fromCashHistory: boolean,
cashBefore: { amount: number; realizedPnL: number } | null = null,
fundSigBefore = '',
): void {
if (this.dailyPnL[accountId] === undefined || this.fundTransactions[accountId] === undefined) {
this.historyLoadedIds.delete(accountId);
@@ -1465,80 +1520,108 @@ export class TradovateClient {
}
this.historyLoadedIds.add(accountId);
const now = Date.now();
if (!this.historyHolds.has(accountId) && loadAccountMeta(accountId, HOLD_META) === 'wipe') {
this.hold(accountId, 'wipe', ''); // a wipe from before a restart
}
const hold = this.historyHolds.get(accountId);
if (!this.balanceBaseline[accountId]) {
const saved = this.savedBaseline(accountId);
if (saved) this.balanceBaseline[accountId] = saved;
}
if (!fromCashHistory) {
if (hold) {
hold.retryAt = now + HOLD_RETRY_MS;
hold.attempts++;
hold.retryAt = now + holdRetryMs(hold.attempts);
if (hold.reason === 'balance' && now - hold.since >= HOLD_UNCHECKED_RELEASE_MS) {
console.warn(`${this.tag} ${this.accountName(accountId)}: its balance change stayed unchecked for`
+ ` ${Math.round((now - hold.since) / 60_000)}m — Cash History never came back; releasing it`);
this.historyHolds.delete(accountId);
this.setBaseline(accountId, this.accountCashBalances[accountId] ?? null);
}
return;
}
// Keep the baseline a Cash History set — this process's, or the
// last one on record (a restart onto a failing report server) —
// rather than take today's balance, which may hold an unbooked
// payout; the drift check then judges it.
if (!this.balanceBaseline[accountId]) {
const saved = loadAccountMeta(accountId, 'cash_baseline');
try {
const parsed = saved ? JSON.parse(saved) as CashBaseline : null;
if (parsed && Number.isFinite(parsed.dayStart) && Number.isFinite(parsed.realized)) {
this.balanceBaseline[accountId] = parsed;
}
} catch { /* unreadable: judged from the live balance below */ }
}
// last one on record — rather than take today's balance, which may
// hold an unbooked payout; the drift check then judges it.
this.balanceDrifted(accountId);
return;
}
const fundSigNow = fundSignature(this.fundTransactions[accountId]);
const cashAfter = this.accountCashBalances[accountId];
const before = cashBefore ?? cashAfter ?? null;
const base: CashBaseline | null = before
? { dayStart: before.amount - before.realizedPnL, realized: before.realizedPnL }
: null;
const movedDuringFetch = base !== null && cashAfter !== undefined && cashExplained(base, cashAfter) === null;
const name = this.accountList.find((a) => a.id === accountId)?.name ?? accountId;
if (hold) {
hold.checks++;
const booked = (this.fundTransactions[accountId]?.length ?? 0) > hold.fundCount;
if (hold.reason === 'balance' && !booked) {
hold.attempts++;
if (hold.reason === 'balance' && fundSigNow === hold.fundSig) {
if (hold.checks < HOLD_MAX_CHECKS || now - hold.since < HOLD_RELEASE_MS) {
hold.retryAt = now + HOLD_RETRY_MS;
hold.retryAt = now + holdRetryMs(hold.attempts);
return;
}
console.warn(`${this.tag} ${name}: ${hold.checks} Cash History reports over`
console.warn(`${this.tag} ${this.accountName(accountId)}: ${hold.checks} Cash History reports over`
+ ` ${Math.round((now - hold.since) / 60_000)}m booked no fund transaction for its balance change —`
+ ' releasing it (a fee, or an adjustment the report files as trading)');
// Nothing will explain it: take the balance as it now stands.
this.historyHolds.delete(accountId);
this.setBaseline(accountId, cashAfter ?? null);
return;
}
// Booked (balance), or rebuilt from Cash History (wipe).
this.historyHolds.delete(accountId);
if (hold.reason === 'wipe') saveAccountMeta(accountId, HOLD_META, '');
this.setBaseline(accountId, before);
return;
}
if (movedDuringFetch) {
this.historyHolds.set(accountId, {
reason: 'balance',
fundCount: this.fundTransactions[accountId]?.length ?? 0,
since: now, checks: 0, retryAt: now + 60_000,
});
this.balanceBaseline[accountId] = base!;
console.warn(`${this.tag} ${name}: balance moved while its Cash History was in flight — held for the next report`);
// A balance the last vouched-for baseline can't explain, with nothing
// booked by this report: a payout it hasn't caught up with yet.
const known = this.balanceBaseline[accountId];
if (known && before && cashExplained(known, before) === null && fundSigNow === fundSigBefore) {
this.hold(accountId, 'balance', fundSigBefore, now + 60_000);
console.warn(`${this.tag} ${this.accountName(accountId)}: balance moved $`
+ `${Math.round((before.amount - before.realizedPnL - known.dayStart) * 100) / 100}`
+ ' and its Cash History booked nothing yet — held for a later report');
return;
}
const base = before ? { dayStart: before.amount - before.realizedPnL, realized: before.realizedPnL } : null;
if (base && cashAfter && cashExplained(base, cashAfter) === null) {
// Held against the fund transactions from before this report, so
// the next report lifts it if this one already booked the change.
this.hold(accountId, 'balance', fundSigBefore, now + 60_000);
this.balanceBaseline[accountId] = base;
console.warn(`${this.tag} ${this.accountName(accountId)}: balance moved while its Cash History was in flight — held for the next report`);
return;
}
this.setBaseline(accountId, before);
}
/** Set (and keep on record, for a restart that can't reach Cash History)
* the baseline a Cash History vouched for. */
/** The baseline last vouched for, as kept on record across restarts. */
private savedBaseline(accountId: number): CashBaseline | null {
try {
const saved = loadAccountMeta(accountId, BASELINE_META);
const parsed = saved ? JSON.parse(saved) as Partial<CashBaseline> : null;
if (parsed && Number.isFinite(parsed.dayStart) && Number.isFinite(parsed.realized)) {
return { dayStart: parsed.dayStart!, realized: parsed.realized! };
}
} catch { /* unreadable: none */ }
return null;
}
/** Set the baseline and keep it on record, so a restart that can't reach
* Cash History judges the balance against it. */
private setBaseline(accountId: number, cash: { amount: number; realizedPnL: number } | null): void {
if (!cash) {
delete this.balanceBaseline[accountId];
return;
}
const base = { dayStart: cash.amount - cash.realizedPnL, realized: cash.realizedPnL };
this.balanceBaseline[accountId] = base;
saveAccountMeta(accountId, 'cash_baseline', JSON.stringify(base));
this.balanceBaseline[accountId] = { dayStart: cash.amount - cash.realizedPnL, realized: cash.realizedPnL };
this.saveBaseline(accountId);
}
private saveBaseline(accountId: number): void {
const base = this.balanceBaseline[accountId];
if (base) saveAccountMeta(accountId, BASELINE_META, JSON.stringify({ ...base, tradeDate: currentTradeDateCT() }));
}
/**
@@ -1549,33 +1632,33 @@ export class TradovateClient {
* amount and realizedPnL together, and the daily reset folds the day's
* realized P&L into the balance; any other move past a few dollars holds
* the account until a Cash History books it (maybeFetchDaysTraded fetches
* it on the next sync).
* it on the next sync). What the baseline learns is kept on record, so a
* restart judges against the day's last realized P&L, not a stale one.
*/
private balanceDrifted(accountId: number): boolean {
const cash = this.accountCashBalances[accountId];
if (!cash) return false;
const base = this.balanceBaseline[accountId];
if (!base) {
this.balanceBaseline[accountId] = { dayStart: cash.amount - cash.realizedPnL, realized: cash.realizedPnL };
this.setBaseline(accountId, cash);
return false;
}
const how = cashExplained(base, cash);
if (how === 'same') {
base.realized = cash.realizedPnL;
if (Math.abs(cash.realizedPnL - base.realized) >= 1) {
base.realized = cash.realizedPnL;
this.saveBaseline(accountId);
}
return false;
}
if (how === 'folded') {
this.balanceBaseline[accountId] = { dayStart: cash.amount - cash.realizedPnL, realized: cash.realizedPnL };
this.setBaseline(accountId, cash);
return false;
}
if (!this.historyHolds.has(accountId)) {
this.historyHolds.set(accountId, {
reason: 'balance',
fundCount: this.fundTransactions[accountId]?.length ?? 0,
since: Date.now(), checks: 0, retryAt: 0,
});
const name = this.accountList.find((a) => a.id === accountId)?.name ?? accountId;
console.warn(`${this.tag} ${name}: balance moved $${Math.round((cash.amount - cash.realizedPnL - base.dayStart) * 100) / 100}`
this.hold(accountId, 'balance', fundSignature(this.fundTransactions[accountId]));
console.warn(`${this.tag} ${this.accountName(accountId)}: balance moved`
+ ` $${Math.round((cash.amount - cash.realizedPnL - base.dayStart) * 100) / 100}`
+ ' that its history does not explain — held out of trading until Cash History books it');
}
return true;