Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Timeline extract finishes on pages with many large saved versions (#9362) (#9375)
Garry Tane87bfea59dcc
diff --git a/changelog.d/fix-timeline-retract-version-scan.md b/changelog.d/fix-timeline-retract-version-scan.mdnew file mode 100644index 0000000000..43fc9eea5c--- /dev/null+++ b/changelog.d/fix-timeline-retract-version-scan.md@@ -0,0 +1,6 @@+**`gbrain extract timeline --source db` no longer fails on a large page with many stored versions on a managed Postgres brain.**++Retraction used to read every stored version of the page in one statement inside the publication transaction. With hundreds of versions that statement ran past the 5 second statement timeout on every retry, so the page's timeline extract failed with `write_interrupted` on every maintenance run. The version scan now runs before the transaction and reads one version at a time. Inside the transaction gbrain only checks that the versions it read are still the page's, with the same text. A row whose versions changed in between stays until the next run. Reported in #9362.++### Fixed+- `gbrain extract timeline --source db` and the maintenance timeline pass no longer time out with `write_interrupted` on a page with hundreds of stored versions (#9362).diff --git a/docs/architecture/key-files/commands-3-continued.md b/docs/architecture/key-files/commands-3-continued.mdindex 0d72154aff..0b33fcbd65 100644--- a/docs/architecture/key-files/commands-3-continued.md+++ b/docs/architecture/key-files/commands-3-continued.md@@ -5,7 +5,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db] [--source-id <id>]`: batch link/timeline extraction. A lost link or timeline write on the fs walks or the slug-scoped path leaves its pages unstamped and throws `extract_write_lost` after stamping the rest (`core/extract-watermark.ts`); the sync hooks report a page whose link or timeline write failed in `errors`, never `processed`, so the caller leaves it unstamped. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use for live brains with no local checkout). The full fs walk replaces each page's own markdown-derived links through `replacePageFileLinks` (the per-slug contract below; a file with no page row is skipped and not stamped); `extract --stale` replaces through `replaceDerivedLinks`, the db link walk through `replaceDerivedLinksBatch`. Timeline rows are inserted in batches of 100 via `addTimelineEntriesBatch` (`ON CONFLICT DO NOTHING`; `created` counts real inserts) after every path — fs walk, db walk, per-slug, sync hooks, `--stale` — runs `retractRemovedTimelineEntries` for the page. `ExtractOpts.slugs?: string[]` enables incremental extract via `extractForSlugs()` (single combined links+timeline pass); the cycle path threads sync's `pagesAffected` through. Every per-slug extractor — `extractForSlugs` (the cycle's incremental pass) and the two sync hooks `extractLinksForSlugs` / `extractTimelineForSlugs` — resolves each requested slug through `resolveSlugRelPath`: the `buildSlugPathIndex(walkMarkdownFiles(dir))` index first, then — only on an index miss — the legacy `join(dir, slug + '.md')` existence check. The index is the primary because `pathToSlug` lowercases and slugifies, so a reconstructed path only finds files whose names are already slugs; the legacy fallback stays because sync ADMITS files the walker never emits (`_`-prefixed names, dot-dirs waived via `sync.include_hidden`) and those slugs still round-trip to their real path — without it such a page imports with no edges. On a slug collision the index prefers the file whose name IS the slug (else the first walked entry), so resolution never depends on read order. The per-slug extractors reconcile like `put_page`: each page's own markdown-derived links are replaced through `replacePageFileLinks` (`src/core/link-reconciliation.ts`, `replaceDerivedLinks` with `preserveExisting`; other producers survive, a page with no row counts as processed with no links, and a failed replacement is reported on stderr and leaves the page stale), and `retractRemovedTimelineEntries` (`src/core/timeline-extract.ts`) deletes the non-event rows whose (date, source, summary) any stored `page_versions` text of the page produced (under either timeline parser) and its current text no longer does; rows no version produced (enrichment, meeting fan-out, `--infer-dates` anchors) are kept, except an extractor row an older reading of that text filed (`supersededTimelineReadings`, e.g. the pre-#3280 bullet split, #7949), and the version scan only runs when the page holds a row the current text does not produce. `gbrain extract timeline --prune-orphans [--source-id] [--dry-run] [--json]` (`pruneTimelineOrphans`) runs that reconciliation over every page with timeline rows, without inserting — the one-time prune for rows orphaned while extraction was insert-only (managed brains retract each page's rows in a page-locked `withCoordinatedWrite` transaction; these rows are removals, so `gbrain repair timeline` / `timeline_history` exclude them; pinned in `test/persistence-managed-lifecycle.test.ts`); the read-only `timeline_orphans` doctor check (`src/commands/doctor/checks/timeline-orphans.ts`) samples 500 pages and points at it. Pinned by `test/timeline-reconcile-all-paths.test.ts`. `loadSlugAliasTargets` adds renamed pages' old slugs (`slug_aliases`) to `allSlugs` so links written before a rename resolve to the current slug. `extractLinksForSlugs` reads the requested files first, then loads page metadata only for those pages and their link endpoints (`fileLinkOwnership(engine, sourceId, { slugs })`); the unscoped whole-brain load happens only when a DB meeting or attendance origin needs `replaceFileLinks`. Both return `ExtractForSlugsResult { created, processed, errors? }`; `errors` names pages whose read or write failed, and sync reports them as `extract_error` (stderr line plus the `SyncResult` field) instead of swallowing them. `processed` is the subset actually read, and the shared `slugsSafeToStamp(links, timeline)` intersection is what every caller may stamp. Stamping the full requested set marks a silently-skipped page fresh and hides it from `extract --stale` permanently. All three call sites (`src/commands/sync.ts`, `src/core/github-source.ts`, `src/core/google/google-source.ts`) stamp the intersection. `walkMarkdownFiles(brainDir)` still runs to build `allSlugs` for link resolution. `--source-id <id>` scopes extraction to one source on federated brains (resolved via `resolveSourceWithTier()` before any SQL; failures hint `gbrain sources list`). `gbrain extract --stale [--source-id <id>] [--catch-up] [--retry-held] [--dry-run] [--json]` branch (`extractStaleFromDB`; on a managed brain it hands the sweep to `extractManagedStaleLinks` in `src/core/persistence/links-maintenance.ts` for every caller, CLI, cycle, jobs and maintain, and a live PGLite owner runs it through `src/commands/extract-stale-delegate.ts`; failing pages are reported and skipped there, and `--retry-held` clears held ones first) — incremental DB-source link+timeline sweep over pages whose `pages.links_extracted_at` watermark is stale, then the mention pass. Embedded callers (the cycle) pass `quiet: true` so the helper writes nothing to stdout, including its JSON result; standalone CLI output is unchanged. In `jsonMode` every fs path, `extractForSlugs` included, reports a dropped flush as a `{event:'batch_error'}` stderr line rather than going silent. Stale predicate (shared by both engines + the doctor check): `links_extracted_at IS NULL OR links_extracted_at < LINK_EXTRACTOR_VERSION_TS::timestamptz OR updated_at > links_extracted_at` (the `updated_at` arm catches MCP `put_page` / `sync --no-extract` edited-since-extract). Three `BrainEngine` methods (parity in postgres-engine.ts + pglite-engine.ts + bootstrap probes): `countStalePagesForExtraction(opts?)`, `listStalePagesForExtraction({batchSize, afterPageId?, sourceId?, versionTs?})` (returns page CONTENT to avoid N+1 `getPage`; `rowToStalePage` in utils.ts maps the row, `StalePageRow` in types.ts), `markPagesExtractedBatch(refs, defaultExtractedAt)` (3-array unnest `slug[],source_id[],ts[]`; each ref may carry its own `extractedAt`). `STALE_BATCH_SIZE` default 25 (`GBRAIN_EXTRACT_STALE_BATCH`; small because page bodies are unbounded — the LIMIT is the only fetch-time memory bound); `STALE_TIME_BUDGET_MS` 30min wall-clock (`--catch-up` removes the cap). Non-swallowing flush (the unmanaged sweep): a page's throw aborts the sweep after the batch's finished pages are flushed and stamped (S4), so the failing page and later ones stay stale and re-extract idempotently. `extractStaleFromDB` stamps with each row's READ `updated_at` (not `now()`), so a concurrent edit during the sweep keeps the page stale and it re-extracts next run rather than marked fresh-with-old-content. On a managed-persistence brain each page publishes its derived links (`prepareAutomaticLinks`), the coordinator's unrecorded canonical timeline tuples (`unrecordedCanonicalTimeline`, insert-only) and its watermark in one revision-bound `withCoordinatedWrite` transaction; a page edited since the read is skipped and stays stale. Tests: `test/extract-stale-managed.test.ts`, `test/persistence-managed-lifecycle.test.ts`. Source-correct stamping at DB-extract sites via `stampExtracted` (best-effort, never throws); `extractLinksFromDB` only stamps the combined watermark when `subcommand === 'all'` (a links-only run must not hide timeline staleness). `LINK_EXTRACTOR_VERSION_TS` lives in `src/core/link-extraction.ts` (bump like `CHUNKER_VERSION` to invalidate all prior stamps). Migration v112 (`pages_links_extracted_at`) adds nullable `TIMESTAMPTZ` + composite `(source_id, links_extracted_at)` index (CONCURRENTLY + invalid-remnant pre-drop on Postgres, plain on PGLite), NO backfill so the real backlog surfaces on first `gbrain doctor`. Fresh-install schema in `src/schema.sql` (both generated blobs follow via `bun run build:schema`) + `REQUIRED_BOOTSTRAP_COVERAGE`. `src/commands/doctor/checks/extraction-sync.ts:checkLinksExtractionLag` (the `links_extraction_lag` check, also in `doctorReportRemote`) warn-only by default (>`GBRAIN_EXTRACTION_LAG_WARN_PCT`, default 20%; shared `EXTRACTION_LAG_WARN_PCT_DEFAULT` + `EXTRACTION_LAG_MIN_PAGES=100` + exported `_resolveEnvNumber`), hard-fails only when `GBRAIN_EXTRACTION_LAG_FAIL_PCT` is set; vacuous-skips <100 pages (no `--source`); pre-v112 brains graceful-skip via `isUndefinedColumnError`; pages whose attendance marker (`links_attendance_blocked_revision`, migration v180) equals their current `knowledge_revision` are left out of the lag count and reported as `details.attendance_blocked` (both `extract --stale` sweeps set the marker on an unresolved-attendance skip; pre-v180 brains fall back to the unfiltered count); strictly a SQL COUNT (safe on remote/thin-client). `src/commands/sync.ts` carries `--no-extract` (threaded through single-source + `--all` + `syncOneSource`), stamps `links_extracted_at` at the inline-extract call site for the slugs the extractors actually read, and `maybeExtractionNudge` prints a one-line stderr nudge after a `synced | first_sync | up_to_date` sync that leaves a backlog (`shouldNudgeAfterSync` pure predicate; `GBRAIN_SYNC_NO_EXTRACT_NUDGE` suppresses). `src/core/retry.ts` lists `'extract.stale'` in `BATCH_AUDIT_SITES`; `src/core/doctor-categories.ts` lists `links_extraction_lag` in `BRAIN_CHECK_NAMES`. Pinned by `test/extract-stale.test.ts` (incl. edited-after-stamp + crash-contract), `test/sync-inline-extract-stamps.test.ts`, `test/sync-nudge-status-gate.test.ts`, `test/doctor-links-extraction-lag.test.ts`, engine-parity (Postgres↔PGLite) for the 3 methods + v112 round-trip. The stale SELECT in both engines projects a deterministic full-µs UTC string `to_char(updated_at AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.US"Z"') AS updated_at_iso` (carried on `StalePageRow.updated_at_iso`, populated by `rowToStalePage` in utils.ts with an ISO-only fallback — never `String(Date)`, which `::timestamptz` misparses); `extractStaleFromDB` stamps that exact-precision value, not a JS `Date` (which truncates to milliseconds), so on Postgres `links_extracted_at` equals the row's `updated_at` to the microsecond and `links_extraction_lag` clears — a ms-truncated stamp stays strictly below the µs `updated_at` and leaves every page perpetually stale, which `extract --stale` could never satisfy. `to_char` (not raw `::text`, which is `DateStyle`-fragile) keeps the projection deterministic. The `markPagesExtractedBatch` SQL is unchanged, so callers passing an explicit (e.g. backdated) `extractedAt` still control the stamp and the edited-since arm is exact. A deterministic PGLite case in `test/extract-stale.test.ts` injects a µs `updated_at`, runs `--stale`, and asserts the lag is 0 and stays 0. W4.14: one basename index per slug set (rebuilt as it grows). GBRA-133 2b.1: the sync hooks (`extractLinksForSlugs` / `extractTimelineForSlugs`) share one `indexMarkdownTree` walk per run (`src/commands/extract-tree.ts`, with `markdownWalk` behind `walkMarkdownFiles`) (it yields after each directory) and type link endpoints through `withEndpointPageTypes`, which parses only the walked files the pass asks a type for (repeating until nothing new is asked) instead of `loadFsPageTypes` over the whole tree; full `gbrain extract` keeps the whole-tree parse, and the two give the same links (`test/pathological/s-sync-offpath-2b.test.ts`). #6805: `extractTimelineForSlugs` writes each page's rows as one batch through `addProjectedPageTimeline` (same tier, write gate and dedup as the per-row path; an absent or soft-deleted page fails the slug with addTimelineEntry's `PageMissingError`); a batch the database refuses is retried row by row through `addProjectedTimelineEntry`, so the page keeps its other rows and is still reported in `errors`. `created` counts rows inserted (`test/extract-timeline-for-slugs-batch.test.ts`).-- `src/commands/extract-timeline-db.ts` — the `extract timeline --source db` walk (`extractTimelineFromDB`), also used by `runExtractCore` for a timeline-only pass on a managed brain. Managed brains (every fresh `gbrain init` brain) publish each page that needs a change as one database-only maintenance request (`managed_maintenance_timeline_extract`, owner-checked by `maintenancePreflight`, request id bound to the page revision; `prepareTimelineExtract` is dispatched from `prepareMaintenanceMutation`): under the page lock it retracts rows the current text no longer produces and inserts the canonical tuples with no stored row (`unrecordedCanonicalTimeline`); a page with nothing to change admits no request, and a page edited meanwhile is skipped. Unmanaged brains batch raw inserts. A refused write prints `refused: <code>; nothing written; existing timeline rows are untouched`, the summary counts written versus refused (a request still pending counts as not confirmed) with `gbrain extract --stale` and the `write-refusals.md#extract-timeline-refused` anchor, and the CLI exits 1 (the `extract` job throws). Pinned by `test/extract-timeline-db-managed.test.ts`, `test/e2e/extract-timeline-db-postgres.test.ts`, `test/fresh-init-db-extract.serial.test.ts`.+- `src/commands/extract-timeline-db.ts` — the `extract timeline --source db` walk (`extractTimelineFromDB`), also used by `runExtractCore` for a timeline-only pass on a managed brain. Managed brains (every fresh `gbrain init` brain) publish each page that needs a change as one database-only maintenance request (`managed_maintenance_timeline_extract`, owner-checked by `maintenancePreflight`, request id bound to the page revision; `prepareTimelineExtract` is dispatched from `prepareMaintenanceMutation`): under the page lock it retracts rows the current text no longer produces and inserts the canonical tuples with no stored row (`unrecordedCanonicalTimeline`); a page with nothing to change admits no request, and a page edited meanwhile is skipped. The retraction's `page_versions` scan runs before the publication transaction, one version body per statement (`versionTimelineScan`, #9362); inside the transaction it only checks that each version it read is still the page's with the same body, and a row whose versions changed in between stays until the next run. Unmanaged brains batch raw inserts. A refused write prints `refused: <code>; nothing written; existing timeline rows are untouched`, the summary counts written versus refused (a request still pending counts as not confirmed) with `gbrain extract --stale` and the `write-refusals.md#extract-timeline-refused` anchor, and the CLI exits 1 (the `extract` job throws). Pinned by `test/extract-timeline-db-managed.test.ts`, `test/e2e/extract-timeline-db-postgres.test.ts`, `test/fresh-init-db-extract.serial.test.ts`.- `src/commands/extract-stale-options.ts` — `gbrain extract --stale` source-option validation, run by `extract-stale-delegate.ts` before every `--stale` invocation connects (so before direct and owner dispatch alike): `--source` is the fs|db read mode, never a source id; `--source db` is accepted, `--source fs` and any other `--source <x>` refuse `invalid_params` (exit 2) with a fix naming `--source-id <x>`. `assertStaleSourceModeUnambiguous` refuses `--source db` as ambiguous when a registered source is named `db` (direct path in `runExtract`; owner path in `writer_extract_stale`, which also takes `catch_up`). Tests: `test/extraction-nudge-6803.test.ts`.diff --git a/src/commands/extract-timeline-db.ts b/src/commands/extract-timeline-db.tsindex e35ba633a7..f3f050c228 100644--- a/src/commands/extract-timeline-db.ts+++ b/src/commands/extract-timeline-db.ts@@ -19,7 +19,7 @@import type { BrainEngine, TimelineBatchInput } from '../core/engine.ts';import type { Page, PageType } from '../core/types.ts';import { parseTimelineEntries, deriveTimelineAnchor } from '../core/link-extraction.ts';-import { eachStoredTimelineRow, retractRemovedTimelineEntries, segmentedTimelineText, storedTimelineKey, TimelineKeySet, unkeptTimelineRows } from '../core/timeline-extract.ts';+import { eachStoredTimelineRow, retractRemovedTimelineEntries, segmentedTimelineText, storedTimelineKey, TimelineKeySet, unkeptTimelineRows, versionTimelineScan, type VersionTimelineScan } from '../core/timeline-extract.ts';import { timelineKey } from '../core/timeline-marker.ts';import { managedPersistenceEnabled } from '../core/persistence/ownership.ts';import { maintenanceTransaction } from '../core/persistence/attribution.ts';@@ -280,8 +280,8 @@ async function readTimelineBatch(engine: BrainEngine, refs: ReadonlyArray<{ slug* deletes the removed rows; `stop` is asked before each segment read.*/async function plannedTimeline(engine: BrainEngine, page: Page, slug: string, sourceId: string, inferDates: boolean, stored?: StoredTimelineRows[],- opts: { retract?: boolean; stop?: () => boolean } = {}) {- const retraction = { dryRun: opts.retract !== true, ...(opts.stop ? { stop: opts.stop } : {}) };+ opts: { retract?: boolean; stop?: () => boolean; versions?: VersionTimelineScan } = {}) {+ const retraction = { dryRun: opts.retract !== true, ...(opts.stop ? { stop: opts.stop } : {}), ...(opts.versions ? { versions: opts.versions } : {}) };const strip = (rows: StoredTimelineRows[]) => rows.map(({ date_text: _text, ...row }) => row);let entries: Array<{ date: string; source?: string; summary: string; detail?: string }>;let removed: unknown[];@@ -321,9 +321,9 @@ async function plannedTimeline(engine: BrainEngine, page: Page, slug: string, soreturn { removed: removed.length, entries };}-/** Under the coordinator's page lock: retract removed rows, insert the planned tuples. */-async function writePageTimeline(tx: BrainEngine, page: Page, slug: string, sourceId: string, inferDates: boolean): Promise<number> {- const plan = await plannedTimeline(tx, page, slug, sourceId, inferDates, undefined, { retract: true });+/** Under the coordinator's page lock: retract removed rows (checked against `versions`, read before the lock), insert the planned tuples. */+async function writePageTimeline(tx: BrainEngine, page: Page, slug: string, sourceId: string, inferDates: boolean, versions: VersionTimelineScan): Promise<number> {+ const plan = await plannedTimeline(tx, page, slug, sourceId, inferDates, undefined, { retract: true, versions });if (!plan.entries.length) return 0;return tx.addTimelineEntriesBatch(plan.entries.map(entry => ({ slug, date: entry.date, source: entry.source,summary: entry.summary, detail: entry.detail || '', source_id: sourceId })), { auditSite: 'extract.timeline_db' });@@ -386,11 +386,17 @@ export async function prepareTimelineExtract(engine: BrainEngine, row: WriteRequif (!snapshot || snapshot.page.id !== Number(row.page_id)) throw timelinePageChanged(row, 'The page was deleted or replaced before its timeline was extracted.');await authorizeWrite(engine, row.authority, 'submit_job', row.slug);const inferDates = (row.intent as { infer_dates?: unknown } | null)?.infer_dates === true;+ // #9362: the page's stored versions are read here, one body per statement, not inside the publication transaction.+ // A segmented page's own snapshots are capped (pruneSegmentedVersions) and read segment by segment there, so its+ // body is planned here only when it also keeps inline snapshots, whose number nothing caps.+ const versions = versionTimelineScan();+ if (!isSegmentedPage(snapshot.page) || (await engine.executeRaw(`SELECT 1 FROM page_versions WHERE page_id = $1 AND body_storage = 'inline' LIMIT 1`,+ [snapshot.page.id])).length) await plannedTimeline(engine, snapshot.page, row.slug, row.source_id, inferDates, undefined, { versions });return { observedRevision: snapshot.revision, noop: true,validate: async tx => { await authorizeWrite(tx, row.authority, 'submit_job', row.slug); },apply: async tx => {const current = await tx.readPageSnapshot(row.slug, { sourceId: row.source_id });if (!current) throw timelinePageChanged(row, 'The page disappeared before its timeline was extracted.');- return { status: 'completed', added: await writePageTimeline(tx, current.page, row.slug, row.source_id, inferDates) };+ return { status: 'completed', added: await writePageTimeline(tx, current.page, row.slug, row.source_id, inferDates, versions) };} };}diff --git a/src/core/timeline-extract.ts b/src/core/timeline-extract.tsindex cdbeb6ae74..68cdda236d 100644--- a/src/core/timeline-extract.ts+++ b/src/core/timeline-extract.ts@@ -16,7 +16,7 @@ import { isDatedTimelineLine, isValidDate, supersededInlineCitationEntries } froimport { createHash } from 'node:crypto';import type { BrainEngine } from './engine.ts';import { firstMaterializedMarkerIndex } from './timeline-marker.ts';-import { bodyBlocks, detachedValue, retainedGenerationBlocks } from './page-body/segment-blocks.ts';+import { BodyReadStopped, bodyBlocks, detachedValue, retainedGenerationBlocks } from './page-body/segment-blocks.ts';export interface ExtractedTimelineEntry {slug: string;@@ -338,6 +338,24 @@ export async function unkeptTimelineRows(engine: Pick<BrainEngine, 'executeRaw'>return out;}+/** A stored version a timeline key was read from, and what must still hold for it to count in a later transaction. */+type VersionWitness = { id: number; storage: 'inline'; digests: [string, string] }+ | { id: number; storage: 'segmented'; generation: number; hash: string | null };++/**+ * What a page's stored versions say about the timeline keys retraction asked about (#9362): for each key, the versions+ * whose text produces it (`produced`) or files it under an older reading (`superseded`), newest first. `done` once+ * filled: a caller fills it with a dry run before its publication transaction, and the retraction inside it then reads no+ * version body, only checks that a version it names is still the page's with the same body.+ */+export interface VersionTimelineScan {+ done: boolean;+ produced: Map<string, VersionWitness[]>;+ superseded: Map<string, VersionWitness[]>;+}++export const versionTimelineScan = (): VersionTimelineScan => ({ done: false, produced: new Map(), superseded: new Map() });+/*** Reconcile a page's timeline rows with its current text: retract every row* an earlier version of the page produced that the current text no longer@@ -345,7 +363,9 @@ export async function unkeptTimelineRows(engine: Pick<BrainEngine, 'executeRaw'>* version of the page ever produced (enrichment, meeting fan-out, inferred* anchors) and event-page projections are never touched. The page_versions* scan only runs when the page holds a row the current text does not produce,- * and keeps only the keys of those rows. A retained segmented generation is read+ * reads one version body per statement and keeps only the keys of those rows.+ * `versions` carries that scan from a dry run before a+ * publication transaction into it (VersionTimelineScan). A retained segmented generation is read* run by run (retainedGenerationBlocks), skipping each run whose key the live body* or an earlier generation already read; `stop` is asked before each of its segment reads.* Returns the orphaned rows; they are deleted unless `dryRun`. `storedRows`@@ -359,33 +379,92 @@ export async function retractRemovedTimelineEntries(slug: string,sourceId: string,current: string | TimelineTextKeys,- opts: { dryRun?: boolean; storedRows?: ReadonlyArray<StoredTimelineTuple>; stop?: () => boolean; expectedRevision?: string } = {},+ opts: { dryRun?: boolean; storedRows?: ReadonlyArray<StoredTimelineTuple>; stop?: () => boolean; expectedRevision?: string; versions?: VersionTimelineScan } = {},): Promise<Array<TimelineTuple & { id: number }>> {const kept = typeof current === 'string' ? markdownTimelineKeys(current, slug) : current.kept;const within = typeof current === 'string' ? undefined : current.within;const rows = opts.storedRows ?? await storedTimelineRows(engine, sourceId, slug);const extra = rows.filter(row => !kept.has(tupleKey(row)) && (!within || within.has(tupleKey(row))));- if (!extra.length) return [];- const versions = await engine.executeRaw<{ compiled_truth: string | null; timeline: string | null; body_storage: string; page_id: number; generation: number | null }>(- `SELECT DISTINCT v.compiled_truth, v.timeline, v.body_storage, v.page_id, v.body_generation AS generation FROM page_versions v JOIN pages p ON p.id = v.page_id- WHERE p.source_id = $1 AND p.slug = $2`, [sourceId, slug]);- const wanted = new Set(extra.map(tupleKey));- const produced = new Set<string>();- // #6226, #7949: rows an older reading filed for this text or a stored version.- const superseded = new Set(typeof current === 'string' ? supersededTimelineReadings(current).map(tupleKey)- : [...wanted].filter(key => current.superseded.has(key)));- const addText = (text: string) => {- for (const key of markdownTimelineKeys(text, slug)) if (wanted.has(key)) produced.add(key);- for (const entry of supersededTimelineReadings(text)) if (wanted.has(tupleKey(entry))) superseded.add(tupleKey(entry));+ // #9362: a filled `versions` (read before this transaction) is checked, not read again; a row it was not asked about stays.+ const replay = opts.versions?.done === true;+ if (!extra.length) {+ if (opts.versions) opts.versions.done = true;+ return [];+ }+ // The page's stored versions, one body per statement (never every body in one result), newest first; identical+ // bodies are parsed once and `stop` is asked before each body read.+ const readVersions = async (wanted: ReadonlySet<string>, seen: Set<string>): Promise<VersionTimelineScan> => {+ const read = versionTimelineScan();+ const parsed = new Map<string, { produced: string[]; superseded: string[] }>();+ const keysIn = async (texts: Iterable<string> | AsyncIterable<string>) => {+ const found = { produced: new Set<string>(), superseded: new Set<string>() };+ for await (const text of texts) {+ for (const key of markdownTimelineKeys(text, slug)) if (wanted.has(key)) found.produced.add(key);+ for (const entry of supersededTimelineReadings(text)) if (wanted.has(tupleKey(entry))) found.superseded.add(tupleKey(entry));+ }+ return { produced: [...found.produced], superseded: [...found.superseded] };+ };+ const versions = await engine.executeRaw<{ id: number; body_storage: string; page_id: number; generation: number | null; content_hash: string | null }>(+ `SELECT v.id, v.body_storage, v.page_id, v.body_generation AS generation, v.content_hash FROM page_versions v JOIN pages p ON p.id = v.page_id+ WHERE p.source_id = $1 AND p.slug = $2 ORDER BY v.id DESC`, [sourceId, slug]);+ for (const version of versions) {+ let witness: VersionWitness;+ let found: { produced: string[]; superseded: string[] };+ if (version.body_storage !== 'segmented') {+ if (opts.stop?.()) throw new BodyReadStopped(slug);+ const [body] = await engine.executeRaw<{ compiled_truth: string; timeline: string | null; head: string; tail: string }>(+ `SELECT compiled_truth, timeline, md5(compiled_truth) AS head, md5(coalesce(timeline, '')) AS tail FROM page_versions WHERE id = $1`, [version.id]);+ if (!body) continue;+ witness = { id: Number(version.id), storage: 'inline', digests: [body.head, body.tail] };+ const cached = `inline:${body.head}:${body.tail}`;+ found = parsed.get(cached) ?? await keysIn([`${body.compiled_truth}\n${body.timeline ?? ''}`]);+ parsed.set(cached, found);+ } else {+ witness = { id: Number(version.id), storage: 'segmented', generation: Number(version.generation), hash: version.content_hash };+ const cached = `segmented:${version.page_id}:${version.generation}`;+ found = parsed.get(cached) ?? await keysIn(retainedGenerationBlocks(engine, { id: Number(version.page_id), slug, sourceId }, Number(version.generation),+ { seen, ...(opts.stop ? { stop: opts.stop } : {}) }));+ parsed.set(cached, found);+ }+ for (const [into, keys] of [[read.produced, found.produced], [read.superseded, found.superseded]] as const) {+ for (const key of keys) {+ const witnesses = into.get(key);+ if (witnesses) witnesses.push(witness);+ else into.set(key, [witness]);+ }+ }+ }+ read.done = true;+ return read;};- const seen = new Set(typeof current === 'string' ? [] : current.runs ?? []);- for (const version of versions) {- if (version.body_storage !== 'segmented') { addText(`${version.compiled_truth}\n${version.timeline ?? ''}`); continue; }- for await (const block of retainedGenerationBlocks(engine, { id: Number(version.page_id), slug, sourceId }, Number(version.generation),- { seen, ...(opts.stop ? { stop: opts.stop } : {}) })) addText(block);+ // Inside a later transaction: the witness version is still this page's, with the body it was read with (no body is returned).+ const stillHolds = async (witness: VersionWitness) => (witness.storage === 'inline'+ ? await engine.executeRaw(`SELECT 1 FROM page_versions v JOIN pages p ON p.id = v.page_id WHERE v.id = $1 AND p.source_id = $2 AND p.slug = $3+ AND v.body_storage = 'inline' AND md5(v.compiled_truth) = $4 AND md5(coalesce(v.timeline, '')) = $5`, [witness.id, sourceId, slug, ...witness.digests])+ : await engine.executeRaw(`SELECT 1 FROM page_versions v JOIN pages p ON p.id = v.page_id WHERE v.id = $1 AND p.source_id = $2 AND p.slug = $3+ AND v.body_storage = 'segmented' AND v.body_generation = $4 AND v.content_hash IS NOT DISTINCT FROM $5`, [witness.id, sourceId, slug, witness.generation, witness.hash])).length > 0;+ const scan = replay ? opts.versions! : await readVersions(new Set(extra.map(tupleKey)), new Set(typeof current === 'string' ? [] : current.runs ?? []));+ if (opts.versions && !replay) Object.assign(opts.versions, scan);+ const holds = new Map<number, boolean>();+ const confirmed = async (witnesses: VersionWitness[] | undefined) => {+ if (!witnesses?.length) return false;+ if (!replay) return true;+ for (const witness of witnesses) {+ if (!holds.has(witness.id)) holds.set(witness.id, await stillHolds(witness));+ if (holds.get(witness.id)) return true;+ }+ return false;+ };+ // #6226, #7949: rows an older reading filed for this text or a stored version.+ const supersededNow = typeof current === 'string' ? new Set(supersededTimelineReadings(current).map(tupleKey)) : current.superseded;+ const found: Array<TimelineTuple & { id: number }> = [];+ for (const row of extra) {+ const key = tupleKey(row);+ if (!await confirmed(scan.produced.get(key))+ && !(hasExtractorDetail(row) && (supersededNow.has(key) || await confirmed(scan.superseded.get(key))))) continue;+ const { detail: _detail, ...tuple } = row;+ found.push(tuple);}- const found = extra.filter(row => produced.has(tupleKey(row)) || (superseded.has(tupleKey(row)) && hasExtractorDetail(row)))- .map(({ detail: _detail, ...row }) => row);if (!found.length) return [];// W1 #6783 §12 (a): a row more trusted than the page's restatement, or held by an open contest, stays; the owner decides.const [{ contestHeldRows, heldRows }, { derivedWriteTrust }, { storedTrustTier }] = await Promise.all([diff --git a/test/e2e/timeline-retract-version-scan-postgres.test.ts b/test/e2e/timeline-retract-version-scan-postgres.test.tsnew file mode 100644index 0000000000..ec40207dd9--- /dev/null+++ b/test/e2e/timeline-retract-version-scan-postgres.test.ts@@ -0,0 +1,15 @@+/**+ * #9362, Postgres arm: a maintenance timeline extract on a page with hundreds of large stored versions used to read+ * every version body in one statement inside the publication transaction, hit its statement timeout (57014) and fail+ * write_interrupted on every run. Scenarios: test/helpers/timeline-version-scan-scenarios.ts.+ */+import { describe, test } from 'bun:test';+import { manyLargeVersionsPublish, scanBeforeTransactionNeverRetractsMore, segmentedPageRetractsLikeMaster, smallPageRetractsLikeMaster } from '../helpers/timeline-version-scan-scenarios.ts';++const url = process.env.DATABASE_URL;+describe.skipIf(!url)('Postgres timeline retraction over many stored versions (#9362)', () => {+ test('a page with 300 versions of ~300 KB publishes its timeline extract over an 8 MB/s link', () => manyLargeVersionsPublish(url!), 360_000);+ test('a small page retracts what master retracts', () => smallPageRetractsLikeMaster(url), 180_000);+ test('a page stored in segments retracts what master retracts', () => segmentedPageRetractsLikeMaster(url), 180_000);+ test('a scan read before the transaction never retracts more than one inside it', () => scanBeforeTransactionNeverRetractsMore(url), 180_000);+});diff --git a/test/helpers/timeline-version-scan-scenarios.ts b/test/helpers/timeline-version-scan-scenarios.tsnew file mode 100644index 0000000000..7b93541c47--- /dev/null+++ b/test/helpers/timeline-version-scan-scenarios.ts@@ -0,0 +1,214 @@+/**+ * #9362: timeline retraction on a page with many stored versions. Shared by the PGLite unit test+ * (test/timeline-retract-version-scan.test.ts) and the Postgres E2E (test/e2e/timeline-retract-version-scan-postgres.test.ts).+ */+import { expect } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { createConnection, createServer, type AddressInfo, type Socket } from 'node:net';+import type { BrainEngine } from '../../src/core/engine.ts';+import type { OperationContext } from '../../src/core/ops/contract.ts';+import { submitPageMutation } from '../../src/core/persistence/page-mutations.ts';+import { disposePersistenceConsumer } from '../../src/core/persistence/service.ts';+import { withCoordinatedWrite } from '../../src/core/persistence/context.ts';+import { isTerminalWriteState } from '../../src/core/persistence/types.ts';+import { publishManagedPageTimeline } from '../../src/commands/extract-timeline-db.ts';+import { retractRemovedTimelineEntries, versionTimelineScan } from '../../src/core/timeline-extract.ts';+import { BodyReadStopped } from '../../src/core/page-body/segment-blocks.ts';+import { TEST_WRITE_ATTRIBUTION } from './write-attribution.ts';+import { managedBrain } from './managed-brain.ts';+import { conversationBody } from './conversation-page.ts';++export const SLUG = 'people/alice-example';+const KEPT = '- **2026-01-02** | test — Met Acme';+const REMOVED = '- **2026-02-03** | test — Joined Acme';+const ALSO_REMOVED = '- **2026-03-04** | test — Left Acme';++const put = async (ctx: OperationContext, body: string) => {+ const current = await ctx.engine.readPageSnapshot(SLUG, { sourceId: 'default' });+ return submitPageMutation(ctx, { operation: 'put_page', params: { slug: SLUG, request_id: randomUUID(),+ content: `---\ntype: person\ntitle: Alice\n---\n\n${body}\n`, ...(current ? { expected_revision: current.revision } : {}) } });+};++const guarded = <T>(engine: BrainEngine, fn: (tx: BrainEngine) => Promise<T>) =>+ engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => fn(tx), TEST_WRITE_ATTRIBUTION));++export const summaries = async (engine: BrainEngine) => (await engine.executeRaw<{ summary: string }>(+ `SELECT t.summary FROM timeline_entries t JOIN pages p ON p.id = t.page_id WHERE p.slug = $1 AND t.event_page_id IS NULL ORDER BY t.date, t.summary`, [SLUG]))+ .map(row => row.summary);++/** Stored rows the current text does not produce: two an earlier version held, one (enrichment) no version ever held. */+async function storeRows(engine: BrainEngine, rows: Array<[string, string]>) {+ await guarded(engine, tx => tx.executeRaw(`INSERT INTO timeline_entries (page_id, date, source, summary, detail)+ SELECT p.id, r.d::date, 'test', r.s, '' FROM pages p, jsonb_to_recordset($2::text::jsonb) AS r(d text, s text) WHERE p.slug = $1`,+ [SLUG, JSON.stringify(rows.map(([d, s]) => ({ d, s })))]));+}++/** A page whose first save held three bullets and whose two later saves hold one, with the removed rows stored again. */+async function smallPage(engine: BrainEngine, ctx: OperationContext) {+ await engine.setConfig('auto_link', 'false');+ await put(ctx, `A person.\n\n## Timeline\n\n${KEPT}\n${REMOVED}\n${ALSO_REMOVED}`);+ await put(ctx, `A person, edited.\n\n## Timeline\n\n${KEPT}`);+ await put(ctx, `A person, edited again.\n\n## Timeline\n\n${KEPT}`);+ await disposePersistenceConsumer(engine);+ await storeRows(engine, [['2026-02-03', 'Joined Acme'], ['2026-03-04', 'Left Acme'], ['2026-04-05', 'Enriched note']]);+ const [{ n }] = await engine.executeRaw<{ n: number }>('SELECT count(*)::int AS n FROM page_versions v JOIN pages p ON p.id = v.page_id WHERE p.slug = $1', [SLUG]);+ expect(n).toBeGreaterThan(0);+ expect(await summaries(engine)).toEqual(['Met Acme', 'Joined Acme', 'Left Acme', 'Enriched note']);+}++/** Master's outcome on a small page: rows an earlier version produced are retracted, the rest stay. */+export async function smallPageRetractsLikeMaster(databaseUrl?: string) {+ await managedBrain(async ({ engine, ctx }) => {+ await smallPage(engine, ctx);+ await publishManagedPageTimeline(engine, SLUG, 'default');+ expect(await summaries(engine)).toEqual(['Met Acme', 'Enriched note']);+ }, { databaseUrl });+}++/**+ * The same outcome on a page now stored in segments: a row its inline history produced and a row only a retained+ * segmented generation produced are both retracted, a row no version produced stays.+ */+export async function segmentedPageRetractsLikeMaster(databaseUrl?: string) {+ await managedBrain(async ({ engine, ctx }) => {+ await engine.setConfig('auto_link', 'false');+ await engine.setConfig('pages.segmented_storage', 'on');+ const large = (bullets: string[]) => `${conversationBody(1_200_000, { seed: 41 })}\n\n${conversationBody(1_200_000, { seed: 42 })}\n\n## Timeline\n\n${bullets.join('\n')}`;+ await put(ctx, `A person.\n\n## Timeline\n\n${KEPT}\n${REMOVED}`);+ await put(ctx, large([KEPT, ALSO_REMOVED]));+ await put(ctx, large([KEPT]));+ await put(ctx, large([KEPT, '- **2026-06-07** | test — Joined again']));+ await put(ctx, large([KEPT]));+ await disposePersistenceConsumer(engine);+ const storage = await engine.executeRaw<{ body_storage: string }>(`SELECT DISTINCT v.body_storage FROM page_versions v JOIN pages p ON p.id = v.page_id+ WHERE p.slug = $1 ORDER BY 1`, [SLUG]);+ expect(storage.map(row => row.body_storage)).toEqual(['inline', 'segmented']);+ await storeRows(engine, [['2026-02-03', 'Joined Acme'], ['2026-03-04', 'Left Acme'], ['2026-04-05', 'Enriched note'], ['2026-06-07', 'Joined again']]);+ await publishManagedPageTimeline(engine, SLUG, 'default');+ expect(await summaries(engine)).toEqual(['Met Acme', 'Enriched note']);+ }, { databaseUrl });+}++const textOf = async (engine: BrainEngine) => {+ const [page] = await engine.executeRaw<{ compiled_truth: string; timeline: string | null }>('SELECT compiled_truth, timeline FROM pages WHERE slug = $1', [SLUG]);+ return `${page!.compiled_truth}\n${page!.timeline ?? ''}`;+};++const ids = (rows: Array<{ id: number }>) => JSON.stringify(rows.map(row => Number(row.id)));++/**+ * The retraction a scan read before the transaction makes is never more than a scan inside it would make (master's+ * read): an interrupted scan retracts nothing, a version rewritten or deleted since the scan vouches for nothing, and a+ * row the scan was not asked about stays.+ */+export async function scanBeforeTransactionNeverRetractsMore(databaseUrl?: string) {+ await managedBrain(async ({ engine, ctx }) => {+ await smallPage(engine, ctx);+ const before = await summaries(engine);+ const text = await textOf(engine);+ const holding = () => engine.executeRaw<{ id: number }>(`SELECT v.id FROM page_versions v JOIN pages p ON p.id = v.page_id+ WHERE p.slug = $1 AND strpos(v.compiled_truth || coalesce(v.timeline, ''), 'Left Acme') > 0`, [SLUG]);+ const inside = async () => (await guarded(engine, tx => retractRemovedTimelineEntries(tx, SLUG, 'default', text, { dryRun: true })))+ .map(row => row.summary);+ expect((await holding()).length).toBeGreaterThan(0);++ let reads = 0;+ const interrupted = versionTimelineScan();+ expect(await retractRemovedTimelineEntries(engine, SLUG, 'default', text, { versions: interrupted, stop: () => ++reads > 1 })+ .then(() => null, (error: unknown) => error)).toBeInstanceOf(BodyReadStopped);+ expect(reads).toBe(2);+ expect(interrupted.done).toBe(false);+ expect(await summaries(engine)).toEqual(before);++ const rewritten = versionTimelineScan();+ expect((await retractRemovedTimelineEntries(engine, SLUG, 'default', text, { dryRun: true, versions: rewritten })).map(row => row.summary))+ .toEqual(['Joined Acme', 'Left Acme']);+ const witnesses = ids(await holding());+ await guarded(engine, tx => tx.executeRaw(`UPDATE page_versions SET compiled_truth = replace(compiled_truth, $2, ''), timeline = replace(timeline, $2, '')+ WHERE id IN (SELECT jsonb_array_elements_text($1::text::jsonb)::int)`, [witnesses, REMOVED]));+ await storeRows(engine, [['2026-05-06', 'Added after the scan']]);+ expect(await inside()).toEqual(['Left Acme']);+ expect(await guarded(engine, tx => retractRemovedTimelineEntries(tx, SLUG, 'default', text, { versions: rewritten }))).toEqual([]);++ const deleted = versionTimelineScan();+ expect((await retractRemovedTimelineEntries(engine, SLUG, 'default', text, { dryRun: true, versions: deleted })).map(row => row.summary))+ .toEqual(['Left Acme']);+ await guarded(engine, tx => tx.executeRaw('DELETE FROM page_versions WHERE id IN (SELECT jsonb_array_elements_text($1::text::jsonb)::int)', [witnesses]));+ expect(await inside()).toEqual([]);+ expect(await guarded(engine, tx => retractRemovedTimelineEntries(tx, SLUG, 'default', text, { versions: deleted }))).toEqual([]);+ expect(await summaries(engine)).toEqual([...before, 'Added after the scan']);+ }, { databaseUrl });+}++/**+ * A TCP proxy to Postgres that delivers at most `bytesPerSecond` toward the client, as a remote managed database's link+ * does. Reading slower makes the server's send block, which the server's statement timeout still counts.+ */+async function throttledPostgres(databaseUrl: string, bytesPerSecond: number) {+ const target = new URL(databaseUrl);+ const sockets = new Set<Socket>();+ const server = createServer(client => {+ const upstream = createConnection({ host: target.hostname, port: Number(target.port || 5432) });+ for (const socket of [client, upstream]) {+ sockets.add(socket);+ socket.on('close', () => sockets.delete(socket));+ socket.on('error', () => { client.destroy(); upstream.destroy(); });+ }+ client.pipe(upstream);+ let next = 0;+ upstream.on('data', (chunk: Buffer) => {+ client.write(chunk);+ next = Math.max(next, performance.now()) + chunk.length / bytesPerSecond * 1000;+ const wait = next - performance.now();+ if (wait > 1) { upstream.pause(); setTimeout(() => upstream.resume(), wait); }+ });+ upstream.on('end', () => client.end());+ });+ await new Promise<void>(done => server.listen(0, '127.0.0.1', done));+ const url = new URL(databaseUrl);+ url.hostname = '127.0.0.1';+ url.port = String((server.address() as AddressInfo).port);+ return { url: url.toString(), close: async () => {+ for (const socket of sockets) socket.destroy();+ await new Promise<void>(done => server.close(() => done()));+ } };+}++/**+ * The reported shape (#9362): a page body of ~300 KB with 300 stored versions of about that size (~90 MB), a stored row+ * the current text does not produce, and a database link of 8 MB/s (the report's own reads ran at about 10 MB/s: 129 MB+ * in 12.7 s, 46 MB in 4.3 s). Master read every version body in one statement inside the publication transaction, which+ * its 5 s statement timeout cancelled on every retry. The extract now publishes and retracts the row only an old+ * version produced.+ */+export async function manyLargeVersionsPublish(databaseUrl: string) {+ const link = await throttledPostgres(databaseUrl, 8 * 1024 * 1024);+ try {+ await managedBrain(async ({ engine, ctx }) => {+ await engine.setConfig('auto_link', 'false');+ const filler = Array.from({ length: 32_000 }, (_, i) => `w${(i * 2654435761 % 4294967296).toString(16)}`).join(' ');+ await put(ctx, `${filler}\n\n## Timeline\n\n${KEPT}`);+ await disposePersistenceConsumer(engine);+ const [{ body }] = await engine.executeRaw<{ body: number }>('SELECT octet_length(compiled_truth)::int AS body FROM pages WHERE slug = $1', [SLUG]);+ expect(body).toBeGreaterThan(250_000);+ await guarded(engine, tx => tx.executeRaw(`INSERT INTO page_versions (page_id, compiled_truth, timeline)+ SELECT p.id, CASE WHEN i = 1 THEN $2 ELSE '' END || (SELECT string_agg(md5(i::text || '-' || j::text), ' ') FROM generate_series(1, 9000) j), ''+ FROM pages p, generate_series(1, 300) i WHERE p.slug = $1`, [SLUG, `## Timeline\n\n${REMOVED}\n\n`]));+ await storeRows(engine, [['2026-02-03', 'Joined Acme'], ['2026-04-05', 'Enriched note']]);+ const [{ n, bytes }] = await engine.executeRaw<{ n: number; bytes: number }>(`SELECT count(*)::int AS n, sum(octet_length(v.compiled_truth))::bigint AS bytes+ FROM page_versions v JOIN pages p ON p.id = v.page_id WHERE p.slug = $1`, [SLUG]);+ expect(n).toBeGreaterThanOrEqual(300);+ expect(Number(bytes)).toBeGreaterThan(80_000_000);+ await publishManagedPageTimeline(engine, SLUG, 'default');+ const extracts = () => engine.executeRaw<{ state: string; error_code: string | null; message: string | null }>(`SELECT state, error_code,+ left(error_message, 200) AS message FROM persistence_requests WHERE slug = $1 AND intent->>'kind' = 'managed_maintenance_timeline_extract'`, [SLUG]);+ let requests = await extracts();+ for (const deadline = performance.now() + 200_000; requests.some(row => !isTerminalWriteState(row.state as 'committed')) && performance.now() < deadline;) {+ await Bun.sleep(1_000);+ requests = await extracts();+ }+ expect(requests).toEqual([{ state: 'committed', error_code: null, message: null }]);+ expect(await summaries(engine)).toEqual(['Met Acme', 'Enriched note']);+ }, { databaseUrl: link.url });+ } finally { await link.close(); }+}diff --git a/test/timeline-retract-version-scan.test.ts b/test/timeline-retract-version-scan.test.tsnew file mode 100644index 0000000000..5321ad3b20--- /dev/null+++ b/test/timeline-retract-version-scan.test.ts@@ -0,0 +1,10 @@+/**+ * #9362, PGLite arm: timeline retraction reads stored versions one body at a time before the publication transaction+ * and only checks them inside it. Postgres arm (with the reported page shape): test/e2e/timeline-retract-version-scan-postgres.test.ts.+ */+import { test } from 'bun:test';+import { scanBeforeTransactionNeverRetractsMore, segmentedPageRetractsLikeMaster, smallPageRetractsLikeMaster } from './helpers/timeline-version-scan-scenarios.ts';++test('a small page retracts what master retracts', () => smallPageRetractsLikeMaster(), 120_000);+test('a page stored in segments retracts what master retracts', () => segmentedPageRetractsLikeMaster(), 120_000);+test('a scan read before the transaction never retracts more than one inside it', () => scanBeforeTransactionNeverRetractsMore(), 120_000);