Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Keep session files whose facts land on a large entity page, and drop open loops of purged Gmail threads
garrytan97f8bfaaff40
diff --git a/behavior-changes.d/corpus_segmented_entity_9279.md b/behavior-changes.d/corpus_segmented_entity_9279.mdnew file mode 100644index 0000000000..2267749f66--- /dev/null+++ b/behavior-changes.d/corpus_segmented_entity_9279.md@@ -0,0 +1 @@+Captured session files whose facts name a large entity page (one stored in segments, over about 1 MiB) are now extracted: facts inferred from a mention of that page were refused after the model call, so those files were retried five times and then stopped. The first sweep after this upgrade retries each file stopped that way once, at the cost of one model call per stopped window, with no command needed. From now on a refusal that retrying cannot change stops its window after one paid attempt instead of five, retention keeps the session file instead of deleting it at the un-ingested limit, and gbrain doctor (memory_writeback) counts these files as held. gbrain sweep --retry-failed --apply still retries them by hand.diff --git a/changelog.d/capy-segmented-corpus-sweep.md b/changelog.d/capy-segmented-corpus-sweep.mdnew file mode 100644index 0000000000..d57b88bfde--- /dev/null+++ b/changelog.d/capy-segmented-corpus-sweep.md@@ -0,0 +1,11 @@+---+important: data-loss+---+**Captured session files that mention a large entity page are extracted again instead of being stopped and later deleted.**++### Fixed+- The session sweep no longer fails every window whose facts name an entity page stored in segments (a page over about 1 MiB). Facts the extractor left without an entity and that mention such a page were checked against the page's whole body, which a segmented page refuses (`page_body_segmented`) after the model call; the window failed five times, stopped, and the session file was deleted at the un-ingested retention limit. The check now reads the page the way its writer does, so the window commits its facts onto the page's `## Facts` table, on a Git source and on a repo-less brain. Reported by @dvasselin (#9279).+- A refusal that retrying cannot change now stops its window after the one paid attempt instead of five, retention keeps the session file it holds, and `gbrain doctor` (`memory_writeback`) counts these files as held. The first sweep after an upgrade retries each such window once, so a brain with files stopped by this bug recovers with no command.+- A Gmail thread you purged while its import was still waiting to commit no longer leaves its open loops behind. `gbrain waiting` could keep listing the thread's subject and evidence quote after the purge; the sync now drops loops for threads it refused as purged in the same run when no page for them exists (#9011).++<!-- behavior-change: corpus_segmented_entity_9279 -->diff --git a/docs/architecture/key-files/agent-bootstrap-continued.md b/docs/architecture/key-files/agent-bootstrap-continued.mdindex 28e2ead893..e9230df1bc 100644--- a/docs/architecture/key-files/agent-bootstrap-continued.md+++ b/docs/architecture/key-files/agent-bootstrap-continued.md@@ -24,7 +24,7 @@- Diagnostics: `src/commands/doctor/checks/memory-writeback.ts` (`memory_writeback`, BRAIN category) — quiet-ok when off, but the OFF branch still probes receipt + canonical instruction paths and WARNS on a lingering block (off-but-still-instructing) with the converge command; plane-compare warns when the file mirror disagrees with the DB row (the Stop hook acting on the wrong truth) naming the re-sync; a read_error reports the state as UNKNOWN, never "off (default)"; resolved bundle + BOTH visibility postures + audience/reasons; on both branches an explicit private `facts.default_visibility` on a non-shared brain with remote readers (a harness receipt, or HTTP MCP clients active in 30d; listed in `private_default_remote_readers`) warns with the shared `PRIVATE_DEFAULT_REMOTE_CONSEQUENCE` text (`writeback-config.ts`), the same text `bootstrap harness` prints before consent; instruction blocks receipt-vs-live-probe-vs-drift compare (drift fix names the `config set` + `bootstrap harness` combo — the config set re-stamps the posture mirror so the re-render converges even after an out-of-band visibility flip) + AGENTS.override.md detection — and a receipt target in state `failed` is a STANDING warn regardless of the live probe (the one physical-survival path — a smoke-rollback strip that itself threw — leaves a current-looking block directing sessions at a rolled-back endpoint, and only the receipt state knows) naming the `bootstrap harness --yes` / `--remove` converge; validity-lapsed count; 7d counters (verbs usage sidecar's additive `remember_status` [stamped in `src/mcp/dispatch.ts`, ALL-MCP-callers semantics labeled honestly] + `writeback`/`writeback-bank` heartbeat events; `turns_banked` counts flush_skip_* — banked, enqueue declined); when at least 10 serve-side harvests finished (ok + error; a `writer_busy_requeued` retry is not finished) in those 7d and more than 20% failed, a warn names the failure count, the top error reason and the `gbrain sweep --once` recovery; `details.corpus_backlog` counts corpus files with no `.ingested` sidecar (pending, oldest age, past retention, deletion day) and, with writeback on, warns once one is past plain retention (E-N1) with the local/lossy disclosure. Null-engine runs report the file mirror honestly. `cross_lane_duplicates_7d` / `near_duplicates_shadow_7d` sum the `writeback_dedup` heartbeat events in both the on and off branches.- Tests: `test/writeback-config.test.ts`, `test/mcp-instructions-writeback.test.ts`, `test/facts/writeback-gate.test.ts`, `test/hook-writeback-stop.serial.test.ts`, writeback-lane cases in `test/checkpoint-harvest.serial.test.ts`, `test/bootstrap-instructions-block.test.ts` + harness serial additions, `test/writeback-nudge.serial.test.ts`, `test/doctor-memory-writeback.test.ts`, the hermetic 5-step `test/ambient-writeback-lifecycle.serial.test.ts`, and the NON-GATING real-codex door (`test/e2e/bootstrap-real-codex.serial.test.ts`, ambient-writeback describe).- `src/core/sweep.ts` + `src/commands/sweep.ts` — the serve-resident maintenance sweep (the lock owner closes the persistence loop): facts-fence reconciliation (zero-LLM, reuses the cycle extractor with a slug subset), deterministic link/timeline extraction over recent workspace pages (the same cores as `gbrain extract` — remote put_page deliberately skips these, the sweep is where the graph compounds), and spend-gated corpus ingest (skipped keyless; turn-boundary windows through `src/core/context/corpus-windows.ts`, at most `CORPUS_WINDOWS_PER_SWEEP` (8) per file and `CORPUS_WINDOWS_PER_SWEEP_TOTAL` (32, env `GBRAIN_CORPUS_WINDOWS_PER_SWEEP`) per sweep, each windowed file reported in `corpus_files[]` as `{file, windows_done, windows_remaining}`; `.ingested` is written when the last window of the current snapshot finishes, and a finished file re-enters when its stat no longer matches its `.progress`; writeback `.wb-` turn files stay one call; dream outputs and claude-cli self-captures classified by `isClaudeCliSelfSessionId` get a terminal `.ingested` sidecar with `skipped: dream_output` / `self_capture` (the shared `selfCaptureSidecarJson` writer) and are never extracted; the extractor input is `corpusTextForExtraction`, so pasted blocks never reach it). Bounded, fail-soft, never throws; the wall-clock budget stops the corpus pass between files and windows, never mid-extraction (an in-flight extraction finishes and writes its sidecar, E-N2, pinned by `test/sweep-budget-between-items.test.ts`); armed at serve startup (3s, best-effort) and on 10-min idle ticks through the injectable timer seam, everything unref'd; `serve --http` arms `armCorpusDrain` instead (X8: every `CORPUS_DRAIN_INTERVAL_MS`, 10 min, one sweep on a `CORPUS_DRAIN_BUDGET_MS` 60 s budget while a corpus `.txt` has no final `.ingested` sidecar; never overlaps itself; its sweep diagnostics and thrown ticks reach stderr, each distinct line once per run, plus the skip reasons whenever their set changes (#6785); pinned by `test/serve-http-corpus-drain.test.ts`); `GBRAIN_SWEEP=0` kills both. `bootstrap verify` runs the sweep in-process with `corpus: false` (#6786: its engine has no configured gateway, so every window answered `chat_unavailable`). Before any model call the corpus pass runs the facts-absorb owner-readiness preflight per source (`assertSourceLegacyWriter` in `persistence/filesystem-guard.ts`): a claimed, not activated canonical worktree skips the file as `writer_coordinator_required` and logs `gbrain sources writer status <id>` (#6785; the compact and writeback harvest lanes do the same). A `chat_unavailable` result is not terminal: the windowed file or `.wb-` turn keeps no sidecar, nothing is recorded, and the pass stops (`corpus_chat_unavailable`). Failures thrown after a chat call are counted per window (`<file>.failures.json`, see `corpus-windows.ts`) and skipped as `corpus_window_backoff` / `corpus_window_dead_lettered`; `gbrain sweep --retry-failed [--apply]` previews or clears them (doctor `memory_writeback` names the paid retry as its fix). Pinned by `test/sweep-corpus-chat-unavailable-6786.test.ts` and `test/sweep-corpus-failure-cap-6785.test.ts`. `gbrain sweep --once` is the trusted CLI seam for operator catch-up runs (CLI-only, never over MCP). The link pass threads the same `link_resolution.cross_source` opt-in and configured `sources.default` the CLI extract lanes pass into `resolveCandidateSources`, so an edge into another source is created (flag on) or counted as `cross_source_link` in `skipped` (flag off) — never silently dropped, and the reconcile never deletes a cross-source edge `extract links --source db` created. A corpus window's facts batch is keyed by its text (the sweep passes `reAdmitFileRefusals`), so when the canonical file check refused some of its entity requests, `resumeManagedFacts` admits their retained facts as a follow-up batch once every refused page passes that check again: no new extraction, the page's current revision, never wider than the current default visibility, committed requests reported rather than re-admitted, the stored and current writer grants re-checked, at most three follow-ups. Pinned by `test/managed-sweep-refused-window-retry.test.ts`.-- `src/core/context/corpus-windows.ts` — windowed fact extraction of session-corpus files (#5887). `retireCorpusFile` / `retireCorpusTurns` (#6091) record a file or its off-period turns as finished without extracting (`skipped_reason`; a retire overrides another extractor's lease, and a later rewrite extracts only unrecorded turns). `parseCorpusTurns` (defined in the provider-free `corpus-turns.ts`, re-exported here) splits only at the exact `\n\n[user]\n` / `\n\n[assistant]\n` markers `toCorpusText` emits (text before the first marker is a header-less preamble); each user turn is paste-stripped whole before windowing; `planCorpusWindows` packs whole turns into windows of at most `MAX_TURN_TEXT_CHARS` (exported from `facts/extract.ts`) and splits a turn that cannot fit a fresh window at a newline or space, never inside a surrogate pair, repeating its `[role]` header on every piece. `<file>.progress` (suffix constants in `corpus-segments.ts`, GC'd by the session-end hook with the `.txt`, never deleted by its resume rewrite) holds `{version, generation, turns[{start,end,sha256}], continuation{turn_sha256, stripped_offset}|null, windows_done, finished(stat)|null, totals, entity_slugs, skipped_reason?, lease}`: `resumePoint` resumes at the first turn whose hash is unrecorded (appends, compaction remainders and same-size or shorter rewrites cost only their new turns). Every write is compare-and-set under the O_EXCL `<file>.progress.lock` (stale after 30 s): a window commits only when generation, last turn hash and lease owner still match and progress grows; the lease (stale after an hour, like the claim) stops a sweep whose `.in-progress` claim the hook deleted from extracting a window another extractor holds. `runCorpusWindows` returns `complete`/`partial`/`aborted`/`contended`/`changed`/`source_changed`/`unavailable`/`backoff`/`dead_lettered` with cumulative counts; an aborted or `unavailable` window records nothing, and a progress a pre-#6786 build finished with `skipped_reason: chat_unavailable` is rewound under the lease CAS to its retired turns (source binding, totals and links kept). Transport and write errors release the lease and throw; one thrown after a chat call (`extractUnderFailureCount`, observed through the AI invocation preflight) counts against the window's start key (`<turn sha256>@<offset>`) and source incarnation in `<file>.failures.json`: retried on the next sweep, then after 10, 20 and 40 minutes (`CORPUS_WINDOW_RETRY_BASE_MS`), stopped at `CORPUS_WINDOW_FAILURE_CAP` (5); tail growth and error labels never reset it, a commit of that window removes it, and one change of `extractionConfigFingerprint` (model, provider, key presence) re-arms a stopped window once (#6785). `corpusFileCompletion` in `corpus-segments.ts` is the one completion classifier (selection, the under-claim re-check, retention GC; `corpusSidecarState` for the drain and the doctor backlog): a `chat_unavailable` sidecar is pending, and only `done`/`legacy` files get the plain retention (#6786); `adoptFinishedCorpusFile` records a pre-window `.ingested` file at its current turns once (only when the sidecar is newer than the file). `checkpoint-harvest.ts` (`hook:compact`) extracts window 1 only: when more remains it publishes window 1's links, releases the receipt and leaves `.progress` for the sweep; an existing `.progress` means window 1 is done. Pinned by `test/corpus-windows.test.ts`, `test/sweep-corpus-windows.test.ts` and `test/managed-sweep-corpus-windows.test.ts` (managed coordinator, PGLite and Postgres).+- `src/core/context/corpus-windows.ts` — windowed fact extraction of session-corpus files (#5887). `retireCorpusFile` / `retireCorpusTurns` (#6091) record a file or its off-period turns as finished without extracting (`skipped_reason`; a retire overrides another extractor's lease, and a later rewrite extracts only unrecorded turns). `parseCorpusTurns` (defined in the provider-free `corpus-turns.ts`, re-exported here) splits only at the exact `\n\n[user]\n` / `\n\n[assistant]\n` markers `toCorpusText` emits (text before the first marker is a header-less preamble); each user turn is paste-stripped whole before windowing; `planCorpusWindows` packs whole turns into windows of at most `MAX_TURN_TEXT_CHARS` (exported from `facts/extract.ts`) and splits a turn that cannot fit a fresh window at a newline or space, never inside a surrogate pair, repeating its `[role]` header on every piece. `<file>.progress` (suffix constants in `corpus-segments.ts`, GC'd by the session-end hook with the `.txt`, never deleted by its resume rewrite) holds `{version, generation, turns[{start,end,sha256}], continuation{turn_sha256, stripped_offset}|null, windows_done, finished(stat)|null, totals, entity_slugs, skipped_reason?, lease}`: `resumePoint` resumes at the first turn whose hash is unrecorded (appends, compaction remainders and same-size or shorter rewrites cost only their new turns). Every write is compare-and-set under the O_EXCL `<file>.progress.lock` (stale after 30 s): a window commits only when generation, last turn hash and lease owner still match and progress grows; the lease (stale after an hour, like the claim) stops a sweep whose `.in-progress` claim the hook deleted from extracting a window another extractor holds. `runCorpusWindows` returns `complete`/`partial`/`aborted`/`contended`/`changed`/`source_changed`/`unavailable`/`backoff`/`dead_lettered` with cumulative counts; an aborted or `unavailable` window records nothing, and a progress a pre-#6786 build finished with `skipped_reason: chat_unavailable` is rewound under the lease CAS to its retired turns (source binding, totals and links kept). Transport and write errors release the lease and throw; one thrown after a chat call (`extractUnderFailureCount`, observed through the AI invocation preflight) counts against the window's start key (`<turn sha256>@<offset>`) and source incarnation in `<file>.failures.json`: retried on the next sweep, then after 10, 20 and 40 minutes (`CORPUS_WINDOW_RETRY_BASE_MS`), stopped at `CORPUS_WINDOW_FAILURE_CAP` (5); tail growth and error labels never reset it, a commit of that window removes it, and one change of `extractionConfigFingerprint` (model, provider, key presence) re-arms a stopped window once (#6785). #9279: a failure code retrying cannot change (`CORPUS_DETERMINISTIC_FAILURE_CODES` in `corpus-segments.ts`, `page_body_segmented`) stops the window after that one paid attempt, stamped with the build (`build`); the first sweep on another gbrain build runs it once, retention GC keeps its file past the un-ingested ceiling (`corpusFailureHeld`), and doctor `memory_writeback` counts it (`held`); `facts/subject-infer-write.ts` reads an inferred target's fence as its writer does, so a segmented page no longer throws that code. Pinned by `test/sweep-segmented-entity-9279.test.ts`. `corpusFileCompletion` in `corpus-segments.ts` is the one completion classifier (selection, the under-claim re-check, retention GC; `corpusSidecarState` for the drain and the doctor backlog): a `chat_unavailable` sidecar is pending, and only `done`/`legacy` files get the plain retention (#6786); `adoptFinishedCorpusFile` records a pre-window `.ingested` file at its current turns once (only when the sidecar is newer than the file). `checkpoint-harvest.ts` (`hook:compact`) extracts window 1 only: when more remains it publishes window 1's links, releases the receipt and leaves `.progress` for the sweep; an existing `.progress` means window 1 is done. Pinned by `test/corpus-windows.test.ts`, `test/sweep-corpus-windows.test.ts` and `test/managed-sweep-corpus-windows.test.ts` (managed coordinator, PGLite and Postgres).- `src/core/context/corpus-source.ts` — #6268, the session's source for every corpus file. Writers (`hook.ts` compact/stop/session-end, the OpenClaw `compact()`) call `openSessionCorpus`, which freezes one source per session at first use (GBRAIN_SOURCE, then the `.gbrain-source` dotfile, else unresolved) into `<sid>.source.json` in the `sourced/` spool and stamps it into the file name (`.src-<id>`, `default` included; `_unresolved` cannot collide with a real id). Pre-stamp sweepers never list the spool. `resolveCorpusFileSource` is the sweep/harvest/serve and dream-synthesize/extract_atoms read: the stamp or frozen record, else tier 4 alone (`resolveRegisteredLocalPathSource` in `source-resolver.ts`: a registered source whose local_path contains the recorded cwd, active over archived; frozen, tier `engine`), else on a single-source brain that source unfrozen (`sole_source`), else held `corpus_source_unresolved`, never a fallback tier (#7868); `{ readOnly: true }` (dry runs, the synthesis child re-check) freezes and binds nothing; legacy top-level files are held (`corpus_legacy_unstamped`) unless a legacy wb stamp, a frozen record or a single-source brain proves the source, never assigned the pass source. The first engine resolution binds the source incarnation; a recreated source holds the session (`corpus_source_incarnation_changed`). `gbrain sweep --assign-corpus <source>` is the operator mapping (preview, then `--apply`; it lists what routing holds, `.txt` and `.md` files in nested folders included, and sessions the sweep already extracted, because dream synthesis and extract_atoms hold those until their source is mapped, #7805, #7846, #7849); it releases only `corpus_legacy_unstamped` and `corpus_source_unresolved` holds, and the phases print each other reason's own fix (`corpusHoldRelease` in `synthesize-source-routing.ts`); doctor `memory_writeback` reports the same listing as `details.corpus_source_holds`. Tests: `test/corpus-source-stamp.test.ts`.- `src/core/context/sync-ipc.ts` + `src/core/serve-sync-runner.ts` + `src/commands/sync-delegate.ts` — serve-delegated sync. `sync-ipc.ts` is the LEAF wire module: `DELEGATED_SYNC_OPTION_FIELDS` (the fields `SYNC_FLAGS` in `src/core/sync-flags.ts` forwards through the serve, plus `sourceId`, `embedDeferred`, `timeoutSeconds` and `origin`) drives the validator, the CLI wire-builder, and the serve-side SyncOpts builder; `validateDelegatedSyncOptions` rejects unknown keys fail-closed (repoPath/skipLock/lockId/concurrency are unreachable from the socket), requires `timeoutSeconds` (0 = the explicit `--no-hard-deadline` unbounded encoding; everything else clamps to 24h), and `toWireSyncResult` truncates `pagesAffected` to 50 + a true total under the 256KB message cap. `serve-sync-runner.ts` is the serve-side module singleton: one job at a time (correctness still rests on the `gbrain-sync:<source>` row lock inside performSync), clientToken attach (a lost-ack retry finds its own running OR retained-terminal job instead of duplicate-running), per-job deadline timer, `[serve-sync]` lifecycle stderr lines, an idempotent shared `shutdownDelegatedSync()` both serve shutdown paths await BEFORE `engine.disconnect()` (the disconnect-mode drain is allowAbort:false after `_db` is nulled, so settle writes need the live engine; a registered drainer remains as backstop), and `maybeDrainDeferredEmbeds` (delegated jobs always run noEmbed — the cost gate lives in runSync — and the serve drains stale embeds afterwards via `runEmbedCore`, keyless-safe, cleared only when a drain finds nothing left; wire `noEmbed` records that the USER declined embeds and suppresses the drain). `sync-delegate.ts` is the CLI half: read-only holder probe (`probeLivePgliteHolder`), DEFAULT-DENY argv gate (any unclassified token refuses by name — a silently dropped `--exclude` would perform the wrong sync), engine-free source tiers (`resolveSourceIdEngineFree`), 1s poll loop tolerant of event-loop-blocked serves with a PID-probe backstop, `unknown_job` → serve-restarted resume hint, Ctrl-C → `sync_abort` (second Ctrl-C exits 130), and runSync's `pull_failed` verdict mirroring. Mounts never delegate; opt-outs `--no-delegate` / `GBRAIN_SYNC_NO_DELEGATE=1` (client) and `GBRAIN_SERVE_SYNC_IPC=0` (serve). #6317 (D1 a′), managed Postgres brains: `maybeDelegateManagedSyncToServe(engine, args, opts)` runs post-connect from `runSingleSourceSync` right before `performSync`; it probes the host's live full `serve` consumer through `persistence_consumers` (`consumer-heartbeat.ts`, the database, never a socket), admits the catch-up's flags (`--no-bulk`, `--lanes`/`--no-lanes`, `--json`, the processing flags), sends the CLI's durable `cli` registration in `sync_start`, prints the drain's own numbered lines from `sync_status` (`managed`, `lines`, `drain`, `next`) and returns the SyncResult to the in-process printers, so exit codes and the JSON envelope do not change. The serve verifies the registration (`withVerifiedLocalRegistration`) and runs `performSync({ drain: true })` as that writer under `withHumanLineSink`; the shared-secret lane stays under `withLegacySyncDelegation` and is still refused on a managed brain; a denied, revoked or stdio registration answers `sync_start` with `permission_denied` and the envelope (`refusal`), which the CLI rethrows. Every other rung (no owner, a non-serve owner, an unsupported flag, no socket, the kill switch, `stale_serve` / `unsupported_kind` from an older serve, `source_mismatch`, a serve lost mid-drain) runs without the serve with one actionable line; only the opt-out, a serve not answering its socket and a `stale_serve` serve call `forceOwnConsumer` (#6543), the rest admit writes the serve consumes under `persistence.single_consumer`. #6712 PR 1: neither builder sends `explicitProcessing` any more; the serve validates one from an older CLI by shape and drops it (`WIRE_SKEW_WINDOWS` in `src/core/wire-skew.ts`), and a resume inherits the cursor's processing opt-outs. #6712 PR 2: `WIRE_BOOL_FLAGS` / `VALUE_FLAGS` / `IGNORED_FLAGS` and the managed pick list are views of `SYNC_FLAGS`; a retired field the validator drops comes back as `ignored`, and the serve logs `retiredWireNotice` and streams it to the older client as a progress line (`test/sync-flag-table-6712.test.ts`). The serve's delegated run follows the source's drain lease as holder `serve` (`followDrainLease`, `drainHolder`). Wire `embedDeferred` (sent only when true) carries the cost gate's runtime deferral, so adopting a cursor that stores `noEmbed=false` never embeds in that run. `maybeDelegateManagedSyncToServe` refuses `--skip-failed` / `--include-gitignored` on a managed source with `managed_sync_flag_refused` before any probe or socket call (`test/sync-delegated-options-6696.serial.test.ts`, `test/sync-managed-forbidden-flags-6696.test.ts`). Pinned by `test/context/sync-ipc-validation.test.ts`, `test/sync-delegate-ladder.test.ts`, `test/serve-sync-runner.serial.test.ts`, `test/serve-sync-handoff.serial.test.ts`, `test/context/resolve-ipc-sync-kinds.test.ts`, `test/e2e/sync-delegation-under-serve.serial.test.ts` (gating tier1 step) and `test/e2e/serve-delegated-managed-sync.serial.test.ts` (Postgres, spawned serve, MCP latency printed). GBRA-133 2b.3: the deferred-embed drain re-arms itself until the backlog is gone: a pass that left work retries after `GBRAIN_DEFERRED_EMBED_RETRY_MS` (default 5000), a failing provider backs off 1x/3x/12x/60x, a running delegated sync defers it at most `DEFERRED_EMBED_BUSY_MAX` (3) times before it drains beside the sync, and a keyless brain keeps the backlog with no provider call or timer; `deferredEmbedDecisions` records each decision. #7243: `startDelegatedSync`'s in-process `logLines` option (never on the wire) also writes a verified job's human lines to the serve log, for the resident notifier's runs that nobody polls.- `src/core/capability.ts` — config-plane keyless/keyed detection + the honest capability report (per-provider keyless banner: OpenAI = semantic search + auto-extraction, Voyage = semantic search, Anthropic = auto-extraction) rendered by verify and the runbook. The extraction probe resolves through the SAME shared `resolveEffectiveChatModel` the gateway's reconfigure fallback uses (`GBRAIN_MODEL` > servable file pin > key-aware tier default), so a stale unservable pin degrades identically in the report and at runtime. Key/env fold comes from `mergedProviderEnv` (`src/core/ai/provider-env.ts`). Accepted limitation (documented in the module): DB-plane overrides (`models.default`, `facts.extraction_model`) are invisible to this engine-less probe; the runtime gate in `facts/extract.ts` and the engine-aware pre-enqueue gate in `facts/backstop.ts` are the backstops.diff --git a/scripts/page-body-readers-baseline.tsv b/scripts/page-body-readers-baseline.tsvindex 65bdba5706..a8b10da238 100644--- a/scripts/page-body-readers-baseline.tsv+++ b/scripts/page-body-readers-baseline.tsv@@ -264,7 +264,7 @@ src/core/facts/relink-publish.ts prepareRelinkMutation call:serializePageToMarkdsrc/core/facts/relink-publish.ts prepareRelinkMutation field SEGWsrc/core/facts/retire.ts retireFactsFromPage call:serializePageToMarkdown INLINE a segmented page is skipped (its rows counted failed) before its body is readsrc/core/facts/retire.ts retireFactsFromPage field INLINE a segmented page is skipped (its rows counted failed) before its body is read-src/core/facts/subject-infer-write.ts linkAllowed field STREAM+src/core/facts/subject-infer-write.ts targetFenceClean field INLINE the body materializedSnapshot read back within PAGE_MAX_BYTES on the managed path; a segmented page over the ceiling or on the unmanaged path is refused before any readsrc/core/facts/unfenced-facts.ts auditUnadoptableFacts call:serializePageToMarkdown SEGWsrc/core/facts/unfenced-facts.ts fenceDatabaseOnlyPage call:serializePageToMarkdown SEGWsrc/core/facts/unfenced-facts.ts planFence field NONE reads compiled_truth from the object withHeldFactsFence returns for the body string its caller passed; no stored page body is read herediff --git a/scripts/postgres-arm-weights.json b/scripts/postgres-arm-weights.jsonindex 03b9e61508..ea141ee49f 100644--- a/scripts/postgres-arm-weights.json+++ b/scripts/postgres-arm-weights.json@@ -597,6 +597,7 @@"test/stats-health-source-scope.test.ts": 5779,"test/status-snapshot-source-scope.test.ts": 5781,"test/strict-background-writers-gst2492.test.ts": 36888,+ "test/sweep-segmented-entity-9279.test.ts": 36900,"test/sync-absent-from-worktree-s27.test.ts": 21022,"test/sync-admission-6712.test.ts": 4821,"test/sync-apply-held-deletions-gbra133.test.ts": 12112,diff --git a/src/commands/doctor/checks/memory-writeback.ts b/src/commands/doctor/checks/memory-writeback.tsindex f86f20d308..95894a4808 100644--- a/src/commands/doctor/checks/memory-writeback.ts+++ b/src/commands/doctor/checks/memory-writeback.ts@@ -40,7 +40,7 @@ import { classifyBrainAudience } from '../../../core/facts/writeback-audience.tsimport { readVerbUsage } from '../../../core/verbs/usage-log.ts';import { readClientOpUsage } from '../../../core/mcp-usage.ts';import { OPENCLAW_COMPACT_EVENT, readHeartbeatTail } from '../../../core/context/hook-heartbeat.ts';-import { CORPUS_UNINGESTED_RETENTION_FACTOR, CORPUS_WINDOW_FAILURE_CAP, corpusBacklog, corpusSpoolDir, listCorpusFailures } from '../../../core/context/corpus-segments.ts';+import { CORPUS_UNINGESTED_RETENTION_FACTOR, CORPUS_WINDOW_FAILURE_CAP, corpusBacklog, corpusFailureHeld, corpusSpoolDir, listCorpusFailures } from '../../../core/context/corpus-segments.ts';import { listCorpusSourceHolds } from '../../../core/context/corpus-source.ts';import { readHarnessReceiptState } from '../../../core/bootstrap/format.ts';import { mutedFirstRunDecisionsNotice } from '../../../core/onboard/mcp-onboarding.ts';@@ -168,13 +168,14 @@ async function corpusBacklogProblem(engine: BrainEngine, fileCfg: ReturnType<typconst top = corpusBacklog(dir, days * DAY_MS, now);const spool = corpusBacklog(corpusSpoolDir(dir), days * DAY_MS, now);const oldest = [top.oldestPendingMtimeMs, spool.oldestPendingMtimeMs].filter((m): m is number => m !== null);- const backlog = { pending: top.pending + spool.pending, pastRetention: top.pastRetention + spool.pastRetention, oldestPendingMtimeMs: oldest.length ? Math.min(...oldest) : null };+ const backlog = { pending: top.pending + spool.pending, pastRetention: top.pastRetention + spool.pastRetention, held: top.held + spool.held, oldestPendingMtimeMs: oldest.length ? Math.min(...oldest) : null };const oldestDays = backlog.oldestPendingMtimeMs === null ? null : Math.floor((now - backlog.oldestPendingMtimeMs) / DAY_MS);const purgeDays = days * CORPUS_UNINGESTED_RETENTION_FACTOR;- details.corpus_backlog = { pending: backlog.pending, oldest_pending_days: oldestDays, past_retention: backlog.pastRetention, retention_days: days, deleted_after_days: purgeDays };+ details.corpus_backlog = { pending: backlog.pending, oldest_pending_days: oldestDays, past_retention: backlog.pastRetention, held: backlog.held, retention_days: days, deleted_after_days: purgeDays };if (backlog.pastRetention === 0) return null;+ const kept = backlog.held ? ` (${backlog.held} held by a refusal that retrying cannot change are kept until extracted)` : '';return `${backlog.pastRetention} of ${backlog.pending} captured session file(s) are older than the ${days}-day corpus retention and were never extracted `- + `(oldest ${oldestDays}d); they are deleted at ${purgeDays}d. Extract them now: gbrain sweep --once --budget-ms 600000`;+ + `(oldest ${oldestDays}d); they are deleted at ${purgeDays}d${kept}. Extract them now: gbrain sweep --once --budget-ms 600000`;}/** #6785: the problem text a stopped corpus window carries; the check's fix keys on it. */@@ -193,10 +194,13 @@ async function corpusFailureProblem(engine: BrainEngine, fileCfg: ReturnType<typif (!failed.length) return null;const codes = new Map<string, number>();for (const f of stopped) codes.set(f.record.last_code, (codes.get(f.record.last_code) ?? 0) + 1);- details.corpus_failed_windows = { retrying: failed.length - stopped.length, stopped: stopped.length, last_codes: Object.fromEntries(codes) };+ const held = stopped.filter((f) => corpusFailureHeld(f.record)).length;+ details.corpus_failed_windows = { retrying: failed.length - stopped.length, stopped: stopped.length, held, last_codes: Object.fromEntries(codes) };if (!stopped.length) return null;const [topCode] = [...codes.entries()].sort((a, b) => b[1] - a[1])[0]!;- return `${stopped.length} captured session file(s) stopped extracting after ${CORPUS_WINDOW_FAILURE_CAP} failed attempts at one window (top: ${topCode}); `+ // #9279: a deterministic refusal stops its window after one paid attempt; retention keeps the file, and the first sweep on a new build retries it once.+ const heldLine = held ? ` ${held} of them stopped at once on a refusal that retrying cannot change; retention keeps them and the first sweep after a gbrain upgrade retries each once.` : '';+ return `${stopped.length} captured session file(s) stopped extracting after ${CORPUS_WINDOW_FAILURE_CAP} failed attempts at one window (top: ${topCode});${heldLine} `+ `fix the cause, then retry them (paid): ${CORPUS_RETRY_COMMAND}`;}diff --git a/src/core/context/corpus-segments.ts b/src/core/context/corpus-segments.tsindex 070c64715a..48eb1306fe 100644--- a/src/core/context/corpus-segments.ts+++ b/src/core/context/corpus-segments.ts@@ -724,6 +724,11 @@ export function corpusFileAwaitsExtraction(full: string, name: string): booleanexport const CORPUS_FAILURE_SUFFIX = '.failures.json';/** Failed attempts after which a window is not retried until an operator retry or one extraction-config change. */export const CORPUS_WINDOW_FAILURE_CAP = 5;+/**+ * #9279: failure codes no retry on the same build can change (the write refuses the same way every time). One such+ * failure stops the window at once; the first sweep on another gbrain build retries it once; retention keeps the file.+ */+export const CORPUS_DETERMINISTIC_FAILURE_CODES: readonly string[] = ['page_body_segmented'];export interface CorpusFailureRecord {version: 1;@@ -739,6 +744,8 @@ export interface CorpusFailureRecord {fingerprint: string | null;/** True once a fingerprint change re-armed a capped window; a second change does not. */rearmed: boolean;+ /** #9279: the gbrain version that recorded a deterministic failure (CORPUS_DETERMINISTIC_FAILURE_CODES). */+ build?: string;}/** The failure record, or null when absent or unparseable. Never throws. */@@ -748,13 +755,19 @@ export function readCorpusFailure(full: string): CorpusFailureRecord | null {if (!r || r.version !== 1 || typeof r.key !== 'string' || !Number.isSafeInteger(r.count) || r.count < 1|| !Number.isFinite(r.last_at) || !Number.isFinite(r.first_at) || typeof r.last_code !== 'string'|| !(r.source_incarnation === null || typeof r.source_incarnation === 'string')- || !(r.fingerprint === null || typeof r.fingerprint === 'string') || typeof r.rearmed !== 'boolean') return null;+ || !(r.fingerprint === null || typeof r.fingerprint === 'string') || typeof r.rearmed !== 'boolean'+ || !(r.build === undefined || typeof r.build === 'string')) return null;return r;} catch {return null;}}+/** #9279: a window stopped by a deterministic failure: its file is never extracted until a retry, so retention keeps it. */+export function corpusFailureHeld(record: CorpusFailureRecord | null): boolean {+ return !!record && record.count >= CORPUS_WINDOW_FAILURE_CAP && CORPUS_DETERMINISTIC_FAILURE_CODES.includes(record.last_code);+}+/** Every corpus `.txt` in `dir` with a failure record. Missing dir = empty; never throws. */export function listCorpusFailures(dir: string): Array<{ name: string; full: string; record: CorpusFailureRecord }> {let names: string[];@@ -779,11 +792,13 @@ export interface CorpusBacklog {oldestPendingMtimeMs: number | null;/** Pending files older than retention: kept only by the un-ingested rule, deleted at the ceiling. */pastRetention: number;+ /** #9279: pending files past retention held by a deterministic failure: kept past the ceiling (corpusFailureHeld). */+ held: number;}/** Count corpus turn files still waiting for extraction. Missing dir = empty backlog; never throws. */export function corpusBacklog(dir: string, retentionMs: number, now = Date.now()): CorpusBacklog {- const backlog: CorpusBacklog = { pending: 0, oldestPendingMtimeMs: null, pastRetention: 0 };+ const backlog: CorpusBacklog = { pending: 0, oldestPendingMtimeMs: null, pastRetention: 0, held: 0 };let names: string[];try {names = readdirSync(dir);@@ -802,7 +817,10 @@ export function corpusBacklog(dir: string, retentionMs: number, now = Date.now()}backlog.pending++;if (backlog.oldestPendingMtimeMs === null || mtimeMs < backlog.oldestPendingMtimeMs) backlog.oldestPendingMtimeMs = mtimeMs;- if (mtimeMs < now - retentionMs) backlog.pastRetention++;+ if (mtimeMs < now - retentionMs) {+ backlog.pastRetention++;+ if (corpusFailureHeld(readCorpusFailure(join(dir, name)))) backlog.held++;+ }}return backlog;}@@ -812,8 +830,8 @@ export function corpusBacklog(dir: string, retentionMs: number, now = Date.now()* 'done' or 'legacy') goes at `retentionMs`; one nothing has finished* extracting (no sidecar, a non-terminal skip such as `chat_unavailable`* #6786, windows still pending, or rewritten since) stays until- * `CORPUS_UNINGESTED_RETENTION_FACTOR` x `retentionMs` (E-N1).- * Best-effort per file; never throws.+ * `CORPUS_UNINGESTED_RETENTION_FACTOR` x `retentionMs` (E-N1); one held by a deterministic failure+ * (corpusFailureHeld, #9279) is never extracted until a retry, so it stays. Best-effort per file; never throws.*/export function gcCorpusTurnFiles(dir: string, retentionMs: number, now = Date.now()): void {try {@@ -823,8 +841,9 @@ export function gcCorpusTurnFiles(dir: string, retentionMs: number, now = Date.ntry {const age = now - statSync(p).mtimeMs;if (age <= retentionMs) continue;- const ageLimit = corpusFileFinished(corpusFileCompletion(p, name)) ? retentionMs : retentionMs * CORPUS_UNINGESTED_RETENTION_FACTOR;- if (age > ageLimit) rmSync(p, { force: true });+ const finished = corpusFileFinished(corpusFileCompletion(p, name));+ const ageLimit = finished ? retentionMs : retentionMs * CORPUS_UNINGESTED_RETENTION_FACTOR;+ if (age > ageLimit && (finished || !corpusFailureHeld(readCorpusFailure(p)))) rmSync(p, { force: true });} catch {/* per-file best effort */}diff --git a/src/core/context/corpus-windows.ts b/src/core/context/corpus-windows.tsindex c0cedc1fb1..4d122b031b 100644--- a/src/core/context/corpus-windows.ts+++ b/src/core/context/corpus-windows.ts@@ -34,8 +34,9 @@ import { MAX_TURN_TEXT_CHARS } from '../facts/extract.ts';import type { runFactsPipeline } from '../facts/backstop.ts';import { withAIInvocationGuard } from '../ai/invocation-guard.ts';import { parseCorpusTurns, type CorpusTurn } from './corpus-turns.ts';+import { VERSION } from '../../version.ts';import {- CORPUS_FAILURE_SUFFIX, CORPUS_NON_TERMINAL_SKIPS, CORPUS_PROGRESS_LOCK_SUFFIX, CORPUS_PROGRESS_SUFFIX, CORPUS_WINDOW_FAILURE_CAP,+ CORPUS_DETERMINISTIC_FAILURE_CODES, CORPUS_FAILURE_SUFFIX, CORPUS_NON_TERMINAL_SKIPS, CORPUS_PROGRESS_LOCK_SUFFIX, CORPUS_PROGRESS_SUFFIX, CORPUS_WINDOW_FAILURE_CAP,corpusFileCompletion, corpusFileFinished, parseCorpusProgress, parseWbFileName, readCorpusFailure, type CorpusFailureRecord,} from './corpus-segments.ts';@@ -415,7 +416,10 @@ function windowKey(turns: CorpusTurn[], at: { turn: number; offset: number }): s* #6785: whether the window at `key` may be paid for now. A record for* another window or another source incarnation does not apply. A capped* window stays stopped, except once when the extraction-config fingerprint- * changed since its last failure.+ * changed since its last failure. #9279: a window stopped by a deterministic+ * failure runs once on a gbrain build other than the one that recorded it+ * (a record from before #9279 carries no build), so an upgrade that fixes the+ * refusal extracts it with no command.*/export function corpusFailureGate(record: CorpusFailureRecord | null,@@ -425,7 +429,10 @@ export function corpusFailureGate(nowMs: number,): 'run' | 'backoff' | 'dead_lettered' {if (!record || record.key !== key || record.source_incarnation !== incarnation) return 'run';- if (record.count >= CORPUS_WINDOW_FAILURE_CAP) return rearms(record, fingerprint) ? 'run' : 'dead_lettered';+ if (record.count >= CORPUS_WINDOW_FAILURE_CAP) {+ const rebuilt = CORPUS_DETERMINISTIC_FAILURE_CODES.includes(record.last_code) && record.build !== VERSION;+ return rebuilt || rearms(record, fingerprint) ? 'run' : 'dead_lettered';+ }if (record.count === 1) return 'run';return nowMs < record.last_at + CORPUS_WINDOW_RETRY_BASE_MS * 2 ** (record.count - 2) ? 'backoff' : 'run';}@@ -509,7 +516,10 @@ function failureCode(e: unknown): string {* #6785: count one failed attempt of the window at `key` (caller holds the* file's `.in-progress` claim). Another window or incarnation starts a new* count; the error label never resets it. A capped window re-armed by a- * fingerprint change starts over once. Returns the record written.+ * fingerprint change starts over once. #9279: a deterministic failure+ * (CORPUS_DETERMINISTIC_FAILURE_CODES) stops the window at once, stamped with+ * this build, instead of paying for four more identical refusals. Returns the+ * record written.*/export async function recordCorpusFailure(full: string,@@ -522,16 +532,20 @@ export async function recordCorpusFailure(): Promise<CorpusFailureRecord> {const same = !!prev && prev.key === key && prev.source_incarnation === incarnation;const rearm = same && prev!.count >= CORPUS_WINDOW_FAILURE_CAP && rearms(prev!, fingerprint);+ const code = failureCode(error);+ const deterministic = CORPUS_DETERMINISTIC_FAILURE_CODES.includes(code);+ const count = same && !rearm ? prev!.count + 1 : 1;const record: CorpusFailureRecord = {version: 1,key,source_incarnation: incarnation,- count: same && !rearm ? prev!.count + 1 : 1,+ count: deterministic ? Math.max(count, CORPUS_WINDOW_FAILURE_CAP) : count,first_at: same ? prev!.first_at : nowMs,last_at: nowMs,- last_code: failureCode(error),+ last_code: code,fingerprint,rearmed: same ? prev!.rearmed || rearm : false,+ ...(deterministic ? { build: VERSION } : {}),};const path = full + CORPUS_FAILURE_SUFFIX;const tmp = `${path}.tmp-${process.pid}`;diff --git a/src/core/facts/subject-infer-write.ts b/src/core/facts/subject-infer-write.tsindex ec149b922b..64979bc2c6 100644--- a/src/core/facts/subject-infer-write.ts+++ b/src/core/facts/subject-infer-write.ts@@ -13,7 +13,7 @@* so those facts stay unattributed. A link is also dropped whenever the* writer would refuse it: caller slug fences, the managed batch's grant and* page visibility, a withdrawn claim, or a malformed target fence (which- * would fail the whole write). Remote callers infer only from the page+ * would fail the whole write, read as that writer reads it: #9279). Remote callers infer only from the page* actually being written and never from pages they cannot read.*/@@ -39,12 +39,13 @@ export async function inferMissingSubjects(ctx: FactsBackstopCtx, facts: Extractconst excludePrivate = managed?.authority.excludePrivate === true || await excludesPrivateWrites(engine, remote);const fromPage = remote ? pageSlug : ctx.sourceSlug ?? pageSlug;const out: ExtractedFact[] = [];+ const fences = new Map<string, Promise<boolean>>();for (const f of facts) {// An extractor-named entity is kept even when it has no page: that name// competes with any page the text mentions, so inference never overrides it.if (!isNullLikeEntity(f.entity_slug)) { out.push(f); continue; }const inferred = await inferFactSubject(engine, ctx.sourceId, { fact: f.fact, pageSlug: fromPage, mode: 'write', excludePrivate });- if (inferred.slug !== null && await linkAllowed(ctx, inferred.slug, f.fact, visibility, remote, managed)) {+ if (inferred.slug !== null && await linkAllowed(ctx, inferred.slug, f.fact, visibility, remote, managed, fences)) {out.push({ ...f, entity_slug: inferred.slug, entity_inferred: inferred.via });} else out.push(f);}@@ -58,7 +59,7 @@ async function writesDatabaseOnly(engine: BrainEngine, sourceId: string): Promis}async function linkAllowed(ctx: FactsBackstopCtx, slug: string, fact: string, visibility: string, remote: boolean,- managed: ManagedFactsSession | null): Promise<boolean> {+ managed: ManagedFactsSession | null, fences: Map<string, Promise<boolean>>): Promise<boolean> {const engine = ctx.engine;const { isFactWithdrawn } = await import('./withdrawal.ts');if (await isFactWithdrawn(engine, ctx.sourceId, visibility, fact, slug)) return false;@@ -81,8 +82,21 @@ async function linkAllowed(ctx: FactsBackstopCtx, slug: string, fact: string, viif (error instanceof OperationError) return false;throw error;}- const page = await engine.readPageSnapshot(slug, { sourceId: ctx.sourceId });- if (!page) return false;+ if (!fences.has(slug)) fences.set(slug, targetFenceClean(engine, ctx.sourceId, slug, managed !== null));+ return fences.get(slug)!;+}++/**+ * #9279: whether the target's facts fence parses cleanly, read as its writer reads it. The managed writer reads a page+ * stored in segments back whole up to the single-string ceiling; the unmanaged fence writer records a segmented page's+ * facts database-only, and an over-ceiling page takes no fence row, so neither is a link target there.+ */+async function targetFenceClean(engine: BrainEngine, sourceId: string, slug: string, managed: boolean): Promise<boolean> {+ const snapshot = await engine.readPageSnapshot(slug, { sourceId });+ if (!snapshot) return false;+ const { isSegmentedPage } = await import('../page-body/page-shape.ts');+ const { materializedSnapshot, segmentedOverCeiling } = await import('../persistence/segmented-apply.ts');+ if (isSegmentedPage(snapshot.page) && (!managed || segmentedOverCeiling(snapshot.page))) return false;const { parseFactsFence } = await import('../facts-fence.ts');- return parseFactsFence(page.page.compiled_truth).warnings.length === 0;+ return parseFactsFence((await materializedSnapshot(engine, snapshot))!.page.compiled_truth).warnings.length === 0;}diff --git a/src/core/google/google-source.ts b/src/core/google/google-source.tsindex 913c31311f..c65dbb478e 100644--- a/src/core/google/google-source.ts+++ b/src/core/google/google-source.ts@@ -162,6 +162,22 @@ function loopRecoveryState(state: GoogleSourceState): Record<string, unknown> {return Object.fromEntries(Object.entries(fields).filter(([, value]) => value !== undefined));}+/**+ * #9011: a thread whose import was still pending when processThread ran detection, then refused as purged content,+ * leaves what detection derived from a page that never landed. Drops its extract candidate and grace hold, and the open+ * loops of that slug while no live page holds it (open_loops.page_slug has no foreign key).+ */+async function forgetRefusedPurgedThreads(deps: GoogleSyncDeps, managed: ManagedConnectorSync, state: GoogleSourceState): Promise<void> {+ await managed.drainPending();+ const slugs = managed.refusedPurgedSlugs();+ if (!slugs.length) return;+ const refused = new Set(slugs);+ deps.extractCandidates = deps.extractCandidates.filter((c) => !refused.has(c.slug));+ for (const [tid, hold] of Object.entries(state.loop_grace_holds ?? {})) if (hold.slug && refused.has(hold.slug)) delete state.loop_grace_holds![tid];+ await deps.engine.executeRaw(`DELETE FROM open_loops WHERE source_id=$1 AND page_slug=ANY($2::text[])+ AND NOT EXISTS (SELECT 1 FROM pages p WHERE p.source_id=$1 AND p.slug=open_loops.page_slug AND p.deleted_at IS NULL)`, [deps.sourceId, slugs]);+}+// ── Gmail sweep ──────────────────────────────────────────────────────────────const BACKFILL_BATCH_THREADS = 25;@@ -922,6 +938,7 @@ async function runGoogleSyncInner(engine: BrainEngine, sourceId: string, cfg: Go// Per-service cursors were advanced in-place only on success; persist.// Throws connector_holds_exhausted before any save, so the cursor stays.+ if (managed && !opts.signal?.aborted) await forgetRefusedPurgedThreads(deps, managed, state);state.item_holds = holds.finish();if (managed) {if (summary.status !== 'partial' && gmailSweepOk) await managed.saveState(state, true, new Date(state.gmail_newest_ms ?? Date.now()).toISOString());diff --git a/src/core/persistence/connector-sync.ts b/src/core/persistence/connector-sync.tsindex d9a2b4fd8d..ab39997b18 100644--- a/src/core/persistence/connector-sync.ts+++ b/src/core/persistence/connector-sync.ts@@ -731,7 +731,7 @@ export class ManagedConnectorSync {}}/** Waits, within the run's budget, for every outstanding receipt. */- private async drainPending(): Promise<void> {+ async drainPending(): Promise<void> {await this.refreshPending();while (this.pendingRows.size && this.remainingWait() > 0) {const [oldest] = this.pendingRows.values();@@ -830,6 +830,10 @@ export class ManagedConnectorSync {process.stderr.write(`[connector] ${this.sourceId}: skipped ${slug}: its content matches a page purge tombstone${row ? ` (request ${row.request_id})` : ''}; `+ `the rest of the sync continues. To let it back in: gbrain pages unpurge ${slug}\n`);}+ /** #9011: slugs this run skipped because the pages guard refused their admitted receipt as purged content. */+ refusedPurgedSlugs(): string[] {+ return [...new Set(this.purged.filter(item => item.request_id !== null).map(item => item.slug))];+ }/*** #6831: screens a page import against the source's page purge tombstones before admission, with the content hash* the pages guard compares (`intentCarriesPurgedContent`), so a purged item that keeps being listed costs no refuseddiff --git a/test/google-purged-pending-open-loops-9011.test.ts b/test/google-purged-pending-open-loops-9011.test.tsnew file mode 100644index 0000000000..312689d160--- /dev/null+++ b/test/google-purged-pending-open-loops-9011.test.ts@@ -0,0 +1,107 @@+/**+ * #9011: on a managed Google source, a purged Gmail thread whose import misses the session's purge screen and is still+ * pending when `importMarkdown` returns ran loop detection and wrote an open loop with the thread's subject and evidence+ * quote. The pages guard then refused the write as purged content from the pending path, but nothing removed what+ * detection wrote (`open_loops.page_slug` has no foreign key), so the subject and quote stayed across later runs.+ * Real PGLite and Postgres (when DATABASE_URL is set), the real Google importer, purge and connector session.+ */+import { afterAll, beforeAll, describe, expect, spyOn, test } from 'bun:test';+import type { BrainEngine } from '../src/core/engine.ts';+import { parseGoogleSourceConfig, runGoogleSync } from '../src/core/google/google-source.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { readManagedConnectorState } from '../src/core/persistence/connector-state.ts';+import { CONNECTOR_WAIT_BUDGET_MS, ManagedConnectorSync, connectorWaitBudget } from '../src/core/persistence/connector-sync.ts';+import { createConnectorFixture, json, options, settleConnectorWrites, withGoogleAccount } from './helpers/connector-fixture.ts';+import { withEnv } from './helpers/with-env.ts';++const THREAD = '17ff00000000c011';+const MESSAGE = '18ff00000000c011';+const SUBJECT = 'Contract question';+const QUOTE = 'Could you confirm the acme-example contract terms?';+const CONFIG = { kind: 'google', g_account: '[email protected]', g_services: 'gmail', g_access: 'env', g_token_env: 'CONNECTOR_TEST_TOKEN' };+const DAY_MS = 86_400_000;+const SECOND_MS = 1000;+const SENT_DAYS_AGO = 3;+const NO_ITEM_WAIT_MS = 0;+const LATER_RUNS = 2;+const TEST_TIMEOUT_MS = 240_000;+const SETUP_TIMEOUT_MS = 120_000;+const b64url = (text: string) => Buffer.from(text, 'utf-8').toString('base64url');+const ctx = (engine: BrainEngine, sourceId: string) => ({ engine, sourceId, config: { engine: engine.kind }, remote: false, dryRun: false,+ logger: { info() {}, warn() {}, error() {} } });++const managed = createConnectorFixture();+describe('#9011: a purged Gmail thread refused at publication after a pending import', () => {+ beforeAll(managed.setup, SETUP_TIMEOUT_MS);+ afterAll(managed.teardown);++ test('leaves no open loop with its subject or evidence quote once the pending refusal resolves', async () => withEnv(managed.env, async () => {+ for (const engine of managed.engines) {+ const f = await managed.source(engine, CONFIG);+ const cfg = parseGoogleSourceConfig(CONFIG, f.dir);+ const sentAt = Date.now() - SENT_DAYS_AGO * DAY_MS;+ let historyId = '5000';+ let changed: string[] = [];+ const message = { id: MESSAGE, threadId: THREAD, labelIds: ['INBOX'], internalDate: String(sentAt),+ payload: { mimeType: 'text/plain', headers: [{ name: 'From', value: 'Alice Example <[email protected]>' },+ { name: 'To', value: CONFIG.g_account }, { name: 'Subject', value: SUBJECT }],+ body: { data: b64url(QUOTE) } } };+ const fetch = withGoogleAccount(async (url: string) => {+ const u = new URL(url);+ if (u.pathname.endsWith('/settings/sendAs')) return json({ sendAs: [] });+ if (u.pathname.endsWith('/users/me/profile')) return json({ emailAddress: CONFIG.g_account, historyId: '5000' });+ if (u.pathname.endsWith('/users/me/messages')) return json({ messages: Number(/after:(\d+)/.exec(u.searchParams.get('q') ?? '')?.[1] ?? 0) * SECOND_MS <= sentAt+ && sentAt < Number(/before:(\d+)/.exec(u.searchParams.get('q') ?? '')?.[1] ?? Infinity) * SECOND_MS ? [{ id: MESSAGE, threadId: THREAD }] : [] });+ if (u.pathname.endsWith('/users/me/history')) return json({ historyId, history: u.searchParams.get('startHistoryId') === historyId ? []+ : [{ id: historyId, messages: changed.map(threadId => ({ id: MESSAGE, threadId })) }] });+ if (u.pathname.endsWith(`/users/me/threads/${THREAD}`)) return json({ id: THREAD, messages: [message] });+ throw new Error(`Unexpected Google fixture route: ${url}`);+ }, CONFIG.g_account);+ const run = () => runGoogleSync(engine, f.id, cfg, options, withGoogleAccount(fetch, CONFIG.g_account));+ const subjectLoops = async () => (await engine.executeRaw<{ n: number }>(+ 'SELECT count(*)::int AS n FROM open_loops WHERE source_id=$1 AND (thread_id=$2 OR summary ILIKE $3)', [f.id, THREAD, `%${SUBJECT}%`]))[0]!.n;+ const quotedLoops = async () => (await engine.executeRaw<{ n: number }>(+ 'SELECT count(*)::int AS n FROM open_loops WHERE source_id=$1 AND evidence::text ILIKE $2', [f.id, `%${QUOTE}%`]))[0]!.n;+ const livePages = async () => (await engine.executeRaw<{ slug: string }>('SELECT slug FROM pages WHERE source_id=$1 AND deleted_at IS NULL', [f.id])).map(r => r.slug);++ expect((await run()).status).not.toBe('partial');+ await settleConnectorWrites(engine, f.id);+ await disposePersistenceConsumer(engine);+ const [slug] = await livePages();+ expect(slug).toBeDefined();+ expect(await subjectLoops()).toBe(1);+ const snapshot = (await engine.readPageSnapshot(slug!, { sourceId: f.id }))!;+ await submitPageMutation(ctx(engine, f.id), { operation: 'delete_page', params: { slug: slug!, purge: true, expected_revision: snapshot.revision } });+ await settleConnectorWrites(engine, f.id);+ await disposePersistenceConsumer(engine);+ expect(await subjectLoops()).toBe(0);++ // The thread is listed again; the session's screen misses it and the item wait is over before the owner publishes,+ // so the import returns pending and loop detection runs. The owner then refuses it as purged content.+ historyId = '5010';+ changed = [THREAD];+ const screen = spyOn(ManagedConnectorSync.prototype as unknown as { carriesPurgedPage: () => Promise<boolean> }, 'carriesPurgedPage').mockResolvedValue(false);+ const quiet = spyOn(process.stderr, 'write').mockImplementation(() => true);+ try {+ connectorWaitBudget.ms = NO_ITEM_WAIT_MS;+ try { await run(); } finally { connectorWaitBudget.ms = CONNECTOR_WAIT_BUDGET_MS; }+ await settleConnectorWrites(engine, f.id).catch(() => {});+ await disposePersistenceConsumer(engine);+ expect(await livePages()).toEqual([]);+ // Later runs resolve the refused receipt from the pending set.+ for (let i = 0; i < LATER_RUNS; i++) {+ await run();+ await settleConnectorWrites(engine, f.id).catch(() => {});+ await disposePersistenceConsumer(engine);+ }+ } finally { quiet.mockRestore(); screen.mockRestore(); }+ expect(await livePages()).toEqual([]);+ const [row] = await engine.executeRaw<{ incarnation: string }>('SELECT incarnation::text FROM sources WHERE id=$1', [f.id]);+ const state = await readManagedConnectorState(engine, f.id, row!.incarnation);+ expect(state.purged_items?.filter(item => item.request_id !== null).map(item => item.slug)).toContain(slug!);+ expect(await subjectLoops()).toBe(0);+ expect(await quotedLoops()).toBe(0);+ }+ }), TEST_TIMEOUT_MS);+});diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex 874f285734..ac39782c1e 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -623,6 +623,7 @@ test/stale-plan-heal.test.tstest/stats-health-source-scope.test.tstest/status-snapshot-source-scope.test.tstest/strict-background-writers-gst2492.test.ts+test/sweep-segmented-entity-9279.test.tstest/sync-absent-from-worktree-s27.test.tstest/sync-admission-6712.test.tstest/sync-apply-held-deletions-gbra133.test.tsdiff --git a/test/sweep-segmented-entity-9279.test.ts b/test/sweep-segmented-entity-9279.test.tsnew file mode 100644index 0000000000..ac8eb08da4--- /dev/null+++ b/test/sweep-segmented-entity-9279.test.ts@@ -0,0 +1,236 @@+/**+ * #9279: captured session windows whose facts land on an entity page stored in segments.+ *+ * 1. Protects: a corpus window whose extracted facts resolve to a segmented entity page (named by the extractor, or+ * inferred from a mention, which is how most conversation facts arrive) commits them onto the page's ## Facts+ * table and the session file counts as extracted, on a managed Git source and on a repo-less brain; a refusal that+ * retrying cannot change (page_body_segmented) stops its window after the one paid attempt instead of five, retention+ * keeps the file it holds and doctor counts it; a window an older build stopped on that refusal is extracted by the+ * first sweep on a new build, with no command.+ * 2. Fails when: subject inference reads the whole body of a segmented page to check its fence (the getters throw+ * page_body_segmented after the model call), a deterministic refusal is paid for again on the same build, retention+ * deletes a held file at the un-ingested ceiling, or a stopped window waits for a hand-run retry after the upgrade.+ * 3. Real seams: the maintenance sweep and its corpus window runner, the managed persistence consumer, a page imported+ * large enough to be stored in segments, gateway test transports, PGLite and Postgres (testBackends).+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, utimesSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { claimWorktree } from '../src/core/persistence/ownership.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { opError } from '../src/core/ops/contract.ts';+import { configureGateway, resetGateway, __setChatTransportForTests, __setEmbedTransportForTests, type ChatResult } from '../src/core/ai/gateway.ts';+import { invokeAI } from '../src/core/ai/invocation-guard.ts';+import { CORPUS_INGESTED_SUFFIX, runMaintenanceSweep } from '../src/core/sweep.ts';+import { toCorpusText } from '../src/core/transcripts/claude-code-jsonl.ts';+import { CORPUS_FAILURE_SUFFIX, CORPUS_UNINGESTED_RETENTION_FACTOR, CORPUS_WINDOW_FAILURE_CAP, corpusBacklog, gcCorpusTurnFiles, readCorpusFailure } from '../src/core/context/corpus-segments.ts';+import { corpusFailureGate, corpusFileStat, corpusFrontierKey, runCorpusWindows } from '../src/core/context/corpus-windows.ts';+import { buildMemoryWritebackCheck } from '../src/commands/doctor/checks/memory-writeback.ts';+import { VERSION } from '../src/version.ts';+import type { CapabilityReport } from '../src/core/capability.ts';+import { spoolPath } from './helpers/corpus-spool.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { testBackends } from './helpers/test-backends.ts';+import { waitFor } from './helpers/wait-for.ts';+import { withEnv } from './helpers/with-env.ts';++const KEYED: CapabilityReport = { embeddings: { available: false }, extraction: { available: true, provider: 'anthropic' }, search: 'keyword-only', mode: 'keyed' };+const PERSON = { slug: 'people/alice-example', title: 'Alice Example' };+const NAMED = 'Alice Example ships the example roadmap';+const INFERRED = 'Alice Example will send the deck by Friday';+const DIMS = 1536;+/** Fence rows on the entity page: about 1.5 MB, over the 1 MiB inline bound, so it is stored in segments. */+const FENCE_ROWS = 4000;+const TEST_MS = 240_000;+const SETTLE_MS = 30_000;+const SWEEP_BUDGET_MS = 120_000;+const DAY_MS = 24 * 60 * 60 * 1000;+const RETENTION_DAYS = 30;++let extractions = 0;+const usage = { input_tokens: 5, output_tokens: 5, cache_read_tokens: 0, cache_creation_tokens: 0 };++function entityPage(): string {+ const rows = Array.from({ length: FENCE_ROWS }, (_, i) => '| ' + (i + 1) + ' | Alice Example noted item ' + i + ' of the example roadmap, one of a long run of notes kept on this page over the years, with a longer tail of words that makes each row of the table wider so the page grows past the inline bound | fact | 0.9 | world | medium | 2026-01-01 | | meetings/2026-01-0' + (i % 9 + 1) + ' | a context note that adds some more length to this row ' + i + ' |');+ return '---\ntitle: ' + PERSON.title + '\ntype: person\n---\n# ' + PERSON.title + '\n\n## Facts\n\n<!--- gbrain:facts:begin -->\n'+ + '| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |\n'+ + '|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|\n'+ + rows.join('\n') + '\n<!--- gbrain:facts:end -->\n';+}++const transcript = () => toCorpusText([+ { role: 'user', text: 'Planning notes with Alice Example about the example roadmap. '.repeat(40) },+ { role: 'assistant', text: 'Summary of the example roadmap discussion. '.repeat(40) },+]);++beforeAll(() => {+ configureGateway({ chat_model: 'anthropic:claude-sonnet-4-6', embedding_model: 'openai:text-embedding-3-large', embedding_dimensions: DIMS,+ env: { ANTHROPIC_API_KEY: 'sk-ant-test', OPENAI_API_KEY: 'sk-test' } });+ __setChatTransportForTests(async (): Promise<ChatResult> => {+ extractions++;+ const facts = [+ { fact: NAMED, kind: 'fact', entity: PERSON.slug, confidence: 0.9, notability: 'high' },+ { fact: INFERRED, kind: 'commitment', entity: null, confidence: 0.9, notability: 'high' },+ ];+ return { text: JSON.stringify({ facts }), blocks: [], stopReason: 'end', model: 'anthropic:claude-sonnet-4-6', providerId: 'anthropic', usage };+ });+ __setEmbedTransportForTests((async ({ values }: { values: string[] }) => ({ embeddings: values.map(() => Array.from({ length: DIMS }, (_, i) => (i === 0 ? 1 : 0))) })) as never);+});++afterAll(() => {+ __setChatTransportForTests(null);+ __setEmbedTransportForTests(null);+ resetGateway();+});++for (const kind of testBackends()) {+ describe('corpus sweep onto a segmented entity page (' + kind + ', #9279)', () => {+ let engine: BrainEngine;+ let close: () => Promise<void>;+ const dirs: string[] = [];++ beforeAll(async () => {+ if (kind === 'postgres') ({ engine, close } = await isolatedPersistencePostgres(process.env.DATABASE_URL!));+ else { engine = new PGLiteEngine(); await engine.connect({ engine: 'pglite' } as never); await engine.initSchema(); close = () => engine.disconnect(); }+ }, TEST_MS);++ afterAll(async () => {+ await disposePersistenceConsumer(engine).catch(() => undefined);+ await close?.();+ for (const dir of dirs) rmSync(dir, { recursive: true, force: true });+ }, TEST_MS);++ /** A source (managed Git checkout, or none) holding the segmented entity page, with one captured session file. */+ async function brain(dir: string, mode: 'git' | 'repoless') {+ const root = join(dir, 'checkout'); mkdirSync(root);+ const corpusDir = join(dir, 'sessions'); mkdirSync(corpusDir);+ const sourceId = 'source-' + randomUUID().slice(0, 8);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await engine.executeRaw('INSERT INTO sources(id,name,local_path) VALUES($1,$1,$2)', [sourceId, mode === 'git' ? root : null]);+ for (const [key, value] of [['sync.write_through', 'true'], ['facts.extraction_enabled', 'true'], ['dream.synthesize.session_corpus_dir', corpusDir],+ ['facts.default_visibility', 'world']]) await engine.setConfig(key, value);+ if (mode === 'git') await claimWorktree(engine, sourceId, root);+ const ctx = { engine, sourceId, remote: false as const, config: { engine: engine.kind, embedding_disabled: true },+ dryRun: false, logger: { info() {}, warn() {}, error() {} } } as unknown as OperationContext;+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug: PERSON.slug, request_id: randomUUID(), content: entityPage() } });+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+ const sessionFile = spoolPath(corpusDir, 'example-session.txt', sourceId);+ writeFileSync(sessionFile, transcript());+ const logs: string[] = [];+ const sweep = async () => {+ const report = await runMaintenanceSweep(engine, { sourceId, capabilities: KEYED, budgetMs: SWEEP_BUDGET_MS, log: line => logs.push(line) });+ await waitFor(async () => (await engine.executeRaw<{ n: number }>(+ `SELECT COUNT(*)::int AS n FROM persistence_requests WHERE source_id=$1 AND state IN ('queued','running','recovering')`, [sourceId]))[0].n === 0,+ { timeoutMs: SETTLE_MS, label: 'fact requests settled' });+ return report.corpusIngested;+ };+ const facts = () => engine.executeRaw<{ fact: string; slug: string; fenced: boolean }>(+ `SELECT fact, source_markdown_slug AS slug, row_num IS NOT NULL AS fenced FROM facts WHERE source_id=$1 AND source='sweep:corpus' ORDER BY fact`, [sourceId]);+ const storage = async () => (await engine.executeRaw<{ s: string }>('SELECT body_storage AS s FROM pages WHERE source_id=$1 AND slug=$2', [sourceId, PERSON.slug]))[0]?.s;+ const [{ incarnation }] = await engine.executeRaw<{ incarnation: string }>('SELECT incarnation::text AS incarnation FROM sources WHERE id=$1', [sourceId]);+ return { dir, corpusDir, sourceId, incarnation, sessionFile, logs, sweep, facts, storage };+ }++ /** One test's brain under its own GBRAIN_HOME (the worktree owner's host identity lives there). */+ async function withBrain(mode: 'git' | 'repoless', body: (b: Awaited<ReturnType<typeof brain>>) => Promise<void>) {+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-9279-'));+ dirs.push(dir);+ await withEnv({ GBRAIN_HOME: join(dir, 'home'), GBRAIN_AUDIT_DIR: join(dir, 'audit') }, async () => {+ try { await body(await brain(dir, mode)); } finally {+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ }+ });+ }++ for (const mode of ['git', 'repoless'] as const) {+ test('a window whose named and inferred facts land on the segmented page commits them, fenced, in one paid call (' + mode + ')', async () => {+ extractions = 0;+ await withBrain(mode, async b => {+ expect(await b.storage()).toBe('segmented');+ expect(await b.sweep()).toBe(1);+ expect(b.logs.join('\n')).not.toContain('page_body_segmented');+ expect(extractions).toBe(1);+ expect(existsSync(b.sessionFile + CORPUS_INGESTED_SUFFIX)).toBe(true);+ expect(existsSync(b.sessionFile + CORPUS_FAILURE_SUFFIX)).toBe(false);+ expect(await b.facts()).toEqual([+ { fact: NAMED, slug: PERSON.slug, fenced: true },+ { fact: INFERRED, slug: PERSON.slug, fenced: true },+ ]);+ expect(await b.storage()).toBe('segmented');+ expect(await b.sweep()).toBe(0);+ expect(extractions).toBe(1);+ });+ }, TEST_MS);+ }++ test('a window an older build stopped on page_body_segmented is extracted by the first sweep after the upgrade, with no command', async () => {+ extractions = 0;+ await withBrain('repoless', async b => {+ const key = await corpusFrontierKey(b.sessionFile, 'example-session.txt');+ writeFileSync(b.sessionFile + CORPUS_FAILURE_SUFFIX, JSON.stringify({ version: 1, key, source_incarnation: b.incarnation, count: CORPUS_WINDOW_FAILURE_CAP,+ first_at: Date.now() - DAY_MS, last_at: Date.now() - DAY_MS, last_code: 'page_body_segmented', fingerprint: null, rearmed: false }) + '\n');+ expect(await b.sweep()).toBe(1);+ expect(extractions).toBe(1);+ expect(existsSync(b.sessionFile + CORPUS_FAILURE_SUFFIX)).toBe(false);+ expect((await b.facts()).map(f => f.fact)).toEqual([NAMED, INFERRED]);+ });+ }, TEST_MS);++ test('retention keeps a file held by a deterministic refusal and doctor counts it; an ordinary stopped file still goes', async () => {+ await withBrain('repoless', async b => {+ const old = (Date.now() - (RETENTION_DAYS * CORPUS_UNINGESTED_RETENTION_FACTOR + 1) * DAY_MS) / 1000;+ const record = (code: string) => JSON.stringify({ version: 1, key: 'k@0', source_incarnation: b.incarnation, count: CORPUS_WINDOW_FAILURE_CAP,+ first_at: 1, last_at: 1, last_code: code, fingerprint: null, rearmed: false, build: VERSION }) + '\n';+ const held = spoolPath(b.corpusDir, 'held-session.txt', b.sourceId);+ const ordinary = spoolPath(b.corpusDir, 'ordinary-session.txt', b.sourceId);+ for (const [file, code] of [[held, 'page_body_segmented'], [ordinary, 'provider_error']] as const) {+ writeFileSync(file, transcript());+ writeFileSync(file + CORPUS_FAILURE_SUFFIX, record(code));+ utimesSync(file, old, old);+ }+ const check = await buildMemoryWritebackCheck(engine);+ expect(check.details).toMatchObject({ corpus_failed_windows: { stopped: 2, held: 1 } });+ expect(corpusBacklog(join(b.corpusDir, 'sourced'), RETENTION_DAYS * DAY_MS)).toMatchObject({ pastRetention: 2, held: 1 });+ expect(check.message).toContain('1 of them stopped at once on a refusal that retrying cannot change');+ gcCorpusTurnFiles(join(b.corpusDir, 'sourced'), RETENTION_DAYS * DAY_MS);+ expect(existsSync(held)).toBe(true);+ expect(existsSync(ordinary)).toBe(false);+ });+ }, TEST_MS);+ });+}++describe('a deterministic refusal after the model call (#9279)', () => {+ test('stops its window after the one paid attempt on this build, and runs once on another build', async () => {+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-9279-windows-'));+ try {+ const full = join(dir, 'refused-session.txt');+ writeFileSync(full, transcript());+ let paid = 0;+ const run = async () => runCorpusWindows({+ full, raw: readFileSync(full, 'utf8'), fileStat: corpusFileStat((await import('node:fs')).statSync(full)), maxWindows: 1,+ overBudget: () => false, signal: new AbortController().signal, failures: { fingerprint: null },+ extract: async () => {+ await invokeAI({ operation: 'extract', model: 'anthropic:test', kind: 'chat' }, async () => { paid++; return 'facts'; }, () => null);+ throw opError('page_body_segmented', 'The entity page is stored in segments.', 'Retrying refuses the same way.');+ },+ }).catch((e: unknown) => e);+ await run();+ expect(paid).toBe(1);+ const record = readCorpusFailure(full)!;+ expect(record).toMatchObject({ count: CORPUS_WINDOW_FAILURE_CAP, last_code: 'page_body_segmented', build: VERSION });+ expect(await run()).toMatchObject({ status: 'dead_lettered' });+ expect(paid).toBe(1);+ expect(corpusFailureGate({ ...record, build: '0.0.0.1' }, record.key, null, null, Date.now())).toBe('run');+ expect(corpusFailureGate({ ...record, last_code: 'provider_error' }, record.key, null, null, Date.now())).toBe('dead_lettered');+ } finally { rmSync(dir, { recursive: true, force: true }); }+ }, TEST_MS);+});