Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix: segmented pages no longer block a source's sync (freeze hold, streamed DB-only re-export, generation-consistent reads)

garrytane08f2a3f17e3
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e1ef3a31af..1f2bc20f91 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,23 @@ credits are retained; no result has been reassigned to another provider. Origina
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
+## [Unreleased]
+
+**A page stored in segments no longer stops a source's sync.** A managed sync re-importing a Git-tracked page stored in segments (`pages.segmented_storage`) whose working-tree bytes differ from the pinned commit, as on a live checkout, compared the file against the stored page with a whole-body read. That read refused with `page_body_segmented`, recorded as a run failure in the freeze phase, and every later sync stopped `sync_blocked` at the same file, with a retry command that could not help. The freeze now compares a segmented page up to 24 MiB read back whole, as its publication already did, so the page gets the outcome an inline page gets. A page whose stored body can't be read at the freeze (`page_body_segmented`, or `page_body_rows_unreadable`) is held `page_write_failed`, naming its code, and the rest of the source syncs. `gbrain sync status` classifies both codes instead of reporting them as unknown.
+
+A full sync's DB-only reconcile (#2426) now re-exports a page stored in segments instead of refusing it: the file is streamed from the segments in one snapshot through the staging writer the withdrawal mirror and restore_page use, written under an in-progress name and renamed once whole, with the bytes `gbrain export` writes. Pages whose re-export fails are listed in the result's `reexport_failures`, and the run completes. A segmented read no longer refuses a healthy page `page_body_rows_unreadable` because a new body published between its reads.
+
+### Itemized changes
+
+- `src/core/page-state/import-guard.ts` (`sameCanonicalImport`): a stored body left unread (over 24 MiB) differs from any parsed body, as `sameContentAnyKeyOrder` already said, instead of reading the throwing getter.
+- `src/core/persistence/sync-screen.ts` (`frozenImportScreenInput`): the freeze screen reads the page snapshot (and a rename's base) through `materializedSnapshot`.
+- `src/core/persistence/sync-entry-fault.ts` (`freezeFaultHold`): `page_body_segmented` and `page_body_rows_unreadable` thrown while freezing one entry hold that entry `page_write_failed` (origin `freeze`, `write_failure.code` the page-body code) instead of stopping the run. The item breaker still applies.
+- `src/core/persistence/sync-fault-class.ts`, `public-category.ts`: rows for both codes (`page_body_segmented`: upgrade, a person; `page_body_rows_unreadable`: retry, the segment-table grant).
+- `src/core/write-through.ts` (`writePageThrough`): a page stored in segments is measured and streamed from its segments inside one repeatable-read read-only transaction (`unreadPageFile`, `stageStreamedContent`), then renamed over the target after the file-changed check; frontmatter overrides on such a page are refused with an error result.
+- `src/commands/sync/full.ts`: the DB-only re-export no longer refuses segmented pages, and each failed re-export is returned in `reexport_failures` (slug, path, code, message, fix).
+- `src/core/page-body/body-tiling.ts` (`readTiledManifest`), `ingest.ts`, `read-range.ts`, `projection.ts`, `segment-rewrite.ts`: a live read names its generation first and checks that generation's rows against that generation's size; a live read whose generation a publish retired meanwhile reads the new live generation (up to 3 attempts).
+- Tests: `test/large-pages/managed-sync-segmented-freeze.test.ts` (the live-checkout freeze, with its inline twin, and a page whose rows don't tile held while the rest of the commit imports), `test/sync-reconcile-db-only-segmented.test.ts` (2 MiB and over-24 MiB re-exports byte-equal to `gbrain export`, one failing page listed while the run advances, a plain-directory source), `test/segmented-read-concurrent-publish.test.ts` (a forced publish between a reader's statements). Every case but the plain-directory one fails on master.
+
## [0.60.558.0] - 2026-11-03
**The QM guide and the admin guide now warn that the attest client belongs only on QM's personal-scope memory route.** An attested call reads as the named person, including their own private facts, and gbrain can't tell who else reads the answer. Registering that client as a model-callable MCP connector, or routing it to a shared scope (a channel, group, project, team or the org), could show one person's private facts to a whole room. Shared conversations wait for `x-qm-audience` (wave 2). Docs only: no behavior changes.
diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsv
index eca661562c..a30316f5ad 100644
--- a/scripts/module-size-limits.tsv
+++ b/scripts/module-size-limits.tsv
@@ -12,7 +12,7 @@ src/commands/extract.ts 2700 ratchet +8 GBRA-130 A-1: extract --stale --retry-he
src/commands/jobs.ts 687 ratchet refactor wave 1 split target (W4 jobs, cut line); ceiling rationale: docs/architecture/key-files/commands-4.md entry for src/commands/jobs.ts; help lines: authorize-legacy --select (#5157 T3) and jobs cancel --select (DX-T2); submit --queue-only + PGLite work drain note (agent-first operator wave E5); +2 spend authorization: jobs submit --yes/--max-usd and jobs list --group help lines; +4 fix wave 9: supervisor stop help documents the 45 s wait, --queue and the verified-drain exit codes; +1 #7142: doctor-refresh handler registration (logic in doctor/last-known.ts); +20 #7242 jobs wait: subcommand row, import, usage lines and its --help text with the exit codes (logic in jobs/wait.ts)
src/commands/serve-http.ts 941 ratchet refactor wave 1 W4 split into serve-http-<area>.ts modules; ceiling rationale: docs/architecture/key-files/commands-5.md entry for src/commands/serve-http.ts; +2 agent operator wave A6: ServeHttpContext.noticeLedger (import + field; logic in core/notice-ledger.ts); +2 agent contract F1/F2: ServeHttpContext.readinessCache (probed readiness for initialize tails + gbrain://capabilities); +5 Lane D facts drain: serve --http starts the drain scheduler and stops it before the engine disconnect (logic in core/facts/drain-scheduler.ts); +2 agent-operator follow-up Item 3: ServeHttpOptions.adoptServer (status-only recovery hands its bound listener to listenOrRefuse; logic in serve-http-status.ts); +5 #6317: managed-sync movement watch start/stop (watch in persistence/sync-movement.ts); +1 #6589: imports loadServeConfigs (operations see the DB-plane config read once at startup; logic in mcp/serve-config.ts); +5 #6858 ask 6: on-commit HEAD watcher start/stop (watcher in core/serve-head-watcher.ts); +4 GBRA-199: serve --http --advertise parse, help line and pass-through (logic in mcp/surface.ts parseAdvertiseFlag)
src/core/ai/gateway.ts 4538 ratchet +5 GBRA-274 B1.3: withoutBudgetTracker export (runs a rollout hook outside the run's budget scope; the budget AsyncLocalStorage is gateway-private, so it cannot move); +2 #6997: ChatResult.fallbackReason (interface field + doc; recorded in ai/chat-fallback.ts); +17 #6607: lazy OpenAI model discovery (discoverBeforeOpenAICall, once per reconfigure, at the chat/expand/embed entry points; it re-resolves through the gateway-internal resolveGatewayModels, so it cannot move); +18 W11 item 3: embedQuery query-embedding cache lookup/store + clear at every model-cache clear site and the test transport seam (cache lives in ai/query-embed-cache.ts); +7 fix wave 5 (#5137): provider-key redactor on every upstream error path and the first embedding 401/403 report (logic in ai/key-redact.ts, ai/key-warnings.ts); +20 System One v1: decide() delegating export + transport seam, RerankInput.onMeta/RerankMeta, TypeSafe reranker delegation to ai/decide/rerank-adapter.ts; +3 #5585: reconfigureGatewayWithEngine records expansion/chat model provenance (ai/gateway-model-sources.ts) + clears it with gateway state; lowered #5585: tracker record sites collapsed onto ai/budget-record.ts (recordOnTracker + failedCallUsage) while adding ledger fields (requested model, purpose, failed, estimated, countsAsCall); +20: schema-rejection fallback in chat() (retry once schemaless on a json_schema 4xx, remember the recipe for the process — mirrors expand()); re-summed after merging master v0.50.0.0 with the wave (structured-output + pricing hunks); +48: #4863 ChatOpts.responseSchema → tolerant json_schema Output on the chat() generateText call for openai-compatible recipes that declare structured outputs (facts extraction on Ollama); +74: #3657 post-sunset rerank short-circuit (effective-model check, once-per-process audit row + stderr, injected-clock seam); grown v0.46.26.0 megawave: config-plane probes + halt telemetry (#3387 #4312); +21 lines (OpenAI Responses API reasoning-item echo: ChatBlock 'reasoning' variant, chat()/toModelMessages() round-trip); wave-k absorbs: OCR routing seam (#4107 class), config-snapshot reader block (#3980); +21: #4107 OCR-model routing (getImageOcrModel); +10: keyless OPENAI_BASE_URL embedding override (#4385); wave-k #3622 reimpl: GBRAIN_EMBED_MAX_BATCH_TOKENS env cap for no_batch_cap recipes; +9: DeepSeek judge thinking pin plumbed through chat opts (#4069); +master v0.48 merge true-up (reranker succession: no_key preflight, reranker_skipped stage, readiness); ranker wave v0.48.4.0: ChatOpts.temperature threaded to the transport (+15); +24: #4172 recipe-declared thinking headroom on the chat()/toolLoop() default — shared isThinkingModel predicate (Claude-5 regex || recipe thinking_by_default, fail-closed), also consumed by the subagent handler; +20: #4861 claude-cli expansion routed to the schemaless viaText path (the CLI adapter ignores responseFormat, so the native generateObject branch had no fallback); retired provider integration removed; safety wave: migration reranker input ceilings; lowered by fix wave 3 #5680 (embed request planning peeled to ai/embed-batch-plan.ts); +5 #4616: empty inputs skip the provider and every text/multimodal result is screened for zero-norm/NaN/Inf vectors (logic lives in ai/embedding-guard.ts); fix wave 3 integration: #4616 empty-input filtering feeds the shared #5680 batch planner; +3 #5691: embedQuery takes a per-request queryPrefix (no gateway state); +26 fix wave 7 (#5642): embed spend records bank provider-reported usage.tokens (all sub-batches reported → measured; chars-per-token estimate stays the fallback and the failure charge); +2 #5892: import + load-time call of installAiSdkWarningWriter (writer lives in ai/sdk-warnings.ts); +12 #5490: chat() delegation to ai/chat-fallback.ts when chat_fallback_chain is set (attempt loop lives there), ChatOpts.allowFallback + ToolLoopOpts.allowFallback fields and the toolLoop forward; +2 #5490 review: ChatResult.fallbackFrom (the call model a chat_fallback_chain answer replaced); +1 fix wave lane B3: configureGateway keeps chat_fallback_on_refusal (the refusal switch lives in ai/chat-fallback.ts); +18 fix wave 9 lane B: ChatOpts.thinking (#5331, per-route mapping lives in ai/thinking-off.ts), ChatOpts.timeoutMs + ToolLoopOpts.turnTimeoutMs and the turn forward (#4921); interface fields and their docs cannot move; +2 P8: GBRAIN_AI_CALL_LOG observer install in configureGateway (logic in ai/call-log.ts); +2 W9F item 5: thinkingOffOutputCap export shared by chat() and the judge preflight estimates (capability table lives in ai/thinking-off.ts); +3 A1 (wave 0): ChatResult.usage.reasoning_tokens (interface field + doc) and the chat() reasoning-subset read beside the cache counts; the ai-sdk importer allowlist is shrink-only, so the seam stays here); +9 GOOGLE_GENERATIVE_AI_BASE_URL: resolveNativeBaseUrl('google') + the three native-google construction sites (eval metering proxy); re-summed at the master 6ccf30c3e merge (4417 integration + 4423 master - 4408 base); lowered -5: the expand() prompt template peeled to ai/expansion-prompt.ts (buildExpansionPrompt) so the longmemeval call planner measures the same text; +5 #7176: expand() writes its chat_usage_log row (phase expansion) beside the budget record, and the guardedExpansion instance that names its invocation gateway.expand (ledger in ai/chat-usage.ts); +2 PR 4 (Wave B incremental derivation, transcripts) 4b round 4: chat() forwards a metered page step's providerRetries (BudgetTracker) as the SDK's maxRetries, so every attempt is one it reserved for
-src/commands/sync.ts 559 ratchet refactor wave 1 split target (W4 sync); ceiling rationale: docs/architecture/key-files/commands-6.md entry for src/commands/sync.ts; +2 for the #5751 SyncResult.legacySkips interface field (fix wave 4); +2 fix wave 4 lane B: SyncResult.connectorHolds interface field (held counts for the sync summary); +1 fix wave 5 (#5032): SyncResult.fileRefusals interface field (named per-file Windows colon refusals); +13 #5984: SyncResult.drain / managedCursor and SyncOpts.drain / drainStartedAt interface fields (drain loop lives in persistence/sync-drain.ts); +1 #5984: SyncOpts.noBulk / bulk interface fields; v0.60.48.0 merge: #5984 growth re-summed over v0.60.47.0 (510 measured); +2 #5984 lanes: SyncOpts.lanes field and the lanes progress-event field; +2 #6278: SyncResult.breaker interface field (the preparation-stall breaker verdict); +1 fix wave 13 P4.3 #6386: the JSON-output comment names writeJsonDocument for sync status/unblock; +2 fix wave 13 PR2: the transaction-pooler stderr line (P2.2) and the staged catch-up result field (P2.14, loop in commands/sync/perform.ts); +2 fix wave 15 P4.5 (#5459 adjacent): SyncResult.working_tree_kept interface field (the tree-walk keep lives in persistence/sync-discovery.ts); +4 #6643: SyncResult.retryWait / knockOns interface fields (recovery lives in persistence/sync-run.ts, sync-knock-on.ts); +2 #6696 (A7): SyncOpts.capacityPreflight interface field (preflight lives in persistence/sync-capacity-preflight.ts); +2 #6696 C1: SyncOpts.embedDeferred interface field (the cost gate's runtime embed deferral; an interface field cannot move), composed with PR A's SyncOpts.capacityPreflight; +2 #6693: SyncResult.replayedRemembers interface field (the post-sync replay lives in commands/sync/perform.ts and repair/failed-writes.ts); +12 #6712 PR 1: SyncResult.leaseHeld / inherited_options / cursorAdopted / cursorFanIn and SyncOpts.drainHolder / followDrainLease interface fields (the lease lives in persistence/sync-lease.ts, adoption in persistence/sync-cursor-key.ts and sync-admitted.ts); +2 #6712 PR 2 (L5): SyncResult.tail interface field (the tail lives in persistence/sync-tail.ts); +2 #6712 PR 2 integration: L4 ghostPages and L5 tail SyncResult interface fields both land (each lane raised 542->544 alone); +2 #6948: SyncResult.origin_twins_resolved interface field (the heal lives in repair/origin-twins-heal.ts); +5 #6712 PR 4b: SyncResult.pass / reconcile and SyncOpts.costApproved interface fields (an interface field cannot move; the pass summary and the pass reason union live in persistence/sync-pass-row.ts, the remote reconcile in sync-pass-apply.ts); lowered to 546 #6712 PR 4b lane C: SyncResult.cursorAdopted / cursorFanIn retired with the cursor engine; +2 #7167: SyncResult.writer interface field (a managed dry run reports the writer instead of waiting on it; the read lives in persistence/sync-dry-run-writer.ts); +6 G8a git-canonical: SyncOpts.canonicalOverride interface field and its doc comment (internal: only the conversion apply in persistence/git-conversion.ts passes it; applied in persistence/sync-discovery.ts); +2 #7866: SyncOpts.fullRound interface field and its doc comment (internal: the conversion strict passes in persistence/git-conversion.ts set it; read in persistence/sync-pass-run.ts); +1 #8007: SyncResult.storage_invalid_kept interface field (the keep lives in persistence/sync-discovery.ts)
+src/commands/sync.ts 561 ratchet refactor wave 1 split target (W4 sync); ceiling rationale: docs/architecture/key-files/commands-6.md entry for src/commands/sync.ts; +2 for the #5751 SyncResult.legacySkips interface field (fix wave 4); +2 fix wave 4 lane B: SyncResult.connectorHolds interface field (held counts for the sync summary); +1 fix wave 5 (#5032): SyncResult.fileRefusals interface field (named per-file Windows colon refusals); +13 #5984: SyncResult.drain / managedCursor and SyncOpts.drain / drainStartedAt interface fields (drain loop lives in persistence/sync-drain.ts); +1 #5984: SyncOpts.noBulk / bulk interface fields; v0.60.48.0 merge: #5984 growth re-summed over v0.60.47.0 (510 measured); +2 #5984 lanes: SyncOpts.lanes field and the lanes progress-event field; +2 #6278: SyncResult.breaker interface field (the preparation-stall breaker verdict); +1 fix wave 13 P4.3 #6386: the JSON-output comment names writeJsonDocument for sync status/unblock; +2 fix wave 13 PR2: the transaction-pooler stderr line (P2.2) and the staged catch-up result field (P2.14, loop in commands/sync/perform.ts); +2 fix wave 15 P4.5 (#5459 adjacent): SyncResult.working_tree_kept interface field (the tree-walk keep lives in persistence/sync-discovery.ts); +4 #6643: SyncResult.retryWait / knockOns interface fields (recovery lives in persistence/sync-run.ts, sync-knock-on.ts); +2 #6696 (A7): SyncOpts.capacityPreflight interface field (preflight lives in persistence/sync-capacity-preflight.ts); +2 #6696 C1: SyncOpts.embedDeferred interface field (the cost gate's runtime embed deferral; an interface field cannot move), composed with PR A's SyncOpts.capacityPreflight; +2 #6693: SyncResult.replayedRemembers interface field (the post-sync replay lives in commands/sync/perform.ts and repair/failed-writes.ts); +12 #6712 PR 1: SyncResult.leaseHeld / inherited_options / cursorAdopted / cursorFanIn and SyncOpts.drainHolder / followDrainLease interface fields (the lease lives in persistence/sync-lease.ts, adoption in persistence/sync-cursor-key.ts and sync-admitted.ts); +2 #6712 PR 2 (L5): SyncResult.tail interface field (the tail lives in persistence/sync-tail.ts); +2 #6712 PR 2 integration: L4 ghostPages and L5 tail SyncResult interface fields both land (each lane raised 542->544 alone); +2 #6948: SyncResult.origin_twins_resolved interface field (the heal lives in repair/origin-twins-heal.ts); +5 #6712 PR 4b: SyncResult.pass / reconcile and SyncOpts.costApproved interface fields (an interface field cannot move; the pass summary and the pass reason union live in persistence/sync-pass-row.ts, the remote reconcile in sync-pass-apply.ts); lowered to 546 #6712 PR 4b lane C: SyncResult.cursorAdopted / cursorFanIn retired with the cursor engine; +2 #7167: SyncResult.writer interface field (a managed dry run reports the writer instead of waiting on it; the read lives in persistence/sync-dry-run-writer.ts); +6 G8a git-canonical: SyncOpts.canonicalOverride interface field and its doc comment (internal: only the conversion apply in persistence/git-conversion.ts passes it; applied in persistence/sync-discovery.ts); +2 #7866: SyncOpts.fullRound interface field and its doc comment (internal: the conversion strict passes in persistence/git-conversion.ts set it; read in persistence/sync-pass-run.ts); +1 #8007: SyncResult.storage_invalid_kept interface field (the keep lives in persistence/sync-discovery.ts); +2 GBRA-287: SyncResult.reexport_failures interface field (the re-export lives in commands/sync/full.ts)
src/core/cycle.ts 3345 ratchet +2 GBRA-130 C-2 (#6946): the sync phase reads an owned clone's pull block and reports it as pull_blocked (logic in url-clone.ts urlClonePullBlock/urlClonePullBlockSummary); +6 GBRA-130 A-1: the extract phase warns on failed or breaker-stopped stale drain pages (logic in cycle/connector-extract.ts staleDrainFailures); +1 W1 belief ledger: the extract phase runs the sampled belief drift reconcile (logic in belief/coverage-report.ts); +2 System One v1 S9: extract_facts phase tail runs the conflict sweep (ai/decide/sweep.ts) only when the slot is not off; +2 System One v1: purge phase prunes decision receipts (ai/decide/store.ts); grown v0.46.26.0 megawave (#4102 #2608) + master v0.46.26.0 cycle-start orphan recovery; +10 wave G #3974: cycle-sync uncommitted-drift warn; grown wave-g (#4416): synthesize_concepts source threading; grown wave-g (#4416): synthesize_concepts source threading (same fix absorbed as #4417 on wave-k); wave-k: ); +7: #4077 abort threading; +10: #4745 fullImplicitSourceCycle opt-in contract; +3: #4890 extract phase jsonMode/quiet + pages-not-links totals; +1: wave review — extract phase passes quiet:true + jsonMode:false; +1: extract_facts pagesFailed detail; +8 fix wave 3: one cycle date resolved in runCycle and threaded to synthesize/patterns/drift (C-15); +1 #5632: cycle sync resumes an unfinished managed cursor with its stored options; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +1 at the merge; -41 #5809: anyAbortSignal peeled to abort-signals.ts; +5 #5809: extract_atoms phase gets a real abort signal; +4 #5809: cycle_already_running names the lock holder; +7 #5255: cycle sync runs a managed pull request as no-pull and reports upstream_refresh; +1 #4578: a lock steal during the last selected phase reports lock_stolen, not aborted; +3 #5536: keyless embed phase clean-skips when embedding_disabled is set; +5 F4b: planner-stats refresh after the freshness phases (logic in planner-stats.ts); +2 agent operator wave A1: PhaseResult sibling code/fix; +20 #5876: Life Chronicle phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block; v0.60.46.0 merge: re-summed to the merged line count (3224); +3 retrieval feedback retention in the purge phase (logic in feedback/store.ts pruneRetrievalFeedback); 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); P3 x foundations-2 composition: deltas re-summed at the merge; 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); +15 GBRA-49 temporal typed edges: edge_contradictions phase registration + dispatch block + extract-phase relationship sweep call (logic in cycle/edge-contradictions.ts, link-relationships.ts); P3 x P1 (v0.60.57.0) composition: deltas re-summed at the merge; +1 fix wave 9 lane P (#5151): extract_facts phase passes drain so the facts watermark drain runs (logic in facts/reconcile-watermark.ts); +10 #6188: fence_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in cycle/fence-repair.ts); +19 fix wave 12 #6242: source cycles with brain-wide phases also take the shared gbrain-cycle lease (acquireDbCycleLock lease set, needsMaintenanceLease, skipBrainWidePhases, all-skipped deriveStatus case; lease logic in cycle/lock-set.ts); +16 #6377: content_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK), the REPAIR_LANE_PHASES dispatch helper shared with fence_repair (logic in cycle/content-repair.ts); +10 C4 pinned questions: standing_questions phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in questions/phase.ts); +11 #6696 (A5, A4): runCycle becomes the background-origin wrapper over runCycleUnscoped and adds the capacity notices (logic in persistence/admission-origin.ts, capacity-snapshot.ts) plus the CycleReport.notices interface field; +2 #6297: CycleOpts.directPhases (operator-named phases skip the patterns cadence); +5 #6865: unindexed_files phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its line in the REPAIR_LANE_PHASES dispatch helper (logic in cycle/unindexed-files.ts); +2 fix wave 20 #6954: the extract phase warns on extract_write_lost and its stale drain retries the pages left due; +5 #7241: CycleOpts.onPhaseStart and its best-effort call in timePhase, so the autopilot-cycle job writes its current phase to minion_jobs.progress; +1 GBRA-133: the extract phase passes the stale-drain budget to its targeted pass when sync gave no slug list; +7 #7630 (GBRA-207): the sync phase refuses sync_anchor_relative when no source matches its directory and default's local_path is relative
src/core/cycle/synthesize.ts 3244 ratchet +3 #7805: each captured session synthesizes only into its own frozen source (routing import, the outer wrapper, the routing parameter; logic in cycle/synthesize-source-routing.ts); +26 System One v1 S7/S8: triage and grounding decide delegation (triage-decide.ts, grounding-decide.ts); merged waves; grown v0.46.26.0 megawave + master rolling lease via shared throttled renewer; grown wave-g (#4506): cycle summaries can stay out of the source repo; +18: expired-verdict sweep + DeepSeek judge thinking pin (#4069); +47: #4077 cooperative-abort threading; +110: eval fix wave F6 spend surfacing (withChatPhase wrap, triage/child token telemetry, zero-page counter); +32: F1b quote-verify call site + kill switch; +2: F1a/F3/F4a OUTPUT POLICY rules; +59: F2 verified-segment rescue (gate at report construction, rescue knobs, telemetry); +19: rubric v2 peak-scoring + docblock rescue branch; +50: #4337 summary link-sample cap + dream_created_cycle_date preserve-on-rerun; +5: #4348 resolveCycleDate wiring (local-day summary bucketing); +3 round-3 rebase true-up; +55: #4805 synth-v2 completion skip (findSynthV2Completion + prefix-parameterized loadSuccessfulSynthesisKeys); +10: #4886 mode-aware buildSynthesisPrompt (oneshot children drop tool-use/final-prose guidance); #5105: small legacy writer/refusal and shared revision CAS delegates; +4: merge of #5666 durable synthesis completions with the v0.59.18.0 verify-epoch wiring; +7 #5413/#5471 discovery exclusions (self-capture session ids, dream output directories); +6 fix wave 2 (v0.60.5.0) integration: W7 paid-loop breaker check before submission and released-key daily-cap count on top of the W1b discovery exclusions; +6 v0.60.5.0 master-merge true-up: the wave-2 growth (+13 W1b, +6 W7) re-summed over master v0.59.18.0 at 3222 lines with no slack; +76 fix wave 3: per-run USD gate + fail-closed daily cap (C-13), child first-write ownership for stamp/reverse-write (C-8, #5685), cycle-date prompt hint (C-15), opt-in attribution rule (#5425); fix wave 3 x fix wave 2 (v0.60.5.0) composition: +19 at the merge; +2 #5806 triageNow clock seam forwarded to triage (interface field + call-site option); +4 #5854: a postprocess or summary publish still pending after its wait is deferred (publish_pending warn, no cooldown stamp; helper in synthesize-postprocess.ts); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); Foundations 2 Lane C: maintenanceTransaction import (attributed summary page); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); Foundations 2 Lane C: maintenanceTransaction import (attributed summary page); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); lowered to 3200 by wave 9 lane W #5145: completion-key helpers (findLegacyCompletion, findSynthV2Completion, loadSuccessfulSynthesisKeys) peeled to synthesize-completion.ts with the new pre-triage partition; +13 P2 extraction.date_grounding: transcript observation date line instead of a run-date "Today's date" (config field + prompt arg); +1 fix wave 11 #6160: oneshot_task_prefixes on the child job data; +18 fix wave 12 #6069: unreliable-verdict backoff in runTriagePass (report code, pass backoff count, triage details entry; marker logic in triage-backoff.ts); +1 fix wave 12 #6242: SYNTH_PHASE_FAIL keeps an OperationError's suggestion and code (OperationError import); +3 wave 17 #6616: slug discovery and postprocessing share the canonical committed child outputs (loadCommittedSynthesisOutputs in synthesize-postprocess.ts; one call-site line, one parameter, one lookup line); lowered to 3224 by wave 17 #6642: the summary provenance section moved to summaryProvenanceLines in dream-provenance.ts; +6 GBRA-130 B-6 (#6860): surrogate-safe triage cuts (truncateUtf8) and the per-transcript verdict-cache write call site (logic in triage-backoff.ts VerdictCacheWrites); +2 GBRA-161 PR 4 (#7065): the synthesize phase activates each synthesized transcript's outputs as its generation (activateSynthesisGenerations import and call site; logic in cycle/synthesize-generations.ts); +8 GBRA-161 PR 5 (S-2): the dream summary publishes as a single-item dream_summary generation bound to its pre-run revision (publishSinglePage with a generation_fenced skip); +1 GBRA-60: a managed synthesis child carries dream_output (verify and stamp before put_page; cycle/synthesize-child-verify.ts)
src/core/cycle/extract-atoms.ts 1658 ratchet crossed the 1500 unlisted cap in the atoms/extraction fix wave: +21 embed-aware cost gate (#4878), +65 atoms-array anchor scan (#4913); peel candidate: the parse block (parseAtomsOutcome / parseArrayAtOffset / atomsFromParsedArray) into extract-atoms-parse.ts; +35: #4853 transcript-origin atoms link to their conversation page; +130: #4916 transcript failure-count/tombstone (extract_atoms_transcript_state, TODOS P2(b)); wave composition: #4916 transcript-state skip + #4948 zero-atom prompt + #4943 meetings source-id summed; +3: wave review — a later `[]` is an honest zero-yield, a non-atom-shaped array is a counted failure; +13: wave review — explicit budget keeps the cap over an unpriced embed route ($0-priced); +14: wave review — transcript-state write errors surfaced, missing table warns once; merged reliability/shared-skills source integration; +37 fix wave 3: stable undated atom dates + stale-atom retirement (C-14); +3 #5525: effectiveVisibility stamped on non-managed atoms; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +3 at the merge; +6 fix wave 3 lane A (#5601): accepted-pending atom batches count as progress (helper in persistence/accepted-pending.ts); +4 fix wave 4: #5705 transcript-as-data system prompt lines and its message helper, #5627 response schema import (the atom-type list moved to extract-atoms-schema.ts); +30 #5809/#5832: drain hard/soft stop signals (ExtractAtomsOpts.signal/stopSignal, pre-commit abort checks, no strike or post-stop writes after a hard stop) and the cancellable backlog count; +1 #5812: transcript items are paste-stripped (corpusTextForExtraction import) before the input cut; +1 #5820: extract_atoms discovery passes selfCaptureSessionIds like synthesize; +10 #5856/#5854: drain attempt state (one BudgetTracker + one publish wait across batches) and the opt-in connector email/meeting exclusion (predicate in connector-atoms.ts); +1 Foundations 2 Lane C: maintenanceTransaction import (attributed stale-atom retirement); +5 fix wave 9 lane P (#5041): opt-in atom write-through hook and retired-atom slugs returned for file removal (logic in derived-write-through.ts); the parse block (parseAtomsOutcome / parseArrayAtOffset / atomsFromParsedArray) is peeled into extract-atoms-parse.ts (-193, fix wave 10 lane B); +35: #4853 transcript-origin atoms link to their conversation page; +130: #4916 transcript failure-count/tombstone (extract_atoms_transcript_state, TODOS P2(b)); wave composition: #4916 transcript-state skip + #4948 zero-atom prompt + #4943 meetings source-id summed; +3: wave review — a later `[]` is an honest zero-yield, a non-atom-shaped array is a counted failure; +13: wave review — explicit budget keeps the cap over an unpriced embed route ($0-priced); +14: wave review — transcript-state write errors surfaced, missing table warns once; merged reliability/shared-skills source integration; +37 fix wave 3: stable undated atom dates + stale-atom retirement (C-14); +3 #5525: effectiveVisibility stamped on non-managed atoms; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +3 at the merge; +6 fix wave 3 lane A (#5601): accepted-pending atom batches count as progress (helper in persistence/accepted-pending.ts); +4 fix wave 4: #5705 transcript-as-data system prompt lines and its message helper, #5627 response schema import (the atom-type list moved to extract-atoms-schema.ts); +30 #5809/#5832: drain hard/soft stop signals (ExtractAtomsOpts.signal/stopSignal, pre-commit abort checks, no strike or post-stop writes after a hard stop) and the cancellable backlog count; +1 #5812: transcript items are paste-stripped (corpusTextForExtraction import) before the input cut; +1 #5820: extract_atoms discovery passes selfCaptureSessionIds like synthesize; +10 #5856/#5854: drain attempt state (one BudgetTracker + one publish wait across batches) and the opt-in connector email/meeting exclusion (predicate in connector-atoms.ts); +1 Foundations 2 Lane C: maintenanceTransaction import (attributed stale-atom retirement); +5 fix wave 9 lane P (#5041): opt-in atom write-through hook and retired-atom slugs returned for file removal (logic in derived-write-through.ts); +17 #5031: same-page case-insensitive atom adoption fallback (one page-atom scan per source page per run); +12 P2 atomsPrompt helper: extraction.date_grounding source observation date + rule in the atoms prompt; +4 GBRA-75: octet_length-bounded body-length predicate (minCompiledTruthChars) for discovery and the backlog count; GBRA-130 A-42 (#6804): dot-led atom stems and the legacy-slug adoption guard
@@ -47,4 +47,4 @@ src/commands/doctor/checks/extraction-sync.ts 1500 ratchet +7 GBRA-130 A-1: link
src/core/search/evidence-delivery.ts 1759 ratchet +5 W5: EvidencePlan.rankCut resolved with the plan (logic in search/page-rank-cut.ts); +20 W5 whole_pages: wholeCost prices a block as delivery would emit it whole (needs the private Block/enrichBlock internals) and the cut is wired before allocate (cutWholePages lives in search/page-rank-cut.ts); new row: crossed the 1500 unlisted cap at the #6367 merge (master 1479, its explicit-budget cap) by the C1 date header threaded through every block path, cap included; pagePlanHits (search/page-plan-hits.ts) and the header loader (evidence-date.ts blockDateHeaders) were peeled first; peel candidate: allocateCapped; +8 at the #6396 merge: page evidence text cut through the fence-chunk trust overlay and the caller minTrust floor (re-summed: our 1504 + master +8); +119 Cat 40 next wave F2: the search.long_page_unit policy (withLongPagePolicy, selectLongPages size preflight with page/byte/deadline limits) and the long-page fetch and fallback in deliverEvidence, kept in the shared seam the plan names (gbrain-evals docs/plans/2026-10-11-cat40-next-wave, F2); +17 F2 rework to the frozen-comparison policy: search.long_page_min_chars (rendered-body characters, default 20,000) read in withLongPagePolicy, a character count in the preflight, the rendered-body confirmation in deliverEvidence, and the long-page row exemption in capDeliveredSnippets (re-summed at the 0.60.225.0 merge: master 1537 + F2 136); +39 Wave B PR 2 lane 2b: planSegmentedBlock joins the windows search/segmented-evidence.ts reads around a segmented page's hits into one run-marked document (needs the private Doc/Piece/buildDoc/locateAnchor internals), planBlock's unit cut split into blockFromDoc for both, sections bounded by their run, selectionCost counts a gap between runs, and the fetch skips the whole-body fence overlay for a segmented page; +4 segmented pages read at once before planning (segmentedReads; the read is search/segmented-evidence.ts, the statements page-body/read-range.ts); +17 GBRA-156: tail trim plan flag, DeliverOptions.query, DeliveryMeta.tail_trim, the trim call and the frozen-hit score/query library fields (logic in search/evidence-tail-trim.ts; interface fields cannot move); +7 GBRA-205: EvidencePlan.titleHeader and DeliveryMeta.title_header (interface fields cannot move) and the titles option on the header loader (logic in search/evidence-date.ts; the query budget key lives in search/evidence-packing.ts); +11 #7600: assemble_evidence quotes conversation rows and capEvidenceToBudget counts the quote fence (logic in search/transcript-quote.ts); +2 GBRA-156 /ship review: the cut-path admission and title room count the quote fence; +2 GBRA-156 /ship review pass 2: capEvidenceToBudget counts the fence only for the paths that fence (quote flag; recall counts the text as returned); +4 GST-687: page evidence and the extractive fallback quote below-owner-tier text for untrusted readers (untrusted flag; logic in search/transcript-quote.ts)
src/commands/config.ts 1524 ratchet new row: crossed the 1500 unlisted cap in wave 15 P7.4 #5461: set-time validation of think.pages_budget_chars / think.excerpt_ceiling_chars (range + parser live in core/think/pages-budget.ts), mirroring the cycle.consolidate.cluster_threshold hook; +1 wave 16 G5.1 (#6495): set-time refusal hook for search.keyword_statement_timeout_ms (check lives in commands/config/keyword-timeout-value.ts); +1 wave 18b lane 5 (#5012 item 6): search.contextual_retrieval prints the existing-pages hint after the write (text lives in commands/config/contextual-mode-hint.ts); +1 #6712 PR 2: retired/rollback sync key dispatch line (logic in core/sync-config-keys.ts handleSyncKeyConfig); +1 #6865: sync.slug_conflicts.auto_remove boolean validation at set time (refusal text in repair/slug-autoheal.ts); +1 #6948: sync.origin_twins.auto_resolve boolean validation at set time (refusal text in repair/origin-twins-heal.ts); +2 #6795: config set chat_model/expansion_model help line and the model-pin import (shadow report and file-pin unset live in commands/config/model-pin-shadow.ts); +1 #6712 PR 4b: sync.admission set-time hook (value logic in commands/config/sync-admission-value.ts); +4 #7130: set-time refusal hook for sync.max_deletes / sync.max_delete_ratio (and .source.<id>) / purge.retention_hours (check lives in core/sync-reconcile.ts deleteSafetyConfigRefusal); +1 GBRA-150: sync.metafile_pages set-time route (body in config/sync-metafile-pages-value.ts); re-summed at the v0.60.330.0 merge (1520 measured); +2 #7242 minions.terminal_hook file-plane key and its set branch (logic in minions/terminal-hook.ts); +1 GBRA-161: generations.* set-time route (refusal in commands/config/generations-values.ts); +1 GBRA-207 #7630: sync.repo_path set-time refusal hook (refusal in core/sync-config-keys.ts syncRepoPathRefusal)
src/commands/mcp-expose.ts 1519 ratchet hosted_option: Session carries the gbrain.io coaching notice on the first-receipt run and Tailscale/service friction exits (copy, refs and gating live in core/onboard/hosted-option.ts); +7 GBRA-199: --surface help lines, surfaceChosen option field and the advertise pass-through (registrationAdvertise in core/mcp-registration.ts)
-src/core/behavior-change-notice.ts 1612 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move)
+src/core/behavior-change-notice.ts 1614 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move); +2 GBRA-287: segmented_freeze_hold_gbra287 and segmented_reexport_gbra287 rows (rows are data and cannot move)
diff --git a/src/commands/sync.ts b/src/commands/sync.ts
index 540a4e8c2b..be6b7979cd 100644
--- a/src/commands/sync.ts
+++ b/src/commands/sync.ts
@@ -96,6 +96,8 @@ export interface SyncResult {
extract_error?: string;
/** #5050: full sync re-sealed unchanged pages at the safe-chunk fence (see RunImportResult.resealed). */
resealed?: import('./import.ts').RunImportResult['resealed'];
+ /** GBRA-287: DB-only pages (#2426) a full sync kept but could not re-export to the working tree; the run still completed. */
+ reexport_failures?: import('./sync/full.ts').ReexportFailure[];
/**
* v0.41.13.0 partial-sync fields (only set when status === 'partial').
*
diff --git a/src/commands/sync/full.ts b/src/commands/sync/full.ts
index bffbcca8b3..49a5e3001d 100644
--- a/src/commands/sync/full.ts
+++ b/src/commands/sync/full.ts
@@ -54,6 +54,9 @@ import { sweepOrphanedRenameSentinels } from './rename-reconcile.ts';
import { classifyItemFailure } from '../../core/item-failure.ts';
import { OperationError } from '../../core/ops/contract.ts';
+/** GBRA-287: one DB-only page a full sync kept but could not re-export: its source path, the failure and what to do. */
+export interface ReexportFailure { slug: string; path: string; code: string; message: string; fix: string }
+
export async function performFullSync(
engine: BrainEngine,
// #753/#774: the three roots resolved once at the top of performSyncInner.
@@ -207,7 +210,7 @@ export async function performFullSync(
);
}
- const reconciledDeletes = await reconcileFullSyncDeletes(engine, opts, { company, gitContextRoot, syncScopeRoot, slugRoot, metafilePages, previousCommit });
+ const { deleted: reconciledDeletes, reexportFailures } = await reconcileFullSyncDeletes(engine, opts, { company, gitContextRoot, syncScopeRoot, slugRoot, metafilePages, previousCommit });
// #3479 blocker 2 — the post-gate sweep above ran BEFORE this reconcile,
// so a `<rename:…>` sentinel whose stale row the reconcile just removed
@@ -258,6 +261,7 @@ export async function performFullSync(
...(result.malformedSkipped ? { malformedSkipped: result.malformedSkipped } : {}),
...(result.type_warnings ? { type_warnings: result.type_warnings } : {}),
...(result.resealed ? { resealed: result.resealed } : {}),
+ ...(reexportFailures.length ? { reexport_failures: reexportFailures } : {}),
};
}
@@ -377,7 +381,7 @@ async function reconcileFullSyncDeletes(
opts: SyncOpts,
input: { company: ReturnType<typeof currentCompanyBrainSync>; gitContextRoot: string; syncScopeRoot: string; slugRoot: string | undefined;
metafilePages: MetafilePagesMode; previousCommit: string | null },
-): Promise<number> {
+): Promise<{ deleted: number; reexportFailures: ReexportFailure[] }> {
const { company, gitContextRoot, syncScopeRoot, slugRoot, metafilePages, previousCommit } = input;
// #1970 (F-A): runImport is import-only — it never purges pages whose backing
// file was deleted since the last sync. A full re-import is authoritative for
@@ -399,6 +403,7 @@ async function reconcileFullSyncDeletes(
// Skipped on the legacy no-sourceId path (the batch delete primitives require
// a sourceId; matches every other source-scoped feature).
let reconciledDeletes = 0;
+ const reexportFailures: ReexportFailure[] = [];
if (opts.sourceId) {
const sid = opts.sourceId;
// GBRA-150: a metafile-named row is reconcilable exactly when the walk would collect its file: its bytes on disk, or,
@@ -437,8 +442,8 @@ async function reconcileFullSyncDeletes(
// #5988 (E13): the old page of a held rename keeps its origin until the
// renamed file imports; its file being gone does not make it stale.
const heldOrigins = await heldRenameOrigins(engine, sid);
- const rows = (await engine.executeRaw<{ slug: string; source_path: string | null; body_storage: string }>(
- `SELECT slug, source_path, body_storage FROM pages WHERE source_id = $1 AND source_path IS NOT NULL AND deleted_at IS NULL`,
+ const rows = (await engine.executeRaw<{ slug: string; source_path: string | null }>(
+ `SELECT slug, source_path FROM pages WHERE source_id = $1 AND source_path IS NOT NULL AND deleted_at IS NULL`,
[sid],
)).filter(row => !heldOrigins.has(row.source_path!));
// #774: a scoped full sync is authoritative ONLY for its scope — pages
@@ -491,7 +496,6 @@ async function reconcileFullSyncDeletes(
// history (i.e. was genuinely deleted) are reconcile-deleted.
const everCommitted = company ? null : listEverCommittedPaths(gitContextRoot);
const pathBySlug = new Map(rows.map(r => [r.slug, r.source_path]));
- const segmentedSlugs = new Set(rows.filter(r => r.body_storage === 'segmented').map(r => r.slug));
let deletableSlugs = plan.staleSlugs;
const dbOnlySlugs: string[] = [];
if (everCommitted) {
@@ -521,7 +525,6 @@ async function reconcileFullSyncDeletes(
const { writePageThrough } = await import('../../core/write-through.ts');
for (const slug of dbOnlySlugs) {
try {
- if (segmentedSlugs.has(slug)) throw new OperationError('page_body_segmented', 'Re-export cannot render a segmented page.', 'Read it with gbrain get --body auto; the page stays in the database.');
const r = await writePageThrough(engine, slug, { sourceId: sid, expectedFileBytes: null });
if (r.written) { reExported++; continue; }
if (r.skipped === 'disabled_by_config' || r.skipped === 'url_clone') continue;
@@ -532,9 +535,10 @@ async function reconcileFullSyncDeletes(
const failure = classifyItemFailure(error);
if (failure.scope === 'run') throw error;
reExportFailures++;
- serr(` ${slug} (${failure.code}: ${failure.message}) Kept the database page and any existing file. ` +
- `If the file is newer, import it with gbrain sync --source ${sid} --include-gitignored; ` +
- `if the database is newer, re-save its copy with write-through on; when both changed, ask the user which edit wins.`);
+ const fix = `Kept the database page and any existing file. If the file is newer, import it with gbrain sync --source ${sid} --include-gitignored; ` +
+ `if the database is newer, fix the cause above and run gbrain sync --source ${sid} --full --no-pull to re-export it; when both changed, ask the user which edit wins.`;
+ reexportFailures.push({ slug, path: pathBySlug.get(slug)!, code: failure.code, message: failure.message, fix });
+ serr(` ${slug} (${failure.code}: ${failure.message}) ${fix}`);
}
}
serr(
@@ -585,5 +589,5 @@ async function reconcileFullSyncDeletes(
}
}
}
- return reconciledDeletes;
+ return { deleted: reconciledDeletes, reexportFailures };
}
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index 223941d769..028670a38d 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1414,6 +1414,8 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: '0.60.553.0', id: 'empty_nonstring_args_gst927', text: 'Read tools now treat an optional argument as not provided when its own schema rules the value out: 0 where the range starts above zero (limit: 0, around_chunk_id: 0, schema_version: 0 and others), [] where the list must not be empty (types, only, traverse_graph hops), {} where the object needs a field (get_page lines and bytes), false on get_versions diff, and get_page context without grep. These calls used to refuse with invalid_params; now they answer as if the argument were left out. get_page known_as_of beside window or view arguments still refuses, and the refusal names the arguments to drop.' },
// GBRA-283 Wave A (supersedes #8454): per-engine caches; release:restamp stamps since at the merge slot.
{ since: '0.60.556.0', id: 'engine_keyed_caches_gbra283', text: 'A process serving several brains (mounts) now keeps each brain\'s search telemetry in that brain\'s own database and reads each brain\'s own search.track_retrieval, admission quotas, mcp.strict_params fallback, publication ceiling, capacity-notice throttle and the two vector-search rollback settings. Before, the first brain to be read decided these for every brain in the process, and every brain\'s search telemetry was written into the first brain\'s database. A process with one brain behaves as before.' },
+ { since: 'TBD', id: 'segmented_freeze_hold_gbra287', text: 'A managed sync whose freeze compares a Git-tracked page stored in segments with working-tree bytes that differ from the pinned commit now reads a page up to 24 MiB back whole, as its publication does, instead of refusing with `page_body_segmented` and stopping the source (`sync_blocked`) on every later sync. A page whose stored body cannot be read while it is frozen (`page_body_segmented` or `page_body_rows_unreadable`) is held `page_write_failed` with that code named, and the rest of the source syncs; `gbrain sync status` classifies both codes (`page_body_segmented`: public category `defect`, upgrade; `page_body_rows_unreadable`: `storage`, retry) instead of reporting an unknown code.' },
+ { since: 'TBD', id: 'segmented_reexport_gbra287', text: 'A full sync re-exports a DB-only page (#2426) stored in segments, streaming its file with the bytes `gbrain export` writes, instead of listing it as a `page_body_segmented` failure; a page whose re-export fails is returned in the result\'s `reexport_failures` and the run completes. Write-through of a page stored in segments with frontmatter overrides returns an error result instead of a `page_body_segmented` one. A segmented read whose page published a new body between its reads reads one generation (or the new live one) instead of refusing `page_body_rows_unreadable`.' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --git a/src/core/page-body/body-tiling.ts b/src/core/page-body/body-tiling.ts
index c20fcc53a9..db44e6b6a2 100644
--- a/src/core/page-body/body-tiling.ts
+++ b/src/core/page-body/body-tiling.ts
@@ -7,6 +7,7 @@
*/
import type { BrainEngine } from '../engine.ts';
import { opError } from '../ops/contract.ts';
+import type { ManifestRow } from './types.ts';
export interface RowSpan { byteStart: number; byteLen: number }
@@ -35,6 +36,34 @@ export async function generationBodyBytes(engine: Pick<BrainEngine, 'executeRaw'
return version ? Number(version.body_bytes) : null;
}
+/** Attempts of a live read whose generation a publish retired between its reads (`readTiledManifest`). */
+export const LIVE_READ_ATTEMPTS = 3;
+
+/**
+ * GBRA-287: the manifest rows of `generation`, checked to tile it. A live read (no `generation`) names the live generation
+ * first and reads that generation's rows, so the rows and the size they are checked against are one generation's even
+ * when a body is published between the statements; reading the live rows and the page in two statements compared one
+ * generation's rows with the next one's size and refused a healthy page `page_body_rows_unreadable`. A live read whose
+ * generation no longer tiles because a publish retired it meanwhile (its rows gone) reads the new live generation, up to
+ * LIVE_READ_ATTEMPTS times; an explicit generation, or a live one that did not move, refuses as before.
+ */
+export async function readTiledManifest(engine: Pick<BrainEngine, 'executeRaw' | 'readManifest'>, pageId: number,
+ generation?: number): Promise<{ generation: number | null; rows: ManifestRow[] }> {
+ const live = async () => (await engine.executeRaw<{ body_generation: unknown }>('SELECT body_generation FROM pages WHERE id=$1', [pageId]))[0];
+ for (let attempt = 1; ; attempt++) {
+ const page = generation === undefined ? await live() : null;
+ const at = generation ?? (page ? Number(page.body_generation) : null);
+ const rows = await engine.readManifest(pageId, at === null ? {} : { generation: at });
+ try {
+ await assertGenerationTiles(engine, pageId, at ?? undefined, rows);
+ return { generation: at, rows };
+ } catch (error) {
+ const now = generation === undefined && attempt < LIVE_READ_ATTEMPTS ? await live() : undefined;
+ if (!now || Number(now.body_generation) === at) throw error;
+ }
+ }
+}
+
/** The tiling check for a reader that read `rows` of `generation` (default live) through readManifest. */
export async function assertGenerationTiles(engine: Pick<BrainEngine, 'executeRaw'>, pageId: number, generation: number | undefined, rows: readonly RowSpan[]): Promise<void> {
const [page] = await engine.executeRaw<{ id: number; slug: string; source_id: string; body_generation: unknown; body_bytes: unknown }>(
diff --git a/src/core/page-body/ingest.ts b/src/core/page-body/ingest.ts
index 662add9bec..c770a91b6e 100644
--- a/src/core/page-body/ingest.ts
+++ b/src/core/page-body/ingest.ts
@@ -34,7 +34,7 @@ import { newFenceChunkBudget, type FenceChunkBudget } from '../markdown-chunks.t
import { loadFenceChunkOverlay } from '../eligibility/fence-overlay.ts';
import { withdrawalFenceBlocks } from '../facts/withdrawal-overlay.ts';
import { overlaySegmentText, readBodyLedger, type BodyLedger } from './segment-ledger.ts';
-import { assertGenerationTiles } from './body-tiling.ts';
+import { readTiledManifest } from './body-tiling.ts';
import { hasStaleChunks } from './projection.ts';
const utf8 = new TextDecoder('utf-8', { fatal: true, ignoreBOM: true });
@@ -466,8 +466,7 @@ export async function stageOccurrenceChunks(engine: BrainEngine, input: { pageId
/** The stored body of `pageId` at `generation` (default live), region by region, one segment at a time. */
export async function* readSegmentedBody(engine: BrainEngine, pageId: number, sourceId: string,
opts: { generation?: number } = {}): AsyncIterable<{ region: BodyRegion; bytes: Uint8Array }> {
- const rows = await engine.readManifest(pageId, opts.generation === undefined ? {} : { generation: opts.generation });
- await assertGenerationTiles(engine, pageId, opts.generation, rows);
+ const { rows } = await readTiledManifest(engine, pageId, opts.generation);
const batchBytes = STAGE_BATCH_BYTES[engine.kind];
for (let start = 0; start < rows.length;) {
let end = start, bytes = 0;
diff --git a/src/core/page-body/projection.ts b/src/core/page-body/projection.ts
index 78a74316bf..6b31c4f493 100644
--- a/src/core/page-body/projection.ts
+++ b/src/core/page-body/projection.ts
@@ -187,7 +187,7 @@ async function buildClaimedProjection(engine: BrainEngine, ref: { sourceId: stri
claim: ProjectionClaim): Promise<SegmentedProjectionResult> {
const generation = Number(page.body_generation);
const pageId = Number(page.id);
- const liveRows = await engine.readManifest(pageId);
+ const liveRows = await engine.readManifest(pageId, { generation });
await assertGenerationTiles(engine, pageId, generation, liveRows);
const { retired, moved } = await retireStaleChunks(engine, ref, pageId, generation, liveRows.map(row => Number(row.occurrenceId)));
if (moved) return { chunks: 0, retired, sealed: false };
diff --git a/src/core/page-body/read-range.ts b/src/core/page-body/read-range.ts
index 61869b8593..1053a88be4 100644
--- a/src/core/page-body/read-range.ts
+++ b/src/core/page-body/read-range.ts
@@ -37,7 +37,7 @@ import { credentialSafeProjection } from '../credential-projection.ts';
import { loadFenceChunkOverlay, markFenceChunk, renderFenceLowTier, type FenceChunkOverlay, type FenceLowTierRow } from '../eligibility/fence-overlay.ts';
import { partialFenceRuns, withdrawalFenceBlocks } from '../facts/withdrawal-overlay.ts';
import { FACTS_FENCE_BEGIN, FACTS_FENCE_END } from '../facts-fence.ts';
-import { assertRowsTile, generationBodyBytes } from './body-tiling.ts';
+import { assertRowsTile, generationBodyBytes, LIVE_READ_ATTEMPTS } from './body-tiling.ts';
import { readEntryState } from './entry-state.ts';
import { FENCE_MAX_BYTES, SEGMENT_MAX_BYTES } from './limits.ts';
import { ledgerApplies, readBodyLedger, overlaySegmentText, type BodyLedger } from './segment-ledger.ts';
@@ -186,10 +186,12 @@ function manifestRows(engine: BrainEngine, pageId: number, generation: number |
*
* The page row, the manifest rows and (given `expect`, the slug and source the caller already holds) the ledger are
* read at once: rows read for another generation than the page row names (a body published between the statements)
- * are read again, and a ledger read under a name the page row does not carry is read again under its own.
+ * are read again, and a ledger read under a name the page row does not carry is read again under its own. A live read
+ * whose generation a publish retired before its rows were read (they no longer tile it) opens the new live generation,
+ * up to LIVE_READ_ATTEMPTS times (GBRA-287).
*/
export async function openSegmentedBody(engine: BrainEngine, opts: { pageId: number; generation?: number; ledger?: BodyLedger; pin?: BodyPin;
- expect?: { slug: string; sourceId: string } }): Promise<SegmentedBody> {
+ expect?: { slug: string; sourceId: string } }, attempt = 1): Promise<SegmentedBody> {
const requested = opts.pin?.generation ?? opts.generation ?? null;
const [[page], read, expected] = await Promise.all([
engine.executeRaw<Record<string, unknown>>(`SELECT id, source_id, slug, body_storage, body_generation, body_bytes, body_lines, truth_bytes,
@@ -209,7 +211,14 @@ export async function openSegmentedBody(engine: BrainEngine, opts: { pageId: num
byteStart: Number(r.byte_start), byteLen: Number(r.byte_len), lineStart: Number(r.line_start), lineCount: Number(r.line_count), staticProjBytes: Number(r.static_proj_bytes),
staticProjLines: Number(r.static_proj_lines), outlineCount: Number(r.outline_count), hasFacts: r.has_facts === true, hasDate: r.has_date === true,
}));
- assertRowsTile(bodyRows, { slug, sourceId, total: await generationBodyBytes(engine, { id: opts.pageId, body_generation: page.body_generation, body_bytes: page.body_bytes }, generation) });
+ try {
+ assertRowsTile(bodyRows, { slug, sourceId, total: await generationBodyBytes(engine, { id: opts.pageId, body_generation: page.body_generation, body_bytes: page.body_bytes }, generation) });
+ } catch (error) {
+ if (requested !== null || attempt >= LIVE_READ_ATTEMPTS) throw error;
+ const [now] = await engine.executeRaw<{ body_generation: unknown }>('SELECT body_generation FROM pages WHERE id=$1', [opts.pageId]);
+ if (!now || Number(now.body_generation) === generation) throw error;
+ return openSegmentedBody(engine, opts, attempt + 1);
+ }
const ledger = opts.ledger ?? (expected && opts.expect!.slug === slug && opts.expect!.sourceId === sourceId ? expected : await readBodyLedger(engine, sourceId, slug));
return {
pageId: opts.pageId, sourceId, slug, generation, bodyBytes: Number(page.body_bytes ?? 0), bodyLines: Number(page.body_lines ?? 0), truthBytes: Number(page.truth_bytes ?? 0),
diff --git a/src/core/page-body/segment-rewrite.ts b/src/core/page-body/segment-rewrite.ts
index e0f3ee04ab..6feaf161b5 100644
--- a/src/core/page-body/segment-rewrite.ts
+++ b/src/core/page-body/segment-rewrite.ts
@@ -178,11 +178,11 @@ export async function rewriteLiveSegmentedFences(tx: BrainEngine, pageId: number
COALESCE((SELECT jsonb_agg(t.tag ORDER BY t.tag) FROM tags t WHERE t.page_id=p.id),'[]'::jsonb) AS tags FROM pages p WHERE p.id=$1`, [pageId]);
if (page?.body_storage !== 'segmented') return null;
const sourceId = page.source_id, slug = page.slug, frontmatter = page.frontmatter ?? {};
- const rows = await tx.readManifest(pageId);
- await assertGenerationTiles(tx, pageId, undefined, rows);
+ const generation = Number(page.body_generation);
+ const rows = await tx.readManifest(pageId, { generation });
+ await assertGenerationTiles(tx, pageId, generation, rows);
const units = await changedUnits(tx, sourceId, rows, transform);
if (!units.length) return null;
- const generation = Number(page.body_generation);
const prepared = await prepareSegmentedBody(tx, { sourceId, slug, title: page.title, type: page.type, frontmatter, tags: page.tags,
live: { id: pageId, generation, storage: 'segmented', title: page.title, type: page.type, flags: page.body_flags },
truth: rewrittenRegion(tx, sourceId, rows, units, 'truth'), timeline: rewrittenRegion(tx, sourceId, rows, units, 'timeline'),
diff --git a/src/core/page-state/import-guard.ts b/src/core/page-state/import-guard.ts
index ee6d671dd4..6865f59f80 100644
--- a/src/core/page-state/import-guard.ts
+++ b/src/core/page-state/import-guard.ts
@@ -26,9 +26,13 @@ export async function assertImportBase(tx: BrainEngine, slug: string, sourceId:
const canonicalFields = (p: Page | ParsedPage) => ({ type: p.type, title: p.title, compiled_truth: p.compiled_truth,
timeline: p.timeline ?? '', frontmatter: p.frontmatter });
-/** Content hashes omit volatile metadata; a prepared no-op must compare all canonical fields. */
+/**
+ * Content hashes omit volatile metadata; a prepared no-op must compare all canonical fields. A stored body left unread
+ * (over the single-string ceiling) differs from any parsed body, which is within it (GBRA-287: reading it threw
+ * `page_body_segmented` and blocked the source's sync).
+ */
export function sameCanonicalImport(existing: PageSnapshot | null, parsed: ParsedPage): boolean {
- if (!existing) return false;
+ if (!existing || bodyUnread(existing.page)) return false;
const tags = [...new Set([...existing.tags, ...parsed.tags])].sort();
return stableJson(canonicalFields(existing.page)) === stableJson(canonicalFields(parsed)) && stableJson([...existing.tags].sort()) === stableJson(tags);
}
diff --git a/src/core/persistence/public-category.ts b/src/core/persistence/public-category.ts
index 682695d41b..b7c9814f34 100644
--- a/src/core/persistence/public-category.ts
+++ b/src/core/persistence/public-category.ts
@@ -35,7 +35,7 @@ export const PUBLIC_BY_TERMINAL: Readonly<Record<TerminalCategory, PublicCategor
export const PUBLIC_BY_CODE: Readonly<Record<string, PublicCategory>> = {
worktree_dirty: 'drift', file_drift: 'drift', file_drift_unknown: 'drift', source_changed: 'drift', pinned_git_worktree_conflict: 'drift',
concurrent_write: 'conflict', revision_conflict: 'conflict', page_identity_changed: 'conflict', legacy_slug_conflict: 'conflict', origin_conflict: 'conflict', origin_conflict_decision: 'conflict', page_identity_decision: 'conflict',
- settle_mismatch: 'defect', page_write_failed: 'defect',
+ settle_mismatch: 'defect', page_write_failed: 'defect', page_body_segmented: 'defect', page_body_rows_unreadable: 'storage',
invalid_fence: 'content_refused', invalid_frontmatter: 'content_refused', frontmatter_slug_conflict: 'content_refused', rename_held: 'content_refused',
rename_identity_unreadable: 'content_refused', rename_reconcile_failed: 'conflict',
file_too_large: 'content_refused', content_rejected: 'content_refused', managed_image_sync_unsupported: 'content_refused',
diff --git a/src/core/persistence/sync-entry-fault.ts b/src/core/persistence/sync-entry-fault.ts
index 3ab5b3e389..15a2b2a6dd 100644
--- a/src/core/persistence/sync-entry-fault.ts
+++ b/src/core/persistence/sync-entry-fault.ts
@@ -96,15 +96,23 @@ function runScoped(error: unknown): boolean {
|| isRetryableConnError(error) || isStatementTimeoutError(error) || classifyItemFailure(error).scope !== 'item';
}
+/**
+ * GBRA-287: a stored page body this freeze could not read (`page_body_segmented`: a whole-body read of a page stored in
+ * segments; `page_body_rows_unreadable`: segment rows that do not tile the body) is that page's fault, never the run's.
+ */
+const PAGE_BODY_FAULT_CODES: ReadonlySet<string> = new Set(['page_body_segmented', 'page_body_rows_unreadable']);
+
/**
* S-63/S-64/S-33/S-7: the hold a throw while freezing one entry earns when holds are on, or null to rethrow it. A
- * `git_object_missing` or `path_not_utf8` refusal is held under its code; any other `OperationError` is a deliberate refusal and keeps
+ * `git_object_missing` or `path_not_utf8` refusal is held under its code; a page-body fault (`PAGE_BODY_FAULT_CODES`) is held
+ * `page_write_failed` like an unexpected throw, naming its code; any other `OperationError` is a deliberate refusal and keeps
* its route. An unexpected throw is held `page_write_failed` (origin `freeze`) unless it is run-scoped or the run
* looks systemic (`ITEM_BREAKER_MIN_ITEMS` faults and at least as many faults as clean freezes).
*/
export function freezeFaultHold(run: object, entry: HoldEntry, slug: string, pageId: number | null, error: unknown, blob: () => TreeBlob | null): HeldFault | null {
const missing = error instanceof OperationError && (error.code === 'git_object_missing' || error.code === 'path_not_utf8');
- if (!missing && (error instanceof OperationError || runScoped(error))) return null;
+ const pageBody = error instanceof OperationError && PAGE_BODY_FAULT_CODES.has(error.code);
+ if (!missing && !pageBody && (error instanceof OperationError || runScoped(error))) return null;
if (!missing) {
const tally = tallies.get(run) ?? { faults: 0, frozen: 0 };
tally.faults++; tallies.set(run, tally);
@@ -117,12 +125,12 @@ export function freezeFaultHold(run: object, entry: HoldEntry, slug: string, pag
...(rename ? { rename_from: rename } : {}), ...(entry.action === 'delete' ? { deleted: true as const } : {}) };
if (missing) return { path: entry.path, source_path: entry.sourcePath, slug, page_id: pageId, code: (error as OperationError).code as 'git_object_missing' | 'path_not_utf8',
message: `${(error as OperationError).message} ${(error as OperationError).suggestion ?? ''}`.trim(), upstream_version: null, meta };
- const name = error instanceof Error ? error.name : typeof error;
+ const name = pageBody ? (error as OperationError).code : error instanceof Error ? error.name : typeof error;
const errno = (error as { code?: unknown } | null)?.code;
const text = (error instanceof Error ? error.message : String(error)).replace(/[\x00-\x1f\x7f]+/g, ' ').slice(0, 300);
return { path: entry.path, source_path: entry.sourcePath, slug, page_id: pageId, code: 'page_write_failed',
message: `${entry.path}: sync could not prepare its ${entry.action === 'delete' ? 'deletion' : 'import'} (${name}: ${text}), so it is held and the rest of the source synced. `
+ 'The next syncs re-test it; if it stays held, report it with the gbrain version or fix the file when the error names it, then gbrain sources retry-held.',
upstream_version: null,
- meta: { ...meta, write_failure: { code: 'storage_error', origin: 'freeze', error_class: name, errno: typeof errno === 'string' ? errno.slice(0, 32) : null, request_id: null, attempts: 1, gbrain_version: VERSION } } };
+ meta: { ...meta, write_failure: { code: pageBody ? (error as OperationError).code : 'storage_error', origin: 'freeze', error_class: name, errno: typeof errno === 'string' ? errno.slice(0, 32) : null, request_id: null, attempts: 1, gbrain_version: VERSION } } };
}
diff --git a/src/core/persistence/sync-fault-class.ts b/src/core/persistence/sync-fault-class.ts
index fdecfda11e..fe0a3bb232 100644
--- a/src/core/persistence/sync-fault-class.ts
+++ b/src/core/persistence/sync-fault-class.ts
@@ -89,6 +89,11 @@ export const SYNC_FAULT_TABLE: ReadonlyArray<SyncFaultRule> = [
{ code: 'settle_mismatch', class: 'page', safe_actions: ['none'], needs_human: true, escalate: 'always: the pass cannot tell why the path does not settle',
human_reason: 'Sync settled this file three passes in a row but it never read settled afterwards; read gbrain doctor --json and gbrain sources writer status <source> (read-only), fix what they name, then run gbrain sources retry-held <source> to re-test it.' },
// GBRA-130 A-6: one path's write failed the same unclassifiable way on repeated attempts; held so the rest of the source moves.
+ // GBRA-287: a stored page body a sync path could not read: whole (a page stored in segments), or at all (rows that do not tile it).
+ { code: 'page_body_segmented', class: 'page', safe_actions: ['upgrade'], needs_human: true, escalate: 'always: a sync path read a page stored in segments whole',
+ human_reason: 'A sync path read this page stored in segments as one string, so its file is held and the rest of the source synced; upgrade gbrain, then run gbrain sources retry-held <source>, and report it with the gbrain version if it stays held.' },
+ { code: 'page_body_rows_unreadable', class: 'page', safe_actions: ['retry'], needs_human: false, escalate: 'the page stays unreadable across syncs (a grant or row-level security policy hides its segment rows)',
+ human_reason: 'Give the role that reads the brain the same access to page_body_manifest and page_segment_texts it has to pages, then run gbrain sources retry-held <source>.' },
{ code: 'page_write_failed', class: 'page', safe_actions: ['upgrade'], needs_human: true, escalate: 'always: the writer cannot place the failure',
human_reason: 'The write of this file keeps failing with an error gbrain cannot classify (an owner exception or an unplaced refusal); read its receipt (gbrain write-request <request_id>), report it with the gbrain version or fix the file when the error names its content, then run gbrain sources retry-held <source>.' },
// GBRA-130 A-7: the file is not a readable regular file inside the source root (a symlink, a non-regular file, a permission error).
diff --git a/src/core/persistence/sync-screen.ts b/src/core/persistence/sync-screen.ts
index 71da5710a3..5ac5c925fb 100644
--- a/src/core/persistence/sync-screen.ts
+++ b/src/core/persistence/sync-screen.ts
@@ -32,6 +32,7 @@ import { loadActivePackForEngine } from '../schema-pack/engine-resolution.ts';
import { VERSION } from '../../version.ts';
import { sha256 } from './digest.ts';
import { readPageBodyPolicy, type PageBodyPolicy } from '../page-body/limits.ts';
+import { materializedSnapshot } from './segmented-apply.ts';
import { readPagePurgeTombstones } from './page-purge.ts';
import { listTrustAllowRules } from '../trust/allow-rules.ts';
import { readSyncContent, readSyncFile, syncGitPath, UNREADABLE_SYNC_FILE, type SyncDiscovery, type SyncEntry } from './sync-discovery.ts';
@@ -313,9 +314,11 @@ export interface FrozenImportScreen {
*/
export async function frozenImportScreenInput(engine: BrainEngine,
input: Pick<FrozenImportScreen, 'cursor' | 'entry' | 'slug' | 'snapshot' | 'rawHash' | 'lineEndingOnly'> & { content: string }, run: SyncScreenRun): Promise<SyncImportScreenInput> {
- const { entry, slug, snapshot } = input;
+ const { entry, slug } = input;
const moved = entry.renameFrom && entry.renameFrom.slug !== slug ? entry.renameFrom : undefined;
- const base = moved ? await engine.readPageSnapshot(moved.slug, { sourceId: input.cursor.sourceId, includeDeleted: true }) : snapshot;
+ // GBRA-287: a page stored in segments up to the single-string ceiling is compared read back whole, as its publication compares it.
+ const snapshot = await materializedSnapshot(engine, input.snapshot);
+ const base = moved ? await materializedSnapshot(engine, await engine.readPageSnapshot(moved.slug, { sourceId: input.cursor.sourceId, includeDeleted: true })) : snapshot;
return { content: input.content, rawHash: input.rawHash, lineEndingOnly: input.lineEndingOnly, slug, sourcePath: entry.sourcePath, path: entry.path,
root: input.cursor.root, snapshot, base, renamed: !!moved, activePack: run.activePack, sanity: run.sanity,
purgedPages: await runPurgeTombstones(engine, run, input.cursor.sourceId), writeGateSource: await runWriteGateSource(engine, run, input.cursor.sourceId) };
diff --git a/src/core/write-through.ts b/src/core/write-through.ts
index 05df5b0b74..7ed2e2180b 100644
--- a/src/core/write-through.ts
+++ b/src/core/write-through.ts
@@ -22,9 +22,11 @@ import { hasSourceFilesystemLock, withSourceFilesystemLock, assertSourceFilesyst
* only does "row exists + repo is a real dir → render + atomic write".
*/
-import { existsSync, statSync, mkdirSync, unlinkSync, readdirSync, readFileSync } from 'fs';
+import { existsSync, statSync, mkdirSync, unlinkSync, readdirSync, readFileSync, renameSync, rmSync } from 'fs';
import { basename, dirname, isAbsolute, join, posix, relative, resolve, sep, win32 } from 'path';
-import { atomicWriteFileSync } from './atomic-write.ts';
+import { atomicStagingPath, atomicWriteFileSync } from './atomic-write.ts';
+import { flushDirectory } from './fs-durable.ts';
+import { isSegmentedPage } from './page-body/page-shape.ts';
import type { BrainEngine } from './engine.ts';
import { isRelativeFileUri, serializePageToMarkdown, resolvePageFilePath, resolveSourceLocalFilePath } from './markdown.ts';
import { isWriteTargetContained, msysToNativePath } from './path-confine.ts';
@@ -47,7 +49,7 @@ export interface WriteThroughLogger {
export interface WriteThroughResult {
written: boolean;
path?: string;
- /** The bytes written to `path` (set with `written: true`), so a caller can tell its own write from a later edit. */
+ /** The bytes written to `path` (set with `written: true` for an inline page; a page stored in segments is streamed and never held whole), so a caller can tell its own write from a later edit. */
content?: string;
/**
* True when the write was also committed to git (#2426). Only attempted on
@@ -465,13 +467,16 @@ export async function writePageThrough(
return { written: false, skipped: 'page_not_found_after_write' };
}
- const tags = await engine.getTags(slug, { sourceId });
- const md = serializePageToMarkdown(writtenPage, tags, {
+ const segmented = isSegmentedPage(writtenPage);
+ if (segmented && opts.frontmatterOverrides) {
+ return { written: false, error: `page ${slug} is stored in segments, and its file is streamed as gbrain export writes it, without frontmatter overrides` };
+ }
+ const md = segmented ? null : serializePageToMarkdown(writtenPage, await engine.getTags(slug, { sourceId }), {
frontmatterOverrides: opts.frontmatterOverrides,
});
// The pre-push secret scan refuses to push any file it cannot scan, so a
// file over its cap would block every later push of the brain repo.
- if (Buffer.byteLength(md, 'utf-8') > PUSH_SCAN_MAX_FILE_BYTES) {
+ if (md !== null && Buffer.byteLength(md, 'utf-8') > PUSH_SCAN_MAX_FILE_BYTES) {
opts.logger?.warn(`[write-through] ${slug}: the rendered file is over the ${PUSH_SCAN_MAX_FILE_BYTES}-byte pre-push secret-scan cap — file not written (DB row is intact)`);
return { written: false, skipped: 'over_push_scan_cap' };
}
@@ -514,8 +519,16 @@ export async function writePageThrough(
assertSourceFilesystemActive();
if (fileChanged()) return { written: false, skipped: 'file_changed' };
const changedWhileStaging = new Error('file_changed');
+ const verify = () => { assertSourceFilesystemActive(); if (fileChanged()) throw changedWhileStaging; };
try {
- atomicWriteFileSync(filePath, md, { verify: () => { assertSourceFilesystemActive(); if (fileChanged()) throw changedWhileStaging; } });
+ if (md !== null) atomicWriteFileSync(filePath, md, { verify });
+ else {
+ const streamed = await writeSegmentedPageFile(engine, slug, sourceId, filePath, verify);
+ if (streamed === 'over_push_scan_cap') {
+ opts.logger?.warn(`[write-through] ${slug}: the rendered file is over the ${PUSH_SCAN_MAX_FILE_BYTES}-byte pre-push secret-scan cap — file not written (DB row is intact)`);
+ }
+ if (streamed !== 'written') return { written: false, skipped: streamed };
+ }
} catch (error) {
if (error === changedWhileStaging) return { written: false, skipped: 'file_changed' };
throw error;
@@ -544,7 +557,7 @@ export async function writePageThrough(
return {
written: true,
path: filePath,
- content: md,
+ ...(md !== null ? { content: md } : {}),
...(committed ? { committed, pushed, lastPushStatus } : {}),
};
} catch (e) {
@@ -554,6 +567,35 @@ export async function writePageThrough(
}
}
+/**
+ * The file of a page stored in segments, with the bytes `gbrain export` writes for it: measured and streamed from its
+ * segments in one repeatable-read snapshot, so a body published meanwhile is never mixed in, through the streamed staging
+ * writer the withdrawal mirror and restore_page use (stageStreamedContent: an in-progress name, renamed once whole), then
+ * renamed over `filePath` after `verify`. Never held as one string, whatever its size.
+ */
+async function writeSegmentedPageFile(engine: BrainEngine, slug: string, sourceId: string, filePath: string,
+ verify: () => void): Promise<'written' | 'over_push_scan_cap' | 'page_not_found_after_write'> {
+ const [{ unreadPageFile }, { stageStreamedContent }] = await Promise.all([import('./persistence/page-prepare.ts'), import('./persistence/large-file.ts')]);
+ return engine.transaction(async tx => {
+ await tx.executeRaw('SET TRANSACTION ISOLATION LEVEL REPEATABLE READ READ ONLY');
+ const snapshot = await tx.readPageSnapshot(slug, { sourceId });
+ if (!snapshot) return 'page_not_found_after_write';
+ const content = await unreadPageFile(tx, snapshot);
+ if (content.bytes > PUSH_SCAN_MAX_FILE_BYTES) return 'over_push_scan_cap';
+ const stage = { path: atomicStagingPath(filePath), bytes: content.bytes, hash: content.hash };
+ await stageStreamedContent(tx, filePath, content, stage, null);
+ try {
+ verify();
+ renameSync(stage.path, filePath);
+ } catch (error) {
+ rmSync(stage.path, { force: true });
+ throw error;
+ }
+ flushDirectory(dirname(filePath));
+ return 'written';
+ });
+}
+
export interface DeleteThroughResult {
/** True when an artifact existed and was unlinked. */
removed: boolean;
diff --git a/test/large-pages/managed-sync-segmented-freeze.test.ts b/test/large-pages/managed-sync-segmented-freeze.test.ts
new file mode 100644
index 0000000000..d15dac2832
--- /dev/null
+++ b/test/large-pages/managed-sync-segmented-freeze.test.ts
@@ -0,0 +1,94 @@
+/**
+ * GBRA-287: a managed sync re-importing a git-tracked page stored in segments never blocks the source.
+ *
+ * 1. Protects: the freeze of an import whose working-tree bytes differ from the pinned commit (a live checkout) compares
+ * the stored page read back whole, as its publication does, so a segmented page up to the single-string ceiling earns
+ * exactly the outcome its inline twin earns (here the `worktree_dirty` hold) and the run completes. A page whose
+ * stored body cannot be read (segment rows that do not tile it) is held at the freeze, naming
+ * `page_body_rows_unreadable`, while the rest of the commit imports and the run completes.
+ * 2. Fails when: the freeze reads the snapshot's throwing body getter (`rowToPage.compiled_truth`, `page_body_segmented`)
+ * and the run stops `blocked` with a recorded failure, or a page-body fault blocks the run instead of holding the page.
+ * 3. PGLite.
+ */
+import { afterAll, beforeAll, expect, test } from 'bun:test';
+import { randomUUID } from 'node:crypto';
+import { execFileSync } from 'node:child_process';
+import { appendFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
+import { claimWorktree } from '../../src/core/persistence/ownership.ts';
+import { performManagedSync } from '../../src/core/persistence/sync-run.ts';
+import { disposePersistenceConsumer } from '../../src/core/persistence/service.ts';
+import { withEnv } from '../helpers/with-env.ts';
+import { withManagedFixtureWrite } from '../helpers/managed-e2e-fixture-write.ts';
+import { conversationBody } from '../helpers/conversation-page.ts';
+
+const home = mkdtempSync(join(tmpdir(), 'gbrain-segmented-freeze-'));
+const env = { GBRAIN_HOME: home, OPENAI_API_KEY: undefined, VOYAGE_API_KEY: undefined, ANTHROPIC_API_KEY: undefined };
+let engine: PGLiteEngine;
+
+function git(root: string, ...args: string[]): string { return execFileSync('git', ['-C', root, ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim(); }
+function commit(root: string): void { git(root, 'add', '-A'); git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'commit', '-qm', 'test content'); }
+const person = (bytes: number) => '---\ntype: person\ntitle: Person Example\n---\n\n# Person Example\n\n' + conversationBody(bytes, { seed: 7 }) + '\n<!-- timeline -->\n- **2026-10-01** | manual — first\n';
+
+beforeAll(async () => { engine = new PGLiteEngine(); await engine.connect({}); await engine.initSchema(); }, 120_000);
+afterAll(async () => {
+ await withEnv(env, async () => { await disposePersistenceConsumer(engine); await engine.disconnect(); });
+ rmSync(home, { recursive: true, force: true });
+});
+
+async function fixture(segmented: boolean, bytes: number) {
+ await disposePersistenceConsumer(engine);
+ await engine.setConfig('pages.segmented_storage', segmented ? 'on' : 'off');
+ const sourceId = `freeze-${randomUUID().slice(0, 8)}`;
+ const root = join(home, sourceId);
+ mkdirSync(join(root, 'people'), { recursive: true });
+ git(root, 'init', '-q');
+ writeFileSync(join(root, 'people/person-example.md'), person(bytes));
+ writeFileSync(join(root, 'people/other-example.md'), '---\ntype: person\ntitle: Other Example\n---\n\nfirst\n');
+ commit(root);
+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');
+ await engine.executeRaw("INSERT INTO sources(id,name,local_path,config) VALUES($1,$1,$2,'{}')", [sourceId, root]);
+ await claimWorktree(engine, sourceId, root);
+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');
+ expect(await performManagedSync(engine, { sourceId, noPull: true })).toMatchObject({ status: 'first_sync', filesImported: 2 });
+ const [row] = await engine.executeRaw<{ id: number; body_storage: string }>("SELECT id, body_storage FROM pages WHERE source_id=$1 AND slug='people/person-example'", [sourceId]);
+ expect(row!.body_storage).toBe(segmented ? 'segmented' : 'inline');
+ return { sourceId, root, pageId: Number(row!.id) };
+}
+
+/** A committed edit plus uncommitted bytes past it: the freeze compares the working tree with the stored page. */
+async function liveCheckoutSync(segmented: boolean, bytes: number) {
+ const f = await fixture(segmented, bytes);
+ appendFileSync(join(f.root, 'people/person-example.md'), '- **2026-11-02** | meeting — second\n');
+ commit(f.root);
+ appendFileSync(join(f.root, 'people/person-example.md'), '- **2026-11-03** | meeting — third, not yet committed\n');
+ return performManagedSync(engine, { sourceId: f.sourceId, noPull: true, noEmbed: true, noExtract: true });
+}
+
+test('a live checkout re-import of a 1.26 MiB segmented page completes with the outcome its inline twin gets', async () => withEnv(env, async () => {
+ const inline = await liveCheckoutSync(false, 64 * 1024);
+ const segmented = await liveCheckoutSync(true, 1260 * 1024);
+ expect(inline.status).toBe('synced');
+ expect(segmented.status).toBe('synced');
+ expect(segmented.failures).toBeUndefined();
+ expect((segmented.held ?? []).map(hold => [hold.path, hold.code])).toEqual((inline.held ?? []).map(hold => [hold.path, hold.code]));
+ expect((segmented.held ?? []).map(hold => hold.code)).toEqual(['worktree_dirty']);
+}), 300_000);
+
+test('a segmented page whose rows do not tile its body is held at the freeze; the rest of the commit imports', async () => withEnv(env, async () => {
+ const f = await fixture(true, 1260 * 1024);
+ await withManagedFixtureWrite(engine, [f.sourceId], tx => tx.executeRaw('DELETE FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NULL AND seq=(SELECT max(seq) FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NULL)', [f.pageId]));
+ appendFileSync(join(f.root, 'people/person-example.md'), '- **2026-11-02** | meeting — second\n');
+ writeFileSync(join(f.root, 'people/other-example.md'), '---\ntype: person\ntitle: Other Example\n---\n\nsecond\n');
+ commit(f.root);
+ const result = await performManagedSync(engine, { sourceId: f.sourceId, noPull: true, noEmbed: true, noExtract: true });
+ expect(result.status).toBe('synced');
+ expect(result.failures).toBeUndefined();
+ expect((result.held ?? []).map(hold => [hold.path, hold.code])).toEqual([['people/person-example.md', 'page_write_failed']]);
+ expect(result.held![0]!.message).toContain('page_body_rows_unreadable');
+ const [other] = await engine.executeRaw<{ compiled_truth: string }>("SELECT compiled_truth FROM pages WHERE source_id=$1 AND slug='people/other-example'", [f.sourceId]);
+ expect(other!.compiled_truth).toContain('second');
+ expect(await engine.executeRaw('SELECT last_commit FROM sources WHERE id=$1', [f.sourceId])).toEqual([{ last_commit: git(f.root, 'rev-parse', 'HEAD') }]);
+}), 300_000);
diff --git a/test/segmented-read-concurrent-publish.test.ts b/test/segmented-read-concurrent-publish.test.ts
new file mode 100644
index 0000000000..815ef45c07
--- /dev/null
+++ b/test/segmented-read-concurrent-publish.test.ts
@@ -0,0 +1,112 @@
+/**
+ * GBRA-287: a segmented page read while a new body publishes (forced probe).
+ *
+ * 1. Protects: a reader of a page stored in segments never refuses a healthy page `page_body_rows_unreadable` because a
+ * publish landed between its statements. A live read names its generation first and checks that generation's rows
+ * against that generation's size (`readSegmentedBody`, behind readPageSnapshot's read-back), and a live open whose
+ * generation a publish retired before its rows were read opens the new live generation (`openSegmentedBody`).
+ * 2. Fails when: the rows of one generation are checked against the next generation's body_bytes, or a retired
+ * generation's missing rows refuse the read instead of reading the live body.
+ * 3. PGLite. The interleaving is forced by an engine wrapper that publishes a new body (and, for the open, retires the old
+ * generation's rows as segment maintenance does once no version references them) at the exact point between reads.
+ */
+import { afterAll, beforeAll, beforeEach, afterEach, describe, expect, test } from 'bun:test';
+import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { resetPgliteState } from './helpers/reset-pglite.ts';
+import type { BrainEngine } from '../src/core/engine.ts';
+import { importFromFile } from '../src/core/import-file.ts';
+import { readPageBodyPolicy } from '../src/core/page-body/limits.ts';
+import { materializeSegmentedBody } from '../src/core/page-body/ingest.ts';
+import { openSegmentedBody } from '../src/core/page-body/read-range.ts';
+
+let engine: PGLiteEngine;
+let dir: string;
+const SLUG = 'notes/live-example';
+
+const line = (tag: string, i: number) => '- ' + tag + String(i).padStart(9, '0') + ' acme-example journal line with enough text to fill a segment\n';
+const truth = (tag: string, bytes: number) => '# Live Example\n\n' + Array.from({ length: Math.ceil(bytes / line(tag, 0).length) }, (_, i) => line(tag, i)).join('');
+
+async function publish(tag: string, bytes: number): Promise<void> {
+ const file = join(dir, tag + '.md');
+ writeFileSync(file, '---\ntype: concept\ntitle: Live Example\n---\n\n' + truth(tag, bytes));
+ expect(await importFromFile(engine, file, SLUG + '.md', { noEmbed: true, bodyPolicy: await readPageBodyPolicy(engine) })).toMatchObject({ status: 'imported' });
+}
+
+async function pageRow(): Promise<{ id: number; generation: number; bytes: number }> {
+ const [row] = await engine.executeRaw<{ id: number; body_generation: number; body_bytes: number; body_storage: string }>(
+ 'SELECT id, body_generation, body_bytes, body_storage FROM pages WHERE slug=$1', [SLUG]);
+ expect(row!.body_storage).toBe('segmented');
+ return { id: Number(row!.id), generation: Number(row!.body_generation), bytes: Number(row!.body_bytes) };
+}
+
+/** The engine with `hook` run once, at the first call of `method` whose arguments `when` accepts, before or after the real call. */
+function interleaved(method: 'readManifest' | 'executeRaw', when: (args: unknown[]) => boolean, at: 'before' | 'after', hook: () => Promise<void>): BrainEngine {
+ let armed = true;
+ return new Proxy(engine, {
+ get(target, key) {
+ const value = Reflect.get(target, key, target);
+ if (key !== method) return typeof value === 'function' ? value.bind(target) : value;
+ return async (...args: unknown[]) => {
+ const fire = armed && when(args);
+ if (fire) armed = false;
+ if (fire && at === 'before') await hook();
+ const out = await (value as (...a: unknown[]) => Promise<unknown>).apply(target, args);
+ if (fire && at === 'after') await hook();
+ return out;
+ };
+ },
+ }) as unknown as BrainEngine;
+}
+
+beforeAll(async () => {
+ engine = new PGLiteEngine();
+ await engine.connect({});
+ await engine.initSchema();
+}, 60_000);
+
+afterAll(async () => {
+ if (engine) await engine.disconnect();
+}, 60_000);
+
+beforeEach(async () => {
+ await resetPgliteState(engine);
+ await engine.setConfig('pages.segmented_storage', 'on');
+ dir = mkdtempSync(join(tmpdir(), 'gbrain-seg-race-'));
+ await publish('a', 2 * 1024 * 1024);
+});
+
+afterEach(() => {
+ if (dir) rmSync(dir, { recursive: true, force: true });
+});
+
+describe('GBRA-287: segmented reads across a concurrent publish', () => {
+ test('a live read-back keeps one generation when a body publishes between its rows and its size check', async () => {
+ const before = await pageRow();
+ const reader = interleaved('readManifest', () => true, 'after', () => publish('b', 3 * 1024 * 1024));
+ const body = await materializeSegmentedBody(reader, before.id, 'default');
+ const after = await pageRow();
+ expect(after.generation).toBeGreaterThan(before.generation);
+ expect(after.bytes).not.toBe(before.bytes);
+ expect(body.compiled_truth).toBe(truth('a', 2 * 1024 * 1024).trimEnd() === body.compiled_truth ? body.compiled_truth : truth('a', 2 * 1024 * 1024));
+ expect(body.compiled_truth.startsWith('# Live Example\n\n- a000000000')).toBe(true);
+ expect(body.compiled_truth).not.toContain('- b000000000');
+ }, 120_000);
+
+ test('a live open whose generation a publish retired opens the new live body', async () => {
+ const before = await pageRow();
+ const isManifestRead = (args: unknown[]) => typeof args[0] === 'string' && (args[0] as string).includes('JOIN page_body_manifest m ON m.page_id=$1');
+ const reader = interleaved('executeRaw', isManifestRead, 'before', async () => {
+ await publish('b', 3 * 1024 * 1024);
+ const live = await pageRow();
+ await engine.executeRaw('DELETE FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NULL AND gen_to IS NOT NULL AND gen_to<=$2', [live.id, live.generation]);
+ });
+ const body = await openSegmentedBody(reader, { pageId: before.id });
+ const after = await pageRow();
+ expect(after.generation).toBeGreaterThan(before.generation);
+ expect(body.generation).toBe(after.generation);
+ expect(body.rows.reduce((sum, row) => sum + row.byteLen, 0)).toBe(after.bytes);
+ }, 120_000);
+});
diff --git a/test/sync-reconcile-db-only-segmented.test.ts b/test/sync-reconcile-db-only-segmented.test.ts
new file mode 100644
index 0000000000..c6e3bfe722
--- /dev/null
+++ b/test/sync-reconcile-db-only-segmented.test.ts
@@ -0,0 +1,149 @@
+/**
+ * GBRA-287: a full sync's DB-only reconcile (#2426) re-exports a page stored in segments.
+ *
+ * 1. Protects: a stale page whose file Git never committed, stored in segments (`pages.segmented_storage`), is re-exported
+ * to the working tree by streaming its body through the staging writer the withdrawal mirror and restore_page use:
+ * the file holds the bytes `gbrain export` writes for the page, at 2 MiB and above the 24 MiB single-string ceiling,
+ * is left uncommitted for Git's own door, and leaves no staging file behind. A page whose re-export fails is listed
+ * (`reexport_failures`) and the run still completes and advances. A plain directory (repo-less) source re-exports
+ * nothing and keeps the page.
+ * 2. Fails when: the reconcile refuses a segmented page (`page_body_segmented`), writes bytes that differ from the export,
+ * leaves a staging file, or one bad page stops the run or goes unreported.
+ * 3. PGLite.
+ */
+import { afterAll, beforeAll, beforeEach, afterEach, describe, expect, test } from 'bun:test';
+import { existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { resetPgliteState } from './helpers/reset-pglite.ts';
+import { gitExec } from './helpers/git-exec.ts';
+import { importFromFile } from '../src/core/import-file.ts';
+import { readPageBodyPolicy, SEGMENT_MAX_BYTES } from '../src/core/page-body/limits.ts';
+import { largeFileWork } from '../src/core/persistence/large-file.ts';
+import { PAGE_MAX_BYTES } from '../src/core/page-size-policy.ts';
+import { parseMarkdown, serializePageToMarkdown } from '../src/core/markdown.ts';
+
+let engine: PGLiteEngine;
+let dir: string;
+let repoPath: string;
+
+const line = (i: number) => '- ' + String(i).padStart(9, '0') + ' acme-example journal line with enough text to fill a segment quickly\n';
+const page = (title: string, bytes: number) => '---\ntype: concept\ntitle: ' + title + '\n---\n\n# ' + title + '\n\n'
+ + Array.from({ length: Math.ceil(bytes / line(0).length) }, (_, i) => line(i)).join('') + '\n<!-- timeline -->\n- **2026-10-01** | manual — first entry\n';
+
+/** A page stored in segments whose source_path names a file Git never committed and the checkout lacks. */
+async function dbOnlySegmented(slug: string, bytes: number, sourceId = 'default'): Promise<void> {
+ const parsed = parseMarkdown(page('Example ' + slug, bytes), slug + '.md');
+ const outside = join(dir, 'outside', slug + '.md');
+ mkdirSync(join(outside, '..'), { recursive: true });
+ writeFileSync(outside, serializePageToMarkdown(parsed as never, parsed.tags));
+ expect(await importFromFile(engine, outside, slug + '.md', { sourceId, noEmbed: true, bodyPolicy: await readPageBodyPolicy(engine) })).toMatchObject({ status: 'imported' });
+ const [row] = await engine.executeRaw<{ body_storage: string; source_path: string }>('SELECT body_storage, source_path FROM pages WHERE slug=$1 AND source_id=$2', [slug, sourceId]);
+ expect(row).toEqual({ body_storage: 'segmented', source_path: slug + '.md' });
+}
+
+async function exportedBytes(slug: string): Promise<Buffer> {
+ const { runExport } = await import('../src/commands/export.ts');
+ const out = mkdtempSync(join(dir, 'export-'));
+ rmSync(out, { recursive: true, force: true });
+ await runExport(engine, ['--dir', out, '--source', 'default', '--slug-prefix', slug]);
+ return readFileSync(join(out, slug + '.md'));
+}
+
+const stagingLeft = (root: string): string[] => readdirSync(root, { recursive: true, encoding: 'utf8' }).filter(name => /\.tmp\.|\.streaming$/.test(name));
+
+async function fullSync() {
+ const { performSync } = await import('../src/commands/sync.ts');
+ return performSync(engine, { repoPath, full: true, sourceId: 'default', noPull: true, noEmbed: true, noExtract: true });
+}
+
+beforeAll(async () => {
+ engine = new PGLiteEngine();
+ await engine.connect({});
+ await engine.initSchema();
+}, 60_000);
+
+afterAll(async () => {
+ if (engine) await engine.disconnect();
+}, 60_000);
+
+beforeEach(async () => {
+ await resetPgliteState(engine);
+ await engine.setConfig('pages.segmented_storage', 'on');
+ dir = mkdtempSync(join(tmpdir(), 'gbrain-dbonly-seg-'));
+ repoPath = join(dir, 'repo');
+ mkdirSync(join(repoPath, 'topics'), { recursive: true });
+ gitExec('git init -q', repoPath);
+ gitExec('git config user.email "[email protected]"', repoPath);
+ gitExec('git config user.name "T"', repoPath);
+ writeFileSync(join(repoPath, 'topics/keep.md'), '---\ntype: concept\ntitle: Keep\n---\n\nstill here\n');
+ gitExec('git add -A && git commit -q -m initial', repoPath);
+});
+
+afterEach(() => {
+ if (dir) rmSync(dir, { recursive: true, force: true });
+});
+
+describe('GBRA-287: full-sync re-export of a DB-only page stored in segments', () => {
+ test('a 2 MiB segmented page is streamed to its file with the export bytes, left for Git to commit', async () => {
+ expect(['first_sync', 'synced']).toContain((await fullSync()).status);
+ await dbOnlySegmented('memories/big', 2 * 1024 * 1024);
+ const result = await fullSync();
+ expect(result.status).toBe('first_sync');
+ expect(result.reexport_failures).toBeUndefined();
+ const file = join(repoPath, 'memories/big.md');
+ expect(existsSync(file)).toBe(true);
+ expect(readFileSync(file).equals(await exportedBytes('memories/big'))).toBe(true);
+ expect(stagingLeft(repoPath)).toEqual([]);
+ expect(gitExec('git status --porcelain --untracked-files=all', repoPath)).toContain('?? memories/big.md');
+ expect(await engine.getPage('memories/big')).not.toBeNull();
+ }, 180_000);
+
+ test('a segmented page over the 24 MiB single-string ceiling is streamed, never rendered whole', async () => {
+ expect(['first_sync', 'synced']).toContain((await fullSync()).status);
+ await dbOnlySegmented('memories/huge', PAGE_MAX_BYTES + 256 * 1024);
+ const [row] = await engine.executeRaw<{ body_bytes: number }>("SELECT body_bytes FROM pages WHERE slug='memories/huge'");
+ expect(Number(row!.body_bytes)).toBeGreaterThan(PAGE_MAX_BYTES);
+ Object.assign(largeFileWork, { bytesWritten: 0, largestPiece: 0 });
+ const result = await fullSync();
+ expect(result.status).toBe('first_sync');
+ expect(result.reexport_failures).toBeUndefined();
+ const file = join(repoPath, 'memories/huge.md');
+ expect(readFileSync(file).equals(await exportedBytes('memories/huge'))).toBe(true);
+ expect(largeFileWork.bytesWritten).toBe(readFileSync(file).length);
+ expect(largeFileWork.largestPiece).toBeLessThanOrEqual(SEGMENT_MAX_BYTES);
+ expect(stagingLeft(repoPath)).toEqual([]);
+ }, 600_000);
+
+ test('one page whose re-export fails is listed; the run completes, advances and re-exports the rest', async () => {
+ expect(['first_sync', 'synced']).toContain((await fullSync()).status);
+ await dbOnlySegmented('memories/good', 2 * 1024 * 1024);
+ await dbOnlySegmented('memories/bad', 2 * 1024 * 1024);
+ mkdirSync(join(repoPath, 'memories/bad.md'), { recursive: true });
+ const result = await fullSync();
+ expect(result.status).toBe('first_sync');
+ expect(readFileSync(join(repoPath, 'memories/good.md')).equals(await exportedBytes('memories/good'))).toBe(true);
+ expect(result.reexport_failures).toEqual([expect.objectContaining({ slug: 'memories/bad', path: 'memories/bad.md' })]);
+ expect(result.reexport_failures![0]!.fix).toContain('gbrain');
+ const head = gitExec('git rev-parse HEAD', repoPath).trim();
+ const [source] = await engine.executeRaw<{ last_commit: string }>("SELECT last_commit FROM sources WHERE id='default'");
+ expect(source!.last_commit).toBe(head);
+ expect(await engine.getPage('memories/bad')).not.toBeNull();
+ expect(stagingLeft(repoPath)).toEqual([]);
+ }, 300_000);
+
+ test('a repo-less (plain directory) source re-exports nothing and keeps the page', async () => {
+ const plain = join(dir, 'plain');
+ mkdirSync(join(plain, 'topics'), { recursive: true });
+ writeFileSync(join(plain, 'topics/keep.md'), '---\ntype: concept\ntitle: Keep\n---\n\nstill here\n');
+ await engine.executeRaw("INSERT INTO sources(id,name,local_path) VALUES('plain','plain',$1)", [plain]);
+ await dbOnlySegmented('memories/big', 2 * 1024 * 1024, 'plain');
+ const { performSync } = await import('../src/commands/sync.ts');
+ const result = await performSync(engine, { sourceId: 'plain', noPull: true, noEmbed: true, noExtract: true });
+ expect(result.status).not.toBe('blocked_by_failures');
+ expect(existsSync(join(plain, 'memories/big.md'))).toBe(false);
+ const [row] = await engine.executeRaw<{ deleted_at: string | null; body_storage: string }>("SELECT deleted_at, body_storage FROM pages WHERE slug='memories/big' AND source_id='plain'");
+ expect(row).toEqual({ deleted_at: null, body_storage: 'segmented' });
+ }, 180_000);
+});