Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(GST-1209): the repair republishes only what the next sync would admit
garrytan5fa2fda09c9d
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5b0709c452..c6b0166f84 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -19,7 +19,7 @@ identifiers and attribution are available in the pre-removal Git revision- `src/core/engine-sql/page-body.ts`: `setBodyStagingState` moves only a live staging and refuses `body_staging_expired` for a move on to prepared or committed of an aborted or expired one; `flipBodyGeneration` refuses, inside its transaction, a generation whose live rows do not cover `body_bytes` from 0 with no gap (`bodyGenerationIncompleteError`, detail reason `generation_incomplete`).- `src/core/page-body/schema.ts` (`liveTilingSql`, `untiledSql`), `body-tiling.ts` (`untiledSegmentedPages`): the tiling of a page's live generation as SQL, shared by the flip guard, doctor and the repair.- Doctor `segmented_tiling` (BRAIN, host only, read-only): lists live segmented pages whose live generation does not tile, with the repair's preview as the fix.-- `gbrain repair segmented-tiling --source <id> [--slug <slug>]`: explicit-only, preview-bound, trusted local CLI on the owner host. Republishes each listed page from the blob HEAD holds as a new generation, versions the damaged generation, keeps the page's metadata and trust tier, clears the previewed sync hold and writes a receipt and an audit line. A source with no Git checkout refuses `not_a_git_repo`. See `docs/guides/repair.md#segmented-tiling`.+- `gbrain repair segmented-tiling --source <id> [--slug <slug>]`: explicit-only, preview-bound, trusted local CLI on the owner host. Republishes each listed page from the blob HEAD holds as a new generation, after the import screen the next sync would run and the purge and withdrawal overlay (a file another sync hold withholds, or one the screen refuses, is listed held), versions the damaged generation, keeps the page's metadata and trust tier, clears the previewed sync hold and writes a receipt and an audit line. A source with no Git checkout refuses `not_a_git_repo`. See `docs/guides/repair.md#segmented-tiling`.- Tests: `test/helpers/segmented-publish-tiling-cases.ts` on PGLite (`test/large-pages/segmented-publish-tiling.serial.test.ts`) and Postgres (`test/e2e/segmented-publish-tiling-postgres.test.ts`): a purge abort, a sweep expiry and staged rows deleted under a live staging while the write prepares; each refuses and the page keeps its previous generation. `test/large-pages/segmented-tiling-repair.test.ts`: doctor lists a damaged managed page read-only, the repair's preview, refusals and apply, the next sync completes, and a repo-less source refuses. Doctor goldens regenerated for the new check.## [0.60.560.0] - 2026-11-03diff --git a/docs/architecture/key-files/core-services-2-continued.md b/docs/architecture/key-files/core-services-2-continued.mdindex e625f6670b..0c3cf76aa7 100644--- a/docs/architecture/key-files/core-services-2-continued.md+++ b/docs/architecture/key-files/core-services-2-continued.md@@ -35,7 +35,7 @@ The page-state concurrency foundation under `src/core/page-state/` and the share- `src/core/persistence/large-file.ts` — Wave B PR 2 fix lane D: canonical files over 24 MiB (`LARGE_FILE_MIN_BYTES`) on the publication path, never one string or Buffer. `hashFile`/`hashDescriptor` (the persistence file hash, read in one-segment pieces), `FileBytes` (a file read on demand through one descriptor, the `ByteSource` the drift check's `fileBodyDigests` takes; `close` refuses `source_changed` when the file changed while read), `StreamedFileContent` (restore_page's file: export's serializer over the segments, measured once in preparation, produced again by `stageStreamedContent` into the reserved publication stage after the recovery record is durable and before the publication transaction: streamed into an in-progress sibling (`streamingStagePath`, `<stage>.streaming`), verified, fsynced and only then renamed to the stage's name, so a stream a crash cut short is never at that name and recovery removes it unread; a mismatch removes the stream and refuses `revision_conflict`; seam `large_file:stage_streaming` after each piece), and `stageBeforeCopy` (the prior file copied by the kernel into its `before` stage, hashed and fsynced; the recovery record names it by path and SHA-256 instead of embedding base64, and reserves the file once). `largeFileWork` (@internal) counts bytes read, written and copied and the largest piece. Coordinator: `pageRecoveryRecord` → `largeRecoveryRecord`, `stageLargeFiles`, `publishStagedContent`; a group publication never takes such a member (`groupable`). Flip lane A: the withdrawal mirror of a segmented page (`effects.ts` `mirrorPage`, any size) renders its file with `page-prepare.ts` `unreadPageFile` (a `StreamedFileContent`) and writes it with `stageStreamedContent` before its recovery transaction (`effect-recovery.ts` `stageStreamedMirror`); `existsAt` is the lstat-based leaf check it uses. `prepareFileTarget` reads the prior file through `FileBytes` at any size for a page stored in segments or a streamed target (never `readFileSync` whole), so its drift check reads in pieces of at most a segment. Test: `test/large-pages/segmented-withdrawal-mirror.test.ts`. Tests: `test/large-pages/segmented-delete-restore-file.slow.test.ts` (27 and 90 MiB round trips byte for byte, both engines), `test/large-pages/segmented-delete-crash.slow.test.ts` (SIGKILL between file removal and commit, and after the restoration rename).- `src/core/page-body/body-digests.ts` — Wave B PR 2: the digests the drift check of a segmented page over the single-string ceiling compares (`page-prepare.ts` `segmentedFileMatches`). `fileBodyDigests(bytes, reading, filePath, opts)` runs the streamed importer's pass 1 (`BodyScanner`) over one reading of the file's bytes (an inferred-frontmatter prefix, then the bytes past a BOM), then SHA-256 over each region's byte ranges, and reports a facts fence marker; `storedBodyDigests(engine, pageId, ledger)` hashes each region of the trusted stored body (`readBodyRange`, one segment per read, `readGarbageCollector`). `bodyDigestWork` counts file and stored bytes and the largest piece handed to a scan or hash (at most a segment). `overlaidFileBodyDigests` overlays the file one complete fence block at a time; inside an open block a read stops at FENCE_MAX_BYTES plus `FENCE_READ_LOOKAHEAD` (4 bytes), so an over-bound fence refuses `page_body_segmented` without reading a further segment (`test/pathological/lp-g-file-digest-fence-bound.test.ts`). In `page-prepare.ts`, `prepareFileTarget` compares a page whose body was not read back through `segmentedFileMatches` (the raw reading, then the inferred one from `streamed-import.ts` `inferredHeadPrefix`; the same normalizations as `fileMatchesSnapshot` via `importedFileType`); a file body with a facts fence under a withdrawal ledger that applies refuses `page_body_segmented` instead of reading as drift. `restore_page` of such a page (`prepareUnreadRestore`) versions it, clears `deleted_at` and reseals (or leaves the projection queued when chunks are missing) without re-importing, and publishes `serializePageToMarkdown`'s bytes streamed from the segments by `segmentedExportMarkdown` into one buffer (under the 25 MiB pre-push scan cap); a core page and an external_untrusted writer refuse `page_body_segmented`, and an agent-tier writer's advisory write-gate scan is not re-run over the unchanged body. Tests: `test/large-pages/segmented-file-drift.test.ts`.- `src/core/page-body/body-tiling.ts` (`untiledSegmentedPages`) + `schema.ts` (`liveTilingSql`, `untiledSql`) + `src/core/engine-sql/page-body.ts` — GST-1209: a live generation tiles its body when its manifest rows, as every reader selects them, cover `body_bytes` from byte 0 with no gap. `flipBodyGeneration` checks the generation it made live inside its transaction and refuses one that does not tile (`body_staging_expired`, detail reason `generation_incomplete`, nothing published). `setBodyStagingState` moves only a live staging (open, sealed, prepared): a move on to prepared or committed of a staging a fact purge aborted or a sweep expired (its staged rows deleted) refuses `body_staging_expired` instead of reviving it. `untiledSegmentedPages` lists live segmented pages that do not tile (read-only, one aggregate per page), for doctor `segmented_tiling` (`src/commands/doctor/checks/segmented-tiling.ts`, BRAIN, host only) and the repair.-- `src/core/repair/segmented-tiling.ts` — `gbrain repair segmented-tiling --source <id> [--slug <slug>]` (GST-1209), explicit-only, preview-bound, trusted local CLI on the owner host, `checks: ['segmented_tiling']`. The preview lists each untiled page with its file, HEAD's blob (oid, SHA-256, size) and sync hold under a hash; a page with no file, a file HEAD lacks or one over `pages.max_bytes` is listed held; a source with no Git checkout refuses `not_a_git_repo`. `--apply --expect` re-checks revision, generation and blob (`preview_changed`), then per page in one transaction under the key lock (coordinated maintenance on a managed brain) versions the damaged generation, stages the blob through `prepareSegmentedBody` without reading the damaged body, flips with `applySegmentedBody` (title, type, frontmatter, tags and trust tier kept), clears the previewed hold, calls `requestGitHoldRetry` and writes an `op_checkpoints` receipt (`segmented-tiling-receipt`) and a `segmented-tiling` audit line. Tests: `test/large-pages/segmented-tiling-repair.test.ts`.+- `src/core/repair/segmented-tiling.ts` — `gbrain repair segmented-tiling --source <id> [--slug <slug>]` (GST-1209), explicit-only, preview-bound, trusted local CLI on the owner host, `checks: ['segmented_tiling']`. The preview lists each untiled page with its file, HEAD's blob (oid, SHA-256, size) and sync hold under a hash; a page with no file, a file HEAD lacks or one over `pages.max_bytes`, a file another sync hold withholds (any but the unreadable-body `page_write_failed` one) or one `unreadPageImportScreen` (sync-screen.ts, the next sync's screen against the unread snapshot) refuses is listed held; a source with no Git checkout refuses `not_a_git_repo`. `--apply --expect` re-checks revision, generation and blob (`preview_changed`), then per page in one transaction under the key lock (coordinated maintenance on a managed brain) versions the damaged generation, stages the screened body after `preserveWithdrawnFenceRows` (purged claims stay out) through `prepareSegmentedBody` without reading the damaged body, flips with `applySegmentedBody` (title, type, frontmatter, tags and trust tier kept), clears the previewed hold, calls `requestGitHoldRetry` and writes an `op_checkpoints` receipt (`segmented-tiling-receipt`) and a `segmented-tiling` audit line. Tests: `test/large-pages/segmented-tiling-repair.test.ts`.- `src/core/page-body/segment-facts.ts` + `segment-conversation.ts` — Wave B PR 2 lane 2e2 (spec 8, full-pass mode): the facts fence reconcile and conversation facts over a segmented body. `segmentedFactsFenceView` is `factsFenceView` (facts-fence.ts: each section's parsed fence, which sections hold one, and the #3625 timeline marker) read from the segments: one SQL query selects the live manifest rows that carry facts slots, a forced cut, or a begin/end marker (bytea `position`, nothing transferred), and `bodyBlocks`' `want` option reads only the runs holding them; the segmenter never cuts inside a fence, so a fence lies in one block (a longer one is a forced run, refused above `BLOCK_MAX_BYTES` with `page_body_segmented`), and joining one across blocks is a bounded fallback. `segmentedBodyMatchesFile` is `canonicalCacheState`'s stale-cache check for a segmented page: the canonical file streamed through `BodyScanner` against the stored regions, by length and SHA-256. `readSegmentedConversationInput` gives `readConversationInput` (conversation-parser/body.ts) the summary body through ranged reads: whole within `CONVERSATION_WINDOW_BYTES` (`PAGE_MAX_BYTES`), else the newest whole segments that fit (never starting inside a forced run) with the window's `fallbackDate` from its first segment's entry state; extract-conversation-facts parses that window, passes the date to the parse and the extractor, and writes a `tail_window` receipt (`window_rule: no_prior_facts`), but only on a page with no conversation facts: one that has any keeps them and is skipped (`skipped_too_large`, `window_rule: prior_facts_kept`), since the run clears every prior fact. Each loop counts its reads in `segmentedPassWork`. Consumers: `runExtractFacts` (`planFenceReconcile`, the stale-cache guard, the managed fence-facts request), the sweep's fence candidates (segmented pages with facts slots), `extract-conversation-facts`. Tests: `test/large-pages/segmented-facts-reconcile.test.ts`, `segmented-conversation-facts.test.ts`, `segmented-derived-linear.test.ts`.- `src/core/page-window/bounded-lead.ts` — leads, snippets and synopses read a bounded prefix, never the whole body: fan-out SQL selects `boundedBodyPrefixSql(alias)` (`left(coalesce(alias.compiled_truth, alias.truth_lead), 65536)`: a segmented page's NULL body falls back to its stored lead; the one COALESCE of body and lead the page body reader ratchet allows by name), `boundedRawPrefix` cuts a body at 64 KiB on a code point, and `overlaidBoundedRows` / `overlaidBoundedPrefix` replace a prefix holding a `gbrain:facts:begin` marker with the prefix of the withdrawal-overlaid body (one batched `readPageSnapshotsBatch` read; a page it no longer returns, or a failed read, gets an empty lead). A segmented page's row carries its stored `truth_lead`; when that lead holds a facts fence it is overlaid alone with the withdrawal ledger the snapshot read returns (`overlayCanonicalBodies`, then the same 64 KiB cut), never by reading segments. `pageLeadSource` gives getPage callers (exact-slug and alias snippets) a page's compiled truth, or a segmented page's `truth_lead` without touching its throwing body getters. `projectedLead` is that prefix through `boundedProjectedPrefix` (`document.ts`). Consumers: the exact-slug tier (`exact-lookup.ts`) and alias injection (`alias-hop.ts`) snippets, relational hydrate, the reflex pointer synopsis (`retrieval-reflex.ts` `safeSynopsis`: world-only reads take the projected prefix, the trusted entity-card widening the raw prefix through its own fence strip), the entity card's hit row and `entity names` leads, and think's anchor and window-floor rows (`think/gather.ts`: the raw overlaid prefix for the trusted local CLI, the projected prefix otherwise). Small pages render as before. Pinned by `test/bounded-lead.test.ts` and `test/large-pages/segmented-leads.test.ts`.- `src/core/page-window/document.ts` — the one read projection every remote window, outline and lead slices: `projectPageText` (fence overlay split, `sanitizeRemoteBody` + `credentialSafeProjection`, timeline after `TIMELINE_SEPARATOR`), `pageDocument` / `readPageDocument` (UTF-8 bytes + timeline byte offset, with the page's fence overlay), `chunkDocumentSpan` + `spanHoldsChunk` (a chunk's stored `projected_byte_*` span mapped into the document and verified), `boundedProjectedPrefix`, `locateChunksByStoredSpans` (byte spans to string offsets in one pass), `DEFAULT_INLINE_BODY_BYTES` (256 KB).diff --git a/docs/guides/repair.md b/docs/guides/repair.mdindex b72353eb25..f292f2e7ae 100644--- a/docs/guides/repair.md+++ b/docs/guides/repair.md@@ -173,7 +173,7 @@ work behind exits 0, so scripts should also check `results[].complete`.| `missing-tables` | `schema_missing_tables` | Explicit-only, preview-bound. Recreates, empty, a rebuildable cache table (`query_cache`, `code_traversal_cache`) that the applied migrations created but the brain no longer has, on a brain whose schema version already covers its creating migration, so `apply-migrations` never recreates it (#6911). DDL only (`IF NOT EXISTS`), no journal admission, brain-wide. See [missing tables](#missing-tables). | `user_data_tables`: a missing table that holds user data; this repair never recreates it, so restore its rows from a backup first. `other_tables`: a missing table that holds no user data; `gbrain init --migrate-only` recreates it if the base schema defines it. || `url-clone-fences` | none (`gbrain sync --source <id>` stops at the pull with `pull_blocked_local_changes`) | Explicit-only, preview-bound, destructive consent. Clears the `## Facts` fence rows an older gbrain appended to files of a clone it owns (`gbrain sources add --url`), which block every `git pull --ff-only` that touches those files (#6946). The preview proves each changed file differs from HEAD only by appended fence rows; `--apply --expect <hash> --yes` backs each file up, keeps its rows database-only and restores the file from HEAD. Unmanaged brains only. See [URL clone fences](#url-clone-fences). | `non_fence_edit`, `unparsed_fence`, `staged_change`, `deleted`, `not_in_head`, `unreadable_checkout`: a file with any other difference from HEAD, named with why; a file whose HEAD would not import as its page (`visibility_noncanonical` and the other import holds): apply keeps its rows database-only and restores the file so the clone can pull, but does not import HEAD and reports `held`; files that changed after the preview (`changed_since_preview`). || `segmented-mirror` | none (doctor does not read segmented files; run the preview once if the brain ran a version before v0.60.299.0) | Explicit-only, preview-bound, destructive consent. Rewrites the canonical file of a page stored in segments that an older binary's withdrawal mirror passed by (`page_body_segmented`), so the file still holds a row `forget`, `forget_fact` or `purge_fact` removed from the database. The file becomes the page's canonical render (`gbrain export`'s bytes), written by the withdrawal mirror and committed by its Git effect. See [segmented mirror](#segmented-mirror). | `file_database_drift`: the file also differs from the database for another reason; it is listed held and never rewritten. `not_owner_host`: the preview ran off the source's owner host and read none of its files. |-| `segmented-tiling` | `segmented_tiling` | Explicit-only, preview-bound, trusted local CLI on the owner host. Republishes, from the file Git holds at HEAD, each live page stored in segments whose live generation's rows do not cover its body (every read of it refuses `page_body_rows_unreadable` and a managed sync holds its file). `--source <id>` (`--slug <slug>` narrows), then `--apply --expect <hash>`. See [segmented tiling](#segmented-tiling). | Pages with no file, a file HEAD does not hold or one over `pages.max_bytes` (listed `held`); pages that changed after the preview (`changed_since_preview`). A source with no Git checkout, a repo-less brain included, refuses (`not_a_git_repo`). |+| `segmented-tiling` | `segmented_tiling` | Explicit-only, preview-bound, trusted local CLI on the owner host. Republishes, from the file Git holds at HEAD, each live page stored in segments whose live generation's rows do not cover its body (every read of it refuses `page_body_rows_unreadable` and a managed sync holds its file). `--source <id>` (`--slug <slug>` narrows), then `--apply --expect <hash>`. See [segmented tiling](#segmented-tiling). | Pages with no file, a file HEAD does not hold or one over `pages.max_bytes`, a file another sync hold withholds or one the next sync's import screen would hold (listed `held`); pages that changed after the preview (`changed_since_preview`). A source with no Git checkout, a repo-less brain included, refuses (`not_a_git_repo`). || `secret-spans` | none (the operator names the pages) | Explicit-only, preview-bound, trusted local CLI. Replaces the credentials the secret scanner finds in the database copies of the named pages with `<REDACTED:rule>` markers and changes nothing else: title, frontmatter values (never keys), the body (inline or segmented), timeline, fact, take and link rows, raw data; chunks holding a value are rebuilt. Database only: no file write, no Git commit, and a sync hold stays. `--source <id> --slug <slug>... [--rules <list>]`, then `--apply --expect <hash>`. See [secret spans](#secret-spans). | `clean` (no match), `refused` (`page_not_found`, `page_kind_unsupported`, `page_too_large`, `proof_failed`, `fact_value_unsupported`, `timeline_rows_collide`); pages that moved after the preview (`preview_changed`, `changed_since_preview`). |Timeline rows that an earlier version of a page produced and its current text@@ -444,7 +444,11 @@ source's checkout, and writes nothing. For each page it lists the livegeneration, the bytes its rows cover, the page's file, HEAD's commit, thefile's blob at HEAD (object id, SHA-256 and size) and the sync hold on it. Apage with no file, a file HEAD does not hold or a file over `pages.max_bytes`-is listed `held` and never written. A source without a Git checkout, a+is listed `held` and never written. So is a file that another sync hold+withholds (any hold but the `page_write_failed` one a sync opens when it+cannot read the stored body), and a file the next sync's import screen would+hold: purged page content, a write-gate reject, private visibility it would+not store, a fence it refuses. A source without a Git checkout, arepo-less brain included, refuses with `not_a_git_repo`: this repairrepublishes the copy Git holds, and there the database has no other copy.On such a brain, rewrite the page from a copy you hold (`put_page`).@@ -453,7 +457,9 @@ The apply needs the preview's hash. It checks each page again (revision,generation and blob) and refuses `preview_changed` when one moved. Then, perpage, in one transaction under the page's key lock (a coordinated maintenancewrite on a managed brain), it versions the damaged generation by reference,-publishes the blob's body as a new generation through the same flip that+screens the blob again, publishes the body the screen's fence step and the+purge and withdrawal overlay admit (a claim the owner purged stays out even+when the file still carries it) as a new generation through the same flip thatrefuses a generation that does not cover its body, keeps the page's title,type, frontmatter, tags and trust tier, clears the sync hold the preview saw,asks the next sync to screen the file again (so a file that differs from thediff --git a/scripts/page-body-readers-baseline.tsv b/scripts/page-body-readers-baseline.tsvindex 6c41410e6d..ab78dfb516 100644--- a/scripts/page-body-readers-baseline.tsv+++ b/scripts/page-body-readers-baseline.tsv@@ -556,7 +556,8 @@ src/core/repair/secret-spans.ts readFields field INLINE inline body columns; a ssrc/core/repair/secret-spans.ts writePage call:contentHash INLINE runs only for an inline page; a segmented body is restaged through prepareSegmentedBody and applySegmentedBodysrc/core/repair/secret-spans.ts writePage sql INLINE runs only for an inline page; a segmented body is restaged through prepareSegmentedBody and applySegmentedBodysrc/core/repair/segmented-mirror.ts fileHoldsWithdrawnRows field NONE the overlay result of one file fragment (a facts fence block) for the region digests (body-digests.ts); no page body is read-src/core/repair/segmented-tiling.ts republish field NONE reads the compiled_truth and timeline parseMarkdown returns for the file blob HEAD holds; no stored page body is read (the damaged one is never read)+src/core/repair/segmented-tiling.ts republish field NONE destructures the truth and timeline strings stageable built from the blob HEAD holds; no stored page body is read+src/core/repair/segmented-tiling.ts stageable field NONE reads the compiled_truth and timeline the import screen (or parseMarkdown) returns for the file blob HEAD holds; no stored page body is read (the damaged one is never read)src/core/repair/slug-conflicts.ts pageContent call:serializePageToMarkdown SEGWsrc/core/repair/split-facts.ts mergeOne call:serializePageToMarkdown SEGWsrc/core/repair/split-facts.ts mergeSplitFacts call:serializePageToMarkdown SEGWdiff --git a/src/core/persistence/sync-screen.ts b/src/core/persistence/sync-screen.tsindex 5ac5c925fb..d25744fc53 100644--- a/src/core/persistence/sync-screen.ts+++ b/src/core/persistence/sync-screen.ts@@ -330,6 +330,20 @@ export async function screenFrozenContent(screenInput: SyncImportScreenInput, rureturn screenNormalized(screen) && !await run.normalize() ? screenSyncImport({ ...screenInput, normalize: false }).screen : screen;}+/**+ * GST-1209: the screen the next sync gives a file imported over a page whose stored body cannot be read (the segmented-tiling+ * repair's input): the run's schema pack, sanity settings, purge tombstones and write gate, the coordinated fence step and+ * the private-visibility hold, against the page's snapshot unread. A hold policy of `fail` still screens.+ */+export async function unreadPageImportScreen(engine: BrainEngine, input: { sourceId: string; root: string; path: string; slug: string; content: string;+ snapshot: SyncImportScreenInput['snapshot'] }): Promise<ImportScreenResult> {+ const activePack = (await loadActivePackForEngine(engine, { remote: false, sourceId: input.sourceId }).catch(() => null))?.manifest;+ const run: SyncScreenRun = { policy: await readSyncHoldPolicy(engine), sanity: await loadImportSanityConfig(engine), activePack, normalize: () => fencesNormalizeEnabled(engine) };+ return screenFrozenContent({ content: input.content, rawHash: sha256(input.content), lineEndingOnly: false, slug: input.slug, sourcePath: input.path, path: input.path,+ root: input.root, snapshot: input.snapshot, base: input.snapshot, renamed: false, activePack, sanity: run.sanity,+ purgedPages: await readPagePurgeTombstones(engine, input.sourceId), writeGateSource: await runWriteGateSource(engine, run, input.sourceId) }, run);+}+/*** The hold a frozen import earns, or null to admit it. Throws* `sync_parser_regression` (policy `stop`, opt-in) when gbrain would hold bytesdiff --git a/src/core/repair/registry.ts b/src/core/repair/registry.tsindex e764676289..8c7be71a94 100644--- a/src/core/repair/registry.ts+++ b/src/core/repair/registry.ts@@ -561,10 +561,10 @@ const SPECS: Record<RepairKind, Omit<RepairKindSpec, 'kind'>> = {summary: 'Republish, from the file Git holds at HEAD, each live page stored in segments whose live generation\'s rows do not cover its body (GST-1209; '+ 'doctor segmented_tiling): such a page refuses every read (page_body_rows_unreadable) and a managed sync holds its file on every retry. Needs '+ '--source; --slug <slug> narrows. The preview, on the host that owns the checkout, lists each page with its generation, the bytes its rows '- + 'cover, its file, HEAD\'s commit, the blob\'s object id, SHA-256 and size and its sync hold; a page with no file or a file HEAD does not hold is '- + 'listed held. A source without a Git checkout (a repo-less brain) refuses (not_a_git_repo). --apply --expect <hash> re-checks each page, then '+ + 'cover, its file, HEAD\'s commit, the blob\'s object id, SHA-256 and size and its sync hold; a page with no file or a file HEAD does not hold, '+ + 'another sync hold withholds or the next sync\'s import screen would hold is listed held. A source without a Git checkout (a repo-less brain) refuses (not_a_git_repo). --apply --expect <hash> re-checks each page, then '+ 'per page, in one transaction under its key lock (a coordinated maintenance write on a managed brain): versions the damaged generation by '- + 'reference, publishes the blob\'s body as a new generation (title, type, frontmatter, tags and trust tier kept), clears the previewed sync hold, '+ + 'reference, publishes the body the import screen and the purge overlay admit as a new generation (title, type, frontmatter, tags and trust tier kept), clears the previewed sync hold, '+ 'asks the next sync to re-screen the file and writes the page\'s receipt and audit line. Trusted local CLI only; runs only when named; never on upgrade.',},'uncommitted-deletions': {diff --git a/src/core/repair/segmented-tiling.ts b/src/core/repair/segmented-tiling.tsindex 98a445410e..ae03e416a3 100644--- a/src/core/repair/segmented-tiling.ts+++ b/src/core/repair/segmented-tiling.ts@@ -8,12 +8,15 @@* Preview (read-only; trusted local CLI, on the host that owns the source's checkout): each damaged page with its live* generation, the bytes its rows cover, its file, HEAD's commit, the file's blob at HEAD (object id, SHA-256 and size) and* the sync hold on it. The hash binds the pages, their revisions and generations, the blobs and the holds. A page with no- * file, a file HEAD does not hold or one over pages.max_bytes is listed held and never written. A source without a Git+ * file, a file HEAD does not hold or one over pages.max_bytes is listed held and never written, as is one another sync+ * hold withholds (any but the unreadable-body `page_write_failed` hold) or one the next sync's import screen would hold+ * (purged page content, the write gate, private visibility, a fence it refuses). A source without a Git* checkout (a repo-less brain) refuses: this repair republishes the copy Git holds, and the database has no other.** `--apply --expect <hash>` re-checks every page (`preview_changed`), then per page, in one transaction under the page's* key lock (a coordinated maintenance write on a managed brain; refused while a sync or write in flight names the file):- * versions the damaged generation by reference, stages the blob's body and publishes it as a new generation through the+ * versions the damaged generation by reference, screens the blob again and stages the body the screen's fence step and+ * the purge and withdrawal overlay admit (a purged claim the file still carries stays out), publishes it as a new generation through the* generation flip (which refuses one that does not tile), keeps the page's title, type, frontmatter, tags and trust tier,* clears the sync hold the preview saw and asks the next sync to re-screen the file (so a file that differs from the* stored page in anything else imports normally), and writes the page's receipt, also appended to the `segmented-tiling`@@ -33,7 +36,10 @@ import { getWorktreeBinding } from '../persistence/ownership.ts';import { checkOwner, ownerUnavailableError } from '../persistence/owner-refusal.ts';import { localHostId } from '../persistence/identity.ts';import { loadRepairBusySet, repairBusy, repairBusyError } from '../persistence/repair-busy.ts';-import { clearGitHold, readGitHold, requestGitHoldRetry } from '../persistence/sync-holds.ts';+import { clearGitHold, readGitHold, requestGitHoldRetry, type GitHoldRecord } from '../persistence/sync-holds.ts';+import { unreadPageImportScreen } from '../persistence/sync-screen.ts';+import { PAGE_BODY_FAULT_CODES } from '../persistence/sync-fault-class.ts';+import { preserveWithdrawnFenceRows } from '../facts/withdrawal.ts';import { clearApprovedSet, loadApprovedSet, previewChangedError, previewHash, saveApprovedSet } from '../persistence/preview-approval.ts';import { applySegmentedBody, sealSegmentedPage, segmentedLivePage } from '../persistence/segmented-apply.ts';import { withPageTierKept } from '../trust/fence-append.ts';@@ -114,6 +120,28 @@ function selectionOf(scope: RepairScope, slugs: readonly string[] | undefined):const describe = (page: HealPage) => `generation ${page.body_generation}: ${page.rows} row(s) cover ${page.covered_bytes ?? 'with a gap'}${page.covered_bytes === null ? '' : ' bytes'} of ${page.body_bytes}; `+ `republish ${page.source_path} from HEAD ${page.head.slice(0, 12)} (blob ${page.blob_oid.slice(0, 12)}, ${page.blob_bytes} bytes)${page.hold_observed_at ? '; clears its sync hold' : ''}`;+/** A sync hold this repair may clear: the one a managed sync opens when it cannot read the page's stored body. */+const segmentedReadHold = (hold: GitHoldRecord) => hold.code === 'page_write_failed'+ && PAGE_BODY_FAULT_CODES.has(String((hold.meta as { write_failure?: { code?: unknown } } | null)?.write_failure?.code ?? ''));++/**+ * The body this repair publishes from `content` (the blob HEAD holds), or why the page is held: a sync hold other than the+ * unreadable-body one withholds the file, or the screen the next sync gives the file refuses it. Admitted bytes take the+ * fence step and the purge and withdrawal overlay an import applies, so a purged claim the file still carries stays out.+ */+async function stageable(engine: BrainEngine, page: { source_id: string; slug: string; source_path: string }, checkout: { root: string; incarnation: string },+ content: string): Promise<{ held: string } | { truth: string; timeline: string; hold: GitHoldRecord | null }> {+ const hold = await readGitHold(engine, page.source_id, checkout.incarnation, page.source_path);+ if (hold && !segmentedReadHold(hold)) return { held: `its sync hold (${hold.code}) withholds the file; resolve that hold first (gbrain sources status ${page.source_id})` };+ const screen = await unreadPageImportScreen(engine, { sourceId: page.source_id, root: checkout.root, path: page.source_path, slug: page.slug, content,+ snapshot: await engine.readPageSnapshot(page.slug, { sourceId: page.source_id }) });+ if (screen.status === 'refused') return { held: `the next sync would hold the file (${screen.refusal.code}): ${screen.refusal.message}` };+ const parsed = screen.status === 'importable' && screen.parsed ? screen.parsed : parseMarkdown(content, `${page.slug}.md`);+ const body = screen.status === 'importable' && screen.fences ? screen.fences.after : { compiled_truth: parsed.compiled_truth, timeline: parsed.timeline ?? '' };+ return { hold, truth: (await preserveWithdrawnFenceRows(engine, page.source_id, body.compiled_truth, page.slug)).trim(),+ timeline: (await preserveWithdrawnFenceRows(engine, page.source_id, body.timeline ?? '', page.slug)).trim() };+}+/** Every damaged page of the selection, as the preview lists it (republishable pages and held ones). */async function survey(engine: BrainEngine, selection: Selection): Promise<{ pages: HealPage[]; listing: RepairListing[]; held: number }> {const checkout = await sourceCheckout(engine, selection.source_id);@@ -126,17 +154,18 @@ async function survey(engine: BrainEngine, selection: Selection): Promise<{ pageconst blob = page.source_path ? headBlob(checkout.root, page.source_path, maxBytes) : { held: 'the page has no file, so Git holds no copy of it' };if ('held' in blob) { held++; listing.push({ item: label, class: 'held', detail: blob.held }); continue; }const bytes = readBlob(checkout.root, blob.oid, blob.bytes);- const hold = await readGitHold(engine, page.source_id, checkout.incarnation, page.source_path!);+ const staged = await stageable(engine, { ...page, source_path: page.source_path! }, checkout, bytes.toString('utf8'));+ if ('held' in staged) { held++; listing.push({ item: label, class: 'held', detail: staged.held }); continue; }const heal: HealPage = { ...page, source_path: page.source_path!, incarnation: checkout.incarnation, head: blob.head, blob_oid: blob.oid, blob_sha256: sha256(bytes),- blob_bytes: bytes.length, hold_observed_at: hold?.observed_at ?? null };+ blob_bytes: bytes.length, hold_observed_at: staged.hold?.observed_at ?? null };pages.push(heal);listing.push({ item: label, class: 'republish', detail: describe(heal) });}return { pages: pages.sort((a, b) => a.page_id - b.page_id), listing, held };}-/** Republishes one page's body from `content` in the caller's transaction; false when the page changed since the preview. */-async function republish(tx: BrainEngine, page: HealPage, content: string): Promise<{ generation: number; unsealed: boolean; hold_cleared: boolean } | false> {+/** Republishes one page's body (`stageable`'s) in the caller's transaction and clears `hold`; false when the page changed since the preview. */+async function republish(tx: BrainEngine, page: HealPage, body: { truth: string; timeline: string }, hold: GitHoldRecord | null): Promise<{ generation: number; unsealed: boolean; hold_cleared: boolean } | false> {await tx.lockPageKeys([{ sourceId: page.source_id, slug: page.slug }]);const [now] = await untiledSegmentedPages(tx, { sourceIds: [page.source_id], slugs: [page.slug] });if (!now || now.page_id !== page.page_id || now.revision !== page.revision || now.body_generation !== page.body_generation) return false;@@ -144,8 +173,7 @@ async function republish(tx: BrainEngine, page: HealPage, content: string): Promcorpus_generation: string | null; tags: string[] }>(`SELECT p.title, p.type, p.page_kind, p.frontmatter, p.contextual_retrieval_mode, p.corpus_generation,COALESCE((SELECT jsonb_agg(t.tag ORDER BY t.tag) FROM tags t WHERE t.page_id=p.id), '[]'::jsonb) AS tags FROM pages p WHERE p.id=$1`, [page.page_id]);const frontmatter = row!.frontmatter ?? {};- const parsed = parseMarkdown(content, `${page.slug}.md`);- const truth = parsed.compiled_truth, timeline = parsed.timeline ?? '';+ const { truth, timeline } = body;const { principal } = await maintenanceAttribution(tx);await tx.createVersion(page.slug, { sourceId: page.source_id });const prepared = await prepareSegmentedBody(tx, { sourceId: page.source_id, slug: page.slug, live: await segmentedLivePage(tx, page.page_id),@@ -159,8 +187,8 @@ async function republish(tx: BrainEngine, page: HealPage, content: string): PrompageId: page.page_id, prepared, promotion: false, retireMaxRows: flipRetireMaxRows,meta: { title: row!.title, type: row!.type as never, frontmatter: frontmatter as never, page_kind: row!.page_kind as never } }));if (applied.complete) await sealSegmentedPage(tx, page.page_id);- const hold_cleared = page.hold_observed_at !== null- && await clearGitHold(tx, { sourceId: page.source_id, incarnation: page.incarnation, path: page.source_path, observedAt: page.hold_observed_at });+ const hold_cleared = hold !== null+ && await clearGitHold(tx, { sourceId: page.source_id, incarnation: page.incarnation, path: page.source_path, observedAt: hold.observed_at });await requestGitHoldRetry(tx, page.source_id, page.incarnation, [page.source_path]);return { generation: prepared.newGeneration, unsealed: !applied.complete, hold_cleared };}@@ -221,6 +249,8 @@ export const segmentedTilingRepair: RepairHandler = {const checkout = await sourceCheckout(ctx.engine, page.source_id);const bytes = readBlob(checkout.root, page.blob_oid, page.blob_bytes);if (sha256(bytes) !== page.blob_sha256) return { applied: false, outcome: 'changed_since_preview', reason: 'the blob no longer has the previewed bytes' };+ const staged = await stageable(ctx.engine, page, checkout, bytes.toString('utf8'));+ if ('held' in staged) return { applied: false, outcome: 'held', reason: staged.held };if (checkout.managed) {const busy = await loadRepairBusySet(ctx.engine, page.source_id, page.incarnation);if (repairBusy(busy, { sourcePath: page.source_path, slug: page.slug })) throw repairBusyError(busy, page.source_id, page.source_path);@@ -228,7 +258,7 @@ export const segmentedTilingRepair: RepairHandler = {const written = await ctx.engine.transaction(async tx => {const attribution = await maintenanceAttribution(tx);const write = async () => {- const result = await republish(tx, page, bytes.toString('utf8'));+ const result = await republish(tx, page, staged, staged.hold);if (!result) return null;const [after] = await tx.executeRaw<{ revision: string }>('SELECT knowledge_revision::text AS revision FROM pages WHERE id=$1', [page.page_id]);const receipt = { kind: 'segmented-tiling', hash, source_id: page.source_id, slug: page.slug, page_id: page.page_id, revision_before: page.revision,diff --git a/test/helpers/segmented-publish-tiling-cases.ts b/test/helpers/segmented-publish-tiling-cases.tsindex 34389aabc5..17b1d05320 100644--- a/test/helpers/segmented-publish-tiling-cases.ts+++ b/test/helpers/segmented-publish-tiling-cases.ts@@ -92,4 +92,18 @@ export function defineSegmentedPublishTilingCases(opts: { getEngine: () => Brainexpect(await coveredBytes(engine, retried)).toBe(retried.body_bytes);}, 300_000);}++ test('a staging that is no longer live keeps its state: an abort leaves a committed one and a move on of a terminal one refuses', async () => {+ const engine = opts.getEngine();+ await importFromContent(engine, SLUG, original, { sourceId: 'default', noEmbed: true });+ const [{ id }] = await engine.executeRaw<{ id: string }>("SELECT id FROM page_body_stagings WHERE slug=$1 AND state='committed'", [SLUG]);+ const state = async () => (await engine.executeRaw<{ state: string }>('SELECT state FROM page_body_stagings WHERE id=$1', [id]))[0]!.state;+ await engine.setBodyStagingState(id, 'aborted');+ await engine.setBodyStagingState(id, 'expired');+ expect(await state()).toBe('committed');+ await expect(engine.setBodyStagingState(id, 'prepared')).rejects.toMatchObject({ code: 'body_staging_expired' });+ await engine.executeRaw("UPDATE page_body_stagings SET state='aborted' WHERE id=$1", [id]);+ await expect(engine.setBodyStagingState(id, 'committed')).rejects.toMatchObject({ code: 'body_staging_expired' });+ expect(await state()).toBe('aborted');+ }, 300_000);}diff --git a/test/large-pages/segmented-tiling-repair.test.ts b/test/large-pages/segmented-tiling-repair.test.tsindex 5970c4ccd5..83e9c031a0 100644--- a/test/large-pages/segmented-tiling-repair.test.ts+++ b/test/large-pages/segmented-tiling-repair.test.ts@@ -20,7 +20,7 @@ import { PGLiteEngine } from '../../src/core/pglite-engine.ts';import { claimWorktree } from '../../src/core/persistence/ownership.ts';import { performManagedSync } from '../../src/core/persistence/sync-run.ts';import { disposePersistenceConsumer } from '../../src/core/persistence/service.ts';-import { readGitHold } from '../../src/core/persistence/sync-holds.ts';+import { readGitHold, writeGitHold } from '../../src/core/persistence/sync-holds.ts';import { importFromContent } from '../../src/core/import-file.ts';import { materializeSegmentedBody } from '../../src/core/page-body/ingest.ts';import { untiledSegmentedPages } from '../../src/core/page-body/body-tiling.ts';@@ -41,7 +41,12 @@ let engine: PGLiteEngine;function git(root: string, ...args: string[]): string { return execFileSync('git', ['-C', root, ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim(); }function commit(root: string): void { git(root, 'add', '-A'); git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'commit', '-qm', 'test content'); }-const person = (bytes: number) => '---\ntype: person\ntitle: Person Example\n---\n\n# Person Example\n\n' + conversationBody(bytes, { seed: 7 }) + '\n<!-- timeline -->\n- **2026-10-01** | manual — first\n';+const person = (bytes: number, fence = '') => '---\ntype: person\ntitle: Person Example\n---\n\n# Person Example\n\n' + fence + conversationBody(bytes, { seed: 7 }) + '\n<!-- timeline -->\n- **2026-10-01** | manual — first\n';+const PURGED = 'Person Example keeps the spare key under the blue mat.';+const KEPT = 'Person Example prefers the green notebook.';+const FENCE = ['## Facts', '', '<!--- gbrain:facts:begin -->', '| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |',+ '|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|', `| 1 | ${PURGED} | fact | 1.0 | world | medium | 2026-01-01 | | chat | |`,+ `| 2 | ${KEPT} | fact | 1.0 | world | medium | 2026-01-01 | | chat | |`, '<!--- gbrain:facts:end -->', '', ''].join('\n');/** Deletes the page's last live manifest row: its live generation no longer covers its body. */const damage = (sourceId: string, pageId: number) => withManagedFixtureWrite(engine, [sourceId], tx => tx.executeRaw('DELETE FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NULL AND seq=(SELECT max(seq) FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NULL)', [pageId]));@@ -55,14 +60,14 @@ afterAll(async () => {rmSync(home, { recursive: true, force: true });});-async function managedFixture() {+async function managedFixture(content = person(1260 * 1024)) {await disposePersistenceConsumer(engine);await engine.setConfig('pages.segmented_storage', 'on');const sourceId = `tiling-${randomUUID().slice(0, 8)}`;const root = join(home, sourceId);mkdirSync(join(root, 'people'), { recursive: true });git(root, 'init', '-q');- writeFileSync(join(root, FILE), person(1260 * 1024));+ writeFileSync(join(root, FILE), content);commit(root);await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');await engine.executeRaw("INSERT INTO sources(id,name,local_path,config) VALUES($1,$1,$2,'{}')", [sourceId, root]);@@ -129,6 +134,58 @@ test('doctor lists the damaged page read-only; the repair republishes it from HEexpect((await run(false)).listing ?? []).toEqual([]);}), 300_000);+test('a page HEAD no longer holds, or whose file another sync hold withholds, is listed held and never written', async () => withEnv(env, async () => {+ const f = await managedFixture();+ await damage(f.sourceId, f.pageId);+ const scope = await resolveRepairScope(engine, f.sourceId);+ const run = (apply: boolean, expect?: string) => repairRunner(engine, { apply, logger }).then(runner => runner.run('segmented-tiling', scope, { explicit: true, sourceFlag: f.sourceId, ...(expect ? { expect } : {}) }));+ const [damaged] = await untiledSegmentedPages(engine, { sourceIds: [f.sourceId] });+ git(f.root, 'rm', '-q', FILE);+ commit(f.root);+ const gone = await run(false);+ expect(gone.listing).toEqual([{ item: `${f.sourceId}:${SLUG} (revision ${damaged!.revision})`, class: 'held', detail: expect.stringContaining(`holds no file at ${FILE}`) }]);+ expect(await run(true, gone.preview_hash!)).toMatchObject({ applied: 0 });+ expect(await untiledSegmentedPages(engine, { sourceIds: [f.sourceId] })).toEqual([damaged!]);++ git(f.root, 'checkout', '-q', 'HEAD~1', '--', FILE);+ commit(f.root);+ const [source] = await engine.executeRaw<{ incarnation: string }>('SELECT incarnation::text AS incarnation FROM sources WHERE id=$1', [f.sourceId]);+ await writeGitHold(engine, { source_id: f.sourceId, incarnation: source!.incarnation, path: FILE, source_path: FILE, slug: SLUG, page_id: f.pageId, code: 'write_gate_rejected',+ message: 'The write gate rejected this file.', upstream_version: null, observed_at: new Date().toISOString(), run_id: null, mode: 'managed', meta: { recovery_version: 1 } });+ const gated = await run(false);+ expect(gated.listing).toEqual([expect.objectContaining({ class: 'held', detail: expect.stringContaining('sync hold (write_gate_rejected)') })]);+ expect(await run(true, gated.preview_hash!)).toMatchObject({ applied: 0 });+ expect(await untiledSegmentedPages(engine, { sourceIds: [f.sourceId] })).toEqual([damaged!]);+ expect(await readGitHold(engine, f.sourceId, source!.incarnation, FILE)).toMatchObject({ code: 'write_gate_rejected' });+}), 300_000);++test('the apply refuses preview_changed when HEAD moved after the preview, and writes nothing', async () => withEnv(env, async () => {+ const f = await managedFixture();+ await damage(f.sourceId, f.pageId);+ const scope = await resolveRepairScope(engine, f.sourceId);+ const run = (apply: boolean, expect?: string) => repairRunner(engine, { apply, logger }).then(runner => runner.run('segmented-tiling', scope, { explicit: true, sourceFlag: f.sourceId, ...(expect ? { expect } : {}) }));+ const preview = await run(false);+ appendFileSync(join(f.root, FILE), '- **2026-11-03** | meeting — third\n');+ commit(f.root);+ const before = await counts();+ await expect(run(true, preview.preview_hash!)).rejects.toMatchObject({ code: 'preview_changed' });+ expect(await counts()).toEqual(before);+ expect(await untiledSegmentedPages(engine, { sourceIds: [f.sourceId] })).toHaveLength(1);+}), 300_000);++test('a claim the owner purged stays out: the apply republishes the file through the purge overlay', async () => withEnv(env, async () => {+ const f = await managedFixture(person(1260 * 1024, FENCE));+ await engine.executeRaw("INSERT INTO fact_purges(source_id, visibility, subject, fact_hash) VALUES ($1, 'world', '*', gbrain_fact_fingerprint($2))", [f.sourceId, PURGED]);+ await damage(f.sourceId, f.pageId);+ const scope = await resolveRepairScope(engine, f.sourceId);+ const run = (apply: boolean, expect?: string) => repairRunner(engine, { apply, logger }).then(runner => runner.run('segmented-tiling', scope, { explicit: true, sourceFlag: f.sourceId, ...(expect ? { expect } : {}) }));+ expect(await run(true, (await run(false)).preview_hash!)).toMatchObject({ applied: 1 });+ const body = await materializeSegmentedBody(engine, f.pageId, f.sourceId);+ expect(body.compiled_truth).toContain(KEPT);+ expect(body.compiled_truth).not.toContain(PURGED);+ expect(await engine.executeRaw('SELECT count(*)::int AS n FROM content_chunks WHERE page_id=$1 AND chunk_text LIKE $2', [f.pageId, `%${PURGED}%`])).toEqual([{ n: 0 }]);+}), 300_000);+test('a source without a Git checkout refuses the preview: Git is the only copy it republishes', async () => withEnv(env, async () => {await disposePersistenceConsumer(engine);await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');