Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(page-body): a write cannot publish a segmented generation without its rows; doctor segmented_tiling and gbrain repair segmented-tiling (GST-1209)

garrytan56e17fd8b5cc
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 3a084c7a14..5b0709c452 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,18 @@ credits are retained; no result has been reassigned to another provider. Origina
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
+## [Unreleased]
+
+**A write to a page stored in segments can no longer publish a body nobody can read.** When a fact purge or a staging sweep collected a write's staging while the write was preparing, the write's last preparation step moved the staging back to `prepared`, so the publish passed its staging check, closed the page's old segment rows and published none. Every read of the page then refused `page_body_rows_unreadable` and a managed sync held its file on every retry. The write now refuses `body_staging_expired` (send it again) and the page keeps its previous body; the publish also refuses any generation whose rows do not cover the body it records. A new doctor check, `segmented_tiling`, lists pages an older version left like this, and `gbrain repair segmented-tiling --source <id>` republishes each from its file at HEAD, preview first, only when applied with the preview's hash.
+
+### Itemized changes
+
+- `src/core/engine-sql/page-body.ts`: `setBodyStagingState` moves only a live staging and refuses `body_staging_expired` for a move on to prepared or committed of an aborted or expired one; `flipBodyGeneration` refuses, inside its transaction, a generation whose live rows do not cover `body_bytes` from 0 with no gap (`bodyGenerationIncompleteError`, detail reason `generation_incomplete`).
+- `src/core/page-body/schema.ts` (`liveTilingSql`, `untiledSql`), `body-tiling.ts` (`untiledSegmentedPages`): the tiling of a page's live generation as SQL, shared by the flip guard, doctor and the repair.
+- Doctor `segmented_tiling` (BRAIN, host only, read-only): lists live segmented pages whose live generation does not tile, with the repair's preview as the fix.
+- `gbrain repair segmented-tiling --source <id> [--slug <slug>]`: explicit-only, preview-bound, trusted local CLI on the owner host. Republishes each listed page from the blob HEAD holds as a new generation, versions the damaged generation, keeps the page's metadata and trust tier, clears the previewed sync hold and writes a receipt and an audit line. A source with no Git checkout refuses `not_a_git_repo`. See `docs/guides/repair.md#segmented-tiling`.
+- Tests: `test/helpers/segmented-publish-tiling-cases.ts` on PGLite (`test/large-pages/segmented-publish-tiling.serial.test.ts`) and Postgres (`test/e2e/segmented-publish-tiling-postgres.test.ts`): a purge abort, a sweep expiry and staged rows deleted under a live staging while the write prepares; each refuses and the page keeps its previous generation. `test/large-pages/segmented-tiling-repair.test.ts`: doctor lists a damaged managed page read-only, the repair's preview, refusals and apply, the next sync completes, and a repo-less source refuses. Doctor goldens regenerated for the new check.
+
## [0.60.560.0] - 2026-11-03
**A page stored in segments no longer stops a source's sync.** A managed sync re-importing a Git-tracked page stored in segments (`pages.segmented_storage`) whose working-tree bytes differ from the pinned commit, as on a live checkout, compared the file against the stored page with a whole-body read. That read refused with `page_body_segmented`, recorded as a run failure in the freeze phase, and every later sync stopped `sync_blocked` at the same file, with a retry command that could not help. The freeze now compares a segmented page up to 24 MiB read back whole, as its publication already did, so the page gets the outcome an inline page gets. A page whose stored body can't be read at the freeze (`page_body_segmented`, or `page_body_rows_unreadable`) is held `page_write_failed`, naming its code, and the rest of the source syncs. `gbrain sync status` classifies such a hold by that code (`page_body_rows_unreadable`: retry, no person needed), not as an unplaced `page_write_failed`. A run that does stop before any write request is admitted (discovery, freeze or admission) now shows `last_error.request_id: null` in `gbrain sync status --json` instead of leaving the field out: there is no receipt, root request or `write_error_detail` to look for, and the phase, code and message name the error.
diff --git a/docs/architecture/coordinated-writes.tsv b/docs/architecture/coordinated-writes.tsv
index 071c46deb9..1a87c84760 100644
--- a/docs/architecture/coordinated-writes.tsv
+++ b/docs/architecture/coordinated-writes.tsv
@@ -35,6 +35,7 @@ src/core/repair/frontmatter-keys.ts apply 1 projection Strips parser-reserved ke
src/core/repair/ontology-facts.ts apply 1 facts_lane Restores ontology-observation facts; G4 (GBRA-190) gives ontology observations a page home.
src/core/repair/segmented-mirror.ts admitRemirror 1 door Admits a journaled re-mirror request and queues its withdrawal-mirror and Git effects, which rewrite the canonical file.
src/core/repair/secret-spans.ts apply 1 projection Replaces only the credential spans the secret scanner finds in a named page's stored rows with <REDACTED:rule> markers and adds no content; it is the database side of a redaction whose file edit goes through Git, and chunks are rebuilt from the redacted body.
+src/core/repair/segmented-tiling.ts apply 1 projection Republishes a named page whose live segmented generation lost its rows from its file at HEAD (the committed markdown) as a new generation; it writes no file and adds no content Git does not hold.
src/core/repair/timeline-comments.ts cleanRows 1 projection Rewrites or deletes timeline rows still carrying markup so they equal the tuple the parser reads from the page text.
src/core/shared-skills/publication.ts validate 1 operational Prunes retained shared-skill revisions inside a skill-bundle publication; retention bookkeeping, the bundle files are published separately.
src/core/shared-skills/retention.ts pruneSharedSkillRevisions 1 operational Prunes retained shared-skill revisions past the retention limits; retention bookkeeping, not memory content.
diff --git a/docs/architecture/key-files/core-services-2-continued.md b/docs/architecture/key-files/core-services-2-continued.md
index c10aa2991a..e625f6670b 100644
--- a/docs/architecture/key-files/core-services-2-continued.md
+++ b/docs/architecture/key-files/core-services-2-continued.md
@@ -34,6 +34,8 @@ The page-state concurrency foundation under `src/core/page-state/` and the share
- `src/core/page-body/segment-blocks.ts` + `segment-links.ts` + `segment-takes.ts` — Wave B PR 2 (spec 8, full-pass mode): the derived passes over a segmented body, O(page) work and O(segment) memory, not yet incremental. `bodyBlocks(engine, page, { regions?, separator?, want? })` yields the trusted, withdrawal-overlaid joined body one block at a time (a segment, or a run joined across forced cuts, refused above `BLOCK_MAX_BYTES` (read-range.ts) with `page_body_segmented`; `want` skips a run unread), so each block starts where the parsers start fresh; `truthLead` reads a bounded lead; `readGarbageCollector` forces a collection every 4 MiB read; `detachedValue` copies a result kept past its block so it pins no segment text; `segmentedPassWork` counts the passes' work for the linearity test. `extractSegmentedPageLinks` runs `extractBodyLinks` over each block in a window (`bodyWindows`: the whole blocks before it, 4096 chars after, the headings still open), with the role phrases a first pass found (`segmentedRoleText`), and merges candidates in `extractPageLinks` order, plus the temporal inputs (`TemporalBody`). `parseTakesFenceBlocks` gives `parseTakesFence` of the joined body. Consumers: link preparation and the links effect, stale-links maintenance, source reconciliation, the sweep, `extract links|timeline|takes --source db`, `extract --stale` (the stale list includes segmented pages since PR 2 fix lane J1; test `test/large-pages/segmented-extraction-stale.test.ts`), the mention pass (`pageMentions`), the canonical prior view (`segmentedPriorView`), `segmentedTimelineText`, the chronicle judge and the facts planner (`planSegmentedExtraction`). Tests: `test/large-pages/segmented-derived-parity.test.ts`, `segmented-derived-passes.test.ts`, `segmented-derived-linear.test.ts`, `segmented-derived-memory.slow.test.ts`, `segmented-derived-work.test.ts`. `segment-lines.ts` serves the line-local maintenance scans: `bodyLines` yields the joined text's complete lines a block at a time (a line cut by a block boundary is carried, counted in `segmentedPassWork.carry`), consumed by `gbrain integrity` and `reconcile-links`; `segmentedRegionContains` searches one region for literal markers across block boundaries (repair `connector-fences`). Test: `test/large-pages/segmented-maintenance-commands.test.ts`.
- `src/core/persistence/large-file.ts` — Wave B PR 2 fix lane D: canonical files over 24 MiB (`LARGE_FILE_MIN_BYTES`) on the publication path, never one string or Buffer. `hashFile`/`hashDescriptor` (the persistence file hash, read in one-segment pieces), `FileBytes` (a file read on demand through one descriptor, the `ByteSource` the drift check's `fileBodyDigests` takes; `close` refuses `source_changed` when the file changed while read), `StreamedFileContent` (restore_page's file: export's serializer over the segments, measured once in preparation, produced again by `stageStreamedContent` into the reserved publication stage after the recovery record is durable and before the publication transaction: streamed into an in-progress sibling (`streamingStagePath`, `<stage>.streaming`), verified, fsynced and only then renamed to the stage's name, so a stream a crash cut short is never at that name and recovery removes it unread; a mismatch removes the stream and refuses `revision_conflict`; seam `large_file:stage_streaming` after each piece), and `stageBeforeCopy` (the prior file copied by the kernel into its `before` stage, hashed and fsynced; the recovery record names it by path and SHA-256 instead of embedding base64, and reserves the file once). `largeFileWork` (@internal) counts bytes read, written and copied and the largest piece. Coordinator: `pageRecoveryRecord` → `largeRecoveryRecord`, `stageLargeFiles`, `publishStagedContent`; a group publication never takes such a member (`groupable`). Flip lane A: the withdrawal mirror of a segmented page (`effects.ts` `mirrorPage`, any size) renders its file with `page-prepare.ts` `unreadPageFile` (a `StreamedFileContent`) and writes it with `stageStreamedContent` before its recovery transaction (`effect-recovery.ts` `stageStreamedMirror`); `existsAt` is the lstat-based leaf check it uses. `prepareFileTarget` reads the prior file through `FileBytes` at any size for a page stored in segments or a streamed target (never `readFileSync` whole), so its drift check reads in pieces of at most a segment. Test: `test/large-pages/segmented-withdrawal-mirror.test.ts`. Tests: `test/large-pages/segmented-delete-restore-file.slow.test.ts` (27 and 90 MiB round trips byte for byte, both engines), `test/large-pages/segmented-delete-crash.slow.test.ts` (SIGKILL between file removal and commit, and after the restoration rename).
- `src/core/page-body/body-digests.ts` — Wave B PR 2: the digests the drift check of a segmented page over the single-string ceiling compares (`page-prepare.ts` `segmentedFileMatches`). `fileBodyDigests(bytes, reading, filePath, opts)` runs the streamed importer's pass 1 (`BodyScanner`) over one reading of the file's bytes (an inferred-frontmatter prefix, then the bytes past a BOM), then SHA-256 over each region's byte ranges, and reports a facts fence marker; `storedBodyDigests(engine, pageId, ledger)` hashes each region of the trusted stored body (`readBodyRange`, one segment per read, `readGarbageCollector`). `bodyDigestWork` counts file and stored bytes and the largest piece handed to a scan or hash (at most a segment). `overlaidFileBodyDigests` overlays the file one complete fence block at a time; inside an open block a read stops at FENCE_MAX_BYTES plus `FENCE_READ_LOOKAHEAD` (4 bytes), so an over-bound fence refuses `page_body_segmented` without reading a further segment (`test/pathological/lp-g-file-digest-fence-bound.test.ts`). In `page-prepare.ts`, `prepareFileTarget` compares a page whose body was not read back through `segmentedFileMatches` (the raw reading, then the inferred one from `streamed-import.ts` `inferredHeadPrefix`; the same normalizations as `fileMatchesSnapshot` via `importedFileType`); a file body with a facts fence under a withdrawal ledger that applies refuses `page_body_segmented` instead of reading as drift. `restore_page` of such a page (`prepareUnreadRestore`) versions it, clears `deleted_at` and reseals (or leaves the projection queued when chunks are missing) without re-importing, and publishes `serializePageToMarkdown`'s bytes streamed from the segments by `segmentedExportMarkdown` into one buffer (under the 25 MiB pre-push scan cap); a core page and an external_untrusted writer refuse `page_body_segmented`, and an agent-tier writer's advisory write-gate scan is not re-run over the unchanged body. Tests: `test/large-pages/segmented-file-drift.test.ts`.
+- `src/core/page-body/body-tiling.ts` (`untiledSegmentedPages`) + `schema.ts` (`liveTilingSql`, `untiledSql`) + `src/core/engine-sql/page-body.ts` — GST-1209: a live generation tiles its body when its manifest rows, as every reader selects them, cover `body_bytes` from byte 0 with no gap. `flipBodyGeneration` checks the generation it made live inside its transaction and refuses one that does not tile (`body_staging_expired`, detail reason `generation_incomplete`, nothing published). `setBodyStagingState` moves only a live staging (open, sealed, prepared): a move on to prepared or committed of a staging a fact purge aborted or a sweep expired (its staged rows deleted) refuses `body_staging_expired` instead of reviving it. `untiledSegmentedPages` lists live segmented pages that do not tile (read-only, one aggregate per page), for doctor `segmented_tiling` (`src/commands/doctor/checks/segmented-tiling.ts`, BRAIN, host only) and the repair.
+- `src/core/repair/segmented-tiling.ts` — `gbrain repair segmented-tiling --source <id> [--slug <slug>]` (GST-1209), explicit-only, preview-bound, trusted local CLI on the owner host, `checks: ['segmented_tiling']`. The preview lists each untiled page with its file, HEAD's blob (oid, SHA-256, size) and sync hold under a hash; a page with no file, a file HEAD lacks or one over `pages.max_bytes` is listed held; a source with no Git checkout refuses `not_a_git_repo`. `--apply --expect` re-checks revision, generation and blob (`preview_changed`), then per page in one transaction under the key lock (coordinated maintenance on a managed brain) versions the damaged generation, stages the blob through `prepareSegmentedBody` without reading the damaged body, flips with `applySegmentedBody` (title, type, frontmatter, tags and trust tier kept), clears the previewed hold, calls `requestGitHoldRetry` and writes an `op_checkpoints` receipt (`segmented-tiling-receipt`) and a `segmented-tiling` audit line. Tests: `test/large-pages/segmented-tiling-repair.test.ts`.
- `src/core/page-body/segment-facts.ts` + `segment-conversation.ts` — Wave B PR 2 lane 2e2 (spec 8, full-pass mode): the facts fence reconcile and conversation facts over a segmented body. `segmentedFactsFenceView` is `factsFenceView` (facts-fence.ts: each section's parsed fence, which sections hold one, and the #3625 timeline marker) read from the segments: one SQL query selects the live manifest rows that carry facts slots, a forced cut, or a begin/end marker (bytea `position`, nothing transferred), and `bodyBlocks`' `want` option reads only the runs holding them; the segmenter never cuts inside a fence, so a fence lies in one block (a longer one is a forced run, refused above `BLOCK_MAX_BYTES` with `page_body_segmented`), and joining one across blocks is a bounded fallback. `segmentedBodyMatchesFile` is `canonicalCacheState`'s stale-cache check for a segmented page: the canonical file streamed through `BodyScanner` against the stored regions, by length and SHA-256. `readSegmentedConversationInput` gives `readConversationInput` (conversation-parser/body.ts) the summary body through ranged reads: whole within `CONVERSATION_WINDOW_BYTES` (`PAGE_MAX_BYTES`), else the newest whole segments that fit (never starting inside a forced run) with the window's `fallbackDate` from its first segment's entry state; extract-conversation-facts parses that window, passes the date to the parse and the extractor, and writes a `tail_window` receipt (`window_rule: no_prior_facts`), but only on a page with no conversation facts: one that has any keeps them and is skipped (`skipped_too_large`, `window_rule: prior_facts_kept`), since the run clears every prior fact. Each loop counts its reads in `segmentedPassWork`. Consumers: `runExtractFacts` (`planFenceReconcile`, the stale-cache guard, the managed fence-facts request), the sweep's fence candidates (segmented pages with facts slots), `extract-conversation-facts`. Tests: `test/large-pages/segmented-facts-reconcile.test.ts`, `segmented-conversation-facts.test.ts`, `segmented-derived-linear.test.ts`.
- `src/core/page-window/bounded-lead.ts` — leads, snippets and synopses read a bounded prefix, never the whole body: fan-out SQL selects `boundedBodyPrefixSql(alias)` (`left(coalesce(alias.compiled_truth, alias.truth_lead), 65536)`: a segmented page's NULL body falls back to its stored lead; the one COALESCE of body and lead the page body reader ratchet allows by name), `boundedRawPrefix` cuts a body at 64 KiB on a code point, and `overlaidBoundedRows` / `overlaidBoundedPrefix` replace a prefix holding a `gbrain:facts:begin` marker with the prefix of the withdrawal-overlaid body (one batched `readPageSnapshotsBatch` read; a page it no longer returns, or a failed read, gets an empty lead). A segmented page's row carries its stored `truth_lead`; when that lead holds a facts fence it is overlaid alone with the withdrawal ledger the snapshot read returns (`overlayCanonicalBodies`, then the same 64 KiB cut), never by reading segments. `pageLeadSource` gives getPage callers (exact-slug and alias snippets) a page's compiled truth, or a segmented page's `truth_lead` without touching its throwing body getters. `projectedLead` is that prefix through `boundedProjectedPrefix` (`document.ts`). Consumers: the exact-slug tier (`exact-lookup.ts`) and alias injection (`alias-hop.ts`) snippets, relational hydrate, the reflex pointer synopsis (`retrieval-reflex.ts` `safeSynopsis`: world-only reads take the projected prefix, the trusted entity-card widening the raw prefix through its own fence strip), the entity card's hit row and `entity names` leads, and think's anchor and window-floor rows (`think/gather.ts`: the raw overlaid prefix for the trusted local CLI, the projected prefix otherwise). Small pages render as before. Pinned by `test/bounded-lead.test.ts` and `test/large-pages/segmented-leads.test.ts`.
- `src/core/page-window/document.ts` — the one read projection every remote window, outline and lead slices: `projectPageText` (fence overlay split, `sanitizeRemoteBody` + `credentialSafeProjection`, timeline after `TIMELINE_SEPARATOR`), `pageDocument` / `readPageDocument` (UTF-8 bytes + timeline byte offset, with the page's fence overlay), `chunkDocumentSpan` + `spanHoldsChunk` (a chunk's stored `projected_byte_*` span mapped into the document and verified), `boundedProjectedPrefix`, `locateChunksByStoredSpans` (byte spans to string offsets in one pass), `DEFAULT_INLINE_BODY_BYTES` (256 KB).
diff --git a/docs/guides/error-codes.md b/docs/guides/error-codes.md
index 55b53c736c..760d172532 100644
--- a/docs/guides/error-codes.md
+++ b/docs/guides/error-codes.md
@@ -3115,7 +3115,7 @@ More: [docs/guides/troubleshooting.md#owner-unavailable](../../docs/guides/troub
| Meaning | Why | Next step | Who acts · consent | Verify | Exit | Retryable |
|---|---|---|---|---|---|---|
-| This large page's segment rows readable on this connection do not cover its body, so the read returned nothing rather than a short or empty body. | A segmented page keeps its body in manifest rows that must tile the stored body from its first byte to body_bytes; a role that sees the page row but not its manifest rows (a row-level security policy or grant that covers pages only), or rows missing from the database, would otherwise read as an empty or truncated page with nothing to say so. | Give the role that reads the brain the same access to page_body_manifest and page_segment_texts it has to pages (the brain host's operator), then read again; run gbrain doctor --json to check the segment tables. | host_admin | `gbrain doctor --json` | 1 | no |
+| This large page's segment rows readable on this connection do not cover its body, so the read returned nothing rather than a short or empty body. | A segmented page keeps its body in manifest rows that must tile the stored body from its first byte to body_bytes; a role that sees the page row but not its manifest rows (a row-level security policy or grant that covers pages only), or rows missing from the database, would otherwise read as an empty or truncated page with nothing to say so. | Give the role that reads the brain the same access to page_body_manifest and page_segment_texts it has to pages (the brain host's operator), then read again; run gbrain doctor --json to check the segment tables. When doctor segmented_tiling lists the page, its rows are missing: preview gbrain repair segmented-tiling --source <id>, which republishes it from Git. | host_admin | `gbrain doctor --json` | 1 | no |
More: [docs/guides/windowed-reads.md#page_body_rows_unreadable](../../docs/guides/windowed-reads.md#page_body_rows_unreadable)
diff --git a/docs/guides/repair.md b/docs/guides/repair.md
index 90020e3281..b72353eb25 100644
--- a/docs/guides/repair.md
+++ b/docs/guides/repair.md
@@ -173,6 +173,7 @@ work behind exits 0, so scripts should also check `results[].complete`.
| `missing-tables` | `schema_missing_tables` | Explicit-only, preview-bound. Recreates, empty, a rebuildable cache table (`query_cache`, `code_traversal_cache`) that the applied migrations created but the brain no longer has, on a brain whose schema version already covers its creating migration, so `apply-migrations` never recreates it (#6911). DDL only (`IF NOT EXISTS`), no journal admission, brain-wide. See [missing tables](#missing-tables). | `user_data_tables`: a missing table that holds user data; this repair never recreates it, so restore its rows from a backup first. `other_tables`: a missing table that holds no user data; `gbrain init --migrate-only` recreates it if the base schema defines it. |
| `url-clone-fences` | none (`gbrain sync --source <id>` stops at the pull with `pull_blocked_local_changes`) | Explicit-only, preview-bound, destructive consent. Clears the `## Facts` fence rows an older gbrain appended to files of a clone it owns (`gbrain sources add --url`), which block every `git pull --ff-only` that touches those files (#6946). The preview proves each changed file differs from HEAD only by appended fence rows; `--apply --expect <hash> --yes` backs each file up, keeps its rows database-only and restores the file from HEAD. Unmanaged brains only. See [URL clone fences](#url-clone-fences). | `non_fence_edit`, `unparsed_fence`, `staged_change`, `deleted`, `not_in_head`, `unreadable_checkout`: a file with any other difference from HEAD, named with why; a file whose HEAD would not import as its page (`visibility_noncanonical` and the other import holds): apply keeps its rows database-only and restores the file so the clone can pull, but does not import HEAD and reports `held`; files that changed after the preview (`changed_since_preview`). |
| `segmented-mirror` | none (doctor does not read segmented files; run the preview once if the brain ran a version before v0.60.299.0) | Explicit-only, preview-bound, destructive consent. Rewrites the canonical file of a page stored in segments that an older binary's withdrawal mirror passed by (`page_body_segmented`), so the file still holds a row `forget`, `forget_fact` or `purge_fact` removed from the database. The file becomes the page's canonical render (`gbrain export`'s bytes), written by the withdrawal mirror and committed by its Git effect. See [segmented mirror](#segmented-mirror). | `file_database_drift`: the file also differs from the database for another reason; it is listed held and never rewritten. `not_owner_host`: the preview ran off the source's owner host and read none of its files. |
+| `segmented-tiling` | `segmented_tiling` | Explicit-only, preview-bound, trusted local CLI on the owner host. Republishes, from the file Git holds at HEAD, each live page stored in segments whose live generation's rows do not cover its body (every read of it refuses `page_body_rows_unreadable` and a managed sync holds its file). `--source <id>` (`--slug <slug>` narrows), then `--apply --expect <hash>`. See [segmented tiling](#segmented-tiling). | Pages with no file, a file HEAD does not hold or one over `pages.max_bytes` (listed `held`); pages that changed after the preview (`changed_since_preview`). A source with no Git checkout, a repo-less brain included, refuses (`not_a_git_repo`). |
| `secret-spans` | none (the operator names the pages) | Explicit-only, preview-bound, trusted local CLI. Replaces the credentials the secret scanner finds in the database copies of the named pages with `<REDACTED:rule>` markers and changes nothing else: title, frontmatter values (never keys), the body (inline or segmented), timeline, fact, take and link rows, raw data; chunks holding a value are rebuilt. Database only: no file write, no Git commit, and a sync hold stays. `--source <id> --slug <slug>... [--rules <list>]`, then `--apply --expect <hash>`. See [secret spans](#secret-spans). | `clean` (no match), `refused` (`page_not_found`, `page_kind_unsupported`, `page_too_large`, `proof_failed`, `fact_value_unsupported`, `timeline_rows_collide`); pages that moved after the preview (`preview_changed`, `changed_since_preview`). |
Timeline rows that an earlier version of a page produced and its current text
@@ -411,6 +412,57 @@ earlier Git commits; this repair does not rewrite history.
The repair runs only when named and never on upgrade, `apply-migrations` or the
maintenance cycle.
+<a id="segmented-tiling"></a>
+### Segmented tiling
+
+A page stored in segments is read from the manifest rows of its live
+generation, and those rows must cover the page's recorded body size from
+byte 0 with no gap. Before this release a write could publish a generation
+without its rows: a fact purge's abort, or a sweep's expiry, collected the
+write's staging (and deleted its staged rows) while the write prepared, and
+the write's last preparation step moved the staging back to `prepared`. The
+publish then passed its staging check, closed the page's old rows and
+published none. Every read of such a page refuses `page_body_rows_unreadable`,
+and a managed sync holds its file `page_write_failed` on every retry.
+Current versions refuse both steps with `body_staging_expired` (send the write
+again), but upgrading rewrites nothing, so a page damaged earlier stays
+unreadable until this repair runs.
+
+Doctor `segmented_tiling` lists such pages: source, slug, live generation,
+how many rows it has, the bytes they cover (or that they leave a gap) and the
+body size. It reads the manifest only and writes nothing.
+
+To republish them from Git, preview, then apply after the user agrees:
+
+```bash
+gbrain repair segmented-tiling --source default # preview: each page, its file, HEAD's blob and its sync hold, with a hash
+gbrain repair segmented-tiling --source default --apply --expect <hash>
+```
+
+The preview runs only from the trusted local CLI, on the host that owns the
+source's checkout, and writes nothing. For each page it lists the live
+generation, the bytes its rows cover, the page's file, HEAD's commit, the
+file's blob at HEAD (object id, SHA-256 and size) and the sync hold on it. A
+page with no file, a file HEAD does not hold or a file over `pages.max_bytes`
+is listed `held` and never written. A source without a Git checkout, a
+repo-less brain included, refuses with `not_a_git_repo`: this repair
+republishes the copy Git holds, and there the database has no other copy.
+On such a brain, rewrite the page from a copy you hold (`put_page`).
+
+The apply needs the preview's hash. It checks each page again (revision,
+generation and blob) and refuses `preview_changed` when one moved. Then, per
+page, in one transaction under the page's key lock (a coordinated maintenance
+write on a managed brain), it versions the damaged generation by reference,
+publishes the blob's body as a new generation through the same flip that
+refuses a generation that does not cover its body, keeps the page's title,
+type, frontmatter, tags and trust tier, clears the sync hold the preview saw,
+asks the next sync to screen the file again (so a file that differs from the
+stored page in anything else imports normally) and writes the page's receipt
+(`op_checkpoints` op `segmented-tiling-receipt`) and a `segmented-tiling` audit
+line. A rerun with the same hash skips pages that already have a receipt.
+The repair runs only when named, never from doctor, a remediation plan or an
+upgrade.
+
<a id="request-growth"></a>
### Request-table growth
diff --git a/docs/guides/windowed-reads.md b/docs/guides/windowed-reads.md
index 879a5d5878..a6df18897a 100644
--- a/docs/guides/windowed-reads.md
+++ b/docs/guides/windowed-reads.md
@@ -169,7 +169,10 @@ backup and restore file writes, body digests, fence rewrites and purges, and
timeline extraction. An export with such a page fails instead of writing a
short file. The brain host's operator gives the reading role the same
access to `page_body_manifest` and `page_segment_texts` it has to `pages`;
-`gbrain doctor --json` checks the segment tables.
+`gbrain doctor --json` checks the segment tables. A page whose rows are
+missing (an older write could publish a body without them) is listed by
+doctor `segmented_tiling`; [`gbrain repair segmented-tiling`](repair.md#segmented-tiling)
+republishes it from its file in Git.
## page_projection_stale
diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsv
index 8f0e8933f5..a747a37829 100644
--- a/scripts/module-size-limits.tsv
+++ b/scripts/module-size-limits.tsv
@@ -47,4 +47,4 @@ src/commands/doctor/checks/extraction-sync.ts 1500 ratchet +7 GBRA-130 A-1: link
src/core/search/evidence-delivery.ts 1759 ratchet +5 W5: EvidencePlan.rankCut resolved with the plan (logic in search/page-rank-cut.ts); +20 W5 whole_pages: wholeCost prices a block as delivery would emit it whole (needs the private Block/enrichBlock internals) and the cut is wired before allocate (cutWholePages lives in search/page-rank-cut.ts); new row: crossed the 1500 unlisted cap at the #6367 merge (master 1479, its explicit-budget cap) by the C1 date header threaded through every block path, cap included; pagePlanHits (search/page-plan-hits.ts) and the header loader (evidence-date.ts blockDateHeaders) were peeled first; peel candidate: allocateCapped; +8 at the #6396 merge: page evidence text cut through the fence-chunk trust overlay and the caller minTrust floor (re-summed: our 1504 + master +8); +119 Cat 40 next wave F2: the search.long_page_unit policy (withLongPagePolicy, selectLongPages size preflight with page/byte/deadline limits) and the long-page fetch and fallback in deliverEvidence, kept in the shared seam the plan names (gbrain-evals docs/plans/2026-10-11-cat40-next-wave, F2); +17 F2 rework to the frozen-comparison policy: search.long_page_min_chars (rendered-body characters, default 20,000) read in withLongPagePolicy, a character count in the preflight, the rendered-body confirmation in deliverEvidence, and the long-page row exemption in capDeliveredSnippets (re-summed at the 0.60.225.0 merge: master 1537 + F2 136); +39 Wave B PR 2 lane 2b: planSegmentedBlock joins the windows search/segmented-evidence.ts reads around a segmented page's hits into one run-marked document (needs the private Doc/Piece/buildDoc/locateAnchor internals), planBlock's unit cut split into blockFromDoc for both, sections bounded by their run, selectionCost counts a gap between runs, and the fetch skips the whole-body fence overlay for a segmented page; +4 segmented pages read at once before planning (segmentedReads; the read is search/segmented-evidence.ts, the statements page-body/read-range.ts); +17 GBRA-156: tail trim plan flag, DeliverOptions.query, DeliveryMeta.tail_trim, the trim call and the frozen-hit score/query library fields (logic in search/evidence-tail-trim.ts; interface fields cannot move); +7 GBRA-205: EvidencePlan.titleHeader and DeliveryMeta.title_header (interface fields cannot move) and the titles option on the header loader (logic in search/evidence-date.ts; the query budget key lives in search/evidence-packing.ts); +11 #7600: assemble_evidence quotes conversation rows and capEvidenceToBudget counts the quote fence (logic in search/transcript-quote.ts); +2 GBRA-156 /ship review: the cut-path admission and title room count the quote fence; +2 GBRA-156 /ship review pass 2: capEvidenceToBudget counts the fence only for the paths that fence (quote flag; recall counts the text as returned); +4 GST-687: page evidence and the extractive fallback quote below-owner-tier text for untrusted readers (untrusted flag; logic in search/transcript-quote.ts)
src/commands/config.ts 1524 ratchet new row: crossed the 1500 unlisted cap in wave 15 P7.4 #5461: set-time validation of think.pages_budget_chars / think.excerpt_ceiling_chars (range + parser live in core/think/pages-budget.ts), mirroring the cycle.consolidate.cluster_threshold hook; +1 wave 16 G5.1 (#6495): set-time refusal hook for search.keyword_statement_timeout_ms (check lives in commands/config/keyword-timeout-value.ts); +1 wave 18b lane 5 (#5012 item 6): search.contextual_retrieval prints the existing-pages hint after the write (text lives in commands/config/contextual-mode-hint.ts); +1 #6712 PR 2: retired/rollback sync key dispatch line (logic in core/sync-config-keys.ts handleSyncKeyConfig); +1 #6865: sync.slug_conflicts.auto_remove boolean validation at set time (refusal text in repair/slug-autoheal.ts); +1 #6948: sync.origin_twins.auto_resolve boolean validation at set time (refusal text in repair/origin-twins-heal.ts); +2 #6795: config set chat_model/expansion_model help line and the model-pin import (shadow report and file-pin unset live in commands/config/model-pin-shadow.ts); +1 #6712 PR 4b: sync.admission set-time hook (value logic in commands/config/sync-admission-value.ts); +4 #7130: set-time refusal hook for sync.max_deletes / sync.max_delete_ratio (and .source.<id>) / purge.retention_hours (check lives in core/sync-reconcile.ts deleteSafetyConfigRefusal); +1 GBRA-150: sync.metafile_pages set-time route (body in config/sync-metafile-pages-value.ts); re-summed at the v0.60.330.0 merge (1520 measured); +2 #7242 minions.terminal_hook file-plane key and its set branch (logic in minions/terminal-hook.ts); +1 GBRA-161: generations.* set-time route (refusal in commands/config/generations-values.ts); +1 GBRA-207 #7630: sync.repo_path set-time refusal hook (refusal in core/sync-config-keys.ts syncRepoPathRefusal)
src/commands/mcp-expose.ts 1519 ratchet hosted_option: Session carries the gbrain.io coaching notice on the first-receipt run and Tailscale/service friction exits (copy, refs and gating live in core/onboard/hosted-option.ts); +7 GBRA-199: --surface help lines, surfaceChosen option field and the advertise pass-through (registrationAdvertise in core/mcp-registration.ts)
-src/core/behavior-change-notice.ts 1614 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move); +2 GBRA-287: segmented_freeze_hold_gbra287 and segmented_reexport_gbra287 rows (rows are data and cannot move)
+src/core/behavior-change-notice.ts 1615 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move); +2 GBRA-287: segmented_freeze_hold_gbra287 and segmented_reexport_gbra287 rows (rows are data and cannot move); +1 GST-1209: segmented_tiling_gst1209 row (rows are data and cannot move)
diff --git a/scripts/serial-files.tsv b/scripts/serial-files.tsv
index a03761f778..8725cdb64d 100644
--- a/scripts/serial-files.tsv
+++ b/scripts/serial-files.tsv
@@ -435,6 +435,7 @@ test/write-path-no-egress.serial.test.ts global-state global-state: binds a loca
test/write-path-zero-llm.serial.test.ts R1 R1: mutates process.env (ANTHROPIC_API_KEY, OPENAI_API_KEY) so gated chat and embedding paths are live
test/writeback-nudge.serial.test.ts global-state global-state: patches process stdout/stderr/console
test/writeback-review-fixes.serial.test.ts global-state global-state: process-wide test seams __resetCheckpointHarvestForTests; patches process stdout/stderr/console
+test/large-pages/segmented-publish-tiling.serial.test.ts global-state global-state: installs the process-wide persistence fault hook (installFaultHook) to collect a write staging while the write prepares
test/large-pages/segmented-think-compile-leads.serial.test.ts global-state global-state: process-wide test seams __setEmbedTransportForTests
test/resolve-ipc-rebind.serial.test.ts global-state global-state: binds real Unix sockets and spawns a child serve that is SIGKILLed; withEnv scopes GBRAIN_HOME
test/backup-preflight-6649.serial.test.ts global-state global-state: R3: constructs PGLiteEngine inside the restore-drill and fixture-brain steps (each disconnected in finally); creates symlinked homes and 8 GiB sparse files
diff --git a/src/commands/doctor/checks/graph-health.ts b/src/commands/doctor/checks/graph-health.ts
index b9e9328315..dba14a82a0 100644
--- a/src/commands/doctor/checks/graph-health.ts
+++ b/src/commands/doctor/checks/graph-health.ts
@@ -361,6 +361,7 @@ export const timelineHistoryEntry: DoctorEntry = {
'unbound_source',
'writer_version',
'segmented_storage',
+ 'segmented_tiling',
'self_capture',
'vector_plan',
'keyword_plan',
diff --git a/src/commands/doctor/checks/segmented-tiling.ts b/src/commands/doctor/checks/segmented-tiling.ts
new file mode 100644
index 0000000000..097a48f720
--- /dev/null
+++ b/src/commands/doctor/checks/segmented-tiling.ts
@@ -0,0 +1,31 @@
+import type { BrainEngine } from '../../../core/engine.ts';
+import type { Check } from '../../doctor.ts';
+import { agentFix, checkError } from '../check-fix.ts';
+import { untiledSegmentedPages } from '../../../core/page-body/body-tiling.ts';
+
+const name = 'segmented_tiling';
+const DOCS = 'docs/guides/repair.md#segmented-tiling';
+const SAMPLE = 20;
+
+/**
+ * GST-1209: live pages stored in segments whose live generation's rows do not cover their body (`untiledSegmentedPages`),
+ * so every read of them refuses `page_body_rows_unreadable` and a managed sync holds their file. Read-only: one aggregate
+ * per page over its live manifest rows. The fix is the repair's preview, which writes nothing; republishing a page from
+ * its file at HEAD needs the user's agreement and the preview hash.
+ */
+export async function segmentedTilingCheck(engine: BrainEngine, sourceIds?: readonly string[]): Promise<Check> {
+ try {
+ const found = await untiledSegmentedPages(engine, { sourceIds, limit: SAMPLE + 1 });
+ if (!found.length) return { name, status: 'ok', message: 'Every live page stored in segments has segment rows that cover its whole body.' };
+ const pages = found.slice(0, SAMPLE).map(({ source_id, slug, body_generation, body_bytes, rows, covered_bytes }) => ({ source_id, slug, body_generation, body_bytes, rows, covered_bytes }));
+ const sources = [...new Set(pages.map(page => page.source_id))].sort();
+ const listed = pages.map(page => `${page.source_id}:${page.slug} (generation ${page.body_generation}: ${page.rows} row(s) covering ${page.covered_bytes ?? 'with a gap'}${page.covered_bytes === null ? '' : ' bytes'} of ${page.body_bytes})`);
+ return { name, status: 'fail', details: { pages, truncated: found.length > SAMPLE, sources, docs: DOCS },
+ message: `${found.length > SAMPLE ? `More than ${SAMPLE}` : found.length} live page(s) stored in segments have segment rows that do not cover their body, so every read of them refuses `
+ + `(page_body_rows_unreadable) and a sync holds their file: ${listed.join('; ')}. Preview the repair; it republishes each page from its file at HEAD only when applied with the user's agreement.`,
+ fix: agentFix(['gbrain', 'repair', 'segmented-tiling', ...(sources.length === 1 ? ['--source', sources[0]!] : [])],
+ 'Previews republishing each listed page from its file at HEAD and writes nothing; applying it needs the user\'s agreement and the printed preview hash.', name, { docs: DOCS }) };
+ } catch (error) {
+ return checkError(name, 'read the segment rows of pages stored in segments', error);
+ }
+}
diff --git a/src/commands/doctor/wave-checks.ts b/src/commands/doctor/wave-checks.ts
index da1c192da4..1b07f07e3a 100644
--- a/src/commands/doctor/wave-checks.ts
+++ b/src/commands/doctor/wave-checks.ts
@@ -258,6 +258,14 @@ export const WAVE_CHECKS: readonly WaveCheckSpec[] = [
count: d => (Array.isArray(d.blockers) ? d.blockers.length : 0),
run: async engine => (await import('./checks/segmented-storage.ts')).segmentedStorageCheck(engine),
},
+ {
+ id: 'segmented_tiling', resolution: 'operator', registration: 'wave',
+ hostOnly: 'It lists page slugs, and the repair it names runs only from the trusted local CLI on the brain host.',
+ impact: 'Some pages stored in segments have segment rows that do not cover their body, so every read of them refuses and sync holds their files',
+ instruction: 'Preview `gbrain repair segmented-tiling --source <id>` (read-only), show the user the listing, and apply it with the printed --expect hash only after they agree; it republishes each page from its file at HEAD and never runs automatically.',
+ count: d => (Array.isArray(d.pages) ? d.pages.length : 0),
+ run: async (engine, scope) => (await import('./checks/segmented-tiling.ts')).segmentedTilingCheck(engine, scope.sourceIds),
+ },
{
id: 'self_capture', resolution: 'operator', registration: 'wave',
count: d => Number(d.classified ?? 0),
diff --git a/src/commands/repair.ts b/src/commands/repair.ts
index 6b73bafd6a..2b02367bc3 100644
--- a/src/commands/repair.ts
+++ b/src/commands/repair.ts
@@ -98,6 +98,7 @@ export const REPAIR_HELP = `Usage: gbrain repair [<kind>] [--apply] [--source <i
gbrain repair fact-claim-dates --source <id> [--day YYYY-MM-DD] [--apply --expect <preview-hash> --yes] [--json]
gbrain repair provenance [--source <id>] [--slug <slug>]... [--include-ambiguous] [--apply --expect <preview-hash> --yes] [--json]
gbrain repair secret-spans --source <id> --slug <slug>... [--rules <rule,...>] [--no-embed] [--apply --expect <preview-hash>] [--json]
+ gbrain repair segmented-tiling --source <id> [--slug <slug>]... [--apply --expect <preview-hash>] [--json]
${REPAIR_ORDER_HELP}
gbrain repair rename-identity --source <id> [--slug <new-slug> [--from <old-slug>] [--close]
[--apply --expect <preview-hash> --yes]] [--json]
@@ -206,7 +207,7 @@ const VALUE_FLAGS = new Set(['--rules', '--source', '--limit', '--expect', '--on
const KIND_FLAGS: Record<string, readonly string[]> = {
'--only': ['frontmatter', 'fences', 'slug-conflicts', 'file-drift', CONTENT_LANE], '--skip': ['frontmatter', 'fences', 'slug-conflicts', 'file-drift', CONTENT_LANE], '--diff': ['frontmatter', 'fences', 'slug-conflicts', CONTENT_LANE], '--yes': ['frontmatter', 'conversation-labels', 'stuck-recovery', 'ghost-pages', 'legacy-twins', 'rename-identity', 'url-clone-fences'],
'--include-cross-entity': ['duplicate-facts'],
- '--slug': ['fences', 'file-drift', 'rename-identity', 'overwritten-edits', 'origin-twins', 'timeline-order', 'provenance', 'secret-spans'], '--from': ['rename-identity'], '--close': ['rename-identity'], '--prefer': ['file-drift'], '--request': ['file-drift'], '--all-drifted': ['file-drift'], '--class': ['file-drift', 'visibility'], '--since': ['file-drift', 'generations'],
+ '--slug': ['fences', 'file-drift', 'rename-identity', 'overwritten-edits', 'origin-twins', 'timeline-order', 'provenance', 'secret-spans', 'segmented-tiling'], '--from': ['rename-identity'], '--close': ['rename-identity'], '--prefer': ['file-drift'], '--request': ['file-drift'], '--all-drifted': ['file-drift'], '--class': ['file-drift', 'visibility'], '--since': ['file-drift', 'generations'],
'--held': ['file-drift', 'generations'], '--side': ['file-drift'], '--state': ['file-drift'], '--cursor': ['file-drift'], '--provenance': ['file-drift'],
'--keep': ['stuck-recovery'], '--request-id': ['stuck-recovery'],
'--older-than': ['receipt-compaction'], '--include-failures': ['receipt-compaction'], '--force': ['receipt-compaction'],
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index de691cf5fe..391951699e 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1416,6 +1416,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: '0.60.556.0', id: 'engine_keyed_caches_gbra283', text: 'A process serving several brains (mounts) now keeps each brain\'s search telemetry in that brain\'s own database and reads each brain\'s own search.track_retrieval, admission quotas, mcp.strict_params fallback, publication ceiling, capacity-notice throttle and the two vector-search rollback settings. Before, the first brain to be read decided these for every brain in the process, and every brain\'s search telemetry was written into the first brain\'s database. A process with one brain behaves as before.' },
{ since: '0.60.560.0', id: 'segmented_freeze_hold_gbra287', text: 'A managed sync whose freeze compares a Git-tracked page stored in segments with working-tree bytes that differ from the pinned commit now reads a page up to 24 MiB back whole, as its publication does, instead of refusing with `page_body_segmented` and stopping the source (`sync_blocked`) on every later sync. A page whose stored body cannot be read while it is frozen (`page_body_segmented` or `page_body_rows_unreadable`) is held `page_write_failed` with that code named, and the rest of the source syncs; `gbrain sync status` classifies such a hold by that code (`page_body_segmented`: upgrade, a person; `page_body_rows_unreadable`: retry, no person needed) instead of as an unplaced `page_write_failed`. A sync that stops before admitting any write request shows `last_error.request_id: null` in `gbrain sync status --json` instead of omitting it.' },
{ since: '0.60.560.0', id: 'segmented_reexport_gbra287', text: 'A full sync re-exports a DB-only page (#2426) stored in segments, streaming its file with the bytes `gbrain export` writes, instead of listing it as a `page_body_segmented` failure; a page whose re-export fails is returned in the result\'s `reexport_failures` and the run completes. Write-through of a page stored in segments keeps the existing file\'s mode, skips a file over the pre-push scan cap before binding `source_path`, and with frontmatter overrides returns an error result instead of a `page_body_segmented` one; the derived (generated-page) writer skips such a page (`segmented_page`). A segmented read whose page published a new body between its reads reads one generation (or the new live one) instead of refusing `page_body_rows_unreadable`.' },
+ { since: 'TBD', id: 'segmented_tiling_gst1209', text: 'A write to a page stored in segments whose staging a fact purge aborted or a sweep expired while the write prepared now refuses `body_staging_expired` (send the write again) and the page keeps its previous body, instead of publishing a generation with no segment rows, after which every read refused `page_body_rows_unreadable` and a managed sync held the file on every retry. A publish also refuses any generation whose rows do not cover the body it records. New doctor check `segmented_tiling` lists pages an older version left like this, and the explicit-only repair `gbrain repair segmented-tiling --source <id>` republishes each from its file at HEAD after a preview, with `--apply --expect <hash>`; a source without a Git checkout refuses it (`not_a_git_repo`).' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --git a/src/core/doctor-categories.ts b/src/core/doctor-categories.ts
index 0513efa87e..e56e6d3c0a 100644
--- a/src/core/doctor-categories.ts
+++ b/src/core/doctor-categories.ts
@@ -71,6 +71,8 @@ export const BRAIN_CHECK_NAMES: ReadonlySet<string> = new Set([
'connector_checkpoints',
'connector_held_items',
'git_held_files',
+ // GST-1209: live pages stored in segments whose live rows do not cover their body (doctor/checks/segmented-tiling.ts).
+ 'segmented_tiling',
'fence_integrity',
// GBRA-221: live fence rows on imported pages whose body is outside-authored text (doctor/checks/outside-content-fences.ts).
'outside_content_fences',
diff --git a/src/core/engine-sql/page-body.ts b/src/core/engine-sql/page-body.ts
index 95af6b37a5..bf0a4b9764 100644
--- a/src/core/engine-sql/page-body.ts
+++ b/src/core/engine-sql/page-body.ts
@@ -34,7 +34,7 @@ import { jsonbParam, type SqlExecutor } from './executor.ts';
import type { LegacyUnscopedRead } from './brands.ts';
import { joinFragments, sqlFragment, trustedSql, type SqlFragment } from './fragment.ts';
import { compileRowNormalizer } from './normalize.ts';
-import { stagingNeedsTextSql } from '../page-body/schema.ts';
+import { liveTilingSql, stagingNeedsTextSql, untiledSql } from '../page-body/schema.ts';
import { DerivationWriteRefused, type SegmentDerivationInput, type SegmentDerivationKind, type SegmentDerivationRow, type SegmentDerivationState } from '../page-body/derivations.ts';
import { RELEASE_STAGED_CTE } from '../persistence/limits.ts';
@@ -342,6 +342,13 @@ export async function flipBodyGeneration(exec: SqlExecutor, input: BodyFlipInput
source_uri = COALESCE(${m.sourceUri}::text, source_uri), ingested_via = COALESCE(${m.ingestedVia}::text, ingested_via),
ingested_at = COALESCE(${m.ingestedAt}::timestamptz, ingested_at)
WHERE id = ${input.pageId}`);
+ // GST-1209: the generation this flip made live must tile the body it recorded, as every reader checks it; rows the
+ // write staged that went missing would otherwise commit a body nothing can read.
+ const { rows: untiled } = await tx.run<{ n: unknown; covered: unknown; gapless: boolean }>(sqlFragment`SELECT t.n, t.covered, t.gapless
+ FROM pages p CROSS JOIN LATERAL (${trustedSql(liveTilingSql('p'))}) t WHERE p.id = ${input.pageId} AND ${trustedSql(untiledSql('p', 't'))}`);
+ if (untiled[0]) {
+ throw bodyGenerationIncompleteError(page.slug, input.stagingId, { rows: Number(untiled[0].n), covered: Number(untiled[0].covered), gapless: untiled[0].gapless }, b);
+ }
return { retiredChunks: retired, retireDeferred };
});
}
@@ -355,6 +362,16 @@ export function bodyStagingGoneError(slug: string, stagingId: string, state: str
{ detail: JSON.stringify({ staging_id: stagingId, state }) });
}
+/** GST-1209: the rows a flip would publish do not tile the body it records (rows it staged went missing), so the flip refuses. */
+export function bodyGenerationIncompleteError(slug: string, stagingId: string, tiled: { rows: number; covered: number; gapless: boolean },
+ body: { body_bytes: number; segment_count: number }) {
+ const covered = tiled.gapless ? `${tiled.covered} bytes` : 'with a gap';
+ return opError('body_staging_expired', `The staged body of this write to ${slug} does not cover its ${body.body_bytes}-byte body (${tiled.rows} of ${body.segment_count} segment rows, ${covered}), so nothing was published.`,
+ 'Segment rows this write staged are missing, so publishing it would leave a body no reader can read. Send the write again: it stages a fresh copy.',
+ { detail: JSON.stringify({ staging_id: stagingId, reason: 'generation_incomplete', rows: tiled.rows, segment_count: body.segment_count,
+ covered_bytes: tiled.gapless ? tiled.covered : null, body_bytes: body.body_bytes }) });
+}
+
export async function openBodyStaging(exec: SqlExecutor, input: BodyStagingInput): Promise<void> {
await exec.run(sqlFragment`INSERT INTO page_body_stagings (id, kind, principal_kind, principal_id, client_request_id, source_id,
source_incarnation, slug, page_id, mode, base_revision, base_generation, declared_bytes, declared_sha256, reserved_bytes,
@@ -365,13 +382,21 @@ export async function openBodyStaging(exec: SqlExecutor, input: BodyStagingInput
${input.idleExpiresAt}, ${input.hardExpiresAt})`);
}
-/** Moving a staging to a live state restarts its idle deadline (never past its hard one). */
+/**
+ * Moves a live staging to `state`; a move to a live state restarts its idle deadline (never past its hard one). A staging
+ * that is no longer live stays as it is (GST-1209): a fact purge's abort or a sweep's expiry deleted its staged rows, so a
+ * write moving it on (to prepared or committed) refuses body_staging_expired instead of reviving it, and an abort or expiry
+ * of it changes nothing.
+ */
export async function setBodyStagingState(exec: SqlExecutor, id: string, state: BodyStagingState, opts: { commitRequestId?: string | null } = {}): Promise<void> {
- await exec.run(sqlFragment`UPDATE page_body_stagings SET state = ${state}, updated_at = now(),
+ const { rows } = await exec.run<{ id: string }>(sqlFragment`UPDATE page_body_stagings SET state = ${state}, updated_at = now(),
idle_expires_at = CASE WHEN ${state} = ANY(${LIVE_STAGING_STATES}::text[])
THEN LEAST(hard_expires_at, now() + (${BODY_STAGING_IDLE_MS}::bigint * interval '1 millisecond')) ELSE idle_expires_at END,
commit_request_id = COALESCE(${opts.commitRequestId ?? null}::uuid, commit_request_id)
- WHERE id = ${id}::uuid`);
+ WHERE id = ${id}::uuid AND state = ANY(${LIVE_STAGING_STATES}::text[]) RETURNING id`);
+ if (rows.length || state === 'aborted' || state === 'expired') return;
+ const { rows: gone } = await exec.run<{ slug: string; state: string }>(sqlFragment`SELECT slug, state FROM page_body_stagings WHERE id = ${id}::uuid`);
+ throw bodyStagingGoneError(gone[0]?.slug ?? id, id, gone[0]?.state ?? null);
}
/**
diff --git a/src/core/error-registry.ts b/src/core/error-registry.ts
index 3bfd94f6ea..d77cc3703b 100644
--- a/src/core/error-registry.ts
+++ b/src/core/error-registry.ts
@@ -535,7 +535,7 @@ export const CODES = {
page_identity_changed: { class: 'caller', summary: "The page was deleted or replaced while the operation ran.", fix: { argv: ['gbrain', 'get', '--source', '{source_id}', '--', '{slug}'], mcp: { tool: 'get_page', arguments: { slug: '{slug}', source_id: '{source_id}' } }, consent: [], actor: 'agent', why: "Shows which page holds the slug now and its revision; a new attempt needs a new request_id.", requires_exclusive: false } },
page_projection_conflict: { class: 'retryable', summary: "Another worker installed a newer search projection of the page while this command prepared its own.", why: "The projection guard keeps the newer installation instead of overwriting it; the canonical page revision did not change.", docs: 'docs/guides/repair.md#page-projection-conflict', suggestion: "Re-run the same command; it re-reads the current projection. If it conflicts again, wait for the other worker to finish." },
page_not_found: { class: 'caller', summary: "No page with that slug exists in the selected source.", legacy_error: 'page_identity_changed', fix: { argv: ['gbrain', 'get', '--source', '{source_id}', '--', '{slug}'], mcp: { tool: 'get_page', arguments: { slug: '{slug}', source_id: '{source_id}' } }, consent: [], actor: 'agent', why: "Confirms whether the page exists in this source before anything is recreated.", requires_exclusive: false } },
- page_body_rows_unreadable: { class: 'server', summary: "This large page's segment rows readable on this connection do not cover its body, so the read returned nothing rather than a short or empty body.", why: "A segmented page keeps its body in manifest rows that must tile the stored body from its first byte to body_bytes; a role that sees the page row but not its manifest rows (a row-level security policy or grant that covers pages only), or rows missing from the database, would otherwise read as an empty or truncated page with nothing to say so.", docs: 'docs/guides/windowed-reads.md#page_body_rows_unreadable', suggestion: "Give the role that reads the brain the same access to page_body_manifest and page_segment_texts it has to pages (the brain host's operator), then read again; run gbrain doctor --json to check the segment tables." },
+ page_body_rows_unreadable: { class: 'server', summary: "This large page's segment rows readable on this connection do not cover its body, so the read returned nothing rather than a short or empty body.", why: "A segmented page keeps its body in manifest rows that must tile the stored body from its first byte to body_bytes; a role that sees the page row but not its manifest rows (a row-level security policy or grant that covers pages only), or rows missing from the database, would otherwise read as an empty or truncated page with nothing to say so.", docs: 'docs/guides/windowed-reads.md#page_body_rows_unreadable', suggestion: "Give the role that reads the brain the same access to page_body_manifest and page_segment_texts it has to pages (the brain host's operator), then read again; run gbrain doctor --json to check the segment tables. When doctor segmented_tiling lists the page, its rows are missing: preview gbrain repair segmented-tiling --source <id>, which republishes it from Git." },
page_projection_stale: { class: 'server', summary: "A segment of this large page has projection metadata gbrain cannot use, so the read returned nothing for it rather than unsanitized text.", why: "Windows of a segmented page are cut from each segment's stored protected regions; when they are stale (another segmenter version, a text that does not match its hash) gbrain recomputes them from the segment's carried parser state, and refuses when that state is missing or of another version, or when a facts fence block crosses a forced cut while the source has withdrawals.", docs: 'docs/guides/windowed-reads.md#page_projection_stale', suggestion: 'Rewrite the page body (import or sync its file again), which segments it with this gbrain version, then read it again; other windows and the outline still work.' },
page_body_segmented: { class: 'caller', summary: "This page's body is stored in segments, and the path that hit it reads or writes the whole body as one string, so it stopped before reading or changing anything.", why: "With pages.segmented_storage on, a page over 1 MiB keeps its body in segments instead of the page row; a whole-body path would otherwise see an empty body and could overwrite the page or drop the unread rest. Windowed reads and segment-aware writers replace these paths.", docs: 'docs/guides/windowed-reads.md#page_body_segmented', suggestion: "Retrying refuses the same way. Read the page in pieces: get_page body: 'auto' returns its outline and first window, and body: 'window' reads any lines or bytes.", fix: { argv: ['gbrain', 'get', '--source', '{source_id}', '--body', 'auto', '--', '{slug}'], mcp: { tool: 'get_page', arguments: { slug: '{slug}', source_id: '{source_id}', body: 'auto' } }, consent: [], actor: 'agent', why: "Returns the page's outline and its first window, with next_cursor to read on.", requires_exclusive: false } },
page_shrink_refused: { class: 'caller', summary: "put_page refused to replace a large page with content much smaller than the stored body; nothing was written.", why: "Above the inline read size (mcp.inline_body_bytes, 256 KB) a get_page read returns a window, not the body, so a replacement far smaller than the page is most often a window sent back as the whole page, which would delete the unread rest.", docs: 'docs/guides/windowed-reads.md#page_shrink_refused', suggestion: "Change part of the page with edit_page (revision from get_page body: 'none'). Pass allow_shrink: true only when the smaller content is the intended complete page.", fix: { argv: ['gbrain', 'get', '--source', '{source_id}', '--', '{slug}'], mcp: { tool: 'get_page', arguments: { slug: '{slug}', source_id: '{source_id}', body: 'none' } }, consent: [], actor: 'agent', why: "Returns the page's revision and size without its body; edit_page with that revision changes only the text it names.", requires_exclusive: false } },
diff --git a/src/core/page-body/body-tiling.ts b/src/core/page-body/body-tiling.ts
index d2160cdf03..a4939fe21e 100644
--- a/src/core/page-body/body-tiling.ts
+++ b/src/core/page-body/body-tiling.ts
@@ -8,6 +8,7 @@
import type { BrainEngine } from '../engine.ts';
import { opError } from '../ops/contract.ts';
import type { ManifestRow } from './types.ts';
+import { liveTilingSql, untiledSql } from './schema.ts';
export interface RowSpan { byteStart: number; byteLen: number }
@@ -20,7 +21,7 @@ export function assertRowsTile(rows: readonly RowSpan[], at: { slug: string; sou
}
if (end >= 0 && (at.total === null ? rows.length > 0 : end === at.total)) return;
throw opError('page_body_rows_unreadable', `Page ${at.slug} (source ${at.sourceId}): the segment rows readable here do not cover its ${at.total ?? 'stored'}-byte body, so nothing was read.`,
- 'Give the role that reads the brain the same access to page_body_manifest and page_segment_texts it has to pages, then read again.',
+ 'Give the role that reads the brain the same access to page_body_manifest and page_segment_texts it has to pages, then read again. When gbrain doctor --only segmented_tiling lists the page, its rows are missing: preview gbrain repair segmented-tiling --source <id>.',
{ detail: JSON.stringify({ rows: rows.length, covered_bytes: end >= 0 ? end : null, body_bytes: at.total }) });
}
@@ -77,3 +78,30 @@ export async function assertGenerationTiles(engine: Pick<BrainEngine, 'executeRa
const at = generation ?? Number(page.body_generation);
assertRowsTile(rows, { slug: page.slug, sourceId: page.source_id, total: bodyBytes ?? await generationBodyBytes(engine, { ...page, id: pageId }, at) });
}
+
+/** GST-1209: a live page stored in segments whose live generation's rows do not tile its body (`untiledSegmentedPages`). */
+export interface UntiledPage {
+ source_id: string; slug: string; page_id: number; revision: string; body_generation: number; body_bytes: number;
+ segment_count: number | null; source_path: string | null;
+ /** The live generation's rows, and the bytes they cover gapless from 0 (null: they leave a gap). */
+ rows: number; covered_bytes: number | null;
+}
+
+/**
+ * GST-1209: the live (not deleted) pages stored in segments whose live generation's rows do not cover body_bytes gapless
+ * from 0, so every read of them refuses page_body_rows_unreadable. One aggregate per page over its live rows, read-only;
+ * `sourceIds` and `slugs` narrow the scan, `limit` bounds the list (ordered by source and slug).
+ */
+export async function untiledSegmentedPages(engine: Pick<BrainEngine, 'executeRaw'>, opts: { sourceIds?: readonly string[]; slugs?: readonly string[]; limit?: number } = {}):
+ Promise<UntiledPage[]> {
+ const rows = await engine.executeRaw<{ id: number; source_id: string; slug: string; revision: string; body_generation: unknown; body_bytes: unknown;
+ segment_count: number | null; source_path: string | null; n: unknown; covered: unknown; gapless: boolean }>(
+ `SELECT p.id, p.source_id, p.slug, p.knowledge_revision::text AS revision, p.body_generation, p.body_bytes, p.segment_count, p.source_path, t.n, t.covered, t.gapless
+ FROM pages p CROSS JOIN LATERAL (${liveTilingSql('p')}) t
+ WHERE p.body_storage = 'segmented' AND p.deleted_at IS NULL AND ($1::text[] IS NULL OR p.source_id = ANY($1::text[]))
+ AND ($2::text[] IS NULL OR p.slug = ANY($2::text[])) AND ${untiledSql('p', 't')}
+ ORDER BY p.source_id, p.slug LIMIT $3`, [opts.sourceIds ? [...opts.sourceIds] : null, opts.slugs ? [...opts.slugs] : null, opts.limit ?? 1000]);
+ return rows.map(row => ({ source_id: row.source_id, slug: row.slug, page_id: Number(row.id), revision: row.revision, body_generation: Number(row.body_generation),
+ body_bytes: Number(row.body_bytes), segment_count: row.segment_count === null ? null : Number(row.segment_count), source_path: row.source_path,
+ rows: Number(row.n), covered_bytes: row.gapless ? Number(row.covered) : null }));
+}
diff --git a/src/core/page-body/schema.ts b/src/core/page-body/schema.ts
index 87bc974968..efd5a01455 100644
--- a/src/core/page-body/schema.ts
+++ b/src/core/page-body/schema.ts
@@ -71,6 +71,23 @@ export function stagingNeedsTextSql(t: string): string {
WHERE r.source_id = ${t}.source_id AND r.sha256 = ${t}.sha256 AND ${liveStagingSql('rs')})`;
}
+/**
+ * GST-1209: the tiling of page `p`'s live generation (an alias of pages), as a LATERAL subquery: its manifest rows as every
+ * reader selects them, their count (`n`), the bytes they cover (`covered`) and whether each starts where the previous one
+ * ends, the first at byte 0 (`gapless`). Constant SQL over the alias, no values.
+ */
+export function liveTilingSql(p: string): string {
+ return `SELECT count(*)::bigint AS n, COALESCE(sum(r.byte_len), 0)::bigint AS covered, COALESCE(bool_and(r.byte_start = r.prior), true) AS gapless
+ FROM (SELECT m.byte_start, m.byte_len, COALESCE(sum(m.byte_len) OVER (ORDER BY m.seq ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING), 0) AS prior
+ FROM page_body_manifest m WHERE m.page_id = ${p}.id AND m.staging_id IS NULL AND m.gen_from <= ${p}.body_generation
+ AND (m.gen_to IS NULL OR m.gen_to > ${p}.body_generation)) r`;
+}
+
+/** Page `p`'s live generation, tiled as `t` (liveTilingSql), does not cover its body_bytes gapless from 0: no reader can read it. */
+export function untiledSql(p: string, t: string): string {
+ return `(NOT ${t}.gapless OR ${t}.covered <> ${p}.body_bytes)`;
+}
+
/** The staging `s` (an alias of page_body_stagings) is live in stagingNeedsTextSql's sense: a write that may still publish it. */
export function liveStagingSql(s: string): string {
return `(${s}.state IN ('open','sealed','prepared')
diff --git a/src/core/persistence/preview-approval.ts b/src/core/persistence/preview-approval.ts
index 6ba5bba4c1..18c980470f 100644
--- a/src/core/persistence/preview-approval.ts
+++ b/src/core/persistence/preview-approval.ts
@@ -40,7 +40,7 @@ export const PREVIEW_APPROVAL_OP = 'repair-approval';
/** The default `purgeStaleCheckpoints` TTL; an older approved set may already be gone. */
export const APPROVAL_MAX_AGE_DAYS = 7;
-export type PreviewApprovalCommand = 'authorize-legacy' | 'jobs-cancel' | 'stale-atoms' | 'atoms-restore' | 'extractor-facts' | 'conversation-labels' | 'captured-facts' | 'agent-session-facts' | 'redacted-facts' | 'loop-facts' | 'ontology-facts' | 'facts-rehome' | 'consolidate-ties' | 'duplicate-facts' | 'failed-writes' | 'split-facts' | 'frontmatter' | 'fences' | 'slug-conflicts' | 'managed-sync-orphans' | 'frontmatter-keys' | 'file-drift' | 'stuck-recovery' | 'ghost-pages' | 'origin-twins' | 'receipt-compaction' | 'accepted-takes' | 'rename-identity' | 'overwritten-edits' | 'legacy-twins' | 'missing-tables' | 'segmented-mirror' | 'uncommitted-deletions' | 'visibility' | 'generations' | 'fact-claim-dates' | 'timeline-order' | 'provenance' | 'secret-spans' | 'rows-behind';
+export type PreviewApprovalCommand = 'authorize-legacy' | 'jobs-cancel' | 'stale-atoms' | 'atoms-restore' | 'extractor-facts' | 'conversation-labels' | 'captured-facts' | 'agent-session-facts' | 'redacted-facts' | 'loop-facts' | 'ontology-facts' | 'facts-rehome' | 'consolidate-ties' | 'duplicate-facts' | 'failed-writes' | 'split-facts' | 'frontmatter' | 'fences' | 'slug-conflicts' | 'managed-sync-orphans' | 'frontmatter-keys' | 'file-drift' | 'stuck-recovery' | 'ghost-pages' | 'origin-twins' | 'receipt-compaction' | 'accepted-takes' | 'rename-identity' | 'overwritten-edits' | 'legacy-twins' | 'missing-tables' | 'segmented-mirror' | 'segmented-tiling' | 'uncommitted-deletions' | 'visibility' | 'generations' | 'fact-claim-dates' | 'timeline-order' | 'provenance' | 'secret-spans' | 'rows-behind';
export interface ApprovedSetKey {
command: PreviewApprovalCommand;
diff --git a/src/core/repair/core.ts b/src/core/repair/core.ts
index 240de4a52d..c083a45d1f 100644
--- a/src/core/repair/core.ts
+++ b/src/core/repair/core.ts
@@ -51,7 +51,7 @@ import { classifyItemFailure, ItemRun, type ItemFailure } from '../item-failure.
import type { ProgressReporter } from '../progress.ts';
import type { GenerationsSelection } from '../generations/repair.ts';
-export const REPAIR_KINDS = ['timeline', 'visibility', 'safe-chunks', 'contextual-mode', 'connector-checkpoints', 'request-indexes', 'connector-fences', 'take-supersession', 'accepted-takes', 'orphan-bindings', 'embedding-effects', 'attribution-backfill', 'planner-stats', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'extractor-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'fences', 'slug-conflicts', 'timeline-comments', 'timeline-order', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'uncommitted-deletions', 'generations', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind', 'access'] as const;
+export const REPAIR_KINDS = ['timeline', 'visibility', 'safe-chunks', 'contextual-mode', 'connector-checkpoints', 'request-indexes', 'connector-fences', 'take-supersession', 'accepted-takes', 'orphan-bindings', 'embedding-effects', 'attribution-backfill', 'planner-stats', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'extractor-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'fences', 'slug-conflicts', 'timeline-comments', 'timeline-order', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'segmented-tiling', 'uncommitted-deletions', 'generations', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind', 'access'] as const;
export type RepairKind = typeof REPAIR_KINDS[number];
export interface RepairScope { brain_id: string; source_ids: string[] }
diff --git a/src/core/repair/registry.ts b/src/core/repair/registry.ts
index 6032892f91..e764676289 100644
--- a/src/core/repair/registry.ts
+++ b/src/core/repair/registry.ts
@@ -84,6 +84,7 @@ import { overwrittenEditsRepair } from './overwritten-edits.ts';
import { missingTablesRepair } from './missing-tables.ts';
import { urlCloneFencesRepair } from './url-clone-fences.ts';
import { segmentedMirrorRepair } from './segmented-mirror.ts';
+import { segmentedTilingRepair } from './segmented-tiling.ts';
import { uncommittedDeletionsRepair, UNCOMMITTED_DELETIONS_CONSENT_TEXT } from './uncommitted-deletions.ts';
import { factClaimDatesRepair } from './fact-claim-dates.ts';
import { provenanceRepair } from './provenance.ts';
@@ -555,6 +556,17 @@ const SPECS: Record<RepairKind, Omit<RepairKindSpec, 'kind'>> = {
+ '(file_database_drift) and never rewritten. Preview-bound and destructive: --apply --expect <hash> --yes re-checks each page and commits, per '
+ 'page, one receipted request whose withdrawal-mirror and Git effects rewrite and commit the file. Runs only when named; never on upgrade.',
},
+ 'segmented-tiling': {
+ handler: segmentedTilingRepair, embeds: 'none', checks: ['segmented_tiling'], explicit_only: true, preview_bound: true,
+ summary: 'Republish, from the file Git holds at HEAD, each live page stored in segments whose live generation\'s rows do not cover its body (GST-1209; '
+ + 'doctor segmented_tiling): such a page refuses every read (page_body_rows_unreadable) and a managed sync holds its file on every retry. Needs '
+ + '--source; --slug <slug> narrows. The preview, on the host that owns the checkout, lists each page with its generation, the bytes its rows '
+ + 'cover, its file, HEAD\'s commit, the blob\'s object id, SHA-256 and size and its sync hold; a page with no file or a file HEAD does not hold is '
+ + 'listed held. A source without a Git checkout (a repo-less brain) refuses (not_a_git_repo). --apply --expect <hash> re-checks each page, then '
+ + 'per page, in one transaction under its key lock (a coordinated maintenance write on a managed brain): versions the damaged generation by '
+ + 'reference, publishes the blob\'s body as a new generation (title, type, frontmatter, tags and trust tier kept), clears the previewed sync hold, '
+ + 'asks the next sync to re-screen the file and writes the page\'s receipt and audit line. Trusted local CLI only; runs only when named; never on upgrade.',
+ },
'uncommitted-deletions': {
handler: uncommittedDeletionsRepair, embeds: 'none', checks: ['uncommitted_sync_deletions'], explicit_only: true, preview_bound: true, consent: 'destructive',
consentText: UNCOMMITTED_DELETIONS_CONSENT_TEXT,
diff --git a/src/core/repair/segmented-tiling.ts b/src/core/repair/segmented-tiling.ts
new file mode 100644
index 0000000000..049597f7a2
--- /dev/null
+++ b/src/core/repair/segmented-tiling.ts
@@ -0,0 +1,256 @@
+/**
+ * `gbrain repair segmented-tiling --source <id> [--slug <slug>]...` (GST-1209): republish, from the file Git holds at HEAD,
+ * each live page stored in segments whose live generation's rows do not cover its body (doctor `segmented_tiling`). Such
+ * a page refuses every read (`page_body_rows_unreadable`) and a managed sync holds its file on every retry: a write once
+ * committed a generation without its rows (its staging, collected by a fact purge's abort or a sweep's expiry while the
+ * write prepared, was moved back to `prepared`; that move and such a flip now refuse).
+ *
+ * Preview (read-only; trusted local CLI, on the host that owns the source's checkout): each damaged page with its live
+ * generation, the bytes its rows cover, its file, HEAD's commit, the file's blob at HEAD (object id, SHA-256 and size) and
+ * the sync hold on it. The hash binds the pages, their revisions and generations, the blobs and the holds. A page with no
+ * file, a file HEAD does not hold or one over pages.max_bytes is listed held and never written. A source without a Git
+ * checkout (a repo-less brain) refuses: this repair republishes the copy Git holds, and the database has no other.
+ *
+ * `--apply --expect <hash>` re-checks every page (`preview_changed`), then per page, in one transaction under the page's
+ * key lock (a coordinated maintenance write on a managed brain; refused while a sync or write in flight names the file):
+ * versions the damaged generation by reference, stages the blob's body and publishes it as a new generation through the
+ * generation flip (which refuses one that does not tile), keeps the page's title, type, frontmatter, tags and trust tier,
+ * clears the sync hold the preview saw and asks the next sync to re-screen the file (so a file that differs from the
+ * stored page in anything else imports normally), and writes the page's receipt, also appended to the `segmented-tiling`
+ * audit log. Explicit-only: never from doctor, a remediation plan or an upgrade.
+ */
+import { execFileSync } from 'node:child_process';
+import { createHash } from 'node:crypto';
+import { join } from 'node:path';
+import type { BrainEngine } from '../engine.ts';
+import { OperationError, opError } from '../ops/contract.ts';
+import { trustedCliRequired } from '../ops/op-fix.ts';
+import { parseMarkdown } from '../markdown.ts';
+import { maintenanceAttribution } from '../persistence/attribution.ts';
+import { withCoordinatedWrite, withWriteAttribution } from '../persistence/context.ts';
+import { readManagedMode } from '../persistence/managed-mode.ts';
+import { getWorktreeBinding } from '../persistence/ownership.ts';
+import { checkOwner, ownerUnavailableError } from '../persistence/owner-refusal.ts';
+import { localHostId } from '../persistence/identity.ts';
+import { loadRepairBusySet, repairBusy, repairBusyError } from '../persistence/repair-busy.ts';
+import { clearGitHold, readGitHold, requestGitHoldRetry } from '../persistence/sync-holds.ts';
+import { clearApprovedSet, loadApprovedSet, previewChangedError, previewHash, saveApprovedSet } from '../persistence/preview-approval.ts';
+import { applySegmentedBody, sealSegmentedPage, segmentedLivePage } from '../persistence/segmented-apply.ts';
+import { withPageTierKept } from '../trust/fence-append.ts';
+import { prepareSegmentedBody, stringBodyFlags, stringRegion } from '../page-body/ingest.ts';
+import { readPageBodyPolicy } from '../page-body/limits.ts';
+import { buildSegmentedProjection } from '../page-body/projection.ts';
+import { untiledSegmentedPages, type UntiledPage } from '../page-body/body-tiling.ts';
+import { isQuarantined } from '../quarantine.ts';
+import { createAuditWriter } from '../audit/audit-writer.ts';
+import { afterCursor, type RepairHandler, type RepairItem, type RepairItemOutcome, type RepairListing, type RepairPlan, type RepairScope } from './core.ts';
+
+const DOCS = 'docs/guides/repair.md#segmented-tiling';
+export const SEGMENTED_TILING_RECEIPT_OP = 'segmented-tiling-receipt';
+
+/** A damaged page the apply republishes: its state at preview time and the blob it republishes. */
+export interface HealPage extends UntiledPage {
+ source_path: string; incarnation: string; head: string; blob_oid: string; blob_sha256: string; blob_bytes: number; hold_observed_at: string | null;
+}
+type Selection = { source_id: string; slugs: string[] };
+type HealItem = RepairItem & { page: HealPage; hash: string; last: boolean };
+
+const previewCommand = (selection: Selection) => `gbrain repair segmented-tiling --source ${selection.source_id}${selection.slugs.map(slug => ` --slug ${slug}`).join('')}`;
+const receiptKey = (hash: string, pageId: number) => `${hash}:${pageId}`;
+const audit = createAuditWriter<{ ts: string } & Record<string, unknown>>({ featureName: 'segmented-tiling', errorLabel: 'segmented-tiling-audit', errorTrailer: '; repair continues' });
+const sha256 = (bytes: Uint8Array) => createHash('sha256').update(bytes).digest('hex');
+const git = (root: string, args: string[], maxBuffer = 1024 * 1024) => execFileSync('git', ['-C', root, ...args], { maxBuffer, stdio: ['ignore', 'pipe', 'pipe'] });
+const gitText = (root: string, args: string[]): string | null => { try { return git(root, args).toString('utf8').trim(); } catch { return null; } };
+
+/**
+ * The checkout this host reads the source's files from: the owner's worktree on a managed source (refused off the owner
+ * host), else the source's local path. Refuses a source with no Git checkout: a repo-less brain has no copy to republish.
+ */
+async function sourceCheckout(engine: BrainEngine, sourceId: string): Promise<{ root: string; incarnation: string; managed: boolean }> {
+ const [source] = await engine.executeRaw<{ local_path: string | null; incarnation: string }>('SELECT local_path, incarnation::text AS incarnation FROM sources WHERE id=$1', [sourceId]);
+ if (!source) throw new OperationError('invalid_params', `Source ${sourceId} does not exist.`, 'List the sources with gbrain sources list, then preview again with one of them.', DOCS);
+ const managed = await readManagedMode(engine);
+ let root = source.local_path;
+ if (managed) {
+ const binding = await getWorktreeBinding(engine, sourceId);
+ if (binding) {
+ const owner = checkOwner(binding, source.incarnation, localHostId());
+ if (owner.reason) throw ownerUnavailableError({ sourceId, reason: owner.reason, binding, incarnation: source.incarnation, hostId: localHostId(), remote: false, work: 'the segmented-tiling repair' });
+ root = join(owner.binding.local_path, owner.binding.relative_path);
+ }
+ }
+ if (!root || gitText(root, ['rev-parse', '--verify', '--quiet', 'HEAD^{commit}']) === null) {
+ throw opError('not_a_git_repo', `Source ${sourceId} has no Git checkout with a commit, so the pages it lists cannot be republished from Git.`,
+ 'This repair republishes the copy of each page Git holds at HEAD; a repo-less source has no copy outside the database, so nothing was read or written. Rewrite the page from a copy you hold (put_page), or restore an earlier version of it (gbrain history).',
+ { docs: DOCS });
+ }
+ return { root, incarnation: source.incarnation, managed };
+}
+
+/** HEAD's commit and the blob it holds at the page's file, or why there is none. */
+function headBlob(root: string, path: string, maxBytes: number): { head: string; oid: string; bytes: number } | { held: string } {
+ const head = gitText(root, ['rev-parse', '--verify', 'HEAD^{commit}'])!;
+ const oid = gitText(root, ['rev-parse', '--verify', '--quiet', `HEAD:./${path}`]);
+ if (!oid || gitText(root, ['cat-file', '-t', oid]) !== 'blob') return { held: `HEAD (${head.slice(0, 12)}) holds no file at ${path}` };
+ const bytes = Number(gitText(root, ['cat-file', '-s', oid]));
+ if (bytes > maxBytes) return { held: `the file at HEAD is ${bytes} bytes, over pages.max_bytes (${maxBytes})` };
+ return { head, oid, bytes };
+}
+
+const readBlob = (root: string, oid: string, bytes: number) => git(root, ['cat-file', 'blob', oid], bytes + 1024 * 1024);
+
+const hashOf = (scope: RepairScope, selection: Selection, pages: readonly HealPage[]) => previewHash({ kind: 'segmented-tiling-v1', brain_id: scope.brain_id, selection,
+ pages: pages.map(page => ({ page_id: page.page_id, slug: page.slug, revision: page.revision, generation: page.body_generation, head: page.head, blob: page.blob_oid,
+ blob_sha256: page.blob_sha256, hold: page.hold_observed_at })) });
+
+function selectionOf(scope: RepairScope, slugs: readonly string[] | undefined): Selection {
+ if (scope.source_ids.length !== 1) {
+ throw new OperationError('invalid_params', 'gbrain repair segmented-tiling works on one source at a time; pass --source <id>.',
+ 'Run doctor (gbrain doctor --only segmented_tiling) to see which sources hold damaged pages, then preview each with --source.', DOCS);
+ }
+ return { source_id: scope.source_ids[0]!, slugs: [...new Set(slugs ?? [])].sort() };
+}
+
+const describe = (page: HealPage) => `generation ${page.body_generation}: ${page.rows} row(s) cover ${page.covered_bytes ?? 'with a gap'}${page.covered_bytes === null ? '' : ' bytes'} of ${page.body_bytes}; `
+ + `republish ${page.source_path} from HEAD ${page.head.slice(0, 12)} (blob ${page.blob_oid.slice(0, 12)}, ${page.blob_bytes} bytes)${page.hold_observed_at ? '; clears its sync hold' : ''}`;
+
+/** Every damaged page of the selection, as the preview lists it (republishable pages and held ones). */
+async function survey(engine: BrainEngine, selection: Selection): Promise<{ pages: HealPage[]; listing: RepairListing[]; held: number }> {
+ const checkout = await sourceCheckout(engine, selection.source_id);
+ const { maxBytes } = await readPageBodyPolicy(engine);
+ const pages: HealPage[] = [];
+ const listing: RepairListing[] = [];
+ let held = 0;
+ for (const page of await untiledSegmentedPages(engine, { sourceIds: [selection.source_id], ...(selection.slugs.length ? { slugs: selection.slugs } : {}) })) {
+ const label = `${page.source_id}:${page.slug} (revision ${page.revision})`;
+ const blob = page.source_path ? headBlob(checkout.root, page.source_path, maxBytes) : { held: 'the page has no file, so Git holds no copy of it' };
+ if ('held' in blob) { held++; listing.push({ item: label, class: 'held', detail: blob.held }); continue; }
+ const bytes = readBlob(checkout.root, blob.oid, blob.bytes);
+ const hold = await readGitHold(engine, page.source_id, checkout.incarnation, page.source_path!);
+ const heal: HealPage = { ...page, source_path: page.source_path!, incarnation: checkout.incarnation, head: blob.head, blob_oid: blob.oid, blob_sha256: sha256(bytes),
+ blob_bytes: bytes.length, hold_observed_at: hold?.observed_at ?? null };
+ pages.push(heal);
+ listing.push({ item: label, class: 'republish', detail: describe(heal) });
+ }
+ return { pages: pages.sort((a, b) => a.page_id - b.page_id), listing, held };
+}
+
+/** Republishes one page's body from `content` in the caller's transaction; false when the page changed since the preview. */
+async function republish(tx: BrainEngine, page: HealPage, content: string): Promise<{ generation: number; unsealed: boolean; hold_cleared: boolean } | false> {
+ await tx.lockPageKeys([{ sourceId: page.source_id, slug: page.slug }]);
+ const [now] = await untiledSegmentedPages(tx, { sourceIds: [page.source_id], slugs: [page.slug] });
+ if (!now || now.page_id !== page.page_id || now.revision !== page.revision || now.body_generation !== page.body_generation) return false;
+ const [row] = await tx.executeRaw<{ title: string; type: string; page_kind: string; frontmatter: Record<string, unknown> | null; contextual_retrieval_mode: string | null;
+ corpus_generation: string | null; tags: string[] }>(`SELECT p.title, p.type, p.page_kind, p.frontmatter, p.contextual_retrieval_mode, p.corpus_generation,
+ COALESCE((SELECT jsonb_agg(t.tag ORDER BY t.tag) FROM tags t WHERE t.page_id=p.id), '[]'::jsonb) AS tags FROM pages p WHERE p.id=$1`, [page.page_id]);
+ const frontmatter = row!.frontmatter ?? {};
+ const parsed = parseMarkdown(content, `${page.slug}.md`);
+ const truth = parsed.compiled_truth, timeline = parsed.timeline ?? '';
+ const { principal } = await maintenanceAttribution(tx);
+ await tx.createVersion(page.slug, { sourceId: page.source_id });
+ const prepared = await prepareSegmentedBody(tx, { sourceId: page.source_id, slug: page.slug, live: await segmentedLivePage(tx, page.page_id),
+ title: row!.title, type: row!.type, frontmatter, tags: row!.tags, truth: stringRegion(truth), timeline: stringRegion(timeline), hasTimeline: timeline.length > 0,
+ flags: stringBodyFlags(truth, timeline), declaredBytes: Buffer.byteLength(truth, 'utf8') + Buffer.byteLength(timeline, 'utf8'),
+ owner: { kind: 'import', principalKind: principal.kind, principalId: principal.id, baseRevision: page.revision },
+ chunking: { quarantined: isQuarantined(frontmatter), ...(row!.contextual_retrieval_mode === 'title'
+ ? { tier: 'title' as const, corpusGeneration: row!.corpus_generation } : { tier: 'none' as const, corpusGeneration: null }) } });
+ const { flipRetireMaxRows } = await readPageBodyPolicy(tx);
+ const applied = await withPageTierKept(tx, { sourceId: page.source_id, slug: page.slug }, () => applySegmentedBody(tx, { slug: page.slug, sourceId: page.source_id,
+ pageId: page.page_id, prepared, promotion: false, retireMaxRows: flipRetireMaxRows,
+ meta: { title: row!.title, type: row!.type as never, frontmatter: frontmatter as never, page_kind: row!.page_kind as never } }));
+ if (applied.complete) await sealSegmentedPage(tx, page.page_id);
+ const hold_cleared = page.hold_observed_at !== null
+ && await clearGitHold(tx, { sourceId: page.source_id, incarnation: page.incarnation, path: page.source_path, observedAt: page.hold_observed_at });
+ await requestGitHoldRetry(tx, page.source_id, page.incarnation, [page.source_path]);
+ return { generation: prepared.newGeneration, unsealed: !applied.complete, hold_cleared };
+}
+
+async function receiptsOf(engine: BrainEngine, hash: string, pages: readonly HealPage[]): Promise<Set<number>> {
+ const rows = await engine.executeRaw<{ fingerprint: string }>('SELECT fingerprint FROM op_checkpoints WHERE op=$1 AND fingerprint=ANY($2::text[])',
+ [SEGMENTED_TILING_RECEIPT_OP, pages.map(page => receiptKey(hash, page.page_id))]);
+ return new Set(rows.map(row => Number(row.fingerprint.slice(hash.length + 1))));
+}
+
+function item(page: HealPage, hash: string, last: boolean): HealItem {
+ return { cursor: { phase: 0, id: page.page_id }, source_id: page.source_id, slug: page.slug, chars: 0, action: describe(page), page, hash, last };
+}
+
+export const segmentedTilingRepair: RepairHandler = {
+ kind: 'segmented-tiling',
+ publication: 'projection',
+ async plan(engine, scope, after, opts): Promise<RepairPlan> {
+ if (opts?.remote !== false) throw trustedCliRequired('gbrain repair segmented-tiling reads the source\'s Git checkout and republishes stored pages, so it runs only from the trusted local CLI on the brain host.');
+ const selection = selectionOf(scope, opts.slugs);
+ const command = previewCommand(selection);
+ if (!opts.apply) {
+ const { pages, listing, held } = await survey(engine, selection);
+ const hash = hashOf(scope, selection, pages);
+ if (pages.length) await saveApprovedSet<HealPage & { selection: Selection }>(engine, { command: 'segmented-tiling', hash }, pages.map(page => ({ ...page, selection })));
+ return { items: pages.map((page, index) => item(page, hash, index === pages.length - 1)), preview_hash: hash, listing,
+ residuals: { pages_to_republish: pages.length, held }, details: { pages } };
+ }
+ if (!opts.expect) {
+ throw new OperationError('invalid_params', 'gbrain repair segmented-tiling --apply republishes only the set a preview printed.',
+ `Preview first: ${command} — show the user the listing, then run the apply command it prints: ${command} --apply --expect <preview-hash>`, DOCS);
+ }
+ const expect = opts.expect;
+ const approved = (await loadApprovedSet<HealPage & { selection: Selection }>(engine, { command: 'segmented-tiling', hash: expect, previewCommand: command,
+ emptyHash: async () => hashOf(scope, selection, []) })).items;
+ if (approved.some(page => page.selection.source_id !== selection.source_id || page.selection.slugs.join('\n') !== selection.slugs.join('\n'))) throw previewChangedError(expect, command);
+ const pages = approved.map(({ selection: _selection, ...page }) => page);
+ const done = await receiptsOf(engine, expect, pages);
+ const pending = pages.filter(page => !done.has(page.page_id));
+ if (pending.length) {
+ const checkout = await sourceCheckout(engine, selection.source_id);
+ const { maxBytes } = await readPageBodyPolicy(engine);
+ const now = new Map((await untiledSegmentedPages(engine, { sourceIds: [selection.source_id], slugs: pending.map(page => page.slug) })).map(page => [page.page_id, page]));
+ for (const page of pending) {
+ const live = now.get(page.page_id);
+ const blob = headBlob(checkout.root, page.source_path, maxBytes);
+ if (!live || live.revision !== page.revision || live.body_generation !== page.body_generation || 'held' in blob || blob.oid !== page.blob_oid) throw previewChangedError(expect, command);
+ }
+ }
+ return { items: pages.map((page, index) => item(page, expect, index === pages.length - 1)).filter(entry => afterCursor(entry.cursor, after)),
+ preview_hash: expect, residuals: {} };
+ },
+ async apply(ctx, entry): Promise<RepairItemOutcome> {
+ if (ctx.remote !== false) throw trustedCliRequired('gbrain repair segmented-tiling runs only from the trusted local CLI on the brain host.');
+ const { page, hash, last } = entry as HealItem;
+ try {
+ if ((await receiptsOf(ctx.engine, hash, [page])).has(page.page_id)) return { applied: true, outcome: 'republished', detail: { generation_before: page.body_generation } };
+ const checkout = await sourceCheckout(ctx.engine, page.source_id);
+ const bytes = readBlob(checkout.root, page.blob_oid, page.blob_bytes);
+ if (sha256(bytes) !== page.blob_sha256) return { applied: false, outcome: 'changed_since_preview', reason: 'the blob no longer has the previewed bytes' };
+ if (checkout.managed) {
+ const busy = await loadRepairBusySet(ctx.engine, page.source_id, page.incarnation);
+ if (repairBusy(busy, { sourcePath: page.source_path, slug: page.slug })) throw repairBusyError(busy, page.source_id, page.source_path);
+ }
+ const written = await ctx.engine.transaction(async tx => {
+ const attribution = await maintenanceAttribution(tx);
+ const write = async () => {
+ const result = await republish(tx, page, bytes.toString('utf8'));
+ if (!result) return null;
+ const [after] = await tx.executeRaw<{ revision: string }>('SELECT knowledge_revision::text AS revision FROM pages WHERE id=$1', [page.page_id]);
+ const receipt = { kind: 'segmented-tiling', hash, source_id: page.source_id, slug: page.slug, page_id: page.page_id, revision_before: page.revision,
+ revision_after: after?.revision ?? null, generation_before: page.body_generation, generation_after: result.generation, path: page.source_path, head: page.head,
+ blob: page.blob_oid, blob_sha256: page.blob_sha256, hold_cleared: result.hold_cleared };
+ await tx.executeRaw(`INSERT INTO op_checkpoints(op,fingerprint,completed_keys,updated_at) VALUES ($1,$2,$3::text::jsonb,now())
+ ON CONFLICT (op,fingerprint) DO UPDATE SET completed_keys=EXCLUDED.completed_keys,updated_at=now()`, [SEGMENTED_TILING_RECEIPT_OP, receiptKey(hash, page.page_id), JSON.stringify([receipt])]);
+ return { receipt, unsealed: result.unsealed };
+ };
+ return checkout.managed ? withCoordinatedWrite(tx, [page.source_id], write, attribution) : withWriteAttribution(tx, attribution, write);
+ });
+ if (!written) return { applied: false, outcome: 'changed_since_preview', reason: 'the page changed since the preview' };
+ audit.log(written.receipt);
+ let projection = 'sealed';
+ if (written.unsealed) await buildSegmentedProjection(ctx.engine, { sourceId: page.source_id, slug: page.slug }).catch(() => { projection = 'queued'; });
+ return { applied: true, outcome: 'republished', detail: { generation_before: page.body_generation, generation_after: written.receipt.generation_after,
+ hold_cleared: written.receipt.hold_cleared, projection } };
+ } finally {
+ if (last) await clearApprovedSet(ctx.engine, { command: 'segmented-tiling', hash });
+ }
+ },
+ render(details) {
+ return ((details.pages ?? []) as HealPage[]).map(page => ` ${page.source_id}:${page.slug}: ${describe(page)}`);
+ },
+};
diff --git a/test/e2e/segmented-publish-tiling-postgres.test.ts b/test/e2e/segmented-publish-tiling-postgres.test.ts
new file mode 100644
index 0000000000..d1271bea3e
--- /dev/null
+++ b/test/e2e/segmented-publish-tiling-postgres.test.ts
@@ -0,0 +1,15 @@
+/**
+ * GST-1209 Postgres arm of test/helpers/segmented-publish-tiling-cases.ts: the race runs on separate pool connections,
+ * so the sweep and the preparation commit independently as they do on a live brain.
+ */
+import { afterAll, beforeAll, describe } from 'bun:test';
+import type { PostgresEngine } from '../../src/core/postgres-engine.ts';
+import { defineSegmentedPublishTilingCases } from '../helpers/segmented-publish-tiling-cases.ts';
+import { hasDatabase, setupDB, teardownDB } from './helpers.ts';
+
+(hasDatabase() ? describe : describe.skip)('segmented publish tiling [postgres]', () => {
+ let engine: PostgresEngine;
+ beforeAll(async () => { engine = await setupDB(); }, 120_000);
+ afterAll(async () => { await teardownDB(); });
+ defineSegmentedPublishTilingCases({ getEngine: () => engine });
+});
diff --git a/test/fix-wave-5-integration.test.ts b/test/fix-wave-5-integration.test.ts
index 3ee52db0a3..7fa1524f0e 100644
--- a/test/fix-wave-5-integration.test.ts
+++ b/test/fix-wave-5-integration.test.ts
@@ -198,6 +198,7 @@ for (const backend of testBackends()) {
frontmatter: 'gbrain repair frontmatter', 'timeline-comments': 'gbrain repair timeline-comments', 'timeline-order': 'gbrain repair timeline-order', 'managed-sync-orphans': 'gbrain repair managed-sync-orphans', 'content-receipt-root': 'gbrain repair content-receipt-root', 'frontmatter-keys': 'gbrain repair frontmatter-keys', 'file-drift': 'gbrain repair file-drift',
'stuck-recovery': 'gbrain repair stuck-recovery', 'ghost-pages': 'gbrain repair ghost-pages', 'origin-twins': 'gbrain repair origin-twins', 'receipt-compaction': 'gbrain repair receipt-compaction', aliases: 'gbrain repair aliases', 'rename-identity': 'gbrain repair rename-identity',
'overwritten-edits': 'gbrain repair overwritten-edits', 'legacy-twins': 'gbrain repair legacy-twins', 'missing-tables': 'gbrain repair missing-tables', 'url-clone-fences': 'gbrain repair url-clone-fences', 'segmented-mirror': 'gbrain repair segmented-mirror',
+ 'segmented-tiling': 'gbrain repair segmented-tiling',
'uncommitted-deletions': 'gbrain repair uncommitted-deletions',
// GBRA-161: the generations preview carries its ledger kind and the restore window, so the printed command runs as is.
generations: 'gbrain repair generations --kind atom --since 30d', 'fact-claim-dates': 'gbrain repair fact-claim-dates',
diff --git a/test/fixtures/goldens/doctor/content-pglite-human.json b/test/fixtures/goldens/doctor/content-pglite-human.json
index 927d33a088..cf6662c67a 100644
--- a/test/fixtures/goldens/doctor/content-pglite-human.json
+++ b/test/fixtures/goldens/doctor/content-pglite-human.json
@@ -128,6 +128,7 @@
" [OK] unbound_source: No page was written database-only to an unbound source.",
" [OK] writer_version: Every writer that admitted or published a request in the last 7 days is v0.60.5.0 or newer.",
" [OK] segmented_storage: Segmented page storage is on: pages over 1 MiB are stored in segments, up to 134217728 bytes on this PGLite brain.",
+ " [OK] segmented_tiling: Every live page stored in segments has segment rows that cover its whole body.",
" [OK] self_capture: No session corpus is configured; nothing to classify.",
" [OK] vector_plan: Skipped: PGLite runs vector search in-process; the plan check applies to Postgres brains.",
" [OK] keyword_plan: Skipped: PGLite runs keyword search with no statement timeout; the canary applies to Postgres brains.",
diff --git a/test/fixtures/goldens/doctor/content-pglite-json.json b/test/fixtures/goldens/doctor/content-pglite-json.json
index 52914748c1..3dc28a60df 100644
--- a/test/fixtures/goldens/doctor/content-pglite-json.json
+++ b/test/fixtures/goldens/doctor/content-pglite-json.json
@@ -1708,6 +1708,13 @@
"name": "segmented_storage",
"status": "ok"
},
+ {
+ "category": "brain",
+ "elapsed_ms": "<volatile>",
+ "message": "Every live page stored in segments has segment rows that cover its whole body.",
+ "name": "segmented_tiling",
+ "status": "ok"
+ },
{
"category": "brain",
"details": {
diff --git a/test/fixtures/goldens/doctor/hot-checks-pglite-json.json b/test/fixtures/goldens/doctor/hot-checks-pglite-json.json
index 12201d2942..423577ce69 100644
--- a/test/fixtures/goldens/doctor/hot-checks-pglite-json.json
+++ b/test/fixtures/goldens/doctor/hot-checks-pglite-json.json
@@ -1887,6 +1887,13 @@
"name": "segmented_storage",
"status": "ok"
},
+ {
+ "category": "brain",
+ "elapsed_ms": "<volatile>",
+ "message": "Every live page stored in segments has segment rows that cover its whole body.",
+ "name": "segmented_tiling",
+ "status": "ok"
+ },
{
"category": "brain",
"details": {
@@ -6851,6 +6858,13 @@
"name": "segmented_storage",
"status": "ok"
},
+ {
+ "category": "brain",
+ "elapsed_ms": "<volatile>",
+ "message": "Every live page stored in segments has segment rows that cover its whole body.",
+ "name": "segmented_tiling",
+ "status": "ok"
+ },
{
"category": "brain",
"details": {
diff --git a/test/fixtures/goldens/doctor/hot-checks-postgres-json.json b/test/fixtures/goldens/doctor/hot-checks-postgres-json.json
index 65e4b7e83a..3303758190 100644
--- a/test/fixtures/goldens/doctor/hot-checks-postgres-json.json
+++ b/test/fixtures/goldens/doctor/hot-checks-postgres-json.json
@@ -1879,6 +1879,13 @@
"name": "segmented_storage",
"status": "ok"
},
+ {
+ "category": "brain",
+ "elapsed_ms": "<volatile>",
+ "message": "Every live page stored in segments has segment rows that cover its whole body.",
+ "name": "segmented_tiling",
+ "status": "ok"
+ },
{
"category": "brain",
"details": {
@@ -6844,6 +6851,13 @@
"name": "segmented_storage",
"status": "ok"
},
+ {
+ "category": "brain",
+ "elapsed_ms": "<volatile>",
+ "message": "Every live page stored in segments has segment rows that cover its whole body.",
+ "name": "segmented_tiling",
+ "status": "ok"
+ },
{
"category": "brain",
"details": {
diff --git a/test/fixtures/goldens/doctor/json-pglite-degraded.json b/test/fixtures/goldens/doctor/json-pglite-degraded.json
index 3885dd16cd..20c729377f 100644
--- a/test/fixtures/goldens/doctor/json-pglite-degraded.json
+++ b/test/fixtures/goldens/doctor/json-pglite-degraded.json
@@ -1585,6 +1585,13 @@
"name": "segmented_storage",
"status": "ok"
},
+ {
+ "category": "brain",
+ "elapsed_ms": "<volatile>",
+ "message": "Every live page stored in segments has segment rows that cover its whole body.",
+ "name": "segmented_tiling",
+ "status": "ok"
+ },
{
"category": "brain",
"details": {
diff --git a/test/fixtures/goldens/doctor/json-pglite-fresh.json b/test/fixtures/goldens/doctor/json-pglite-fresh.json
index a4c0ccefa0..ee62910248 100644
--- a/test/fixtures/goldens/doctor/json-pglite-fresh.json
+++ b/test/fixtures/goldens/doctor/json-pglite-fresh.json
@@ -1641,6 +1641,13 @@
"name": "segmented_storage",
"status": "ok"
},
+ {
+ "category": "brain",
+ "elapsed_ms": "<volatile>",
+ "message": "Every live page stored in segments has segment rows that cover its whole body.",
+ "name": "segmented_tiling",
+ "status": "ok"
+ },
{
"category": "brain",
"details": {
diff --git a/test/fixtures/goldens/doctor/json-postgres-fresh.json b/test/fixtures/goldens/doctor/json-postgres-fresh.json
index 143798ccae..cac0b5a293 100644
--- a/test/fixtures/goldens/doctor/json-postgres-fresh.json
+++ b/test/fixtures/goldens/doctor/json-postgres-fresh.json
@@ -1633,6 +1633,13 @@
"name": "segmented_storage",
"status": "ok"
},
+ {
+ "category": "brain",
+ "elapsed_ms": "<volatile>",
+ "message": "Every live page stored in segments has segment rows that cover its whole body.",
+ "name": "segmented_tiling",
+ "status": "ok"
+ },
{
"category": "brain",
"details": {
diff --git a/test/fixtures/goldens/doctor/registry-build-checks.json b/test/fixtures/goldens/doctor/registry-build-checks.json
index da3e4c0c44..96ccb84c1c 100644
--- a/test/fixtures/goldens/doctor/registry-build-checks.json
+++ b/test/fixtures/goldens/doctor/registry-build-checks.json
@@ -557,6 +557,10 @@
"category": "ops",
"name": "segmented_storage"
},
+ {
+ "category": "brain",
+ "name": "segmented_tiling"
+ },
{
"category": "brain",
"name": "self_capture"
diff --git a/test/helpers/segmented-publish-tiling-cases.ts b/test/helpers/segmented-publish-tiling-cases.ts
new file mode 100644
index 0000000000..34389aabc5
--- /dev/null
+++ b/test/helpers/segmented-publish-tiling-cases.ts
@@ -0,0 +1,95 @@
+/**
+ * GST-1209: a segmented publish commits only a generation whose rows tile its body. Shared by the PGLite arm
+ * (test/large-pages/segmented-publish-tiling.serial.test.ts) and the Postgres arm (test/e2e/segmented-publish-tiling-postgres.test.ts).
+ *
+ * 1. Protects: a staging that a fact purge aborted, or a sweep expired, while its write was still preparing (the sweep then
+ * deletes the staging's staged rows) stays terminal: the preparation's move to `prepared` refuses (body_staging_expired)
+ * instead of reviving it, so the flip never closes the live rows and publishes none. Independently, the flip refuses a
+ * publication whose new generation does not tile its body (here: staged rows deleted under a live staging). Either way
+ * the page stays at its previous generation, whole and readable.
+ * 2. Fails when: the write commits a generation whose rows cover less than its body_bytes (the damaged shape: live rows
+ * closed, new rows missing), or a refusal leaves the page at a different generation.
+ * 3. Serial: installs the process-wide fault hook.
+ */
+import { afterEach, beforeEach, expect, test } from 'bun:test';
+import type { BrainEngine } from '../../src/core/engine.ts';
+import { importFromContent } from '../../src/core/import-file.ts';
+import { installFaultHook } from '../../src/core/persistence/fault-points.ts';
+import { BODY_STAGING_IDLE_MS } from '../../src/core/page-body/limits.ts';
+import { materializeSegmentedBody } from '../../src/core/page-body/ingest.ts';
+import { readTiledManifest } from '../../src/core/page-body/body-tiling.ts';
+import { conversationBody, conversationPage } from './conversation-page.ts';
+
+const SLUG = 'notes/tiling-example';
+const original = conversationPage(2 * 1024 * 1024, { seed: 31 });
+const rewritten = `${original.trimEnd()}\n\n${conversationBody(32 * 1024, { seed: 32 })}\n`;
+
+type Live = { id: number; body_generation: number; body_bytes: number; content_hash: string };
+async function livePage(engine: BrainEngine): Promise<Live> {
+ const [row] = await engine.executeRaw<Live>('SELECT id, body_generation, body_bytes, content_hash FROM pages WHERE slug=$1', [SLUG]);
+ return { ...row!, id: Number(row!.id), body_generation: Number(row!.body_generation), body_bytes: Number(row!.body_bytes) };
+}
+
+/** Bytes the live generation's rows cover, gapless from 0. */
+async function coveredBytes(engine: BrainEngine, page: Live): Promise<number> {
+ const rows = await engine.executeRaw<{ byte_start: string; byte_len: number }>(`SELECT byte_start, byte_len FROM page_body_manifest
+ WHERE page_id=$1 AND staging_id IS NULL AND gen_from<=$2 AND (gen_to IS NULL OR gen_to>$2) ORDER BY seq`, [page.id, page.body_generation]);
+ let end = 0;
+ for (const row of rows) { if (Number(row.byte_start) !== end) return -1; end += row.byte_len; }
+ return end;
+}
+
+export function defineSegmentedPublishTilingCases(opts: { getEngine: () => BrainEngine }): void {
+ beforeEach(async () => {
+ const engine = opts.getEngine();
+ await engine.executeRaw('DELETE FROM pages WHERE slug=$1', [SLUG]);
+ await engine.setConfig('pages.segmented_storage', 'on');
+ });
+ afterEach(() => { installFaultHook(undefined); });
+
+ /** Each race, the refusal it earns, and the state the write's staging ends in (a terminal one stays as the race left it). */
+ const races: Array<[string, string, string, (engine: BrainEngine, pageId: number) => Promise<void>]> = [
+ ['a fact purge aborts the staging and sweeps it', 'is aborted, so nothing was published', 'aborted', async (engine, pageId) => {
+ await engine.executeRaw("UPDATE page_body_stagings SET state='aborted', updated_at=now() WHERE page_id=$1 AND state IN ('open','sealed','prepared')", [pageId]);
+ await engine.sweepSegments({ pageId });
+ }],
+ ['a sweep expires the staging past its idle deadline', 'is expired, so nothing was published', 'expired', async (engine, pageId) => {
+ await engine.sweepSegments({ pageId, expireStagings: true, now: new Date(Date.now() + BODY_STAGING_IDLE_MS + 60_000) });
+ }],
+ ['staged rows go missing under a live staging (the flip guard)', '-byte body (', 'aborted', async (engine, pageId) => {
+ await engine.executeRaw('DELETE FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NOT NULL', [pageId]);
+ }],
+ ];
+
+ for (const [name, refusal, staging, race] of races) {
+ test(`${name} while the write prepares: the write refuses and the page keeps its previous generation`, async () => {
+ const engine = opts.getEngine();
+ await importFromContent(engine, SLUG, original, { sourceId: 'default', noEmbed: true });
+ const before = await livePage(engine);
+ expect(await coveredBytes(engine, before)).toBe(before.body_bytes);
+ const bodyBefore = await materializeSegmentedBody(engine, before.id, 'default');
+
+ installFaultHook(async reached => {
+ if (reached !== 'segmented:chunks_staged') return;
+ installFaultHook(undefined);
+ await race(engine, before.id);
+ });
+ const error = await importFromContent(engine, SLUG, rewritten, { sourceId: 'default', noEmbed: true }).then(() => null, (e: unknown) => e as { code?: string; message?: string });
+ installFaultHook(undefined);
+
+ const after = await livePage(engine);
+ expect({ code: error?.code, generation: after.body_generation, covered: await coveredBytes(engine, after) })
+ .toEqual({ code: 'body_staging_expired', generation: before.body_generation, covered: before.body_bytes });
+ expect(error?.message).toContain(refusal);
+ expect(after.content_hash).toBe(before.content_hash);
+ expect(await engine.executeRaw("SELECT state FROM page_body_stagings WHERE page_id=$1 AND state <> 'committed'", [after.id])).toEqual([{ state: staging }]);
+ expect((await readTiledManifest(engine, after.id)).generation).toBe(before.body_generation);
+ expect(await materializeSegmentedBody(engine, after.id, 'default')).toEqual(bodyBefore);
+
+ await importFromContent(engine, SLUG, rewritten, { sourceId: 'default', noEmbed: true });
+ const retried = await livePage(engine);
+ expect(retried.body_generation).toBe(before.body_generation + 1);
+ expect(await coveredBytes(engine, retried)).toBe(retried.body_bytes);
+ }, 300_000);
+ }
+}
diff --git a/test/large-pages/segmented-publish-tiling.serial.test.ts b/test/large-pages/segmented-publish-tiling.serial.test.ts
new file mode 100644
index 0000000000..798a22977e
--- /dev/null
+++ b/test/large-pages/segmented-publish-tiling.serial.test.ts
@@ -0,0 +1,13 @@
+/**
+ * GST-1209 PGLite arm of test/helpers/segmented-publish-tiling-cases.ts. Serial: installs the process-wide fault hook.
+ */
+import { afterAll, beforeAll, describe } from 'bun:test';
+import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
+import { defineSegmentedPublishTilingCases } from '../helpers/segmented-publish-tiling-cases.ts';
+
+describe('segmented publish tiling [pglite]', () => {
+ let engine: PGLiteEngine;
+ beforeAll(async () => { engine = new PGLiteEngine(); await engine.connect({}); await engine.initSchema(); }, 120_000);
+ afterAll(async () => { await engine.disconnect(); });
+ defineSegmentedPublishTilingCases({ getEngine: () => engine });
+});
diff --git a/test/large-pages/segmented-tiling-repair.test.ts b/test/large-pages/segmented-tiling-repair.test.ts
new file mode 100644
index 0000000000..5970c4ccd5
--- /dev/null
+++ b/test/large-pages/segmented-tiling-repair.test.ts
@@ -0,0 +1,146 @@
+/**
+ * GST-1209: doctor `segmented_tiling` and `gbrain repair segmented-tiling`.
+ *
+ * 1. Protects: doctor lists a live segmented page whose live rows do not cover its body and writes nothing. The repair's
+ * preview names the page, its file and HEAD's blob under a hash; `--apply` refuses without that hash; the apply
+ * republishes the page from the blob at HEAD as a new generation that tiles, keeps the damaged generation as a
+ * version by reference, writes a receipt, clears the sync hold, and the next managed sync completes with no hold.
+ * A source with no Git checkout refuses the preview (not_a_git_repo); a remote caller is refused.
+ * 2. Fails when: doctor misses the page or writes, an apply runs without the previewed hash or against a changed page,
+ * the republished generation does not tile, the hold survives, or a repo-less source reads as republishable.
+ * 3. PGLite; the managed fixture follows test/large-pages/managed-sync-segmented-freeze.test.ts.
+ */
+import { afterAll, beforeAll, expect, test } from 'bun:test';
+import { randomUUID } from 'node:crypto';
+import { execFileSync } from 'node:child_process';
+import { appendFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
+import { claimWorktree } from '../../src/core/persistence/ownership.ts';
+import { performManagedSync } from '../../src/core/persistence/sync-run.ts';
+import { disposePersistenceConsumer } from '../../src/core/persistence/service.ts';
+import { readGitHold } from '../../src/core/persistence/sync-holds.ts';
+import { importFromContent } from '../../src/core/import-file.ts';
+import { materializeSegmentedBody } from '../../src/core/page-body/ingest.ts';
+import { untiledSegmentedPages } from '../../src/core/page-body/body-tiling.ts';
+import { resolveRepairScope, runRepair } from '../../src/core/repair/core.ts';
+import { AUTO_REPAIR_REGISTRY, repairRunner } from '../../src/core/repair/registry.ts';
+import { segmentedTilingRepair, SEGMENTED_TILING_RECEIPT_OP } from '../../src/core/repair/segmented-tiling.ts';
+import { segmentedTilingCheck } from '../../src/commands/doctor/checks/segmented-tiling.ts';
+import { withEnv } from '../helpers/with-env.ts';
+import { withManagedFixtureWrite } from '../helpers/managed-e2e-fixture-write.ts';
+import { conversationBody } from '../helpers/conversation-page.ts';
+
+const home = mkdtempSync(join(tmpdir(), 'gbrain-segmented-tiling-'));
+const env = { GBRAIN_HOME: home, OPENAI_API_KEY: undefined, VOYAGE_API_KEY: undefined, ANTHROPIC_API_KEY: undefined };
+const logger = { info() {}, warn() {}, error() {} };
+const SLUG = 'people/person-example';
+const FILE = 'people/person-example.md';
+let engine: PGLiteEngine;
+
+function git(root: string, ...args: string[]): string { return execFileSync('git', ['-C', root, ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim(); }
+function commit(root: string): void { git(root, 'add', '-A'); git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'commit', '-qm', 'test content'); }
+const person = (bytes: number) => '---\ntype: person\ntitle: Person Example\n---\n\n# Person Example\n\n' + conversationBody(bytes, { seed: 7 }) + '\n<!-- timeline -->\n- **2026-10-01** | manual — first\n';
+/** Deletes the page's last live manifest row: its live generation no longer covers its body. */
+const damage = (sourceId: string, pageId: number) => withManagedFixtureWrite(engine, [sourceId], tx => tx.executeRaw(
+ 'DELETE FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NULL AND seq=(SELECT max(seq) FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NULL)', [pageId]));
+const counts = async () => (await engine.executeRaw<Record<string, number>>(`SELECT (SELECT count(*)::int FROM pages) AS pages, (SELECT count(*)::int FROM page_body_manifest) AS manifest,
+ (SELECT count(*)::int FROM page_body_stagings) AS stagings, (SELECT count(*)::int FROM page_versions) AS versions, (SELECT count(*)::int FROM op_checkpoints) AS checkpoints,
+ (SELECT max(knowledge_revision::text) FROM pages) AS revision`))[0];
+
+beforeAll(async () => { engine = new PGLiteEngine(); await engine.connect({}); await engine.initSchema(); }, 120_000);
+afterAll(async () => {
+ await withEnv(env, async () => { await disposePersistenceConsumer(engine); await engine.disconnect(); });
+ rmSync(home, { recursive: true, force: true });
+});
+
+async function managedFixture() {
+ await disposePersistenceConsumer(engine);
+ await engine.setConfig('pages.segmented_storage', 'on');
+ const sourceId = `tiling-${randomUUID().slice(0, 8)}`;
+ const root = join(home, sourceId);
+ mkdirSync(join(root, 'people'), { recursive: true });
+ git(root, 'init', '-q');
+ writeFileSync(join(root, FILE), person(1260 * 1024));
+ commit(root);
+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');
+ await engine.executeRaw("INSERT INTO sources(id,name,local_path,config) VALUES($1,$1,$2,'{}')", [sourceId, root]);
+ await claimWorktree(engine, sourceId, root);
+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');
+ expect(await performManagedSync(engine, { sourceId, noPull: true })).toMatchObject({ status: 'first_sync', filesImported: 1 });
+ const [row] = await engine.executeRaw<{ id: number }>('SELECT id FROM pages WHERE source_id=$1 AND slug=$2', [sourceId, SLUG]);
+ return { sourceId, root, pageId: Number(row!.id) };
+}
+
+test('doctor lists the damaged page read-only; the repair republishes it from HEAD and the next sync completes', async () => withEnv(env, async () => {
+ const f = await managedFixture();
+ expect((await segmentedTilingCheck(engine, [f.sourceId])).status).toBe('ok');
+ await damage(f.sourceId, f.pageId);
+ appendFileSync(join(f.root, FILE), '- **2026-11-02** | meeting — second\n');
+ commit(f.root);
+ const held = await performManagedSync(engine, { sourceId: f.sourceId, noPull: true, noEmbed: true, noExtract: true });
+ expect((held.held ?? []).map(hold => [hold.path, hold.code])).toEqual([[FILE, 'page_write_failed']]);
+ const [source] = await engine.executeRaw<{ incarnation: string }>('SELECT incarnation::text AS incarnation FROM sources WHERE id=$1', [f.sourceId]);
+ expect(await readGitHold(engine, f.sourceId, source!.incarnation, FILE)).not.toBeNull();
+
+ const before = await counts();
+ const check = await segmentedTilingCheck(engine, [f.sourceId]);
+ expect(await counts()).toEqual(before);
+ expect(check).toMatchObject({ status: 'fail', details: { pages: [expect.objectContaining({ source_id: f.sourceId, slug: SLUG, rows: expect.any(Number) })] } });
+ expect(check.fix?.argv).toEqual(['gbrain', 'repair', 'segmented-tiling', '--source', f.sourceId]);
+ const [damaged] = await untiledSegmentedPages(engine, { sourceIds: [f.sourceId] });
+
+ expect(AUTO_REPAIR_REGISTRY.map(spec => spec.kind)).not.toContain('segmented-tiling');
+ const scope = await resolveRepairScope(engine, f.sourceId);
+ const run = (apply: boolean, expect?: string) => repairRunner(engine, { apply, logger }).then(runner => runner.run('segmented-tiling', scope, { explicit: true, sourceFlag: f.sourceId, ...(expect ? { expect } : {}) }));
+ await expect(repairRunner(engine, { apply: false, logger }).then(runner => runner.run('segmented-tiling', scope, {}))).rejects.toThrow(/explicit/);
+ const remote = { engine, config: { engine: engine.kind }, logger, dryRun: true, remote: true, sourceId: f.sourceId } as never;
+ await expect(runRepair(remote, segmentedTilingRepair, scope, { apply: false, explicit: true })).rejects.toMatchObject({ code: 'permission_denied' });
+ await expect(run(true)).rejects.toThrow(/preview/i);
+
+ const preview = await run(false);
+ const hash = preview.preview_hash!;
+ const blob = git(f.root, 'rev-parse', `HEAD:${FILE}`);
+ expect(preview.listing).toEqual([{ item: `${f.sourceId}:${SLUG} (revision ${damaged!.revision})`, class: 'republish', detail: expect.stringContaining(`blob ${blob.slice(0, 12)}`) }]);
+ expect((await untiledSegmentedPages(engine, { sourceIds: [f.sourceId] }))).toEqual([damaged!]);
+ await expect(run(true, 'f'.repeat(64))).rejects.toMatchObject({ code: 'preview_changed' });
+
+ const applied = await run(true, hash);
+ expect(applied.applied).toBe(1);
+ expect(applied.outcomes).toEqual({ republished: 1 });
+ expect(await untiledSegmentedPages(engine, { sourceIds: [f.sourceId] })).toEqual([]);
+ const [page] = await engine.executeRaw<{ body_generation: number }>('SELECT body_generation FROM pages WHERE id=$1', [f.pageId]);
+ expect(Number(page!.body_generation)).toBe(damaged!.body_generation + 1);
+ expect(await engine.executeRaw('SELECT body_generation::int AS g, body_bytes::int AS b FROM page_versions WHERE page_id=$1 AND body_generation=$2',
+ [f.pageId, damaged!.body_generation])).toContainEqual({ g: damaged!.body_generation, b: damaged!.body_bytes });
+ const body = await materializeSegmentedBody(engine, f.pageId, f.sourceId);
+ expect(body.timeline).toContain('meeting — second');
+ const [receipt] = await engine.executeRaw<{ completed_keys: Array<Record<string, unknown>> }>('SELECT completed_keys FROM op_checkpoints WHERE op=$1 AND fingerprint=$2',
+ [SEGMENTED_TILING_RECEIPT_OP, `${hash}:${f.pageId}`]);
+ expect(receipt!.completed_keys[0]).toMatchObject({ slug: SLUG, generation_before: damaged!.body_generation, blob, hold_cleared: true });
+ expect(await readGitHold(engine, f.sourceId, source!.incarnation, FILE)).toBeNull();
+
+ const after = await performManagedSync(engine, { sourceId: f.sourceId, noPull: true, noEmbed: true, noExtract: true });
+ expect(after.status).toBe('synced');
+ expect(after.failures).toBeUndefined();
+ expect(after.held ?? []).toEqual([]);
+ expect((await segmentedTilingCheck(engine, [f.sourceId])).status).toBe('ok');
+ expect((await run(false)).listing ?? []).toEqual([]);
+}), 300_000);
+
+test('a source without a Git checkout refuses the preview: Git is the only copy it republishes', async () => withEnv(env, async () => {
+ await disposePersistenceConsumer(engine);
+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');
+ const sourceId = `repoless-${randomUUID().slice(0, 8)}`;
+ await engine.executeRaw("INSERT INTO sources(id,name,config) VALUES($1,$1,'{}')", [sourceId]);
+ await engine.setConfig('pages.segmented_storage', 'on');
+ await importFromContent(engine, 'notes/repoless-example', person(1260 * 1024), { sourceId, noEmbed: true });
+ const [row] = await engine.executeRaw<{ id: number }>('SELECT id FROM pages WHERE source_id=$1 AND slug=$2', [sourceId, 'notes/repoless-example']);
+ await engine.executeRaw('DELETE FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NULL AND seq=(SELECT max(seq) FROM page_body_manifest WHERE page_id=$1 AND staging_id IS NULL)', [row!.id]);
+ expect((await segmentedTilingCheck(engine, [sourceId])).status).toBe('fail');
+ const scope = await resolveRepairScope(engine, sourceId);
+ await expect(repairRunner(engine, { apply: false, logger }).then(runner => runner.run('segmented-tiling', scope, { explicit: true, sourceFlag: sourceId })))
+ .rejects.toMatchObject({ code: 'not_a_git_repo' });
+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');
+}), 300_000);
diff --git a/test/receipt-coverage.test.ts b/test/receipt-coverage.test.ts
index b09692a9bc..2912491861 100644
--- a/test/receipt-coverage.test.ts
+++ b/test/receipt-coverage.test.ts
@@ -49,6 +49,7 @@ const RECEIPT_COVERAGE: Record<string, Coverage> = {
'src/core/trust/backfill.ts': { sites: 3, tables: ['pages', 'facts', 'takes', 'timeline_entries'], todo: 'TODO(receipts): gbrain trust backfill and claim-sources --relower (#6533) set legacy trust_tier/write_origin (no content column) as attributed coordinated maintenance without a request; publish each batch as a maintenance intent' },
'src/core/repair/frontmatter-keys.ts': { sites: 1, tables: ['pages'], todo: 'TODO(receipts): gbrain repair frontmatter-keys (#6454) strips the four parser-reserved keys from pages.frontmatter (no content column) in withCoordinatedWrite batches as explicit-only maintenance without a request; publish each batch as a maintenance intent' },
'src/core/repair/secret-spans.ts': { sites: 1, tables: ['pages', 'facts', 'takes', 'timeline_entries'], todo: 'TODO(receipts): gbrain repair secret-spans (GST-327) replaces credential spans in preview-approved pages, their facts, takes and timeline rows in withCoordinatedWrite as explicit-only maintenance without a request, with its own per-page op_checkpoints receipt; publish each page as a maintenance intent' },
+ 'src/core/repair/segmented-tiling.ts': { sites: 1, tables: ['pages'], todo: 'TODO(receipts): gbrain repair segmented-tiling (GST-1209) republishes the body of a preview-approved page from its file at HEAD in withCoordinatedWrite as explicit-only maintenance without a request, with its own per-page op_checkpoints receipt; publish each page as a maintenance intent' },
'src/core/cycle/concept-publication.ts': { sites: 1, tables: [], other: 'concept provenance links (links is not a guarded table)' },
'src/core/persistence/links-maintenance.ts': { sites: 1, tables: [], other: 'derived links and the pages attendance-blocked stamp (a physical column the guard exempts)' },
'src/core/persistence/database-write.ts': { sites: 1, tables: [], other: 'manual links and chronicle ontology observations (unguarded tables)' },
diff --git a/test/repair-explicit-only.test.ts b/test/repair-explicit-only.test.ts
index 42c226789d..630d337060 100644
--- a/test/repair-explicit-only.test.ts
+++ b/test/repair-explicit-only.test.ts
@@ -24,7 +24,7 @@ import { REPAIR_HELP, parseRepairArgs, runRepairCommand } from '../src/commands/
import { _resetCliExitVerdictForTests, currentExitCode } from '../src/core/cli-force-exit.ts';
import { withEnv } from './helpers/with-env.ts';
-const EXPLICIT: RepairKind[] = ['accepted-takes', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'extractor-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'timeline-comments', 'timeline-order', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'uncommitted-deletions', 'generations', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind'];
+const EXPLICIT: RepairKind[] = ['accepted-takes', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'extractor-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'timeline-comments', 'timeline-order', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'segmented-tiling', 'uncommitted-deletions', 'generations', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind'];
// GBRA-161: the generations preview carries its ledger kind and the restore window, so the printed command runs as is.
const PREVIEW = (kind: RepairKind): string => kind === 'generations' ? 'gbrain repair generations --kind atom --since 30d' : `gbrain repair ${kind}`;
let engine: PGLiteEngine;