Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Admit a managed readers run's pages together so their files share commits (GST-2476)

garrytan2e06f18fc2fd
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 242af1ebbf..48321ef308 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,7 +14,7 @@ identifiers and attribution are available in the pre-removal Git revision
**Making a page private through an operator repair no longer lowers its trust or rewrites its file.** On a managed brain, an operator who added `visibility: private` to one page through a local `put_page`, or ran `gbrain repair visibility`, saw the page drop from "your notes" (operator_curated) to "written by an agent", a pending `lower_page` proposal only a typed token could settle, and a re-rendered file: a `slug` key removed, a `source_kind` value overwritten with `put_page`, keys reordered, and provenance stamps and a `trust_tier` marker added. Both repairs and `gbrain repair private-labels` now write through one label-only door: the page keeps its tier, no proposal is filed, and its file changes only on the label lines, with every other key, its order and the body kept byte for byte. A new slug-bound form, `gbrain repair visibility --slug <page>`, is the operator's single-page way to make a page private (preview first, then `--apply --expect <hash>`). A plain `put_page`, local or remote, is still an agent edit and lowers the page as before. `gbrain trust backfill --label-lowerings` lists every `lower_page` proposal as label-only or content, counted by proposer, so an operator can see which lowerings came from label-only writes; it changes nothing, and raising a page back stays a trust decision.
-**The brain host can now set who reads a page without lowering its trust.** `gbrain repair visibility --slug <page> --readers alice-example,group:team-a` (or `--slugs-file <file>` for a long list, `--readers none` to remove the field) sets a page's `readable_by` through the same label-only door: the page keeps its tier and provenance, no `lower_page` proposal is filed, and its file changes only on the `readable_by` line. Before, the only way was a `put_page`, which lowered the page to "written by an agent". Readers are checked first: an unknown or disabled member or an unknown group refuses with nothing written, and a list that leaves out the brain owner refuses unless `--allow-without-owner` is passed, because `readable_by` binds the owner too. Each page is its own write, so a list of thousands holds no run-wide lock; on a managed brain the Git effect commits the files up to 100 per commit and `--json` counts them by effect state in `details.git`. A managed page with no file outside a declared `storage.db_only` path is now held `canonical_file_missing` with nothing written (it was refused as a failed `source_changed`), for every `repair visibility` write.
+**The brain host can now set who reads a page without lowering its trust.** `gbrain repair visibility --slug <page> --readers alice-example,group:team-a` (or `--slugs-file <file>` for a long list, `--readers none` to remove the field) sets a page's `readable_by` through the same label-only door: the page keeps its tier and provenance, no `lower_page` proposal is filed, and its file changes only on the `readable_by` line. Before, the only way was a `put_page`, which lowered the page to "written by an agent". Readers are checked first: an unknown or disabled member or an unknown group refuses with nothing written, and a list that leaves out the brain owner refuses unless `--allow-without-owner` is passed, because `readable_by` binds the owner too. A list of thousands holds no run-wide lock: on a managed brain each admission window of pages is admitted together (Postgres publishes them in groups of up to 8), the Git effect commits the ready files together (up to 100 per commit), and `--json` counts them by effect state in `details.git`. A managed page with no file outside a declared `storage.db_only` path is now held `canonical_file_missing` with nothing written (it was refused as a failed `source_changed`), for every `repair visibility` write.
### Itemized changes
@@ -27,7 +27,7 @@ identifiers and attribution are available in the pre-removal Git revision
- `src/core/trust/label-lowerings.ts` (new), `src/cli/commands/trust.ts`: `gbrain trust backfill --label-lowerings [--source ID] [--json]`, read-only.
- Tests: `test/label-only-repairs.test.ts` (managed Git source and repo-less brain, PGLite and Postgres); `test/owner-member-boundary.test.ts` pins that the owner reads its private pages and pages that list it, and never another member's restricted page, on every read path.
- `src/core/persistence/label-edit.ts`: the label edit also takes `readable_by`, written as a one-line list (`readable_by: [a, b]`); block lists and duplicate keys are declined. `LABEL_KEYS` is unchanged, so `gbrain trust backfill --label-lowerings` reads as before.
-- `src/core/repair/readers.ts` (new), `src/core/repair/visibility.ts`, `src/commands/repair.ts`: `gbrain repair visibility (--slug <slug>... | --slugs-file <file>) --readers <ids|none> [--allow-without-owner]`, preview-bound (`--apply --expect <hash>`), up to 10,000 pages per run; reader validation (`access_member_invalid`, owner rule); `details.git` effect counts on apply; a managed page with no canonical file outside `storage.db_only` held `canonical_file_missing`.
+- `src/core/repair/readers.ts` (new), `src/core/repair/visibility.ts`, `src/commands/repair.ts`: `gbrain repair visibility (--slug <slug>... | --slugs-file <file>) --readers <ids|none> [--allow-without-owner]`, preview-bound (`--apply --expect <hash>`), up to 10,000 pages per run; reader validation (`access_member_invalid`, owner rule); `details.git` effect counts on apply; a managed run admits each window of pages together (`applyBatch`, `publishMaintenancePages` under the run's batch id); a managed page with no canonical file outside `storage.db_only` held `canonical_file_missing`.
- `src/core/repair/label-write.ts`: a readers run publishes its pages under one maintenance batch.
- Tests: `test/readers-label-setter.test.ts` (managed Git source with tracked files, a `storage.db_only` page and a page with no file; repo-less brain with remote reads by member, a 150-page slugs file, refusals and `preview_changed`).
diff --git a/docs/architecture/canonical-writers.tsv b/docs/architecture/canonical-writers.tsv
index 753cb09564..d099566c8a 100644
--- a/docs/architecture/canonical-writers.tsv
+++ b/docs/architecture/canonical-writers.tsv
@@ -33,7 +33,7 @@ src/core/persistence/twin-merge.ts 10 coordinator The #6948 twin merge (links in
src/core/repair/origin-twins.ts 1 coordinator gbrain repair origin-twins and the post-sync heal resolve each verified twin as a receipted managed_maintenance_resolve_origin_twin request (submitDatabaseMaintenanceIntent, owner host only) on managed brains and in one maintenanceTransaction on unmanaged brains; the verdict is recomputed inside the write and any drift holds the file instead of merging.
src/core/repair/unwithdrawn-facts.ts 1 coordinator `gbrain repair unwithdrawn-facts` (#8819) puts the ledger before-state back on preview-approved facts a fact_fence generation retired though no fence held them. On a managed brain the UPDATE runs only inside the managed_maintenance_restore_unwithdrawn_facts publication; an unmanaged or repo-less brain classifies every candidate excluded (unmanaged_reconcile), so a set previewed while it was managed is reclassified in one maintenanceTransaction under the page lock after the revision recheck and restores nothing.
src/core/repair/private-labels.ts 1 coordinator `gbrain repair private-labels` (GST-2059) removes or re-adds only the previewed privacy keys on preview-approved pages. On a managed brain each page goes through publishMaintenancePage (maintenancePreflight owner check, revision-bound, keep_trust; the Git door on a repo source); an unmanaged or repo-less brain re-imports it in one maintenanceTransaction under the page lock after the revision recheck, except a page whose file sits in an unmanaged source's checkout, which is held (file_backed) and never written.
-src/core/repair/label-write.ts 1 coordinator The label-only page write of `gbrain repair visibility` and `gbrain repair private-labels` (GST-2468). On a managed brain it publishes through publishMaintenancePage (maintenancePreflight owner check, revision-bound, keep_trust, label_edit), or through publishMaintenancePages with a maintenance batch for repair visibility --readers (GST-2476), so the Git effect commits a run's files together. Only an unmanaged or repo-less brain calls importFromContent (importAtRevision), in one maintenanceTransaction under the page lock after the revision recheck and under withTrustKeep; for a page whose file sits in an unmanaged checkout (writeLabelFile) the file must still hold the page (ownerRepairFile), and writePageThrough writes the spliced bytes back with their preimage checked inside that transaction, so a refused or failed file write rolls the import back.
+src/core/repair/label-write.ts 1 coordinator The label-only page write of `gbrain repair visibility` and `gbrain repair private-labels` (GST-2468). On a managed brain it publishes through publishMaintenancePage (maintenancePreflight owner check, revision-bound, keep_trust, label_edit), or through publishMaintenancePages with a maintenance batch for repair visibility --readers (GST-2476; visibility.ts applyBatch admits a window of pages together), so the Git effect commits a window's ready files together. Only an unmanaged or repo-less brain calls importFromContent (importAtRevision), in one maintenanceTransaction under the page lock after the revision recheck and under withTrustKeep; for a page whose file sits in an unmanaged checkout (writeLabelFile) the file must still hold the page (ownerRepairFile), and writePageThrough writes the spliced bytes back with their preimage checked inside that transaction, so a refused or failed file write rolls the import back.
src/core/search-vector-backfill.ts 2 projection backfillPageVectors (reindex-search-vector, the fts_cjk_boundary migration) rewrites only pages.search_vector, through the seal's pageSearchVectorSql over the sanitized timeline and guarded by the knowledge revision it read (its read is an UPDATE-to-self of id); canonical fields are untouched and the managed writer guard passes search_vector-only updates.
src/commands/sources-set-path.ts 1 coordinator Managed rebind uses native exclusion, exact manifest verification and a retained source-incarnation receipt; unmanaged pointer repair remains explicit.
src/commands/sources.ts 10 coordinator Managed add/archive/restore/remove/purge/reclone route through source lifecycle; remaining legacy canonical SQL stays fenced.
diff --git a/docs/architecture/key-files/core-persistence-continued.md b/docs/architecture/key-files/core-persistence-continued.md
index 2ed24af7f6..e7cf026a50 100644
--- a/docs/architecture/key-files/core-persistence-continued.md
+++ b/docs/architecture/key-files/core-persistence-continued.md
@@ -42,4 +42,4 @@ The #6188 fence check and fence repair live in [fence repair](core-fence-repair.
- `src/core/postgres-engine/reserved-transactions.ts` with `isPersistenceDeclaration` (`src/core/persistence/protocol.ts`) — BEGIN pipelining. `isPersistenceDeclaration` (GBRA-156) accepts exactly the `protocol.ts` declarations (transaction-local `set_config` calls and the brain row FOR SHARE). Admission, the FIFO claim, the effect claims and source guards, the receipt true-up, the single publication (`coordinator.ts`) and the consumer's lent lane transactions (`consumer.ts` `onLane`, through `reservedTransactions`) pass it as `TransactionOptions.pipelineFirst`, so behind a transaction-mode pooler the declaration goes out with BEGIN (one round trip less per transaction) while every later statement still waits for BEGIN's answer; if BEGIN fails, only the declaration has run, in autocommit, where it changes nothing. A reserved transaction gates every statement its body can reach, savepoint and nested-transaction bodies and `file()` included. A sync lane group's publication passes none: its begun mark must follow BEGIN's answer (`sync-lanes.ts` `awaitLaneBegin`). Tests: `test/persistence-pipeline-first.test.ts` (the predicate and the reviewed call sites), `test/e2e/pgbouncer-transaction-pipeline-6930.test.ts`.
- `src/core/links-fence.ts` + `src/core/links-fence-projection.ts` — the `## Links` fence (G5): the markdown form of authored (manual) edges, `<!--- gbrain:links:begin -->` / `end` around `| to | type | source | valid_from | valid_until | context |`, one row per (to, type, source), source defaulting to `manual`; a reconciliation-managed provenance (`MANAGED_LINK_SOURCES`, exported here and re-exported by `ops/links.ts`), a bad date or a duplicate key is a skipped row with a warning. `parseLinksFence` (state `absent` / `parsed` / `malformed`), the deterministic `renderLinksFence` (rows sorted by key), `upsertLinksFenceRow` (omitted context and dates keep the stored ones; identical row = unchanged body), `removeLinksFenceRows` (leaves an empty fence), `replaceOrInsertLinksFence` (a `## Links` section at the end of compiled_truth, above an unclosed code block) and `maskLinksFence`, which `extractBodyLinks` applies so the fence cells are never also extracted as markdown, wikilink or bare-slug edges. A rewrite of a fence that parses with warnings throws `LinksFenceRewriteRefusal`. Cells use `escapeFenceCell` plus the same one-level marker encoding for `gbrain:links:*`. `projectLinksFence(tx, page, rows)` reconciles only what the page's fence owns: edges with origin_page_id = the page and origin_field `links_fence` (deleted, context-updated, inserted by key) and `manual` transitions with origin_page_id = the page, compared per row by (to, type, link_source, kind, date) and written with the row's source in `link_source`, then refreshes the touched relationships; a row whose target is not a live page in the source yields no edge. `linksFenceProjectionRows` returns null for an absent or malformed fence, so such a body changes no edge (an absent fence retires nothing, as for facts; unlike facts no held generation is recorded, so after a hand deletion of the whole section the database keeps edges git no longer has until `remove_link` or the G3 indexer), and drops rows typed with a link type the active pack does not declare. Callers: `prepareCanonicalProjections` (every projected page publication, quarantined and segmented-unread bodies excepted), `prepareAutomaticLinks` (put_page auto-links and the managed links maintenance, so `extract --stale` restores a lost fence edge) and `persistence/manual-links.ts`. Database-only manual edges (origin_field NULL) are never touched by any of them. Pinned by `test/links-fence.test.ts` and `test/managed-links-fence.test.ts`.
- `src/core/persistence/source-lifecycle.ts` retire batches with `topology-transaction.ts` (#7702/#8905) — every `topologyTransaction` runs under `statement_timeout=5s` and `lock_timeout=1s` and maps SQLSTATE 57014/55P03 to `write_pending`, so a managed `remove` or `purge` first deletes the source's pages in batches of `SOURCE_RETIRE_BATCH_PAGES` (500; about 110 ms per batch with chunks and links on Postgres) under the native locks `withTopologyLocks` holds, each batch its own `topologyTransaction`. A batch re-checks the final transaction's preconditions (admin state, `lockTopologyRows`, the incarnation read before the locks, archived or `EXPIRED_ARCHIVE_SQL` due, no prior receipt other than this request's own admission, no `oauth_clients` reference, `assertTopologyRequestsSettled` from `topology-locks.ts`, `lockTopologyPrincipal`, and on the first batch `admitTopologyRetire` from `topology-receipts.ts`, which checks the receipt quotas at the largest receipt and records the admission) and, when one would refuse or replay, stops without deleting so the final transaction reports it, or throws that refusal before deleting; a batch that fails `write_pending` is retried at half the size (floor 1), so heavy pages still fit the bound; a refusal after some batches committed says how many pages that run already deleted; a dry run, a missing checkout, an unconfirmed operation and an admission-hooked call never batch. The final transaction's `sources` delete runs `removedSourceScrubCtes` (#9068: the source's retrieval and volunteer rows, and its slugs in eval candidates), so a run that stops after some batches leaves those rows until a retry commits. Each batch deletes inside `withCoordinatedWrite` with local-CLI attribution (the `pages` guard needs the source in `gbrain.write_sources`; only the final `sources` delete sets `gbrain.topology_change`). Committed batches are durable: a retry (same or new request_id) resumes, and the receipt's `pages_deleted` counts the pages deleted by the run that commits it. #8984: the first batch's transaction records the admission, a `persistence_topology_changes` row in state `recovering` with no recovery record, bound to the source incarnation it deletes from (`isRetireAdmission`); `recordTopologyChange` settles it `committed`, and a remove or purge that deletes that incarnation under another request, or any other lifecycle operation that commits a change to it (archive, restore, rebind, claim), settles it `failed` (`supersedeTopologyRetires`, which skips only the caller's own principal and request_id pair). A retry of the request_id against another incarnation (the source was removed and re-added) refuses `source_retire_superseded`, settles the admission `failed` and deletes nothing, also when a since-replaced CLI registration (another principal) admitted that request_id for a remove or purge, since idempotency is per principal; an admission bound to the current incarnation still resumes under the new registration; deactivation and graduation list an unsettled admission as a blocker that names its request_id and asks the user whether to finish it or keep what is left; an automatic expired-archive purge's admission (`expired_only` in its outcome) is left to the next automatic run with `needsUser: false`, and `sources writer status --json` lists the stopped and superseded requests (`topology_requests`). Tests: `test/source-retire-batched.test.ts` (PGLite), `test/e2e/source-retire-batched-postgres.test.ts` (60,000 pages), `test/source-retire-incarnation.test.ts` and `test/e2e/source-retire-incarnation-postgres.test.ts`.
-- `src/core/persistence/label-edit.ts` + `src/core/repair/label-write.ts` (GST-2468) — the label-only page write of operator repairs (`repair visibility`, `repair private-labels`). `label-edit.ts`: `LABEL_KEYS` (visibility, access, confidentiality), `readLabelEdit` validates a `managed_maintenance_page` intent's `label_edit` (`set` plain-word values, `remove` keys), `spliceLabelLines` removes, replaces or appends label lines in a file's frontmatter text and declines (null) a block or multi-line value, a duplicate key or a file with no frontmatter block. `page-prepare.ts` `labelEditFile` uses it in `compareFileTarget` when the file matches the snapshot: the edited file (or its edited head over the write's body, for a fence row) must read back as the file the write would render, else the write refuses `invalid_params`; the file is never re-rendered under a label edit. `label-write.ts`: `writeLabelPage` publishes through the maintenance door with `keep_trust` and the edit on a managed brain, else `importAtRevision` (maintenance transaction, page lock, `withTrustKeep`); `filesInCheckout` names the unmanaged pages whose checkout file would undo a database-only write; for those, `writeLabelFile` applies `labelEditFile` to the file's own bytes (it must still hold the page, `ownerRepairFile`), imports it under `withTrustKeep` and writes it back through `writePageThrough` (`fileText`, preimage checked) in one transaction; drift or a file edited mid-write throws `source_changed`, and a file it cannot write (`sync.write_through` off, `url_clone`, an unresolvable target, a write error) returns a held `file_backed` outcome with the import rolled back. `repair visibility` calls it directly; `private-labels` still holds those pages as `file_backed`. GST-2476: `label-edit.ts` also edits `readable_by` (`READERS_KEY`, `EDIT_KEYS`; `set` takes a plain-word list, rendered `readable_by: [a, b]`), and `writeLabelPage` takes a maintenance `batch` (then `publishMaintenancePages([page], {batch})`). `src/core/repair/readers.ts` is `repair visibility --readers`: `checkedReaders` (members exist and are enabled, groups exist, else `access_member_invalid`; a list that leaves out the owner refuses unless `allowWithoutOwner`), `planReaders` (named pages or a `--slugs-file`, up to `READERS_MAX_PAGES`, approved set bound to each page's live readers) and `applyReaders` (phase `READERS_PHASE`, one label-only write per page, a segmented page held); `readersGitCounts` reports the run's Git effect states. `visibility.ts` `labelWrite` holds a managed page with no canonical file outside `storage.db_only` as `canonical_file_missing`.
+- `src/core/persistence/label-edit.ts` + `src/core/repair/label-write.ts` (GST-2468) — the label-only page write of operator repairs (`repair visibility`, `repair private-labels`). `label-edit.ts`: `LABEL_KEYS` (visibility, access, confidentiality), `readLabelEdit` validates a `managed_maintenance_page` intent's `label_edit` (`set` plain-word values, `remove` keys), `spliceLabelLines` removes, replaces or appends label lines in a file's frontmatter text and declines (null) a block or multi-line value, a duplicate key or a file with no frontmatter block. `page-prepare.ts` `labelEditFile` uses it in `compareFileTarget` when the file matches the snapshot: the edited file (or its edited head over the write's body, for a fence row) must read back as the file the write would render, else the write refuses `invalid_params`; the file is never re-rendered under a label edit. `label-write.ts`: `writeLabelPage` publishes through the maintenance door with `keep_trust` and the edit on a managed brain, else `importAtRevision` (maintenance transaction, page lock, `withTrustKeep`); `filesInCheckout` names the unmanaged pages whose checkout file would undo a database-only write; for those, `writeLabelFile` applies `labelEditFile` to the file's own bytes (it must still hold the page, `ownerRepairFile`), imports it under `withTrustKeep` and writes it back through `writePageThrough` (`fileText`, preimage checked) in one transaction; drift or a file edited mid-write throws `source_changed`, and a file it cannot write (`sync.write_through` off, `url_clone`, an unresolvable target, a write error) returns a held `file_backed` outcome with the import rolled back. `repair visibility` calls it directly; `private-labels` still holds those pages as `file_backed`. GST-2476: `label-edit.ts` also edits `readable_by` (`READERS_KEY`, `EDIT_KEYS`; `set` takes a plain-word list, rendered `readable_by: [a, b]`), and `writeLabelPage` takes a maintenance `batch` (then `publishMaintenancePages([page], {batch})`). `src/core/repair/readers.ts` is `repair visibility --readers`: `checkedReaders` (members exist and are enabled, groups exist, else `access_member_invalid`; a list that leaves out the owner refuses unless `allowWithoutOwner`), `planReaders` (named pages or a `--slugs-file`, up to `READERS_MAX_PAGES`, approved set bound to each page's live readers) `readersWrite` (re-checks a page, holds a segmented one, renders the edit) and `applyReaders` (phase `READERS_PHASE`, one label-only write); `visibility.ts` `applyBatch` admits a managed source's run of readers items together through `publishMaintenancePages` under the run's batch (Postgres publishes them in groups of up to 8), so the Git effect commits a window's ready files together; `readersGitCounts` reports the run's Git effect states. `visibility.ts` `labelWrite` holds a managed page with no canonical file outside `storage.db_only` as `canonical_file_missing`.
diff --git a/docs/guides/repair.md b/docs/guides/repair.md
index d21e4b8b5f..4f052cdb76 100644
--- a/docs/guides/repair.md
+++ b/docs/guides/repair.md
@@ -112,9 +112,14 @@ and its own keys would stop reading the page; a group the owner belongs to does
not count. Pass `--allow-without-owner` when that is what you want. A page whose
readers changed after the preview is skipped.
-Each page is its own write, so no lock or transaction spans the run and a long
-list does not hold up other writers. On a managed brain the Git effect commits
-the tracked files, grouping queued files up to 100 per commit; `--json` reports
+No lock or transaction spans the run, so a long list does not hold up other
+writers. On a managed brain the apply admits a source's pages together, one
+admission window at a time; on Postgres the owner publishes them in groups of
+up to 8 pages (PGLite publishes one at a time). The Git effect then commits the
+files that are ready together, up to 100 per commit, and waits briefly while
+more of the run's writes are still publishing, so a long list lands in a few
+large commits rather than one per page. Each page still succeeds or fails on
+its own. `--json` reports
`details.git`, the run's files by effect state (`committed`, `pending`,
`failed`, and `skipped` for a `storage.db_only` page, which has no file). A
managed page with no file, outside any declared `storage.db_only` path, is held
diff --git a/scripts/page-body-readers-baseline.tsv b/scripts/page-body-readers-baseline.tsv
index 18148af924..5911c5790f 100644
--- a/scripts/page-body-readers-baseline.tsv
+++ b/scripts/page-body-readers-baseline.tsv
@@ -582,7 +582,7 @@ src/core/repair/overwritten-edits.ts versionMarkdown call:serializePageToMarkdow
src/core/repair/overwritten-edits.ts versionMarkdown field SEGW inline versions only: overwriteContent returns null for a segmented version, which answers version_changed
src/core/repair/overwritten-edits.ts versionMarkdown sql SEGW selects body_storage; a segmented version answers version_changed
src/core/repair/private-labels.ts prepareWrite call:serializePageToMarkdown SEGW a segmented page returns segmented_page before the read and keeps its label
-src/core/repair/readers.ts applyReaders call:serializePageToMarkdown SEGW a segmented page returns segmented_page before the read and keeps its readers
+src/core/repair/readers.ts readersWrite call:serializePageToMarkdown SEGW a segmented page returns segmented_page before the read and keeps its readers
src/core/repair/redacted-facts.ts pageTexts field INLINE inline branch only (a segmented page's body is read through readBodyRange in segmentedText)
src/core/repair/redacted-facts.ts pageTexts sql INLINE page_versions rows that hold a body copy (compiled_truth IS NOT NULL); a segmented version has none and is not read
src/core/repair/redacted-facts.ts restoreFencedRows call:serializePageToMarkdown INLINE inline pages only: a segmented page returns before the read, and its fenced rows preview as class segmented
diff --git a/src/core/repair/readers.ts b/src/core/repair/readers.ts
index acce08b6a5..c6de1392b3 100644
--- a/src/core/repair/readers.ts
+++ b/src/core/repair/readers.ts
@@ -114,7 +114,7 @@ async function planPages(engine: BrainEngine, selection: Selection) {
}
/** The maintenance batch id an approved readers apply carries on its managed writes (one per approved set and source). */
-const readersBatch = (hash: string, sourceId: string) => digest(['visibility-readers-batch-v1', hash, sourceId]);
+export const readersBatch = (hash: string, sourceId: string) => digest(['visibility-readers-batch-v1', hash, sourceId]);
/**
* How the managed writes of the approved set `hash` reached Git: their Git effects by state. The Git effect runner commits
@@ -181,24 +181,45 @@ export async function planReaders(engine: BrainEngine, scope: RepairScope, after
listing: items.map(item => ({ item: item.source_id + ':' + item.slug, class: 'readers', detail: item.action })) };
}
+/** A readers item ready to write: the page as read and its rendering with only `readable_by` changed. */
+export interface ReadersWrite { snapshot: PageSnapshot; content: string; edit: LabelEdit }
+
+/**
+ * One page's readers write, re-checked against the page: false when its readers moved since the plan (skipped), the held
+ * outcome of a page stored in segments, else the page and its rendering with only `readable_by` changed.
+ */
+export async function readersWrite(engine: BrainEngine, item: ReadersItem): Promise<false | RepairItemOutcome | ReadersWrite> {
+ const snapshot = await engine.readPageSnapshot(item.slug, { sourceId: item.source_id });
+ if (!snapshot || Number(snapshot.page.id) !== item.target.id || !same(normalizeReaders(snapshot.page.frontmatter), item.readers.from)) return false;
+ if (isSegmentedPage(snapshot.page)) return { applied: false, outcome: 'segmented_page', reason: 'The page body is stored in segments; edit its readable_by line by hand on the brain host.' };
+ const { [READERS_KEY]: _readers, ...rest } = snapshot.page.frontmatter;
+ const to = item.readers.to;
+ const frontmatter = to ? { ...snapshot.page.frontmatter, [READERS_KEY]: to } : rest;
+ const edit: LabelEdit = to ? { set: { [READERS_KEY]: to }, remove: [] } : { set: {}, remove: [READERS_KEY] };
+ return { snapshot, content: serializePageToMarkdown({ ...snapshot.page, frontmatter }, snapshot.tags), edit };
+}
+
+/** The outcome of a written readers item. */
+export const readersOutcome = (item: ReadersItem): RepairItemOutcome => ({ applied: true, outcome: item.readers.to ? 'readers_set' : 'readers_removed' });
+
+/** Retires the approved set once its last item has been tried. */
+export async function retireReaders(engine: BrainEngine, items: ReadersItem[]): Promise<void> {
+ const last = items.find(item => item.last && item.hash);
+ if (last) await clearApprovedSet(engine, { command: 'visibility', hash: last.hash! });
+}
+
/**
- * One page's readers write: re-checked against the page (a page whose readers moved since the plan is skipped), then the
- * label-only write (`write`, visibility.ts labelWrite) of the page with only `readable_by` changed. Its held outcome
- * when the write could not be made; `readers_set` or `readers_removed` once written.
+ * One page's readers write on its own (visibility.ts applyBatch admits a managed run's pages together): readersWrite, then
+ * the label-only write (`write`, visibility.ts labelWrite). Its held outcome when the write could not be made;
+ * `readers_set` or `readers_removed` once written.
*/
export async function applyReaders(engine: BrainEngine, item: ReadersItem,
write: (snapshot: PageSnapshot, content: string, edit: LabelEdit, batch?: string) => Promise<RepairItemOutcome | null>): Promise<boolean | RepairItemOutcome> {
try {
- const snapshot = await engine.readPageSnapshot(item.slug, { sourceId: item.source_id });
- if (!snapshot || Number(snapshot.page.id) !== item.target.id || !same(normalizeReaders(snapshot.page.frontmatter), item.readers.from)) return false;
- if (isSegmentedPage(snapshot.page)) return { applied: false, outcome: 'segmented_page', reason: 'The page body is stored in segments; edit its readable_by line by hand on the brain host.' };
- const { [READERS_KEY]: _readers, ...rest } = snapshot.page.frontmatter;
- const to = item.readers.to;
- const frontmatter = to ? { ...snapshot.page.frontmatter, [READERS_KEY]: to } : rest;
- const edit: LabelEdit = to ? { set: { [READERS_KEY]: to }, remove: [] } : { set: {}, remove: [READERS_KEY] };
- const held = await write(snapshot, serializePageToMarkdown({ ...snapshot.page, frontmatter }, snapshot.tags), edit, item.hash ? readersBatch(item.hash, item.source_id) : undefined);
- return held ?? { applied: true, outcome: to ? 'readers_set' : 'readers_removed' };
+ const ready = await readersWrite(engine, item);
+ if (ready === false || !('snapshot' in ready)) return ready;
+ return await write(ready.snapshot, ready.content, ready.edit, item.hash ? readersBatch(item.hash, item.source_id) : undefined) ?? readersOutcome(item);
} finally {
- if (item.last && item.hash) await clearApprovedSet(engine, { command: 'visibility', hash: item.hash });
+ await retireReaders(engine, [item]);
}
}
diff --git a/src/core/repair/visibility.ts b/src/core/repair/visibility.ts
index c1d4877b02..232135542a 100644
--- a/src/core/repair/visibility.ts
+++ b/src/core/repair/visibility.ts
@@ -49,12 +49,12 @@ import type { PreparedMutation } from '../persistence/coordinator.ts';
import { digest } from '../persistence/digest.ts';
import type { WriteRequest } from '../persistence/model.ts';
import { managedPersistenceEnabled } from '../persistence/ownership.ts';
-import { maintenancePreflight, submitDatabaseMaintenanceIntent } from '../persistence/prepared-maintenance.ts';
+import { maintenancePreflight, publishMaintenancePages, submitDatabaseMaintenanceIntent } from '../persistence/prepared-maintenance.ts';
import { CANONICAL_FILE_MISSING } from '../persistence/publication-failure.ts';
import { isSegmentedPage } from '../page-body/page-shape.ts';
-import { afterCursor, repairRequestId, type RepairHandler, type RepairItem, type RepairItemOutcome, type RepairPlan, type RepairPlanOptions, type RepairScope, type RepairCursor } from './core.ts';
+import { afterCursor, repairRequestId, type RepairHandler, type RepairItem, type RepairItemOutcome, type RepairPlan, type RepairPlanOptions, type RepairScope, type RepairSettlement, type RepairCursor } from './core.ts';
import { filesInCheckout, writeLabelFile, writeLabelPage } from './label-write.ts';
-import { applyReaders, planReaders, readersGitCounts, READERS_PHASE, type ReadersItem } from './readers.ts';
+import { applyReaders, planReaders, readersBatch, readersGitCounts, readersOutcome, readersWrite, retireReaders, READERS_PHASE, type ReadersItem, type ReadersWrite } from './readers.ts';
import type { LabelEdit } from '../persistence/label-edit.ts';
import type { PageSnapshot } from '../page-state/types.ts';
import { clearApprovedSet, loadApprovedSet, previewChangedError, previewHash, saveApprovedSet } from '../persistence/preview-approval.ts';
@@ -299,6 +299,18 @@ async function applyFactItem(ctx: Parameters<RepairHandler['apply']>[0], item: R
return leftWorld(fact, 'its fence row was written private, but the original row is more trusted than the repair and was kept beside it');
}
+/**
+ * The held outcome of a managed page whose canonical file is missing outside any declared storage.db_only path (the
+ * maintenance door refused it before admission, nothing written); any other error is rethrown.
+ */
+function missingFileHeld(item: Pick<RepairItem, 'source_id' | 'slug'>, error: unknown): RepairItemOutcome {
+ if (!(error instanceof OperationError) || error.detail !== CANONICAL_FILE_MISSING) throw error;
+ return { applied: false, outcome: CANONICAL_FILE_MISSING, detail: { path: `${item.slug}.md` },
+ reason: `${item.source_id}:${item.slug} has no file in its source's checkout and is not under a declared storage.db_only path, so a label write would put the `
+ + `database ahead of Git; nothing was written. If the page belongs in the database only, declare its folder under storage.db_only in gbrain.yml, commit it `
+ + `and preview again; if its file was removed by mistake, restore it (gbrain repair ghost-pages --source ${item.source_id} lists pages whose file is gone).` };
+}
+
/**
* GST-2468: one item's page write is a label-only maintenance write (repair/label-write.ts): it keeps the page's trust
* tier, files no lower_page proposal and changes only the label lines of the page's file (a fact item: its fence row).
@@ -317,11 +329,7 @@ async function labelWrite(ctx: Parameters<RepairHandler['apply']>[0], item: Repa
written = path ? await writeLabelFile(ctx.engine, item, { ...write, path })
: await writeLabelPage(ctx, item, { ...write, requestId: await repairRequestId(ctx, 'visibility', item, snapshot.revision) });
} catch (error) {
- if (!(error instanceof OperationError) || error.detail !== CANONICAL_FILE_MISSING) throw error;
- return { applied: false, outcome: CANONICAL_FILE_MISSING, detail: { path: `${item.slug}.md` },
- reason: `${item.source_id}:${item.slug} has no file in its source's checkout and is not under a declared storage.db_only path, so a label write would put the `
- + `database ahead of Git; nothing was written. If the page belongs in the database only, declare its folder under storage.db_only in gbrain.yml, commit it `
- + `and preview again; if its file was removed by mistake, restore it (gbrain repair ghost-pages --source ${item.source_id} lists pages whose file is gone).` };
+ return missingFileHeld(item, error);
}
if (written === null) throw opError('revision_conflict', `${item.source_id}:${item.slug} changed after it was read, so it was not written.`,
'Preview gbrain repair visibility again; the page is listed with its current state.');
@@ -458,6 +466,41 @@ export const visibilityRepair: RepairHandler = {
const git = details.git as Record<string, number> | undefined;
return git ? [` git: ${Object.entries(git).map(([key, value]) => `${key}=${value}`).join(', ')} (file(s) by Git effect state; queued files commit together, up to 100 per commit)`] : [];
},
+ /**
+ * GST-2476: a run of readers items of one managed source at once. Each page is re-checked as apply does; the pages are
+ * admitted together under the run's maintenance batch (publishMaintenancePages; Postgres publishes them in groups of up to
+ * 8), so the Git effect commits their ready files together (up to 100 per commit), each page its own fate. Other classes, a source
+ * without managed persistence, and a run of one apply item by item (no settlement leaves an item to apply).
+ */
+ async applyBatch(ctx, entries) {
+ const items = entries as ReadersItem[];
+ const sourceId = items[0]?.source_id;
+ const end = items.findIndex(item => (item.target ?? item.cursor).phase !== READERS_PHASE || item.source_id !== sourceId || !item.hash);
+ const run = end < 0 ? items : items.slice(0, end);
+ if (run.length < 2) return [];
+ const authority = await maintenancePreflight(ctx.engine, sourceId!);
+ if (!authority) return [];
+ const out: RepairSettlement[] = [];
+ const pages: Array<{ index: number; item: ReadersItem; ready: ReadersWrite; requestId: string }> = [];
+ try {
+ for (const [index, item] of run.entries()) {
+ const ready = await readersWrite(ctx.engine, item).catch((error: unknown) => ({ error }));
+ if (ready !== false && 'snapshot' in ready) pages.push({ index, item, ready, requestId: await repairRequestId(ctx, 'visibility', { ...item, cursor: item.target }, ready.snapshot.revision) });
+ else out[index] = ready !== false && 'error' in ready ? { error: ready.error } : { value: ready };
+ }
+ if (!pages.length) return out;
+ const published = await publishMaintenancePages(ctx.engine, authority, pages.map(({ item, ready, requestId }) => ({ slug: item.slug, content: ready.content,
+ expectedRevision: ready.snapshot.revision, requestId, keepTrust: true as const, labelEdit: ready.edit })), { batch: readersBatch(run[0]!.hash!, sourceId!) });
+ pages.forEach(({ index, item }, position) => {
+ const settled = published[position]!;
+ if (!('error' in settled)) { out[index] = { value: readersOutcome(item) }; return; }
+ try { out[index] = { value: missingFileHeld(item, settled.error) }; } catch (error) { out[index] = { error }; }
+ });
+ return out;
+ } finally {
+ await retireReaders(ctx.engine, run);
+ }
+ },
async apply(ctx, entry) {
const item = entry as VisibilityItem;
const target = item.target ?? item.cursor;
diff --git a/test/readers-label-setter.test.ts b/test/readers-label-setter.test.ts
index 7ad042845c..80ac2f3bac 100644
--- a/test/readers-label-setter.test.ts
+++ b/test/readers-label-setter.test.ts
@@ -208,8 +208,13 @@ for (const backend of testBackends()) describe('[' + backend + '] repair visibil
expect(text).toContain('outcomes: readers_set=3');
expect(text).toMatch(/git: committed=\d+, pending=\d+, failed=0, skipped=\d+/);
await settleManagedGitEffects(engine);
- // The two tracked files reach Git through the Git effect (the one door), committed together; the db_only page has no file.
- expect(git(root, 'rev-list', '--count', head + '..HEAD').trim()).toBe('1');
+ // The two tracked files reach Git through the Git effect (the one door); the db_only page has no file. The run admits its pages
+ // together: Postgres publishes them as one group, so both files land in one commit. PGLite publishes one page at a time and
+ // its Git effects share a commit only when both are ready, so there each file lands in at most its own commit.
+ const commits = git(root, 'rev-list', head + '..HEAD').trim().split(NL);
+ if (backend === 'postgres') expect(commits).toHaveLength(1);
+ else expect(commits.length).toBeLessThanOrEqual(filed.length);
+ for (const commit of commits) expect(filed.map(slug => slug + '.md')).toEqual(expect.arrayContaining(git(root, 'show', '--name-only', '--format=', commit).trim().split(NL)));
expect(git(root, 'diff', '--name-only', head, 'HEAD').trim().split(NL).sort()).toEqual(filed.map(slug => slug + '.md'));
expect(await readersGitCounts(engine, { brain_id: preview.scope?.brain_id ?? '', source_ids: ['default'] }, preview.preview_hash)).toEqual({ committed: 2, pending: 0, failed: 0, skipped: 1 });
for (const slug of filed) expect(onDisk(root, slug)).toBe(withLine(before[slug]!, readersLine('group:team-example', OWNER)));