Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Document label repairs over derived stamps and label_edit_refused (GST-2603)
garrytan9c10bc3b4ffa
diff --git a/behavior-changes.d/label_repairs_hold_refusals_gst2603.md b/behavior-changes.d/label_repairs_hold_refusals_gst2603.mdnew file mode 100644index 0000000000..0986d26c1a--- /dev/null+++ b/behavior-changes.d/label_repairs_hold_refusals_gst2603.md@@ -0,0 +1 @@+gbrain repair visibility and gbrain repair private-labels now ignore stamps gbrain keeps only in the database (a pack-renamed type, the validate: false grandfather stamp, a legacy_type retype record) when they edit a page's label line, so those pages are labelled. A page whose file still differs in a real key, or whose label line cannot be edited as one line, is held by the preview with the blocking key and its next step and is left out of the approved set. A page that starts refusing after the preview is declined with label_edit_refused, nothing is written for it, the run exits 0 and it no longer counts toward item_failures_systemic.diff --git a/changelog.d/capy-label-repair-derived-stamps.md b/changelog.d/capy-label-repair-derived-stamps.mdnew file mode 100644index 0000000000..8a1a1c75c9--- /dev/null+++ b/changelog.d/capy-label-repair-derived-stamps.md@@ -0,0 +1,9 @@+### Label repairs no longer refuse pages over gbrain's own stamps++**`gbrain repair visibility` and `gbrain repair private-labels` now label a page whose file differs from the database only by stamps gbrain keeps in the database, and they hold the pages they can't edit instead of failing the run (GST-2603).** A page retyped by a schema pack, a page with the `validate: false` grandfather stamp, or a page with a `legacy_type` retype record used to refuse every label repair, either as "cannot be made to the page's file line by line" or as `source_changed`. Enough of them in one window stopped the whole run as `item_failures_systemic`.+- Those stamps no longer count as a difference. The page is labelled and its file still gains only the `visibility` line. A real difference (a date, the tags, the title, the body) still blocks the edit.+- The preview checks every page in the window. A page whose file differs in a real key, or whose label line can't be edited as one line, is listed as held under `details.held` with the key that blocks it and its next step: `gbrain sources reconcile <id> <slug> --preview` for a file edited outside gbrain, or the line to fix. Held pages stay out of the preview hash and the approved set, so the rest of the window applies.+- A page that starts refusing after the preview is declined with the new `label_edit_refused` code and nothing written. It never counts toward `item_failures_systemic`, the run exits 0, and the receipt names the pages that applied.+- A brain without a repository writes the label to the database only, as before.++<!-- behavior-change: label_repairs_hold_refusals_gst2603 -->diff --git a/docs/architecture/key-files/commands-5.md b/docs/architecture/key-files/commands-5.mdindex 77116fc7b5..9486341335 100644--- a/docs/architecture/key-files/commands-5.md+++ b/docs/architecture/key-files/commands-5.md@@ -3,7 +3,7 @@[Subsystem index](../KEY_FILES.md). Read only the entries relevant to your change.Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.-- `src/commands/repair.ts` — `gbrain repair [<kind>] [--apply] [--source <id>] [--limit <n>] [--no-embed] [--json]` and `gbrain repair --all`. Kinds `timeline` (#5567 materialize), `visibility` (#5525 tighten-only backfill) `safe-chunks` (#5050/#5247 projection-only re-seal of pages of every kind below the safe-chunk fence; `--no-embed` defers embedding to `embed --stale`), `contextual-mode` (#5621 projection-only stamp of the import-time contextual retrieval mode on pages that have none; honors `--no-embed`) `connector-checkpoints` (#5686 brain-wide cleanup of connector checkpoint rows no source can load), `take-supersession` (#5886 relinks takes supersession chains from journal, stored-pointer or forced fence-structure evidence; ambiguous pages are reported with the manual edit, never rewritten; `src/core/repair/take-supersession.ts`), `orphan-bindings` (#5732 brain-wide cleanup of persistence source bindings whose source or incarnation is gone) and `embedding-effects` (#5629/#5734 settlement of stale queued and failed embedding effects: reconciled, superseded, retry_queued or blocked, reported per item in `outcomes` / `outcome_items`) and `google-file-modes` (clears group/other bits on gbrain's own files under Google source dirs outside `~/.gbrain`; filesystem only); no kind previews every kind. Dry run unless `--apply` (no prompt; `--yes` refused). `gbrain repair fences [--only <path>]... [--skip <path>]... [--slug <slug>]... [--diff] [--no-llm] [--max-usd <n>] [--apply [--expect <hash>]]` repairs malformed facts and takes fences (#6188, `src/core/repair/fences.ts`). `parseRepairArgs` is strict: any unknown option or a second positional refuses `invalid_params` before anything runs; flags a kind does not take name the kinds that do (`--only`/`--skip`/`--diff`: frontmatter, fences; `--slug`: fences; `--yes`: frontmatter); `--expect` is accepted for explicit-only and preview-bound kinds, `--include-ambiguous` for explicit-only kinds; `--max-usd <n>` and `--no-llm` only for `spends: 'llm'` kinds (a non-negative USD amount that only lowers the kind's cap), and on any other kind `--max-usd` names the capped route `gbrain doctor --remediate --yes --include-repairs --max-usd <n> --expect <plan_hash>` (hash from `doctor --remediation-plan --json`), as `--max-cost` always does. Help lines come from `REPAIR_REGISTRY`; `--json` adds `paid` per kind and `paid_kinds`. An apply with a report hook prints `repaired N; still waiting: <reason>=<count>` (never a repair count it did not make) and a partial scan; a stop prints its `fix` command and the consent it asks for. Output carries the resolved scope, affected count, a capped `source:slug` sample, residual counts, request-ID / receipt-byte / re-embedding cost (plus, for a `spends: 'llm'` kind, the paid-model estimate or spend and what is left under today's cap) and capacity, and the apply command; `--json` has the same fields. The dry run lists the kinds that may queue paid embeddings and, when a result carries `cost.llm_usd`, `Kinds that may call a paid model` with the summed estimate and the cap left. Host-only: thin clients refuse via `THIN_CLIENT_REFUSED_COMMANDS`. Logic lives in `src/core/repair/`. Segmented pages (Wave B) are not repaired by the whole-body kinds: `timeline`, `timeline-comments`, `connector-fences` (timeline region searched with `segmentedRegionContains`), `split-facts` (manifest facts regions) and `conversation-labels` report them as `segmented_*` residuals with their slugs (`test/large-pages/segmented-repair-scans.test.ts`). An item whose apply fails item-scoped (any failure `classifyItemFailure` does not call run-scoped, a superset of `REPAIR_ITEM_FAILURE_CODES` in `src/core/repair/core.ts`) is a per-item verdict: `runRepair` records it in `result.failed[]` (`item`, `source_id`, `slug`, `reason` the classified code, redacted `message`, `retry: 'next_full_scan'`, `fix` the kind's read-only preview for the source) plus a `failed` outcome, writes no cursor for it and continues; a later success moves the cursor past it, so a resumed run skips it and the next full scan retries it (the `revision_conflict` rule). The command prints `failed N`, each `FAILED <item> (<reason>)` line and the full-scan command, and `repairExitVerdict` exits 1 on any failure or stop. `runRepairKinds` is the kind loop of the no-kind preview and `--all`: a kind whose plan or apply throws item-scoped is listed as a result stopped with reason `error`, its redacted message and the kind's preview as `fix` (`kindErrorResult`, `src/core/remediation/repairs.ts`), and the later kinds still run; a run-scoped error, or most kinds failing (`item_failures_systemic`, ItemRun site `repair.all_kinds`), stops the command; a named kind's error is the command's error. Consumers never read failures as an all-clear: `runRepairSteps` (`RepairStepResult.failed`, status `failed`), the content lane (`ContentLaneResult.failed[]`, exit 1), and the `fence_repair` / `content_repair` phases (`warn`). Test: `test/repair-apply-continue-6459.test.ts`. `gbrain repair stuck-recovery` takes `--request-id <uuid>` and, with `--apply --expect <hash> --yes`, `--keep prior|disk` (both refused for other kinds, the value checked once the kind is known); the consent argv carries them. `gbrain repair ghost-pages --source <id>` (#6696 B5) accepts `--yes` with `--apply --expect <hash>` (destructive consent; `src/core/repair/ghost-pages.ts`). `gbrain repair file-drift` also takes `--only`/`--skip` (canonical file path globs, passed to the kind like the other path kinds'), and `--keep` on it refuses `invalid_params` with the same command rewritten to `--prefer database` as a read-only preview fix (`fileDriftKeepRefusal`).+- `src/commands/repair.ts` — `gbrain repair [<kind>] [--apply] [--source <id>] [--limit <n>] [--no-embed] [--json]` and `gbrain repair --all`. Kinds `timeline` (#5567 materialize), `visibility` (#5525 tighten-only backfill) `safe-chunks` (#5050/#5247 projection-only re-seal of pages of every kind below the safe-chunk fence; `--no-embed` defers embedding to `embed --stale`), `contextual-mode` (#5621 projection-only stamp of the import-time contextual retrieval mode on pages that have none; honors `--no-embed`) `connector-checkpoints` (#5686 brain-wide cleanup of connector checkpoint rows no source can load), `take-supersession` (#5886 relinks takes supersession chains from journal, stored-pointer or forced fence-structure evidence; ambiguous pages are reported with the manual edit, never rewritten; `src/core/repair/take-supersession.ts`), `orphan-bindings` (#5732 brain-wide cleanup of persistence source bindings whose source or incarnation is gone) and `embedding-effects` (#5629/#5734 settlement of stale queued and failed embedding effects: reconciled, superseded, retry_queued or blocked, reported per item in `outcomes` / `outcome_items`) and `google-file-modes` (clears group/other bits on gbrain's own files under Google source dirs outside `~/.gbrain`; filesystem only); no kind previews every kind. Dry run unless `--apply` (no prompt; `--yes` refused). `gbrain repair fences [--only <path>]... [--skip <path>]... [--slug <slug>]... [--diff] [--no-llm] [--max-usd <n>] [--apply [--expect <hash>]]` repairs malformed facts and takes fences (#6188, `src/core/repair/fences.ts`). `parseRepairArgs` is strict: any unknown option or a second positional refuses `invalid_params` before anything runs; flags a kind does not take name the kinds that do (`--only`/`--skip`/`--diff`: frontmatter, fences; `--slug`: fences; `--yes`: frontmatter); `--expect` is accepted for explicit-only and preview-bound kinds, `--include-ambiguous` for explicit-only kinds; `--max-usd <n>` and `--no-llm` only for `spends: 'llm'` kinds (a non-negative USD amount that only lowers the kind's cap), and on any other kind `--max-usd` names the capped route `gbrain doctor --remediate --yes --include-repairs --max-usd <n> --expect <plan_hash>` (hash from `doctor --remediation-plan --json`), as `--max-cost` always does. Help lines come from `REPAIR_REGISTRY`; `--json` adds `paid` per kind and `paid_kinds`. An apply with a report hook prints `repaired N; still waiting: <reason>=<count>` (never a repair count it did not make) and a partial scan; a stop prints its `fix` command and the consent it asks for. Output carries the resolved scope, affected count, a capped `source:slug` sample, residual counts, request-ID / receipt-byte / re-embedding cost (plus, for a `spends: 'llm'` kind, the paid-model estimate or spend and what is left under today's cap) and capacity, and the apply command; `--json` has the same fields. The dry run lists the kinds that may queue paid embeddings and, when a result carries `cost.llm_usd`, `Kinds that may call a paid model` with the summed estimate and the cap left. Host-only: thin clients refuse via `THIN_CLIENT_REFUSED_COMMANDS`. Logic lives in `src/core/repair/`. Segmented pages (Wave B) are not repaired by the whole-body kinds: `timeline`, `timeline-comments`, `connector-fences` (timeline region searched with `segmentedRegionContains`), `split-facts` (manifest facts regions) and `conversation-labels` report them as `segmented_*` residuals with their slugs (`test/large-pages/segmented-repair-scans.test.ts`). An item whose apply fails item-scoped (any failure `classifyItemFailure` does not call run-scoped, a superset of `REPAIR_ITEM_FAILURE_CODES` in `src/core/repair/core.ts`) is a per-item verdict: `runRepair` records it in `result.failed[]` (`item`, `source_id`, `slug`, `reason` the classified code, redacted `message`, `retry: 'next_full_scan'`, `fix` the kind's read-only preview for the source) plus a `failed` outcome, writes no cursor for it and continues; a later success moves the cursor past it, so a resumed run skips it and the next full scan retries it (the `revision_conflict` rule). The command prints `failed N`, each `FAILED <item> (<reason>)` line and the full-scan command, and `repairExitVerdict` exits 1 on any failure or stop. `runRepairKinds` is the kind loop of the no-kind preview and `--all`: a kind whose plan or apply throws item-scoped is listed as a result stopped with reason `error`, its redacted message and the kind's preview as `fix` (`kindErrorResult`, `src/core/remediation/repairs.ts`), and the later kinds still run; a run-scoped error, or most kinds failing (`item_failures_systemic`, ItemRun site `repair.all_kinds`), stops the command; a named kind's error is the command's error. Consumers never read failures as an all-clear: `runRepairSteps` (`RepairStepResult.failed`, status `failed`), the content lane (`ContentLaneResult.failed[]`, exit 1), and the `fence_repair` / `content_repair` phases (`warn`). Test: `test/repair-apply-continue-6459.test.ts`. A `label_edit_refused` item (GST-2603, a label repair whose file cannot take the line edit) counts as declined (`REPAIR_DECLINE_CODES`), exits 0 and never counts toward `item_failures_systemic`. `gbrain repair stuck-recovery` takes `--request-id <uuid>` and, with `--apply --expect <hash> --yes`, `--keep prior|disk` (both refused for other kinds, the value checked once the kind is known); the consent argv carries them. `gbrain repair ghost-pages --source <id>` (#6696 B5) accepts `--yes` with `--apply --expect <hash>` (destructive consent; `src/core/repair/ghost-pages.ts`). `gbrain repair file-drift` also takes `--only`/`--skip` (canonical file path globs, passed to the kind like the other path kinds'), and `--keep` on it refuses `invalid_params` with the same command rewritten to `--prefer database` as a read-only preview fix (`fileDriftKeepRefusal`).- `src/commands/repair-content.ts` — `gbrain repair content [--source <id>] [--only <path>]... [--skip <path>]... [--no-llm] [--max-usd <n>] [--diff] [--apply [--expect <h1>[,<h2>]]] [--json]` (#6377): the content-repair lane as one command. `content` is parsed by `repair.ts`'s strict flag grammar like a kind (`CONTENT_LANE`; `--only`/`--skip`/`--diff`, `--no-llm`, `--max-usd` and `--expect` are accepted for it, `--slug`, `--yes`, `--include-ambiguous` and `--all` refuse) and dispatched before the kind check; it is a lane, not a `RepairKind`: no registry entry, no cursor of its own, listed under "Lanes:" in `REPAIR_HELP`. `runRepairContentCommand` resolves the scope, runs `runContentLane` and prints each kind's result through `renderRepairResult` plus a lane line (spend, allowance left, each kind's hash, every stop, the apply command); `--json` prints `{ lane, scope, mode, kinds, results[] (with `paid`), preview_hashes, cost, stopped, paid_kinds, apply_command }`. A kind's stop sets exit verdict 1; a wrong `--expect` count is `invalid_params` with the preview as its fix. Test: `test/repair-content.test.ts`.- `src/commands/reindex.ts` — `gbrain reindex --markdown [--type PAGE_TYPE] [--limit N] [--workers N] [--dry-run] [--json] [--no-embed] [--yes] [--max-usd USD] [--repo PATH]`. W4.5: re-embedding is paid, so `runReindex(engine, args, { authorized?, interactive? })` calls `requireReindexConsent` (`src/core/reindex-consent.ts`) after the dry-run return and before the managed dispatch for every caller unless `--no-embed` or `authorized` (the upgrade path's TTY yes, the job handler's stored spend record); `src/cli/commands/reindex.ts` renders the exit-3 consent refusal and `cost_cap_exceeded`. The approved finite cap runs the inline pass (and the managed dispatch) under a `BudgetTracker` via `withBudgetTracker`; once it is exhausted (an import swallows the refused embedding) the loop stops at the next page and `reportBudgetStop` exits 1 with `budget_exhausted` in `--json`. `countPending` is exported for `jobs submit reindex` and the mode-switch prompt. The writer pool is sized once from the run's page target (pending, capped by `--limit`) through `resolveWorkersWithClamp`, so Postgres runs over 100 pages default to `DEFAULT_PARALLEL_WORKERS` while PGLite stays at one; `--workers` overrides and the used count is reported as `workers` in the result and JSON (pinned by `test/e2e/reindex-auto-workers-5181.test.ts`). Walks markdown pages with stale `chunker_version` (plus unstamped contextual-retrieval mode when embedding is enabled; `--no-embed` still stamps the mode of every page it re-chunks) in 100-row id-keyset batches; a failed row cannot starve later batches in the same invocation. `--type` adds a bound-parameter `pages.type = $N` scope for focused backfills such as atom pages and is rejected by reindex modes that do not consume it. Current-version chunkless healing deliberately remains owned by the native, bounded `embed --stale` path. Rows with non-null `source_path` re-import via `importFromFile`; rows without fall back to `importFromContent`. Both paths pass `forceRechunk: true` to bypass `importFromContent`'s `content_hash` short-circuit — without it the chunker version bump never reaches pages whose source content hasn't changed, AND the stripFactsFence privacy strip never applies to chunks stored without it. Wired into `src/commands/upgrade.ts:runPostUpgrade` after `apply-migrations`. The DB-only fallback (no source file on disk) does NOT pass body-only `compiled_truth` to `importFromContent` (that path re-parses with EMPTY frontmatter and OVERWRITES the page's real frontmatter/title/timeline); it `getPage`+`getTags`, reconstructs FULL markdown via `serializeMarkdown(frontmatter, compiled_truth, timeline, {type, title, tags})`, and re-imports THAT so re-chunking a DB-only page preserves everything while bumping `chunker_version`. Pinned by `test/reindex-preserve-tags.test.ts` and `test/reindex.test.ts`. #5377: on a managed brain (`managedPersistenceEnabled`) the `--markdown` sweep delegates to the projection repairs through `runRepair` (`safe-chunks`, then `contextual-mode` unless `--no-embed`), so it never takes the legacy writer and changes no page revision, journal request or canonical file; `--type` is refused there with the repair commands, and pages the repairs cannot rebuild are reported as `unsupported_pages`; a page the repairs held (`projection_unbuildable`) or failed per item is counted in `failed` (`--json` adds `failures`, exit 1) instead of failing the command. The doctor, upgrade and post-upgrade hints that name `gbrain reindex --markdown` therefore work on both brain modes (C-NEW-5).diff --git a/docs/architecture/key-files/core-persistence-continued.md b/docs/architecture/key-files/core-persistence-continued.mdindex 13e048b4ad..4a0ca51f9f 100644--- a/docs/architecture/key-files/core-persistence-continued.md+++ b/docs/architecture/key-files/core-persistence-continued.md@@ -43,5 +43,5 @@ The #6188 fence check and fence repair live in [fence repair](core-fence-repair.- `src/core/postgres-engine/reserved-transactions.ts` with `isPersistenceDeclaration` (`src/core/persistence/protocol.ts`) — BEGIN pipelining. `isPersistenceDeclaration` (GBRA-156) accepts exactly the `protocol.ts` declarations (transaction-local `set_config` calls and the brain row FOR SHARE). Admission, the FIFO claim, the effect claims and source guards, the receipt true-up, the single publication (`coordinator.ts`) and the consumer's lent lane transactions (`consumer.ts` `onLane`, through `reservedTransactions`) pass it as `TransactionOptions.pipelineFirst`, so behind a transaction-mode pooler the declaration goes out with BEGIN (one round trip less per transaction) while every later statement still waits for BEGIN's answer; if BEGIN fails, only the declaration has run, in autocommit, where it changes nothing. A reserved transaction gates every statement its body can reach, savepoint and nested-transaction bodies and `file()` included. A sync lane group's publication passes none: its begun mark must follow BEGIN's answer (`sync-lanes.ts` `awaitLaneBegin`). Tests: `test/persistence-pipeline-first.test.ts` (the predicate and the reviewed call sites), `test/e2e/pgbouncer-transaction-pipeline-6930.test.ts`.- `src/core/links-fence.ts` + `src/core/links-fence-projection.ts` — the `## Links` fence (G5): the markdown form of authored (manual) edges, `<!--- gbrain:links:begin -->` / `end` around `| to | type | source | valid_from | valid_until | context |`, one row per (to, type, source), source defaulting to `manual`; a reconciliation-managed provenance (`MANAGED_LINK_SOURCES`, exported here and re-exported by `ops/links.ts`), a bad date or a duplicate key is a skipped row with a warning. `parseLinksFence` (state `absent` / `parsed` / `malformed`), the deterministic `renderLinksFence` (rows sorted by key), `upsertLinksFenceRow` (omitted context and dates keep the stored ones; identical row = unchanged body), `removeLinksFenceRows` (leaves an empty fence), `replaceOrInsertLinksFence` (a `## Links` section at the end of compiled_truth, above an unclosed code block) and `maskLinksFence`, which `extractBodyLinks` applies so the fence cells are never also extracted as markdown, wikilink or bare-slug edges. A rewrite of a fence that parses with warnings throws `LinksFenceRewriteRefusal`. Cells use `escapeFenceCell` plus the same one-level marker encoding for `gbrain:links:*`. `projectLinksFence(tx, page, rows)` reconciles only what the page's fence owns: edges with origin_page_id = the page and origin_field `links_fence` (deleted, context-updated, inserted by key) and `manual` transitions with origin_page_id = the page, compared per row by (to, type, link_source, kind, date) and written with the row's source in `link_source`, then refreshes the touched relationships; a row whose target is not a live page in the source yields no edge. `linksFenceProjectionRows` returns null for an absent or malformed fence, so such a body changes no edge (an absent fence retires nothing, as for facts; unlike facts no held generation is recorded, so after a hand deletion of the whole section the database keeps edges git no longer has until `remove_link` or the G3 indexer), and drops rows typed with a link type the active pack does not declare. Callers: `prepareCanonicalProjections` (every projected page publication, quarantined and segmented-unread bodies excepted), `prepareAutomaticLinks` (put_page auto-links and the managed links maintenance, so `extract --stale` restores a lost fence edge) and `persistence/manual-links.ts`. Database-only manual edges (origin_field NULL) are never touched by any of them. Pinned by `test/links-fence.test.ts` and `test/managed-links-fence.test.ts`.- `src/core/persistence/source-lifecycle.ts` retire batches with `topology-transaction.ts` (#7702/#8905) — every `topologyTransaction` runs under `statement_timeout=5s` and `lock_timeout=1s` and maps SQLSTATE 57014/55P03 to `write_pending`, so a managed `remove` or `purge` first deletes the source's pages in batches of `SOURCE_RETIRE_BATCH_PAGES` (500; about 110 ms per batch with chunks and links on Postgres) under the native locks `withTopologyLocks` holds, each batch its own `topologyTransaction`. A batch re-checks the final transaction's preconditions (admin state, `lockTopologyRows`, the incarnation read before the locks, archived or `EXPIRED_ARCHIVE_SQL` due, no prior receipt other than this request's own admission, no `oauth_clients` reference, `assertTopologyRequestsSettled` from `topology-locks.ts`, `lockTopologyPrincipal`, and on the first batch `admitTopologyRetire` from `topology-receipts.ts`, which checks the receipt quotas at the largest receipt and records the admission) and, when one would refuse or replay, stops without deleting so the final transaction reports it, or throws that refusal before deleting; a batch that fails `write_pending` is retried at half the size (floor 1), so heavy pages still fit the bound; a refusal after some batches committed says how many pages that run already deleted; a dry run, a missing checkout, an unconfirmed operation and an admission-hooked call never batch. The final transaction's `sources` delete runs `removedSourceScrubCtes` (#9068: the source's retrieval and volunteer rows, and its slugs in eval candidates), so a run that stops after some batches leaves those rows until a retry commits. Each batch deletes inside `withCoordinatedWrite` with local-CLI attribution (the `pages` guard needs the source in `gbrain.write_sources`; only the final `sources` delete sets `gbrain.topology_change`). Committed batches are durable: a retry (same or new request_id) resumes, and the receipt's `pages_deleted` counts the pages deleted by the run that commits it. #8984: the first batch's transaction records the admission, a `persistence_topology_changes` row in state `recovering` with no recovery record, bound to the source incarnation it deletes from (`isRetireAdmission`); `recordTopologyChange` settles it `committed`, and a remove or purge that deletes that incarnation under another request, or any other lifecycle operation that commits a change to it (archive, restore, rebind, claim), settles it `failed` (`supersedeTopologyRetires`, which skips only the caller's own principal and request_id pair). A retry of the request_id against another incarnation (the source was removed and re-added) refuses `source_retire_superseded`, settles the admission `failed` and deletes nothing, also when a since-replaced CLI registration (another principal) admitted that request_id for a remove or purge, since idempotency is per principal; an admission bound to the current incarnation still resumes under the new registration; deactivation and graduation list an unsettled admission as a blocker that names its request_id and asks the user whether to finish it or keep what is left; an automatic expired-archive purge's admission (`expired_only` in its outcome) is left to the next automatic run with `needsUser: false`, and `sources writer status --json` lists the stopped and superseded requests (`topology_requests`). Tests: `test/source-retire-batched.test.ts` (PGLite), `test/e2e/source-retire-batched-postgres.test.ts` (60,000 pages), `test/source-retire-incarnation.test.ts` and `test/e2e/source-retire-incarnation-postgres.test.ts`.-- `src/core/persistence/label-edit.ts` + `src/core/repair/label-write.ts` (GST-2468) — the label-only page write of operator repairs (`repair visibility`, `repair private-labels`). `label-edit.ts`: `LABEL_KEYS` (visibility, access, confidentiality), `readLabelEdit` validates a `managed_maintenance_page` intent's `label_edit` (`set` plain-word values, `remove` keys), `spliceLabelLines` removes, replaces or appends label lines in a file's frontmatter text and declines (null) a block or multi-line value, a duplicate key or a file with no frontmatter block. `page-prepare.ts` `labelEditFile` uses it in `compareFileTarget` when the file matches the snapshot: the edited file (or its edited head over the write's body, for a fence row) must read back as the file the write would render, else the write refuses `invalid_params`; the file is never re-rendered under a label edit. `label-write.ts`: `writeLabelPage` publishes through the maintenance door with `keep_trust` and the edit on a managed brain, else `importAtRevision` (maintenance transaction, page lock, `withTrustKeep`); `filesInCheckout` names the unmanaged pages whose checkout file would undo a database-only write; for those, `writeLabelFile` applies `labelEditFile` to the file's own bytes (it must still hold the page, `ownerRepairFile`), imports it under `withTrustKeep` and writes it back through `writePageThrough` (`fileText`, preimage checked) in one transaction; drift or a file edited mid-write throws `source_changed`, and a file it cannot write (`sync.write_through` off, `url_clone`, an unresolvable target, a write error) returns a held `file_backed` outcome with the import rolled back. `repair visibility` calls it directly; `private-labels` still holds those pages as `file_backed`. GST-2476: `label-edit.ts` also edits `readable_by` (`READERS_KEY`, `EDIT_KEYS`; `set` takes a plain-word list, rendered `readable_by: [a, b]`), and `writeLabelPage` takes a maintenance `batch` (then `publishMaintenancePages([page], {batch})`). `src/core/repair/readers.ts` is `repair visibility --readers`: `checkedReaders` (members exist and are enabled, groups exist, else `access_member_invalid`; a list that leaves out the owner refuses unless `allowWithoutOwner`), `planReaders` (named pages or a `--slugs-file`, up to `READERS_MAX_PAGES`, approved set bound to each page's live readers) `readersWrite` (re-checks a page, holds a segmented one, renders the edit) and `applyReaders` (phase `READERS_PHASE`, one label-only write); `visibility.ts` `applyBatch` admits a managed source's run of readers items together through `publishMaintenancePages` under the run's batch (Postgres publishes them in groups of up to 8), so the Git effect commits a window's ready files together; `readersGitCounts` reports the run's Git effect states. `visibility.ts` `labelWrite` holds a managed page with no canonical file outside `storage.db_only` as `canonical_file_missing`. GST-2519, labels under `storage.db_only`: `sync-classify.ts` `gitVerdict` keeps a strict source's `db_ahead` path when `state.dbOnly` (a label repair on a page whose file Git still tracks there survives later syncs; a commit that moved still imports); `sync-prepare.ts` imports a db_only file over a live page through `withStoredLabels(text, stored, incoming)`, which adds each `EDIT_KEYS` label the row holds and the file does not state, so an import there never widens (a label the file states applies, wider included); `page-prepare.ts` `prepareFileTarget` lets a label repair of a strict page stamped `git_ignored`, with no recorded file and a path Git still ignores, write the database only instead of holding it `canonical_file_missing`. Tests: `test/db-only-labels-gst2519.test.ts`, `test/sync-classify-strict-g3.test.ts`.+- `src/core/persistence/label-edit.ts` + `src/core/repair/label-write.ts` (GST-2468) — the label-only page write of operator repairs (`repair visibility`, `repair private-labels`). `label-edit.ts`: `LABEL_KEYS` (visibility, access, confidentiality), `readLabelEdit` validates a `managed_maintenance_page` intent's `label_edit` (`set` plain-word values, `remove` keys), `spliceLabelLines` removes, replaces or appends label lines in a file's frontmatter text and declines (null) a block or multi-line value, a duplicate key or a file with no frontmatter block. `page-prepare.ts` `labelEditFile` uses it in `compareFileTarget` when the file matches the snapshot: the edited file (or its edited head over the write's body, for a fence row) must read back as the file the write would render, else the write refuses `label_edit_refused` (GST-2603; reason `file_differs` with the differing keys in `detail`, or `label_line`); the file is never re-rendered under a label edit. GST-2603: `labelEditFit` (also used by `writeLabelFile`) aligns the file with the stored page before comparing, and `align-stored.ts` `keepStoredOnlyStamps` (used by `alignWithStored` and sync's `keepStoredStamps`) keeps the stamps gbrain holds only in the database (the `validate: false` grandfather stamp and a stored `legacy_type` whose aligned type is the page's), so they never count as a difference; `fileSnapshotDifference` lists the differing keys (frontmatter keys, title, type, tags, body, timeline) for the raw and inferred reading and `fileMatchesSnapshot` is its empty case. `label-write.ts` `labelFileRefusal` is a write-free preflight (managed: `prepareFileTarget`, with the drift's keys from `error.localEdit` and `gbrain sources reconcile <id> <slug> --preview` as next step; unmanaged: `ownerRepairFile`, next step `gbrain sync --source <id>`; repo-less or no file: null); `visibility.ts` holds refusing pages in the preview (`details.held`, `residuals.held_file_drift` / `held_label_edit`, outside the hash and approved set) and `labelWrite` / `private-labels.ts` `prepareWrite` throw only its `label_edit_refused` before writing. `repair/core.ts` counts `label_edit_refused` as declined and never toward `item_failures_systemic`. Test: `test/label-repair-derived-stamps-gst2603.test.ts`. `label-write.ts`: `writeLabelPage` publishes through the maintenance door with `keep_trust` and the edit on a managed brain, else `importAtRevision` (maintenance transaction, page lock, `withTrustKeep`); `filesInCheckout` names the unmanaged pages whose checkout file would undo a database-only write; for those, `writeLabelFile` applies `labelEditFile` to the file's own bytes (it must still hold the page, `ownerRepairFile`), imports it under `withTrustKeep` and writes it back through `writePageThrough` (`fileText`, preimage checked) in one transaction; drift or a file edited mid-write throws `source_changed`, and a file it cannot write (`sync.write_through` off, `url_clone`, an unresolvable target, a write error) returns a held `file_backed` outcome with the import rolled back. `repair visibility` calls it directly; `private-labels` still holds those pages as `file_backed`. GST-2476: `label-edit.ts` also edits `readable_by` (`READERS_KEY`, `EDIT_KEYS`; `set` takes a plain-word list, rendered `readable_by: [a, b]`), and `writeLabelPage` takes a maintenance `batch` (then `publishMaintenancePages([page], {batch})`). `src/core/repair/readers.ts` is `repair visibility --readers`: `checkedReaders` (members exist and are enabled, groups exist, else `access_member_invalid`; a list that leaves out the owner refuses unless `allowWithoutOwner`), `planReaders` (named pages or a `--slugs-file`, up to `READERS_MAX_PAGES`, approved set bound to each page's live readers) `readersWrite` (re-checks a page, holds a segmented one, renders the edit) and `applyReaders` (phase `READERS_PHASE`, one label-only write); `visibility.ts` `applyBatch` admits a managed source's run of readers items together through `publishMaintenancePages` under the run's batch (Postgres publishes them in groups of up to 8), so the Git effect commits a window's ready files together; `readersGitCounts` reports the run's Git effect states. `visibility.ts` `labelWrite` holds a managed page with no canonical file outside `storage.db_only` as `canonical_file_missing`. GST-2519, labels under `storage.db_only`: `sync-classify.ts` `gitVerdict` keeps a strict source's `db_ahead` path when `state.dbOnly` (a label repair on a page whose file Git still tracks there survives later syncs; a commit that moved still imports); `sync-prepare.ts` imports a db_only file over a live page through `withStoredLabels(text, stored, incoming)`, which adds each `EDIT_KEYS` label the row holds and the file does not state, so an import there never widens (a label the file states applies, wider included); `page-prepare.ts` `prepareFileTarget` lets a label repair of a strict page stamped `git_ignored`, with no recorded file and a path Git still ignores, write the database only instead of holding it `canonical_file_missing`. Tests: `test/db-only-labels-gst2519.test.ts`, `test/sync-classify-strict-g3.test.ts`.- `src/core/persistence/page-edit.ts` — `edit_page` (#5616): splits the canonical serialization into editable text and protected takes/facts segments whose remote view matches get_page's sanitized `content` byte for byte; `applyPageEdits` applies ordered exact-once replacements inside editable text only (typed `edit_no_match`/`edit_ambiguous_match`/`edit_protected_span`/`edit_invalid`), so canonical fences never move; `editDiff` caps the caller-view diff at 8 KB; `boundedUnifiedDiff` keeps its LCS table under `EDIT_PAGE_DIFF_CELL_BUDGET` (4M cells) by diffing only the window between common leading/trailing lines, and a window still over budget becomes one coarse hunk flagged `diff_coarse` (pinned by `test/large-pages/edit-diff-bounded.test.ts`). Called at submission (`page-mutations.ts`, before admission; skipped for a segmented page) and again on the locked snapshot in `page-prepare.ts`, which gives edit_page put_page's revision-bound `editing` timeline semantics. Pinned by `test/edit-page.test.ts`. `within` and the segmented path: [large page writes](core-services-2-continued.md#large-page-writes).diff --git a/docs/guides/repair.md b/docs/guides/repair.mdindex f9a75b40fc..519d675a32 100644--- a/docs/guides/repair.md+++ b/docs/guides/repair.md@@ -87,9 +87,20 @@ gbrain repair visibility --slug notes/a-example --readers none # remove readabEvery visibility write is label-only: the page keeps its trust tier, files no`lower_page` proposal, and its file changes only on the `visibility` line (a fact-item: its fence row). A file that cannot take the change without being-re-rendered (a label with a block value or a duplicate key, no frontmatter-block) is refused with nothing written; make that edit by hand. `--slug` is the operator's way to make one page private;+item: its fence row). Stamps gbrain keeps only in the database (a type a schema+pack renamed, the `validate: false` grandfather stamp, a `legacy_type` retype+record) do not count as a difference: such a page is labelled and its file+still gains only the label line (GST-2603). A page whose file cannot take the+change line by line (a label with a block value or a duplicate key, no+frontmatter block, or a file that differs from the page in a real key such as a+date, the tags, the title or the body) is held by the preview: it is listed+under `details.held` with the key that blocks it and its next step+(`gbrain sources reconcile <id> <slug> --preview` for a file edited outside+gbrain, the field to fix for a label line), and it stays out of the hash and+the approved set. A page that starts refusing after the preview is declined+with [`label_edit_refused`](write-refusals.md#label_edit_refused) and nothing+written; it never counts toward `item_failures_systemic`, so the rest of the+window applies and the receipt names the pages that did. `--slug` is the operator's way to make one page private;a `put_page` that adds the label is an agent edit and lowers the page. On a brainwithout managed persistence, a page whose file sits in its source's checkout getsthe same edit in that file and in the database together, so the next sync reads@@ -2294,7 +2305,10 @@ database, so a page whose file sits in its source's checkout is held withoutcome `file_backed`. The next sync or re-import would put the label back fromthe file, so edit the file's frontmatter, commit it, and run`gbrain sync --source <id>`. Repo-less pages clear as usual. A clear or restore-changes only the label lines of the page's file and keeps its trust tier; a+changes only the label lines of the page's file and keeps its trust tier, and+gbrain's own database-only stamps never block it; a file that cannot take the+edit line by line is declined with `label_edit_refused`, as in+`repair visibility`; arestore whose label value isn't a plain word is held as `label_not_plain`, soadd that label to the file by hand.diff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.mdindex 234e10362b..5efd781a19 100644--- a/docs/guides/write-refusals.md+++ b/docs/guides/write-refusals.md@@ -53,6 +53,7 @@ of making a duplicate.| --- | --- | --- | --- || <a id="file_database_drift"></a>`file_database_drift` | `source_changed` | The page's canonical file and its database copy disagree, usually because the file was edited outside a coordinated write. Neither copy was overwritten. The receipt records the cause as `error_detail.origin: file_database_drift` (GST-1601). A file with no `type:` line keeps the stored type, and titles compare trimmed, so those alone do not cause this. Drift only in ordinary frontmatter keys (not a policy, privacy, lineage or trust key such as `readable_by`, `derived_from` or `trust_tier`, nor title, type or tags) does not refuse a write that leaves the frontmatter unchanged, such as `remember`: the write lands and the file keeps its own frontmatter, so that drift is left for reconcile (#8109). With `gbrain config set persistence.adopt_local_edits.source.<source> true` (off by default), a `remember`, `capture` or `add_timeline_entry` that meets a one-sided local edit (Git `HEAD` still holds the database copy; on a repo-less brain, any edit) is not refused: once the file has been unchanged for `persistence.adopt_quiet_ms` (default 5000), the owner commits that file on its own (`gbrain: adopt local edit to <path>`, trailers `gbrain-adopted:` and `gbrain-author: unknown`; page history reason `adopted_local_edit`), then applies the write on top, and the receipt names `adopted`. A two-sided edit, a file that does not import, an edit to the title, type or a protected key, `put_page` and every other verb still refuse; doctor `local_edit_adoptions` lists adoptions of the last 24 h (GST-1601). The refusal also stays for a file over the page size limit or a page stored in segments, an unmerged path or a merge, cherry-pick, revert, rebase, `git bisect` (`BISECT_LOG`) or uncommitted `git merge --squash` (`SQUASH_MSG`) in progress, and a stale editor save (a file equal to an earlier stored revision, or missing a Facts or timeline row the last write added); the quiet wait never passes 30 s, since writes behind it on the same checkout wait meanwhile (#8755). It also stays when `git rev-parse --absolute-git-dir` answers with anything but one absolute, existing directory in valid UTF-8; a `git cherry-pick -n` leaves no marker, and its staged paths stay out of the adoption's commit (#8826). An adoption whose commit landed before its publication failed is published on its replay even after the write it was made for ended, and one that ends without committing after that commit names it on its receipt (`outcome.persistence.git_commit`, `landed_before_row: true`; #8824, #8767). | The receipt's fix (`gbrain sources writer failures <source>`, `gbrain write-request <request_id> --json`) previews keeping the database copy for that one page with `gbrain repair file-drift --source <source> --slug <slug> --prefer database` (`--prefer file` previews importing the file instead); the preview names any file edit or deletion sync never imported that the database copy would overwrite (in bulk such pages are kept, `kept_newer_file` / `kept_newer_delete`); a person picks the side, and the apply the preview prints asks for their consent (exit 3), taking `--yes` only after they agree ([newer file changes](repair.md#file-drift-newer)). A group of such receipts gets `--all-drifted --since <instant>` with the oldest receipt's time, so only pages that drifted since then are selected. For a hand merge: `gbrain sources reconcile <source> <slug> --brain <brain> --preview`, review the preview, resolve it, then `--apply <resolved-preview-file> --request-id <new uuid>`. Retry the original write with a **new** request ID. A page that records no file of its own (no `source_path`, typically written by an older release whose write-through also put `ingested_via` / `ingested_at` / `source_kind` in the file only) reconciles against the file at its slug path only when you add `--adopt-slug-path` to the preview; the refusal names the flag in that case, and apply records that file as the page's origin. See [repair a file/database disagreement](concurrent-writes.md#repair-a-filedatabase-disagreement). Many pages at once: `gbrain sources reconcile <source> --brain host --audit --conflicts` lists them and `gbrain repair file-drift` resolves them, preferring the file or the database copy ([file drift](repair.md#file-drift)). || <a id="canonical_file_missing"></a>`canonical_file_missing` | `source_changed` (a caller write; maintenance refuses before admission) | The page's recorded canonical file is gone while its database page is still live (a ghost page: the file was moved or deleted outside gbrain). A caller write refuses with "The canonical file was removed outside coordinated publication." and nothing is written; retrying it refuses the same way. A maintenance write (fact-fence adoption, enrichment, `dream`) is refused before admission: no request id, no receipt (before #6696 each cycle settled a `conflict` receipt). `gbrain sources writer failures <source>` reports such receipts as category `canonical_file_missing`. Neither copy was changed. | Decide which copy is right. A visibility edit cannot publish while the file is gone, so to keep the page and only narrow who reads it, add a page rule on the brain host: `gbrain access rule add --slug <slug> --readers <member> --source <source>` (it previews; `--yes` applies; list the owner's member to keep the owner reading; removing the rule undoes it; upgrade every gbrain host that shares the brain first). No file is written for the page. If the file should exist, restore it at its recorded path and preview the page with `gbrain sources reconcile <source> <slug> --brain host --preview`, or sync. If the page should go, preview the ghost-page repair with `gbrain repair ghost-pages --source <source>` (read-only; it prints the apply command with its hash) and apply it with `--apply --expect <hash> --yes` only after the user agrees; a soft-deleted page can be restored with `gbrain restore <slug> --force` until the autopilot purge 72 h later. `gbrain pages ghosts --source <source>` and doctor `ghost_pages` list them, doctor `maintenance_write_drift` lists held-back maintenance writes. See [ghost pages](repair.md#ghost-pages). |+| <a id="label_edit_refused"></a>`label_edit_refused` | `label_edit_refused` (reason `file_differs` or `label_line`) | An operator label repair (`gbrain repair visibility`, `gbrain repair private-labels`) left a page unchanged because its file holds the page but cannot take the label edit line by line (GST-2603). These repairs change only the label lines and never re-render a file, so a person's formatting and comments survive. `file_differs`: once its label lines are edited, the file still differs from the page in a real key, named in `detail` (a frontmatter key such as a date, the title, the type, the tags or the body); stamps gbrain keeps only in the database (a pack-renamed type, `validate: false`, `legacy_type`) never count. `label_line`: the label line has a block or multi-line value or a duplicate key, or the file has no frontmatter block. Nothing was written. The repair counts the page as declined, never toward `item_failures_systemic`, and the next preview holds it under `details.held` with its next step. | A file edited outside gbrain: preview it with `gbrain sources reconcile <source> <slug> --preview` (managed) or run `gbrain sync --source <source>` (unmanaged), then preview the repair again. A label line: write the label as one plain line (`visibility: private`), commit it and sync the source. || <a id="refusal_breaker"></a>`refusal_breaker` | the code of the earlier refusals | The same maintenance write (same source, slug, page revision, planner and intent) was refused `persistence.maintenance.refusal_breaker_threshold` times (default 3; `0` turns it off) in the last 24 hours, so maintenance submits nothing for it instead of adding another receipt (#6696). It is tried again once the page, the planner or the write changes, or when the oldest refusal is 24 hours old. | Read the last refusal with `gbrain write-request -- <request_id>` (named in the message) and fix what it names. Doctor `maintenance_write_drift` lists every held-back write with its exit. || <a id="canonical_file_absent"></a>`canonical_file_absent` | `source_changed` (`restore_page` / `gbrain restore`) | On a brain without managed persistence, a restore that writes no file (`sync.write_through` off, a sandbox, a database-only source) of a soft-deleted page whose recorded file is no longer among the files sync imports (#6609). A plain restore would be undone by the next `gbrain sync --full`, so nothing was restored. Not retryable; the fix asks the user. | Ask the user. To keep the page with its file: put the file back at its recorded path, commit it and `gbrain sync --source <source>`. To keep it database-only: `gbrain restore <slug> --source-id <source> --detach` (MCP `restore_page` with `detach: true`), which forgets the recorded file and stamps `database_only_reason='detached'`. See [restoring a page whose file is gone](../storage-tiering.md#restoring-a-page-whose-file-is-gone). || <a id="pending_import"></a>`pending_import` | `source_changed` (detail `file_database_drift`, reason `pending_import`) | The page's canonical file holds a change committed through Git (another agent's commit, a pull) that managed sync has not imported yet: the file on disk is exactly the blob `HEAD` holds for its path and not the one the source's last imported commit holds. The message is "The canonical file holds a committed Git change that sync has not imported yet." Nothing was overwritten. Since #6693 the receipt records origin `pending_import`; a write refused because sync holds the file records origin `sync_held` (see [held files](#held-files-and-content-refusals)). | A `remember` needs nothing from the caller: its `fix.next` is `wait`, the fact is kept in the receipt, and the first managed sync that imports the file (any outcome, a partial run included) makes one replay attempt on the original caller's lane with the fact's original `valid_from` and expiry (the sync result's `replayedRemembers` counts examined, replayed, waiting, skipped, remaining; at most 50 per run and 30 s). Do not save the fact again. A remember refused while sync held its file is replayed the same way once the repaired file is imported. A replay that is refused (authority revoked, page rewritten) stays listed by `gbrain repair failed-writes --source <source>` for a person. Any other write (`put_page`, `add_timeline_entry`) is never replayed automatically, because its body would replace the imported change: wait for the source's running sync owner (`gbrain serve`, autopilot or a sync job) to import the commit, which the refusal's read-only fix `gbrain sync status --source <source> --json` shows, then read the page again and resubmit with a new request ID. Neither refusal tells the caller to start a sync, because a second sync beside a running owner competes with it; when status shows no owner running, ask the user to sync the source on its host. An uncommitted hand edit still refuses `file_database_drift` and needs reconcile. With `persistence.refresh_barrier` on, a `put_page`, `capture`, `remember` or `add_timeline_entry` waits for the import instead of being refused ([deferred writes](#pending_import_expired)). |