Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

docs: convergence kill-point list and the chained-write claim rule (ship docs audit)

garrytan81e19adba34d
diff --git a/docs/architecture/key-files/core-persistence-continued.md b/docs/architecture/key-files/core-persistence-continued.md
index 9f55d3d392..267331b824 100644
--- a/docs/architecture/key-files/core-persistence-continued.md
+++ b/docs/architecture/key-files/core-persistence-continued.md
@@ -37,7 +37,7 @@ The #6188 fence check and fence repair live in [fence repair](core-fence-repair.
- `src/core/persistence/switches.ts` + `preadmit-cache.ts` + `config-snapshot.ts` — the single-write fast path for foreground page writes on Postgres. `switches.ts` reads every write-path kill switch (`persistence.single_write_group` / `GBRAIN_SINGLE_WRITE_GROUP`, `persistence.preadmit_cache` / `GBRAIN_PREADMIT_CACHE`, `sync.waive_batch` / `GBRAIN_SYNC_WAIVE_BATCH`, `sync.foreground_priority` / `GBRAIN_SYNC_FOREGROUND_PRIORITY`, and `persistence.preparation_deadlines` / `GBRAIN_PREPARATION_DEADLINES` (#6278: off restores the old preparation predicate, see persistence-stalls.md); on by default, `0`/`false` off, env over config) in one `SELECT key,value FROM config WHERE key = ANY(...)` per engine, kept 5 s (the consumer tick reads it under its phase deadline), together with the four preparation budget keys (`readWriteSwitchSnapshot` returns `{ switches, preparation }`; `readPreparationPolicy` is the budgets alone); read-through views register their engine (`registerEngineView`) to share it. With `single_write_group` on: the consumer publishes an eligible single page write (put_page, capture, edit/delete/restore/revert without an internal `intent.kind` or group key) as a group of one through `publishGroup` (`publishSingleWrite`; its intent carries no group marker, its effects queue without `deferBatchReconcile`, and a group that does not commit falls back to `publishMutation`, reclaiming a claim its file restoration released); PGLite and refused shapes keep `publishMutation`. The waiter's own wake (`wake(true)`) claims without the periodic scans (recovery records still block the claim through `claimableWriteSql`); preparation reads the config table once (`config-snapshot.ts`) and sends its independent reads together; the coordinated (`coordinated: true`) put_page apply reuses the publisher's locked read as the import base, pipelines its follow-up statements and reads the page once after its last write for the read-back check, seal, core notice and receipt. Publication, the own admission and the recovery transactions run on the consumer's reserved lane (`onLane`: the idle-probe connection, taken while two other pool connections stay free and kept across writes; never lent while an idle probe runs on it, and probes skip it while lent), so their prepared statements stay warm; the waiter is told of the commit before the recovery record is cleared. `preadmit-cache.ts` (with `preadmit_cache` on, Postgres) answers the brain id, source row, worktree binding and local-writer reads of `submitPageMutation` from a per-engine cache (30 s, keyed by brain id; absences never cached); admission rechecks each under lock and the brain id against its protocol statement, and a refusal the cache could explain (`source_changed`, `permission_denied`, a changed brain) drops the cache and runs the whole preparation once more uncached. Pinned by `test/e2e/single-write-fast-path-postgres.test.ts` (switches, cache rechecks, replay, restoration after a failure past the file rename, the recovery-blocked direct claim) and `test/single-write-group-parity.test.ts` + `test/e2e/single-write-group-parity-postgres.test.ts` (row-for-row parity with the switches off); measured by `scripts/bench/foreground-put-page.ts`.
-- Foreground priority (#5984 Phase 4.5, `foreground_priority` switch, `journal.ts`): `claimableWriteSql` lets a non-sync request pass earlier queued sync rows of its root that have not started and do not name its page (slug, page id or `intent.renameFrom.slug`), claims it first, and holds a passed sync row while that request runs; `claimNextLaneHead` starts no lane head while such a request is unfinished, except one head per foreground commit since the run's last head claim (`seen_sequence`). Both claims find that commit through `FOREGROUND_WAITING_ON`'s probe, served by `persistence_requests_foreground_committed` (#7649, a partial index on committed non-sync rows by worktree and sequence, see [request indexes](core-persistence-request-indexes.md)), so a consumer's first claims after a restart, when the bound is 0, stay index probes on a large request table. A consumer whose claimed write passed sync rows (`passed_sync`) claims plain FIFO on that root after it commits (`owesSyncTurn`), so a stream of writes cannot starve the drain. The sync side's pauses (no admit-ahead for 60 s after a write, no group while one is queued, the 25-page credit wait) apply only with the switch off. Receipt health judges a claimed write only against earlier started writes. Pinned by `test/managed-sync-foreground-priority.test.ts` (two processes) and `test/persistence-diagnostics.test.ts`.
+- Foreground priority (#5984 Phase 4.5, `foreground_priority` switch, `journal.ts`): `claimableWriteSql` lets a non-sync request pass earlier queued sync rows of its root that have not started and do not name its page (slug, page id or `intent.renameFrom.slug`), claims it first, and holds a passed sync row while that request runs; `claimNextLaneHead` starts no lane head while such a request is unfinished, except one head per foreground commit since the run's last head claim (`seen_sequence`). A write queued behind an earlier queued write that waits on a sync row naming its page holds back nothing, since it cannot go first either (pinned by `test/claim-fifo-foreground-waiting.test.ts`). Both claims find that commit through `FOREGROUND_WAITING_ON`'s probe, served by `persistence_requests_foreground_committed` (#7649, a partial index on committed non-sync rows by worktree and sequence, see [request indexes](core-persistence-request-indexes.md)), so a consumer's first claims after a restart, when the bound is 0, stay index probes on a large request table. A consumer whose claimed write passed sync rows (`passed_sync`) claims plain FIFO on that root after it commits (`owesSyncTurn`), so a stream of writes cannot starve the drain. The sync side's pauses (no admit-ahead for 60 s after a write, no group while one is queued, the 25-page credit wait) apply only with the switch off. Receipt health judges a claimed write only against earlier started writes. Pinned by `test/managed-sync-foreground-priority.test.ts` (two processes) and `test/persistence-diagnostics.test.ts`.
- `src/core/postgres-engine/reserved-transactions.ts` with `isPersistenceDeclaration` (`src/core/persistence/protocol.ts`) — BEGIN pipelining. `isPersistenceDeclaration` (GBRA-156) accepts exactly the `protocol.ts` declarations (transaction-local `set_config` calls and the brain row FOR SHARE). Admission, the FIFO claim, the effect claims and source guards, the receipt true-up, the single publication (`coordinator.ts`) and the consumer's lent lane transactions (`consumer.ts` `onLane`, through `reservedTransactions`) pass it as `TransactionOptions.pipelineFirst`, so behind a transaction-mode pooler the declaration goes out with BEGIN (one round trip less per transaction) while every later statement still waits for BEGIN's answer; if BEGIN fails, only the declaration has run, in autocommit, where it changes nothing. A reserved transaction gates every statement its body can reach, savepoint and nested-transaction bodies and `file()` included. A sync lane group's publication passes none: its begun mark must follow BEGIN's answer (`sync-lanes.ts` `awaitLaneBegin`). Tests: `test/persistence-pipeline-first.test.ts` (the predicate and the reviewed call sites), `test/e2e/pgbouncer-transaction-pipeline-6930.test.ts`.
- `src/core/links-fence.ts` + `src/core/links-fence-projection.ts` — the `## Links` fence (G5): the markdown form of authored (manual) edges, `<!--- gbrain:links:begin -->` / `end` around `| to | type | source | valid_from | valid_until | context |`, one row per (to, type, source), source defaulting to `manual`; a reconciliation-managed provenance (`MANAGED_LINK_SOURCES`, exported here and re-exported by `ops/links.ts`), a bad date or a duplicate key is a skipped row with a warning. `parseLinksFence` (state `absent` / `parsed` / `malformed`), the deterministic `renderLinksFence` (rows sorted by key), `upsertLinksFenceRow` (omitted context and dates keep the stored ones; identical row = unchanged body), `removeLinksFenceRows` (leaves an empty fence), `replaceOrInsertLinksFence` (a `## Links` section at the end of compiled_truth, above an unclosed code block) and `maskLinksFence`, which `extractBodyLinks` applies so the fence cells are never also extracted as markdown, wikilink or bare-slug edges. A rewrite of a fence that parses with warnings throws `LinksFenceRewriteRefusal`. Cells use `escapeFenceCell` plus the same one-level marker encoding for `gbrain:links:*`. `projectLinksFence(tx, page, rows)` reconciles only what the page's fence owns: edges with origin_page_id = the page and origin_field `links_fence` (deleted, context-updated, inserted by key) and `manual` transitions with origin_page_id = the page, then refreshes the touched relationships; a row whose target is not a live page in the source yields no edge. `linksFenceProjectionRows` returns null for an absent or malformed fence, so such a body changes no edge (an absent fence retires nothing, as for facts; unlike facts no held generation is recorded, so after a hand deletion of the whole section the database keeps edges git no longer has until `remove_link` or the G3 indexer), and drops rows typed with a link type the active pack does not declare. Callers: `prepareCanonicalProjections` (every projected page publication, quarantined and segmented-unread bodies excepted), `prepareAutomaticLinks` (put_page auto-links and the managed links maintenance, so `extract --stale` restores a lost fence edge) and `persistence/manual-links.ts`. Database-only manual edges (origin_field NULL) are never touched by any of them. Pinned by `test/links-fence.test.ts` and `test/managed-links-fence.test.ts`.
diff --git a/scripts/persistence/README.md b/scripts/persistence/README.md
index ece6cc1226..628942232e 100644
--- a/scripts/persistence/README.md
+++ b/scripts/persistence/README.md
@@ -563,7 +563,7 @@ The chaos modes, each named in failures and selectable with
| Mode | What happens | Expected end state | On master |
| --- | --- | --- | --- |
-| (a) client kills, (b) serve kills | A kill loop of `GBRAIN_TEST_CONVERGENCE_KILLS` change rounds, each ending in one `kill -9` at a point of the curated list (`publication:before_commit`, `publication:after_commit`, `consumer:preparing`, `lane:applied`, `sync:before_group_admission`, `sync:mid_checkpoint`), cycling a `--no-delegate` client, a serve running a delegated catch-up, and a serve with its delegating client | strict convergence | runs |
+| (a) client kills, (b) serve kills | A kill loop of `GBRAIN_TEST_CONVERGENCE_KILLS` change rounds, each ending in one `kill -9` at a point of the curated list (`publication:before_commit`, `publication:after_commit`, `consumer:preparing`, `lane:applied`, `sync:before_group_admission`, `sync:mid_checkpoint`, `sync:held_settled`, `sync:passed_over`, `sync:batch_entries`, `sync:tail_started`), cycling a `--no-delegate` client, a serve running a delegated catch-up, and a serve with its delegating client | strict convergence | runs |
| (c) concurrent writers | During the kill loop: `gbrain put` rewrites of 5% of the notes, `remember` facts on an entity page, `gbrain extract timeline --source db` (a managed maintenance job) | strict convergence; every acknowledged write in the ledger | runs |
| (c-db) DB-only competitor | With `sync.write_through=false`, a DB-only `put` lands while the run is stalled on that page's frozen entry, and another on a page whose file did not change | the raced page is held `concurrent_write` (needs a person), its file keeps the committed bytes and its page the DB write; both pages are left out of disk == DB | runs |
| (d) out-of-band Git | While a run is stalled mid-manifest: `git mv` and `git rm` of 1% of the files, committed, and deletion of 1% uncommitted | moves and removals converge strictly; each uncommitted deletion is listed by `gbrain pages ghosts` with the `gbrain repair ghost-pages` exit and left out of disk == DB | runs |