Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
docs: conditional git_target_changed hold; reparse under an open episode only for pages not already current
garrytanbe1372204d18
diff --git a/docs/architecture/key-files/core-persistence-continued.md b/docs/architecture/key-files/core-persistence-continued.mdindex a19cc38244..89a30d1416 100644--- a/docs/architecture/key-files/core-persistence-continued.md+++ b/docs/architecture/key-files/core-persistence-continued.md@@ -46,5 +46,5 @@ The #6188 fence check and fence repair live in [fence repair](core-fence-repair.- `src/core/postgres-engine/reserved-transactions.ts` with `isPersistenceDeclaration` (`src/core/persistence/protocol.ts`) — BEGIN pipelining. `isPersistenceDeclaration` (GBRA-156) accepts exactly the `protocol.ts` declarations (transaction-local `set_config` calls and the brain row FOR SHARE). Admission, the FIFO claim, the effect claims and source guards, the receipt true-up, the single publication (`coordinator.ts`) and the consumer's lent lane transactions (`consumer.ts` `onLane`, through `reservedTransactions`) pass it as `TransactionOptions.pipelineFirst`, so behind a transaction-mode pooler the declaration goes out with BEGIN (one round trip less per transaction) while every later statement still waits for BEGIN's answer; if BEGIN fails, only the declaration has run, in autocommit, where it changes nothing. A reserved transaction gates every statement its body can reach, savepoint and nested-transaction bodies and `file()` included. A sync lane group's publication passes none: its begun mark must follow BEGIN's answer (`sync-lanes.ts` `awaitLaneBegin`). Tests: `test/persistence-pipeline-first.test.ts` (the predicate and the reviewed call sites), `test/e2e/pgbouncer-transaction-pipeline-6930.test.ts`.- `src/core/persistence/checkout-state.ts` (`readCheckoutState`) — Q1: a canonical checkout against its last fetched upstream, read-only and time-bounded (no fetch): ahead/behind, `diverged`, each commit the upstream lacks (sha, author, date, subject, paths; at most 20) and porcelain counts. `sources writer status` reports it as `checkouts`; doctor `git_convergence` adds the commits and counts for a managed checkout and warns at once on a diverged one.- `src/core/links-fence.ts` + `src/core/links-fence-projection.ts` — the `## Links` fence (G5): the markdown form of authored (manual) edges, `<!--- gbrain:links:begin -->` / `end` around `| to | type | source | valid_from | valid_until | context |`, one row per (to, type, source), source defaulting to `manual`; a reconciliation-managed provenance (`MANAGED_LINK_SOURCES`, exported here and re-exported by `ops/links.ts`), a bad date or a duplicate key is a skipped row with a warning. `parseLinksFence` (state `absent` / `parsed` / `malformed`), the deterministic `renderLinksFence` (rows sorted by key), `upsertLinksFenceRow` (omitted context and dates keep the stored ones; identical row = unchanged body), `removeLinksFenceRows` (leaves an empty fence), `replaceOrInsertLinksFence` (a `## Links` section at the end of compiled_truth, above an unclosed code block) and `maskLinksFence`, which `extractBodyLinks` applies so the fence cells are never also extracted as markdown, wikilink or bare-slug edges. A rewrite of a fence that parses with warnings throws `LinksFenceRewriteRefusal`. Cells use `escapeFenceCell` plus the same one-level marker encoding for `gbrain:links:*`. `projectLinksFence(tx, page, rows)` reconciles only what the page's fence owns: edges with origin_page_id = the page and origin_field `links_fence` (deleted, context-updated, inserted by key) and `manual` transitions with origin_page_id = the page, then refreshes the touched relationships; a row whose target is not a live page in the source yields no edge. `linksFenceProjectionRows` returns null for an absent or malformed fence, so such a body changes no edge (an absent fence retires nothing, as for facts; unlike facts no held generation is recorded, so after a hand deletion of the whole section the database keeps edges git no longer has until `remove_link` or the G3 indexer), and drops rows typed with a link type the active pack does not declare. Callers: `prepareCanonicalProjections` (every projected page publication, quarantined and segmented-unread bodies excepted), `prepareAutomaticLinks` (put_page auto-links and the managed links maintenance, so `extract --stale` restores a lost fence edge) and `persistence/manual-links.ts`. Database-only manual edges (origin_field NULL) are never touched by any of them. Pinned by `test/links-fence.test.ts` and `test/managed-links-fence.test.ts`.-- `src/core/persistence/git-door.ts`: G2, the door of git-canonical memory (Git is the source of truth, the database derives from it). A source with `sources.config.canonical = "git"` (strict, default off; nothing sets it yet) commits each published file before its database row. `publishMutation` (`coordinator.ts`) and `publishGroup` (`group-publish.ts`, one commit per group, after its lane turn) call `commitThroughDoor` under the writer lock, between the file rename and the transaction commit. It commits by plumbing without the checkout's index: `hash-object -w` per file, the trees along the changed paths rebuilt from HEAD (`cat-file tree`, `hash-object -w -t tree`), `commit-tree` with `commitGitTargets`' message, `update-ref` compare-and-swap (one rebuild if the branch moved, then a refusal), then `update-index --cacheinfo` for exactly the published paths (an entry someone staged during the write is left as it is, and the write says its commit landed); detached HEAD and unborn branches refuse typed. Each `DoorFile` carries its planned state (`planned`: the sha256 of the bytes the publication wrote, or null for a removal), and the door commits exactly that state or nothing: the file must hash to the plan before and after `hash-object -w` (else `git_target_changed`, which leaves the edit on disk and holds the write in recovery as `unexpected_file_bytes`), and each target's index entry must equal HEAD's (else `git_index_conflict`, index untouched). An entry that still holds the version its path's last commit replaced (an interrupted door refresh) counts as clean: the write restores it to HEAD and reports the paths as `outcome.persistence.git_index_restored` and the `git_index_restored` notice (`indexRestoredNotice`, emitted by `page-mutations.ts` `emitFenceNotice`). The step deadline is sized from one stat of the index (`gitStepTimeoutMs`), and `refuseIgnored` asks `check-ignore` only about new targets `ignore-rules.ts` cannot clear. Door commits are not coalesced across single writes (each holds its worktree lock exclusively and the consumer runs one per root); a batch commits once through publishGroup. The publication stamps the commit as `outcome.persistence.git_commit` (`git_state: committed`); a target Git refuses rolls the publication back. The Git effect carries `push_only` and joins the push without staging (`effects.ts`). The recovery record carries `door` (checkout and starting HEAD); `recoverPublication` keeps a file whose commit landed (`doorCommitLanded`) instead of restoring it, and the replay completes against it: `replayAfterDoorCommit` in `publishMutation` and the strict exceptions in `page-prepare.ts` (own bytes committed, or a removal HEAD already holds). Old binaries ignore `door`; strict stays off until conversion fences them. Tests: `test/git-door-strict-gbra133.test.ts` (both engines), `test/git-door-plumbing-gbra133.test.ts`, `test/git-door-planned-state.test.ts`, `test/git-door-write-cost.test.ts`; per-write overhead bench `scripts/bench/door-write-overhead.ts` (opt-in, smoke `test/bench-door-write-overhead.test.ts`).+- `src/core/persistence/git-door.ts`: G2, the door of git-canonical memory (Git is the source of truth, the database derives from it). A source with `sources.config.canonical = "git"` (strict, default off; nothing sets it yet) commits each published file before its database row. `publishMutation` (`coordinator.ts`) and `publishGroup` (`group-publish.ts`, one commit per group, after its lane turn) call `commitThroughDoor` under the writer lock, between the file rename and the transaction commit. It commits by plumbing without the checkout's index: `hash-object -w` per file, the trees along the changed paths rebuilt from HEAD (`cat-file tree`, `hash-object -w -t tree`), `commit-tree` with `commitGitTargets`' message, `update-ref` compare-and-swap (one rebuild if the branch moved, then a refusal), then `update-index --cacheinfo` for exactly the published paths (an entry someone staged during the write is left as it is, and the write says its commit landed); detached HEAD and unborn branches refuse typed. Each `DoorFile` carries its planned state (`planned`: the sha256 of the bytes the publication wrote, or null for a removal), and the door commits exactly that state or nothing: the file must hash to the plan before and after `hash-object -w` (else `git_target_changed`: nothing is committed and the edit stays on disk; the write is held in recovery as `unexpected_file_bytes` when the file holds neither its bytes nor the version before it, and fails with nothing held when the file holds the version before it), and each target's index entry must equal HEAD's (else `git_index_conflict`, index untouched). An entry that still holds the version its path's last commit replaced (an interrupted door refresh) counts as clean: the write restores it to HEAD and reports the paths as `outcome.persistence.git_index_restored` and the `git_index_restored` notice (`indexRestoredNotice`, emitted by `page-mutations.ts` `emitFenceNotice`). The step deadline is sized from one stat of the index (`gitStepTimeoutMs`), and `refuseIgnored` asks `check-ignore` only about new targets `ignore-rules.ts` cannot clear. Door commits are not coalesced across single writes (each holds its worktree lock exclusively and the consumer runs one per root); a batch commits once through publishGroup. The publication stamps the commit as `outcome.persistence.git_commit` (`git_state: committed`); a target Git refuses rolls the publication back. The Git effect carries `push_only` and joins the push without staging (`effects.ts`). The recovery record carries `door` (checkout and starting HEAD); `recoverPublication` keeps a file whose commit landed (`doorCommitLanded`) instead of restoring it, and the replay completes against it: `replayAfterDoorCommit` in `publishMutation` and the strict exceptions in `page-prepare.ts` (own bytes committed, or a removal HEAD already holds). Old binaries ignore `door`; strict stays off until conversion fences them. Tests: `test/git-door-strict-gbra133.test.ts` (both engines), `test/git-door-plumbing-gbra133.test.ts`, `test/git-door-planned-state.test.ts`, `test/git-door-write-cost.test.ts`; per-write overhead bench `scripts/bench/door-write-overhead.ts` (opt-in, smoke `test/bench-door-write-overhead.test.ts`).- `src/core/persistence/ignore-rules.ts` — the ignore filter in front of `git check-ignore`; Git stays the only judge of an ignored path. `ignoreCandidates(root, paths)` returns every path some positive ignore rule could match, read from every `.gitignore` from the top level down to the path, `info/exclude` and `core.excludesFile`; a path it drops is one no rule could match. It over-approximates: negated rules are dropped, every wildcard matches any run of characters including `/`, directory-only rules match files, letters compare case-insensitively, each rule is tested against the path and every directory above it in its own and NFC spelling, and an unreadable or non-UTF-8 ignore file or a rule with a backslash or a character outside printable ASCII matches everything under its directory. Every path goes to Git on a sparse checkout or skip-worktree entry, a split or sparse index, an unresolvable `core.excludesFile` or include, a gitlink or symlink along the path, or a failed probe. The probe (rev-parse layout and the configuration with its origin files) is cached per checkout and taken again when a configuration file it read or could read, HEAD or the configuration environment changes; ignore files are re-read on every call and parsed once per content. Callers: `git-door.ts` `refuseIgnored` and `sync-origin-evidence.ts` `gitIgnoresUntracked`. Tests: `test/ignore-rules.test.ts`.diff --git a/docs/designs/GIT_CANONICAL_MEMORY.md b/docs/designs/GIT_CANONICAL_MEMORY.mdindex 09c54182f5..ac76207a15 100644--- a/docs/designs/GIT_CANONICAL_MEMORY.md+++ b/docs/designs/GIT_CANONICAL_MEMORY.md@@ -32,7 +32,7 @@ flowchart LR2. **Recovery follows git.** A crash after the commit and before the database transaction is finished by indexing the committed file, not by restoring the previous bytes.3. **The indexer reads commits, not the worktree.** For a strict source the managed sync pass reads the level at HEAD alone and imports each committed blob the database does not hold, keyed by path and blob id so replays are idempotent: a database version sync never saw, a drifted page and a page under an uncommitted edit all take the committed bytes, and the edit stays on disk. A committed removal deletes the page. An uncommitted absence means nothing. There is no hold classification, because there is nothing to classify: git is right by definition. The pass refuses `--working-tree`, the level it keeps between passes is dropped when `canonical` flips, and a pass that runs between a door commit and its database transaction leaves that write's replay to finish it (the door's own bytes are already committed, so the replay publishes them once). `gbrain sources reindex <id> --from-git` previews every database version Git replaces, bound to HEAD by a hash, and `--apply --expect <hash>` runs that pass; doctor `git_index_lag` reports how many commits and seconds a strict source's index trails HEAD.- The one place the working tree reaches Git on a strict source is the door commit itself. `commitThroughDoor` commits each written path in its planned state (the bytes the write published) or not at all: an edit made to that path after the write's rename and before its commit refuses `git_target_changed`, leaves the edit on disk and holds the write for `gbrain repair stuck-recovery`, and a staged change of the path refuses `git_index_conflict` with the index untouched. Nothing else, sync included, reads the working tree of a strict source.+ The one place the working tree reaches Git on a strict source is the door commit itself. `commitThroughDoor` commits each written path in its planned state (the bytes the write published) or not at all: an edit made to that path after the write's rename and before its commit refuses `git_target_changed` and leaves the edit on disk (the write is held for `gbrain repair stuck-recovery` when the file holds neither the write's bytes nor the version before it, and fails with nothing held when it holds the version before it), and a staged change of the path refuses `git_index_conflict` with the index untouched. Nothing else, sync included, reads the working tree of a strict source.4. **Push is off the write path.** Pushes are batched, integrate a moved remote in memory, and skip changes that are patch-equivalent to commits this checkout already made. A genuine two-sided conflict on a page path parks only the push; local commits and writes continue, and doctor reports it.5. **Enforced in the database.** For a strict source, a page-content change that does not come from the door or the indexer is refused, so an older binary cannot write database-first.diff --git a/docs/guides/sync-unblock-runbook.md b/docs/guides/sync-unblock-runbook.mdindex acc7235b4f..743f1bd7c0 100644--- a/docs/guides/sync-unblock-runbook.md+++ b/docs/guides/sync-unblock-runbook.md@@ -429,7 +429,8 @@ it, and a `--full` continues until every path it covers is settled.- `owner`, sync finishes it by itself: `in_flight` (admitted, not yetpublished), `import`, `reparse` (an unchanged file whose page is importedagain because its parser or schema pack changed, its origin moved, or an- open episode covers it), `delete`, `keep_db` and `keep_db_pending` (at most 200+ open episode covers it and the page is not already current: same blob and+ revision, chunks and text projection current, no hold), `delete`, `keep_db` and `keep_db_pending` (at most 200DB-wins writes per pass; the rest wait for the next pass),`legacy_unverified` (the legacy sweep checks it), and `paused_keep_db`(which waits for the owner's `sync.admission` change, as paused deletes