Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

docs(key-files): split the Git writer and git-canonical entries out of core-persistence-continued.md (59,974 of 60,000 bytes) into core-persistence-git.md

garrytana3fbff4eae98
diff --git a/docs/architecture/KEY_FILES.md b/docs/architecture/KEY_FILES.md
index 36971e126f..9314885c14 100644
--- a/docs/architecture/KEY_FILES.md
+++ b/docs/architecture/KEY_FILES.md
@@ -24,7 +24,7 @@ link it here rather than raising the cap.
| Subsystem | Entry range / scope |
|---|---|
| [Page identity and writer administration](key-files/page-identity-and-administration.md) | Opaque result IDs, current grants, state-bound ownership changes |
-| [Canonical reconciliation](key-files/canonical-reconciliation.md) | Exact-page repair, retained originals, [derived generations](key-files/generations.md), sync discovery and the origin rule (`sync-origin.ts`, `file-claims.ts`); [rows behind the checkpoint](key-files/canonical-reconciliation-continued.md) |
+| [Canonical reconciliation](key-files/canonical-reconciliation.md) | Exact-page repair, retained originals, [derived generations](key-files/generations.md), sync discovery and the origin rule; [rows behind the checkpoint](key-files/canonical-reconciliation-continued.md) |
| [Company-brain ingestion](key-files/company-brain.md) | Inspection, admission, receipts, derived relationships and schema; [operator guide](../guides/company-brain-ingestion.md) |
| [Agent operator contract](key-files/agent-contract.md) | `agent-output.ts`, error registry + docs, notice ledger, `isCallable`, `--json` guard, contract scanner |
| [Commands (1/6)](key-files/commands-1.md) | `src/commands/agent-logs.ts` through `db-repair.ts` |
@@ -46,7 +46,7 @@ link it here rather than raising the cap.
| [Core Minions (1/2)](key-files/core-minions-1.md) | `src/core/minions/` through `src/core/minions/protected-names.ts` |
| [Core Minions (2/2)](key-files/core-minions-2.md) | `src/core/minions/queue.ts` through `src/core/minions/worker.ts` |
| [Core Persistence](key-files/core-persistence.md) | `src/core/persistence/` journal, coordinator, effects, managed sync |
-| [Core Persistence (continued)](key-files/core-persistence-continued.md) | worktree-refresh*, connectors, holds, receipts; [fence repair](key-files/core-fence-repair.md) |
+| [Core Persistence (continued)](key-files/core-persistence-continued.md) | connectors, holds, receipts; [Git writer](key-files/core-persistence-git.md); [fence repair](key-files/core-fence-repair.md) |
| [Persistence capacity](key-files/core-persistence-capacity.md) | write caps, true-up |
| [Core Persistence (seams)](key-files/core-persistence-request-indexes.md) | `checkpoint-validation`, `fault-points`, `native-file-target`, `admission-retry`, `unbound-source` |
| [Core Persistence (managed facts)](key-files/core-persistence-facts.md) | `facts-maintenance.ts`, `facts-prepare.ts`, `derived-facts.ts`, `same-page-shadow.ts` |
diff --git a/docs/architecture/key-files/core-persistence-continued.md b/docs/architecture/key-files/core-persistence-continued.md
index 8f25bdc923..1da48fe65c 100644
--- a/docs/architecture/key-files/core-persistence-continued.md
+++ b/docs/architecture/key-files/core-persistence-continued.md
@@ -2,11 +2,7 @@
[Subsystem index](../KEY_FILES.md) · [Previous entries](core-persistence.md). Read only the entries relevant to your change.
Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.
-The #6188 fence check and fence repair live in [fence repair](core-fence-repair.md). Managed connector ingestion, connector item holds and the no-op kernel (checkpoint validation and the fault seams: [request indexes and fault seams](core-persistence-request-indexes.md)). Also the persistence verb types and errors, database-write and manual links, write attribution and loop-fact retirement; the managed purge of deleted pages is under [memory purge](memory-purge.md).
-
-- `src/core/persistence/{worktree-refresh,worktree-refresh-schema}.ts` — `gbrain sources refresh` (F0), the one sanctioned managed Git writer. `persistence_worktree_refreshes` holds one row per refresh with a partial unique index (one active refresh per worktree; its 23505 maps to `refresh_in_progress`). `refreshWorktree` prechecks with no database state (managed, owner host, upstream, no active refresh, no recovery, no unexhausted managed-sync cursor, hardened `git fetch` bounded by `sources.refresh_fetch_timeout_ms`, ancestry, dirty paths intersected with the incoming diff), then: `draining` (inserted under `withTopologyLocks` + `lockTopologyRows`; admission in `journal.ts` refuses `worktree_refreshing` for every request with that `worktree_id`, and `assertManagedSyncAllowed` refuses a managed sync before discovery), poll until no queued/running/recovering request, no queued/running `git`/`withdrawal-mirror` effect and no recovery record remain (else `refresh_drain_timeout`, aborted), `fenced` (both native locks held, rechecked in one transaction; `refreshFenceClear` stops request and effect claims), HEAD verified, dirt rechecked, `git merge --ff-only`, HEAD verified again, then one transaction `merged` + upstream observation + `syncing`; native locks are released and the managed `--no-pull` sync runs per member in one budget (`sources.refresh_sync_budget_ms`; past it a member is `deferred`) (only `managed_sync_*` `submit_job` admissions pass during `syncing`); `completed` when every member's `last_commit` is the target or descends from it, or with `outcome.sync_blocked` / `sync_deferred` when a member blocks or is deferred. A per-worktree `refresh-<worktree>.lock` held for the whole run proves liveness: `resumeWorktreeRefreshes` (called from `recoverSourceTopologies` on every consumer topology scan) touches interrupted `draining`/`fenced`/`merged` rows whose lock is free and applies the section 1.5 table (draining or fenced at the old HEAD abort; HEAD at the verified target moves to `syncing`; anything else is `recovery_required`, which admission refuses with `refresh_recovery_required`). Admission also completes a `syncing` row whose members all reached the target. #7679: each row records its holder (`outcome.holder`: host id, hostname, pid, process start; `previous_holder` after a recovery adopts it). A row whose lock is free is orphaned: `resumeWorktreeRefreshes` also completes an orphaned `syncing` row once every member's `last_commit` reached the target or a descendant, and a plain `gbrain sources refresh` on the owner host finishes any orphaned non-`recovery_required` row through the `--resume` path (`adoptOrphanedRefresh` + `resumeHeld`) before its own refresh, reporting it as `recovered`. A holder recorded on another host is never adopted. `refreshHolderReport` (writer status `worktree_refreshes`, sync status `worktree_refresh`) names the holder, `liveness` (`live` / `orphaned` / `unknown` off the lock's host) and a fix naming `--resume`. `--resume` converges and runs the member syncs; `--abandon` lifts a fence only with HEAD back at the start commit. `gbrain sources writer status` lists active refreshes as `worktree_refreshes` (state, heads, the next command). Doctor `worktree_refresh_stuck` lists rows active over 15 minutes. CLI body: `src/commands/sources-refresh.ts` (#7685: `--json` prints a refusal's structured `fix` when it has one; `refresh_in_progress` carries the writer-status read); while a resident PGLite owner (`gbrain serve`, stdio MCP) holds the database, `src/commands/sources-refresh-delegate.ts` runs the refresh inside it through the `writer_refresh` administration operation (trusted CLI registration only) and prints the same result or refusal. The consumer's idle probe applies `refreshFenceClear` to effects like the claims, so a fenced worktree's queued effects do not wake it. Tests: `test/persistence-worktree-refresh.test.ts`, `test/persistence-worktree-refresh-restart.test.ts` (child-process crash after merge), `test/worktree-refresh-orphan-7679.test.ts` (SIGKILL mid-syncing, live-holder no-steal), `test/oversize-hold-rescreen-8072.test.ts` (budget), Postgres via `test/e2e/worktree-refresh-*postgres.test.ts`, owner delegation in `test/persistence-sync-stdio-owner.test.ts`.
-- `src/core/persistence/git-gap.ts` + `src/commands/sources-git-gap.ts` + `src/commands/doctor/checks/db-ahead-of-git.ts` — git-canonical memory, wave G1 (`docs/designs/GIT_CANONICAL_MEMORY.md#the-git-gap-census`), read-only. `declaredDatabaseOnly` is the one predicate for pages that may not have a tracked file, tested in order: a read-only mirror source, a private page (`isPrivatePage`), an alias-private page (`isDerivationPrivate`), lineage-private (computed in SQL from `privatePagesFilterFragment`), `database_only_reason`, a `storage.db_only` prefix (`isSourceDbOnlySlug`, refusing an invalid gbrain.yml), a never-filed derive-phase page (page-prepare.ts `isNeverFiledDerivedPage`: no `source_path`/`source_uri` under `DERIVE_PHASE_DB_ONLY_DEFAULTS`), a path the checkout's Git ignores (`ignoredPaths`: filter-free `check-ignore -z --stdin`, tracked files never reported, exit 1 = none, null when Git cannot answer). The equality contract compares a committed blob with the page's export view (`readExportPage`, withdrawals overlaid; the census reads a batch's views with `readExportPages`, the same snapshots in two statements, #7617): an inline blob parsed as reconcile parses a canonical file, aligned with the stored type, subtype and #6429 stamp and compared by `reconcileCanonical` digest; a segmented page by the Git blob id of its streamed export. HEAD is read with hardened plumbing only (`readHeadTree`: one `ls-tree -r -l -z --full-tree` under the source's directory; `readBlobs`: `cat-file --batch` in calls of at most 64 MiB) and the working tree by hashing raw bytes (`worktreeBlobOid`), never `git status`/`git diff`, so no repository filter, textconv or fsmonitor runs. `gbrain sources git-gap <id> [--sample n] [--json]` (trusted local CLI, not an MCP op) puts every live page in one state (`equal`, `database_ahead`, `git_ahead` when the sync import record's `blob_oid`/`disk_blob_oid` differs from HEAD, else folded into `database_ahead` and counted in `database_ahead_without_import_record`, `declared_database_only` per class, `undeclared_database_only`, `worktree_only`, `unparseable`, `unsafe_path`), reports the working tree separately, and counts rows with no Git home (unfenced active facts, fenced facts on a missing page or a page without a committed file, links whose `link_source` is not in `DERIVED_LINK_SOURCES`, timeline rows on pages without a file), open `sync-hold` records by code and parked effect targets by kind and code. Each 256-page batch reads in its own read-only transaction. Doctor `db_ahead_of_git` runs the census per non-archived source whose directory is a Git checkout (others are only counted) under `GBRAIN_DOCTOR_GIT_GAP_TIMEOUT_MS` (default 10 s) and the entry deadline; it is always `ok` (informational) and lists totals only, a source the budget stopped or never reached as partial. Tests: `test/git-gap-census.test.ts` (PGLite and Postgres).
-- `src/core/persistence/git-declared.ts` + `src/core/persistence/git-canonical-schema.ts` (migration `git_canonical_declared`) — git-canonical memory, wave G7a: the declared database-only classes of a strict source (`sources.config.canonical = "git"`, default off). `strictPageClass` judges a page by its slug and prior file (`declaredPathClass`: `db_only_prefix` for a `storage.db_only` dir, `derive_folder` for a never-filed derive-phase page; G3's indexer reads the same function) and then its rendered content (`privacyClass`: `private`, `alias_private`, or `lineage_private` from the same SQL remote reads use, `privatePagesFilterFragment`, over the page's own values). `page-prepare.ts` `prepareFileTarget` asks it before it builds a strict file target: a declared page gets no target (no file, no door commit, no Git effect), an untracked ignored target is `git_ignored`, a page turning private gets the removal of the file it still has in the worktree or at HEAD, and a path whose `filter` attribute is `lfs` passes `assertLfsTarget` only with `storage.git_lfs` and a checkout that configures the LFS filter (`git_lfs_not_enabled`). `preparePrivateTransition` (page-prepare.ts) is that removal for other writers (GBRA-60). In `publishMutation` and `publishGroup`, `assertNoPrivateCommit` refuses a file target carrying a private page's bytes before anything is written (`git_private_commit_refused`; a group rolls back whole), and `declareStrictPublication` (after apply, before the door commit) refuses a declared page's file or an undeclared file-less page (`git_home_required`), stamps `pages.database_only_reason` and, for a privacy class, clears `source_path`/`source_uri` in the transaction of the door commit that removed the file. A managed import builds its own file target, so it meets only these publication checks. `assertStrictGitHome` is the refusal for writers with no file form yet (G4, G5). The managed writer guard (`writer-guard-schema.ts`) refuses rows of a strict source unless the writer declares `gbrain.git_protocol` (`withCoordinatedWrite` does), behind the `git_canonical_brain.strict_sources` counter kept by `sources` triggers (`git_protocol_required`, mapped in `publication-failure.ts`). The guard reads its state as the writing role and fails closed: a hidden or missing `persistence_brain` or `git_canonical_brain` row, or a source or page row hidden by row-level security, raises `writer_coordinator_required` (`managed_writer_guard:unreadable_state`), and the counter refuses a recount under RLS on `sources` (`test/writer-fence-fail-closed-7636.test.ts`). The census (`git-gap.ts`) adds `database_only_source` (a source with no checkout) and maps the stamped reasons to their own classes. Tests: `test/git-canonical-declared-g7a.test.ts` (both engines).
+The #6188 fence check and fence repair live in [fence repair](core-fence-repair.md). Managed connector ingestion, connector item holds and the no-op kernel (checkpoint validation and the fault seams: [request indexes and fault seams](core-persistence-request-indexes.md)). The sources refresh writer, the git gap census, declared classes, the git door and ignore rules live in [Core Persistence (Git)](core-persistence-git.md). Also the persistence verb types and errors, database-write and manual links, write attribution and loop-fact retirement; the managed purge of deleted pages is under [memory purge](memory-purge.md).
- `src/core/persistence/{types,params,preconditions,verb-errors}.ts` — public mutation receipt states, error reasons and revision/idempotency parameter contracts. Receipt serialization explicitly selects public fields; queued content and execution credentials stay internal. `frozenVerbWriteError` uses MEMORY_VERBS v1's existing error enum with additive `write_request`/`write_error`; `page_body_segmented` is a write error code, so a verb refused by a segmented target keeps it as `write_error` and canonical code under the frozen `invalid_params` (caller class, not retryable) on both the receipt and the pre-admission path (`runMemoryWrite`). `remember` reads a segmented entity page up to PAGE_MAX_BYTES back whole (`materializedPage`) for its inferred-target checks and fence write; over it, a named or inferred entity refuses before admission with `segmentedRememberRefusal` (fix: remember with `infer_entity: false`), never a silent `NO_ENTITY` (`test/large-pages/segmented-remember-refusal.test.ts`); `committedVerbOutcome` refuses pending or failed receipts. `OperationError` and the thin MCP client preserve this metadata, including when a timeout races an already-received receipt. The registry-safe parameter/schema leaf never imports operation values. Pinned by `test/persistence-write-contracts.test.ts`, `test/mcp-client-hardening.test.ts` and a real HTTP envelope in `test/mcp-client.test.ts`.
- `src/core/persistence/database-write.ts` — `coordinatedDatabaseWrite` is the managed path for rows with no canonical file; `ontology_propose` (one entity page key) uses it. On a managed brain it checks the caller's durable write authority for the authority page (`submissionAuthority`, then `authorizeWrite` FOR SHARE inside the transaction), revalidates the source incarnation and commits inside `withCoordinatedWrite` holding the given page keys. Unmanaged brains keep the engine calls. Tests: `test/managed-ontology-propose.test.ts`.
@@ -44,7 +40,4 @@ The #6188 fence check and fence repair live in [fence repair](core-fence-repair.
- Foreground priority (#5984 Phase 4.5, `foreground_priority` switch, `journal.ts`): `claimableWriteSql` lets a non-sync request pass earlier queued sync rows of its root that have not started and do not name its page (slug, page id or `intent.renameFrom.slug`), claims it first, and holds a passed sync row while that request runs; `claimNextLaneHead` starts no lane head while such a request is unfinished, except one head per foreground commit since the run's last head claim (`seen_sequence`). Both claims find that commit through `FOREGROUND_WAITING_ON`'s probe, served by `persistence_requests_foreground_committed` (#7649, a partial index on committed non-sync rows by worktree and sequence, see [request indexes](core-persistence-request-indexes.md)), so a consumer's first claims after a restart, when the bound is 0, stay index probes on a large request table. A consumer whose claimed write passed sync rows (`passed_sync`) claims plain FIFO on that root after it commits (`owesSyncTurn`), so a stream of writes cannot starve the drain. The sync side's pauses (no admit-ahead for 60 s after a write, no group while one is queued, the 25-page credit wait) apply only with the switch off. Receipt health judges a claimed write only against earlier started writes. Pinned by `test/managed-sync-foreground-priority.test.ts` (two processes) and `test/persistence-diagnostics.test.ts`.
- `src/core/postgres-engine/reserved-transactions.ts` with `isPersistenceDeclaration` (`src/core/persistence/protocol.ts`) — BEGIN pipelining. `isPersistenceDeclaration` (GBRA-156) accepts exactly the `protocol.ts` declarations (transaction-local `set_config` calls and the brain row FOR SHARE). Admission, the FIFO claim, the effect claims and source guards, the receipt true-up, the single publication (`coordinator.ts`) and the consumer's lent lane transactions (`consumer.ts` `onLane`, through `reservedTransactions`) pass it as `TransactionOptions.pipelineFirst`, so behind a transaction-mode pooler the declaration goes out with BEGIN (one round trip less per transaction) while every later statement still waits for BEGIN's answer; if BEGIN fails, only the declaration has run, in autocommit, where it changes nothing. A reserved transaction gates every statement its body can reach, savepoint and nested-transaction bodies and `file()` included. A sync lane group's publication passes none: its begun mark must follow BEGIN's answer (`sync-lanes.ts` `awaitLaneBegin`). Tests: `test/persistence-pipeline-first.test.ts` (the predicate and the reviewed call sites), `test/e2e/pgbouncer-transaction-pipeline-6930.test.ts`.
-- `src/core/persistence/checkout-state.ts` (`readCheckoutState`) — Q1: a canonical checkout against its last fetched upstream, read-only and time-bounded (no fetch): ahead/behind, `diverged`, each commit the upstream lacks (sha, author, date, subject, paths; at most 20) and porcelain counts. `sources writer status` reports it as `checkouts`; doctor `git_convergence` adds the commits and counts for a managed checkout and warns at once on a diverged one.
- `src/core/links-fence.ts` + `src/core/links-fence-projection.ts` — the `## Links` fence (G5): the markdown form of authored (manual) edges, `<!--- gbrain:links:begin -->` / `end` around `| to | type | source | valid_from | valid_until | context |`, one row per (to, type, source), source defaulting to `manual`; a reconciliation-managed provenance (`MANAGED_LINK_SOURCES`, exported here and re-exported by `ops/links.ts`), a bad date or a duplicate key is a skipped row with a warning. `parseLinksFence` (state `absent` / `parsed` / `malformed`), the deterministic `renderLinksFence` (rows sorted by key), `upsertLinksFenceRow` (omitted context and dates keep the stored ones; identical row = unchanged body), `removeLinksFenceRows` (leaves an empty fence), `replaceOrInsertLinksFence` (a `## Links` section at the end of compiled_truth, above an unclosed code block) and `maskLinksFence`, which `extractBodyLinks` applies so the fence cells are never also extracted as markdown, wikilink or bare-slug edges. A rewrite of a fence that parses with warnings throws `LinksFenceRewriteRefusal`. Cells use `escapeFenceCell` plus the same one-level marker encoding for `gbrain:links:*`. `projectLinksFence(tx, page, rows)` reconciles only what the page's fence owns: edges with origin_page_id = the page and origin_field `links_fence` (deleted, context-updated, inserted by key) and `manual` transitions with origin_page_id = the page, then refreshes the touched relationships; a row whose target is not a live page in the source yields no edge. `linksFenceProjectionRows` returns null for an absent or malformed fence, so such a body changes no edge (an absent fence retires nothing, as for facts; unlike facts no held generation is recorded, so after a hand deletion of the whole section the database keeps edges git no longer has until `remove_link` or the G3 indexer), and drops rows typed with a link type the active pack does not declare. Callers: `prepareCanonicalProjections` (every projected page publication, quarantined and segmented-unread bodies excepted), `prepareAutomaticLinks` (put_page auto-links and the managed links maintenance, so `extract --stale` restores a lost fence edge) and `persistence/manual-links.ts`. Database-only manual edges (origin_field NULL) are never touched by any of them. Pinned by `test/links-fence.test.ts` and `test/managed-links-fence.test.ts`.
-- `src/core/persistence/git-door.ts`: G2, the door of git-canonical memory (Git is the source of truth, the database derives from it). A source with `sources.config.canonical = "git"` (strict, default off; set by `gbrain sources canonical <id> --to git --apply --expect <hash>`, `git-conversion.ts`) commits each published file before its database row. `publishMutation` (`coordinator.ts`) and `publishGroup` (`group-publish.ts`, one commit per group, after its lane turn) call `commitThroughDoor` under the writer lock, between the file rename and the transaction commit. It commits by plumbing without the checkout's index: `hash-object -w` per file, the trees along the changed paths rebuilt from HEAD (`cat-file tree`, `hash-object -w -t tree`), `commit-tree` with `commitGitTargets`' message, `update-ref` compare-and-swap (one rebuild if the branch moved, then a refusal), then `update-index --cacheinfo` for exactly the published paths (an entry someone staged during the write is left as it is, and the write says its commit landed). That refresh runs under Git's own index lock: `syncIndex` creates `<index>.lock` exclusively (an existing lock the #6932 proof finds abandoned is reclaimed under the caller's writer lock through `reclaimAbandonedIndexLock` in `effect-git.ts` and taken again; any other existing lock refuses with the landed failure, which says why it was not reclaimed), compares the entries, updates a hard-linked copy of the index through `GIT_INDEX_FILE`, and renames the copy onto the lock and the lock onto the index, so no writer that takes the lock can stage between the compare and the write; detached HEAD and unborn branches refuse typed. Each `DoorFile` carries its planned state (`planned`: the sha256 of the bytes the publication wrote, or null for a removal), and the door commits exactly that state or nothing: the file must hash to the plan before and after `hash-object -w` (else `git_target_changed`: nothing is committed and the edit stays on disk; the write is held in recovery as `unexpected_file_bytes` when the file holds neither its bytes nor the version before it, and fails with nothing held when the file holds the version before it), and each target's index entry must equal HEAD's (else `git_index_conflict`, index untouched). An entry that still holds the version its path's last commit replaced (an interrupted door refresh) counts as clean: the write restores it to HEAD and reports the paths as `outcome.persistence.git_index_restored` and the `git_index_restored` notice (`indexRestoredNotice`, emitted by `page-mutations.ts` `emitFenceNotice`). The step deadline is sized from one stat of the index (`gitStepTimeoutMs`), and `refuseIgnored` asks `check-ignore` only about new targets `ignore-rules.ts` cannot clear. Door commits are not coalesced across single writes (each holds its worktree lock exclusively and the consumer runs one per root); a batch commits once through publishGroup. The publication stamps the commit as `outcome.persistence.git_commit` (`git_state: committed`); a target Git refuses rolls the publication back. The Git effect carries `push_only` and joins the push without staging (`effects.ts`). The recovery record carries `door` (checkout and starting HEAD); `recoverPublication` keeps a file whose commit landed (`doorCommitLanded`) instead of restoring it, and the replay completes against it: `replayAfterDoorCommit` in `publishMutation` and the strict exceptions in `page-prepare.ts` (own bytes committed, or a removal HEAD already holds). Old binaries ignore `door`; strict stays off until conversion fences them. Tests: `test/git-door-strict-gbra133.test.ts` (both engines), `test/git-door-plumbing-gbra133.test.ts`, `test/git-door-planned-state.test.ts`, `test/git-door-write-cost.test.ts`, `test/git-door-index-lock-7742.test.ts` (both engines); per-write overhead bench `scripts/bench/door-write-overhead.ts` (opt-in, smoke `test/bench-door-write-overhead.test.ts`).
-- `src/core/persistence/ignore-rules.ts` — the ignore filter in front of `git check-ignore`; Git stays the only judge of an ignored path. `ignoreCandidates(root, paths)` returns every path some positive ignore rule could match, read from every `.gitignore` from the top level down to the path, `info/exclude` and `core.excludesFile`; a path it drops is one no rule could match. It over-approximates: negated rules are dropped, every wildcard matches any run of characters including `/`, directory-only rules match files, letters compare case-insensitively, each rule is tested against the path and every directory above it in its own and NFC spelling, and an unreadable or non-UTF-8 ignore file or a rule with a backslash or a character outside printable ASCII matches everything under its directory. Every path goes to Git on a sparse checkout or skip-worktree entry, a split or sparse index, an unresolvable `core.excludesFile` or include, a gitlink or symlink along the path, or a failed probe. The probe (rev-parse layout and the configuration with its origin files) is cached per checkout and taken again when a configuration file it read or could read, HEAD or the configuration environment changes; ignore files are re-read on every call and parsed once per content. Callers: `git-door.ts` `refuseIgnored` and `sync-origin-evidence.ts` `gitIgnoresUntracked`. Tests: `test/ignore-rules.test.ts`.
diff --git a/docs/architecture/key-files/core-persistence-git.md b/docs/architecture/key-files/core-persistence-git.md
new file mode 100644
index 0000000000..7b2bb2b005
--- /dev/null
+++ b/docs/architecture/key-files/core-persistence-git.md
@@ -0,0 +1,12 @@
+# Key files: Core Persistence (Git)
+
+[Subsystem index](../KEY_FILES.md) · [Core Persistence](core-persistence.md) · [continued](core-persistence-continued.md). Read only the entries relevant to your change.
+Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.
+The managed Git writer (`gbrain sources refresh`), git-canonical memory (the git gap census, declared database-only classes, the door) and the checkout reads beside them.
+
+- `src/core/persistence/{worktree-refresh,worktree-refresh-schema}.ts` — `gbrain sources refresh` (F0), the one sanctioned managed Git writer. `persistence_worktree_refreshes` holds one row per refresh with a partial unique index (one active refresh per worktree; its 23505 maps to `refresh_in_progress`). `refreshWorktree` prechecks with no database state (managed, owner host, upstream, no active refresh, no recovery, no unexhausted managed-sync cursor, hardened `git fetch` bounded by `sources.refresh_fetch_timeout_ms`, ancestry, dirty paths intersected with the incoming diff), then: `draining` (inserted under `withTopologyLocks` + `lockTopologyRows`; admission in `journal.ts` refuses `worktree_refreshing` for every request with that `worktree_id`, and `assertManagedSyncAllowed` refuses a managed sync before discovery), poll until no queued/running/recovering request, no queued/running `git`/`withdrawal-mirror` effect and no recovery record remain (else `refresh_drain_timeout`, aborted), `fenced` (both native locks held, rechecked in one transaction; `refreshFenceClear` stops request and effect claims), HEAD verified, dirt rechecked, `git merge --ff-only`, HEAD verified again, then one transaction `merged` + upstream observation + `syncing`; native locks are released and the managed `--no-pull` sync runs per member in one budget (`sources.refresh_sync_budget_ms`; past it a member is `deferred`) (only `managed_sync_*` `submit_job` admissions pass during `syncing`); `completed` when every member's `last_commit` is the target or descends from it, or with `outcome.sync_blocked` / `sync_deferred` when a member blocks or is deferred. A per-worktree `refresh-<worktree>.lock` held for the whole run proves liveness: `resumeWorktreeRefreshes` (called from `recoverSourceTopologies` on every consumer topology scan) touches interrupted `draining`/`fenced`/`merged` rows whose lock is free and applies the section 1.5 table (draining or fenced at the old HEAD abort; HEAD at the verified target moves to `syncing`; anything else is `recovery_required`, which admission refuses with `refresh_recovery_required`). Admission also completes a `syncing` row whose members all reached the target. #7679: each row records its holder (`outcome.holder`: host id, hostname, pid, process start; `previous_holder` after a recovery adopts it). A row whose lock is free is orphaned: `resumeWorktreeRefreshes` also completes an orphaned `syncing` row once every member's `last_commit` reached the target or a descendant, and a plain `gbrain sources refresh` on the owner host finishes any orphaned non-`recovery_required` row through the `--resume` path (`adoptOrphanedRefresh` + `resumeHeld`) before its own refresh, reporting it as `recovered`. A holder recorded on another host is never adopted. `refreshHolderReport` (writer status `worktree_refreshes`, sync status `worktree_refresh`) names the holder, `liveness` (`live` / `orphaned` / `unknown` off the lock's host) and a fix naming `--resume`. `--resume` converges and runs the member syncs; `--abandon` lifts a fence only with HEAD back at the start commit. `gbrain sources writer status` lists active refreshes as `worktree_refreshes` (state, heads, the next command). Doctor `worktree_refresh_stuck` lists rows active over 15 minutes. CLI body: `src/commands/sources-refresh.ts` (#7685: `--json` prints a refusal's structured `fix` when it has one; `refresh_in_progress` carries the writer-status read); while a resident PGLite owner (`gbrain serve`, stdio MCP) holds the database, `src/commands/sources-refresh-delegate.ts` runs the refresh inside it through the `writer_refresh` administration operation (trusted CLI registration only) and prints the same result or refusal. The consumer's idle probe applies `refreshFenceClear` to effects like the claims, so a fenced worktree's queued effects do not wake it. Tests: `test/persistence-worktree-refresh.test.ts`, `test/persistence-worktree-refresh-restart.test.ts` (child-process crash after merge), `test/worktree-refresh-orphan-7679.test.ts` (SIGKILL mid-syncing, live-holder no-steal), `test/oversize-hold-rescreen-8072.test.ts` (budget), Postgres via `test/e2e/worktree-refresh-*postgres.test.ts`, owner delegation in `test/persistence-sync-stdio-owner.test.ts`.
+- `src/core/persistence/git-gap.ts` + `src/commands/sources-git-gap.ts` + `src/commands/doctor/checks/db-ahead-of-git.ts` — git-canonical memory, wave G1 (`docs/designs/GIT_CANONICAL_MEMORY.md#the-git-gap-census`), read-only. `declaredDatabaseOnly` is the one predicate for pages that may not have a tracked file, tested in order: a read-only mirror source, a private page (`isPrivatePage`), an alias-private page (`isDerivationPrivate`), lineage-private (computed in SQL from `privatePagesFilterFragment`), `database_only_reason`, a `storage.db_only` prefix (`isSourceDbOnlySlug`, refusing an invalid gbrain.yml), a never-filed derive-phase page (page-prepare.ts `isNeverFiledDerivedPage`: no `source_path`/`source_uri` under `DERIVE_PHASE_DB_ONLY_DEFAULTS`), a path the checkout's Git ignores (`ignoredPaths`: filter-free `check-ignore -z --stdin`, tracked files never reported, exit 1 = none, null when Git cannot answer). The equality contract compares a committed blob with the page's export view (`readExportPage`, withdrawals overlaid; the census reads a batch's views with `readExportPages`, the same snapshots in two statements, #7617): an inline blob parsed as reconcile parses a canonical file, aligned with the stored type, subtype and #6429 stamp and compared by `reconcileCanonical` digest; a segmented page by the Git blob id of its streamed export. HEAD is read with hardened plumbing only (`readHeadTree`: one `ls-tree -r -l -z --full-tree` under the source's directory; `readBlobs`: `cat-file --batch` in calls of at most 64 MiB) and the working tree by hashing raw bytes (`worktreeBlobOid`), never `git status`/`git diff`, so no repository filter, textconv or fsmonitor runs. `gbrain sources git-gap <id> [--sample n] [--json]` (trusted local CLI, not an MCP op) puts every live page in one state (`equal`, `database_ahead`, `git_ahead` when the sync import record's `blob_oid`/`disk_blob_oid` differs from HEAD, else folded into `database_ahead` and counted in `database_ahead_without_import_record`, `declared_database_only` per class, `undeclared_database_only`, `worktree_only`, `unparseable`, `unsafe_path`), reports the working tree separately, and counts rows with no Git home (unfenced active facts, fenced facts on a missing page or a page without a committed file, links whose `link_source` is not in `DERIVED_LINK_SOURCES`, timeline rows on pages without a file), open `sync-hold` records by code and parked effect targets by kind and code. Each 256-page batch reads in its own read-only transaction. Doctor `db_ahead_of_git` runs the census per non-archived source whose directory is a Git checkout (others are only counted) under `GBRAIN_DOCTOR_GIT_GAP_TIMEOUT_MS` (default 10 s) and the entry deadline; it is always `ok` (informational) and lists totals only, a source the budget stopped or never reached as partial. Tests: `test/git-gap-census.test.ts` (PGLite and Postgres).
+- `src/core/persistence/git-declared.ts` + `src/core/persistence/git-canonical-schema.ts` (migration `git_canonical_declared`) — git-canonical memory, wave G7a: the declared database-only classes of a strict source (`sources.config.canonical = "git"`, default off). `strictPageClass` judges a page by its slug and prior file (`declaredPathClass`: `db_only_prefix` for a `storage.db_only` dir, `derive_folder` for a never-filed derive-phase page; G3's indexer reads the same function) and then its rendered content (`privacyClass`: `private`, `alias_private`, or `lineage_private` from the same SQL remote reads use, `privatePagesFilterFragment`, over the page's own values). `page-prepare.ts` `prepareFileTarget` asks it before it builds a strict file target: a declared page gets no target (no file, no door commit, no Git effect), an untracked ignored target is `git_ignored`, a page turning private gets the removal of the file it still has in the worktree or at HEAD, and a path whose `filter` attribute is `lfs` passes `assertLfsTarget` only with `storage.git_lfs` and a checkout that configures the LFS filter (`git_lfs_not_enabled`). `preparePrivateTransition` (page-prepare.ts) is that removal for other writers (GBRA-60). In `publishMutation` and `publishGroup`, `assertNoPrivateCommit` refuses a file target carrying a private page's bytes before anything is written (`git_private_commit_refused`; a group rolls back whole), and `declareStrictPublication` (after apply, before the door commit) refuses a declared page's file or an undeclared file-less page (`git_home_required`), stamps `pages.database_only_reason` and, for a privacy class, clears `source_path`/`source_uri` in the transaction of the door commit that removed the file. A managed import builds its own file target, so it meets only these publication checks. `assertStrictGitHome` is the refusal for writers with no file form yet (G4, G5). The managed writer guard (`writer-guard-schema.ts`) refuses rows of a strict source unless the writer declares `gbrain.git_protocol` (`withCoordinatedWrite` does), behind the `git_canonical_brain.strict_sources` counter kept by `sources` triggers (`git_protocol_required`, mapped in `publication-failure.ts`). The guard reads its state as the writing role and fails closed: a hidden or missing `persistence_brain` or `git_canonical_brain` row, or a source or page row hidden by row-level security, raises `writer_coordinator_required` (`managed_writer_guard:unreadable_state`), and the counter refuses a recount under RLS on `sources` (`test/writer-fence-fail-closed-7636.test.ts`). The census (`git-gap.ts`) adds `database_only_source` (a source with no checkout) and maps the stamped reasons to their own classes. Tests: `test/git-canonical-declared-g7a.test.ts` (both engines).
+- `src/core/persistence/checkout-state.ts` (`readCheckoutState`) — Q1: a canonical checkout against its last fetched upstream, read-only and time-bounded (no fetch): ahead/behind, `diverged`, each commit the upstream lacks (sha, author, date, subject, paths; at most 20) and porcelain counts. `sources writer status` reports it as `checkouts`; doctor `git_convergence` adds the commits and counts for a managed checkout and warns at once on a diverged one.
+- `src/core/persistence/git-door.ts`: G2, the door of git-canonical memory (Git is the source of truth, the database derives from it). A source with `sources.config.canonical = "git"` (strict, default off; set by `gbrain sources canonical <id> --to git --apply --expect <hash>`, `git-conversion.ts`) commits each published file before its database row. `publishMutation` (`coordinator.ts`) and `publishGroup` (`group-publish.ts`, one commit per group, after its lane turn) call `commitThroughDoor` under the writer lock, between the file rename and the transaction commit. It commits by plumbing without the checkout's index: `hash-object -w` per file, the trees along the changed paths rebuilt from HEAD (`cat-file tree`, `hash-object -w -t tree`), `commit-tree` with `commitGitTargets`' message, `update-ref` compare-and-swap (one rebuild if the branch moved, then a refusal), then `update-index --cacheinfo` for exactly the published paths (an entry someone staged during the write is left as it is, and the write says its commit landed). That refresh runs under Git's own index lock: `syncIndex` creates `<index>.lock` exclusively (an existing lock the #6932 proof finds abandoned is reclaimed under the caller's writer lock through `reclaimAbandonedIndexLock` in `effect-git.ts` and taken again; any other existing lock refuses with the landed failure, which says why it was not reclaimed), compares the entries, updates a hard-linked copy of the index through `GIT_INDEX_FILE`, and renames the copy onto the lock and the lock onto the index, so no writer that takes the lock can stage between the compare and the write; detached HEAD and unborn branches refuse typed. Each `DoorFile` carries its planned state (`planned`: the sha256 of the bytes the publication wrote, or null for a removal), and the door commits exactly that state or nothing: the file must hash to the plan before and after `hash-object -w` (else `git_target_changed`: nothing is committed and the edit stays on disk; the write is held in recovery as `unexpected_file_bytes` when the file holds neither its bytes nor the version before it, and fails with nothing held when the file holds the version before it), and each target's index entry must equal HEAD's (else `git_index_conflict`, index untouched). An entry that still holds the version its path's last commit replaced (an interrupted door refresh) counts as clean: the write restores it to HEAD and reports the paths as `outcome.persistence.git_index_restored` and the `git_index_restored` notice (`indexRestoredNotice`, emitted by `page-mutations.ts` `emitFenceNotice`). The step deadline is sized from one stat of the index (`gitStepTimeoutMs`), and `refuseIgnored` asks `check-ignore` only about new targets `ignore-rules.ts` cannot clear. Door commits are not coalesced across single writes (each holds its worktree lock exclusively and the consumer runs one per root); a batch commits once through publishGroup. The publication stamps the commit as `outcome.persistence.git_commit` (`git_state: committed`); a target Git refuses rolls the publication back. The Git effect carries `push_only` and joins the push without staging (`effects.ts`). The recovery record carries `door` (checkout and starting HEAD); `recoverPublication` keeps a file whose commit landed (`doorCommitLanded`) instead of restoring it, and the replay completes against it: `replayAfterDoorCommit` in `publishMutation` and the strict exceptions in `page-prepare.ts` (own bytes committed, or a removal HEAD already holds). Old binaries ignore `door`; strict stays off until conversion fences them. Tests: `test/git-door-strict-gbra133.test.ts` (both engines), `test/git-door-plumbing-gbra133.test.ts`, `test/git-door-planned-state.test.ts`, `test/git-door-write-cost.test.ts`, `test/git-door-index-lock-7742.test.ts` (both engines); per-write overhead bench `scripts/bench/door-write-overhead.ts` (opt-in, smoke `test/bench-door-write-overhead.test.ts`).
+- `src/core/persistence/ignore-rules.ts` — the ignore filter in front of `git check-ignore`; Git stays the only judge of an ignored path. `ignoreCandidates(root, paths)` returns every path some positive ignore rule could match, read from every `.gitignore` from the top level down to the path, `info/exclude` and `core.excludesFile`; a path it drops is one no rule could match. It over-approximates: negated rules are dropped, every wildcard matches any run of characters including `/`, directory-only rules match files, letters compare case-insensitively, each rule is tested against the path and every directory above it in its own and NFC spelling, and an unreadable or non-UTF-8 ignore file or a rule with a backslash or a character outside printable ASCII matches everything under its directory. Every path goes to Git on a sparse checkout or skip-worktree entry, a split or sparse index, an unresolvable `core.excludesFile` or include, a gitlink or symlink along the path, or a failed probe. The probe (rev-parse layout and the configuration with its origin files) is cached per checkout and taken again when a configuration file it read or could read, HEAD or the configuration environment changes; ignore files are re-read on every call and parsed once per content. Callers: `git-door.ts` `refuseIgnored` and `sync-origin-evidence.ts` `gitIgnoresUntracked`. Tests: `test/ignore-rules.test.ts`.