Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
docs(key-files): keep canonical-reconciliation.md under its 60000-byte cap after the train merges
garrytan7d4e28485203
diff --git a/docs/architecture/key-files/canonical-reconciliation.md b/docs/architecture/key-files/canonical-reconciliation.mdindex 63fc8cbce6..03d863dc2b 100644--- a/docs/architecture/key-files/canonical-reconciliation.md+++ b/docs/architecture/key-files/canonical-reconciliation.md@@ -17,7 +17,7 @@ See [generations](generations.md) and [continued entries](canonical-reconciliati- `src/core/persistence/managed-mode.ts` — the one reader of the brain's persistence mode (#6482). `readManagedMode` / `readPersistenceBrain` accept exactly one `persistence_brain` row with a boolean `enabled` (optionally `FOR SHARE` / `FOR UPDATE`) and refuse anything else (`storage_error`, `detail: persistence_mode_unknown`, fix `gbrain sources writer status --probe --json`); only a well-formed `enabled: false` row selects the unmanaged path, so a missing, duplicated or malformed row can never route a write around the owner. Every non-test reader of the mode goes through it (`test/persistence-mode-reader-guard.test.ts` lists the few schema and admin-intent files allowed their own statement); `ownership.ts` re-exports it for callers that already import ownership.- `src/core/persistence/sync-slices.ts` — #6349 path-sliced staging for a managed target that exceeds the cursor bound with no commit axis to stage on (a first sync, a `--full` tree walk, one commit above the bound). `sliceEntries` cuts the whole resolved manifest in `(phase, code-unit path)` order: every import slice, then bounded deletion slices, the longest prefix the bound admits. The slice cursor `{ phase, after, head, target }` lives in `op_checkpoints` (`managed-sync-slice`, keyed by source, incarnation and the run's cursor options so a resumed invocation with other exclusions starts over), written or cleared by the slice's checkpoint in the same transaction that leaves `last_commit` untouched until the last slice settles; `sync-discovery.ts` resumes an unfinished target pinned at its recorded commit.-- **`src/core/persistence/sync-discovery.ts`** + **`sync-origin.ts`** — managed owner-only Git discovery with source-relative exclusions and one-statement page identity/revision/liveness snapshots. A non-Git root refuses `sync_not_applicable`. #6517: `selectOriginClaimant` (sync-origin.ts) is the one origin rule, applied by the discovery census and by `assertSyncPageOrigin` at preparation and publication: a live page wins over tombstones, several live pages refuse `page_identity_changed` naming the soft-delete exit (`sharedOriginExit`), a single tombstone may be reused, several tombstones stay ambiguous. Reserved skillpack paths (`skills/` at any depth and `skillpack.json`, `src/core/skill-reserved-paths.ts`) never enter discovery: the shared skill publisher owns them and `managedImportContent` refuses them, so a seeded pack cannot block a sync. Nested full discovery filters source scope before slicing paths. Windows separator aliases retain their page identity; ambiguous aliases refuse, while POSIX backslashes/case remain exact. Under `source-root` mode a stored `<scope>/<path>` origin (the Git-root form older write-through minted from the slug, recognized because `<path>` names the page slug) is accepted as `<path>` at discovery, freeze, preparation and the page-origin query, and an import rewrites it in place; when both readings name existing files the stored origin refuses with `detail: ambiguous_source_path`. A genuine origin mismatch refuses with `detail: sync_origin_mismatch` and the filled paths. Deletions must be absent from the pinned Git target or their explicitly selected working tree. Deleting a file that no page records as its origin (for example a skipped slug twin) while its slug's page keeps another origin commits an explicit no-op receipt (`unownedDeletion`, outcome reason `unowned_deleted_path`) that preserves that live page or tombstone; freeze and publication re-prove that no page owns the path and that the file is still absent. Existing cursors reuse their accepted manifests, with unknown old deletion provenance refused rather than rewritten. Active Git holds of the source are re-screened when their blob or working bytes changed, the file left, a `sources retry-held` request names them, the hold predates the current `RECOVERY_VERSION`, or the code is `frontmatter_slug_conflict`, `file_too_large`, `rename_held` or `parser_regression`; holds of now-excluded paths are released at the checkpoint. #6278 (B7): `retryTaken` lists every scheduled path the discovery read (screened as an entry, as a candidate, or no longer eligible); `sync-run.ts` clears them in the transaction that saves the fresh cursor. A held rename destination (or a held path renamed again) carries the hold's `renameFrom` when the old page is unchanged, else `renameHeld` (held as `rename_held` / `rename_source_changed`), and a full walk never deletes an origin a hold's `rename_from` names. Imports whose origin has no page claim their slug together: a live page whose own file is still in the tree keeps the slug, otherwise the file named like the slug (else the first path) wins, and the losers leave the manifest and are reported as `slugCollisions` (the library path's `slug_collision` skip) instead of failing the run. A winner whose slug is held by a page with a vanished or deleted origin takes that page over (`renameFrom`, same slug). A Git-detected rename to an unoccupied slug becomes one import carrying `renameFrom` in place of a delete plus a create. Origins for one slug that differ only by case or separator spelling refuse, except a case-only respelling whose old file the same sync removes. Code files keep exact per-origin identity. On Windows a path containing `:` is reported in `SyncDiscovery.fileRefusals` (code `colon_slug_windows_write_through`, with its filled `gbrain sync --source <id> --no-pull` hint) and kept out of the run and its origin checks, so one such file no longer fails the whole sync (#5032). `resolveManagedSyncContext` refuses `skipFailed` / `includeGitignored` on a managed source with `managed_sync_flag_refused` (`managed-sync-flags.ts`, `detail` = the flag, fix = the same `gbrain sync` argv without it), the one refusal jobs, MCP and the cycle reach; `gbrain sync` raises it at parse, before the legacy ledger acknowledgement and the cost gate, `--all` included (#6696). #6865: the per-path eligibility (scope, `sync.exclude`, `sync.include_hidden`, strategy, reserved skillpack paths) and the origin a path maps to under the slug mode come from one exported `loadSyncEligibility(engine, context, opts?, company?)`, which the unindexed-file detector shares.+- **`src/core/persistence/sync-discovery.ts`** + **`sync-origin.ts`** — managed owner-only Git discovery with source-relative exclusions and one-statement page identity/revision/liveness snapshots. A non-Git root refuses `sync_not_applicable`. #6517: `selectOriginClaimant` (sync-origin.ts) is the one origin rule, applied by the discovery census and by `assertSyncPageOrigin` at preparation and publication: a live page wins over tombstones, several live pages refuse `page_identity_changed` naming the soft-delete exit (`sharedOriginExit`), a single tombstone may be reused, several tombstones stay ambiguous. Reserved skillpack paths (`skills/` at any depth and `skillpack.json`, `src/core/skill-reserved-paths.ts`) never enter discovery: the shared skill publisher owns them and `managedImportContent` refuses them, so a seeded pack cannot block a sync. Nested full discovery filters source scope before slicing paths. Windows separator aliases retain their page identity; ambiguous aliases refuse, while POSIX backslashes/case remain exact. Under `source-root` mode a stored `<scope>/<path>` origin (the Git-root form older write-through minted from the slug, recognized because `<path>` names the page slug) is accepted as `<path>` at discovery, freeze, preparation and the page-origin query, and an import rewrites it in place; when both readings name existing files the stored origin refuses with `detail: ambiguous_source_path`. A genuine origin mismatch refuses with `detail: sync_origin_mismatch` and the filled paths. Deletions must be absent from the pinned Git target or their explicitly selected working tree. Deleting a file that no page records as its origin while its slug's page keeps another origin commits an explicit no-op receipt (`unownedDeletion`, outcome reason `unowned_deleted_path`) that preserves that live page or tombstone; freeze and publication re-prove that no page owns the path and that the file is still absent. Existing cursors reuse their accepted manifests, with unknown old deletion provenance refused rather than rewritten. Active Git holds of the source are re-screened when their blob or working bytes changed, the file left, a `sources retry-held` request names them, the hold predates the current `RECOVERY_VERSION`, or the code is `frontmatter_slug_conflict`, `file_too_large`, `rename_held` or `parser_regression`; holds of now-excluded paths are released at the checkpoint. #6278 (B7): `retryTaken` lists every scheduled path the discovery read (screened as an entry, as a candidate, or no longer eligible); `sync-run.ts` clears them in the transaction that saves the fresh cursor. A held rename destination (or a held path renamed again) carries the hold's `renameFrom` when the old page is unchanged, else `renameHeld` (held as `rename_held` / `rename_source_changed`), and a full walk never deletes an origin a hold's `rename_from` names. Imports whose origin has no page claim their slug together: a live page whose own file is still in the tree keeps the slug, otherwise the file named like the slug (else the first path) wins, and the losers leave the manifest and are reported as `slugCollisions` (the library path's `slug_collision` skip) instead of failing the run. A winner whose slug is held by a page with a vanished or deleted origin takes that page over (`renameFrom`, same slug). A Git-detected rename to an unoccupied slug becomes one import carrying `renameFrom` in place of a delete plus a create. Origins for one slug that differ only by case or separator spelling refuse, except a case-only respelling whose old file the same sync removes. Code files keep exact per-origin identity. On Windows a path containing `:` is reported in `SyncDiscovery.fileRefusals` (code `colon_slug_windows_write_through`, with its filled `gbrain sync --source <id> --no-pull` hint) and kept out of the run and its origin checks (#5032). `resolveManagedSyncContext` refuses `skipFailed` / `includeGitignored` on a managed source with `managed_sync_flag_refused` (`managed-sync-flags.ts`, `detail` = the flag, fix = the same `gbrain sync` argv without it), the one refusal jobs, MCP and the cycle reach; `gbrain sync` raises it at parse, before the legacy ledger acknowledgement and the cost gate, `--all` included (#6696). #6865: the per-path eligibility (scope, `sync.exclude`, `sync.include_hidden`, strategy, reserved skillpack paths) and the origin a path maps to under the slug mode come from one exported `loadSyncEligibility(engine, context, opts?, company?)`, which the unindexed-file detector shares.- `src/core/hardened-git.ts` (#6712 additions; base entry in [persistence-stalls](persistence-stalls.md)) — `assertHardenedGitArgs` checks exact options in every wrapper: `cat-file` with at most `--batch` (never `--filters`, `--textconv` or `--batch-command`), `hash-object --no-filters` optionally with `--stdin-paths`, `check-attr`/`check-ignore` with `-z`/`--stdin`, and `diff-tree` only as `-r -z --no-renames`. `hardenedGitStream` is a third wrapper: it spawns async, streams stdout to a callback, feeds stdin with backpressure, SIGKILLs past its timeout or byte cap and rejects when the consumer or stdin iterable throws; `NulRecordSplitter` and `hardenedStreamComplete` give the completeness rule (exit 0, no signal, output ending at a NUL or empty). Tests: `test/hardened-git-stream-6712.test.ts`, `test/hardened-git-filters.test.ts`.- `src/core/persistence/sync-git-stream.ts` — the managed-sync reconcile's Git level reads (#6712), all through `hardenedGitStream` (allow-listed, filter-free plumbing; the checkout's clean/process filters and textconv never run) and parsed as they stream, so memory is the result maps. `readTreeLevel` (`ls-tree -r -z <commit> -- <scope>`: blobs plus symlinks and gitlinks with their mode), `readIndexLevel` (`ls-files -s -z`, the index only; a conflicted path keeps its highest stage) and `readRenameHints` (`diff-tree -r -z --no-renames`: a deleted and an added regular file paired only when their blob OID is unique to them; hints, never candidates) throw `SyncTreeReadIncomplete` (code `reconcile_read_incomplete`) unless Git exited 0, was not signalled and its output ended at a NUL. `readBlobs` reads contents through one `cat-file --batch`, leaving out missing, oversized and over-total objects. `probeDirtiness` is the batched `hardenedPathDirty` (`Dp`): per batch of 2,000 paths one `hash-object --no-filters --stdin-paths` and one `check-attr -z --stdin`, presence and bytes from `readSyncFile`; verdicts in order `untracked`, `staged` (index and tree differ in OID or mode, a non-zero stage, or one side missing), `deleted`, `unknown` (symlink or gitlink, unreadable file, a path with a newline or a leading quote, any Git failure, an OID that disagrees with the bytes read), then `clean` (raw OID equals the tree with the executable bit matching), `clean_eol` (only `text`/`eol` set and the CRLF-folded bytes hash to the tree), `clean_lfs` (a valid pointer whose sha256 and size match the bytes) or `modified`; another filter, `working-tree-encoding`, an executable-bit-only change or CRLF bytes that fold to the tree with no attribute are `unknown`, never clean. Tests: `test/hardened-git-stream-6712.test.ts` (every verdict, completeness, and a forced probe that 1,000 passes of every read never run a configured filter or textconv while `git status`/`git diff` on the same checkout do). Working mode: `probeDirtiness(..., { working: true })` never reads the index (a path with no file is `deleted`, a present file outside the tree `untracked` with its raw OID, every other present file judged from its bytes against the tree through the same `clean_eol`/`clean_lfs` domains), so a staged edit, a staged new file, `git rm --cached`, a staged deletion and a staged file removed from disk probe as the same disk does with a pristine index: the working level is the physical bytes (design v3.2 §2.1). `hashWorkingLevel` hashes every path of a working round that way with hash-then-restat (a path whose bigint `lstat` moved across the hash is rehashed, up to three attempts) and returns the stat cache it leaves: `WorkingStatCache` is plain JSON (`{v: 1, root, entries: {path: 'dev:ino:mode:uid:size:mtime_ns:ctime_ns' | 'absent'}}`), an entry is written only for a stable stat whose `mtime` and `ctime` are older than 2 s before the attempt began (Git's racy rule) and a probe with a level, and at most `WORKING_STAT_CACHE_MAX_ENTRIES` (100,000) entries are kept in path order. `workingChanged(cache, paths)` is the idle tick's check: a path the cache does not hold, a moved stat, a failed stat, a zero `ctime`, a missing cache or a moved root all read as changed, never as unchanged. `parseWorkingStatCache` validates a serialized copy (junk is null, so everything reads changed); `rememberWorkingStatCache`/`recallWorkingStatCache` keep up to 64 caches per process. Working-mode tests: `test/sync-working-level-6712.test.ts` (probe 41, on the Git reads and on a managed PGLite pass), `test/sync-working-stat-cache-6712.test.ts` (probe 42); the probe 28 marker test also runs `hashWorkingLevel`.- `src/core/persistence/sync-classify.ts` — the managed-sync level classifier (#6712 PR 4, design v3.2 §2.3): pure (no I/O), total, ordered, first match over one path's facts (`SyncPathState`: mode, `T[p]`, the level `L`, presence, `Dp`, the page and its import record, the attestation of its revision, hold, deletion marker, baseline blob, open work). Rows in order: `held`, `spelling`, `untracked`, `kept` (and an absent db_only page), `delete`, `delete_drift`, `gone`, `deleted_same`, `deleted_edited`, `legacy_same`, `dirty`, `matched`, `adopt`, `reparse`, `db_ahead` (sub-labels `pending`/`db_only`/`unproven`; keep_db writes only `db_only` on a clean committed path; an open full obligation skips `db_only`/`unproven`), `import`, `legacy_unverified`, `keep_db`, `drift` (side `both` or `unknown`); then the overlay (unsettled with open work is `in_flight`). `syncClassAction` is the glossary tag (`none`, `owner`, `repair`); `syncPassKind` decides `idle`, `hold_only` or `full` from the converged state and every obligation. Tests: `test/sync-classify-6712.test.ts` (each row, and a cross product against a reference model).