Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(verify): ledger read helper outside the loop, operation markers probed in the adoption, stale-save versions matched by hash (GST-1623, #8755)
garrytand390c79528f7
diff --git a/docs/architecture/key-files/core-persistence-request-indexes.md b/docs/architecture/key-files/core-persistence-request-indexes.mdindex 079234a828..a397ffac65 100644--- a/docs/architecture/key-files/core-persistence-request-indexes.md+++ b/docs/architecture/key-files/core-persistence-request-indexes.md@@ -17,4 +17,4 @@ The online-built request-table indexes (the managed sync checkpoint pair, the pa- `src/core/persistence/hosted-repo.ts` with `src/commands/sources-host-repo.ts` — G7b-2, `gbrain sources host-repo <id> --dir <path> [--push-remote <url>] [--dry-run | --expect <plan_hash>]`: a hosted brain gives an empty, unbound source a bare repository (`<dir>/repo.git`, branch `main`, one commit holding an empty `.gitignore`, repository-local identity `gbrain`) and checks it out as a linked worktree (`<dir>/worktree`) claimed through `runManagedSourceLifecycle` (`add` or `claim`, every physical-root and manifest check), with `config.canonical = "git"` and `config.hosted_repo` set in the claim's own transaction. The door then commits there unchanged. A pre-receive hook (`core.hooksPath` pinned to the repository's `hooks/`) refuses every push, since Git's checked-out-branch refusal ends once the worktree is pruned. The preview (the default without `--expect`) prints the plan in words and a `plan_hash` over the source incarnation, canonical setting, directory state, push remote and hook; the apply refuses `preview_changed` when it moved. Refuses `strict_requires_managed` on an unmanaged brain, `hosted_repo_refused` (`source_bound`, `source_has_path`, `source_has_pages`, `source_archived`, `dir_not_empty`) and `overlapping_path` (a directory in a checkout or a source's directory). On a hosted source the preview reports the hook (`ok`/`missing`/`changed`) and the worktree link, and the apply reinstalls the hook. Trusted local CLI only; no operation. Tests: `test/hosted-repo-g7b2.test.ts` (both engines).- `src/core/persistence/worktree-link.ts` `linkedWorktreeProblem` / `assertLinkedWorktreeIntact` — a claim (`localRoot` in `source-lifecycle.ts`) refuses `git_worktree_link_broken` for a linked worktree whose `.git` file names a repository entry that is gone (pruned) or whose entry names another path (the checkout moved); a submodule's `.git` file is checked for existence only. Tests: `test/hosted-repo-g7b2.test.ts`.- `src/core/persistence/sync-receipt-wait.ts` — how long a managed sync pass waits on its own admitted write, in one place: `syncReceiptWait()` returns `pass` (5 s), `checkpoint` (8 s), `held` (30 s, while this process's consumer still holds the write) and `drain` (30 s, a drain's re-entry). `sync-run.ts` reads every wait from it. `__setSyncReceiptWaitForTests(partial | null)` is the test seam (returns a restore function), so a test outlasts a wait in milliseconds instead of sleeping past 30 s.-- **`src/core/persistence/local-edit-adoption.ts`** — GST-1601 part 2: with `persistence.adopt_local_edits.source.<id>` on, `finishUnpublishedFailure` (`coordinator.ts`) hands a `remember` / `capture` / `add_timeline_entry` refused `file_database_drift` to `adoptLocalEdit`, which reads the drifted file the refusal carries (`localEdit`, set in `page-prepare.ts` compareFileTarget), waits until it is quiet (`persistence.adopt_quiet_ms`, empty reads as the 5000 default, at most three windows and never past 30 s: the caller's claim holds its worktree queue meanwhile), and adopts only a one-sided, safe edit (#8755: a file over `PAGE_MAX_BYTES`, checked by stat before reading, a segmented page, an unmerged path or an operation in progress (`pending-import.ts` `unmergedPath` / `mergeInProgress`), and a stale editor save (`staleEditorSave`: the file equals an earlier `page_versions` body, or lacks a list or table row the current revision has and its newest archived version had not) keep the refusal; the repo-less test walks up for `.git`): HEAD's blob matches the database copy (`fileMatchesSnapshot`; a repo-less brain skips this), `protectedLocalEdit` finds no title, type or protected-key change, and `prepareRepairPublication` is `ready`. The adoption is a `managed_file_repair` (`file-repair.ts`) carrying `adoptLocalEdit` (`{ blob, replaced_revision }`), admitted under the CLI writer with a request id derived from the caller's (one adoption per write), claimed and published inline (`publishMutation`, retrying a busy worktree lock for 10 s) so it lands before the caller: it publishes the file's unchanged bytes with the `localEditAdoptionCommit` note, stamps the archived replaced revision `change_reason = 'adopted_local_edit'`, and commits that one file inside its publication transaction (`afterEffects` → `commitAdoptedLocalEdit`; a strict source's door commits it instead). #8755: a publication that fails after that commit is requeued by recovery and `publishAdoption` publishes it again (it retries any release within its 10 s): `adoptedHeadHolds` accepts HEAD holding the adopted blob and the replay names the landed commit (`landed_before_row`). The intent names the caller's claim (`adoptLocalEdit.caller`); `assertAdoptionCaller` (`file-repair.ts` preparation) refuses `write_claim_lost` once that claim is gone, so an adoption orphaned by a crash publishes nothing; its own claim carries the owner stamp. `settleAdoption` waits up to 120 s more on an adoption whose publication already started; a caller whose claim was lost is returned as `claimLost`, never `released`; an admission failure logs `[persistence] phase=adoption`. The caller is then released with `intent.adopted_local_edit` and prepared again; `withAdoptedLocalEdit` (`service.ts` preparePersistedMutation) puts `adopted` on its receipt. A file changed under the adoption ends both `raw_file_changed`. Commit-ledger lines carry `adopted` (`adoptedBy`, `effect-model.ts`). Doctor `local_edit_adoptions` (`src/commands/doctor/checks/local-edit-adoptions.ts`) lists adoptions of the last 24 h per source from `outcome.adopted_local_edit` (kept by compaction), always ok. Test: `test/adopt-local-edit.test.ts`.+- **`src/core/persistence/local-edit-adoption.ts`** — GST-1601 part 2: with `persistence.adopt_local_edits.source.<id>` on, `finishUnpublishedFailure` (`coordinator.ts`) hands a `remember` / `capture` / `add_timeline_entry` refused `file_database_drift` to `adoptLocalEdit`, which reads the drifted file the refusal carries (`localEdit`, set in `page-prepare.ts` compareFileTarget), waits until it is quiet (`persistence.adopt_quiet_ms`, empty reads as the 5000 default, at most three windows and never past 30 s: the caller's claim holds its worktree queue meanwhile), and adopts only a one-sided, safe edit (#8755: a file over `PAGE_MAX_BYTES`, checked by stat before reading, a segmented page, an unmerged path or an operation in progress (`pending-import.ts` `unmergedPath` / `operationMarkerPaths`), and a stale editor save (`staleEditorSave`: the file equals an earlier `page_versions` body, or lacks a list or table row the current revision has and its newest archived version had not) keep the refusal; the repo-less test walks up for `.git`): HEAD's blob matches the database copy (`fileMatchesSnapshot`; a repo-less brain skips this), `protectedLocalEdit` finds no title, type or protected-key change, and `prepareRepairPublication` is `ready`. The adoption is a `managed_file_repair` (`file-repair.ts`) carrying `adoptLocalEdit` (`{ blob, replaced_revision }`), admitted under the CLI writer with a request id derived from the caller's (one adoption per write), claimed and published inline (`publishMutation`, retrying a busy worktree lock for 10 s) so it lands before the caller: it publishes the file's unchanged bytes with the `localEditAdoptionCommit` note, stamps the archived replaced revision `change_reason = 'adopted_local_edit'`, and commits that one file inside its publication transaction (`afterEffects` → `commitAdoptedLocalEdit`; a strict source's door commits it instead). #8755: a publication that fails after that commit is requeued by recovery and `publishAdoption` publishes it again (it retries any release within its 10 s): `adoptedHeadHolds` accepts HEAD holding the adopted blob and the replay names the landed commit (`landed_before_row`). The intent names the caller's claim (`adoptLocalEdit.caller`); `assertAdoptionCaller` (`file-repair.ts` preparation) refuses `write_claim_lost` once that claim is gone, so an adoption orphaned by a crash publishes nothing; its own claim carries the owner stamp. `settleAdoption` waits up to 120 s more on an adoption whose publication already started; a caller whose claim was lost is returned as `claimLost`, never `released`; an admission failure logs `[persistence] phase=adoption`. The caller is then released with `intent.adopted_local_edit` and prepared again; `withAdoptedLocalEdit` (`service.ts` preparePersistedMutation) puts `adopted` on its receipt. A file changed under the adoption ends both `raw_file_changed`. Commit-ledger lines carry `adopted` (`adoptedBy`, `effect-model.ts`). Doctor `local_edit_adoptions` (`src/commands/doctor/checks/local-edit-adoptions.ts`) lists adoptions of the last 24 h per source from `outcome.adopted_local_edit` (kept by compaction), always ok. Test: `test/adopt-local-edit.test.ts`.diff --git a/scripts/page-body-readers-baseline.tsv b/scripts/page-body-readers-baseline.tsvindex 06b68e76a8..716bdde82c 100644--- a/scripts/page-body-readers-baseline.tsv+++ b/scripts/page-body-readers-baseline.tsv@@ -459,6 +459,8 @@ src/core/persistence/large-page-guard.ts assertLargePageAdmission sql META storesrc/core/persistence/links-maintenance.ts extractManagedStaleLinks field NONE not the page body: result.timeline countersrc/core/persistence/links-maintenance.ts formatManagedStaleExtraction field NONE not the page body: result.timeline countersrc/core/persistence/links-preparation.ts prepareAutomaticLinks field STREAM reads the incoming parsed page, not a stored body+src/core/persistence/local-edit-adoption.ts staleEditorSave field INLINE #8755 stale editor save: adoptionIntent refuses a segmented page before and assertInlineBody asserts it; the equality with earlier versions is an md5 match in SQL, and the row check needs the whole current body and the newest inline version (rows can sit anywhere, and inline versions carry no content_hash)+src/core/persistence/local-edit-adoption.ts staleEditorSave sql INLINE #8755 stale editor save: adoptionIntent refuses a segmented page before and assertInlineBody asserts it; the equality with earlier versions is an md5 match in SQL, and the row check needs the whole current body and the newest inline version (rows can sit anywhere, and inline versions carry no content_hash)src/core/persistence/loop-fact-retirement.ts prepareLoopFactRetirement call:serializePageToMarkdown SEGWsrc/core/persistence/loop-fact-retirement.ts prepareLoopFactRetirement field SEGWsrc/core/persistence/manual-links.ts prepareManualLinkMutation call:serializePageToMarkdown SEGW wholeBodySnapshot first: a segmented page within 24 MiB is read back whole, a larger one refusesdiff --git a/src/core/git/commit-ledger.ts b/src/core/git/commit-ledger.tsindex 51a3c630af..fda616b5d3 100644--- a/src/core/git/commit-ledger.ts+++ b/src/core/git/commit-ledger.ts@@ -168,6 +168,12 @@ export async function recordIntentAsync(root: string, intent: LedgerIntent): Proawait writeLockedAsync(root, loc => intentRecord(loc, intent));}+/** One ledger file's text; null when it does not exist, and any other read error is thrown (GST-1623). */+function readLedgerFile(file: string): string | null {+ try { return readFileSync(file, 'utf8'); }+ catch (error) { if ((error as NodeJS.ErrnoException).code === 'ENOENT') return null; throw error; }+}+/*** The retained ledger lines, oldest first (the rotated file, then the current one); unparseable lines are skipped. Only a* missing file reads as empty: any other read error throws (GST-1623), so doctor never reads an unreadable ledger as@@ -176,10 +182,8 @@ export async function recordIntentAsync(root: string, intent: LedgerIntent): Proexport function readLedger(common: string): LedgerRecord[] {const files = ledgerFiles(common);const out: LedgerRecord[] = [];- for (const file of [files.rotated, files.current]) {- let text: string;- try { text = readFileSync(file, 'utf8'); }- catch (error) { if ((error as NodeJS.ErrnoException).code === 'ENOENT') continue; throw error; }+ for (const text of [readLedgerFile(files.rotated), readLedgerFile(files.current)]) {+ if (text === null) continue;for (const raw of text.split('\n')) {if (!raw) continue;try {diff --git a/src/core/persistence/local-edit-adoption.ts b/src/core/persistence/local-edit-adoption.tsindex 4e896fd88a..fda8b9ff8b 100644--- a/src/core/persistence/local-edit-adoption.ts+++ b/src/core/persistence/local-edit-adoption.ts@@ -36,7 +36,7 @@ import type { BrainEngine } from '../engine.ts';import type { GBrainConfig } from '../config.ts';import { OperationError, opError, type OperationContext } from '../ops/contract.ts';import { PAGE_MAX_BYTES } from '../page-size-policy.ts';-import { isSegmentedPage } from '../page-body/page-shape.ts';+import { assertInlineBody, isSegmentedPage } from '../page-body/page-shape.ts';import { parseMarkdown } from '../markdown.ts';import type { PageSnapshot } from '../page-state/types.ts';import { gitBlobOid, sha256 } from './digest.ts';@@ -50,7 +50,7 @@ import { declarePersistenceProtocol } from './protocol.ts';import type { WorktreeBinding } from './ownership.ts';import { getWorktreeBinding } from './ownership.ts';import { fileMatchesSnapshot, protectedLocalEdit } from './page-prepare.ts';-import { blobText, headBlob, headCommit, landedCommit, mergeInProgress, unmergedPath } from './pending-import.ts';+import { blobText, headBlob, headCommit, landedCommit, operationMarkerPaths, unmergedPath } from './pending-import.ts';import { claimOwner } from './claim-phase.ts';import type { PreparedMutation } from './coordinator.ts';import type { GitCommitNote } from './effect-model.ts';@@ -117,19 +117,25 @@ function fenceRows(...texts: Array<string | null | undefined>): Set<string> {return new Set(texts.flatMap(text => (text ?? '').split('\n')).map(line => line.trim()).filter(line => /^(- |\|)/.test(line) && !/^\|[\s:|-]*\|?$/.test(line)));}/**- * #8755: a stale editor save: the file equals an earlier stored revision's body, or lacks a Facts or timeline row the last- * revision added (rows the page holds that its newest archived version, the revision before, did not).+ * #8755: a stale editor save: the file equals one of the last STALE_REVISIONS stored revisions' bodies (matched by hash in+ * SQL, no version body is loaded), or lacks a Facts or timeline row the last revision added (rows the page holds that its+ * newest archived version, the revision before, did not). Inline pages only: adoptionIntent refuses a segmented page first.*/async function staleEditorSave(engine: BrainEngine, slug: string, text: string, snapshot: PageSnapshot, activePack: unknown): Promise<boolean> {+ assertInlineBody(snapshot.page, 'staleEditorSave');const file = parseMarkdown(text, slug, { activePack } as never);- const body = (compiled: string | null | undefined, timeline: string | null | undefined) => `${(compiled ?? '').trim()}\0${(timeline ?? '').trim()}`;- const versions = await engine.executeRaw<{ compiled_truth: string | null; timeline: string | null }>(- 'SELECT compiled_truth, timeline FROM page_versions WHERE page_id=$1 AND compiled_truth IS NOT NULL ORDER BY id DESC LIMIT $2', [snapshot.page.id, STALE_REVISIONS]);- const current = body(snapshot.page.compiled_truth, snapshot.page.timeline);- const mine = body(file.compiled_truth, file.timeline);- if (mine !== current && versions.some(version => body(version.compiled_truth, version.timeline) === mine)) return true;- if (!versions.length) return false;- const before = fenceRows(versions[0]!.compiled_truth, versions[0]!.timeline);+ const [compiled, timeline] = [(file.compiled_truth ?? '').trim(), (file.timeline ?? '').trim()];+ if (compiled !== (snapshot.page.compiled_truth ?? '').trim() || timeline !== (snapshot.page.timeline ?? '').trim()) {+ const [earlier] = await engine.executeRaw<{ same: boolean }>(`SELECT EXISTS (SELECT 1 FROM (SELECT compiled_truth, timeline FROM page_versions+ WHERE page_id=$1 AND compiled_truth IS NOT NULL ORDER BY id DESC LIMIT $2) v+ WHERE md5(btrim(v.compiled_truth, E' \\t\\r\\n'))=md5($3) AND md5(btrim(COALESCE(v.timeline, ''), E' \\t\\r\\n'))=md5($4)) AS same`,+ [snapshot.page.id, STALE_REVISIONS, compiled, timeline]);+ if (earlier?.same) return true;+ }+ const [previous] = await engine.executeRaw<{ compiled_truth: string; timeline: string | null }>(+ 'SELECT compiled_truth, timeline FROM page_versions WHERE page_id=$1 AND compiled_truth IS NOT NULL ORDER BY id DESC LIMIT 1', [snapshot.page.id]);+ if (!previous) return false;+ const before = fenceRows(previous.compiled_truth, previous.timeline);const kept = fenceRows(file.compiled_truth, file.timeline);return [...fenceRows(snapshot.page.compiled_truth, snapshot.page.timeline)].some(line => !before.has(line) && !kept.has(line));}@@ -150,7 +156,7 @@ async function adoptionIntent(engine: BrainEngine, row: WriteRequest, target: Loconst head = commit ? await headBlob(target.root, target.path) : undefined;if (head === null) return null;// Conflict markers from a merge, stash pop, cherry-pick or rebase are never adopted (`git add` would mark them resolved).- if (commit && (await unmergedPath(target.root, target.path) !== false || await mergeInProgress(target.root) !== false)) return null;+ if (commit && (await unmergedPath(target.root, target.path) !== false || (await operationMarkerPaths(target.root))?.every(marker => !existsSync(marker)) !== true)) return null;const { prepareRepairPublication, repairScreenConfig } = await import('./file-repair.ts');const screen = await repairScreenConfig(engine, row.source_id);// A checkout whose HEAD holds the database's copy: only the worktree file moved. A repo-less brain has no HEAD to ask.diff --git a/src/core/persistence/pending-import.ts b/src/core/persistence/pending-import.tsindex 841c548c93..5237c9b305 100644--- a/src/core/persistence/pending-import.ts+++ b/src/core/persistence/pending-import.ts@@ -17,7 +17,6 @@* is resubmitted by its caller after that sync.*/import { spawn } from '../spawn.ts';-import { existsSync } from 'node:fs';import { relative, sep } from 'node:path';import type { BrainEngine } from '../engine.ts';import type { Action } from '../agent-output.ts';@@ -127,11 +126,15 @@ export async function unmergedPath(root: string, path: string, signal?: AbortSig.then(out => out.length > 0, error => { if (signal?.aborted) throw error; return null; });}-/** #8755: whether a merge, cherry-pick, revert or rebase is in progress in `root`'s checkout; null when Git cannot say. */-export async function mergeInProgress(root: string, signal?: AbortSignal): Promise<boolean | null> {+/**+ * #8755: the absolute paths of the markers a merge, cherry-pick, revert or rebase in progress leaves in `root`'s git dir+ * (MERGE_HEAD, CHERRY_PICK_HEAD, REVERT_HEAD, rebase-merge, rebase-apply); null when Git cannot name them. The caller+ * checks which exist.+ */+export async function operationMarkerPaths(root: string, signal?: AbortSignal): Promise<string[] | null> {const markers = ['MERGE_HEAD', 'CHERRY_PICK_HEAD', 'REVERT_HEAD', 'rebase-merge', 'rebase-apply'];return gitRead(root, ['rev-parse', '--path-format=absolute', ...markers.flatMap(marker => ['--git-path', marker])], signal)- .then(out => { const paths = out.split('\n').filter(Boolean); return paths.length === markers.length ? paths.some(path => existsSync(path)) : null; },+ .then(out => { const paths = out.split('\n').filter(Boolean); return paths.length === markers.length ? paths : null; },error => { if (signal?.aborted) throw error; return null; });}