Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Count a blob sync adopted as bytes the database held, and name the files a deleted-side database-wins apply removes in its consent (GST-2579)
garrytanc4f4946245c1
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex be51574d88..940bdb3d38 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -14,11 +14,11 @@ identifiers and attribution are available in the pre-removal Git revision### A database-wins file-drift repair no longer discards a newer file edit-**`gbrain repair file-drift --prefer database` and `gbrain sources reconcile --apply` could overwrite a change to a page's file that sync had not imported yet, or bring back a file another writer had just deleted, with nothing in the preview saying so.** gbrain's own hold messages pointed agents at that command, so the loss was easy to walk into. A repair now checks each page's file against sync's import record before it takes the database copy.+**`gbrain repair file-drift --prefer database` and `gbrain sources reconcile --apply` could overwrite a change to a page's file that sync had not imported yet, or bring back a file another writer had just deleted, with nothing in the preview saying so.** A repair now checks each page's file against sync's import record before it takes the database copy.-- `src/core/persistence/unimported-file.ts` (new): the database copy may win only when the database itself held the version the file records: the bytes sync's import record names, or gbrain's own write of a revision the page has held (live or in its version history). Anything else is newer, whoever committed it, including gbrain's own writer commit of a revision whose database row never landed and a page with no import record at all; a wrong "keep" costs a hold and a wrong "take" costs an edit. A database edit made after the file's last commit still wins. The preview names the last commit that touched the file (sha, author, date).+- `src/core/persistence/unimported-file.ts` (new): the database copy may win only when the database itself held the version the file records: the bytes sync's import record names (or the blob a sync adopted into it), or gbrain's own write of a revision the page has held (live or in its version history). Anything else is newer, whoever committed it, including gbrain's own writer commit of a revision whose database row never landed and a page with no import record at all; a wrong "keep" costs a hold and a wrong "take" costs an edit. A database edit made after the file's last commit still wins. The preview names the last commit that touched the file (sha, author, date).- `gbrain repair file-drift --prefer database` with `--held` or `--all-drifted` leaves such pages held and lists them `kept_newer_file`; a file that another writer's delete commit removed is `kept_newer_delete` instead of being recreated. Naming one page with `--slug` still previews it, and the preview and the consent text name what it would discard.-- Every `gbrain repair file-drift` apply now asks for the user's consent (`consent: 'destructive'` in the repair registry), and the consent text lists the changes it would discard. A script that applied without `--yes` now exits 3.+- Every `gbrain repair file-drift` apply now asks for the user's consent (`consent: 'destructive'` in the repair registry), and the consent text lists the changes it would discard and the files a `tombstone_edited` or `purged_differs` `--prefer database` apply removes. A script that applied without `--yes` now exits 3.- `gbrain sources reconcile` (preview and `--apply`, and the `writer_reconcile_preview` / `writer_reconcile_apply` operations) refuses `source_changed` when its result takes the database copy over such a file.- Hold texts, `fix` steps and the docs that print a database-wins repair now give the preview as the step and say the apply needs the user's consent.- `docs/guides/repair.md` explains the new behaviour and how to find edits an earlier repair overwrote, from the reconcile backups it kept.diff --git a/docs/guides/repair.md b/docs/guides/repair.mdindex 540453a801..873d95c1b6 100644--- a/docs/guides/repair.md+++ b/docs/guides/repair.md@@ -3314,7 +3314,7 @@ wrote the page over its file even when the file held another writer's committhat sync had not imported yet, and the preview said "exactly the preferredside". Now a database-wins write goes ahead only when the database itself heldthe version the file's bytes record: the bytes the page's last import read or left-on disk (sync's import record), or bytes a Git effect of one of the page's own+on disk (sync's import record, including a blob a sync adopted into it), or bytes a Git effect of one of the page's ownwrites put in the file for a revision the page has held (its live revision or onein its version history). So a database edit made after the file's last commitstill wins, shorter or not. Anything else is a change the database never held,@@ -3329,8 +3329,9 @@ import record and no write of its own.Every `gbrain repair file-drift` apply now needs the user's consent: run theprinted apply without `--yes` and it exits 3 with the consent message, which-lists each newer change the database copy overwrites; add `--yes` only after-the user agrees. `gbrain sources reconcile` (and MCP+lists each newer change the database copy overwrites and each file a+`tombstone_edited` or `purged_differs` `--prefer database` apply removes; add+`--yes` only after the user agrees. `gbrain sources reconcile` (and MCP`writer_reconcile_preview` / `writer_reconcile_apply`) refuses`source_changed` (detail `unimported_file_edit`) when a decision takes thedatabase copy over what such a file says (`take_database`, `delete` ordiff --git a/src/core/persistence/unimported-file.ts b/src/core/persistence/unimported-file.tsindex a05a059616..17e6962659 100644--- a/src/core/persistence/unimported-file.ts+++ b/src/core/persistence/unimported-file.ts@@ -6,7 +6,8 @@** The bytes count as known only when the database itself held the version they render: the bytes the page's last* recorded import read or left on disk (sync's import record: raw_sha256, disk_sha256, written in the publication that- * stored them), or bytes a Git effect of one of the page's own writes put in that file (expected_hash) for a revision the+ * stored them), the blob a sync adopted into that record for the revision the page held then (blob_oid, disk_blob_oid,+ * written only while the page was at that revision), or bytes a Git effect of one of the page's own writes put in that file (expected_hash) for a revision the* page has held, live or in its version history. gbrain's own writer commit over a database row that never took that* revision (the Git commit landed and the row did not) is newer, as is anything else, including a page with no import* record and no write of its own: a wrong "newer" keeps a hold, a wrong "known" loses an edit. Git only describes the@@ -78,6 +79,7 @@ export async function unimportedFile(engine: BrainEngine, state: Pick<ReconcileSif (pins.raw_file_hash === ABSENT_FILE_HASH) return null;const record = await readSyncImportProvenance(engine, pins.source_id, pins.source_incarnation, Number(pins.page_id));if (record && (record.raw_sha256 === pins.raw_file_hash || record.disk_sha256 === pins.raw_file_hash)) return null;+ if (record && [record.blob_oid, record.disk_blob_oid].some(oid => oid && gitBlobOid(state.raw, oid.length === 64 ? 'sha256' : 'sha1') === oid)) return null;if (await gbrainWroteHeldBytes(engine, pins)) return null;const path = pins.relative_path.split('\\').join('/');const head = await headBlob(state.root, state.path);diff --git a/src/core/repair/file-drift.ts b/src/core/repair/file-drift.tsindex b526f9a8af..f332164003 100644--- a/src/core/repair/file-drift.ts+++ b/src/core/repair/file-drift.ts@@ -743,16 +743,22 @@ function item(page: ApprovedPage, index: number, hash: string, last: boolean, pr*/export function fileDriftConsentText(hash: string, sourceId: string | undefined): { what: string; why: string; risk: string; user_message: string } {let discards: UnimportedFile[] = [];- try { discards = [...readArtifact(hash, `gbrain repair file-drift --source ${sourceId ?? '<source>'}`).values()].flatMap(page => 'discards' in page && page.discards ? [page.discards] : []); }- catch { /* the apply itself refuses a missing or changed artifact */ }- const named = discards.slice(0, 10).map(unimportedFileText).join('; ') + (discards.length > 10 ? `; and ${discards.length - 10} more (listed in the preview)` : '');+ let removals: string[] = [];+ try {+ const pages = [...readArtifact(hash, `gbrain repair file-drift --source ${sourceId ?? '<source>'}`).values()];+ discards = pages.flatMap(page => 'discards' in page && page.discards ? [page.discards] : []);+ removals = pages.flatMap(page => 'remove_file' in page ? [page.remove_file.path] : []);+ } catch { /* the apply itself refuses a missing or changed artifact */ }+ const list = (items: string[]) => items.slice(0, 10).join('; ') + (items.length > 10 ? `; and ${items.length - 10} more (listed in the preview)` : '');+ const named = list(discards.map(unimportedFileText));+ const removing = removals.length ? ` It also removes ${removals.length} file(s) that changed after their page was deleted, and the page does not hold what they say: ${list(removals)}.` : '';return {what: 'Apply the previewed file-drift resolutions: each listed page takes the preferred side, so either the canonical file is rewritten from the database copy or the page is replaced from the file.',- why: `The preview ${hash} listed each page and the side that wins it${discards.length ? `, and the file edits sync never imported that the database copy overwrites (${discards.length})` : ''}.`,- risk: 'The losing side is replaced: a file rewrite is a new commit (git revert brings the edit back) and reconcile keeps a private backup of the replaced copy; a page that changed since the preview is skipped.',- user_message: discards.length+ why: `The preview ${hash} listed each page and the side that wins it${discards.length ? `, and the file edits sync never imported that the database copy overwrites (${discards.length})` : ''}${removals.length ? `, and the files it removes (${removals.length})` : ''}.`,+ risk: 'The losing side is replaced: a file rewrite is a new commit (git revert brings the edit back) and reconcile keeps a private backup of the replaced copy; a removed file comes back with git revert only when it was committed; a page that changed since the preview is skipped.',+ user_message: (discards.length? `Apply the previewed file-drift repair? It overwrites file edits sync never imported with the database copy: ${named}.`- : 'Apply the previewed file-drift repair? Each listed page takes the side the preview showed, and the other copy is replaced.',+ : 'Apply the previewed file-drift repair? Each listed page takes the side the preview showed, and the other copy is replaced.') + removing,};}@@ -904,11 +910,13 @@ export const fileDriftRepair: RepairHandler = {const appended = pages.flatMap(page => 'preview' in page && page.file_appended ? [page.preview.preconditions.relative_path] : []);// #6712 PR 4b: a removed file is gone only once its deletion is committed; without Git durability the next sync finds the committed file held again.const removed = pages.flatMap(page => 'remove_file' in page ? [page.remove_file.path] : []);+ const removalWarning = removed.length ? [`The apply removes ${removed.length} file(s) that changed after their page was deleted; the deleted page does not hold what they say: ${removed.slice(0, 10).join(' ')}${removed.length > 10 ? ` and ${removed.length - 10} more` : ''}`] : [];let warnings: string[] = nextCommand ? [`This preview covered ${previewed} of the ${held!.targets.length} selected held pages${opts.cursor ? ' after its cursor' : ''} (one window stops at ${HELD_WINDOW_PAGES} pages or ${HELD_WINDOW_BUDGET_MS / 1000} s); its hash and apply command cover only these. `+ `After applying this window (or deciding not to), preview the next one: ${nextCommand}`] : [];const discarded = pages.flatMap(page => 'preview' in page && page.discards ? [page.discards] : []);if (discarded.length) warnings.unshift(`This apply writes the database copy over ${discarded.length} file change(s) sync never imported, and needs the user's consent: ${discarded.map(unimportedFileText).join('; ')}. `+ 'Each write is a new commit, so git revert brings the change back; reconcile also keeps the replaced bytes in its private backup.');+ warnings.push(...removalWarning);const releases = pages.filter(page => 'release_hold' in page).length;if (releases) warnings.push(`The apply only schedules a re-screen of ${releases} held file(s) whose page and file agree; the next sync (gbrain sync --source ${sourceId}) releases each that still agrees.`);let undoneBySync: { recreated: number; removed: number } | null = null;diff --git a/test/db-wins-unimported-gst2579.test.ts b/test/db-wins-unimported-gst2579.test.tsindex a1c4cbc75b..773b7c5f0d 100644--- a/test/db-wins-unimported-gst2579.test.ts+++ b/test/db-wins-unimported-gst2579.test.ts@@ -10,8 +10,9 @@* - gbrain's own writer commit counts the same as anyone's: a file holding gbrain's write of a revision the database* never took (the Git commit landed, the row fell back) is kept_newer_file in bulk and named with --slug;* - a database copy that is provably newer still wins: the file holds the bytes sync imported, or gbrain's write of a- * revision the page has since moved past (a later, shorter database edit included, --all-drifted --only), and the- * repair is listed and applied as before.+ * revision the page has since moved past (a later, shorter database edit included, --all-drifted --only), or a blob+ * sync adopted into the page's import record (with the version history pruned), and the repair is listed and applied+ * as before.* Real PGLite (and Postgres when DATABASE_URL is set), a managed worktree with real Git commits; no seams.*/import { expect, test } from 'bun:test';@@ -157,6 +158,10 @@ for (const backend of testBackends()) {expect(kept).toMatchObject({ format_version: 1, retention: 'pre_admission_or_terminal' });expect(Buffer.from(kept.preview.preimages.file_base64, 'base64').toString('utf8')).toBe(EDIT);expect(kept.preview.decisions).toContainEqual({ path: '/compiled_truth', action: 'take_database' });+ // The guide's read-only lookup (docs/guides/repair.md#file-drift-past-losses) finds the overwritten commit from the backup.+ expect([typeof kept.request_id, kept.preview.preconditions.slug, kept.preview.preconditions.relative_path]).toEqual(['string', 'notes/a', 'notes/a.md']);+ const blob = execFileSync('git', ['-C', root, 'hash-object', '--no-filters', '--stdin'], { input: Buffer.from(kept.preview.preimages.file_base64, 'base64') }).toString().trim();+ expect(git(root, 'log', '--all', `--find-object=${blob}`, '--format=%H %an')).toContain(`${commit} Other Writer`);}), 240_000);test(`${backend}: sources reconcile refuses to take the database copy over the unimported commit, in its preview and its apply`, () => brain(backend, async ({ engine, root, commit }) => {@@ -283,6 +288,32 @@ for (const backend of testBackends()) {await settleManagedGitEffects(engine);expect(readFileSync(join(root, 'notes/a.md'), 'utf8')).not.toContain(WRITER_LINE);}, backendOpts(backend)), 240_000);+ test(`${backend}: a file holding the blob sync adopted for gbrain's own write still loses to a later database edit when the version history no longer keeps that revision`, () => managedBrain(async ({ engine, ctx, root }) => {+ await registerLocalWriter(engine, 'cli');+ expect((await performManagedSync(engine, SYNC)).status).toBe('first_sync');+ await engine.executeRaw("UPDATE persistence_host_bindings SET git_durability='enabled'");+ const first = (await engine.readPageSnapshot('notes/a', { sourceId: 'default' }))!;+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug: 'notes/a', content: WRITER, request_id: randomUUID(), expected_revision: first.revision } });+ await settleManagedGitEffects(engine);+ // Sync adopts gbrain's committed write into the page's import record.+ await performManagedSync(engine, SYNC);+ await engine.setConfig('sync.write_through', 'false');+ const written = (await engine.readPageSnapshot('notes/a', { sourceId: 'default' }))!;+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug: 'notes/a', content: FIRST, request_id: randomUUID(), expected_revision: written.revision } });+ await engine.setConfig('sync.write_through', 'true');+ expect(readFileSync(join(root, 'notes/a.md'), 'utf8')).toContain(WRITER_LINE);+ const newer = (await engine.readPageSnapshot('notes/a', { sourceId: 'default' }))!;+ expect((await refusal(() => submitPageMutation(ctx, { operation: 'put_page', params: { slug: 'notes/a', content: '---\ntype: concept\ntitle: A\n---\n\nA refused edit.\n',+ request_id: randomUUID(), expected_revision: newer.revision } }))).code).toBe('source_changed');+ // Retention pruned the version history, so no page_versions row keeps the revision gbrain's write carried.+ await engine.executeRaw("DELETE FROM page_versions WHERE page_id=(SELECT id FROM pages WHERE source_id='default' AND slug='notes/a')");+ const preview = await runner(engine, { allDrifted: true, only: ['notes/**'], prefer: 'database' });+ expect(preview.residuals.kept_newer_file).toBeUndefined();+ expect(preview.listing!.find(entry => entry.item === 'default:notes/a')!.class).toBe('prefer_database');+ expect((await runner(engine, { allDrifted: true, only: ['notes/**'], prefer: 'database', expect: preview.preview_hash })).outcomes).toEqual({ repaired: 1 });+ await settleManagedGitEffects(engine);+ expect(readFileSync(join(root, 'notes/a.md'), 'utf8')).not.toContain(WRITER_LINE);+ }, backendOpts(backend)), 240_000);test(`${backend}: --held --side deleted --state file_gone --prefer database keeps another writer's unimported delete (kept_newer_delete)`, () => brain(backend, async ({ engine, root }) => {git(root, 'rm', '-q', '--', 'notes/a.md'); git(root, 'commit', '-qm', 'other writer delete');const deleteCommit = git(root, 'rev-parse', 'HEAD');diff --git a/test/repair-file-drift-held-6712.test.ts b/test/repair-file-drift-held-6712.test.tsindex e449ddaf55..f034af8d81 100644--- a/test/repair-file-drift-held-6712.test.ts+++ b/test/repair-file-drift-held-6712.test.ts@@ -30,6 +30,7 @@ import { SYNC_ORIGIN_DELETED_OP } from '../src/core/persistence/sync-pass-row.tsimport { gitHoldItem, readGitHold, writeGitHold, type FileDriftDeletedState, type FileDriftSide } from '../src/core/persistence/sync-holds.ts';import { performManagedSync } from '../src/core/persistence/sync-run.ts';import { repairRunner } from '../src/core/repair/registry.ts';+import { fileDriftConsentText } from '../src/core/repair/file-drift.ts';import { resolveRepairScope } from '../src/core/repair/core.ts';import { withEnv } from './helpers/with-env.ts';import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';@@ -218,7 +219,10 @@ for (const backend of testBackends()) describe(`#6712 PR 4b: repair file-drift -expect(preview).toMatchObject({ mode: 'dry_run', affected: 1 });expect(preview.details!.pages).toEqual([expect.objectContaining({ slug: 'notes/tomb', action: 'remove_file', state: 'tombstone_edited', verb: 'keep the deletion and remove the file' })]);expect(preview.listing).toEqual(expect.arrayContaining([expect.objectContaining({ detail: `keep the deletion and remove the file: ${path}` })]));- expect(preview.warnings).toEqual([expect.stringContaining("leaves each removed file's deletion uncommitted")]);+ expect(preview.warnings).toEqual([expect.stringContaining("leaves each removed file's deletion uncommitted"),+ `The apply removes 1 file(s) that changed after their page was deleted; the deleted page does not hold what they say: ${path}`]);+ // GST-2579: the consent names each file the apply removes.+ expect(fileDriftConsentText(preview.preview_hash!, f.sourceId).user_message).toContain(`It also removes 1 file(s) that changed after their page was deleted, and the page does not hold what they say: ${path}.`);expect(existsSync(join(f.root, path))).toBe(true);const applied = await run(f.sourceId, { side: 'deleted', state: 'tombstone_edited', prefer: 'database', expect: preview.preview_hash! });expect(applied.outcomes).toEqual({ repaired: 1 });