Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Count a blob sync adopted as bytes the database held, and name the files a deleted-side database-wins apply removes in its consent (GST-2579)

garrytanc4f4946245c1
diff --git a/CHANGELOG.md b/CHANGELOG.md
index be51574d88..940bdb3d38 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,11 +14,11 @@ identifiers and attribution are available in the pre-removal Git revision
### A database-wins file-drift repair no longer discards a newer file edit
-**`gbrain repair file-drift --prefer database` and `gbrain sources reconcile --apply` could overwrite a change to a page's file that sync had not imported yet, or bring back a file another writer had just deleted, with nothing in the preview saying so.** gbrain's own hold messages pointed agents at that command, so the loss was easy to walk into. A repair now checks each page's file against sync's import record before it takes the database copy.
+**`gbrain repair file-drift --prefer database` and `gbrain sources reconcile --apply` could overwrite a change to a page's file that sync had not imported yet, or bring back a file another writer had just deleted, with nothing in the preview saying so.** A repair now checks each page's file against sync's import record before it takes the database copy.
-- `src/core/persistence/unimported-file.ts` (new): the database copy may win only when the database itself held the version the file records: the bytes sync's import record names, or gbrain's own write of a revision the page has held (live or in its version history). Anything else is newer, whoever committed it, including gbrain's own writer commit of a revision whose database row never landed and a page with no import record at all; a wrong "keep" costs a hold and a wrong "take" costs an edit. A database edit made after the file's last commit still wins. The preview names the last commit that touched the file (sha, author, date).
+- `src/core/persistence/unimported-file.ts` (new): the database copy may win only when the database itself held the version the file records: the bytes sync's import record names (or the blob a sync adopted into it), or gbrain's own write of a revision the page has held (live or in its version history). Anything else is newer, whoever committed it, including gbrain's own writer commit of a revision whose database row never landed and a page with no import record at all; a wrong "keep" costs a hold and a wrong "take" costs an edit. A database edit made after the file's last commit still wins. The preview names the last commit that touched the file (sha, author, date).
- `gbrain repair file-drift --prefer database` with `--held` or `--all-drifted` leaves such pages held and lists them `kept_newer_file`; a file that another writer's delete commit removed is `kept_newer_delete` instead of being recreated. Naming one page with `--slug` still previews it, and the preview and the consent text name what it would discard.
-- Every `gbrain repair file-drift` apply now asks for the user's consent (`consent: 'destructive'` in the repair registry), and the consent text lists the changes it would discard. A script that applied without `--yes` now exits 3.
+- Every `gbrain repair file-drift` apply now asks for the user's consent (`consent: 'destructive'` in the repair registry), and the consent text lists the changes it would discard and the files a `tombstone_edited` or `purged_differs` `--prefer database` apply removes. A script that applied without `--yes` now exits 3.
- `gbrain sources reconcile` (preview and `--apply`, and the `writer_reconcile_preview` / `writer_reconcile_apply` operations) refuses `source_changed` when its result takes the database copy over such a file.
- Hold texts, `fix` steps and the docs that print a database-wins repair now give the preview as the step and say the apply needs the user's consent.
- `docs/guides/repair.md` explains the new behaviour and how to find edits an earlier repair overwrote, from the reconcile backups it kept.
diff --git a/docs/guides/repair.md b/docs/guides/repair.md
index 540453a801..873d95c1b6 100644
--- a/docs/guides/repair.md
+++ b/docs/guides/repair.md
@@ -3314,7 +3314,7 @@ wrote the page over its file even when the file held another writer's commit
that sync had not imported yet, and the preview said "exactly the preferred
side". Now a database-wins write goes ahead only when the database itself held
the version the file's bytes record: the bytes the page's last import read or left
-on disk (sync's import record), or bytes a Git effect of one of the page's own
+on disk (sync's import record, including a blob a sync adopted into it), or bytes a Git effect of one of the page's own
writes put in the file for a revision the page has held (its live revision or one
in its version history). So a database edit made after the file's last commit
still wins, shorter or not. Anything else is a change the database never held,
@@ -3329,8 +3329,9 @@ import record and no write of its own.
Every `gbrain repair file-drift` apply now needs the user's consent: run the
printed apply without `--yes` and it exits 3 with the consent message, which
-lists each newer change the database copy overwrites; add `--yes` only after
-the user agrees. `gbrain sources reconcile` (and MCP
+lists each newer change the database copy overwrites and each file a
+`tombstone_edited` or `purged_differs` `--prefer database` apply removes; add
+`--yes` only after the user agrees. `gbrain sources reconcile` (and MCP
`writer_reconcile_preview` / `writer_reconcile_apply`) refuses
`source_changed` (detail `unimported_file_edit`) when a decision takes the
database copy over what such a file says (`take_database`, `delete` or
diff --git a/src/core/persistence/unimported-file.ts b/src/core/persistence/unimported-file.ts
index a05a059616..17e6962659 100644
--- a/src/core/persistence/unimported-file.ts
+++ b/src/core/persistence/unimported-file.ts
@@ -6,7 +6,8 @@
*
* The bytes count as known only when the database itself held the version they render: the bytes the page's last
* recorded import read or left on disk (sync's import record: raw_sha256, disk_sha256, written in the publication that
- * stored them), or bytes a Git effect of one of the page's own writes put in that file (expected_hash) for a revision the
+ * stored them), the blob a sync adopted into that record for the revision the page held then (blob_oid, disk_blob_oid,
+ * written only while the page was at that revision), or bytes a Git effect of one of the page's own writes put in that file (expected_hash) for a revision the
* page has held, live or in its version history. gbrain's own writer commit over a database row that never took that
* revision (the Git commit landed and the row did not) is newer, as is anything else, including a page with no import
* record and no write of its own: a wrong "newer" keeps a hold, a wrong "known" loses an edit. Git only describes the
@@ -78,6 +79,7 @@ export async function unimportedFile(engine: BrainEngine, state: Pick<ReconcileS
if (pins.raw_file_hash === ABSENT_FILE_HASH) return null;
const record = await readSyncImportProvenance(engine, pins.source_id, pins.source_incarnation, Number(pins.page_id));
if (record && (record.raw_sha256 === pins.raw_file_hash || record.disk_sha256 === pins.raw_file_hash)) return null;
+ if (record && [record.blob_oid, record.disk_blob_oid].some(oid => oid && gitBlobOid(state.raw, oid.length === 64 ? 'sha256' : 'sha1') === oid)) return null;
if (await gbrainWroteHeldBytes(engine, pins)) return null;
const path = pins.relative_path.split('\\').join('/');
const head = await headBlob(state.root, state.path);
diff --git a/src/core/repair/file-drift.ts b/src/core/repair/file-drift.ts
index b526f9a8af..f332164003 100644
--- a/src/core/repair/file-drift.ts
+++ b/src/core/repair/file-drift.ts
@@ -743,16 +743,22 @@ function item(page: ApprovedPage, index: number, hash: string, last: boolean, pr
*/
export function fileDriftConsentText(hash: string, sourceId: string | undefined): { what: string; why: string; risk: string; user_message: string } {
let discards: UnimportedFile[] = [];
- try { discards = [...readArtifact(hash, `gbrain repair file-drift --source ${sourceId ?? '<source>'}`).values()].flatMap(page => 'discards' in page && page.discards ? [page.discards] : []); }
- catch { /* the apply itself refuses a missing or changed artifact */ }
- const named = discards.slice(0, 10).map(unimportedFileText).join('; ') + (discards.length > 10 ? `; and ${discards.length - 10} more (listed in the preview)` : '');
+ let removals: string[] = [];
+ try {
+ const pages = [...readArtifact(hash, `gbrain repair file-drift --source ${sourceId ?? '<source>'}`).values()];
+ discards = pages.flatMap(page => 'discards' in page && page.discards ? [page.discards] : []);
+ removals = pages.flatMap(page => 'remove_file' in page ? [page.remove_file.path] : []);
+ } catch { /* the apply itself refuses a missing or changed artifact */ }
+ const list = (items: string[]) => items.slice(0, 10).join('; ') + (items.length > 10 ? `; and ${items.length - 10} more (listed in the preview)` : '');
+ const named = list(discards.map(unimportedFileText));
+ const removing = removals.length ? ` It also removes ${removals.length} file(s) that changed after their page was deleted, and the page does not hold what they say: ${list(removals)}.` : '';
return {
what: 'Apply the previewed file-drift resolutions: each listed page takes the preferred side, so either the canonical file is rewritten from the database copy or the page is replaced from the file.',
- why: `The preview ${hash} listed each page and the side that wins it${discards.length ? `, and the file edits sync never imported that the database copy overwrites (${discards.length})` : ''}.`,
- risk: 'The losing side is replaced: a file rewrite is a new commit (git revert brings the edit back) and reconcile keeps a private backup of the replaced copy; a page that changed since the preview is skipped.',
- user_message: discards.length
+ why: `The preview ${hash} listed each page and the side that wins it${discards.length ? `, and the file edits sync never imported that the database copy overwrites (${discards.length})` : ''}${removals.length ? `, and the files it removes (${removals.length})` : ''}.`,
+ risk: 'The losing side is replaced: a file rewrite is a new commit (git revert brings the edit back) and reconcile keeps a private backup of the replaced copy; a removed file comes back with git revert only when it was committed; a page that changed since the preview is skipped.',
+ user_message: (discards.length
? `Apply the previewed file-drift repair? It overwrites file edits sync never imported with the database copy: ${named}.`
- : 'Apply the previewed file-drift repair? Each listed page takes the side the preview showed, and the other copy is replaced.',
+ : 'Apply the previewed file-drift repair? Each listed page takes the side the preview showed, and the other copy is replaced.') + removing,
};
}
@@ -904,11 +910,13 @@ export const fileDriftRepair: RepairHandler = {
const appended = pages.flatMap(page => 'preview' in page && page.file_appended ? [page.preview.preconditions.relative_path] : []);
// #6712 PR 4b: a removed file is gone only once its deletion is committed; without Git durability the next sync finds the committed file held again.
const removed = pages.flatMap(page => 'remove_file' in page ? [page.remove_file.path] : []);
+ const removalWarning = removed.length ? [`The apply removes ${removed.length} file(s) that changed after their page was deleted; the deleted page does not hold what they say: ${removed.slice(0, 10).join(' ')}${removed.length > 10 ? ` and ${removed.length - 10} more` : ''}`] : [];
let warnings: string[] = nextCommand ? [`This preview covered ${previewed} of the ${held!.targets.length} selected held pages${opts.cursor ? ' after its cursor' : ''} (one window stops at ${HELD_WINDOW_PAGES} pages or ${HELD_WINDOW_BUDGET_MS / 1000} s); its hash and apply command cover only these. `
+ `After applying this window (or deciding not to), preview the next one: ${nextCommand}`] : [];
const discarded = pages.flatMap(page => 'preview' in page && page.discards ? [page.discards] : []);
if (discarded.length) warnings.unshift(`This apply writes the database copy over ${discarded.length} file change(s) sync never imported, and needs the user's consent: ${discarded.map(unimportedFileText).join('; ')}. `
+ 'Each write is a new commit, so git revert brings the change back; reconcile also keeps the replaced bytes in its private backup.');
+ warnings.push(...removalWarning);
const releases = pages.filter(page => 'release_hold' in page).length;
if (releases) warnings.push(`The apply only schedules a re-screen of ${releases} held file(s) whose page and file agree; the next sync (gbrain sync --source ${sourceId}) releases each that still agrees.`);
let undoneBySync: { recreated: number; removed: number } | null = null;
diff --git a/test/db-wins-unimported-gst2579.test.ts b/test/db-wins-unimported-gst2579.test.ts
index a1c4cbc75b..773b7c5f0d 100644
--- a/test/db-wins-unimported-gst2579.test.ts
+++ b/test/db-wins-unimported-gst2579.test.ts
@@ -10,8 +10,9 @@
* - gbrain's own writer commit counts the same as anyone's: a file holding gbrain's write of a revision the database
* never took (the Git commit landed, the row fell back) is kept_newer_file in bulk and named with --slug;
* - a database copy that is provably newer still wins: the file holds the bytes sync imported, or gbrain's write of a
- * revision the page has since moved past (a later, shorter database edit included, --all-drifted --only), and the
- * repair is listed and applied as before.
+ * revision the page has since moved past (a later, shorter database edit included, --all-drifted --only), or a blob
+ * sync adopted into the page's import record (with the version history pruned), and the repair is listed and applied
+ * as before.
* Real PGLite (and Postgres when DATABASE_URL is set), a managed worktree with real Git commits; no seams.
*/
import { expect, test } from 'bun:test';
@@ -157,6 +158,10 @@ for (const backend of testBackends()) {
expect(kept).toMatchObject({ format_version: 1, retention: 'pre_admission_or_terminal' });
expect(Buffer.from(kept.preview.preimages.file_base64, 'base64').toString('utf8')).toBe(EDIT);
expect(kept.preview.decisions).toContainEqual({ path: '/compiled_truth', action: 'take_database' });
+ // The guide's read-only lookup (docs/guides/repair.md#file-drift-past-losses) finds the overwritten commit from the backup.
+ expect([typeof kept.request_id, kept.preview.preconditions.slug, kept.preview.preconditions.relative_path]).toEqual(['string', 'notes/a', 'notes/a.md']);
+ const blob = execFileSync('git', ['-C', root, 'hash-object', '--no-filters', '--stdin'], { input: Buffer.from(kept.preview.preimages.file_base64, 'base64') }).toString().trim();
+ expect(git(root, 'log', '--all', `--find-object=${blob}`, '--format=%H %an')).toContain(`${commit} Other Writer`);
}), 240_000);
test(`${backend}: sources reconcile refuses to take the database copy over the unimported commit, in its preview and its apply`, () => brain(backend, async ({ engine, root, commit }) => {
@@ -283,6 +288,32 @@ for (const backend of testBackends()) {
await settleManagedGitEffects(engine);
expect(readFileSync(join(root, 'notes/a.md'), 'utf8')).not.toContain(WRITER_LINE);
}, backendOpts(backend)), 240_000);
+ test(`${backend}: a file holding the blob sync adopted for gbrain's own write still loses to a later database edit when the version history no longer keeps that revision`, () => managedBrain(async ({ engine, ctx, root }) => {
+ await registerLocalWriter(engine, 'cli');
+ expect((await performManagedSync(engine, SYNC)).status).toBe('first_sync');
+ await engine.executeRaw("UPDATE persistence_host_bindings SET git_durability='enabled'");
+ const first = (await engine.readPageSnapshot('notes/a', { sourceId: 'default' }))!;
+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug: 'notes/a', content: WRITER, request_id: randomUUID(), expected_revision: first.revision } });
+ await settleManagedGitEffects(engine);
+ // Sync adopts gbrain's committed write into the page's import record.
+ await performManagedSync(engine, SYNC);
+ await engine.setConfig('sync.write_through', 'false');
+ const written = (await engine.readPageSnapshot('notes/a', { sourceId: 'default' }))!;
+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug: 'notes/a', content: FIRST, request_id: randomUUID(), expected_revision: written.revision } });
+ await engine.setConfig('sync.write_through', 'true');
+ expect(readFileSync(join(root, 'notes/a.md'), 'utf8')).toContain(WRITER_LINE);
+ const newer = (await engine.readPageSnapshot('notes/a', { sourceId: 'default' }))!;
+ expect((await refusal(() => submitPageMutation(ctx, { operation: 'put_page', params: { slug: 'notes/a', content: '---\ntype: concept\ntitle: A\n---\n\nA refused edit.\n',
+ request_id: randomUUID(), expected_revision: newer.revision } }))).code).toBe('source_changed');
+ // Retention pruned the version history, so no page_versions row keeps the revision gbrain's write carried.
+ await engine.executeRaw("DELETE FROM page_versions WHERE page_id=(SELECT id FROM pages WHERE source_id='default' AND slug='notes/a')");
+ const preview = await runner(engine, { allDrifted: true, only: ['notes/**'], prefer: 'database' });
+ expect(preview.residuals.kept_newer_file).toBeUndefined();
+ expect(preview.listing!.find(entry => entry.item === 'default:notes/a')!.class).toBe('prefer_database');
+ expect((await runner(engine, { allDrifted: true, only: ['notes/**'], prefer: 'database', expect: preview.preview_hash })).outcomes).toEqual({ repaired: 1 });
+ await settleManagedGitEffects(engine);
+ expect(readFileSync(join(root, 'notes/a.md'), 'utf8')).not.toContain(WRITER_LINE);
+ }, backendOpts(backend)), 240_000);
test(`${backend}: --held --side deleted --state file_gone --prefer database keeps another writer's unimported delete (kept_newer_delete)`, () => brain(backend, async ({ engine, root }) => {
git(root, 'rm', '-q', '--', 'notes/a.md'); git(root, 'commit', '-qm', 'other writer delete');
const deleteCommit = git(root, 'rev-parse', 'HEAD');
diff --git a/test/repair-file-drift-held-6712.test.ts b/test/repair-file-drift-held-6712.test.ts
index e449ddaf55..f034af8d81 100644
--- a/test/repair-file-drift-held-6712.test.ts
+++ b/test/repair-file-drift-held-6712.test.ts
@@ -30,6 +30,7 @@ import { SYNC_ORIGIN_DELETED_OP } from '../src/core/persistence/sync-pass-row.ts
import { gitHoldItem, readGitHold, writeGitHold, type FileDriftDeletedState, type FileDriftSide } from '../src/core/persistence/sync-holds.ts';
import { performManagedSync } from '../src/core/persistence/sync-run.ts';
import { repairRunner } from '../src/core/repair/registry.ts';
+import { fileDriftConsentText } from '../src/core/repair/file-drift.ts';
import { resolveRepairScope } from '../src/core/repair/core.ts';
import { withEnv } from './helpers/with-env.ts';
import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';
@@ -218,7 +219,10 @@ for (const backend of testBackends()) describe(`#6712 PR 4b: repair file-drift -
expect(preview).toMatchObject({ mode: 'dry_run', affected: 1 });
expect(preview.details!.pages).toEqual([expect.objectContaining({ slug: 'notes/tomb', action: 'remove_file', state: 'tombstone_edited', verb: 'keep the deletion and remove the file' })]);
expect(preview.listing).toEqual(expect.arrayContaining([expect.objectContaining({ detail: `keep the deletion and remove the file: ${path}` })]));
- expect(preview.warnings).toEqual([expect.stringContaining("leaves each removed file's deletion uncommitted")]);
+ expect(preview.warnings).toEqual([expect.stringContaining("leaves each removed file's deletion uncommitted"),
+ `The apply removes 1 file(s) that changed after their page was deleted; the deleted page does not hold what they say: ${path}`]);
+ // GST-2579: the consent names each file the apply removes.
+ expect(fileDriftConsentText(preview.preview_hash!, f.sourceId).user_message).toContain(`It also removes 1 file(s) that changed after their page was deleted, and the page does not hold what they say: ${path}.`);
expect(existsSync(join(f.root, path))).toBe(true);
const applied = await run(f.sourceId, { side: 'deleted', state: 'tombstone_edited', prefer: 'database', expect: preview.preview_hash! });
expect(applied.outcomes).toEqual({ repaired: 1 });