Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(sources canonical): a conversion converges with a live writer committing through the door

garrytana26c0d6d7dec
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 3ef9158c78..ecaf394f81 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -12,6 +12,10 @@ identifiers and attribution are available in the pre-removal Git revision
## [Unreleased]
+**A conversion to git-canonical now finishes while a live writer commits to the source.**
+
+Door commits (`gbrain git commit-push`) that landed after the apply's second strict pass read its level left Git ahead of the database at those paths, so the closing preview never reached 0 and the apply stopped with `git_conversion_incomplete`, however fast it ran. Nobody can promise a quiet window on a source that scheduled ingest and agents write to. The apply now runs up to three closing strict passes (`strict_pass_closing`) while those commits are all that is left, previewing again after each; the result reports `closing_passes`. No write is held or delayed. If Git is still ahead after the last pass, or anything else is open, the apply stops with `git_conversion_incomplete` and its fix as before, the source stays git-canonical, and a plain `gbrain sync` imports the rest.
+
**`gbrain sources canonical <id> --to git` on a managed source with no Git checkout now refuses with a typed error and a fix.**
When a managed source's folder was a plain directory, or a `git init` with no commit yet, the preview and the apply failed with `internal_error` and a raw git message. They now refuse with `git_checkout_required` or `git_branch_unborn`, read and write nothing, leave the source usable as it is, and carry a fix that asks the user before the folder becomes a Git repository (then `gbrain sources canonical <id> --to git --dry-run --json` previews again).
diff --git a/docs/designs/GIT_CANONICAL_MEMORY.md b/docs/designs/GIT_CANONICAL_MEMORY.md
index 9f46dc8152..4a9e224450 100644
--- a/docs/designs/GIT_CANONICAL_MEMORY.md
+++ b/docs/designs/GIT_CANONICAL_MEMORY.md
@@ -90,7 +90,7 @@ G8a ships it as `gbrain sources canonical <source-id> --to git|mixed [--dry-run
| parked effects | Retried once, or, for targets Git ignores, the page is stamped `git_ignored` (declared) in the same transaction that marks each parked target `declared: git_ignored`; the target stays as a record and the next preview no longer lists it. |
| discard | Refused: `git_conversion_blocked` names each category (both sides moved, a both-sides drift hold, an unknown-side drift hold whose row differs from the committed file, a database edit Git deleted, an uncommitted working-tree edit that differs from HEAD and the export, an ignored target, a target whose ignore status Git cannot read (`ignore_status_unknown`: beyond a symlink, outside the checkout, a failed check; only that path is refused), unparseable, unsafe path, segmented page, export mismatch, or a row a strict pass would overwrite unplanned). There is no override. |
-`--apply --expect <hash>` re-plans and refuses `preview_changed` on any difference, adopts group by group (a crash leaves the source mixed and the rerun preview lists only what is left), withholds group by group, settles the parked effects, rechecks what a strict pass would overwrite, clears the holds only after that recheck passes, runs the full strict-equivalent pass, then sets `canonical: git` in the transaction that records the conversion. Writes can land between that pass and the flag, so it runs one genuine strict pass and previews again: the conversion is complete only when nothing is left; otherwise it reports `git_conversion_incomplete` with what remains and the source stays strict, because Git wins. Where durability applies the apply pushes its door commits itself, after the groups and once at the end; a push that does not land leaves them local, the receipt says so, and the next page write's push-only effect sends them. The apply prints a progress line per phase and every 10 seconds on stderr, and SIGTERM, SIGINT or SIGHUP prints a partial report (phase, groups committed, items scanned, canonical state); `GBRAIN_MEMORY_TRACE=<seconds>` (1 to 3600) adds a memory line on stderr every that many seconds, and `gbrain sync` honors it too. `--to mixed` removes the flag and changes nothing else; Git keeps every door commit. Both strict passes read the whole level (a full round) without opening a `--full` re-import episode, so a page already equal to its blob is not re-imported, and a re-import that changes nothing queues no embedding effect when every chunk already holds a vector for its text (#7866). Each pass drains under the source's drain lease as `gbrain sync` does, so a write of its own that outlives one receipt wait does not stop it; an unrelated write that never settles, or another drainer, still stops it with `git_conversion_incomplete` before the flag. Every `git_conversion_incomplete` carries `fix` (preview again; verify with `gbrain sync status --source <id> --json`), and with `--json` the refusal carries `progress`. In a strict pass, progress `total` is the planned paths for the whole pass; `round`, `candidates`, `batch` of `batches` and `scanned` say where the round stands.
+`--apply --expect <hash>` re-plans and refuses `preview_changed` on any difference, adopts group by group (a crash leaves the source mixed and the rerun preview lists only what is left), withholds group by group, settles the parked effects, rechecks what a strict pass would overwrite, clears the holds only after that recheck passes, runs the full strict-equivalent pass, then sets `canonical: git` in the transaction that records the conversion. Writes can land between that pass and the flag, so it runs one genuine strict pass and previews again. A live writer's door commits (`gbrain git commit-push`) that land after that pass read its level leave Git ahead of the database at their paths and nothing else; the source is git-canonical by then, so the apply runs up to three closing strict passes (phase `strict_pass_closing`, incremental), previewing again after each, and reports `closing_passes`. The conversion is complete only when nothing is left; otherwise it reports `git_conversion_incomplete` with what remains and the source stays strict, because Git wins. Where durability applies the apply pushes its door commits itself, after the groups and once at the end; a push that does not land leaves them local, the receipt says so, and the next page write's push-only effect sends them. The apply prints a progress line per phase and every 10 seconds on stderr, and SIGTERM, SIGINT or SIGHUP prints a partial report (phase, groups committed, items scanned, canonical state); `GBRAIN_MEMORY_TRACE=<seconds>` (1 to 3600) adds a memory line on stderr every that many seconds, and `gbrain sync` honors it too. `--to mixed` removes the flag and changes nothing else; Git keeps every door commit. Both strict passes read the whole level (a full round) without opening a `--full` re-import episode, so a page already equal to its blob is not re-imported, and a re-import that changes nothing queues no embedding effect when every chunk already holds a vector for its text (#7866). Each pass drains under the source's drain lease as `gbrain sync` does, so a write of its own that outlives one receipt wait does not stop it; an unrelated write that never settles, or another drainer, still stops it with `git_conversion_incomplete` before the flag. Every `git_conversion_incomplete` carries `fix` (preview again; verify with `gbrain sync status --source <id> --json`), and with `--json` the refusal carries `progress`. In a strict pass, progress `total` is the planned paths for the whole pass; `round`, `candidates`, `batch` of `batches` and `scanned` say where the round stands.
## The git-gap census
diff --git a/docs/guides/error-codes.md b/docs/guides/error-codes.md
index 787acb31c2..9d31184dec 100644
--- a/docs/guides/error-codes.md
+++ b/docs/guides/error-codes.md
@@ -1697,7 +1697,7 @@ More: [docs/designs/GIT_CANONICAL_MEMORY.md#conversion](../../docs/designs/GIT_C
| Meaning | Why | Next step | Who acts · consent | Verify | Exit | Retryable |
|---|---|---|---|---|---|---|
-| A git-canonical conversion stopped because its strict pass did not finish, or finished with work left after the flag was set. | The conversion runs a full strict-equivalent pass before it sets canonical: git, and one genuine strict pass after. A pass that ends partial, blocked or deferred stops the conversion where it is: before the flag the source stays as it was, after it the source stays strict (Git wins) and the plan lists what is left. | Wait until gbrain sync status --source <id> --json shows the source idle, then preview again with gbrain sources canonical <id> --to git --dry-run and apply the new hash. | agent | `repeat the read that failed` | 1 | yes |
+| A git-canonical conversion stopped because its strict pass did not finish, or finished with work left after the flag was set. | The conversion runs a full strict-equivalent pass before it sets canonical: git, and one genuine strict pass after, then up to three closing strict passes while only door commits a live writer landed meanwhile are left (Git ahead of the database). A pass that ends partial, blocked or deferred stops the conversion where it is: before the flag the source stays as it was, after it the source stays strict (Git wins) and the plan lists what is left. | Wait until gbrain sync status --source <id> --json shows the source idle, then preview again with gbrain sources canonical <id> --to git --dry-run and apply the new hash. | agent | `repeat the read that failed` | 1 | yes |
More: [docs/designs/GIT_CANONICAL_MEMORY.md#conversion](../../docs/designs/GIT_CANONICAL_MEMORY.md#conversion)
diff --git a/src/commands/sources-canonical.ts b/src/commands/sources-canonical.ts
index cd1b02fcb9..874f1dea7e 100644
--- a/src/commands/sources-canonical.ts
+++ b/src/commands/sources-canonical.ts
@@ -140,7 +140,8 @@ export async function runSourcesCanonical(engine: BrainEngine, args: string[]):
const result = await withConversionReport(engine, parsed.sourceId, json,
onProgress => applyGitConversion(engine, parsed.sourceId, parsed.expect!, { noEmbed: parsed.noEmbed, noExtract: parsed.noExtract, onProgress }));
if (result.status === 'incomplete') {
- const error = opError('git_conversion_incomplete', `Source ${parsed.sourceId} is git-canonical, and ${result.after.remaining} item(s) were still open after its strict pass.`,
+ const closing = result.closing_passes ? ` and ${result.closing_passes} closing pass(es), with Git still ahead at ${result.after.git_wins.count} path(s)` : '';
+ const error = opError('git_conversion_incomplete', `Source ${parsed.sourceId} is git-canonical, and ${result.after.remaining} item(s) were still open after its strict passes${closing}.`,
`Writes landed during the conversion. Git wins on the source now; preview again with gbrain sources canonical ${parsed.sourceId} --to git --dry-run and apply the new hash.`,
{ docs: DOCS, detail: 'work_left', fix: conversionIncompleteFix(parsed.sourceId) });
print({ status: 'incomplete', result, ...toAgentError(error, { transport: 'cli', command: 'sources canonical', render: cliRenderContext() }) }, describeResult(result));
diff --git a/src/core/error-registry.ts b/src/core/error-registry.ts
index da734ae67b..92ba04cbc5 100644
--- a/src/core/error-registry.ts
+++ b/src/core/error-registry.ts
@@ -335,7 +335,7 @@ export const CODES = {
git_unavailable: { class: 'retryable', summary: "A git command did not complete." },
git_object_missing: { class: 'caller', summary: "A committed file's Git object is missing from the checkout and its working-tree copy differs or is gone, so managed sync holds that file alone.", why: "An interrupted gc, a partial clone without its promisor remote or disk corruption can drop an object a commit still names. Sync imports the working-tree bytes when they hash to the committed blob id; otherwise it imports nothing from a guess, and the rest of the source keeps syncing.", docs: 'docs/guides/write-refusals.md#git_object_missing', suggestion: 'Run git fsck in the checkout to see what is missing, restore it with git fetch (a partial or shallow clone) or from a backup, then sync again; every sync re-tests the hold.' },
git_conversion_blocked: { class: 'consent', actor: 'user', summary: "Converting a source to git-canonical would discard database content or cannot place it, so the apply wrote nothing.", reasons: ['both_moved', 'held_drift_both', 'held_drift_unknown_differs', 'deleted_in_git', 'uncommitted_edit', 'ignored_target', 'unparseable', 'unsafe_path', 'segmented_page', 'export_mismatch', 'git_overwrites_database', 'ignore_status_unknown'], why: "gbrain sources canonical --to git applies only when every database row will be in Git or in a declared database-only class afterwards. The preview counts what would be lost or cannot be placed by category (detail lists them): both_moved and held_drift_both, a page Git and the database both changed; deleted_in_git, a page Git deleted that the database edited since; uncommitted_edit, a working-tree edit the adoption would overwrite; ignored_target, a page whose file Git ignores without a declared class; unparseable and unsafe_path, a committed file or path sync cannot read; segmented_page, a segmented page conversion does not adopt yet; export_mismatch, a page whose export does not read back equal; git_overwrites_database, a page version a strict pass would replace that nothing accepted; ignore_status_unknown, a target whose ignore status Git cannot read (beyond a symlink, outside the checkout, a timed-out check). There is no override.", docs: 'docs/designs/GIT_CANONICAL_MEMORY.md#conversion', suggestion: 'Relay the categories and their paths (the --dry-run preview lists them) and ask the user how each should be resolved; preview again afterwards.' },
- git_conversion_incomplete: { class: 'retryable', summary: "A git-canonical conversion stopped because its strict pass did not finish, or finished with work left after the flag was set.", why: "The conversion runs a full strict-equivalent pass before it sets canonical: git, and one genuine strict pass after. A pass that ends partial, blocked or deferred stops the conversion where it is: before the flag the source stays as it was, after it the source stays strict (Git wins) and the plan lists what is left.", docs: 'docs/designs/GIT_CANONICAL_MEMORY.md#conversion', suggestion: 'Wait until gbrain sync status --source <id> --json shows the source idle, then preview again with gbrain sources canonical <id> --to git --dry-run and apply the new hash.' },
+ git_conversion_incomplete: { class: 'retryable', summary: "A git-canonical conversion stopped because its strict pass did not finish, or finished with work left after the flag was set.", why: "The conversion runs a full strict-equivalent pass before it sets canonical: git, and one genuine strict pass after, then up to three closing strict passes while only door commits a live writer landed meanwhile are left (Git ahead of the database). A pass that ends partial, blocked or deferred stops the conversion where it is: before the flag the source stays as it was, after it the source stays strict (Git wins) and the plan lists what is left.", docs: 'docs/designs/GIT_CANONICAL_MEMORY.md#conversion', suggestion: 'Wait until gbrain sync status --source <id> --json shows the source idle, then preview again with gbrain sources canonical <id> --to git --dry-run and apply the new hash.' },
git_home_required: { class: 'caller', summary: "A git-canonical source would store content in the database without a file in Git; nothing was written.", reasons: ['no_file_form', 'declared_target', 'facts_lane'], why: "On a source with canonical: git, page content reaches the database only after Git commits its file. The declared database-only classes (private, alias-private and lineage-private pages, storage.db_only directories, never-filed derive-phase pages, paths Git ignores, detached pages and origin twins) are the only exceptions. no_file_form: the write has no file and is in no declared class; declared_target: a publication would commit a file for a declared page; facts_lane: a writer with no file form yet.", docs: 'docs/guides/write-refusals.md#git_home_required', suggestion: 'Ask the user which git home to configure (a path Git tracks, or a storage.db_only directory in gbrain.yml), then submit the write again with a new request_id.' },
git_private_commit_refused: { class: 'caller', summary: "A git-canonical source refused to commit the file of a private, alias-private or lineage-private page; nothing was written and HEAD is unchanged.", reasons: ['private', 'alias_private', 'lineage_private'], why: "Private pages of a source with canonical: git are database-only: their bytes never enter Git history. A page turning private is published as the door commit that removes its file, never as a commit of its bytes.", docs: 'docs/guides/write-refusals.md#git_private_commit_refused', suggestion: 'The page stays database-only. Write it through put_page (which publishes a private page without a file), or remove visibility: private if it belongs in Git, then submit again with a new request_id.' },
git_protocol_required: { class: 'host_only', summary: "A write reached a git-canonical source from a process that does not declare the git-canonical protocol (an older gbrain, or gbrain.write_sources set by hand); nothing was written.", docs: 'docs/guides/write-refusals.md#git_protocol_required', suggestion: 'Upgrade or stop the gbrain process that wrote it (gbrain upgrade on its host), then retry with a new request_id.' },
diff --git a/src/core/persistence/git-conversion.ts b/src/core/persistence/git-conversion.ts
index df4418d813..dec0425154 100644
--- a/src/core/persistence/git-conversion.ts
+++ b/src/core/persistence/git-conversion.ts
@@ -83,9 +83,9 @@ const SAMPLE = 20;
const LOCK_WAIT_MS = 30_000;
const MOVED_BATCH = 10_000;
-/** The steps of an apply, in order (`ConversionProgress.phase`). */
-export const CONVERSION_PHASES = ['adopt', 'withhold', 'recheck', 'holds', 'strict_pass_1', 'strict_pass_2'] as const;
-export type ConversionPhase = typeof CONVERSION_PHASES[number];
+/** The steps of every apply, in order (`ConversionProgress.phase`); `strict_pass_closing` follows only when door commits landed (CLOSING_PASSES). */
+export type ConversionPhase = 'adopt' | 'withhold' | 'recheck' | 'holds' | 'strict_pass_1' | 'strict_pass_2' | 'strict_pass_closing';
+export const CONVERSION_PHASES: readonly ConversionPhase[] = ['adopt', 'withhold', 'recheck', 'holds', 'strict_pass_1', 'strict_pass_2'];
/**
* Where an apply stands: its phase, the door commits it made (adoption and withholding groups), and the items the phase
* went through of its total. In a strict pass (#7866), `total` is the paths the strict level holds (the planned page count,
@@ -490,10 +490,20 @@ export interface GitConversionResult {
parked: { retried: number; reclassified: number };
/** #7644: the push of the door commits, as it stood when the apply ended. */
push: ConversionPush;
- /** The plan after the flag was set and a strict pass ran: nothing left on a completed conversion. */
+ /** The closing strict passes that imported door commits landed after strict_pass_2 read its level (CLOSING_PASSES). */
+ closing_passes: number;
+ /** The plan after the flag was set and the strict passes ran: nothing left on a completed conversion. */
after: GitConversionPreview;
}
+/**
+ * A live writer's door commits (`gbrain git commit-push`) that land after strict_pass_2 read its level leave the closing
+ * plan with Git ahead of the database at their paths (git_wins) and nothing else. The source is git-canonical by then and
+ * those commits are Git's, so an incremental strict pass imports them and loses nothing: the apply runs up to this many
+ * such passes, planning again after each. Any other open item, or Git still ahead after the last one, ends incomplete.
+ */
+export const CLOSING_PASSES = 3;
+
const blockedError = (preview: GitConversionPreview): OperationError => {
const categories = DISCARD_CATEGORIES.filter(category => preview.discard.by[category] > 0);
return opError('git_conversion_blocked', `Converting source ${preview.source_id} would discard database content or cannot place it (${categories.map(category => `${category} ${preview.discard.by[category]}`).join(', ')}); nothing was written.`,
@@ -541,10 +551,10 @@ export async function applyGitConversion(engine: BrainEngine, sourceId: string,
}
const processing = { noEmbed: opts.noEmbed, noExtract: opts.noExtract };
const planned = recheck?.paths.length ?? 0;
- const pass = (phase: 'strict_pass_1' | 'strict_pass_2') => {
- report({ phase, scanned: 0, total: planned, ...ROUND_RESET });
- return strictPass(engine, sourceId, { sourceId, noPull: true, fullRound: true, ...processing, ...(phase === 'strict_pass_1' ? { canonicalOverride: 'git' as const } : {}),
- onProgress: passProgress(report) });
+ const pass = (phase: 'strict_pass_1' | 'strict_pass_2' | 'strict_pass_closing', total = planned) => {
+ report({ phase, scanned: 0, total, ...ROUND_RESET });
+ return strictPass(engine, sourceId, { sourceId, noPull: true, fullRound: phase !== 'strict_pass_closing', ...processing,
+ ...(phase === 'strict_pass_1' ? { canonicalOverride: 'git' as const } : {}), onProgress: passProgress(report) });
};
passOrThrow(sourceId, 'strict_pass_1', await pass('strict_pass_1'), progress);
await faultPoint('conversion:before_flip', { sourceId });
@@ -553,8 +563,13 @@ export async function applyGitConversion(engine: BrainEngine, sourceId: string,
const final = await pushConversion(engine, plan.context);
const push = final.reason === 'nothing_to_push' ? progress.push! : final;
report({ push });
- const after = (await planGitConversion(engine, sourceId)).preview;
- return { status: after.remaining === 0 ? 'converted' : 'incomplete', source_id: sourceId, head: after.head, committed, withheld, holds_cleared: holdsCleared, parked, push, after };
+ let after = (await planGitConversion(engine, sourceId)).preview, closing = 0;
+ for (; after.remaining > 0 && after.remaining === after.git_wins.count && closing < CLOSING_PASSES; closing++) {
+ passOrThrow(sourceId, 'strict_pass_closing', await pass('strict_pass_closing', after.git_wins.count), progress);
+ after = (await planGitConversion(engine, sourceId)).preview;
+ }
+ return { status: after.remaining === 0 ? 'converted' : 'incomplete', source_id: sourceId, head: after.head, committed, withheld, holds_cleared: holdsCleared, parked, push,
+ closing_passes: closing, after };
}
/**
diff --git a/test/git-canonical-conversion-g8a.test.ts b/test/git-canonical-conversion-g8a.test.ts
index b9217321f3..7f6091b3e2 100644
--- a/test/git-canonical-conversion-g8a.test.ts
+++ b/test/git-canonical-conversion-g8a.test.ts
@@ -817,7 +817,7 @@ for (const backend of testBackends()) describe(`[${backend}] G8a: sources canoni
let result: Record<string, any>;
try { result = await cli(engine, ['default', '--to', 'git', '--apply', '--expect', preview.preview_hash, '--no-embed', '--no-extract']); }
finally { installFaultHook(undefined); }
- expect(result).toMatchObject({ status: 'incomplete', code: 'git_conversion_incomplete', result: { status: 'incomplete', committed: { pages: 1 },
+ expect(result).toMatchObject({ status: 'incomplete', code: 'git_conversion_incomplete', result: { status: 'incomplete', committed: { pages: 1 }, closing_passes: 0,
after: { strict: true, remaining: 1, parked_effects: { retry: 1 } } } });
expect(await strict(engine)).toBe(true);
expect(conversionCommits(root, base)).toHaveLength(1);
diff --git a/test/git-canonical-conversion-live-writer.test.ts b/test/git-canonical-conversion-live-writer.test.ts
new file mode 100644
index 0000000000..4b04cd9ed8
--- /dev/null
+++ b/test/git-canonical-conversion-live-writer.test.ts
@@ -0,0 +1,106 @@
+/**
+ * A live writer during a conversion: door commits (`gbrain git commit-push` in a managed checkout, file-first and under the
+ * checkout's commit lock) that land after strict_pass_2 read its level used to leave the closing plan with Git ahead of the
+ * database and the apply incomplete (`git_conversion_incomplete`), however fast it ran. The apply now runs bounded closing
+ * strict passes that import them, and refuses with the operator contract only when Git is still ahead after the last one.
+ */
+import { describe, expect, test } from 'bun:test';
+import { execFileSync } from 'node:child_process';
+import { mkdirSync, writeFileSync } from 'node:fs';
+import { join } from 'node:path';
+import type { BrainEngine } from '../src/core/engine.ts';
+import { performManagedSync } from '../src/core/persistence/sync-run.ts';
+import { installFaultHook } from '../src/core/persistence/fault-points.ts';
+import { applyGitConversion, CLOSING_PASSES, planGitConversion, type ConversionProgress } from '../src/core/persistence/git-conversion.ts';
+import { runSourcesCanonical } from '../src/commands/sources-canonical.ts';
+import { _resetCliExitVerdictForTests } from '../src/core/cli-force-exit.ts';
+import { acquireCommitLockAsync } from '../src/core/git/commit-push.ts';
+import { managedBrain } from './helpers/managed-brain.ts';
+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';
+
+const N = 12;
+const git = (root: string, ...args: string[]) => execFileSync('git', ['-C', root, '-c', 'commit.gpgsign=false', ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();
+const note = (i: number, body = `Committed body ${i}.`) => `---\ntype: concept\ntitle: note ${i}\n---\n\n${body}\n`;
+const slugOf = (i: number) => `notes/n${String(i).padStart(2, '0')}`;
+const setup = ({ root }: { root: string }) => {
+ mkdirSync(join(root, 'notes'), { recursive: true }); git(root, 'init', '-q');
+ git(root, 'config', 'user.name', 'Example Writer'); git(root, 'config', 'user.email', '[email protected]');
+ for (let i = 0; i < N; i++) writeFileSync(join(root, `${slugOf(i)}.md`), note(i));
+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'notes');
+};
+/** A live writer's door commit: files written, then committed under the checkout's commit lock, as the door does. */
+async function doorCommit(root: string, pages: number[], body: string): Promise<void> {
+ for (const i of pages) writeFileSync(join(root, `${slugOf(i)}.md`), note(i, body));
+ const lock = await acquireCommitLockAsync(root, 10_000);
+ try { git(root, 'add', '--', ...pages.map(i => `${slugOf(i)}.md`)); git(root, 'commit', '-qm', `report: ${body}`); } finally { await lock.release(); }
+}
+const bodies = async (engine: BrainEngine) => Object.fromEntries((await engine.executeRaw<{ slug: string; body: string }>(
+ "SELECT slug, compiled_truth AS body FROM pages WHERE source_id='default' AND deleted_at IS NULL")).map(row => [row.slug, row.body.trim()]));
+const strict = async (engine: BrainEngine) => (await engine.executeRaw("SELECT 1 FROM sources WHERE id='default' AND config->>'canonical'='git'")).length === 1;
+
+for (const backend of testBackends()) describe(`[${backend}] a conversion with a live writer committing through the door`, () => {
+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;
+
+ // Value: protects=door commits that land after strict_pass_2 read its level are imported by closing passes and the apply converts; fails_when=the apply ends incomplete on them (git_ahead) or drops their content; why_new=no test wrote through the door during a strict pass; seam=fault hook at sync:level_read; no_claim=database-first writes (put_page), whose Git effects commit their own files
+ test('door commits after strict_pass_2 read its level, and during a closing pass, are imported and the source converts', () => managedBrain(async ({ engine, root }) => {
+ expect((await performManagedSync(engine, { sourceId: 'default', noPull: true, noExtract: true })).status).toBe('first_sync');
+ const seen: ConversionProgress[] = [];
+ const writes: Array<{ phase: string; pages: number[]; body: string }> = [
+ { phase: 'strict_pass_2', pages: [2, 3, 4], body: 'Report written during strict pass 2.' },
+ { phase: 'strict_pass_closing', pages: [4, 5], body: 'Report written during the closing pass.' }];
+ // The second lands at the closing pass's first level read, so the round after it imports it: one closing pass is enough.
+ installFaultHook(async point => {
+ const at = writes.findIndex(write => write.phase === seen.at(-1)?.phase);
+ if (point !== 'sync:level_read' || at < 0) return;
+ const [write] = writes.splice(at, 1);
+ await doorCommit(root, write!.pages, write!.body);
+ });
+ let result: Awaited<ReturnType<typeof applyGitConversion>>;
+ try {
+ const { preview } = await planGitConversion(engine, 'default');
+ result = await applyGitConversion(engine, 'default', preview.preview_hash, { noExtract: true, onProgress: progress => seen.push(progress) });
+ } finally { installFaultHook(undefined); }
+ expect(result).toMatchObject({ status: 'converted', closing_passes: 1, after: { strict: true, remaining: 0, git_wins: { count: 0 } } });
+ expect(writes).toEqual([]);
+ expect(await strict(engine)).toBe(true);
+ const after = await bodies(engine);
+ expect([2, 3].map(i => after[slugOf(i)])).toEqual(['Report written during strict pass 2.', 'Report written during strict pass 2.']);
+ expect([4, 5].map(i => after[slugOf(i)])).toEqual(['Report written during the closing pass.', 'Report written during the closing pass.']);
+ // The closing pass names the three paths Git was ahead at when the apply planned again.
+ expect([...new Set(seen.filter(progress => progress.phase === 'strict_pass_closing').map(progress => progress.total))]).toEqual([3]);
+ }, { databaseUrl, setup }), 300_000);
+
+ // Value: protects=a writer that commits after every pass read its last level ends the apply after CLOSING_PASSES closing passes with the operator contract, and none of its commits is lost; fails_when=the closing loop is unbounded, refuses without fix, or a door commit's content never reaches the database; why_new=the bound and its refusal; seam=fault hook at sync:level_read; no_claim=how long the passes take
+ test('a writer that commits after every pass stops the apply after the bounded closing passes, with a fix, and loses nothing', () => managedBrain(async ({ engine, root }) => {
+ expect((await performManagedSync(engine, { sourceId: 'default', noPull: true, noExtract: true })).status).toBe('first_sync');
+ let commits = 0, phase = '';
+ installFaultHook(async point => {
+ // After strict_pass_2 read its level, and after each closing pass released its drain lease (its last level read).
+ if (!(point === 'sync:level_read' && phase === 'strict_pass_2' && commits === 0) && !(point === 'sync:tail_started' && phase === 'strict_pass_closing')) return;
+ commits++;
+ await doorCommit(root, [commits % N], `Report ${commits}.`);
+ });
+ const lines: string[] = [];
+ const log = console.log, write = process.stderr.write, priorExitCode = process.exitCode;
+ console.log = (line: unknown) => { lines.push(String(line)); };
+ process.stderr.write = ((chunk: string | Uint8Array) => {
+ for (const line of String(chunk).split('\n')) if (line.startsWith('{')) phase = (JSON.parse(line) as { phase?: string }).phase ?? phase;
+ return true;
+ }) as typeof process.stderr.write;
+ try {
+ const { preview } = await planGitConversion(engine, 'default');
+ await runSourcesCanonical(engine, ['default', '--to', 'git', '--apply', '--expect', preview.preview_hash, '--no-extract', '--json']);
+ } finally { console.log = log; process.stderr.write = write; installFaultHook(undefined); _resetCliExitVerdictForTests(); process.exitCode = priorExitCode ?? 0; }
+ const out = JSON.parse(lines.at(-1)!);
+ expect(out).toMatchObject({ status: 'incomplete', code: 'git_conversion_incomplete', result: { status: 'incomplete', closing_passes: CLOSING_PASSES, after: { strict: true } },
+ fix: { argv: ['gbrain', 'sources', 'canonical', 'default', '--to', 'git', '--dry-run', '--json'] } });
+ expect(out.result.after.remaining).toBe(out.result.after.git_wins.count);
+ expect(out.message).toContain(`${CLOSING_PASSES} closing pass(es), with Git still ahead`);
+ expect(commits).toBe(1 + CLOSING_PASSES);
+ expect(await strict(engine)).toBe(true);
+ // Every door commit is Git's: a plain sync of the git-canonical source imports the last one, and the preview is clean.
+ expect((await performManagedSync(engine, { sourceId: 'default', noPull: true, noExtract: true })).status).toBe('synced');
+ expect((await bodies(engine))[slugOf(commits % N)]).toBe(`Report ${commits}.`);
+ expect((await planGitConversion(engine, 'default')).preview.remaining).toBe(0);
+ }, { databaseUrl, setup }), 300_000);
+});
diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txt
index 290eadc3f2..a7efd0b33d 100644
--- a/test/postgres-unit-arms.txt
+++ b/test/postgres-unit-arms.txt
@@ -84,6 +84,7 @@ test/forecast-journal.test.ts
test/frontmatter-reserved-keys-6454.test.ts
test/get-chunks-paging.test.ts
test/git-canonical-conversion-g8a.test.ts
+test/git-canonical-conversion-live-writer.test.ts
test/git-canonical-conversion-no-checkout-7885.test.ts
test/git-canonical-conversion-progress.test.ts
test/git-canonical-conversion-strict-pass-7866.test.ts