Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(conversion): strict passes stop re-embedding unchanged pages and refusing writer_pending on their own writes (#7866)
garrytan20be27233bda
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex edfb069199..75dd5949f6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,16 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++**Converting a large source to git-canonical no longer re-embeds unchanged pages or stops on its own writes.**++On a source with pages written by an older gbrain, `gbrain sources canonical <id> --to git --apply` re-imported every page twice, once per strict pass, and queued an embedding job for each even though nothing had changed. It then stopped with `git_conversion_incomplete` (`writer_pending`) whenever one of its own writes took longer than its wait, so a long run could end having converted nothing. Now the strict passes only import what Git actually changed, a re-import that changes nothing queues no embedding job when the page's vectors already match its text, and each pass keeps going until it finishes, as `gbrain sync` does. A write from someone else that never settles still stops the conversion before the source is flipped, and nothing is lost.++- Progress lines name the round, its candidates and the import batch; `total` is the planned path count and no longer changes mid-pass.+- Every `git_conversion_incomplete` refusal carries a `fix` (preview again, verified by `gbrain sync status --source <id> --json`), and with `--json` a `progress` object with the counts where it stopped.+- `scripts/bench/conversion-strict-pass.ts` measures a no-change conversion on a synthetic source (opt-in, never run in CI).+## [0.60.481.0] - 2026-10-28**Synthesis stops filing a session into the wrong source, and a write that asks for a private page can no longer quietly publish it.**diff --git a/docs/designs/GIT_CANONICAL_MEMORY.md b/docs/designs/GIT_CANONICAL_MEMORY.mdindex 10369ac44c..9f46dc8152 100644--- a/docs/designs/GIT_CANONICAL_MEMORY.md+++ b/docs/designs/GIT_CANONICAL_MEMORY.md@@ -90,7 +90,7 @@ G8a ships it as `gbrain sources canonical <source-id> --to git|mixed [--dry-run| parked effects | Retried once, or, for targets Git ignores, the page is stamped `git_ignored` (declared) in the same transaction that marks each parked target `declared: git_ignored`; the target stays as a record and the next preview no longer lists it. || discard | Refused: `git_conversion_blocked` names each category (both sides moved, a both-sides drift hold, an unknown-side drift hold whose row differs from the committed file, a database edit Git deleted, an uncommitted working-tree edit that differs from HEAD and the export, an ignored target, a target whose ignore status Git cannot read (`ignore_status_unknown`: beyond a symlink, outside the checkout, a failed check; only that path is refused), unparseable, unsafe path, segmented page, export mismatch, or a row a strict pass would overwrite unplanned). There is no override. |-`--apply --expect <hash>` re-plans and refuses `preview_changed` on any difference, adopts group by group (a crash leaves the source mixed and the rerun preview lists only what is left), withholds group by group, settles the parked effects, rechecks what a strict pass would overwrite, clears the holds only after that recheck passes, runs the full strict-equivalent pass, then sets `canonical: git` in the transaction that records the conversion. Writes can land between that pass and the flag, so it runs one genuine strict pass and previews again: the conversion is complete only when nothing is left; otherwise it reports `git_conversion_incomplete` with what remains and the source stays strict, because Git wins. Where durability applies the apply pushes its door commits itself, after the groups and once at the end; a push that does not land leaves them local, the receipt says so, and the next page write's push-only effect sends them. The apply prints a progress line per phase and every 10 seconds on stderr, and SIGTERM, SIGINT or SIGHUP prints a partial report (phase, groups committed, items scanned, canonical state); `GBRAIN_MEMORY_TRACE=<seconds>` (1 to 3600) adds a memory line on stderr every that many seconds, and `gbrain sync` honors it too. `--to mixed` removes the flag and changes nothing else; Git keeps every door commit.+`--apply --expect <hash>` re-plans and refuses `preview_changed` on any difference, adopts group by group (a crash leaves the source mixed and the rerun preview lists only what is left), withholds group by group, settles the parked effects, rechecks what a strict pass would overwrite, clears the holds only after that recheck passes, runs the full strict-equivalent pass, then sets `canonical: git` in the transaction that records the conversion. Writes can land between that pass and the flag, so it runs one genuine strict pass and previews again: the conversion is complete only when nothing is left; otherwise it reports `git_conversion_incomplete` with what remains and the source stays strict, because Git wins. Where durability applies the apply pushes its door commits itself, after the groups and once at the end; a push that does not land leaves them local, the receipt says so, and the next page write's push-only effect sends them. The apply prints a progress line per phase and every 10 seconds on stderr, and SIGTERM, SIGINT or SIGHUP prints a partial report (phase, groups committed, items scanned, canonical state); `GBRAIN_MEMORY_TRACE=<seconds>` (1 to 3600) adds a memory line on stderr every that many seconds, and `gbrain sync` honors it too. `--to mixed` removes the flag and changes nothing else; Git keeps every door commit. Both strict passes read the whole level (a full round) without opening a `--full` re-import episode, so a page already equal to its blob is not re-imported, and a re-import that changes nothing queues no embedding effect when every chunk already holds a vector for its text (#7866). Each pass drains under the source's drain lease as `gbrain sync` does, so a write of its own that outlives one receipt wait does not stop it; an unrelated write that never settles, or another drainer, still stops it with `git_conversion_incomplete` before the flag. Every `git_conversion_incomplete` carries `fix` (preview again; verify with `gbrain sync status --source <id> --json`), and with `--json` the refusal carries `progress`. In a strict pass, progress `total` is the planned paths for the whole pass; `round`, `candidates`, `batch` of `batches` and `scanned` say where the round stands.## The git-gap censusdiff --git a/scripts/bench/conversion-strict-pass.ts b/scripts/bench/conversion-strict-pass.tsnew file mode 100644index 0000000000..0d702ca85d--- /dev/null+++ b/scripts/bench/conversion-strict-pass.ts@@ -0,0 +1,116 @@+#!/usr/bin/env bun+/**+ * #7866: the speed and side effects of `gbrain sources canonical --to git --apply` on a synthetic production-shaped source.+ * Opt-in, never run in CI.+ *+ * bun scripts/bench/conversion-strict-pass.ts [--pages 25000] [--timebox-min 30] [--legacy 1]+ * [--database-url postgres://postgres:postgres@localhost:5434/gbrain_test] [--work ~/.capy/work/conversion-bench] [--label <name>] [--out <file.json>]+ *+ * It builds a Git checkout of N markdown pages in nested directories, syncs it into a fresh Postgres database (created and+ * dropped), and with --legacy 1 leaves the pages as an older gbrain did: every chunk holding a vector for its text and no+ * contextual-mode stamp. It then plans the conversion and applies it with nothing changed, timing each step and counting+ * the write requests and embedding effects the apply made. An apply still running at --timebox-min is stopped and its+ * rate read from the pages it settled. It prints one JSON report on stdout and writes it to --out.+ */+import { execFileSync } from 'node:child_process';+import { mkdirSync, rmSync, writeFileSync } from 'node:fs';+import { homedir } from 'node:os';+import { join, resolve } from 'node:path';++const REPO = resolve(import.meta.dir, '../..');+function flag(name: string, fallback: string): string {+ const i = process.argv.indexOf(`--${name}`);+ return i >= 0 && process.argv[i + 1] ? process.argv[i + 1]! : fallback;+}+const PAGES = Number(flag('pages', '25000'));+const TIMEBOX_MS = Number(flag('timebox-min', '30')) * 60_000;+const LEGACY = flag('legacy', '1') === '1';+const DATABASE_URL = flag('database-url', process.env.DATABASE_URL ?? 'postgres://postgres:postgres@localhost:5434/gbrain_test');+const WORK = resolve(flag('work', join(homedir(), '.capy', 'work', 'conversion-bench')));+const git = (root: string, ...args: string[]) => execFileSync('git', ['-C', root, '-c', 'commit.gpgsign=false', ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], maxBuffer: 256 * 1024 * 1024 }).trim();+const LABEL = flag('label', git(REPO, 'rev-parse', '--short', 'HEAD'));+const OUT = resolve(flag('out', join(WORK, `conversion-${LABEL.replace(/[^\w.-]/g, '_')}-${PAGES}.json`)));+const log = (msg: string) => console.error(`[conversion-bench ${new Date().toISOString().slice(11, 19)}] ${msg}`);+const seconds = (ms: number) => Math.round(ms / 100) / 10;++function checkout(): string {+ const root = join(WORK, `checkout-${PAGES}`);+ rmSync(root, { recursive: true, force: true });+ mkdirSync(root, { recursive: true });+ git(root, 'init', '-q'); git(root, 'config', 'user.name', 'Example Writer'); git(root, 'config', 'user.email', '[email protected]');+ for (let i = 0; i < PAGES; i++) {+ const dir = join(root, 'notes', `d${String(Math.floor(i / 500)).padStart(3, '0')}`);+ if (i % 500 === 0) mkdirSync(dir, { recursive: true });+ const para = `Paragraph about example topic ${i % 97}, written for a synthetic page. `.repeat(12);+ writeFileSync(join(dir, `p${String(i).padStart(6, '0')}.md`), `---\ntype: concept\ntitle: Synthetic page ${i}\n---\n\n# Synthetic page ${i}\n\n${para}\n\n## Notes\n\n${para}\n`);+ }+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'synthetic pages');+ return root;+}++async function main() {+ mkdirSync(WORK, { recursive: true });+ const { isolatedPersistencePostgres } = await import('../../test/helpers/persistence-postgres.ts');+ const { claimWorktree } = await import('../../src/core/persistence/ownership.ts');+ const { activateSharedSkillPersistence } = await import('../../src/core/persistence/skill-activation.ts');+ const { disposePersistenceConsumer } = await import('../../src/core/persistence/service.ts');+ const { drainManagedSync } = await import('../../src/core/persistence/sync-drain.ts');+ const { withDrainLease } = await import('../../src/core/persistence/sync-lease.ts');+ const { applyGitConversion, planGitConversion } = await import('../../src/core/persistence/git-conversion.ts');+ const home = join(WORK, `home-${process.pid}`);+ rmSync(home, { recursive: true, force: true }); mkdirSync(home, { recursive: true });+ process.env.GBRAIN_HOME = home; delete process.env.DATABASE_URL; delete process.env.GBRAIN_DATABASE_URL;+ let t = performance.now();+ const root = checkout();+ const report: Record<string, unknown> = { label: LABEL, pages: PAGES, legacy: LEGACY, checkout_s: seconds(performance.now() - t) };+ log(`checkout of ${PAGES} pages in ${report.checkout_s}s`);+ const { engine, close } = await isolatedPersistencePostgres(DATABASE_URL);+ const counts = async () => (await engine.executeRaw<{ embedding: number; imports: number; committed: number }>(`SELECT+ (SELECT count(*)::int FROM persistence_effects WHERE kind='embedding') AS embedding,+ (SELECT count(*)::int FROM persistence_requests WHERE intent->>'kind'='managed_sync_import') AS imports,+ (SELECT count(*)::int FROM persistence_requests WHERE intent->>'kind'='managed_sync_import' AND state='committed') AS committed`))[0]!;+ try {+ await engine.executeRaw("UPDATE sources SET local_path=$1 WHERE id='default'", [root]);+ await claimWorktree(engine, 'default', root);+ await activateSharedSkillPersistence(engine, { confirmQuiesced: true });+ t = performance.now();+ const first = await withDrainLease(engine, 'default', { holder: { kind: 'cli', id: null }, remote: false },+ () => drainManagedSync(engine, { sourceId: 'default', noPull: true, noExtract: true }, false));+ report.first_sync = { status: first.status, outcome: first.drain?.outcome, s: seconds(performance.now() - t) };+ log(`first sync ${JSON.stringify(report.first_sync)}`);+ if (LEGACY) {+ const [col] = await engine.executeRaw<{ t: string }>("SELECT format_type(atttypid, atttypmod) AS t FROM pg_attribute WHERE attrelid='content_chunks'::regclass AND attname='embedding'");+ const dims = Number(col!.t.match(/\d+/)![0]);+ await engine.executeRaw(`UPDATE content_chunks SET embedding=('[' || array_to_string(array_fill(0.1::real, ARRAY[${dims}]), ',') || ']')::vector, embedded_at=now(), embedded_text_hash=md5(chunk_text)`);+ await engine.executeRaw('UPDATE pages SET contextual_retrieval_mode=NULL');+ }+ t = performance.now();+ const { preview } = await planGitConversion(engine, 'default');+ report.plan = { s: seconds(performance.now() - t), discard: preview.discard.count, adopt: preview.adopt.count, pages: preview.pages };+ log(`plan ${JSON.stringify(report.plan)}`);+ const before = await counts();+ let last: Record<string, unknown> = {};+ t = performance.now();+ const timer = setInterval(() => log(`apply +${seconds(performance.now() - t)}s ${JSON.stringify(last)}`), 30_000);+ const outcome = await Promise.race([+ applyGitConversion(engine, 'default', preview.preview_hash, { noExtract: true, onProgress: progress => { last = { ...progress, push: undefined }; } })+ .then(result => ({ status: result.status }), (error: { code?: string; message?: string }) => ({ status: 'refused', code: error.code, message: error.message })),+ new Promise<{ status: string }>(resolve => setTimeout(() => resolve({ status: 'timebox' }), TIMEBOX_MS).unref?.()),+ ]);+ clearInterval(timer);+ const elapsed = performance.now() - t, after = await counts();+ const settled = after.committed - before.committed;+ report.apply = { ...outcome, s: seconds(elapsed), last_progress: last, import_requests: after.imports - before.imports, imports_committed: settled,+ embedding_effects: after.embedding - before.embedding,+ pages_per_s: outcome.status === 'converted' ? Math.round((PAGES / (elapsed / 1000)) * 10) / 10 : Math.round((settled / (elapsed / 1000)) * 10) / 10 };+ log(`apply ${JSON.stringify(report.apply)}`);+ } finally {+ await Promise.race([disposePersistenceConsumer(engine), new Promise(resolve => setTimeout(resolve, 10_000))]);+ await Promise.race([close(), new Promise(resolve => setTimeout(resolve, 10_000))]);+ rmSync(home, { recursive: true, force: true });+ }+ writeFileSync(OUT, JSON.stringify(report, null, 2));+ console.log(JSON.stringify(report));+ process.exit(0);+}+await main();diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsvindex f0d7b1da76..3c8d565e0a 100644--- a/scripts/module-size-limits.tsv+++ b/scripts/module-size-limits.tsv@@ -12,7 +12,7 @@ src/commands/extract.ts 2700 ratchet +8 GBRA-130 A-1: extract --stale --retry-hesrc/commands/jobs.ts 687 ratchet refactor wave 1 split target (W4 jobs, cut line); ceiling rationale: docs/architecture/key-files/commands-4.md entry for src/commands/jobs.ts; help lines: authorize-legacy --select (#5157 T3) and jobs cancel --select (DX-T2); submit --queue-only + PGLite work drain note (agent-first operator wave E5); +2 spend authorization: jobs submit --yes/--max-usd and jobs list --group help lines; +4 fix wave 9: supervisor stop help documents the 45 s wait, --queue and the verified-drain exit codes; +1 #7142: doctor-refresh handler registration (logic in doctor/last-known.ts); +20 #7242 jobs wait: subcommand row, import, usage lines and its --help text with the exit codes (logic in jobs/wait.ts)src/commands/serve-http.ts 937 ratchet refactor wave 1 W4 split into serve-http-<area>.ts modules; ceiling rationale: docs/architecture/key-files/commands-5.md entry for src/commands/serve-http.ts; +2 agent operator wave A6: ServeHttpContext.noticeLedger (import + field; logic in core/notice-ledger.ts); +2 agent contract F1/F2: ServeHttpContext.readinessCache (probed readiness for initialize tails + gbrain://capabilities); +5 Lane D facts drain: serve --http starts the drain scheduler and stops it before the engine disconnect (logic in core/facts/drain-scheduler.ts); +2 agent-operator follow-up Item 3: ServeHttpOptions.adoptServer (status-only recovery hands its bound listener to listenOrRefuse; logic in serve-http-status.ts); +5 #6317: managed-sync movement watch start/stop (watch in persistence/sync-movement.ts); +1 #6589: imports loadServeConfigs (operations see the DB-plane config read once at startup; logic in mcp/serve-config.ts); +5 #6858 ask 6: on-commit HEAD watcher start/stop (watcher in core/serve-head-watcher.ts)src/core/ai/gateway.ts 4533 ratchet +2 #6997: ChatResult.fallbackReason (interface field + doc; recorded in ai/chat-fallback.ts); +17 #6607: lazy OpenAI model discovery (discoverBeforeOpenAICall, once per reconfigure, at the chat/expand/embed entry points; it re-resolves through the gateway-internal resolveGatewayModels, so it cannot move); +18 W11 item 3: embedQuery query-embedding cache lookup/store + clear at every model-cache clear site and the test transport seam (cache lives in ai/query-embed-cache.ts); +7 fix wave 5 (#5137): provider-key redactor on every upstream error path and the first embedding 401/403 report (logic in ai/key-redact.ts, ai/key-warnings.ts); +20 System One v1: decide() delegating export + transport seam, RerankInput.onMeta/RerankMeta, TypeSafe reranker delegation to ai/decide/rerank-adapter.ts; +3 #5585: reconfigureGatewayWithEngine records expansion/chat model provenance (ai/gateway-model-sources.ts) + clears it with gateway state; lowered #5585: tracker record sites collapsed onto ai/budget-record.ts (recordOnTracker + failedCallUsage) while adding ledger fields (requested model, purpose, failed, estimated, countsAsCall); +20: schema-rejection fallback in chat() (retry once schemaless on a json_schema 4xx, remember the recipe for the process — mirrors expand()); re-summed after merging master v0.50.0.0 with the wave (structured-output + pricing hunks); +48: #4863 ChatOpts.responseSchema → tolerant json_schema Output on the chat() generateText call for openai-compatible recipes that declare structured outputs (facts extraction on Ollama); +74: #3657 post-sunset rerank short-circuit (effective-model check, once-per-process audit row + stderr, injected-clock seam); grown v0.46.26.0 megawave: config-plane probes + halt telemetry (#3387 #4312); +21 lines (OpenAI Responses API reasoning-item echo: ChatBlock 'reasoning' variant, chat()/toModelMessages() round-trip); wave-k absorbs: OCR routing seam (#4107 class), config-snapshot reader block (#3980); +21: #4107 OCR-model routing (getImageOcrModel); +10: keyless OPENAI_BASE_URL embedding override (#4385); wave-k #3622 reimpl: GBRAIN_EMBED_MAX_BATCH_TOKENS env cap for no_batch_cap recipes; +9: DeepSeek judge thinking pin plumbed through chat opts (#4069); +master v0.48 merge true-up (reranker succession: no_key preflight, reranker_skipped stage, readiness); ranker wave v0.48.4.0: ChatOpts.temperature threaded to the transport (+15); +24: #4172 recipe-declared thinking headroom on the chat()/toolLoop() default — shared isThinkingModel predicate (Claude-5 regex || recipe thinking_by_default, fail-closed), also consumed by the subagent handler; +20: #4861 claude-cli expansion routed to the schemaless viaText path (the CLI adapter ignores responseFormat, so the native generateObject branch had no fallback); retired provider integration removed; safety wave: migration reranker input ceilings; lowered by fix wave 3 #5680 (embed request planning peeled to ai/embed-batch-plan.ts); +5 #4616: empty inputs skip the provider and every text/multimodal result is screened for zero-norm/NaN/Inf vectors (logic lives in ai/embedding-guard.ts); fix wave 3 integration: #4616 empty-input filtering feeds the shared #5680 batch planner; +3 #5691: embedQuery takes a per-request queryPrefix (no gateway state); +26 fix wave 7 (#5642): embed spend records bank provider-reported usage.tokens (all sub-batches reported → measured; chars-per-token estimate stays the fallback and the failure charge); +2 #5892: import + load-time call of installAiSdkWarningWriter (writer lives in ai/sdk-warnings.ts); +12 #5490: chat() delegation to ai/chat-fallback.ts when chat_fallback_chain is set (attempt loop lives there), ChatOpts.allowFallback + ToolLoopOpts.allowFallback fields and the toolLoop forward; +2 #5490 review: ChatResult.fallbackFrom (the call model a chat_fallback_chain answer replaced); +1 fix wave lane B3: configureGateway keeps chat_fallback_on_refusal (the refusal switch lives in ai/chat-fallback.ts); +18 fix wave 9 lane B: ChatOpts.thinking (#5331, per-route mapping lives in ai/thinking-off.ts), ChatOpts.timeoutMs + ToolLoopOpts.turnTimeoutMs and the turn forward (#4921); interface fields and their docs cannot move; +2 P8: GBRAIN_AI_CALL_LOG observer install in configureGateway (logic in ai/call-log.ts); +2 W9F item 5: thinkingOffOutputCap export shared by chat() and the judge preflight estimates (capability table lives in ai/thinking-off.ts); +3 A1 (wave 0): ChatResult.usage.reasoning_tokens (interface field + doc) and the chat() reasoning-subset read beside the cache counts; the ai-sdk importer allowlist is shrink-only, so the seam stays here); +9 GOOGLE_GENERATIVE_AI_BASE_URL: resolveNativeBaseUrl('google') + the three native-google construction sites (eval metering proxy); re-summed at the master 6ccf30c3e merge (4417 integration + 4423 master - 4408 base); lowered -5: the expand() prompt template peeled to ai/expansion-prompt.ts (buildExpansionPrompt) so the longmemeval call planner measures the same text; +5 #7176: expand() writes its chat_usage_log row (phase expansion) beside the budget record, and the guardedExpansion instance that names its invocation gateway.expand (ledger in ai/chat-usage.ts); +2 PR 4 (Wave B incremental derivation, transcripts) 4b round 4: chat() forwards a metered page step's providerRetries (BudgetTracker) as the SDK's maxRetries, so every attempt is one it reserved for-src/commands/sync.ts 556 ratchet refactor wave 1 split target (W4 sync); ceiling rationale: docs/architecture/key-files/commands-6.md entry for src/commands/sync.ts; +2 for the #5751 SyncResult.legacySkips interface field (fix wave 4); +2 fix wave 4 lane B: SyncResult.connectorHolds interface field (held counts for the sync summary); +1 fix wave 5 (#5032): SyncResult.fileRefusals interface field (named per-file Windows colon refusals); +13 #5984: SyncResult.drain / managedCursor and SyncOpts.drain / drainStartedAt interface fields (drain loop lives in persistence/sync-drain.ts); +1 #5984: SyncOpts.noBulk / bulk interface fields; v0.60.48.0 merge: #5984 growth re-summed over v0.60.47.0 (510 measured); +2 #5984 lanes: SyncOpts.lanes field and the lanes progress-event field; +2 #6278: SyncResult.breaker interface field (the preparation-stall breaker verdict); +1 fix wave 13 P4.3 #6386: the JSON-output comment names writeJsonDocument for sync status/unblock; +2 fix wave 13 PR2: the transaction-pooler stderr line (P2.2) and the staged catch-up result field (P2.14, loop in commands/sync/perform.ts); +2 fix wave 15 P4.5 (#5459 adjacent): SyncResult.working_tree_kept interface field (the tree-walk keep lives in persistence/sync-discovery.ts); +4 #6643: SyncResult.retryWait / knockOns interface fields (recovery lives in persistence/sync-run.ts, sync-knock-on.ts); +2 #6696 (A7): SyncOpts.capacityPreflight interface field (preflight lives in persistence/sync-capacity-preflight.ts); +2 #6696 C1: SyncOpts.embedDeferred interface field (the cost gate's runtime embed deferral; an interface field cannot move), composed with PR A's SyncOpts.capacityPreflight; +2 #6693: SyncResult.replayedRemembers interface field (the post-sync replay lives in commands/sync/perform.ts and repair/failed-writes.ts); +12 #6712 PR 1: SyncResult.leaseHeld / inherited_options / cursorAdopted / cursorFanIn and SyncOpts.drainHolder / followDrainLease interface fields (the lease lives in persistence/sync-lease.ts, adoption in persistence/sync-cursor-key.ts and sync-admitted.ts); +2 #6712 PR 2 (L5): SyncResult.tail interface field (the tail lives in persistence/sync-tail.ts); +2 #6712 PR 2 integration: L4 ghostPages and L5 tail SyncResult interface fields both land (each lane raised 542->544 alone); +2 #6948: SyncResult.origin_twins_resolved interface field (the heal lives in repair/origin-twins-heal.ts); +5 #6712 PR 4b: SyncResult.pass / reconcile and SyncOpts.costApproved interface fields (an interface field cannot move; the pass summary and the pass reason union live in persistence/sync-pass-row.ts, the remote reconcile in sync-pass-apply.ts); lowered to 546 #6712 PR 4b lane C: SyncResult.cursorAdopted / cursorFanIn retired with the cursor engine; +2 #7167: SyncResult.writer interface field (a managed dry run reports the writer instead of waiting on it; the read lives in persistence/sync-dry-run-writer.ts); +6 G8a git-canonical: SyncOpts.canonicalOverride interface field and its doc comment (internal: only the conversion apply in persistence/git-conversion.ts passes it; applied in persistence/sync-discovery.ts)+src/commands/sync.ts 558 ratchet refactor wave 1 split target (W4 sync); ceiling rationale: docs/architecture/key-files/commands-6.md entry for src/commands/sync.ts; +2 for the #5751 SyncResult.legacySkips interface field (fix wave 4); +2 fix wave 4 lane B: SyncResult.connectorHolds interface field (held counts for the sync summary); +1 fix wave 5 (#5032): SyncResult.fileRefusals interface field (named per-file Windows colon refusals); +13 #5984: SyncResult.drain / managedCursor and SyncOpts.drain / drainStartedAt interface fields (drain loop lives in persistence/sync-drain.ts); +1 #5984: SyncOpts.noBulk / bulk interface fields; v0.60.48.0 merge: #5984 growth re-summed over v0.60.47.0 (510 measured); +2 #5984 lanes: SyncOpts.lanes field and the lanes progress-event field; +2 #6278: SyncResult.breaker interface field (the preparation-stall breaker verdict); +1 fix wave 13 P4.3 #6386: the JSON-output comment names writeJsonDocument for sync status/unblock; +2 fix wave 13 PR2: the transaction-pooler stderr line (P2.2) and the staged catch-up result field (P2.14, loop in commands/sync/perform.ts); +2 fix wave 15 P4.5 (#5459 adjacent): SyncResult.working_tree_kept interface field (the tree-walk keep lives in persistence/sync-discovery.ts); +4 #6643: SyncResult.retryWait / knockOns interface fields (recovery lives in persistence/sync-run.ts, sync-knock-on.ts); +2 #6696 (A7): SyncOpts.capacityPreflight interface field (preflight lives in persistence/sync-capacity-preflight.ts); +2 #6696 C1: SyncOpts.embedDeferred interface field (the cost gate's runtime embed deferral; an interface field cannot move), composed with PR A's SyncOpts.capacityPreflight; +2 #6693: SyncResult.replayedRemembers interface field (the post-sync replay lives in commands/sync/perform.ts and repair/failed-writes.ts); +12 #6712 PR 1: SyncResult.leaseHeld / inherited_options / cursorAdopted / cursorFanIn and SyncOpts.drainHolder / followDrainLease interface fields (the lease lives in persistence/sync-lease.ts, adoption in persistence/sync-cursor-key.ts and sync-admitted.ts); +2 #6712 PR 2 (L5): SyncResult.tail interface field (the tail lives in persistence/sync-tail.ts); +2 #6712 PR 2 integration: L4 ghostPages and L5 tail SyncResult interface fields both land (each lane raised 542->544 alone); +2 #6948: SyncResult.origin_twins_resolved interface field (the heal lives in repair/origin-twins-heal.ts); +5 #6712 PR 4b: SyncResult.pass / reconcile and SyncOpts.costApproved interface fields (an interface field cannot move; the pass summary and the pass reason union live in persistence/sync-pass-row.ts, the remote reconcile in sync-pass-apply.ts); lowered to 546 #6712 PR 4b lane C: SyncResult.cursorAdopted / cursorFanIn retired with the cursor engine; +2 #7167: SyncResult.writer interface field (a managed dry run reports the writer instead of waiting on it; the read lives in persistence/sync-dry-run-writer.ts); +6 G8a git-canonical: SyncOpts.canonicalOverride interface field and its doc comment (internal: only the conversion apply in persistence/git-conversion.ts passes it; applied in persistence/sync-discovery.ts); +2 #7866: SyncOpts.fullRound interface field and its doc comment (internal: the conversion strict passes in persistence/git-conversion.ts set it; read in persistence/sync-pass-run.ts)src/core/cycle.ts 3345 ratchet +2 GBRA-130 C-2 (#6946): the sync phase reads an owned clone's pull block and reports it as pull_blocked (logic in url-clone.ts urlClonePullBlock/urlClonePullBlockSummary); +6 GBRA-130 A-1: the extract phase warns on failed or breaker-stopped stale drain pages (logic in cycle/connector-extract.ts staleDrainFailures); +1 W1 belief ledger: the extract phase runs the sampled belief drift reconcile (logic in belief/coverage-report.ts); +2 System One v1 S9: extract_facts phase tail runs the conflict sweep (ai/decide/sweep.ts) only when the slot is not off; +2 System One v1: purge phase prunes decision receipts (ai/decide/store.ts); grown v0.46.26.0 megawave (#4102 #2608) + master v0.46.26.0 cycle-start orphan recovery; +10 wave G #3974: cycle-sync uncommitted-drift warn; grown wave-g (#4416): synthesize_concepts source threading; grown wave-g (#4416): synthesize_concepts source threading (same fix absorbed as #4417 on wave-k); wave-k: ); +7: #4077 abort threading; +10: #4745 fullImplicitSourceCycle opt-in contract; +3: #4890 extract phase jsonMode/quiet + pages-not-links totals; +1: wave review — extract phase passes quiet:true + jsonMode:false; +1: extract_facts pagesFailed detail; +8 fix wave 3: one cycle date resolved in runCycle and threaded to synthesize/patterns/drift (C-15); +1 #5632: cycle sync resumes an unfinished managed cursor with its stored options; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +1 at the merge; -41 #5809: anyAbortSignal peeled to abort-signals.ts; +5 #5809: extract_atoms phase gets a real abort signal; +4 #5809: cycle_already_running names the lock holder; +7 #5255: cycle sync runs a managed pull request as no-pull and reports upstream_refresh; +1 #4578: a lock steal during the last selected phase reports lock_stolen, not aborted; +3 #5536: keyless embed phase clean-skips when embedding_disabled is set; +5 F4b: planner-stats refresh after the freshness phases (logic in planner-stats.ts); +2 agent operator wave A1: PhaseResult sibling code/fix; +20 #5876: Life Chronicle phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block; v0.60.46.0 merge: re-summed to the merged line count (3224); +3 retrieval feedback retention in the purge phase (logic in feedback/store.ts pruneRetrievalFeedback); 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); P3 x foundations-2 composition: deltas re-summed at the merge; 12 Lane D: facts_drain phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch line (logic in cycle/facts-drain.ts); +15 GBRA-49 temporal typed edges: edge_contradictions phase registration + dispatch block + extract-phase relationship sweep call (logic in cycle/edge-contradictions.ts, link-relationships.ts); P3 x P1 (v0.60.57.0) composition: deltas re-summed at the merge; +1 fix wave 9 lane P (#5151): extract_facts phase passes drain so the facts watermark drain runs (logic in facts/reconcile-watermark.ts); +10 #6188: fence_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in cycle/fence-repair.ts); +19 fix wave 12 #6242: source cycles with brain-wide phases also take the shared gbrain-cycle lease (acquireDbCycleLock lease set, needsMaintenanceLease, skipBrainWidePhases, all-skipped deriveStatus case; lease logic in cycle/lock-set.ts); +16 #6377: content_repair phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK), the REPAIR_LANE_PHASES dispatch helper shared with fence_repair (logic in cycle/content-repair.ts); +10 C4 pinned questions: standing_questions phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its runCycle dispatch block (logic in questions/phase.ts); +11 #6696 (A5, A4): runCycle becomes the background-origin wrapper over runCycleUnscoped and adds the capacity notices (logic in persistence/admission-origin.ts, capacity-snapshot.ts) plus the CycleReport.notices interface field; +2 #6297: CycleOpts.directPhases (operator-named phases skip the patterns cadence); +5 #6865: unindexed_files phase registration (CyclePhase, ALL_PHASES, NEEDS_LOCK) and its line in the REPAIR_LANE_PHASES dispatch helper (logic in cycle/unindexed-files.ts); +2 fix wave 20 #6954: the extract phase warns on extract_write_lost and its stale drain retries the pages left due; +5 #7241: CycleOpts.onPhaseStart and its best-effort call in timePhase, so the autopilot-cycle job writes its current phase to minion_jobs.progress; +1 GBRA-133: the extract phase passes the stale-drain budget to its targeted pass when sync gave no slug list; +7 #7630 (GBRA-207): the sync phase refuses sync_anchor_relative when no source matches its directory and default's local_path is relativesrc/core/cycle/synthesize.ts 3244 ratchet +3 #7805: each captured session synthesizes only into its own frozen source (routing import, the outer wrapper, the routing parameter; logic in cycle/synthesize-source-routing.ts); +26 System One v1 S7/S8: triage and grounding decide delegation (triage-decide.ts, grounding-decide.ts); merged waves; grown v0.46.26.0 megawave + master rolling lease via shared throttled renewer; grown wave-g (#4506): cycle summaries can stay out of the source repo; +18: expired-verdict sweep + DeepSeek judge thinking pin (#4069); +47: #4077 cooperative-abort threading; +110: eval fix wave F6 spend surfacing (withChatPhase wrap, triage/child token telemetry, zero-page counter); +32: F1b quote-verify call site + kill switch; +2: F1a/F3/F4a OUTPUT POLICY rules; +59: F2 verified-segment rescue (gate at report construction, rescue knobs, telemetry); +19: rubric v2 peak-scoring + docblock rescue branch; +50: #4337 summary link-sample cap + dream_created_cycle_date preserve-on-rerun; +5: #4348 resolveCycleDate wiring (local-day summary bucketing); +3 round-3 rebase true-up; +55: #4805 synth-v2 completion skip (findSynthV2Completion + prefix-parameterized loadSuccessfulSynthesisKeys); +10: #4886 mode-aware buildSynthesisPrompt (oneshot children drop tool-use/final-prose guidance); #5105: small legacy writer/refusal and shared revision CAS delegates; +4: merge of #5666 durable synthesis completions with the v0.59.18.0 verify-epoch wiring; +7 #5413/#5471 discovery exclusions (self-capture session ids, dream output directories); +6 fix wave 2 (v0.60.5.0) integration: W7 paid-loop breaker check before submission and released-key daily-cap count on top of the W1b discovery exclusions; +6 v0.60.5.0 master-merge true-up: the wave-2 growth (+13 W1b, +6 W7) re-summed over master v0.59.18.0 at 3222 lines with no slack; +76 fix wave 3: per-run USD gate + fail-closed daily cap (C-13), child first-write ownership for stamp/reverse-write (C-8, #5685), cycle-date prompt hint (C-15), opt-in attribution rule (#5425); fix wave 3 x fix wave 2 (v0.60.5.0) composition: +19 at the merge; +2 #5806 triageNow clock seam forwarded to triage (interface field + call-site option); +4 #5854: a postprocess or summary publish still pending after its wait is deferred (publish_pending warn, no cooldown stamp; helper in synthesize-postprocess.ts); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); Foundations 2 Lane C: maintenanceTransaction import (attributed summary page); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); Foundations 2 Lane C: maintenanceTransaction import (attributed summary page); +14 #5490 review: a triage verdict a chat_fallback_chain model gave is used for the run but not cached (TriageResult.answeredBy, judge-client marker, guarded putDreamVerdict); lowered to 3200 by wave 9 lane W #5145: completion-key helpers (findLegacyCompletion, findSynthV2Completion, loadSuccessfulSynthesisKeys) peeled to synthesize-completion.ts with the new pre-triage partition; +13 P2 extraction.date_grounding: transcript observation date line instead of a run-date "Today's date" (config field + prompt arg); +1 fix wave 11 #6160: oneshot_task_prefixes on the child job data; +18 fix wave 12 #6069: unreliable-verdict backoff in runTriagePass (report code, pass backoff count, triage details entry; marker logic in triage-backoff.ts); +1 fix wave 12 #6242: SYNTH_PHASE_FAIL keeps an OperationError's suggestion and code (OperationError import); +3 wave 17 #6616: slug discovery and postprocessing share the canonical committed child outputs (loadCommittedSynthesisOutputs in synthesize-postprocess.ts; one call-site line, one parameter, one lookup line); lowered to 3224 by wave 17 #6642: the summary provenance section moved to summaryProvenanceLines in dream-provenance.ts; +6 GBRA-130 B-6 (#6860): surrogate-safe triage cuts (truncateUtf8) and the per-transcript verdict-cache write call site (logic in triage-backoff.ts VerdictCacheWrites); +2 GBRA-161 PR 4 (#7065): the synthesize phase activates each synthesized transcript's outputs as its generation (activateSynthesisGenerations import and call site; logic in cycle/synthesize-generations.ts); +8 GBRA-161 PR 5 (S-2): the dream summary publishes as a single-item dream_summary generation bound to its pre-run revision (publishSinglePage with a generation_fenced skip); +1 GBRA-60: a managed synthesis child carries dream_output (verify and stamp before put_page; cycle/synthesize-child-verify.ts)src/core/cycle/extract-atoms.ts 1658 ratchet crossed the 1500 unlisted cap in the atoms/extraction fix wave: +21 embed-aware cost gate (#4878), +65 atoms-array anchor scan (#4913); peel candidate: the parse block (parseAtomsOutcome / parseArrayAtOffset / atomsFromParsedArray) into extract-atoms-parse.ts; +35: #4853 transcript-origin atoms link to their conversation page; +130: #4916 transcript failure-count/tombstone (extract_atoms_transcript_state, TODOS P2(b)); wave composition: #4916 transcript-state skip + #4948 zero-atom prompt + #4943 meetings source-id summed; +3: wave review — a later `[]` is an honest zero-yield, a non-atom-shaped array is a counted failure; +13: wave review — explicit budget keeps the cap over an unpriced embed route ($0-priced); +14: wave review — transcript-state write errors surfaced, missing table warns once; merged reliability/shared-skills source integration; +37 fix wave 3: stable undated atom dates + stale-atom retirement (C-14); +3 #5525: effectiveVisibility stamped on non-managed atoms; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +3 at the merge; +6 fix wave 3 lane A (#5601): accepted-pending atom batches count as progress (helper in persistence/accepted-pending.ts); +4 fix wave 4: #5705 transcript-as-data system prompt lines and its message helper, #5627 response schema import (the atom-type list moved to extract-atoms-schema.ts); +30 #5809/#5832: drain hard/soft stop signals (ExtractAtomsOpts.signal/stopSignal, pre-commit abort checks, no strike or post-stop writes after a hard stop) and the cancellable backlog count; +1 #5812: transcript items are paste-stripped (corpusTextForExtraction import) before the input cut; +1 #5820: extract_atoms discovery passes selfCaptureSessionIds like synthesize; +10 #5856/#5854: drain attempt state (one BudgetTracker + one publish wait across batches) and the opt-in connector email/meeting exclusion (predicate in connector-atoms.ts); +1 Foundations 2 Lane C: maintenanceTransaction import (attributed stale-atom retirement); +5 fix wave 9 lane P (#5041): opt-in atom write-through hook and retired-atom slugs returned for file removal (logic in derived-write-through.ts); the parse block (parseAtomsOutcome / parseArrayAtOffset / atomsFromParsedArray) is peeled into extract-atoms-parse.ts (-193, fix wave 10 lane B); +35: #4853 transcript-origin atoms link to their conversation page; +130: #4916 transcript failure-count/tombstone (extract_atoms_transcript_state, TODOS P2(b)); wave composition: #4916 transcript-state skip + #4948 zero-atom prompt + #4943 meetings source-id summed; +3: wave review — a later `[]` is an honest zero-yield, a non-atom-shaped array is a counted failure; +13: wave review — explicit budget keeps the cap over an unpriced embed route ($0-priced); +14: wave review — transcript-state write errors surfaced, missing table warns once; merged reliability/shared-skills source integration; +37 fix wave 3: stable undated atom dates + stale-atom retirement (C-14); +3 #5525: effectiveVisibility stamped on non-managed atoms; fix wave 3 x fix wave 2 (v0.60.5.0) composition: +3 at the merge; +6 fix wave 3 lane A (#5601): accepted-pending atom batches count as progress (helper in persistence/accepted-pending.ts); +4 fix wave 4: #5705 transcript-as-data system prompt lines and its message helper, #5627 response schema import (the atom-type list moved to extract-atoms-schema.ts); +30 #5809/#5832: drain hard/soft stop signals (ExtractAtomsOpts.signal/stopSignal, pre-commit abort checks, no strike or post-stop writes after a hard stop) and the cancellable backlog count; +1 #5812: transcript items are paste-stripped (corpusTextForExtraction import) before the input cut; +1 #5820: extract_atoms discovery passes selfCaptureSessionIds like synthesize; +10 #5856/#5854: drain attempt state (one BudgetTracker + one publish wait across batches) and the opt-in connector email/meeting exclusion (predicate in connector-atoms.ts); +1 Foundations 2 Lane C: maintenanceTransaction import (attributed stale-atom retirement); +5 fix wave 9 lane P (#5041): opt-in atom write-through hook and retired-atom slugs returned for file removal (logic in derived-write-through.ts); +17 #5031: same-page case-insensitive atom adoption fallback (one page-atom scan per source page per run); +12 P2 atomsPrompt helper: extraction.date_grounding source observation date + rule in the atoms prompt; +4 GBRA-75: octet_length-bounded body-length predicate (minCompiledTruthChars) for discovery and the backlog count; GBRA-130 A-42 (#6804): dot-led atom stems and the legacy-slug adoption guarddiff --git a/src/commands/sources-canonical.ts b/src/commands/sources-canonical.tsindex 97c2ccbd7d..22e4bb63f3 100644--- a/src/commands/sources-canonical.ts+++ b/src/commands/sources-canonical.ts@@ -10,7 +10,7 @@ import { OperationError, opError } from '../core/ops/contract.ts';import { cliRenderContext, toAgentError } from '../core/agent-output.ts';import { registerSignalOwner, triggerCleanupAndExit } from '../core/process-cleanup.ts';import { startMemoryTrace } from '../core/memory-trace.ts';-import { applyGitConversion, applyMixedConversion, assertConversionCaller, CONVERSION_GROUP_MAX, planGitConversion, planMixedConversion,+import { applyGitConversion, applyMixedConversion, assertConversionCaller, CONVERSION_GROUP_MAX, conversionIncompleteFix, planGitConversion, planMixedConversion,type ConversionProgress, type GitConversionPreview, type GitConversionResult, type MixedConversionPreview } from '../core/persistence/git-conversion.ts';const USAGE = 'Usage: gbrain sources canonical <source-id> --to git|mixed [--dry-run | --apply --expect <hash> [--no-embed] [--no-extract]] [--json]';@@ -71,8 +71,12 @@ function describeMixed(p: MixedConversionPreview): string {: `Source ${p.source_id} is not git-canonical; there is nothing to change.`;}-const progressText = (p: ConversionProgress) =>- `[sources canonical] ${p.phase}${p.round ? ` round ${p.round}` : ''}: ${p.scanned}/${p.total} scanned, ${p.groups_committed} group(s) committed`;+const roundText = (p: ConversionProgress) => p.round ? ` round ${p.round}${p.batches ? ` batch ${p.batch}/${p.batches}` : ''}${p.candidates !== undefined ? ` (${p.scanned}/${p.candidates} candidate(s) settled)` : ''}` : '';+const progressText = (p: ConversionProgress) => p.phase.startsWith('strict_pass')+ ? `[sources canonical] ${p.phase}${roundText(p)}: ${p.total} planned path(s), ${p.groups_committed} group(s) committed`+ : `[sources canonical] ${p.phase}: ${p.scanned}/${p.total} scanned, ${p.groups_committed} group(s) committed`;+/** #7866: where an apply stood when it was refused, so the refusal carries its counts. */+const stoppedAt = new WeakMap<object, ConversionProgress>();/** The source's canonical state as its row says now: `git` once the flag is set, else `mixed`; `unknown` when the read fails or takes over 2 s. */async function canonicalState(engine: BrainEngine, sourceId: string): Promise<'git' | 'mixed' | 'unknown'> {@@ -103,7 +107,7 @@ async function withConversionReport<T>(engine: BrainEngine, sourceId: string, jsconst at = progress ?? { phase: 'adopt', groups_committed: 0, scanned: 0, total: 0 };const error = opError('git_conversion_incomplete', `The conversion of source ${sourceId} was interrupted by ${signal} during ${at.phase}; the source is ${canonical === 'git' ? 'git-canonical' : canonical === 'mixed' ? 'still mixed' : 'in an unknown state'}.`,`Preview again with gbrain sources canonical ${sourceId} --to git --dry-run once gbrain sync status --source ${sourceId} --json shows the source idle, then apply the new hash; the groups already committed read equal.`,- { docs: DOCS, detail: 'interrupted' });+ { docs: DOCS, detail: 'interrupted', fix: conversionIncompleteFix(sourceId) });const report = { status: 'interrupted', signal, source_id: sourceId, ...at, canonical };process.stdout.write(`${json ? JSON.stringify({ ...report, ...toAgentError(error, { transport: 'cli', command: 'sources canonical', render: cliRenderContext() }) }): `Conversion interrupted (${signal}) during ${at.phase}${at.round ? ` round ${at.round}` : ''}: ${at.scanned}/${at.total} scanned, ${at.groups_committed} group(s) committed; source canonical state: ${canonical}.${at.push?.state === 'pending' ? `\nPush pending (${at.push.code}): ${at.push.notice}` : ''}\nNext: ${error.suggestion}`}\n`);@@ -114,6 +118,7 @@ async function withConversionReport<T>(engine: BrainEngine, sourceId: string, jsconst release = registerSignalOwner('sources canonical --apply', interrupted);process.on('SIGINT', onSigint);try { return await apply(onProgress); }+ catch (error) { if (error instanceof OperationError && progress) stoppedAt.set(error, progress); throw error; }finally { clearInterval(timer); stopTrace(); release(); process.off('SIGINT', onSigint); }}@@ -136,7 +141,7 @@ export async function runSourcesCanonical(engine: BrainEngine, args: string[]):if (result.status === 'incomplete') {const error = opError('git_conversion_incomplete', `Source ${parsed.sourceId} is git-canonical, and ${result.after.remaining} item(s) were still open after its strict pass.`,`Writes landed during the conversion. Git wins on the source now; preview again with gbrain sources canonical ${parsed.sourceId} --to git --dry-run and apply the new hash.`,- { docs: DOCS, detail: 'work_left' });+ { docs: DOCS, detail: 'work_left', fix: conversionIncompleteFix(parsed.sourceId) });print({ status: 'incomplete', result, ...toAgentError(error, { transport: 'cli', command: 'sources canonical', render: cliRenderContext() }) }, describeResult(result));setCliExitVerdict(1);return;@@ -145,7 +150,9 @@ export async function runSourcesCanonical(engine: BrainEngine, args: string[]):} catch (error) {if (!(error instanceof OperationError)) throw error;const env = toAgentError(error, { transport: 'cli', command: 'sources canonical', render: cliRenderContext() });- console.log(json ? JSON.stringify({ status: 'refused', ...env }) : `Conversion refused (${error.code}): ${error.message}\nFix: ${error.suggestion ?? USAGE}`);+ const at = stoppedAt.get(error);+ console.log(json ? JSON.stringify({ status: 'refused', ...env, ...(at ? { progress: at } : {}) })+ : `Conversion refused (${error.code}): ${error.message}\nFix: ${error.suggestion ?? USAGE}${env.fix?.command ? `\nNext: ${env.fix.command}` : ''}`);setCliExitVerdict(1);}}diff --git a/src/commands/sync.ts b/src/commands/sync.tsindex d09f503f18..e584dcbe01 100644--- a/src/commands/sync.ts+++ b/src/commands/sync.ts@@ -208,6 +208,8 @@ export interface SyncOpts {* working-tree mode (resolveManagedSyncContext refuses `--working-tree` for a strict-classified pass).*/canonicalOverride?: 'git';+ /** #7866, internal (the conversion's strict passes): a full round without a `--full` episode, so a page equal to its blob is not re-imported. */+ fullRound?: boolean;/*** v0.18.0 Step 5 — sync a specific named source. When set, sync reads* local_path + last_commit from the sources table (not the global@@ -371,7 +373,7 @@ export interface SyncOpts {* breadcrumbs already cover that surface).*/onProgress?: (p: { phase: string; bankedFiles?: number; total?: number; waived?: boolean; group?: number;- /** #5984 lanes (phase `managed_sync.lanes`). */ lanes?: { effective: number; stepDown: string | null; overlapped: number; fallbacks: number }; /** #7243: a pass round (`managed_sync.round_level`, `managed_sync.round_admit`). */ round?: { round: number; target: string; paths: number; read_ms?: number; imports?: number; deletes?: number; in_flight?: number } }) => void;+ /** #5984 lanes (phase `managed_sync.lanes`). */ lanes?: { effective: number; stepDown: string | null; overlapped: number; fallbacks: number }; /** #7243: a pass round (`managed_sync.round_level`, `managed_sync.round_admit`; #7866 `managed_sync.round_batch`: `batch` of `batches`, starting at candidate `base`). */ round?: { round: number; target: string; paths: number; read_ms?: number; imports?: number; deletes?: number; in_flight?: number; batch?: number; batches?: number; base?: number } }) => void;/*** #5984: managed sync only. Re-enter the single-pass managed sync until the* cursor is done, the caller's signal/deadline stops it, or it is blockeddiff --git a/src/core/persistence/coordinator.ts b/src/core/persistence/coordinator.tsindex d298a171e5..8c51d3f913 100644--- a/src/core/persistence/coordinator.ts+++ b/src/core/persistence/coordinator.ts@@ -67,6 +67,8 @@ interface PreparedMutationBase {/** #5470 screening only: the content is unchanged, but publication still runs (and queues effects). */contentUnchanged?: boolean;deferEmbedding?: boolean;+ /** #7866: a content no-op whose chunks all hold a vector for their text: publication queues no embedding effect (`embedding_state: current`). */+ embeddingCurrent?: boolean;/** Why a page write bound to a worktree publishes no file (receipt `write_through.skipped`). */databaseOnlyReason?: 'db_only' | 'unbound_source' | 'mirror_read_only' | 'sync_excluded' | 'shared_origin' | 'origin_twin' | 'detached';/** #6517: the receipt's `write_through.warning` for a database-only publication, set by `validate` under the publication guards. */diff --git a/src/core/persistence/effect-journal.ts b/src/core/persistence/effect-journal.tsindex a0f0eae5a7..69d2d401b6 100644--- a/src/core/persistence/effect-journal.ts+++ b/src/core/persistence/effect-journal.ts@@ -47,8 +47,8 @@ export async function queuePublicationEffects(tx: BrainEngine, row: EffectRequesif (!opts.pushOnly && outcome.persistence && typeof outcome.persistence === 'object') Object.assign(outcome.persistence, { git_state: 'queued' });}if (snapshot && !snapshot.page.deleted_at) {- if (!prepared?.deferEmbedding) queue('embedding');- outcome.embedding_state = await embeddingDisabled(tx) ? 'disabled' : prepared?.deferEmbedding ? 'deferred' : 'queued';+ if (!prepared?.deferEmbedding && !prepared?.embeddingCurrent) queue('embedding');+ outcome.embedding_state = await embeddingDisabled(tx) ? 'disabled' : prepared?.deferEmbedding ? 'deferred' : prepared?.embeddingCurrent ? 'current' : 'queued';if ((outcome.facts_backstop as { queued?: boolean } | undefined)?.queued) {if (!(await isFactsExtractionEnabled(tx))) outcome.facts_backstop = { skipped: 'extraction_disabled' };else queue('facts-backstop', { visibility: await resolveDefaultVisibility(tx) });diff --git a/src/core/persistence/git-conversion.ts b/src/core/persistence/git-conversion.tsindex 9ae10476b7..c6d4e3ad7a 100644--- a/src/core/persistence/git-conversion.ts+++ b/src/core/persistence/git-conversion.ts@@ -42,7 +42,10 @@import { lstatSync, mkdirSync, readFileSync, realpathSync, renameSync, rmSync, writeFileSync } from 'node:fs';import { dirname, join, relative, sep } from 'node:path';import type { BrainEngine } from '../engine.ts';-import type { SyncOpts } from '../../commands/sync.ts';+import type { SyncOpts, SyncResult } from '../../commands/sync.ts';+import type { Action } from '../agent-output.ts';+import { LockUnavailableError, syncLockId } from '../db-lock.ts';+import { syncOutcome } from './sync-drain.ts';import { loadConfig } from '../config.ts';import { resolveSourceId } from '../source-resolver.ts';import { opError, OperationError } from '../ops/contract.ts';@@ -84,10 +87,13 @@ export const CONVERSION_PHASES = ['adopt', 'withhold', 'recheck', 'holds', 'striexport type ConversionPhase = typeof CONVERSION_PHASES[number];/*** Where an apply stands: its phase, the door commits it made (adoption and withholding groups), and the items the phase- * went through of its total. In a strict pass, `round` is the pass round, `total` the paths its level read until it- * admits, then its batch, and `scanned` the batch entries settled.+ * went through of its total. In a strict pass (#7866), `total` is the paths the strict level holds (the planned page count,+ * set from the recheck's read and kept for the whole pass), `round` the pass round, `candidates` the paths that round+ * imports or deletes, `batch` of `batches` the import batch it is draining, and `scanned` the candidates of the round+ * settled so far, across its batches.*/-export interface ConversionProgress { phase: ConversionPhase; groups_committed: number; scanned: number; total: number; round?: number; push?: ConversionPush }+export interface ConversionProgress { phase: ConversionPhase; groups_committed: number; scanned: number; total: number; round?: number; candidates?: number;+ batch?: number; batches?: number; push?: ConversionPush }/*** Where the push of the apply's door commits stands: `pushed`; `pending` (the commits are local and the next page write's* push-only effect pushes them; `code` and `notice` say why it did not land); `skipped` (`reason`: durability is off,@@ -508,13 +514,16 @@ export async function applyGitConversion(engine: BrainEngine, sourceId: string,report({ scanned: progress.scanned + 1 });}const processing = { noEmbed: opts.noEmbed, noExtract: opts.noExtract };- const { performManagedSync } = await import('./sync-run.ts');- report({ phase: 'strict_pass_1', scanned: 0, total: 0 });- passOrThrow(sourceId, await performManagedSync(engine, { sourceId, noPull: true, full: true, ...processing, canonicalOverride: 'git', onProgress: passProgress(report) }));+ const planned = recheck?.paths.length ?? 0;+ const pass = (phase: 'strict_pass_1' | 'strict_pass_2') => {+ report({ phase, scanned: 0, total: planned, round: undefined, candidates: undefined, batch: undefined, batches: undefined });+ return strictPass(engine, sourceId, { sourceId, noPull: true, fullRound: true, ...processing, ...(phase === 'strict_pass_1' ? { canonicalOverride: 'git' as const } : {}),+ onProgress: passProgress(report) });+ };+ passOrThrow(sourceId, 'strict_pass_1', await pass('strict_pass_1'), progress);await faultPoint('conversion:before_flip', { sourceId });await recordConversion(engine, sourceId, plan.incarnation, 'git', expect, plan.preview.head);- report({ phase: 'strict_pass_2', scanned: 0, total: 0, round: undefined });- passOrThrow(sourceId, await performManagedSync(engine, { sourceId, noPull: true, full: true, ...processing, onProgress: passProgress(report) }));+ passOrThrow(sourceId, 'strict_pass_2', await pass('strict_pass_2'), progress);const final = await pushConversion(engine, plan.context);const push = final.reason === 'nothing_to_push' ? progress.push! : final;report({ push });@@ -546,11 +555,38 @@ async function pushConversion(engine: BrainEngine, context: ManagedSyncContext):}}-/** A strict pass's progress events as the apply's: a round's level read sets the total, its batch the scanned entries. */+/**+ * #7866: a strict pass of the apply. It drains (sync-drain.ts) under the source's drain lease, as `gbrain sync` does: the+ * single pass (`performManagedSync`) stops `writer_pending` whenever one of its own writes outlives its 5 s wait, which on a+ * large source ended the conversion on work it had queued itself. The drain re-enters on that and stops only on its own+ * rules (no progress, contention, a blocked head), so an unrelated write that never settles still stops the apply.+ */+async function strictPass(engine: BrainEngine, sourceId: string, opts: SyncOpts): Promise<SyncResult> {+ const { withDrainLease, leaseHeldResult, readDrainHolder } = await import('./sync-lease.ts');+ const { drainManagedSync } = await import('./sync-drain.ts');+ const { withSyncTail } = await import('./sync-tail.ts');+ try {+ const drained = await withDrainLease(engine, sourceId, { holder: { kind: 'cli', id: null }, remote: false },+ signal => drainManagedSync(engine, signal ? { ...opts, signal } : opts, false));+ return await withSyncTail(engine, sourceId, drained, { signal: opts.signal });+ } catch (error) {+ if (!(error instanceof LockUnavailableError) || error.lockId !== syncLockId(sourceId)) throw error;+ return leaseHeldResult(sourceId, await readDrainHolder(engine, sourceId).catch(() => null));+ }+}++/**+ * A strict pass's progress events as the apply's (#7866): `total` stays the planned paths; a round names its candidates+ * and each import batch, and `scanned` counts the round's settled candidates across its batches.+ */function passProgress(report: ConversionReporter): NonNullable<SyncOpts['onProgress']> {+ let base = 0;return event => {- if (event.round) report({ round: event.round.round, scanned: 0, total: event.phase === 'managed_sync.round_admit' ? (event.round.imports ?? 0) + (event.round.deletes ?? 0) : event.round.paths });- else if (event.bankedFiles !== undefined && event.phase !== 'managed_sync.level_progress' && event.phase !== 'managed_sync.legacy_swept') report({ scanned: event.bankedFiles, ...(event.total !== undefined ? { total: event.total } : {}) });+ const round = event.round;+ if (round && event.phase === 'managed_sync.round_level') { base = 0; report({ round: round.round, scanned: 0, candidates: undefined, batch: undefined, batches: undefined }); }+ else if (round && event.phase === 'managed_sync.round_admit') report({ round: round.round, scanned: 0, candidates: (round.imports ?? 0) + (round.deletes ?? 0) });+ else if (round && event.phase === 'managed_sync.round_batch') { base = round.base ?? 0; report({ round: round.round, scanned: base, batch: round.batch, batches: round.batches }); }+ else if (event.bankedFiles !== undefined && event.phase !== 'managed_sync.level_progress' && event.phase !== 'managed_sync.legacy_swept') report({ scanned: base + event.bankedFiles });};}@@ -569,11 +605,19 @@ async function movedSincePlan(engine: BrainEngine, sourceId: string, compared: M}const PASS_DONE = new Set(['first_sync', 'synced', 'up_to_date']);-function passOrThrow(sourceId: string, result: { status: string; reason?: string }): void {- if (PASS_DONE.has(result.status)) return;- throw opError('git_conversion_incomplete', `The strict pass of source ${sourceId} ended ${result.status}${result.reason ? ` (${result.reason})` : ''}, so the conversion stopped there.`,- `Rerun gbrain sources canonical ${sourceId} --to git --dry-run once gbrain sync status --source ${sourceId} --json shows the source idle, then apply the new hash.`,- { docs: DOCS, detail: result.status });+/** The operator step every git_conversion_incomplete refusal carries (#7866): preview again, verified by the source's sync status. */+export function conversionIncompleteFix(sourceId: string): Action {+ return { argv: ['gbrain', 'sources', 'canonical', sourceId, '--to', 'git', '--dry-run', '--json'], consent: [], actor: 'agent', requires_exclusive: false,+ why: 'Plans the conversion again from where it stopped (the door groups already committed read equal); read-only. Apply its new hash once the source is idle.',+ verify: { argv: ['gbrain', 'sync', 'status', '--source', sourceId, '--json'] } };+}+function passOrThrow(sourceId: string, phase: ConversionPhase, result: SyncResult, at: ConversionProgress): void {+ if (PASS_DONE.has(result.status) && syncOutcome(result) === 'synced') return;+ const stop = result.drain?.stop_reason ?? result.reason;+ const written = result.added + result.modified + result.deleted + result.renamed;+ throw opError('git_conversion_incomplete', `The ${phase} of source ${sourceId} ended ${result.status}${stop ? ` (${stop})` : ''} after ${written} page write(s), at round ${at.round ?? 0}, ${at.scanned} of ${at.candidates ?? 0} candidate(s) settled of ${at.total} planned path(s); the conversion stopped there${phase === 'strict_pass_1' ? ' and the source is still mixed' : ' and the source stays git-canonical'}.`,+ `Run gbrain sync status --source ${sourceId} --json until the source is idle, then preview again with gbrain sources canonical ${sourceId} --to git --dry-run and apply the new hash.`,+ { docs: DOCS, reason: stop ?? result.status, detail: `${result.status}${stop ? `:${stop}` : ''}`, fix: conversionIncompleteFix(sourceId) });}/**diff --git a/src/core/persistence/noop-kernel.ts b/src/core/persistence/noop-kernel.tsindex f92eec7d61..fefb69b242 100644--- a/src/core/persistence/noop-kernel.ts+++ b/src/core/persistence/noop-kernel.ts@@ -34,6 +34,8 @@ import type { WriteRequest } from './model.ts';import type { PageSnapshot } from '../page-state/types.ts';import { belowSafeChunkFence, chunkGenerationFilter, chunkManifestReads } from '../search/safe-chunks.ts';import { fileContentHash } from './large-file.ts';+import { embeddingWriteTarget } from '../page-state/projections.ts';+import { quoteIdentifier } from '../search/embedding-column.ts';export interface NoopKernelResult {contentUnchanged: boolean;@@ -113,3 +115,17 @@ export async function inspectUnchanged(engine: Pick<BrainEngine, 'kind' | 'execu}return result(undefined, waived.length ? { waived } : {});}++/**+ * #7866: whether a content no-op leaves the page no embedding work: every chunk of its current generation has a vector in+ * the active column for its exact text. Such a no-op queues no embedding effect (a re-sync, a conversion's strict pass);+ * a page with a chunk never embedded, or embedded for other text, still gets one, as before. A model change keeps+ * the column's vectors until `gbrain embed --stale`, as for any page that was not written.+ */+export async function chunksEmbeddedAsIs(engine: Pick<BrainEngine, 'kind' | 'executeRaw'>, pageId: number): Promise<boolean> {+ const { column } = await embeddingWriteTarget(engine);+ const [row] = await engine.executeRaw<{ owed: boolean }>(`SELECT EXISTS (SELECT 1 FROM content_chunks c JOIN pages p ON p.id=c.page_id+ WHERE p.id=$1 AND ${chunkGenerationFilter('p', 'c', await chunkManifestReads(engine))}+ AND (c.${quoteIdentifier(column.name)} IS NULL OR c.embedded_at IS NULL OR c.embedded_text_hash IS DISTINCT FROM md5(c.chunk_text))) AS owed`, [pageId]);+ return row?.owed === false;+}diff --git a/src/core/persistence/sync-classify.ts b/src/core/persistence/sync-classify.tsindex 4dbbffacbf..f074b2ce3f 100644--- a/src/core/persistence/sync-classify.ts+++ b/src/core/persistence/sync-classify.ts@@ -272,6 +272,8 @@ export interface SyncPassKindInput {rescreenDueHold: boolean;/** `--full`. */full: boolean;+ /** #7866: a full round asked for without an episode (SyncOpts.fullRound). */+ fullRound?: boolean;}/** Full rounds run at least this often (v3.2 §2.2, [estimated]). */@@ -286,7 +288,7 @@ export function syncPassKind(input: SyncPassKindInput): { kind: 'idle' | 'hold_oelse {if (c.target !== input.head) blockers.push('target_moved');if (c.parserFp !== input.parserFp || c.pack !== input.pack || c.selectionFp !== input.selectionFp) blockers.push('fingerprint_changed');- if (c.passesSinceFull >= SYNC_FULL_ROUND_MAX_PASSES || input.now - c.lastFullAt >= SYNC_FULL_ROUND_MAX_MS) blockers.push('full_round_due');+ if (input.fullRound || c.passesSinceFull >= SYNC_FULL_ROUND_MAX_PASSES || input.now - c.lastFullAt >= SYNC_FULL_ROUND_MAX_MS) blockers.push('full_round_due');}if (input.full || input.openEpisode) blockers.push('open_episode');if (input.owedTail) blockers.push('owed_tail');diff --git a/src/core/persistence/sync-pass-run.ts b/src/core/persistence/sync-pass-run.tsindex 9ed708e77a..1bbfcd8d3a 100644--- a/src/core/persistence/sync-pass-run.ts+++ b/src/core/persistence/sync-pass-run.ts@@ -437,7 +437,7 @@ async function pinPass(engine: BrainEngine, input: SyncPassInput, lease: NonNullhead, parserFp, pack, selectionFp, now: started, openEpisode: !!episode, owedTail: tail !== null, retryRow: retry.length > 0,notConvergedStreak: obligations.not_converged_streak, mismatchStreak: Object.keys(obligations.mismatch).length,requestNewer: probe?.newer === true, openAtStampFinished: !!converged && (converged.open_at_stamp === null || (probe?.stillOpen ?? 0) < converged.open_at_stamp.length),- writebackOpen: probe?.writebackOpen === true, workingChange: working.changed, rescreenDueHold: holds.some(hold => holdRescreenDue(hold, retry.includes(hold.path))), full: opts.full === true });+ writebackOpen: probe?.writebackOpen === true, workingChange: working.changed, rescreenDueHold: holds.some(hold => holdRescreenDue(hold, retry.includes(hold.path))), full: opts.full === true, fullRound: opts.fullRound === true });// Design §2.6: a withheld mass delete is applied by a full round once the owner host's env valve allows it.const valveOpen = !remote && !!obligations.deletes_withheld && massReconcileAllowed();// Design §4: keep_db writes the previous pass deferred (`keep_db_pending`: its cap, or write capacity) are owed to this pass, so it is@@ -964,7 +964,7 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): Prconst fold = passFold(level, levelCache, { imports, entries: built.entries, unsettled: deletes.length + inFlight.length + owed.length + mismatch.length + legacyWaived.size + retry.length,remote, episodeFullId: episode?.full_id ?? null });for (let i = 0; i < built.entries.length; i += PASS_BATCH_MAX) {- const chunk = built.entries.slice(i, i + PASS_BATCH_MAX), last = !!fold && i + chunk.length >= built.entries.length;+ const chunk = built.entries.slice(i, i + PASS_BATCH_MAX), last = !!fold && i + chunk.length >= built.entries.length; opts.onProgress?.({ phase: 'managed_sync.round_batch', round: { round, target: head, paths: level.paths.length, batch: i / PASS_BATCH_MAX + 1, batches: Math.ceil(built.entries.length / PASS_BATCH_MAX), base: i } });// A folded stamp writes the pass row with the obligations as they stand when its import is frozen.const stopped = await drain(chunk, last ? { ...await stampFor(level), fold: { ...fold, obligations: structuredClone(value.obligations), batch: chunk.length } } : undefined);if (stopped) return stop(stopped);diff --git a/src/core/persistence/sync-prepare.ts b/src/core/persistence/sync-prepare.tsindex 52b34e4b60..d2550731d4 100644--- a/src/core/persistence/sync-prepare.ts+++ b/src/core/persistence/sync-prepare.ts@@ -2,6 +2,7 @@ import { existsSync, lstatSync, realpathSync } from 'node:fs';import { sizeStateField } from '../page-embedding-budget.ts';import { enterClaimStep, type ClaimPhaseClock } from './claim-phase.ts';import { boundedReads } from './bounded-reads.ts';+import { chunksEmbeddedAsIs } from './noop-kernel.ts';import { basename, dirname, join } from 'node:path';import type { BrainEngine } from '../engine.ts';import type { GBrainConfig } from '../config.ts';@@ -912,7 +913,7 @@ async function prepareSyncMutation(unbounded: BrainEngine, row: WriteRequest, _cpageUnchanged: ready.noop && !moved,...(renamed ? { additionalPageKeys: [{ sourceId: row.source_id, slug: renamed.slug }] } : {}),validate: async tx => { await validate(tx); await ready.validate(tx); },- deferEmbedding: p.processingOptions?.noEmbed || p.embedDeferred === true,+ deferEmbedding: p.processingOptions?.noEmbed || p.embedDeferred === true, ...(!(p.processingOptions?.noEmbed || p.embedDeferred === true) && ready.noop && !moved && snapshot && await chunksEmbeddedAsIs(engine, snapshot.page.id) ? { embeddingCurrent: true } : {}),...(ready.afterCommit || renamed ? { afterCommit: async (target: BrainEngine) => { await published.afterCommit?.(target); } } : {}),...(writeback ? { file: { root, path: join(root, p.path), content: canonicalFile, expectedBeforeHash: p.rawHash } } : {}),...(mirrorReadOnly ? { databaseOnlyReason: 'mirror_read_only' as const } : {}),diff --git a/test/git-canonical-conversion-progress.test.ts b/test/git-canonical-conversion-progress.test.tsindex d586356d90..0c35d4e08c 100644--- a/test/git-canonical-conversion-progress.test.ts+++ b/test/git-canonical-conversion-progress.test.ts@@ -64,13 +64,14 @@ for (const backend of testBackends()) describe('[' + backend + '] #7644: a convefor (const phase of ['strict_pass_1', 'strict_pass_2'] as const) {const pass = seen.filter(progress => progress.phase === phase);expect(pass.every(progress => progress.groups_committed === 2 && progress.scanned <= progress.total)).toBe(true);- // A round's level read sets the total.+ // #7866: the total is the planned paths, the same on every event of the pass.expect(pass.some(progress => progress.round === 1 && progress.total >= BULK)).toBe(true);+ expect(new Set(pass.map(progress => progress.total)).size).toBe(1);}- // The first pass settles its batch entries up to the total; the second admits every page, now current, in place, so its round has nothing to settle.- expect(seen.some(progress => progress.phase === 'strict_pass_1' && progress.round === 1 && progress.scanned >= BULK)).toBe(true);- expect(seen.filter(progress => progress.phase === 'strict_pass_2').every(progress => progress.scanned === 0)).toBe(true);- expect(seen.some(progress => progress.phase === 'strict_pass_2' && progress.round === 1 && progress.total === 0)).toBe(true);+ // The first pass settles its candidates, every adopted page, in one batch; the second admits every page, now current, in place, so its round has no candidate.+ expect(seen.some(progress => progress.phase === 'strict_pass_1' && progress.round === 1 && progress.candidates === BULK && progress.batch === 1 && progress.batches === 1 && progress.scanned >= BULK)).toBe(true);+ expect(seen.filter(progress => progress.phase === 'strict_pass_2').every(progress => progress.scanned === 0 && (progress.candidates ?? 0) === 0)).toBe(true);+ expect(seen.some(progress => progress.phase === 'strict_pass_2' && progress.round === 1)).toBe(true);}, { databaseUrl, setup: bulkSetup }), 300_000);test('the CLI prints a progress line on stderr at each phase, and a JSON line with --json', () => managedBrain(async ({ engine }) => {diff --git a/test/git-canonical-conversion-strict-pass-7866.test.ts b/test/git-canonical-conversion-strict-pass-7866.test.tsnew file mode 100644index 0000000000..aa15e9d5fc--- /dev/null+++ b/test/git-canonical-conversion-strict-pass-7866.test.ts@@ -0,0 +1,143 @@+/**+ * #7866: a conversion's strict passes on a production-shaped source. On master a no-change `sources canonical --to git`+ * re-imported every page its `--full` episode found not derived-current (pages written before contextual retrieval had+ * no mode stamp), queued an embedding effect for each, did it again in the second pass, and stopped+ * `git_conversion_incomplete` whenever one of its own writes outlived the single pass's 5 s wait. These tests pin the+ * fixed behaviour: a full round without an episode, no embedding effect for a no-op whose chunks hold their vectors, a+ * drained pass, progress whose total is the planned paths, and refusals that carry the operator contract.+ */+import { describe, expect, test } from 'bun:test';+import { execFileSync } from 'node:child_process';+import { mkdirSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import type { SyncResult } from '../src/commands/sync.ts';+import { performManagedSync } from '../src/core/persistence/sync-run.ts';+import { withDrainLease } from '../src/core/persistence/sync-lease.ts';+import { installFaultHook } from '../src/core/persistence/fault-points.ts';+import { applyGitConversion, planGitConversion, type ConversionProgress } from '../src/core/persistence/git-conversion.ts';+import { runSourcesCanonical } from '../src/commands/sources-canonical.ts';+import { _resetCliExitVerdictForTests } from '../src/core/cli-force-exit.ts';+import { managedBrain } from './helpers/managed-brain.ts';+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';++const N = 30;+const git = (root: string, ...args: string[]) => execFileSync('git', ['-C', root, '-c', 'commit.gpgsign=false', ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();+const note = (i: number, body = `Committed body ${i}.`) => `---\ntype: concept\ntitle: note ${i}\n---\n\n${body}\n`;+const setup = ({ root }: { root: string }) => {+ mkdirSync(join(root, 'notes'), { recursive: true }); git(root, 'init', '-q');+ git(root, 'config', 'user.name', 'Example Writer'); git(root, 'config', 'user.email', '[email protected]');+ for (let i = 0; i < N; i++) writeFileSync(join(root, `notes/n${String(i).padStart(2, '0')}.md`), note(i));+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'notes');+};+/** Pages as an older gbrain left them: every chunk embedded for its text, and no contextual-mode stamp. */+async function legacyEmbedded(engine: BrainEngine): Promise<void> {+ const [col] = await engine.executeRaw<{ t: string }>("SELECT format_type(atttypid, atttypmod) AS t FROM pg_attribute WHERE attrelid='content_chunks'::regclass AND attname='embedding'");+ const dims = Number(col!.t.match(/\d+/)![0]);+ await engine.executeRaw(`UPDATE content_chunks SET embedding=('[' || array_to_string(array_fill(0.1::real, ARRAY[${dims}]), ',') || ']')::vector, embedded_at=now(), embedded_text_hash=md5(chunk_text)`);+ await engine.executeRaw('UPDATE pages SET contextual_retrieval_mode=NULL');+}+const counts = async (engine: BrainEngine) => {+ const [row] = await engine.executeRaw<{ embedding: number; imports: number }>(`SELECT (SELECT count(*)::int FROM persistence_effects WHERE kind='embedding') AS embedding,+ (SELECT count(*)::int FROM persistence_requests WHERE intent->>'kind'='managed_sync_import') AS imports`);+ return row!;+};+async function firstSync(engine: BrainEngine): Promise<void> {+ expect((await performManagedSync(engine, { sourceId: 'default', noPull: true, noExtract: true })).status).toBe('first_sync');+}+async function convert(engine: BrainEngine, seen: ConversionProgress[] = []) {+ const { preview } = await planGitConversion(engine, 'default');+ expect(preview.discard.count).toBe(0);+ return applyGitConversion(engine, 'default', preview.preview_hash, { noExtract: true, onProgress: progress => seen.push(progress) });+}+const strictEvents = (seen: ConversionProgress[]) => seen.filter(progress => progress.phase.startsWith('strict_pass'));++for (const backend of testBackends()) describe(`[${backend}] #7866: the conversion's strict passes`, () => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;++ test('a no-change conversion of pages an older gbrain embedded makes no embedding effect and no write', () => managedBrain(async ({ engine }) => {+ await firstSync(engine);+ await legacyEmbedded(engine);+ const before = await counts(engine);+ const seen: ConversionProgress[] = [];+ expect(await convert(engine, seen)).toMatchObject({ status: 'converted' });+ expect(await counts(engine)).toEqual(before);+ // Progress: the total is the planned paths in both passes and never changes within them.+ expect(strictEvents(seen).length).toBeGreaterThan(0);+ expect(strictEvents(seen).every(progress => progress.total === N)).toBe(true);+ expect(await engine.executeRaw("SELECT 1 FROM sources WHERE id='default' AND config->>'canonical'='git'")).toHaveLength(1);+ }, { databaseUrl, setup }), 300_000);++ test('a no-op re-import queues no embedding effect when every chunk holds a vector for its text, and still queues one for a page whose vector is for other text', () => managedBrain(async ({ engine }) => {+ await firstSync(engine);+ await legacyEmbedded(engine);+ // A parser this binary no longer records owes every page a re-import; the re-import changes no content.+ await engine.executeRaw(`UPDATE op_checkpoints SET completed_keys=jsonb_set(completed_keys, '{0,parser}', '"parser-of-an-older-release"'::jsonb) WHERE completed_keys->0 ? 'parser'`);+ const [unembedded] = await engine.executeRaw<{ id: number }>("SELECT id FROM pages WHERE slug='notes/n07'");+ await engine.executeRaw("UPDATE content_chunks SET embedded_text_hash=md5('text an earlier revision held') WHERE page_id=$1", [unembedded!.id]);+ const before = await counts(engine);+ const [{ last }] = await engine.executeRaw<{ last: string }>('SELECT COALESCE(max(id), 0)::text AS last FROM persistence_effects') as [{ last: string }];+ expect(await convert(engine)).toMatchObject({ status: 'converted' });+ const after = await counts(engine);+ expect(after.imports - before.imports).toBe(N);+ const queued = await engine.executeRaw<{ page_id: number }>("SELECT (data->>'page_id')::int AS page_id FROM persistence_effects WHERE kind='embedding' AND id > $1::bigint", [last]);+ expect(queued.map(row => Number(row.page_id))).toEqual([Number(unembedded!.id)]);+ }, { databaseUrl, setup }), 300_000);++ test('a strict pass whose own write outlives the single pass\'s receipt wait drains on instead of refusing writer_pending', () => managedBrain(async ({ engine, root }) => {+ await firstSync(engine);+ writeFileSync(join(root, 'notes/n03.md'), note(3, 'Edited in Git after the last sync.'));+ git(root, 'commit', '-qam', 'edit n03');+ const seen: ConversionProgress[] = [];+ let stalled = false;+ // Once the strict pass froze its import, the consumer stops claiming for 33 s: the admitted request waits queued past+ // the single pass's receipt wait (5 s, held to 30 s while queued here), as it does behind a busy owner on a production host.+ let armed = false;+ installFaultHook(async point => {+ if (point === 'sync:after_freeze' && seen.at(-1)?.phase === 'strict_pass_1') armed = true;+ if (point !== 'consumer:claiming' || !armed || stalled) return;+ stalled = true;+ await Bun.sleep(33_000);+ });+ try {+ expect(await convert(engine, seen)).toMatchObject({ status: 'converted' });+ } finally { installFaultHook(undefined); }+ expect(stalled).toBe(true);+ const [page] = await engine.executeRaw<{ compiled_truth: string }>("SELECT compiled_truth FROM pages WHERE slug='notes/n03'");+ expect(page!.compiled_truth).toContain('Edited in Git after the last sync.');+ // The round names its candidate and its one import batch; the total stays the planned paths.+ const pass1 = seen.filter(progress => progress.phase === 'strict_pass_1');+ expect(pass1.every(progress => progress.total === N)).toBe(true);+ expect(pass1.some(progress => progress.round === 1 && progress.candidates === 1 && progress.batch === 1 && progress.batches === 1)).toBe(true);+ expect(pass1.every(progress => progress.scanned <= (progress.candidates ?? 0))).toBe(true);+ }, { databaseUrl, setup }), 300_000);++ test('a concurrent drainer of the source refuses the apply with the operator contract and counts, before anything is flipped', () => managedBrain(async ({ engine, root }) => {+ await firstSync(engine);+ writeFileSync(join(root, 'notes/n05.md'), note(5, 'Edited in Git after the last sync.'));+ git(root, 'commit', '-qam', 'edit n05');+ const { preview } = await planGitConversion(engine, 'default');+ let release!: () => void, acquired!: () => void;+ const holding = new Promise<void>(resolve => { release = resolve; }), held = new Promise<void>(resolve => { acquired = resolve; });+ const holder = withDrainLease(engine, 'default', { holder: { kind: 'cli', id: null }, remote: false }, async () => { acquired(); await holding; return { status: 'up_to_date' } as SyncResult; });+ await held;+ const out: string[] = [];+ const log = console.log, write = process.stderr.write, priorExitCode = process.exitCode;+ console.log = (line: unknown) => { out.push(String(line)); };+ process.stderr.write = (() => true) as typeof process.stderr.write;+ try { await runSourcesCanonical(engine, ['default', '--to', 'git', '--apply', '--expect', preview.preview_hash, '--no-extract', '--json']); }+ finally { console.log = log; process.stderr.write = write; _resetCliExitVerdictForTests(); process.exitCode = priorExitCode ?? 0; release(); await holder; }+ const refused = JSON.parse(out.at(-1)!);+ expect(refused).toMatchObject({ status: 'refused', code: 'git_conversion_incomplete',+ fix: { next: 'run', argv: ['gbrain', 'sources', 'canonical', 'default', '--to', 'git', '--dry-run', '--json'], verify: { argv: ['gbrain', 'sync', 'status', '--source', 'default', '--json'] } },+ progress: { phase: 'strict_pass_1', total: N } });+ expect(refused.message).toContain('still mixed');+ // Nothing flipped, and the page Git moved still has the database's version for the next apply to take.+ expect(await engine.executeRaw("SELECT 1 FROM sources WHERE id='default' AND config->>'canonical'='git'")).toHaveLength(0);+ const [page] = await engine.executeRaw<{ compiled_truth: string }>("SELECT compiled_truth FROM pages WHERE slug='notes/n05'");+ expect(page!.compiled_truth).toContain('Committed body 5.');+ // The same apply runs once the drainer is gone.+ const { preview: again } = await planGitConversion(engine, 'default');+ expect(await applyGitConversion(engine, 'default', again.preview_hash, { noExtract: true })).toMatchObject({ status: 'converted' });+ }, { databaseUrl, setup }), 300_000);+});diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex 2658f7d8bc..3ddc103718 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -85,6 +85,7 @@ test/frontmatter-reserved-keys-6454.test.tstest/get-chunks-paging.test.tstest/git-canonical-conversion-g8a.test.tstest/git-canonical-conversion-progress.test.ts+test/git-canonical-conversion-strict-pass-7866.test.tstest/git-canonical-declared-g7a.test.tstest/git-canonical-indexer-g3.test.tstest/git-door-index-lock-7742.test.ts