Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Large read-only mirror sources stop parking as abandoned while their passes re-check old holds (GST-2843) (#9509)
Garry Tan36bd05806a3f
diff --git a/behavior-changes.d/held_compare_complete_window_gst2843.md b/behavior-changes.d/held_compare_complete_window_gst2843.mdnew file mode 100644index 0000000000..c6c0b0b0d6--- /dev/null+++ b/behavior-changes.d/held_compare_complete_window_gst2843.md@@ -0,0 +1,4 @@+---+kind: fix+---+A gbrain repair file-drift --held dry run (a comparison, or a --prefer preview) now reports complete: false while held pages remain past its window, and details.window.next_command names the command for the next window. Before, complete was true after every window, even with thousands of held pages left. Applies keep the old meaning: every item of the approved window was attempted.diff --git a/behavior-changes.d/hold_by_code_counts_all_gst2843.md b/behavior-changes.d/hold_by_code_counts_all_gst2843.mdnew file mode 100644index 0000000000..38417102ea--- /dev/null+++ b/behavior-changes.d/hold_by_code_counts_all_gst2843.md@@ -0,0 +1,4 @@+---+kind: fix+---+gbrain sources status --json now counts git_holds.by_code over every hold of the source. Before, the counts came only from the held files the listing read (at most 500), so a source with more holds under-counted each code. A code with no held file in the listing has no example. If the per-code count cannot be read, by_code falls back to the listed files and by_code_complete is false.diff --git a/behavior-changes.d/mirror_compare_reads_file_gst2843.md b/behavior-changes.d/mirror_compare_reads_file_gst2843.mdnew file mode 100644index 0000000000..2240fd3326--- /dev/null+++ b/behavior-changes.d/mirror_compare_reads_file_gst2843.md@@ -0,0 +1,4 @@+---+kind: fix+---+On a read-only mirror source, gbrain repair file-drift --held without --prefer now compares each held page with its file. A page that differs from its file reads prefer_file or needs_human with its evidence (what differs and which side is newer), and one that matches reads release_hold. Only a page whose file is gone reads no_drift with database_only. Before, every held page on a mirror read no_drift with database_only, whether or not it matched its file. Every compared page in details.pages now carries an evidence field, null when nothing was read. The comparison still writes nothing, and sync holds the same pages as before.diff --git a/changelog.d/gbra-568-mirror-parked.md b/changelog.d/gbra-568-mirror-parked.mdnew file mode 100644index 0000000000..136aefdebc--- /dev/null+++ b/changelog.d/gbra-568-mirror-parked.md@@ -0,0 +1,15 @@+**A large read-only mirror source whose sync passes only re-check old holds no longer parks as `pass_abandoned` with nothing settled, and its status and repair output now say what is actually held.**++### Fixed+- A sync pass that stopped while it re-checked old holds now counts as progress when it cleared a hold's re-check trigger, so a few such passes no longer build the abandoned streak. Passes that re-check the same holds every time (slug conflicts) still end `pass_abandoned`, and so do passes that only re-hold a page edited again before each of them: once that hold has been held three times, `gbrain sync status` reports `needs_human: true` with that page's own decision.+- A pass now reads the earlier Git versions of the held pages it re-checks one batch at a time, just before that batch commits, instead of for the whole round before its first commit. On a source with many such holds the first commit came only after the whole history read.+- `gbrain sync status` leads with the pass's own next step when the holds are not what stops the source, and `next.why` for `pass_interrupted` and `pass_abandoned` names the step the stopped pass was last seen in (the level read, or the history read with how far it got).+- On a read-only mirror, `gbrain repair file-drift --held` compares each held page with its file. A differing page reads `prefer_file` or `needs_human` with its evidence, a matching one `release_hold`, and only a page whose file is gone reads `no_drift` with `database_only`. The comparison still writes nothing.+- A windowed held dry run reports `complete: false` while held pages remain past its window, and `details.window.next_command` names the next window.+- `gbrain sources status --json` counts `git_holds.by_code` over every hold of the source, not only the listed ones.++Larger sources still need a per-class decision flow for their held pages; this change only makes the counts and the comparison true.++<!-- behavior-change: mirror_compare_reads_file_gst2843 -->+<!-- behavior-change: held_compare_complete_window_gst2843 -->+<!-- behavior-change: hold_by_code_counts_all_gst2843 -->diff --git a/src/core/persistence/connector-status.ts b/src/core/persistence/connector-status.tsindex f76306cf8f..7de159b54a 100644--- a/src/core/persistence/connector-status.ts+++ b/src/core/persistence/connector-status.ts@@ -18,7 +18,7 @@ import { classifySyncFault, type SyncFaultVerdict } from './sync-fault-class.ts'import { cliRenderContext, renderAction, type Action } from '../agent-output.ts';import { managedBrain, readAllSourceHolds } from '../connectors/item-holds-store.ts';import type { ItemHoldRecord } from '../connectors/item-holds.ts';-import { fenceAutoRepairFor, gitHoldItem, readGitHoldListing, readSyncConversions, readSyncHoldPolicy, type GitHoldItem, type SyncConversion } from './sync-holds.ts';+import { fenceAutoRepairFor, gitHoldItem, readGitHoldCodeCounts, readGitHoldListing, readSyncConversions, readSyncHoldPolicy, type GitHoldItem, type SyncConversion } from './sync-holds.ts';import { holdLine } from '../../commands/sync-diagnostics.ts';import { pageSortedRows, type ListingRequest } from '../bounded-listing.ts';@@ -150,6 +150,8 @@ export interface GitHoldStatus {count: number;items: GitHoldItem[];truncated?: true;+ /** GST-2843: holds per code over every hold of a truncated listing; absent when the listing is complete (its items count) or the count read failed. */+ code_counts?: Record<string, number>;/** Blocked sync requests a recent run converted in place (held or re-frozen) instead of staying blocked. */recent_conversions?: SyncConversion[];/** The command that runs the next sync of this source (managed brains sync with --no-pull). */@@ -169,11 +171,14 @@ export async function readGitHoldStatuses(engine: Pick<BrainEngine, 'executeRaw'const managed = await managedBrain(engine);const auto = await fenceAutoRepairFor(engine, sources.flatMap(source => source.holds));const counts = new Map(sources.map(source => [source.sourceId, source]));+ const truncated = sources.filter(source => source.count > source.holds.length).map(source => source.sourceId);+ const codeCounts = truncated.length ? await readGitHoldCodeCounts(engine, truncated).catch(() => null) : null;for (const sourceId of sourceIds) {const source = counts.get(sourceId), converted = conversions.get(sourceId);if (!source && !converted) continue;const items = (source?.holds ?? []).map(hold => gitHoldItem(hold, auto));- out.set(sourceId, { count: source?.count ?? 0, items, ...((source?.count ?? 0) > items.length ? { truncated: true as const } : {}),+ const perCode = codeCounts?.get(sourceId);+ out.set(sourceId, { count: source?.count ?? 0, items, ...((source?.count ?? 0) > items.length ? { truncated: true as const, ...(perCode ? { code_counts: perCode } : {}) } : {}),...(converted ? { recent_conversions: converted } : {}),sync_argv: ['gbrain', 'sync', '--source', sourceId, ...(managed ? ['--no-pull'] : [])] });}@@ -187,23 +192,28 @@ export async function readGitHoldStatuses(engine: Pick<BrainEngine, 'executeRaw'*/export type GitHoldListing = { mode: 'summary' } | { mode: 'all' } | { mode: 'page'; request: ListingRequest };-/** Held items per code, largest first, each with its first item as the example (its fix and docs apply to the rest). */-function holdsByCode<T extends { code: string }>(items: readonly T[]): Array<{ code: string; count: number; example: T }> {- const groups = new Map<string, { code: string; count: number; example: T }>();+/**+ * Held items per code, largest first, each with its first item as the example (its fix and docs apply to the rest). GST-2843:+ * `counts` (every hold per code) replaces the listed items' counts, so a truncated listing still counts each code in full; a code+ * none of the listed items has carries no example.+ */+function holdsByCode<T extends { code: string }>(items: readonly T[], counts?: Record<string, number>): Array<{ code: string; count: number; example?: T }> {+ const groups = new Map<string, { code: string; count: number; example?: T }>();for (const item of items) {const group = groups.get(item.code);if (group) group.count++;else groups.set(item.code, { code: item.code, count: 1, example: item });}+ if (counts) for (const [code, count] of Object.entries(counts)) groups.set(code, { ...groups.get(code), code, count });return [...groups.values()].sort((a, b) => b.count - a.count || (a.code < b.code ? -1 : 1));}/** A source's `git_holds` for `sources status --json`, summary first and items last, bounded by `listing`; each fix rendered with the CLI routing pin (#9319). */export function gitHoldStatusJson(sourceId: string, status: GitHoldStatus, listing: GitHoldListing) {const ctx = cliRenderContext();- const { items: held, ...rest } = status;+ const { items: held, code_counts: codeCounts, ...rest } = status;const items = held.map(item => ({ ...item, fix: renderAction(item.fix, ctx) }));- const head = { ...rest, by_code: holdsByCode(items) };+ const head = { ...rest, by_code: holdsByCode(items, codeCounts), ...(rest.truncated && !codeCounts ? { by_code_complete: false } : {}) };if (listing.mode === 'all') return { ...head, items };if (listing.mode === 'page') {const page = pageSortedRows(items, item => [item.path], listing.request, `sources_status_git_holds:${sourceId}`);diff --git a/src/core/persistence/sync-classify.ts b/src/core/persistence/sync-classify.tsindex f4442649c1..9339a7a28b 100644--- a/src/core/persistence/sync-classify.ts+++ b/src/core/persistence/sync-classify.ts@@ -63,8 +63,9 @@ export interface SyncPathState {* An active hold at `p`: whether it is at level `L` in its own domain, re-screen due, or in the retry set. #7199 `unscreened`:* a side-`unknown` drift hold an older gbrain wrote before the drift screen compared the page with the file at the baseline commit.* GST-2201 `deletionSide`: a `file_drift` hold on the deletion side (`tombstone_edited`, `purged_differs`, `file_gone`).+ * GST-2843 `moved`: the hold's basis or drift moved since it was written (`holdBasisMoved`, `movedDriftHolds`).*/- hold: { atLevel: boolean; rescreenDue: boolean; retried: boolean; unscreened?: boolean; deletionSide?: boolean } | null;+ hold: { atLevel: boolean; rescreenDue: boolean; retried: boolean; unscreened?: boolean; deletionSide?: boolean; moved?: boolean } | null;/*** A deletion marker at the origin: `deleted` (with the blob ids it held) or `legacy` (the baseline blob of a page deleted* before PR 4; `deletion` is the committed deleting request that proves it, and a legacy marker without one settles nothing).diff --git a/src/core/persistence/sync-discovery.ts b/src/core/persistence/sync-discovery.tsindex 3d406bbe02..da6a3c74f3 100644--- a/src/core/persistence/sync-discovery.ts+++ b/src/core/persistence/sync-discovery.ts@@ -62,6 +62,8 @@ export interface SyncEntry { path: string; sourcePath: string; action: 'import'driftBaseline?: string;/** #7199: for an older side-`unknown` hold, the file's earlier committed blobs; the freeze imports when the page still holds what one imports. */driftHistory?: string[];+ /** GST-2843: the pass re-checks this held path because the hold's basis or drift moved; a hold the entry gets is marked `basis_moved`. */+ basisMoved?: true;/** #6727: live pages with no recorded origin whose slugs fold to this file's slug, several or one the importer cannot write; the freeze holds it `legacy_slug_conflict`. */legacySlugs?: string[];/** #6729 x #6727: the one legacy page carries an explicit frontmatter id that differs from the file's, so it is another document; held `legacy_slug_conflict` (`explicit_id_conflict`). */diff --git a/src/core/persistence/sync-holds.ts b/src/core/persistence/sync-holds.tsindex 3ceb00b29d..e13c1db92d 100644--- a/src/core/persistence/sync-holds.ts+++ b/src/core/persistence/sync-holds.ts@@ -152,6 +152,8 @@ export interface GitHoldMeta {competing_request_id?: string;competing_operation?: string;page_revision?: string;+ /** GST-2843: the pass rewrote this hold because its basis or drift moved, not because a re-screen came due; a later rewrite that is not a move drops it. */+ basis_moved?: true;/** #6712 PR 1 `file_drift`: which side moved since the page's last recorded import (`unknown`: no usable record); `page_revision` is the page's revision when held. */sides?: 'database' | 'both' | 'unknown';/** GBRA-133 `file_drift`: the drift screen that held it; an unknown-side hold from an older screen is re-screened (#7340). */@@ -600,6 +602,17 @@ export async function readGitHoldListing(engine: Exec, sourceIds: string[], limireturn [...out.values()];}+/** GST-2843: each listed source's holds counted per code, over every hold of its current incarnation (not a listing's first rows). */+export async function readGitHoldCodeCounts(engine: Exec, sourceIds: string[]): Promise<Map<string, Record<string, number>>> {+ const out = new Map<string, Record<string, number>>();+ if (!sourceIds.length) return out;+ const rows = await engine.executeRaw<{ source_id: string; code: string; n: number | string }>(`SELECT s.id AS source_id, x.completed_keys->0->>'code' AS code, count(*) AS n+ FROM sources s JOIN op_checkpoints x ON x.op=$1 AND x.completed_keys->0->>'source_id'=s.id AND x.completed_keys->0->>'incarnation'=s.incarnation::text+ WHERE s.id=ANY($2::text[]) GROUP BY 1,2`, [GIT_HOLD_OP, sourceIds]);+ for (const row of rows) out.set(row.source_id, { ...out.get(row.source_id), [row.code]: Number(row.n) });+ return out;+}+/*** #6712: the held paths whose verdict rests on something that has since moved: a dependency-edge hold (`blocked_on`)* whose predecessor's hold is gone, and a deterministic `concurrent_write` hold whose page's revision moved (a@@ -649,8 +662,11 @@ export async function holdBasisMoved(engine: Pick<BrainEngine, 'executeRaw'>, so*/export const FILE_DRIFT_VERSION = 3;+/** Hold codes every sync re-screens, whatever else changed (holdRescreenDue). GST-2843: re-holding one is never progress (sync-pass-status.ts passBanked). */+export const EVERY_PASS_RESCREEN_CODES: readonly string[] = ['frontmatter_slug_conflict', 'rename_held', 'parser_regression', 'content_rejected', 'write_gate_rejected', 'purged_content', 'legacy_slug_conflict', 'origin_conflict', 'page_identity_changed', 'git_object_missing'];+export function holdRescreenDue(record: Pick<GitHoldRecord, 'code' | 'meta'>, retried: boolean): boolean {- return retried || (record.code === 'file_too_large' && record.meta.streamed?.gbrain_version !== VERSION) || ['frontmatter_slug_conflict', 'rename_held', 'parser_regression', 'content_rejected', 'write_gate_rejected', 'purged_content', 'legacy_slug_conflict', 'origin_conflict', 'page_identity_changed', 'git_object_missing'].includes(record.code)+ return retried || (record.code === 'file_too_large' && record.meta.streamed?.gbrain_version !== VERSION) || EVERY_PASS_RESCREEN_CODES.includes(record.code)// #6712: a transient hold (the write owner stalled, not the file) is retried by every sync until it has been held HOLD_ATTEMPTS_NEEDS_HUMAN times.|| (outOfAttemptsHold(record.code) && (record.meta.attempts ?? 1) < HOLD_ATTEMPTS_NEEDS_HUMAN)|| (record.code === 'managed_image_sync_unsupported' && isMultimodalEnabled())diff --git a/src/core/persistence/sync-pass-apply.ts b/src/core/persistence/sync-pass-apply.tsindex e1978e654e..cd5cbe9936 100644--- a/src/core/persistence/sync-pass-apply.ts+++ b/src/core/persistence/sync-pass-apply.ts@@ -402,7 +402,7 @@ async function writeHolds(engine: BrainEngine, level: ShadowLevel, input: Pick<Pawait writeGitHold(tx, { source_id: level.sourceId, incarnation: level.incarnation, path, source_path: sourcePath, slug: hold.slug,page_id: hold.pageId !== undefined ? hold.pageId : hold.entry.state.page?.live ? hold.entry.pageId : null,code: hold.code, message: hold.message, upstream_version: hold.entry.state.levelSha256, observed_at: input.observedAt, run_id: input.passId, mode: 'managed',- meta: { recovery_version: RECOVERY_VERSION, ...levelMeta(level, hold.entry), ...hold.meta } });+ meta: { recovery_version: RECOVERY_VERSION, ...levelMeta(level, hold.entry), ...hold.meta, ...(hold.entry.state.hold?.moved ? { basis_moved: true as const } : {}) } });}, slice => {for (const hold of slice) {state.holds[hold.kind]++;diff --git a/src/core/persistence/sync-pass-row.ts b/src/core/persistence/sync-pass-row.tsindex 2de9907e46..e457550244 100644--- a/src/core/persistence/sync-pass-row.ts+++ b/src/core/persistence/sync-pass-row.ts@@ -123,6 +123,12 @@ export interface ManagedSyncPassValue {/** #6712 PR 4b (design §9): the newest round's classes (`{count, action}`) and level summary, for `sync status`'s live `convergence`. */classes?: Record<string, { count: number; action: 'none' | 'owner' | 'repair' }>;summary?: PassRoundSummary;+ /**+ * GST-2843: the long step before a batch drains that an open pass was last seen in (the level read, or the Git history+ * read of re-screened holds with how many of the batch's paths it read), written by the 30 s beat and cleared when the+ * step ends; status names it for a pass that stopped there.+ */+ step?: { name: 'level_read' | 'drift_history'; at: string; done?: number; total?: number };};}diff --git a/src/core/persistence/sync-pass-run.ts b/src/core/persistence/sync-pass-run.tsindex f66702f0ce..15c102b0d4 100644--- a/src/core/persistence/sync-pass-run.ts+++ b/src/core/persistence/sync-pass-run.ts@@ -147,6 +147,26 @@ function entryPaths(level: ShadowLevel, gitPath: string): { path: string; sourcereturn { path, sourcePath: level.slugMode === 'source-root' && level.scope ? path : gitPath };}+/**+ * #7199 / GST-2843: the earlier committed versions of each entry of `chunk` whose held page is re-screened against them+ * (`historyOwed`), read in one batched Git read for the batch that drains next and reported per path, so the read never+ * looks stalled and a pass over many such holds commits each batch before it reads the next one's history. While it reads,+ * the pass row's `telemetry.step` says how far it got (the beat saves it), so status can name the step a killed pass stopped in.+ */+async function readDriftHistories(level: ShadowLevel, chunk: PassEntry[], owed: ReadonlyMap<PassEntry, ShadowPath>, value: ManagedSyncPassValue, onHistoryRead: () => void): Promise<void> {+ const due = chunk.flatMap(entry => { const shadow = owed.get(entry); return shadow ? [[entry, shadow] as const] : []; });+ if (!due.length) return;+ let done = 0;+ const read = await pathBlobHistories(level.gitRoot, level.baselineCommit ?? level.head, due.map(([, shadow]) => shadow.path), DRIFT_HISTORY_DEPTH, () => {+ value.telemetry.step = { name: 'drift_history', at: new Date().toISOString(), done: ++done, total: due.length };+ onHistoryRead();+ }).finally(() => { delete value.telemetry.step; });+ for (const [entry, shadow] of due) {+ const versions = [...new Set(read.get(shadow.path) ?? [])].filter(oid => oid !== shadow.state.level && oid !== shadow.state.baseline).slice(0, DRIFT_HISTORY_VERSIONS);+ if (versions.length) entry.driftHistory = versions;+ }+}+/*** The round's candidates as batch entries, with the identities discovery resolves (`discoverManagedSync`): a live page* at the origin is the import's page; a new origin takes the slug its path names, and the page holding that slug is@@ -154,9 +174,9 @@ function entryPaths(level: ShadowLevel, gitPath: string): { path: string; source* stays; one new origin whose bytes equal a page leaving at another origin is that page's Git rename (`renameFrom`, the* page moves to the new slug). Deletes first, then imports.*/-async function batchEntries(engine: BrainEngine, level: ShadowLevel, deletes: ShadowPath[], imports: ShadowPath[], onHistoryRead?: (path: string) => void):+async function batchEntries(engine: BrainEngine, level: ShadowLevel, deletes: ShadowPath[], imports: ShadowPath[]):Promise<{ entries: PassEntry[]; collisions: SyncSlugCollision[]; skipped: ShadowPath[]; downgrades: DowngradedRename[];- identity: Array<{ entry: ShadowPath; origin: ShadowPath | null; slug: string; ids: { page: string; file: string } }> }> {+ identity: Array<{ entry: ShadowPath; origin: ShadowPath | null; slug: string; ids: { page: string; file: string } }>; historyOwed: Map<PassEntry, ShadowPath> }> {await faultPoint('sync:batch_entries', { sourceId: level.sourceId });const working = (entry: ShadowPath) => level.mode === 'working' && entry.state.level !== entry.state.tree;const committedOnly = level.strict ? { committedOnly: true as const } : {};@@ -209,18 +229,20 @@ async function batchEntries(engine: BrainEngine, level: ShadowLevel, deletes: Sh: entry.result.side === 'unknown' ? entry.state.baseline ?? undefined : undefined;// #7199: a side-`unknown` hold an older gbrain wrote after the pass that held it moved the source's last commit past the one// the classic sync read; its re-screen also compares the page with the file's last few committed versions before the held one.- // GST-2515: one batched read for all of them, reporting each path it finished, so thousands of such holds neither start a Git- // process each nor read as a stalled pass.- const unscreened = imports.filter(entry => entry.result.side === 'unknown' && entry.pageId !== null && entry.state.hold?.unscreened);- const read = unscreened.length ? await pathBlobHistories(level.gitRoot, level.baselineCommit ?? level.head, unscreened.map(entry => entry.path), DRIFT_HISTORY_DEPTH, onHistoryRead) : null;- const history = new Map(unscreened.map(entry => [entry.path, [...new Set(read?.get(entry.path) ?? [])].filter(oid => oid !== entry.state.level && oid !== entry.state.baseline).slice(0, DRIFT_HISTORY_VERSIONS)]));+ // GST-2843: those versions are read per batch, just before the batch drains (readDriftHistories), so a pass over tens of+ // thousands of such holds commits each batch's holds before it reads the next batch's history.+ const historyOwed = new Map<PassEntry, ShadowPath>();const entries: PassEntry[] = [], collisions: SyncSlugCollision[] = [], skipped: ShadowPath[] = [], identity: Array<{ entry: ShadowPath; origin: ShadowPath | null; slug: string; ids: { page: string; file: string } }> = [];const claims = new Map<string, ShadowPath[]>();for (const entry of imports) {if (step.due()) await step.pause();const paths = entryPaths(level, entry.path);- const base = { ...paths, ...committedOnly, action: 'import' as const, working: working(entry), level: entry.state.level, ...(entry.result.cls === 'drift' && entry.result.side && entry.result.side !== 'deleted' ? { drift: entry.result.side, ...(driftBaseline(entry) ? { driftBaseline: driftBaseline(entry) } : {}), ...(history.get(entry.path)?.length ? { driftHistory: history.get(entry.path) } : {}) } : {}) };- if (entry.state.page) { entries.push({ ...base, slug: entry.slug!, pageId: entry.pageId, revision: entry.revision }); continue; }+ const base = { ...paths, ...committedOnly, action: 'import' as const, working: working(entry), level: entry.state.level, ...(entry.state.hold?.moved ? { basisMoved: true as const } : {}), ...(entry.result.cls === 'drift' && entry.result.side && entry.result.side !== 'deleted' ? { drift: entry.result.side, ...(driftBaseline(entry) ? { driftBaseline: driftBaseline(entry) } : {}) } : {}) };+ if (entry.state.page) {+ const built: PassEntry = { ...base, slug: entry.slug!, pageId: entry.pageId, revision: entry.revision };+ if (entry.result.cls === 'drift' && entry.result.side === 'unknown' && entry.pageId !== null && entry.state.hold?.unscreened) historyOwed.set(built, entry);+ entries.push(built); continue;+ }const slug = slugOf.get(entry.path);if (!slug) { skipped.push(entry); continue; }claims.set(slug, [...claims.get(slug) ?? [], entry]);@@ -304,7 +326,7 @@ async function batchEntries(engine: BrainEngine, level: ShadowLevel, deletes: Shconst batch = [...deleteEntries.values(), ...entries.sort((a, b) => a.path.localeCompare(b.path))];// The drain freezes the batch's first entry (and reads its blob window) right after: it starts on a turn of its own too.await step.pause();- return { entries: batch, collisions, skipped, downgrades, identity };+ return { entries: batch, collisions, skipped, downgrades, identity, historyOwed };}/**@@ -791,7 +813,7 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): Pr}value.telemetry = { ...value.telemetry, outcome, stop_reason: stopReason, settled: S.size + (yieldSettled || roundSettled + inBatch), counts: { ...counts, ...passApplyCounts(applied) },classes: applied.classes, ...(applied.summary ? { summary: applied.summary } : {}),- ...(ended ? { ended_at: new Date().toISOString(), duration_ms: Date.now() - started } : {}) };+ ...(ended ? { ended_at: new Date().toISOString(), duration_ms: Date.now() - started, step: undefined } : {}) };if (outcome === 'blocked' || stopReason === 'not_converged') value.obligations.stuck_since ??= new Date().toISOString();await engine.transaction(async tx => { if (fenced) await assertDrainLease(tx, sourceId); await writePassRow(tx, value, { pin, onlyOpen }); });};@@ -892,8 +914,8 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): PrwriteIds: value.obligations.converged?.writeback ?? [] })) return synced(true);const levelInput = { sourceId, context, workingTree: mode === 'working',pass: { noSchemaPack: processingOptions.noSchemaPack, episodeFullId: episode?.full_id ?? null, retryConsumed: round > 1 ? retryConsumed : undefined, exclude: opts.exclude, includeHidden: opts.includeHidden, strategy: opts.strategy, bootstrap: markersOwed(),- slugMode: value.obligations.slug_mode, cache: levelCache, levelReuse, passId, recorded: applied.recorded, onProgress: beat.report, assertActive, ...(opts.conversionBinding ? { conversion: opts.conversionBinding } : {}) } };- const level = await readLevelReporting(engine, levelInput, round, opts.onProgress);+ slugMode: value.obligations.slug_mode, cache: levelCache, levelReuse, passId, recorded: applied.recorded, onProgress: () => { value.telemetry.step = { name: 'level_read', at: new Date().toISOString() }; beat.report(); }, assertActive, ...(opts.conversionBinding ? { conversion: opts.conversionBinding } : {}) } };+ const level = await readLevelReporting(engine, levelInput, round, opts.onProgress).finally(() => { delete value.telemetry.step; });await faultPoint('sync:level_read', { sourceId, round, level: level ? { statements: level.statements, confirm: level.confirm ?? null, reused: level.reused ?? 0, shadow: level, input: levelInput } : undefined });assertActive();if (!level) throw new Error(`Source ${sourceId} became a company-brain source during its sync pass.`);@@ -970,7 +992,7 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): Prreturn synced(false);}opts.onProgress?.({ phase: 'managed_sync.round_admit', round: { round, target: head, paths: level.paths.length, imports: imports.length, deletes: deletes.length, in_flight: inFlight.length } });- const built = await batchEntries(engine, level, deletes, imports, () => { assertActive(); beat.report(); }).catch(async (error: unknown) => {+ const built = await batchEntries(engine, level, deletes, imports).catch(async (error: unknown) => {if ((error as { code?: string } | null | undefined)?.code === 'page_identity_changed') await save('blocked', 'page_identity_changed');throw error;});@@ -992,7 +1014,7 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): Prfor (let i = 0; i < built.entries.length; i += PASS_BATCH_MAX) {const chunk = built.entries.slice(i, i + PASS_BATCH_MAX), last = !!fold && i + chunk.length >= built.entries.length; opts.onProgress?.({ phase: 'managed_sync.round_batch', round: { round, target: head, paths: level.paths.length, batch: i / PASS_BATCH_MAX + 1, batches: Math.ceil(built.entries.length / PASS_BATCH_MAX), base: i } });// A folded stamp writes the pass row with the obligations as they stand when its import is frozen.- const stopped = await drain(chunk, last ? { ...await stampFor(level), fold: { ...fold, obligations: structuredClone(value.obligations), batch: chunk.length } } : undefined);+ const stopped = await readDriftHistories(level, chunk, built.historyOwed, value, () => { assertActive(); beat.report(); }).then(async () => drain(chunk, last ? { ...await stampFor(level), fold: { ...fold, obligations: structuredClone(value.obligations), batch: chunk.length } } : undefined));if (stopped) return stop(stopped);await consumeRetry(chunk.map(entry => entry.path));value.telemetry.remaining = Math.max(0, built.entries.length - i - chunk.length) + inFlight.length + owed.length + mismatch.length;diff --git a/src/core/persistence/sync-pass-status.ts b/src/core/persistence/sync-pass-status.tsindex e7f48113cd..50ac90a87f 100644--- a/src/core/persistence/sync-pass-status.ts+++ b/src/core/persistence/sync-pass-status.ts@@ -8,7 +8,7 @@ import type { BrainEngine } from '../engine.ts';import type { Action } from '../agent-output.ts';import type { SyncAdmissionStatus } from './sync-admission.ts';import { digest } from './digest.ts';-import { fileDriftSide, GIT_HOLD_OP, type GitHoldRecord } from './sync-holds.ts';+import { EVERY_PASS_RESCREEN_CODES, fileDriftSide, GIT_HOLD_OP, holdRescreenDue, type GitHoldRecord } from './sync-holds.ts';import { readDeletesWithheld, type DeletesWithheld } from './sync-markers.ts';import { ABANDONED_QUIET_MS, ABANDONED_REPORT_STREAK, MANAGED_SYNC_PASS_OP, NOT_CONVERGED_REPORT_STREAK, passHealth, passReadClock, transientEnding, type PassRoundSummary,type PassRow } from './sync-pass-row.ts';@@ -70,17 +70,37 @@ export async function passConvergence(engine: Pick<BrainEngine, 'executeRaw'>, iduration_ms: t.duration_ms, timings_ms: t.timings_ms, last_full_at: o.converged?.last_full_at ?? null };}+/** GST-2843: how many of a pass's rewritten holds `passBanked` reads per page while it looks for one that settled its re-screen. */+const RESCREEN_PAGE = 500;+/*** GBRA-542: whether a pass committed work: a sync write it admitted committed, or it held a path it had not held before. Read* from durable rows, never from the pass row's heartbeat, so a pass that only reported progress is not counted.+ * GST-2843: or it re-screened a held path and the rewrite settled it: the hold is no longer due for a re-screen (an older+ * gbrain's hold now carries this screen's versions) and a drift hold names the page's current revision. A re-screen that+ * leaves the hold due (a code every pass re-screens) is not progress, so passes that only re-hold those still build the streak.+ * Nor is a rewrite marked `basis_moved`: the pass re-checked the path because its page or file moved again, so passes that+ * only re-hold a page that keeps moving build the streak too.*/export async function passBanked(engine: Pick<BrainEngine, 'executeRaw'>, sourceId: string, incarnation: string, telemetry: { pass_id: string; started_at: string }): Promise<boolean> {const [row] = await engine.executeRaw<{ banked: boolean }>(`SELECT EXISTS (SELECT 1 FROM persistence_requests r JOIN persistence_source_bindings b ON b.worktree_id=r.worktree_idWHERE b.source_id=$1 AND b.source_incarnation::text=$6 AND r.source_incarnation=b.source_incarnation AND r.state='committed' AND COALESCE(r.intent->>'kind','') LIKE 'managed_sync_%'AND r.completed_at >= $5::timestamptz AND r.intent->>'passId'=$2)- OR EXISTS (SELECT 1 FROM op_checkpoints WHERE op=$3 AND left(fingerprint, length($4))=$4 AND completed_keys->0->>'run_id'=$2 AND (completed_keys->0->>'held_at')::timestamptz >= $5::timestamptz) AS banked`,+ OR EXISTS (SELECT 1 FROM op_checkpoints WHERE op=$3 AND left(fingerprint, length($4))=$4 AND completed_keys->0->>'run_id'=$2 AND (completed_keys->0->>'held_at')::timestamptz >= $5::timestamptz+ AND completed_keys->0->'meta'->'basis_moved' IS NULL) AS banked`,[sourceId, telemetry.pass_id, GIT_HOLD_OP, `${sourceId}:${incarnation}:`, telemetry.started_at, incarnation]);- return row?.banked === true;+ if (row?.banked === true) return true;+ for (let after = ''; ;) {+ const rewritten = await engine.executeRaw<{ fingerprint: string; record: GitHoldRecord; revision: string | null }>(`SELECT h.fingerprint, h.completed_keys->0 AS record, p.knowledge_revision::text AS revision+ FROM op_checkpoints h LEFT JOIN pages p ON p.source_id=$1 AND p.id=(h.completed_keys->0->>'page_id')::bigint+ WHERE h.op=$2 AND left(h.fingerprint, length($3))=$3 AND h.completed_keys->0->>'run_id'=$4 AND (h.completed_keys->0->>'updated_at')::timestamptz >= $5::timestamptz+ AND NOT (h.completed_keys->0->>'code' = ANY($6::text[])) AND h.completed_keys->0->'meta'->'basis_moved' IS NULL AND h.fingerprint > $7 ORDER BY h.fingerprint LIMIT $8`,+ [sourceId, GIT_HOLD_OP, `${sourceId}:${incarnation}:`, telemetry.pass_id, telemetry.started_at, [...EVERY_PASS_RESCREEN_CODES], after, RESCREEN_PAGE]);+ if (rewritten.some(({ record, revision }) => !holdRescreenDue(record, false)+ && (record.code !== 'file_drift' || record.page_id == null || (record.meta.page_revision ?? null) === revision))) return true;+ if (rewritten.length < RESCREEN_PAGE) return false;+ after = rewritten.at(-1)!.fingerprint;+ }}export type PassStatusReason = 'pass_unfinished' | 'pass_interrupted' | 'pass_abandoned' | 'not_converged' | 'settle_mismatch' | 'deletes_withheld' | 'selection_scoped' | 'admission_paused' | 'writeback_failed';@@ -127,6 +147,9 @@ export function passStatusState(row: PassRow, withheld: DeletesWithheld | null,export function passStatusNext(sourceId: string, reason: PassStatusReason, row: PassRow, resume: string[], writebackFailed: string | null = null, interrupted?: boolean, banked?: boolean): Action {const verify = { argv: ['gbrain', 'sync', 'status', '--source', sourceId, '--json'] };const sync: Action = { argv: ['gbrain', 'sync', ...resume], consent: [], actor: 'agent', requires_exclusive: false, verify, why: '' };+ const step = row.value.telemetry.outcome === 'open' ? row.value.telemetry.step : undefined;+ const seen = step && ` last seen at ${step.at} ${step.name === 'level_read' ? 'reading the files and pages of the source (the level read)'+ : `reading the Git history of held pages it re-checks (${step.done} of ${step.total} paths of its batch), before that batch commits`}.`;switch (reason) {case 'not_converged':return (row.value.obligations.not_converged_streak ?? 0) >= NOT_CONVERGED_REPORT_STREAK@@ -148,9 +171,9 @@ export function passStatusNext(sourceId: string, reason: PassStatusReason, row:why: 'The preview names the parked effect and what a retry runs (read-only); until it commits, the source is never idle.' };case 'pass_abandoned':return { argv: ['gbrain', 'doctor', '--json'], consent: [], actor: 'agent', requires_exclusive: false, verify,- why: `The last ${abandonedPasses(row, passReadClock(row), interrupted, banked)} sync passes of ${sourceId} stopped before they ended (the drain abandoned them for making no progress, or their process was killed) or on a transient stop that recurs (a statement timeout, a lost connection), so another plain sync is likely to stop the same way. Report what doctor shows, with the gbrain version and the newest sync run's drain.stall (read-only).` };+ why: `The last ${abandonedPasses(row, passReadClock(row), interrupted, banked)} sync passes of ${sourceId} stopped before they ended (the drain abandoned them for making no progress, or their process was killed) or on a transient stop that recurs (a statement timeout, a lost connection), so another plain sync is likely to stop the same way. Report what doctor shows, with the gbrain version and the newest sync run's drain.stall (read-only).${seen ? ` The newest pass was${seen}` : ''}` };case 'pass_interrupted':- return { ...sync, why: `The newest pass of ${sourceId} stopped before it ended and no process holds its drain lease: its process was killed or crashed (an out-of-memory kill or a timeout), or stopped on an error it does not record (a dropped connection), so nothing recorded a failure. The plain sync continues it from the current level; if that pass stops the same way, status reads pass_abandoned.` };+ return { ...sync, why: `The newest pass of ${sourceId} stopped before it ended and no process holds its drain lease: its process was killed or crashed (an out-of-memory kill or a timeout), or stopped on an error it does not record (a dropped connection), so nothing recorded a failure. The plain sync continues it from the current level; if that pass stops the same way, status reads pass_abandoned.${seen ? ` It was${seen}` : ''}` };case 'selection_scoped':return { ...sync, why: 'The last pass converged at a narrower selection and did not advance last_commit; the plain sync runs the configured selection.' };default:diff --git a/src/core/persistence/sync-pass.ts b/src/core/persistence/sync-pass.tsindex ad39658039..1ab04a4c13 100644--- a/src/core/persistence/sync-pass.ts+++ b/src/core/persistence/sync-pass.ts@@ -835,7 +835,7 @@ export async function readShadowLevel(engine: BrainEngine, input: { sourceId: strecord: record ? { blobOids: recordBlobOids, revision: recordRevision != null ? String(recordRevision) : null, parserSame: sameParser(pages.parserOf(at!), fp(path)) } : null,attested, levelSha256: probe?.rawSha256 ?? null,published: !!attested?.hash && probe?.rawSha256 === attested.hash,- hold: hold ? { atLevel: holdAtLevel(hold, level, probe?.rawSha256 ?? null) && !moved.has(path), rescreenDue: holdRescreenDue(hold, retry.has(hold.path)), retried: retry.has(hold.path), ...(unscreenedDrift(hold) ? { unscreened: true } : {}), ...(hold.code === 'file_drift' && fileDriftSide(hold.meta) === 'deleted' ? { deletionSide: true } : {}) } : null,+ hold: hold ? { atLevel: holdAtLevel(hold, level, probe?.rawSha256 ?? null) && !moved.has(path), rescreenDue: holdRescreenDue(hold, retry.has(hold.path)), retried: retry.has(hold.path), ...(unscreenedDrift(hold) ? { unscreened: true } : {}), ...(hold.code === 'file_drift' && fileDriftSide(hold.meta) === 'deleted' ? { deletionSide: true } : {}), ...(moved.has(path) ? { moved: true } : {}) } : null,marker: marker ?? virtualMarker, baseline: needsBaseline ? baselineOf(path) : null,spelling: legacySpelling.has(path) ? 'legacy' : twins.has(path),episodeUnsatisfied: input.pass ? !!input.pass.episodeFullId && live && (record ? pages.fullEpisode[at!] : undefined) !== input.pass.episodeFullIddiff --git a/src/core/persistence/sync-run.ts b/src/core/persistence/sync-run.tsindex 19a5bb2499..da2790ab20 100644--- a/src/core/persistence/sync-run.ts+++ b/src/core/persistence/sync-run.ts@@ -537,7 +537,7 @@ async function freezeNext(engine: BrainEngine, cursor: Cursor, key: string, asseif (blockedOn && 'requestId' in frozen && cursor.entries[index]) frozen = { hold: dependentHold(cursor.entries[index]!, frozen.slug, frozen.pageId, frozen.intent, blockedOn) };if ('hold' in frozen) {return { held: true, saved: await saveCursor(engine, key, cursor, { ...withHeldKeys(advanceHeld(cursor, undefined, index), frozen.hold), ...(frozen.overtaken ? { overtaken: true as const } : {}) },- false, assertActive, heldWrite(cursor, frozen.hold, observedAt)) };+ false, assertActive, heldWrite(cursor, frozen.hold, observedAt, cursor.entries[index])) };}return { frozen, ...(deferredAt !== undefined ? { deferredAt } : {}) };}@@ -581,8 +581,8 @@ async function managedDryRun(engine: BrainEngine, value: Cursor, base: Cursor, rconst holdClear = (cursor: Cursor, path: string | null | undefined, observedAt: string) => path? (tx: BrainEngine) => clearGitHold(tx, { sourceId: cursor.sourceId, incarnation: cursor.incarnation, path, observedAt }) : undefined;/** #5988: the hold write that commits with the cursor step passing a held entry. */-const heldWrite = (held: Cursor, hold: HeldEntry, observedAt: string) => (tx: BrainEngine) => writeGitHold(tx, { source_id: held.sourceId, incarnation: held.incarnation, ...hold,- observed_at: observedAt, run_id: held.runId, mode: 'managed' });+const heldWrite = (held: Cursor, hold: HeldEntry, observedAt: string, entry: { basisMoved?: true } | undefined) => (tx: BrainEngine) => writeGitHold(tx, { source_id: held.sourceId,+ incarnation: held.incarnation, ...hold, ...(entry?.basisMoved ? { meta: { ...hold.meta, basis_moved: true as const } } : {}), observed_at: observedAt, run_id: held.runId, mode: 'managed' });/** #6278: the terminal code an owner stamps on a request whose preparation never settled within its attempts (Lane A, consumer.ts). */export const PREPARATION_STALLED = 'preparation_stalled';@@ -620,7 +620,7 @@ async function holdUnderGate(engine: BrainEngine, input: { cursor: Cursor; key:AND ${SYNC_PAGE_KINDS_SQL} AND intent->>'index'=$3 AND state IN ('failed','cancelled','conflict')`, [cursor.binding.worktree_id, cursor.runId, String(index)]);const converted = [...new Set([...(cursor.convertedFromFailed ?? []), ...stale.map(row => row.request_id), pending.requestId])];return writeCursor(tx, key, cursor, withHeldKeys(advanceHeld(base, converted, index), input.hold), async inner => {- await heldWrite(cursor, input.hold, input.observedAt)(inner);+ await heldWrite(cursor, input.hold, input.observedAt, cursor.entries[index])(inner);await recordSyncConversion(inner, cursor.sourceId, cursor.incarnation, { request_id: pending.requestId, path: pending.intent.path ?? null, slug: pending.slug, run_id: cursor.runId, outcome: 'held' });}, true);});diff --git a/src/core/persistence/sync-status.ts b/src/core/persistence/sync-status.tsindex 76eaf386cc..b3c4f61d1b 100644--- a/src/core/persistence/sync-status.ts+++ b/src/core/persistence/sync-status.ts@@ -447,7 +447,11 @@ export async function readSyncStatus(engine: BrainEngine, sourceId: string, opts? await passBanked(engine, sourceId, source.incarnation, passRow.value.telemetry).catch(() => undefined) : undefined;const passState = passRow ? passStatusState(passRow, withheld, source.writeback_failed, passReadClock(passRow), interrupted, banked) : null;const resume = !header && passRow ? passResumeArgs(sourceId, passRow) : managedSyncResumeArgs({ sourceId, processingOptions: header?.processingOptions, syncOptions: header?.syncOptions });- const human = holds.find(hold => hold.needs_human) ?? (liveError?.needs_human ? liveError : undefined);+ // GST-2843: passes parked because they only re-hold a page that keeps moving under them (each rewrote its hold for a moved basis,+ // up to the attempt cap): that hold's own ask is the decision, not the pass step.+ const moving = passState?.[1] === 'pass_abandoned' ? holds.find((hold, i) => hold.needs_human && hold.attempts >= HOLD_ATTEMPTS_NEEDS_HUMAN+ && records[i]!.meta.basis_moved && records[i]!.run_id === passRow!.value.telemetry.pass_id) : undefined;+ const human = moving ?? holds.find(hold => hold.needs_human) ?? (liveError?.needs_human ? liveError : undefined);const refreshRow = typeof source.refresh === 'string' ? JSON.parse(source.refresh) as RefreshProbeRow : source.refresh;const worktreeRefresh = refreshRow ? { refresh_id: refreshRow.id, state: refreshRow.state, updated_at: new Date(refreshRow.updated_at).toISOString(),...await (await import('./worktree-refresh.ts')).refreshHolderReport(engine, refreshRow) } : null;@@ -499,13 +503,15 @@ export async function readSyncStatus(engine: BrainEngine, sourceId: string, optsconst admission = await syncAdmissionStatus(engine, sourceId, source.incarnation,{ stored: source.admission_stored, row_value: source.admission_row_value, set_at: source.admission_set_at, pinned: source.admission_pinned, destructive: source.admission_destructive });const passNext = passRow && passState && passState[1] === stateReason ? passStatusNext(sourceId, passState[1], passRow, resume, source.writeback_failed, interrupted, banked) : null;+ // GST-2843: while the pass is what keeps the source from converging, its step leads; a held page's decision stays in holds and human_reason.+ const passLeads = passNext !== null && liveJob === null && !moving && (!!human || (!refreshFix && actionable.length > 0));return { source_id: sourceId, last_commit: source.last_commit, last_sync_at: iso(source.last_sync_at), last_committed_at: iso(committed?.last_committed_at), live: holder ? { ...holder, job_id: holderJob, stalled: holderStalled(holder, lease?.db_now) } : null,interrupted: interrupted && passRow ? { pass_id: passRow.value.telemetry.pass_id, since: passRow.updated_at, holder: lease?.holder ?? null } : null, run_id: header?.runId ?? null,cursor: header ? { index: Number(header.index), total: Number(header.total), pinned_target: header.target ?? null, last_advance_at: p ? new Date(p.lastAt).toISOString() : null, done: !!header.done } : null,- committed_last_10m: Number(committed?.n ?? 0), ...estimate, live_job_id: liveJob ?? holderJob, holds, ...(carriedHolds(records).length ? { carried: carriedHolds(records) } : {}), identity_downgrades: await withIdentityRemedies(engine, await listIdentityDowngrades(engine, [sourceId])), last_error: lastError, needs_human: !!human, ...(human ? { human_reason: human.human_reason } : {}),+ committed_last_10m: Number(committed?.n ?? 0), ...estimate, live_job_id: liveJob ?? holderJob, holds, ...(carriedHolds(records).length ? { carried: carriedHolds(records) } : {}), identity_downgrades: await withIdentityRemedies(engine, await listIdentityDowngrades(engine, [sourceId])), last_error: lastError, needs_human: passLeads && passNext ? passNext.ask_user === true : !!human, ...(human ? { human_reason: human.human_reason } : {}),needs_human_from: !human ? null : 'path' in human && 'held_since' in human ? { kind: 'hold', path: human.path, slug: human.slug, code: human.code } : { kind: 'last_error', code: human.code },resume_argv: resume, ...blockers, cursors: detail ? await liveCursors(engine, sourceId, source.incarnation, selected?.key ?? null, failures) : [],- terminal_failures: terminal, connector_last_error: connectorLastError, git_push_holds: gitPushHolds, fence_holds: fenceHolds, worktree_refresh: worktreeRefresh, next: next ?? recovery_next ?? passNext ?? (stateReason === 'request_pending' && oldest.status === 'ok' && oldest.request ? { argv: ['gbrain', 'sync', ...resume], consent: [], actor: 'agent', requires_exclusive: false, verify,+ terminal_failures: terminal, connector_last_error: connectorLastError, git_push_holds: gitPushHolds, fence_holds: fenceHolds, worktree_refresh: worktreeRefresh, next: (passLeads && passNext ? passNext : next) ?? recovery_next ?? passNext ?? (stateReason === 'request_pending' && oldest.status === 'ok' && oldest.request ? { argv: ['gbrain', 'sync', ...resume], consent: [], actor: 'agent', requires_exclusive: false, verify,why: `Write request ${oldest.request.request_id} of this source is still ${oldest.request.state}${oldest.request.step ? ` (step ${oldest.request.step})` : ''}; a running owner finishes it on its own, and the next sync of this source drains it.` } satisfies Action: stateReason === 'commits_waiting' ? { argv: ['gbrain', 'sync', ...resume], consent: [], actor: 'agent', requires_exclusive: false, verify,why: `The checkout's HEAD is ${waiting} commit(s) past the last synced commit ${source.last_commit?.slice(0, 12)}; this sync imports them.` } satisfies Actiondiff --git a/src/core/repair/core.ts b/src/core/repair/core.tsindex 80e23c9fda..c785ae4c69 100644--- a/src/core/repair/core.ts+++ b/src/core/repair/core.ts@@ -99,6 +99,8 @@ export interface RepairPlan {llm?: { usd: number | null; cap_remaining_usd: number | null };/** Discovery kinds: when the scan the plan read last finished (`fresh_at`) and whether it is still partial. */scan?: { fresh_at: string | null; partial: boolean };+ /** GST-2843: a windowed dry run whose selection holds more than this window: false, so its result is not complete. Applies ignore it. */+ complete?: boolean;/*** The items whose write re-embeds, when the kind knows which do (GBRA-443: private-labels re-embeds only a page stored* under per-chunk synopsis); the cost counts the pending ones among them. Default: every pending item, none under embeds: false.@@ -462,7 +464,7 @@ async function runRepairScoped(ctx: OperationContext, handler: RepairHandler, scif (plan.listing) result.listing = plan.listing;if (plan.preview_hash) result.preview_hash = plan.preview_hash;if (plan.details) result.details = plan.details;- result.complete = pending.length === plan.items.length;+ result.complete = pending.length === plan.items.length && plan.complete !== false;return finish();}// A resumed run keeps its id, so work it authorized is replayed, not authorized twice.diff --git a/src/core/repair/file-drift-compare.ts b/src/core/repair/file-drift-compare.tsindex b44a74f5a4..08ae1b9f7d 100644--- a/src/core/repair/file-drift-compare.ts+++ b/src/core/repair/file-drift-compare.ts@@ -80,7 +80,7 @@ export function newerPageParagraph(sourceId: string, slug: string, evidence: Dri}/** One selected page, compared: a held page by its hold, any other by its slug. Never writes. */-export async function comparePage(engine: BrainEngine, sourceId: string, target: { slug: string | null; hold?: GitHoldRecord }, assessmentAt: string): Promise<ComparedPage> {+export async function comparePage(engine: BrainEngine, sourceId: string, target: { slug: string | null; hold?: GitHoldRecord; mirror?: boolean }, assessmentAt: string): Promise<ComparedPage> {const { slug, hold } = target;const item = `${sourceId}:${slug ?? hold?.path ?? ''}`;const side = hold ? fileDriftSide(hold.meta) : undefined;@@ -100,8 +100,16 @@ export async function comparePage(engine: BrainEngine, sourceId: string, target:why: !hold ? 'the page and its file agree' : due ? 'the page and its file agree once gbrain\'s own normalisation is read; the next sync releases the hold with no write': `the page and its file agree; gbrain repair file-drift --source ${sourceId} --slug ${slug} --prefer file previews the release` };}- if (live.still_present !== true) return { item, slug, ...held, would: live.still_present === false ? 'no_drift' : 'unknown', why: live.reason };- const { path, ...evidence } = await readDriftEvidence(engine, sourceId, slug, assessmentAt);+ // GST-2843: a read-only mirror never writes its files, so the live check calls every page database-only; a held page whose+ // file sync still imports is compared with that file instead, and only a file that is gone stays database-only.+ const mirrorHeld = live.still_present === false && live.reason === 'database_only' && !!hold && side !== 'deleted' && target.mirror === true;+ if (live.still_present !== true && !mirrorHeld) return { item, slug, ...held, would: live.still_present === false ? 'no_drift' : 'unknown', why: live.reason };+ const read = await readDriftEvidence(engine, sourceId, slug, assessmentAt).catch(error => {+ if (mirrorHeld && error instanceof OperationError && error.detail === 'canonical_file_missing') return null;+ throw error;+ });+ if (!read) return { item, slug, ...held, would: 'no_drift', why: live.reason };+ const { path, ...evidence } = read;const base = { item, slug, ...held, path, evidence };if (evidence.differs === 'none') return { ...base, would: hold ? 'release_hold' : 'no_drift', why: 'nothing differs once gbrain\'s own normalisation is read' };if (evidence.newer === 'file') return { ...base, would: 'prefer_file', why: `${path} was committed after the page last changed, so the file wins (Git is the parent): gbrain repair file-drift --source ${sourceId} --slug ${slug} --prefer file` };diff --git a/src/core/repair/file-drift.ts b/src/core/repair/file-drift.tsindex 926792464d..e904190e65 100644--- a/src/core/repair/file-drift.ts+++ b/src/core/repair/file-drift.ts@@ -97,6 +97,7 @@ import { authorizeReconcile, runReconcileApply } from '../persistence/reconcile.import { discardedFilePaths, unimportedFile, unimportedFileText, type UnimportedFile } from '../persistence/unimported-file.ts';import { acceptedPendingReceipt } from '../persistence/accepted-pending.ts';import { recordsNoFileOrigin } from '../persistence/unbound-pages.ts';+import { sourceMirrorReadOnly } from '../persistence/mirror-read-only.ts';import { comparedListing, comparePage, linesNotCarried, newerPageParagraph, readDriftEvidence, type ComparedPage } from './file-drift-compare.ts';import { serializePageToMarkdown } from '../markdown.ts';import { afterCursor, type RepairHandler, type RepairItem, type RepairItemOutcome, type RepairPlan, type RepairPlanOptions, type RepairScope } from './core.ts';@@ -789,7 +790,8 @@ function item(page: ApprovedPage, index: number, hash: string, last: boolean, prasync function compareSelection(engine: BrainEngine, sourceId: string, opts: RepairPlanOptions): Promise<RepairPlan> {const held = opts.held ? await selectHeld(engine, sourceId, opts) : null;const selected = held ? null : await selectSlugs(engine, sourceId, opts);- const targets: { slug: string | null; hold?: GitHoldRecord }[] = held ? [...held.legacy.map(hold => ({ slug: hold.slug, hold })), ...held.targets.slice(0, HELD_WINDOW_PAGES).map(target => ({ slug: target.hold.slug, hold: target.hold }))]+ const mirror = held ? await sourceMirrorReadOnly(engine, sourceId) : false;+ const targets: { slug: string | null; hold?: GitHoldRecord; mirror?: boolean }[] = held ? [...held.legacy.map(hold => ({ slug: hold.slug, hold, mirror })), ...held.targets.slice(0, HELD_WINDOW_PAGES).map(target => ({ slug: target.hold.slug, hold: target.hold, mirror }))]: selected!.slugs.map(slug => ({ slug }));const residuals: Record<string, number> = { ...(held?.notSelected || selected?.notSelected ? { not_selected: held?.notSelected ?? selected!.notSelected } : {}), ...(held?.before ? { before_cursor: held.before } : {}) };const count = (key: string) => { residuals[key] = (residuals[key] ?? 0) + 1; };@@ -816,9 +818,9 @@ async function compareSelection(engine: BrainEngine, sourceId: string, opts: Repconst filesNewer = pages.filter(page => page.would === 'prefer_file').length;const next = [...(filesNewer ? [`Preview the file side for the ${filesNewer} page(s) whose file is newer: ${previewCommand(sourceId, { ...opts, prefer: 'file' })} (pages newer than their file are left held, kept_newer_page)`] : []),...(nextCursor ? [`Compare the next window: ${shellQuote(compareArgv(sourceId, { ...opts, cursor: nextCursor }).slice(0, -1))}`] : [])];- return { items: [], residuals, listing, details: { compare: true, writes: 'none',- pages: pages.map(({ item: _item, ...page }) => page), ...(next.length ? { next_actions: next } : {}),- ...(held ? { window: { previewed: windowed, remaining, complete: remaining === 0, ...(nextCursor ? { next_cursor: nextCursor } : {}) } } : {}),+ return { items: [], residuals, listing, ...(held ? { complete: remaining === 0 } : {}), details: { compare: true, writes: 'none',+ pages: pages.map(({ item: _item, ...page }) => ({ evidence: null, ...page })), ...(next.length ? { next_actions: next } : {}),+ ...(held ? { window: { previewed: windowed, remaining, complete: remaining === 0, ...(nextCursor ? { next_cursor: nextCursor, next_command: shellQuote(compareArgv(sourceId, { ...opts, cursor: nextCursor })) } : {}) } } : {}),evidence: 'newer compares the file\'s last commit time at HEAD with the page\'s updated_at; unknown when the file has uncommitted bytes or either time is missing.' } };}@@ -1051,7 +1053,7 @@ export const fileDriftRepair: RepairHandler = {artifact_digest: digest(page), selection })));}return { items: pages.map((page, index) => item({ source_id: sourceId, slug: page.slug, artifact_digest: digest(page), selection }, index, hash, index === pages.length - 1, opts.prefer)),- preview_hash: hash, residuals, listing, ...(warnings.length ? { warnings } : {}),+ preview_hash: hash, residuals, listing, ...(held ? { complete: remaining === 0 } : {}), ...(warnings.length ? { warnings } : {}),details: { prefer: opts.prefer, ...(artifact ? { artifact } : {}),...(undoneBySync ? { undone_by_next_sync_unless_committed: undoneBySync } : {}),...(held ? { window: { previewed, remaining, complete: remaining === 0, ...(nextCursor ? { next_cursor: nextCursor, next_command: nextCommand } : {}) } } : {}),diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex 82f363ee2a..c111428d60 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -709,6 +709,7 @@ test/sync-legacy-sweep-resume-7189.test.tstest/sync-level-reuse-across-gbra133.test.tstest/sync-metafile-foreign-slug-7341.test.tstest/sync-mirror-catchup-gst2515.test.ts+test/sync-mirror-parked-gst2843.test.tstest/sync-origin-live-wins-6517.test.tstest/sync-page-identity-hold-gbra130.test.tstest/sync-page-write-failed-gbra130.test.tsdiff --git a/test/sources-status-hold-fix-routing-9319.test.ts b/test/sources-status-hold-fix-routing-9319.test.tsindex 398261cdea..2826bc39ea 100644--- a/test/sources-status-hold-fix-routing-9319.test.ts+++ b/test/sources-status-hold-fix-routing-9319.test.ts@@ -17,6 +17,6 @@ describe('#9319: sources status --json renders hold fixes with the routing pin',const out = gitHoldStatusJson('source-a', { count: 1, items: [item], sync_argv: ['gbrain', 'sync', '--source', 'source-a', '--no-pull'] }, { mode: 'summary' });const pinned = ['gbrain', 'sources', 'reconcile', 'source-a', 'notes/alice-example', '--preview', '--brain', 'host'];expect(out.items[0]!.fix.argv).toEqual(pinned);- expect(out.by_code[0]!.example.fix.argv).toEqual(pinned);+ expect(out.by_code[0]!.example!.fix.argv).toEqual(pinned);});});diff --git a/test/sources-status-holds-bounded-7326.test.ts b/test/sources-status-holds-bounded-7326.test.tsindex a9951c7cbd..5e207c5d21 100644--- a/test/sources-status-holds-bounded-7326.test.ts+++ b/test/sources-status-holds-bounded-7326.test.ts@@ -21,7 +21,7 @@ describe('#7326: sources status --json bounds git_holds', () => {expect(out.items).toHaveLength(HELD_STATUS_LIMIT);expect(out).toMatchObject({ items_complete: false, items_command: ['gbrain', 'sources', 'status', 'source-a', '--json', '--holds'] });expect(out.by_code.map(g => [g.code, g.count])).toEqual([['invalid_frontmatter', 333], ['invalid_fence', 167]]);- expect(out.by_code[0]!.example.fix).toBeDefined();+ expect(out.by_code[0]!.example!.fix).toBeDefined();expect(Object.keys(out).at(-1)).toBe('items');const full = JSON.stringify(gitHoldStatusJson('source-a', holds(500), { mode: 'all' }));expect(JSON.stringify(out).length).toBeLessThan(full.length / 10);diff --git a/test/sync-mirror-parked-gst2843.test.ts b/test/sync-mirror-parked-gst2843.test.tsnew file mode 100644index 0000000000..e6b9b750bd--- /dev/null+++ b/test/sync-mirror-parked-gst2843.test.ts@@ -0,0 +1,321 @@+/**+ * GST-2843: a read-only mirror whose every held page came due for one re-check (holds an older gbrain wrote) stayed+ * blocked / pass_abandoned after GST-2515.+ * - A stopped pass that re-checked held pages and cleared their re-check trigger saved work, so it does not build the+ * abandoned streak; a pass that re-checks the same holds on every pass still does, and still ends abandoned.+ * - While the pass is what keeps the source from converging, status names the pass's next step, not a held page's decision.+ * - On a read-only mirror the held comparison reads the file: a page that differs is compared like a writable source's,+ * and only a file that is gone reads no_drift / database_only. It still writes nothing, and sync holds the same paths.+ * - The held comparison is not complete while held pages remain past its window.+ * - sources status counts every hold per code, not only the listed ones.+ * PGLite always; Postgres when DATABASE_URL is set (managedBrain). Synthetic content only.+ */+import { describe, expect, setSystemTime, spyOn, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import * as gitStream from '../src/core/persistence/sync-git-stream.ts';+import { execFileSync } from 'node:child_process';+import { mkdirSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { RECOVERY_VERSION } from '../src/core/markdown.ts';+import { withCoordinatedWrite } from '../src/core/persistence/context.ts';+import { installFaultHook } from '../src/core/persistence/fault-points.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { PASS_BATCH_KEY_PREFIX, readPassBatch } from '../src/core/persistence/sync-pass-batch.ts';+import { registerLocalWriter, withVerifiedLocalRegistration } from '../src/core/persistence/identity.ts';+import { FILE_DRIFT_VERSION, writeGitHold } from '../src/core/persistence/sync-holds.ts';+import { readPassRow } from '../src/core/persistence/sync-pass-row.ts';+import { passBanked } from '../src/core/persistence/sync-pass-status.ts';+import { performManagedSync } from '../src/core/persistence/sync-run.ts';+import { readSyncStatus } from '../src/core/persistence/sync-status.ts';+import { gitHoldStatusJson, readGitHoldStatuses } from '../src/core/persistence/connector-status.ts';+import { resolveRepairScope, type RepairPlanOptions } from '../src/core/repair/core.ts';+import { repairRunner } from '../src/core/repair/registry.ts';+import { managedBrain } from './helpers/managed-brain.ts';+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';+import { TEST_WRITE_ATTRIBUTION } from './helpers/write-attribution.ts';+import { waitFor } from './helpers/wait-for.ts';+import { withEnv } from './helpers/with-env.ts';++const git = (root: string, ...args: string[]) => execFileSync('git', ['-C', root, '-c', 'user.name=Example', '-c', '[email protected]',+ '-c', 'commit.gpgsign=false', ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();+const OPTS = { sourceId: 'default', noPull: true, noEmbed: true, noExtract: true };+const logger = { info() {}, warn() {}, error() {} };+const doc = (title: string, body: string, extra = '') => `---\ntype: concept\ntitle: ${title}\n${extra}---\n\n${body}\n`;+const commitAll = (root: string, message: string) => { git(root, 'add', '-A'); git(root, 'commit', '-qm', message); };+const files = (root: string, entries: Record<string, string>) => {+ for (const [slug, content] of Object.entries(entries)) { mkdirSync(join(root, slug, '..'), { recursive: true }); writeFileSync(join(root, `${slug}.md`), content); }+};+const setupWith = (entries: Record<string, string>) => ({ root }: { root: string }) => { git(root, 'init', '-q'); files(root, entries); commitAll(root, 'pages'); };+const coordinated = (engine: BrainEngine, sql: string, params: unknown[] = []) =>+ engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => tx.executeRaw(sql, params), TEST_WRITE_ATTRIBUTION));+const mirror = (engine: BrainEngine) => engine.executeRaw(`UPDATE sources SET config = config || '{"mirror_read_only": true}'::jsonb WHERE id='default'`);+const incarnation = async (engine: BrainEngine) => (await engine.executeRaw<{ i: string }>("SELECT incarnation::text AS i FROM sources WHERE id='default'"))[0]!.i;+const holdRows = async (engine: BrainEngine) => (await engine.executeRaw<{ r: { path: string; code: string; upstream_version: string | null; meta: Record<string, unknown> } }>(+ "SELECT completed_keys->0 AS r FROM op_checkpoints WHERE op='sync-hold' ORDER BY fingerprint")).map(({ r }) => r);+/** Pages as an older gbrain left them: no import record, and a revision no import or write request produced. */+const legacy = async (engine: BrainEngine) => {+ await coordinated(engine, "DELETE FROM op_checkpoints WHERE op='sync-import-provenance'");+ await coordinated(engine, "UPDATE pages SET knowledge_revision=gen_random_uuid(), revision_write_request_id=NULL WHERE source_id='default'");+};+class Killed extends Error {}+/** One sync stopped after `after` cursor checkpoints, as a killed process leaves it: its pass row open and no failure recorded. */+async function stoppedSync(engine: BrainEngine, after: number) {+ let n = 0;+ installFaultHook(async point => { if (point === 'sync:mid_checkpoint' && ++n > after) throw new Killed('killed'); });+ try { await performManagedSync(engine, OPTS); } catch (e) { if (!(e instanceof Killed) && !String(e).includes('killed')) throw e; } finally { installFaultHook(undefined); }+ await engine.executeRaw("DELETE FROM op_checkpoints WHERE op='managed-sync-failure'");+}+async function repair(engine: BrainEngine, opts: Partial<RepairPlanOptions>) {+ return withVerifiedLocalRegistration(engine, await registerLocalWriter(engine, 'cli'), async () =>+ (await repairRunner(engine, { apply: false, logger })).run('file-drift', await resolveRepairScope(engine, 'default'), { explicit: true, sourceFlag: 'default', ...opts } as never));+}+type Plan = { complete: boolean; listing?: Array<{ item: string; class: string; detail?: string }>; details?: Record<string, any> };+const classes = (plan: unknown) => Object.fromEntries(((plan as Plan).listing ?? []).map(entry => [entry.item.replace('default:', ''), entry.class]));++/** A mirror whose `count` pages changed in the database, lost their import records and had their files rewritten upstream: one side-unknown hold each. */+async function unknownHolds(engine: BrainEngine, root: string, count: number): Promise<void> {+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');+ await mirror(engine);+ await coordinated(engine, "UPDATE pages SET compiled_truth=compiled_truth||' Edited in the database.' WHERE source_id='default' AND slug LIKE 'notes/%'");+ await legacy(engine);+ files(root, Object.fromEntries(Array.from({ length: count }, (_, i) => [`notes/n${i}`, doc(`notes/n${i}`, `Body ${i}, rewritten upstream.`)])));+ commitAll(root, 'upstream rewrites');+ await performManagedSync(engine, OPTS);+ expect((await holdRows(engine)).filter(h => h.code === 'file_drift' && h.meta.side === 'unknown')).toHaveLength(count);+}+const notes = (count: number, extra: Record<string, string> = {}) => setupWith({ ...Object.fromEntries(Array.from({ length: count }, (_, i) => [`notes/n${i}`, doc(`notes/n${i}`, `Body ${i}.`)])), ...extra });++describe.each(testBackends())('GST-2843: stopped passes that only re-check holds (%s)', backend => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;++ test('a stopped pass that re-checked old holds and cleared their trigger saved work: no abandoned streak, and the plain sync finishes', () => managedBrain(async ({ engine, root }) => {+ await unknownHolds(engine, root, 12);+ // An older gbrain wrote these holds before the drift screen's current version, so every one is due for one re-check.+ await engine.executeRaw("UPDATE op_checkpoints SET completed_keys=jsonb_set(completed_keys,'{0,meta,drift_version}','2'::jsonb) WHERE op='sync-hold'");+ const inc = await incarnation(engine);+ for (let pass = 0; pass < 3; pass++) await stoppedSync(engine, 2);+ const due = (await holdRows(engine)).filter(h => (h.meta.drift_version as number) < FILE_DRIFT_VERSION).length;+ expect(due).toBeLessThan(12);+ expect((await readPassRow(engine, 'default', inc))?.value.obligations.abandoned_streak ?? 0).toBe(0);+ const status: any = await readSyncStatus(engine, 'default');+ expect(status.reason).not.toBe('pass_abandoned');+ const finished: any = await performManagedSync(engine, OPTS);+ expect(finished.status).toBe('synced');+ expect((await holdRows(engine)).every(h => h.meta.drift_version === FILE_DRIFT_VERSION)).toBe(true);+ }, { databaseUrl, setup: notes(12) }), 600_000);++ test('the earlier versions old holds are re-checked against are read per batch, after the batch is admitted, not for the whole round first', () => managedBrain(async ({ engine, root }) => {+ await unknownHolds(engine, root, 6);+ await engine.executeRaw("UPDATE op_checkpoints SET completed_keys=jsonb_set(completed_keys,'{0,meta,drift_version}','1'::jsonb) WHERE op='sync-hold'");+ const events: string[] = [];+ const real = gitStream.pathBlobHistories;+ const reads = spyOn(gitStream, 'pathBlobHistories').mockImplementation(async (...args: Parameters<typeof real>) => {+ events.push(`history:${args[2].length}`);+ return real(...args);+ });+ try {+ const result: any = await performManagedSync(engine, { ...OPTS, onProgress: (e: { phase?: string }) => { if (e.phase === 'managed_sync.round_admit' || e.phase === 'managed_sync.round_batch') events.push(e.phase); } } as never);+ expect(result.status).toBe('synced');+ } finally { reads.mockRestore(); }+ expect(events.filter(e => e.startsWith('history:'))).toEqual(['history:6']);+ expect(events.indexOf('history:6')).toBeGreaterThan(events.indexOf('managed_sync.round_batch'));+ }, { databaseUrl, setup: notes(6) }), 600_000);++ test('a pass killed while it reads the Git history of re-checked holds leaves that step on its row, and status names it', () => managedBrain(async ({ engine, root }) => {+ await unknownHolds(engine, root, 3);+ await engine.executeRaw("UPDATE op_checkpoints SET completed_keys=jsonb_set(completed_keys,'{0,meta,drift_version}','1'::jsonb) WHERE op='sync-hold'");+ const inc = await incarnation(engine);+ const reads = spyOn(gitStream, 'pathBlobHistories').mockImplementation(async (...args: Parameters<typeof gitStream.pathBlobHistories>) => {+ const onPath = args[4]!;+ onPath(args[2][0]!);+ setSystemTime(new Date(Date.now() + 31_000));+ onPath(args[2][1]!);+ for (let i = 0; i < 200 && !(await readPassRow(engine, 'default', inc))?.value.telemetry.step; i++) await Bun.sleep(50);+ throw new Killed('killed');+ });+ try { await performManagedSync(engine, OPTS); } catch (e) { if (!String(e).includes('killed')) throw e; } finally { reads.mockRestore(); setSystemTime(); }+ await engine.executeRaw("DELETE FROM op_checkpoints WHERE op='managed-sync-failure'");+ expect((await readPassRow(engine, 'default', inc))?.value.telemetry).toMatchObject({ outcome: 'open', step: { name: 'drift_history', done: 2, total: 3 } });+ const status: any = await readSyncStatus(engine, 'default');+ expect(status.reason).toBe('pass_interrupted');+ expect(status.next.why).toContain('reading the Git history of held pages it re-checks (2 of 3 paths of its batch)');+ expect((await performManagedSync(engine, OPTS)).status).toBe('synced');+ expect((await readPassRow(engine, 'default', inc))?.value.telemetry.step).toBeUndefined();+ }, { databaseUrl, setup: notes(3) }), 600_000);++ test('passes that re-check the same holds every time still build the streak, end abandoned, and status then names the pass step over a held page', () => managedBrain(async ({ engine, root }) => {+ await unknownHolds(engine, root, 1);+ // Slug conflicts re-check on every pass by design: a pass that only re-holds them moves nothing.+ files(root, Object.fromEntries(Array.from({ length: 8 }, (_, i) => [`clash/c${i}`, doc(`clash/c${i}`, 'Slug conflict.', 'slug: notes/n0\n')])));+ commitAll(root, 'upstream adds conflicting slugs');+ await performManagedSync(engine, OPTS);+ expect((await holdRows(engine)).filter(h => h.code === 'frontmatter_slug_conflict').length).toBeGreaterThan(2);+ const inc = await incarnation(engine);+ for (let pass = 0; pass < 3; pass++) await stoppedSync(engine, 1);+ expect((await readPassRow(engine, 'default', inc))?.value.obligations.abandoned_streak).toBeGreaterThanOrEqual(2);+ const status: any = await readSyncStatus(engine, 'default');+ expect([status.state, status.reason]).toEqual(['blocked', 'pass_abandoned']);+ expect(status.next.argv).toEqual(['gbrain', 'doctor', '--json']);+ expect(status.next.user_message ?? '').not.toContain('needs a decision');+ expect(status.needs_human).toBe(false);+ expect(status.needs_human_from).toMatchObject({ kind: 'hold', code: 'file_drift' });+ }, { databaseUrl, setup: notes(1) }), 600_000);+});++describe.each(testBackends())('GST-2843: the held comparison on a read-only mirror (%s)', backend => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ /** The `file_drift` hold an older screen wrote for a page, at its current revision. */+ async function seedHold(engine: BrainEngine, slug: string, meta: Record<string, unknown>): Promise<void> {+ const [page] = await engine.executeRaw<{ id: string; revision: string; incarnation: string }>(`SELECT p.id::text AS id, p.knowledge_revision::text AS revision,+ s.incarnation::text AS incarnation FROM pages p JOIN sources s ON s.id=p.source_id WHERE p.source_id='default' AND p.slug=$1`, [slug]);+ await engine.transaction(tx => writeGitHold(tx, { source_id: 'default', incarnation: page!.incarnation, path: `${slug}.md`, source_path: `${slug}.md`, slug,+ page_id: Number(page!.id), code: 'file_drift', message: `${slug}.md is held: file drift`, upstream_version: null, observed_at: new Date().toISOString(),+ run_id: null, mode: 'managed', meta: { recovery_version: RECOVERY_VERSION, page_revision: page!.revision, ...meta } as never }));+ }+ /** Four held pages on a mirror: same (agrees with its file), newer-file, newer-page (each differs in body), gone (its file left Git). */+ async function heldPages(engine: BrainEngine, root: string): Promise<void> {+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');+ await coordinated(engine, "UPDATE pages SET compiled_truth='An old database body.', updated_at=now() - interval '2 hours' WHERE source_id='default' AND slug='notes/newer-file'");+ files(root, { 'notes/newer-file': doc('notes/newer-file', 'A newer file body.') });+ git(root, 'rm', '-q', '--', 'notes/gone.md');+ commitAll(root, 'edit and remove');+ await coordinated(engine, "UPDATE pages SET compiled_truth='A newer database body.', updated_at=now() + interval '1 minute' WHERE source_id='default' AND slug='notes/newer-page'");+ await legacy(engine);+ await mirror(engine);+ for (const slug of ['notes/same', 'notes/newer-file', 'notes/newer-page']) await seedHold(engine, slug, { sides: 'unknown', side: 'unknown', drift_version: FILE_DRIFT_VERSION });+ await seedHold(engine, 'notes/gone', { side: 'deleted', state: 'file_gone' });+ }+ const heldSetup = setupWith(Object.fromEntries(['same', 'newer-file', 'newer-page', 'gone'].map(name => [`notes/${name}`, doc(`notes/${name}`, `The committed body of notes/${name}.`)])));++ test('a held page that differs from its file is compared, not read as no drift; only a gone file is database-only', () => managedBrain(async ({ engine, root }) => {+ await heldPages(engine, root);+ const compared = await repair(engine, { held: true }) as Plan;+ expect(classes(compared)).toEqual({ 'notes/gone': 'no_drift', 'notes/newer-file': 'prefer_file', 'notes/newer-page': 'needs_human', 'notes/same': 'release_hold' });+ const pages = Object.fromEntries((compared.details!.pages as Array<{ slug: string; evidence?: { differs: string; newer: string } | null }>).map(page => [page.slug, page.evidence]));+ expect([pages['notes/newer-file'], pages['notes/newer-page']].map(found => [found!.differs, found!.newer])).toEqual([['body', 'file'], ['body', 'database']]);+ expect(pages['notes/gone']).toBeNull();+ }, { databaseUrl, setup: heldSetup }), 240_000);++ test('the comparison changes nothing sync does: a sync after it holds the same paths with the same records', () => managedBrain(async ({ engine, root }) => {+ await heldPages(engine, root);+ await performManagedSync(engine, OPTS);+ const strip = (rows: Awaited<ReturnType<typeof holdRows>>) => rows.map(({ path, code, upstream_version, meta }) => ({ path, code, upstream_version, side: meta.side, state: meta.state }));+ const before = strip(await holdRows(engine));+ const revisions = async () => JSON.stringify(await engine.executeRaw("SELECT slug, knowledge_revision::text, deleted_at IS NULL AS live FROM pages ORDER BY slug"));+ const pages = await revisions();+ await repair(engine, { held: true });+ await performManagedSync(engine, OPTS);+ expect(strip(await holdRows(engine))).toEqual(before);+ expect(await revisions()).toBe(pages);+ expect(git(root, 'status', '--porcelain')).toBe('');+ }, { databaseUrl, setup: heldSetup }), 240_000);+});++describe.each(testBackends())('GST-2843: counts past a window or a listing limit (%s)', backend => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ test('the held comparison is complete only at its last window, and sources status counts every hold per code', () => managedBrain(async ({ engine, root }) => {+ await unknownHolds(engine, root, 230);+ const first = await repair(engine, { held: true }) as Plan;+ expect(first.details!.window).toMatchObject({ previewed: 200, remaining: 30, complete: false });+ expect(first.complete).toBe(false);+ expect(first.details!.window.next_command).toContain(`--held --cursor ${first.details!.window.next_cursor}`);+ const last = await repair(engine, { held: true, cursor: first.details!.window.next_cursor }) as Plan;+ expect(last.details!.window).toMatchObject({ remaining: 0, complete: true });+ expect(last.complete).toBe(true);+ // A listing that reads 50 holds (the sync result's detail cap) still counts all 230 per code.+ const status = (await withEnv({ GBRAIN_SYNC_HOLD_CAP: '50' }, () => readGitHoldStatuses(engine, ['default']))).get('default')!;+ const json: any = gitHoldStatusJson('default', status, { mode: 'summary' });+ expect([json.count, json.truncated, status.items.length]).toEqual([230, true, 50]);+ expect(json.by_code).toEqual([expect.objectContaining({ code: 'file_drift', count: 230 })]);+ expect(json.by_code_complete).toBeUndefined();+ }, { databaseUrl, setup: notes(230) }), 900_000);+});++describe.each(testBackends())('GST-2843: a pass that settled one re-check among many still-due ones (%s)', backend => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ test('the settled re-check counts as progress even behind more than 50 rewritten holds that stay due', () => managedBrain(async ({ engine }) => {+ const inc = await incarnation(engine);+ const before = '2026-01-01T00:00:00.000Z', started = '2026-01-02T00:00:00.000Z', during = '2026-01-02T00:05:00.000Z';+ const hold = (path: string, code: string, meta: Record<string, unknown>) => [`default:${inc}:${path}`, JSON.stringify([{ version: 1, source_id: 'default', incarnation: inc, path,+ source_path: path, slug: null, page_id: null, code, message: 'synthetic', upstream_version: null, observed_at: during, held_at: before, updated_at: during,+ run_id: 'pass-1', mode: 'managed', meta: { recovery_version: RECOVERY_VERSION, ...meta } }])];+ // Rewritten by the pass and still due (a transient hold below its attempt cap), then one hold whose re-screen settled.+ const rows = [...Array.from({ length: 520 }, (_, i) => hold(`a/n${String(i).padStart(3, '0')}.md`, 'preparation_stalled', { attempts: 1 })), hold('z/settled.md', 'invalid_frontmatter', {})];+ await coordinated(engine, "INSERT INTO op_checkpoints(op,fingerprint,completed_keys) SELECT 'sync-hold', f, k::jsonb FROM unnest($1::text[], $2::text[]) AS t(f, k)",+ [rows.map(([fingerprint]) => fingerprint), rows.map(([, keys]) => keys)]);+ expect(await passBanked(engine, 'default', inc, { pass_id: 'pass-1', started_at: started })).toBe(true);+ }, { databaseUrl }), 120_000);+});++describe.each(testBackends())('GST-2843: a held page that keeps moving under killed passes (%s)', backend => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ const SLUG = 'people/alice-example', PATH = 'people/alice-example.md';+ const person = (body: string, extra = '') => `---\ntype: person\ntitle: Alice Example\n${extra}---\n\n${body}\n`;+ const setup = ({ root }: { root: string }) => { git(root, 'init', '-q'); files(root, { 'people/alice-example': person('Works at acme-example.') }); commitAll(root, 'pages'); };+ const theHold = async (engine: BrainEngine) => (await holdRows(engine)).find(h => h.path === PATH)!;+ /** A database-only write raced the import of a newer Git version (#6194), then eight files claim the page's slug, so every pass re-checks them. */+ async function movingHold(engine: BrainEngine, ctx: OperationContext, root: string) {+ await engine.setConfig('sync.write_through', 'false');+ const put = async (text: string) => {+ const id = randomUUID(), current = await engine.readPageSnapshot(SLUG, { sourceId: 'default' });+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug: SLUG, request_id: id, expected_revision: current!.revision, content: person(text) } });+ await waitFor(async () => (await engine.executeRaw<{ state: string }>('SELECT state FROM persistence_requests WHERE request_id=$1::uuid', [id]))[0]?.state === 'committed', { timeoutMs: 20_000, label: 'put commits' });+ };+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');+ files(root, { 'people/alice-example': person('Works at acme-example; the Git edit.') });+ commitAll(root, 'git edit');+ let fired = false;+ installFaultHook(async (point, detail) => {+ if (point !== 'sync:mid_checkpoint' || detail.sourceId !== 'default' || fired) return;+ const [row] = await engine.executeRaw<{ pass_id: string | null }>("SELECT completed_keys->0->'telemetry'->>'pass_id' AS pass_id FROM op_checkpoints WHERE op='managed-sync-pass' AND completed_keys->0->>'source_id'='default'");+ if (!row?.pass_id || !Array.from({ length: 8 }, (_, b) => readPassBatch<{ pending?: { slug?: string } | null }>(`${PASS_BATCH_KEY_PREFIX}${row.pass_id}:${b}`)?.pending?.slug).includes(SLUG)) return;+ fired = true;+ await put('A database edit.');+ });+ try { expect((await performManagedSync(engine, OPTS)).held?.[0]?.code).toBe('concurrent_write'); } finally { installFaultHook(undefined); }+ files(root, Object.fromEntries(Array.from({ length: 8 }, (_, i) => [`people/clash${i}`, person('Slug conflict.', `slug: ${SLUG}\n`)])));+ commitAll(root, 'upstream adds conflicting slugs');+ await performManagedSync(engine, OPTS);+ return put;+ }++ test('passes that only re-hold a page edited before each of them build the streak, end abandoned, and status names the page with its ask', () => managedBrain(async ({ engine, ctx, root }) => {+ const put = await movingHold(engine, ctx as OperationContext, root);+ const inc = await incarnation(engine);+ const passes: Array<{ code: string; rewritten: boolean; moved: unknown; banked: boolean }> = [];+ for (let pass = 0; pass < 3; pass++) {+ await put(`A database edit before pass ${pass}.`);+ await stoppedSync(engine, 6);+ const telemetry = (await readPassRow(engine, 'default', inc))!.value.telemetry, hold: any = (await engine.executeRaw<{ r: unknown }>(+ "SELECT completed_keys->0 AS r FROM op_checkpoints WHERE op='sync-hold' AND completed_keys->0->>'path'=$1", [PATH]))[0]!.r;+ passes.push({ code: hold.code, rewritten: hold.run_id === telemetry.pass_id, moved: hold.meta.basis_moved, banked: await passBanked(engine, 'default', inc, telemetry) });+ }+ const status: any = await readSyncStatus(engine, 'default');+ expect([status.state, status.reason]).toEqual(['blocked', 'pass_abandoned']);+ expect(status.needs_human).toBe(true);+ const hold = status.holds.find((h: { path: string }) => h.path === PATH);+ expect(hold.needs_human).toBe(true);+ expect(status.needs_human_from).toEqual({ kind: 'hold', path: PATH, slug: SLUG, code: hold.code });+ expect(status.human_reason).toBe(hold.human_reason);+ expect(status.next.user_message).toContain(`needs a decision: page ${SLUG}`);+ // Each pass re-checked the page because it moved, and rewrote its hold marked so; none of them counts as saved work.+ expect(passes).toEqual(Array.from({ length: 3 }, () => ({ code: hold.code, rewritten: true, moved: true, banked: false })));+ }, { databaseUrl, setup }), 600_000);++ test('a later re-check that is not a move clears the mark, so that hold counts as saved work again', () => managedBrain(async ({ engine, ctx, root }) => {+ const put = await movingHold(engine, ctx as OperationContext, root);+ const inc = await incarnation(engine);+ await put('A second database edit.');+ await stoppedSync(engine, 6);+ expect((await theHold(engine)).meta).toMatchObject({ basis_moved: true });+ // The page stops moving; an older drift screen's version makes the hold due for one re-check that is not a move.+ await coordinated(engine, "UPDATE op_checkpoints SET completed_keys=jsonb_set(completed_keys,'{0,meta,drift_version}','1'::jsonb) WHERE op='sync-hold' AND completed_keys->0->>'path'=$1", [PATH]);+ await stoppedSync(engine, 6);+ const hold = await theHold(engine);+ expect(hold.meta.drift_version).toBe(FILE_DRIFT_VERSION);+ expect(hold.meta.basis_moved).toBeUndefined();+ expect(await passBanked(engine, 'default', inc, (await readPassRow(engine, 'default', inc))!.value.telemetry)).toBe(true);+ }, { databaseUrl, setup }), 600_000);+});