Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
test(worktree-refresh): move the #7679 state matrices 7c and 7d into their own files (#8025)
garrytane22ddf7f164e
diff --git a/docs/architecture/key-files/core-persistence-continued.md b/docs/architecture/key-files/core-persistence-continued.mdindex b303cb0d36..c5461ae539 100644--- a/docs/architecture/key-files/core-persistence-continued.md+++ b/docs/architecture/key-files/core-persistence-continued.md@@ -4,7 +4,7 @@Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.The #6188 fence check and fence repair live in [fence repair](core-fence-repair.md). Managed connector ingestion, connector item holds and the no-op kernel (checkpoint validation and the fault seams: [request indexes and fault seams](core-persistence-request-indexes.md)). Also the persistence verb types and errors, database-write and manual links, write attribution and loop-fact retirement; the managed purge of deleted pages is under [memory purge](memory-purge.md).-- `src/core/persistence/{worktree-refresh,worktree-refresh-schema}.ts` — `gbrain sources refresh` (F0), the one sanctioned managed Git writer. `persistence_worktree_refreshes` holds one row per refresh with a partial unique index (one active refresh per worktree; its 23505 maps to `refresh_in_progress`). `refreshWorktree` prechecks with no database state (managed, owner host, upstream, no active refresh, no recovery, no unexhausted managed-sync cursor, hardened `git fetch` bounded by `sources.refresh_fetch_timeout_ms`, ancestry, dirty paths intersected with the incoming diff), then: `draining` (inserted under `withTopologyLocks` + `lockTopologyRows`; admission in `journal.ts` refuses `worktree_refreshing` for every request with that `worktree_id`, and `assertManagedSyncAllowed` refuses a managed sync before discovery), poll until no queued/running/recovering request, no queued/running `git`/`withdrawal-mirror` effect and no recovery record remain (else `refresh_drain_timeout`, aborted), `fenced` (both native locks held, rechecked in one transaction; `refreshFenceClear` stops request and effect claims), HEAD verified, dirt rechecked, `git merge --ff-only`, HEAD verified again, then one transaction `merged` + upstream observation + `syncing`; native locks are released and the managed `--no-pull` sync runs per member (only `managed_sync_*` `submit_job` admissions pass during `syncing`); `completed` when every member's `last_commit` is the target or descends from it, or with `outcome.sync_blocked` when a member sync blocks. A per-worktree `refresh-<worktree>.lock` held for the whole run proves liveness: `resumeWorktreeRefreshes` (called from `recoverSourceTopologies` on every consumer topology scan) touches interrupted `draining`/`fenced`/`merged` rows whose lock is free and applies the section 1.5 table (draining or fenced at the old HEAD abort; HEAD at the verified target moves to `syncing`; anything else is `recovery_required`, which admission refuses with `refresh_recovery_required`). Admission also completes a `syncing` row whose members all reached the target. #7679: each row records its holder (`outcome.holder`: host id, hostname, pid, process start; `previous_holder` after a recovery adopts it). A row whose lock is free is orphaned: `resumeWorktreeRefreshes` (every consumer topology scan) also completes an orphaned `syncing` row once every member's `last_commit` reached the target or a descendant, and a plain `gbrain sources refresh` on the owner host finishes any orphaned non-`recovery_required` row through the `--resume` path (`adoptOrphanedRefresh` + `resumeHeld`) before its own refresh, reporting it as `recovered`. A holder recorded on another host is never adopted. `refreshHolderReport` (writer status `worktree_refreshes`, sync status `worktree_refresh`) names the holder, `liveness` (`live` / `orphaned` / `unknown` off the lock's host, from an instant probe of the refresh lock) and a fix naming `--resume`. `--resume` converges and runs the member syncs; `--abandon` lifts a fence only with HEAD back at the start commit. `gbrain sources writer status` lists active refreshes as `worktree_refreshes` (state, heads, the next command), which the refusal hints point at. Doctor `worktree_refresh_stuck` lists rows active over 15 minutes. CLI body: `src/commands/sources-refresh.ts` (#7685: `--json` prints a refusal's structured fix as the v1 `fix` RenderedAction when it has one, else the legacy command string; `refresh_in_progress` carries the writer-status read); while a resident PGLite owner (`gbrain serve`, stdio MCP) holds the database, `src/commands/sources-refresh-delegate.ts` runs the refresh inside it through the `writer_refresh` administration operation (trusted CLI registration only) and prints the same result or refusal. The consumer's idle probe applies `refreshFenceClear` to effects like the claims, so a fenced worktree's queued effects do not wake it. Tests: `test/persistence-worktree-refresh.test.ts`, `test/persistence-worktree-refresh-restart.test.ts` (child-process crash after merge), `test/worktree-refresh-orphan-7679.test.ts` (SIGKILL mid-syncing, live-holder no-steal), Postgres via `test/e2e/worktree-refresh-postgres.test.ts`, owner delegation in `test/persistence-sync-stdio-owner.test.ts`.+- `src/core/persistence/{worktree-refresh,worktree-refresh-schema}.ts` — `gbrain sources refresh` (F0), the one sanctioned managed Git writer. `persistence_worktree_refreshes` holds one row per refresh with a partial unique index (one active refresh per worktree; its 23505 maps to `refresh_in_progress`). `refreshWorktree` prechecks with no database state (managed, owner host, upstream, no active refresh, no recovery, no unexhausted managed-sync cursor, hardened `git fetch` bounded by `sources.refresh_fetch_timeout_ms`, ancestry, dirty paths intersected with the incoming diff), then: `draining` (inserted under `withTopologyLocks` + `lockTopologyRows`; admission in `journal.ts` refuses `worktree_refreshing` for every request with that `worktree_id`, and `assertManagedSyncAllowed` refuses a managed sync before discovery), poll until no queued/running/recovering request, no queued/running `git`/`withdrawal-mirror` effect and no recovery record remain (else `refresh_drain_timeout`, aborted), `fenced` (both native locks held, rechecked in one transaction; `refreshFenceClear` stops request and effect claims), HEAD verified, dirt rechecked, `git merge --ff-only`, HEAD verified again, then one transaction `merged` + upstream observation + `syncing`; native locks are released and the managed `--no-pull` sync runs per member (only `managed_sync_*` `submit_job` admissions pass during `syncing`); `completed` when every member's `last_commit` is the target or descends from it, or with `outcome.sync_blocked` when a member sync blocks. A per-worktree `refresh-<worktree>.lock` held for the whole run proves liveness: `resumeWorktreeRefreshes` (called from `recoverSourceTopologies` on every consumer topology scan) touches interrupted `draining`/`fenced`/`merged` rows whose lock is free and applies the section 1.5 table (draining or fenced at the old HEAD abort; HEAD at the verified target moves to `syncing`; anything else is `recovery_required`, which admission refuses with `refresh_recovery_required`). Admission also completes a `syncing` row whose members all reached the target. #7679: each row records its holder (`outcome.holder`: host id, hostname, pid, process start; `previous_holder` after a recovery adopts it). A row whose lock is free is orphaned: `resumeWorktreeRefreshes` (every consumer topology scan) also completes an orphaned `syncing` row once every member's `last_commit` reached the target or a descendant, and a plain `gbrain sources refresh` on the owner host finishes any orphaned non-`recovery_required` row through the `--resume` path (`adoptOrphanedRefresh` + `resumeHeld`) before its own refresh, reporting it as `recovered`. A holder recorded on another host is never adopted. `refreshHolderReport` (writer status `worktree_refreshes`, sync status `worktree_refresh`) names the holder, `liveness` (`live` / `orphaned` / `unknown` off the lock's host, from an instant probe of the refresh lock) and a fix naming `--resume`. `--resume` converges and runs the member syncs; `--abandon` lifts a fence only with HEAD back at the start commit. `gbrain sources writer status` lists active refreshes as `worktree_refreshes` (state, heads, the next command), which the refusal hints point at. Doctor `worktree_refresh_stuck` lists rows active over 15 minutes. CLI body: `src/commands/sources-refresh.ts` (#7685: `--json` prints a refusal's structured fix as the v1 `fix` RenderedAction when it has one, else the legacy command string; `refresh_in_progress` carries the writer-status read); while a resident PGLite owner (`gbrain serve`, stdio MCP) holds the database, `src/commands/sources-refresh-delegate.ts` runs the refresh inside it through the `writer_refresh` administration operation (trusted CLI registration only) and prints the same result or refusal. The consumer's idle probe applies `refreshFenceClear` to effects like the claims, so a fenced worktree's queued effects do not wake it. Tests: `test/persistence-worktree-refresh.test.ts`, `test/persistence-worktree-refresh-restart.test.ts` (child-process crash after merge), `test/worktree-refresh-orphan-7679.test.ts` (SIGKILL mid-syncing, live-holder no-steal), the #7679 state matrices `test/worktree-refresh-settle-matrix-7679.test.ts` (7c) and `test/worktree-refresh-unmoved-matrix-7679.test.ts` (7d), Postgres via `test/e2e/worktree-refresh-postgres.test.ts`, `test/e2e/worktree-refresh-settle-matrix-postgres.test.ts` and `test/e2e/worktree-refresh-unmoved-matrix-postgres.test.ts`, owner delegation in `test/persistence-sync-stdio-owner.test.ts`.- `src/core/persistence/git-gap.ts` + `src/commands/sources-git-gap.ts` + `src/commands/doctor/checks/db-ahead-of-git.ts` — git-canonical memory, wave G1 (`docs/designs/GIT_CANONICAL_MEMORY.md#the-git-gap-census`), read-only. `declaredDatabaseOnly` is the one predicate for pages that may not have a tracked file, tested in order: a read-only mirror source, a private page (`isPrivatePage`), an alias-private page (`isDerivationPrivate`), lineage-private (computed in SQL from `privatePagesFilterFragment`), `database_only_reason`, a `storage.db_only` prefix (`isSourceDbOnlySlug`, refusing an invalid gbrain.yml), a never-filed derive-phase page (page-prepare.ts `isNeverFiledDerivedPage`: no `source_path`/`source_uri` under `DERIVE_PHASE_DB_ONLY_DEFAULTS`), a path the checkout's Git ignores (`ignoredPaths`: filter-free `check-ignore -z --stdin`, tracked files never reported, exit 1 = none, null when Git cannot answer). The equality contract compares a committed blob with the page's export view (`readExportPage`, withdrawals overlaid; the census reads a batch's views with `readExportPages`, the same snapshots in two statements, #7617): an inline blob parsed as reconcile parses a canonical file, aligned with the stored type, subtype and #6429 stamp and compared by `reconcileCanonical` digest; a segmented page by the Git blob id of its streamed export. HEAD is read with hardened plumbing only (`readHeadTree`: one `ls-tree -r -l -z --full-tree` under the source's directory; `readBlobs`: `cat-file --batch` in calls of at most 64 MiB) and the working tree by hashing raw bytes (`worktreeBlobOid`), never `git status`/`git diff`, so no repository filter, textconv or fsmonitor runs. `gbrain sources git-gap <id> [--sample n] [--json]` (trusted local CLI, not an MCP op) puts every live page in one state (`equal`, `database_ahead`, `git_ahead` when the sync import record's `blob_oid`/`disk_blob_oid` differs from HEAD, else folded into `database_ahead` and counted in `database_ahead_without_import_record`, `declared_database_only` per class, `undeclared_database_only`, `worktree_only`, `unparseable`, `unsafe_path`), reports the working tree separately, and counts rows with no Git home (unfenced active facts, fenced facts on a missing page or a page without a committed file, links whose `link_source` is not in `DERIVED_LINK_SOURCES`, timeline rows on pages without a file), open `sync-hold` records by code and parked effect targets by kind and code. Each 256-page batch reads in its own read-only transaction. Doctor `db_ahead_of_git` runs the census per non-archived source whose directory is a Git checkout (others are only counted) under `GBRAIN_DOCTOR_GIT_GAP_TIMEOUT_MS` (default 10 s) and the entry deadline; it is always `ok` (informational) and lists totals only, a source the budget stopped or never reached as partial. Tests: `test/git-gap-census.test.ts` (PGLite and Postgres).- `src/core/persistence/git-declared.ts` + `src/core/persistence/git-canonical-schema.ts` (migration `git_canonical_declared`) — git-canonical memory, wave G7a: the declared database-only classes of a strict source (`sources.config.canonical = "git"`, default off). `strictPageClass` judges a page by its slug and prior file (`declaredPathClass`: `db_only_prefix` for a `storage.db_only` dir, `derive_folder` for a never-filed derive-phase page; G3's indexer reads the same function) and then its rendered content (`privacyClass`: `private`, `alias_private`, or `lineage_private` from the same SQL remote reads use, `privatePagesFilterFragment`, over the page's own values). `page-prepare.ts` `prepareFileTarget` asks it before it builds a strict file target: a declared page gets no target (no file, no door commit, no Git effect), an untracked ignored target is `git_ignored`, a page turning private gets the removal of the file it still has in the worktree or at HEAD, and a path whose `filter` attribute is `lfs` passes `assertLfsTarget` only with `storage.git_lfs` and a checkout that configures the LFS filter (`git_lfs_not_enabled`). `preparePrivateTransition` (page-prepare.ts) is that removal for other writers (GBRA-60). In `publishMutation` and `publishGroup`, `assertNoPrivateCommit` refuses a file target carrying a private page's bytes before anything is written (`git_private_commit_refused`; a group rolls back whole), and `declareStrictPublication` (after apply, before the door commit) refuses a declared page's file or an undeclared file-less page (`git_home_required`), stamps `pages.database_only_reason` and, for a privacy class, clears `source_path`/`source_uri` in the transaction of the door commit that removed the file. A managed import builds its own file target, so it meets only these publication checks. `assertStrictGitHome` is the refusal for writers with no file form yet (G4, G5). The managed writer guard (`writer-guard-schema.ts`) refuses rows of a strict source unless the writer declares `gbrain.git_protocol` (`withCoordinatedWrite` does), behind the `git_canonical_brain.strict_sources` counter kept by `sources` triggers (`git_protocol_required`, mapped in `publication-failure.ts`). The guard reads its state as the writing role and fails closed: a hidden or missing `persistence_brain` or `git_canonical_brain` row, or a source or page row hidden by row-level security, raises `writer_coordinator_required` (`managed_writer_guard:unreadable_state`), and the counter refuses a recount under RLS on `sources` (`test/writer-fence-fail-closed-7636.test.ts`). The census (`git-gap.ts`) adds `database_only_source` (a source with no checkout) and maps the stamped reasons to their own classes. Tests: `test/git-canonical-declared-g7a.test.ts` (both engines).diff --git a/test/e2e/worktree-refresh-settle-matrix-postgres.test.ts b/test/e2e/worktree-refresh-settle-matrix-postgres.test.tsnew file mode 100644index 0000000000..e12ff7f5ed--- /dev/null+++ b/test/e2e/worktree-refresh-settle-matrix-postgres.test.ts@@ -0,0 +1,4 @@+import { registerPostgresTests } from '../helpers/test-backends.ts';++// #7679 state matrix 7c of `gbrain sources refresh` on Postgres, split from worktree-refresh-postgres.test.ts (#8025).+await registerPostgresTests(() => import('../worktree-refresh-settle-matrix-7679.test.ts'));diff --git a/test/e2e/worktree-refresh-unmoved-matrix-postgres.test.ts b/test/e2e/worktree-refresh-unmoved-matrix-postgres.test.tsnew file mode 100644index 0000000000..273488c0de--- /dev/null+++ b/test/e2e/worktree-refresh-unmoved-matrix-postgres.test.ts@@ -0,0 +1,4 @@+import { registerPostgresTests } from '../helpers/test-backends.ts';++// #7679 state matrix 7d of `gbrain sources refresh` on Postgres, split from worktree-refresh-postgres.test.ts (#8025).+await registerPostgresTests(() => import('../worktree-refresh-unmoved-matrix-7679.test.ts'));diff --git a/test/persistence-worktree-refresh.test.ts b/test/persistence-worktree-refresh.test.tsindex 57c1756e35..21d510a86a 100644--- a/test/persistence-worktree-refresh.test.ts+++ b/test/persistence-worktree-refresh.test.ts@@ -35,7 +35,7 @@ import { runManagedSourceLifecycle } from '../src/core/persistence/source-lifecyimport { runPersistenceAdministration } from '../src/core/persistence/administration.ts';import { performManagedSync } from '../src/core/persistence/sync-run.ts';import { readSyncStatus } from '../src/core/persistence/sync-status.ts';-import { refreshLockPath, refreshWorktree, resumeWorktreeRefreshes } from '../src/core/persistence/worktree-refresh.ts';+import { refreshWorktree, resumeWorktreeRefreshes } from '../src/core/persistence/worktree-refresh.ts';import { commitAndPush } from '../src/core/git/commit-push.ts';import { gitConvergenceCheck } from '../src/commands/doctor/checks/git-convergence.ts';import { MANAGED_SYNC_PASS_OP, writePassRow, type ManagedSyncPassValue } from '../src/core/persistence/sync-pass-row.ts';@@ -373,168 +373,6 @@ test('7b. #7679: a plain refresh rolls back an orphan that died before merging,expect(await f.lastCommit(f.alpha)).toBe(target);}), 180_000);-test('7c. #7679 state matrix: an interrupted refresh of each kind, settled by the owner scan, a plain refresh and --resume', async () => {- // Value: protects=each kind of interrupted refresh row leaves the row section 1.5 names when the owner scan, a plain refresh (adoption) or- // --resume (resumeHeld) settles it; fails_when=a kind falls through to a sibling's path (an unverified HEAD adopted as the target, a syncing- // row with an unrelated HEAD completed or refreshed over, a dead drain left standing); why_new=the #7679 tests cover syncing orphans with HEAD at- // the target only; seam=none; no_claim=a real process death (the crash is a hook throw; the refresh lock is released as on exit).- type Kind = 'draining' | 'fenced_old' | 'fenced_target' | 'fenced_unrelated' | 'merged_target' | 'merged_unrelated' | 'syncing_at_target' | 'syncing_descendant'- | 'syncing_unrelated' | 'syncing_unrelated_synced';- type Action = 'owner_scan' | 'plain' | 'resume';- const kinds: Kind[] = ['draining', 'fenced_old', 'fenced_target', 'fenced_unrelated', 'merged_target', 'merged_unrelated', 'syncing_at_target', 'syncing_descendant',- 'syncing_unrelated', 'syncing_unrelated_synced'];- const stateOf = (kind: Kind) => kind.split('_')[0] as 'draining' | 'fenced' | 'merged' | 'syncing';- const crashAt = { draining: 'drained', fenced: 'fenced', merged: 'fenced', syncing: 'syncing' } as const;- // The owner scan's row and moved count; a merged row's bookkeeping leaves it syncing, and a syncing row with an unrelated HEAD waits for --resume.- const scanned: Record<Kind, [string, number]> = { draining: ['aborted', 1], fenced_old: ['aborted', 1], fenced_target: ['syncing', 1], fenced_unrelated: ['recovery_required', 1],- merged_target: ['syncing', 1], merged_unrelated: ['recovery_required', 1], syncing_at_target: ['completed', 1], syncing_descendant: ['syncing', 0], syncing_unrelated: ['syncing', 0], syncing_unrelated_synced: ['syncing', 0] };- const cells = kinds.flatMap(kind => (['owner_scan', 'plain', 'resume'] as Action[]).map(action => [kind, action] as [Kind, Action]));- for (const engine of engines) for (const [kind, action] of cells) await withRefreshFixture(engine, async f => {- const label = `${kind} x ${action}`;- const target = f.push('alpha/two.md', page('Alpha two', 'Upstream.'));- const old = f.head();- await expect(refreshWorktree(f.engine, f.alpha, { hooks: { boundary: point => { if (point === crashAt[stateOf(kind)]) throw new Error(`simulated crash at ${point}`); } } }))- .rejects.toThrow('simulated crash');- await disposePersistenceConsumer(f.engine);- const [orphan] = await f.refreshRows();- // The resident consumer's owner scan may already have settled it; restore what the crash left.- await f.engine.executeRaw(`UPDATE persistence_worktree_refreshes SET state=$2,completed_at=NULL WHERE id=$1::uuid`, [orphan.id, stateOf(kind)]);- if (kind === 'fenced_target' || kind === 'merged_target') git(f.root, 'merge', '-q', '--ff-only', target);- if (kind === 'syncing_descendant') commitFile(f.root, 'beta/local.md', page('Local', 'Committed on top of the target.'));- if (kind.includes('_unrelated')) { git(f.root, 'reset', '-q', '--hard', old); commitFile(f.root, 'beta/unrelated.md', page('Unrelated', 'Not the upstream commit.')); }- if (kind === 'syncing_at_target' || kind === 'syncing_unrelated_synced') for (const member of [f.alpha, f.beta]) await f.engine.transaction(tx => withCoordinatedWrite(tx, [member],- () => tx.executeRaw('UPDATE sources SET last_commit=$2 WHERE id=$1', [member, target]), TEST_WRITE_ATTRIBUTION));- const head = f.head();- const rows = async () => (await f.refreshRows()).map(r => r.state);- if (kind === 'draining') {- // Ship audit (Step 7). Value: protects=writer status tells an orphaned draining refresh's writers to --resume it, never to retry (only a live- // drain finishes on its own); fails_when=the draining branch of next stops checking liveness (master's wording); why_new=test 10 reads a live- // drain only; seam=none; no_claim=a real process death (the crash is a hook throw; the refresh lock is released as on exit).- const [status] = (await runPersistenceAdministration(f.engine, 'writer_status', { source_id: f.alpha }) as { worktree_refreshes: Array<Record<string, any>> }).worktree_refreshes;- expect(status, label).toMatchObject({ refresh_id: orphan.id, state: 'draining', liveness: 'orphaned', auto_recovery: 'next_refresh',- fix: { argv: ['gbrain', 'sources', 'refresh', f.alpha, '--resume'] }, next: `gbrain sources refresh ${f.alpha} --resume` });- }- if (action === 'owner_scan') {- const [state, moved] = scanned[kind];- expect(await resumeWorktreeRefreshes(f.engine), label).toBe(moved);- expect(await rows(), label).toEqual([state]);- expect(f.head(), label).toBe(head);- return;- }- const run = refreshWorktree(f.engine, f.alpha, action === 'resume' ? { resume: true } : {});- if (kind.includes('_unrelated')) {- await refusedWith(run, 'refresh_recovery_required');- expect(await rows(), label).toEqual(['recovery_required']);- expect(f.head(), label).toBe(head);- return;- }- if (kind === 'syncing_descendant' && action === 'plain') {- // The adoption completes the orphan at the descendant HEAD; the refresh itself then refuses the unpushed local commit.- await refusedWith(run, 'refresh_diverged');- expect(await rows(), label).toEqual(['completed']);- expect(f.head(), label).toBe(head);- expect(await f.lastCommit(f.alpha), label).toBe(head);- expect(await f.lastCommit(f.beta), label).toBe(head);- return;- }- const result = await run;- const rolledBack = kind === 'draining' || kind === 'fenced_old';- if (action === 'resume') {- expect(result, label).toMatchObject({ status: rolledBack ? 'abandoned' : 'completed', refresh_id: orphan.id });- expect(await rows(), label).toEqual([rolledBack ? 'aborted' : 'completed']);- expect(f.head(), label).toBe(head);- return;- }- if (rolledBack) {- expect(result, label).toMatchObject({ status: 'completed', target_head: target, recovered: { refresh_id: orphan.id, state: 'aborted', status: 'abandoned' } });- expect(await rows(), label).toEqual(['aborted', 'completed']);- } else {- expect(result, label).toMatchObject({ status: 'already_current', recovered: { refresh_id: orphan.id, state: 'completed', status: 'completed', previous_holder: { pid: process.pid } } });- expect(await rows(), label).toEqual(['completed']);- }- const synced = kind === 'syncing_descendant' ? head : target;- expect(f.head(), label).toBe(synced);- expect(await f.lastCommit(f.alpha), label).toBe(synced);- expect(await f.lastCommit(f.beta), label).toBe(synced);- });-}, 900_000);--test('7d. #7679 state matrix: an interrupted refresh that the unattended paths cannot verify, or that a live or remote holder owns, is never moved', async () => {- // Value: protects=a plain refresh leaves an orphan untouched (state, holder, previous_holder, HEAD) when git cannot read HEAD (refusing- // refresh_in_progress), when publication/topology recovery is pending or the worktree is not active (refusing refresh_recovery_required, the- // blocker that must clear first), when a live process holds the refresh lock, or when its holder is on another host; the owner scan aborts a dead- // drain whatever HEAD reads and leaves every other kind on an unreadable HEAD; fails_when=adoption runs recovery on a null HEAD, ahead of the- // recovery gate, against a live lock or another host's holder, or the scan escalates an unreadable HEAD; why_new=7c covers readable HEADs with- // free locks and no pending recovery only; seam=.git/HEAD overwritten with garbage, a recovering topology change row, the worktree state column,- // the refresh lock held in this process, a foreign holder host id; no_claim=the race where HEAD becomes unreadable between adoption's two reads.- type Kind = 'draining' | 'fenced_old' | 'fenced_target' | 'merged_target' | 'syncing_lagging' | 'syncing_at_target' | 'recovery_required';- type Action = 'plain_head_unreadable' | 'scan_head_unreadable' | 'plain_recovery_pending' | 'plain_worktree_inactive' | 'live_holder' | 'other_host_holder';- const kinds: Kind[] = ['draining', 'fenced_old', 'fenced_target', 'merged_target', 'syncing_lagging', 'syncing_at_target', 'recovery_required'];- const actions: Action[] = ['plain_head_unreadable', 'scan_head_unreadable', 'plain_recovery_pending', 'plain_worktree_inactive', 'live_holder', 'other_host_holder'];- const stateOf = (kind: Kind) => kind === 'recovery_required' ? 'recovery_required' : kind.split('_')[0];- const crashAt = (kind: Kind) => kind === 'draining' ? 'drained' : kind.startsWith('syncing') ? 'syncing' : 'fenced';- const cells = kinds.flatMap(kind => actions.map(action => [kind, action] as [Kind, Action]));- for (const engine of engines) for (const [kind, action] of cells) await withRefreshFixture(engine, async f => {- const label = `${kind} x ${action}`;- const target = f.push('alpha/two.md', page('Alpha two', 'Upstream.'));- await expect(refreshWorktree(f.engine, f.alpha, { hooks: { boundary: point => { if (point === crashAt(kind)) throw new Error(`simulated crash at ${point}`); } } }))- .rejects.toThrow('simulated crash');- await disposePersistenceConsumer(f.engine);- const [orphan] = await f.refreshRows();- const state = stateOf(kind);- await f.engine.executeRaw(`UPDATE persistence_worktree_refreshes SET state=$2,completed_at=NULL WHERE id=$1::uuid`, [orphan.id, state]);- if (kind === 'fenced_target' || kind === 'merged_target') git(f.root, 'merge', '-q', '--ff-only', target);- if (kind === 'syncing_at_target') for (const member of [f.alpha, f.beta]) await f.engine.transaction(tx => withCoordinatedWrite(tx, [member],- () => tx.executeRaw('UPDATE sources SET last_commit=$2 WHERE id=$1', [member, target]), TEST_WRITE_ATTRIBUTION));- const head = f.head();- const settled = async () => (await f.refreshRows()).map(r => [r.state, (r.outcome.holder as { pid?: number } | undefined)?.pid, r.outcome.previous_holder]);- const before = await settled();- const plainCode = kind === 'recovery_required' ? 'refresh_recovery_required' : 'refresh_in_progress';- const writerStatus = async () => (await runPersistenceAdministration(f.engine, 'writer_status', { source_id: f.alpha }) as { worktree_refreshes: Array<Record<string, any>> }).worktree_refreshes[0];- if (action === 'plain_recovery_pending' || action === 'plain_worktree_inactive') {- if (action === 'plain_recovery_pending') await f.engine.executeRaw(`INSERT INTO persistence_topology_changes(principal_id,request_id,digest,operation,source_id,worktree_ids,state,recovery)- VALUES($1::uuid,$2::uuid,'example','reclone',$3,ARRAY[$4::uuid],'recovering',$5::text::jsonb)`,- [randomUUID(), randomUUID(), f.alpha, f.worktreeId, JSON.stringify({ worktreeId: f.worktreeId })]);- else await f.engine.executeRaw(`UPDATE persistence_worktrees SET state='recovering' WHERE id=$1::uuid`, [f.worktreeId]);- await refusedWith(refreshWorktree(f.engine, f.alpha), 'refresh_recovery_required');- // Status does not promise that the next refresh finishes an orphan the recovery gate holds back.- if (kind !== 'recovery_required') expect(await writerStatus(), label).toMatchObject({ liveness: 'orphaned', auto_recovery: 'none',- fix: { argv: ['gbrain', 'sources', 'writer', 'status', '--source', f.alpha, '--json'] } });- } else if (action === 'live_holder') {- const lock = await tryAcquireNativeLock(refreshLockPath(f.worktreeId));- expect(lock, label).not.toBeNull();- try {- expect(await resumeWorktreeRefreshes(f.engine), label).toBe(0);- await refusedWith(refreshWorktree(f.engine, f.alpha), plainCode);- await refusedWith(refreshWorktree(f.engine, f.alpha, { resume: true }), 'refresh_in_progress');- expect(await writerStatus(), label).toMatchObject({ liveness: 'live', auto_recovery: 'none' });- } finally { await lock!.release(); }- } else if (action === 'other_host_holder') {- await f.engine.executeRaw(`UPDATE persistence_worktree_refreshes SET outcome=jsonb_set(outcome,'{holder,host_id}',to_jsonb($2::text)) WHERE id=$1::uuid`, [orphan.id, randomUUID()]);- await refusedWith(refreshWorktree(f.engine, f.alpha), plainCode);- if (kind !== 'recovery_required') expect(await writerStatus(), label).toMatchObject({ liveness: 'unknown', auto_recovery: 'unknown', fix: { actor: 'host_admin' },- next: `gbrain sources writer status --source ${f.alpha} --json` });- } else {- const headFile = join(f.root, '.git', 'HEAD');- const saved = readFileSync(headFile, 'utf8');- writeFileSync(headFile, 'not a ref\n');- try {- if (action === 'plain_head_unreadable') {- const refused = await refusedWith(refreshWorktree(f.engine, f.alpha), plainCode);- if (kind !== 'recovery_required') expect(refused.suggestion, label).toContain(`gbrain sources writer status --source ${f.alpha} --json`);- } else {- // The scan aborts a dead drain (it does not depend on HEAD) and leaves every other kind for a scan that can read HEAD.- expect(await resumeWorktreeRefreshes(f.engine), label).toBe(kind === 'draining' ? 1 : 0);- expect((await f.refreshRows()).map(r => r.state), label).toEqual([kind === 'draining' ? 'aborted' : state]);- }- } finally { writeFileSync(headFile, saved); }- if (action === 'scan_head_unreadable') { expect(f.head(), label).toBe(head); return; }- }- expect(await settled(), label).toEqual(before);- expect(f.head(), label).toBe(head);- });-}, 900_000);-test('7e. #7679: the owner scan leaves an orphaned syncing refresh while one member lags behind its target, and completes it once both reach it', () => each(async f => {// Ship audit (Step 7). Value: protects=the owner scan completes an orphaned syncing row only when every member reached the target, even when// the SQL prefilter admits it (a member past old_head but short of the target); fails_when=completeSyncedOrphan or membersAtTarget stopsdiff --git a/test/worktree-refresh-settle-matrix-7679.test.ts b/test/worktree-refresh-settle-matrix-7679.test.tsnew file mode 100644index 0000000000..28408622f3--- /dev/null+++ b/test/worktree-refresh-settle-matrix-7679.test.ts@@ -0,0 +1,130 @@+/**+ * #7679 state matrix 7c of `gbrain sources refresh`: an interrupted refresh row of each kind, settled by the owner scan, a+ * plain refresh and --resume. Split from persistence-worktree-refresh.test.ts with 7d+ * (worktree-refresh-unmoved-matrix-7679.test.ts) so each Postgres file fits run-e2e.sh's per-file cap (#8025). Postgres runs+ * through test/e2e/worktree-refresh-settle-matrix-postgres.test.ts.+ */+import { afterAll, beforeAll, expect, test } from 'bun:test';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { runPersistenceAdministration } from '../src/core/persistence/administration.ts';+import { refreshWorktree, resumeWorktreeRefreshes } from '../src/core/persistence/worktree-refresh.ts';+import { withCoordinatedWrite } from '../src/core/persistence/context.ts';+import { TEST_WRITE_ATTRIBUTION } from './helpers/write-attribution.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { testBackends } from './helpers/test-backends.ts';+import { commitFile, git, page, withRefreshFixture } from './helpers/worktree-refresh-fixture.ts';++const backends = testBackends();+const engines: BrainEngine[] = [];+let closePostgres: (() => Promise<void>) | undefined;++beforeAll(async () => {+ if (backends.includes('pglite')) {+ const engine = new PGLiteEngine();+ await engine.connect({}); await engine.initSchema(); engines.push(engine);+ }+ if (backends.includes('postgres')) {+ const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);+ engines.push(pg.engine); closePostgres = pg.close;+ }+}, 120_000);+afterAll(async () => {+ for (const engine of engines) { await disposePersistenceConsumer(engine); if (engine.kind === 'pglite') await engine.disconnect(); }+ await closePostgres?.();+});++const refusedWith = async (promise: Promise<unknown>, code: string) => {+ const error = await promise.then(() => null, e => e as { code?: string; suggestion?: string; message?: string; docs?: string });+ expect(error, `expected refusal ${code}`).not.toBeNull();+ expect(error!.code).toBe(code);+ return error!;+};++test('7c. #7679 state matrix: an interrupted refresh of each kind, settled by the owner scan, a plain refresh and --resume', async () => {+ // Value: protects=each kind of interrupted refresh row leaves the row section 1.5 names when the owner scan, a plain refresh (adoption) or+ // --resume (resumeHeld) settles it; fails_when=a kind falls through to a sibling's path (an unverified HEAD adopted as the target, a syncing+ // row with an unrelated HEAD completed or refreshed over, a dead drain left standing); why_new=the #7679 tests cover syncing orphans with HEAD at+ // the target only; seam=none; no_claim=a real process death (the crash is a hook throw; the refresh lock is released as on exit).+ type Kind = 'draining' | 'fenced_old' | 'fenced_target' | 'fenced_unrelated' | 'merged_target' | 'merged_unrelated' | 'syncing_at_target' | 'syncing_descendant'+ | 'syncing_unrelated' | 'syncing_unrelated_synced';+ type Action = 'owner_scan' | 'plain' | 'resume';+ const kinds: Kind[] = ['draining', 'fenced_old', 'fenced_target', 'fenced_unrelated', 'merged_target', 'merged_unrelated', 'syncing_at_target', 'syncing_descendant',+ 'syncing_unrelated', 'syncing_unrelated_synced'];+ const stateOf = (kind: Kind) => kind.split('_')[0] as 'draining' | 'fenced' | 'merged' | 'syncing';+ const crashAt = { draining: 'drained', fenced: 'fenced', merged: 'fenced', syncing: 'syncing' } as const;+ // The owner scan's row and moved count; a merged row's bookkeeping leaves it syncing, and a syncing row with an unrelated HEAD waits for --resume.+ const scanned: Record<Kind, [string, number]> = { draining: ['aborted', 1], fenced_old: ['aborted', 1], fenced_target: ['syncing', 1], fenced_unrelated: ['recovery_required', 1],+ merged_target: ['syncing', 1], merged_unrelated: ['recovery_required', 1], syncing_at_target: ['completed', 1], syncing_descendant: ['syncing', 0], syncing_unrelated: ['syncing', 0], syncing_unrelated_synced: ['syncing', 0] };+ const cells = kinds.flatMap(kind => (['owner_scan', 'plain', 'resume'] as Action[]).map(action => [kind, action] as [Kind, Action]));+ for (const engine of engines) for (const [kind, action] of cells) await withRefreshFixture(engine, async f => {+ const label = `${kind} x ${action}`;+ const target = f.push('alpha/two.md', page('Alpha two', 'Upstream.'));+ const old = f.head();+ await expect(refreshWorktree(f.engine, f.alpha, { hooks: { boundary: point => { if (point === crashAt[stateOf(kind)]) throw new Error(`simulated crash at ${point}`); } } }))+ .rejects.toThrow('simulated crash');+ await disposePersistenceConsumer(f.engine);+ const [orphan] = await f.refreshRows();+ // The resident consumer's owner scan may already have settled it; restore what the crash left.+ await f.engine.executeRaw(`UPDATE persistence_worktree_refreshes SET state=$2,completed_at=NULL WHERE id=$1::uuid`, [orphan.id, stateOf(kind)]);+ if (kind === 'fenced_target' || kind === 'merged_target') git(f.root, 'merge', '-q', '--ff-only', target);+ if (kind === 'syncing_descendant') commitFile(f.root, 'beta/local.md', page('Local', 'Committed on top of the target.'));+ if (kind.includes('_unrelated')) { git(f.root, 'reset', '-q', '--hard', old); commitFile(f.root, 'beta/unrelated.md', page('Unrelated', 'Not the upstream commit.')); }+ if (kind === 'syncing_at_target' || kind === 'syncing_unrelated_synced') for (const member of [f.alpha, f.beta]) await f.engine.transaction(tx => withCoordinatedWrite(tx, [member],+ () => tx.executeRaw('UPDATE sources SET last_commit=$2 WHERE id=$1', [member, target]), TEST_WRITE_ATTRIBUTION));+ const head = f.head();+ const rows = async () => (await f.refreshRows()).map(r => r.state);+ if (kind === 'draining') {+ // Ship audit (Step 7). Value: protects=writer status tells an orphaned draining refresh's writers to --resume it, never to retry (only a live+ // drain finishes on its own); fails_when=the draining branch of next stops checking liveness (master's wording); why_new=test 10 reads a live+ // drain only; seam=none; no_claim=a real process death (the crash is a hook throw; the refresh lock is released as on exit).+ const [status] = (await runPersistenceAdministration(f.engine, 'writer_status', { source_id: f.alpha }) as { worktree_refreshes: Array<Record<string, any>> }).worktree_refreshes;+ expect(status, label).toMatchObject({ refresh_id: orphan.id, state: 'draining', liveness: 'orphaned', auto_recovery: 'next_refresh',+ fix: { argv: ['gbrain', 'sources', 'refresh', f.alpha, '--resume'] }, next: `gbrain sources refresh ${f.alpha} --resume` });+ }+ if (action === 'owner_scan') {+ const [state, moved] = scanned[kind];+ expect(await resumeWorktreeRefreshes(f.engine), label).toBe(moved);+ expect(await rows(), label).toEqual([state]);+ expect(f.head(), label).toBe(head);+ return;+ }+ const run = refreshWorktree(f.engine, f.alpha, action === 'resume' ? { resume: true } : {});+ if (kind.includes('_unrelated')) {+ await refusedWith(run, 'refresh_recovery_required');+ expect(await rows(), label).toEqual(['recovery_required']);+ expect(f.head(), label).toBe(head);+ return;+ }+ if (kind === 'syncing_descendant' && action === 'plain') {+ // The adoption completes the orphan at the descendant HEAD; the refresh itself then refuses the unpushed local commit.+ await refusedWith(run, 'refresh_diverged');+ expect(await rows(), label).toEqual(['completed']);+ expect(f.head(), label).toBe(head);+ expect(await f.lastCommit(f.alpha), label).toBe(head);+ expect(await f.lastCommit(f.beta), label).toBe(head);+ return;+ }+ const result = await run;+ const rolledBack = kind === 'draining' || kind === 'fenced_old';+ if (action === 'resume') {+ expect(result, label).toMatchObject({ status: rolledBack ? 'abandoned' : 'completed', refresh_id: orphan.id });+ expect(await rows(), label).toEqual([rolledBack ? 'aborted' : 'completed']);+ expect(f.head(), label).toBe(head);+ return;+ }+ if (rolledBack) {+ expect(result, label).toMatchObject({ status: 'completed', target_head: target, recovered: { refresh_id: orphan.id, state: 'aborted', status: 'abandoned' } });+ expect(await rows(), label).toEqual(['aborted', 'completed']);+ } else {+ expect(result, label).toMatchObject({ status: 'already_current', recovered: { refresh_id: orphan.id, state: 'completed', status: 'completed', previous_holder: { pid: process.pid } } });+ expect(await rows(), label).toEqual(['completed']);+ }+ const synced = kind === 'syncing_descendant' ? head : target;+ expect(f.head(), label).toBe(synced);+ expect(await f.lastCommit(f.alpha), label).toBe(synced);+ expect(await f.lastCommit(f.beta), label).toBe(synced);+ });+}, 900_000);+diff --git a/test/worktree-refresh-unmoved-matrix-7679.test.ts b/test/worktree-refresh-unmoved-matrix-7679.test.tsnew file mode 100644index 0000000000..5d33474aad--- /dev/null+++ b/test/worktree-refresh-unmoved-matrix-7679.test.ts@@ -0,0 +1,123 @@+/**+ * #7679 state matrix 7d of `gbrain sources refresh`: an interrupted refresh that the unattended paths cannot verify, or+ * that a live or remote holder owns, is never moved. Split from persistence-worktree-refresh.test.ts with 7c+ * (worktree-refresh-settle-matrix-7679.test.ts) so each Postgres file fits run-e2e.sh's per-file cap (#8025). Postgres runs+ * through test/e2e/worktree-refresh-unmoved-matrix-postgres.test.ts.+ */+import { afterAll, beforeAll, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { readFileSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { tryAcquireNativeLock } from '../src/core/persistence/native-lock.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { runPersistenceAdministration } from '../src/core/persistence/administration.ts';+import { refreshLockPath, refreshWorktree, resumeWorktreeRefreshes } from '../src/core/persistence/worktree-refresh.ts';+import { withCoordinatedWrite } from '../src/core/persistence/context.ts';+import { TEST_WRITE_ATTRIBUTION } from './helpers/write-attribution.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { testBackends } from './helpers/test-backends.ts';+import { git, page, withRefreshFixture } from './helpers/worktree-refresh-fixture.ts';++const backends = testBackends();+const engines: BrainEngine[] = [];+let closePostgres: (() => Promise<void>) | undefined;++beforeAll(async () => {+ if (backends.includes('pglite')) {+ const engine = new PGLiteEngine();+ await engine.connect({}); await engine.initSchema(); engines.push(engine);+ }+ if (backends.includes('postgres')) {+ const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);+ engines.push(pg.engine); closePostgres = pg.close;+ }+}, 120_000);+afterAll(async () => {+ for (const engine of engines) { await disposePersistenceConsumer(engine); if (engine.kind === 'pglite') await engine.disconnect(); }+ await closePostgres?.();+});++const refusedWith = async (promise: Promise<unknown>, code: string) => {+ const error = await promise.then(() => null, e => e as { code?: string; suggestion?: string; message?: string; docs?: string });+ expect(error, `expected refusal ${code}`).not.toBeNull();+ expect(error!.code).toBe(code);+ return error!;+};++test('7d. #7679 state matrix: an interrupted refresh that the unattended paths cannot verify, or that a live or remote holder owns, is never moved', async () => {+ // Value: protects=a plain refresh leaves an orphan untouched (state, holder, previous_holder, HEAD) when git cannot read HEAD (refusing+ // refresh_in_progress), when publication/topology recovery is pending or the worktree is not active (refusing refresh_recovery_required, the+ // blocker that must clear first), when a live process holds the refresh lock, or when its holder is on another host; the owner scan aborts a dead+ // drain whatever HEAD reads and leaves every other kind on an unreadable HEAD; fails_when=adoption runs recovery on a null HEAD, ahead of the+ // recovery gate, against a live lock or another host's holder, or the scan escalates an unreadable HEAD; why_new=7c covers readable HEADs with+ // free locks and no pending recovery only; seam=.git/HEAD overwritten with garbage, a recovering topology change row, the worktree state column,+ // the refresh lock held in this process, a foreign holder host id; no_claim=the race where HEAD becomes unreadable between adoption's two reads.+ type Kind = 'draining' | 'fenced_old' | 'fenced_target' | 'merged_target' | 'syncing_lagging' | 'syncing_at_target' | 'recovery_required';+ type Action = 'plain_head_unreadable' | 'scan_head_unreadable' | 'plain_recovery_pending' | 'plain_worktree_inactive' | 'live_holder' | 'other_host_holder';+ const kinds: Kind[] = ['draining', 'fenced_old', 'fenced_target', 'merged_target', 'syncing_lagging', 'syncing_at_target', 'recovery_required'];+ const actions: Action[] = ['plain_head_unreadable', 'scan_head_unreadable', 'plain_recovery_pending', 'plain_worktree_inactive', 'live_holder', 'other_host_holder'];+ const stateOf = (kind: Kind) => kind === 'recovery_required' ? 'recovery_required' : kind.split('_')[0];+ const crashAt = (kind: Kind) => kind === 'draining' ? 'drained' : kind.startsWith('syncing') ? 'syncing' : 'fenced';+ const cells = kinds.flatMap(kind => actions.map(action => [kind, action] as [Kind, Action]));+ for (const engine of engines) for (const [kind, action] of cells) await withRefreshFixture(engine, async f => {+ const label = `${kind} x ${action}`;+ const target = f.push('alpha/two.md', page('Alpha two', 'Upstream.'));+ await expect(refreshWorktree(f.engine, f.alpha, { hooks: { boundary: point => { if (point === crashAt(kind)) throw new Error(`simulated crash at ${point}`); } } }))+ .rejects.toThrow('simulated crash');+ await disposePersistenceConsumer(f.engine);+ const [orphan] = await f.refreshRows();+ const state = stateOf(kind);+ await f.engine.executeRaw(`UPDATE persistence_worktree_refreshes SET state=$2,completed_at=NULL WHERE id=$1::uuid`, [orphan.id, state]);+ if (kind === 'fenced_target' || kind === 'merged_target') git(f.root, 'merge', '-q', '--ff-only', target);+ if (kind === 'syncing_at_target') for (const member of [f.alpha, f.beta]) await f.engine.transaction(tx => withCoordinatedWrite(tx, [member],+ () => tx.executeRaw('UPDATE sources SET last_commit=$2 WHERE id=$1', [member, target]), TEST_WRITE_ATTRIBUTION));+ const head = f.head();+ const settled = async () => (await f.refreshRows()).map(r => [r.state, (r.outcome.holder as { pid?: number } | undefined)?.pid, r.outcome.previous_holder]);+ const before = await settled();+ const plainCode = kind === 'recovery_required' ? 'refresh_recovery_required' : 'refresh_in_progress';+ const writerStatus = async () => (await runPersistenceAdministration(f.engine, 'writer_status', { source_id: f.alpha }) as { worktree_refreshes: Array<Record<string, any>> }).worktree_refreshes[0];+ if (action === 'plain_recovery_pending' || action === 'plain_worktree_inactive') {+ if (action === 'plain_recovery_pending') await f.engine.executeRaw(`INSERT INTO persistence_topology_changes(principal_id,request_id,digest,operation,source_id,worktree_ids,state,recovery)+ VALUES($1::uuid,$2::uuid,'example','reclone',$3,ARRAY[$4::uuid],'recovering',$5::text::jsonb)`,+ [randomUUID(), randomUUID(), f.alpha, f.worktreeId, JSON.stringify({ worktreeId: f.worktreeId })]);+ else await f.engine.executeRaw(`UPDATE persistence_worktrees SET state='recovering' WHERE id=$1::uuid`, [f.worktreeId]);+ await refusedWith(refreshWorktree(f.engine, f.alpha), 'refresh_recovery_required');+ // Status does not promise that the next refresh finishes an orphan the recovery gate holds back.+ if (kind !== 'recovery_required') expect(await writerStatus(), label).toMatchObject({ liveness: 'orphaned', auto_recovery: 'none',+ fix: { argv: ['gbrain', 'sources', 'writer', 'status', '--source', f.alpha, '--json'] } });+ } else if (action === 'live_holder') {+ const lock = await tryAcquireNativeLock(refreshLockPath(f.worktreeId));+ expect(lock, label).not.toBeNull();+ try {+ expect(await resumeWorktreeRefreshes(f.engine), label).toBe(0);+ await refusedWith(refreshWorktree(f.engine, f.alpha), plainCode);+ await refusedWith(refreshWorktree(f.engine, f.alpha, { resume: true }), 'refresh_in_progress');+ expect(await writerStatus(), label).toMatchObject({ liveness: 'live', auto_recovery: 'none' });+ } finally { await lock!.release(); }+ } else if (action === 'other_host_holder') {+ await f.engine.executeRaw(`UPDATE persistence_worktree_refreshes SET outcome=jsonb_set(outcome,'{holder,host_id}',to_jsonb($2::text)) WHERE id=$1::uuid`, [orphan.id, randomUUID()]);+ await refusedWith(refreshWorktree(f.engine, f.alpha), plainCode);+ if (kind !== 'recovery_required') expect(await writerStatus(), label).toMatchObject({ liveness: 'unknown', auto_recovery: 'unknown', fix: { actor: 'host_admin' },+ next: `gbrain sources writer status --source ${f.alpha} --json` });+ } else {+ const headFile = join(f.root, '.git', 'HEAD');+ const saved = readFileSync(headFile, 'utf8');+ writeFileSync(headFile, 'not a ref\n');+ try {+ if (action === 'plain_head_unreadable') {+ const refused = await refusedWith(refreshWorktree(f.engine, f.alpha), plainCode);+ if (kind !== 'recovery_required') expect(refused.suggestion, label).toContain(`gbrain sources writer status --source ${f.alpha} --json`);+ } else {+ // The scan aborts a dead drain (it does not depend on HEAD) and leaves every other kind for a scan that can read HEAD.+ expect(await resumeWorktreeRefreshes(f.engine), label).toBe(kind === 'draining' ? 1 : 0);+ expect((await f.refreshRows()).map(r => r.state), label).toEqual([kind === 'draining' ? 'aborted' : state]);+ }+ } finally { writeFileSync(headFile, saved); }+ if (action === 'scan_head_unreadable') { expect(f.head(), label).toBe(head); return; }+ }+ expect(await settled(), label).toEqual(before);+ expect(f.head(), label).toBe(head);+ });+}, 900_000);