Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: resume proven initialized retirement journals automatically
[._.]/ Adam Eivy14e971a4afa6
diff --git a/docs/features/branch-reconciliation.md b/docs/features/branch-reconciliation.mdindex 2778166dd6..9122f52716 100644--- a/docs/features/branch-reconciliation.md+++ b/docs/features/branch-reconciliation.md@@ -217,6 +217,57 @@ A move-based restore retry refuses when both originals are already restored andits working-move proof is present, preserving that evidence for this separatefinalization path instead of replacing it with no-op observations.+### Automatic continuation of initialized retirement journals++CoS discovers existing private initialized-retirement journals at startup evaluation+and every 30 minutes, independently of branch-reconcile scheduling or free agent+slots. It acts only while the daemon is running, unpaused, and CoS autonomy is+`execute`. Discovery never starts a retirement or grants authority from a journal+phase alone. The bounded drain handles at most two transactions per pass, rotates+past held work, and refreshes evidence on subsequent passes. It creates no agent,+worktree, or additional persistent queue.++Automatic continuation requires a valid original transaction, a bound preservation+manifest, the original exact reservation, and fresh service/native proofs. The+local tip must equal the merged PR head; the complete scoped comparison must be+`integrated` with no differences. The native classifier permits only the archived+residual-compression removal and root-provenance addition described below. Every+other source field must still match the preserved baseline. A successful review+is passed to the same purpose-specific operation with its exact digest; the+operation recomputes the proof before moving anything. After its internal scan,+the native helper waits for an exact, fresh server acknowledgement before rename;+pause, a lost executor, invalid replies, and parent exit grant nothing. The branch+receiver guard runs after preservation and fsyncs deletion intent before invoking+Git. Before lengthy branch verification, `branch-retirement-prepared` durably+binds the new exact reservation without granting deletion. Restart obtains a fresh+branch review under that same cut. The daemon never accepts+supersession. Branch retirement requires its own subsequent fresh review.++A plain `quarantined` journal whose reservation release was interrupted can finish+through this same proof without moving either original again. Durably confirmed+branch deletion can replay its receipt without another Git deletion; it still+requires archive, placement, absent-ref, ownership, and reservation checks. A+completed receipt or completed restore with an already released reservation needs+no further action and consumes no replay slot. A missing owner file inside an+existing cut is uncertainty, not a released reservation.++Unknown metadata, changed sources, superseded work, active owners, manual policy+holds, unsupported pre-move/restoration phases, and uncertain deletion results+remain held for explicit review. CoS does not restore originals automatically,+transfer a cut, infer ownership from age/PIDs, or turn ref absence into a deletion+receipt. Existing schedule status exposes `reconcileOutcomes[appId].initializedRetirements`+with each transaction, branch, observed journal digest, phase, and reason. These+observations are diagnostics, never cached approval. Invalid unbound journals are+reported in the global reconcile outcome. The journals and archives remain the+recovery evidence after restart. A crash between cut creation and publication of+its preparation record remains an explicit reservation-mismatch hold; the daemon+never adopts an unrecorded owner. A pause or interruption after deletion intent+was fsynced remains uncertain even when no receiver result was observed.++The narrow machine-verified continuation above does not require another runtime+agent or human approval. The explicit operations below remain available for+independent inspection, supersession decisions, and unsupported recovery cases.+### Completing an interrupted quarantineWhen the working directory has already moved into quarantine, restoring it@@ -280,6 +331,6 @@ age, timer or automatic retry clears this evidence. Reservation release atomicalmoves the intact owner into a durable private release record, preserving evidenceacross a crash without leaving an ownerless cut. Receipt replay keeps the originalarchive identity and timestamp so a newer checkout obligation cannot be satisfied-by replaying an older retirement. An interrupted transaction-requires explicit status and recovery; ambiguous identities remain held. The+by replaying an older retirement. Interrupted transactions outside the machine-verified continuation above+require explicit status and recovery; ambiguous identities remain held. Theprivate journal and all copies remain available after success or recovery.diff --git a/scripts/lib/initialized_checkout_archive.py b/scripts/lib/initialized_checkout_archive.pyindex 1f37623a15..4a89e442b6 100644--- a/scripts/lib/initialized_checkout_archive.py+++ b/scripts/lib/initialized_checkout_archive.py@@ -1350,6 +1350,29 @@ def verify_retired_quarantine(request):return verified_quarantine_resumption(request, completed=True, retired=True)+def await_mutation_permit(request):+ """Automatic callers recheck the live daemon gate after native verification.++ The backend lock stays held while waiting. EOF, missing/invalid replies and+ parent death grant nothing; explicit operations retain their old protocol.+ """+ token = request.get('mutationPermit')+ if token is None:+ return+ if not isinstance(token, str) or not re.fullmatch(r'[a-f0-9-]{36}', token):+ fail('mutation-permit-invalid')+ print(json.dumps({'mutationReady': token, 'part': 'administration'}), flush=True)+ reply = sys.stdin.readline(4097)+ try:+ accepted = json.loads(reply)+ except (ValueError, TypeError):+ fail('mutation-permit-missing')+ if (len(reply) > 4096 or not reply.endswith('\n') or not isinstance(accepted, dict)+ or set(accepted) != {'mutationPermit', 'allow'}+ or accepted['mutationPermit'] != token or accepted['allow'] is not True):+ fail('mutation-permit-invalid')++def resume_quarantine(request):verified = verify_quarantine_resumption(request)if verified['placements']['administration']['state'] == 'quarantine':@@ -1360,6 +1383,9 @@ def resume_quarantine(request):source, destination = Path(facts['privateGitDir']), root / 'quarantine/administration'safe_path(str(source))safe_path(str(destination.parent))+ if identity(source) != facts['identity']['administration'] or os.path.lexists(destination):+ fail('move-lease-or-destination-changed')+ await_mutation_permit(request)if identity(source) != facts['identity']['administration'] or os.path.lexists(destination):fail('move-lease-or-destination-changed')exclusive_rename(source, destination)diff --git a/scripts/lib/initialized_checkout_archive_test.py b/scripts/lib/initialized_checkout_archive_test.pyindex 5454a36fec..413647c069 100644--- a/scripts/lib/initialized_checkout_archive_test.py+++ b/scripts/lib/initialized_checkout_archive_test.py@@ -780,6 +780,43 @@ class NativeArchive(unittest.TestCase):self.assertFalse(self.work.exists())self.assertEqual((self.txn / 'manifest.json').read_bytes(), baseline)+ def test_automatic_quarantine_requires_fresh_exact_permit_after_native_verification(self):+ request = {**self.quarantine_resumption_fixture(), 'command': 'resume-quarantine',+ 'mutationPermit': '11111111-1111-1111-1111-111111111111'}+ saved = archive.load_transaction(request)[1]+ source = Path(saved['facts']['privateGitDir'])+ original = archive.identity(source)+ destination = self.txn / 'quarantine/administration'+ for reply in ('', '{}\n', json.dumps({'mutationPermit': request['mutationPermit'], 'allow': False}) + '\n',+ json.dumps({'mutationPermit': request['mutationPermit'], 'allow': 1}) + '\n'):+ with subprocess.Popen([sys.executable, str(Path(archive.__file__))], env=self.env,+ stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,+ text=True) as child:+ child.stdin.write(json.dumps(request) + '\n')+ child.stdin.flush()+ ready = json.loads(child.stdout.readline())+ self.assertEqual(ready, {'mutationReady': request['mutationPermit'], 'part': 'administration'})+ # This readiness record is emitted after full native verification,+ # while the exact original and backend lock remain in place.+ self.assertEqual(archive.identity(source), original)+ self.assertFalse(destination.exists())+ output, _ = child.communicate(reply, timeout=30)+ self.assertNotEqual(child.returncode, 0)+ self.assertFalse(json.loads(output)['ok'])+ self.assertEqual(archive.identity(source), original)+ self.assertFalse(destination.exists())+ with subprocess.Popen([sys.executable, str(Path(archive.__file__))], env=self.env,+ stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,+ text=True) as child:+ child.stdin.write(json.dumps(request) + '\n')+ child.stdin.flush()+ self.assertEqual(json.loads(child.stdout.readline())['mutationReady'], request['mutationPermit'])+ output, _ = child.communicate(json.dumps({'mutationPermit': request['mutationPermit'], 'allow': True}) + '\n', timeout=30)+ self.assertEqual(child.returncode, 0, output)+ self.assertTrue(json.loads(output)['moved'])+ self.assertEqual(archive.identity(destination), original)+ self.assertFalse(source.exists())+def test_quarantine_resumption_recovers_postmove_evidence_interruption_without_replaying_rename(self):request = self.quarantine_resumption_fixture()write = archive.private_jsondiff --git a/server/routes/cosScheduleRoutes.js b/server/routes/cosScheduleRoutes.jsindex 87a2efae9d..030c7beb66 100644--- a/server/routes/cosScheduleRoutes.js+++ b/server/routes/cosScheduleRoutes.js@@ -314,7 +314,8 @@ router.post('/schedule/branch-reconcile/checkouts/retire', asyncHandler(async (rres.json(result);}));-// Explicit operator capability; automatic cleanup retains its initialized-module guard.+// Initial retirement remains explicit. Only existing proof-bound journals can be+// continued by the deterministic daemon; ordinary cleanup keeps its module guard.router.post('/schedule/branch-reconcile/initialized-checkouts/review', asyncHandler(async (req, res) => {const { appId, prUrl, ...release } = validateRequest(heldCheckoutReviewBodySchema.strict(), req.body);const { reviewInitializedCheckout } = await import('../services/initializedCheckoutRecovery.js');diff --git a/server/services/cos.js b/server/services/cos.jsindex 0a2b986e2c..966464172a 100644--- a/server/services/cos.js+++ b/server/services/cos.js@@ -562,6 +562,20 @@ async function bootDaemon(state) {// set to `execute` and the daemon is not paused.// The handoff follow-up poll (prWatcher.followUpHandoffReceipt) shares the// gate and is cancelled with the daemon.+ // Existing retirement journals must recover even when branch-reconcile is+ // disabled or every agent slot is occupied. The service reuses its exact cut;+ // it never goes through normal agent/worktree admission.+ const { INITIALIZED_RECOVERY_INTERVAL_MS, sweepInitializedCheckoutRecoveries } = await import('./initializedCheckoutRecoverySweep.js');+ scheduleEvent({+ id: 'cos-initialized-retirement-recovery',+ type: 'interval',+ intervalMs: INITIALIZED_RECOVERY_INTERVAL_MS,+ handler: async () => {+ await sweepInitializedCheckoutRecoveries(pendingMergeSweepBlocked)+ .catch(error => console.warn(`⚠️ Initialized retirement recovery deferred: ${error.message}`));+ },+ metadata: { description: 'Resume verified initialized retirement journals' },+ });const { PENDING_MERGE_SWEEP_INTERVAL_MS, setHandoffFollowUpGate } = await import('./prWatcher.js');setHandoffFollowUpGate(pendingMergeSweepBlocked);scheduleEvent({@@ -661,6 +675,7 @@ async function teardownDaemon() {cancelEvent('cos-rehabilitation-check');cancelEvent('cos-improvement-check');cancelEvent('cos-pending-merge-sweep');+ cancelEvent('cos-initialized-retirement-recovery');(await import('./prWatcher.js')).cancelHandoffFollowUps();await unregisterJobSchedules();diff --git a/server/services/cos.startStop.test.js b/server/services/cos.startStop.test.jsindex 1de12667d2..5bc76054e3 100644--- a/server/services/cos.startStop.test.js+++ b/server/services/cos.startStop.test.js@@ -13,12 +13,22 @@import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';const scheduled = new Set();+const handlers = new Map();vi.mock('./eventScheduler.js', async (importOriginal) => ({...(await importOriginal()),- schedule: vi.fn(({ id }) => { scheduled.add(id); return id; }),+ schedule: vi.fn(({ id, handler }) => { scheduled.add(id); handlers.set(id, handler); return id; }),cancel: vi.fn((id) => scheduled.delete(id)),}));+const recoveryDrain = vi.hoisted(() => ({ calls: [] }));+vi.mock('./initializedCheckoutRecoverySweep.js', () => ({+ INITIALIZED_RECOVERY_INTERVAL_MS: 30 * 60_000,+ sweepInitializedCheckoutRecoveries: async blocked => {+ const reason = await blocked();+ if (!reason) recoveryDrain.calls.push('replay');+ return { skipped: reason };+ },+}));let persisted = { running: false, paused: false, agents: {}, config: { healthCheckIntervalMs: 900000 } };vi.mock('./cosState.js', async (importOriginal) => ({...(await importOriginal()),@@ -153,6 +163,25 @@ describe('CoS daemon start/stop lifecycle (#11860)', () => {expect(handoffFollowUp.gate).toBe(null);});+ it('arms journal recovery independently of agent slots and branch tasks, and enforces live autonomy on every timer', async () => {+ recoveryDrain.calls.length = 0;+ persisted = { ...persisted, config: { ...persisted.config, maxConcurrentAgents: 0, domainAutonomy: { cos: 'execute' } } };+ const starting = start();+ await waitFor(() => typeof releaseRecovery === 'function');+ releaseRecovery(); await starting;+ expect(cosEventIds()).toContain('cos-initialized-retirement-recovery');+ const tick = handlers.get('cos-initialized-retirement-recovery');+ await tick();+ expect(recoveryDrain.calls).toEqual(['replay']);+ persisted.paused = true; await tick();+ persisted.paused = false;+ for (const mode of ['off', 'dry-run']) { persisted.config.domainAutonomy.cos = mode; await tick(); }+ expect(recoveryDrain.calls).toEqual(['replay']);+ await stop(); await tick();+ expect(recoveryDrain.calls).toEqual(['replay']);+ expect(cosEventIds()).not.toContain('cos-initialized-retirement-recovery');+ });+it('a boot step that throws rolls the daemon back so the next start() succeeds', async () => {boot.cleanupOrphanedAgents = vi.fn(async () => { throw new Error('agent state unreadable'); });await expect(start()).rejects.toThrow('agent state unreadable');diff --git a/server/services/cosTaskGenerator.js b/server/services/cosTaskGenerator.jsindex 9d1822a8d5..13b9565594 100644--- a/server/services/cosTaskGenerator.js+++ b/server/services/cosTaskGenerator.js@@ -882,6 +882,14 @@ export async function evaluateTasks(options) {// meters agent RUNS — a deterministic merge spawns nothing and consumes none// of it — while the mode is the user's "may PortOS act on its own" switch.if (!paused && getDomainMode(state.config, 'cos') === 'execute') {+ const { sweepInitializedCheckoutRecoveries } = await import('./initializedCheckoutRecoverySweep.js');+ // This drain has its own cadence/single-flight guard and rechecks autonomy+ // between stages, including after native archive verification.+ void sweepInitializedCheckoutRecoveries(async () => {+ if (!isDaemonRunning()) return 'daemon-stopped';+ const current = await loadState();+ return current.paused ? 'daemon-paused' : getDomainMode(current.config, 'cos') !== 'execute' ? 'autonomy-not-execute' : null;+ }).catch(error => console.warn(`⚠️ Initialized retirement recovery deferred: ${error.message}`));const prWatcher = await import('./prWatcher.js');const sweep = await prWatcher.sweepPendingMergePrs();if (sweep.merged || sweep.escalated || sweep.timedOut) {diff --git a/server/services/initializedCheckoutNativePermit.test.js b/server/services/initializedCheckoutNativePermit.test.jsnew file mode 100644index 0000000000..d44c5ee941--- /dev/null+++ b/server/services/initializedCheckoutNativePermit.test.js@@ -0,0 +1,70 @@+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';+import { EventEmitter } from 'node:events';+import { PassThrough } from 'node:stream';+import { pinPlatform } from '../lib/testHelper.js';+vi.mock('../lib/childProcess.js', () => ({ spawn: vi.fn() }));+vi.mock('./cosClaimOwnershipState.js', () => ({ withClaimOwnershipLock: work => work() }));+import { spawn } from '../lib/childProcess.js';+import { _runInitializedCheckoutNative } from './initializedCheckoutRecovery.js';++let restore, child, sent;+beforeEach(() => {+ restore = pinPlatform('darwin');+ sent = [];+ child = Object.assign(new EventEmitter(), { stdout: new PassThrough(), stderr: new PassThrough(), stdin: new PassThrough(),+ kill: vi.fn(() => { queueMicrotask(() => child.emit('close', 1)); }) });+ child.stdin.on('data', value => sent.push(JSON.parse(value.toString())));+ spawn.mockReturnValue(child);+});+afterEach(() => { restore(); vi.clearAllMocks(); });++const request = { command: 'resume-quarantine', transactionDir: '/fixture/transaction' };+const ready = () => child.stdout.write(`${JSON.stringify({ mutationReady: sent[0].mutationPermit, part: 'administration' })}\n`);++describe('native mutation permission transport', () => {+ it('keeps stdin open until post-verification readiness and obtains a fresh gate decision before acknowledging', async () => {+ const gate = vi.fn(async () => {});+ const result = _runInitializedCheckoutNative(request, { beforeMutation: gate });+ expect(sent[0]).toMatchObject({ ...request, mutationPermit: expect.any(String) });+ expect(child.stdin.writableEnded).toBe(false);+ expect(gate).not.toHaveBeenCalled();+ ready();+ await vi.waitFor(() => expect(sent).toHaveLength(2));+ expect(sent[1]).toEqual({ mutationPermit: sent[0].mutationPermit, allow: true });+ child.stdout.write('{"ok":true,"moved":true}\n'); child.emit('close', 0);+ expect(await result).toEqual({ ok: true, moved: true });+ expect(gate).toHaveBeenCalledTimes(1);+ });+ it('sends no permit when the daemon pauses during native verification, and waits for child exit', async () => {+ const gate = vi.fn(async () => { throw new Error('daemon-paused'); });+ const result = _runInitializedCheckoutNative(request, { beforeMutation: gate });+ const rejected = expect(result).rejects.toThrow('daemon-paused');+ ready();+ await rejected;+ expect(sent).toHaveLength(1);+ expect(child.kill).toHaveBeenCalledWith('SIGTERM');+ });+ it('refuses a mismatched readiness token without calling the gate or acknowledging', async () => {+ const gate = vi.fn();+ const result = _runInitializedCheckoutNative(request, { beforeMutation: gate });+ const rejected = expect(result).rejects.toThrow('retirement-native-permit-invalid');+ child.stdout.write('{"mutationReady":"foreign","part":"administration"}\n');+ await rejected;+ expect(gate).not.toHaveBeenCalled(); expect(sent).toHaveLength(1);+ });+ it('does not acknowledge after child exit while the gate is still resolving', async () => {+ let finish;+ const result = _runInitializedCheckoutNative(request, { beforeMutation: () => new Promise(resolve => { finish = resolve; }) });+ const rejected = expect(result).rejects.toThrow();+ ready(); await vi.waitFor(() => expect(finish).toBeTypeOf('function'));+ child.emit('close', 1); await rejected;+ finish(); await Promise.resolve(); await Promise.resolve();+ expect(sent).toHaveLength(1);+ });+ it('preserves the explicit/manual single-request protocol without a mutation permit', async () => {+ const result = _runInitializedCheckoutNative(request);+ expect(sent).toEqual([request]); expect(child.stdin.writableEnded).toBe(true);+ child.stdout.write('{"ok":true,"moved":true}\n'); child.emit('close', 0);+ expect(await result).toMatchObject({ ok: true, moved: true });+ });+});diff --git a/server/services/initializedCheckoutRecovery.js b/server/services/initializedCheckoutRecovery.jsindex 501374c290..019169bc2c 100644--- a/server/services/initializedCheckoutRecovery.js+++ b/server/services/initializedCheckoutRecovery.js@@ -5,8 +5,8 @@*/import { spawn } from '../lib/childProcess.js';import { createHash, randomUUID } from 'node:crypto';-import { link, lstat, mkdir, open, readFile, readdir } from 'node:fs/promises';-import { join } from 'node:path';+import { link, lstat, mkdir, open, opendir, readFile, readdir } from 'node:fs/promises';+import { isAbsolute, join } from 'node:path';import { type as hostOsType } from 'node:os';import { fileURLToPath } from 'node:url';import { PATHS } from '../lib/paths.js';@@ -17,13 +17,16 @@ import { withClaimOwnershipLock } from './cosClaimOwnershipState.js';const helper = fileURLToPath(new URL('../../scripts/lib/initialized_checkout_archive.py', import.meta.url));const hash = value => createHash('sha256').update(JSON.stringify(value)).digest('hex');const fail = reason => { throw new Error(reason); };-const sameRelease = (a, b) => ['path', 'head', 'branch', 'locked', 'lockReason'].every(key => (a[key] ?? null) === (b[key] ?? null));+const sameRelease = (a = {}, b = {}) => ['path', 'head', 'branch', 'locked', 'lockReason'].every(key => (a[key] ?? null) === (b[key] ?? null));const transactionRoot = () => join(PATHS.cos, 'initialized-checkout-retirements');/** Native helper holds its own flock until the actual child exits. */-function runInitializedCheckoutNative(input, { keepOpen = false } = {}) {+export function _runInitializedCheckoutNative(input, { keepOpen = false, beforeMutation } = {}) {if (process.platform !== 'darwin') return Promise.reject(new Error('retirement-platform-unsupported'));return new Promise((resolve, reject) => {+ const permit = beforeMutation ? randomUUID() : null;+ if (permit && (keepOpen || input.command !== 'resume-quarantine')) return reject(new Error('retirement-native-permit-command-invalid'));+ let permissionRequested = false;const child = spawn('/usr/bin/python3', [helper], { stdio: ['pipe', 'pipe', 'pipe'], env: {PATH: process.env.PATH, HOME: process.env.HOME, TMPDIR: process.env.TMPDIR,PYTHONDONTWRITEBYTECODE: '1',@@ -56,6 +59,21 @@ function runInitializedCheckoutNative(input, { keepOpen = false } = {}) {child.stdout.on('data', data => {output += data;if (output.length > 4 * 1024 * 1024) return stop(new Error('retirement-native-output-limit'));+ if (permit && !permissionRequested && output.includes('\n')) {+ const newline = output.indexOf('\n');+ let ready;+ try { ready = JSON.parse(output.slice(0, newline)); } catch { return stop(new Error('retirement-native-permit-invalid')); }+ // A refused native verification may finish without requesting permission.+ if (ready.ok === false) return;+ if (ready.mutationReady !== permit || ready.part !== 'administration'+ || Object.keys(ready).length !== 2) return stop(new Error('retirement-native-permit-invalid'));+ permissionRequested = true;+ output = output.slice(newline + 1);+ Promise.resolve().then(beforeMutation).then(() => {+ if (closed || failure) return;+ child.stdin.end(`${JSON.stringify({ mutationPermit: permit, allow: true })}\n`);+ }).catch(stop);+ }if (keepOpen && !acquired && output.includes('\n')) {try {const result = JSON.parse(output.slice(0, output.indexOf('\n')));@@ -70,8 +88,8 @@ function runInitializedCheckoutNative(input, { keepOpen = false } = {}) {// Metadata and native diagnostics stay out of public logs.child.stderr.resume();child.stdin.on('error', error => { failure ||= error; });- child.stdin.write(`${JSON.stringify(input)}\n`);- if (!keepOpen) child.stdin.end();+ child.stdin.write(`${JSON.stringify(permit ? { ...input, mutationPermit: permit } : input)}\n`);+ if (!keepOpen && !permit) child.stdin.end();});}@@ -107,14 +125,27 @@ async function appendStep(directory, state, phase, patch = {}) {return next;}-async function readJournal(directory) {+async function readJournal(directory, { bounded = false, budget } = {}) {await privateDirectory(directory);- const files = (await readdir(directory)).filter(name => /^\d{6}\.json$/.test(name)).sort();- let previous = null;+ const names = [];+ if (bounded) {+ const listing = await opendir(directory);+ for await (const entry of listing) {+ if (names.length >= 1024) fail('retirement-journal-budget-exceeded');+ names.push(entry.name);+ }+ } else names.push(...await readdir(directory));+ const files = names.filter(name => /^\d{6}\.json$/.test(name)).sort();+ if (bounded && files.length > 128) fail('retirement-journal-budget-exceeded');+ let previous = null, bytes = 0;for (const [sequence, name] of files.entries()) {const info = await lstat(join(directory, name));if (!info.isFile() || info.isSymbolicLink() || (hostOsType() !== 'Windows_NT' && (info.mode & 0o777) !== 0o600)) fail('retirement-journal-unsafe');+ bytes += info.size;+ if (budget) { budget.remaining -= info.size; if (budget.remaining < 0) fail('retirement-discovery-budget-exceeded'); }+ if (bounded && (info.size > 16 * 1024 * 1024 || bytes > 64 * 1024 * 1024)) fail('retirement-journal-budget-exceeded');const row = JSON.parse(await readFile(join(directory, name), 'utf8'));+ if (bounded && previous && ['id', 'appId', 'repoPath', 'release', 'review'].some(key => JSON.stringify(row[key]) !== JSON.stringify(previous[key]))) fail('retirement-journal-identity-changed');if (row.version !== 1 || row.sequence !== sequence || row.previous !== (previous ? hash(previous) : null)) fail('retirement-journal-invalid');previous = row;}@@ -135,7 +166,7 @@ async function defaultContext(appId, release) {/** Factory allows workflow tests to replace native IO and forge/network boundaries. */export function _createInitializedCheckoutRecovery(deps = {}) {- const native = deps.native || runInitializedCheckoutNative;+ const native = deps.native || _runInitializedCheckoutNative;const root = deps.root || transactionRoot;const context = deps.context || defaultContext;const fenced = deps.fenced || withClaimOwnershipLock;@@ -252,8 +283,8 @@ export function _createInitializedCheckoutRecovery(deps = {}) {quarantineResumption: accepted.quarantineResumption };};const branchReview = async ({ directory, state, ctx }) => {- const transitioned = state.phase === 'quarantined-with-metadata-transitions';- if ((!transitioned && state.phase !== 'quarantined') || state.branchDeleted) fail('retirement-branch-phase-not-ready');+ const transitioned = Boolean(state.quarantineResumptionReview);+ if (!['quarantined', 'quarantined-with-metadata-transitions', 'branch-retirement-prepared'].includes(state.phase) || state.branchDeleted) fail('retirement-branch-phase-not-ready');const policy = await ensureOwner(ctx, state.release, false);const proof = await native(transitioned? { command: 'verify-completed-quarantine', ...quarantineProofInput({ directory, state, ctx }) }@@ -279,17 +310,24 @@ export function _createInitializedCheckoutRecovery(deps = {}) {const executor = await native({ command: 'lock', transactionDir: location(input.transactionId) }, { keepOpen: true });try {const bound = await loadBoundTransaction(input);- return await (await admission()).withCheckoutPublication(bound.ctx.repoPath, async () => {+ const gate = await admission();+ const work = async () => {executor.assertHeld();if (hash(await readJournal(bound.directory)) !== hash(bound.state)) fail('retirement-journal-changed');return { ok: true, review: await branchReview(bound) };- });+ };+ if (bound.state.phase === 'branch-retirement-prepared') {+ const reservation = await gate.readCheckoutReservation(bound.ctx.repoPath, bound.state.id);+ if (JSON.stringify(reservation) !== JSON.stringify(bound.state.reservation)) fail('retirement-reservation-changed');+ return await gate.withCheckoutReservation(reservation, work);+ }+ return await gate.withCheckoutPublication(bound.ctx.repoPath, work);} finally { await executor.release(); }}).catch(error => ({ ok: false, reason: error.message }));// A receiver success was fsynced before recording the schedule receipt. Replay// only those completed phases: this never issues a Git mutation or recreates// a ref, and still refuses a new branch/holder or changed archive.- const finishBranchReceipt = async ({ directory, ctx, state }) => {+ const finishBranchReceipt = async ({ directory, ctx, state }, beforeMutation) => {const executor = await native({ command: 'lock', transactionDir: directory }, { keepOpen: true });const gate = await admission();try {@@ -323,12 +361,14 @@ export function _createInitializedCheckoutRecovery(deps = {}) {if (reservation && JSON.stringify(reservation) !== JSON.stringify(state.reservation)) fail('retirement-reservation-changed');const finish = async () => {await verify();+ await beforeMutation();if (state.phase === 'branch-retired') {const record = deps.recordRetirement || (await import('./taskSchedule.js')).recordHeldCheckoutRetirement;await record(state.appId, { path: state.release.path, branch: state.release.branch,head: state.release.head, branchDeleted: true, archiveDir: directory });state = await appendStep(directory, state, 'retired', { receiptRecorded: true });}+ await beforeMutation();if (reservation) await gate.releaseCheckoutReservation(reservation);return { ok: true, transactionId: state.id, branchDeleted: true, receiptRecorded: true, archiveDir: directory };};@@ -336,11 +376,11 @@ export function _createInitializedCheckoutRecovery(deps = {}) {: await gate.withCheckoutPublication(ctx.repoPath, finish);} finally { await executor.release(); }};- const retireBranch = input => fenced(async () => {+ const retireBranch = (input, { beforeMutation = async () => {} } = {}) => fenced(async () => {const bound = await loadBoundTransaction(input);const { directory, ctx } = bound;let { state } = bound;- if (['branch-retired', 'retired'].includes(state.phase) && state.branchDeleted === true) return finishBranchReceipt(bound);+ if (['branch-retired', 'retired'].includes(state.phase) && state.branchDeleted === true) return finishBranchReceipt(bound, beforeMutation);// An interrupted receiver may have completed without its journal result.// It is an explicit uncertainty hold, never permission to recreate a ref.if (state.phase === 'branch-retirement-intent') fail('retirement-branch-deletion-uncertain');@@ -349,21 +389,54 @@ export function _createInitializedCheckoutRecovery(deps = {}) {try {if (hash(await readJournal(directory)) !== hash(state)) fail('retirement-journal-changed');executor.assertHeld();- const current = await branchReview(bound);- assertReview(current, input.review);- const reservation = await gate.reserveCheckoutRetirement(ctx.repoPath, state.id);+ let reservation, current;+ const prepared = state.phase === 'branch-retirement-prepared';+ if (prepared) {+ reservation = await gate.readCheckoutReservation(ctx.repoPath, state.id);+ if (JSON.stringify(reservation) !== JSON.stringify(state.reservation)) fail('retirement-reservation-changed');+ } else {+ current = await branchReview(bound);+ assertReview(current, input.review);+ await beforeMutation();+ reservation = await gate.reserveCheckoutRetirement(ctx.repoPath, state.id);+ // Durable preparation binds this new cut before lengthy verification or+ // branch archival. It grants no deletion and can be reviewed after restart.+ state = await appendStep(directory, state, 'branch-retirement-prepared', { reservation });+ }return await gate.withCheckoutReservation(reservation, async () => {- assertReview(await branchReview(bound), input.review);+ const fresh = await branchReview({ ...bound, state });+ if (prepared) assertReview(fresh, input.review);+ else {+ const evidence = value => { const { digest: _digest, journalDigest: _journal, ...rest } = value; return rest; };+ // Only our just-published journal transition changed. Every substantive+ // source/archive/forge/owner field must equal the accepted review.+ if (JSON.stringify(evidence(fresh)) !== JSON.stringify(evidence(current))) fail('retirement-evidence-changed');+ }+ current = fresh;executor.assertHeld();- state = await appendStep(directory, state, 'branch-retirement-intent', { reservation, branchReview: current });+ await beforeMutation();const remove = deps.deleteRef || (await import('./worktreeManager.js')).deleteCompletedLocalBranch;- const deleted = await remove(ctx.repoPath, state.release.branch, state.release.head);+ let receiverAuthorized = false;+ const deleted = await remove(ctx.repoPath, state.release.branch, state.release.head, { beforeMutation: async () => {+ executor.assertHeld();+ if (JSON.stringify(await ensureOwner(ctx, state.release, false)) !== JSON.stringify(current.policy)+ || JSON.stringify(await forge(ctx, state.release, state.review.forge.prUrl)) !== JSON.stringify(current.forge)) fail('retirement-evidence-changed');+ await beforeMutation();+ executor.assertHeld();+ state = await appendStep(directory, state, 'branch-retirement-intent', { reservation, branchReview: current });+ receiverAuthorized = true;+ await beforeMutation();+ executor.assertHeld();+ } });if (!deleted) fail('retirement-branch-delete-refused-or-uncertain');+ if (!receiverAuthorized) fail('retirement-branch-receiver-intent-missing');state = await appendStep(directory, state, 'branch-retired', { branchDeleted: true, branchRetainedReason: null });+ await beforeMutation();const record = deps.recordRetirement || (await import('./taskSchedule.js')).recordHeldCheckoutRetirement;await record(state.appId, { path: state.release.path, branch: state.release.branch,head: state.release.head, branchDeleted: true, archiveDir: directory });state = await appendStep(directory, state, 'retired', { receiptRecorded: true });+ await beforeMutation();await gate.releaseCheckoutReservation(reservation);return { ok: true, transactionId: state.id, branchDeleted: true, receiptRecorded: true, archiveDir: directory };});@@ -397,6 +470,7 @@ export function _createInitializedCheckoutRecovery(deps = {}) {...(unboundPreservation ? { recoveryRequired: true, recoveryAction: 'restore', reason: 'preservation-proof-unbound' } : {}),...(state.phase === 'branch-retirement-intent' ? { branchDeletion: 'uncertain', branchRef, recoveryRequired: true } : {}),...(state.phase === 'branch-retired' ? { receiptPending: true, recoveryRequired: true } : {}),+ ...(state.phase === 'branch-retirement-prepared' ? { recoveryRequired: true, recoveryAction: 'review-branch' } : {}),...(state.quarantineResumptionReview ? { quarantineResumption: state.quarantineResumptionReview.quarantineResumption,quarantineResumptionComplete: state.phase === 'quarantined-with-metadata-transitions' } : {}),...(state.metadataRestoreReview ? { metadataRecovery: state.metadataRestoreReview.metadataRecovery,@@ -566,7 +640,7 @@ export function _createInitializedCheckoutRecovery(deps = {}) {const quarantineReview = async (bound, reservation) => {const { state, directory, ctx } = bound;const allowed = ['quarantine-working-intent', 'quarantined-working', 'quarantine-administration-intent',- 'quarantined-administration', 'quarantine-resumption-approved', 'quarantine-resumption-administration-intent',+ 'quarantined-administration', 'quarantined', 'quarantine-resumption-approved', 'quarantine-resumption-administration-intent','quarantine-resumption-administration-complete', 'quarantined-with-metadata-transitions'];if (!allowed.includes(state.phase) || state.branchDeleted || !state.proof?.manifestSha256) fail('quarantine-resumption-phase-not-ready');if (JSON.stringify(reservation) !== JSON.stringify(state.reservation)) fail('quarantine-resumption-reservation-changed');@@ -587,9 +661,11 @@ export function _createInitializedCheckoutRecovery(deps = {}) {};const reviewQuarantineResumption = input => withMetadataRestore(input,async (bound, { reservation }) => ({ ok: true, review: await quarantineReview(bound, reservation) }));- const resumeQuarantine = input => withMetadataRestore(input, async (bound, { executor, gate, reservation }) => {+ const resumeQuarantine = (input, { beforeMutation: mutationGuard } = {}) => withMetadataRestore(input, async (bound, { executor, gate, reservation }) => {+ const beforeMutation = mutationGuard || (async () => {});const current = await quarantineReview(bound, reservation);assertReview(current, input.review);+ await beforeMutation();const { directory } = bound;let state = await appendStep(directory, bound.state, 'quarantine-resumption-approved',{ quarantineResumptionReview: current, quarantineResumptionObservation: null });@@ -598,13 +674,19 @@ export function _createInitializedCheckoutRecovery(deps = {}) {const fresh = await quarantineSafety({ ...bound, state }, administrationQuarantined);if (hash(fresh) !== current.safetyEvidenceDigest) fail('retirement-evidence-changed');executor.assertHeld();+ await beforeMutation();};await assertSafety(current.initialPlacements.administration === 'quarantine');await native({ command: 'verify-quarantine-resumption', ...proof });if (current.initialPlacements.administration === 'original') {await assertSafety(false);state = await appendStep(directory, state, 'quarantine-resumption-administration-intent');- const moved = await native({ command: 'resume-quarantine', ...proof });+ await beforeMutation();+ const moved = await native({ command: 'resume-quarantine', ...proof }, mutationGuard ? { beforeMutation: async () => {+ executor.assertHeld();+ await assertSafety(false);+ executor.assertHeld();+ } } : undefined);if (moved.moved !== true || !moved.postMoveObservation?.file|| !/^[a-f0-9]{64}$/.test(moved.postMoveObservation.sha256 || '')|| !/^[a-f0-9]{64}$/.test(moved.postMoveObservation.digest || '')) fail('quarantine-postmove-evidence-missing');@@ -616,6 +698,7 @@ export function _createInitializedCheckoutRecovery(deps = {}) {await assertSafety(true);state = await appendStep(directory, state, 'quarantined-with-metadata-transitions',{ branchDeleted: false, branchRetainedReason: 'quarantined-checkout-rollback-ref' });+ await beforeMutation();await gate.releaseCheckoutReservation(reservation);return { ok: true, transactionId: state.id, phase: state.phase, originalsQuarantined: true,branchDeleted: false, branchRetainedReason: state.branchRetainedReason, archiveDir: directory };@@ -700,7 +783,140 @@ export function _createInitializedCheckoutRecovery(deps = {}) {archivesRetained: true, metadataTransitionsPreserved: true, restoredFinalization: current.restoredFinalization,archiveDir: directory };}, true);- return { review, retire, status, restore, verifyArchive, reviewBranch, retireBranch, reviewMetadataRestore, restoreMetadata,+ const recoveryBinding = async (id, expectedDigest, budget) => {+ const directory = location(id);+ const state = await readJournal(directory, { bounded: true, budget });+ const original = state.review;+ const unsigned = { ...original }; delete unsigned.digest;+ if (state.id !== id || typeof state.appId !== 'string' || !state.appId+ || !isAbsolute(state.repoPath || '') || !isAbsolute(state.release?.path || '')+ || typeof state.release?.branch !== 'string' || !state.release.branch+ || !/^[a-f0-9]{40}(?:[a-f0-9]{24})?$/.test(state.release?.head || '')+ || original?.mode !== 'initialized-private-modules' || original.digest !== hash(unsigned)+ || !sameRelease(original.release, state.release)) fail('retirement-journal-binding-invalid');+ const ctx = await context(state.appId);+ if (ctx.repoPath !== state.repoPath) fail('retirement-transaction-app-mismatch');+ if (expectedDigest && hash(state) !== expectedDigest) fail('retirement-journal-changed');+ return { directory, state, ctx };+ };+ const completedWithoutCut = async ({ state, ctx }) => {+ const complete = (state.phase === 'retired' && state.branchDeleted === true && state.receiptRecorded === true)+ || ['restored', 'restored-with-metadata-transitions', 'restored-with-reviewed-finalization'].includes(state.phase);+ if (!complete) return false;+ const gate = await admission();+ const cut = await gate.readCheckoutReservation(ctx.repoPath, state.id).catch(async error => {+ if (error.code === 'CHECKOUT_RETIREMENT_RESERVED') {+ // A durably completed historical transaction must not consume replay+ // slots merely because a different transaction now owns this repo.+ // Validate and re-read that owner; never enter or release its cut.+ const { directory } = await gate._checkoutAdmission(ctx.repoPath);+ const ownerPath = join(directory, 'cut', 'owner.json');+ const info = await lstat(ownerPath);+ if (!info.isFile() || info.isSymbolicLink() || info.size > 4096+ || (hostOsType() !== 'Windows_NT' && (info.mode & 0o777) !== 0o600)) fail('retirement-reservation-owner-invalid');+ const owner = JSON.parse(await readFile(ownerPath, 'utf8'));+ if (owner.version !== 1 || !/^[a-f0-9-]{36}$/.test(owner.id || '') || owner.id === state.id+ || owner.generation !== owner.id || owner.kind !== 'snapshot' || !Number.isInteger(owner.pid)+ || typeof owner.startedAt !== 'string' || !Number.isFinite(Date.parse(owner.startedAt))) fail('retirement-reservation-owner-invalid');+ const fresh = await gate.readCheckoutReservation(ctx.repoPath, owner.id);+ if (JSON.stringify(fresh.owner) !== JSON.stringify(owner)) fail('retirement-reservation-changed');+ return null;+ }+ if (error.code !== 'ENOENT') throw error;+ // Missing owner.json inside an extant cut is uncertainty, not release.+ const { directory } = await gate._checkoutAdmission(ctx.repoPath);+ const exists = await lstat(join(directory, 'cut')).catch(readError => {+ if (readError.code === 'ENOENT') return null;+ throw readError;+ });+ if (exists) fail('retirement-reservation-owner-missing');+ return null;+ });+ return cut === null;+ };+ // Discovery reads only journals; expensive native/forge checks happen on a+ // bounded cadence in the daemon drain. A journal is a locator, never a grant.+ const discover = async () => {+ const rows = [];+ const exists = await lstat(root()).catch(error => { if (error.code === 'ENOENT') return null; throw error; });+ if (!exists) return { ok: true, transactions: [] };+ await privateDirectory(root());+ const entries = await opendir(root());+ const budget = { remaining: 256 * 1024 * 1024 };+ let count = 0;+ for await (const entry of entries) {+ if (++count > 1000) return { ok: false, reason: 'retirement-discovery-budget-exceeded', transactions: [] };+ if (!/^[a-f0-9-]{36}$/.test(entry.name)) {+ rows.push({ transactionId: null, reason: 'retirement-journal-name-invalid' });+ continue;+ }+ const row = await recoveryBinding(entry.name, undefined, budget).then(async bound => {+ const { state } = bound;+ const settled = await completedWithoutCut(bound).catch(() => false);+ return { transactionId: state.id, appId: state.appId, repoPath: state.repoPath,+ phase: state.phase, journalDigest: hash(state), release: state.release, settled };+ }).catch(error => ({ transactionId: entry.name, reason: error.message }));+ if (row.reason === 'retirement-discovery-budget-exceeded') return { ok: false, reason: row.reason, transactions: [] };+ rows.push(row);+ }+ return { ok: true, transactions: rows.sort((a, b) => String(a.transactionId).localeCompare(String(b.transactionId))) };+ };+ const automaticReview = value => {+ if (!value || value.release?.head !== value.forge?.prHead) fail('automatic-retirement-exact-pr-head-required');+ if (value.comparison?.disposition !== 'integrated' || value.comparison.difference !== '') fail('automatic-retirement-independent-review-required');+ };+ // Only the deterministic daemon calls this continuation. It never starts a+ // transaction, restores originals, adopts another cut, or accepts supersession.+ const recoverAutomatically = async (input, { blocked = async () => 'automatic-retirement-gate-required' } = {}) => {+ const permitted = async () => { const reason = await blocked(); if (reason) fail(reason); };+ const bound = await recoveryBinding(input.transactionId, input.journalDigest);+ const { state, ctx } = bound;+ if (state.appId !== input.appId) fail('retirement-transaction-app-mismatch');+ await permitted();+ if (await completedWithoutCut(bound)) return { ok: true, alreadyComplete: true, phase: state.phase, transactionId: state.id };+ if (!/^[a-f0-9]{64}$/.test(state.proof?.manifestSha256 || '')) fail('preservation-proof-unbound');+ if (state.phase === 'branch-retirement-intent') fail('retirement-branch-deletion-uncertain');+ if (['branch-retired', 'retired'].includes(state.phase) && state.branchDeleted === true) {+ // Confirmed deletion is already authorized and durable. A changed default+ // branch cannot prevent recording its receipt; this path never deletes.+ return retireBranch(input, { beforeMutation: permitted });+ }+ const finishBranch = async current => {+ await permitted();+ const reviewed = await reviewBranch(current);+ if (!reviewed.ok) return reviewed;+ automaticReview(reviewed.review);+ await permitted();+ return retireBranch({ ...current, review: { digest: reviewed.review.digest } }, { beforeMutation: permitted });+ };+ if (state.phase === 'branch-retirement-prepared') return finishBranch(input);+ const resumable = ['quarantine-working-intent', 'quarantined-working', 'quarantine-administration-intent',+ 'quarantined-administration', 'quarantined', 'quarantine-resumption-approved',+ 'quarantine-resumption-administration-intent', 'quarantine-resumption-administration-complete',+ 'quarantined-with-metadata-transitions'];+ if (!resumable.includes(state.phase) || state.branchDeleted) fail('automatic-retirement-phase-requires-review');+ const gate = await admission();+ const reservation = await gate.readCheckoutReservation(ctx.repoPath, state.id).catch(error => {+ if (error.code === 'ENOENT' && ['quarantined', 'quarantined-with-metadata-transitions'].includes(state.phase)) return null;+ throw error;+ });+ let current = input;+ if (reservation) {+ if (JSON.stringify(reservation) !== JSON.stringify(state.reservation)) fail('retirement-reservation-changed');+ const reviewed = await reviewQuarantineResumption(current);+ if (!reviewed.ok) return reviewed;+ automaticReview(reviewed.review);+ await permitted();+ const resumed = await resumeQuarantine({ ...current, review: { digest: reviewed.review.digest } }, { beforeMutation: permitted });+ if (!resumed.ok) return resumed;+ const fresh = await recoveryBinding(state.id);+ current = { ...current, journalDigest: hash(fresh.state) };+ }+ return finishBranch(current);+ };+ const automatic = async (input, options) => recoverAutomatically(input, options)+ .catch(error => ({ ok: false, reason: error.message, transactionId: input.transactionId, recoveryRequired: true }));+ return { discover, recoverAutomatically: automatic, review, retire, status, restore, verifyArchive, reviewBranch, retireBranch, reviewMetadataRestore, restoreMetadata,reviewRestoredFinalization, finalizeRestored, reviewQuarantineResumption, resumeQuarantine };}@@ -721,3 +937,6 @@ export const finalizeInitializedCheckoutRestored = recovery.finalizeRestored;export const reviewInitializedCheckoutQuarantineResumption = recovery.reviewQuarantineResumption;export const resumeInitializedCheckoutQuarantine = recovery.resumeQuarantine;++export const discoverInitializedCheckoutRecoveries = recovery.discover;+export const recoverInitializedCheckoutAutomatically = recovery.recoverAutomatically;diff --git a/server/services/initializedCheckoutRecovery.test.js b/server/services/initializedCheckoutRecovery.test.jsindex 1a8c9b3e1f..4527fafbf3 100644--- a/server/services/initializedCheckoutRecovery.test.js+++ b/server/services/initializedCheckoutRecovery.test.js@@ -1,6 +1,6 @@import { pinPlatform } from '../lib/testHelper.js';import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';-import { mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises';+import { mkdir, mkdtemp, open, readFile, readdir, rm, writeFile } from 'node:fs/promises';import { join } from 'node:path';import { tmpdir } from 'node:os';import { createHash } from 'node:crypto';@@ -71,10 +71,11 @@ beforeEach(async () => {fixture = { cut: null, facts, snapshot, headExists: true, placement: { working: 'original', administration: 'original' }, failPart: null,metadataToken: 'f'.repeat(64), finalizationToken: '7'.repeat(64), quarantineToken: '6'.repeat(64) };const admission = {+ _checkoutAdmission: async () => ({ directory: join(root, 'admission') }),withCheckoutPublication: async (_repo, work) => { if (fixture.cut) throw new Error('checkout-retirement-reserved'); return work(); },reserveCheckoutRetirement: vi.fn(async (_repo, id) => {if (fixture.cut) throw new Error('checkout-retirement-reserved');- fixture.cut = { owner: { id }, directory: '/fixture/admission' }; return fixture.cut;+ fixture.cut = { owner: { version: 1, id, generation: id, kind: 'snapshot', pid: process.pid, startedAt: new Date().toISOString() }, directory: '/fixture/admission' }; return fixture.cut;}),withCheckoutReservation: async (_cut, work) => work(),readCheckoutReservation: async () => {@@ -161,11 +162,11 @@ beforeEach(async () => {ownership: vi.fn(async () => ({ ok: true, policy: null })),forge: async () => ({ ...fixture.facts }),comparison: async (_ctx, _release, _url, _snapshot, policy, forge) => ({- forge, policy, comparison: { disposition: fixture.superseded ? 'review-supersession' : 'ancestor' },+ release: _release, forge, policy, comparison: { disposition: fixture.superseded ? 'review-supersession' : 'integrated', difference: fixture.superseded ? 'review the changed work' : '' },}),listWorktrees: async () => fixture.newHolder ? [{ path: '/fixture/new', branch: release.branch }] : [],execGit: vi.fn(async () => ({ exitCode: fixture.headExists ? 0 : 1, stdout: fixture.headExists ? release.head : '' })),- deleteRef: vi.fn(async () => { fixture.headExists = false; return true; }),+ deleteRef: vi.fn(async (_repo, _branch, _head, options) => { await options.beforeMutation(); fixture.headExists = false; return true; }),recordRetirement: vi.fn(async () => {}),};fixture.deps = deps; fixture.service = _createInitializedCheckoutRecovery(deps);@@ -618,7 +619,7 @@ describe('initialized checkout transaction', () => {it('never promotes absent-ref quarantine deletion intent to success', async () => {const input = await resumedQuarantine();const prepared = await fixture.service.reviewBranch(input);- fixture.deps.deleteRef.mockImplementationOnce(async () => { fixture.headExists = false; throw new Error('receiver-outcome-lost'); });+ fixture.deps.deleteRef.mockImplementationOnce(async (_repo, _branch, _head, options) => { await options.beforeMutation(); fixture.headExists = false; throw new Error('receiver-outcome-lost'); });await fixture.service.retireBranch({ ...input, review: { digest: prepared.review.digest } });const status = await fixture.service.status(input);expect(status.phase).toBe('branch-retirement-intent');@@ -869,7 +870,7 @@ describe('initialized checkout transaction', () => {const input = await quarantinedReview();const result = await fixture.service.retireBranch(input);expect(result).toMatchObject({ ok: true, branchDeleted: true, receiptRecorded: true });- expect(fixture.deps.deleteRef).toHaveBeenCalledWith('/fixture/repo', release.branch, release.head);+ expect(fixture.deps.deleteRef).toHaveBeenCalledWith('/fixture/repo', release.branch, release.head, { beforeMutation: expect.any(Function) });expect(fixture.deps.recordRetirement).toHaveBeenCalledWith('app', { path: release.path, branch: release.branch,head: release.head, branchDeleted: true, archiveDir: join(root, input.transactionId) });expect(fixture.cut).toBeNull();@@ -893,7 +894,7 @@ describe('initialized checkout transaction', () => {});it('retains the delete intent and reports uncertainty after deletion without a journal outcome', async () => {const input = await quarantinedReview();- fixture.deps.deleteRef.mockImplementation(async () => { fixture.headExists = false; throw new Error('receiver-result-lost'); });+ fixture.deps.deleteRef.mockImplementation(async (_repo, _branch, _head, options) => { await options.beforeMutation(); fixture.headExists = false; throw new Error('receiver-result-lost'); });expect(await fixture.service.retireBranch(input)).toMatchObject({ ok: false, reason: 'receiver-result-lost', recoveryRequired: true });const restarted = _createInitializedCheckoutRecovery(fixture.deps);const status = await restarted.status(input);@@ -1069,3 +1070,252 @@ describe('initialized checkout transaction', () => {await expect(fixture.service.status({ appId: 'app', transactionId: result.transactionId })).rejects.toThrow('retirement-journal-invalid');});});++// These exercise persisted journal discovery through the actual service+// continuation, not a synthetic phase-to-command mapping.+describe('automatic initialized retirement continuation', () => {+ const automaticInput = async () => {+ const result = await fixture.service.discover();+ expect(result.ok).toBe(true);+ expect(result.transactions).toHaveLength(1);+ const row = result.transactions[0];+ expect(row.reason).toBeUndefined();+ return { appId: row.appId, transactionId: row.transactionId, journalDigest: row.journalDigest };+ };+ const permitted = { blocked: async () => null };+ const interrupted = async () => {+ fixture.failPart = 'administration';+ expect(await fixture.service.retire(await reviewed())).toMatchObject({ ok: false, recoveryRequired: true });+ fixture.failPart = null;+ fixture.service = _createInitializedCheckoutRecovery(fixture.deps);+ return automaticInput();+ };+ it('discovers after restart and finishes archive-backed quarantine plus a separate fresh branch retirement', async () => {+ const input = await interrupted();+ const originalReservation = structuredClone(fixture.cut);+ await expect(fixture.deps.admission.withCheckoutPublication('/fixture/repo', () => {}))+ .rejects.toThrow('checkout-retirement-reserved');+ const before = fixture.deps.native.mock.calls.length;+ expect(await fixture.service.recoverAutomatically(input, permitted))+ .toMatchObject({ ok: true, branchDeleted: true, receiptRecorded: true });+ expect(fixture.deps.admission.releaseCheckoutReservation.mock.calls[0][0]).toEqual(originalReservation);+ expect(fixture.placement).toEqual({ working: 'quarantined', administration: 'quarantined' });+ expect(fixture.deps.native.mock.calls.slice(before).some(([arg]) => ['inspect', 'preserve', 'move'].includes(arg.command))).toBe(false);+ expect(fixture.deps.deleteRef).toHaveBeenCalledTimes(1);+ expect(fixture.deps.recordRetirement).toHaveBeenCalledTimes(1);+ expect(fixture.cut).toBeNull();+ });+ it('finishes the plain quarantined journal left before releasing its exact reservation', async () => {+ fixture.deps.admission.releaseCheckoutReservation.mockRejectedValueOnce(new Error('release-interrupted'));+ const result = await fixture.service.retire(await reviewed());+ expect(result).toMatchObject({ ok: false, reason: 'release-interrupted' });+ expect((await fixture.service.status({ appId: 'app', transactionId: result.transactionId })).phase).toBe('quarantined');+ const moves = fixture.deps.native.mock.calls.filter(([arg]) => arg.command === 'move').length;+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted)).toMatchObject({ ok: true, receiptRecorded: true });+ expect(fixture.deps.native.mock.calls.filter(([arg]) => arg.command === 'move')).toHaveLength(moves);+ expect(fixture.deps.native.mock.calls.some(([arg]) => arg.command === 'resume-quarantine')).toBe(false);+ });+ it.each(['superseded', 'different-pr-head', 'unknown-metadata', 'active-owner', 'new-holder', 'archive-changed', 'executor-busy', 'reservation-changed'])(+ 'retains the original reservation and all sources when fresh evidence is %s', async cause => {+ const input = await interrupted();+ if (cause === 'superseded') fixture.superseded = true;+ if (cause === 'different-pr-head') fixture.facts.prHead = '9'.repeat(40);+ if (cause === 'unknown-metadata') fixture.forbiddenMetadata = true;+ if (cause === 'active-owner') fixture.deps.ownership.mockResolvedValue({ ok: false, reason: 'active-owner' });+ if (cause === 'new-holder') fixture.newHolder = true;+ if (cause === 'archive-changed') fixture.failArchive = true;+ if (cause === 'executor-busy') fixture.deps.native.mockImplementationOnce(async () => { throw new Error('retirement-executor-busy'); });+ if (cause === 'reservation-changed') fixture.cut.owner.id = 'other-transaction';+ expect(await fixture.service.recoverAutomatically(input, permitted)).toMatchObject({ ok: false });+ expect(fixture.cut).not.toBeNull();+ expect(fixture.placement.administration).toBe('original');+ expect(fixture.deps.deleteRef).not.toHaveBeenCalled();+ expect(fixture.deps.recordRetirement).not.toHaveBeenCalled();+ });+ it.each(['pause', 'default', 'owner'])('rechecks %s after preparing a review and before the remaining move', async changed => {+ const input = await interrupted();+ let gates = 0;+ const blocked = async () => {+ if (++gates !== 2) return null;+ if (changed === 'pause') return 'daemon-paused';+ if (changed === 'default') fixture.facts.defaultHead = '8'.repeat(40);+ if (changed === 'owner') fixture.deps.ownership.mockResolvedValue({ ok: false, reason: 'active-owner' });+ return null;+ };+ expect(await fixture.service.recoverAutomatically(input, { blocked })).toMatchObject({ ok: false });+ expect(fixture.placement.administration).toBe('original');+ expect(fixture.cut).not.toBeNull();+ });+ it.each(['remaining-rename', 'branch-delete'])('honors a pause arriving inside the second native review before %s', async stage => {+ const input = stage === 'remaining-rename' ? await interrupted() : await (async () => {+ await fixture.service.retire(await reviewed()); return automaticInput();+ })();+ let paused = false, checks = 0;+ const native = fixture.deps.native.getMockImplementation();+ fixture.deps.native.mockImplementation(async request => {+ const result = await native(request);+ if (stage === 'remaining-rename' && request.command === 'verify-quarantine-resumption') paused = true;+ if (stage === 'branch-delete' && request.command === 'verify' && request.mode === 'restore' && ++checks === 3) paused = true;+ return result;+ });+ expect(await fixture.service.recoverAutomatically(input, { blocked: async () => paused ? 'daemon-paused' : null }))+ .toMatchObject({ ok: false, reason: 'daemon-paused' });+ expect(fixture.deps.deleteRef).not.toHaveBeenCalled();+ expect(fixture.cut).not.toBeNull();+ if (stage === 'remaining-rename') expect(fixture.placement.administration).toBe('original');+ });+ it('restarts a durably prepared branch retirement under the exact original cut without creating another reservation', async () => {+ await fixture.service.retire(await reviewed());+ fixture.deps.deleteRef.mockRejectedValueOnce(new Error('branch-archive-interrupted'));+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted)).toMatchObject({ ok: false, reason: 'branch-archive-interrupted' });+ const input = await automaticInput();+ expect((await fixture.service.status(input)).phase).toBe('branch-retirement-prepared');+ const reservation = structuredClone(fixture.cut);+ const reservations = fixture.deps.admission.reserveCheckoutRetirement.mock.calls.length;+ fixture.service = _createInitializedCheckoutRecovery(fixture.deps);+ expect(await fixture.service.recoverAutomatically(input, permitted)).toMatchObject({ ok: true, receiptRecorded: true });+ expect(fixture.deps.admission.reserveCheckoutRetirement).toHaveBeenCalledTimes(reservations);+ expect(fixture.deps.admission.releaseCheckoutReservation.mock.calls.at(-1)[0]).toEqual(reservation);+ });+ it.each(['owner', 'archive', 'forge', 'reservation'])('holds prepared retirement after fresh %s evidence changes', async changed => {+ await fixture.service.retire(await reviewed());+ fixture.deps.deleteRef.mockRejectedValueOnce(new Error('branch-archive-interrupted'));+ await fixture.service.recoverAutomatically(await automaticInput(), permitted);+ const input = await automaticInput();+ if (changed === 'owner') fixture.deps.ownership.mockResolvedValue({ ok: false, reason: 'active-owner' });+ if (changed === 'archive') fixture.failArchive = true;+ if (changed === 'forge') fixture.facts.prHead = '9'.repeat(40);+ if (changed === 'reservation') fixture.cut.owner.id = 'foreign';+ expect(await fixture.service.recoverAutomatically(input, permitted)).toMatchObject({ ok: false });+ expect(fixture.deps.deleteRef).toHaveBeenCalledTimes(1);+ expect(fixture.cut).not.toBeNull(); expect(fixture.headExists).toBe(true);+ });+ it('rechecks pause after durable intent publication and retains uncertainty without calling the receiver', async () => {+ await fixture.service.retire(await reviewed());+ let received = false;+ fixture.deps.deleteRef.mockImplementation(async (_repo, _branch, _head, options) => {+ await options.beforeMutation(); received = true; fixture.headExists = false; return true;+ });+ const input = await automaticInput();+ const blocked = async () => {+ const status = await fixture.service.status(input);+ return status.phase === 'branch-retirement-intent' ? 'daemon-paused' : null;+ };+ expect(await fixture.service.recoverAutomatically(input, { blocked })).toMatchObject({ ok: false, reason: 'daemon-paused' });+ expect(received).toBe(false); expect(fixture.headExists).toBe(true); expect(fixture.cut).not.toBeNull();+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted))+ .toMatchObject({ ok: false, reason: 'retirement-branch-deletion-uncertain' });+ });+ it.each(['restored', 'restored-with-metadata-transitions', 'restored-with-reviewed-finalization'])(+ 'treats completed %s with released reservation as settled, never as new retirement intent', async phase => {+ const input = await interrupted();+ if (phase === 'restored') {+ expect(await fixture.service.restore(input)).toMatchObject({ ok: true });+ } else {+ const review = await fixture.service.reviewMetadataRestore(input);+ fixture.delayedRestoreDrift = phase === 'restored-with-reviewed-finalization';+ expect(await fixture.service.restoreMetadata({ ...input, review: { digest: review.review.digest } }))+ .toMatchObject({ ok: !fixture.delayedRestoreDrift });+ if (phase === 'restored-with-reviewed-finalization') {+ const current = await automaticInput();+ const final = await fixture.service.reviewRestoredFinalization(current);+ expect(await fixture.service.finalizeRestored({ ...current, review: { digest: final.review.digest } })).toMatchObject({ ok: true });+ }+ }+ const discovered = await fixture.service.discover();+ expect(discovered.transactions[0]).toMatchObject({ phase, settled: true });+ const commands = fixture.deps.native.mock.calls.length;+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted)).toMatchObject({ ok: true, alreadyComplete: true, phase });+ expect(fixture.deps.native).toHaveBeenCalledTimes(commands);+ expect(fixture.deps.deleteRef).not.toHaveBeenCalled();+ });+ it('skips completed historical journals beside a fresh foreign cut without starving its exact owner', async () => {+ for (let i = 0; i < 3; i++) {+ const retired = await fixture.service.retire(await reviewed());+ expect(await fixture.service.restore({ appId: 'app', transactionId: retired.transactionId })).toMatchObject({ ok: true });+ }+ fixture.failPart = 'administration';+ const pending = await fixture.service.retire(await reviewed());+ fixture.failPart = null;+ const originalCut = fixture.cut;+ const owner = originalCut.owner;+ await mkdir(join(root, 'admission/cut'), { recursive: true, mode: 0o700 });+ await writeFile(join(root, 'admission/cut/owner.json'), JSON.stringify(owner), { mode: 0o600 });+ fixture.deps.admission.readCheckoutReservation = async (_repo, id) => {+ if (id !== pending.transactionId) throw Object.assign(new Error('checkout-retirement-reserved'), { code: 'CHECKOUT_RETIREMENT_RESERVED' });+ return { ...originalCut, owner };+ };+ // Discovery proves terminal history is unrelated to the fresh owner. Its+ // sweep slot must go to that owner, not any of the three completed rows.+ const discovered = await fixture.service.discover();+ expect(discovered.transactions.filter(row => row.settled)).toHaveLength(3);+ const { _createInitializedRecoverySweep } = await import('./initializedCheckoutRecoverySweep.js');+ const recover = vi.fn(async () => ({ ok: false, reason: 'fixture-current-owner-hold' }));+ await _createInitializedRecoverySweep({ discover: () => fixture.service.discover(), recover })(async () => null);+ expect(recover.mock.calls.map(([input]) => input.transactionId)).toEqual([pending.transactionId]);+ expect(fixture.cut).toBe(originalCut);+ expect(fixture.deps.deleteRef).not.toHaveBeenCalled();+ // Unknown or same-owner metadata is never treated as proof of release.+ await writeFile(join(root, 'admission/cut/owner.json'), '{}', { mode: 0o600 });+ expect((await fixture.service.discover()).transactions.some(row => row.settled)).toBe(false);+ });+ it('replays a durably confirmed receipt without another deletion, but retains uncertain deletion', async () => {+ await fixture.service.retire(await reviewed());+ fixture.deps.recordRetirement.mockRejectedValueOnce(new Error('receipt-disk-full'));+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted)).toMatchObject({ ok: false });+ fixture.superseded = true; // Receipt replay does not delete superseded work.+ fixture.service = _createInitializedCheckoutRecovery(fixture.deps);+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted)).toMatchObject({ ok: true, receiptRecorded: true });+ expect(fixture.deps.deleteRef).toHaveBeenCalledTimes(1);+ expect(fixture.cut).toBeNull();+ const calls = fixture.deps.native.mock.calls.length;+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted)).toMatchObject({ ok: true, alreadyComplete: true });+ expect(fixture.deps.native).toHaveBeenCalledTimes(calls);+ });+ it('never interprets an absent ref as a completed deletion without the committed result', async () => {+ await fixture.service.retire(await reviewed());+ fixture.deps.deleteRef.mockImplementation(async (_repo, _branch, _head, options) => { await options.beforeMutation(); fixture.headExists = false; throw new Error('receiver-result-lost'); });+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted)).toMatchObject({ ok: false });+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted))+ .toMatchObject({ ok: false, reason: 'retirement-branch-deletion-uncertain' });+ expect(fixture.deps.deleteRef).toHaveBeenCalledTimes(1);+ expect(fixture.deps.recordRetirement).not.toHaveBeenCalled();+ });+ it('reports unsupported pre-move phases and corrupt journals without creating a fresh transaction', async () => {+ fixture.failPreserve = true;+ await fixture.service.retire(await reviewed());+ const input = await automaticInput();+ expect(await fixture.service.recoverAutomatically(input, permitted)).toMatchObject({ ok: false, reason: 'preservation-proof-unbound' });+ const first = join(root, input.transactionId, '000000.json');+ await writeFile(first, '{', { mode: 0o600 });+ const found = await fixture.service.discover();+ expect(found.transactions[0]).toMatchObject({ transactionId: input.transactionId, reason: expect.any(String) });+ expect(fixture.deps.native.mock.calls.some(([arg]) => arg.command === 'move')).toBe(false);+ expect(fixture.cut).not.toBeNull();+ });+ it('does not call a terminal receipt settled when its owner file disappeared but the cut still exists', async () => {+ await fixture.service.retire(await reviewed());+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted)).toMatchObject({ ok: true });+ await mkdir(join(root, 'admission/cut'), { recursive: true, mode: 0o700 });+ const found = await fixture.service.discover();+ expect(found.transactions[0].settled).toBe(false);+ expect(await fixture.service.recoverAutomatically(await automaticInput(), permitted))+ .toMatchObject({ ok: false, reason: 'retirement-reservation-owner-missing' });+ expect(fixture.deps.deleteRef).toHaveBeenCalledTimes(1);+ });+ it('holds an oversized journal before reading or dispatching its contents', async () => {+ const input = await interrupted();+ const file = await open(join(root, input.transactionId, '000000.json'), 'r+');+ try { await file.truncate(16 * 1024 * 1024 + 1); } finally { await file.close(); }+ const found = await fixture.service.discover();+ expect(found.transactions[0]).toMatchObject({ transactionId: input.transactionId, reason: 'retirement-journal-budget-exceeded' });+ expect(fixture.deps.deleteRef).not.toHaveBeenCalled();+ });+ it('refuses stale discovery digests and requires an explicit running gate', async () => {+ const input = await interrupted();+ expect(await fixture.service.recoverAutomatically(input)).toMatchObject({ ok: false, reason: 'automatic-retirement-gate-required' });+ expect(await fixture.service.recoverAutomatically({ ...input, journalDigest: '0'.repeat(64) }, permitted))+ .toMatchObject({ ok: false, reason: 'retirement-journal-changed' });+ expect(fixture.deps.deleteRef).not.toHaveBeenCalled();+ });+});diff --git a/server/services/initializedCheckoutRecoverySweep.js b/server/services/initializedCheckoutRecoverySweep.jsnew file mode 100644index 0000000000..e66b9438eb--- /dev/null+++ b/server/services/initializedCheckoutRecoverySweep.js@@ -0,0 +1,77 @@+/** Deterministic continuation of existing private retirement transactions.+ * The journal stays authoritative; this bounded in-memory retry cache grants+ * nothing and is rebuilt after restart. No task, agent, or checkout is created.+ */+export const INITIALIZED_RECOVERY_INTERVAL_MS = 30 * 60_000;+const MAX_ATTEMPTS = 2;++export function _createInitializedRecoverySweep({ discover, recover, now = Date.now, report = () => {} }) {+ let running = null;+ let nextScan = 0;+ let cursor = '';+ const attempted = new Map();+ const sweep = async blocked => {+ const reason = await blocked();+ if (reason) return { skipped: reason, outcomes: [] };+ if (now() < nextScan) return { skipped: 'recovery-cadence', outcomes: [] };+ nextScan = now() + INITIALIZED_RECOVERY_INTERVAL_MS;+ const found = await discover();+ if (!found.ok) {+ await report({ transactionId: null, reason: found.reason });+ return { skipped: found.reason, outcomes: [] };+ }+ const outcomes = [];+ const valid = [];+ for (const row of found.transactions) {+ if (row.reason) { await report(row); outcomes.push(row); }+ else if (row.settled) {+ const outcome = { transactionId: row.transactionId, appId: row.appId, branch: row.release?.branch,+ phase: row.phase, journalDigest: row.journalDigest, ok: true, reason: 'retirement-recovery-completed' };+ await report(outcome); outcomes.push(outcome);+ } else valid.push(row);+ }+ const ids = new Set(valid.map(row => row.transactionId));+ for (const key of attempted.keys()) if (!ids.has(key)) attempted.delete(key);+ // Rotate bounded batches so an old held transaction cannot starve later+ // journals. Every retry obtains fresh evidence; cached holds expire.+ const sorted = [...valid.filter(row => row.transactionId > cursor), ...valid.filter(row => row.transactionId <= cursor)];+ let count = 0;+ for (const row of sorted) {+ const prior = attempted.get(row.transactionId);+ if (prior?.digest === row.journalDigest && now() < prior.retryAt) continue;+ const stopped = await blocked();+ if (stopped) return { skipped: stopped, outcomes };+ if (++count > MAX_ATTEMPTS) break;+ cursor = row.transactionId;+ const result = await recover({ appId: row.appId, transactionId: row.transactionId,+ journalDigest: row.journalDigest }, { blocked });+ const outcome = { transactionId: row.transactionId, appId: row.appId,+ phase: result.receiptRecorded ? 'retired' : result.phase || row.phase,+ branch: row.release?.branch, journalDigest: row.journalDigest, ok: result.ok, reason: result.reason || (result.ok ? 'retirement-recovery-completed' : 'retirement-recovery-held') };+ attempted.set(row.transactionId, { digest: row.journalDigest, retryAt: now() + INITIALIZED_RECOVERY_INTERVAL_MS });+ await report(outcome); outcomes.push(outcome);+ }+ return { outcomes };+ };+ return blocked => {+ if (running) return running;+ running = sweep(blocked).finally(() => { running = null; });+ return running;+ };+}++const drain = _createInitializedRecoverySweep({+ discover: async () => (await import('./initializedCheckoutRecovery.js')).discoverInitializedCheckoutRecoveries(),+ recover: async (...args) => (await import('./initializedCheckoutRecovery.js')).recoverInitializedCheckoutAutomatically(...args),+ report: async row => {+ const { recordInitializedRetirementRecovery } = await import('./taskSchedule.js');+ const { appId, ...observation } = row;+ await recordInitializedRetirementRecovery(appId || null, observation);+ // No paths, native diagnostics, patches, or raw metadata in daemon logs.+ const reason = /^[a-z][a-z0-9-]+$/.test(row.reason) ? row.reason : 'retirement-recovery-held';+ const message = `Initialized retirement ${row.transactionId || 'discovery'}: ${reason}`;+ console.info(`🔁 ${message}`);+ },+});++export const sweepInitializedCheckoutRecoveries = drain;diff --git a/server/services/initializedCheckoutRecoverySweep.test.js b/server/services/initializedCheckoutRecoverySweep.test.jsnew file mode 100644index 0000000000..f5f436fad7--- /dev/null+++ b/server/services/initializedCheckoutRecoverySweep.test.js@@ -0,0 +1,64 @@+import { describe, expect, it, vi } from 'vitest';+import { _createInitializedRecoverySweep, INITIALIZED_RECOVERY_INTERVAL_MS } from './initializedCheckoutRecoverySweep.js';++const transaction = id => ({ transactionId: id, appId: 'app', journalDigest: `${id}-digest`,+ phase: 'quarantined-working', release: { branch: `claim/${id}` } });++describe('initialized retirement recovery drain', () => {+ it('serializes overlapping triggers, bounds batches and rotates past held work with fresh retry evidence', async () => {+ let time = 0, release;+ const rows = ['a', 'b', 'c'].map(transaction);+ const discover = vi.fn(async () => ({ ok: true, transactions: rows }));+ const recover = vi.fn(async () => ({ ok: false, reason: 'automatic-retirement-independent-review-required' }));+ recover.mockImplementationOnce(() => new Promise(resolve => { release = resolve; }));+ const report = vi.fn();+ const drain = _createInitializedRecoverySweep({ discover, recover, report, now: () => time });+ const first = drain(async () => null);+ expect(drain(async () => null)).toBe(first);+ await vi.waitFor(() => expect(release).toBeTypeOf('function'));+ release({ ok: false, reason: 'active-owner' });+ await first;+ expect(recover.mock.calls.map(([input]) => input.transactionId)).toEqual(['a', 'b']);+ expect(report.mock.calls[0][0]).toMatchObject({ transactionId: 'a', branch: 'claim/a', reason: 'active-owner' });+ expect(await drain(async () => null)).toMatchObject({ skipped: 'recovery-cadence' });+ time += INITIALIZED_RECOVERY_INTERVAL_MS;+ rows[0].journalDigest = 'new-journal';+ await drain(async () => null);+ expect(recover.mock.calls.slice(2).map(([input]) => input.transactionId)).toEqual(['c', 'a']);+ expect(recover.mock.calls.at(-1)[0].journalDigest).toBe('new-journal');+ expect(discover).toHaveBeenCalledTimes(2);+ });+ it('clears completed diagnostics without letting terminal journals consume replay slots', async () => {+ const report = vi.fn();+ const recover = vi.fn(async () => ({ ok: false, reason: 'active-owner' }));+ const rows = ['a', 'b', 'c'].map(id => ({ ...transaction(id), settled: true, phase: 'restored-with-reviewed-finalization' }));+ const drain = _createInitializedRecoverySweep({ discover: async () => ({ ok: true, transactions: [...rows, transaction('d')] }), recover, report });+ await drain(async () => null);+ expect(recover.mock.calls.map(([input]) => input.transactionId)).toEqual(['d']);+ expect(report.mock.calls.slice(0, 3).every(([row]) => row.ok && row.reason === 'retirement-recovery-completed')).toBe(true);+ });+ it('does no discovery while stopped, paused, off or dry-run and checks the gate again before replay', async () => {+ const discover = vi.fn(async () => ({ ok: true, transactions: [transaction('a')] }));+ const recover = vi.fn();+ const drain = _createInitializedRecoverySweep({ discover, recover });+ for (const reason of ['daemon-stopped', 'daemon-paused', 'autonomy-off', 'autonomy-dry-run']) {+ expect(await drain(async () => reason)).toMatchObject({ skipped: reason });+ }+ expect(discover).not.toHaveBeenCalled();+ const gate = vi.fn().mockResolvedValueOnce(null).mockResolvedValue('daemon-paused');+ expect(await drain(gate)).toMatchObject({ skipped: 'daemon-paused' });+ expect(recover).not.toHaveBeenCalled();+ });+ it('surfaces invalid journals and refuses an incomplete discovery without invoking recovery', async () => {+ const report = vi.fn();+ const recover = vi.fn();+ const discover = vi.fn(async () => ({ ok: true, transactions: [{ transactionId: 'invalid', reason: 'retirement-journal-invalid' }] }));+ const drain = _createInitializedRecoverySweep({ discover, recover, report, now: () => 0 });+ await drain(async () => null);+ expect(report).toHaveBeenCalledWith({ transactionId: 'invalid', reason: 'retirement-journal-invalid' });+ const capped = _createInitializedRecoverySweep({ discover: async () => ({ ok: false,+ reason: 'retirement-discovery-budget-exceeded', transactions: [transaction('a')] }), recover, report });+ expect(await capped(async () => null)).toMatchObject({ skipped: 'retirement-discovery-budget-exceeded' });+ expect(recover).not.toHaveBeenCalled();+ });+});diff --git a/server/services/taskSchedule.js b/server/services/taskSchedule.jsindex ac42b424a6..5ccf9469b0 100644--- a/server/services/taskSchedule.js+++ b/server/services/taskSchedule.js@@ -762,6 +762,24 @@ export async function recordBranchReconcileOutcome(appId, { status, reason, rema});}+/** Observations are diagnostic only; journals and fresh service proofs own recovery. */+export async function recordInitializedRetirementRecovery(appId, observation) {+ return updateSchedule(async schedule => {+ const record = ensureExecutionRecord(schedule, 'branch-reconcile', appId);+ const previous = record.branchReconcileOutcome || {};+ const rows = previous.initializedRetirements || [];+ const index = rows.findIndex(row => row.transactionId === observation.transactionId);+ if (index < 0 && rows.length >= 1000) throw new Error('retirement-observation-budget-exceeded');+ const prior = index < 0 ? null : rows[index];+ if (prior && Object.entries(observation).every(([key, value]) => prior[key] === value)) return { result: prior, changed: false };+ const next = [...rows];+ const row = { ...observation, observedAt: new Date().toISOString() };+ if (index < 0) next.push(row); else next[index] = row;+ record.branchReconcileOutcome = { ...previous, initializedRetirements: next };+ return { result: row, changed: true };+ });+}+export async function getBranchReconcileOutcome(appId) {const schedule = await loadSchedule();return resolveExecutionRecord(schedule, 'branch-reconcile', appId)?.branchReconcileOutcome ?? null;diff --git a/server/services/taskSchedule.test.js b/server/services/taskSchedule.test.jsindex 121d86bd34..0351f609c7 100644--- a/server/services/taskSchedule.test.js+++ b/server/services/taskSchedule.test.js@@ -143,6 +143,7 @@ import {getPerpetualDrainState,recordPerpetualDispatch,recordBranchReconcileOutcome,+ recordInitializedRetirementRecovery,recordHeldCheckoutRetirement,applyOnDemandRunResets,queuePerpetualRefill,@@ -3187,6 +3188,11 @@ describe('taskSchedule', () => {writeFile.mockImplementation(async (_path, contents) => { saved = JSON.parse(contents); });const remaining = [{ branch: 'fix/example', path: '/example/checkout', reason: 'worktree-locked' }];await recordBranchReconcileOutcome('app-1', { status: 'running', reason: 'batch-pending', remaining, nextAction: 'Continue.' });+ await recordInitializedRetirementRecovery('app-1', { transactionId: 'transaction-a', branch: 'claim/held',+ phase: 'quarantined-working', journalDigest: 'digest-a', ok: false, reason: 'automatic-retirement-independent-review-required' });+ expect((await getScheduleStatus()).tasks['branch-reconcile'].reconcileOutcomes['app-1'].initializedRetirements)+ .toEqual([expect.objectContaining({ transactionId: 'transaction-a', branch: 'claim/held', reason: 'automatic-retirement-independent-review-required' })]);+const observedAt = saved.executions['task:branch-reconcile'].perApp['app-1'].branchReconcileOutcome.observedAt;await recordBranchReconcileOutcome('app-1', { status: 'incomplete', reason: 'agent-failed', nextAction: 'Resolve the provider failure and run again.' });expect(saved.executions['task:branch-reconcile'].perApp['app-1'].branchReconcileOutcome)diff --git a/server/services/worktreeManager.js b/server/services/worktreeManager.jsindex 100123a359..81bea75fbf 100644--- a/server/services/worktreeManager.js+++ b/server/services/worktreeManager.js@@ -248,7 +248,8 @@ async function queueWorktreeCreate(repo, fn) {* policy still apply. Share the creation lane so managed attachments cannot* enter between the receiver's holder check and ref transaction.*/-export function deleteCompletedLocalBranch(repoPath, branch, expectedHead) {+export function deleteCompletedLocalBranch(repoPath, branch, expectedHead, { beforeMutation = async () => {} } = {}) {+ let callbackError;return queueWorktreeCreate(repoPath, async () => {if (PROTECTED_BRANCHES.includes(branch)) return false;const { getDefaultBranch } = await import('./git.js');@@ -261,11 +262,12 @@ export function deleteCompletedLocalBranch(repoPath, branch, expectedHead) {if (!isAbsolute(common)) return false;const preservation = await preserveBranchRetirement(repoPath, common, branch, expectedHead);if (!await preservation.verify()) return false;+ await Promise.resolve().then(beforeMutation).catch(error => { callbackError = error; throw error; });const result = await execGit(['push','--receive-pack=git -c receive.denyCurrentBranch=refuse -c receive.denyDeleteCurrent=refuse -c receive.autogc=false receive-pack',`--force-with-lease=${ref}:${expectedHead}`, '--', common, `:${ref}`], repoPath, { ignoreExitCode: true });return result.exitCode === 0;- }).catch(() => false);+ }).catch(error => { if (error === callbackError) throw error; return false; });}/**@@ -417,7 +419,7 @@ export async function forceRemoveWorktreeDir(repo, worktreePath, opts = {}) {.catch(error => ({ removed: false, reason: error.message, warning: 'Checkout retained: repository reservation unavailable' }));}-async function forceRemoveWorktreeDirAdmitted(repo, worktreePath, { label, log = 'remove', subject = worktreePath, verifyRemoval } = {}) {+async function forceRemoveWorktreeDirAdmitted(repo, worktreePath, { label, log = 'remove', subject = worktreePath, verifyRemoval, beforeMutation = async () => {} } = {}) {const logAll = label && log === 'all';let lastFailure = null;let heldReason = null;@@ -439,6 +441,7 @@ async function forceRemoveWorktreeDirAdmitted(repo, worktreePath, { label, log =return null;};heldReason = await check();+ if (!heldReason) await beforeMutation();if (!heldReason) await execGit(['worktree', 'remove', worktreePath, '--force'], repo, { timeout: WORKTREE_ADD_TIMEOUT_MS }).catch(async (err) => {lastFailure = errorExcerpt(err);if (label) console.log(`⚠️ ${label}: ${err.message}`);@@ -448,6 +451,7 @@ async function forceRemoveWorktreeDirAdmitted(repo, worktreePath, { label, log =// bypass a lock acquired after our last read, even with a caller guard.const directory = await lstat(worktreePath).then(() => 'present', err => err.code === 'ENOENT' ? 'absent' : 'unknown');if (directory !== 'absent') { heldReason = 'worktree-removal-refused'; return; }+ await beforeMutation();await execGit(['worktree', 'prune'], repo).catch((pruneErr) => {lastFailure = errorExcerpt(pruneErr);if (logAll) console.log(`⚠️ Worktree prune failed for ${subject}: ${pruneErr.message}`);diff --git a/server/services/worktreeManager.test.js b/server/services/worktreeManager.test.jsindex d2f2f9f40a..deeaa1a9cf 100644--- a/server/services/worktreeManager.test.js+++ b/server/services/worktreeManager.test.js@@ -50,6 +50,7 @@ vi.mock('./git.js', () => ({const {shouldRefuseDefaultBranchMerge,deleteCompletedLocalBranch,+ forceRemoveWorktreeDir,isHumanClaimWorktree,classifyWorktreeDirt,isGitLockError,@@ -86,6 +87,48 @@ describe('completed branch archival boundary', () => {afterEach(() => {preserveBranchMock.mockReset().mockResolvedValue({ archiveDir: '/branch-archive', verify: async () => true });});+ it('runs the trusted receiver guard after archival verification and preserves its refusal reason without pushing', async () => {+ let ready = false;+ preserveBranchMock.mockResolvedValue({ verify: async () => { ready = true; return true; } });+ const beforeMutation = vi.fn(async () => {+ expect(ready).toBe(true);+ throw new Error('daemon-paused');+ });+ await expect(deleteCompletedLocalBranch('/repo', 'done', head, { beforeMutation })).rejects.toThrow('daemon-paused');+ expect(beforeMutation).toHaveBeenCalledOnce();+ expect(execGitMock.mock.calls.some(([args]) => args[0] === 'push')).toBe(false);+ });+ it('checks automatic permission after final registration refresh before checkout removal', async () => {+ let reads = 0, verified = false;+ execGitMock.mockImplementation(async args => {+ if (args[0] === 'worktree' && args[1] === 'list') {+ reads++;+ return { stdout: `worktree /checkout\nHEAD ${head}\nbranch refs/heads/done\n\n`, exitCode: 0 };+ }+ return { stdout: '', exitCode: 0 };+ });+ const beforeMutation = vi.fn(async () => {+ expect(verified).toBe(true); expect(reads).toBe(2);+ throw new Error('daemon-paused');+ });+ expect(await forceRemoveWorktreeDir('/repo', '/checkout', {+ verifyRemoval: async () => { verified = true; return null; }, beforeMutation,+ })).toMatchObject({ removed: false, reason: 'daemon-paused' });+ expect(beforeMutation).toHaveBeenCalledOnce();+ expect(execGitMock.mock.calls.some(([args]) => args[0] === 'worktree' && ['remove', 'prune'].includes(args[1]))).toBe(false);+ });+ it('rechecks automatic permission before pruning an already absent checkout after a failed removal', async () => {+ execGitMock.mockImplementation(async args => {+ if (args[0] === 'worktree' && args[1] === 'remove') throw new Error('remove interrupted');+ return { stdout: '', exitCode: 0 };+ });+ lstat.mockRejectedValueOnce(Object.assign(new Error('missing'), { code: 'ENOENT' }));+ const beforeMutation = vi.fn().mockResolvedValueOnce(undefined).mockRejectedValueOnce(new Error('daemon-stopped'));+ expect(await forceRemoveWorktreeDir('/repo', '/checkout', { verifyRemoval: async () => null, beforeMutation }))+ .toMatchObject({ removed: false, reason: 'daemon-stopped' });+ expect(beforeMutation).toHaveBeenCalledTimes(2);+ expect(execGitMock.mock.calls.some(([args]) => args[0] === 'worktree' && args[1] === 'prune')).toBe(false);+ });it.each(['archive failure', 'archive verification failure'])('never sends the deletion when there is an %s', async failure => {if (failure === 'archive failure') preserveBranchMock.mockRejectedValue(new Error('archive disk full'));else preserveBranchMock.mockResolvedValue({ archiveDir: '/branch-archive', verify: async () => false });