Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

fix([issue-12740]): stamp dead-owner publication reservations unsettled (#12759)

[._.]/e41621536ff3
diff --git a/docs/STORAGE.md b/docs/STORAGE.md
index 6813d0e6c0..1a568834df 100644
--- a/docs/STORAGE.md
+++ b/docs/STORAGE.md
@@ -46,11 +46,22 @@ certify the old work finished. Do not delete this journal to claim readiness; pr
with the related run/job records for recovery. A backup restored on another installation
therefore starts conservatively if it contains a hold or outstanding operations.
+On server boot, and again when orphaned agents are swept, a settled `Output publication`
+reservation whose admitting pid is dead is marked unsettled and given an uncertainty
+stamp. That does not release it. A live pid is left untouched, and an already-unsettled
+stamp is not rotated. A dead-owner `cli-run` is released only when that run's metadata
+already has an end time, which is published before the reservation is finished. A
+dead-owner `mind-turn` is released only when the persisted mind record has no active
+turn, including after restart recovery of an orphaned turn. A dead-owner `provider`
+reservation names a caller label, not a run, so it is left in place. Other kinds are
+not inferred from a dead pid alone.
+
Publication admission keeps a maintenance reservation while waiting for a backup
cut. A boundary-owned callback-start check retires a refused admission without
claiming any output was saved; failures after callback entry remain unresolved.
Legacy generic `Output publication` reservations lack destination/batch identity.
-They cannot be swept automatically. After explicit operator disposition of
+They cannot be swept as finished work. A dead admitting process is recorded as
+uncertainty, not deleted. After explicit operator disposition of
inspected evidence, an owning service may use `reconcilePublicationRefusal` with
the exact original operation and hold. Its synchronous publisher must preserve a
durable, replayable evidence receipt before retirement; the journal validates and
diff --git a/server/lib/maintenanceAdmission.js b/server/lib/maintenanceAdmission.js
index e1f40b8187..00378a6518 100644
--- a/server/lib/maintenanceAdmission.js
+++ b/server/lib/maintenanceAdmission.js
@@ -310,6 +310,21 @@ export function createMaintenanceAdmission(dataDir = PATHS.data, { io = fs, asse
&& seen.resource === op.resource && seen.pid === op.pid && seen.startedAt === op.startedAt));
return before - state.operations.length;
});
+ // Fail-closed stamp for records the caller already inspected. Already-unsettled
+ // rows keep their stamp: rotating it would invalidate an operator receipt in flight.
+ const markOperationsUncertain = expected => transaction(state => {
+ let marked = 0;
+ for (const op of state.operations) {
+ if (op.unsettled) continue;
+ const seen = expected.some(item => item.id === op.id && item.kind === op.kind
+ && item.resource === op.resource && item.pid === op.pid && item.startedAt === op.startedAt);
+ if (!seen) continue;
+ op.unsettled = true;
+ op.uncertaintyStamp = randomUUID();
+ marked += 1;
+ }
+ return marked;
+ });
// Owning-service recovery only. Validation and durable receipt publication
// run under the same journal lock as the exact reservation removal.
const reconcileAbandonedAgent = ({ hold, agentId, validate, publish, replay }) => transaction(state => {
@@ -326,7 +341,7 @@ export function createMaintenanceAdmission(dataDir = PATHS.data, { io = fs, asse
return receipt;
});
const { observeIdle, claimReady, getExclusive, transitionExclusive, settleExclusive, issueExecutionCapability } = exclusive;
- return { directory, status, held, assertOpen, admit, tryAdmit, recoverOwned, reconcilePublicationRefusal, operations, releaseOperations, run, currentId, finish, finishResource, withResource, markResourceUnsettled, markCurrentUnsettled, continueSettlement, begin, resume, events,
+ return { directory, status, held, assertOpen, admit, tryAdmit, recoverOwned, reconcilePublicationRefusal, operations, releaseOperations, markOperationsUncertain, run, currentId, finish, finishResource, withResource, markResourceUnsettled, markCurrentUnsettled, continueSettlement, begin, resume, events,
observeIdle, claimReady, getExclusive, transitionExclusive, settleExclusive, issueExecutionCapability, reconcileAbandonedAgent };
}
diff --git a/server/services/README.md b/server/services/README.md
index 3ebd91707b..9fb8836ae6 100644
--- a/server/services/README.md
+++ b/server/services/README.md
@@ -6,6 +6,7 @@ image-generation paths below are useful entry points when tracing render jobs.
| Module | Responsibility |
| --- | --- |
+| `deadOwnerReservations.js` | Boot and orphan-sweep reconcile for reservations whose admitting process died: stamps settled Output publication rows unsettled for receipt recovery, and releases only cli-run or mind-turn residue whose owning record proves completion. |
| `branchReconcileRetirement.js` | Retires exact-head, integrated checkouts through fresh fenced archive proof; reserves bounded cognitive review only for source/history differences. |
| `repoCleanupRecovery.js` | Reserves one bounded exact-owner review or code-repair obligation, rechecks policy and delivery evidence before publication, and never queues a provider for routine holds. |
| `repoCleanupObservation.js` | Read-only exact-ref presence/absence accounting; malformed, changing or unsupported metadata remains unknown. |
diff --git a/server/services/agentManagement.js b/server/services/agentManagement.js
index 16c5255921..a1592545fe 100644
--- a/server/services/agentManagement.js
+++ b/server/services/agentManagement.js
@@ -3,6 +3,7 @@ import { hasClaimFlowContract } from '../lib/claimFlowTaskTypes.js';
import { isFalsyMeta, isTruthyMeta } from '../lib/metadataFlags.js';
import { maintenance } from '../lib/maintenanceAdmission.js';
import { reconcileStaleAgentReservations, releaseAgentReservation } from './agentMaintenanceReservations.js';
+import { reconcileDeadOwnerReservations } from './deadOwnerReservations.js';
/**
* Agent Management
*
@@ -1425,6 +1426,9 @@ async function runCleanupOrphanedAgents() {
// dead owner pid cannot belong to an agent spawned meanwhile, so no re-read is needed.
const retired = new Set(retiredAgentIds);
await reconcileStaleAgentReservations({ agents: agents.map(a => retired.has(a.id) ? { ...a, status: 'completed' } : a) });
+ // Same sweep, different kinds: publication residue is stamped, and only a
+ // cli-run or mind-turn whose owning record proves completion is released.
+ reconcileDeadOwnerReservations();
// Consume the marker. Doing it AFTER the retry pass (not before) means a crash
// mid-sweep leaves it in place and the next boot still classifies these agents
diff --git a/server/services/agentManagement.test.js b/server/services/agentManagement.test.js
index bbc649c922..8066217959 100644
--- a/server/services/agentManagement.test.js
+++ b/server/services/agentManagement.test.js
@@ -122,6 +122,12 @@ vi.mock('../lib/hostShutdown.js', async (importOriginal) => ({
}));
// Real admission with only the settlement release observable: the default instance
// resolves the live data root, so a spy keeps this suite off real journal state.
+// The sweep also reconciles dead-owner reservations. This file's maintenance
+// proxy forwards every other method to the live journal, so the reconcile must
+// stay stubbed or a test would stamp the install's publication residue.
+vi.mock('./deadOwnerReservations.js', () => ({
+ reconcileDeadOwnerReservations: vi.fn(() => ({ marked: 0, released: 0 })),
+}));
vi.mock('../lib/maintenanceAdmission.js', async (importOriginal) => {
const actual = await importOriginal();
const finishResource = vi.fn().mockResolvedValue([]);
diff --git a/server/services/bootstrap.js b/server/services/bootstrap.js
index f5cc94378d..490242cbc7 100644
--- a/server/services/bootstrap.js
+++ b/server/services/bootstrap.js
@@ -68,6 +68,7 @@ import { recordCompletedRunUsage } from './usageReconciler.js';
import { setAIToolkit as setProvidersToolkit } from './providers.js';
import { setAIToolkit as setProviderStatusToolkit } from './providerStatus.js';
import { setAIToolkit as setRunnerToolkit, executeCliRun as executeCliRunFixed } from './runner.js';
+import { reconcileDeadOwnerReservations } from './deadOwnerReservations.js';
import { setAIToolkit as setPromptsToolkit } from './promptService.js';
import { executeTuiRun as executeTuiRunFixed } from './tuiPromptRunner.js';
import { initAutoFixer } from './autoFixer.js';
@@ -419,6 +420,11 @@ export const bootstrapServices = async ({ io, dataDir, dataReferenceDir, serverD
* server from listening; each logs its own failure and the boot continues.
*/
const startBackgroundServices = ({ spawnerReady, io, httpsEnabled, port }) => {
+ // A crashed process leaks settled reservations that finish() never reached.
+ // Stamp uncertain publications before later boot steps read the journal.
+ // Sync and non-throwing; CoS may be off. Smoke boot skips this whole function.
+ reconcileDeadOwnerReservations();
+
// Keep the opt-in agent API key file current (Settings > Security). A no-op
// beyond one settings read while the key is off. Disabled under smoke boot
// with the rest of this function, so a smoke run never writes to $HOME.
diff --git a/server/services/deadOwnerReservations.js b/server/services/deadOwnerReservations.js
new file mode 100644
index 0000000000..25dc6a0281
--- /dev/null
+++ b/server/services/deadOwnerReservations.js
@@ -0,0 +1,98 @@
+/**
+ * Reservations left when the admitting process dies inside the boundary.
+ * `run()` finishes in a finally, so only process death leaks a settled row.
+ * Boot and the orphan sweep call this. It never throws: a journal that needs
+ * recovery must not abort the caller's startup.
+ *
+ * A dead pid is not proof the work was saved. `settlement` / `Output publication`
+ * is stamped unsettled so the operator receipt path can see it, and is never
+ * released here. `cli-run` is released only when that run's metadata already
+ * has an end time (published before `permit.finish()`). `mind-turn` is released
+ * only when the mind record has no active turn. `provider` names a caller
+ * label, not a run, so it cannot be matched to a finished call and stays.
+ */
+import { readFileSync } from 'node:fs';
+import { createRequire } from 'node:module';
+import { join } from 'node:path';
+import { maintenance } from '../lib/maintenanceAdmission.js';
+import { PATHS } from '../lib/paths.js';
+
+const require = createRequire(import.meta.url);
+
+// heavyJobClaim computes its claim path at import. Load it on use so a suite
+// that stubs fileUtils without a data root can still import this module.
+function defaultPidAlive(pid) {
+ return require('../lib/heavyJobClaim.js').isPidAlive(pid);
+}
+
+// Run ids are minted as UUIDs. Reject anything that could escape the runs directory.
+const SAFE_RUN_ID = /^[A-Za-z0-9][A-Za-z0-9._-]{0,200}$/;
+
+function cliRunRecordEnded(runId, runsDir) {
+ if (typeof runId !== 'string' || !SAFE_RUN_ID.test(runId)) return false;
+ try {
+ const metadata = JSON.parse(readFileSync(join(runsDir, runId, 'metadata.json'), 'utf8'));
+ return typeof metadata?.endTime === 'string' && metadata.endTime.length > 0;
+ } catch {
+ return false;
+ }
+}
+
+/** True only for a persisted mind object that is not inside a turn. */
+function mindRecordHasNoActiveTurn(state) {
+ const mind = state?.persistentMind;
+ if (!mind || typeof mind !== 'object' || Array.isArray(mind)) return false;
+ return mind.activeTurn == null;
+}
+
+function readMindIdle(mindStatePath = join(PATHS.cos, 'state.json')) {
+ try {
+ return mindRecordHasNoActiveTurn(JSON.parse(readFileSync(mindStatePath, 'utf8')));
+ } catch {
+ return false;
+ }
+}
+
+/**
+ * @param {{ admission?: object, pidAlive?: (pid: number) => boolean, runsDir?: string,
+ * cliRunEnded?: (runId: string) => boolean, mindStatePath?: string,
+ * mindTurnIdle?: () => boolean }} [options]
+ * @returns {{ marked: number, released: number }}
+ */
+export function reconcileDeadOwnerReservations({
+ admission = maintenance,
+ pidAlive = defaultPidAlive,
+ runsDir = PATHS.runs,
+ cliRunEnded = (runId) => cliRunRecordEnded(runId, runsDir),
+ mindStatePath,
+ mindTurnIdle = () => readMindIdle(mindStatePath),
+} = {}) {
+ try {
+ const dead = admission.operations().filter(op => op.unsettled !== true && !pidAlive(op.pid));
+ if (dead.length === 0) return { marked: 0, released: 0 };
+ const publications = dead.filter(op => op.kind === 'settlement' && op.resource === 'Output publication');
+ const marked = publications.length === 0 ? 0 : admission.markOperationsUncertain(publications);
+ let mindIdle;
+ const proved = (op) => {
+ if (op.kind === 'cli-run') {
+ try { return cliRunEnded(op.resource) === true; } catch { return false; }
+ }
+ if (op.kind === 'mind-turn') {
+ if (mindIdle === undefined) {
+ try { mindIdle = mindTurnIdle() === true; } catch { mindIdle = false; }
+ }
+ return mindIdle;
+ }
+ // provider: nothing in the reservation identifies the call that was saved.
+ return false;
+ };
+ const releasable = dead.filter(op => op.kind !== 'settlement' && proved(op));
+ const released = releasable.length === 0 ? 0 : admission.releaseOperations(releasable);
+ if (marked > 0) console.log(`🧹 Marked ${marked} dead-owner publication reservation(s) unsettled`);
+ if (released > 0) console.log(`🧹 Released ${released} dead-owner reservation(s) with proved completion`);
+ return { marked, released };
+ } catch (err) {
+ console.warn(`⚠️ Dead-owner reservation reconcile skipped: ${err.message}`);
+ return { marked: 0, released: 0 };
+ }
+}
diff --git a/server/services/deadOwnerReservations.test.js b/server/services/deadOwnerReservations.test.js
new file mode 100644
index 0000000000..16f5eb8ef5
--- /dev/null
+++ b/server/services/deadOwnerReservations.test.js
@@ -0,0 +1,181 @@
+import { afterEach, describe, expect, it, vi } from 'vitest';
+import * as fs from 'node:fs';
+import { join } from 'node:path';
+import { tmpdir } from 'node:os';
+import { createMaintenanceAdmission } from '../lib/maintenanceAdmission.js';
+import { reconcileDeadOwnerReservations } from './deadOwnerReservations.js';
+
+const roots = [];
+const DEAD_PID = 2147483646;
+const gate = () => {
+ const root = fs.mkdtempSync(join(tmpdir(), 'dead-owner-reservations-'));
+ roots.push(root);
+ return { root, admission: createMaintenanceAdmission(root) };
+};
+const readOps = (admission) => JSON.parse(fs.readFileSync(join(admission.directory, 'state.json'), 'utf8')).operations;
+const rewritePid = (admission, id, pid) => {
+ const file = join(admission.directory, 'state.json');
+ const state = JSON.parse(fs.readFileSync(file, 'utf8'));
+ state.operations.find(op => op.id === id).pid = pid;
+ fs.writeFileSync(file, JSON.stringify(state));
+};
+afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); });
+
+describe('reconcileDeadOwnerReservations', () => {
+ const reconcile = (admission, extra = {}) => reconcileDeadOwnerReservations({
+ admission,
+ pidAlive: (pid) => pid === process.pid,
+ cliRunEnded: () => false,
+ mindTurnIdle: () => false,
+ ...extra,
+ });
+
+ it('marks a dead-owner settled publication unsettled so receipt recovery can drain it, and leaves a live owner', () => {
+ const { admission } = gate();
+ const dead = admission.admit('settlement', 'Output publication');
+ const live = admission.admit('settlement', 'Output publication');
+ rewritePid(admission, dead.id, DEAD_PID);
+ const before = readOps(admission).find(op => op.id === dead.id);
+ const hold = admission.begin({ reason: 'Inspect refused publication', owner: 'Operator' }).hold;
+ const receipt = { disposition: 'operator-reconciled' };
+ const publish = vi.fn(() => receipt);
+ const replay = vi.fn(() => receipt);
+ // Settled residue is invisible to the receipt path. Skipping the stamp leaves this refusal in place.
+ expect(() => admission.reconcilePublicationRefusal({ hold, expected: before, publish, replay })).toThrow(/uncertain publication/);
+ expect(publish).not.toHaveBeenCalled();
+
+ expect(reconcile(admission)).toMatchObject({ marked: 1, released: 0 });
+ const stamped = readOps(admission).find(op => op.id === dead.id);
+ expect(stamped).toMatchObject({ unsettled: true, pid: DEAD_PID, resource: 'Output publication' });
+ expect(stamped.uncertaintyStamp).toMatch(/^[0-9a-f-]{36}$/i);
+ expect(readOps(admission).find(op => op.id === live.id).unsettled).toBeUndefined();
+
+ expect(admission.reconcilePublicationRefusal({ hold, expected: stamped, publish, replay })).toBe(receipt);
+ expect(publish).toHaveBeenCalledOnce();
+ expect(replay).not.toHaveBeenCalled();
+ expect(readOps(admission).some(op => op.id === dead.id)).toBe(false);
+ expect(readOps(admission).some(op => op.id === live.id)).toBe(true);
+ });
+
+ it('does not rotate an uncertainty stamp or stamp a non-publication settlement', () => {
+ const { admission } = gate();
+ const stamped = admission.admit('settlement', 'Output publication');
+ stamped.markUnsettled();
+ const other = admission.admit('settlement', 'Saving');
+ rewritePid(admission, stamped.id, DEAD_PID);
+ rewritePid(admission, other.id, DEAD_PID);
+ const stamp = readOps(admission).find(op => op.id === stamped.id).uncertaintyStamp;
+ expect(reconcile(admission)).toEqual({ marked: 0, released: 0 });
+ expect(readOps(admission).find(op => op.id === stamped.id).uncertaintyStamp).toBe(stamp);
+ expect(readOps(admission).find(op => op.id === other.id).unsettled).toBeUndefined();
+ });
+
+ it('releases only the dead-owner kinds whose owning record proves completion', () => {
+ const { admission } = gate();
+ const ended = admission.admit('cli-run', 'fixture-run-ended');
+ const open = admission.admit('cli-run', 'fixture-run-open');
+ const liveRun = admission.admit('cli-run', 'fixture-run-live');
+ const idleTurn = admission.admit('mind-turn', 'persistent-mind');
+ const provider = admission.admit('provider', 'fixture-source');
+ const media = admission.admit('media', 'fixture-render');
+ for (const op of [ended, open, idleTurn, provider, media]) rewritePid(admission, op.id, DEAD_PID);
+
+ expect(reconcile(admission, {
+ cliRunEnded: (runId) => runId === 'fixture-run-ended',
+ mindTurnIdle: () => true,
+ })).toEqual({ marked: 0, released: 2 });
+ const resources = admission.operations().map(op => op.resource).sort();
+ expect(resources).toEqual(['fixture-render', 'fixture-run-live', 'fixture-run-open', 'fixture-source']);
+ expect(admission.operations().find(op => op.id === liveRun.id)).toMatchObject({
+ resource: 'fixture-run-live',
+ pid: process.pid,
+ });
+ });
+
+ it('keeps a dead mind-turn while a turn is still active, and an unsettled cli-run even after it ended', () => {
+ const { admission } = gate();
+ const turn = admission.admit('mind-turn', 'persistent-mind');
+ const run = admission.admit('cli-run', 'fixture-run-ended');
+ run.markUnsettled();
+ rewritePid(admission, turn.id, DEAD_PID);
+ rewritePid(admission, run.id, DEAD_PID);
+ expect(reconcile(admission, {
+ cliRunEnded: () => true,
+ mindTurnIdle: () => false,
+ })).toEqual({ marked: 0, released: 0 });
+ expect(admission.operations().map(op => op.resource).sort()).toEqual(['fixture-run-ended', 'persistent-mind']);
+ });
+
+ it('skips quietly when the journal needs recovery', () => {
+ const { admission } = gate();
+ admission.admit('settlement', 'Output publication');
+ fs.writeFileSync(join(admission.directory, 'state.json'), '{not json');
+ expect(reconcile(admission)).toEqual({ marked: 0, released: 0 });
+ });
+});
+
+describe('owning-record proofs', () => {
+ it('treats a non-empty end time as a finished cli run and ignores escapes', () => {
+ const { admission } = gate();
+ const root = fs.mkdtempSync(join(tmpdir(), 'dead-owner-runs-'));
+ roots.push(root);
+ const write = (id, endTime) => {
+ fs.mkdirSync(join(root, id));
+ fs.writeFileSync(join(root, id, 'metadata.json'), JSON.stringify({ endTime }));
+ };
+ write('fixture-run-ended', '2026-01-01T00:00:00.000Z');
+ write('fixture-run-open', null);
+ for (const id of ['fixture-run-ended', 'fixture-run-open', 'fixture-run-missing', '../fixture-run-ended', '']) {
+ rewritePid(admission, admission.admit('cli-run', id).id, DEAD_PID);
+ }
+ expect(reconcileDeadOwnerReservations({
+ admission,
+ pidAlive: (pid) => pid === process.pid,
+ runsDir: root,
+ mindTurnIdle: () => false,
+ })).toEqual({ marked: 0, released: 1 });
+ expect(admission.operations().map(op => op.resource).sort()).toEqual(
+ ['', '../fixture-run-ended', 'fixture-run-missing', 'fixture-run-open'],
+ );
+ });
+
+ it('requires a mind object with no active turn', () => {
+ const { admission } = gate();
+ const root = fs.mkdtempSync(join(tmpdir(), 'dead-owner-mind-'));
+ roots.push(root);
+ const file = join(root, 'state.json');
+ const turn = admission.admit('mind-turn', 'persistent-mind');
+ rewritePid(admission, turn.id, DEAD_PID);
+ const kept = (state) => {
+ fs.writeFileSync(file, JSON.stringify(state));
+ expect(reconcileDeadOwnerReservations({
+ admission,
+ pidAlive: (pid) => pid === process.pid,
+ cliRunEnded: () => false,
+ mindStatePath: file,
+ })).toEqual({ marked: 0, released: 0 });
+ expect(admission.operations().some(op => op.id === turn.id)).toBe(true);
+ };
+ kept(null);
+ kept({});
+ kept({ persistentMind: [] });
+ kept({ persistentMind: { activeTurn: { id: 'fixture-turn' } } });
+ fs.writeFileSync(file, JSON.stringify({ persistentMind: {} }));
+ expect(reconcileDeadOwnerReservations({
+ admission,
+ pidAlive: (pid) => pid === process.pid,
+ cliRunEnded: () => false,
+ mindStatePath: file,
+ })).toEqual({ marked: 0, released: 1 });
+ expect(admission.operations().some(op => op.id === turn.id)).toBe(false);
+ const again = admission.admit('mind-turn', 'persistent-mind');
+ rewritePid(admission, again.id, DEAD_PID);
+ fs.writeFileSync(file, JSON.stringify({ persistentMind: { activeTurn: null } }));
+ expect(reconcileDeadOwnerReservations({
+ admission,
+ pidAlive: (pid) => pid === process.pid,
+ cliRunEnded: () => false,
+ mindStatePath: file,
+ })).toEqual({ marked: 0, released: 1 });
+ });
+});
diff --git a/server/services/persistentMindSupervisor.js b/server/services/persistentMindSupervisor.js
index b6c9790e5c..ad261dc5e3 100644
--- a/server/services/persistentMindSupervisor.js
+++ b/server/services/persistentMindSupervisor.js
@@ -1,4 +1,5 @@
import { maintenance } from '../lib/maintenanceAdmission.js';
+import { reconcileDeadOwnerReservations } from './deadOwnerReservations.js';
/**
* Persistent CoS mind supervisor.
*
@@ -1763,6 +1764,11 @@ export async function initializePersistentMindSupervisor() {
},
});
}
+ // The turn record is durable now. A dead mind-turn reservation is releasable
+ // only once this recovery has cleared an orphaned active turn.
+ reconcileDeadOwnerReservations({
+ mindTurnIdle: () => recovered.state != null && recovered.state.activeTurn == null,
+ });
if (recovered.state.enabled && recovered.state.started) {
armWatchdog();
await scheduleNextWake();
diff --git a/server/services/persistentMindSupervisor.test.js b/server/services/persistentMindSupervisor.test.js
index 8dcca4ca13..072457ee95 100644
--- a/server/services/persistentMindSupervisor.test.js
+++ b/server/services/persistentMindSupervisor.test.js
@@ -1,4 +1,6 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import { readFileSync, writeFileSync } from 'node:fs';
+import { join } from 'node:path';
import { createDefaultPersistentMindState, normalizePersistentMindState, PERSISTENT_MIND_LIMITS } from '../lib/persistentMind.js';
import { buildPersistentMindCallDenial } from '../lib/persistentMindTrajectory.js';
import { PROVIDER_USAGE_LIMIT_PAUSE_REASON } from '../lib/persistentMindUsageLimit.js';
@@ -212,6 +214,37 @@ describe('persistent mind supervisor', () => {
expect(run).not.toHaveBeenCalled();
});
+ it('releases a dead mind-turn after restart recovery and leaves an unproved provider', async () => {
+ const deadPid = 2147483646;
+ const turn = maintenance.admit('mind-turn', 'persistent-mind');
+ const live = maintenance.admit('mind-turn', 'persistent-mind');
+ const provider = maintenance.admit('provider', 'fixture-source');
+ const file = join(maintenance.directory, 'state.json');
+ const journal = JSON.parse(readFileSync(file, 'utf8'));
+ for (const id of [turn.id, provider.id]) journal.operations.find(op => op.id === id).pid = deadPid;
+ writeFileSync(file, JSON.stringify(journal));
+ mock.root.persistentMind = {
+ ...createDefaultPersistentMindState(),
+ enabled: true,
+ started: true,
+ status: 'thinking',
+ activeTurn: {
+ id: 'fixture-turn',
+ wake: { kind: 'message', message: { id: 'fixture-message', text: 'Ordinary.', createdAt: new Date(1).toISOString() } },
+ startedAt: new Date(1).toISOString(),
+ },
+ };
+ try {
+ const recovered = await supervisor.initializePersistentMindSupervisor();
+ expect(recovered.activeTurn).toBeNull();
+ const left = maintenance.operations().filter(op => [turn.id, live.id, provider.id].includes(op.id));
+ expect(left.map(op => op.id).sort()).toEqual([live.id, provider.id].sort());
+ } finally {
+ await maintenance.finishResource('mind-turn', 'persistent-mind');
+ await maintenance.finishResource('provider', 'fixture-source');
+ }
+ });
+
afterEach(() => vi.useRealTimers());
it.each([