Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

fix(eidoverse): retain completed controller delivery evidence (#10818)

[._.]/ Adam Eivy92c7efa10858
diff --git a/docs/STORAGE.md b/docs/STORAGE.md
index ad43b76eb3..47dc1f8b10 100644
--- a/docs/STORAGE.md
+++ b/docs/STORAGE.md
@@ -165,7 +165,7 @@ does not recover nonempty output or general failed saves; those still require th
- App scaffolds / cloned repos / browser profiles — `data/repos`, `data/browser-profile` — inherently filesystem-oriented.
- Eidoverse PortOS integration and world logs — `data/eidoverse/portos-world.json` stores the PortOS-owned private-world identity, versioned design selection, explicit display aliases keyed by opaque resource identity (never backfilled from records), user overrides, deterministic asset-resolution lock (paths, fingerprints, size, and provenance only; never model bytes), migration report, and last-good reconciliation checkpoint; `data/eidoverse/worlds` stores the external runtime's append-only world files. Both are `file-primary`, included in filesystem backups, and intentionally machine-local — never federated by PortOS. Guest travel sessions and live chat cursors are ephemeral transport state held only in memory; they are not a new record store or sync category. Explicit guest conversation follows the [guest-chat ADR](decisions/2026-09-05-eidoverse-guest-chat.md). PortOS selects the runtime's world-store location through `.env.portos` but does not edit the external checkout to build content. The separately licensed git checkouts and Eidoverse-owned asset library/cache live under the existing re-cloneable `data/repos/` backup class.
- Eidoverse world foundations ledger — `data/eidoverse/foundations.json` (#7455, #7461). The install-local record of every durable world foundation this instance authored OR inherited: its ownership layer (`vernacular` by default, `baseline` once promoted or inherited), the promotable `body`, the vernacular `style` layer that never leaves, provenance (opaque instance id + author kind), the last agent-free resilience verdict, and the last packaged promote candidate. A record this install pulled from a peer additionally carries an `inheritance` edge (`{ originInstanceId, foundationId, fingerprint, sourceInstanceId, inheritedAt }`) and is stored under a `peer:<originInstanceId>:<foundationId>` key disjoint from any locally-authored id, so it can never shadow or collide with a same-id local vernacular foundation. A write under that key is REFUSED, not applied, when the record already there was inherited from a different peer: the key is chosen by an origin the sender claims and hashes into its own fingerprint, so without that check a peer could replace this install's genuine copy of a third install's foundation (#7631). A record this install AUTHORED by building on an inherited one instead carries a `derivedFrom` edge (`{ originInstanceId, foundationId, fingerprint, derivedAt }`), which is the one provenance field that also rides the promote envelope — so a body copied from a peer is refused unless it says what it was derived from, and what it publishes is "derived from that install's X" rather than local authorship. The proposal → commit → promote/inherit lineage is not stored — `foundationLineage()` derives it on every read from fields the record already carries. `file-primary` and **intentionally machine-local — never federated**: the only thing authorized to leave the install is a packaged candidate envelope, and only for a foundation somebody explicitly promoted. `services/sharing/peerEidoverseFoundationSync.js` is the transport: it serves those envelopes (and only those — never a record, so never the `style` layer) at `GET /api/peer-sync/eidoverse-foundations` to registered outbound-enabled peers, and pulls a full-sync peer's offering into `recordEidoverseFoundationInheritance()`, which re-runs the whole accept-side gate and writes nothing on a refusal. Version-gated as `PORTOS_SCHEMA_VERSIONS.eidoverseFoundations`. No `data.reference/` seed and no migration — an absent file IS the empty ledger every install starts from, and the `inheritance`/`lineage`/`derivedFrom` additions are additive/nullable so an older file on disk reads back unchanged. Backed up with the rest of `data/eidoverse/`.
-- Eidoverse executable world controllers — `data/eidoverse/controllers.json` (#7456). The install-local set of controllers attached to this instance's private world: which shipped controller each install names (by REGISTRY ID — never a module path, so an install can never become an import), its bounded config, its cadence, whether it is armed, whether its effects are allowed to reach the world, and the durable state the controller advances one step per supervised tick. `file-primary` and **intentionally machine-local — never federated**: an install names this world's own entities and carries its author's intent, which the [machine-local privacy ADR](decisions/2026-08-08-privacy-records-machine-local.md) keeps off the federation layer; a controller is shared with a peer only as the BODY of a promoted foundation, which carries no install. No `data.reference/` seed and no migration — an absent file IS the empty set every install starts from. Backed up with the rest of `data/eidoverse/`.
+- Eidoverse executable world controllers — `data/eidoverse/controllers.json` (#7456). The install-local set of controllers attached to this instance's private world: which shipped controller each install names (by REGISTRY ID — never a module path, so an install can never become an import), its bounded config, its cadence, whether it is armed, whether its effects are allowed to reach the world, and the durable state the controller advances one step per supervised tick. `file-primary` and **intentionally machine-local — never federated**: an install names this world's own entities and carries its author's intent, which the [machine-local privacy ADR](decisions/2026-08-08-privacy-records-machine-local.md) keeps off the federation layer; a controller is shared with a peer only as the BODY of a promoted foundation, which carries no install. No `data.reference/` seed — an absent file IS the empty set every install starts from. Migration 425 adds nullable completed outbound-delivery history to existing stores without inferring old acknowledgements. Backed up with the rest of `data/eidoverse/`.
- Eidoverse observation visit marker — `data/eidoverse/observation.json` (#7457). What this install's mind had already seen the last time it toured its own world: the foundation ids, opaque peer ids, per-district status, and already-failing controller ids observed at that moment, plus when. It exists so "what is new since I last looked" survives a WAKE BOUNDARY — the nearest in-memory equivalents do not (the world chat cursor lives in a ring buffer a restart empties; the peer sync's `lastOfferingListHash` is per-peer change detection for a background sweep). `file-primary` and **intentionally machine-local — never federated**: a marker records this install's own mind's behavior, which the [machine-local privacy ADR](decisions/2026-08-08-privacy-records-machine-local.md) keeps off the federation layer, and it holds no body, style, or record content. No `data.reference/` seed and no migration — an absent file IS the never-observed state every install starts from, and that state is deliberately distinct from an empty one: the first observation reports `firstObservation: true` rather than declaring a settled world new. A marker stamped by a NEWER PortOS degrades to that same never-observed state rather than being diffed against fields this build does not understand. Backed up with the rest of `data/eidoverse/`.
- Sprite animation-track definitions — `data/sprites/animation-tracks.json` (#3152). A small hand-editable authoring config: which animation types exist beyond the compiled-in `walk` (label, directionality, frame/fps bounds, prompt template, and the on-disk `setKind` strings). `file-primary` rather than `db-primary` because it is **machine-local and inseparable from the on-disk sprite tree it describes** — a row names the `setKind` an approved set under `data/sprites/{id}/` already carries, so the two travel together or neither means anything — and because it has no cross-record queries, no relationships beyond `kinds` strings, and no sync cursor. Read synchronously and cached per process (`server/services/sprites/animationTrackStore.js`): `server/lib/validation.js` builds sprite Zod ranges from it at module load, so it must resolve without `await`. Seeded from `data.reference/sprites/animation-tracks.json` (migration 211), which is also the fallback read when no user copy exists yet. Backed up in full by the rsync snapshot.
- Quota-burn plan — `data/cos/quota-burn.json` (#3390). The install's burn plan: master switch, poll interval, and per-provider-family windows + an ordered list of steps, each a REFERENCE to a scheduled task (`data/cos/task-schedule.json` or the app job store) plus its per-invocation overrides. `file-primary` and **intentionally machine-local — never federated**: quota belongs to a particular machine and provider account, so a synced plan would have each peer spending against the other's window budget, and a step's scheduled-task reference names task types and managed apps that only exist on this machine. No sync cursor, no tombstone. Its four companions are `ephemeral-file` — regenerable telemetry, all safe to delete: `data/cos/quota-burn-dispatches.json` (per-window dispatch counts, self-pruning at 30 days), `data/cos/quota-burn-runs.json` (capped run log), `data/cos/quota-burn-inflight.json` (entries a burn job has enqueued but whose renders have not completed, self-pruning at 6 hours — deleting it only risks re-queueing a render already in flight), and `data/cos/quota-burn-denials.json` (per-family blocks from an observed provider refusal, cleared by the next successful burn or a 5-hour TTL — deleting it only risks one dispatch into a still-exhausted window). Backed up with the rest of `data/cos/`; a restored plan simply re-applies on this machine. See [Quota Burn](./QUOTA-BURN.md).
diff --git a/docs/features/eidoverse.md b/docs/features/eidoverse.md
index 8ab6bc4917..2f1828c581 100644
--- a/docs/features/eidoverse.md
+++ b/docs/features/eidoverse.md
@@ -1395,6 +1395,22 @@ rather than being erased by it.
local, never federated (`docs/STORAGE.md`). A controller crosses to a peer only
as the *body* of a promoted foundation, which carries no install.
+**Delivery evidence (#10818).** Both the common list and semantic inspect
+projection expose nullable `lastCompletedDelivery: { at, tick, ok, delivered,
+reason }`. It changes only after a `say` or `augment` delivery finishes and
+still matches the installed record's latest tick. `at` is that tick's time;
+read it and `tick` to assess the age of the acknowledgement. A refusal (including
+partial delivery) or thrown error replaces prior success. Quiet ticks, local
+notes, disabled delivery, step failures and in-flight requests leave the prior
+completed verdict intact. Arming/config edits preserve it; reinstall resets it.
+
+`lastDelivery` retains its latest-tick contract: quiet ticks report unknown
+(`ok: null`). Neither historical acknowledgement nor pulse/failure counters or
+`recentEffects` proposals prove current world state or visible rendering.
+Migration 425 upgrades older stores to schema version 2 with unknown (`null`)
+completed history, preserving their state and counters rather than manufacturing
+success from ambiguous old outcomes. There is no shipped store seed.
+
**Shipped controllers.** `ambient-beacon` counts ticks and marks a pulse every
Nth one (`announce` decides whether that pulse is spoken into the world or kept
as a note). `lantern-keeper` re-issues the `light` verb for a fixed set of world
diff --git a/scripts/lib/migrationOwnedPaths.js b/scripts/lib/migrationOwnedPaths.js
index e98b62940e..85ba2d94f9 100644
--- a/scripts/lib/migrationOwnedPaths.js
+++ b/scripts/lib/migrationOwnedPaths.js
@@ -37,6 +37,8 @@ export const MIGRATION_OWNED_PATHS = new Set([
// authored (or peer-inherited) work; a shipped seed would land first and the
// migration would rewrite shipped defaults where their ledger should be.
'eidoverse/foundations.json',
+ // Migration 425 retains completed-delivery history in the install-local store.
+ 'eidoverse/controllers.json',
// Migration 406 derives the CoS archive's completion-order projection from the
// install's own agent metadata. A shipped seed would land first (setup-data
// runs before migrations) and the backfill would then merge this install's
diff --git a/scripts/migrations/425-eidoverse-controller-completed-delivery.js b/scripts/migrations/425-eidoverse-controller-completed-delivery.js
new file mode 100644
index 0000000000..cf473260f3
--- /dev/null
+++ b/scripts/migrations/425-eidoverse-controller-completed-delivery.js
@@ -0,0 +1,45 @@
+/**
+ * Retain completed outbound delivery independently of latest-tick outcomes.
+ * Older outcomes/proposal summaries cannot prove a completed acknowledgement,
+ * so history starts unknown. Offline and provider-free; no derived-store seed.
+ */
+import { readFile } from 'node:fs/promises';
+import { join } from 'node:path';
+import { writeJsonAtomic } from './_lib.js';
+
+export default {
+ async up({ rootDir }) {
+ const file = join(rootDir, 'data', 'eidoverse', 'controllers.json');
+ const raw = await readFile(file, 'utf8').catch((error) => {
+ if (error.code === 'ENOENT') return null;
+ throw error;
+ });
+ if (raw === null) return { updated: 0, reason: 'no-controller-store' };
+ let parsed;
+ try {
+ parsed = JSON.parse(raw);
+ } catch {
+ console.warn('⚠️ migration 425: unreadable controller store; leaving it untouched');
+ return { updated: 0, reason: 'unreadable-store' };
+ }
+ if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)
+ || !parsed.installs || typeof parsed.installs !== 'object' || Array.isArray(parsed.installs)) {
+ return { updated: 0, reason: 'invalid-store' };
+ }
+ if (Number.isInteger(parsed.schemaVersion) && parsed.schemaVersion > 2) {
+ return { updated: 0, reason: 'newer-schema' };
+ }
+ let updated = 0;
+ for (const record of Object.values(parsed.installs)) {
+ if (!record || typeof record !== 'object' || Array.isArray(record)) continue;
+ if (!Object.hasOwn(record, 'lastCompletedDelivery')) {
+ record.lastCompletedDelivery = null;
+ updated += 1;
+ }
+ }
+ if (updated === 0 && parsed.schemaVersion === 2) return { updated: 0, reason: 'already-current' };
+ await writeJsonAtomic(file, { ...parsed, schemaVersion: 2 });
+ console.log(`🔁 migration 425: initialized unknown delivery history for ${updated} Eidoverse controller(s)`);
+ return { updated, reason: 'upgraded' };
+ },
+};
diff --git a/scripts/migrations/425-eidoverse-controller-completed-delivery.test.js b/scripts/migrations/425-eidoverse-controller-completed-delivery.test.js
new file mode 100644
index 0000000000..9d3d5d52c3
--- /dev/null
+++ b/scripts/migrations/425-eidoverse-controller-completed-delivery.test.js
@@ -0,0 +1,46 @@
+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { afterEach, beforeEach, describe, expect, it } from 'vitest';
+import migration from './425-eidoverse-controller-completed-delivery.js';
+
+let rootDir;
+let file;
+beforeEach(() => {
+ rootDir = mkdtempSync(join(tmpdir(), 'portos-migration-425-'));
+ mkdirSync(join(rootDir, 'data', 'eidoverse'), { recursive: true });
+ file = join(rootDir, 'data', 'eidoverse', 'controllers.json');
+});
+afterEach(() => rmSync(rootDir, { recursive: true, force: true }));
+const write = (value) => writeFileSync(file, JSON.stringify(value));
+const read = () => JSON.parse(readFileSync(file, 'utf8'));
+
+describe('migration 425 — completed controller delivery history', () => {
+ it('preserves legacy state/counters while leaving ambiguous delivery unknown, and reruns without writing', async () => {
+ const legacy = { tick: 8, state: { pulses: 4 }, consecutiveFailures: 1, consecutiveDeliveryFailures: 2,
+ lastOutcome: { effects: 1, delivered: 1, deliveryError: null }, recentEffects: [{ kind: 'say', summary: 'example pulse' }] };
+ write({ schemaVersion: 1, installs: { beacon: legacy } });
+ expect(await migration.up({ rootDir })).toEqual({ updated: 1, reason: 'upgraded' });
+ expect(read()).toEqual({ schemaVersion: 2, installs: { beacon: { ...legacy, lastCompletedDelivery: null } } });
+ const bytes = readFileSync(file, 'utf8');
+ expect(await migration.up({ rootDir })).toEqual({ updated: 0, reason: 'already-current' });
+ expect(readFileSync(file, 'utf8')).toBe(bytes);
+ });
+
+ it('does not replace existing completed history when upgrading an unstamped store', async () => {
+ const lastCompletedDelivery = { at: '2026-03-04T00:02:00.000Z', tick: 2, ok: false, delivered: 1, reason: 'partial refusal' };
+ write({ installs: { beacon: { lastCompletedDelivery } } });
+ await migration.up({ rootDir });
+ expect(read()).toEqual({ schemaVersion: 2, installs: { beacon: { lastCompletedDelivery } } });
+ });
+
+ it('leaves missing, malformed and newer stores untouched rather than creating or downgrading them', async () => {
+ expect(await migration.up({ rootDir })).toMatchObject({ reason: 'no-controller-store' });
+ expect(existsSync(file)).toBe(false);
+ for (const bytes of ['{broken', JSON.stringify({ installs: [] }), JSON.stringify({ schemaVersion: 3, installs: { beacon: { tick: 1 } } })]) {
+ writeFileSync(file, bytes);
+ await migration.up({ rootDir });
+ expect(readFileSync(file, 'utf8')).toBe(bytes);
+ }
+ });
+});
diff --git a/server/lib/eidoverseControllers.js b/server/lib/eidoverseControllers.js
index 3d7a49f86c..7d5543315a 100644
--- a/server/lib/eidoverseControllers.js
+++ b/server/lib/eidoverseControllers.js
@@ -179,6 +179,16 @@ const controllerTickOutcomeSchema = z.object({
deliveryError: z.string().trim().min(1).max(EIDOVERSE_CONTROLLER_LIMITS.reasonMax).nullable().default(null),
}).strict();
+/** A completed outbound attempt, retained independently of quiet/failed ticks.
+ * `at` is the originating tick's time; no legacy proposal implies a verdict. */
+const controllerCompletedDeliverySchema = z.object({
+ at: isoDateSchema,
+ tick: z.number().int().min(0),
+ ok: z.boolean(),
+ delivered: z.number().int().min(0),
+ reason: z.string().trim().min(1).max(EIDOVERSE_CONTROLLER_LIMITS.reasonMax).nullable().default(null),
+}).strict();
+
/** The persisted install record. Machine-local; nothing here federates. */
export const eidoverseControllerRecordSchema = eidoverseControllerInstallSchema.extend({
installedBy: installedBySchema,
@@ -192,6 +202,7 @@ export const eidoverseControllerRecordSchema = eidoverseControllerInstallSchema.
nextTickAt: isoDateSchema,
lastTickAt: isoDateSchema.nullable().default(null),
lastOutcome: controllerTickOutcomeSchema.nullable().default(null),
+ lastCompletedDelivery: controllerCompletedDeliverySchema.nullable().default(null),
recentEffects: z.array(z.object({
at: isoDateSchema,
tick: z.number().int().min(0),
@@ -432,6 +443,7 @@ export function summarizeControllerInstall(record, { includeState = false } = {}
// produced nothing to deliver" — neither is a failure, so neither
// collapses into `false`.
lastDelivery: summarizeLastDelivery(record),
+ lastCompletedDelivery: record?.lastCompletedDelivery ?? null,
consecutiveDeliveryFailures: record?.consecutiveDeliveryFailures ?? 0,
disarmedReason: record?.disarmedReason ?? null,
note: record?.note ?? null,
diff --git a/server/services/cosToolRegistry.js b/server/services/cosToolRegistry.js
index 13d5e6385a..b9d4aa20b7 100644
--- a/server/services/cosToolRegistry.js
+++ b/server/services/cosToolRegistry.js
@@ -546,8 +546,8 @@ const eidoverseCreativeTools = [
// that can build in the world should be able to see what is already ticking in
// it, and seeing is what makes the writes usable rather than guesswork.
const eidoverseControllerTools = [
- ['controllers', 'List the executable world controllers PortOS ships and the ones installed here — each install\'s cadence, whether it is armed, whether its effects reach the world, its tick count, and why the supervisor disarmed it if it did. Read this before installing: `controllerId` must be one of the registry ids it returns. This list never carries `config`/`state` — use eidoverse.inspect-controller for one install\'s full record.', z.object({}).strict(), ['manageEidoverse'], 'read'],
- ['inspect-controller', 'Read one installed controller\'s full record by its install id, including its `config` and its accumulated `state` — the detail eidoverse.controllers deliberately omits. Read this before deciding whether to arm-controller, retire-controller, or re-install over an id you did not author.', eidoverseControllerIdParamSchema, ['manageEidoverse'], 'read'],
+ ['controllers', 'List the executable world controllers PortOS ships and the ones installed here — each install\'s cadence, whether it is armed, whether its effects reach the world, its tick count, and why the supervisor disarmed it if it did. Read this before installing: `controllerId` must be one of the registry ids it returns. Read `lastCompletedDelivery` for the last completed outbound verdict and its tick/time age; it is historical acknowledgement, not current world state or visible rendering. `lastDelivery` describes only the latest tick and is unknown on quiet ticks. This list never carries `config`/`state` — use eidoverse.inspect-controller for one install\'s full record.', z.object({}).strict(), ['manageEidoverse'], 'read'],
+ ['inspect-controller', 'Read one installed controller\'s full record by its install id, including its `config` and its accumulated `state` — the detail eidoverse.controllers deliberately omits. Read `lastCompletedDelivery` for the last completed outbound verdict and its tick/time age; historical acknowledgement does not prove current world state or visible rendering. `lastDelivery` describes only the latest tick. Read this before deciding whether to arm-controller, retire-controller, or re-install over an id you did not author.', eidoverseControllerIdParamSchema, ['manageEidoverse'], 'read'],
['install-controller', 'Attach a bounded controller to the private world so it keeps ticking between your wakes. `controllerId` names one of the ids eidoverse.controllers returns — a controller is never a path or code you supply. Every tick is synchronous and provider-free, and `deliverEffects` (default false) decides whether its effects reach the world at all. This is a REQUEST: the result is `outcome: "installed"` only when it landed, and any other outcome means nothing is running — read `reasons`. The first tick is one interval away, never immediate. Re-installing an existing id REBUILDS its state from the new config — inspect it first with eidoverse.inspect-controller if you want to keep what it has accumulated.', eidoverseControllerInstallSchema, ['manageEidoverse', 'installEidoverseControllers'], 'write'],
['arm-controller', 'Pause or resume one installed controller without losing the state it has accumulated. Use this to resume a controller the supervisor disarmed after repeated failures, once you have fixed what it was failing on — re-installing would work too but starts its state over.', eidoverseControllerArmSchema, ['manageEidoverse', 'installEidoverseControllers'], 'write'],
['retire-controller', 'Stop and remove one installed controller by its install id. Retiring deletes the install and its accumulated state; re-installing starts it fresh.', eidoverseControllerIdParamSchema, ['manageEidoverse', 'installEidoverseControllers'], 'write'],
diff --git a/server/services/eidoverseControllerRuntime.js b/server/services/eidoverseControllerRuntime.js
index 47aa091c99..f2c18934f7 100644
--- a/server/services/eidoverseControllerRuntime.js
+++ b/server/services/eidoverseControllerRuntime.js
@@ -36,7 +36,7 @@
* this install's world entities and carries its author's intent, and the
* machine-local privacy ADR keeps records off the federation layer. There is
* no `data.reference/` seed — an absent file IS the empty set every install
- * starts from, so no migration is owed.
+ * starts from. Migration 425 adds unknown completed-delivery history to older stores.
*/
import { join } from 'node:path';
@@ -55,7 +55,7 @@ import { findControllerDefinitionById } from './eidoverseControllerRegistry.js';
import { cancel, getEvent, schedule } from './eventScheduler.js';
/** Storage-layout version stamped on `data/eidoverse/controllers.json`. */
-const STORE_SCHEMA_VERSION = 1;
+const STORE_SCHEMA_VERSION = 2;
const SCHEDULER_EVENT_ID = 'eidoverse-controller-tick';
@@ -106,6 +106,13 @@ async function readInstalls() {
const raw = await readJSONFile(storeFile(), null, { allowArray: false, strict: true });
const installs = raw && typeof raw === 'object' && raw.installs && typeof raw.installs === 'object' ? { ...raw.installs } : {};
const schemaVersion = raw && typeof raw === 'object' && Number.isInteger(raw.schemaVersion) ? raw.schemaVersion : STORE_SCHEMA_VERSION;
+ // Additive read compatibility even before the offline migration has run.
+ // Never derive acknowledgements from old outcomes or proposal summaries.
+ if (schemaVersion <= STORE_SCHEMA_VERSION) {
+ for (const [id, record] of Object.entries(installs)) {
+ installs[id] = { lastCompletedDelivery: null, ...record };
+ }
+ }
return { schemaVersion, installs };
}
@@ -208,6 +215,7 @@ export async function installEidoverseController(input, {
nextTickAt: nextControllerTickAt(authored, nowMs),
lastTickAt: existing?.lastTickAt ?? null,
lastOutcome: null,
+ lastCompletedDelivery: null,
recentEffects: [],
consecutiveFailures: 0,
consecutiveDeliveryFailures: 0,
@@ -361,19 +369,22 @@ async function deliverControllerEffects(effects, { signal } = {}) {
let delivered = 0;
let error = null;
for (const effect of outbound) {
- if (effect.kind === 'say') {
- // `sayInEidoverseWorld` has no rewrite/refusal dimension of its own —
- // it either resolves (the world acked it) or throws, which the caller
- // in `deliverPassEffects` already treats as a delivery failure.
- await world.sayInEidoverseWorld(effect.text, { signal });
- delivered += 1;
- continue;
- }
- const result = await world.augmentEidoverseWorld(effect.operations, { signal });
- delivered += result.applied;
- if (result.success === false && error === null) {
- const refusal = result.operations.find((operation) => operation.outcome === 'refused');
- error = refusal?.reason ?? 'Eidoverse refused part of this augment batch.';
+ try {
+ if (effect.kind === 'say') {
+ // A resolved say is acknowledged; preserve that count if a later
+ // effect throws, rather than losing already-committed world work.
+ await world.sayInEidoverseWorld(effect.text, { signal });
+ delivered += 1;
+ continue;
+ }
+ const result = await world.augmentEidoverseWorld(effect.operations, { signal });
+ delivered += result.applied;
+ if (result.success === false && error === null) {
+ const refusal = result.operations.find((operation) => operation.outcome === 'refused');
+ error = refusal?.reason ?? 'Eidoverse refused part of this augment batch.';
+ }
+ } catch (failure) {
+ return { delivered, error: String(failure?.message ?? failure) || 'World delivery failed.' };
}
}
return { delivered, error };
@@ -505,14 +516,17 @@ async function stepInstall(record, { nowMs, at, resolveDefinition, storeSchemaVe
async function deliverPassEffects(stepped, { deliver, signal }) {
const deliveries = [];
for (const { record, effects } of stepped) {
- if (effects.length === 0) continue;
+ const outbound = effects.filter((effect) => effect.kind === 'say' || effect.kind === 'augment');
+ if (outbound.length === 0) continue;
+ const at = record.lastOutcome.at;
try {
- const { delivered, error } = await deliver(effects, { signal });
- deliveries.push({ id: record.id, tick: record.tick, delivered, error });
+ const { delivered, error } = await deliver(outbound, { signal });
+ const reason = error == null ? null : (String(error).trim() || 'World delivery failed.').slice(0, EIDOVERSE_CONTROLLER_LIMITS.reasonMax);
+ deliveries.push({ id: record.id, at, tick: record.tick, delivered, error: reason });
} catch (error) {
- const message = String(error.message).slice(0, EIDOVERSE_CONTROLLER_LIMITS.reasonMax);
+ const message = (String(error?.message ?? error).trim() || 'World delivery failed.').slice(0, EIDOVERSE_CONTROLLER_LIMITS.reasonMax);
console.error(`❌ ${LOG_PREFIX}: "${record.id}" tick ${record.tick} stepped but could not reach the world: ${message}`);
- deliveries.push({ id: record.id, tick: record.tick, delivered: 0, error: message });
+ deliveries.push({ id: record.id, at, tick: record.tick, delivered: 0, error: message });
}
}
return deliveries;
@@ -521,8 +535,8 @@ async function deliverPassEffects(stepped, { deliver, signal }) {
/**
* Fold delivery outcomes back onto the records they belong to.
*
- * Each patch is applied only while the record's `lastOutcome.tick` still
- * matches the tick that produced those effects: an install or retire landing
+ * Each patch is applied only while the record's `lastOutcome.tick` and `at`
+ * still match the tick that produced those effects: an install or retire landing
* between the two phases has replaced what the delivery was about, and
* stamping a delivery count onto it would describe work that record never did.
*
@@ -541,9 +555,9 @@ async function recordDeliveries(deliveries) {
const { schemaVersion: storeSchemaVersion, installs } = await readInstalls();
let changed = false;
let gateMoved = false;
- for (const { id, tick, delivered, error } of deliveries) {
+ for (const { id, at, tick, delivered, error } of deliveries) {
const current = installs[id];
- if (!current || current.lastOutcome?.tick !== tick) continue;
+ if (!current || current.lastOutcome?.tick !== tick || current.lastOutcome.at !== at) continue;
const consecutiveDeliveryFailures = error ? (current.consecutiveDeliveryFailures ?? 0) + 1 : 0;
const exhausted = error && consecutiveDeliveryFailures >= EIDOVERSE_CONTROLLER_LIMITS.maxConsecutiveFailures;
if (exhausted) {
@@ -552,6 +566,7 @@ async function recordDeliveries(deliveries) {
installs[id] = {
...current,
lastOutcome: { ...current.lastOutcome, delivered, deliveryError: error },
+ lastCompletedDelivery: { at, tick, ok: !error, delivered, reason: error },
consecutiveDeliveryFailures,
...(exhausted ? {
armed: false,
diff --git a/server/services/eidoverseControllerRuntime.test.js b/server/services/eidoverseControllerRuntime.test.js
index eba0502bbe..117d1a3a58 100644
--- a/server/services/eidoverseControllerRuntime.test.js
+++ b/server/services/eidoverseControllerRuntime.test.js
@@ -21,7 +21,7 @@ import { rmSync } from 'node:fs';
import { join } from 'node:path';
import { z } from 'zod';
import { cleanupTempDataRoots, lazyTempDataRoot, makePathsProxy } from '../lib/mockPathsDataRoot.js';
-import { summarizeControllerInstall } from '../lib/eidoverseControllers.js';
+import { eidoverseControllerRecordSchema, summarizeControllerInstall } from '../lib/eidoverseControllers.js';
vi.mock('../lib/fileUtils.js', async (importOriginal) => makePathsProxy(await importOriginal(), {
dataRoot: () => lazyTempDataRoot('portos-eidoverse-controllers-'),
@@ -214,7 +214,7 @@ describe('the supervised tick path', () => {
it('counts the step and records the failure when the world cannot be reached', async () => {
const deliver = vi.fn(async () => { throw new Error('world host is down'); });
- await install({ config: { label: 'plaza', pulseEveryTicks: 1 }, deliverEffects: true });
+ await install({ config: { label: 'plaza', pulseEveryTicks: 1, announce: true }, deliverEffects: true });
await runSupervisorPasses(5, { deliver });
@@ -260,6 +260,7 @@ describe('the supervised tick path', () => {
const summary = summarizeControllerInstall(record);
expect(summary.lastTickOk).toBe(true);
expect(summary.lastDelivery).toEqual({ ok: false, delivered: 0, reason: 'unknown entity id' });
+ expect(summary.lastCompletedDelivery).toEqual({ at: at(3 * MINUTE), tick: 3, ok: false, delivered: 0, reason: 'unknown entity id' });
});
it('does not disarm on a rewritten delivery — the world landed different args, not nothing', async () => {
@@ -520,3 +521,104 @@ describe('updateEidoverseControllerConfig (#7629)', () => {
expect((await updateEidoverseControllerConfig('nope', {})).outcome).toBe('unknown-install');
});
});
+
+// #10818: proposals and latest-tick health cannot stand in for retained,
+// completed outbound evidence. Drive persistence and projections together.
+describe('completed outbound delivery evidence', () => {
+ it('survives multiple quiet/failed ticks and restart, preserves arming/config edits, and resets on reinstall', async () => {
+ await install({ tickIntervalMs: MINUTE, config: { label: 'plaza', pulseEveryTicks: 2, announce: true }, deliverEffects: true });
+ const deliver = vi.fn(async () => ({ delivered: 1, error: null }));
+ await runSupervisorPasses(3, { deliver });
+ const completed = { at: at(2 * MINUTE), tick: 2, ok: true, delivered: 1, reason: null };
+ expect(deliver).toHaveBeenCalledTimes(1);
+ const quiet = summarizeControllerInstall(await getEidoverseControllerInstall('plaza-beacon'));
+ expect(quiet.lastDelivery).toEqual({ ok: null, delivered: 0, reason: null });
+ expect(quiet.lastCompletedDelivery).toEqual(completed);
+ await updateEidoverseControllerConfig('plaza-beacon', { label: 'plaza', pulseEveryTicks: 100, announce: true }, { now: at(3 * MINUTE) });
+ await runSupervisorPasses(2, { fromMs: 3 * MINUTE, deliver });
+ const broken = { configSchema: z.record(z.unknown()), step: () => { throw new Error('example step failure'); } };
+ await runSupervisorPasses(1, { fromMs: 5 * MINUTE, resolveDefinition: () => broken, deliver });
+ expect((await getEidoverseControllerInstall('plaza-beacon')).lastCompletedDelivery).toEqual(completed);
+ await setEidoverseControllerArmed('plaza-beacon', false, { now: at(6 * MINUTE) });
+ await setEidoverseControllerArmed('plaza-beacon', true, { now: at(6 * MINUTE) });
+ __resetEidoverseControllerRuntimeForTests();
+ vi.resetModules();
+ const restarted = await import('./eidoverseControllerRuntime.js');
+ const record = await restarted.getEidoverseControllerInstall('plaza-beacon');
+ expect(record.lastCompletedDelivery).toEqual(completed);
+ expect(eidoverseControllerRecordSchema.safeParse(record).success).toBe(true);
+ const listed = (await restarted.listEidoverseControllers()).installs[0];
+ for (const projection of [summarizeControllerInstall(listed), summarizeControllerInstall(record, { includeState: true })]) {
+ expect(projection.lastCompletedDelivery).toEqual(completed);
+ expect(projection.lastDelivery).toEqual({ ok: null, delivered: 0, reason: null });
+ }
+ expect((await readJSONFile(storeFile(), null)).installs['plaza-beacon'].lastCompletedDelivery).toEqual(completed);
+ const reinstalled = await restarted.installEidoverseController({ id: 'plaza-beacon', controllerId: 'ambient-beacon', config: {} }, { now: at(7 * MINUTE) });
+ expect(reinstalled.install.lastCompletedDelivery).toBeNull();
+ restarted.__resetEidoverseControllerRuntimeForTests();
+ });
+
+ it('never creates completed outbound evidence from local notes or disabled delivery', async () => {
+ const deliver = vi.fn(async () => ({ delivered: 1, error: null }));
+ await install({ tickIntervalMs: MINUTE, config: { pulseEveryTicks: 1, announce: false }, deliverEffects: true });
+ await install({ id: 'disabled-beacon', tickIntervalMs: MINUTE, config: { pulseEveryTicks: 1, announce: true }, deliverEffects: false });
+ await runSupervisorPasses(2, { deliver });
+ expect(deliver).not.toHaveBeenCalled();
+ for (const record of (await listEidoverseControllers()).installs) {
+ expect(record.state.pulses).toBe(2);
+ expect(record.recentEffects).toHaveLength(2);
+ expect(record.lastCompletedDelivery).toBeNull();
+ }
+ });
+
+ it('replaces success with partial refusal and then thrown failure while retaining acknowledged counts', async () => {
+ const definition = { id: 'example-outbound', configSchema: z.object({}).strict(), createState: () => ({}), step: (state) => ({ state, effects: [
+ { kind: 'say', text: 'example acknowledgement' },
+ { kind: 'augment', operations: [{ verb: 'light', args: { id: 'example-lamp' } }] },
+ ] }) };
+ await install({ controllerId: definition.id, tickIntervalMs: MINUTE, config: {}, deliverEffects: true }, { resolveDefinition: () => definition });
+ sayInEidoverseWorldMock.mockResolvedValue(undefined);
+ augmentEidoverseWorldMock.mockResolvedValueOnce({ success: true, applied: 1, operations: [] })
+ .mockResolvedValueOnce({ success: false, applied: 1, operations: [{ outcome: 'refused', reason: 'example partial refusal' }] })
+ .mockRejectedValue(new Error('example transport failure'));
+ await runSupervisorPasses(1, { resolveDefinition: () => definition });
+ expect((await getEidoverseControllerInstall('plaza-beacon')).lastCompletedDelivery).toEqual({ at: at(MINUTE), tick: 1, ok: true, delivered: 2, reason: null });
+ await runSupervisorPasses(1, { fromMs: MINUTE, resolveDefinition: () => definition });
+ expect((await getEidoverseControllerInstall('plaza-beacon')).lastCompletedDelivery).toEqual({ at: at(2 * MINUTE), tick: 2, ok: false, delivered: 2, reason: 'example partial refusal' });
+ await runSupervisorPasses(2, { fromMs: 2 * MINUTE, resolveDefinition: () => definition });
+ const record = await getEidoverseControllerInstall('plaza-beacon');
+ expect(record.lastCompletedDelivery).toEqual({ at: at(4 * MINUTE), tick: 4, ok: false, delivered: 1, reason: 'example transport failure' });
+ expect(record.consecutiveDeliveryFailures).toBe(3);
+ expect(record.armed).toBe(false);
+ expect(isEidoverseControllerSupervisorRegistered()).toBe(false);
+ });
+
+ it('keeps in-flight history unchanged and rejects completion after reinstall or a different outcome with the same tick', async () => {
+ await install({ tickIntervalMs: MINUTE, config: { pulseEveryTicks: 1, announce: true }, deliverEffects: true });
+ await runSupervisorPasses(1, { deliver: async () => ({ delivered: 1, error: null }) });
+ const completed = (await getEidoverseControllerInstall('plaza-beacon')).lastCompletedDelivery;
+ for (const replace of ['different-outcome', 'reinstall']) {
+ let finish;
+ let started;
+ const entered = new Promise((resolve) => { started = resolve; });
+ const pending = tickEidoverseControllers({ now: at(replace === 'reinstall' ? 3 * MINUTE : 2 * MINUTE), deliver: () => {
+ started();
+ return new Promise((resolve) => { finish = resolve; });
+ } });
+ await entered;
+ expect((await getEidoverseControllerInstall('plaza-beacon')).lastCompletedDelivery).toEqual(completed);
+ if (replace === 'reinstall') {
+ await install({ tickIntervalMs: MINUTE, deliverEffects: true }, { now: at(3 * MINUTE) });
+ } else {
+ const raw = await readJSONFile(storeFile(), null);
+ raw.installs['plaza-beacon'].lastOutcome.at = at(2 * MINUTE + 1);
+ await atomicWrite(storeFile(), raw);
+ }
+ finish({ delivered: 2, error: null });
+ await pending;
+ const record = await getEidoverseControllerInstall('plaza-beacon');
+ expect(record.lastCompletedDelivery).toEqual(replace === 'reinstall' ? null : completed);
+ expect(record.lastOutcome?.delivered ?? 0).toBe(0);
+ }
+ });
+});