Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
/**
* The install-local ledger of Eidoverse world foundations — which durable
* artifacts this instance authored or inherited, which ownership layer each
* one sits in, and the last promote candidate packaged from (or inheritance
* edge built for) it (#7455, #7461, epic #7453).
*
* `server/lib/eidoverseFoundations.js` owns the ownership model and the
* packaging/validation gate; this module is the persistence, clock, and
* assay-execution shell around it. Everything AUTHORED here lands at the
* `vernacular` layer, so an artifact is local until somebody promotes it on
* purpose. The one other way a record enters this ledger is as a local copy
* of a foundation a PEER promoted — `recordEidoverseFoundationInheritance()` —
* which lands directly at `baseline` and carries an `inherited-from` edge
* rather than this install's own authorship.
*
* Between the two sits DERIVATION (#7631): a record this install authored by
* building on an inherited copy. It is local work — it packages and promotes
* like any other — but it carries, and publishes, a `derived-from` edge back to
* the origin. `recordEidoverseFoundation()` refuses a body that matches an
* inherited one and names no such edge, which is what closed the path from
* 'load a peer's foundation into the authoring form' to 're-share it as this
* install's own work'.
*
* **The promote gate runs the agent-free assay; it never accepts a verdict.**
* `packageEidoverseFoundationCandidate()` DERIVES the sandbox from the
* foundation's own body (`lib/eidoverseFoundationSandbox.js`, #7625), replays it
* through `runResilienceAssay()` (#7460), and packages against the verdict it
* just produced. Deriving rather than resolving a caller-supplied
* `contributionId` is what makes 'what was evaluated' and 'what gets promoted'
* the same object: the id used to be free text, so a local build could name a
* shipped demo fixture and promote on evidence about the fixture.
*
* A caller — a route, a mind tool — therefore cannot assert that
* its build survived its author's absence; it can only ask for the check. That
* is the same proposal-versus-consequence separation #7454 gave the
* construction tools, applied to promotion.
*
* Storage is `data/eidoverse/foundations.json` — `file-primary` and MACHINE
* LOCAL, the same class as `portos-world.json` beside it (`docs/STORAGE.md`).
* PortOS never federates this file: the only thing authorized to leave the
* install is a packaged candidate envelope, and even that leaves only once
* somebody promotes the foundation — `listPromotedFoundationCandidates()`
* below is the whole of what a peer can ever read
* (`services/sharing/peerEidoverseFoundationSync.js`). There is no `data.reference/` seed — an absent file is
* an empty ledger, which is the correct state for every install that has never
* authored or inherited a foundation, so no migration is owed.
*
* **Withdrawal is a first-class act, and it propagates (#7632).** Promotion used
* to be one-way: a peer that pulled a foundation kept that copy forever, because
* the offering only ever ADDED and the sweep converged upward toward the
* sender's list. Re-authoring already de-promoted a record locally, which
* silently dropped it from the offering and left every receiver holding a
* `baseline` copy of bytes the author had retracted. `withdrawEidoverseFoundation()`
* below makes that retraction explicit and records a TOMBSTONE — keyed on the
* candidate's content-addressed fingerprint, the one identifier that means the
* same thing on every install — which rides the offering beside the candidates
* so a receiver DROPS the copy instead of merely ceasing to re-pull it.
* `applyEidoverseFoundationTombstones()` is that receiving half.
*/
import { join } from 'node:path';
import { PATHS, atomicWrite, readJSONFile } from '../lib/fileUtils.js';
import { createMutex } from '../lib/asyncMutex.js';
import { getPortosVersion } from '../lib/schemaVersions.js';
import {
DEFAULT_EIDOVERSE_FOUNDATION_LAYER,
assayEvidenceFromVerdict,
derivedContributionId,
eidoverseFoundationInputSchema,
foundationFromInheritedCandidate,
foundationLedgerKey,
foundationLineage,
inheritedFoundationStorageKey,
packageFoundationCandidate,
planFoundationAdoption,
resolveFoundationDerivation,
verifyFoundationCandidate,
} from '../lib/eidoverseFoundations.js';
import { ServerError } from '../lib/errorHandler.js';
import { generateDistrictTemplatePlacement } from '../lib/eidoverseCreativeToolkit.js';
import {
clearTombstone,
normalizeTombstones,
recordTombstone,
tombstoneTimestamp,
} from '../lib/tombstones.js';
import { RESILIENCE_DISTURBANCES, runResilienceAssay } from './eidoverseResilienceAssay.js';
import { foundationSandbox } from '../lib/eidoverseFoundationSandbox.js';
import { findControllerDefinitionById } from './eidoverseControllerRegistry.js';
/**
* Storage-layout version stamped on `data/eidoverse/foundations.json`.
*
* Still 1 with the `tombstones` list added (#7632): the key is ADDITIVE and
* compatible in both directions — a PortOS that predates it ignores the key,
* and this one reads a file without it as an empty tombstone list, which is
* exactly right for an install that has never withdrawn anything. Bumping
* would advertise an incompatibility that does not exist. The one asymmetry
* worth naming: an older PortOS that WRITES this file back drops the
* tombstones, so a downgrade-then-upgrade loses pending retractions that had
* not yet reached a peer. That is a lost withdrawal, not a corrupted ledger,
* and re-withdrawing the record records it again.
*/
const LEDGER_SCHEMA_VERSION = 1;
/**
* Tombstones are keyed on the promote candidate's `fingerprint`, NOT on the
* foundation id. The id is local and two installs can legitimately carry the
* same one; the fingerprint is a sha256 over the canonicalized envelope, so it
* names one specific published body and means the same thing everywhere. It is
* also exactly what the receiver already stores on its inherited copy
* (`inheritance.fingerprint`), so a match needs no extra index.
*/
const TOMBSTONE_KEY_FIELD = 'fingerprint';
/**
* Capped independently of `OFFERING_ENTRY_CAP` in the transport, and far above
* any realistic withdrawal history. A retraction must never be displaced to
* make room for a publication — the whole point is that it outranks one.
*/
const TOMBSTONE_LIMIT = 200;
const tombstoneOptions = { keyField: TOMBSTONE_KEY_FIELD, limit: TOMBSTONE_LIMIT };
// Read through `PATHS` per call, NOT `dataPath()`: a suite redirects the data
// root by proxying this module's `fileUtils` import, and `dataPath()` resolves
// against `paths.js`'s own `PATHS` binding, which that proxy never sees — so
// the helper would send every test at the live install's ledger. Per call
// rather than captured at module load for the same reason.
const ledgerFile = () => join(PATHS.data, 'eidoverse', 'foundations.json');
const withLedgerLock = createMutex();
/**
* Strict read: a `foundations.json` this process cannot parse must NOT read as
* "no foundations yet", because the very next write would then replace the
* user's ledger with an empty one. `strict: true` throws on unreadable bytes,
* while a genuinely ABSENT file still reads as the empty ledger it is.
*/
async function readLedger() {
const raw = await readJSONFile(ledgerFile(), null, { allowArray: false, strict: true });
const stored = raw && typeof raw === 'object' ? raw : {};
return {
foundations: stored.foundations && typeof stored.foundations === 'object' ? { ...stored.foundations } : {},
// Normalized on READ as well as write: this list is the one part of the
// ledger a peer's offering also writes, and `foundations.json` is a file a
// human can edit. An entry without a usable key or stamp could never win a
// comparison anyway, so dropping it here keeps every downstream reader from
// having to re-check the shape.
tombstones: normalizeTombstones(stored.tombstones, TOMBSTONE_KEY_FIELD),
};
}
/** The foundations map alone, for the read-only paths that never write back. */
const readFoundations = async () => (await readLedger()).foundations;
// Both halves always written together: a `writeFoundations(foundations)` that
// defaulted the tombstones would erase every pending retraction on the next
// ordinary authoring write, which is precisely the silent-loss shape #7632
// exists to close.
async function writeLedger({ foundations, tombstones }) {
await atomicWrite(ledgerFile(), {
schemaVersion: LEDGER_SCHEMA_VERSION,
foundations,
tombstones: normalizeTombstones(tombstones, TOMBSTONE_KEY_FIELD),
});
}
/**
* Every foundation this install knows about, most recently updated first —
* both what this install authored and any local copy it holds of a peer's
* promoted foundation (#7461). Each entry carries its derived `lineage`
* (proposal → commit → promote/inherit), computed fresh on every read rather
* than stored, per `foundationLineage()`'s own header.
*/
export async function listEidoverseFoundations() {
const foundations = Object.values(await readFoundations())
.sort((a, b) => String(b.updatedAt || '').localeCompare(String(a.updatedAt || '')));
const counts = foundations.reduce((totals, entry) => ({
vernacular: totals.vernacular + (entry.layer === 'vernacular' ? 1 : 0),
baseline: totals.baseline + (entry.layer === 'baseline' ? 1 : 0),
candidates: totals.candidates + (entry.candidate ? 1 : 0),
// A subset of `baseline`: this install pulled the copy from a peer rather
// than promoting its own work into it.
inherited: totals.inherited + (entry.inheritance ? 1 : 0),
}), { vernacular: 0, baseline: 0, candidates: 0, inherited: 0 });
return {
schemaVersion: LEDGER_SCHEMA_VERSION,
counts,
foundations: foundations.map((entry) => ({ ...entry, lineage: foundationLineage(entry) })),
};
}
/**
* The promote envelopes this install OFFERS to its federated peers (#7455) —
* the read behind `GET /api/peer-sync/eidoverse-foundations`.
*
* Three filters, each load-bearing:
*
* - `layer === 'baseline'` — only what somebody deliberately promoted. A
* `vernacular` record may carry a packaged candidate (packaging is the step
* BEFORE promoting); packaging is "show me whether this would pass", not
* "publish this", and serving those would turn a dry run into a broadcast.
* - `!inheritance` — `baseline` covers BOTH what this install promoted and a
* local copy of a peer's, so offering every `baseline` record would re-share
* another install's work. That matches `packageEidoverseFoundationCandidate`,
* which already refuses an inherited record ("promotion re-shares only
* foundations this install authored"): a peer that wants a third install's
* foundation pulls it from the install that authored it.
* - the candidate still VERIFIES — the envelope was gated when it was
* promoted, but `foundations.json` is a file a human can edit and a partial
* hand-edit is exactly how a machine name or a local path would end up
* inside a body that was clean when it was packaged. Refuse-never-redact
* applies outbound as well as inbound, so a candidate that no longer passes
* is dropped from the offering (and logged) rather than served.
*
* Returns the envelopes only — never the ledger records, which carry the
* `style` layer that is not authorized to leave this install.
*/
export async function listPromotedFoundationCandidates() {
const { foundations, tombstones } = await readLedger();
const offerable = Object.values(foundations)
.filter((entry) => entry?.layer === 'baseline' && !entry.inheritance && entry.candidate);
const candidates = [];
for (const entry of offerable) {
// A fourth filter (#7632), and a backstop rather than a live path: promoting
// clears the fingerprint's tombstone, so the two lists cannot normally
// disagree. A hand-edited ledger can make them disagree, and an offering
// that both published and retracted one fingerprint would resolve
// differently on every receiver depending on which list it applied first.
// The retraction wins, because refuse-never-redact runs outbound too.
if (tombstoneTimestamp(tombstones, entry.candidate.fingerprint, TOMBSTONE_KEY_FIELD) !== null) {
console.warn(`⚠️ Eidoverse foundation "${entry.id}" is promoted but its candidate is withdrawn — withholding it from the peer offering`);
continue;
}
const verified = verifyFoundationCandidate(entry.candidate, { requiredDisturbances: RESILIENCE_DISTURBANCES });
if (!verified.valid) {
console.warn(`⚠️ Eidoverse foundation "${entry.id}" is promoted but its stored candidate no longer passes the promote gate — withholding it from the peer offering (${verified.reasons.length} reason(s))`);
continue;
}
candidates.push(entry.candidate);
}
// Fingerprint order: content-addressed, so the offering (and the listHash a
// receiver short-circuits on) is stable across ledger read order.
return candidates.sort((a, b) => String(a.fingerprint).localeCompare(String(b.fingerprint)));
}
export async function getEidoverseFoundation(id) {
const found = (await readFoundations())[id] || null;
return found ? { ...found, lineage: foundationLineage(found) } : null;
}
/**
* One foundation by REFERENCE — `{ id, originInstanceId }` — which is the only
* way an inherited copy of a peer's foundation is addressable (#7626).
*
* `getEidoverseFoundation(id)` above reaches exactly the records this install
* AUTHORED, because a plain id is the ledger key for those and only those. A
* local copy of a peer's foundation lives under
* `inheritedFoundationStorageKey()`'s disjoint `peer:` namespace, so until
* this existed every read-one caller — the HTTP route, and therefore every
* mind surface built on it — could reach a local record and nothing else. The
* list endpoint returned whole records, so a HUMAN could read an inherited
* body in the panel while a Mind on the same install could not obtain it at
* all.
*
* Deliberately NOT a widening of the id: `originInstanceId` is its own field,
* and `null` means "the record authored here", never "whichever matches".
*/
export async function getEidoverseFoundationByRef(ref) {
return getEidoverseFoundation(foundationLedgerKey(ref));
}
/**
* ADOPT a foundation this install inherited from a peer — stand its `body` up
* as something that actually runs here, carrying a `derived-from` edge back to
* the origin (#7626).
*
* This is the verb epic #7453's success signal ("a mind uses what another mind
* left") always needed and never had. Inheriting stored a row: no runtime on
* either install read a foundation's `body`, so the only way to use a peer's
* contribution was a human reading raw JSON out of the Foundations panel and
* retyping it — which is precisely the "go read the author's transcript" path
* the epic set out to replace, and which keeps no attribution at all.
*
* What adoption does is decided per KIND by `planFoundationAdoption()`, and a
* kind with no interpreter here is REFUSED BY NAME rather than silently
* "adopted" into nothing. Only `controller` is adoptable today, and even then
* the peer names WHICH SHIPPED CONTROLLER to run, never code: `definitionId` is
* resolved against this install's own fixed registry by
* `installEidoverseController()`, so adopting can never execute a peer's
* bytes. The install lands DISARMED with `deliverEffects: false` — arming a
* peer's controller and letting it speak in the world stay separate, local
* decisions.
*
* A refusal is a RESULT with its reasons, the shape every other gate in this
* module uses.
*
* @returns {Promise<{ outcome: 'adopted'|'refused'|'unknown-foundation', install: object|null, reasons: string[] }>}
*/
export async function adoptEidoverseFoundation(ref, { installedBy = 'user' } = {}) {
const key = foundationLedgerKey(ref);
const record = await getEidoverseFoundation(key);
if (!record) return { outcome: 'unknown-foundation', install: null, reasons: [`no foundation is recorded under "${key}"`] };
const planned = planFoundationAdoption(record);
if (planned.outcome !== 'plan') return { outcome: 'refused', install: null, reasons: planned.reasons };
const { getEidoverseControllerInstall, installEidoverseController } = await import('./eidoverseControllerRuntime.js');
// The install id is the foundation's own id, so re-adopting the same
// foundation updates one install instead of accumulating copies. That makes
// it possible for the id to name a controller this install already stood up
// for some OTHER reason, and installing over it would silently retarget a
// running controller at a peer's config. Refuse that by name; only an
// install already derived from this same foundation is re-adoptable.
const occupant = await getEidoverseControllerInstall(planned.plan.install.id);
const sameSource = occupant?.derivedFrom
&& occupant.derivedFrom.originInstanceId === planned.plan.derivedFrom.originInstanceId
&& occupant.derivedFrom.foundationId === planned.plan.derivedFrom.foundationId;
if (occupant && !sameSource) {
return {
outcome: 'refused',
install: null,
reasons: [`a controller is already installed under "${planned.plan.install.id}" and was not adopted from this foundation — retire it first rather than letting an adopt overwrite a controller already running here`],
};
}
const result = await installEidoverseController(planned.plan.install, { installedBy, derivedFrom: planned.plan.derivedFrom });
if (result.outcome !== 'installed') return { outcome: 'refused', install: null, reasons: result.reasons };
console.log(`🧬 Eidoverse: adopted foundation "${record.id}" from origin ${planned.plan.derivedFrom.originInstanceId} as disarmed controller "${result.install.id}"`);
return { outcome: 'adopted', install: result.install, reasons: [] };
}
/**
* A `district-template` foundation's `body.placement` is DERIVED, never
* trusted verbatim from the caller (#7627): `generateDistrictTemplatePlacement()`
* is deterministic and seeded, so replaying `{layoutId, anchor, seed}` here —
* the same replay a peer runs on inherit — is what makes the module's "same
* inputs reproduce the same geometry" claim actually hold on the install that
* authored it. A caller-supplied `placement` that disagrees with the
* derivation is REPLACED wholesale, not merged: a stale or hand-edited value
* must never ride along looking like the derivation's own output.
*
* A body missing `layoutId` or `anchor` (an older shape, or a foundation kind
* this doesn't apply to) is returned unchanged — this only expands the one
* documented generative shape, never invents one.
*/
function deriveDistrictTemplateBody(authored) {
if (authored.kind !== 'district-template') return authored.body;
const { layoutId, anchor, propCount, seed, facing } = authored.body;
if (typeof layoutId !== 'string' || !Array.isArray(anchor)) return authored.body;
const placement = generateDistrictTemplatePlacement({ layoutId, anchor, propCount, seed, facing });
return { ...authored.body, placement };
}
/**
* Record (or re-author) a local vernacular foundation.
*
* `originInstanceId` is supplied by the caller rather than read here so this
* module stays a single-purpose store; `getInstanceId()` from
* `services/instanceIdentity.js` is what every caller passes.
*
* Re-authoring an existing id keeps its original `createdAt` and origin — an
* artifact's provenance is part of its identity — and CLEARS the packaged
* candidate and the recorded assay, because both described the previous body.
* Carrying a passing verdict across an edit is exactly the "it worked when I
* narrated it" claim the agent-free harness exists to refuse.
*
* For the same reason a re-authored `baseline` foundation drops back to
* `vernacular` with `promotedAt` cleared: the body that was promoted no longer
* exists on this install, so a record still claiming to be part of the shared
* baseline would be describing bytes nobody has. It is local work again until
* somebody promotes the new body. (Keeping the `baseline` stamp instead would
* also be a dead end — `layerPromoteRefusal` refuses to package a `baseline`
* foundation, so the edit could never be published.)
*/
export async function recordEidoverseFoundation(input, { originInstanceId, now = new Date().toISOString() } = {}) {
const authored = eidoverseFoundationInputSchema.parse(input);
return withLedgerLock(async () => {
const { foundations, tombstones } = await readLedger();
const existing = foundations[authored.id] || null;
// The republish guard (#7631). `inheritance` below is a field THIS path
// clears, so it could never have stopped a peer's foundation being saved
// back through the authoring surface and promoted as local work; the body
// itself is what gets checked, against every inherited copy this install
// holds. A refusal is thrown rather than returned because both callers —
// the route and the mind tool — treat this function's return value as the
// recorded foundation, and a verdict shape would make "refused" look like
// a saved record to anything that did not read the new field.
// Digest the body as it will be STORED, not as it arrived (#7627 + #7631).
// A district-template's `placement` is derived here rather than supplied, so
// a caller who re-submits only the `{layoutId, anchor, seed}` recipe of an
// inherited template — which is exactly what `eidoverse.draft-foundation`
// hands back — would otherwise digest a body the inherited copy never had,
// slip past the republish guard, and then be stored byte-identical to it.
const body = deriveDistrictTemplateBody(authored);
const derivation = resolveFoundationDerivation({
claim: authored.derivedFrom,
body,
inheritedRecords: Object.values(foundations),
existingEdge: existing?.derivedFrom || null,
now,
});
if (derivation.refusal) throw new ServerError(derivation.refusal, { status: 409 });
const record = {
id: authored.id,
layer: DEFAULT_EIDOVERSE_FOUNDATION_LAYER,
kind: authored.kind,
title: authored.title,
summary: authored.summary,
// DERIVED, never authored (#7625): the assay replays this foundation's
// own body, so the label that binds evidence to it is computed from that
// body — the STORED one derived just above, which is what the sandbox
// will actually replay. A caller that could name the label could point
// the gate at a shipped demo fixture and inherit its passing verdict.
contributionId: derivedContributionId({ kind: authored.kind, id: authored.id, body }),
body,
style: authored.style,
provenance: existing?.provenance || { originInstanceId, authorKind: authored.authorKind, createdAt: now },
disclosure: authored.disclosure,
assay: null,
candidate: null,
promotedAt: null,
// Always `null` here: this path authors LOCAL work. A ledger key this
// function writes is always the plain id, never
// `inheritedFoundationStorageKey()`'s namespace, so it can never
// overwrite (or be confused with) a copy pulled from a peer.
inheritance: null,
derivedFrom: derivation.edge,
updatedAt: now,
};
foundations[authored.id] = record;
// Re-authoring a PROMOTED foundation is a retraction of the published body,
// and it is the one the issue that prompted #7632 opens with: the record
// silently dropped back to `vernacular`, left the offering, and every peer
// that had pulled it kept a `baseline` copy of bytes this install no longer
// has. The candidate being replaced is what those peers hold, so tombstone
// its fingerprint — the new body publishes under a new one when (and if)
// somebody promotes it.
await writeLedger({ foundations, tombstones: tombstoneForReplacedCandidate(tombstones, existing, now) });
return record;
});
}
/**
* Tombstone the candidate a write is about to replace, but ONLY when that
* candidate was actually published. A merely PACKAGED candidate on a
* `vernacular` record never left the install — `listPromotedFoundationCandidates()`
* refuses to serve one — so tombstoning it would broadcast a retraction for a
* fingerprint no peer could ever hold, burning a tombstone slot for nothing.
*/
const wasPublished = (entry) => entry?.layer === 'baseline' && !entry.inheritance && typeof entry.candidate?.fingerprint === 'string';
function tombstoneForReplacedCandidate(tombstones, existing, now) {
if (!wasPublished(existing)) return tombstones;
return recordTombstone(tombstones, existing.candidate.fingerprint, { ...tombstoneOptions, deletedAt: now });
}
const verdict = (outcome, reasons) => ({ outcome, candidate: null, assay: null, reasons, findings: [] });
const unknownFoundation = (id) => verdict('unknown-foundation', [`no foundation is recorded under "${id}"`]);
/**
* Run the agent-free assay against a recorded foundation and, if it and every
* other gate pass, package the promote candidate onto its record.
*
* A refusal is a RESULT, not an exception: "this build is not ready to leave
* the install" is ordinary, expected output the caller shows the author with
* its reasons. The assay verdict is persisted either way — a failing verdict is
* the diagnostic that says what to fix — while a refusal clears any candidate
* packaged earlier, since the verdict that vouched for it no longer holds.
*
* @returns {Promise<{ outcome: 'packaged'|'refused'|'unknown-foundation', candidate: object|null, assay: object|null, reasons: string[], findings: Array }>}
*/
export async function packageEidoverseFoundationCandidate(id, { now = new Date().toISOString() } = {}) {
const existing = await getEidoverseFoundation(id);
if (!existing) return unknownFoundation(id);
// Refused before resolving or replaying ANYTHING: `packageFoundationCandidate`
// (the pure lib) refuses an inherited record too, but only after running the
// assay against the body it carries. That body came from a PEER — replaying
// it here would be exactly the "run arbitrary code pulled from a peer" this
// install's own assay harness exists to keep off every OTHER install, even
// though the derivation only ever executes code PortOS itself ships.
if (existing.inheritance) {
return verdict('refused', [`inherited from another install (${existing.inheritance.originInstanceId}) — promotion re-shares only foundations this install authored`]);
}
// Outside the ledger lock: replaying a contribution is the slow part, and it
// reads nothing from the ledger. The lock below re-reads the record and
// re-checks that the body has not been re-authored underneath the verdict.
//
// The sandbox is derived from THIS FOUNDATION'S OWN BODY (#7625). It used to
// be resolved by the id the author typed, so the gate replayed whatever
// module was named and the verdict described that module rather than this
// build. A body with no derivable sandbox is refused with the reason, which
// is the honest answer: nothing can replay it without its author.
const { contribution, refusal } = await foundationSandbox(existing, { findControllerDefinition: findControllerDefinitionById });
if (refusal) return verdict('refused', [refusal]);
const assay = assayEvidenceFromVerdict(runResilienceAssay(contribution), { ranAt: now });
// Read after the early returns: `getPortosVersion()` re-reads and re-parses
// package.json on every call, and a request naming an id this install never
// authored should not pay for it.
const portosVersion = await getPortosVersion();
return withLedgerLock(async () => {
const { foundations, tombstones } = await readLedger();
const current = foundations[id];
if (!current) return unknownFoundation(id);
if (current.updatedAt !== existing.updatedAt) {
return verdict('refused', ['the foundation was re-authored while the assay was running — package it again']);
}
// `contributionId` is re-derived here rather than read off the record, so a
// record written before #7625 (or restored from a backup that predates it)
// heals on its next package attempt instead of failing the pure gate's
// binding backstop forever.
const result = packageFoundationCandidate({
record: { ...current, contributionId: contribution.id, assay, candidate: null },
requiredDisturbances: RESILIENCE_DISTURBANCES,
portosVersion,
now,
});
// `updatedAt` tracks the BODY's authorship, not assay runs: bumping it here
// would reorder the list on a refusal and make the optimistic check above
// report a re-authoring that never happened. A refusal also clears any
// previously packaged candidate — the verdict that vouched for it no
// longer holds, even though the bytes are unchanged.
foundations[id] = { ...current, contributionId: contribution.id, assay, candidate: result.candidate };
await writeLedger({ foundations, tombstones });
return { ...result, assay };
});
}
/**
* Promote a packaged foundation into this install's shared baseline population.
*
* Promotion re-packages first and publishes the candidate it just produced, so
* it can never rest on a stored verdict: every gate — ownership, the agent-free
* assay, the style-leak scan, federation safety, the content-addressed
* fingerprint — is re-run against the body as it stands right now. A refusal is
* a RESULT with its reasons, exactly as packaging is; nothing moves layer.
*
* `baseline` means "this install offers this foundation to the shared
* population", and it is literally the set served from:
* `listPromotedFoundationCandidates()` reads it for the peer-facing offering
* at `GET /api/peer-sync/eidoverse-foundations`. The promoted record keeps its
* `style` locally — only the candidate envelope, which has no style layer at
* all, is authorized to cross, per the federated Eidoverse foundations ADR
* (docs/decisions/2026-09-18-federated-eidoverse-foundations.md).
*
* @returns {Promise<{ outcome: 'promoted'|'refused'|'unknown-foundation', promoted: boolean, foundation: object|null, candidate: object|null, assay: object|null, reasons: string[], findings: Array }>}
*/
export async function promoteEidoverseFoundation(id, { now = new Date().toISOString() } = {}) {
const packaged = await packageEidoverseFoundationCandidate(id, { now });
if (packaged.outcome !== 'packaged') return { ...packaged, promoted: false, foundation: null };
return withLedgerLock(async () => {
const { foundations, tombstones } = await readLedger();
const current = foundations[id];
if (!current) return { ...unknownFoundation(id), promoted: false, foundation: null };
// Packaging released the ledger lock before this one was taken, so an
// author (or a mind) could have re-authored the body in the gap — which
// clears the candidate. Publishing then would ship bytes nobody gated.
if (current.candidate?.fingerprint !== packaged.candidate.fingerprint) {
return {
outcome: 'refused',
promoted: false,
foundation: null,
candidate: null,
assay: packaged.assay,
reasons: ['the foundation was re-authored while it was being packaged — promote it again'],
findings: [],
};
}
const foundation = { ...current, layer: 'baseline', promotedAt: now };
foundations[id] = foundation;
// Re-publishing a body that was withdrawn earlier clears its tombstone, or
// the offering would carry a retraction and the republication of the very
// same fingerprint at once and every receiver would reap what it just
// inherited, forever. This is `tombstones.js`'s re-create case: the
// fingerprint is content-addressed, so an identical body promoted again IS
// the same key, and only an explicit clear can distinguish "I changed my
// mind" from "I never withdrew it".
await writeLedger({ foundations, tombstones: clearTombstone(tombstones, packaged.candidate.fingerprint, TOMBSTONE_KEY_FIELD) });
return { ...packaged, outcome: 'promoted', promoted: true, foundation };
});
}
/**
* Accept a promote candidate this install pulled from a peer and store it as
* a local baseline copy, carrying an `inherited-from` edge back to its origin
* (#7461).
*
* The caller is the pull/inherit transport in
* `services/sharing/peerEidoverseFoundationSync.js` (#7455), which has fetched
* the envelope from a full-sync peer and can vouch for `sourceInstanceId` —
* the peer it pulled FROM, which differs from the candidate's own
* `originInstanceId` on a foundation re-shared through more than one hop (see
* `foundationFromInheritedCandidate()`).
*
* `localInstanceId` is supplied by the caller rather than read here, matching
* `recordEidoverseFoundation()`'s own convention: this module stays a
* single-purpose store, and the future transport already has to resolve its
* own instance id to make the pull in the first place.
*
* Every gate `promoteEidoverseFoundation()` runs on the SENDING side —
* schema, content-addressed fingerprint, the assay evidence already recorded
* in the envelope, and federation safety (PII, credentials, machine identity)
* — runs again here on the RECEIVING side: a payload handed over by a peer is
* exactly the untrusted input `verifyFoundationCandidate()` exists for. A
* candidate that fails is refused outright and nothing is written — never
* stored redacted, and never allowed to shadow (or read as) a local
* vernacular foundation of the same id, because it is stored under
* `inheritedFoundationStorageKey()`'s disjoint namespace instead.
*
* @returns {Promise<{ outcome: 'inherited'|'refused', foundation: object|null, reasons: string[], findings: Array }>}
*/
export async function recordEidoverseFoundationInheritance(candidate, { sourceInstanceId, localInstanceId, now = new Date().toISOString() } = {}) {
const built = foundationFromInheritedCandidate({
candidate, requiredDisturbances: RESILIENCE_DISTURBANCES, sourceInstanceId, localInstanceId, now,
});
if (built.outcome !== 'inherited') return built;
return withLedgerLock(async () => {
const { foundations, tombstones } = await readLedger();
const originInstanceId = built.foundation.provenance.originInstanceId;
const key = inheritedFoundationStorageKey(originInstanceId, built.foundation.id);
// The storage key is chosen by `provenance.originInstanceId` — a field the
// SENDER wrote and hashed into its own fingerprint, so re-fingerprinting an
// altered origin costs a forger nothing and `verifyFoundationCandidate()`
// (which only checks self-consistency) cannot see it. Without this check a
// peer offering an envelope that claims ANOTHER install's origin silently
// replaced this install's genuine copy of that install's foundation (#7631).
//
// Divergence is the whole signal: `listPromotedFoundationCandidates()`
// filters out inherited records, so no shipped install re-shares a
// foundation it did not author, and a genuine record under this key can
// therefore only ever have come from the origin itself. Re-offering from
// the SAME peer stays an ordinary update.
const held = foundations[key];
if (held?.inheritance && held.inheritance.sourceInstanceId !== sourceInstanceId) {
console.warn(`⚠️ Eidoverse foundation "${built.foundation.id}" attributed to origin ${originInstanceId} is already held from peer ${held.inheritance.sourceInstanceId} — refusing the copy offered by peer ${sourceInstanceId} rather than overwriting it`);
return {
outcome: 'refused',
foundation: null,
reasons: [`this install already holds "${built.foundation.id}" from origin ${originInstanceId} by way of peer ${held.inheritance.sourceInstanceId}; peer ${sourceInstanceId} offered a different copy under that same origin — refusing rather than overwriting the record already attributed to it`],
findings: [],
};
}
foundations[key] = built.foundation;
await writeLedger({ foundations, tombstones });
return { outcome: 'inherited', foundation: { ...built.foundation, lineage: foundationLineage(built.foundation) }, reasons: [], findings: [] };
});
}
/**
* Withdraw a foundation this install PROMOTED, retracting it from the shared
* population (#7632).
*
* Two things happen, and the second is the one that makes this different from
* every de-promotion that came before it:
*
* 1. The record drops back to `vernacular` with `promotedAt` and `candidate`
* cleared — the same transition re-authoring performs. It leaves the
* offering immediately, because `listPromotedFoundationCandidates()` serves
* `baseline` records only.
* 2. The withdrawn candidate's fingerprint is TOMBSTONED, and the tombstone
* rides the offering. Without it, a peer that already pulled the foundation
* keeps its `baseline` copy indefinitely: the sweep converges upward toward
* whatever the sender currently lists and has no way to express "and drop
* what I no longer list", because a foundation legitimately absent from one
* offering is indistinguishable from one the sender never had.
*
* The body is KEPT. Withdrawal un-publishes; deleting the local work is
* `deleteEidoverseFoundation()` and a separate decision.
*
* A peer running a version that predates the tombstone key keeps its copy and
* says so in its log. That is unavoidable by construction — it is why the
* retraction is documented as best-effort rather than a guarantee — and it is
* the correct failure mode: it degrades to today's behavior rather than
* mis-reading a key it does not know.
*
* @returns {Promise<{ outcome: 'withdrawn'|'not-promoted'|'refused'|'unknown-foundation', foundation: object|null, reasons: string[] }>}
*/
export async function withdrawEidoverseFoundation(id, { now = new Date().toISOString() } = {}) {
return withLedgerLock(async () => {
const { foundations, tombstones } = await readLedger();
const current = foundations[id];
if (!current) return { outcome: 'unknown-foundation', foundation: null, reasons: [`no foundation is recorded under "${id}"`] };
if (current.inheritance) {
return {
outcome: 'refused',
foundation: null,
reasons: [`"${id}" is a local copy of a foundation install ${current.inheritance.originInstanceId} promoted — this install never published it, so it has nothing to withdraw (delete the copy instead)`],
};
}
if (current.layer !== 'baseline') {
return { outcome: 'not-promoted', foundation: { ...current, lineage: foundationLineage(current) }, reasons: [`"${id}" is not promoted, so there is nothing to retract`] };
}
// `updatedAt` is untouched: it tracks the BODY's authorship, and a
// withdrawal changes what this install publishes, not what it wrote.
const foundation = { ...current, layer: 'vernacular', candidate: null, promotedAt: null };
foundations[id] = foundation;
await writeLedger({ foundations, tombstones: tombstoneForReplacedCandidate(tombstones, current, now) });
console.log(`🚫 Withdrew Eidoverse foundation "${current.title}" from this install's promoted population — peers drop their copy on the next sweep`);
return { outcome: 'withdrawn', foundation: { ...foundation, lineage: foundationLineage(foundation) }, reasons: [] };
});
}
/**
* The retractions this install advertises beside its offering — read by
* `buildEidoverseFoundationOffering()` in the peer transport.
*
* Deliberately the WHOLE capped list, not a delta: a receiver that was offline
* across several withdrawals has no cursor, and a tombstone for a fingerprint
* nobody holds is a silent no-op on the receiving side, so re-sending one costs
* nothing but bytes.
*/
export async function listWithdrawnFoundationTombstones() {
return (await readLedger()).tombstones;
}
/**
* Apply a peer's retractions: drop every local copy this install inherited FROM
* that peer whose fingerprint the peer has tombstoned (#7632). The receiving
* half of `withdrawEidoverseFoundation()`.
*
* **Scoped to the peer that sent them.** A tombstone authorizes dropping only a
* record this install pulled from that same peer (`inheritance.sourceInstanceId`).
* Otherwise any registered peer could retract a third install's foundation by
* naming its fingerprint — a fingerprint is public inside the federation the
* moment it is offered, so it is an identifier, never a credential.
*
* A tombstone for a fingerprint this install does not hold is a no-op, not an
* error: a receiver that never pulled the foundation (or already dropped it) is
* already in the state the retraction asks for.
*
* The peer's tombstones are NOT persisted here. This install never re-offers an
* inherited record, so it can never relay one; and the sender keeps advertising
* the tombstone while its candidate stays absent, so there is no window in
* which a dropped copy could be re-inherited. Storing them would only let a
* peer grow this install's capped list and push out its own retractions.
*
* @returns {Promise<{ removed: number }>}
*/
export async function applyEidoverseFoundationTombstones(peerTombstones, { sourceInstanceId } = {}) {
const incoming = normalizeTombstones(peerTombstones, TOMBSTONE_KEY_FIELD);
if (incoming.length === 0 || !sourceInstanceId) return { removed: 0 };
return withLedgerLock(async () => {
const { foundations, tombstones } = await readLedger();
const doomed = Object.entries(foundations).filter(([, entry]) => entry?.inheritance
&& entry.inheritance.sourceInstanceId === sourceInstanceId
&& tombstoneTimestamp(incoming, entry.inheritance.fingerprint, TOMBSTONE_KEY_FIELD) !== null);
if (doomed.length === 0) return { removed: 0 };
for (const [key, entry] of doomed) {
delete foundations[key];
console.log(`🗑️ Dropped inherited Eidoverse foundation "${entry.title}" — install ${entry.inheritance.originInstanceId} withdrew it`);
}
await writeLedger({ foundations, tombstones });
return { removed: doomed.length };
});
}
/**
* Delete a foundation record outright — local work or an inherited copy.
*
* An inherited copy is addressed as `{ id, originInstanceId }` rather than by
* its raw `peer:<origin>:<id>` storage key, so the ledger's key grammar stays
* an implementation detail of this module instead of something a caller has to
* assemble (and could assemble wrong).
*
* **Deleting a PROMOTED local record withdraws it first.** Dropping the row
* without a tombstone would orphan every peer's copy permanently, which is the
* exact failure withdrawal exists to prevent — and it would be reached by the
* most natural gesture a regretful author makes ("remove this").
*
* @returns {Promise<{ outcome: 'deleted'|'unknown-foundation', foundation: object|null, withdrawn: boolean }>}
*/
export async function deleteEidoverseFoundation(id, { originInstanceId = null, now = new Date().toISOString() } = {}) {
return withLedgerLock(async () => {
const { foundations, tombstones } = await readLedger();
const key = originInstanceId ? inheritedFoundationStorageKey(originInstanceId, id) : id;
const current = foundations[key];
if (!current) return { outcome: 'unknown-foundation', foundation: null, withdrawn: false };
const withdrawn = wasPublished(current);
delete foundations[key];
await writeLedger({ foundations, tombstones: tombstoneForReplacedCandidate(tombstones, current, now) });
console.log(`🗑️ Deleted Eidoverse foundation "${current.title}"${withdrawn ? ' and withdrew it from the peer offering' : ''}`);
return { outcome: 'deleted', foundation: current, withdrawn };
});
}