Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: merge eye exam copies by connected identity and keep retired ids as aliases (#12432)
[._.]/ Adam Eivyb4820896d8d4
diff --git a/scripts/migrations/429-eye-exam-legacy-row-ids.js b/scripts/migrations/429-eye-exam-legacy-row-ids.jsindex 5e5491c551..67eff3f0ed 100644--- a/scripts/migrations/429-eye-exam-legacy-row-ids.js+++ b/scripts/migrations/429-eye-exam-legacy-row-ids.js@@ -2,8 +2,8 @@* (#12432). Before this, each machine stamped an id-less eyes.json row with its* own `randomUUID()`, so syncing peers appended each other's copy of the same* exam. Rows without `createdAt` whose content is identical are copies of one- * exam: they collapse to one row carrying the `legacy-` id every peer derives- * from that content. A single row keeps its id — it may have been minted before+ * exam: they collapse to one row under the smallest of their ids, the others+ * kept as `aliasIds`. A single row keeps its id — it may have been minted before* #12432 and edited — and still meets a peer's differently-stamped copy through* the sync merge's content key. Input-gated on eyes.json; ships no seed.* Idempotent: the output has no id-less rows and no duplicates left. */diff --git a/scripts/migrations/429-eye-exam-legacy-row-ids.test.js b/scripts/migrations/429-eye-exam-legacy-row-ids.test.jsindex 79679d045f..34abfae41b 100644--- a/scripts/migrations/429-eye-exam-legacy-row-ids.test.js+++ b/scripts/migrations/429-eye-exam-legacy-row-ids.test.js@@ -16,7 +16,7 @@ const seed = async (doc) => {};const read = async () => JSON.parse(await readFile(eyesPath(), 'utf8'));-it('collapses copies of one legacy exam to its content id, stamps id-less rows, keeps every other id, and is idempotent', async () => {+it('collapses copies of one legacy exam under its smallest id, stamps id-less rows, keeps every other id, and is idempotent', async () => {const legacy = { date: '2024-05-01', leftSphere: -1, rightSphere: -1.25 };const idless = { date: '2023-02-01', leftSphere: -0.5 };// A pre-#12432 addEyeExam row: random id, no createdAt, edited since. Its id is kept.@@ -39,7 +39,8 @@ it('collapses copies of one legacy exam to its content id, stamps id-less rows,expect(doc.exams).toEqual([{ id: legacyRowId(idless), ...idless },preFixEdited,- { id: legacyRowId(legacy), ...legacy },+ // The peer's stamp stays reachable as an alias, so its edits and deletes still land.+ { id: '11111111-0000-4000-8000-000000000001', aliasIds: ['33333333-0000-4000-8000-000000000003'], ...legacy },minted,]);diff --git a/server/lib/README.md b/server/lib/README.mdindex 7b5f773f13..4549f1cf9c 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -250,7 +250,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `modelInventory.js` | Identity AND durable shape of a downloaded-model inventory row. `modelInventoryId(backend, key)` mints the `hf:` / `lora:` / `ollama:` / `lmstudio:` id and `parseModelInventoryId` splits it back on the FIRST colon (an Ollama tag like `qwen3:8b` carries one of its own). `hfInventoryRow` / `loraInventoryRow` / `localModelInventoryRow` build the row a backend contributes, and `presentInventoryRow` re-derives the live fields (residency, inventory-verified) that a STORED row can never answer. Shared by the scan that mints rows from disk (`services/systemResources.js`) and the manifest that records them at the install chokepoints (`services/modelManifest.js`) — a drifted copy of a row's risk, manage path or delete action is a delete button pointed at the wrong place. Pure and dependency-free, so the install services reach the builders without pulling the scan's module graph. || `videoFailure.js` | `normalizeVideoFailure` classifies and scrubs local video causes; `createVideoDiagnosticTail` retains bounded stdout/stderr diagnostics without grouping bare exit codes. || `maintenanceSequence.js` | The maintenance ladder shared by both runners: `MAINTENANCE_TASK_ORDER` / `MAINTENANCE_ORDER_GUIDANCE`, the interleaved `MAINTENANCE_SEQUENCE_TYPES` (a perpetual `claim-issue` drain between every pair of audits), `maintenanceStepParams` (audits file issues, documentation does the work), and `buildMaintenanceSteps` (fix mode runs audits directly with one final drain; issue filing can omit claim jobs) which emits quota-burn-shaped run-once steps so `quotaBurnInvoke.js` can run a step from the Schedule tab's manual run or a Quota Burn family plan alike. Pure. |-| `meatspaceRowIdentity.js` | Cross-machine identity for MeatSpace health rows (#12432): `legacyRowId` derives a `legacy-<sha256>` key from a row's content, `rowIdentityKeys` lists the keys a row answers to (its id, plus the content key when it has no `createdAt`), `mergeRowCopies` collapses copies of one row for the `row-union` snapshot merge, `preferRowCopy` is the total per-row `updatedAt` LWW order, `normalizeLegacyRows` stamps and dedupes legacy eye exams (also migration 429), and `ROW_TOMBSTONES_KEY` names the per-file deletion markers. Pure. |+| `meatspaceRowIdentity.js` | Cross-machine identity for MeatSpace health rows (#12432): `legacyRowId` derives a `legacy-<sha256>` key from a row's content, `rowIdentityKeys` lists the keys a row answers to (its id and `aliasIds`, plus the content key when it has no `createdAt`), `mergeRowCopies` collapses every group of rows sharing a key (transitively) to one row under its smallest id for the `row-union` snapshot merge, `preferRowCopy` is the total per-row `updatedAt` LWW order, `normalizeLegacyRows` stamps and dedupes legacy eye exams (also migration 429), and `ROW_TOMBSTONES_KEY` names the per-file deletion markers. Pure. || `meetingUrl.js` | Selects the one cacheable join URL (`selectMeetingUrl`) out of a raw Google Calendar event — first http(s) `video` entry point, else `hangoutLink` — and owns the shared `MEETING_URL_MAX` bound. Three-state: `undefined` = the producer never described conferencing, `null` = described with none usable. || `minimaxH3Memory.js` | The declared weight-placement table for the three MiniMax H3 entries (#5420) — H3 is the one video model family whose components fit nowhere unassisted, so "does this box have enough" is a render gate, not a UI fact. `MINIMAX_H3_MEMORY_PROFILES` maps entry id → `{ shippedRepo, shippedRevision, profiles }`, each profile carrying an honest `minMemoryGb` host floor and (CUDA only) a `minVramGb` device floor, ordered best-first. Every capacity number is HOISTED from what already existed — the CUDA tiers are `resolve_offload_profile()`'s own thresholds in `scripts/generate_minimax_h3_cuda.py`, the host floors are the entries' `memoryGb` — the sole new number being `MINIMAX_H3_HOST_RESERVE_GB`, a policy reserve held back for the OS. `applyMiniMaxH3MemoryProfiles(list)` is the load-time backfill (twin of migration 317) and guards BOTH `repo` and `revision`, like the speed-profile decorator. `selectMiniMaxH3MemoryProfile({ model, totalMemoryGb })` picks the best profile the HOST can hold (VRAM is the runner's call — the server has no synchronous device view); `miniMaxH3MemoryDeclineReason()` RETURNS the fail-closed reason so the render path can 400 it and a status surface can show it, and returns `null` on an UNMEASURED host — "not measured" is a deferral to the runner, never the same as zero. `validateMiniMaxH3MemoryProfileTable` / `sanitizeMiniMaxH3MemoryProfiles` warn + strip a hand-edited table (NaN floor, duplicate/reserved id, mis-ordered tiers) at load. || `videoContinuity.js` | How chunk N+1 of a chained video render is conditioned on chunk N. `resolveContextFrames(requested)` normalizes the tail-window size (absent → `DEFAULT_CONTEXT_FRAMES` = 22 ≈ 1s @ 24fps; an explicit `0` is preserved as "last frame only", NOT collapsed into the default) and clamps to `MIN_CONTEXT_FRAMES`..`MAX_CONTEXT_FRAMES`. `resolveContinuityStrategy({model, contextFrames})` picks `'window'` (LTX-2 `extend_from_video` conditioned on the prior chunk's last N frames — motion, not just a pose) or `'frame'` (extract the last frame, run i2v), degrading to `'frame'` on any runtime outside `CONTEXT_WINDOW_RUNTIMES` rather than rejecting. `extendLatentFrames` / `extendedPixelFrames` convert across the VAE's `LATENT_FRAME_STRIDE` (8 pixel frames per latent), `contextPrefixFrames({totalFrames, extendLatents})` measures how much of an extend render is echoed context to trim back off before stitching (0 = leave it alone), `tailWindowStartFrame` gives the cut index for the window itself, and `planExtendSplice({sourceFrames, outputFrames, extendLatents})` decides where a plain native Extend joins the ORIGINAL source to the render's new frames (`null` = layout doesn't verify, keep the model output). Pure — importable from `prepareParams.js` without dragging in `local.js`. Mirrored for the picker in `client/src/lib/videoGenParams.js`, pinned by `videoContinuity.parity.test.js`. |diff --git a/server/lib/meatspaceRowIdentity.js b/server/lib/meatspaceRowIdentity.jsindex a6771e4f79..c506db3b7e 100644--- a/server/lib/meatspaceRowIdentity.js+++ b/server/lib/meatspaceRowIdentity.js@@ -8,8 +8,9 @@* their `id` alone is their identity. A row without `createdAt` has unknown* provenance — an id-less legacy row, a random stamp, or an id minted before* #12432 — so it answers to its own id AND to `legacy-<sha256>` of its content,- * which every machine computes alike. Two copies that meet only through the- * content key converge on that key as their id.+ * which every machine computes alike. Copies of one record that carry different+ * ids converge on the smallest of them and keep the rest as `aliasIds`, so an+ * edit or delete still addressed by a retired id finds the row.** Pure. Migration 429 calls `normalizeLegacyRows`, so `legacyRowId` IS an* on-disk id: changing it would key every peer's rows differently.@@ -29,14 +30,19 @@ const hasStringId = (row) => typeof row?.id === 'string' && row.id !== '';/** Deterministic id for a legacy row: a hash of its content without identity or clock fields. */export function legacyRowId(row) {- const { id: _id, createdAt: _createdAt, updatedAt: _updatedAt, ...content } = row;+ const { id: _id, aliasIds: _aliasIds, createdAt: _createdAt, updatedAt: _updatedAt, ...content } = row;const digest = createHash('sha256').update(canonicalStringify(content)).digest('hex');return `${LEGACY_ROW_ID_PREFIX}${digest.slice(0, 24)}`;}-/** Every key a row answers to; the content key last. A delete tombstones all of them. */+const rowIds = (row) => [+ ...(hasStringId(row) ? [row.id] : []),+ ...(Array.isArray(row.aliasIds) ? row.aliasIds.filter((id) => typeof id === 'string' && id !== '') : []),+];++/** Every key a row answers to: id, aliases, then the content key. A delete tombstones all of them. */export function rowIdentityKeys(row) {- const keys = hasStringId(row) ? [row.id] : [];+ const keys = [...new Set(rowIds(row))];if (row.createdAt === undefined) {const contentKey = legacyRowId(row);if (!keys.includes(contentKey)) keys.push(contentKey);@@ -65,46 +71,76 @@ export function preferRowCopy(a, b) {return canonicalStringify(a) >= canonicalStringify(b) ? a : b;}+// One record from a group of its copies: the preferred copy's fields, the+// smallest id as its id and every other id as an alias. Both choices are+// order-free, so every machine reduces the same copies to the same row.+function reduceCopies(members) {+ if (members.length === 1) return members[0];+ const winner = members.map((m) => m.row).reduce(preferRowCopy);+ const ids = [...new Set(members.flatMap((m) => rowIds(m.row)))].sort();+ let row = winner;+ if (ids.length > 0) {+ const { aliasIds: _aliasIds, ...fields } = winner;+ row = ids.length > 1 ? { ...fields, id: ids[0], aliasIds: ids.slice(1) } : { ...fields, id: ids[0] };+ }+ const keys = new Set([...members.flatMap((m) => [...m.keys]), ...rowIdentityKeys(row)]);+ return { row, keys };+}+/**- * Collapse copies of one record across `localRows` then `remoteRows`. Each- * entry of the result is `{ row, keys }` (`keys` is null for a non-object row,- * which is kept only from the local side). Copies resolve to `preferRowCopy`;- * copies with different ids that met through the content key take that key as- * their id.+ * Collapse copies of one record across `localRows` then `remoteRows`. Rows are+ * copies when they share any identity key, transitively, so a row bridging two+ * others joins all three whichever side is local. Each result entry is+ * `{ row, keys }` (`keys` is null for a non-object row, kept only from the+ * local side), in first-occurrence order.** `stampIds` gives an id-less row its content key as its id (eyes.json, whose- * rows are addressed by id). `keepLocalDuplicates` keeps rows of `localRows`- * that share a key as separate rows (blood/epigenetic files, whose id-less rows- * were never deduped locally); only a remote copy is matched against them.+ * rows are addressed by id). `keepLocalDuplicates` keeps a group's local rows+ * as they are and drops its remote copies (blood/epigenetic files: id-less rows+ * keyed by content alone, which were never deduped locally).*/export function mergeRowCopies(localRows, remoteRows = [], { stampIds = false, keepLocalDuplicates = false } = {}) {- const entries = [];- const indexByKey = new Map();- const register = (keys, index) => {- for (const key of keys) if (!indexByKey.has(key)) indexByKey.set(key, index);- };- const add = (row, fromLocal) => {+ const items = [];+ const add = (row, local) => {if (!isPlainObject(row)) {- if (fromLocal) entries.push({ row, keys: null });+ if (local) items.push({ row, keys: null, local });return;}const keys = rowIdentityKeys(row);const keyed = stampIds && !hasStringId(row) ? { ...row, id: keys[keys.length - 1] } : row;- const matchKey = keys.find((key) => indexByKey.has(key));- if (matchKey === undefined || (keepLocalDuplicates && fromLocal)) {- register(keys, entries.length);- entries.push({ row: keyed, keys: new Set(keys) });- return;- }- const entry = entries[indexByKey.get(matchKey)];- const preferred = preferRowCopy(entry.row, keyed);- const id = entry.row.id === keyed.id ? preferred.id : matchKey;- entry.row = preferred.id === id ? preferred : { ...preferred, id };- for (const key of [...keys, ...rowIdentityKeys(entry.row)]) entry.keys.add(key);- register(entry.keys, indexByKey.get(matchKey));+ items.push({ row: keyed, keys: new Set(keys), local });};for (const row of Array.isArray(localRows) ? localRows : []) add(row, true);for (const row of Array.isArray(remoteRows) ? remoteRows : []) add(row, false);++ const parent = items.map((_, i) => i);+ const find = (i) => {+ while (parent[i] !== i) { parent[i] = parent[parent[i]]; i = parent[i]; }+ return i;+ };+ const ownerByKey = new Map();+ items.forEach((item, i) => {+ for (const key of item.keys ?? []) {+ if (!ownerByKey.has(key)) { ownerByKey.set(key, i); continue; }+ const a = find(ownerByKey.get(key));+ const b = find(i);+ if (a !== b) parent[Math.max(a, b)] = Math.min(a, b);+ }+ });++ const groups = new Map();+ items.forEach((item, i) => {+ const root = item.keys ? find(i) : `passthrough-${i}`;+ if (!groups.has(root)) groups.set(root, []);+ groups.get(root).push(item);+ });+ const entries = [];+ for (const members of groups.values()) {+ if (members[0].keys === null) { entries.push({ row: members[0].row, keys: null }); continue; }+ const locals = members.filter((m) => m.local);+ if (keepLocalDuplicates && locals.length > 0) entries.push(...locals.map(({ row, keys }) => ({ row, keys })));+ else entries.push(reduceCopies(members.map(({ row, keys }) => ({ row, keys }))));+ }return entries;}diff --git a/server/services/dataSync.meatspace.test.js b/server/services/dataSync.meatspace.test.jsindex 291962b4b8..aa5437f351 100644--- a/server/services/dataSync.meatspace.test.js+++ b/server/services/dataSync.meatspace.test.js@@ -127,7 +127,38 @@ describe('applyRemote meatspace eyes.json identity', () => {await apply('eyes.json', { exams: [{ id: legacyRowId(legacy), ...legacy }] });const exams = (await readDoc('eyes.json')).exams;- expect(exams).toEqual([{ id: legacyRowId(legacy), ...legacy }]);+ expect(exams).toHaveLength(1);+ expect([exams[0].id, ...exams[0].aliasIds].sort())+ .toEqual(['0d9b4c1e-0000-4000-8000-000000000001', legacyRowId(legacy)].sort());+ });++ // A machine that has not yet seen the convergence edits the copy under the id+ // the merge retired; the alias still routes the edit to the one row.+ it('applies an edit addressed by an id that convergence turned into an alias', async () => {+ const legacy = { date: '2025-02-02', leftSphere: -1 };+ const edited = { id: 'stamp-z', ...legacy, leftSphere: -1.75, updatedAt: '2026-07-01T00:00:00.000Z' };+ await persist('eyes.json', { exams: [edited] });+ await apply('eyes.json', { exams: [{ id: 'stamp-b', aliasIds: ['stamp-z'], ...legacy }] });+ expect((await readDoc('eyes.json')).exams).toEqual([{ ...edited, id: 'stamp-b', aliasIds: ['stamp-z'] }]);+ });++ // A row whose id matches one exam and whose content matches another joins all+ // three, whichever machine runs the merge.+ it('merges a row bridging two exams to the same result in both sync directions', async () => {+ const a = { id: 'stamp-a', date: '2025-03-03', leftSphere: -1 };+ const b = { id: 'stamp-b', date: '2025-03-03', leftSphere: -2 };+ const bridge = { id: 'stamp-a', date: '2025-03-03', leftSphere: -2, updatedAt: '2026-07-01T00:00:00.000Z' };++ await persist('eyes.json', { exams: [a, b] });+ await apply('eyes.json', { exams: [bridge] });+ const forward = (await readDoc('eyes.json')).exams;++ await persist('eyes.json', { exams: [bridge] });+ await apply('eyes.json', { exams: [a, b] });+ const backward = (await readDoc('eyes.json')).exams;++ expect(forward).toEqual([{ ...bridge, aliasIds: ['stamp-b'] }]);+ expect(backward).toEqual(forward);});// An exam minted before #12432 has an id but no createdAt. Two machines that