Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
address review (claude): never adopt an unreadable legacy ledger
[._.]/ Adam Eivyde59284f6772
diff --git a/client/src/components/BackupWidget.jsx b/client/src/components/BackupWidget.jsxindex 9290c6f86f..5608c48991 100644--- a/client/src/components/BackupWidget.jsx+++ b/client/src/components/BackupWidget.jsx@@ -273,7 +273,9 @@ function RestorePanel({ snapshot, onClose, restoring, onRestoreStateChange }) {<p className="mb-2 flex items-start gap-1.5 rounded border border-port-warning/40 bg-port-warning/10 p-2 text-xs text-port-warning" role="alert"><AlertTriangle size={13} className="mt-0.5 shrink-0" /><span className="min-w-0 break-words">- This snapshot keeps its migration record in the older <code>migrations/.applied.json</code> location. Restoring installs it as this install's record (the current one is kept as <code>migrations.applied.json.pre-restore-<time></code>) so the next start re-runs every migration the snapshot predates.+ {preview.restoredLegacyLedger.status === 'unreadable'+ ? <>This snapshot's migration record (<code>migrations/.applied.json</code>) is unreadable, so this install keeps its own record and no migration re-runs over the restored files; repair the snapshot copy or run the migrations by hand afterwards.</>+ : <>This snapshot keeps its migration record in the older <code>migrations/.applied.json</code> location. Restoring installs it as this install's record (the current one is kept as <code>migrations.applied.json.pre-restore-<time></code>) so the next start re-runs every migration the snapshot predates.</>}</span></p>)}diff --git a/client/src/components/BackupWidget.test.jsx b/client/src/components/BackupWidget.test.jsxindex 1975249d2a..b805eb409e 100644--- a/client/src/components/BackupWidget.test.jsx+++ b/client/src/components/BackupWidget.test.jsx@@ -402,6 +402,19 @@ describe('BackupWidget snapshots', () => {));});+ it('warns that an unreadable legacy ledger keeps this install\'s record', async () => {+ mockRestoreBackup.mockResolvedValueOnce({+ dryRun: true,+ changedFiles: ['migrations/.applied.json'],+ restoredLegacyLedger: { status: 'unreadable', parkedAs: null },+ });+ renderWidget();+ await openRestorePanel();+ fireEvent.click(screen.getByRole('button', { name: 'Preview changes' }));++ expect(await screen.findByRole('alert')).toHaveTextContent(/is unreadable, so this install keeps its own record/);+ });+it('shows no migration warning for a gap that is empty or unknown', async () => {mockRestoreBackup.mockResolvedValueOnce({ changedFiles: ['cos/quota-burn.json'], ledgerGap: { count: 0, names: [] } })diff --git a/docs/BACKUP.md b/docs/BACKUP.mdindex 9f9c79cba3..3149accb3f 100644--- a/docs/BACKUP.md+++ b/docs/BACKUP.md@@ -517,7 +517,7 @@ A snapshot taken before the agent records moved into PostgreSQL (#12738) also caKeep the original backup intact. Copy it into a disposable offline install running the compatible historical version, upgrade that isolated install through the file-to-database transition, and verify the imported records and linked files there. Make a current-format file and PostgreSQL backup from the converted install, then use the normal verified restore workflow. Do not convert against the live database or delete its markers. Current file-primary siblings (such as manuscript reviews, draft Markdown and media assets) and parked `.imported` recovery evidence remain eligible for file restore. Selecting one of those siblings is allowed even if another part of the snapshot contains legacy metadata.-A full restore also installs the snapshot's `migrations.applied.json`, so the next start re-runs every file migration the snapshot predates. A snapshot taken before migrations moved out of `data/` (May 2026) carries that ledger at `migrations/.applied.json` instead; the transfer lands it beside the live `migrations.applied.json`, which the runner would then keep, so a full restore adopts it by hand after the transfer: the live ledger is parked as `migrations.applied.json.pre-restore-<timestamp>` and the restored one takes its place. Preview and result carry `restoredLegacyLedger` (`{ status, parkedAs }`, `pending` in a preview and `adopted` after execution); the widget warns before restore and asks for a restart afterwards. A snapshot that also carries a current-path ledger is left to the transfer. That re-run only upgrades restored files for migrations gated on their input. The store-split migrations (059 for media collections, 200 for the Brain stores) gate on their output, a type index already at schema v1, and file restore leaves the live index in place, so their re-run is a no-op over a restored monolith. Restore therefore converts those stores itself (#12626): when the scope covers a store (full restore, `brain`, `brain/<type>.json`, `media-collections` or `media-collections.json`) and the snapshot carries its monolith (`brain/<type>.json`, `media-collections.json`) without `<store>/index.json`, a successful transfer moves the live `<store>/` directory to `<store>.pre-restore-<timestamp>/` (and a live `.bak-200`/`.bak-059` backup beside it), runs the frozen split migration against the install, and invalidates the Brain caches. The restored records, tombstones included, then load immediately, for folder restores too. A snapshot that has the split layout plus a stray monolith (setup-data re-seeds those after an upgrade) is current and is not converted. A monolith the split would refuse (unparseable, or a malformed records container) is restored as-is and never parks the live store. Preview and result list these stores in `restoredLegacyStores` (`{ store, legacyFile, status, parkedAs }`, with `status` `pending` in a preview and `converted` or `unreadable` after execution); the widget warns before restore and names the parked directory afterwards. A folder restore (`subdirFilter`) keeps the live ledger, so restored files can come back in a shape older than migrations no runner will revisit. Its preview and result therefore carry `ledgerGap`: `{ count, names }` listing the migrations the live ledger records and the snapshot's lacks, or `null` when either ledger is missing or unreadable. A full restore omits the field. The widget warns on a non-empty gap and recommends a full restore, but does not refuse: a folder no migration touched restores correctly. The quota-burn plan is converted on the spot: a restore covering `cos/quota-burn.json` (full, `cos`, or that file) runs any legacy `jobType` steps through the same conversion as migration 359 (`reconcileRestoredQuotaBurnPlan`), reusing each step's deterministic `job-burn-<family>-<step>` custom job, so a repeated restore creates no duplicates.+A full restore also installs the snapshot's `migrations.applied.json`, so the next start re-runs every file migration the snapshot predates. A snapshot taken before migrations moved out of `data/` (May 2026) carries that ledger at `migrations/.applied.json` instead; the transfer lands it beside the live `migrations.applied.json`, which the runner would then keep, so a full restore adopts it by hand after the transfer: the live ledger is parked as `migrations.applied.json.pre-restore-<timestamp>` and the restored one takes its place. Preview and result carry `restoredLegacyLedger` (`{ status, parkedAs }`, `pending` in a preview and `adopted` after execution; `unreadable` when the snapshot's legacy ledger does not parse as a list, in which case the live ledger is kept); the widget warns before restore and asks for a restart afterwards. A snapshot that also carries a current-path ledger is left to the transfer. That re-run only upgrades restored files for migrations gated on their input. The store-split migrations (059 for media collections, 200 for the Brain stores) gate on their output, a type index already at schema v1, and file restore leaves the live index in place, so their re-run is a no-op over a restored monolith. Restore therefore converts those stores itself (#12626): when the scope covers a store (full restore, `brain`, `brain/<type>.json`, `media-collections` or `media-collections.json`) and the snapshot carries its monolith (`brain/<type>.json`, `media-collections.json`) without `<store>/index.json`, a successful transfer moves the live `<store>/` directory to `<store>.pre-restore-<timestamp>/` (and a live `.bak-200`/`.bak-059` backup beside it), runs the frozen split migration against the install, and invalidates the Brain caches. The restored records, tombstones included, then load immediately, for folder restores too. A snapshot that has the split layout plus a stray monolith (setup-data re-seeds those after an upgrade) is current and is not converted. A monolith the split would refuse (unparseable, or a malformed records container) is restored as-is and never parks the live store. Preview and result list these stores in `restoredLegacyStores` (`{ store, legacyFile, status, parkedAs }`, with `status` `pending` in a preview and `converted` or `unreadable` after execution); the widget warns before restore and names the parked directory afterwards. A folder restore (`subdirFilter`) keeps the live ledger, so restored files can come back in a shape older than migrations no runner will revisit. Its preview and result therefore carry `ledgerGap`: `{ count, names }` listing the migrations the live ledger records and the snapshot's lacks, or `null` when either ledger is missing or unreadable. A full restore omits the field. The widget warns on a non-empty gap and recommends a full restore, but does not refuse: a folder no migration touched restores correctly. The quota-burn plan is converted on the spot: a restore covering `cos/quota-burn.json` (full, `cos`, or that file) runs any legacy `jobType` steps through the same conversion as migration 359 (`reconcileRestoredQuotaBurnPlan`), reusing each step's deterministic `job-burn-<family>-<step>` custom job, so a repeated restore creates no duplicates.**Every nonzero rsync exit fails a restore, including 24.** A backup tolerates exit 24 ("some source files vanished") because it reads the live data tree, where a file deleted mid-scan is normal. A restore reads a snapshot that must not change under it, so a vanished source file means live data may now mix restored and older files: preview and execution both reject it. An execution failure is reported as a partial restore — the error says some files may already have been overwritten because file restore is not transactional — and caches (settings, Brain, media index) are reconciled before the error is returned, exactly as for any other failed transfer.diff --git a/server/services/backup.js b/server/services/backup.jsindex 45dfeb3fbe..df0c0b4008 100644--- a/server/services/backup.js+++ b/server/services/backup.js@@ -1986,7 +1986,7 @@ async function restoreHeldSnapshot({ snapshotDir, currentSource }, snapshotId, {transferFiles,});restoredLegacyStores = await convertRestoredLegacySplitStores(legacyStores);- if (legacyLedger) restoredLegacyLedger = await adoptRestoredLegacyLedger();+ if (legacyLedger) restoredLegacyLedger = await adoptRestoredLegacyLedger(legacyLedger);return changed;},}));@@ -2018,11 +2018,15 @@ async function readSnapshotMigrationLedger(srcDir) {* then keeps the live ledger (a current ledger is authoritative over a legacy* one), so nothing would re-run over the restored records (#12355 assumed the* two paths coincide). Null when the snapshot carries a current ledger or none.+ * A legacy ledger that does not parse as a list is reported `unreadable` and+ * never adopted: the live ledger is worth more than a broken one, which the+ * runner would rename aside and rebuild from scratch.*/async function snapshotLegacyLedger(srcDir, subdirFilter) {if (subdirFilter) return null;const { current, legacy } = ledgerPaths(srcDir);- return !(await fileExists(current)) && await fileExists(legacy) ? { status: 'pending', parkedAs: null } : null;+ if (await fileExists(current) || !await fileExists(legacy)) return null;+ return { status: await readMigrationLedger(legacy) ? 'pending' : 'unreadable', parkedAs: null };}/**@@ -2031,13 +2035,18 @@ async function snapshotLegacyLedger(srcDir, subdirFilter) {* like a parked split store), and the next migration run starts from the* snapshot's list, exactly as a full restore with a current-path ledger does.*/-async function adoptRestoredLegacyLedger() {+async function adoptRestoredLegacyLedger(legacyLedger) {+ if (legacyLedger.status !== 'pending') {+ console.warn(`⚠️ Restore left the live migration ledger in place: the snapshot's ${LEGACY_APPLIED_LEDGER_RELPATH} is unreadable`);+ return legacyLedger;+ }const { current, legacy } = ledgerPaths(PATHS.data);- // Confirmed before the live ledger moves: parking it and then failing the- // rename would leave no ledger at all, which the next boot reads as a fresh- // install (every migration re-runs, purge migrations are disarmed).- if (!await fileExists(legacy)) {- throw new Error(`Restore transferred no ${LEGACY_APPLIED_LEDGER_RELPATH}; the live migration ledger was left unchanged`);+ // Confirmed on the transferred copy before the live ledger moves: parking it+ // and then failing would leave no ledger (or a broken one), which the next+ // boot reads as a fresh install: every migration re-runs and purge migrations+ // are disarmed.+ if (!await readMigrationLedger(legacy)) {+ throw new Error(`Restore transferred no readable ${LEGACY_APPLIED_LEDGER_RELPATH}; the live migration ledger was left unchanged`);}const stamp = new Date().toISOString().replace(/[:.]/g, '-');const parkedAs = `${basename(current)}.pre-restore-${stamp}`;diff --git a/server/services/backup.test.js b/server/services/backup.test.jsindex d883dc76bd..969ccaa86d 100644--- a/server/services/backup.test.js+++ b/server/services/backup.test.js@@ -2616,6 +2616,19 @@ describe('restoreSnapshot manifest verification', () => {expect(await parkedLedgers()).toEqual([]);});+ it('reports an unreadable legacy ledger and keeps the live ledger', async () => {+ await writeManifest({ 'migrations/.applied.json': await writeSnapshotFile('migrations/.applied.json', '{ not a ledger') });+ await realFs.writeFile(liveLedgerPath(), JSON.stringify(['001-example.js', '359-example.js']));+ await realFs.mkdir(joinPath(PATHS.data, 'migrations'), { recursive: true });+ await realFs.writeFile(liveLegacyLedgerPath(), '{ not a ledger');++ await expect(finishRestore({ dryRun: true })).resolves.toMatchObject({ restoredLegacyLedger: { status: 'unreadable', parkedAs: null } });+ spawn.mockReset();+ await expect(finishRestore({ dryRun: false })).resolves.toMatchObject({ restoredLegacyLedger: { status: 'unreadable', parkedAs: null } });+ expect(JSON.parse(await realFs.readFile(liveLedgerPath(), 'utf8'))).toEqual(['001-example.js', '359-example.js']);+ expect(await parkedLedgers()).toEqual([]);+ });+it('leaves a snapshot that also carries a current-path ledger to rsync', async () => {await seedLegacySnapshot({ alsoCurrent: true });await realFs.writeFile(liveLedgerPath(), JSON.stringify(['001-example.js', '359-example.js']));