Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

address review (claude): never adopt an unreadable legacy ledger

[._.]/ Adam Eivyde59284f6772
diff --git a/client/src/components/BackupWidget.jsx b/client/src/components/BackupWidget.jsx
index 9290c6f86f..5608c48991 100644
--- a/client/src/components/BackupWidget.jsx
+++ b/client/src/components/BackupWidget.jsx
@@ -273,7 +273,9 @@ function RestorePanel({ snapshot, onClose, restoring, onRestoreStateChange }) {
<p className="mb-2 flex items-start gap-1.5 rounded border border-port-warning/40 bg-port-warning/10 p-2 text-xs text-port-warning" role="alert">
<AlertTriangle size={13} className="mt-0.5 shrink-0" />
<span className="min-w-0 break-words">
- This snapshot keeps its migration record in the older <code>migrations/.applied.json</code> location. Restoring installs it as this install&apos;s record (the current one is kept as <code>migrations.applied.json.pre-restore-&lt;time&gt;</code>) so the next start re-runs every migration the snapshot predates.
+ {preview.restoredLegacyLedger.status === 'unreadable'
+ ? <>This snapshot&apos;s migration record (<code>migrations/.applied.json</code>) is unreadable, so this install keeps its own record and no migration re-runs over the restored files; repair the snapshot copy or run the migrations by hand afterwards.</>
+ : <>This snapshot keeps its migration record in the older <code>migrations/.applied.json</code> location. Restoring installs it as this install&apos;s record (the current one is kept as <code>migrations.applied.json.pre-restore-&lt;time&gt;</code>) so the next start re-runs every migration the snapshot predates.</>}
</span>
</p>
)}
diff --git a/client/src/components/BackupWidget.test.jsx b/client/src/components/BackupWidget.test.jsx
index 1975249d2a..b805eb409e 100644
--- a/client/src/components/BackupWidget.test.jsx
+++ b/client/src/components/BackupWidget.test.jsx
@@ -402,6 +402,19 @@ describe('BackupWidget snapshots', () => {
));
});
+ it('warns that an unreadable legacy ledger keeps this install\'s record', async () => {
+ mockRestoreBackup.mockResolvedValueOnce({
+ dryRun: true,
+ changedFiles: ['migrations/.applied.json'],
+ restoredLegacyLedger: { status: 'unreadable', parkedAs: null },
+ });
+ renderWidget();
+ await openRestorePanel();
+ fireEvent.click(screen.getByRole('button', { name: 'Preview changes' }));
+
+ expect(await screen.findByRole('alert')).toHaveTextContent(/is unreadable, so this install keeps its own record/);
+ });
+
it('shows no migration warning for a gap that is empty or unknown', async () => {
mockRestoreBackup
.mockResolvedValueOnce({ changedFiles: ['cos/quota-burn.json'], ledgerGap: { count: 0, names: [] } })
diff --git a/docs/BACKUP.md b/docs/BACKUP.md
index 9f9c79cba3..3149accb3f 100644
--- a/docs/BACKUP.md
+++ b/docs/BACKUP.md
@@ -517,7 +517,7 @@ A snapshot taken before the agent records moved into PostgreSQL (#12738) also ca
Keep the original backup intact. Copy it into a disposable offline install running the compatible historical version, upgrade that isolated install through the file-to-database transition, and verify the imported records and linked files there. Make a current-format file and PostgreSQL backup from the converted install, then use the normal verified restore workflow. Do not convert against the live database or delete its markers. Current file-primary siblings (such as manuscript reviews, draft Markdown and media assets) and parked `.imported` recovery evidence remain eligible for file restore. Selecting one of those siblings is allowed even if another part of the snapshot contains legacy metadata.
-A full restore also installs the snapshot's `migrations.applied.json`, so the next start re-runs every file migration the snapshot predates. A snapshot taken before migrations moved out of `data/` (May 2026) carries that ledger at `migrations/.applied.json` instead; the transfer lands it beside the live `migrations.applied.json`, which the runner would then keep, so a full restore adopts it by hand after the transfer: the live ledger is parked as `migrations.applied.json.pre-restore-<timestamp>` and the restored one takes its place. Preview and result carry `restoredLegacyLedger` (`{ status, parkedAs }`, `pending` in a preview and `adopted` after execution); the widget warns before restore and asks for a restart afterwards. A snapshot that also carries a current-path ledger is left to the transfer. That re-run only upgrades restored files for migrations gated on their input. The store-split migrations (059 for media collections, 200 for the Brain stores) gate on their output, a type index already at schema v1, and file restore leaves the live index in place, so their re-run is a no-op over a restored monolith. Restore therefore converts those stores itself (#12626): when the scope covers a store (full restore, `brain`, `brain/<type>.json`, `media-collections` or `media-collections.json`) and the snapshot carries its monolith (`brain/<type>.json`, `media-collections.json`) without `<store>/index.json`, a successful transfer moves the live `<store>/` directory to `<store>.pre-restore-<timestamp>/` (and a live `.bak-200`/`.bak-059` backup beside it), runs the frozen split migration against the install, and invalidates the Brain caches. The restored records, tombstones included, then load immediately, for folder restores too. A snapshot that has the split layout plus a stray monolith (setup-data re-seeds those after an upgrade) is current and is not converted. A monolith the split would refuse (unparseable, or a malformed records container) is restored as-is and never parks the live store. Preview and result list these stores in `restoredLegacyStores` (`{ store, legacyFile, status, parkedAs }`, with `status` `pending` in a preview and `converted` or `unreadable` after execution); the widget warns before restore and names the parked directory afterwards. A folder restore (`subdirFilter`) keeps the live ledger, so restored files can come back in a shape older than migrations no runner will revisit. Its preview and result therefore carry `ledgerGap`: `{ count, names }` listing the migrations the live ledger records and the snapshot's lacks, or `null` when either ledger is missing or unreadable. A full restore omits the field. The widget warns on a non-empty gap and recommends a full restore, but does not refuse: a folder no migration touched restores correctly. The quota-burn plan is converted on the spot: a restore covering `cos/quota-burn.json` (full, `cos`, or that file) runs any legacy `jobType` steps through the same conversion as migration 359 (`reconcileRestoredQuotaBurnPlan`), reusing each step's deterministic `job-burn-<family>-<step>` custom job, so a repeated restore creates no duplicates.
+A full restore also installs the snapshot's `migrations.applied.json`, so the next start re-runs every file migration the snapshot predates. A snapshot taken before migrations moved out of `data/` (May 2026) carries that ledger at `migrations/.applied.json` instead; the transfer lands it beside the live `migrations.applied.json`, which the runner would then keep, so a full restore adopts it by hand after the transfer: the live ledger is parked as `migrations.applied.json.pre-restore-<timestamp>` and the restored one takes its place. Preview and result carry `restoredLegacyLedger` (`{ status, parkedAs }`, `pending` in a preview and `adopted` after execution; `unreadable` when the snapshot's legacy ledger does not parse as a list, in which case the live ledger is kept); the widget warns before restore and asks for a restart afterwards. A snapshot that also carries a current-path ledger is left to the transfer. That re-run only upgrades restored files for migrations gated on their input. The store-split migrations (059 for media collections, 200 for the Brain stores) gate on their output, a type index already at schema v1, and file restore leaves the live index in place, so their re-run is a no-op over a restored monolith. Restore therefore converts those stores itself (#12626): when the scope covers a store (full restore, `brain`, `brain/<type>.json`, `media-collections` or `media-collections.json`) and the snapshot carries its monolith (`brain/<type>.json`, `media-collections.json`) without `<store>/index.json`, a successful transfer moves the live `<store>/` directory to `<store>.pre-restore-<timestamp>/` (and a live `.bak-200`/`.bak-059` backup beside it), runs the frozen split migration against the install, and invalidates the Brain caches. The restored records, tombstones included, then load immediately, for folder restores too. A snapshot that has the split layout plus a stray monolith (setup-data re-seeds those after an upgrade) is current and is not converted. A monolith the split would refuse (unparseable, or a malformed records container) is restored as-is and never parks the live store. Preview and result list these stores in `restoredLegacyStores` (`{ store, legacyFile, status, parkedAs }`, with `status` `pending` in a preview and `converted` or `unreadable` after execution); the widget warns before restore and names the parked directory afterwards. A folder restore (`subdirFilter`) keeps the live ledger, so restored files can come back in a shape older than migrations no runner will revisit. Its preview and result therefore carry `ledgerGap`: `{ count, names }` listing the migrations the live ledger records and the snapshot's lacks, or `null` when either ledger is missing or unreadable. A full restore omits the field. The widget warns on a non-empty gap and recommends a full restore, but does not refuse: a folder no migration touched restores correctly. The quota-burn plan is converted on the spot: a restore covering `cos/quota-burn.json` (full, `cos`, or that file) runs any legacy `jobType` steps through the same conversion as migration 359 (`reconcileRestoredQuotaBurnPlan`), reusing each step's deterministic `job-burn-<family>-<step>` custom job, so a repeated restore creates no duplicates.
**Every nonzero rsync exit fails a restore, including 24.** A backup tolerates exit 24 ("some source files vanished") because it reads the live data tree, where a file deleted mid-scan is normal. A restore reads a snapshot that must not change under it, so a vanished source file means live data may now mix restored and older files: preview and execution both reject it. An execution failure is reported as a partial restore — the error says some files may already have been overwritten because file restore is not transactional — and caches (settings, Brain, media index) are reconciled before the error is returned, exactly as for any other failed transfer.
diff --git a/server/services/backup.js b/server/services/backup.js
index 45dfeb3fbe..df0c0b4008 100644
--- a/server/services/backup.js
+++ b/server/services/backup.js
@@ -1986,7 +1986,7 @@ async function restoreHeldSnapshot({ snapshotDir, currentSource }, snapshotId, {
transferFiles,
});
restoredLegacyStores = await convertRestoredLegacySplitStores(legacyStores);
- if (legacyLedger) restoredLegacyLedger = await adoptRestoredLegacyLedger();
+ if (legacyLedger) restoredLegacyLedger = await adoptRestoredLegacyLedger(legacyLedger);
return changed;
},
}));
@@ -2018,11 +2018,15 @@ async function readSnapshotMigrationLedger(srcDir) {
* then keeps the live ledger (a current ledger is authoritative over a legacy
* one), so nothing would re-run over the restored records (#12355 assumed the
* two paths coincide). Null when the snapshot carries a current ledger or none.
+ * A legacy ledger that does not parse as a list is reported `unreadable` and
+ * never adopted: the live ledger is worth more than a broken one, which the
+ * runner would rename aside and rebuild from scratch.
*/
async function snapshotLegacyLedger(srcDir, subdirFilter) {
if (subdirFilter) return null;
const { current, legacy } = ledgerPaths(srcDir);
- return !(await fileExists(current)) && await fileExists(legacy) ? { status: 'pending', parkedAs: null } : null;
+ if (await fileExists(current) || !await fileExists(legacy)) return null;
+ return { status: await readMigrationLedger(legacy) ? 'pending' : 'unreadable', parkedAs: null };
}
/**
@@ -2031,13 +2035,18 @@ async function snapshotLegacyLedger(srcDir, subdirFilter) {
* like a parked split store), and the next migration run starts from the
* snapshot's list, exactly as a full restore with a current-path ledger does.
*/
-async function adoptRestoredLegacyLedger() {
+async function adoptRestoredLegacyLedger(legacyLedger) {
+ if (legacyLedger.status !== 'pending') {
+ console.warn(`⚠️ Restore left the live migration ledger in place: the snapshot's ${LEGACY_APPLIED_LEDGER_RELPATH} is unreadable`);
+ return legacyLedger;
+ }
const { current, legacy } = ledgerPaths(PATHS.data);
- // Confirmed before the live ledger moves: parking it and then failing the
- // rename would leave no ledger at all, which the next boot reads as a fresh
- // install (every migration re-runs, purge migrations are disarmed).
- if (!await fileExists(legacy)) {
- throw new Error(`Restore transferred no ${LEGACY_APPLIED_LEDGER_RELPATH}; the live migration ledger was left unchanged`);
+ // Confirmed on the transferred copy before the live ledger moves: parking it
+ // and then failing would leave no ledger (or a broken one), which the next
+ // boot reads as a fresh install: every migration re-runs and purge migrations
+ // are disarmed.
+ if (!await readMigrationLedger(legacy)) {
+ throw new Error(`Restore transferred no readable ${LEGACY_APPLIED_LEDGER_RELPATH}; the live migration ledger was left unchanged`);
}
const stamp = new Date().toISOString().replace(/[:.]/g, '-');
const parkedAs = `${basename(current)}.pre-restore-${stamp}`;
diff --git a/server/services/backup.test.js b/server/services/backup.test.js
index d883dc76bd..969ccaa86d 100644
--- a/server/services/backup.test.js
+++ b/server/services/backup.test.js
@@ -2616,6 +2616,19 @@ describe('restoreSnapshot manifest verification', () => {
expect(await parkedLedgers()).toEqual([]);
});
+ it('reports an unreadable legacy ledger and keeps the live ledger', async () => {
+ await writeManifest({ 'migrations/.applied.json': await writeSnapshotFile('migrations/.applied.json', '{ not a ledger') });
+ await realFs.writeFile(liveLedgerPath(), JSON.stringify(['001-example.js', '359-example.js']));
+ await realFs.mkdir(joinPath(PATHS.data, 'migrations'), { recursive: true });
+ await realFs.writeFile(liveLegacyLedgerPath(), '{ not a ledger');
+
+ await expect(finishRestore({ dryRun: true })).resolves.toMatchObject({ restoredLegacyLedger: { status: 'unreadable', parkedAs: null } });
+ spawn.mockReset();
+ await expect(finishRestore({ dryRun: false })).resolves.toMatchObject({ restoredLegacyLedger: { status: 'unreadable', parkedAs: null } });
+ expect(JSON.parse(await realFs.readFile(liveLedgerPath(), 'utf8'))).toEqual(['001-example.js', '359-example.js']);
+ expect(await parkedLedgers()).toEqual([]);
+ });
+
it('leaves a snapshot that also carries a current-path ledger to rsync', async () => {
await seedLegacySnapshot({ alsoCurrent: true });
await realFs.writeFile(liveLedgerPath(), JSON.stringify(['001-example.js', '359-example.js']));