Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: preserve backup retention and restored metadata
[._.]/ Adam Eivy5954235f17ca
diff --git a/docs/BACKUP.md b/docs/BACKUP.mdindex 80595bd910..00a00c3e0f 100644--- a/docs/BACKUP.md+++ b/docs/BACKUP.md@@ -509,6 +509,10 @@ After the preflight, rsync copies `<snapshot>/data/` back to `./data/`. RestoreFile restore copies what the snapshot holds and leaves live files the snapshot lacks in place. One consequence is handled explicitly: a snapshot taken before the CoS config moved to its own file (#6182) carries its CoS settings inside `cos/state.json` and has no `cos/config.json`. When such a restore lands that legacy slice, the live `cos/config.json` (newer than the restore point) is moved aside to `cos/config.pre-restore-<timestamp>.json` and the settings are recovered from the restored `state.json`; otherwise the newer file would keep winning every read.+**Historical file-to-database metadata (#12397).** On a PostgreSQL install, full and selective file restores refuse selected legacy metadata with `409 BACKUP_LEGACY_DB_METADATA` before preview or execution starts a transfer. The diagnostic names only affected domains: universes (record `index.json` and type-level `config.runs`), pipeline issues/series, Story Builder, Writers Room (`folders.json`, `exercises.json`, work `manifest.json`), and Creative Director projects. These records now live in PostgreSQL. A live domain import marker can suppress re-import after file restore; restoring the older file-migration ledger does not reset that separate marker. Clearing markers is unsafe too: the importers preserve existing rows with `ON CONFLICT DO NOTHING`, so they cannot provide replacement-restore semantics for overlapping IDs.++Keep the original backup intact. Copy it into a disposable offline install running the compatible historical version, upgrade that isolated install through the file-to-database transition, and verify the imported records and linked files there. Make a current-format file and PostgreSQL backup from the converted install, then use the normal verified restore workflow. Do not convert against the live database or delete its markers. Current file-primary siblings (such as manuscript reviews, draft Markdown and media assets) and parked `.imported` recovery evidence remain eligible for file restore. Selecting one of those siblings is allowed even if another part of the snapshot contains legacy metadata.+A full restore also installs the snapshot's `migrations.applied.json`, so the next start re-runs every file migration the snapshot predates. A folder restore (`subdirFilter`) keeps the live ledger, so restored files can come back in a shape older than migrations no runner will revisit. Its preview and result therefore carry `ledgerGap`: `{ count, names }` listing the migrations the live ledger records and the snapshot's lacks, or `null` when either ledger is missing or unreadable. A full restore omits the field. The widget warns on a non-empty gap and recommends a full restore, but does not refuse: a folder no migration touched restores correctly. The quota-burn plan is converted on the spot: a restore covering `cos/quota-burn.json` (full, `cos`, or that file) runs any legacy `jobType` steps through the same conversion as migration 359 (`reconcileRestoredQuotaBurnPlan`), reusing each step's deterministic `job-burn-<family>-<step>` custom job, so a repeated restore creates no duplicates.**Every nonzero rsync exit fails a restore, including 24.** A backup tolerates exit 24 ("some source files vanished") because it reads the live data tree, where a file deleted mid-scan is normal. A restore reads a snapshot that must not change under it, so a vanished source file means live data may now mix restored and older files: preview and execution both reject it. An execution failure is reported as a partial restore — the error says some files may already have been overwritten because file restore is not transactional — and caches (settings, Brain, media index) are reconciled before the error is returned, exactly as for any other failed transfer.@@ -572,7 +576,7 @@ Left unbounded, `snapshots/<hostname>/` grows forever — every prior full DB du- **`retentionCount` absent or explicitly `null` both mean unlimited** — no pruning runs. This is why an install that predates this setting keeps every snapshot until the operator saves a choice: it has no stored value, and absence resolves to unlimited, not to the new-install default.- **Pruning runs only after a run reaches a completed snapshot with a usable database dump** (`pg_dump` succeeded, or was explicitly skipped by the file-backend escape hatch). A degraded run whose database dump failed keeps older snapshots and reports zero pruned snapshots; retention resumes after a later successful dump. A run that fails before completion never prunes, and a prune failure is logged without failing an otherwise-successful backup.- **Pruning is scoped to the CURRENT machine's namespace** (`snapshots/<hostname>/`) and only ever deletes snapshots whose `snapshotState()` reports neither `incomplete` nor `failed`. It never touches another machine's namespace in a shared destination, the legacy pre-namespace root, or an in-progress/failed snapshot — those stay until the operator deletes them explicitly. It also skips a snapshot that a restore or download is still reading (see [Snapshot source lifetime](#how-restore-works)); the skipped snapshot is not counted as pruned and becomes a candidate again on a later run.-- **Unreadable or corrupt manifests are protected in addition to the retention count.** They never displace readable completed snapshots, even when an incorrect clock gave a protected snapshot a future-dated ID. Snapshot IDs determine age; manifest timestamps do not change that ordering. Legacy completed snapshots with no manifest remain eligible for retention as before.+- **Unreadable or structurally invalid manifests are protected in addition to the retention count.** Retention uses the same field, count, path and SHA-256-shape validator as file restore, without hashing old payloads. Existing JSON `null`, arrays, malformed objects and invalid entries are not legacy absence. They never displace structurally valid completed snapshots, even when an incorrect clock gave a protected snapshot a future-dated ID. Snapshot IDs determine age; manifest timestamps do not change that ordering. Legacy completed snapshots with no manifest remain eligible for retention as before.- **`DELETE /api/backup/snapshots/:snapshotId?source=<source>`** (`backup.deleteSnapshot()`) permanently removes exactly the selected source/ID pair, immediately — there is no undo and it is never automatic. It shares `resolveSnapshotPath()`'s path-traversal and symlink guards with restore/download, and refuses a snapshot that is still being written (`SNAPSHOT_INCOMPLETE`) or that a restore or download is still reading (`SNAPSHOT_IN_USE`); unlike restore, a `.failed` snapshot **is** deletable. The Backup settings tab's snapshot history exposes this as a per-row delete action behind a confirmation dialog.## Scheduling & statusdiff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.mdindex bbd718d91a..119548d168 100644--- a/docs/TROUBLESHOOTING.md+++ b/docs/TROUBLESHOOTING.md@@ -514,6 +514,12 @@ npm run pm2:startThe split never overwrites a record directory that already exists. If a later Postgres import (`server/scripts/migrateUniversesToDB.js` and similar) already ran, re-run that explicit legacy-import tool after the split rather than resetting import markers or schema versions.+### Collection images or pinned covers lost after migration 428++Migration 428 inferred deletion from missing local image bytes. That is invalid for peer-hosted media, whose bytes intentionally remain on the hosting peer, and for temporarily unavailable files. The migration runner now records pending `428-prune-dangling-collection-images.js` without executing its frozen transform, including when the ledger is missing or rebuilt. This policy is persisted before any other pending migration runs, so an interrupted upgrade cannot re-arm it. Explicit gallery deletion still removes the references for the image being deleted.++If 428 already ran, preserve the current collection and recover membership from an unaffected peer or retained backup. Union synchronization may re-add the image items, but the migration also stamped a newer null `coverKey`; scalar last-write-wins synchronization cannot automatically restore that pinned cover. Select the intended cover explicitly after recovery. The peer-hosted media index does not record collection ownership and cannot reconstruct memberships. Do not rerun absence-based cleanup or overwrite newer edits with reconstructed guesses.+### "Another migration run holds data/migrations.lock"**Symptom**: `npm run migrations` or server boot exits with `Another migration run (pid N) holds …/data/migrations.lock`.diff --git a/scripts/run-migrations.js b/scripts/run-migrations.jsindex 26c6db1bd5..40735246d1 100644--- a/scripts/run-migrations.js+++ b/scripts/run-migrations.js@@ -270,6 +270,17 @@ async function runPendingMigrations({ rootDir, migrationsDir, appliedFile }) {await writeJsonAtomic(appliedFile, applied);};+ // Retire this frozen transform before ANY pending migration executes, even+ // with a rebuilt ledger or a later failure. Local absence cannot prove image+ // deletion: host-mode peers intentionally retain metadata without bytes.+ let disarmed = 0;+ const retired = '428-prune-dangling-collection-images.js';+ if (files.includes(retired) && !applied.includes(retired)) {+ applied.push(retired);+ disarmed++;+ console.warn(`⏭️ Retired migration ${retired}: local image absence is not deletion evidence; recorded without running (#12398)`);+ }+// Disarm EVERY pending purge migration up front, not as each is reached in// the run loop: if this rebuilt-from-empty run aborted on an earlier// throwing migration (the documented repair-and-reboot flow), the partial@@ -277,7 +288,6 @@ async function runPendingMigrations({ rootDir, migrationsDir, appliedFile }) {// would then execute destructively against the rebuilt install. The probe// reads source text, never importing a module before its turn in the loop.if (ledgerStartedEmpty) {- let disarmed = 0;for (const file of files) {if (applied.includes(file)) continue;if (!sourceDeclaresPurge(await readFile(join(migrationsDir, file), 'utf-8'))) continue;@@ -285,8 +295,10 @@ async function runPendingMigrations({ rootDir, migrationsDir, appliedFile }) {applied.push(file);disarmed++;}- if (disarmed > 0) await persistAppliedList();}+ // Publish both policies together: persisting retirement first would leave a+ // nonempty ledger after a crash, re-arming pending purge-class migrations.+ if (disarmed > 0) await persistAppliedList();let ran = 0;for (const file of files) {diff --git a/scripts/run-migrations.test.js b/scripts/run-migrations.test.jsindex dd86d86208..6ef6d24d8b 100644--- a/scripts/run-migrations.test.js+++ b/scripts/run-migrations.test.js@@ -66,6 +66,34 @@ export default {const corruptSiblings = () =>readdirSync(dataDir).filter((f) => f.startsWith('migrations.applied.json.corrupt-'));+ it.each(['pending', 'missing', 'corrupt', 'applied'])('retires unsafe 428 with a %s ledger before earlier failures (#12398)', async state => {+ const retired = '428-prune-dangling-collection-images.js';+ // Exercise the shipped, frozen transform through the real runner. If the+ // retirement regresses it removes both kinds of unresolved image pointer.+ writeFileSync(join(migrationsDir, retired), `export { default } from ${JSON.stringify(new URL('./migrations/428-prune-dangling-collection-images.js', import.meta.url).href)};`);+ mkdirSync(join(dataDir, 'images'));+ const collectionDir = join(dataDir, 'media-collections', 'collection-example');+ mkdirSync(collectionDir, { recursive: true });+ const original = JSON.stringify({ items: [{ kind: 'image', ref: 'hosted.png' }, { kind: 'image', ref: 'temporarily-missing.png' }], coverKey: 'image:hosted.png', updatedAt: '2026-01-01T00:00:00Z' });+ writeFileSync(join(collectionDir, 'index.json'), original);+ writeFileSync(join(dataDir, 'peer-hosted-media.json'), JSON.stringify({ version: 1, entries: { 'image:hosted.png': { peerId: 'example-peer', at: '2026-01-01T00:00:00Z' } } }));+ if (state === 'pending') writeFileSync(appliedFile, '["000-earlier.js"]');+ if (state === 'corrupt') writeFileSync(appliedFile, '{broken');+ if (state === 'applied') writeFileSync(appliedFile, JSON.stringify([retired]));+ writeFileSync(join(migrationsDir, '002-failure.js'), 'export async function up() { throw new Error("fixture interruption"); }');+ if (state === 'missing' || state === 'corrupt') {+ writeFileSync(join(migrationsDir, '429-purge.js'), 'export const purge = true; export async function up() { throw new Error("purge must stay disarmed"); }');+ }+ await expect(runMigrations({ rootDir, migrationsDir })).rejects.toThrow('fixture interruption');+ expect(JSON.parse(readFileSync(appliedFile, 'utf8'))).toContain(retired);+ expect(readFileSync(join(collectionDir, 'index.json'), 'utf8')).toBe(original);+ // A retry with a now nonempty ledger cannot re-arm the retired transform.+ rmSync(join(migrationsDir, '002-failure.js'));+ expect(await runMigrations({ rootDir, migrationsDir })).toBe(0);+ expect(readFileSync(join(collectionDir, 'index.json'), 'utf8')).toBe(original);+ if (state !== 'applied') expect(warnSpy).toHaveBeenCalledWith(expect.stringContaining('Retired migration 428'));+ });+it('renames truncated JSON aside and rebuilds from scratch', async () => {writeFileSync(appliedFile, '[\n "001-fixture.js",\n "002-'); // truncated mid-writediff --git a/server/services/backup.js b/server/services/backup.jsindex dac4eeb687..744e5d2c26 100644--- a/server/services/backup.js+++ b/server/services/backup.js@@ -1148,22 +1148,23 @@ async function pruneOldSnapshots(destPath, retentionCount) {const snapshotDir = join(sourceRoot, id);const { incomplete, failed } = await snapshotState(snapshotDir, id, true);if (incomplete || failed) return null;- // A manifest that exists but cannot be read must not date or delete the- // snapshot: fail toward keeping data (#11859).- const manifestRead = await readJSONFileStrict(join(snapshotDir, 'manifest.json'), null, { logError: false });- return { id, snapshotDir, createdAt: manifestRead.value?.generatedAt ?? null, protectedFromPrune: !manifestRead.ok };+ // Share restore's structural admission, without hashing old payloads.+ // Only confirmed absence is legacy; JSON null is an invalid manifest.+ const manifestRead = await readJSONFileStrict(join(snapshotDir, 'manifest.json'), MANIFEST_ABSENT, { logError: false });+ const valid = manifestRead.ok && (manifestRead.value === MANIFEST_ABSENT+ || manifestDataEntries(manifestRead.value, join(snapshotDir, 'data'), null) !== null);+ return { id, snapshotDir, protectedFromPrune: !valid };}))).filter(Boolean);// Newest-first by snapshot id (an ISO-style timestamp that never depends on- // reading the manifest); manifest.generatedAt is only a tiebreak.- descriptors.sort((a, b) => b.id.localeCompare(a.id)- || (b.createdAt ?? '').localeCompare(a.createdAt ?? ''));+ // reading the manifest). IDs are unique within this source namespace.+ descriptors.sort((a, b) => b.id.localeCompare(a.id));// Keep protected snapshots in addition to the readable retention budget,// even when a bad clock placed a protected snapshot ahead of healthy ones.const toDelete = descriptors.filter(d => {if (d.protectedFromPrune) {- console.warn(`⚠️ Backup retention: kept snapshot ${d.id} (manifest unreadable)`);+ console.warn(`⚠️ Backup retention: kept snapshot ${d.id} (manifest unreadable or invalid)`);return false;}return true;@@ -1577,6 +1578,36 @@ async function snapshotScopeInventory(srcDir, subdirFilter) {return { paths, largestFileBytes };}+// These exact metadata shapes belonged to six marker-gated file→DB imports.+// Sibling assets and parked .imported evidence remain file-primary recovery data.+const LEGACY_DB_METADATA = [+ ['universes', /^universes\/[^/]+\/index\.json$/],+ ['pipeline-issues', /^pipeline-issues\/iss-[A-Za-z0-9-]+\/index\.json$/],+ ['pipeline-series', /^pipeline-series\/ser-[A-Za-z0-9-]+\/index\.json$/],+ ['story-builder', /^story-builder\/stb-[A-Za-z0-9-]+\/index\.json$/],+ ['writers-room', /^writers-room\/(?:(?:folders|exercises)\.json|works\/[^/]+\/manifest\.json)$/],+ ['creative-director', /^creative-director-projects\.json$/],+];++async function assertRestoreStorageCompatible(srcDir, inventory, snapshotId) {+ // Match dumpPostgres: an uninitialized backend still requires PostgreSQL.+ if (process.env.MEMORY_BACKEND === 'file' || getBackendName() === 'file') return;+ if (!inventory) throw snapshotFileIntegrityError(snapshotId);+ const domains = new Set(LEGACY_DB_METADATA+ .filter(([, pattern]) => inventory.paths.some(path => pattern.test(path)))+ .map(([domain]) => domain));+ if (inventory.paths.includes('universes/index.json')) {+ const index = await readJSONFileStrict(join(srcDir, 'universes', 'index.json'), null, { logError: false });+ if (!index.ok || index.value?.config?.runs !== undefined) domains.add('universes');+ }+ if (domains.size) {+ throw new ServerError(+ `Historical file metadata for ${[...domains].join(', ')} cannot restore database-primary records. Convert this backup in a disposable offline upgrade install, verify its records, then restore a current-format file/database backup (docs/BACKUP.md).`,+ { status: 409, code: 'BACKUP_LEGACY_DB_METADATA', context: { domains: [...domains] } },+ );+ }+}+async function verifySnapshotFiles(snapshotDir, srcDir, snapshotId, subdirFilter) {// A unique fallback distinguishes a genuinely absent legacy manifest from an// existing file whose parsed value is JSON null.@@ -1593,10 +1624,11 @@ async function verifySnapshotFiles(snapshotDir, srcDir, snapshotId, subdirFilter}if (manifestRead.value === MANIFEST_ABSENT) {// No manifest to verify against, but the restore's idle-deadline floor- // still needs the largest in-scope file. Walk it best-effort: a scope that- // can't be surveyed keeps the default deadline rather than failing a- // restore that would previously have run.+ // still needs the largest in-scope file. PostgreSQL also needs a complete+ // inventory to reject ignored legacy metadata; only the file escape hatch+ // can retain the old best-effort behavior when inventory fails.const legacyInventory = await snapshotScopeInventory(srcDir, subdirFilter).catch(() => null);+ await assertRestoreStorageCompatible(srcDir, legacyInventory, snapshotId);return {status: 'unverified',reason: 'manifest_absent',@@ -1620,6 +1652,7 @@ async function verifySnapshotFiles(snapshotDir, srcDir, snapshotId, subdirFilterconst inventory = await snapshotScopeInventory(srcDir, subdirFilter).catch(() => null);if (!inventory) throw snapshotFileIntegrityError(snapshotId);+ await assertRestoreStorageCompatible(srcDir, inventory, snapshotId);const { paths: snapshotPaths, largestFileBytes } = inventory;const unmanifested = snapshotPaths.find(entry => !dataPaths.has(entry));if (unmanifested) throw snapshotFileIntegrityError(snapshotId, unmanifested);diff --git a/server/services/backup.test.js b/server/services/backup.test.jsindex 8e0f194743..2aa8cf8310 100644--- a/server/services/backup.test.js+++ b/server/services/backup.test.js@@ -2273,6 +2273,82 @@ describe('restoreSnapshot manifest verification', () => {return pending;}+ it.each([+ ['universes', 'universes/universe-example/index.json', { id: 'universe-example' }],+ ['universes', 'universes/index.json', { config: { runs: [{ id: 'run-example' }] } }],+ ['pipeline-issues', 'pipeline-issues/iss-example/index.json', { id: 'iss-example' }],+ ['pipeline-series', 'pipeline-series/ser-example/index.json', { id: 'ser-example' }],+ ['story-builder', 'story-builder/stb-example/index.json', { id: 'stb-example' }],+ ['writers-room', 'writers-room/folders.json', [{ id: 'folder-example' }]],+ ['writers-room', 'writers-room/exercises.json', [{ id: 'exercise-example' }]],+ ['writers-room', 'writers-room/works/wr-work-aabb/manifest.json', { id: 'wr-work-aabb' }],+ ['creative-director', 'creative-director-projects.json', [{ id: 'project-example' }]],+ ])('refuses %s legacy metadata at %s before preview or execution (#12397)', async (domain, path, record) => {+ getBackendName.mockReturnValue('postgres');+ const savedBackend = process.env.MEMORY_BACKEND;+ delete process.env.MEMORY_BACKEND;+ const markerPath = joinPath(PATHS.data, `${domain === 'creative-director' ? 'creative-director-projects' : domain}.migrated.json`);+ await realFs.writeFile(markerPath, '{"imported":1}');+ try {+ const hash = await writeSnapshotFile(path, JSON.stringify(record));+ await writeManifest({ [path]: hash });+ // Both empty and overlapping destination databases must refuse: this is+ // source compatibility, not an opportunistic ON CONFLICT merge.+ for (const rows of [[], [{ id: 'existing-record' }]]) {+ query.mockResolvedValue({ rows });+ for (const dryRun of [true, false]) {+ for (const subdirFilter of [null, path.split('/')[0], path]) {+ await expect(restoreSnapshot(tmpRoot, 'snap-1', { dryRun, subdirFilter }))+ .rejects.toMatchObject({ code: 'BACKUP_LEGACY_DB_METADATA', context: { domains: [domain] } });+ }+ }+ }+ expect(spawn).not.toHaveBeenCalled();+ expect(query).not.toHaveBeenCalled();+ expect(await realFs.readFile(markerPath, 'utf8')).toBe('{"imported":1}');+ // Pre-manifest snapshots get the same compatibility preflight.+ await realFs.rm(joinPath(snapshotDir, 'manifest.json'));+ await expect(restoreSnapshot(tmpRoot, 'snap-1', { dryRun: false }))+ .rejects.toMatchObject({ code: 'BACKUP_LEGACY_DB_METADATA' });+ } finally {+ if (savedBackend === undefined) delete process.env.MEMORY_BACKEND;+ else process.env.MEMORY_BACKEND = savedBackend;+ getBackendName.mockReturnValue(null);+ query.mockResolvedValue({ rows: [] });+ await realFs.rm(markerPath);+ }+ });++ it('admits current file-primary siblings and parked recovery evidence on PostgreSQL', async () => {+ getBackendName.mockReturnValue('postgres');+ const savedBackend = process.env.MEMORY_BACKEND;+ delete process.env.MEMORY_BACKEND;+ try {+ const paths = [+ 'universes/universe-example/assets/sheet.png', 'universes.imported/universe-example/index.json',+ 'universes/index.json', 'pipeline-issues/iss-example/render.png',+ 'pipeline-series/ser-example/manuscript-review.json', 'pipeline-series/ser-example/index.json.imported',+ 'story-builder.imported/stb-example/index.json', 'writers-room/folders.imported.json',+ 'writers-room/exercises.imported.json', 'writers-room/works/wr-work-aabb/manifest.imported.json',+ 'writers-room/works/wr-work-aabb/drafts/draft-example.md', 'creative-director-projects.imported.json',+ ];+ const files = {};+ for (const path of paths) files[path] = await writeSnapshotFile(path, '{}');+ await writeManifest(files);+ await expect(finishRestore({ dryRun: true })).resolves.toMatchObject({ verification: { status: 'verified' } });+ // A mixed snapshot still permits selecting only the current draft body.+ files['writers-room/folders.json'] = await writeSnapshotFile('writers-room/folders.json', '[]');+ await writeManifest(files);+ spawn.mockReset();+ await expect(finishRestore({ dryRun: false, subdirFilter: 'writers-room/works/wr-work-aabb/drafts' }))+ .resolves.toMatchObject({ verification: { status: 'verified', checkedFiles: 1 } });+ } finally {+ if (savedBackend === undefined) delete process.env.MEMORY_BACKEND;+ else process.env.MEMORY_BACKEND = savedBackend;+ getBackendName.mockReturnValue(null);+ }+ });+it('rechecks snapshot bytes on execution and refuses a post-preview change before rsync', async () => {const relativePath = 'restore-integrity/example.json';const originalHash = await writeSnapshotFile(relativePath, 'trusted backup');@@ -4901,7 +4977,7 @@ describe('runBackup lifecycle', () => {const dir = joinPath(destRoot, 'snapshots', source, id);await fsp.mkdir(joinPath(dir, 'data'), { recursive: true });if (generatedAt) {- await fsp.writeFile(joinPath(dir, 'manifest.json'), JSON.stringify({ generatedAt, fileCount: 0 }));+ await fsp.writeFile(joinPath(dir, 'manifest.json'), JSON.stringify({ generatedAt, fileCount: 0, files: {} }));}if (marker) await fsp.writeFile(joinPath(dir, marker), '');return dir;@@ -5053,7 +5129,7 @@ describe('runBackup lifecycle', () => {it('retains the readable snapshot budget in addition to corrupt manifests (#12340)', async () => {const fsp = await actualFs();- const oldest = await seedSnapshot(machineHost, '2020-01-01T00-00-00', { generatedAt: { malformed: true } });+ const oldest = await seedSnapshot(machineHost, '2020-01-01T00-00-00', { generatedAt: '2020-01-01T00:00:00.000Z' });const readable = await seedSnapshot(machineHost, '2020-01-02T00-00-00', { generatedAt: '2099-01-01T00:00:00.000Z' });const corrupt = await seedSnapshot(machineHost, '2020-01-03T00-00-00');await fsp.writeFile(joinPath(corrupt, 'manifest.json'), '{truncated');@@ -5074,7 +5150,7 @@ describe('runBackup lifecycle', () => {await expect(fsp.stat(joinPath(destRoot, 'snapshots', machineHost, result.snapshotId))).resolves.toBeDefined();await expect(fsp.stat(corrupt)).resolves.toBeDefined();await expect(fsp.stat(unreadable)).resolves.toBeDefined();- expect(warn).toHaveBeenCalledWith('⚠️ Backup retention: kept snapshot 2019-12-31T00-00-00 (manifest unreadable)');+ expect(warn).toHaveBeenCalledWith('⚠️ Backup retention: kept snapshot 2019-12-31T00-00-00 (manifest unreadable or invalid)');});it('keeps a completed backup after clock correction when a future snapshot manifest cannot be read (#12340)', async () => {@@ -5097,12 +5173,50 @@ describe('runBackup lifecycle', () => {expect(result).toMatchObject({ status: 'ok', prunedSnapshots: 0 });await expect(fsp.stat(joinPath(destRoot, 'snapshots', machineHost, result.snapshotId))).resolves.toBeDefined();await expect(fsp.stat(future)).resolves.toBeDefined();- expect(warn).toHaveBeenCalledWith('⚠️ Backup retention: kept snapshot 2099-01-01T00-00-00 (manifest unreadable)');+ expect(warn).toHaveBeenCalledWith('⚠️ Backup retention: kept snapshot 2099-01-01T00-00-00 (manifest unreadable or invalid)');} finally {vi.useRealTimers();}});+ it('excludes every invalid manifest shape from a successful dump retention budget (#12396)', async () => {+ const fsp = await actualFs();+ const valid = { generatedAt: '2099-01-01T00:00:00Z', fileCount: 0, files: {} };+ const invalid = [null, [], {}, { ...valid, generatedAt: {} }, { ...valid, fileCount: 1 },+ { ...valid, fileCount: 1, files: { '../escape': 'a'.repeat(64) } },+ { ...valid, fileCount: 1, files: { 'record.json': 'bad-hash' } }];+ const protectedIds = [];+ for (const [index, manifest] of invalid.entries()) {+ const id = `2099-01-0${index + 1}T00-00-00`;+ const dir = await seedSnapshot(machineHost, id);+ await fsp.writeFile(joinPath(dir, 'manifest.json'), JSON.stringify(manifest));+ protectedIds.push(id);+ await expect(restoreSnapshot(destRoot, id, { dryRun: true }))+ .rejects.toMatchObject({ code: 'BACKUP_MANIFEST_INVALID' });+ }+ const legacy = await seedSnapshot(machineHost, '2019-01-01T00-00-00');+ const healthy = await seedSnapshot(machineHost, '2020-01-01T00-00-00', { generatedAt: valid.generatedAt });+ checkHealth.mockResolvedValue({ connected: true, hasSchema: true });+ getBackendName.mockReturnValue('postgres');+ const rsync = fakeProc();+ const dump = fakeProc();+ spawn.mockReturnValueOnce(rsync).mockReturnValueOnce(dump);+ const pending = runBackup(destRoot, null, { retentionCount: 1 });+ await waitFor(() => spawn.mock.calls.length === 1, 'rsync spawn');+ rsync.emit('close', 0);+ await waitFor(() => spawn.mock.calls.length === 2, 'pg_dump spawn');+ const args = spawn.mock.calls[1][1];+ await fsp.writeFile(args[args.indexOf('-f') + 1], 'CREATE TABLE example (id integer);');+ dump.emit('close', 0);+ const result = await pending;+ expect(result).toMatchObject({ status: 'ok', pgBackup: { status: 'ok' }, prunedSnapshots: 2 });+ for (const id of [...protectedIds, result.snapshotId]) {+ await expect(fsp.stat(joinPath(destRoot, 'snapshots', machineHost, id))).resolves.toBeDefined();+ }+ await expect(fsp.stat(legacy)).rejects.toThrow();+ await expect(fsp.stat(healthy)).rejects.toThrow();+ });+it('does not prune anything when retentionCount is null (unlimited — the legacy/unset default)', async () => {const fsp = await actualFs();const old1 = await seedSnapshot(machineHost, '2020-01-01T00-00-00', { generatedAt: '2020-01-01T00:00:00.000Z' });