Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

fix: preserve backup retention and restored metadata

[._.]/ Adam Eivy5954235f17ca
diff --git a/docs/BACKUP.md b/docs/BACKUP.md
index 80595bd910..00a00c3e0f 100644
--- a/docs/BACKUP.md
+++ b/docs/BACKUP.md
@@ -509,6 +509,10 @@ After the preflight, rsync copies `<snapshot>/data/` back to `./data/`. Restore
File restore copies what the snapshot holds and leaves live files the snapshot lacks in place. One consequence is handled explicitly: a snapshot taken before the CoS config moved to its own file (#6182) carries its CoS settings inside `cos/state.json` and has no `cos/config.json`. When such a restore lands that legacy slice, the live `cos/config.json` (newer than the restore point) is moved aside to `cos/config.pre-restore-<timestamp>.json` and the settings are recovered from the restored `state.json`; otherwise the newer file would keep winning every read.
+**Historical file-to-database metadata (#12397).** On a PostgreSQL install, full and selective file restores refuse selected legacy metadata with `409 BACKUP_LEGACY_DB_METADATA` before preview or execution starts a transfer. The diagnostic names only affected domains: universes (record `index.json` and type-level `config.runs`), pipeline issues/series, Story Builder, Writers Room (`folders.json`, `exercises.json`, work `manifest.json`), and Creative Director projects. These records now live in PostgreSQL. A live domain import marker can suppress re-import after file restore; restoring the older file-migration ledger does not reset that separate marker. Clearing markers is unsafe too: the importers preserve existing rows with `ON CONFLICT DO NOTHING`, so they cannot provide replacement-restore semantics for overlapping IDs.
+
+Keep the original backup intact. Copy it into a disposable offline install running the compatible historical version, upgrade that isolated install through the file-to-database transition, and verify the imported records and linked files there. Make a current-format file and PostgreSQL backup from the converted install, then use the normal verified restore workflow. Do not convert against the live database or delete its markers. Current file-primary siblings (such as manuscript reviews, draft Markdown and media assets) and parked `.imported` recovery evidence remain eligible for file restore. Selecting one of those siblings is allowed even if another part of the snapshot contains legacy metadata.
+
A full restore also installs the snapshot's `migrations.applied.json`, so the next start re-runs every file migration the snapshot predates. A folder restore (`subdirFilter`) keeps the live ledger, so restored files can come back in a shape older than migrations no runner will revisit. Its preview and result therefore carry `ledgerGap`: `{ count, names }` listing the migrations the live ledger records and the snapshot's lacks, or `null` when either ledger is missing or unreadable. A full restore omits the field. The widget warns on a non-empty gap and recommends a full restore, but does not refuse: a folder no migration touched restores correctly. The quota-burn plan is converted on the spot: a restore covering `cos/quota-burn.json` (full, `cos`, or that file) runs any legacy `jobType` steps through the same conversion as migration 359 (`reconcileRestoredQuotaBurnPlan`), reusing each step's deterministic `job-burn-<family>-<step>` custom job, so a repeated restore creates no duplicates.
**Every nonzero rsync exit fails a restore, including 24.** A backup tolerates exit 24 ("some source files vanished") because it reads the live data tree, where a file deleted mid-scan is normal. A restore reads a snapshot that must not change under it, so a vanished source file means live data may now mix restored and older files: preview and execution both reject it. An execution failure is reported as a partial restore — the error says some files may already have been overwritten because file restore is not transactional — and caches (settings, Brain, media index) are reconciled before the error is returned, exactly as for any other failed transfer.
@@ -572,7 +576,7 @@ Left unbounded, `snapshots/<hostname>/` grows forever — every prior full DB du
- **`retentionCount` absent or explicitly `null` both mean unlimited** — no pruning runs. This is why an install that predates this setting keeps every snapshot until the operator saves a choice: it has no stored value, and absence resolves to unlimited, not to the new-install default.
- **Pruning runs only after a run reaches a completed snapshot with a usable database dump** (`pg_dump` succeeded, or was explicitly skipped by the file-backend escape hatch). A degraded run whose database dump failed keeps older snapshots and reports zero pruned snapshots; retention resumes after a later successful dump. A run that fails before completion never prunes, and a prune failure is logged without failing an otherwise-successful backup.
- **Pruning is scoped to the CURRENT machine's namespace** (`snapshots/<hostname>/`) and only ever deletes snapshots whose `snapshotState()` reports neither `incomplete` nor `failed`. It never touches another machine's namespace in a shared destination, the legacy pre-namespace root, or an in-progress/failed snapshot — those stay until the operator deletes them explicitly. It also skips a snapshot that a restore or download is still reading (see [Snapshot source lifetime](#how-restore-works)); the skipped snapshot is not counted as pruned and becomes a candidate again on a later run.
-- **Unreadable or corrupt manifests are protected in addition to the retention count.** They never displace readable completed snapshots, even when an incorrect clock gave a protected snapshot a future-dated ID. Snapshot IDs determine age; manifest timestamps do not change that ordering. Legacy completed snapshots with no manifest remain eligible for retention as before.
+- **Unreadable or structurally invalid manifests are protected in addition to the retention count.** Retention uses the same field, count, path and SHA-256-shape validator as file restore, without hashing old payloads. Existing JSON `null`, arrays, malformed objects and invalid entries are not legacy absence. They never displace structurally valid completed snapshots, even when an incorrect clock gave a protected snapshot a future-dated ID. Snapshot IDs determine age; manifest timestamps do not change that ordering. Legacy completed snapshots with no manifest remain eligible for retention as before.
- **`DELETE /api/backup/snapshots/:snapshotId?source=<source>`** (`backup.deleteSnapshot()`) permanently removes exactly the selected source/ID pair, immediately — there is no undo and it is never automatic. It shares `resolveSnapshotPath()`'s path-traversal and symlink guards with restore/download, and refuses a snapshot that is still being written (`SNAPSHOT_INCOMPLETE`) or that a restore or download is still reading (`SNAPSHOT_IN_USE`); unlike restore, a `.failed` snapshot **is** deletable. The Backup settings tab's snapshot history exposes this as a per-row delete action behind a confirmation dialog.
## Scheduling & status
diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md
index bbd718d91a..119548d168 100644
--- a/docs/TROUBLESHOOTING.md
+++ b/docs/TROUBLESHOOTING.md
@@ -514,6 +514,12 @@ npm run pm2:start
The split never overwrites a record directory that already exists. If a later Postgres import (`server/scripts/migrateUniversesToDB.js` and similar) already ran, re-run that explicit legacy-import tool after the split rather than resetting import markers or schema versions.
+### Collection images or pinned covers lost after migration 428
+
+Migration 428 inferred deletion from missing local image bytes. That is invalid for peer-hosted media, whose bytes intentionally remain on the hosting peer, and for temporarily unavailable files. The migration runner now records pending `428-prune-dangling-collection-images.js` without executing its frozen transform, including when the ledger is missing or rebuilt. This policy is persisted before any other pending migration runs, so an interrupted upgrade cannot re-arm it. Explicit gallery deletion still removes the references for the image being deleted.
+
+If 428 already ran, preserve the current collection and recover membership from an unaffected peer or retained backup. Union synchronization may re-add the image items, but the migration also stamped a newer null `coverKey`; scalar last-write-wins synchronization cannot automatically restore that pinned cover. Select the intended cover explicitly after recovery. The peer-hosted media index does not record collection ownership and cannot reconstruct memberships. Do not rerun absence-based cleanup or overwrite newer edits with reconstructed guesses.
+
### "Another migration run holds data/migrations.lock"
**Symptom**: `npm run migrations` or server boot exits with `Another migration run (pid N) holds …/data/migrations.lock`.
diff --git a/scripts/run-migrations.js b/scripts/run-migrations.js
index 26c6db1bd5..40735246d1 100644
--- a/scripts/run-migrations.js
+++ b/scripts/run-migrations.js
@@ -270,6 +270,17 @@ async function runPendingMigrations({ rootDir, migrationsDir, appliedFile }) {
await writeJsonAtomic(appliedFile, applied);
};
+ // Retire this frozen transform before ANY pending migration executes, even
+ // with a rebuilt ledger or a later failure. Local absence cannot prove image
+ // deletion: host-mode peers intentionally retain metadata without bytes.
+ let disarmed = 0;
+ const retired = '428-prune-dangling-collection-images.js';
+ if (files.includes(retired) && !applied.includes(retired)) {
+ applied.push(retired);
+ disarmed++;
+ console.warn(`⏭️ Retired migration ${retired}: local image absence is not deletion evidence; recorded without running (#12398)`);
+ }
+
// Disarm EVERY pending purge migration up front, not as each is reached in
// the run loop: if this rebuilt-from-empty run aborted on an earlier
// throwing migration (the documented repair-and-reboot flow), the partial
@@ -277,7 +288,6 @@ async function runPendingMigrations({ rootDir, migrationsDir, appliedFile }) {
// would then execute destructively against the rebuilt install. The probe
// reads source text, never importing a module before its turn in the loop.
if (ledgerStartedEmpty) {
- let disarmed = 0;
for (const file of files) {
if (applied.includes(file)) continue;
if (!sourceDeclaresPurge(await readFile(join(migrationsDir, file), 'utf-8'))) continue;
@@ -285,8 +295,10 @@ async function runPendingMigrations({ rootDir, migrationsDir, appliedFile }) {
applied.push(file);
disarmed++;
}
- if (disarmed > 0) await persistAppliedList();
}
+ // Publish both policies together: persisting retirement first would leave a
+ // nonempty ledger after a crash, re-arming pending purge-class migrations.
+ if (disarmed > 0) await persistAppliedList();
let ran = 0;
for (const file of files) {
diff --git a/scripts/run-migrations.test.js b/scripts/run-migrations.test.js
index dd86d86208..6ef6d24d8b 100644
--- a/scripts/run-migrations.test.js
+++ b/scripts/run-migrations.test.js
@@ -66,6 +66,34 @@ export default {
const corruptSiblings = () =>
readdirSync(dataDir).filter((f) => f.startsWith('migrations.applied.json.corrupt-'));
+ it.each(['pending', 'missing', 'corrupt', 'applied'])('retires unsafe 428 with a %s ledger before earlier failures (#12398)', async state => {
+ const retired = '428-prune-dangling-collection-images.js';
+ // Exercise the shipped, frozen transform through the real runner. If the
+ // retirement regresses it removes both kinds of unresolved image pointer.
+ writeFileSync(join(migrationsDir, retired), `export { default } from ${JSON.stringify(new URL('./migrations/428-prune-dangling-collection-images.js', import.meta.url).href)};`);
+ mkdirSync(join(dataDir, 'images'));
+ const collectionDir = join(dataDir, 'media-collections', 'collection-example');
+ mkdirSync(collectionDir, { recursive: true });
+ const original = JSON.stringify({ items: [{ kind: 'image', ref: 'hosted.png' }, { kind: 'image', ref: 'temporarily-missing.png' }], coverKey: 'image:hosted.png', updatedAt: '2026-01-01T00:00:00Z' });
+ writeFileSync(join(collectionDir, 'index.json'), original);
+ writeFileSync(join(dataDir, 'peer-hosted-media.json'), JSON.stringify({ version: 1, entries: { 'image:hosted.png': { peerId: 'example-peer', at: '2026-01-01T00:00:00Z' } } }));
+ if (state === 'pending') writeFileSync(appliedFile, '["000-earlier.js"]');
+ if (state === 'corrupt') writeFileSync(appliedFile, '{broken');
+ if (state === 'applied') writeFileSync(appliedFile, JSON.stringify([retired]));
+ writeFileSync(join(migrationsDir, '002-failure.js'), 'export async function up() { throw new Error("fixture interruption"); }');
+ if (state === 'missing' || state === 'corrupt') {
+ writeFileSync(join(migrationsDir, '429-purge.js'), 'export const purge = true; export async function up() { throw new Error("purge must stay disarmed"); }');
+ }
+ await expect(runMigrations({ rootDir, migrationsDir })).rejects.toThrow('fixture interruption');
+ expect(JSON.parse(readFileSync(appliedFile, 'utf8'))).toContain(retired);
+ expect(readFileSync(join(collectionDir, 'index.json'), 'utf8')).toBe(original);
+ // A retry with a now nonempty ledger cannot re-arm the retired transform.
+ rmSync(join(migrationsDir, '002-failure.js'));
+ expect(await runMigrations({ rootDir, migrationsDir })).toBe(0);
+ expect(readFileSync(join(collectionDir, 'index.json'), 'utf8')).toBe(original);
+ if (state !== 'applied') expect(warnSpy).toHaveBeenCalledWith(expect.stringContaining('Retired migration 428'));
+ });
+
it('renames truncated JSON aside and rebuilds from scratch', async () => {
writeFileSync(appliedFile, '[\n "001-fixture.js",\n "002-'); // truncated mid-write
diff --git a/server/services/backup.js b/server/services/backup.js
index dac4eeb687..744e5d2c26 100644
--- a/server/services/backup.js
+++ b/server/services/backup.js
@@ -1148,22 +1148,23 @@ async function pruneOldSnapshots(destPath, retentionCount) {
const snapshotDir = join(sourceRoot, id);
const { incomplete, failed } = await snapshotState(snapshotDir, id, true);
if (incomplete || failed) return null;
- // A manifest that exists but cannot be read must not date or delete the
- // snapshot: fail toward keeping data (#11859).
- const manifestRead = await readJSONFileStrict(join(snapshotDir, 'manifest.json'), null, { logError: false });
- return { id, snapshotDir, createdAt: manifestRead.value?.generatedAt ?? null, protectedFromPrune: !manifestRead.ok };
+ // Share restore's structural admission, without hashing old payloads.
+ // Only confirmed absence is legacy; JSON null is an invalid manifest.
+ const manifestRead = await readJSONFileStrict(join(snapshotDir, 'manifest.json'), MANIFEST_ABSENT, { logError: false });
+ const valid = manifestRead.ok && (manifestRead.value === MANIFEST_ABSENT
+ || manifestDataEntries(manifestRead.value, join(snapshotDir, 'data'), null) !== null);
+ return { id, snapshotDir, protectedFromPrune: !valid };
}))).filter(Boolean);
// Newest-first by snapshot id (an ISO-style timestamp that never depends on
- // reading the manifest); manifest.generatedAt is only a tiebreak.
- descriptors.sort((a, b) => b.id.localeCompare(a.id)
- || (b.createdAt ?? '').localeCompare(a.createdAt ?? ''));
+ // reading the manifest). IDs are unique within this source namespace.
+ descriptors.sort((a, b) => b.id.localeCompare(a.id));
// Keep protected snapshots in addition to the readable retention budget,
// even when a bad clock placed a protected snapshot ahead of healthy ones.
const toDelete = descriptors.filter(d => {
if (d.protectedFromPrune) {
- console.warn(`⚠️ Backup retention: kept snapshot ${d.id} (manifest unreadable)`);
+ console.warn(`⚠️ Backup retention: kept snapshot ${d.id} (manifest unreadable or invalid)`);
return false;
}
return true;
@@ -1577,6 +1578,36 @@ async function snapshotScopeInventory(srcDir, subdirFilter) {
return { paths, largestFileBytes };
}
+// These exact metadata shapes belonged to six marker-gated file→DB imports.
+// Sibling assets and parked .imported evidence remain file-primary recovery data.
+const LEGACY_DB_METADATA = [
+ ['universes', /^universes\/[^/]+\/index\.json$/],
+ ['pipeline-issues', /^pipeline-issues\/iss-[A-Za-z0-9-]+\/index\.json$/],
+ ['pipeline-series', /^pipeline-series\/ser-[A-Za-z0-9-]+\/index\.json$/],
+ ['story-builder', /^story-builder\/stb-[A-Za-z0-9-]+\/index\.json$/],
+ ['writers-room', /^writers-room\/(?:(?:folders|exercises)\.json|works\/[^/]+\/manifest\.json)$/],
+ ['creative-director', /^creative-director-projects\.json$/],
+];
+
+async function assertRestoreStorageCompatible(srcDir, inventory, snapshotId) {
+ // Match dumpPostgres: an uninitialized backend still requires PostgreSQL.
+ if (process.env.MEMORY_BACKEND === 'file' || getBackendName() === 'file') return;
+ if (!inventory) throw snapshotFileIntegrityError(snapshotId);
+ const domains = new Set(LEGACY_DB_METADATA
+ .filter(([, pattern]) => inventory.paths.some(path => pattern.test(path)))
+ .map(([domain]) => domain));
+ if (inventory.paths.includes('universes/index.json')) {
+ const index = await readJSONFileStrict(join(srcDir, 'universes', 'index.json'), null, { logError: false });
+ if (!index.ok || index.value?.config?.runs !== undefined) domains.add('universes');
+ }
+ if (domains.size) {
+ throw new ServerError(
+ `Historical file metadata for ${[...domains].join(', ')} cannot restore database-primary records. Convert this backup in a disposable offline upgrade install, verify its records, then restore a current-format file/database backup (docs/BACKUP.md).`,
+ { status: 409, code: 'BACKUP_LEGACY_DB_METADATA', context: { domains: [...domains] } },
+ );
+ }
+}
+
async function verifySnapshotFiles(snapshotDir, srcDir, snapshotId, subdirFilter) {
// A unique fallback distinguishes a genuinely absent legacy manifest from an
// existing file whose parsed value is JSON null.
@@ -1593,10 +1624,11 @@ async function verifySnapshotFiles(snapshotDir, srcDir, snapshotId, subdirFilter
}
if (manifestRead.value === MANIFEST_ABSENT) {
// No manifest to verify against, but the restore's idle-deadline floor
- // still needs the largest in-scope file. Walk it best-effort: a scope that
- // can't be surveyed keeps the default deadline rather than failing a
- // restore that would previously have run.
+ // still needs the largest in-scope file. PostgreSQL also needs a complete
+ // inventory to reject ignored legacy metadata; only the file escape hatch
+ // can retain the old best-effort behavior when inventory fails.
const legacyInventory = await snapshotScopeInventory(srcDir, subdirFilter).catch(() => null);
+ await assertRestoreStorageCompatible(srcDir, legacyInventory, snapshotId);
return {
status: 'unverified',
reason: 'manifest_absent',
@@ -1620,6 +1652,7 @@ async function verifySnapshotFiles(snapshotDir, srcDir, snapshotId, subdirFilter
const inventory = await snapshotScopeInventory(srcDir, subdirFilter)
.catch(() => null);
if (!inventory) throw snapshotFileIntegrityError(snapshotId);
+ await assertRestoreStorageCompatible(srcDir, inventory, snapshotId);
const { paths: snapshotPaths, largestFileBytes } = inventory;
const unmanifested = snapshotPaths.find(entry => !dataPaths.has(entry));
if (unmanifested) throw snapshotFileIntegrityError(snapshotId, unmanifested);
diff --git a/server/services/backup.test.js b/server/services/backup.test.js
index 8e0f194743..2aa8cf8310 100644
--- a/server/services/backup.test.js
+++ b/server/services/backup.test.js
@@ -2273,6 +2273,82 @@ describe('restoreSnapshot manifest verification', () => {
return pending;
}
+ it.each([
+ ['universes', 'universes/universe-example/index.json', { id: 'universe-example' }],
+ ['universes', 'universes/index.json', { config: { runs: [{ id: 'run-example' }] } }],
+ ['pipeline-issues', 'pipeline-issues/iss-example/index.json', { id: 'iss-example' }],
+ ['pipeline-series', 'pipeline-series/ser-example/index.json', { id: 'ser-example' }],
+ ['story-builder', 'story-builder/stb-example/index.json', { id: 'stb-example' }],
+ ['writers-room', 'writers-room/folders.json', [{ id: 'folder-example' }]],
+ ['writers-room', 'writers-room/exercises.json', [{ id: 'exercise-example' }]],
+ ['writers-room', 'writers-room/works/wr-work-aabb/manifest.json', { id: 'wr-work-aabb' }],
+ ['creative-director', 'creative-director-projects.json', [{ id: 'project-example' }]],
+ ])('refuses %s legacy metadata at %s before preview or execution (#12397)', async (domain, path, record) => {
+ getBackendName.mockReturnValue('postgres');
+ const savedBackend = process.env.MEMORY_BACKEND;
+ delete process.env.MEMORY_BACKEND;
+ const markerPath = joinPath(PATHS.data, `${domain === 'creative-director' ? 'creative-director-projects' : domain}.migrated.json`);
+ await realFs.writeFile(markerPath, '{"imported":1}');
+ try {
+ const hash = await writeSnapshotFile(path, JSON.stringify(record));
+ await writeManifest({ [path]: hash });
+ // Both empty and overlapping destination databases must refuse: this is
+ // source compatibility, not an opportunistic ON CONFLICT merge.
+ for (const rows of [[], [{ id: 'existing-record' }]]) {
+ query.mockResolvedValue({ rows });
+ for (const dryRun of [true, false]) {
+ for (const subdirFilter of [null, path.split('/')[0], path]) {
+ await expect(restoreSnapshot(tmpRoot, 'snap-1', { dryRun, subdirFilter }))
+ .rejects.toMatchObject({ code: 'BACKUP_LEGACY_DB_METADATA', context: { domains: [domain] } });
+ }
+ }
+ }
+ expect(spawn).not.toHaveBeenCalled();
+ expect(query).not.toHaveBeenCalled();
+ expect(await realFs.readFile(markerPath, 'utf8')).toBe('{"imported":1}');
+ // Pre-manifest snapshots get the same compatibility preflight.
+ await realFs.rm(joinPath(snapshotDir, 'manifest.json'));
+ await expect(restoreSnapshot(tmpRoot, 'snap-1', { dryRun: false }))
+ .rejects.toMatchObject({ code: 'BACKUP_LEGACY_DB_METADATA' });
+ } finally {
+ if (savedBackend === undefined) delete process.env.MEMORY_BACKEND;
+ else process.env.MEMORY_BACKEND = savedBackend;
+ getBackendName.mockReturnValue(null);
+ query.mockResolvedValue({ rows: [] });
+ await realFs.rm(markerPath);
+ }
+ });
+
+ it('admits current file-primary siblings and parked recovery evidence on PostgreSQL', async () => {
+ getBackendName.mockReturnValue('postgres');
+ const savedBackend = process.env.MEMORY_BACKEND;
+ delete process.env.MEMORY_BACKEND;
+ try {
+ const paths = [
+ 'universes/universe-example/assets/sheet.png', 'universes.imported/universe-example/index.json',
+ 'universes/index.json', 'pipeline-issues/iss-example/render.png',
+ 'pipeline-series/ser-example/manuscript-review.json', 'pipeline-series/ser-example/index.json.imported',
+ 'story-builder.imported/stb-example/index.json', 'writers-room/folders.imported.json',
+ 'writers-room/exercises.imported.json', 'writers-room/works/wr-work-aabb/manifest.imported.json',
+ 'writers-room/works/wr-work-aabb/drafts/draft-example.md', 'creative-director-projects.imported.json',
+ ];
+ const files = {};
+ for (const path of paths) files[path] = await writeSnapshotFile(path, '{}');
+ await writeManifest(files);
+ await expect(finishRestore({ dryRun: true })).resolves.toMatchObject({ verification: { status: 'verified' } });
+ // A mixed snapshot still permits selecting only the current draft body.
+ files['writers-room/folders.json'] = await writeSnapshotFile('writers-room/folders.json', '[]');
+ await writeManifest(files);
+ spawn.mockReset();
+ await expect(finishRestore({ dryRun: false, subdirFilter: 'writers-room/works/wr-work-aabb/drafts' }))
+ .resolves.toMatchObject({ verification: { status: 'verified', checkedFiles: 1 } });
+ } finally {
+ if (savedBackend === undefined) delete process.env.MEMORY_BACKEND;
+ else process.env.MEMORY_BACKEND = savedBackend;
+ getBackendName.mockReturnValue(null);
+ }
+ });
+
it('rechecks snapshot bytes on execution and refuses a post-preview change before rsync', async () => {
const relativePath = 'restore-integrity/example.json';
const originalHash = await writeSnapshotFile(relativePath, 'trusted backup');
@@ -4901,7 +4977,7 @@ describe('runBackup lifecycle', () => {
const dir = joinPath(destRoot, 'snapshots', source, id);
await fsp.mkdir(joinPath(dir, 'data'), { recursive: true });
if (generatedAt) {
- await fsp.writeFile(joinPath(dir, 'manifest.json'), JSON.stringify({ generatedAt, fileCount: 0 }));
+ await fsp.writeFile(joinPath(dir, 'manifest.json'), JSON.stringify({ generatedAt, fileCount: 0, files: {} }));
}
if (marker) await fsp.writeFile(joinPath(dir, marker), '');
return dir;
@@ -5053,7 +5129,7 @@ describe('runBackup lifecycle', () => {
it('retains the readable snapshot budget in addition to corrupt manifests (#12340)', async () => {
const fsp = await actualFs();
- const oldest = await seedSnapshot(machineHost, '2020-01-01T00-00-00', { generatedAt: { malformed: true } });
+ const oldest = await seedSnapshot(machineHost, '2020-01-01T00-00-00', { generatedAt: '2020-01-01T00:00:00.000Z' });
const readable = await seedSnapshot(machineHost, '2020-01-02T00-00-00', { generatedAt: '2099-01-01T00:00:00.000Z' });
const corrupt = await seedSnapshot(machineHost, '2020-01-03T00-00-00');
await fsp.writeFile(joinPath(corrupt, 'manifest.json'), '{truncated');
@@ -5074,7 +5150,7 @@ describe('runBackup lifecycle', () => {
await expect(fsp.stat(joinPath(destRoot, 'snapshots', machineHost, result.snapshotId))).resolves.toBeDefined();
await expect(fsp.stat(corrupt)).resolves.toBeDefined();
await expect(fsp.stat(unreadable)).resolves.toBeDefined();
- expect(warn).toHaveBeenCalledWith('⚠️ Backup retention: kept snapshot 2019-12-31T00-00-00 (manifest unreadable)');
+ expect(warn).toHaveBeenCalledWith('⚠️ Backup retention: kept snapshot 2019-12-31T00-00-00 (manifest unreadable or invalid)');
});
it('keeps a completed backup after clock correction when a future snapshot manifest cannot be read (#12340)', async () => {
@@ -5097,12 +5173,50 @@ describe('runBackup lifecycle', () => {
expect(result).toMatchObject({ status: 'ok', prunedSnapshots: 0 });
await expect(fsp.stat(joinPath(destRoot, 'snapshots', machineHost, result.snapshotId))).resolves.toBeDefined();
await expect(fsp.stat(future)).resolves.toBeDefined();
- expect(warn).toHaveBeenCalledWith('⚠️ Backup retention: kept snapshot 2099-01-01T00-00-00 (manifest unreadable)');
+ expect(warn).toHaveBeenCalledWith('⚠️ Backup retention: kept snapshot 2099-01-01T00-00-00 (manifest unreadable or invalid)');
} finally {
vi.useRealTimers();
}
});
+ it('excludes every invalid manifest shape from a successful dump retention budget (#12396)', async () => {
+ const fsp = await actualFs();
+ const valid = { generatedAt: '2099-01-01T00:00:00Z', fileCount: 0, files: {} };
+ const invalid = [null, [], {}, { ...valid, generatedAt: {} }, { ...valid, fileCount: 1 },
+ { ...valid, fileCount: 1, files: { '../escape': 'a'.repeat(64) } },
+ { ...valid, fileCount: 1, files: { 'record.json': 'bad-hash' } }];
+ const protectedIds = [];
+ for (const [index, manifest] of invalid.entries()) {
+ const id = `2099-01-0${index + 1}T00-00-00`;
+ const dir = await seedSnapshot(machineHost, id);
+ await fsp.writeFile(joinPath(dir, 'manifest.json'), JSON.stringify(manifest));
+ protectedIds.push(id);
+ await expect(restoreSnapshot(destRoot, id, { dryRun: true }))
+ .rejects.toMatchObject({ code: 'BACKUP_MANIFEST_INVALID' });
+ }
+ const legacy = await seedSnapshot(machineHost, '2019-01-01T00-00-00');
+ const healthy = await seedSnapshot(machineHost, '2020-01-01T00-00-00', { generatedAt: valid.generatedAt });
+ checkHealth.mockResolvedValue({ connected: true, hasSchema: true });
+ getBackendName.mockReturnValue('postgres');
+ const rsync = fakeProc();
+ const dump = fakeProc();
+ spawn.mockReturnValueOnce(rsync).mockReturnValueOnce(dump);
+ const pending = runBackup(destRoot, null, { retentionCount: 1 });
+ await waitFor(() => spawn.mock.calls.length === 1, 'rsync spawn');
+ rsync.emit('close', 0);
+ await waitFor(() => spawn.mock.calls.length === 2, 'pg_dump spawn');
+ const args = spawn.mock.calls[1][1];
+ await fsp.writeFile(args[args.indexOf('-f') + 1], 'CREATE TABLE example (id integer);');
+ dump.emit('close', 0);
+ const result = await pending;
+ expect(result).toMatchObject({ status: 'ok', pgBackup: { status: 'ok' }, prunedSnapshots: 2 });
+ for (const id of [...protectedIds, result.snapshotId]) {
+ await expect(fsp.stat(joinPath(destRoot, 'snapshots', machineHost, id))).resolves.toBeDefined();
+ }
+ await expect(fsp.stat(legacy)).rejects.toThrow();
+ await expect(fsp.stat(healthy)).rejects.toThrow();
+ });
+
it('does not prune anything when retentionCount is null (unlimited — the legacy/unset default)', async () => {
const fsp = await actualFs();
const old1 = await seedSnapshot(machineHost, '2020-01-01T00-00-00', { generatedAt: '2020-01-01T00:00:00.000Z' });