Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix: purge reports a blob missing from settings.storage as remaining, never deleted

garrytanfcb01eb99c88
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7eadf4e271..29871c4b1d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -18,7 +18,7 @@ identifiers and attribution are available in the pre-removal Git revision
- `src/core/ops/files.ts`: `file_register` accepts only the upload layout path (`<page_slug>/<filename>`, or `unsorted/<first 8 hash chars>-<filename>`), refuses `page_not_found` for a page that does not exist or that the caller's read policy hides (member access and privacy, the private-pages hatch included), and records nothing unless the stored object matches. Rows record their store's backend and prefix. Registering the same path again updates its row.
- A store that does not answer is the new `store_unavailable` code, never a file reported missing and never a row recorded.
-- On an engine opened from settings, `delete_page` purge deletes a page's files from `settings.storage` too, and reports them remaining when the engine has none. Rows an embedded engine's `file_upload` wrote before this release point at the process store, so `file_url` and `file_verify` look for them in `settings.storage`.
+- On an engine opened from settings, `delete_page` purge deletes a page's files from `settings.storage` too. A file it can't find there, or a store that doesn't answer, is reported in `blobs_remaining`, never as deleted; files an embedded engine's `file_upload` wrote before this release live in the process store, so delete those remaining paths by hand. Rows an embedded engine's `file_upload` wrote before this release point at the process store, so `file_url` and `file_verify` look for them in `settings.storage`.
- `openEngine` refuses `engine_settings_invalid` for a `storage` setting missing a value its backend needs (so no backend falls back to a default path or environment credentials), an unknown field, a backend other than local or s3, or a prefix that is not one segment; an s3 store must name its endpoint, so no AWS_* variable or ~/.aws file picks it; `embeddedSettings()` reads back `hasSecret` instead of the secret.
- `src/core/storage/local.ts` and `s3.ts` stream an object's size and SHA-256 (`hashObject`); other backends hash one download.
- `test/file-register.test.ts`: real PGLite engines and a real local store; the unreachable store is a closed local port.
diff --git a/src/core/persistence/page-purge.ts b/src/core/persistence/page-purge.ts
index 48811d3283..52ea131e5d 100644
--- a/src/core/persistence/page-purge.ts
+++ b/src/core/persistence/page-purge.ts
@@ -379,10 +379,16 @@ export async function finishPagePurge(ctx: Pick<OperationContext, 'engine' | 'co
if (purge.blobs?.length) {
try {
const { createStorage, settingsStorage } = await import('../storage.ts');
- const storage = isEmbeddedEngine(ctx.engine) ? await settingsStorage(ctx.engine)
+ const embedded = isEmbeddedEngine(ctx.engine);
+ const storage = embedded ? await settingsStorage(ctx.engine)
: await createStorage((ctx.config as { storage?: unknown } | undefined)?.storage as never);
if (!storage) throw new Error('This engine was opened with no settings.storage.');
- for (const path of purge.blobs) { try { await storage.delete(path); } catch { remaining.push(path); } }
+ for (const path of purge.blobs) {
+ try {
+ if (embedded && !(await storage.exists(path))) { remaining.push(path); continue; }
+ await storage.delete(path);
+ } catch { remaining.push(path); }
+ }
} catch { remaining.push(...purge.blobs); }
}
const { verifyPagePurge } = await import('../facts/purge-verify.ts');
diff --git a/test/file-register.test.ts b/test/file-register.test.ts
index bff91199c4..7062b194a6 100644
--- a/test/file-register.test.ts
+++ b/test/file-register.test.ts
@@ -167,6 +167,19 @@ describe('file_register', () => {
expect([kept.purge.blobs_remaining, existsSync(join(processDir, 'notes/plan/p.txt'))]).toEqual([['notes/plan/p.txt'], true]);
});
+ test('a purge never counts a blob missing from settings.storage as deleted (an older row whose object is in the process store)', async () => {
+ const engine = await embedded({ storage: { ...storage, prefix: 'brain-a' } });
+ const processDir = join(root, 'process-store-4');
+ mkdirSync(join(processDir, 'notes/plan'), { recursive: true });
+ writeFileSync(join(processDir, 'notes/plan/old.txt'), 'uploaded before this release');
+ store('brain-a/notes/plan/new.txt', Buffer.from('embedded'));
+ const { finishPagePurge } = await import('../src/core/persistence/page-purge.ts');
+ const purged = { status: 'purged', source_id: 'default', slug: 'notes/plan', request_id: randomUUID(), purge: { blobs: ['notes/plan/old.txt', 'notes/plan/new.txt'], removed: {} } };
+ const result = await finishPagePurge({ engine, config: { storage: { backend: 'local', bucket: 'test', localPath: processDir } } } as never, purged) as { purge: { blobs_remaining: string[]; blobs_deleted: number } };
+ expect([result.purge.blobs_remaining, result.purge.blobs_deleted]).toEqual([['notes/plan/old.txt'], 1]);
+ expect([existsSync(join(processDir, 'notes/plan/old.txt')), existsSync(join(storageRoot, 'brain-a/notes/plan/new.txt'))]).toEqual([true, false]);
+ });
+
test('a store that cannot be reached is store_unavailable and records nothing', async () => {
const engine = await embedded({ storage: unreachable });
const bytes = Buffer.from('x');