Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(storage): prefixedStorage confines every key to its prefix
garrytanf2076dd1ad32
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex b05fbf5d85..ff6478af19 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -12,7 +12,7 @@ identifiers and attribution are available in the pre-removal Git revision## [Unreleased]-**A brain opened from settings keeps its files in its own store.** On an engine opened from settings, every file operation (`file_upload`, `file_url`, page purge and the new `file_register` and `file_verify`) uses the engine's own `settings.storage`: a local path, or s3 with its endpoint and keys, and an optional one-segment key `prefix` so brains sharing a bucket each keep their own keys. The process storage config is never used there. `file_register` records the files row for an object the host already wrote at the path `file_upload` would use, after reading it back and matching its size and SHA-256, and `file_verify` lists which rows still have their object or re-hashes one. A brain the CLI or a server opens is unchanged, and there both refuse `embedded_engine_required`; neither is ever an MCP tool. Contributed by @bradgessler (#8457).+**A brain opened from settings keeps its files in its own store.** On an engine opened from settings, every file operation (`file_upload`, `file_url`, page purge and the new `file_register` and `file_verify`) uses the engine's own `settings.storage`: a local path, or s3 with its endpoint and keys, and an optional one-segment key `prefix` so brains sharing a bucket each keep their own keys. The process storage config is never used there. `file_register` records the files row for an object the host already wrote at the path `file_upload` would use, after reading it back and matching its size and SHA-256, and `file_verify` lists which rows still have their object or re-hashes one. A brain the CLI or a server opens is unchanged, and there both refuse `embedded_engine_required`; neither is ever an MCP tool. A registration never moves a row to another page, compared by id as well as slug. On every engine, `file_upload` now records each row's source and page, so purging a page in any source removes its attachments (#8513), and a prefixed store refuses any key that would leave its prefix. Contributed by @bradgessler (#8457).### Itemized changes@@ -22,6 +22,7 @@ identifiers and attribution are available in the pre-removal Git revision- A store that does not answer is the new `store_unavailable` code, never a file reported missing and never a row recorded.- On an engine opened from settings, `delete_page` purge deletes a page's files from `settings.storage` too. A file it can't find there, or a store that doesn't answer, is reported in `blobs_remaining`, never as deleted; files an embedded engine's `file_upload` wrote before this release live in the process store, so delete those remaining paths by hand. Rows an embedded engine's `file_upload` wrote before this release point at the process store, so `file_url` and `file_verify` look for them in `settings.storage`.- `openEngine` refuses `engine_settings_invalid` for a `storage` setting missing a value its backend needs (so no backend falls back to a default path or environment credentials), an unknown field, a backend other than local or s3, or a prefix that is not one segment; an s3 store must name its endpoint, so no AWS_* variable or ~/.aws file picks it; `embeddedSettings()` reads back `hasSecret` instead of the secret.+- `src/core/storage.ts`: `prefixedStorage` refuses a key with an empty, `.` or `..` segment or a leading `/` (`Path traversal blocked`), so on a local store a path such as `../brain-b/x` can never reach another brain's prefix.- `src/core/storage/local.ts` and `s3.ts` stream an object's size and SHA-256 (`hashObject`); other backends hash one download.- `test/embedded-file-store.test.ts` and `test/file-register.test.ts`: real PGLite engines and a real local store; the unreachable store is a closed local port.diff --git a/src/core/storage.ts b/src/core/storage.tsindex c1c55ed40f..c2bff4eed7 100644--- a/src/core/storage.ts+++ b/src/core/storage.ts@@ -87,19 +87,28 @@ export async function settingsStorage(engine: object): Promise<StorageBackend |return prefixedStorage(await createStorage({ bucket: '', ...config }), prefix);}-/** `backend` with every object key under `<prefix>/`; list() returns keys without it. No prefix returns `backend` itself. */+/**+ * `backend` with every object key under `<prefix>/`; list() returns keys without it. No prefix returns `backend` itself.+ * A key with an empty, `.` or `..` segment (a leading `/` included) is refused, so no key reaches outside the prefix.+ */export function prefixedStorage(backend: StorageBackend, prefix: string | undefined): StorageBackend {if (!prefix) return backend;- const key = (path: string) => `${prefix}/${path}`;+ const key = (path: string, listing = false) => {+ const segments = listing && path === '' ? [] : path.split('/');+ if (segments.some(segment => segment === '' || segment === '.' || segment === '..')) {+ throw new Error(`Path traversal blocked: ${path} resolves outside storage prefix ${prefix}`);+ }+ return `${prefix}/${path}`;+ };return {- upload: (path, data, mime) => backend.upload(key(path), data, mime),- uploadFile: (path, filePath, mime, sha256) => backend.uploadFile(key(path), filePath, mime, sha256),- download: path => backend.download(key(path)),- delete: path => backend.delete(key(path)),- exists: path => backend.exists(key(path)),- list: async listPrefix => (await backend.list(key(listPrefix))).map(path => path.slice(prefix.length + 1)),- getUrl: path => backend.getUrl(key(path)),- ...(backend.hashObject ? { hashObject: (path: string) => backend.hashObject!(key(path)) } : {}),+ upload: async (path, data, mime) => backend.upload(key(path), data, mime),+ uploadFile: async (path, filePath, mime, sha256) => backend.uploadFile(key(path), filePath, mime, sha256),+ download: async path => backend.download(key(path)),+ delete: async path => backend.delete(key(path)),+ exists: async path => backend.exists(key(path)),+ list: async listPrefix => (await backend.list(key(listPrefix, true))).map(path => path.slice(prefix.length + 1)),+ getUrl: async path => backend.getUrl(key(path)),+ ...(backend.hashObject ? { hashObject: async (path: string) => backend.hashObject!(key(path)) } : {}),};}diff --git a/test/embedded-file-store.test.ts b/test/embedded-file-store.test.tsindex 2b1fdb347a..524a3fba40 100644--- a/test/embedded-file-store.test.ts+++ b/test/embedded-file-store.test.ts@@ -153,6 +153,25 @@ describe('settings.storage', () => {});});+describe('prefixedStorage', () => {+ test("refuses a key that would leave its prefix, so one brain never reaches another brain's objects", async () => {+ const { LocalStorage } = await import('../src/core/storage/local.ts');+ const { prefixedStorage } = await import('../src/core/storage.ts');+ store('brain-b/notes/x.txt', Buffer.from('brain b bytes'));+ const a = prefixedStorage(new LocalStorage(storageRoot), 'brain-a');+ for (const path of ['../brain-b/notes/x.txt', 'notes/../../brain-b/notes/x.txt', './notes/x.txt', 'notes//x.txt', '/notes/x.txt', '', 'notes/x.txt/']) {+ for (const call of [() => a.exists(path), () => a.download(path), () => a.delete(path), () => a.upload(path, Buffer.from('overwrite')), () => a.getUrl(path)]) {+ expect([path, (await failure(call))?.message]).toEqual([path, `Path traversal blocked: ${path} resolves outside storage prefix brain-a`]);+ }+ }+ expect((await failure(() => a.list('../brain-b')))?.message).toBe('Path traversal blocked: ../brain-b resolves outside storage prefix brain-a');+ expect(readFileSync(join(storageRoot, 'brain-b/notes/x.txt'), 'utf8')).toBe('brain b bytes');+ await a.upload('notes/x.txt', Buffer.from('brain a bytes'));+ expect([await a.exists('notes/x.txt'), readFileSync(join(storageRoot, 'brain-a/notes/x.txt'), 'utf8')]).toEqual([true, 'brain a bytes']);+ expect(await a.list('notes')).toEqual(['notes/x.txt']);+ });+});+describe('hashing a stored object', () => {test('S3 streams the object body into its size and SHA-256', async () => {const { S3Storage } = await import('../src/core/storage/s3.ts');