Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix: pre-landing review fixes
garrytanba2de3bd8def
diff --git a/docs/guides/error-codes.md b/docs/guides/error-codes.mdindex 22a33253b0..8d14a36ac3 100644--- a/docs/guides/error-codes.md+++ b/docs/guides/error-codes.md@@ -5476,9 +5476,9 @@ More: [docs/guides/write-refusals.md#storage-manifest-refusals](../../docs/guide| Meaning | Why | Next step | Who acts · consent | Verify | Exit | Retryable ||---|---|---|---|---|---|---|-| A storage card field looks like a secret or has a credential shape; nothing was written and the value is not echoed. | Cards are read by agents and copied into the stores; credentials stay in the environment of the skill that restores an item. | Remove the secret, environment assignment, credential variable, URL password or signed-URL parameter from the named field. | agent | `repeat the read that failed` | 1 | no |+| A storage card field looks like a secret or has a credential shape; nothing was written and the value is not echoed. | Cards are read by agents and copied into the stores; credentials stay in the environment of the skill that restores an item. | Remove the secret, environment assignment, credential variable, URL password, signed-URL parameter or credential-named field from the card. | agent | `repeat the read that failed` | 1 | no |-Reasons: `secret_scanner`, `env_assignment`, `credential_variable`, `url_userinfo`, `presigned_url`.+Reasons: `secret_scanner`, `env_assignment`, `credential_variable`, `url_userinfo`, `presigned_url`, `credential_key`.More: [docs/guides/write-refusals.md#storage-manifest-refusals](../../docs/guides/write-refusals.md#storage-manifest-refusals)diff --git a/docs/guides/storage-manifest.md b/docs/guides/storage-manifest.mdindex 9c73ffbdad..78747370c3 100644--- a/docs/guides/storage-manifest.md+++ b/docs/guides/storage-manifest.md@@ -56,8 +56,8 @@ Agents that archived before the shelf existed often kept interim pages: `type: s2. The printed apply, `--apply --expect <plan_hash>` with one `--drop-field <dotted.name>` per refused field, runsafter the user agrees to leave those fields out. It registers each card with `origin: adopt` through the samepath `storage-register` uses, then tags the original `storage-manifest-adopted`. It deletes nothing. A- credential-shaped value (a secret, an environment assignment, a credential variable, a URL password, a signed URL)- always refuses, whatever is dropped: fix the source page first.+ credential-shaped value (a secret, an environment assignment, a credential variable, a URL password, a signed URL,+ a value under a credential-named key such as `restore.args.password`) always refuses, whatever is dropped: fix the source page first.3. `gbrain storage adopt --finalize` previews soft-deleting the tagged originals; its `--expect <plan_hash>` run deletesthem (restorable for 72 hours with `gbrain restore`), only when every adopted card is on the shelf and matches itsoriginal's mapped card. A restore recorded with `storage-update` since adoption is not a difference. The previewdiff --git a/src/core/engine-sql/sources.ts b/src/core/engine-sql/sources.tsindex d81d904675..fe03be345b 100644--- a/src/core/engine-sql/sources.ts+++ b/src/core/engine-sql/sources.ts@@ -20,14 +20,15 @@ export async function listAllSources(exec: LegacyUnscopedRead, opts?: {// how many archived rows exist. ORDER BY (id='default') DESC, id// matches sources-ops.listSources for operator-output stability.// localPathOnly skips pure-DB sources so autopilot fan-out doesn't- // dispatch jobs that would fall back to the global sync.repo_path.+ // dispatch jobs that would fall back to the global sync.repo_path, and+ // the storage shelf, whose cycle is always skipped (GST-1943).const includeArchived = opts?.includeArchived === true;const localPathOnly = opts?.localPathOnly === true;const { rows } = await exec.run(sqlFragment`SELECT id, name, local_path, last_sync_at, configFROM sourcesWHERE (${includeArchived} OR archived IS NOT TRUE)- AND (${!localPathOnly} OR local_path IS NOT NULL)+ AND (${!localPathOnly} OR (local_path IS NOT NULL AND config->>'role' IS DISTINCT FROM 'storage_manifest'))ORDER BY (id = 'default') DESC, id`);return rows.map((r) => ({diff --git a/src/core/error-registry.ts b/src/core/error-registry.tsindex 9d0f6f440f..b18ae934bf 100644--- a/src/core/error-registry.ts+++ b/src/core/error-registry.ts@@ -844,7 +844,7 @@ export const CODES = {storage_card_exists: { class: 'caller', summary: "A storage card with this id is already on the storage shelf; nothing was written.", why: 'Registering never overwrites a card, so two archives cannot silently share one id; a change to an existing card names the revision it read.', docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Change the card with gbrain storage-update <id> --expected-revision <revision from storage_get>, or register the new item under another id.' },storage_manifest_missing: { class: 'consent', actor: 'user', summary: "This brain has no storage shelf to write to (or the storage source ID belongs to another source); nothing was written.", why: 'On a brain with a Git checkout the shelf is a hosted git-canonical repository, created only after the user previews it, and hosting needs managed persistence. A repo-less brain gets its database-only shelf on the first register.', docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Preview gbrain storage init --dir <absolute path> --dry-run and apply the command it prints after the user agrees.' },storage_record_invalid: { class: 'caller', summary: "A storage card failed validation (a missing or malformed field, a command field, or a field gbrain stamps itself); nothing was written.", why: 'Every card must say where the item is, how big it is and how to restore it, and carries no commands, so a card is complete and safe to act on when it is found months later.', reasons: ['schema', 'command_field', 'stamped_field', 'not_json', 'not_object', 'too_large', 'id_change', 'no_record', 'duplicate_id'], docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Fix the field the message names and send the card again; gbrain storage-register --help lists the card fields.' },- storage_record_secret: { class: 'caller', summary: "A storage card field looks like a secret or has a credential shape; nothing was written and the value is not echoed.", why: 'Cards are read by agents and copied into the stores; credentials stay in the environment of the skill that restores an item.', reasons: ['secret_scanner', 'env_assignment', 'credential_variable', 'url_userinfo', 'presigned_url'], docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Remove the secret, environment assignment, credential variable, URL password or signed-URL parameter from the named field.' },+ storage_record_secret: { class: 'caller', summary: "A storage card field looks like a secret or has a credential shape; nothing was written and the value is not echoed.", why: 'Cards are read by agents and copied into the stores; credentials stay in the environment of the skill that restores an item.', reasons: ['secret_scanner', 'env_assignment', 'credential_variable', 'url_userinfo', 'presigned_url', 'credential_key'], docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Remove the secret, environment assignment, credential variable, URL password, signed-URL parameter or credential-named field from the card.' },storage_revision_conflict: { class: 'caller', summary: "A storage card changed since the revision the update named was read; nothing was written.", why: 'Updates bind to the revision they read, so two writers never silently overwrite each other; the refusal carries the current revision in detail.', docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Read the card again with storage_get, apply the change to what it says now, and update once with its revision.' },storage_source_managed: { class: 'caller', summary: "A page write targeted the storage shelf outside the storage verbs; nothing was written.", why: 'The storage source holds only validated cards, so put_page, remember and the other page writers are refused there.', docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Register cards with gbrain storage-register and change them with gbrain storage-update.' },storage_error: { class: 'server', summary: "Reading or writing durable storage failed.",diff --git a/src/core/ops/storage-manifest.ts b/src/core/ops/storage-manifest.tsindex ff83b6cdbb..c57a7cbad4 100644--- a/src/core/ops/storage-manifest.ts+++ b/src/core/ops/storage-manifest.ts@@ -91,6 +91,10 @@ async function writeCard(ctx: OperationContext, card: Card, opts: { expectedRevi...(opts.requestId ? { request_id: opts.requestId } : {}) } });return response;} catch (error) {+ if (error instanceof OperationError && error.code === 'revision_required' && !opts.expectedRevision) {+ const current = await ctx.engine.readPageSnapshot(card.id, { sourceId: STORAGE_SOURCE_ID });+ throw cardExists(card.id, current?.revision ?? 'unknown');+ }if (error instanceof OperationError && error.code === 'revision_conflict') {const current = await ctx.engine.readPageSnapshot(card.id, { sourceId: STORAGE_SOURCE_ID });throw opError('storage_revision_conflict', `Card ${card.id} changed since revision ${opts.expectedRevision} was read; nothing was written.`,@@ -101,13 +105,17 @@ async function writeCard(ctx: OperationContext, card: Card, opts: { expectedRevi}}+function cardExists(id: string, revision: string): OperationError {+ return opError('storage_card_exists', `A card with id ${id} is already on the storage shelf; nothing was written.`,+ `Change it with gbrain storage-update ${id} --expected-revision ${revision}, or register the new item under another id.`,+ { fix: getFix(id), detail: revision });+}+/** Register a new card. Exported for `storage adopt` and `storage import` (origin adopt / import). */export async function registerCard(ctx: OperationContext, input: CardInput, origin: (typeof ORIGINS)[number], requestId?: string): Promise<Record<string, unknown>> {await ensureStorageSource(ctx.engine);const existing = await ctx.engine.readPageSnapshot(input.id, { sourceId: STORAGE_SOURCE_ID });- if (existing) throw opError('storage_card_exists', `A card with id ${input.id} is already on the storage shelf; nothing was written.`,- `Change it with gbrain storage-update ${input.id} --expected-revision ${existing.revision}, or register the new item under another id.`,- { fix: getFix(input.id), detail: existing.revision });+ if (existing) throw cardExists(input.id, existing.revision);const card: Card = { ...input, registered_by: await principalLabel(ctx), origin };const response = await writeCard(ctx, card, { requestId });return { id: card.id, source_id: STORAGE_SOURCE_ID, origin, write: response };diff --git a/src/core/storage-manifest/adopt.ts b/src/core/storage-manifest/adopt.tsindex 37477230de..7360190d95 100644--- a/src/core/storage-manifest/adopt.ts+++ b/src/core/storage-manifest/adopt.ts@@ -30,7 +30,7 @@ import { boundedBodyPrefixSql } from '../page-window/bounded-lead.ts';import { BOUNDED_PREFIX_BYTES } from '../page-window/document.ts';import { submitPageMutation } from '../persistence/page-mutations.ts';import { readShelfCards } from './find.ts';-import { CARD_KEYS, cardContent, cardDigest, credentialShape, isCommandKey, parseCardInput, STAMPED_KEYS, STORED_OBJECT_TYPE, type CardInput } from './schema.ts';+import { CARD_KEYS, cardContent, cardDigest, credentialShape, holdsCredential, isCommandKey, parseCardInput, STAMPED_KEYS, STORED_OBJECT_TYPE, type CardInput } from './schema.ts';import { shelfWritable, STORAGE_SOURCE_ID } from './source.ts';export const ADOPT_DEFAULT_PREFIX = 'storage-manifest/';@@ -136,6 +136,7 @@ function credentialAt(value: unknown, path: string): { field: string; shape: strfor (const [key, item] of items) {const keyShape = Array.isArray(value) ? null : credentialShape(key);if (keyShape) return { field: `${path || '(record)'} (a key)`, shape: keyShape };+ if (!Array.isArray(value) && holdsCredential(key, item)) return { field: path ? `${path}.${key}` : key, shape: 'credential_key' };const found = credentialAt(item, path ? `${path}.${key}` : key);if (found) return found;}diff --git a/src/core/storage-manifest/schema.ts b/src/core/storage-manifest/schema.tsindex 66a4940a7e..a39eea539f 100644--- a/src/core/storage-manifest/schema.ts+++ b/src/core/storage-manifest/schema.ts@@ -88,6 +88,8 @@ export type Card = CardInput & { registered_by: string; origin: (typeof ORIGINS)const COMMAND_KEYS = new Set(['command', 'cmd', 'script', 'exec', 'shell', 'instructions']);export const STAMPED_KEYS = ['registered_by', 'origin'] as const;const CREDENTIAL_NAME = /(?:KEY|SECRET|TOKEN|PASSWORD|PASSWD|PASS|CRED|AUTH)/i;+const CREDENTIAL_KEY = /(?:^|_)(?:password|passwd|passphrase|secret|token|credentials?|(?:api|access|secret|application|account|private|auth|master|encryption)_?key)(?:_|$)/;+const REFERENCE_KEY = /_(?:env|var|variable|file|path|name|ref|id)$/;const ENV_ASSIGNMENT = /(?:^|[\s;&|(])[A-Za-z_][A-Za-z0-9_]{1,}=\S/;const VAR_REFERENCE = /\$\{?([A-Za-z_][A-Za-z0-9_]*)/g;const URL_USERINFO = /[a-z][a-z0-9+.-]*:\/\/[^/\s@]*:[^/\s@]*@/i;@@ -106,6 +108,7 @@ function walk(value: unknown, path: string, visit: (path: string, text: string)for (const [key, item] of Object.entries(value)) {const at = path ? `${path}.${key}` : key;if (isCommandKey(key)) throw invalid(at, 'a card carries no commands; restore runs through restore.skill and restore.args', 'command_field');+ if (holdsCredential(key, item)) throw credentialKeyRefusal(at);visit(`${path || '(card)'} (a key)`, key);walk(item, at, visit);}@@ -128,6 +131,19 @@ export function isCommandKey(key: string): boolean {return COMMAND_KEYS.has(key.toLowerCase());}+/** A non-empty string or number under a key named for a credential (password, application_key), unless the key names where it lives (password_file, token_env). */+export function holdsCredential(key: string, value: unknown): boolean {+ if (!(typeof value === 'number' || (typeof value === 'string' && value.trim() !== ''))) return false;+ const name = key.replace(/([a-z0-9])([A-Z])/g, '$1_$2').replace(/-/g, '_').toLowerCase();+ return CREDENTIAL_KEY.test(name) && !REFERENCE_KEY.test(name);+}++function credentialKeyRefusal(path: string): OperationError {+ return opError('storage_record_secret', `Storage card field ${path} is named for a credential and holds a value; nothing was written.`,+ 'Remove that field. Cards name where an item is and how to restore it; the restore skill reads credentials from its own environment (name the variable, e.g. token_env, instead).',+ { reason: 'credential_key' });+}+/** Throws storage_record_secret without echoing the value. */function assertNoCredential(path: string, value: string): void {const shape = credentialShape(value);diff --git a/test/storage-manifest-adopt.test.ts b/test/storage-manifest-adopt.test.tsindex e10f39fcf3..89a275f26c 100644--- a/test/storage-manifest-adopt.test.ts+++ b/test/storage-manifest-adopt.test.ts@@ -139,6 +139,11 @@ describe('storage adopt on a repo-less brain', () => {const keyed = (await previewAdopt(ctx(), opts)).entries.find((e) => e.slug === 'storage-manifest/dataset-k')!;expect(keyed).toMatchObject({ state: 'blocked', refused: [], problem: { code: 'storage_record_secret', reason: 'secret_scanner' } });expect(JSON.stringify(await previewAdopt(ctx(), opts))).not.toContain('AKIAIOSFODNN7EXAMPLE');++ await interim('storage-manifest/dataset-p', record('dataset-p', { details: { password: 'fake-pass-value' } }));+ const named = (await previewAdopt(ctx(), opts)).entries.find((e) => e.slug === 'storage-manifest/dataset-p')!;+ expect(named).toMatchObject({ state: 'blocked', card: null, problem: { code: 'storage_record_secret', reason: 'credential_key' } });+ expect(JSON.stringify(named)).not.toContain('fake-pass-value');}, 60_000);test('an unreadable record and a duplicate card id block the apply', async () => {diff --git a/test/storage-manifest.test.ts b/test/storage-manifest.test.tsindex b0a85a6a79..00d3721386 100644--- a/test/storage-manifest.test.ts+++ b/test/storage-manifest.test.ts@@ -5,8 +5,8 @@* - storage_find ranks exact id, then exact alias, then most of the phrase's words; a miss says not_found and lists* the closest cards; storage_list filters by store and status;* - validation refuses a missing field, a command field, a stamped field, a scanner secret (in a value or an object- * key) and every credential shape (env assignment, credential variable, URL userinfo, presigned URL), without- * echoing the value;+ * key) and every credential shape (env assignment, credential variable, URL userinfo, presigned URL, a value under+ * a credential-named key), without echoing the value; a key naming where a credential lives (token_env) passes;* - storage_update keeps the page's access frontmatter (visibility, readable_by) when it rewrites the card;* - the write guard: put_page, remember and forget_fact into the shelf are refused storage_source_managed, a forged* parameter does not open it, and the storage verbs still write;@@ -20,7 +20,9 @@* when its grant names the storage source (an ungranted bound client and an unbound stdio connection are refused);* - derived writers skip the shelf: a dream cycle scoped to the storage source runs no phase, and enrichment* considers no card there (no model call, no page write);- * - a hosted shelf never becomes the sole non-default source that bare writes route to;+ * - a hosted shelf never becomes the sole non-default source that bare writes route to, and stays out of the+ * local-path source list autopilot fans out over and cycle_freshness reads (its cycle is always skipped);+ * - two concurrent registers of one id land once; the other is refused storage_card_exists;* - a fresh keyless brain, whose default source sits in the gbrain-owned content directory init creates (no Git),* counts as repo-less: the first register creates the shelf; a Git checkout there needs storage init.*/@@ -131,14 +133,18 @@ describe('storage shelf on a repo-less brain', () => {[card('a-8', { locations: [{ store: 's3', path: 'https://user:@example.invalid/x', tool: 'object', size_bytes: 1 }] }), 'storage_record_secret', 'url_userinfo'],[card('a-9', { locations: [{ store: 's3', path: 'https://example.invalid/x?X-Amz-Signature=abc', tool: 'object', size_bytes: 1 }] }), 'storage_record_secret', 'presigned_url'],[card('a-10', { details: { AKIAIOSFODNN7EXAMPLE: 'b2 key' } }), 'storage_record_secret', 'secret_scanner'],+ [card('a-11', { restore: { skill: 'restore-data', args: { id: 'a-11', applicationKey: 'K005abcdef123ghijkl456mnopqr789' } } }), 'storage_record_secret', 'credential_key'],+ [card('a-12', { details: { restic_password: 'abcdef123-correct-horse' } }), 'storage_record_secret', 'credential_key'],];for (const [input, code, reason] of cases) {const r = await refusal(call('storage_register', { card: input }));expect({ id: input.id, code: r?.code, reason: r?.reason }).toEqual({ id: input.id, code, reason });expect(r!.message).not.toContain('abcdef123');expect(r!.message).not.toContain('AKIAIOSFODNN7EXAMPLE');+ expect(r!.message).not.toContain('K005abcdef123');}expect(await engine.executeRaw("SELECT 1 FROM pages WHERE source_id='storage'")).toEqual([]);+ await call('storage_register', { card: card('a-13', { restore: { skill: 'restore-data', args: { id: 'a-13', key: 'backups/a-13.tar', password_file: '/etc/restic/pass', token_env: 'B2_APPLICATION_KEY' } } }) });}, 30_000);test('the write guard refuses put_page and remember into the shelf, forged parameters included', async () => {@@ -209,6 +215,9 @@ describe('storage shelf on a repo-less brain', () => {test('register never overwrites; update binds to its revision and records restores newest first', async () => {await call('storage_register', { card: card('item-b') });expect((await refusal(call('storage_register', { card: card('item-b') })))?.code).toBe('storage_card_exists');+ const raced = await Promise.allSettled([call('storage_register', { card: card('item-r', { title: 'First' }) }), call('storage_register', { card: card('item-r', { title: 'Second' }) })]);+ const lost = raced.find((r) => r.status === 'rejected') as PromiseRejectedResult | undefined;+ expect({ landed: raced.filter((r) => r.status === 'fulfilled').length, code: lost?.reason?.code }).toEqual({ landed: 1, code: 'storage_card_exists' });const first = await call('storage_get', { id: 'item-b' });await call('storage_update', { id: 'item-b', expected_revision: first.revision, restored_to: '/scratch/item-b', restored_verified: true });const stale = await refusal(call('storage_update', { id: 'item-b', expected_revision: first.revision, patch: { status: 'retired' } }));@@ -270,6 +279,8 @@ describe('storage shelf on a repo-less brain', () => {expect(await resolveSourceWithTier(engine, null, tmpdir())).toMatchObject({ source_id: 'default', tier: 'seed_default' });await engine.executeRaw("INSERT INTO sources (id, name, local_path, config) VALUES ('vault-a', 'vault-a', $1, '{}'::jsonb)", [join(home, 'vault-a')]);expect(await resolveSourceWithTier(engine, null, tmpdir())).toMatchObject({ source_id: 'vault-a', tier: 'sole_non_default' });+ expect((await engine.listAllSources({ localPathOnly: true })).map((s) => s.id)).toEqual(['vault-a']);+ expect((await engine.listAllSources()).map((s) => s.id)).toContain('storage');}, 30_000);test('a brain with a Git checkout needs storage init first, and init needs managed persistence', async () => {