Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix: refuse more on the storage shelf: credential-named keys and remote-written records
garrytanf60d3d827947
diff --git a/changelog.d/capy-storage-manifest-adopt.md b/changelog.d/capy-storage-manifest-adopt.mdindex c3057e0738..2c14a61db0 100644--- a/changelog.d/capy-storage-manifest-adopt.md+++ b/changelog.d/capy-storage-manifest-adopt.md@@ -1,16 +1,16 @@**A storage manifest: agents can register what they archived to object storage and find it again later.** A new `storage` source holds one validated card per archived item: what it is, where every copy lives (store, bucket, path, restic repository and snapshot, size, file count, hash and its kind, when it was verified), where it came from and which skill restores it. `gbrain storage-register` adds a card (from stdin or `--card`) and never overwrites one; `gbrain storage-update <id> --expected-revision <rev>` patches a card bound to the revision it read, keeps the page's access labels and records restores newest first; `gbrain storage-find "<words>"` ranks the exact id, then an exact alias, then the cards holding most of the words, and a miss lists the closest cards; `gbrain storage-get` and `gbrain storage-list` read. On a brain with a Git checkout, `gbrain storage init` hosts the shelf in its own repository first (preview, then `--expect <plan_hash>`; it needs managed persistence), so each card is a commit before it is indexed. A repo-less brain gets a database-only shelf on the first register.-Cards never carry credentials or commands. A card is refused when any field looks like a secret, an environment assignment, a credential variable, a URL with user info or a presigned URL, or when any key is named `command`, `cmd`, `script`, `exec`, `shell` or `instructions`; object keys are checked too, a value under a key whose last word names a credential (`password`, `restic_password`, `application_key`, `access_token`) refuses while any other key passes (`token_env`, `password_file`, `token_count`, `api_key_last4`), and the refusal never echoes the value. A first shelf creation that stopped part way is finished by the next register or `storage init`, and an archived shelf refuses until it is restored. A restore is `{ skill, args }` with flat scalar arguments. gbrain stamps `registered_by` and `origin` itself, and holds no store credentials.+Cards never carry credentials or commands. A card is refused when any field looks like a secret, an environment assignment, a credential variable, a URL with user info or a presigned URL, or when any key is named `command`, `cmd`, `script`, `exec`, `shell` or `instructions`; object keys are checked too, a value under a key named for a credential refuses wherever the word sits (`password`, `restic_password`, `token_value`, `token_count`, `application_key`) unless the key ends in a suffix that names where something lives (`_env`, `_file`, `_path`, `_name`, `_ref`, `_id`, `_var`, `_variable`), and the refusal says to rename a key that holds no credential that way, and the refusal never echoes the value. A first shelf creation that stopped part way is finished by the next register or `storage init`, and an archived shelf refuses until it is restored. A restore is `{ skill, args }` with flat scalar arguments. gbrain stamps `registered_by` and `origin` itself, and holds no store credentials.The shelf is written only through the storage verbs. `put_page`, `remember` and every other page writer are refused `storage_source_managed` there, a request parameter cannot open the door, register and update run only from the trusted local CLI, and a dream cycle or enrichment run scoped to the shelf skips it. The source is not federated, so a remote connection reads cards only when its grant names `storage`.-**Agents find the shelf, and older archive records move onto it.** The brain-first lookup table every harness reads gains one row, "Find something archived to B2, Supabase or S3" (`gbrain storage-find` / `storage_find`), and a new `storage-manifest` skill says when to register, to find before restoring, to restore through the agent's own restore skill with the card's `restore.skill` and `restore.args`, and to record the restore with `gbrain storage-update`. `gbrain storage adopt` moves interim records (`stored-object` pages under `storage-manifest/` holding their record as JSON under `## Record`) onto the shelf: the preview shows each mapped card and every field a card refuses with its reason; `--apply --expect <plan_hash>` registers each card with `origin: adopt` and tags the original `storage-manifest-adopted`, deleting nothing; each card takes its original's access labels (`visibility`, `access`, `confidentiality`, `readable_by`, with `readable_by` narrowed to the readers of every folder rule over the original), so a card is never read wider than its original, and an original whose readers and folder rules share no reader, or whose rules are still being applied, is blocked; and refuses until each refused field is named with `--drop-field` (a credential-shaped value always refuses); `--finalize` previews, then soft-deletes the adopted originals only when every card matches its original (a restore recorded since is not a difference); the preview names any other section an original holds and flags one longer than the 64 KiB it reads, and an original edited after the preview, even just before its delete, refuses. `gbrain storage export --jsonl` and `gbrain storage import --jsonl <file>` (preview per line, then `--apply --expect`) rebuild a shelf from a file, registering missing cards with `origin: import` and never overwriting a differing card; each line carries its card's access labels under `labels` and import restores them (a card on the shelf with other labels is a conflict); export leaves out and names a stored card that fails the card rules. Applies are idempotent and report `side_effects` and, when a run stops part way, the command that finishes it; a write gbrain accepted that has not committed yet is listed under `pending` with its receipt and the `gbrain write-request <id>` poll (exit 10), never reported as no change. The new doctor check `storage_manifest` warns about cards that fail the card schema, interim records not adopted yet and a hosted shelf whose directory is gone. On a brain with a Git checkout, adopt and import wait for `gbrain storage init`.+**Agents find the shelf, and older archive records move onto it.** The brain-first lookup table every harness reads gains one row, "Find something archived to B2, Supabase or S3" (`gbrain storage-find` / `storage_find`), and a new `storage-manifest` skill says when to register, to find before restoring, to restore through the agent's own restore skill with the card's `restore.skill` and `restore.args`, and to record the restore with `gbrain storage-update`. `gbrain storage adopt` moves interim records (`stored-object` pages under `storage-manifest/` holding their record as JSON under `## Record`) onto the shelf: the preview shows each mapped card and every field a card refuses with its reason; `--apply --expect <plan_hash>` registers each card with `origin: adopt` and tags the original `storage-manifest-adopted`, deleting nothing; a record whose page a remote (MCP) client last wrote is blocked and never registered, so only records written on this machine or by sync become cards; each card takes its original's access labels (`visibility`, `access`, `confidentiality`, `readable_by`, with `readable_by` narrowed to the readers of every folder rule over the original), so a card is never read wider than its original, and an original whose readers and folder rules share no reader, or whose rules are still being applied, is blocked; and refuses until each refused field is named with `--drop-field` (a credential-shaped value always refuses); `--finalize` previews, then soft-deletes the adopted originals only when every card matches its original (a restore recorded since is not a difference); the preview names any other section an original holds and flags one longer than the 64 KiB it reads, and an original edited after the preview, even just before its delete, refuses. `gbrain storage export --jsonl` and `gbrain storage import --jsonl <file>` (preview per line, then `--apply --expect`) rebuild a shelf from a file, registering missing cards with `origin: import` and never overwriting a differing card; each line carries its card's access labels under `labels` and import restores them (a card on the shelf with other labels is a conflict); export leaves out and names a stored card that fails the card rules. Applies are idempotent and report `side_effects` and, when a run stops part way, the command that finishes it; a write gbrain accepted that has not committed yet is listed under `pending` with its receipt and the `gbrain write-request <id>` poll (exit 10), never reported as no change. The new doctor check `storage_manifest` warns about cards that fail the card schema, interim records not adopted yet and a hosted shelf whose directory is gone. On a brain with a Git checkout, adopt and import wait for `gbrain storage init`.### Itemized changes- `src/core/storage-manifest/adopt.ts`, `src/core/storage-manifest/jsonl.ts`, `src/commands/storage.ts`: `gbrain storage adopt` (preview, apply, finalize) and `gbrain storage export|import --jsonl`, each preview-first and bound to its plan_hash (a stale one refuses `preview_changed`). `schema.ts` exports the credential-shape rules, the command-key test and the card keys adopt maps onto, plus `cardContent`/`cardDigest` for comparing copies; `source.ts` gains `shelfWritable` (the refusal a storage write would get, without creating anything); `find.ts` gains `readShelfCards`.- `src/commands/doctor/checks/storage-manifest.ts`, `src/commands/doctor/registry.ts`, `src/core/doctor-categories.ts`: the read-only `storage_manifest` check (brain category), `not_applicable` with no shelf and no interim record. Doctor goldens regenerated for the new check.-- `src/core/error-registry.ts`, `docs/guides/write-refusals.md`, `docs/guides/error-codes.md`: `storage_adopt_field_refused` (consent: the user agrees to leave refused fields out) and `storage_adopt_unverified`; `storage_record_invalid` gains reasons `no_record`, `duplicate_id`, `access_unpreservable`, `access_pending` and `labels`.+- `src/core/error-registry.ts`, `docs/guides/write-refusals.md`, `docs/guides/error-codes.md`: `storage_adopt_field_refused` (consent: the user agrees to leave refused fields out) and `storage_adopt_unverified`; `storage_record_invalid` gains reasons `no_record`, `duplicate_id`, `access_unpreservable`, `access_pending`, `labels` and `remote_written`.- `skills/conventions/brain-first.md` (and its generated copies in `AGENTS.md`, `INSTALL_FOR_AGENTS.md`, `skills/RESOLVER.md`, the bootstrap template and the plugin trees), `skills/storage-manifest/SKILL.md`, `skills/RESOLVER.md`, `skills/manifest.json`, `openclaw.plugin.json`: the lookup row and the skill.- `docs/guides/storage-manifest.md`: the guide (verbs, repo-less and Git-checkout brains, adopt, export and import, doctor, privacy).- Tests: `test/storage-manifest-adopt.test.ts`, `test/storage-manifest-jsonl.test.ts` and `test/doctor-storage-manifest.test.ts` (PGLite, repo-less) and `test/storage-manifest-adopt-hosted.test.ts` (managed PGLite with a Git checkout).diff --git a/docs/guides/error-codes.md b/docs/guides/error-codes.mdindex 56755ed734..ccb4af6862 100644--- a/docs/guides/error-codes.md+++ b/docs/guides/error-codes.md@@ -5466,7 +5466,7 @@ More: [docs/guides/write-refusals.md#storage-manifest-refusals](../../docs/guide|---|---|---|---|---|---|---|| A storage card failed validation (a missing or malformed field, a command field, or a field gbrain stamps itself); nothing was written. | Every card must say where the item is, how big it is and how to restore it, and carries no commands, so a card is complete and safe to act on when it is found months later. | Fix the field the message names and send the card again; gbrain storage-register --help lists the card fields. For an adopt blocked on access_pending or access_unpreservable, settle the access rules over the interim page (gbrain access status) or give it readers its folder rules also list, then preview again. | agent | `repeat the read that failed` | 1 | no |-Reasons: `schema`, `command_field`, `stamped_field`, `not_json`, `not_object`, `too_large`, `id_change`, `no_record`, `duplicate_id`, `access_unpreservable`, `access_pending`, `labels`.+Reasons: `schema`, `command_field`, `stamped_field`, `not_json`, `not_object`, `too_large`, `id_change`, `no_record`, `duplicate_id`, `access_unpreservable`, `access_pending`, `labels`, `remote_written`.More: [docs/guides/write-refusals.md#storage-manifest-refusals](../../docs/guides/write-refusals.md#storage-manifest-refusals)@@ -5476,7 +5476,7 @@ More: [docs/guides/write-refusals.md#storage-manifest-refusals](../../docs/guide| Meaning | Why | Next step | Who acts · consent | Verify | Exit | Retryable ||---|---|---|---|---|---|---|-| A storage card field looks like a secret or has a credential shape; nothing was written and the value is not echoed. | Cards are read by agents and copied into the stores; credentials stay in the environment of the skill that restores an item. | Remove the secret, environment assignment, credential variable, URL password, signed-URL parameter or credential-named field from the card. | agent | `repeat the read that failed` | 1 | no |+| A storage card field looks like a secret or has a credential shape; nothing was written and the value is not echoed. | Cards are read by agents and copied into the stores; credentials stay in the environment of the skill that restores an item. | Remove the secret, environment assignment, credential variable, URL password, signed-URL parameter or credential-named field from the card; a credential-named field that holds no credential can be renamed to end in _id, _ref or another suffix the rule passes. | agent | `repeat the read that failed` | 1 | no |Reasons: `secret_scanner`, `env_assignment`, `credential_variable`, `url_userinfo`, `presigned_url`, `credential_key`.diff --git a/docs/guides/storage-manifest.md b/docs/guides/storage-manifest.mdindex d92626607e..ffbf411554 100644--- a/docs/guides/storage-manifest.md+++ b/docs/guides/storage-manifest.md@@ -57,8 +57,11 @@ Agents that archived before the shelf existed often kept interim pages: `type: safter the user agrees to leave those fields out. It registers each card with `origin: adopt` through the samepath `storage-register` uses, then tags the original `storage-manifest-adopted`. It deletes nothing. Acredential-shaped value (a secret, an environment assignment, a credential variable, a URL password, a signed URL,- a value under a key whose last word names a credential, such as `restore.args.password` or `restic_password`) always- refuses, whatever is dropped: fix the source page first. Each card takes its original's access labels (`visibility`,+ a value under a credential-named key such as `restore.args.password`, `restic_password` or `token_count`, unless the+ key ends in `_id`, `_ref`, `_name`, `_path`, `_file`, `_env`, `_var` or `_variable`) always refuses, whatever is+ dropped: fix the source page first, renaming a key that holds no credential to end in one of those suffixes. A+ record whose page a remote (MCP) client last wrote is blocked (`remote_written`) and never becomes a card: check it+ and register the item yourself with `storage-register`. Each card takes its original's access labels (`visibility`,`access`, `confidentiality`, `readable_by`), with `readable_by` narrowed to the readers that every folder rule overthe original also lists (readers match literally, so a member is not matched to its groups), so no card is readwider than its original. An original whose readers and folder rules share no reader (`access_unpreservable`), ordiff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.mdindex 4510f041f7..ae91c8cebf 100644--- a/docs/guides/write-refusals.md+++ b/docs/guides/write-refusals.md@@ -1122,7 +1122,7 @@ An adopt or import apply whose plan changed since its preview refuses `preview_c| Code | What happened | What to do ||---|---|---|| `storage_source_managed` | `put_page`, `remember` or another page writer targeted the shelf. | Register with `gbrain storage-register`, change with `gbrain storage-update`. |-| `storage_record_invalid` | A card field is missing or malformed, names a command (`command`, `script`, `exec` and similar), or is one gbrain sets itself (`registered_by`, `origin`). On adopt, reason `access_unpreservable` means an interim page's `readable_by` and the folder rules over it share no reader, and `access_pending` that a rule over it is still being applied, so no card could carry the same restriction; on import, `labels` means a line's `labels` holds something other than access labels. `storage_get` refuses it too for a stored card that breaks these rules, such as a hand-edited card file. | Fix the field the message names. Restore runs through `restore.skill` and `restore.args`. For `access_pending`, finish the rule change (`gbrain access status`) and preview again; for `access_unpreservable`, give the page readers its folder rules also list. For a stored card, `gbrain doctor --only storage_manifest` lists the invalid cards, and the refusal carries the card's revision for `gbrain storage-update <id> --expected-revision <rev> --patch '<json>'`. |+| `storage_record_invalid` | A card field is missing or malformed, names a command (`command`, `script`, `exec` and similar), or is one gbrain sets itself (`registered_by`, `origin`). On adopt, reason `access_unpreservable` means an interim page's `readable_by` and the folder rules over it share no reader, and `access_pending` that a rule over it is still being applied, so no card could carry the same restriction; on import, `labels` means a line's `labels` holds something other than access labels; `remote_written` means a remote (MCP) client last wrote the interim page, so adopt never registers it (check it and use `gbrain storage-register`). `storage_get` refuses it too for a stored card that breaks these rules, such as a hand-edited card file. | Fix the field the message names. Restore runs through `restore.skill` and `restore.args`. For `access_pending`, finish the rule change (`gbrain access status`) and preview again; for `access_unpreservable`, give the page readers its folder rules also list. For a stored card, `gbrain doctor --only storage_manifest` lists the invalid cards, and the refusal carries the card's revision for `gbrain storage-update <id> --expected-revision <rev> --patch '<json>'`. || `storage_record_secret` | A field looks like a secret or has a credential shape: an environment assignment, a credential-named variable, a URL password or a signed-URL parameter. The value is not echoed. | Remove it; the restore skill reads credentials from its own environment. || `storage_card_exists` | A card with that id is already registered. | Update it with the revision `storage_get` returns, or use another id. || `storage_revision_conflict` | The card changed since its revision was read; `detail` holds the current revision. | Read it again, apply the change, update once. |diff --git a/src/core/error-registry.ts b/src/core/error-registry.tsindex 0bb54e49e0..7733fab145 100644--- a/src/core/error-registry.ts+++ b/src/core/error-registry.ts@@ -843,8 +843,8 @@ export const CODES = {storage_adopt_unverified: { class: 'caller', summary: "An adopted interim original has no card on the storage shelf that matches its record, so finalize deleted nothing.", why: 'Finalize soft-deletes an original only when its card holds everything the record mapped to, so no archive record is lost.', docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Compare the original with its card (gbrain storage-get), fix the card or the record, then preview gbrain storage adopt --finalize again.' },storage_card_exists: { class: 'caller', summary: "A storage card with this id is already on the storage shelf; nothing was written.", why: 'Registering never overwrites a card, so two archives cannot silently share one id; a change to an existing card names the revision it read.', docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Change the card with gbrain storage-update <id> --expected-revision <revision from storage_get>, or register the new item under another id.' },storage_manifest_missing: { class: 'consent', actor: 'user', summary: "This brain has no storage shelf to write to (or the storage source ID belongs to another source); nothing was written.", why: 'On a brain with a Git checkout the shelf is a hosted git-canonical repository, created only after the user previews it, and hosting needs managed persistence. A repo-less brain gets its database-only shelf on the first register.', docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Preview gbrain storage init --dir <absolute path> --dry-run and apply the command it prints after the user agrees.' },- storage_record_invalid: { class: 'caller', summary: "A storage card failed validation (a missing or malformed field, a command field, or a field gbrain stamps itself); nothing was written.", why: 'Every card must say where the item is, how big it is and how to restore it, and carries no commands, so a card is complete and safe to act on when it is found months later.', reasons: ['schema', 'command_field', 'stamped_field', 'not_json', 'not_object', 'too_large', 'id_change', 'no_record', 'duplicate_id', 'access_unpreservable', 'access_pending', 'labels'], docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Fix the field the message names and send the card again; gbrain storage-register --help lists the card fields. For an adopt blocked on access_pending or access_unpreservable, settle the access rules over the interim page (gbrain access status) or give it readers its folder rules also list, then preview again.' },- storage_record_secret: { class: 'caller', summary: "A storage card field looks like a secret or has a credential shape; nothing was written and the value is not echoed.", why: 'Cards are read by agents and copied into the stores; credentials stay in the environment of the skill that restores an item.', reasons: ['secret_scanner', 'env_assignment', 'credential_variable', 'url_userinfo', 'presigned_url', 'credential_key'], docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Remove the secret, environment assignment, credential variable, URL password, signed-URL parameter or credential-named field from the card.' },+ storage_record_invalid: { class: 'caller', summary: "A storage card failed validation (a missing or malformed field, a command field, or a field gbrain stamps itself); nothing was written.", why: 'Every card must say where the item is, how big it is and how to restore it, and carries no commands, so a card is complete and safe to act on when it is found months later.', reasons: ['schema', 'command_field', 'stamped_field', 'not_json', 'not_object', 'too_large', 'id_change', 'no_record', 'duplicate_id', 'access_unpreservable', 'access_pending', 'labels', 'remote_written'], docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Fix the field the message names and send the card again; gbrain storage-register --help lists the card fields. For an adopt blocked on access_pending or access_unpreservable, settle the access rules over the interim page (gbrain access status) or give it readers its folder rules also list, then preview again.' },+ storage_record_secret: { class: 'caller', summary: "A storage card field looks like a secret or has a credential shape; nothing was written and the value is not echoed.", why: 'Cards are read by agents and copied into the stores; credentials stay in the environment of the skill that restores an item.', reasons: ['secret_scanner', 'env_assignment', 'credential_variable', 'url_userinfo', 'presigned_url', 'credential_key'], docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Remove the secret, environment assignment, credential variable, URL password, signed-URL parameter or credential-named field from the card; a credential-named field that holds no credential can be renamed to end in _id, _ref or another suffix the rule passes.' },storage_revision_conflict: { class: 'caller', summary: "A storage card changed since the revision the update named was read; nothing was written.", why: 'Updates bind to the revision they read, so two writers never silently overwrite each other; the refusal carries the current revision in detail.', docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Read the card again with storage_get, apply the change to what it says now, and update once with its revision.' },storage_source_managed: { class: 'caller', summary: "A page write targeted the storage shelf outside the storage verbs; nothing was written.", why: 'The storage source holds only validated cards, so put_page, remember and the other page writers are refused there.', docs: 'docs/guides/write-refusals.md#storage-manifest-refusals', suggestion: 'Register cards with gbrain storage-register and change them with gbrain storage-update.' },storage_error: { class: 'server', summary: "Reading or writing durable storage failed.",diff --git a/src/core/storage-manifest/adopt.ts b/src/core/storage-manifest/adopt.tsindex b8a6f48848..083b33d2e2 100644--- a/src/core/storage-manifest/adopt.ts+++ b/src/core/storage-manifest/adopt.ts@@ -78,11 +78,11 @@ export interface FinalizeApplyResult {deleted: string[]; skipped: string[]; failure?: StepFailure; pending?: PendingWrite[]; rerun?: string[];}-interface InterimAccess { folder_rule_ids?: string[] | null; folder_readers?: unknown; access_pending?: boolean }+interface InterimAccess { folder_rule_ids?: string[] | null; folder_readers?: unknown; access_pending?: boolean; write_channel?: string | null }interface InterimRow extends InterimAccess { slug: string; title: string; frontmatter: unknown; body: string | null; content_hash: string | null; long_body: boolean; tagged: boolean; deleted: boolean }const INTERIM_COLUMNS = `p.slug, p.title, p.frontmatter, ${boundedBodyPrefixSql('p')} AS body, p.content_hash,- coalesce(p.body_bytes, 0) > ${BOUNDED_PREFIX_BYTES} AS long_body, p.folder_rule_ids,+ coalesce(p.body_bytes, 0) > ${BOUNDED_PREFIX_BYTES} AS long_body, p.folder_rule_ids, p.write_origin->>'channel' AS write_channel,(SELECT jsonb_agg(r.readable_by) FROM access_rules r WHERE r.source_id || ':' || r.rule_id = ANY(p.folder_rule_ids)) AS folder_readers,EXISTS (SELECT 1 FROM access_pending_coverage pc WHERE pc.source_id = p.source_id AND ${ruleCoversSql('p.slug', 'pc.prefix')}) AS access_pending`;@@ -191,6 +191,7 @@ function originalLabels(row: InterimAccess, frontmatter: Record<string, unknown>/** Map one interim page onto a card. Pure: reads nothing beyond the row. */export function mapInterimPage(row: Pick<InterimRow, 'slug' | 'title' | 'frontmatter' | 'body'> & InterimAccess, prefix: string): Pick<AdoptEntry, 'card' | 'labels' | 'refused' | 'problem'> {+ if (row.write_channel?.startsWith('mcp:')) return { card: null, labels: null, refused: [], problem: invalidRecord('remote_written', `the page was last written by a remote client (${row.write_channel}); adopt takes only records written on this machine or by sync`) };const text = recordJson(row.body ?? '');if (text === null) return { card: null, labels: null, refused: [], problem: invalidRecord('no_record', 'the page has no json block under a ## Record heading in its first 64 KiB') };let record: unknown;@@ -279,8 +280,10 @@ function blockedRefusal(opts: AdoptOptions, blocked: AdoptEntry[]): OperationErrconst list = blocked.slice(0, 5).map((e) => `${e.slug} (${e.problem!.message})`).join('; ');return opError(first.problem!.code, `${blocked.length} interim record(s) cannot become cards: ${list}${blocked.length > 5 ? '; and more' : ''}. Nothing was written.`,first.problem!.code === 'storage_record_secret'- ? 'Remove the credential from the record on each named page (the restore skill reads credentials from its own environment), then preview the adopt again.'- : 'Fix the record on each named page so it is a complete card, then preview the adopt again.',+ ? 'Remove the credential from the record on each named page (the restore skill reads credentials from its own environment); a field refused only for its name (credential_key) that holds no credential can instead be renamed to end in _id, _ref or another suffix the rule passes. Then preview the adopt again.'+ : first.problem!.reason === 'remote_written'+ ? 'Adopt never registers a record a remote client wrote. Check each named page yourself and register the item with gbrain storage-register, or leave it as it is.'+ : 'Fix the record on each named page so it is a complete card, then preview the adopt again.',{ reason: first.problem!.reason, fix: previewFix(opts) });}diff --git a/src/core/storage-manifest/schema.ts b/src/core/storage-manifest/schema.tsindex 011997e3bc..255e725160 100644--- a/src/core/storage-manifest/schema.ts+++ b/src/core/storage-manifest/schema.ts@@ -89,7 +89,8 @@ export type Card = CardInput & { registered_by: string; origin: (typeof ORIGINS)const COMMAND_KEYS = new Set(['command', 'cmd', 'script', 'exec', 'shell', 'instructions']);export const STAMPED_KEYS = ['registered_by', 'origin'] as const;const CREDENTIAL_NAME = /(?:KEY|SECRET|TOKEN|PASSWORD|PASSWD|PASS|CRED|AUTH)/i;-const CREDENTIAL_KEY = /(?:^|_)(?:password|passwd|passphrase|secret|token|credentials?|(?:api|access|secret|application|account|private|auth|master|encryption)_?key)$/;+const CREDENTIAL_KEY = /(?:^|_)(?:password|passwd|passphrase|secret|token|credentials?|(?:api|access|secret|application|account|private|auth|master|encryption)_?key)(?:_|$)/;+const REFERENCE_KEY = /_(?:env|var|variable|file|path|name|ref|id)$/;const ENV_ASSIGNMENT = /(?:^|[\s;&|(])[A-Za-z_][A-Za-z0-9_]{1,}=\S/;const VAR_REFERENCE = /\$\{?([A-Za-z_][A-Za-z0-9_]*)/g;const URL_USERINFO = /[a-z][a-z0-9+.-]*:\/\/[^/\s@]*:[^/\s@]*@/i;@@ -131,19 +132,16 @@ export function isCommandKey(key: string): boolean {return COMMAND_KEYS.has(key.toLowerCase());}-/**- * A non-empty string or number under a key whose last word names a credential (password, restic_password, application_key).- * A key that only starts with one (token_env, password_file, token_count, api_key_last4) names something else.- */+/** A non-empty string or number under a key named for a credential (password, application_key), unless the key names where it lives (password_file, token_env). */export function holdsCredential(key: string, value: unknown): boolean {if (!(typeof value === 'number' || (typeof value === 'string' && value.trim() !== ''))) return false;const name = key.replace(/([a-z0-9])([A-Z])/g, '$1_$2').replace(/-/g, '_').toLowerCase();- return CREDENTIAL_KEY.test(name);+ return CREDENTIAL_KEY.test(name) && !REFERENCE_KEY.test(name);}function credentialKeyRefusal(path: string): OperationError {return opError('storage_record_secret', `Storage card field ${path} is named for a credential and holds a value; nothing was written.`,- 'Remove that field. Cards name where an item is and how to restore it; the restore skill reads credentials from its own environment (name the variable, e.g. token_env, instead).',+ 'If it holds a credential, remove it: the restore skill reads credentials from its own environment (name the variable, e.g. token_env, instead). If it holds none (a count, a version, a date), rename the key so it ends in a suffix this rule passes: _id, _ref, _name, _path, _file, _env, _var or _variable (token_count becomes token_count_id, say).',{ reason: 'credential_key' });}diff --git a/test/storage-manifest-adopt.test.ts b/test/storage-manifest-adopt.test.tsindex 78b5d9fe0e..b7835470f5 100644--- a/test/storage-manifest-adopt.test.ts+++ b/test/storage-manifest-adopt.test.ts@@ -14,7 +14,8 @@* - each card takes its original's access labels, readable_by narrowed to the readers of the folder rules over it, so a* storage-only remote caller reads no card its original was kept from; no common reader blocks the page, and a card* that lost the labels is a conflict whose original finalize never deletes;- * - a write accepted but still pending is reported with its receipt (side_effects partial, never none).+ * - a write accepted but still pending is reported with its receipt (side_effects partial, never none);+ * - a stored-object page a remote (MCP) client wrote is blocked at preview and apply, and nothing it holds reaches a card.*/import { afterAll, beforeAll, beforeEach, describe, expect, test } from 'bun:test';import { randomUUID } from 'node:crypto';@@ -303,6 +304,25 @@ describe('storage adopt on a repo-less brain', () => {expect((await previewAdopt(ctx(), opts)).entries.map((e) => e.state)).toEqual(['conflict']);}, 60_000);+ test('a stored-object page a remote client wrote is blocked at preview and apply, and its restore.args never reach a card', async () => {+ await interim('storage-manifest/dataset-l', record('dataset-l'));+ await engine.executeRaw("INSERT INTO oauth_clients(client_id,client_secret_hash,client_name,scope,source_id) VALUES('client-w','fixture-hash','example-client','read write','default')");+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+ const remote = ctx({ remote: true, transport: 'http', auth: { token: 'fixture', clientId: 'client-w', principal: { kind: 'oauth_client', id: 'client-w' }, scopes: ['read', 'write'], sourceId: 'default', allowedSources: ['default'] } } as Partial<OperationContext>);+ const rec = JSON.stringify(record('dataset-w', { restore: { skill: 'restore-data', args: { target: '/srv/remote-chosen-target' } } }));+ await call('put_page', { slug: 'storage-manifest/dataset-w', request_id: randomUUID(), content: `---\ntype: stored-object\ntitle: Remote record\n---\n## Record\n\n${FENCE}json\n${rec}\n${FENCE}\n` }, remote);+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ const plan = await previewAdopt(ctx(), opts);+ expect(plan.entries.map((e) => [e.slug, e.state, e.problem?.reason ?? null])).toEqual([['storage-manifest/dataset-l', 'register', null], ['storage-manifest/dataset-w', 'blocked', 'remote_written']]);+ expect(plan.apply).toBeUndefined();+ const refused = await refusal(applyAdopt(ctx(), { ...opts, expect: plan.plan_hash, dropFields: [] }));+ expect({ code: refused?.code, reason: refused?.reason }).toEqual({ code: 'storage_record_invalid', reason: 'remote_written' });+ expect(await engine.executeRaw("SELECT 1 FROM pages WHERE source_id = 'storage'")).toEqual([]);+ expect(JSON.stringify(await engine.executeRaw("SELECT frontmatter FROM pages WHERE source_id = 'storage'"))).not.toContain('remote-chosen-target');+ expect(await tagged()).toEqual([]);+ }, 60_000);+test('a write accepted but still pending is reported with its receipt, never as no change, and the rerun finishes it', async () => {await interim('storage-manifest/dataset-a', record('dataset-a'));const plan = await previewAdopt(ctx(), opts);diff --git a/test/storage-manifest.test.ts b/test/storage-manifest.test.tsindex 15b811a18d..d54a9a0da6 100644--- a/test/storage-manifest.test.ts+++ b/test/storage-manifest.test.ts@@ -6,7 +6,7 @@* the closest cards; storage_list filters by store and status;* - validation refuses a missing field, a command field, a stamped field, a scanner secret (in a value or an object* key) and every credential shape (env assignment, credential variable, URL userinfo, presigned URL, a value under- * a credential-named key), without echoing the value; a key whose last word names no credential (token_env, token_count, api_key_last4) passes;+ * a credential-named key), without echoing the value; a credential word anywhere in a key refuses (token_value, token_count) unless the key ends in a reference suffix (token_env, token_count_id), and the refusal says to rename it;* - storage_update keeps the page's access frontmatter (visibility, readable_by) when it rewrites the card;* - the write guard: put_page, remember and forget_fact into the shelf are refused storage_source_managed, a forged* parameter does not open it, and the storage verbs still write;@@ -137,6 +137,11 @@ describe('storage shelf on a repo-less brain', () => {[card('a-12', { details: { restic_password: 'abcdef123-correct-horse' } }), 'storage_record_secret', 'credential_key'],[card('a-14', { restore: { skill: 'restore-data', args: { id: 'a-14', access_token: 'abcdef123ghijklmno' } } }), 'storage_record_secret', 'credential_key'],[card('a-15', { details: { secretAccessKey: 'abcdef123/pqrstuvw' } }), 'storage_record_secret', 'credential_key'],+ [card('a-16', { details: { token_value: 'abcdef123-token' } }), 'storage_record_secret', 'credential_key'],+ [card('a-17', { details: { secret_value: 'abcdef123-secret' } }), 'storage_record_secret', 'credential_key'],+ [card('a-18', { details: { password_value: 'abcdef123-horse' } }), 'storage_record_secret', 'credential_key'],+ [card('a-19', { details: { restic_password_b64: 'abcdef123-b64' } }), 'storage_record_secret', 'credential_key'],+ [card('a-20', { details: { token_count: 1200 } }), 'storage_record_secret', 'credential_key'],];for (const [input, code, reason] of cases) {const r = await refusal(call('storage_register', { card: input }));@@ -147,7 +152,9 @@ describe('storage shelf on a repo-less brain', () => {}expect(await engine.executeRaw("SELECT 1 FROM pages WHERE source_id='storage'")).toEqual([]);await call('storage_register', { card: card('a-13', { restore: { skill: 'restore-data', args: { id: 'a-13', key: 'backups/a-13.tar', password_file: '/etc/restic/pass', token_env: 'B2_APPLICATION_KEY' } },- details: { token_count: 1200, secret_version: 'v2', password_changed_at: '2026-11-01T00:00:00Z', api_key_last4: '7c2f' } }) });+ details: { token_count_id: 1200, secret_ref: 'vault/b2-archive' } }) });+ const renamed = await operationsByName.storage_register!.handler(ctx(), { card: card('a-21', { details: { token_count: 1200 } }) }).then(() => null, (e: { suggestion?: string }) => e.suggestion ?? '');+ expect(renamed).toContain('rename the key so it ends in a suffix this rule passes: _id, _ref');}, 30_000);test('the write guard refuses put_page and remember into the shelf, forged parameters included', async () => {