Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Merge remote-tracking branch 'origin/capy/storage-manifest-shelf' into capy/storage-manifest-adopt
garrytan8bbaa0b24e3c
diff --cc CHANGELOG.mdindex df30825276,d13815328c..9d0aba9e89--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -18,16 -18,8 +18,16 @@@ Cards never carry credentials or commanThe shelf is written only through the storage verbs. `put_page`, `remember` and every other page writer are refused `storage_source_managed` there, a request parameter cannot open the door, register and update run only from the trusted local CLI, and a dream cycle or enrichment run scoped to the shelf skips it. The source is not federated: a remote connection reads cards only when its grant names `storage`.+**Agents find the shelf, and older archive records move onto it.** The brain-first lookup table every harness reads gains one row, "Find something archived to B2, Supabase or S3" (`gbrain storage-find` / `storage_find`), and a new `storage-manifest` skill says when to register, to find before restoring, to restore through the agent's own restore skill with the card's `restore.skill` and `restore.args`, and to record the restore with `gbrain storage-update`. `gbrain storage adopt` moves interim records (`stored-object` pages under `storage-manifest/` holding their record as JSON under `## Record`) onto the shelf: the preview shows each mapped card and every field a card refuses with its reason; `--apply --expect <plan_hash>` registers each card with `origin: adopt` and tags the original `storage-manifest-adopted`, deleting nothing, and refuses until each refused field is named with `--drop-field` (a credential-shaped value always refuses); `--finalize` previews, then soft-deletes the adopted originals only when every card matches its original. `gbrain storage export --jsonl` and `gbrain storage import --jsonl <file>` (preview per line, then `--apply --expect`) rebuild a shelf from a file, registering missing cards with `origin: import` and never overwriting a differing card. Applies are idempotent and report `side_effects` and, when a run stops part way, the command that finishes it. The new doctor check `storage_manifest` warns about cards that fail the card schema, interim records not adopted yet and a hosted shelf whose directory is gone. On a brain with a Git checkout, adopt and import wait for `gbrain storage init`.+### Itemized changes+- `src/core/storage-manifest/adopt.ts`, `src/core/storage-manifest/jsonl.ts`, `src/commands/storage.ts`: `gbrain storage adopt` (preview, apply, finalize) and `gbrain storage export|import --jsonl`, each preview-first and bound to its plan_hash (a stale one refuses `preview_changed`). `schema.ts` exports the credential-shape rules, the command-key test and the card keys adopt maps onto, plus `cardContent`/`cardDigest` for comparing copies; `source.ts` gains `shelfWritable` (the refusal a storage write would get, without creating anything); `find.ts` gains `readShelfCards`.+- `src/commands/doctor/checks/storage-manifest.ts`, `src/commands/doctor/registry.ts`, `src/core/doctor-categories.ts`: the read-only `storage_manifest` check (brain category), `not_applicable` with no shelf and no interim record. Doctor goldens regenerated for the new check.+- `src/core/error-registry.ts`, `docs/guides/write-refusals.md`, `docs/guides/error-codes.md`: `storage_adopt_field_refused` (consent: the user agrees to leave refused fields out) and `storage_adopt_unverified`; `storage_record_invalid` gains reasons `no_record` and `duplicate_id`.+- `skills/conventions/brain-first.md` (and its generated copies in `AGENTS.md`, `INSTALL_FOR_AGENTS.md`, `skills/RESOLVER.md`, the bootstrap template and the plugin trees), `skills/storage-manifest/SKILL.md`, `skills/RESOLVER.md`, `skills/manifest.json`, `openclaw.plugin.json`: the lookup row and the skill.+- `docs/guides/storage-manifest.md`: the guide (verbs, repo-less and Git-checkout brains, adopt, export and import, doctor, privacy).- - Tests: `test/storage-manifest-adopt.test.ts`, `test/storage-manifest-jsonl.test.ts` and `test/doctor-storage-manifest.test.ts` (PGLite, repo-less) and `test/storage-manifest-adopt-hosted.serial.test.ts` (managed PGLite with a Git checkout).++- Tests: `test/storage-manifest-adopt.test.ts`, `test/storage-manifest-jsonl.test.ts` and `test/doctor-storage-manifest.test.ts` (PGLite, repo-less) and `test/storage-manifest-adopt-hosted.test.ts` (managed PGLite with a Git checkout).- `src/core/storage-manifest/` (`schema.ts`, `door.ts`, `source.ts`, `find.ts`) and `src/core/ops/storage-manifest.ts`: the card schema and secret checks, the write door, the source lifecycle (`storage init`), keyless ranked lookup (exact id, `page_aliases`, word matches, `pg_trgm` closest cards) and the five ops `storage_find`, `storage_get`, `storage_list`, `storage_register`, `storage_update`.- `src/core/persistence/page-mutations.ts`, `src/core/persistence/memory-mutations.ts`: the page door and `remember`/`forget` refuse writers other than the storage verbs on a source with `role: storage_manifest`; the owner's import of commits already in the hosted repository still syncs.- `src/core/cycle.ts`, `src/commands/enrich.ts`: a cycle scoped to the storage source returns `skipped` with reason `storage_manifest_source`; enrichment considers no card there.diff --cc docs/architecture/key-files/commands-5-continued.mdindex a55f6cd419,c7fc6f5090..fddb89ccd1--- a/docs/architecture/key-files/commands-5-continued.md+++ b/docs/architecture/key-files/commands-5-continued.md@@@ -17,5 -17,4 +17,5 @@@ Current behavior and load-bearing invar- `src/commands/status-funnel.ts` — `gbrain status --funnel [--json] [--enable | --delete]` (D8 adoption funnel): engine-free, dispatched from `src/cli/main.ts` before any connect so it works before `gbrain init` and on a thin client; reads and writes only the local record through `src/core/funnel.ts`. Guide: `docs/guides/adoption-funnel.md`. Tests: `test/funnel.test.ts`, `test/funnel-cli.test.ts`.-- `src/commands/storage.ts` — `gbrain storage status [--repo P] [--json]`. Split into pure data (`getStorageStatus`) + JSON formatter + human formatter (ASCII-only) matching the `orphans.ts` pattern. `PageCountsByTier` and `DiskUsageByTier` are distinct nominal types so swaps fail at compile time. The repo and the counted source come from `resolveRestoreTarget` (`src/core/restore-target.ts`, the `export --restore-only` rule), and a page is listed missing exactly when that export would restore it (`isDbOnly`, then `restoreFilePath` in the page source's `scannerSlugRootMode` and `nativeFileTarget`: the recorded `source_path`, else `<slug>.md`, absent; the repo walk answers without a per-page syscall whenever it saw that exact file as a regular file under real directories, and a recorded path starting with the repo's path inside its git checkout, which export re-anchors, takes the full check), so the printed `Use: gbrain export --restore-only --source <id> --repo <path>` (the path passed through `shellQuote`, `src/core/shell-quote.ts`) writes exactly the listed files into `--dir` (default `./export`, not the repo). An unsafe recorded path or a symlinked/ambiguous file target (`OperationError` from `nativeFileTarget`) becomes one warning per reason (lowest slug plus a count of further pages) plus the refusal export would give, printed even when no file is missing; a repo whose own owner (`resolveSourceForRepoPath`: dotfile, then longest registered `local_path`) is archived is refused too (unless an active source is registered at the repo's exact path, which owns it outright), although export's exact-path rule would fall back to the only active source; a refused rule prints `Cannot suggest a restore command:` and falls back to the `--repo` / default-source path, counting the active repo owner's pages (dotfile, then longest registered `local_path`) or every source's pages when no active source owns it. `restoreSource` / `restoreRefusal` carry this in the JSON result. Pinned by `test/storage-restore-hint.test.ts`.+- `src/commands/storage.ts` — `gbrain storage status [--repo P] [--json]`. Split into pure data (`getStorageStatus`) + JSON formatter + human formatter (ASCII-only) matching the `orphans.ts` pattern. `PageCountsByTier` and `DiskUsageByTier` are distinct nominal types so swaps fail at compile time. The repo and the counted source come from `resolveRestoreTarget` (`src/core/restore-target.ts`, the `export --restore-only` rule), and a page is listed missing exactly when that export would restore it (`isDbOnly`, then `restoreFilePath` in the page source's `scannerSlugRootMode` and `nativeFileTarget`: the recorded `source_path`, else `<slug>.md`, absent; the repo walk answers without a per-page syscall whenever it saw that exact file as a regular file under real directories, and a recorded path starting with the repo's path inside its git checkout, which export re-anchors, takes the full check), so the printed `Use: gbrain export --restore-only --source <id> --repo <path>` (the path passed through `shellQuote`, `src/core/shell-quote.ts`) writes exactly the listed files into `--dir` (default `./export`, not the repo). An unsafe recorded path or a symlinked/ambiguous file target (`OperationError` from `nativeFileTarget`) becomes one warning per reason (lowest slug plus a count of further pages) plus the refusal export would give, printed even when no file is missing; a repo whose own owner (`resolveSourceForRepoPath`: dotfile, then longest registered `local_path`) is archived is refused too (unless an active source is registered at the repo's exact path, which owns it outright), although export's exact-path rule would fall back to the only active source; a refused rule prints `Cannot suggest a restore command:` and falls back to the `--repo` / default-source path, counting the active repo owner's pages (dotfile, then longest registered `local_path`) or every source's pages when no active source owns it. `restoreSource` / `restoreRefusal` carry this in the JSON result. Pinned by `test/storage-restore-hint.test.ts`. The storage shelf subcommands (GST-1943) are `storage init`, `storage adopt` (preview, `--apply --expect`, `--finalize [--expect]`; logic in `src/core/storage-manifest/adopt.ts`) and `storage export|import --jsonl` (`src/core/storage-manifest/jsonl.ts`); each builds a trusted local `OperationContext` and renders refusals with `writeCliError`.- - `src/core/storage-manifest/adopt.ts`, `src/core/storage-manifest/jsonl.ts` and `src/commands/doctor/checks/storage-manifest.ts` — GST-1943 storage shelf recovery. Adopt reads interim `stored-object` pages under a prefix (never `<prefix>index`) from the bounded body prefix (`boundedBodyPrefixSql`), takes the JSON block under `## Record`, strips every field the card refuses (`command_field`, `unknown_field`, `stamped_field`; `CARD_KEYS` in `schema.ts`) and validates the rest with `parseCardInput`; a credential-shaped value (`credentialShape`) blocks the page whatever is dropped. Apply registers through `registerCard` (`origin: adopt`) and tags the original `storage-manifest-adopted` through `submitPageMutation`; it never deletes. Finalize soft-deletes tagged originals only when `cardDigest` of the shelf card equals the mapped card. Every plan_hash covers end state (mapped cards, refusals, blocks, conflicts), never progress, so the printed rerun of a partial apply still applies. JSONL import registers `origin: import` and never overwrites. The doctor check `storage_manifest` is read-only: schema-failing cards, unadopted interim records outside the shelf, a hosted shelf directory gone; `not_applicable` with neither shelf nor interim pages. Pinned by `test/storage-manifest-adopt.test.ts`, `test/storage-manifest-jsonl.test.ts`, `test/doctor-storage-manifest.test.ts` and `test/storage-manifest-adopt-hosted.serial.test.ts`.++- `src/core/storage-manifest/adopt.ts`, `src/core/storage-manifest/jsonl.ts` and `src/commands/doctor/checks/storage-manifest.ts` — GST-1943 storage shelf recovery. Adopt reads interim `stored-object` pages under a prefix (never `<prefix>index`) from the bounded body prefix (`boundedBodyPrefixSql`), takes the JSON block under `## Record`, strips every field the card refuses (`command_field`, `unknown_field`, `stamped_field`; `CARD_KEYS` in `schema.ts`) and validates the rest with `parseCardInput`; a credential-shaped value (`credentialShape`) blocks the page whatever is dropped. Apply registers through `registerCard` (`origin: adopt`) and tags the original `storage-manifest-adopted` through `submitPageMutation`; it never deletes. Finalize soft-deletes tagged originals only when `cardDigest` of the shelf card equals the mapped card. Every plan_hash covers end state (mapped cards, refusals, blocks, conflicts), never progress, so the printed rerun of a partial apply still applies. JSONL import registers `origin: import` and never overwrites. The doctor check `storage_manifest` is read-only: schema-failing cards, unadopted interim records outside the shelf, a hosted shelf directory gone; `not_applicable` with neither shelf nor interim pages. Pinned by `test/storage-manifest-adopt.test.ts`, `test/storage-manifest-jsonl.test.ts`, `test/doctor-storage-manifest.test.ts` and `test/storage-manifest-adopt-hosted.test.ts`.diff --cc test/storage-manifest-adopt-hosted.test.tsindex d5ebf0df82,0000000000..d5ebf0df82mode 100644,000000..100644--- a/test/storage-manifest-adopt-hosted.test.ts+++ b/test/storage-manifest-adopt-hosted.test.ts