Skip to content

The fastest, cheapest, most efficient open-source agent harness. Run more than 500 agents on a $10 VPS.

docs(storage): update storage architecture documentation

Steven Enamakelc898eeda64ba
diff --git a/crates/openhuman-core/src/storage/README.md b/crates/openhuman-core/src/storage/README.md
index 92bf55360a..f38424fe15 100644
--- a/crates/openhuman-core/src/storage/README.md
+++ b/crates/openhuman-core/src/storage/README.md
@@ -355,7 +355,7 @@ storage itself.
## Schema, scopes and boot
- `scope.rs`: the one scope grammar (`tenant_scope`, `scope_for_profile`, `scope_for_agent`, `scope_for_module`), built on tsd's `Scope::tenant/local/agent/module`.
-- `schema.rs`: `owner_schemas()` and the composed `registry()`; `module_items(owner)` feeds the generated module grants.
-- `boot.rs`: `prepare_and_migrate`, called by `open`: prepare, scope move, then per scope the importers and `Registry::migrate`.
+- `schema.rs`: `owner_collections()` composes and validates the linked owners' `CollectionSpec` declarations; `module_collections(owner)` feeds module grants.
+- `boot.rs`: `prepare_and_migrate`, called by `open`, ensures the composed collections in the host's local and cluster scopes. Tenant, agent and module scopes are prepared by their owners when opened.
- `scope_move.rs`: the resumable copy of v1 scopes (`profile:`, `<profile>~<agent>`, raw agent ids, `module:<id>@<base>`, the `*-sha256:` fallbacks) into the v2 scopes.
-- The generated reference is `docs/storage-schema.md`; `node scripts/ci/check-storage-schema.mjs` lints it.
+- The generated reference is `docs/storage-schema.md`; `node scripts/ci/check-storage-schema.mjs` lints the current collection names, indexes, expiry fields and search fields.
diff --git a/docs/gitbooks/en/developing/architecture/storage.md b/docs/gitbooks/en/developing/architecture/storage.md
index 35896af77f..efc004d78c 100644
--- a/docs/gitbooks/en/developing/architecture/storage.md
+++ b/docs/gitbooks/en/developing/architecture/storage.md
@@ -170,12 +170,12 @@ CI runs these targets, with the lib tests of the domains on the ports, in the co
## Schema, boot and migration
-Every crate that stores records exports `storage_schema()` (collections named `<owner>.<entity>`, streams, blob and secret prefixes, each with a scope class and the v1 to v2 steps that bring older data forward). The host composes them in `storage::schema` (`Registry::compose`: harness, session, graph, tasks, flows and cron, skills, and with `modules` the mcp, channels, connectors, box and juice owners, with `web3` the wallet, with `tinymemes` the memes owner, with `inbound-queues` the `teq` owner of the inbound event queues). The composed result is documented in [`docs/storage-schema.md`](../../../../storage-schema.md), generated by `scripts/ci/check-storage-schema.mjs` (`pnpm docs:generate`), which also lints names, casing, `*_ms` time fields and single ownership in `pnpm rust:layout`.
+Storage owners expose pure v2 `CollectionSpec` values. The host composes and validates them in `storage::schema::owner_collections`, then ensures those collections in its shared scopes and when an owner opens a scope. The generated [`docs/storage-schema.md`](../../../../storage-schema.md) lists the composed collection names, indexes, expiry fields and search fields; `scripts/ci/check-storage-schema.mjs` checks their names and millisecond timestamp fields. Record shapes, legacy conversion and migration rules remain with the owning domains.
`storage::open` runs `storage::boot::prepare_and_migrate` on the raw backend before the lease fence wraps it:
-1. `Registry::prepare` declares every collection.
-2. **Scope move** (`storage::scope_move`): every v1 scope is copied (documents with their schema, creation time and expiry; streams and their sealed state; blobs) into the v2 scope it maps to and then removed. v1 raw agent ids and `default` become `local/agent:<id>`, `<profile>~<agent>` becomes `u:<profile>/agent:<agent>`, `profile:<id>` becomes `u:<id>`, `module:<id>@<base>` and `<profile>~module:<id>` become `<tenant>/module:<id>`. A hashed v1 scope (`sha256:`, `profile-sha256:`, `module-sha256:`) maps only when a recorded agent, a provisioned profile or a registry module hashes to it; others stay in place and are logged. The move is resumable and recorded in the target scope's `tsd.migrations` (owner `host`, step `scope_move:<source>`).
-3. Per scope: the pre-migrate importer (`import_legacy_ledger`), `Registry::migrate` (each owner's steps), then the post-migrate importers that move key-value state and counters into `tsd.kv` and `tsd.counters` (`import_legacy_kv`, `import_legacy_leases`, `import_legacy_task_cache`, the flows importers). A failing piece is logged and retried at the next boot.
+1. `owner_collections` merges compatible declarations and fails on conflicting specs before the backend is touched.
+2. The host ensures composed collections in its local and cluster scopes. Profile, agent and module scopes are prepared by their owners when opened; no driver-wide schema registry or scope enumeration is used.
+3. Historical workspace and per-domain files are read by the separately gated `legacy-import` path. Its source formats and typed owner conversions are kept separate from normal backend opening.
-A module's storage grant is generated from the module-scoped items of its owner's schema (`modules::storage_grant_schemas`), so it cannot drift from what the module declares. The module's legacy-file importers still write the v1 shapes; the owner's steps carry them forward.
+A module's storage grant is generated from its owner's declared collections (`modules::storage_grant_schemas`), so it cannot drift from what the module declares.
diff --git a/examples/storage_schema_dump.rs b/examples/storage_schema_dump.rs
index 1a62331ef5..5f74ac407e 100644
--- a/examples/storage_schema_dump.rs
+++ b/examples/storage_schema_dump.rs
@@ -13,5 +13,8 @@ use serde_json::json;
fn main() {
let collections = owner_collections().expect("owner collection declarations compose");
- println!("{}", serde_json::to_string_pretty(&json!({ "collections": collections })).unwrap());
+ println!(
+ "{}",
+ serde_json::to_string_pretty(&json!({ "collections": collections })).unwrap()
+ );
}