The fastest, cheapest, most efficient open-source agent harness. Run more than 500 agents on a $10 VPS.
Rust agent-orchestrationai-agentsai-assistantdesktopllmlocal-firstmcppersonal-aiprivacyrustsecond-braintauri
docs(storage): update storage architecture documentation
Steven Enamakelc898eeda64ba
diff --git a/crates/openhuman-core/src/storage/README.md b/crates/openhuman-core/src/storage/README.mdindex 92bf55360a..f38424fe15 100644--- a/crates/openhuman-core/src/storage/README.md+++ b/crates/openhuman-core/src/storage/README.md@@ -355,7 +355,7 @@ storage itself.## Schema, scopes and boot- `scope.rs`: the one scope grammar (`tenant_scope`, `scope_for_profile`, `scope_for_agent`, `scope_for_module`), built on tsd's `Scope::tenant/local/agent/module`.-- `schema.rs`: `owner_schemas()` and the composed `registry()`; `module_items(owner)` feeds the generated module grants.-- `boot.rs`: `prepare_and_migrate`, called by `open`: prepare, scope move, then per scope the importers and `Registry::migrate`.+- `schema.rs`: `owner_collections()` composes and validates the linked owners' `CollectionSpec` declarations; `module_collections(owner)` feeds module grants.+- `boot.rs`: `prepare_and_migrate`, called by `open`, ensures the composed collections in the host's local and cluster scopes. Tenant, agent and module scopes are prepared by their owners when opened.- `scope_move.rs`: the resumable copy of v1 scopes (`profile:`, `<profile>~<agent>`, raw agent ids, `module:<id>@<base>`, the `*-sha256:` fallbacks) into the v2 scopes.-- The generated reference is `docs/storage-schema.md`; `node scripts/ci/check-storage-schema.mjs` lints it.+- The generated reference is `docs/storage-schema.md`; `node scripts/ci/check-storage-schema.mjs` lints the current collection names, indexes, expiry fields and search fields.diff --git a/docs/gitbooks/en/developing/architecture/storage.md b/docs/gitbooks/en/developing/architecture/storage.mdindex 35896af77f..efc004d78c 100644--- a/docs/gitbooks/en/developing/architecture/storage.md+++ b/docs/gitbooks/en/developing/architecture/storage.md@@ -170,12 +170,12 @@ CI runs these targets, with the lib tests of the domains on the ports, in the co## Schema, boot and migration-Every crate that stores records exports `storage_schema()` (collections named `<owner>.<entity>`, streams, blob and secret prefixes, each with a scope class and the v1 to v2 steps that bring older data forward). The host composes them in `storage::schema` (`Registry::compose`: harness, session, graph, tasks, flows and cron, skills, and with `modules` the mcp, channels, connectors, box and juice owners, with `web3` the wallet, with `tinymemes` the memes owner, with `inbound-queues` the `teq` owner of the inbound event queues). The composed result is documented in [`docs/storage-schema.md`](../../../../storage-schema.md), generated by `scripts/ci/check-storage-schema.mjs` (`pnpm docs:generate`), which also lints names, casing, `*_ms` time fields and single ownership in `pnpm rust:layout`.+Storage owners expose pure v2 `CollectionSpec` values. The host composes and validates them in `storage::schema::owner_collections`, then ensures those collections in its shared scopes and when an owner opens a scope. The generated [`docs/storage-schema.md`](../../../../storage-schema.md) lists the composed collection names, indexes, expiry fields and search fields; `scripts/ci/check-storage-schema.mjs` checks their names and millisecond timestamp fields. Record shapes, legacy conversion and migration rules remain with the owning domains.`storage::open` runs `storage::boot::prepare_and_migrate` on the raw backend before the lease fence wraps it:-1. `Registry::prepare` declares every collection.-2. **Scope move** (`storage::scope_move`): every v1 scope is copied (documents with their schema, creation time and expiry; streams and their sealed state; blobs) into the v2 scope it maps to and then removed. v1 raw agent ids and `default` become `local/agent:<id>`, `<profile>~<agent>` becomes `u:<profile>/agent:<agent>`, `profile:<id>` becomes `u:<id>`, `module:<id>@<base>` and `<profile>~module:<id>` become `<tenant>/module:<id>`. A hashed v1 scope (`sha256:`, `profile-sha256:`, `module-sha256:`) maps only when a recorded agent, a provisioned profile or a registry module hashes to it; others stay in place and are logged. The move is resumable and recorded in the target scope's `tsd.migrations` (owner `host`, step `scope_move:<source>`).-3. Per scope: the pre-migrate importer (`import_legacy_ledger`), `Registry::migrate` (each owner's steps), then the post-migrate importers that move key-value state and counters into `tsd.kv` and `tsd.counters` (`import_legacy_kv`, `import_legacy_leases`, `import_legacy_task_cache`, the flows importers). A failing piece is logged and retried at the next boot.+1. `owner_collections` merges compatible declarations and fails on conflicting specs before the backend is touched.+2. The host ensures composed collections in its local and cluster scopes. Profile, agent and module scopes are prepared by their owners when opened; no driver-wide schema registry or scope enumeration is used.+3. Historical workspace and per-domain files are read by the separately gated `legacy-import` path. Its source formats and typed owner conversions are kept separate from normal backend opening.-A module's storage grant is generated from the module-scoped items of its owner's schema (`modules::storage_grant_schemas`), so it cannot drift from what the module declares. The module's legacy-file importers still write the v1 shapes; the owner's steps carry them forward.+A module's storage grant is generated from its owner's declared collections (`modules::storage_grant_schemas`), so it cannot drift from what the module declares.diff --git a/examples/storage_schema_dump.rs b/examples/storage_schema_dump.rsindex 1a62331ef5..5f74ac407e 100644--- a/examples/storage_schema_dump.rs+++ b/examples/storage_schema_dump.rs@@ -13,5 +13,8 @@ use serde_json::json;fn main() {let collections = owner_collections().expect("owner collection declarations compose");- println!("{}", serde_json::to_string_pretty(&json!({ "collections": collections })).unwrap());+ println!(+ "{}",+ serde_json::to_string_pretty(&json!({ "collections": collections })).unwrap()+ );}