Garry's Opinionated OpenClaw/Hermes Agent Brain
Research maps: storage engine overview
These maps describe how gbrain (TypeScript, garrytan/gbrain at commit 8bbfd0b5e8) uses Postgres and PGLite today. They are the input for the Rust storage engine. Every claim cites path:line in the gbrain repo at that commit.
| Map | What it covers |
|---|---|
| brain-engine.md | Every BrainEngine member by family: signature, behavior, SQL, PGLite vs Postgres differences, tests. Also what a Rust engine needs to pass test/e2e/engine-parity.test.ts, and binding options for a third engine in Bun. |
| schema.md | Every table, column, index, function, trigger and migration (132 tables, 313 migrations), with readers and writers. |
| postgres-features.md | Every Postgres-specific feature, where it is used, what depends on it, and embedded Rust replacements with parity risks. |
The other research maps (agent contract, background jobs, test suites, subsystems, gbrain-evals) are listed in the index.
Confirmed first build target
Garry confirmed option B as the first build target: a Rust storage engine that implements the BrainEngine interface as a third engine inside garrytan/gbrain, next to PGLite and Postgres. PGLite and Postgres stay supported. PGLite may be deprecated later if the Rust engine proves itself. The binding (napi-rs, sidecar or another option) is still open; see question 1 below.
Hardest parity risks
- Multi-process access to one brain. Postgres lets the CLI, the MCP server, cron, hooks and job workers write at the same time; coordination is advisory locks, row locks,
SKIP LOCKEDand a lease table. PGLite allows one process per data directory and returnspglite_busyto the others. An embedded Rust engine must pick a model: a single writer daemon, SQLite WAL withBEGIN IMMEDIATEacross processes, or an exclusive file lock. Only the daemon avoidspglite_busy-style failures for concurrent commands. - Logic inside the database. About 25 plpgsql trigger functions maintain derived state (search vectors, revision counters, projection queues, generation clocks) and refuse writes with typed
P0001messages. Some read session GUCs (gbrain.write_trust_tier,gbrain.write_sources). Each one must become an explicit, tested Rust write-path step. A missed trigger breaks search or purge with no error. - Keyword ranking. Keyword search is
websearch_to_tsquery+ts_rankwith A/B/C weights, the Snowball English config, size caps on the tsvector input, and a CJKILIKEfallback. BM25 (FTS5, tantivy) gives different scores and stems. Hybrid RRF uses only list order, so order parity is enough there, butscore_detailsand keyset cursors expose raw scores. - Vector search. pgvector HNSW is approximate and tuned per query (
ef_search, iterative scan), with an exact-scan fallback on Postgres only. A Rust ANN index returns a different top-k at the margin. The exact scan is the parity oracle.halfvecstorage and dimension reconciliation at connect must be kept. - The parity test does not fit a third engine yet.
test/e2e/engine-parity.test.tsneedsDATABASE_URLfor 16 of its 17 blocks, covers 60 of 197 engine members, and makes 46 raw Postgres SQL calls throughexecuteRaw. A Rust engine needs either a SQL compatibility layer or a rewrite of those fixtures into typed methods. executeRawis part of the interface, and it is used everywhere. 1,203 files insrc/outside the engine code callexecuteRawwith Postgres SQL (largest groups:core/persistence232,commands/doctor123,core/repair61,core/facts46,core/minions40). A third engine that implements only the typed methods leaves all of these broken. It must either run that SQL (SQLite cannot run most of it) or the callers must move to typed methods. This is likely the largest single cost of the third-engine plan.- Engine differences that callers can see today. For example,
searchKeywordChunksfilters and thestaleflag differ between PGLite and Postgres, and several interface doc comments disagree with the code. The Rust engine needs one chosen reference behavior per method. - Upgrades from existing brains. 313 migrations include data backfills. In-place upgrade of a PGLite or Postgres brain to the Rust engine would need them; an export-and-import path from the markdown brain plus derived state avoids most of them.
Open architecture questions for Garry
The engine placement is settled (see Confirmed first build target). These questions remain open:
- Binding. napi-rs in-process addon (lowest call cost, reuses the
native/locksprebuild andrequire()embedding pattern, no crash isolation) or a Rust sidecar over a socket (crash isolation, natural single writer for CLI, MCP and cron, per-call serialization and startup cost)? WASM andbun:ffilook weak for this use. See the binding section inbrain-engine.md. - Storage core. SQLite (rusqlite, bundled) with FTS5 and a vector extension, or a key-value store (redb or fjall) with tantivy and an HNSW crate? SQLite keeps SQL and transactions and makes porting the shared
engine-sqlstatements easier. A key-value store gives more control but requires every query to be written by hand. - Concurrency model. Is one writer per brain acceptable (daemon or
BEGIN IMMEDIATE), or must several processes write at the same time as on Postgres today? executeRaw. Can the Rust engine refuse raw Postgres SQL, so that callers must move to typed engine methods first? If not, how much SQL dialect translation is acceptable?- Ranking parity bar. Is order parity on the eval suites (gbrain-evals) enough for keyword and vector search, or must raw scores match within a tolerance?
- Reference behavior. Where PGLite and Postgres differ, should the Rust engine match Postgres (the safer reference) by default?
- Migration path. Should the Rust engine import existing PGLite and Postgres brains in place, or only through export and re-import?
- Access control. Is it acceptable to drop RLS and database roles and rely on application authorization plus file permissions for the embedded engine?