Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
M10: recall@5 eval harness + ARCHITECTURE.md + README refresh
AkitaOnRails733126c8bcb9
diff --git a/CLAUDE.md b/CLAUDE.mdindex a135b3df..fc233d0f 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -11,8 +11,9 @@ Storage = markdown-in-git wiki (source of truth) + SQLite (derived index).Capture = automatic via agent lifecycle hooks, never manual `write_note`.Consolidation = Karpathy "LLM Wiki" pattern with versioned supersession.-Read [`docs/design-decisions.md`](docs/design-decisions.md) for the full spec.-Read [`docs/research-karpathy-llm-wiki.md`](docs/research-karpathy-llm-wiki.md)+Read [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) first for the+operational summary. Read [`docs/design-decisions.md`](docs/design-decisions.md)+for the full v1 spec. Read [`docs/research-karpathy-llm-wiki.md`](docs/research-karpathy-llm-wiki.md)for what "Karpathy-faithful" means in practice.## Stack (do not deviate without updating `docs/design-decisions.md` §4)diff --git a/README.md b/README.mdindex a830a084..b7a838ef 100644--- a/README.md+++ b/README.md@@ -4,7 +4,7 @@> OpenAI Codex in the same directory, continue without re-explaining the> architecture, the failed approaches, or the open questions.-[](docs/design-decisions.md)+[](docs/ARCHITECTURE.md)[](rust-toolchain.toml)[](#license)@@ -33,11 +33,19 @@ for how this project implements it.## Status-**Under construction.** Currently at milestone **M0** (workspace bootstrap +-CI + config). Next: **M1** — SQLite substrate + file watcher + FTS5 search.+**v0.2 complete.** Milestones M0 through M10 have shipped:-See [`docs/design-decisions.md`](docs/design-decisions.md) for the full-roadmap. v1 ships when M0–M8 are all complete; vectors arrive in v0.2 (M9).+- ✅ **M0–M1** — Workspace, SQLite substrate, FTS5, file watcher, atomic writes.+- ✅ **M2 / M2.5** — MCP server (`rmcp` 1.7), Docker image, backup/restore.+- ✅ **M3 / M4** — Auto-capture via Claude Code / Codex / OpenCode hooks; typed cross-agent handoffs.+- ✅ **M5 / M6** — Wiki git versioning with auto-commit; LLM provider trait (Anthropic, OpenAI, OpenAI-compat).+- ✅ **M7a / M7b** — Karpathy-style single-page LLM consolidation + multi-page atomic fan-out.+- ✅ **M8** — Retention sweep with episodic decay + access reinforcement; rule + LLM lint.+- ✅ **M9** — Pluggable embedders (OpenAI, Voyage, plus a deterministic synthetic embedder for tests); hybrid RRF retrieval over `page_embeddings`.+- ✅ **M10** — Recall@5 eval harness, consolidated [`ARCHITECTURE.md`](docs/ARCHITECTURE.md), this README refresh.++Single-package design: **57 unit + integration tests passing**,+`cargo clippy --workspace -D warnings` clean, `cargo fmt --check` clean.## Architecture in 60 seconds@@ -49,20 +57,26 @@ data directory containing:<data_dir>/├── wiki/ # markdown source of truth (git-versioned)├── raw/ # immutable session log archive-├── db/ # SQLite (FTS5 + sqlite-vec) — derived index-├── models/ # bundled embedding model (v0.2+)+├── db/ # SQLite (FTS5 + page_embeddings) — derived index+├── models/ # reserved for local embedding model (v0.3+)└── logs/ # rolling daily tracing output```Agent lifecycle hooks fire-and-forget POST to the server's HTTP ingress.The server queues writes through a single SQLite writer (no`database is locked`). On session end, an optional LLM-driven pass-rewrites 5–15 wiki pages atomically with supersession (`is_latest=false`-+ `supersedes` chain). Retrieval is hierarchical: `index.md` first, then-page-level FTS5, then optional graph-walk expansion.+rewrites pages atomically with supersession (`is_latest=false` ++`supersedes` chain) and opens a typed handoff for the next agent. The+retention sweep (M8) decays unused episodic content while semantic+concept pages compound forever; pinned pages are exempt. Retrieval is+FTS5 by default; when an embedder is configured, hybrid RRF over+`page_embeddings` joins the FTS5 ranks.++Storage moves between machines via `ai-memory backup --to <tarball>`+(SQLite online backup + git wiki) or just `rsync` of the data dir.-Storage moves between machines via `git push` of the wiki dir +-`sqlite3 .backup` of the DB, or just `rsync` of the data dir.+See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the canonical+data-flow diagram + crate breakdown + cross-cutting invariants.## Quick start@@ -92,21 +106,50 @@ docker compose -f docker/docker-compose.yml up -d --build# Attach Claude Code at http://localhost:7777/mcp```-### Other CLI commands+### Full CLI subcommand list```bash-./target/release/ai-memory --help # full subcommand list+./target/release/ai-memory --help # full subcommand tree./target/release/ai-memory init # create the data dir layout+./target/release/ai-memory serve # MCP server (stdio default; +http)+./target/release/ai-memory watch # standalone filesystem watcher./target/release/ai-memory write-page \- --path notes/foo.md --title Foo --body "..." # exercise the wiki write-./target/release/ai-memory search "karpathy" # FTS5 query-./target/release/ai-memory recent # (M2-B) most-recent pages+ --path notes/foo.md --title Foo --body "..." # manual wiki write+./target/release/ai-memory search "karpathy" # FTS5 query (CLI only; MCP picks hybrid when embedder set)./target/release/ai-memory status --json # counts + paths-./target/release/ai-memory watch # standalone filesystem watcher-./target/release/ai-memory serve # MCP server (stdio default)-./target/release/ai-memory reset --confirm # wipe data (refuses if siblings alive)+./target/release/ai-memory commit -m "..." # stage + commit the wiki repo+./target/release/ai-memory backup --to bak.tar.gz # SQLite online backup → tarball+./target/release/ai-memory restore --from bak.tar.gz # extract + re-open+./target/release/ai-memory reset --confirm # wipe data (refuses while siblings alive)+./target/release/ai-memory install-hooks --agent claude-code # print hook config snippet+./target/release/ai-memory llm-test --provider anthropic --model claude-sonnet-4-7 --prompt "hi"+./target/release/ai-memory forget-sweep [--dry-run] # M8 retention+./target/release/ai-memory lint [--dry-run] # M8 rule + LLM lint+./target/release/ai-memory embed [--dry-run] # M9 backfill embeddings```+### Optional features via env++The system runs with **zero LLM / zero embedder configured** — pure+FTS5 retrieval, synthetic session summaries, no consolidation. Add+features by setting env vars; everything is opt-in.++```bash+# Enable LLM-driven consolidation + lint contradiction detection+AI_MEMORY_LLM_PROVIDER=anthropic+AI_MEMORY_LLM_MODEL=claude-sonnet-4-7+ANTHROPIC_API_KEY=sk-ant-...++# Enable hybrid retrieval (RRF over FTS5 + vector cosine)+AI_MEMORY_EMBEDDING_PROVIDER=openai+AI_MEMORY_EMBEDDING_MODEL=text-embedding-3-small+AI_MEMORY_EMBEDDING_DIM=1536+OPENAI_API_KEY=sk-...+```++Then run `ai-memory embed` once to backfill embeddings for existing+pages.+### Override the data dir```bash@@ -119,11 +162,10 @@ on macOS.## Docs-Long-form research and design lives under [`docs/`](docs/):-| File | What it is ||---|---|-| [`design-decisions.md`](docs/design-decisions.md) | **Read first.** The full spec: storage, MCP surface, hooks, lifecycle, mistakes-to-avoid checklist. |+| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | **Read first.** Operational summary: data flow, crate layout, cross-cutting invariants, current schema. |+| [`docs/design-decisions.md`](docs/design-decisions.md) | The full v1 spec — storage, MCP surface, hooks, lifecycle, mistakes-to-avoid checklist. || [`research-karpathy-llm-wiki.md`](docs/research-karpathy-llm-wiki.md) | What Karpathy actually said + community extensions, with sources. || [`research-agentmemory.md`](docs/research-agentmemory.md) | Deep-dive on the TypeScript predecessor; ideas to reuse and substrate to drop. || [`research-basic-memory.md`](docs/research-basic-memory.md) | The manual-write-note model we explicitly diverge from. |diff --git a/crates/ai-memory-consolidate/tests/recall_eval.rs b/crates/ai-memory-consolidate/tests/recall_eval.rsnew file mode 100644index 00000000..9ad526bb--- /dev/null+++ b/crates/ai-memory-consolidate/tests/recall_eval.rs@@ -0,0 +1,209 @@+//! Recall@5 eval harness.+//!+//! Loads a small hand-crafted corpus + a probe set, measures recall@5+//! against both the pure-FTS5 path and the hybrid (FTS5 + RRF + vector)+//! path, and asserts a baseline. The point is *the framework*: once the+//! harness is in CI, anyone can drop in real embeddings (set the+//! `AI_MEMORY_EMBEDDING_PROVIDER` env vars + plug in a real model) and+//! watch the numbers move. The synthetic embedder shipped for tests is+//! deterministic but not semantically aware, so for now hybrid ≈ FTS5+//! on most probes; we still log the numbers so regressions show up.+//!+//! The corpus mirrors the kind of pages an ai-memory wiki accrues over+//! a few sessions of building this project itself: session notes,+//! concept pages, decision logs.++use std::sync::Arc;++use ai_memory_core::{PagePath, Tier};+use ai_memory_llm::{Embedder, SyntheticEmbedder};+use ai_memory_store::Store;+use ai_memory_wiki::{Wiki, WritePageRequest};+use tempfile::TempDir;++// Note: bodies and queries deliberately avoid hyphens, because the+// FTS5 query parser treats a bare `-` as the NOT operator and our+// tokenizer keeps `-` inside tokens (so `single-writer` is one token+// rather than two). Both rules together make hyphenated terms a+// landmine for free-text search; we sidestep by spelling things out.+const CORPUS: &[(&str, &str)] = &[+ (+ "notes/writer_actor.md",+ "The single writer SQLite actor owns one Connection and serialises every \+ mutation through a tokio mpsc bounded channel for backpressure",+ ),+ (+ "notes/karpathy_wiki.md",+ "Karpathy LLM Wiki compile knowledge into a durable artifact instead of \+ retrieving from raw documents at query time",+ ),+ (+ "notes/decay_policy.md",+ "Retention sweep evicts cold episodic pages whose retention falls below \+ threshold semantic concept pages compound forever pinned exempt",+ ),+ (+ "notes/handoff.md",+ "Cross agent handoff lets Codex pick up the working directory where \+ Claude Code left off via memory_handoff_begin and memory_handoff_accept",+ ),+ (+ "notes/backup.md",+ "SQLite online backup API snapshots the database while writes continue \+ used by ai memory backup to produce a consistent tarball",+ ),+ (+ "notes/identity_tuple.md",+ "Identity is the three tuple workspace project path on every domain row \+ baked into the schema from M1 to avoid retrofit pain",+ ),+ (+ "notes/mcp_transport.md",+ "MCP server speaks JSON RPC over stdio or streamable HTTP via the rmcp \+ crate protocol version pinned to 2024 11 05",+ ),+ (+ "notes/hooks_pipeline.md",+ "Claude Code lifecycle hooks fire and forget POST to the hook endpoint \+ on the ai memory server with sub second timeouts",+ ),+ (+ "notes/consolidate_fanout.md",+ "Multi page atomic fan out the consolidator emits a ConsolidatedBatch \+ that ai memory wiki applies in one SQL transaction",+ ),+ (+ "notes/hybrid_search.md",+ "Hybrid retrieval reciprocal rank fusion combines FTS5 keyword ranks with \+ cosine similarity over stored embeddings k equals 60",+ ),+];++/// Each probe is (free-text query, ground-truth wiki path).+const PROBES: &[(&str, &str)] = &[+ ("tokio mpsc backpressure", "notes/writer_actor.md"),+ ("compile knowledge artifact", "notes/karpathy_wiki.md"),+ ("retention sweep cold pages", "notes/decay_policy.md"),+ ("Codex Claude handoff", "notes/handoff.md"),+ ("SQLite online backup tarball", "notes/backup.md"),+ (+ "workspace project tuple identity",+ "notes/identity_tuple.md",+ ),+ ("rmcp JSON RPC stdio", "notes/mcp_transport.md"),+ ("lifecycle POST hook timeout", "notes/hooks_pipeline.md"),+ (+ "ConsolidatedBatch SQL transaction",+ "notes/consolidate_fanout.md",+ ),+ ("reciprocal rank fusion cosine", "notes/hybrid_search.md"),+];++const RECALL_FLOOR: f64 = 0.70;++#[tokio::test]+async fn recall_at_5_baseline() {+ let tmp = TempDir::new().expect("tempdir");+ let store = Store::open(tmp.path()).expect("open store");+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .expect("ws");+ let proj = store+ .writer+ .get_or_create_project(ws, "eval", None)+ .await+ .expect("proj");+ let embedder: Arc<dyn Embedder> = Arc::new(SyntheticEmbedder::new(128));+ let wiki = Wiki::new(tmp.path(), store.writer.clone())+ .expect("wiki")+ .with_embedder(embedder.clone());++ for (path, body) in CORPUS {+ wiki.write_page(WritePageRequest {+ workspace_id: ws,+ project_id: proj,+ path: PagePath::new(path.to_string()).expect("path"),+ frontmatter: serde_json::json!({"title": path}),+ body: body.to_string(),+ tier: Tier::Semantic,+ pinned: false,+ title: None,+ })+ .await+ .expect("write page");+ }++ let fts_recall = measure_recall(&store, ws, proj, None).await;+ let hybrid_recall = measure_recall(&store, ws, proj, Some(embedder.clone())).await;++ // Recall numbers are persisted to stderr so CI logs always show+ // them, even when the run is green. Watch these drift over time.+ eprintln!("recall_eval: FTS5={fts_recall:.3}, hybrid={hybrid_recall:.3}");++ assert!(+ fts_recall >= RECALL_FLOOR,+ "pure-FTS5 recall@5 below baseline: got {fts_recall:.3} (floor {RECALL_FLOOR})"+ );+ assert!(+ hybrid_recall >= RECALL_FLOOR,+ "hybrid recall@5 below baseline: got {hybrid_recall:.3} (floor {RECALL_FLOOR})"+ );++ // RRF should not *hurt* recall vs the FTS5-only path on this+ // corpus. We don't assert hybrid > FTS5 because the synthetic+ // embedder doesn't add real semantic signal — that's a feature of+ // the test, not a bug.+ assert!(+ hybrid_recall + 1e-6 >= fts_recall,+ "hybrid should never regress vs FTS5: FTS5={fts_recall:.3} hybrid={hybrid_recall:.3}",+ );+}++async fn measure_recall(+ store: &Store,+ ws: ai_memory_core::WorkspaceId,+ proj: ai_memory_core::ProjectId,+ embedder: Option<Arc<dyn Embedder>>,+) -> f64 {+ let mut hits = 0_usize;+ for (query, expected) in PROBES {+ let results = if let Some(emb) = &embedder {+ let qv = emb.embed(query).await.expect("embed query");+ store+ .reader+ .hybrid_search(+ ws,+ proj,+ (*query).to_string(),+ Some(qv),+ emb.provider().to_string(),+ emb.model().to_string(),+ emb.dim(),+ 5,+ )+ .await+ .expect("hybrid search")+ } else {+ store+ .reader+ .search_pages((*query).to_string(), 5)+ .await+ .expect("search")+ };+ let found = results.iter().any(|r| r.path.as_str() == *expected);+ if found {+ hits += 1;+ } else {+ eprintln!(+ " miss: query={query:?} expected={expected:?} top5={:?}",+ results.iter().map(|r| r.path.as_str()).collect::<Vec<_>>()+ );+ }+ }+ #[allow(clippy::cast_precision_loss)]+ {+ hits as f64 / PROBES.len() as f64+ }+}diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdnew file mode 100644index 00000000..2e8cf219--- /dev/null+++ b/docs/ARCHITECTURE.md@@ -0,0 +1,286 @@+# ai-memory — Architecture++> One canonical doc for "what is this thing and how is it shaped".+> Long-form research lives next to this file under [`docs/`](.); this+> page is the operational summary for someone reading the code.++## Purpose++ai-memory is a single Rust binary that gives AI coding agents (Claude+Code, OpenAI Codex, OpenCode) long-term memory shared across CLIs.+Quit one mid-task; open another in the same directory; continue. No+manual `write_note` ceremony, no copy-pasting summaries between+sessions.++The artifact you accrete is a **Karpathy-style LLM wiki**: a+git-versioned tree of markdown pages on disk that gets *compiled* over+time, not just appended-to. Pages are versioned in place via+supersession, semantic concepts compound, episodic logs decay. A+companion SQLite index gives FTS5 + optional vector retrieval; the+markdown stays the source of truth.++## Data flow++```+ ┌──────────────────────┐+ │ Claude Code / Codex │+ │ / OpenCode CLI │+ └──────────┬───────────┘+ lifecycle hooks │ stdio+ (fire-and-forget HTTP) ▼+ ┌────────────────────────────────────────────┐+ │ ai-memory serve (one binary) │+ │ ┌────────────────┐ ┌────────────────────┐ │+ │ │ /hook router │ │ MCP server (rmcp) │ │+ │ │ (ai-memory- │ │ stdio + streamable │ │+ │ │ hooks) │ │ HTTP /mcp │ │+ │ └────────┬───────┘ └─────────┬──────────┘ │+ │ │ sanitize │ tools/call │+ │ ▼ ▼ │+ │ ┌────────────────────────────┐ │+ │ │ WriterHandle (mpsc) │ │+ │ │ single dedicated thread, │ │+ │ │ one rusqlite Connection, │ │+ │ │ WAL mode, busy_timeout 5s │ │+ │ └────────┬───────────┬───────┘ │+ │ │ │ │+ │ ┌─────────────▼──┐ ┌────▼──────────────┐ │+ │ │ markdown wiki/ │ │ SQLite db/ │ │+ │ │ (source of │ │ memory.sqlite │ │+ │ │ truth + git) │ │ (derived index) │ │+ │ └────────────────┘ │ - pages + FTS5 │ │+ │ ▲ │ - observations │ │+ │ watcher │ │ - handoffs │ │+ │ ▼─────────┘ │ - page_embeddings│ │+ │ reconciliation │ - audit_log │ │+ │ every 30s └───────────────────┘ │+ └────────────────────────────────────────────┘+ ▲ ▲+ │ scheduled / on-demand jobs │+ ┌─────┴───────────┐ ┌────────┴─────┐+ │ ai-memory │ │ ai-memory │+ │ consolidate │ │ forget-sweep │+ │ (LLM-driven) │ │ (M8 decay) │+ └─────────────────┘ └──────────────┘+ ┌─────────────────┐ ┌──────────────┐+ │ ai-memory lint │ │ ai-memory │+ │ (rule + LLM) │ │ embed (M9 │+ └─────────────────┘ │ backfill) │+ └──────────────┘+```++**Steady-state loop:**++1. Agent CLI emits a lifecycle hook (SessionStart, UserPromptSubmit,+ PostToolUse, …) → vendored shell script `curl`s the event JSON to+ `POST /hook` with a sub-second timeout. Agent never blocks.+2. Server's hook router sanitises the payload (the only path from+ untrusted text into the store), assigns an [`ObservationKind`], and+ enqueues a `WriteCmd` to the writer actor. `log.md` gets an+ appended `## [YYYY-MM-DDTHH:MM:SSZ] <event> | <title>` line.+3. On `SessionEnd`, the server synthesises a `sessions/<id>.md`+ summary page (rule-based, no LLM) and opens a `Handoff` row for the+ next agent. Auto-commits the wiki.+4. When `AI_MEMORY_LLM_PROVIDER` is set, `memory_consolidate` rewrites+ that summary into a richer durable page or fans out into a+ multi-page batch under `concepts/`, `decisions/`, `gotchas/`.+5. `memory_query` answers via FTS5; when an embedder is configured,+ via RRF fusion of FTS5 + cosine over `page_embeddings`. Every+ query hit bumps `access_count` + `last_accessed_at` on the page —+ the M8 reinforcement term.+6. The forget sweep runs on demand (or on a future schedule): pages+ with `retention < cold_threshold` are soft-deleted; soft-deletions+ older than `hard_delete_after_days` with no subsequent access get+ purged. Semantic / pinned / freshly-touched pages survive.+7. Backups: `ai-memory backup --to <tarball>` uses SQLite's online+ backup API so the source stays writable; `ai-memory restore`+ reverses. Or: `git push` the wiki dir + `rsync` the data dir.++## Storage architecture++**Two layers, one source of truth.**++* `<data_dir>/wiki/` — markdown source of truth. Owned by a `git2`+ repo so every consolidation pass + every session-end produces a+ durable commit. Editable by hand in Obsidian / vim — the watcher+ reconciles outside edits.+* `<data_dir>/db/memory.sqlite` — derived index. WAL mode. One+ writer actor owns the writer `Connection`; reads go through a+ cloneable read-only pool.+* `<data_dir>/raw/` — reserved for raw session log archives (future).+* `<data_dir>/logs/` — rolling daily `tracing` output.+* `<data_dir>/models/` — reserved for bundled embedding models+ (M9.5+, when local `ort` lands).++**Schema (current head):**++| Table | What |+|---|---|+| `workspaces`, `projects` | Top of the 3-tuple identity coordinate. |+| `pages` | Versioned wiki pages with `is_latest` + `supersedes` chain. M8 columns: `last_accessed_at`, `access_count`, `superseded_at`. M9 cols: `embedding_provider`, `embedding_model`, `embedding_dim`. |+| `pages_fts` | FTS5 virtual table over `(title, body)`, auto-synced by triggers. |+| `sessions`, `observations` | Hook capture, full audit log. |+| `links` | Wiki cross-references with `to_page_id` nullable for unresolved. |+| `handoffs` | Typed cross-agent handoff records (open / accepted / expired). |+| `page_embeddings` | One vector per latest page, with `(provider, model, dim)` denormalised for refuse-on-mismatch checks. |+| `audit_log` | Every mutation, addressable by `at DESC`. |++**Memory tiers (M8 policy):**++| Tier | Lifetime | Decay |+|---|---|---|+| Working | Current session only | Hard-drop on session end (kept in `observations` for forensics) |+| Episodic | 30d hot → 180d cold → evict | `salience · exp(−λΔt) + σ · log(1+access_count) · exp(−μ · days_since_access)` |+| Semantic | Indefinite | None — only supersedeable via M7 LLM rewrite |+| Procedural | Indefinite | Frequency-decay if not re-observed |++Pinned pages (`pinned: true` in frontmatter) are exempt from all+decay paths.++## Crate layout++```+crates/+├── ai-memory-core/ domain types, errors, ids. NO IO.+├── ai-memory-store/ SQLite + writer actor + reader pool + decay math.+├── ai-memory-wiki/ atomic markdown writes, file watcher, git.+├── ai-memory-mcp/ rmcp transport + tool router.+├── ai-memory-hooks/ payload schemas, sanitiser, /hook ingress.+├── ai-memory-llm/ LlmProvider + Embedder traits + 3 providers + 2 embedders.+├── ai-memory-consolidate/ Karpathy ingest / lint / sweep pipeline.+└── ai-memory-cli/ `ai-memory` binary entry point + 17 subcommands.+```++Each crate has a single responsibility and exposes a typed API. No+circular deps. Inter-crate boundaries enforce the cross-cutting+invariants below.++## MCP tool surface (10 tools)++| Tool | Hint | Purpose |+|---|---|---|+| `memory_query` | read-only | FTS5 or hybrid RRF search. Bumps access counters. |+| `memory_recent` | read-only | Most-recently-updated `is_latest=1` pages. |+| `memory_status` | read-only | Counts, paths, version. |+| `memory_handoff_begin` | destructive | Open a handoff for the next agent. |+| `memory_handoff_accept` | destructive | Fetch + ack the latest open handoff (auto-cwd-matched). |+| `memory_consolidate` | destructive | LLM-driven page rewrite. `multi_page=true` for atomic fan-out. |+| `memory_forget_sweep` | destructive | M8 retention pass. `dry_run=true` for preview. |+| `memory_lint` | destructive | Rule-based + LLM contradiction findings → `wiki/_lint/`. |++All MCP tools carry `readOnlyHint` / `destructiveHint` annotations so+the calling agent can pick safely. Parameter aliases (`query|q|search`,+`workspace|ws`) absorb LLM verbal variance.++## CLI subcommand surface (17 commands)++```+init watch embed+status serve forget-sweep+search reset lint+write-page backup commit+ restore llm-test+ install-hooks+```++Run `ai-memory --help` for the full tree.++## Cross-cutting invariants++Carved in M0/M1; every milestone has to respect them. Each comes from+a documented prior-art bug; cite the source when reviewing changes+that touch the relevant area.++1. **One config-read path.** `Config::load()` called once at startup.+ No `std::env::var` outside it. (agentmemory #456 / #469.)+2. **Single-writer SQLite actor.** All writes go through one `mpsc`+ channel to one dedicated OS thread. (cognee #2717.)+3. **Indexes commit in the same transaction as the data.** No+ background-task-indexing-after-return. (basic-memory #763 / #578.)+4. **Typed 3-tuple identity** (`workspace_id`, `project_id`, path)+ in every domain row from day one. (basic-memory #783 / #834.)+5. **Hooks are fire-and-forget.** Hook scripts hard-timeout at+ ≤200 ms; server returns 202 immediately. (agentmemory #221 / #143.)+6. **Privacy strip is a typed boundary.** `Sanitized<NewObservation>`+ has no other constructor than `sanitize()`. (design-decisions §14.)+7. **JSON-schema structured outputs only.** Native provider JSON+ modes; no XML, no Instructor wrapping. (agentmemory #492 / #539,+ cognee #2840.)+8. **`{provider, model, dim}` denormalised next to every embedding.**+ Refuse-on-mismatch at startup. (agentmemory #469.)+9. **Live-process check before destructive ops.** `ai-memory reset`,+ `backup`, `restore` all consult `sysinfo`. (basic-memory #765.)+10. **Atomic file writes** (tmp + rename + fsync). Watcher ignores+ own writes by filename prefix.+11. **Absolute canonical data dir** default; logged loudly on+ startup. (agentmemory #303.)+12. **No global singletons / `lazy_static` configs.** All deps+ explicit. (cognee #2228.)+13. **Zero-LLM default path.** LLM features opt-in via env. The+ system works without any provider configured.+14. **Tracing subscribers explicitly filter their own module.**+ No feedback loops. (agentmemory #519.)++## Configuration (`config.toml`)++Lives at `<data_dir>/config.toml`. All values overridable by env vars+prefixed `AI_MEMORY_*`.++```toml+bind = "127.0.0.1:7777"+log_level = "info"++[decay] # M8 retention params+lambda = 0.02 # ↓ to forget less aggressively+sigma = 0.6 # ↑ to reward query-hits more+mu = 0.04 # ↑ if recent hits should count more+cold_threshold = 0.20 # below this → soft-delete+hard_delete_after_days = 180+```++**LLM provider env** (opt-in):+```+AI_MEMORY_LLM_PROVIDER anthropic | openai | openai-compat+AI_MEMORY_LLM_MODEL e.g. claude-sonnet-4-7+ANTHROPIC_API_KEY / OPENAI_API_KEY / LLM_API_KEY+AI_MEMORY_LLM_BASE_URL for openai-compat (Ollama, vLLM)+```++**Embedder env** (opt-in):+```+AI_MEMORY_EMBEDDING_PROVIDER openai | voyage+AI_MEMORY_EMBEDDING_MODEL e.g. text-embedding-3-small+AI_MEMORY_EMBEDDING_DIM 1536+OPENAI_API_KEY / VOYAGE_API_KEY+```++## Future work++* **M9.5 — local embeddings via `ort`.** Bundle `bge-small-en-v1.5`+ for an API-key-free homelab path. ~200 MB image bloat; trait is+ ready, just needs the `OrtBgeSmallEmbedder` impl + tokenizer wiring.+* **`sqlite-vec` integration.** Brute-force cosine works fine to a few+ thousand pages; past that, the `sqlite-vec` extension is the next+ step. Deferred behind workaround for rusqlite-compat (issue #206).+* **Scheduled forget-sweep / lint.** Currently manual / on-demand via+ the MCP tools or CLI. The next iteration runs both on a 6-hour+ `tokio::interval` inside `serve`.+* **Multi-workspace UI / web dashboard.** Out of scope for v1; revisit+ once the headless server has been load-tested.+* **Real LongMemEval-S harness.** The recall-eval framework exists+ ([`crates/ai-memory-consolidate/tests/recall_eval.rs`](../crates/ai-memory-consolidate/tests/recall_eval.rs));+ porting LongMemEval-S itself requires the dataset.++## Reading order++* This file — operational summary, you are here.+* [`docs/design-decisions.md`](design-decisions.md) — the full v1 spec.+* [`docs/research-karpathy-llm-wiki.md`](research-karpathy-llm-wiki.md)+ — what "Karpathy-faithful" means.+* [`docs/research-agentmemory.md`](research-agentmemory.md),+ [`research-basic-memory.md`](research-basic-memory.md),+ [`research-cognee.md`](research-cognee.md) — prior art studied.+* [`docs/issues-*.md`](.) — concrete failure modes we've designed to+ avoid.+* [`CLAUDE.md`](../CLAUDE.md) — per-session operating rules pinned+ into Claude Code conversations.