Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

docs: humanize README and docs

AkitaOnRailsc36e620ba073
diff --git a/README.md b/README.md
index 60829ff4..4a592d69 100644
--- a/README.md
+++ b/README.md
@@ -23,7 +23,7 @@ pages get rewritten as a coherent narrative; when the next agent
starts (Claude Code, Codex, OpenCode, …) it sees a handoff with
"where you left off" already prepended.
-The wiki is plain markdown in a git repo — `grep`-able, openable in
+The wiki is plain markdown in a git repo - `grep`-able, openable in
Obsidian, backed up with `rsync`. No vector database to babysit, no
`write_note` ceremony, no manual context-loading. The full design is
in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md); the influences and
@@ -34,7 +34,7 @@ priors are at the [bottom](#influences-and-prior-art).
- **Zero-friction capture.** Lifecycle hooks fire-and-forget every
prompt + tool call + session boundary. You never type `write_note`.
- **Cross-agent handoffs.** Quit Claude Code mid-task, start Codex
- in the same directory hours later — the next agent sees a
+ in the same directory hours later - the next agent sees a
"where you left off" block before its first prompt.
- **Per-project isolation by construction.** Each project lives at
`<wiki_root>/<workspace_id>/<project_id>/…` keyed by stable UUIDs.
@@ -44,7 +44,7 @@ priors are at the [bottom](#influences-and-prior-art).
at session-end (or PreCompact), not retrieved over raw logs.
Supersession chain + git-versioned markdown means you can
time-travel with `git log`.
-- **Built-in `/web` browser.** Read-only HTML UI for the wiki —
+- **Built-in `/web` browser.** Read-only HTML UI for the wiki -
project list, folder tree, FTS5 search, markdown rendering, dark
mode. Mounted on the same axum server as MCP.
- **Multi-agent + multi-machine ready.** Supported clients: Claude
@@ -53,7 +53,7 @@ priors are at the [bottom](#influences-and-prior-art).
box (LAN/VPN/cloud) with bearer-token auth.
- **Thin-client CLI.** `ai-memory bootstrap`, `purge-project`,
`rename-project`, `lint`, `embed`, `forget-sweep`, `backup` are
- all HTTP clients of the running server — never touch SQLite or
+ all HTTP clients of the running server - never touch SQLite or
wiki files directly. Server is the single source of truth.
- **LLM is opt-in.** Zero-LLM mode still gives you FTS5 search +
rule-based summarisation. Add a provider when you want consolidated
@@ -67,7 +67,7 @@ priors are at the [bottom](#influences-and-prior-art).
next steps, and a session summary.
- **"What did we decide about X six weeks ago?"** Type
`memory_query X` from the agent (or `ai-memory search X` from a
- terminal) — FTS5 over the wiki. Pages are LLM-consolidated, so
+ terminal) - FTS5 over the wiki. Pages are LLM-consolidated, so
the hit is a coherent decision page, not a raw chat log.
- **"This new project has months of history before ai-memory."**
`cd ~/Projects/<that-project> && ai-memory bootstrap` collects
@@ -80,7 +80,7 @@ priors are at the [bottom](#influences-and-prior-art).
pages cleanly separated; the `/web` UI is reachable from a
browser anywhere on the LAN.
- **"Audit what landed before sharing with a teammate."** Browse
- the wiki at `http://<server>:49374/web` — HTTP Basic dialog if
+ the wiki at `http://<server>:49374/web` - HTTP Basic dialog if
auth is on, paste the token as password. Per-project tree view,
rendered markdown, supersession chain visible per page.
- **"Drop an experiment, keep the rest."**
@@ -93,7 +93,7 @@ priors are at the [bottom](#influences-and-prior-art).
You need: Docker + an agent CLI (Claude Code, Codex, OpenCode, Cursor,
or anything else that speaks MCP).
-The default quick-start has **no authentication** — the server binds
+The default quick-start has **no authentication** - the server binds
to loopback only, so on a single-user laptop nothing else can reach
it. Adding a bearer token is a one-line change once you're ready to
expose the server on the LAN; see [Security](#security) below.
@@ -135,13 +135,13 @@ ai-memory install-mcp --client claude-code --apply
ai-memory install-hooks --agent claude-code --apply
```
-That's it. Start a Claude Code session as usual — every prompt and
+That's it. Start a Claude Code session as usual - every prompt and
tool call now lands in ai-memory, and the next session you open in
this project will see a handoff with where you left off.
The `install-mcp` / `install-hooks` commands default to
`http://127.0.0.1:49374` (matching the server above) and no bearer
-token. Both are idempotent — re-runs replace ai-memory's entry,
+token. Both are idempotent - re-runs replace ai-memory's entry,
preserve every other server / hook you have configured, and write a
timestamped `.bak-<ts>` next to the file before each modifying
write. The hook scripts are staged into
@@ -178,7 +178,7 @@ replacement), (2) `docker pull`s the latest image, (3) re-stages
hook scripts under `~/.local/share/ai-memory/hooks/<agent>/` for
every agent you've configured, and (4) tells you how to restart the
server container so the new binary is picked up. The hook refresh
-is idempotent — re-running `install-hooks --apply` replaces the
+is idempotent - re-running `install-hooks --apply` replaces the
seven keys ai-memory owns and leaves every other hook the user has
wired up alone. Set `AI_MEMORY_NO_VERSION_CHECK=1` to silence the
daily check, or `AI_MEMORY_WRAPPER_URL=<url>` to pin the self-upgrade
@@ -199,16 +199,16 @@ source (e.g. a fork or a tagged release).
> read-only); daily use from inside the sandbox needs no binary at
> all (agents reach ai-memory over MCP).
-**For everything else** — Codex, OpenCode, Cursor, Claude Desktop,
+**For everything else** - Codex, OpenCode, Cursor, Claude Desktop,
Gemini CLI, OpenClaw, the curl-based hook installer (no docker
needed), running ai-memory without docker, the full subcommand
-reference, the homelab deploy pattern, security hardening — see
+reference, the homelab deploy pattern, security hardening - see
[**`docs/install.md`**](docs/install.md).
## Security
The default Quick start runs **without authentication** because the
-server is bound to loopback (`127.0.0.1:49374`) — no process outside
+server is bound to loopback (`127.0.0.1:49374`) - no process outside
this machine can reach it. That's the safest default for a personal
laptop and matches the "single-user, single-machine" use case the
project is optimised for.
@@ -287,7 +287,7 @@ sending requests to the server using a different hostname.
### Browser access to `/web`
When the server has **no** bearer token set, visit
-`http://<host>:49374/web` in any browser — no prompt.
+`http://<host>:49374/web` in any browser - no prompt.
When `AI_MEMORY_AUTH_TOKEN` is set, the browser shows a native HTTP
Basic dialog on first visit. Leave the username blank (or any value)
@@ -301,7 +301,7 @@ clients continue to use `Authorization: Bearer <token>`.
## Configuring the CLI
The `ai-memory` binary is a thin HTTP client. It never opens the
-wiki or SQLite directly — every state-touching command goes through
+wiki or SQLite directly - every state-touching command goes through
the running server, which is the sole writer.
Configuration is two environment variables, both **optional**:
@@ -309,7 +309,7 @@ Configuration is two environment variables, both **optional**:
| Variable | Default | When to set it |
|---|---|---|
| `AI_MEMORY_SERVER_URL` | `http://127.0.0.1:49374` | When the server runs somewhere other than this machine (e.g. a homelab at `http://192.168.0.90:49374`). |
-| `AI_MEMORY_AUTH_TOKEN` | unset (no auth) | When the server has bearer auth enabled — see [Security](#security). |
+| `AI_MEMORY_AUTH_TOKEN` | unset (no auth) | When the server has bearer auth enabled - see [Security](#security). |
For the **single-laptop local case** (scenarios A/B) you don't need
either env var: the CLI talks to the loopback server and just works.
@@ -325,12 +325,12 @@ export AI_MEMORY_AUTH_TOKEN="b9a5075d…" # only when server has auth enabled
```
Explicit flags (`--auth-token`, `--server-url`) on `install-mcp` /
-`install-hooks` override env vars when both are set — useful when
+`install-hooks` override env vars when both are set - useful when
you're generating configs for a client that talks to a different
server than the CLI default.
The `init`, `serve`, `install-*`, `generate-auth-token`, and
-`setup-agent` subcommands don't need these env vars — they either
+`setup-agent` subcommands don't need these env vars - they either
set up local files or start the server itself.
## How it works in practice
@@ -370,7 +370,7 @@ even though its raw history is gone.
If you're installing ai-memory in a project you've been working on
for months, the wiki starts empty and the first few sessions are
-net-zero — you're populating, not retrieving. `ai-memory bootstrap`
+net-zero - you're populating, not retrieving. `ai-memory bootstrap`
solves that by LLM-summarising your existing `git log`, README,
`docs/`, and module-level doc-comments into seed wiki pages.
@@ -385,7 +385,7 @@ ai-memory bootstrap
```
The workspace defaults to `default` and the project defaults to the
-current directory's basename — that's almost always what you want, so
+current directory's basename - that's almost always what you want, so
omit `--workspace` / `--project` unless you're deliberately overriding.
Bootstrap produces a per-project `bootstrap.md` manifest (under
@@ -416,7 +416,7 @@ For a more navigable view, start the server with `--enable-web` and
open `http://<host>:49374/web` in any browser. Project-list homepage,
per-project page tree with breadcrumbs, rendered markdown with syntax
highlighting and metadata (tier, kind, pinned, supersedes chain),
-plus FTS5 search — all read-only, no editing. Light/dark theme
+plus FTS5 search - all read-only, no editing. Light/dark theme
follows your OS setting via `prefers-color-scheme`.
![Project list homepage with the LLM-optimised banner; four projects (distrobox-gaming, ai-memory, nes-to-sms, .config) shown as cards with page counts + last activity.](docs/web-projects-home.png)
@@ -435,18 +435,18 @@ ai-memory serve --transport http --bind 127.0.0.1:49374 --enable-web
The web routes are mounted at `/web` on the same axum server as the
MCP endpoint. When the server has bearer auth enabled, the browser
-shows a native HTTP Basic dialog on first visit — leave the username
+shows a native HTTP Basic dialog on first visit - leave the username
blank (or any value) and paste the token as the password; the cookie
persists for 30 days so subsequent navigation doesn't re-prompt.
Loopback-bound servers with no token need no credentials at all. See
[Security → Browser access to /web](#browser-access-to-web) above.
-### Rules vs facts — ai-memory tells you when something belongs in CLAUDE.md
+### Rules vs facts - ai-memory tells you when something belongs in CLAUDE.md
When you type something like "don't forget to never add a function
without a unit test", that's a **durable project rule**, not a
session-level observation. Rules need to fire on every relevant
-action — that's what your project's `CLAUDE.md` / `AGENTS.md` is for
+action - that's what your project's `CLAUDE.md` / `AGENTS.md` is for
(it's loaded into the agent's system prompt every turn), while
ai-memory queries only fire when the agent thinks to call them.
@@ -460,15 +460,15 @@ the agent sees a suggestion:
> project's CLAUDE.md / AGENTS.md so the agent sees it on every
> turn, not just when it remembers to call memory_query.
-ai-memory never edits your `CLAUDE.md` itself — the suggestion is
+ai-memory never edits your `CLAUDE.md` itself - the suggestion is
the whole UX. You copy what's useful, ignore what isn't.
### When the agent reaches for memory
Hooks handle *capture* (every prompt + tool call + session boundary)
and *handoff resume* (SessionStart auto-fetches pending handoffs)
-without you typing anything. **Proactive querying** — the agent
-reaching into the wiki on its own — depends on the agent knowing
+without you typing anything. **Proactive querying** - the agent
+reaching into the wiki on its own - depends on the agent knowing
*when* to call which tool. Concrete patterns once the routing
snippet is installed (see "Nudge the agent" below):
@@ -491,7 +491,7 @@ the next section installs.
The capture side works automatically. For the *query* side, the
agent needs a routing table in the project's rules file
(`CLAUDE.md` for Claude Code; `AGENTS.md` for Codex / OpenCode /
-Cursor / Gemini CLI). Two ways to install it — pick whichever's
+Cursor / Gemini CLI). Two ways to install it - pick whichever's
easier in the moment:
**From the agent** (no terminal needed):
@@ -516,7 +516,7 @@ ai-memory install-instructions --print # preview without writing
The CLI's auto-detect: if `$PWD/CLAUDE.md` exists, extend that;
if `$PWD/AGENTS.md` exists, extend that; if both exist, write to
-both (multi-agent project — keep both files in sync); if neither
+both (multi-agent project - keep both files in sync); if neither
exists, create `CLAUDE.md` and print a hint about `--target
AGENTS.md` for non-Claude agents.
@@ -524,12 +524,12 @@ Both paths produce the same block. Both replace existing markered
blocks in place rather than duplicating, so you can re-run safely
whenever the snippet evolves (e.g. when a new MCP tool ships).
-## LLM provider — recommended defaults
+## LLM provider - recommended defaults
You can run ai-memory entirely without an LLM (FTS5 search +
rule-based summaries, $0). When you *do* configure one, the
options below are ranked by fitness for ai-memory's
-consolidation workload — see
+consolidation workload - see
[`docs/llm-provider-comparison.md`](docs/llm-provider-comparison.md)
for the empirical writeup behind this ranking.
@@ -539,7 +539,7 @@ for the empirical writeup behind this ranking.
> local LLM server and prefer $0 / fully-self-hosted. The
> three are interchangeable; pick once and forget.
-### Option 1 — Claude Haiku 4.5 *(recommended default)*
+### Option 1 - Claude Haiku 4.5 *(recommended default)*
Best balance of speed (~7 s), restraint, and classification
quality. The only model that consistently classifies durable
@@ -563,7 +563,7 @@ AI_MEMORY_LLM_MODEL=anthropic/claude-haiku-4.5
LLM_API_KEY=sk-or-v1-…
```
-### Option 2 — OpenAI GPT-5.4-mini *(cheaper alternative)*
+### Option 2 - OpenAI GPT-5.4-mini *(cheaper alternative)*
~5× cheaper than Haiku, ~2× faster (~4 s avg). Same parse
reliability, same faithfulness. One known weakness: mild
@@ -586,13 +586,13 @@ AI_MEMORY_LLM_MODEL=openai/gpt-5.4-mini
LLM_API_KEY=sk-or-v1-…
```
-### Option 3 — Local Ollama qwen3:32b *(free / self-hosted)*
+### Option 3 - Local Ollama qwen3:32b *(free / self-hosted)*
$0 per consolidation. Requires a machine with at least ~24 GB
of unified or VRAM memory to keep the Q4_K_M weights warm
(~20 GB) plus headroom. Strix Halo / Apple Silicon / a
recent NVIDIA card all work. Latency is ~90 s but
-consolidation is a background job — users never see it.
+consolidation is a background job - users never see it.
One-time setup on the Ollama host:
@@ -615,17 +615,17 @@ LLM_API_KEY=ollama-local # any non-empty value; Ollama doesn't
```
If you bind ai-memory to a non-loopback address, also set
-`AI_MEMORY_ALLOWED_HOSTS` — see [Security → DNS-rebinding guard](#dns-rebinding-guard-ai_memory_allowed_hosts).
+`AI_MEMORY_ALLOWED_HOSTS` - see [Security → DNS-rebinding guard](#dns-rebinding-guard-ai_memory_allowed_hosts).
### What we don't recommend
-- **Claude Sonnet 4.5** — strictly dominated by Haiku for
+- **Claude Sonnet 4.5** - strictly dominated by Haiku for
this task: same parse reliability, 3× cost, hallucinated
details before the prompt was tightened. Use it only if
you specifically need extended reasoning (e.g. cross-page
lint sweeps).
- **Reasoning-mode models** (Kimi-K2.6, Claude with extended
- thinking enabled, GPT-o3, Gemini "thinking" variants) —
+ thinking enabled, GPT-o3, Gemini "thinking" variants) -
these models burn `max_tokens` budget on internal
reasoning before emitting visible content; with the
strict-JSON consolidation prompt they hang or emit empty
@@ -634,7 +634,7 @@ If you bind ai-memory to a non-loopback address, also set
### Embedding provider
The LLM provider drives consolidation + lint. Embeddings are
-a *separate* concern (hybrid retrieval over the wiki — BM25
+a *separate* concern (hybrid retrieval over the wiki - BM25
+ vector RRF). Defaults when `AI_MEMORY_EMBEDDING_PROVIDER`
is set:
@@ -653,7 +653,7 @@ AI_MEMORY_EMBEDDING_DIM=768
OPENAI_API_KEY=ollama-local
```
-Skipping the embedding provider entirely is fine —
+Skipping the embedding provider entirely is fine -
`memory_query` falls back to pure FTS5 (BM25) and still
works; you just lose vector re-ranking.
@@ -702,15 +702,15 @@ data-flow diagram + crate breakdown + cross-cutting invariants.
## Influences and prior art
-- **[Karpathy LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)** — the compile-not-retrieve pattern.
-- **[agentmemory](https://github.com/rohitg00/agentmemory)** — most of the right ideas; this project is the Rust successor.
-- **[basic-memory](https://github.com/basicmachines-co/basic-memory)** — the markdown-on-disk source-of-truth model.
-- **[cognee](https://github.com/topoteretes/cognee)** — pipeline composition and triplet embeddings.
-- **[A-MEM](https://arxiv.org/abs/2502.12110)** — Zettelkasten-style atomic notes with link evolution.
+- **[Karpathy LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)** - the compile-not-retrieve pattern.
+- **[agentmemory](https://github.com/rohitg00/agentmemory)** - most of the right ideas; this project is the Rust successor.
+- **[basic-memory](https://github.com/basicmachines-co/basic-memory)** - the markdown-on-disk source-of-truth model.
+- **[cognee](https://github.com/topoteretes/cognee)** - pipeline composition and triplet embeddings.
+- **[A-MEM](https://arxiv.org/abs/2502.12110)** - Zettelkasten-style atomic notes with link evolution.
## License
-MIT — see [LICENSE](LICENSE).
+MIT - see [LICENSE](LICENSE).
## Acknowledgements
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index 87c452c9..835e255e 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -1,4 +1,4 @@
-# ai-memory — Architecture
+# ai-memory - Architecture
> One canonical doc for "what is this thing and how is it shaped".
> Long-form research lives next to this file under [`docs/`](.); this
@@ -14,7 +14,7 @@ sessions.
The artifact you accrete is a **Karpathy-style LLM wiki**: a
git-versioned tree of markdown pages on disk that gets *compiled* over
-time, not just appended-to. Pages are versioned in place via
+time, appended-to. Pages are versioned in place via
supersession, semantic concepts compound, episodic logs decay. A
companion SQLite index gives FTS5 + optional vector retrieval; the
markdown stays the source of truth.
@@ -86,7 +86,7 @@ markdown stays the source of truth.
multi-page batch under `concepts/`, `decisions/`, `gotchas/`.
5. `memory_query` answers via FTS5; when an embedder is configured,
via RRF fusion of FTS5 + cosine over `page_embeddings`. Every
- query hit bumps `access_count` + `last_accessed_at` on the page —
+ query hit bumps `access_count` + `last_accessed_at` on the page -
the M8 reinforcement term.
6. The forget sweep runs on demand (or on a future schedule): pages
with `retention < cold_threshold` are soft-deleted; soft-deletions
@@ -100,16 +100,16 @@ markdown stays the source of truth.
**Two layers, one source of truth.**
-* `<data_dir>/wiki/` — markdown source of truth. Owned by a `git2`
+* `<data_dir>/wiki/` - markdown source of truth. Owned by a `git2`
repo so every consolidation pass + every session-end produces a
- durable commit. Editable by hand in Obsidian / vim — the watcher
+ durable commit. Editable by hand in Obsidian / vim - the watcher
reconciles outside edits.
-* `<data_dir>/db/memory.sqlite` — derived index. WAL mode. One
+* `<data_dir>/db/memory.sqlite` - derived index. WAL mode. One
writer actor owns the writer `Connection`; reads go through a
cloneable read-only pool.
-* `<data_dir>/raw/` — reserved for raw session log archives (future).
-* `<data_dir>/logs/` — rolling daily `tracing` output.
-* `<data_dir>/models/` — reserved for bundled embedding models
+* `<data_dir>/raw/` - reserved for raw session log archives (future).
+* `<data_dir>/logs/` - rolling daily `tracing` output.
+* `<data_dir>/models/` - reserved for bundled embedding models
(M9.5+, when local `ort` lands).
**Schema (current head):**
@@ -131,7 +131,7 @@ markdown stays the source of truth.
|---|---|---|
| Working | Current session only | Hard-drop on session end (kept in `observations` for forensics) |
| Episodic | 30d hot → 180d cold → evict | `salience · exp(−λΔt) + σ · log(1+access_count) · exp(−μ · days_since_access)` |
-| Semantic | Indefinite | None — only supersedeable via M7 LLM rewrite |
+| Semantic | Indefinite | None - only supersedeable via M7 LLM rewrite |
| Procedural | Indefinite | Frequency-decay if not re-observed |
Pinned pages (`pinned: true` in frontmatter) are exempt from all
@@ -216,7 +216,7 @@ that touch the relevant area.
startup. (agentmemory #303.)
12. **No global singletons / `lazy_static` configs.** All deps
explicit. (cognee #2228.)
-13. **Zero-LLM default path.** LLM features opt-in via env. The
+13. **Zero-LLM default path.** LLM has opt-in via env. The
system works without any provider configured.
14. **Tracing subscribers explicitly filter their own module.**
No feedback loops. (agentmemory #519.)
@@ -256,7 +256,7 @@ OPENAI_API_KEY / VOYAGE_API_KEY
## Future work
-* **M9.5 — local embeddings via `ort`.** Bundle `bge-small-en-v1.5`
+* **M9.5 - local embeddings via `ort`.** Bundle `bge-small-en-v1.5`
for an API-key-free homelab path. ~200 MB image bloat; trait is
ready, just needs the `OrtBgeSmallEmbedder` impl + tokenizer wiring.
* **`sqlite-vec` integration.** Brute-force cosine works fine to a few
@@ -273,14 +273,14 @@ OPENAI_API_KEY / VOYAGE_API_KEY
## Reading order
-* This file — operational summary, you are here.
-* [`docs/design-decisions.md`](design-decisions.md) — the full v1 spec.
+* This file - operational summary, you are here.
+* [`docs/design-decisions.md`](design-decisions.md) - the full v1 spec.
* [`docs/research-karpathy-llm-wiki.md`](research-karpathy-llm-wiki.md)
- — what "Karpathy-faithful" means.
+ - what "Karpathy-faithful" means.
* [`docs/research-agentmemory.md`](research-agentmemory.md),
[`research-basic-memory.md`](research-basic-memory.md),
- [`research-cognee.md`](research-cognee.md) — prior art studied.
-* [`docs/issues-*.md`](.) — concrete failure modes we've designed to
+ [`research-cognee.md`](research-cognee.md) - prior art studied.
+* [`docs/issues-*.md`](.) - concrete failure modes we've designed to
avoid.
-* [`CLAUDE.md`](../CLAUDE.md) — per-session operating rules pinned
+* [`CLAUDE.md`](../CLAUDE.md) - per-session operating rules pinned
into Claude Code conversations.
diff --git a/docs/deploy.md b/docs/deploy.md
index 5f3420b6..53dcb35f 100644
--- a/docs/deploy.md
+++ b/docs/deploy.md
@@ -20,7 +20,7 @@ stripped, and are gitignored.
| `docker/.env.production.example` | `docker/.env.production` | LLM + embedding API keys |
`.gitignore` excludes the live files. If you ever see one staged,
-something has drifted — unstage before committing.
+something has drifted - unstage before committing.
## First-time setup (one-time)
@@ -61,7 +61,7 @@ ssh "$SERVER" "docker inspect --format='{{.State.Health.Status}}' ai-memory"
# Expect: healthy
```
-## Security — bearer-token auth + encrypted transport
+## Security - bearer-token auth + encrypted transport
The default `docker-compose.prod.yml.example` binds to `0.0.0.0:49374`
so the LAN can reach the MCP endpoint. **A LAN-bound server with no
@@ -221,7 +221,7 @@ ssh "$SERVER" "tail -100 $DEPLOY_DIR/data/logs/ai-memory.log.$(date +%F)"
`ai-memory embed --reembed` (after starting with the old config or
doing a manual SQL fix) to migrate.
- **Container restart loop**: check
- `docker logs ai-memory` — the `ai-memory starting` line at the top
+ `docker logs ai-memory` - the `ai-memory starting` line at the top
reports the resolved config; a missing required env var (e.g.
`LLM_API_KEY` with `openai-compat` selected but no model) will fail
here with a clear error.
diff --git a/docs/design-decisions.md b/docs/design-decisions.md
index c7de3f4c..39513299 100644
--- a/docs/design-decisions.md
+++ b/docs/design-decisions.md
@@ -1,4 +1,4 @@
-# ai-memory — Design Decisions (Synthesis)
+# ai-memory - Design Decisions (Synthesis)
> Distills the four research reports (`research-*.md`) and three issue-tracker
> reports (`issues-*.md`) into the concrete decisions this project will make.
@@ -9,7 +9,7 @@
A self-contained Rust binary that:
1. Runs as an **MCP server** (stdio + HTTP/SSE) for coding-agent CLIs (Claude Code, OpenAI Codex, OpenCode, future).
-2. Captures the agent's session **automatically** — no `write_note` ceremony — via hook scripts that the agent CLIs invoke (Claude Code lifecycle hooks, Codex hooks, OpenCode equivalents). Optional transcript-tail fallback for agents without hook APIs.
+2. Captures the agent's session **automatically** - no `write_note` ceremony - via hook scripts that the agent CLIs invoke (Claude Code lifecycle hooks, Codex hooks, OpenCode equivalents). Optional transcript-tail fallback for agents without hook APIs.
3. Maintains a **Karpathy-style wiki**: incrementally-compiled markdown pages with cross-links, supersession, an `index.md` and a `log.md`.
4. Serves retrieval via the MCP `tools/list` to coding agents: a handful of *narrow* tools, not 50.
5. Ships a **Docker image** (`docker run -v ai-memory-data:/data -p 49374:49374 ai-memory`) so it can move between desktop and homelab.
@@ -21,13 +21,13 @@ A self-contained Rust binary that:
- Cargo-format clean.
- Docker-deployable, easy backup, easy move desktop↔homelab.
- MCP server for coding agents.
-- **Automatic** memory capture/fetch — minimal manual tool invocations.
+- **Automatic** memory capture/fetch - minimal manual tool invocations.
- Differentiates **short-term** vs **long-term** memory temporally (like agentmemory).
- Self-healing memory management.
- Helps with handoffs between agent CLIs (resume from Codex where Claude Code left off).
-- Iteratively planned — each feature working before the next starts. No dead code.
+- Iteratively planned - each feature working before the next starts. No dead code.
-## 3. Storage model — the biggest architectural decision
+## 3. Storage model - the biggest architectural decision
Three options surveyed:
@@ -37,12 +37,12 @@ Three options surveyed:
| **B. Markdown-in-git** primary | Files in repo | Derived index | Diff-able, grep-able, portable, Karpathy-faithful | Watcher correctness (basic-memory #580/#758/#798), inode races (#765), startup cost |
| **C. DB-primary with on-demand export** | SQLite | Everything | Best of both | Two formats to keep coherent; user must remember to export |
-**Decision: Option B — markdown in a git repo is source of truth, SQLite is derived index.**
+**Decision: Option B - markdown in a git repo is source of truth, SQLite is derived index.**
**Why:**
-- Backup/move story is trivial — `git clone` or `rsync` a directory. The user explicitly asked for this.
+- Backup/move story is trivial - `git clone` or `rsync` a directory. The user explicitly asked for this.
- Karpathy's pattern *is* the wiki on disk. Faking it with an export step loses the inspect-in-Obsidian property.
-- DB is rebuildable from files — corruption is recoverable.
+- DB is rebuildable from files - corruption is recoverable.
- Cross-tool compatibility for free: any agent that reads `~/.ai-memory/wiki/*.md` works without an MCP integration.
**How we avoid basic-memory's watcher pain:**
@@ -55,7 +55,7 @@ Three options surveyed:
- DB stores `(path, mtime, size, sha256, indexed_at, provider, model, dim)` per page. On startup, fast scan vs. cached SHAs; only changed files re-parsed.
- Embeddings keyed by `sha256(content) + provider + model + dim`. Re-embed only when content changes.
-## 4. Database choice — single SQLite file
+## 4. Database choice - single SQLite file
**Decision: one SQLite file with FTS5 + `sqlite-vec` extension + JSON columns for graph edges.**
@@ -73,7 +73,7 @@ Why not LanceDB/Qdrant/Kuzu/CozoDB/SurrealDB?
**Crates** (research-backed picks):
- `rusqlite` + `rusqlite-extension` for sqlite-vec loading. `bundled-sqlcipher` if we want encryption later.
- `sqlx` for migrations (`sqlx::migrate!`). Async, type-checked.
-- `tantivy` *not* used initially — sqlite FTS5 is sufficient at the corpus sizes we expect (hundreds to low-thousands of pages per project). Revisit only if FTS5 ranking proves inadequate.
+- `tantivy` *not* used initially - sqlite FTS5 is sufficient at the corpus sizes we expect (hundreds to low-thousands of pages per project). Revisit only if FTS5 ranking proves inadequate.
- `petgraph` for in-memory graph algorithms during consolidation.
## 5. Embedding & LLM
@@ -88,22 +88,22 @@ Why not LanceDB/Qdrant/Kuzu/CozoDB/SurrealDB?
- **Off by default**, behaves like agentmemory after #138's fix. Without a provider, the system still works: synthetic compression (rule-based), no LLM-generated summaries, no `memory_consolidate` page-rewrite.
- With a provider, scheduled consolidation runs (1× per session-end + optional 6h timer).
- Provider trait `LlmProvider { complete(...); complete_structured(...) }`. Implementations: `AnthropicProvider`, `OpenAIProvider`, `OllamaProvider`, `OpenAICompatProvider`.
-- **Native HTTP per provider** — no LiteLLM-equivalent. The cognee tracker (#2412/#2430/#2537/#2608/#2749/#2782/#2840/#2842) showed silent-kwarg-drop in a generic gateway is the #1 source of provider bugs. Each provider's typed JSON, errors on unknown fields. Hand-coded but correct.
+- **Native HTTP per provider** - no LiteLLM-equivalent. The cognee tracker (#2412/#2430/#2537/#2608/#2749/#2782/#2840/#2842) showed silent-kwarg-drop in a generic gateway is the #1 source of provider bugs. Each provider's typed JSON, errors on unknown fields. Hand-coded but correct.
- **Structured output via JSON schema, not XML, not Instructor-style wrapping.** Use each provider's native JSON-mode where available; for Anthropic, request a tool-use response with a typed schema. Validate with `serde_json` + `schemars`-derived schemas.
-## 6. Capture model — auto, never `write_note`
+## 6. Capture model - auto, never `write_note`
Three capture surfaces, in priority order:
1. **Lifecycle hooks** (Claude Code, Codex, OpenCode). These are fast, reliable, structured. We ship hook scripts the user installs once. Lessons from agentmemory:
- - Hooks must be **fire-and-forget** (#221). No `await fetch()` blocking session start.
- - Sub-second hard timeouts on the writer side (`tokio::time::timeout`).
- - All hooks → single HTTP/Unix-socket POST → server queues → returns 202 immediately.
- - Privacy strip at the hook boundary, not later (agentmemory `stripPrivateData`).
+ - Hooks must be **fire-and-forget** (#221). No `await fetch()` blocking session start.
+ - Sub-second hard timeouts on the writer side (`tokio::time::timeout`).
+ - All hooks → single HTTP/Unix-socket POST → server queues → returns 202 immediately.
+ - Privacy strip at the hook boundary, not later (agentmemory `stripPrivateData`).
2. **Transcript tail** (universal fallback). Watch `~/.claude/projects/`, `~/.codex/`, `~/.config/opencode/sessions/`. Lossier but works for any agent. Required for the basic-memory #669/#687/#730 demand the tracker has been asking for.
-3. **Manual MCP tool** (`memory_remember`) — only for ad-hoc explicit captures from the user ("remember this"). Not the primary path; not what the agent reaches for by default.
+3. **Manual MCP tool** (`memory_remember`) - only for ad-hoc explicit captures from the user ("remember this"). Not the primary path; not what the agent reaches for by default.
## 7. Memory model (temporal)
@@ -112,8 +112,8 @@ Adopt agentmemory's tier model **but** keep the surface narrow:
| Tier | What it is | Lifetime | Decay |
|---|---|---|---|
| **Working** | Current session: last N observations, last user prompt, current files | Until session end | Drop on session end (kept in DB for forensics, but excluded from default recall) |
-| **Episodic** | Per-session summaries with concept tags, files-touched, decisions made | 30 days hot, 180 days cold, then evict if cold-score < threshold | Salience × exp(-λΔt) + Σ(σ/days_since_access) — agentmemory's formula, validated |
-| **Semantic** | Distilled facts/preferences/architecture notes — the wiki pages themselves | Indefinite, supersedeable | Versioned in place: old `is_latest=false`, new `supersedes=old_id` |
+| **Episodic** | Per-session summaries with concept tags, files-touched, decisions made | 30 days hot, 180 days cold, then evict if cold-score < threshold | Salience × exp(-λΔt) + Σ(σ/days_since_access) - agentmemory's formula, validated |
+| **Semantic** | Distilled facts/preferences/architecture notes - the wiki pages themselves | Indefinite, supersedeable | Versioned in place: old `is_latest=false`, new `supersedes=old_id` |
| **Procedural** | Repeated patterns extracted from episodic clusters (`pattern` type with frequency ≥ 2) | Indefinite | Frequency-decay if not re-observed in N days |
**Implementation note:** the four tiers map to one `pages` table with a `tier` enum column + an `observations` table for raw working/episodic, not four separate tables. Keeps schema migrations sane.
@@ -123,14 +123,14 @@ Adopt agentmemory's tier model **but** keep the surface narrow:
Three scheduled MCP operations:
- **`memory_ingest`** (auto-called by hooks): one observation → write-fan-out to ~5–15 wiki pages. New page if no match; supersede + version if the page already exists. No-LLM fallback: append to a per-day digest page if no provider configured.
-- **`memory_query`** (called by agent on demand): hierarchical — search `index.md` first, then page-level FTS+vector, then optional graph-walk expansion. RRF-fused. Agentmemory hit 95.2% R@5 with this pattern.
+- **`memory_query`** (called by agent on demand): hierarchical - search `index.md` first, then page-level FTS+vector, then optional graph-walk expansion. RRF-fused. Agentmemory hit 95.2% R@5 with this pattern.
- **`memory_lint`** (scheduled hourly + on session-end): scans for contradictions, orphan pages, broken links, stale claims, low-confidence + zero-reinforcement entries. Pure LLM with strict JSON output.
Decay/forget runs as a separate `memory_forget_sweep` job: applies the retention formula; soft-deletes via `is_latest=false` + `superseded_at`; hard-deletes only after 180 days *and* zero accesses. Never silently destroys anything user-pinned.
## 9. Cross-agent handoff
-A first-class typed protocol, not just shared state:
+A first-class typed protocol, shared state:
```rust
struct Handoff {
@@ -151,7 +151,7 @@ MCP tools `memory_handoff_begin` (writes a handoff page tagged `state=open`) and
agentmemory has this informally (`/handoff` skill); we make it explicit from day one because every research report flagged cross-agent as the v0.1 weak spot.
-## 10. MCP tool surface — narrow on purpose
+## 10. MCP tool surface - narrow on purpose
basic-memory has ~25 tools, agentmemory has 53. Both have user confusion as a result. Ship **at most 10 v1 tools**:
@@ -163,14 +163,14 @@ basic-memory has ~25 tools, agentmemory has 53. Both have user confusion as a re
| `memory_handoff_begin` | Mark session boundary, write handoff | destructive |
| `memory_handoff_accept` | Fetch+ack open handoff | destructive |
| `memory_forget` | Explicit user forget | destructive |
-| `memory_session_summary` | Internal — used by stop hook | destructive (internal) |
-| `memory_consolidate` | Internal — used by scheduler | destructive (internal) |
-| `memory_lint` | Internal — used by scheduler | destructive (internal) |
+| `memory_session_summary` | Internal - used by stop hook | destructive (internal) |
+| `memory_consolidate` | Internal - used by scheduler | destructive (internal) |
+| `memory_lint` | Internal - used by scheduler | destructive (internal) |
| `memory_status` | Health, counts, last-consolidation-at | read-only |
-Internal tools are gated by `tools/list` annotation; agents see only the user-visible ones unless `expose=all` is set. Every tool has MCP `readOnlyHint`/`destructiveHint`/`idempotentHint` (basic-memory pattern; lesson from #818 — be careful with `bool | None` aliases on tool params).
+Internal tools are gated by `tools/list` annotation; agents see only the user-visible ones unless `expose=all` is set. Every tool has MCP `readOnlyHint`/`destructiveHint`/`idempotentHint` (basic-memory pattern; lesson from #818 - be careful with `bool | None` aliases on tool params).
-Tool param aliases: accept `query|q|search`, `project|workspace`, `dir|directory` — basic-memory's `AliasChoices` pattern works for LLM resilience.
+Tool param aliases: accept `query|q|search`, `project|workspace`, `dir|directory` - basic-memory's `AliasChoices` pattern works for LLM resilience.
## 11. Identity & project scoping (3-tuple from day one)
@@ -200,8 +200,8 @@ To stay scoped:
- No alternative embedded vector backends (sqlite-vec only).
- No alternative graph DB (SQL recursive CTEs only).
- No multimodal (text only).
-- No "skills" / slash-command bundle in v1 (agentmemory plugin format) — focus on hooks + MCP first.
-- No LongMemEval-style benchmark harness in v1 — add in v0.4.
+- No "skills" / slash-command bundle in v1 (agentmemory plugin format) - focus on hooks + MCP first.
+- No LongMemEval-style benchmark harness in v1 - add in v0.4.
## 14. Mistakes-to-avoid checklist (from issue research)
@@ -224,6 +224,6 @@ Top-line rules carved into the codebase:
15. Default data dir is an absolute canonical platform path (agentmemory #303).
16. No `lru_cache` on configs (cognee #2228/#2853).
17. Datasets/projects are query-time filters, not orchestration-mode-conditional (cognee #2867).
-18. LLM features off by default; opt-in via env (agentmemory #138/#143).
-19. `cargo deny` for transitive license audits (cognee #2807 — FastEmbed removed for license).
+18. LLM has off by default; opt-in via env (agentmemory #138/#143).
+19. `cargo deny` for transitive license audits (cognee #2807 - FastEmbed removed for license).
20. Pin upstream native deps; ship a lockfile (agentmemory #555/#540).
diff --git a/docs/install.md b/docs/install.md
index d993e55a..4351c4f3 100644
--- a/docs/install.md
+++ b/docs/install.md
@@ -75,14 +75,14 @@ two env vars automatically.
Each agent CLI needs two things:
-1. **MCP registration** — so the agent can call `memory_query`,
+1. **MCP registration** - so the agent can call `memory_query`,
`memory_recent`, `memory_handoff_accept`.
-2. **Lifecycle hooks** — so the server auto-captures session events.
+2. **Lifecycle hooks** - so the server auto-captures session events.
Without this, the agent can still query memory but capture
becomes manual.
The five clients on the [MCP install page](mcp-install.md) (Cursor,
-Claude Desktop, Gemini CLI, OpenClaw, pi) get MCP only — no
+Claude Desktop, Gemini CLI, OpenClaw, pi) get MCP only - no
lifecycle hooks ship for them today. For the three flagship CLIs
(Claude Code, Codex, OpenCode), both are wired up.
@@ -142,7 +142,7 @@ docker run --rm -v "$HOME/.ai-memory:/host" \
This works cleanly when the container user's UID matches the host
user's UID (e.g. the homelab where both are 1000). It **fails on
-rootless Docker** and on hosts with `userns-remap` enabled — the
+rootless Docker** and on hosts with `userns-remap` enabled - the
container can't write to a host directory that belongs to a UID
outside the user-namespace mapping.
@@ -154,7 +154,7 @@ known not to remap UIDs.
### Cursor, Claude Desktop, Gemini CLI, OpenClaw, pi
-These are MCP-only — no lifecycle-hook bundles ship for them. See
+These are MCP-only - no lifecycle-hook bundles ship for them. See
[**`docs/mcp-install.md`**](mcp-install.md) for the per-client config
file path and snippet, or one-shot it via:
@@ -242,7 +242,7 @@ the repo's `hooks/` automatically:
./target/release/ai-memory install-hooks --agent claude-code --auth-token "$TOKEN"
```
-(No need for `setup-agent` in this case — the scripts already live
+(No need for `setup-agent` in this case - the scripts already live
at the right host path.)
---
@@ -428,12 +428,12 @@ docker run -d --name ai-memory \
```
Notice the bind: `127.0.0.1:49374`, not `0.0.0.0:49374`. This is the
-critical pairing — **no bearer token AND loopback only** is the only
+critical pairing - **no bearer token AND loopback only** is the only
safe combination. The startup log will warn loudly if you bind to a
LAN address without setting `AI_MEMORY_AUTH_TOKEN`.
Then wire up the agent CLI. Both commands default to no auth and
-`http://127.0.0.1:49374` — no extra flags needed for the local case:
+`http://127.0.0.1:49374` - no extra flags needed for the local case:
```bash
ai-memory install-mcp --client claude-code --apply
@@ -444,9 +444,9 @@ ai-memory install-hooks --agent claude-code --apply
## See also
-- [`docs/deploy.md`](deploy.md) — homelab deploy walkthrough
+- [`docs/deploy.md`](deploy.md) - homelab deploy walkthrough
(`bin/deploy`, cloudflared TLS, env-file management)
-- [`docs/mcp-install.md`](mcp-install.md) — per-client MCP config
+- [`docs/mcp-install.md`](mcp-install.md) - per-client MCP config
reference for Cursor, Claude Desktop, Gemini CLI, OpenClaw, pi
-- [`docs/ARCHITECTURE.md`](ARCHITECTURE.md) — what's actually
+- [`docs/ARCHITECTURE.md`](ARCHITECTURE.md) - what's actually
running inside ai-memory
diff --git a/docs/issues-agentmemory.md b/docs/issues-agentmemory.md
index 46418d26..935008d0 100644
--- a/docs/issues-agentmemory.md
+++ b/docs/issues-agentmemory.md
@@ -1,4 +1,4 @@
-# agentmemory — Issue & PR Pain-Point Synthesis
+# agentmemory - Issue & PR Pain-Point Synthesis
> Source: GitHub `rohitg00/agentmemory`, captured 2026-05-21.
> Repo health: 15.7k stars, very active. ~50 merged PRs in last week.
@@ -6,23 +6,23 @@
## Top recurring pain points (ranked)
-### 1. Install / ops — the single largest bucket
+### 1. Install / ops - the single largest bucket
- **`iii-engine` is a separate native binary** with its own version, config file, and storage layout. Pinning is fragile: `[email protected]` broke routing because `package.json` used `^0.11.2` (#555, fixed by PR #567 pinning exact). Migrating to `iii-database`/iii 0.11.7 is blocking the SQLite migration (#309 comment). The whole stack is held back by an upstream you don't control.
-- **Distroless engine + Docker named volumes** = silent permission denied. UID 65532 can't write to root-owned `/data`; engine logs the error but the wrapper buffers in RAM and looks fine until restart wipes everything (#301 — still OPEN despite an earlier 0.9.7 "fix").
-- **Engine writes `data/` to caller's `cwd`**, so launching from different directories produces different state stores. On Windows users believed memories vanished — they were stranded in `E:\文档\New project\data\state_store.db` while the dashboard read `C:\Users\Lenovo\data\` (#303, still OPEN even after PR #314 added `--data-dir`).
+- **Distroless engine + Docker named volumes** = silent permission denied. UID 65532 can't write to root-owned `/data`; engine logs the error but the wrapper buffers in RAM and looks fine until restart wipes everything (#301 - still OPEN despite an earlier 0.9.7 "fix").
+- **Engine writes `data/` to caller's `cwd`**, so launching from different directories produces different state stores. On Windows users believed memories vanished - they were stranded in `E:\文档\New project\data\state_store.db` while the dashboard read `C:\Users\Lenovo\data\` (#303, still OPEN even after PR #314 added `--data-dir`).
- **Hooks break on Windows when username has spaces** because `hooks.json` doesn't quote `${CLAUDE_PLUGIN_ROOT}` (#477).
-- **Runaway log feedback loop** — `iii::workers::observability` warns about "subscriber lagged", that warn is captured by the same subscriber: 137 GB `daemon.log.new`, system at 98% (#519, still OPEN; `RUST_LOG` not honoured).
+- **Runaway log feedback loop** - `iii::workers::observability` warns about "subscriber lagged", that warn is captured by the same subscriber: 137 GB `daemon.log.new`, system at 98% (#519, still OPEN; `RUST_LOG` not honoured).
### 2. Data integrity / silent loss
- **State persistence buffered behind a 5s `IndexPersistence` debounce**. When `state::set` times out at 30s, the uncaught `IIIInvocationError` crashes the Node process, losing every in-memory BM25/vector update since the last debounce flush (#204).
- **BM25 index `mem%3Aindex%3Abm25.bin` stays at ~96 bytes** because every `state::set` times out at 180s on 10k-observation corpora; each restart pays a 5-minute rebuild (#309, OPEN).
- **Sessions never end on Ctrl-C / SSH-drop / laptop sleep**, so the consolidation + graph-extraction pipeline never fires, then the eviction sweep deletes the entire session (#308, "graph would-have-extracted content is permanently lost").
-- **`AGENTMEMORY_DROP_STALE_INDEX=true` in `.env` did nothing** — the dimension guard read `process.env` directly while everything else reads `getMergedEnv()`. **Two config-read paths in the same codebase** (#456). Combined with #469 (vector index 2048-dim on disk vs 384-dim from provider), stranded users with no working recovery path.
-- **Sessions never created, but observations were** — separate KV scopes; OpenClaw plugin only wrote observations, so `GET /sessions` returned `[]` (#522, OPEN). Masked by `postJson({fallback_on_error:true})` swallowing the 4xx.
+- **`AGENTMEMORY_DROP_STALE_INDEX=true` in `.env` did nothing** - the dimension guard read `process.env` directly while everything else reads `getMergedEnv()`. **Two config-read paths in the same codebase** (#456). Combined with #469 (vector index 2048-dim on disk vs 384-dim from provider), stranded users with no working recovery path.
+- **Sessions never created, but observations were** - separate KV scopes; OpenClaw plugin only wrote observations, so `GET /sessions` returned `[]` (#522, OPEN). Masked by `postJson({fallback_on_error:true})` swallowing the 4xx.
- **`session.summary` and `session.firstPrompt` set to the same truncated title** (#276, OPEN, labelled CRITICAL).
### 3. LLM compression / token-cost
-- **`AGENTMEMORY_AUTO_COMPRESS=true` was the default in v0.8.7**. User olcor1: *"My allocation is busted within 20 minutes."* — the PostToolUse hook called the LLM on every tool call (#138, brown-paper-bag fix in v0.8.8 flipping default to false). Maintainer: *"this is a real bug in the tool's design, not your setup."*
+- **`AGENTMEMORY_AUTO_COMPRESS=true` was the default in v0.8.7**. User olcor1: *"My allocation is busted within 20 minutes."* - the PostToolUse hook called the LLM on every tool call (#138, brown-paper-bag fix in v0.8.8 flipping default to false). Maintainer: *"this is a real bug in the tool's design, not your setup."*
- **`SessionStart` hook was injecting ~1-2 K tokens into every new session**. Maintainer initially blamed Claude Pro caps, then retracted and gated injection behind `AGENTMEMORY_INJECT_CONTEXT=false` default in v0.8.10 (#143). His own retraction: *"I pattern-matched without verifying against the docs."*
- **`mem::compress` silently failed on ~47% of Claude Code tool calls** because `post-tool-use.mjs` read `data.tool_output` but Claude Code sends `tool_response` (#539, fixed PR #561).
- **Graph extraction parser drops self-closing `<entity .../>` tags** (#492). Same family: #338.
@@ -30,12 +30,12 @@
### 4. Retrieval quality / API contract
- **MCP `memory_recall` was aliased to `smart_search` and dropped the `format` param**, so full content was unreachable via MCP no matter what the caller asked for (#440 + #507, fixed PR #516). Six weeks of users getting only compact-mode hits.
- **Viewer/status show 0 memories on real corpora** because `/agentmemory/memories?latest=true` and `/agentmemory/export` materialize the entire list, time out on >8k memories (#544, OPEN).
-- **Event-loop starvation from pure-JS dot products** — VectorIndex.search hangs the loop on 100k vectors; sqlite-vec swap took search from 200-250 ms to 20-40 ms (#195).
+- **Event-loop starvation from pure-JS dot products** - VectorIndex.search hangs the loop on 100k vectors; sqlite-vec swap took search from 200-250 ms to 20-40 ms (#195).
### 5. Agent integration / MCP surface
- **Claude Code requested protocol version 2025-03-26 but the shim pinned 2024-11-05**; Claude Code discarded the tools list as a result (#510, OPEN). Same root in #553 (OpenCode 8/51 tools) and #400.
- **Codex worktrees treated as separate projects**, fragmenting lessons/sessions per ephemeral worktree path (#515, OPEN).
-- **Hooks blocked startup** — `await fetch(... 5000 ms timeout)` on session-start; 10 parallel `claude -p` jobs OOM-killed the engine (#221, fixed PR #222 making hooks fire-and-forget).
+- **Hooks blocked startup** - `await fetch(... 5000 ms timeout)` on session-start; 10 parallel `claude -p` jobs OOM-killed the engine (#221, fixed PR #222 making hooks fire-and-forget).
## Design choices that caused the most issues
@@ -50,20 +50,20 @@
## Maintainer fixes reveal regrets
-- **Auto-compress flipped from default-on to default-off** in v0.8.8 the same day #138 was filed — *"the closest thing to an admission that the headline feature was the headline misfeature"*. *"This is exactly the kind of brown-paper-bag issue."*
-- **Context injection moved behind `AGENTMEMORY_INJECT_CONTEXT=false`** in v0.8.10 (#143) — maintainer publicly retracted a wrong first diagnosis. Two defaults reversed in two minor versions.
+- **Auto-compress flipped from default-on to default-off** in v0.8.8 the same day #138 was filed - *"the closest thing to an admission that the headline feature was the headline misfeature"*. *"This is exactly the kind of brown-paper-bag issue."*
+- **Context injection moved behind `AGENTMEMORY_INJECT_CONTEXT=false`** in v0.8.10 (#143) - maintainer publicly retracted a wrong first diagnosis. Two defaults reversed in two minor versions.
- **`VECTOR_BACKEND=sqlite-vec` introduced behind a flag**, not flipped on, explicitly because *"some Windows / Alpine Docker users will hit install issues we can't preempt"* (#195).
-- **Multiple GH-packages mirror experiments reverted within hours** — PRs #545 → #547 → #548. Lots of try/revert.
+- **Multiple GH-packages mirror experiments reverted within hours** - PRs #545 → #547 → #548. Lots of try/revert.
- **PR #500: rebuildIndex made non-blocking on boot**, PR #504: batch-embed in rebuildIndex (25 h → 3 h on large corpora). Admits the original boot path made the daemon unusable for hours.
- **OpenCode plugin (#236) shipped as separate subsystem** because the Claude Code hook abstraction didn't fit other agents.
## Still-open architectural debts
-- **#309 in-memory BM25/graph → SQLite/FTS5** — blocked on iii v0.11.7. Biggest debt in the repo.
-- **#519 daemon.log feedback loop** — offending warn is inside the closed-source `iii` binary.
-- **#303 cwd-relative state store on Windows** — still leaks the wrong dir despite #314.
-- **#301 distroless docker volume permissions** — still OPEN.
-- **#510 / #553 / #400 MCP protocol-version negotiation** — three reports, same root cause.
+- **#309 in-memory BM25/graph → SQLite/FTS5** - blocked on iii v0.11.7. Biggest debt in the repo.
+- **#519 daemon.log feedback loop** - offending warn is inside the closed-source `iii` binary.
+- **#303 cwd-relative state store on Windows** - still leaks the wrong dir despite #314.
+- **#301 distroless docker volume permissions** - still OPEN.
+- **#510 / #553 / #400 MCP protocol-version negotiation** - three reports, same root cause.
## Seven "do not repeat" lessons for the Rust rewrite
@@ -81,7 +81,7 @@
7. **Don't depend on an unpinned native sidecar.** Statically link the engine (`tantivy` for BM25, `sqlite-vec` via `rusqlite`, `petgraph` for the concept graph) (#301, #519, #555). Half the install issues in the tracker exist because `iii-engine` is a separate binary the wrapper can't fix.
-**Bonus:** Default the data directory to a canonicalized absolute platform path (`dirs::data_local_dir().join("ai-memory")`) and log it loudly on startup — single change would have prevented #303 entirely.
+**Bonus:** Default the data directory to a canonicalized absolute platform path (`dirs::data_local_dir().join("ai-memory")`) and log it loudly on startup - single change would have prevented #303 entirely.
### Driver issues
#138 (auto-compress default), #143 (context-injection default), #195 (CPU-bound JS), #204 (uncaught SDK timeout), #221 (blocking hooks), #274 (lesson discard), #276 (corrupt session fields), #301 (distroless volume), #303 (cwd state), #308 (sessions never end), #309 (in-memory BM25), #338/#492 (XML parser), #440/#507 (MCP recall aliased), #456/#469 (dim mismatch + two env paths), #477 (Windows quoting), #510/#553/#400 (MCP protocol version), #515 (Codex worktrees), #519 (log feedback loop), #522 (silent error swallow), #539 (tool_response vs tool_output), #540 (no lockfile), #544 (unbounded list endpoints), #555 (iii-sdk semver).
diff --git a/docs/issues-basic-memory.md b/docs/issues-basic-memory.md
index 0c3cae8f..09ca0f23 100644
--- a/docs/issues-basic-memory.md
+++ b/docs/issues-basic-memory.md
@@ -1,4 +1,4 @@
-# basic-memory — Issue & PR Pain-Point Synthesis
+# basic-memory - Issue & PR Pain-Point Synthesis
> Source: GitHub `basicmachines-co/basic-memory`. Captured 2026-05-21.
> Tracker is unusually high-signal — small team, deep technical replies.
@@ -6,55 +6,55 @@
## Top recurring pain points (ranked)
-### 1. Sync correctness & file-watcher reliability — the single biggest theme
+### 1. Sync correctness & file-watcher reliability - the single biggest theme
-- **#580** — watch service can go stale while process stays alive (no heartbeat, no liveness signal, `awatch()` blocks forever on macOS FSEvents buffer overflow). Closed with partial fix.
-- **#758** — watch service ignored the `--project` constraint, N concurrent MCP processes produced N overlapping watchers and races. The fix found *three independent bugs* while fixing one.
-- **#798** — watch service silently dropped events under hidden-directory parents (e.g., projects under `~/.claude/...`) because gitignore-style globs treated *every* component starting with `.` as hidden.
-- **#765** — "Stale FTS index entries persist after `reset --reindex`": fixed by detecting **zombie MCP processes holding the old SQLite inode** after `unlink`. > *"That process keeps its connection to the old `memory.db` inode (now unlinked but not freed)... newly-spawned MCP processes attach to the new file. But any tool call that gets routed to a zombie MCP process queries the **old** inode."* Now `bm reset` refuses to run while MCP processes are alive (PR #776).
-- **#763** — `write_note` returns *before* semantic indexing completes. Maintainer defends this as intentional architecture but admits the CLI path loses embeddings on process exit.
-- **#839** (OPEN) — CLI `write-note` prints `CancelledError` traceback because `_log_task_failure` doesn't handle task cancellation on process exit. Same root cause as #763.
-- **#578** — *new entities silently skip embedding generation* after a single sqlite-vec load failure. Background task errors fire-and-forget.
-- **#634** — schema-validate uses stale `entity_metadata` when files edited externally.
-- **#481** — `alembic/env.py` was unconditionally setting `BASIC_MEMORY_ENV="test"` at module import time, which silently disabled the watch service in production.
+- **#580** - watch service can go stale while process stays alive (no heartbeat, no liveness signal, `awatch()` blocks forever on macOS FSEvents buffer overflow). Closed with partial fix.
+- **#758** - watch service ignored the `--project` constraint, N concurrent MCP processes produced N overlapping watchers and races. The fix found *three independent bugs* while fixing one.
+- **#798** - watch service silently dropped events under hidden-directory parents (e.g., projects under `~/.claude/...`) because gitignore-style globs treated *every* component starting with `.` as hidden.
+- **#765** - "Stale FTS index entries persist after `reset --reindex`": fixed by detecting **zombie MCP processes holding the old SQLite inode** after `unlink`. > *"That process keeps its connection to the old `memory.db` inode (now unlinked but not freed)... newly-spawned MCP processes attach to the new file. But any tool call that gets routed to a zombie MCP process queries the **old** inode."* Now `bm reset` refuses to run while MCP processes are alive (PR #776).
+- **#763** - `write_note` returns *before* semantic indexing completes. Maintainer defends this as intentional architecture but admits the CLI path loses embeddings on process exit.
+- **#839** (OPEN) - CLI `write-note` prints `CancelledError` traceback because `_log_task_failure` doesn't handle task cancellation on process exit. Same root cause as #763.
+- **#578** - *new entities silently skip embedding generation* after a single sqlite-vec load failure. Background task errors fire-and-forget.
+- **#634** - schema-validate uses stale `entity_metadata` when files edited externally.
+- **#481** - `alembic/env.py` was unconditionally setting `BASIC_MEMORY_ENV="test"` at module import time, which silently disabled the watch service in production.
-### 2. Multi-project / multi-workspace routing — the v0.20 trauma cluster
+### 2. Multi-project / multi-workspace routing - the v0.20 trauma cluster
A wave of nearly identical bugs in April–May 2026: **#782, #783, #788, #793, #799, #800, #802, #803, #804, #805, #810, #820, #834**. Every one of them is the same shape: an MCP tool ignores or mis-resolves a project/workspace identifier. Maintainer on #783: > *"Permalinks are unique per project but project is not unique per workspace... any tool that holds only a permalink string cannot distinguish between them."* **Architecture catching up to ambition**: permalinks were designed as a 2-tuple (project, path) and later had to grow a third dimension (workspace) under load.
### 3. SQLite-vec / embedding provider install hell
-- **#735 / #767** — duplicates: "no such module: vec0" on Windows in worker connections. Fixed by ensuring `_ensure_sqlite_vec_loaded` is called on *every* session that touches vec0.
-- **#829 / #658** — still-open variants.
-- **#741 / #681** — FastEmbed cache defaulted to `/tmp/fastembed_cache`, wiped in sandboxed runtimes like Codex CLI, causing `ONNXRuntimeError: NO_SUCHFILE` on every subsequent semantic search.
-- **#830** (OPEN) — `docker-compose-postgres.yml` ships plain `postgres:17` instead of `pgvector/pgvector:pg17`; semantic search silently fails.
-- **#831** (OPEN) — `IndexError: pop from an empty deque` during async engine dispose on Postgres/asyncpg.
+- **#735 / #767** - duplicates: "no such module: vec0" on Windows in worker connections. Fixed by ensuring `_ensure_sqlite_vec_loaded` is called on *every* session that touches vec0.
+- **#829 / #658** - still-open variants.
+- **#741 / #681** - FastEmbed cache defaulted to `/tmp/fastembed_cache`, wiped in sandboxed runtimes like Codex CLI, causing `ONNXRuntimeError: NO_SUCHFILE` on every subsequent semantic search.
+- **#830** (OPEN) - `docker-compose-postgres.yml` ships plain `postgres:17` instead of `pgvector/pgvector:pg17`; semantic search silently fails.
+- **#831** (OPEN) - `IndexError: pop from an empty deque` during async engine dispose on Postgres/asyncpg.
### 4. Parser fragility (markdown-as-source-of-truth)
-- **#738** — Parser captured Obsidian callout syntax (`> [!note]`) as observation categories.
-- **#721** — `edit_note` fails on notes with long text around inline wikilinks because `relation_type` exceeds `MaxLen`.
-- **#528** — Cloud sync prepended duplicate frontmatter to files that already had YAML.
-- **#408** — YAML frontmatter parsing fails with unquoted colons in title.
-- **#256** — *"Editing notes causes them to disappear from index"* — the search index DELETE was missing a `project_id` filter, so an edit in one project nuked search rows in every project with the same permalink.
+- **#738** - Parser captured Obsidian callout syntax (`> [!note]`) as observation categories.
+- **#721** - `edit_note` fails on notes with long text around inline wikilinks because `relation_type` exceeds `MaxLen`.
+- **#528** - Cloud sync prepended duplicate frontmatter to files that already had YAML.
+- **#408** - YAML frontmatter parsing fails with unquoted colons in title.
+- **#256** - *"Editing notes causes them to disappear from index"* - the search index DELETE was missing a `project_id` filter, so an edit in one project nuked search rows in every project with the same permalink.
### 5. Search quality / pagination
-- **#693** — `read_note` pagination params were ignored at the API endpoint.
-- **#354** — `tag:tagname` syntax silently treated as literal text.
-- **#686** (OPEN) — user hitting MCP response size limits at 57 pages.
-- **#666, #618, #603** (all OPEN) — reranking, time-decay, length normalization. Search ranking is admitted-weak by maintainers.
+- **#693** - `read_note` pagination params were ignored at the API endpoint.
+- **#354** - `tag:tagname` syntax silently treated as literal text.
+- **#686** (OPEN) - user hitting MCP response size limits at 57 pages.
+- **#666, #618, #603** (all OPEN) - reranking, time-decay, length normalization. Search ranking is admitted-weak by maintainers.
### 6. Backup / undo / git
-- **#124** (OPEN since June 2025) — git-based undo. Maintainer wrote design spec but issue remains open: > *"there hasn't been an obvious way to handle when to commit. Commit on every change? Commit every so often, how often? Push to a remote? How to handle conflicts?"*
-- **#59** — leverage `git diff` to prevent knowledge corruption/deletion. OPEN since March 2025.
+- **#124** (OPEN since June 2025) - git-based undo. Maintainer wrote design spec but issue remains open: > *"there hasn't been an obvious way to handle when to commit. Commit on every change? Commit every so often, how often? Push to a remote? How to handle conflicts?"*
+- **#59** - use `git diff` to prevent knowledge corruption/deletion. OPEN since March 2025.
-### 7. Manual capture friction — present but indirect
+### 7. Manual capture friction - present but indirect
**Nobody filed "I'm tired of telling the agent to remember."** But:
-- **#297** — Cursor + basic-memory produce sensationalized progress logs (`BREAKTHROUGH - Live Data Packets Detected!`) that pollute search results. Maintainer closes as **not basic-memory's fault, it's the LLM's**: > *"It is just the tools. Your LLM is in charge of how to use it."* That's the philosophical commitment that drives the manual `write_note` workflow — transferring all the noise problems onto the user.
-- **#669, #730, #687** (all OPEN) — three separate proposals to add a *sidecar that watches session transcripts and builds the knowledge graph automatically*. **The strongest signal that manual capture is friction.** The maintainer is interested but hasn't started.
+- **#297** - Cursor + basic-memory produce sensationalized progress logs (`BREAKTHROUGH - Live Data Packets Detected!`) that pollute search results. Maintainer closes as **not basic-memory's fault, it's the LLM's**: > *"It is just the tools. Your LLM is in charge of how to use it."* That's the philosophical commitment that drives the manual `write_note` workflow - transferring all the noise problems onto the user.
+- **#669, #730, #687** (all OPEN) - three separate proposals to add a *sidecar that watches session transcripts and builds the knowledge graph automatically*. **The strongest signal that manual capture is friction.** The maintainer is interested but hasn't started.
## Design choices that caused the most issues
@@ -79,14 +79,14 @@ A wave of nearly identical bugs in April–May 2026: **#782, #783, #788, #793, #
- **Docs heavily expanded after confusion**: Postgres setup (#830 still open), `bm cloud setup` (#779 was pointing users at a non-existent command).
- **Out-of-scope back-outs**: #720 (visible_project_ids filter) closed not-fix because *"would leak multi-tenant visibility concerns into a local-first single-user product"*.
-## Still-unsolved open issues — and why they're hard
+## Still-unsolved open issues - and why they're hard
-- **#124 git-based undo** — open 11+ months. Hard because commit cadence is ill-defined and conflicts in markdown trees are nasty.
-- **#834 local project_id routing in mixed cloud mode** — same root cause keeps spawning new symptoms.
-- **#382 / #686 large-context handling** — pagination of search results into LLM context windows is still unsolved.
-- **#740 startup time** — 4.6s for `--help` because FastMCP/onnxruntime/fastembed are imported eagerly. Multi-file lazy-import refactor not shipped.
-- **#830 / #831 / #829 Postgres + sqlite-vec footguns** — install path still surprises users.
-- **#669 / #687 transcript-watching sidecar** — the holy grail; nobody has built it.
+- **#124 git-based undo** - open 11+ months. Hard because commit cadence is ill-defined and conflicts in markdown trees are nasty.
+- **#834 local project_id routing in mixed cloud mode** - same root cause keeps spawning new symptoms.
+- **#382 / #686 large-context handling** - pagination of search results into LLM context windows is still unsolved.
+- **#740 startup time** - 4.6s for `--help` because FastMCP/onnxruntime/fastembed are imported eagerly. Multi-file lazy-import refactor not shipped.
+- **#830 / #831 / #829 Postgres + sqlite-vec footguns** - install path still surprises users.
+- **#669 / #687 transcript-watching sidecar** - the holy grail; nobody has built it.
## Concrete "do not repeat" lessons for the Rust rewrite
@@ -96,7 +96,7 @@ A wave of nearly identical bugs in April–May 2026: **#782, #783, #788, #793, #
3. **Liveness > correctness assumptions for the watcher.** A long-lived file watcher *will* go stale. Build heartbeats, watchdog timers, and "did we miss any events" reconciliation passes from the start. (#580, #758, #798). Treat every notify-rs-style loop as suspect.
-4. **Treat the embedding/vector backend as a fallible plugin, not a default-on assumption.** sqlite-vec, pgvector, FastEmbed, ONNX — every one of them has bitten basic-memory (#735, #767, #741, #681, #830, #831, #658, #829, #578). In Rust, isolate the embedder behind a trait, fail loudly at startup if the backend can't load (don't silently degrade like #578), and don't default cache paths to `/tmp`.
+4. **Treat the embedding/vector backend as a fallible plugin, not a default-on assumption.** sqlite-vec, pgvector, FastEmbed, ONNX - every one of them has bitten basic-memory (#735, #767, #741, #681, #830, #831, #658, #829, #578). In Rust, isolate the embedder behind a trait, fail loudly at startup if the backend can't load (don't silently degrade like #578), and don't default cache paths to `/tmp`.
5. **Make `reset` safe.** Unlinking SQLite while a sibling process holds the inode = mystery phantom search results (#765). Acquire an exclusive advisory lock, or psutil-style live-process check before any destructive op.
diff --git a/docs/issues-cognee.md b/docs/issues-cognee.md
index e62bcb3e..c032439c 100644
--- a/docs/issues-cognee.md
+++ b/docs/issues-cognee.md
@@ -1,4 +1,4 @@
-# cognee — Issue & PR Pain-Point Synthesis
+# cognee - Issue & PR Pain-Point Synthesis
> Source: GitHub `topoteretes/cognee`. Captured 2026-05-21.
> Tracker character: ~40% feature requests (high comment counts), ~40% bugs
@@ -9,7 +9,7 @@
### A. LLM-adapter brittleness (highest volume, highest churn)
Every provider has had a wire-level bug in 2026.
-- Anthropic adapter dropped `max_tokens`, every call HTTP-422 (#2749, #2782 — **two consecutive releases shipped broken Anthropic**).
+- Anthropic adapter dropped `max_tokens`, every call HTTP-422 (#2749, #2782 - **two consecutive releases shipped broken Anthropic**).
- Ollama / LlamaCpp adapters missing `@observe` decorator (#2820).
- vLLM hangs because system message sent after user message (#2537).
- vLLM "custom" provider doesn't forward `LLM_ENDPOINT` to LiteLLM (#2412, #2430, #2842).
@@ -24,11 +24,11 @@ Every provider has had a wire-level bug in 2026.
### B. Multi-store coordination & data integrity bugs
The "graph + vector + relational" tripartite store is the cognee architectural commitment, and it is the source of consistent, high-severity bugs.
-- `EntityAlreadyExistsError` — `Entity("institution")` and `EntityType("institution")` collide on UUID (#2510). Second `cognify` blew up.
+- `EntityAlreadyExistsError` - `Entity("institution")` and `EntityType("institution")` collide on UUID (#2510). Second `cognify` blew up.
- `add_data_points` parallel DB ops cause SQLite `database is locked` deadlock; **still reproducible in 1.0.2**: `multiple PipelineRunErrored ... elapsed 561s before crash` (#2717, OPEN).
- `cognify` exceeds asyncpg bind-argument limit during `upsert_edges` for large batches (~4356 edges) (#2829, fixed via batching in #2798, #2586).
- `add_data_points` crashes with `asyncpg.CharacterNotInRepertoireError` on null bytes (0x00) (#2612, OPEN).
-- `index_data_points`: shallow copy of metadata dict — only the first `index_field` is embedded (#2529, OPEN).
+- `index_data_points`: shallow copy of metadata dict - only the first `index_field` is embedded (#2529, OPEN).
- `get_graph_from_model + copy_model` drops user-defined DataPoint id (#2633, OPEN).
- Edge deduplication in `retrieve_existing_edges` is non-functional (#2557).
- `delete_dataset` fails in non-"public" Postgres schema (#2291).
@@ -37,26 +37,26 @@ The "graph + vector + relational" tripartite store is the cognee architectural c
- KuzuAdapter writes visible in-memory but not persisted to disk on 0.5.1 (#1981).
- Per-collection distance normalization in `brute_force_triplet_search` produces incorrect ranking (#2030, fix #2451 removed normalization, **then #2720 surfaced downstream**).
-### C. Recall quality regressions — the bug a memory server cannot afford
+### C. Recall quality regressions - the bug a memory server cannot afford
-- **#2720 (OPEN)**: "Graph-completion retrieval returns identical subgraph regardless of query". User-built reproducer shows direct LanceDB queries return different top-K for different queries, but cognee's `/api/v1/search` returns ~identical answers — LanceDB IDs are not propagating into the graph projection. User attributes to fallout from #2451: downstream thresholds in `brute_force_triplet_search.py` still expect pre-#2451 [0,1] scale and silently fall back to an unfiltered graph.
+- **#2720 (OPEN)**: "Graph-completion retrieval returns identical subgraph regardless of query". User-built reproducer shows direct LanceDB queries return different top-K for different queries, but cognee's `/api/v1/search` returns ~identical answers - LanceDB IDs are not propagating into the graph projection. User attributes to fallout from #2451: downstream thresholds in `brute_force_triplet_search.py` still expect pre-#2451 [0,1] scale and silently fall back to an unfiltered graph.
- `SearchType.CHUNKS` silently ignores `node_name` filtering (#2815).
-- `CHUNKS/SUMMARIES/GRAPH_COMPLETION` ignores `datasets=` filter when `ENABLE_BACKEND_ACCESS_CONTROL=false` (#2867 — maintainer answer is essentially "datasets only work with access control on").
+- `CHUNKS/SUMMARIES/GRAPH_COMPLETION` ignores `datasets=` filter when `ENABLE_BACKEND_ACCESS_CONTROL=false` (#2867 - maintainer answer is essentially "datasets only work with access control on").
- Cognee-mcp GRAPH_COMPLETION discarded all but first dataset (#2617).
-- `TemporalRetriever` is event-only, blocks ontology-wide temporal filtering (#2429, closed as superseded by an internal Q2 redesign — not shipped).
+- `TemporalRetriever` is event-only, blocks ontology-wide temporal filtering (#2429, closed as superseded by an internal Q2 redesign - not shipped).
- GRAPH_COMPLETION does not search custom DataPoint vector collections (#2495).
### D. Dependency-installation hell
- macOS arm64 + Python 3.14: `kuzu` wheel does not exist; quick-start fails (#2753).
-- `ModuleNotFoundError: No module named 'kuzu' in Docker since v1.0.4` (#2775) — kuzu removed but image not rebuilt.
+- `ModuleNotFoundError: No module named 'kuzu' in Docker since v1.0.4` (#2775) - kuzu removed but image not rebuilt.
- `fastembed` removed from the Docker image entirely because one of its transitive deps was not Apache/MIT (#2807).
- LiteLLMEmbeddingEngine truncated `BAAI/bge-m3` to `bge-m3` (#1915).
-- `embedding_dimensions` defaulted to 3072 regardless of model — every non-3072-dim embedder broke (#2751, fix #2757).
+- `embedding_dimensions` defaulted to 3072 regardless of model - every non-3072-dim embedder broke (#2751, fix #2757).
- LanceDB lance-file writer schema drift / "contained null values" RuntimeError bypassed auto-migration (#2702, #2768).
- Pydantic v1/v2 friction: upper bound conflicts with openai-agents (#2019); `.json()` deprecated (#2042); generic validation issues (#1198).
- Mistral client import error (#2481).
-- `lru_cache` hash invalidation bugs for Vector and Graph configs (#2357), and **`lru_cache` was eventually disabled outright** in PR #2853 — "refactor: reduce lru cache".
+- `lru_cache` hash invalidation bugs for Vector and Graph configs (#2357), and **`lru_cache` was eventually disabled outright** in PR #2853 - "refactor: reduce lru cache".
### E. MCP server bugs vs FastAPI
@@ -64,14 +64,14 @@ The MCP wrapper consistently lags the core API.
- MCP `cognify` with valid local file path returns success but creates no Data item (#2250, OPEN).
- cognee-mcp Quick Start fails on macOS arm64 + Py 3.14 (#2753).
- cognee-mcp `cognify(data=str)` silently dropped all writes after first due to hardcoded `data.txt` filename (#2747).
-- MCP recall in default config fails: `'NoneType' object has no attribute 'id'` — wrapper does not pass user to cognee.recall (#2855).
+- MCP recall in default config fails: `'NoneType' object has no attribute 'id'` - wrapper does not pass user to cognee.recall (#2855).
- cognee-cli `--api-url` doesn't support remember/recall/improve/forget (#2809, OPEN).
- Frontend Docker build broken (#2832), Turbopack import case mismatch (#2605), `cognee-cli -ui` v0.5.5 missing 3 npm deps (#2413), UI compile errors (#2709).
### F. Auth / multi-tenancy regressions
- Token refresh mechanism literally not implemented; maintainer admits *"we had to reimplement it for our cloud deployment. At this point, we can't allocate resources"* (#2065).
-- Request-scoped LLM config impossible because `get_llm_config()` and `get_embedding_config()` use `@lru_cache` — singletons (#2228).
+- Request-scoped LLM config impossible because `get_llm_config()` and `get_embedding_config()` use `@lru_cache` - singletons (#2228).
- Auth disable needs *two* flags: `ENABLE_BACKEND_ACCESS_CONTROL=false` AND `REQUIRE_AUTHENTICATION=False` (#2808, fix #2836).
- `cognee.search` ignored ACL when resolving dataset by name for non-owners (#2845); `cognee.add` silently created a new owner-scoped dataset when a non-owner reused a name (#2846); `cognify` silently skipped data added by non-owners (#2847).
- Agent display name leaked user ID (#2811).
@@ -82,11 +82,11 @@ The MCP wrapper consistently lags the core API.
1. **Singleton config via `@lru_cache`.** Breaks multi-tenancy (#2228), invalidation bugs (#2357), eventually reverted (#2853).
2. **LiteLLM + Instructor as the universal LLM/structured-output layer.** Source of #2412, #2430, #2537, #2608, #2613, #2749, #2782, #2820, #2840, #2842.
3. **SQLite as the default relational backend with greenlet parallelism.** #2717: `OperationalError: database is locked` under parallel cognify, OPEN. Maintainer hedges: *"sqlite is not there for production use-cases"*.
-4. **Kuzu as the default embedded graph DB.** Kuzu was archived upstream (#2098), maintainers chose to replace with **Ladybug** (PR #2755) — a *fork* of Kuzu. Immediate post-replacement bugs: #2768, #2775, WAL corruption (PR #2838). **The forked-DB risk has played out.**
+4. **Kuzu as the default embedded graph DB.** Kuzu was archived upstream (#2098), maintainers chose to replace with **Ladybug** (PR #2755) - a *fork* of Kuzu. Immediate post-replacement bugs: #2768, #2775, WAL corruption (PR #2838). **The forked-DB risk has played out.**
5. **LanceDB as the default vector store** with schema migration assumed to auto-handle drift. Reality: #2702 null-values bypasses migration, #2720 retrieval pipeline drops filter on the path from vector hits to graph subgraph.
6. **Tripartite store with implicit sync (graph + vector + relational + optional ontology).** Source of the entire integrity-bug class in section B. Orchestration is handled inside cognee, not by any transactional layer.
7. **`@lru_cache` + ContextVar mixed model** for tenant isolation. #2228 explains: db config uses ContextVar pattern, but LLM and embedding configs do not.
-8. **Per-collection distance normalization in `brute_force_triplet_search`** (#2030) — fix #2451 removed normalization, breaking downstream threshold assumptions, surfacing as #2720.
+8. **Per-collection distance normalization in `brute_force_triplet_search`** (#2030) - fix #2451 removed normalization, breaking downstream threshold assumptions, surfacing as #2720.
9. **Backend access control is the orchestration plane.** When `ENABLE_BACKEND_ACCESS_CONTROL=false`, *all* dataset-scoped retrieval silently degrades. (#2867, #2845, #2846, #2847, #2808.)
10. **Default-LLM cost coupling**: `embedding_dimensions` defaults to 3072 (text-embedding-3-large), and litellm's `model_cost` table silently overrode user `max_completion_tokens`. Two bugs from "assume OpenAI defaults" (#2751, #2608).
@@ -96,14 +96,14 @@ The MCP wrapper consistently lags the core API.
- **Subprocess mode + Redis** was added explicitly to escape the SQLite-greenlet trap (#2803, #2812).
- **Ladybug is a Kuzu fork** they own. Already shipped "fix: resolve issue with WAL file corruption for ladybug" (#2838).
- **Auto-migrate LanceDB on schema drift** had to be added (#2703) after lance-file writer crashed workers.
-- **Anthropic adapter broken for a full version cycle** — #2749 then re-broken in 1.0.5 as #2782.
+- **Anthropic adapter broken for a full version cycle** - #2749 then re-broken in 1.0.5 as #2782.
- **Defaults flipped**: `fastembed` removed from core (#2807), result-cache logging disabled by default (#2851), embedding dimensions auto-derived not defaulted (#2757), auth gated by single switch (#2836).
- **Feature deprecated, not fixed**: `TemporalRetriever` (#2429).
## Open issues maintainers haven't solved
-- **#2717 SQLite deadlock under parallel cognify** — Reproducible across versions.
-- **#2720 LanceDB filter not propagating to graph projection** — *Correctness* bug in core retrieval path. No assignee.
+- **#2717 SQLite deadlock under parallel cognify** - Reproducible across versions.
+- **#2720 LanceDB filter not propagating to graph projection** - *Correctness* bug in core retrieval path. No assignee.
- **#2840 Thinking-token + Instructor incompatibility.**
- **#2532 N+1 query in `/api/v1/cognify`.**
- **#2612 null-byte crash in asyncpg.**
@@ -138,7 +138,7 @@ The MCP wrapper consistently lags the core API.
1. **Do not put the LLM call behind a generic Python-style gateway that silently drops kwargs.** Each provider gets a typed Rust client that errors on unknown fields rather than dropping them. (#2840, #2608, #2782.)
-2. **Don't use SQLite for write-parallel pipeline state.** Use Postgres or — for embedded — a single-writer actor in front of an LMDB/Sled/SQLite-with-WAL serialized via a queue. (#2717.)
+2. **Don't use SQLite for write-parallel pipeline state.** Use Postgres or - for embedded - a single-writer actor in front of an LMDB/Sled/SQLite-with-WAL serialized via a queue. (#2717.)
3. **Don't pin a forked embedded graph DB as the default.** Either commit to a battle-tested external store (Postgres + AGE, Neo4j) or build the graph primitives directly on the relational store. The Kuzu→Ladybug pivot cost real users (#2098, #2768, #2775, #2753, PR#2838).
@@ -148,16 +148,16 @@ The MCP wrapper consistently lags the core API.
6. **Never default `embedding_dimensions` to a constant.** Derive from `(provider, model)` at startup; refuse to start if collection-dim and model-dim mismatch. (#2751, #2757.)
-7. **Idempotent ingestion with explicit id derivation.** Node IDs must be a function of `(category, name, dataset)` not just `name`. Property tests on cross-run determinism. (#2510, #2557, #2633.)
+7. **Idempotent ingestion with explicit id derivation.** Node IDs must be a function of `(category, name, dataset)` `name`. Property tests on cross-run determinism. (#2510, #2557, #2633.)
8. **Re-ingestion / pipeline-run status must be a state machine, not a flag.** "PipelineRunAlreadyCompleted" prevented re-ingest of deleted files (#2097).
9. **Dataset isolation must work whether or not "access control" is on.** Dataset is a hard query-time filter on every retriever. (#2867, #2845, #2846, #2847.)
-10. **Batch every cross-store mutation.** asyncpg bind-arg limit, SQLite locks, lance writer flushes — all rooted in unbounded fan-out (#2829, #2717, #2702).
+10. **Batch every cross-store mutation.** asyncpg bind-arg limit, SQLite locks, lance writer flushes - all rooted in unbounded fan-out (#2829, #2717, #2702).
11. **Single transactional boundary or a documented eventual-consistency contract.** Cognee's silent skips between graph/vector/relational are the deepest class of bug. Pick one.
-12. **Audit logs and result caches with retention from day one.** Cognee's relational DB grew unbounded — 42k cached results in 9 days (#2548). They eventually disabled by default; do it before the first user hits it.
+12. **Audit logs and result caches with retention from day one.** Cognee's relational DB grew unbounded - 42k cached results in 9 days (#2548). They eventually disabled by default; do it before the first user hits it.
**Calibration**: the single most repeated lesson, weighted by both severity and recurrence: **the LLM/structured-output layer (LiteLLM + Instructor) is fragile, and the multi-store sync (graph + vector + relational) is the deepest source of correctness bugs.** A Rust rewrite that gets either of those wrong inherits cognee's tracker.
diff --git a/docs/issues-mempalace.md b/docs/issues-mempalace.md
index d2e4147d..b2ea100c 100644
--- a/docs/issues-mempalace.md
+++ b/docs/issues-mempalace.md
@@ -1,4 +1,4 @@
-# MemPalace — Issue & Architecture Synthesis
+# MemPalace - Issue & Architecture Synthesis
> Source: GitHub `MemPalace/mempalace` + local checkout at `~/Projects/mempalace`.
> Captured 2026-05-24. Repo: 52.7k stars, MIT, Python, created 2026-04-05,
@@ -23,133 +23,133 @@ no LLM). It is a genuinely strong retrieval system whose pain is almost entirely
## Top recurring pain points (ranked)
-### 1. Concurrent-write corruption of ChromaDB / HNSW / FTS5 — THE dominant theme
+### 1. Concurrent-write corruption of ChromaDB / HNSW / FTS5 - THE dominant theme
This single cluster is the majority of open bugs. Root cause: **multiple
-uncoordinated writer paths to one ChromaDB store** — the Stop hook fires an
+uncoordinated writer paths to one ChromaDB store** - the Stop hook fires an
async `mempalace mine` on every session end, the PreCompact hook fires another,
the MCP server writes on tool calls, and manual `mempalace mine` can run at the
same time. ChromaDB's HNSW index and FTS5 shadow tables are not safe under
concurrent writers, and a writer killed mid-write (SIGTERM during compaction /
permission prompts / window close) leaves a malformed index.
-- **#1596** — *"The Claude Code Stop hook fires `mempalace mine` asynchronously
+- **#1596** - *"The Claude Code Stop hook fires `mempalace mine` asynchronously
on every session end. When multiple Claude sessions close simultaneously …
multiple mine processes start with no mutual-exclusion guard. These
overlapping processes write to FTS5 shadow tables concurrently, and when one
is killed mid-write … it leaves `embedding_fulltext_search` in a malformed
state."* `PRAGMA integrity_check` returns `ok` (rows intact) but
`quick_check` fails (FTS5 index malformed) → `repair` refuses to run.
-- **#1599** — recurring FTS5 corruption under live writes, *after* the
+- **#1599** - recurring FTS5 corruption under live writes, *after* the
transient-vector-failure mitigation that was supposed to fix it.
-- **#1581** — HNSW corruption from concurrent ChromaDB clients (MCP server +
+- **#1581** - HNSW corruption from concurrent ChromaDB clients (MCP server +
hook subprocesses + auto-ingest mine) **within a single Claude Code session**.
-- **#1253** — PreCompact hook double-ingest race corrupts HNSW: *"no
+- **#1253** - PreCompact hook double-ingest race corrupts HNSW: *"no
palace-wide write lock."*
-- **#1564** — even the single-writer sweep can trigger stale HNSW quarantine.
-- **#1514** (community patch, in Chinese) — dual-client concurrent read/write:
+- **#1564** - even the single-writer sweep can trigger stale HNSW quarantine.
+- **#1514** (community patch, in Chinese) - dual-client concurrent read/write:
user shipped a `flock + threading.Lock` double-lock patch themselves.
-- **#1343 / #974 / #965** (referenced) — the original "wrap every ChromaDB
+- **#1343 / #974 / #965** (referenced) - the original "wrap every ChromaDB
write in `mine_palace_lock`" retrofit; corruption persists in edge cases.
-- **#1497** (the tell) — a user asks: *"should MemPalace recommend a
+- **#1497** (the tell) - a user asks: *"should MemPalace recommend a
single-writer/gateway pattern for multi-agent setups? … our reading is:
`mcp-proxy` helps avoid each client spawning its own server, but it does not
by itself guarantee that all palace mutations are serialized."* They sketch a
gateway → write-queue → **single writer**. **They are proposing, in an
issue, the exact architecture ai-memory has by construction.**
-### 2. Silent persistence failure — "filed N drawers" but nothing persisted
+### 2. Silent persistence failure - "filed N drawers" but nothing persisted
The write path returns success *before* the data is durably in ChromaDB. Below
`hnsw:sync_threshold`, the HNSW segment + `index_metadata.pickle` never flush,
so small mines vanish while the CLI cheerfully reports success.
-- **#1597** — *"`mempalace mine` reports `Drawers filed: 40` … but the chromadb
+- **#1597** - *"`mempalace mine` reports `Drawers filed: 40` … but the chromadb
does not persist them. `mempalace status` keeps showing the pre-upgrade
total."* Confirmed not a cache issue (`mempalace_reconnect` agrees with the
stale count). The v3.3.5 notes had specifically claimed *"integrity,
recovery, and cross-process correctness."*
-- **#1579** — mines smaller than `hnsw:sync_threshold` never persist
+- **#1579** - mines smaller than `hnsw:sync_threshold` never persist
`index_metadata.pickle`.
-- **#1526** — silent empty index below batch threshold (one of two HNSW failure
+- **#1526** - silent empty index below batch threshold (one of two HNSW failure
modes).
-- **#1537** — `mempalace mine` reports `Done ✓` with full counts when the mine
+- **#1537** - `mempalace mine` reports `Done ✓` with full counts when the mine
produced an FTS5-corrupt palace.
-- **#1398** — MCP search returns `Error finding id` after bulk-add until WAL
+- **#1398** - MCP search returns `Error finding id` after bulk-add until WAL
flushes; CLI works (read-after-write inconsistency).
-- **#1489** — request to expose `hnsw:sync_threshold` because low-volume MCP
+- **#1489** - request to expose `hnsw:sync_threshold` because low-volume MCP
workloads "never flush metadata."
-### 3. Repair/migrate tooling that cannot recover — or actively destroys data
+### 3. Repair/migrate tooling that cannot recover - or actively destroys data
When corruption happens (theme 1) the recovery path is itself unsafe.
-- **#1394** — *"`mempalace repair --mode legacy` destroys 99% of SQLite
+- **#1394** - *"`mempalace repair --mode legacy` destroys 99% of SQLite
embedding rows on 3.3.4."*
-- **#1545** — `repair --mode from-sqlite` "structurally broken at scale:
+- **#1545** - `repair --mode from-sqlite` "structurally broken at scale:
auto-quarantine destroys rebuild progress mid-run."
-- **#1586** — repair cannot recover combined HNSW-dimensionality + FTS5
+- **#1586** - repair cannot recover combined HNSW-dimensionality + FTS5
corruption (313k drawers).
-- **#1589** — repair *rebuilds* an HNSW segment that ChromaDB 1.5.9 then cannot
+- **#1589** - repair *rebuilds* an HNSW segment that ChromaDB 1.5.9 then cannot
load; the corrupt segment SIGSEGVs in `chromadb_rust_bindings`.
-- **#1595** — `mempalace-health.py` parses `header.bin` as uint32 →
+- **#1595** - `mempalace-health.py` parses `header.bin` as uint32 →
trillion-element HNSW corruption silently passes all checks.
-- **#1266** — HNSW pickle corruption after long-running mine: "unsupported
+- **#1266** - HNSW pickle corruption after long-running mine: "unsupported
opcode" on next read; repair/migrate cannot recover.
-- **#1492 / #1493** — `rebuild_index` deterministically writes
+- **#1492 / #1493** - `rebuild_index` deterministically writes
`dimensionality=None` into ChromaDB's `index_metadata`.
-### 4. Data loss by design — miners and swaps destroy verbatim history
+### 4. Data loss by design - miners and swaps destroy verbatim history
-- **#1593** — *filed by the founder*: "stop delete-on-remine — verbatim history
+- **#1593** - *filed by the founder*: "stop delete-on-remine - verbatim history
should never be destroyed by MemPalace miners." Re-mining a source deletes the
old drawers first; for a system whose whole pitch is "verbatim, never lossy,"
this is an existential bug.
-- **#1533** — MCP-server processes destroy newly-swapped palace files on first
+- **#1533** - MCP-server processes destroy newly-swapped palace files on first
read after an atomic swap (no quarantine re-arm).
-- **#1341** — no `SessionEnd` hook: short sessions lose memories on clean exit
+- **#1341** - no `SessionEnd` hook: short sessions lose memories on clean exit
(only Stop/PreCompact exist, and they don't always fire).
-- **#1535** — `mempalace mine --limit N` truncates files *before* the
+- **#1535** - `mempalace mine --limit N` truncates files *before* the
already-mined skip, so a run can finish with 0 new drawers.
-- **#1329** — Stop hook → 1.9 **TB** palace bloat + ChromaDB Rust-bindings
+- **#1329** - Stop hook → 1.9 **TB** palace bloat + ChromaDB Rust-bindings
segfault (verbatim-everything with no decay → unbounded growth).
### 5. Embedding-model drift & pluggability gaps
-- **#1561** — "persist and verify embedding model metadata in palace
+- **#1561** - "persist and verify embedding model metadata in palace
collections." There is currently **no stored `{provider, model, dim}` to
- refuse on mismatch** — exactly the failure ai-memory's invariant #8 prevents.
-- **#1559** — support external embedding APIs (LM Studio, Ollama,
+ refuse on mismatch** - exactly the failure ai-memory's invariant #8 prevents.
+- **#1559** - support external embedding APIs (LM Studio, Ollama,
OpenAI-compatible).
-- **#1563 / #1261** — embedding model is hardcoded to all-MiniLM-L6-v2; users
+- **#1563 / #1261** - embedding model is hardcoded to all-MiniLM-L6-v2; users
want configuration-driven switching without source edits.
-- **#1380** — hash-embedding fallback mode needs a real BM25 lexical path.
+- **#1380** - hash-embedding fallback mode needs a real BM25 lexical path.
### 6. Native-binding / platform instability (the heavy-dependency tax)
ChromaDB drags in `onnxruntime`, `grpcio`, `numpy`, and a Rust bindings layer.
-- **#1355** — `chromadb_rust_bindings` segfaults on macOS 26.4 ARM64 — **all
+- **#1355** - `chromadb_rust_bindings` segfaults on macOS 26.4 ARM64 - **all
`>=1.5.4,<2` versions affected**. Hard floor on a core dependency.
-- **#1247** — Windows ONNX `bad_alloc` + convo mine crash.
-- **#1488** — MCP server fails on Cyrillic/non-ASCII on Windows (cp1252 stdin).
-- **#1570** — openai-compat provider probe blocked (403) on Cloudflare-fronted
+- **#1247** - Windows ONNX `bad_alloc` + convo mine crash.
+- **#1488** - MCP server fails on Cyrillic/non-ASCII on Windows (cp1252 stdin).
+- **#1570** - openai-compat provider probe blocked (403) on Cloudflare-fronted
endpoints because of the default `Python-urllib` User-Agent.
### 7. MCP correctness & scale
-- **#1574** — MCP returns generic `-32000` instead of `-32602 (Invalid params)`
+- **#1574** - MCP returns generic `-32000` instead of `-32602 (Invalid params)`
on missing required params.
-- **#1580** — `_expand_with_neighbors` stitches *unrelated* chunks across MCP
+- **#1580** - `_expand_with_neighbors` stitches *unrelated* chunks across MCP
drawers that share an empty `source_file` (silent cross-contamination of
results).
-- **#1379** — overview tools time out on large palaces.
-- **#1471** — "HNSW capacity divergence" check runs per-request in the MCP
+- **#1379** - overview tools time out on large palaces.
+- **#1471** - "HNSW capacity divergence" check runs per-request in the MCP
server (perf).
### 8. Capture noise (markdown/transcript parsing)
-- **#1333** — `<local-command-*>` tags and ANSI escapes survive
+- **#1333** - `<local-command-*>` tags and ANSI escapes survive
`strip_noise()`, polluting drawers (same class of problem as basic-memory
#738 and our own Claude-Code-tag stripping).
@@ -175,33 +175,33 @@ issues rather than our own.
| MemPalace pain | ai-memory invariant that prevents it |
|---|---|
-| #1497/#1514/#1343 — debating/retrofitting a single-writer gateway; concurrent writers corrupt HNSW | **#2 Single-writer SQLite actor.** All writes through one mpsc channel, by construction, since M1. No retrofit. |
-| #1597/#1579/#1537 — "filed N" but not persisted; write returns before durable | **#3 Indexes commit in the same transaction as the data.** No background-task-indexing-after-return; tool responses block until durable. |
-| #1561/#1563 — no stored embedding model identity; drift on model change | **#8 `{provider, model, dim}` stored next to every embedding; refuse on mismatch.** Schema column since M1, enforced at startup. |
-| #1394/#1545/#1586 — repair tooling destroys data it should save | **#17 Wiki migrations: no destructive deletes without a graveyard step** (`_graveyard/<migration>/…`) + **#10 atomic tmp+rename+fsync**. |
-| #1593/#1533 — miners/swaps destroy verbatim history | **#10 Atomic file writes** + git-versioned wiki (M5): every page change is a commit, recoverable via `git`. |
-| #1355/#1247/#1488 — native ONNX + Rust-bindings segfaults, platform floors | **Bundled `rusqlite` (static), pure-Rust, zero-LLM default path (#13).** Vectors (sqlite-vec) deferred to v0.2 precisely to avoid this class of dependency risk. |
-| #1596/#1253 — hooks spawn N concurrent writer subprocesses | **#5 Hooks are fire-and-forget POST to one server** with one writer actor — never a process per event. |
-| #1329 — verbatim-everything → 1.9 TB bloat | **Karpathy compile-don't-hoard** + **M8 decay/forget-sweep** + log rotation. We compress, we don't accumulate raw forever. |
-| #1574 — wrong JSON-RPC error codes | rmcp typed `McpError` variants. (Worth a spot-check that we map param errors correctly — see "gotchas".) |
+| #1497/#1514/#1343 - debating/retrofitting a single-writer gateway; concurrent writers corrupt HNSW | **#2 Single-writer SQLite actor.** All writes through one mpsc channel, by construction, since M1. No retrofit. |
+| #1597/#1579/#1537 - "filed N" but not persisted; write returns before durable | **#3 Indexes commit in the same transaction as the data.** No background-task-indexing-after-return; tool responses block until durable. |
+| #1561/#1563 - no stored embedding model identity; drift on model change | **#8 `{provider, model, dim}` stored next to every embedding; refuse on mismatch.** Schema column since M1, enforced at startup. |
+| #1394/#1545/#1586 - repair tooling destroys data it should save | **#17 Wiki migrations: no destructive deletes without a graveyard step** (`_graveyard/<migration>/…`) + **#10 atomic tmp+rename+fsync**. |
+| #1593/#1533 - miners/swaps destroy verbatim history | **#10 Atomic file writes** + git-versioned wiki (M5): every page change is a commit, recoverable via `git`. |
+| #1355/#1247/#1488 - native ONNX + Rust-bindings segfaults, platform floors | **Bundled `rusqlite` (static), pure-Rust, zero-LLM default path (#13).** Vectors (sqlite-vec) deferred to v0.2 precisely to avoid this class of dependency risk. |
+| #1596/#1253 - hooks spawn N concurrent writer subprocesses | **#5 Hooks are fire-and-forget POST to one server** with one writer actor - never a process per event. |
+| #1329 - verbatim-everything → 1.9 TB bloat | **Karpathy compile-don't-hoard** + **M8 decay/forget-sweep** + log rotation. We compress, we don't accumulate raw forever. |
+| #1574 - wrong JSON-RPC error codes | rmcp typed `McpError` variants. (Worth a spot-check that we map param errors correctly - see "gotchas".) |
## What ai-memory could learn / steal from MemPalace
-These are the genuinely useful ideas worth a brainstorm — NOT yet adopted.
+These are the genuinely useful ideas worth a brainstorm - NOT yet adopted.
1. **A verbatim recall tier as a safety net under the compiled wiki.** This is
the deepest idea. MemPalace proves that *raw text + local semantic search*
scores 96.6% R@5 on LongMemEval with zero LLM. ai-memory compiles
- observations into wiki pages — which is higher-signal but **lossy**:
+ observations into wiki pages - which is higher-signal but **lossy**:
consolidation can drop a detail the user later needs. We already keep `raw/`
immutable session logs. When vectors land (v0.2), we could expose semantic
search over the raw observation log as a *fallback recall path* when the
- compiled wiki misses — "compiled-first, verbatim-fallback." Best of both
+ compiled wiki misses - "compiled-first, verbatim-fallback." Best of both
philosophies.
2. **LongMemEval as a first-class, honest eval (our M10).** MemPalace has the
harness wired and *publishes held-out R@5 with retractions* in
- `docs/HISTORY.md`. Their `tests/test_readme_claims.py` — tests that fail the
- build if a public claim isn't backed by code — is a clever, stealable
+ `docs/HISTORY.md`. Their `tests/test_readme_claims.py` - tests that fail the
+ build if a public claim isn't backed by code - is a clever, stealable
pattern. Adopt the methodology (held-out split, no headline-chasing) when we
build M10.
3. **Deterministic IDs for idempotent, resume-safe ingest.** MemPalace's sweeper
@@ -213,7 +213,7 @@ These are the genuinely useful ideas worth a brainstorm — NOT yet adopted.
table; MemPalace's KG (`knowledge_graph.py`) and their #1416 "promote drawer
claims into KG triples at end of day" rhymes with our consolidation. Could be
a future tier for "facts that change over time" (who owns X now vs. before).
-5. **Index-health monitoring + a repair path — done safely.** When sqlite-vec
+5. **Index-health monitoring + a repair path - done safely.** When sqlite-vec
lands in v0.2, monitor index health (MemPalace's link-to-data ratio guard is
smart) AND make repair graveyard-first. Their #1394/#1595 are the cautionary
tales: a repair that destroys rows, and a health check that parses a field
@@ -223,18 +223,18 @@ These are the genuinely useful ideas worth a brainstorm — NOT yet adopted.
## Gotchas to AVOID that bit MemPalace (cautions for our v0.2 vector work)
-- **Never report "indexed/saved" before durable commit** — even for vectors.
+- **Never report "indexed/saved" before durable commit** - even for vectors.
MemPalace #1597 is the whole movie. Invariant #3 already covers this; keep it
ironclad when sqlite-vec writes land.
- **Stamp `{provider, model, dim}` on the collection and refuse on mismatch.**
- MemPalace #1561 shows the cost of not doing it. We have it — don't regress.
+ MemPalace #1561 shows the cost of not doing it. We have it - don't regress.
- **Any re-embed / rebuild-index path must graveyard before delete.** MemPalace
#1394 destroyed 99% of rows in a "repair." Invariant #17 already mandates
this; apply it to vector rebuilds too.
- **Keep hooks one-writer fire-and-forget.** Do not, under load, spawn a worker
per event (MemPalace #1596). Our design is safe; keep it.
- **Watch raw-log growth.** Verbatim is great for recall but #1329 hit 1.9 TB.
- Our compile + decay + log rotation is the antidote — but if we add a verbatim
+ Our compile + decay + log rotation is the antidote - but if we add a verbatim
recall tier (idea #1), give it a retention budget from the start.
- **Spot-check MCP param-error codes.** MemPalace #1574 returns the wrong
JSON-RPC code on missing params. Confirm rmcp maps our missing/invalid params
@@ -251,16 +251,16 @@ name, the install one-liner (`curl … | sh`), and the Docker image. Cheap
pre-emptive moves: claim the obvious domains, document the *only* official
install sources (GitHub repo, the `akitaonrails/ai-memory` Docker Hub image,
the raw wrapper URL on `main`), and pin/sign releases so users can verify. This
-is a "popular-project" problem we don't have yet — but it's free to pre-empt.
+is a "popular-project" problem we don't have yet - but it's free to pre-empt.
## Bottom line
MemPalace is a strong, popular, well-tested retrieval engine with a smart
-verbatim philosophy and admirably honest maintainers — and it is **drowning in
+verbatim philosophy and admirably honest maintainers - and it is **drowning in
exactly the storage/concurrency/durability failures ai-memory designed around
-from the first milestone**. The single most valuable thing to take from it is
+from the first milestone**. The single most useful thing to take from it is
not a fix but a feature idea: a *verbatim semantic-recall fallback* beneath the
compiled wiki, so we get MemPalace's lossless recall without inheriting its
-ChromaDB corruption surface. The single most valuable thing to take as
-reassurance: our invariants #2, #3, #8, #10, #17 are not over-engineering — they
+ChromaDB corruption surface. The single most useful thing to take as
+reassurance: our invariants #2, #3, #8, #10, #17 are not over-engineering - they
are precisely the 241-open-issues someone else is living through.
diff --git a/docs/lifecycle-ops.md b/docs/lifecycle-ops.md
index 3daee233..f923c162 100644
--- a/docs/lifecycle-ops.md
+++ b/docs/lifecycle-ops.md
@@ -4,7 +4,7 @@ Reference for the destructive / state-touching ai-memory commands.
Read this before running anything that mutates wiki + db, especially
on a homelab box where mistakes are harder to undo.
-## TL;DR — safety matrix
+## TL;DR - safety matrix
| Command | Safe with server **running**? | Wipes data? | Reversible? | Notes |
|---|---|---|---|---|
@@ -44,7 +44,7 @@ The mutable **project name** (the human-readable `distrobox-gaming`
or `.config` you see in `/web/`) never appears in any disk path; the
stable **project_id UUID** does. SQLite's `projects.name` column maps
name → id. Two projects can have the exact same `pages.path` (e.g.
-both have `decisions/0001.md`) without colliding on disk — the
+both have `decisions/0001.md`) without colliding on disk - the
namespaced layout guarantees structural isolation.
The git history is rooted at `<wiki_root>` (one repo, all projects
@@ -66,7 +66,7 @@ What happens, in order:
if either is missing.
2. Counts rows that will cascade (`pages`, `sessions`,
`observations`, `handoffs`, `page_embeddings`).
-3. Single `DELETE FROM projects WHERE id = ?` — the V01 + V05
+3. Single `DELETE FROM projects WHERE id = ?` - the V01 + V05
`ON DELETE CASCADE` foreign keys propagate to every dependent
table in one transaction.
4. `std::fs::remove_dir_all(<wiki_root>/<workspace_id>/<project_id>)`
@@ -89,7 +89,7 @@ Why this is safe with the server running:
it against any other writes.
- The on-disk delete touches only the project's UUID-keyed subdir,
which no other project shares files with. No race with the
- watcher even mid-write — at worst the watcher emits delete
+ watcher even mid-write - at worst the watcher emits delete
events for files we just removed, which it ignores (no DB row
to reindex).
@@ -127,7 +127,7 @@ ai-memory backup --output-path /tmp/ai-memory-backup.tar.gz
What happens on the server:
-1. SQLite online-backup API copies the live WAL DB to a temp file —
+1. SQLite online-backup API copies the live WAL DB to a temp file -
guaranteed consistent snapshot without stopping the writer.
2. Server tar-gzips the snapshot + the wiki tree + `config.toml`.
3. Response body IS the gzipped tarball
@@ -135,7 +135,7 @@ What happens on the server:
CLI writes the response body to `--output-path`. For a homelab user
this is the standard "snapshot before doing something dangerous"
-move — `ai-memory backup` first, then proceed.
+move - `ai-memory backup` first, then proceed.
Restoring a backup follows the inverse:
@@ -171,7 +171,7 @@ Order of operations:
Failure modes:
- **Server still running** → exits with "another ai-memory process is
- alive (pid X); stop it before restoring" — same wording as `reset`.
+ alive (pid X); stop it before restoring" - same wording as `reset`.
- **`--confirm` omitted** → exits with usage hint.
- **Data dir not empty + no `--force`** → exits with "data dir not
empty; pass `--force` to overwrite".
@@ -187,13 +187,13 @@ alive. Removes the contents of `wiki/`, `db/`, and `raw/` under the
configured data dir. `config.toml` is preserved.
Identical sysinfo guard to `restore`. The use case is "wipe and start
-over" — typically when changing major version with a breaking
+over" - typically when changing major version with a breaking
migration, or when bootstrapping a new install on top of an old
data dir.
For a docker deploy where the data lives in a host-path bind mount,
you can also just `rm -rf <host-path>/*` after stopping the
-container — but `ai-memory reset` is the cross-platform path that
+container - but `ai-memory reset` is the cross-platform path that
works whether the data dir is local, bind-mounted, or in a named
volume.
diff --git a/docs/llm-provider-comparison.md b/docs/llm-provider-comparison.md
index b9306e0b..7f7e4ca1 100644
--- a/docs/llm-provider-comparison.md
+++ b/docs/llm-provider-comparison.md
@@ -1,4 +1,4 @@
-# LLM provider comparison — local Ollama vs hosted OpenRouter
+# LLM provider comparison - local Ollama vs hosted OpenRouter
> **TL;DR.** ai-memory's consolidation prompt had a latent
> schema-vs-prompt bug that made every provider fail JSON validation.
@@ -37,7 +37,7 @@
When the homelab deploy switched ai-memory off the billed
OpenAI / OpenRouter providers and onto the locally-hosted Ollama
-server, we needed empirical evidence — not a vibes-based claim —
+server, we needed empirical evidence - not a vibes-based claim -
that *consolidation quality didn't degrade*. ai-memory's
consolidator turns a session's raw observations into 1–5 wiki
pages classified as `concept`, `decision`, `gotcha`, or `rule`;
@@ -78,7 +78,7 @@ exactly as the production hook ingress emits them.
Per fixture, the runner calls
[`ai_memory_consolidate::build_batch_request(session_id, &observations)`](../crates/ai-memory-consolidate/src/consolidator.rs)
-— the **same** function the live consolidator uses on every
+- the **same** function the live consolidator uses on every
`memory_consolidate` invocation. That request is then sent
through [`ai_memory_llm::complete_structured`](../crates/ai-memory-llm/src/lib.rs)
(also the live path). Apples-to-apples by construction.
@@ -98,11 +98,11 @@ The home server (`192.168.0.90`) is a Ryzen AI MAX+ 395
(Strix Halo / gfx1151), 96 GB unified memory, ROCm-backed
Ollama with `OLLAMA_KEEP_ALIVE=20m` + `OLLAMA_FLASH_ATTENTION=1`
+ `OLLAMA_KV_CACHE_TYPE=q8_0`. Once a model is loaded into
-unified memory it stays warm for 20 min — so the first
+unified memory it stays warm for 20 min - so the first
request pays a 30–60 s cold-load tax and subsequent ones are
sub-3 s.
-## Run 1 — broken prompts + schema (pre-fix baseline)
+## Run 1 - broken prompts + schema (pre-fix baseline)
Every provider failed schema validation on every fixture:
@@ -121,7 +121,7 @@ extracted faithful summaries, and respected the path
conventions. The failures were **format only**:
- **Kimi** was emitting beautifully formatted markdown
- (`### Update 1` / `**path:**` / `**body:**`) — completely
+ (`### Update 1` / `**path:**` / `**body:**`) - completely
ignoring the request for JSON.
- **qwen3** was emitting clean JSON in code fences, but with
`tier: 1` / `tier: 2` / `tier: 3` (integers) instead of the
@@ -133,7 +133,7 @@ conventions. The failures were **format only**:
Two separate problems, both **on our side**:
-### Bug A — `Tier` had no `JsonSchema` derive
+### Bug A - `Tier` had no `JsonSchema` derive
In `crates/ai-memory-consolidate/src/types.rs`:
@@ -149,18 +149,18 @@ pub struct ConsolidatedPageUpdate {
`schemars` couldn't produce an enum constraint for `tier`
because `Tier` (the actual enum in `ai-memory-core`) didn't
have the `JsonSchema` derive. The generated schema field was
-just `{ "type": "string" }` — no `enum` constraint — so models
+just `{ "type": "string" }` - no `enum` constraint - so models
were free to guess. Both Kimi and qwen3 guessed numeric indices.
-### Bug B — prompt described values, didn't enforce them
+### Bug B - prompt described values, didn't enforce them
The system prompt in
[`build_batch_request`](../crates/ai-memory-consolidate/src/consolidator.rs)
listed the valid `tier` and `kind` values in prose but never
said "use these EXACT string values, never an integer, never a
-synonym, never code fences". Local instruction-tuned models —
+synonym, never code fences". Local instruction-tuned models -
especially when there's no `response_format: json_schema`
-support to enforce — will drift to whatever feels natural.
+support to enforce - will drift to whatever feels natural.
Compounding this: openai-compat providers (Ollama, OpenRouter
passthrough) do **not** expose strict-mode JSON-schema
@@ -185,7 +185,7 @@ Three small changes landed together:
pub enum Tier { Working, Episodic, Semantic, Procedural }
```
-Adds `schemars` as a dep on `ai-memory-core` (acceptable —
+Adds `schemars` as a dep on `ai-memory-core` (acceptable -
schemars is already a workspace dep used by every type that
crosses the LLM boundary).
@@ -232,7 +232,7 @@ Belt-and-suspenders: the schema now *rejects* the bad values,
and the prompt makes it actively hard for the model to produce
them in the first place.
-## Run 2 — schema + first prompt fix
+## Run 2 - schema + first prompt fix
After the schema fix + first prompt iteration, the same five
fixtures produced:
@@ -245,16 +245,16 @@ fixtures produced:
| 02 architecture-decision | ✓ | 31,039 | 4 | ✓ | 122,200 | 5 |
| 03 gotcha-with-rule | ✓ | 19,173 | 4 | ✓ | 98,025 | 4 |
| 04 low-signal-session | ✓ | 6,106 | **1** | ✓ | 51,694 | **1** |
-| 05 multi-topic-session | ✓ | 47,249 | 4 | ✗* | 133,178 | — |
-| **Aggregate** | **5/5** | **avg 26 s** | — | **4/5** | **avg 103 s** | — |
+| 05 multi-topic-session | ✓ | 47,249 | 4 | ✗* | 133,178 | - |
+| **Aggregate** | **5/5** | **avg 26 s** | - | **4/5** | **avg 103 s** | - |
*qwen3's only failure: invented `kind: "concept"` (not in the
-`PageKind` enum — valid values are `decision`/`gotcha`/`rule`/
+`PageKind` enum - valid values are `decision`/`gotcha`/`rule`/
`fact`). Despite the prompt mentioning the valid set, the
model drifted. **This gets fixed in Run 3 below.**
Both models **correctly restrained themselves** on fixture
-04 (low-signal-session) and produced a single update — a
+04 (low-signal-session) and produced a single update - a
non-trivial test the original schema-broken Run 1 couldn't
even reach.
@@ -269,12 +269,12 @@ Same prompt, both Anthropic models side-by-side:
| 03 gotcha-with-rule | ✓ | 24,810 | 4 | ✓ | 14,304 | 4 |
| 04 low-signal-session | ✓ | 5,673 | **1** | ✓ | 4,044 | **1** |
| 05 multi-topic-session | ✓ | 39,189 | 4 | ✓ | 16,026 | 4 |
-| **Aggregate** | **5/5** | **avg 27 s** | — | **5/5** | **avg 13 s** | — |
+| **Aggregate** | **5/5** | **avg 27 s** | - | **5/5** | **avg 13 s** | - |
**Haiku is ~2× faster than Sonnet on every fixture**, hits the
same 5/5 parse rate, and on the gotcha-with-rule fixture
correctly classified the `audit-ignore-with-revisit-date`
-convention as `kind: rule` — which **Sonnet missed**, calling
+convention as `kind: rule` - which **Sonnet missed**, calling
it a generic `gotcha`. The auto-routing to `_rules/<slug>.md`
that the consolidator depends on therefore *only fires under
Haiku* for that fixture, not Sonnet.
@@ -285,7 +285,7 @@ faithfulness than Sonnet even with the loose prompt:
- **Sonnet** invented `Date: 2025-01-23` twice in fixture 5
(no date in the source observations); fabricated an entire
`## Alternatives considered` section listing Alpine/Scratch/
- Debian-slim — none mentioned in the session; added "Better
+ Debian-slim - none mentioned in the session; added "Better
long-term solutions" / "When NOT to ignore" filler.
- **Haiku** had a couple of invented "Options considered"
entries (Alpine, aggressive optimization flags) but
@@ -295,7 +295,7 @@ For consolidation, the headroom Sonnet has over Haiku
expressed itself as *more hallucination*, not better
fidelity.
-### Kimi-K2.6 (OpenRouter) — INELIGIBLE for this task
+### Kimi-K2.6 (OpenRouter) - INELIGIBLE for this task
After the prompt + schema fixes, the Kimi rerun **hung for
16+ minutes on the first fixture** and never returned a parseable
@@ -326,7 +326,7 @@ instructions before *either* emitting JSON or running out of
budget with no content. The eval observed 16 minutes of no
progress on fixture 1 before being killed.
-This is **not a fixable prompt or schema issue** — it's a
+This is **not a fixable prompt or schema issue** - it's a
property of the model's response style. Run 1 only "worked"
on Kimi (in the sense of producing *something*) because the
loose prompt let Kimi emit prose markdown, which used `content`
@@ -336,18 +336,18 @@ reasoning mode and starves the visible response.
**Kimi-K2.6 is not a suitable provider for ai-memory's
consolidation workload.** It would work for the broader
"summarise this for me" use case where formatted prose is
-fine — just not for our JSON-schema-validated path.
+fine - just not for our JSON-schema-validated path.
Other reasoning-mode models (Claude with extended thinking,
GPT-o3, Gemini "thinking" variants) would need the same
caveat: turn off reasoning mode, or budget tokens with
reasoning consumption in mind.
-## Run 3 — tightened anti-hallucination system prompt
+## Run 3 - tightened anti-hallucination system prompt
The Run 2 evidence above showed that Sonnet was hallucinating
dates, fabricating "Alternatives considered" tables, and
-inventing tutorial sections — content that wasn't in the
+inventing tutorial sections - content that wasn't in the
observations. Even Haiku slipped occasionally. The fix wasn't
a model swap; it was tightening the **system prompt** to
demand faithfulness explicitly:
@@ -386,7 +386,7 @@ This change is in
[`crates/ai-memory-consolidate/src/consolidator.rs`](../crates/ai-memory-consolidate/src/consolidator.rs)
under `pub const BATCH_SYSTEM_PROMPT`.
-### Same fixtures, tightened prompt — Haiku vs Sonnet
+### Same fixtures, tightened prompt - Haiku vs Sonnet
| Metric | Sonnet (old prompt) | Sonnet (tightened) | Δ |
|---|---|---|---|
@@ -404,7 +404,7 @@ under `pub const BATCH_SYSTEM_PROMPT`.
| Updates per fixture | 5-4-4-1-4 | 4-2-4-1-3 | fewer manufactured pages |
| Invented "Options considered" filler | a few | 0 | ✓ gone |
-### Same prompt against the local model — Haiku vs qwen3:32b
+### Same prompt against the local model - Haiku vs qwen3:32b
| Fixture | Haiku parse | Haiku ms | Haiku updates | qwen3 parse | qwen3 ms | qwen3 updates |
|---|---|---|---|---|---|---|
@@ -413,18 +413,18 @@ under `pub const BATCH_SYSTEM_PROMPT`.
| 03 gotcha-with-rule | ✓ | 7,610 | 3 | ✓ | 91,307 | 3 |
| 04 low-signal-session | ✓ | 2,922 | **1** | ✓ | 44,502 | **1** |
| 05 multi-topic-session | ✓ | 9,681 | 3 | ✓ | 122,220 | 5 |
-| **Aggregate** | **5/5** | **avg 8 s** | — | **5/5** | **avg 92 s** | — |
+| **Aggregate** | **5/5** | **avg 8 s** | - | **5/5** | **avg 92 s** | - |
-**qwen3 went from 4/5 → 5/5** with the tightened prompt — the
+**qwen3 went from 4/5 → 5/5** with the tightened prompt - the
explicit field-by-field enumeration of legal `kind` values
eliminated the "concept" drift that broke Run 2.
-The tightened-prompt change is the highest-leverage diff in
+The tightened-prompt change is the highest-use diff in
the whole investigation. Same models, no infra changes, ~60%
latency reduction, complete elimination of date hallucination
on Sonnet, parse rate parity restored for qwen3.
-## Run 4 — budget-tier hosted comparison
+## Run 4 - budget-tier hosted comparison
After establishing that Haiku 4.5 dominates Sonnet 4.5 on this
task, the question became "is there an even cheaper hosted
@@ -439,7 +439,7 @@ option that still works?" Tested two:
| 03 gotcha-with-rule | ✓ | 4,212 | 4 | ✓ | 8,211 | 3 |
| 04 low-signal-session | ✓ | 4,636 | **2*** | ✓ | 3,258 | **1** |
| 05 multi-topic-session | ✓ | 3,997 | 3 | ✓ | 10,583 | 4 |
-| **Aggregate** | **5/5** | **avg 4.3 s** | — | **5/5** | **avg 8.0 s** | — |
+| **Aggregate** | **5/5** | **avg 4.3 s** | - | **5/5** | **avg 8.0 s** | - |
*GPT-5.4-mini failed the restraint test on the low-signal
session: it manufactured an extra `decisions/docs-spelling.md`
@@ -464,18 +464,18 @@ invented dates, no fabricated tutorial sections.
| 03 gotcha-with-rule | ✓ | 54,203 | 4 | ✓ | 8,376 | 3 |
| 04 low-signal-session | ✓ | 4,049 | **1** | ✓ | 2,837 | **1** |
| 05 multi-topic-session | ✓ | 15,543 | 5 | ✓ | 7,616 | 3 |
-| **Aggregate** | **5/5** | **avg 21.7 s** | — | **5/5** | **avg 7.4 s** | — |
+| **Aggregate** | **5/5** | **avg 21.7 s** | - | **5/5** | **avg 7.4 s** | - |
-DeepSeek V4 Flash passes every reliability bar — 5/5 parse,
+DeepSeek V4 Flash passes every reliability bar - 5/5 parse,
correct restraint on low-signal, no hallucinated dates (the
"2026-08" that appeared in its output was *legitimately in the
source observations*), correct `kind: rule` classification.
Notable: fixture 3 took 54 s, suggesting variance under load
or extended reasoning. On multi-topic it produced 5 updates
-vs the 3 the other models settled on — slightly more
+vs the 3 the other models settled on - slightly more
exuberant than Haiku.
-## Comprehensive ranking — all six providers
+## Comprehensive ranking - all six providers
Ranking on a 0–5 scale per axis, then aggregated.
**Higher is better** in every column except Cost (where
@@ -484,12 +484,12 @@ Rows sorted by overall fitness for ai-memory.
| # | Provider | Parse | Speed | Cost† | Faithfulness | Restraint | Classification | Fitness |
|---|---|---|---|---|---|---|---|---|
-| 1 | **Haiku 4.5** | 5 | 4 | 3 | 5 | **5** | **5** | **5** — recommended default |
-| 2 | **GPT-5.4-mini** | 5 | **5** | **5** | 5 | 3 | 4 | 4 — cheaper alternative |
-| 3 | **qwen3:32b (Ollama)** | 5 | 1 | **5** ($0) | 5 | **5** | 4 | 4 — free if you have a local server |
-| 4 | DeepSeek V4 Flash | 5 | 2 | 4 | 5 | **5** | **5** | 4 — no edge over GPT-mini or Haiku |
-| 5 | Sonnet 4.5 | 5 | 4 | 1 | 4‡ | 4 | 3 | 3 — displaced by Haiku |
-| 6 | Kimi-K2.6 | 0 | 0 | n/a | n/a | n/a | n/a | **0** — ineligible (reasoning model) |
+| 1 | **Haiku 4.5** | 5 | 4 | 3 | 5 | **5** | **5** | **5** - recommended default |
+| 2 | **GPT-5.4-mini** | 5 | **5** | **5** | 5 | 3 | 4 | 4 - cheaper alternative |
+| 3 | **qwen3:32b (Ollama)** | 5 | 1 | **5** ($0) | 5 | **5** | 4 | 4 - free if you have a local server |
+| 4 | DeepSeek V4 Flash | 5 | 2 | 4 | 5 | **5** | **5** | 4 - no edge over GPT-mini or Haiku |
+| 5 | Sonnet 4.5 | 5 | 4 | 1 | 4‡ | 4 | 3 | 3 - displaced by Haiku |
+| 6 | Kimi-K2.6 | 0 | 0 | n/a | n/a | n/a | n/a | **0** - ineligible (reasoning model) |
† Cost score derived from order-of-magnitude $/M output tokens:
$0 (qwen3) = 5; ~$0.40 (DeepSeek) = 4; ~$1 (GPT-mini) = 5 by
@@ -497,7 +497,7 @@ amortised cost-per-task; ~$5 (Haiku) = 3; ~$15 (Sonnet) = 1.
‡ Sonnet's faithfulness was 2/5 with the loose prompt (invented
dates, fabricated alternatives sections). It recovers to 4
-after the tightened prompt — but Haiku achieved that same level
+after the tightened prompt - but Haiku achieved that same level
without needing the prompt change as much, suggesting Haiku has
better defaults for this task.
@@ -513,7 +513,7 @@ better defaults for this task.
Inexpensive options change the cost from monthly-recurring
to negligible.
- **Faithfulness**: does the model only write what's in the
- observations? Critical for a *memory* wiki — fabrication
+ observations? Critical for a *memory* wiki - fabrication
corrupts the long-term record.
- **Restraint**: does the model resist manufacturing pages
when the session is low-signal? Lack of restraint pollutes
@@ -531,43 +531,43 @@ better defaults for this task.
For ai-memory's consolidation task specifically:
-1. **Haiku 4.5** — **recommended default for most users.**
+1. **Haiku 4.5** - **recommended default for most users.**
Hosted (always available), 7 s avg latency, restraint +
classification top of the field, ~$0.02/run is negligible
for personal use. The benchmark every other option is
measured against.
-2. **GPT-5.4-mini** — **cheaper hosted alternative.** ~5×
+2. **GPT-5.4-mini** - **cheaper hosted alternative.** ~5×
cheaper than Haiku, 2× faster (4 s avg). Only weakness is
mild over-classification on trivial sessions
(manufactures one extra "decisions/" page on a typo-fix
session). If budget matters more than restraint, pick
this.
-3. **qwen3:32b on Ollama** — **free alternative for those
+3. **qwen3:32b on Ollama** - **free alternative for those
with a local server.** $0 per consolidation. ~92 s
latency is invisible because consolidation is a background
job. Restraint + faithfulness match the top hosted models.
Requires Ollama (or compatible OpenAI-compat server) with
`qwen3:32b` pulled and enough RAM/VRAM (~20 GB) to keep
it warm.
-4. **DeepSeek V4 Flash** — **solid but no clear edge.** All
+4. **DeepSeek V4 Flash** - **solid but no clear edge.** All
reliability bars met, faithful, restrained, correctly
classifies rules. But GPT-mini matches it on quality and
beats it on speed; Haiku matches it on quality and beats
it on classification consistency. Pick only if your
workflow is already DeepSeek-leaning.
-5. **Sonnet 4.5** — **strictly dominated by Haiku** for
+5. **Sonnet 4.5** - **strictly dominated by Haiku** for
plain consolidation. 3× the cost for the same parse rate
and only marginally different latency. Reserve for tasks
that *specifically* need extended reasoning (cross-page
lint sweeps that compare contradictory claims across many
pages, or for sparse-observation sessions where you want
the model to infer more aggressively).
-6. **Kimi-K2.6** — **ineligible.** Reasoning model burns
+6. **Kimi-K2.6** - **ineligible.** Reasoning model burns
`max_tokens` budget on internal thinking before emitting
visible content. Hangs indefinitely on strict-JSON
prompts. Same caveat applies to any other reasoning-mode
model (Claude with extended thinking, GPT-o3, Gemini
- "thinking" variants) — turn reasoning off or budget
+ "thinking" variants) - turn reasoning off or budget
tokens with consumption in mind before using them here.
## Qualitative read (Run 2)
@@ -579,7 +579,7 @@ don't capture:
- **Sonnet writes long, comprehensive entries.** A concept
page on Docker multi-stage builds will get 3 KB of well-
organised prose including "When to use" / "When NOT to use"
- / "Gotchas" sections — content that *wasn't in the
+ / "Gotchas" sections - content that *wasn't in the
observations*. The model is generating useful tutorial-style
content, not strictly consolidating what happened.
Sonnet's fixture 05 page invented a `Date: 2025-01-23`
@@ -616,13 +616,13 @@ prompt), the picture is clear:
consolidation is a background job, not interactive.
- **Cost**: **$0 per consolidation** (electricity not modeled).
- **Fidelity**: comparable to or better than the hosted models
- — qwen3 was the most faithful provider in Run 2's old-prompt
+ - qwen3 was the most faithful provider in Run 2's old-prompt
comparisons.
### Best hosted fallback: Claude Haiku 4.5
If the homelab is unreachable, or for one-off complex
-consolidations, **Haiku 4.5 is the right hosted choice — not
+consolidations, **Haiku 4.5 is the right hosted choice - not
Sonnet 4.5**:
- **2× faster** than Sonnet at every fixture.
@@ -634,7 +634,7 @@ Sonnet 4.5**:
identified the rule that Sonnet flattened to a gotcha).
- Same 5/5 parse reliability.
-### Sonnet 4.5 — displaced by Haiku for this task
+### Sonnet 4.5 - displaced by Haiku for this task
Sonnet's reasoning headroom doesn't help consolidation. With
the loose prompt it expressed itself as *more hallucination*
@@ -645,9 +645,9 @@ cheaper. Reserve Sonnet for tasks where the extra reasoning
matters (e.g. cross-page lint sweeps that compare contradictory
claims).
-### Kimi-K2.6 — ineligible
+### Kimi-K2.6 - ineligible
-Reasoning model — burns `max_tokens` budget internally before
+Reasoning model - burns `max_tokens` budget internally before
emitting visible content. Run hung for 16+ minutes on fixture 1
under the strict-JSON prompt. Direct probe confirmed: `content:
null` with the entire token budget consumed by `reasoning`.
@@ -664,7 +664,7 @@ mode models if used in this pipeline.
| DeepSeek V4 Flash (OpenRouter) | ~$0.005 | ~22 s | cheap but slower than GPT-mini |
| Haiku 4.5 (OpenRouter) | ~$0.02 | ~7 s | best restraint/classification |
| Sonnet 4.5 (OpenRouter) | ~$0.06 | ~11 s | 3× cost of Haiku for same task |
-| Kimi-K2.6 (OpenRouter) | n/a | ✗ hangs | reasoning model — ineligible |
+| Kimi-K2.6 (OpenRouter) | n/a | ✗ hangs | reasoning model - ineligible |
\* Rough order of magnitude; ai-memory consolidations land
around 2–3 KB of output with the tightened prompt. Per-run $
@@ -688,7 +688,7 @@ Re-run this harness when any of the following changes:
- Repo checkout + `cargo` toolchain (Rust 1.95+, as pinned in
`rust-toolchain.toml`).
-- An OpenRouter API key, exported as `OPENROUTER_API_KEY` —
+- An OpenRouter API key, exported as `OPENROUTER_API_KEY` -
pays the Kimi + Sonnet legs.
- A reachable Ollama with `qwen3:32b` pulled. The default URL
in the docs assumes the homelab; substitute your own.
@@ -729,7 +729,7 @@ evals/runs/<timestamp>/
```
The `.raw.txt` files are the most informative artifact when a
-parse fails — they show *exactly* what the model said, so you
+parse fails - they show *exactly* what the model said, so you
can tell whether the failure was format (model emitted prose),
schema (model used integer enums), or substance (model
produced nothing useful).
@@ -763,13 +763,13 @@ back to `Other`.
Try to hit one of the four hard cases:
-1. **Multi-page extraction** — does the model split a session
+1. **Multi-page extraction** - does the model split a session
into the right slices?
-2. **Restraint** — does it avoid manufacturing pages when
+2. **Restraint** - does it avoid manufacturing pages when
there's nothing durable?
-3. **Classification** — does it correctly choose `kind: rule`
+3. **Classification** - does it correctly choose `kind: rule`
for project rules?
-4. **Topic separation** — does it produce separate pages per
+4. **Topic separation** - does it produce separate pages per
unrelated topic instead of mashing them?
## What's NOT in this harness (yet)
diff --git a/docs/mcp-install.md b/docs/mcp-install.md
index 3dc5b869..9ad4352a 100644
--- a/docs/mcp-install.md
+++ b/docs/mcp-install.md
@@ -1,4 +1,4 @@
-# MCP install guide — additional clients
+# MCP install guide - additional clients
> All snippets below default to `http://127.0.0.1:49374` (local server). For a
> remote server (homelab, LAN box) substitute the appropriate URL AND add an
@@ -19,7 +19,7 @@
This page documents how to register ai-memory as an MCP server with
the agent CLIs **that are not covered inline in the README**.
-The three flagship clients — Claude Code, OpenAI Codex, OpenCode —
+The three flagship clients - Claude Code, OpenAI Codex, OpenCode -
have lifecycle-hook scripts under [`hooks/`](../hooks) and are
covered in the [main README](../README.md#configure-your-agent-cli).
@@ -32,7 +32,7 @@ without a per-client plugin written upstream. The trade-off:
| | What you get | What you don't get |
|---|---|---|
| **MCP only** | LLM can query the wiki, accept handoffs, run memory_consolidate | No automatic session-end summaries; no auto-handoff at session boundaries |
-| **MCP + hooks** | All of the above *plus* every prompt/tool-call captured automatically; handoffs surface at SessionStart with no human prompting | — |
+| **MCP + hooks** | All of the above *plus* every prompt/tool-call captured automatically; handoffs surface at SessionStart with no human prompting | - |
Most of these clients will land hook integration upstream eventually
(Gemini CLI is the closest); for now, you can still cover the
@@ -216,7 +216,7 @@ plugin and is out of scope for ai-memory v0.2.
---
-## After registering MCP — verify it works
+## After registering MCP - verify it works
Regardless of which client you used, the first sanity check is the
same: ask the model to list its available MCP tools, or to call
@@ -256,13 +256,13 @@ dir on disk.
## When does the auto-handoff actually work?
The cross-agent handoff feature (the "headline" pitch in the README)
-requires both sides — the agent that *ends* a session, and the agent
-that *starts* the next one — to play nicely with ai-memory:
+requires both sides - the agent that *ends* a session, and the agent
+that *starts* the next one - to play nicely with ai-memory:
| Side | What's needed | Covered by |
|---|---|---|
| **Ending side** | The `SessionEnd` hook must fire ai-memory's `/hook` endpoint. | Built-in for Claude Code / Codex / OpenCode via the scripts in `hooks/`. |
-| **Starting side** | Either (a) the `SessionStart` hook auto-injects the handoff via `/handoff`, OR (b) the model proactively calls `memory_handoff_accept` on first turn. | (a) is built-in for Claude Code / Codex / OpenCode. (b) works for any MCP-capable client if you nudge the model — see [the CLAUDE.md snippet](../README.md#nudging-the-agent-to-use-memory-proactively). |
+| **Starting side** | Either (a) the `SessionStart` hook auto-injects the handoff via `/handoff`, OR (b) the model proactively calls `memory_handoff_accept` on first turn. | (a) is built-in for Claude Code / Codex / OpenCode. (b) works for any MCP-capable client if you nudge the model - see [the CLAUDE.md snippet](../README.md#nudging-the-agent-to-use-memory-proactively). |
So a typical mixed workflow looks like:
diff --git a/docs/research-agentmemory.md b/docs/research-agentmemory.md
index eeb53f62..2535c9a9 100644
--- a/docs/research-agentmemory.md
+++ b/docs/research-agentmemory.md
@@ -1,4 +1,4 @@
-# agentmemory — Research Report
+# agentmemory - Research Report
> Source project: `~/Projects/agentmemory` (TypeScript, MCP server built on `iii-engine`).
> This is the user's own existing project. The Rust effort in this repo is a
@@ -8,11 +8,11 @@
agentmemory is **persistent memory infrastructure for AI coding agents**. The core pitch: an agent silently captures what you do during a coding session (tool calls, prompts, decisions, errors), compresses those raw observations into searchable memory, and re-injects relevant context into the *next* session so the user never has to re-explain architecture, preferences, or past bugs. README pitches a tagline ("Your coding agent remembers everything. No more re-explaining.") with claimed retrieval R@5 of 95.2% on LongMemEval-S vs. 86.2% BM25-only fallback, and ~1,900 tokens/session vs. ~22K for raw CLAUDE.md.
-The author explicitly frames it as the *implementation* of Karpathy's "LLM Wiki" pattern, extended with confidence scoring, lifecycle, knowledge graph, and hybrid search — the project page touts a viral gist with 1200+ stars that articulated the design.
+The author explicitly frames it as the *implementation* of Karpathy's "LLM Wiki" pattern, extended with confidence scoring, lifecycle, knowledge graph, and hybrid search - the project page touts a viral gist with 1200+ stars that articulated the design.
Caveat about `DESIGN.md`: that file in agentmemory is a Lamborghini-inspired *visual* design system for the marketing site, not architecture. The real architecture docs are in `AGENTS.md` and `README.md`.
-`ROADMAP.md` confirms the trajectory: Q2 2026 "Depth" (multimodal), Q3 "Breadth" (more agents, OpenSSF), Q4 "Trust" (SSO/RBAC), Q1 2027 v1.0 freeze. Candidate item for Q1 2027: *"Reference implementation in a second language (Rust or Go)"* — directly relevant to this project.
+`ROADMAP.md` confirms the trajectory: Q2 2026 "Depth" (multimodal), Q3 "Breadth" (more agents, OpenSSF), Q4 "Trust" (SSO/RBAC), Q1 2027 v1.0 freeze. Candidate item for Q1 2027: *"Reference implementation in a second language (Rust or Go)"* - directly relevant to this project.
## 2. Architecture
@@ -24,10 +24,10 @@ Caveat about `DESIGN.md`: that file in agentmemory is a Lamborghini-inspired *vi
config:
adapter: { name: kv, config: { store_method: file_based, file_path: ./data/state_store.db } }
```
- The Node code only sees a tiny shim (`src/state/kv.ts:6-46`) wrapping five RPCs (`state::get/set/list/update/delete`). No direct SQLite, no Postgres, no Qdrant, no graph DB — *all* memory types are stored as JSON values under namespaced "scopes" (e.g. `mem:memories`, `mem:semantic`, `mem:graph:nodes`).
+ The Node code only sees a tiny shim (`src/state/kv.ts:6-46`) wrapping five RPCs (`state::get/set/list/update/delete`). No direct SQLite, no Postgres, no Qdrant, no graph DB - *all* memory types are stored as JSON values under namespaced "scopes" (e.g. `mem:memories`, `mem:semantic`, `mem:graph:nodes`).
- **Indices** live in-process: BM25 (`src/state/search-index.ts`), an in-RAM vector index with cosine similarity (`src/state/vector-index.ts`), and persisted snapshots written back into KV via `IndexPersistence` (`src/state/index-persistence.ts`). Hybrid search uses RRF-style fusion of BM25 + vector + graph (`src/state/hybrid-search.ts`).
-- **Surfaces**: REST API on `:3111` (124 endpoints — `src/triggers/api.ts`), MCP server on stdio via `npx @agentmemory/mcp` (`src/mcp/server.ts`, ~62KB, with 53 tools in `src/mcp/tools-registry.ts`), live WebSocket stream on `:3112`, and a real-time viewer HTML on `:3113`.
-- **KV scope catalogue**: `src/state/schema.ts:3-50` lists ~40 scopes (sessions, observations per-session, memories, summaries, semantic, procedural, graph:nodes/edges, insights, lessons, crystals, sketches, sentinels, actions, leases, routines, signals, checkpoints, mesh, slots, retention, accessLog, audit, imageRefs, etc.). The breadth is striking — but it's all one SQLite file.
+- **Surfaces**: REST API on `:3111` (124 endpoints - `src/triggers/api.ts`), MCP server on stdio via `npx @agentmemory/mcp` (`src/mcp/server.ts`, ~62KB, with 53 tools in `src/mcp/tools-registry.ts`), live WebSocket stream on `:3112`, and a real-time viewer HTML on `:3113`.
+- **KV scope catalogue**: `src/state/schema.ts:3-50` lists ~40 scopes (sessions, observations per-session, memories, summaries, semantic, procedural, graph:nodes/edges, insights, lessons, crystals, sketches, sentinels, actions, leases, routines, signals, checkpoints, mesh, slots, retention, accessLog, audit, imageRefs, etc.). The breadth is striking - but it's all one SQLite file.
## 3. Memory Model
@@ -37,18 +37,18 @@ export type ConsolidationTier = "working" | "episodic" | "semantic" | "procedura
```
- **Raw observation** (`RawObservation`, `types.ts:29-42`): captured by hooks on every tool call.
-- **Compressed observation** (`types.ts:44-62`): structured XML output from an LLM (`mem::compress`) with `type`, `title`, `facts`, `narrative`, `concepts`, `files`, `importance` 1–10. *Critically* this LLM compression is OFF by default (`src/index.ts:245-253`, issue #138). Default path uses **synthetic** compression (`src/functions/compress-synthetic.ts`) — zero LLM calls — to keep token bills sane. Set `AGENTMEMORY_AUTO_COMPRESS=true` to opt in.
-- **Memory** (`types.ts:81-101`): consolidated long-term entry with `type` (pattern/preference/architecture/bug/workflow/fact), `strength`, `version`, `supersedes`, `isLatest`. Versioned in place — old memory `isLatest=false`, new one keeps `parentId`/`supersedes` chain (`src/functions/consolidate.ts:159-191`).
+- **Compressed observation** (`types.ts:44-62`): structured XML output from an LLM (`mem::compress`) with `type`, `title`, `facts`, `narrative`, `concepts`, `files`, `importance` 1–10. *Critically* this LLM compression is OFF by default (`src/index.ts:245-253`, issue #138). Default path uses **synthetic** compression (`src/functions/compress-synthetic.ts`) - zero LLM calls - to keep token bills sane. Set `AGENTMEMORY_AUTO_COMPRESS=true` to opt in.
+- **Memory** (`types.ts:81-101`): consolidated long-term entry with `type` (pattern/preference/architecture/bug/workflow/fact), `strength`, `version`, `supersedes`, `isLatest`. Versioned in place - old memory `isLatest=false`, new one keeps `parentId`/`supersedes` chain (`src/functions/consolidate.ts:159-191`).
- **SemanticMemory** (`types.ts:435-446`): individual *facts* with confidence + access counts.
- **ProceduralMemory** (`types.ts:448-462`): named procedures with steps + trigger condition.
- **Lesson / Insight / Crystal**: a higher tier of distillation. Crystals (`crystallize.ts`) summarize chains of completed Actions into narrative + key outcomes + lessons; Lessons feed Reflect which produces Insights from concept clusters.
-- **MemorySlot** (`types.ts:222-232`, `functions/slots.ts:13-83`): pinned editable text blocks (persona, user_preferences, project_context, guidance, pending_items, etc.) — Karpathy-wiki-style human-editable section. Always injected into context via `src/functions/context.ts:43-61`.
+- **MemorySlot** (`types.ts:222-232`, `functions/slots.ts:13-83`): pinned editable text blocks (persona, user_preferences, project_context, guidance, pending_items, etc.) - Karpathy-wiki-style human-editable section. Always injected into context via `src/functions/context.ts:43-61`.
**Automatic operations (no manual writes needed)**:
-- `mem::observe` runs from `PostToolUse` hook on every tool call — privacy-stripped, dedup'd, optionally LLM-compressed, indexed, streamed live (`functions/observe.ts:42-280`).
+- `mem::observe` runs from `PostToolUse` hook on every tool call - privacy-stripped, dedup'd, optionally LLM-compressed, indexed, streamed live (`functions/observe.ts:42-280`).
- `setInterval` timers in `src/index.ts:491-531`: auto-forget every 1h, lesson decay every 24h, insight decay every 24h, consolidation pipeline every 2h.
- `mem::auto-forget` (`functions/auto-forget.ts`): deletes TTL-expired memories, soft-deletes contradiction pairs (Jaccard similarity > 0.9), purges 180-day-old observations with importance ≤ 2.
-- `mem::retention-score` (`functions/retention.ts:80-94`): retention = `salience * exp(-λ·Δt) + reinforcementBoost(accessLog, σ)`. Below the "cold" threshold (0.15), entries become evictable. The reinforcement boost (`computeReinforcementBoost`) sums `1/daysSinceAccess` — classic spaced-repetition.
+- `mem::retention-score` (`functions/retention.ts:80-94`): retention = `salience * exp(-λ·Δt) + reinforcementBoost(accessLog, σ)`. Below the "cold" threshold (0.15), entries become evictable. The reinforcement boost (`computeReinforcementBoost`) sums `1/daysSinceAccess` - classic spaced-repetition.
## 4. Reorganization / Consolidation (the Karpathy bit)
@@ -59,30 +59,30 @@ This is the most interesting part. The consolidation pipeline runs on a 2h cron
3. **Procedural tier**: finds Memory rows of type `pattern` with `frequency >= 2`, extracts named `<procedure>` blocks with steps.
4. **Decay tier**: applies geometric decay `strength *= 0.9^decayPeriods` after a configurable inactivity window (`applyDecay` at consolidation-pipeline.ts:21-43).
-Separately, `mem::consolidate` (`functions/consolidate.ts:65-225`) groups observations by concept, picks the top-N most important per concept, and for each cluster either *creates* a Memory or *evolves* an existing one. Evolution = mark old `isLatest=false`, write a new row pointing at it via `supersedes` and `parentId` (lines 161-189). The old memory isn't deleted — it remains versioned-but-shadowed, with `isLatest` filtering at read time.
+Separately, `mem::consolidate` (`functions/consolidate.ts:65-225`) groups observations by concept, picks the top-N most important per concept, and for each cluster either *creates* a Memory or *evolves* an existing one. Evolution = mark old `isLatest=false`, write a new row pointing at it via `supersedes` and `parentId` (lines 161-189). The old memory isn't deleted - it remains versioned-but-shadowed, with `isLatest` filtering at read time.
`mem::insight-decay-sweep` (`functions/reflect.ts:425-476`) runs weekly: `newConfidence = confidence - decayRate * weeksSince`. Below 0.1 with zero reinforcements, soft-deleted.
-This is genuinely Karpathy-wiki-shaped: **memories aren't append-only — they get rewritten in place via versioned supersession, and unused entries quietly fade.**
+This is genuinely Karpathy-wiki-shaped: **memories aren't append-only - they get rewritten in place via versioned supersession, and unused entries quietly fade.**
## 5. Agent Integration
Three surfaces, depending on what the host supports:
-- **Hooks (Claude Code, Codex)**: 12 hook scripts in `src/hooks/` — standalone Node scripts that read JSON from stdin and POST to `/agentmemory/observe` over HTTP with a 3s timeout. `plugin/hooks/hooks.json` registers all 12 (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, PostToolUseFailure, PreCompact, SubagentStart/Stop, Notification, TaskCompleted, Stop, SessionEnd). Codex gets a 6-hook subset (`README.md:419`).
-- **MCP tools**: 53 tools in `src/mcp/tools-registry.ts` — `memory_recall`, `memory_smart_search`, `memory_save`, `memory_sessions`, `memory_consolidate`, `memory_action_create`, `memory_lease`, `memory_signal_send`, `memory_crystallize`, etc. Only 8 are visible by default (`AGENTMEMORY_TOOLS=all` exposes the rest, AGENTS.md:114).
-- **Skills** (slash commands): `plugin/skills/{recall,remember,handoff,recap,forget,session-history,commit-context,commit-history}/SKILL.md` — each is a markdown file with frontmatter the host parses.
+- **Hooks (Claude Code, Codex)**: 12 hook scripts in `src/hooks/` - standalone Node scripts that read JSON from stdin and POST to `/agentmemory/observe` over HTTP with a 3s timeout. `plugin/hooks/hooks.json` registers all 12 (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, PostToolUseFailure, PreCompact, SubagentStart/Stop, Notification, TaskCompleted, Stop, SessionEnd). Codex gets a 6-hook subset (`README.md:419`).
+- **MCP tools**: 53 tools in `src/mcp/tools-registry.ts` - `memory_recall`, `memory_smart_search`, `memory_save`, `memory_sessions`, `memory_consolidate`, `memory_action_create`, `memory_lease`, `memory_signal_send`, `memory_crystallize`, etc. Only 8 are visible by default (`AGENTMEMORY_TOOLS=all` exposes the rest, AGENTS.md:114).
+- **Skills** (slash commands): `plugin/skills/{recall,remember,handoff,recap,forget,session-history,commit-context,commit-history}/SKILL.md` - each is a markdown file with frontmatter the host parses.
- **System-prompt injection**: pre-tool-use and session-start hooks can write up to 4000 chars of memory context to stdout, which Claude Code prepends to the next turn. **Off by default** (`AGENTMEMORY_INJECT_CONTEXT=false`) because of token-burn complaints (#143, see `src/hooks/pre-tool-use.ts:9-22`).
## 6. Cross-Agent Handoff
Two mechanisms:
-- **`/handoff` skill** (`plugin/skills/handoff/SKILL.md`): finds the most recent session whose `cwd` matches your current directory (with proper path-boundary check), surfaces any unanswered question first, then fetches a recall of top concepts. This works *because all agents write to the same `:3111` server* — a Claude Code session and a Codex session in the same project share KV. Cross-agent recall is implicit.
+- **`/handoff` skill** (`plugin/skills/handoff/SKILL.md`): finds the most recent session whose `cwd` matches your current directory (with proper path-boundary check), surfaces any unanswered question first, then fetches a recall of top concepts. This works *because all agents write to the same `:3111` server* - a Claude Code session and a Codex session in the same project share KV. Cross-agent recall is implicit.
- **Signals** (`functions/signals.ts`): a typed inter-agent message bus with `from`/`to`/`threadId`/`type: handoff|request|response|alert|info` and TTLs. `mem::signal-send` + `mem::signal-read` MCP tools.
- **Mesh sync** (`functions/mesh.ts`) does LWW-merge replication between agentmemory *servers* on different machines, but cross-agent on a single box is just the shared SQLite.
-Roadmap explicitly flags "Cross-agent shared memory namespace" as Q3 2026 candidate and "Agent-to-agent memory handoff protocol" for Q4 2026 — so the current support is informal.
+Roadmap explicitly flags "Cross-agent shared memory namespace" as Q3 2026 candidate and "Agent-to-agent memory handoff protocol" for Q4 2026 - so the current support is informal.
## 7. Self-Healing / Operations
@@ -94,22 +94,22 @@ Roadmap explicitly flags "Cross-agent shared memory namespace" as Q3 2026 candid
- **Audit**: every state-changing operation records to `KV.audit` with a typed operation union (`types.ts:493-539`, ~45 op types). `AGENTS.md:39-41` makes adding new ones mandatory.
- **Resilience**: top-level `unhandledRejection` swallow with throttled logging (`src/index.ts:118-128`) to survive iii-engine timeout spikes (#204). Circuit-breaker + fallback-chain providers (`src/providers/`).
-## 8. What's Good / What's Missing — Honest Take
+## 8. What's Good / What's Missing - Honest Take
**What's clever and worth reusing:**
-- **Two-tier compression (synthetic default, LLM opt-in)**: the #138 fix is the right call. Default zero-LLM keeps token bills sane while preserving BM25/vector search. Critical lesson — don't make the LLM call mandatory.
+- **Two-tier compression (synthetic default, LLM opt-in)**: the #138 fix is the right call. Default zero-LLM keeps token bills sane while preserving BM25/vector search. Critical lesson - don't make the LLM call mandatory.
- **Versioned in-place memory evolution** (`isLatest` + `supersedes` chain): exactly the Karpathy wiki-rewrite pattern. Cheaper than maintaining a separate "graveyard", and `parentId` keeps history queryable.
- **Retention-as-formula** rather than rules: `salience * exp(-λΔt) + Σ(σ/daysSinceAccess)`. Tunable, principled, batchable. Worth borrowing wholesale.
- **Triple-stream RRF retrieval** (BM25 + vector + graph): graph-walk-as-third-signal is unusual and the benchmark numbers (95.2% R@5) suggest it pays off.
- **Git-as-snapshot-backup**: dump state.json, `git commit`. Diffable, restorable, no DB-specific tooling needed.
-- **Slots**: pinned, human-editable, always-injected wiki-style sections. The user can hand-edit `project_context` and it survives forever — a clean escape hatch from purely-emergent memory.
-- **Hook scripts as standalone Node files** (no SDK import, just HTTP): fast startup, fault-tolerant. The 800ms-1500ms hard timeouts (`src/hooks/session-start.ts:27-28`) are explicit — there's a war story about #221 where unbounded hook fan-out OOM-killed iii-engine.
+- **Slots**: pinned, human-editable, always-injected wiki-style sections. The user can hand-edit `project_context` and it survives forever - a clean escape hatch from purely-emergent memory.
+- **Hook scripts as standalone Node files** (no SDK import, just HTTP): fast startup, fault-tolerant. The 800ms-1500ms hard timeouts (`src/hooks/session-start.ts:27-28`) are explicit - there's a war story about #221 where unbounded hook fan-out OOM-killed iii-engine.
- **Privacy filter at observe boundary** (`stripPrivateData` before any persistence): defense in depth.
**What feels overengineered:**
-- **~40 KV scopes, 53 MCP tools, 124 REST endpoints, 50+ iii functions** is a *lot* of surface for v0.9. Many feel speculative (sentinels, sketches, frontier, leases, routines, checkpoints, facets, crystals, mesh, branch-aware, flow-compress, vision-search). The AGENTS.md "you must update ALL of the following" 7-step checklists are a smell — the system is so wide that ordinary changes ripple far.
+- **~40 KV scopes, 53 MCP tools, 124 REST endpoints, 50+ iii functions** is a *lot* of surface for v0.9. Many feel speculative (sentinels, sketches, frontier, leases, routines, checkpoints, facets, crystals, mesh, branch-aware, flow-compress, vision-search). The AGENTS.md "you must update ALL of the following" 7-step checklists are a smell - the system is so wide that ordinary changes ripple far.
- **iii-engine dependency**: every install requires a separate Rust binary, pinned to a specific version (0.11.2; 0.11.6 broke them), with no canonical Windows installer (`README.md:549`). For a Rust rewrite this is a *strong argument to be self-contained*: embed SQLite directly, drop the engine.
- **All-JSON storage under one big SQLite file**: every "memories" or "graph nodes" list operation is `state::list` → return *all* JSON, parse in Node, filter in memory. `auto-forget.ts:67` literally caps at 1000 latest memories because the O(N²) Jaccard loop. Won't scale past ~10K memories without proper indexing.
- **XML-in-LLM-output everywhere**: `<memory>...</memory>`, `<temporal_graph>`, `<insight>`. Fragile regex parsing (`parseCompressionXml`, `parseTemporalGraphXml`). A schema-validated JSON / structured-outputs path would be more robust.
@@ -119,13 +119,13 @@ Roadmap explicitly flags "Cross-agent shared memory namespace" as Q3 2026 candid
1. **First-class embedded store.** Use `rusqlite` (or `sqlx` + SQLite/Postgres) with *proper indices and FTS5* instead of JSON-blob-in-KV. The retention/auto-forget logic deserves real WHERE clauses, not in-memory filter passes. Look at `sqlite-vec` or `lancedb` for vectors so you're not maintaining a Map<String, Vec<f32>>.
2. **Self-contained binary.** No separate engine. A single `agentmemory` binary that *is* the MCP server, REST server, and storage. The iii-engine indirection costs operability for ~nothing the user sees.
-3. **Native MCP transport.** Use a proper Rust MCP SDK; you don't need an HTTP shim between hooks and storage — hooks can speak MCP-over-stdio or a Unix socket. Cuts the 3s HTTP timeout and the auth bearer dance.
+3. **Native MCP transport.** Use a proper Rust MCP SDK; you don't need an HTTP shim between hooks and storage - hooks can speak MCP-over-stdio or a Unix socket. Cuts the 3s HTTP timeout and the auth bearer dance.
4. **First-class structured outputs.** Don't parse `<memory><type>...</type></memory>` regex-strings. Use JSON-schema constrained generation; `serde_json` deserialization with strict validation.
5. **Smaller, sharper tool surface.** Pick the 8–12 tools that demonstrably matter (recall, save, sessions, smart_search, handoff, consolidate, forget, governance_delete) and ship those well. The 53-tool surface invites confusion; the README admits 8 are visible by default for a reason.
-6. **Real graph store for temporal edges.** The temporal-graph design (`tvalid`, `tvalidEnd`, `supersededBy`, edge-history) is *good* — but implementing it on JSON blobs in KV means every "what does Alice prefer as of 2024-06-15" query is a full table scan. Consider `petgraph` in memory backed by a real SQL graph table, or a proper graph DB if you're going big.
-7. **Cross-agent handoff as a designed protocol**, not just shared SQLite. The Q4 2026 roadmap candidate is exactly this — get there day one. Define a `Handoff` type with from-agent, to-agent, context, open-questions, files-touched, model-used; expose `mcp::handoff/begin` and `mcp::handoff/accept`.
-8. **Reproducible eval harness.** They ran LongMemEval-S and got 95.2% R@5 — bake the harness into CI from the start so regressions are caught.
+6. **Real graph store for temporal edges.** The temporal-graph design (`tvalid`, `tvalidEnd`, `supersededBy`, edge-history) is *good* - but implementing it on JSON blobs in KV means every "what does Alice prefer as of 2024-06-15" query is a full table scan. Consider `petgraph` in memory backed by a real SQL graph table, or a proper graph DB if you're going big.
+7. **Cross-agent handoff as a designed protocol**, shared SQLite. The Q4 2026 roadmap candidate is exactly this - get there day one. Define a `Handoff` type with from-agent, to-agent, context, open-questions, files-touched, model-used; expose `mcp::handoff/begin` and `mcp::handoff/accept`.
+8. **Reproducible eval harness.** They ran LongMemEval-S and got 95.2% R@5 - bake the harness into CI from the start so regressions are caught.
9. **Single source of truth for schema.** The AGENTS.md "update all 7 files" checklists indicate the schema is duplicated across types, tools-registry, REST, MCP, tests, README, plugin. In Rust, one `proto`-or-derive-driven definition can generate all of those.
10. **A clear `ARCHITECTURE.md`.** Don't repeat their mistake of scattering architecture across AGENTS, README, and inline comments.
-The big takeaway: **agentmemory's *concepts* are unusually thoughtful** — versioned supersession, retention formulas, slot pinning, hybrid retrieval, opt-in LLM compression, session-cwd-based handoff. The *implementation* is constrained by the iii-engine bet and the all-JSON-in-KV storage choice. A Rust rewrite has a real opportunity to keep the ideas and drop the substrate.
+The big takeaway: **agentmemory's *concepts* are unusually thoughtful** - versioned supersession, retention formulas, slot pinning, hybrid retrieval, opt-in LLM compression, session-cwd-based handoff. The *implementation* is constrained by the iii-engine bet and the all-JSON-in-KV storage choice. A Rust rewrite has a real opportunity to keep the ideas and drop the substrate.
diff --git a/docs/research-basic-memory.md b/docs/research-basic-memory.md
index b78f272d..a400b2e6 100644
--- a/docs/research-basic-memory.md
+++ b/docs/research-basic-memory.md
@@ -1,4 +1,4 @@
-# basic-memory — Research Report
+# basic-memory - Research Report
> Source project: `~/Projects/basic-memory` (Python, MCP-native, markdown-on-disk).
> Studied as inspiration *and* as the manual-write-note model we explicitly
@@ -8,7 +8,7 @@
Basic Memory is a **local-first, MCP-native, Markdown-based personal knowledge graph**. Its tagline is "Your AI never forgets again": notes live as plain Markdown files on disk, both humans (in Obsidian, VS Code) and LLMs (over MCP) read and write them, and a SQLite/Postgres index keeps a knowledge graph in sync.
-**The model is explicitly *manual*.** The user (or, more often, the agent in response to the user) calls `write_note`, `edit_note`, `move_note`, etc. There is **no implicit capture** of conversation content. The "How it works" example in `README.md:317-356` is literally *"Ask the LLM to capture it: 'Make a note on coffee brewing methods.'"* The README even prescribes user prompts ("Create a note about our project architecture decisions") as the activation step. Nothing in the codebase auto-saves conversation turns. The closest thing to automation is the `continue_conversation` *prompt* (`src/basic_memory/mcp/prompts/continue_conversation.py:18-90`), which only *retrieves* — it asks the model to search recent activity and load context; it never writes.
+**The model is explicitly *manual*.** The user (or, more often, the agent in response to the user) calls `write_note`, `edit_note`, `move_note`, etc. There is **no implicit capture** of conversation content. The "How it works" example in `README.md:317-356` is literally *"Ask the LLM to capture it: 'Make a note on coffee brewing methods.'"* The README even prescribes user prompts ("Create a note about our project architecture decisions") as the activation step. Nothing in the codebase auto-saves conversation turns. The closest thing to automation is the `continue_conversation` *prompt* (`src/basic_memory/mcp/prompts/continue_conversation.py:18-90`), which only *retrieves* - it asks the model to search recent activity and load context; it never writes.
## 2. Storage Model
@@ -40,7 +40,7 @@ There is also a `NoteContent` table (`knowledge.py:152-217`) that **materializes
## 3. MCP Tools Exposed
-All tools are registered via the `@mcp.tool` decorator and exported from `src/basic_memory/mcp/tools/__init__.py:9-65`. Every tool is annotated with MCP `readOnlyHint`/`destructiveHint`/`idempotentHint`/`openWorldHint` so agents can pick safely. Every single tool requires **explicit invocation** — none are triggered by the server itself.
+All tools are registered via the `@mcp.tool` decorator and exported from `src/basic_memory/mcp/tools/__init__.py:9-65`. Every tool is annotated with MCP `readOnlyHint`/`destructiveHint`/`idempotentHint`/`openWorldHint` so agents can pick safely. Every single tool requires **explicit invocation** - none are triggered by the server itself.
| Tool | Hint | File |
|---|---|---|
@@ -62,46 +62,46 @@ Two MCP **prompts** ship: `continue_conversation` and `recent_activity` (both re
## 4. Memory Lifecycle
-**There is no lifecycle.** Grep'd the entire `src/` tree for `decay|aging|consolidat|summariz|forget|expire|ttl|prune|archive_old` — zero matches outside of unrelated OAuth-token expiry and SQLAlchemy `expire_on_commit`. Notes are **append-only forever** unless a human or agent explicitly calls `delete_note`, `move_note`, or `edit_note`. There is no auto-summarization, no auto-merge of duplicates, no recency weighting in search ranking (only an `after_date` filter), no "cold storage" tier. `recent_activity` (`tools/recent_activity.py:28`) just queries by `created_at`/`updated_at`; it doesn't shape memory.
+**There is no lifecycle.** Grep'd the entire `src/` tree for `decay|aging|consolidat|summariz|forget|expire|ttl|prune|archive_old` - zero matches outside of unrelated OAuth-token expiry and SQLAlchemy `expire_on_commit`. Notes are **append-only forever** unless a human or agent explicitly calls `delete_note`, `move_note`, or `edit_note`. There is no auto-summarization, no auto-merge of duplicates, no recency weighting in search ranking (only an `after_date` filter), no "cold storage" tier. `recent_activity` (`tools/recent_activity.py:28`) just queries by `created_at`/`updated_at`; it doesn't shape memory.
The only background process is the `WatchService` (`src/basic_memory/sync/watch_service.py:81-145`) + `SyncService` (`src/basic_memory/sync/sync_service.py:153-188`), which reconcile file changes with the DB after a `sync_delay` of 1000 ms (`config.py:338`). That's housekeeping, not lifecycle.
## 5. Cross-Project / Cross-Agent
-Projects are a first-class concept. Config holds a `Dict[str, ProjectEntry]` (`config.py:184-195`), each with a `path`, a `mode` (`LOCAL` or `CLOUD` — per-project routing), optional `workspace_id`, and bisync state. A `default_project` is auto-set to the first project (`config.py:705-711`).
+Projects are a first-class concept. Config holds a `Dict[str, ProjectEntry]` (`config.py:184-195`), each with a `path`, a `mode` (`LOCAL` or `CLOUD` - per-project routing), optional `workspace_id`, and bisync state. A `default_project` is auto-set to the first project (`config.py:705-711`).
**Project resolution is a unified three-tier chain** (`docs/ARCHITECTURE.md:298-324`): explicit `project` argument → default → single-project fallback. Every MCP tool accepts `project` and `project_id` (UUID) parameters. `get_project_client(project, ...)` (`mcp/project_context.py`) routes to local ASGI or cloud HTTP per-project, so you can mix.
-For agent handoffs, there is no special handshake — basic-memory treats all MCP clients identically. The "handoff" story is: agent A writes notes to project `foo`, agent B (any other MCP client pointed at the same project) reads them. The Markdown files on disk are the lingua franca. There is no session/agent identifier, no per-agent scratchpad. `created_by`/`last_updated_by` columns exist (`knowledge.py:99-102`) but are populated only in cloud (user_profile_id), null for local/CLI.
+For agent handoffs, there is no special handshake - basic-memory treats all MCP clients identically. The "handoff" story is: agent A writes notes to project `foo`, agent B (any other MCP client pointed at the same project) reads them. The Markdown files on disk are the lingua franca. There is no session/agent identifier, no per-agent scratchpad. `created_by`/`last_updated_by` columns exist (`knowledge.py:99-102`) but are populated only in cloud (user_profile_id), null for local/CLI.
## 6. Backup & Portability
- **Default location**: `~/basic-memory` for notes (per `BASIC_MEMORY_HOME` env, README:197), `~/.basic-memory/` for SQLite DB + config (`config.py:24, 67-80`). Config in `~/.basic-memory/config.json`, chmod 0600.
-- **Portability**: Excellent for files (they're just Markdown — `git clone`, `rsync`, Syncthing, rclone all work). The DB is a derived index — `bm sync` rebuilds it from the files.
+- **Portability**: Excellent for files (they're just Markdown - `git clone`, `rsync`, Syncthing, rclone all work). The DB is a derived index - `bm sync` rebuilds it from the files.
- **Schema migrations**: 22 Alembic migrations in `src/basic_memory/alembic/versions/`, auto-run on startup via `get_or_create_db` (`services/initialization.py:23-38`). Migrations cover both SQLite and Postgres.
- **Importers** for Claude conversations, ChatGPT exports, and `memory.json` (the original MCP "memory" server format) live in `src/basic_memory/importers/`.
-- **Cloud sync** uses `rclone bisync` (`config.py:136-138, 164-167`), not a custom protocol — another portable choice.
+- **Cloud sync** uses `rclone bisync` (`config.py:136-138, 164-167`), not a custom protocol - another portable choice.
## 7. Strengths Worth Borrowing
1. **Files are source of truth, DB is derived index.** Survives any DB corruption, plays nice with git, version control, and grep. Bidirectional human/AI editing via file-watcher + checksum.
2. **MCP behavior annotations on every tool** (`readOnlyHint`, `destructiveHint`, etc., `tools/write_note.py:23`). Agents can plan multi-step actions without trial-and-error.
3. **Aggressive `AliasChoices` aliasing** of parameter names (`write_note.py:31` accepts `directory|folder|dir|path`; `search.py:574` accepts `query|q|search|text`). LLMs use whatever name their training reaches for; the tool absorbs the variance.
-4. **`memory://` URI scheme + `build_context` graph walker** (`tools/build_context.py:114-247`) — turns wiki-links into navigable context. Cleaner than dumping the whole graph.
-5. **Unresolved relations as first-class state** (`knowledge.py:282`) — `to_id` nullable, resolves later when target appears. Forward references just work.
+4. **`memory://` URI scheme + `build_context` graph walker** (`tools/build_context.py:114-247`) - turns wiki-links into navigable context. Cleaner than dumping the whole graph.
+5. **Unresolved relations as first-class state** (`knowledge.py:282`) - `to_id` nullable, resolves later when target appears. Forward references just work.
6. **Per-project routing** with mixed local/cloud modes per project, not per-server.
7. **Composition root + typed-client pattern** (`docs/ARCHITECTURE.md:14-256`) keeps MCP/CLI/API entrypoints clean and testable.
-8. **The `NoteContent` table** with `file_write_status` state machine (`knowledge.py:155-217`) handles the race between agent writes and on-disk human edits — worth replicating if you keep a DB cache.
+8. **The `NoteContent` table** with `file_write_status` state machine (`knowledge.py:155-217`) handles the race between agent writes and on-disk human edits - worth replicating if you keep a DB cache.
## 8. Weaknesses / Friction (Avoid)
-1. **The manual `write_note` ceremony is the headline friction.** Users must explicitly tell the model "make a note about this," and the model must decide *title*, *directory*, *tags*, *note_type*, and the semantic observation/relation grammar — all on every call. The `write_note` signature has 11 parameters (`write_note.py:25-45`). Skip this entirely: capture should be ambient (post-turn summarization, automatic salience scoring, etc.) — not a tool the model must remember to call.
+1. **The manual `write_note` ceremony is the headline friction.** Users must explicitly tell the model "make a note about this," and the model must decide *title*, *directory*, *tags*, *note_type*, and the semantic observation/relation grammar - all on every call. The `write_note` signature has 11 parameters (`write_note.py:25-45`). Skip this entirely: capture should be ambient (post-turn summarization, automatic salience scoring, etc.) - not a tool the model must remember to call.
2. **Append-only forever.** No decay, no consolidation, no automatic deduplication. Long-running graphs accumulate cruft. Recent v0.20 added a guard so `write_note` *errors* on conflict instead of silently upserting (`write_note.py:240-262`), which protects data but pushes the burden back onto the agent to manage identity. For agent long-term memory, decay/merge/consolidation is essential and absent here.
3. **The semantic grammar is human-authored convention.** `- [category] text #tag (context)` and `- relation_type [[Target]]` are intuitive for humans in Obsidian but require the LLM to *generate* this format correctly every time. Drift is normal. A Rust server can store edges natively and let the LLM emit prose.
4. **Note identity is brittle.** `permalink` is derived from title/path; renames create work (`update_permalinks_on_move` defaults to `False`, `config.py:349-352`). The 11-column `entity` table + separate `note_content` is a lot of machinery to keep in sync with files.
5. **No agent/session model.** No notion of "who wrote this," "which session," or "what was the user's intent." `created_by` exists only for cloud auth (`knowledge.py:99-102`). For multi-agent handoffs, you'd want provenance baked in.
6. **Search ranking is keyword-or-vector with a fixed `min_similarity=0.55`** (`config.py:307-312`). No recency/importance reranking, no usage feedback, no per-query learning.
-7. **The Markdown source-of-truth tradeoff.** Filesystem latency, checksum recomputation, FTS5 rebuilds, and circuit-breaker retry tracking (`sync_service.py:179-281`) are a lot of infrastructure just to keep a DB consistent with files. For agent memory where the files exist only because LLMs wrote them, this is overhead with no payoff — keep the DB as primary and only export Markdown on demand.
+7. **The Markdown source-of-truth tradeoff.** Filesystem latency, checksum recomputation, FTS5 rebuilds, and circuit-breaker retry tracking (`sync_service.py:179-281`) are a lot of infrastructure just to keep a DB consistent with files. For agent memory where the files exist only because LLMs wrote them, this is overhead with no payoff - keep the DB as primary and only export Markdown on demand.
8. **Tool surface is wide (~25 tools).** Agents have to pick among `write_note`/`edit_note`/`move_note`/`delete_note`/`read_note`/`view_note`/`read_content`/`search`/`search_notes`/`build_context`/`recent_activity`/`list_directory`/`list_memory_projects`/`canvas`/`schema_*`. Even with hints, that's a lot of context burned describing tools. Aim for a smaller, more orthogonal set.
-**Bottom line**: borrow the *graph + observations + wiki-link* primitives, the *memory:// URI* navigation, the *typed-client + composition-root* layering, the *MCP annotations*, and the *file-as-portable-export* idea — but invert the capture model (ambient, not invoked), add a lifecycle (decay/consolidation/summarization), keep the DB primary, add agent/session provenance, and ship a much narrower MCP tool surface.
+**Bottom line**: borrow the *graph + observations + wiki-link* primitives, the *memory:// URI* navigation, the *typed-client + composition-root* layering, the *MCP annotations*, and the *file-as-portable-export* idea - but invert the capture model (ambient, not invoked), add a lifecycle (decay/consolidation/summarization), keep the DB primary, add agent/session provenance, and ship a much narrower MCP tool surface.
diff --git a/docs/research-cognee.md b/docs/research-cognee.md
index b367a66f..9275a602 100644
--- a/docs/research-cognee.md
+++ b/docs/research-cognee.md
@@ -1,10 +1,10 @@
-# cognee — Research Report
+# cognee - Research Report
> Source project: `~/Projects/cognee` (Python, knowledge-graph + vector + relational, MCP server).
## 1. Purpose & Scope
-Cognee bills itself as **"the brain behind your agents" — a memory control plane** (`README.md:40,67`). It ingests heterogeneous data (text, PDF, CSV, code, web pages), then continuously builds a hybrid **knowledge graph + vector index + relational catalog** so that agents can retrieve context by both meaning (embeddings) and structure (graph relationships). The current public SDK is intentionally minimal — four verbs: `remember`, `recall`, `forget`, `improve` (README L127). Internally, those map to the older `add`/`cognify`/`search`/`prune` primitives. Cognee positions itself between a RAG retriever and a "company brain," with multi-tenant isolation, ontology grounding, and a Claude Code plugin for capturing agent traces.
+Cognee bills itself as **"the brain behind your agents" - a memory control plane** (`README.md:40,67`). It ingests heterogeneous data (text, PDF, CSV, code, web pages), then continuously builds a hybrid **knowledge graph + vector index + relational catalog** so that agents can retrieve context by both meaning (embeddings) and structure (graph relationships). The current public SDK is intentionally minimal - four verbs: `remember`, `recall`, `forget`, `improve` (README L127). Internally, those map to the older `add`/`cognify`/`search`/`prune` primitives. Cognee positions itself between a RAG retriever and a "company brain," with multi-tenant isolation, ontology grounding, and a Claude Code plugin for capturing agent traces.
## 2. The Cognify Pipeline (heart of the system)
@@ -24,22 +24,22 @@ Step-by-step:
1. **Classify documents** (`cognee/tasks/documents/classify_documents.py`). Wraps each raw `Data` row in a typed `Document` subclass (`PdfDocument`, `TextDocument`, `DltRowDocument`, etc.) so downstream chunkers know how to read content.
-2. **Chunk documents** (`cognee/tasks/documents/extract_chunks_from_documents.py` + `cognee/modules/chunking/TextChunker.py:1-60`). The default `TextChunker` uses `chunk_by_paragraph` to pack paragraphs up to `max_chunk_size` tokens (calculated as `min(embedding_max_completion_tokens, llm_max_completion_tokens // 2)`). Each chunk becomes a `DocumentChunk` DataPoint with `metadata={"index_fields": ["text"]}` — those `index_fields` are how the storage layer later knows what to embed.
+2. **Chunk documents** (`cognee/tasks/documents/extract_chunks_from_documents.py` + `cognee/modules/chunking/TextChunker.py:1-60`). The default `TextChunker` uses `chunk_by_paragraph` to pack paragraphs up to `max_chunk_size` tokens (calculated as `min(embedding_max_completion_tokens, llm_max_completion_tokens // 2)`). Each chunk becomes a `DocumentChunk` DataPoint with `metadata={"index_fields": ["text"]}` - those `index_fields` are how the storage layer later knows what to embed.
3. **Extract graph + summarize** (`cognee/tasks/graph/extract_graph_and_summarize.py:21-37`). Two LLM tasks concurrently on every chunk via `asyncio.gather`:
- - `extract_graph_from_data` (`cognee/tasks/graph/extract_graph_from_data.py:128-222`) calls `extract_content_graph(chunk.text, graph_model, custom_prompt)` for each chunk — an `instructor`/`litellm`-backed structured call that returns a Pydantic `KnowledgeGraph` of `Node`/`Edge`. Edges with missing source/target are filtered (L181-188). Entity nodes are then validated against an ontology via `expand_with_nodes_and_edges(..., ontology_resolver, ...)` (L110-112). Provenance is stamped onto every DataPoint (`_stamp_provenance_deep`, L30-53). Existing edges are deduplicated via `retrieve_existing_edges`.
- - `summarize_text` produces a `TextSummary` for each chunk used by later "SUMMARIES" search.
+ - `extract_graph_from_data` (`cognee/tasks/graph/extract_graph_from_data.py:128-222`) calls `extract_content_graph(chunk.text, graph_model, custom_prompt)` for each chunk - an `instructor`/`litellm`-backed structured call that returns a Pydantic `KnowledgeGraph` of `Node`/`Edge`. Edges with missing source/target are filtered (L181-188). Entity nodes are then validated against an ontology via `expand_with_nodes_and_edges(..., ontology_resolver, ...)` (L110-112). Provenance is stamped onto every DataPoint (`_stamp_provenance_deep`, L30-53). Existing edges are deduplicated via `retrieve_existing_edges`.
+ - `summarize_text` produces a `TextSummary` for each chunk used by later "SUMMARIES" search.
4. **Persist nodes, edges & embeddings** (`cognee/tasks/storage/add_data_points.py:30-149`). `get_graph_from_model` recursively walks the Pydantic graph into `(nodes, edges)` tuples, then `deduplicate_nodes_and_edges` removes duplicates. The pipeline branches on `EngineCapability.HYBRID_WRITE`:
- - Hybrid backend (e.g. Postgres+pgvector): `graph_engine.add_nodes_with_vectors(nodes)` in one transaction.
- - Otherwise, parallel writes: `graph_engine.add_nodes(nodes)` + `index_data_points(...)` to the vector engine.
- - If `embed_triplets=True`, builds `(source -› relation -› target)` text strings and embeds them as additional `Triplet` DataPoints (L184-265). This is what makes graph-walk retrieval finable by similarity.
+ - Hybrid backend (e.g. Postgres+pgvector): `graph_engine.add_nodes_with_vectors(nodes)` in one transaction.
+ - Otherwise, parallel writes: `graph_engine.add_nodes(nodes)` + `index_data_points(...)` to the vector engine.
+ - If `embed_triplets=True`, builds `(source -› relation -› target)` text strings and embeds them as additional `Triplet` DataPoints (L184-265). This is what makes graph-walk retrieval finable by similarity.
-5. **DLT FK edges** — adds deterministic foreign-key edges for ingested SQL/DLT-backed datasets without LLM cost.
+5. **DLT FK edges** - adds deterministic foreign-key edges for ingested SQL/DLT-backed datasets without LLM cost.
There's also a parallel **temporal** pipeline (`cognify.py:347-392`) that swaps the graph extractor for `extract_events_and_timestamps` → `extract_knowledge_graph_from_events` to build a time-aware graph.
-## 3. Storage Backends — Triple-Stack, Pluggable
+## 3. Storage Backends - Triple-Stack, Pluggable
Cognee always uses three stores in parallel, abstracted by a `UnifiedStoreEngine` (`cognee/infrastructure/databases/unified/unified_store_engine.py:11-66`):
@@ -53,13 +53,13 @@ Selection is **environment-driven**: `GRAPH_DATABASE_PROVIDER`, `VECTOR_DB_PROVI
Two `EngineCapability` flags (`HYBRID_WRITE`, `HYBRID_SEARCH`) let the pipeline branch on whether nodes+vectors can land in one transaction (saves a round trip) or need two writes.
-The default local-only stack is therefore: **SQLite + LanceDB + Ladybug/Kuzu**, all file-based — no servers required. For homelab parity, that's the lean setup.
+The default local-only stack is therefore: **SQLite + LanceDB + Ladybug/Kuzu**, all file-based - no servers required. For homelab parity, that's the lean setup.
## 4. Search / Retrieval
Recall is **multi-strategy and auto-routed**. `SearchType` enum (`cognee/modules/search/types/SearchType.py`) lists 16 strategies: `GRAPH_COMPLETION` (default), `GRAPH_COMPLETION_COT`, `GRAPH_COMPLETION_CONTEXT_EXTENSION`, `GRAPH_COMPLETION_DECOMPOSITION`, `GRAPH_SUMMARY_COMPLETION`, `RAG_COMPLETION`, `TRIPLET_COMPLETION`, `CHUNKS`, `CHUNKS_LEXICAL`, `SUMMARIES`, `CYPHER`, `NATURAL_LANGUAGE`, `TEMPORAL`, `FEELING_LUCKY`, `CODING_RULES`, `AGENTIC_COMPLETION`.
-The `recall()` API (`cognee/api/v1/recall/recall.py:314-513`) picks one via `route_query(query_text)` — a **rule-based classifier** in `query_router.py` (regex patterns for "when/before/after" → `TEMPORAL`, keyword fragments → `CHUNKS_LEXICAL`, multi-hop wording → `GRAPH_COMPLETION_COT`, etc.). Default fallback is `GRAPH_COMPLETION`.
+The `recall()` API (`cognee/api/v1/recall/recall.py:314-513`) picks one via `route_query(query_text)` - a **rule-based classifier** in `query_router.py` (regex patterns for "when/before/after" → `TEMPORAL`, keyword fragments → `CHUNKS_LEXICAL`, multi-hop wording → `GRAPH_COMPLETION_COT`, etc.). Default fallback is `GRAPH_COMPLETION`.
The flagship retriever is `GraphCompletionRetriever` (`cognee/modules/retrieval/graph_completion_retriever.py`). It uses **`brute_force_triplet_search`** (`cognee/modules/retrieval/utils/brute_force_triplet_search.py:216-355`), which:
@@ -71,7 +71,7 @@ The flagship retriever is `GraphCompletionRetriever` (`cognee/modules/retrieval/
Recall also supports **session-cache first-pass** (keyword match against recent QA entries in a relational session table) with **fall-through to graph** when no session hit (`recall.py:382-397, 447-457`). That's the "hybrid working-memory + long-term memory" pattern.
-## 5. Memory Lifecycle — Cognee Does Reorganize
+## 5. Memory Lifecycle - Cognee Does Reorganize
Cognee has explicit **memory enrichment** beyond one-shot ingest. The `improve()` API (`cognee/api/v1/improve/improve.py:36-232`) runs up to five stages:
@@ -89,19 +89,19 @@ So: **not one-shot.** There's a clear feedback loop, summary consolidation, and
Yes, in `cognee-mcp/` as a sibling project with its own pyproject. The server (`cognee-mcp/src/server.py`) uses the official `mcp` SDK over stdio/SSE/HTTP. The **publicly exposed tools are deliberately minimal** (L1076-1222):
-- `remember(data, dataset_name?, session_id?, custom_prompt?)` — with `session_id`, fast session-cache write; without, runs full `add + cognify` pipeline.
-- `recall(query, search_type?, datasets?, session_id?, top_k=10)` — auto-routes search type.
-- `forget(dataset?, everything=False)` — deletes across all three stores.
+- `remember(data, dataset_name?, session_id?, custom_prompt?)` - with `session_id`, fast session-cache write; without, runs full `add + cognify` pipeline.
+- `recall(query, search_type?, datasets?, session_id?, top_k=10)` - auto-routes search type.
+- `forget(dataset?, everything=False)` - deletes across all three stores.
There are *internal* tools registered but not exposed in API mode: `cognify`, `search`, `list_data`, `delete`, `prune`, `improve`, plus a UI bundle (`visualize_graph_ui`, `upload_file_ui`, `open_cognee_workspace`, `cognify_file`) that opens an embedded HTML workspace from `src/app_bundles/visualize-graph.html`.
**Notable design choice**: per-MCP-client auto-named datasets (`cursor_vscode_memory`, `claude_code_memory`) so different agents don't pollute each other's memory unless they opt in by passing `dataset_name="main_dataset"`. Toggle with `COGNEE_MCP_AGENT_SCOPED=false`.
-MCP integration is **manual** in the sense that the agent invokes `remember`/`recall` explicitly. The Claude Code plugin (separate repo `cognee-integrations`) automates it via Claude Code lifecycle hooks (`SessionStart`, `PostToolUse`, `UserPromptSubmit`, `PreCompact`, `SessionEnd`) — see README L204.
+MCP integration is **manual** in the sense that the agent invokes `remember`/`recall` explicitly. The Claude Code plugin (separate repo `cognee-integrations`) automates it via Claude Code lifecycle hooks (`SessionStart`, `PostToolUse`, `UserPromptSubmit`, `PreCompact`, `SessionEnd`) - see README L204.
## 7. Operational Concerns
-**Deployment**: A top-level `Dockerfile` builds the FastAPI server; `cognee-mcp/Dockerfile` builds the MCP server. `docker-compose.yml` ships the API on port 8000 with **resource limits of 4 CPUs and 8 GB RAM** — non-trivial. Deployment targets include Cognee Cloud, Modal, Railway, Fly.io, Render, Daytona (`distributed/deploy/`).
+**Deployment**: A top-level `Dockerfile` builds the FastAPI server; `cognee-mcp/Dockerfile` builds the MCP server. `docker-compose.yml` ships the API on port 8000 with **resource limits of 4 CPUs and 8 GB RAM** - non-trivial. Deployment targets include Cognee Cloud, Modal, Railway, Fly.io, Render, Daytona (`distributed/deploy/`).
**LLM cost per ingest**: heavy. For *every* chunk, cognify runs:
- 1 structured-extraction call (entity/relationship → Pydantic KnowledgeGraph)
@@ -116,29 +116,29 @@ A 100-chunk document easily generates 200+ LLM calls plus hundreds of embeddings
## 8. Strongest Ideas to Adopt
-1. **Task-list pipeline composition.** `cognify.py:316-344` reads like a data-flow recipe. Each `Task` is a thin wrapper with a `batch_size` config and a context object. This is the cleanest pattern to port to Rust — represent the pipeline as a typed `Vec<Box<dyn Task>>` and let tasks declare their batching behavior.
+1. **Task-list pipeline composition.** `cognify.py:316-344` reads like a data-flow recipe. Each `Task` is a thin wrapper with a `batch_size` config and a context object. This is the cleanest pattern to port to Rust - represent the pipeline as a typed `Vec<Box<dyn Task>>` and let tasks declare their batching behavior.
-2. **Triplet embeddings as first-class citizens.** Embedding `(source -› relation -› target)` text directly (`add_data_points.py:184-265`) is the trick that lets a graph become searchable by semantic similarity, not just by walking. This is genuinely the bridge between "vector RAG" and "graph RAG."
+2. **Triplet embeddings as first-class citizens.** Embedding `(source -› relation -› target)` text directly (`add_data_points.py:184-265`) is the trick that lets a graph become searchable by semantic similarity, by walking. This is genuinely the bridge between "vector RAG" and "graph RAG."
3. **Capability-flag-driven storage facade.** `UnifiedStoreEngine.has_capability(HYBRID_WRITE)` lets the same pipeline target separate `(Kuzu, LanceDB)` or fused `(Postgres+pgvector)` without conditionals scattered everywhere.
-4. **`improve()` lifecycle with feedback weights.** The `feedback_weight` on graph elements + `used_graph_element_ids` from the retrieval trace is a sharp idea — it gives you graded knowledge without retraining anything.
+4. **`improve()` lifecycle with feedback weights.** The `feedback_weight` on graph elements + `used_graph_element_ids` from the retrieval trace is a sharp idea - it gives you graded knowledge without retraining anything.
5. **Multi-collection vector search at recall time.** Querying `Entity_name`, `TextSummary_text`, `DocumentChunk_text`, and edge collections in parallel and merging by triplet score, rather than picking one index, is the core retrieval move.
-6. **Provenance stamping** (`_stamp_provenance_deep`) — every DataPoint carries `source_pipeline` + `source_task`, making the graph debuggable.
+6. **Provenance stamping** (`_stamp_provenance_deep`) - every DataPoint carries `source_pipeline` + `source_task`, making the graph debuggable.
## 9. Realistic Downsides for a Lean Homelab
-- **Dependency weight**: 40+ core dependencies — `openai`, `litellm`, `instructor`, `sqlalchemy`, `aiosqlite`, `lancedb`, `pylance`, `ladybug`, `networkx`, `pypdf`, `fastapi`, `fastapi-users`, `rdflib`, `langdetect`, `datamodel-code-generator`, `tiktoken`, `tenacity`, `aiolimiter`, `diskcache`, `fakeredis`, plus optional extras (neo4j, chromadb, postgres, anthropic, ollama, huggingface, scraping, dlt, monitoring). Cold install is huge — `poetry.lock` is **1.4 MB**.
-- **Python async everywhere + LRU-cached engine handles** (`closing_lru_cache`) — debugging stale-adapter issues required adding `_GraphEngineHandle` (see the 50-line docstring at `get_graph_engine.py:56-105`). That's complexity you inherit.
-- **Per-cognify LLM cost** is high. There's no obvious caching of "I've already extracted entities from this chunk hash" — re-ingestion repeats work. `incremental_loading=True` helps but only at the dataset level.
+- **Dependency weight**: 40+ core dependencies - `openai`, `litellm`, `instructor`, `sqlalchemy`, `aiosqlite`, `lancedb`, `pylance`, `ladybug`, `networkx`, `pypdf`, `fastapi`, `fastapi-users`, `rdflib`, `langdetect`, `datamodel-code-generator`, `tiktoken`, `tenacity`, `aiolimiter`, `diskcache`, `fakeredis`, plus optional extras (neo4j, chromadb, postgres, anthropic, ollama, huggingface, scraping, dlt, monitoring). Cold install is huge - `poetry.lock` is **1.4 MB**.
+- **Python async everywhere + LRU-cached engine handles** (`closing_lru_cache`) - debugging stale-adapter issues required adding `_GraphEngineHandle` (see the 50-line docstring at `get_graph_engine.py:56-105`). That's complexity you inherit.
+- **Per-cognify LLM cost** is high. There's no obvious caching of "I've already extracted entities from this chunk hash" - re-ingestion repeats work. `incremental_loading=True` helps but only at the dataset level.
- **Three databases to back up**, three to migrate. The "wipe `.cognee_system/` when you flip `ENABLE_BACKEND_ACCESS_CONTROL`" caveat (cognee-mcp/README L518-525) shows the multi-store coordination is fragile.
- **8 GB RAM limit** in docker-compose is the recommended floor. LanceDB + Kuzu + embedding model loaded simultaneously is heavy on a low-end NAS.
-- **Bus factor on `ladybug`** — a 0.16.0 pinned Kuzu fork by the same org. If they stop maintaining it, you're on a single-vendor graph DB.
+- **Bus factor on `ladybug`** - a 0.16.0 pinned Kuzu fork by the same org. If they stop maintaining it, you're on a single-vendor graph DB.
- **Two-tier MCP API**: public 3 tools + 10+ "internal" tools means the API surface is in motion; not stable.
-For a Rust homelab MCP server, the **lean equivalents** would be: SQLite (relational) + a single embedded vector store (e.g. `lancedb-rs` or embedded `sqlite-vec`) + an embedded graph DB (CozoDB, SurrealDB, or just SQL tables with proper indices) — then port the *task pipeline + triplet embedding + capability flag pattern* and skip the FastAPI/users/migrations/UI bundle stack entirely.
+For a Rust homelab MCP server, the **lean equivalents** would be: SQLite (relational) + a single embedded vector store (e.g. `lancedb-rs` or embedded `sqlite-vec`) + an embedded graph DB (CozoDB, SurrealDB, or just SQL tables with proper indices) - then port the *task pipeline + triplet embedding + capability flag pattern* and skip the FastAPI/users/migrations/UI bundle stack entirely.
### Key file references
diff --git a/docs/research-karpathy-llm-wiki.md b/docs/research-karpathy-llm-wiki.md
index dd77a62a..b64337b4 100644
--- a/docs/research-karpathy-llm-wiki.md
+++ b/docs/research-karpathy-llm-wiki.md
@@ -1,11 +1,11 @@
-# Karpathy's "LLM Wiki" — Research Report
+# Karpathy's "LLM Wiki" - Research Report
> The pattern this project is trying to implement faithfully. Primary source
> below; related/competing ideas listed for honest contrast.
## 1. What Karpathy Actually Said
-The canonical primary source is Karpathy's April 2026 gist [`llm-wiki.md`](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f), which he calls an **"idea file"** — explicitly *not* a library or app, but a pattern designed to be copy-pasted into an agent (Claude Code, Codex, OpenCode) so the agent can instantiate it for the user's domain.
+The canonical primary source is Karpathy's April 2026 gist [`llm-wiki.md`](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f), which he calls an **"idea file"** - explicitly *not* a library or app, but a pattern designed to be copy-pasted into an agent (Claude Code, Codex, OpenCode) so the agent can instantiate it for the user's domain.
The original framing came from an X thread on April 2, 2026, paraphrased as: *"using LLMs to build personal knowledge bases for various topics of research interest"*. He followed up two days later with the gist. He then [boosted "Farzapedia"](https://x.com/karpathy/status/2040572272944324650) as a good example of the pattern in the wild.
@@ -19,7 +19,7 @@ The original framing came from an X thread on April 2, 2026, paraphrased as: *"u
> "The tedious part of maintaining a knowledge base is not the reading or the thinking — it's the bookkeeping... LLMs don't get bored, don't forget to update a cross-reference, and can touch 15 files in one pass."
-He explicitly links the idea to Vannevar Bush's 1945 Memex — "a personal, curated knowledge store with associative trails between documents" — arguing the part Bush couldn't solve was *who does the maintenance*; LLMs solve that.
+He explicitly links the idea to Vannevar Bush's 1945 Memex - "a personal, curated knowledge store with associative trails between documents" - arguing the part Bush couldn't solve was *who does the maintenance*; LLMs solve that.
## 2. The Core Principles
@@ -27,13 +27,13 @@ From the gist itself (Karpathy's, not paraphrased):
1. **Compilation, not retrieval.** Knowledge is compiled at ingest time, not re-synthesized at query time. The wiki is the artifact; raw sources are the source of truth.
2. **Three-layer architecture.**
- - **Raw sources** — immutable; LLM reads only.
- - **Wiki** — markdown files; LLM owns and maintains entirely.
- - **Schema** (CLAUDE.md / AGENTS.md) — conventions that turn "a generic chatbot into a disciplined wiki maintainer."
+ - **Raw sources** - immutable; LLM reads only.
+ - **Wiki** - markdown files; LLM owns and maintains entirely.
+ - **Schema** (CLAUDE.md / AGENTS.md) - conventions that turn "a generic chatbot into a disciplined wiki maintainer."
3. **Three operations: Ingest / Query / Lint.**
- - *Ingest*: one source typically touches **10–15 wiki pages**.
- - *Query*: "good answers can be filed back into the wiki as new pages... explorations compound in the knowledge base just like ingested sources do."
- - *Lint*: periodic health check for contradictions, stale claims, orphan pages, missing cross-references, data gaps.
+ - *Ingest*: one source typically touches **10–15 wiki pages**.
+ - *Query*: "good answers can be filed back into the wiki as new pages... explorations compound in the knowledge base just like ingested sources do."
+ - *Lint*: periodic health check for contradictions, stale claims, orphan pages, missing cross-references, data gaps.
4. **Cross-linking is the synthesis.** The wiki is interlinked like Wikipedia or a fan wiki (he cites [Tolkien Gateway](https://tolkiengateway.net/wiki/Main_Page)); the graph *is* the consolidated knowledge.
5. **Two navigation files: `index.md` (content catalog) and `log.md` (chronological append-only ledger).** The log uses a fixed prefix so unix tools (`grep "^## \["`) can parse it.
6. **Division of labor.** Human curates sources and asks good questions; the LLM does "the summarizing, cross-referencing, filing, and bookkeeping." Or in his metaphor: *"Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase."*
@@ -42,16 +42,16 @@ From the gist itself (Karpathy's, not paraphrased):
The community frequently attributes a few additional ideas to him that are **paraphrase / extension**, not in his gist:
-- **Episodic vs. semantic memory tiers** — neuroscience framing. Not in the gist. This comes from extensions like [LLM Wiki v2](https://gist.github.com/rohitg00/2067ab416f7bbe447c1977edaaa681e2) and the broader memory-research literature.
-- **"Sleep-like" consolidation passes** — also extension framing, not Karpathy's. His closest analog is the *Lint* operation (periodic health-check), which is rule-based rather than dream-like.
-- **Confidence scoring, Ebbinghaus decay, supersession semantics** — these are LLM Wiki v2's additions, not the original.
-- **The numbered "1. Explicit. 2. ..." Farzapedia points** — community summaries say Karpathy listed several advantages of *explicit memory artifacts* over the "AI that allegedly gets better the more you use it" status quo. Flagged as **community paraphrase**.
+- **Episodic vs. semantic memory tiers** - neuroscience framing. Not in the gist. This comes from extensions like [LLM Wiki v2](https://gist.github.com/rohitg00/2067ab416f7bbe447c1977edaaa681e2) and the broader memory-research literature.
+- **"Sleep-like" consolidation passes** - also extension framing, not Karpathy's. His closest analog is the *Lint* operation (periodic health-check), which is rule-based rather than dream-like.
+- **Confidence scoring, Ebbinghaus decay, supersession semantics** - these are LLM Wiki v2's additions, not the original.
+- **The numbered "1. Explicit. 2. ..." Farzapedia points** - community summaries say Karpathy listed several advantages of *explicit memory artifacts* over the "AI that allegedly gets better the more you use it" status quo. Flagged as **community paraphrase**.
## 3. Implementation Hints from the Gist
- **Trigger**: human-in-the-loop on ingest ("I prefer to ingest sources one at a time and stay involved"), though batch ingestion is allowed.
- **Format**: plain markdown in a git repo. Optional YAML frontmatter for Dataview queries.
-- **Retrieval at small scale**: `index.md` is "surprisingly good... at ~100 sources, ~hundreds of pages" — no embeddings needed.
+- **Retrieval at small scale**: `index.md` is "surprisingly good... at ~100 sources, ~hundreds of pages" - no embeddings needed.
- **Retrieval at larger scale**: shell out to a local hybrid search tool (he names [`qmd`](https://github.com/tobi/qmd), BM25 + vector + LLM re-rank, available as CLI and MCP).
- **Tooling**: Obsidian as the viewer (graph view to spot orphan/hub pages), Web Clipper to capture sources, version-controlled in git.
@@ -59,23 +59,23 @@ The community frequently attributes a few additional ideas to him that are **par
- **MemGPT / Letta** ([letta.com](https://www.letta.com/blog/benchmarking-ai-agent-memory)): treats the context window as virtual memory; the *agent itself* decides what to page in/out across core, recall, and archival tiers. Stronger on long-horizon episodic coherence; higher lock-in (owns the agent loop).
- **Mem0** ([tokenmix.ai comparison](https://tokenmix.ai/blog/ai-agent-memory-mem0-vs-letta-vs-memgpt-2026)): lightweight memory layer with `extract / store / retrieve`. Extracts memories *passively* from conversations rather than letting the agent self-edit. Low lock-in.
-- **A-MEM** ([arXiv 2502.12110](https://arxiv.org/abs/2502.12110), NeurIPS 2025): explicitly Zettelkasten-inspired. Each memory is an atomic note with structured attributes, keywords, tags; new memories trigger *evolution* of existing notes' representations. This is the closest published research analog to Karpathy's wiki — atomic notes + automatic linking + revision propagation.
-- **ReadAgent** (Google DeepMind, 2024): "gist memory" — compresses long contexts into a tree of summaries with pointers back to detail. Different angle (long-document reading), but shares the "compile, don't re-retrieve" instinct.
+- **A-MEM** ([arXiv 2502.12110](https://arxiv.org/abs/2502.12110), NeurIPS 2025): explicitly Zettelkasten-inspired. Each memory is an atomic note with structured attributes, keywords, tags; new memories trigger *evolution* of existing notes' representations. This is the closest published research analog to Karpathy's wiki - atomic notes + automatic linking + revision propagation.
+- **ReadAgent** (Google DeepMind, 2024): "gist memory" - compresses long contexts into a tree of summaries with pointers back to detail. Different angle (long-document reading), but shares the "compile, don't re-retrieve" instinct.
- **LLM Wiki v2** (Rohit Ghumare): explicit extension with confidence scores, supersession, Ebbinghaus decay, four consolidation tiers (working → episodic → semantic → procedural), event-driven hooks, audit trails. This is basically the agentmemory model.
- **Rowboat / knowledge-graph extension** ([dailydoseofds.com](https://blog.dailydoseofds.com/p/the-next-step-after-karpathys-wiki)): argues the wiki of summaries breaks down for evolving work contexts (deadlines, commitments) and proposes a *typed-entity knowledge graph* (decisions, people, projects as nodes).
## 5. Design Implications for a Rust MCP Server for Coding Agents
-Translating Karpathy faithfully — a "Karpathy-style" backend looks very different from naive vector RAG:
+Translating Karpathy faithfully - a "Karpathy-style" backend looks very different from naive vector RAG:
**What it is, concretely:**
- **Storage = markdown files in a git repo**, not opaque vector blobs. The wiki must be human-inspectable and grep-able. Embeddings can index it but never replace it.
- **Three directories enforced by the MCP server**: `raw/` (append-only, immutable), `wiki/` (LLM-writable, structured), and a schema doc (`AGENTS.md`-style) the server injects into every session.
-- **MCP tools mirror the three operations**: `memory_ingest`, `memory_query`, `memory_lint` — plus low-level primitives (`wiki_read`, `wiki_write`, `wiki_link`, `wiki_supersede`). Not `vector_search` as the headline tool.
-- **Ingest must be a *write fan-out*, not just an insert.** A new observation should *touch ~10–15 existing pages* — updating an entity page, a concept page, a decisions log, a gotchas page. This is the single biggest deviation from vector RAG, which only ever appends.
+- **MCP tools mirror the three operations**: `memory_ingest`, `memory_query`, `memory_lint` - plus low-level primitives (`wiki_read`, `wiki_write`, `wiki_link`, `wiki_supersede`). Not `vector_search` as the headline tool.
+- **Ingest must be a *write fan-out*, an insert.** A new observation should *touch ~10–15 existing pages* - updating an entity page, a concept page, a decisions log, a gotchas page. This is the single biggest deviation from vector RAG, which only ever appends.
- **`index.md` and `log.md` as first-class files.** The log is the audit trail and the consolidation trigger source. Use the prefix convention (`## [YYYY-MM-DD] action | title`) so it's grep-able.
-- **Retrieval is hierarchical, not just nearest-neighbor.** Read `index.md` → narrow to candidate pages → read them → optionally fall back to hybrid search (BM25 + vector, RRF-fused) for novel queries. The index *is* the synthesis; the embeddings are a backstop.
+- **Retrieval is hierarchical, nearest-neighbor.** Read `index.md` → narrow to candidate pages → read them → optionally fall back to hybrid search (BM25 + vector, RRF-fused) for novel queries. The index *is* the synthesis; the embeddings are a backstop.
- **Consolidation is an explicit, scheduled MCP operation**, not a side-effect. `memory_consolidate` is invoked on session-end (Claude Code's stop hook, Codex's session-end), on compaction events, or on a timer. It is LLM-driven (needs a provider key); if absent, it runs no-op as agentmemory does.
- **Cross-agent shared state.** Because the wiki is plain text, Claude Code, Codex, and OpenCode all read/write the *same* artifact. The MCP server is the gatekeeper; the markdown is the contract. No vendor lock.
- **Coding-specific page types**: library gotchas, architectural decisions (ADR-style), failed approaches, repo conventions, environment quirks. Karpathy's example domains were personal/research; for coding agents the high-value pages are *failure modes* and *decisions*, because those are exactly what gets dropped on context compaction.
@@ -86,22 +86,22 @@ Translating Karpathy faithfully — a "Karpathy-style" backend looks very differ
- Not a chronological transcript. The log exists, but it's metadata. The semantic content lives in synthesized pages.
- Not opaque. Every memory the agent has must be openable in Obsidian, diff-able in git, and explainable in prose.
-**Honest tension** worth resolving in design: Karpathy's gist is optimized for *human-curated research wikis* ingested one source at a time with the user watching. A coding agent ingests *continuously and unsupervised* from tool calls. This project inherits Karpathy's structure but needs the lifecycle layer (decay, supersession, confidence) that LLM Wiki v2 proposes — otherwise the wiki will fill with stale, low-signal observations from autonomous runs.
+**Honest tension** worth resolving in design: Karpathy's gist is optimized for *human-curated research wikis* ingested one source at a time with the user watching. A coding agent ingests *continuously and unsupervised* from tool calls. This project inherits Karpathy's structure but needs the lifecycle layer (decay, supersession, confidence) that LLM Wiki v2 proposes - otherwise the wiki will fill with stale, low-signal observations from autonomous runs.
## Sources
-- [Karpathy — `llm-wiki.md` gist (primary source)](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)
-- [Karpathy — Farzapedia tweet](https://x.com/karpathy/status/2040572272944324650)
+- [Karpathy - `llm-wiki.md` gist (primary source)](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)
+- [Karpathy - Farzapedia tweet](https://x.com/karpathy/status/2040572272944324650)
- [Yuchen Jin's summary tweet quoting Karpathy](https://x.com/Yuchenj_UW/status/2040482771576197377)
-- [AkitaOnRails — AI Agent Memory: Karpathy LLM Wiki and agentmemory in Practice](https://akitaonrails.com/en/2026/05/18/ai-agent-memory-karpathy-llm-wiki-agentmemory/)
-- [Rohit Ghumare — LLM Wiki v2 (gist)](https://gist.github.com/rohitg00/2067ab416f7bbe447c1977edaaa681e2)
+- [AkitaOnRails - AI Agent Memory: Karpathy LLM Wiki and agentmemory in Practice](https://akitaonrails.com/en/2026/05/18/ai-agent-memory-karpathy-llm-wiki-agentmemory/)
+- [Rohit Ghumare - LLM Wiki v2 (gist)](https://gist.github.com/rohitg00/2067ab416f7bbe447c1977edaaa681e2)
- [A-MEM: Agentic Memory for LLM Agents (NeurIPS 2025)](https://arxiv.org/abs/2502.12110)
- [Mem0 vs Letta vs MemGPT comparison (TokenMix, 2026)](https://tokenmix.ai/blog/ai-agent-memory-mem0-vs-letta-vs-memgpt-2026)
- [Benchmarking AI Agent Memory: Is a Filesystem All You Need? (Letta)](https://www.letta.com/blog/benchmarking-ai-agent-memory)
-- [The Next Step After Karpathy's Wiki Idea — Avi Chawla](https://blog.dailydoseofds.com/p/the-next-step-after-karpathys-wiki)
+- [The Next Step After Karpathy's Wiki Idea - Avi Chawla](https://blog.dailydoseofds.com/p/the-next-step-after-karpathys-wiki)
- [Gamgee: Why the Future of AI Memory Isn't RAG](https://gamgee.ai/blogs/karpathy-llm-wiki-memory-pattern/)
- [Beyond RAG: How Karpathy's LLM Wiki Pattern Builds Knowledge That Compounds (Plaban Nayak, Level Up Coding)](https://levelup.gitconnected.com/beyond-rag-how-andrej-karpathys-llm-wiki-pattern-builds-knowledge-that-actually-compounds-31a08528665e)
-- [Analytics Vidhya — LLM Wiki Revolution](https://www.analyticsvidhya.com/blog/2026/04/llm-wiki-by-andrej-karpathy/)
-- [Agentpedia — Karpathy's LLM Wiki: Complete Guide to His Idea File](https://agentpedia.codes/blog/karpathy-llm-wiki-idea-file)
+- [Analytics Vidhya - LLM Wiki Revolution](https://www.analyticsvidhya.com/blog/2026/04/llm-wiki-by-andrej-karpathy/)
+- [Agentpedia - Karpathy's LLM Wiki: Complete Guide to His Idea File](https://agentpedia.codes/blog/karpathy-llm-wiki-idea-file)
- [Tolkien Gateway (the fan-wiki Karpathy cites)](https://tolkiengateway.net/wiki/Main_Page)
-- [qmd — local hybrid markdown search (referenced in the gist)](https://github.com/tobi/qmd)
+- [qmd - local hybrid markdown search (referenced in the gist)](https://github.com/tobi/qmd)
diff --git a/docs/v0.3-roadmap.md b/docs/v0.3-roadmap.md
index 72651678..414f5c61 100644
--- a/docs/v0.3-roadmap.md
+++ b/docs/v0.3-roadmap.md
@@ -8,7 +8,7 @@ Nothing here is committed to a milestone yet; this is a parking lot.
### Rule-vs-fact disambiguation in consolidation
**Problem.** Users sometimes type durable project rules into the
-agent — "don't forget to never add a new function without a unit
+agent - "don't forget to never add a new function without a unit
test", "we always run lint before merging", "memory mustn't store
PII". These end up as ordinary observations in ai-memory. But they'd
be more useful as instructions in the project's `CLAUDE.md` /
@@ -32,14 +32,14 @@ Detection heuristics: starts with `always`, `never`, `don't`,
`remember to`, `before X always`, `as a rule`, or contains
`(rule|policy|convention)`. Easy, ~50 lines plus the lint surface.
-The output is a *suggestion only* — ai-memory never auto-edits a
+The output is a *suggestion only* - ai-memory never auto-edits a
user's CLAUDE.md / AGENTS.md / project instructions file.
### Bootstrap an existing project's history into the wiki
**Problem.** Today ai-memory starts empty for any project. For
existing projects with months of decisions in `git log`, `docs/`,
-`README.md`, code comments — that's a lot of warm context the user
+`README.md`, code comments - that's a lot of warm context the user
already paid for once, and we throw it away.
**Sketch.** A `ai-memory bootstrap` one-shot subcommand. Inputs:
diff --git a/docs/wiki-migrations.md b/docs/wiki-migrations.md
index 64883032..2e5dc66a 100644
--- a/docs/wiki-migrations.md
+++ b/docs/wiki-migrations.md
@@ -128,7 +128,7 @@ mod tests {
### No destructive deletes without a graveyard step
-If a migration removes files (not just moves them), copy or move them to
+If a migration removes files (moves them), copy or move them to
`<wiki_root>/_graveyard/<migration_name>/<original_path>` first. This lets
operators recover accidentally-deleted data for at least one release cycle.