Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
docs: catch up with the v0.8.x feature surface
AkitaOnRailsfad242a4fd18
diff --git a/README.md b/README.mdindex 3f14bdf8..352a6e95 100644--- a/README.md+++ b/README.md@@ -10,7 +10,7 @@> re-explaining the architecture, the failed approaches, or the open> questions.-[](docs/ARCHITECTURE.md)+[](docs/ARCHITECTURE.md)[](rust-toolchain.toml)[](LICENSE)@@ -58,11 +58,15 @@ priors are at the [bottom](#influences-and-prior-art)."where you left off" block before its first prompt.- **Per-project isolation by construction.** Each project lives at`<wiki_root>/<workspace_id>/<project_id>/…` keyed by stable UUIDs.- By default `workspace = "default"` and `project = basename($cwd)`.- Drop a [`.ai-memory.toml` marker file](docs/marker-file.md) in any- ancestor directory to override either or opt into repo-root project- identity — perfect for multi-client consultancies, work/personal split,- mono-repos, or linked git worktrees.+ Workspace defaults to `"default"`. Project is derived from `$cwd`:+ CLI subcommands (`bootstrap`, `write-page`, `lint`, …) walk to the+ main git repo root so all worktrees of the same repo share one+ project identity; the hook router defaults to `basename($cwd)` and+ can opt into the repo-root rule. Drop a+ [`.ai-memory.toml` marker file](docs/marker-file.md) in any+ ancestor directory to override either field explicitly — perfect for+ multi-client consultancies, work/personal split, mono-repos, or+ linked git worktrees.Same page path can exist in two projects without collision; arename is one column update; a purge is one `rm -rf`.- **Karpathy-style LLM wiki.** Pages are compiled from observations@@ -104,8 +108,11 @@ priors are at the [bottom](#influences-and-prior-art).on Postgres for X" or "annotate this as a project rule" and it calls`memory_write_page` to write a durable, git-versioned wiki page. Froma terminal it's `ai-memory write-page --path decisions/0007-db.md- --title "Standardised on Postgres" --body "..." --pinned` (`--pinned`- exempts it from the decay sweep). Unlike a handoff (single-use) or an+ --body $'# Standardised on Postgres\n\n...' --pinned`. `--pinned`+ exempts it from the decay sweep; the H1 on the first line of+ `--body` becomes the page title (omit `--title` — it's still+ accepted, but LLM callers trip over JSON-escaping their way through+ it, see issue #67). Unlike a handoff (single-use) or anauto-synthesised session page (rewritten on consolidation), awrite-page note is yours: it shows up in `memory_query`, renders in`/web`, and stays until you change it.diff --git a/docs/admission-webhooks.md b/docs/admission-webhooks.mdindex 8377a88e..d5508857 100644--- a/docs/admission-webhooks.md+++ b/docs/admission-webhooks.md@@ -58,9 +58,18 @@ Webhooks fire on these `op` values today (extensible enum):reject the project move.`delete` / `purge_project` / `move_project` are notifications — there is no-body to mutate; a `Reject`-policy webhook still aborts the operation. Each webhook opts-into the ops it cares about via `events`; the chain checks the op against-`WebhookConfig::events` before dispatching.+body to mutate; a `Reject`-policy webhook still aborts the operation+(admission fires BEFORE the SQL destruction in both the `/admin/purge-project`+and `/admin/move-project` copy-purge paths, so reject leaves the source+intact). Each webhook opts into the ops it cares about via `events`; the+chain checks the op against `WebhookConfig::events` before dispatching.++A copy-purge `/admin/move-project` fires **two** webhook events from+one request: one or more `write_page` notifications as the pages copy+into the destination, then one terminal `purge_project` notification+when the source is torn down. The `purge_project` event carries+`partial_failure: true` if the SQL purge committed but the on-disk+dir removal failed afterwards.### What does NOT fire the chain (by design)@@ -107,7 +116,13 @@ X-Memory-Op: write_page | consolidate | delete | purge_project | move_project"client": "72836f52-...", // DCR client UUID"session_id": "019e6d-..."},- "op": "write_page" // write_page | consolidate | delete | purge_project | move_project+ "op": "write_page", // write_page | consolidate | delete | purge_project | move_project+ "partial_failure": true // purge_project only, and ONLY when set+ // (skipped on the wire when false).+ // true → the DB rows were purged but+ // `remove_project_dir` failed afterwards;+ // a filesystem-tracking mirror (git push)+ // should refuse to drop its own copy.}}```diff --git a/docs/deploy.md b/docs/deploy.mdindex 8fb7c540..ca0ec4a7 100644--- a/docs/deploy.md+++ b/docs/deploy.md@@ -117,6 +117,7 @@ deployment guide, including:- When to add TLS and when to skip it (the loopback + stdio cases honestly don't need it).- Copy-paste docker compose templates in [`docker/compose.tls.caddy.yml`](../docker/compose.tls.caddy.yml) and [`docker/compose.tls.cloudflared.yml`](../docker/compose.tls.cloudflared.yml).- Per-OS trust-store install for the internal-CA path (the load-bearing manual step).+- [Hosting under a subpath](https-via-proxy.md#hosting-under-a-subpath) via `--base-path` / `AI_MEMORY_BASE_PATH` when ai-memory shares a hostname with other apps.- The explicit "what can go wrong" sections so you don't ship security theatre by accident.For the single-user-on-loopback Quick Start, the bearer token alonediff --git a/docs/frontend-api.md b/docs/frontend-api.mdindex 90637d63..a9e8844f 100644--- a/docs/frontend-api.md+++ b/docs/frontend-api.md@@ -331,15 +331,46 @@ across all projects in the workspace:> Note: `last_open_handoff` is **not** consumed by the read API — the> handoff stays "open" and can still be accepted by the next agent.+### 4.9 Cross-project graph++```http+GET /api/v1/graph+```++Returns every resolved wikilink whose endpoints sit in different+projects, with both endpoints' workspace + project + path. Useful for+rendering a project-level dependency view in the SPA.++```json+{+ "edges": [+ {+ "from_workspace": "default",+ "from_project": "ai-memory",+ "from_path": "decisions/0014-storage.md",+ "to_workspace": "default",+ "to_project": "infra",+ "to_path": "runbooks/sqlite-wal.md"+ }+ ]+}+```++Global today (no workspace / project filter); narrower query params+are a follow-up.+## 5. Limits and pagination- All `limit` query params clamp to `1..=100`.- `POST /api/v1/search`: at most **25 scopes** per request.- HTTP body cap: **10 MB** (shared with the MCP body limit; you won'thit this for normal API traffic).-- **No `Cache-Control` / ETag headers yet.** A polling SPA hits the DB- on every request. If your UI polls aggressively, consider client-side- debouncing; server-side caching is a planned iteration.+- **Cache-Control + ETag.** Idempotent read endpoints (workspaces,+ projects, pages list, page read, recent, briefing, overview) send+ `Cache-Control: private, max-age=N` with an N tuned per endpoint+ and a SHA-256 `ETag` derived from the response body. Browsers that+ echo back `If-None-Match` receive a `304 Not Modified` with no body.+ Search responses are not cacheable (request body affects the result).## 6. Custom UI hosting and base paths@@ -385,6 +416,22 @@ With `--base-path /wiki`, the API lives at `/wiki/api/v1`, MCP atdefault web UI at `/wiki/web`. Set `--web-slug /` to mount the web UI or customSPA at the base root (`/wiki`) instead of `/wiki/web`.+**Base-path safety rules.** Both `--base-path` and `--web-slug` go+through the same normaliser. Segments must be RFC 3986 unreserved+characters (`[A-Za-z0-9-._~]`). Three things collapse the prefix to+`""` (root mount) with a startup `WARN` so you can see the+downgrade in the log:++- `.` or `..` segments. Their characters are unreserved on their own,+ but at the segment boundary they mean "current" and "parent" — one+ typo and your prefix is a traversal vector.+- Any character outside the unreserved set (spaces, `<`, `"`, etc.).+- Empty / whitespace-only input.++The trailing-slash redirect at `{base_path}{web_slug}/` →+`{base_path}{web_slug}` keeps the query string. Fragments are+client-side and never reach the server.+When `--web-ui-dir` is **absent**, the built-in server-side `/web`browser is the default (read-only HTML rendering, FTS5 search,project tree). No regression.@@ -459,15 +506,27 @@ Read these:| Auth + middleware layering | `crates/ai-memory-cli/src/commands/serve.rs` (`mount_web_router`, `apply_http_layers`) || Custom-UI dir validation | `crates/ai-memory-cli/src/commands/serve.rs` (`validate_web_ui_args`) |-## 9. Known gaps (planned iterations, not blockers)+## 9. CORS++`/api/v1` accepts cross-origin requests when the operator configures+the allow-list. The CORS layer is scoped to that router only — `/mcp`,+`/hook`, `/admin/*`, and `/web` stay same-origin.++Configure via either `--cors-allow-origin <origin>` (repeatable) on+the `serve` subcommand or `AI_MEMORY_CORS_ALLOW_ORIGINS=<csv>` in the+environment. The list is validated at startup:++- Each entry must be a fully-qualified `scheme://host[:port]` URL.+- No trailing slash, no path, no query, no wildcard (`*`).+- Mixed `http://` + `https://` is fine; pick what your SPA serves.++Invalid origins fail startup with a clear error rather than silently+accepting wildcard. The layer allows `GET / POST / OPTIONS`,+`Authorization` + `Content-Type` headers, and credentials, with a+10-minute preflight cache.++## 10. Known gaps (planned iterations, not blockers)-- **`Cache-Control` + ETags** on idempotent reads. Right now every- `/api/v1/overview` poll hits the DB. Adding `Cache-Control: private,- max-age=N` + ETags is a quick win once polling patterns are observed.-- **CORS.** No `CorsLayer` is mounted, so the API is reachable only- same-origin or via a reverse proxy that adds the headers. A future- `--cors-allow-origin` flag would let separately-hosted frontends talk- to a remote `ai-memory` server directly.- **Write surface.** Browsers can't mutate today (notes, consolidate,lint, purge — all live under `/admin/*` for the CLI or under MCPtools for agents). A thin authenticated write surface ("edit thisdiff --git a/docs/https-via-proxy.md b/docs/https-via-proxy.mdindex e8e6fce4..aab7a4e1 100644--- a/docs/https-via-proxy.md+++ b/docs/https-via-proxy.md@@ -159,6 +159,16 @@ The built-in browser is then at `https://memory.example.com/wiki/web`; add`AI_MEMORY_WEB_SLUG=/` if you want the browser or custom `--web-ui-dir` SPA at`https://memory.example.com/wiki` itself.+**Safety rules on both flags.** `AI_MEMORY_BASE_PATH` and+`AI_MEMORY_WEB_SLUG` go through the same normaliser. Segments must be+RFC 3986 unreserved characters (`[A-Za-z0-9-._~]`). Dot-segments+(`.` / `..`) are rejected — they mean "current" and "parent" at a+segment boundary, so accepting them would let a typo turn the prefix+into traversal. Anything outside the unreserved set falls back to a+root mount, and the startup log says why. The trailing-slash redirect+at `{base_path}{web_slug}/` keeps the query string on its way to the+canonical form.+### MCP client config (Claude Code shown — others follow the same shape)```bashdiff --git a/docs/install.md b/docs/install.mdindex 2ab70a0f..bc01648a 100644--- a/docs/install.md+++ b/docs/install.md@@ -610,6 +610,18 @@ local-data directory on Windows, typicallyTo require bearer-token auth, set `AI_MEMORY_AUTH_TOKEN` in theserver's environment.+#### Optional serve flags++The `serve` subcommand also accepts:++| Flag | Env var | What it does |+|---|---|---|+| `--enable-web` | `AI_MEMORY_ENABLE_WEB=true` | Mount the read-only web browser + `/api/v1` JSON API. |+| `--base-path /wiki` | `AI_MEMORY_BASE_PATH` | Host the entire HTTP surface (`/mcp`, `/hook`, `/admin/*`, `/api/v1`, `/web`) under a configurable subpath — useful behind a reverse proxy sharing a hostname. `.` and `..` segments are rejected; unsafe chars cause a fallback to root with a warning. See [`docs/https-via-proxy.md`](https-via-proxy.md#hosting-under-a-subpath). |+| `--web-slug /web` | `AI_MEMORY_WEB_SLUG` | Where the web UI mounts within the base-path. Default `/web`; set to `/` to mount the UI at the base-path root. |+| `--web-ui-dir <path>` | `AI_MEMORY_WEB_UI_DIR` | Serve a custom SPA from `<path>` instead of the built-in browser. ai-memory injects `<base href>` and `<meta name="ai-memory-base-path">` so the SPA can build relative URLs and API calls under the configured prefix. |+| `--cors-allow-origin <origin>` | `AI_MEMORY_CORS_ALLOW_ORIGINS` (CSV) | Allow listed origins to call `/api/v1`. Layer is scoped only to that route — `/mcp`, `/hook`, `/admin`, and `/web` remain origin-locked. |+On Windows, see [`docs/windows.md`](windows.md). The short version: runthe install commands from the same environment that launches the agent.WSL2-launched agents need WSL paths and POSIX `.sh` hooks. Native Claude Code@@ -903,7 +915,11 @@ remote or uses a custom host/port.```--repo-path <PATH> (default: git rev-parse --show-toplevel)--workspace <NAME> (default: "default")---project <NAME> (default: "scratch")+--project <NAME> (default: derived from cwd — main repo root's+ basename via `git rev-parse --show-toplevel`,+ or basename(cwd) when no repo is found.+ "scratch" only as a defensive fallback for+ hook events with no usable cwd.)--max-input-tokens N (default: 150000; total source budget after prune)--chunk-input-tokens N (default: 24000; per LLM call; 0 = single call)--since "30 days ago" (git log filter; supports "N days/months/years ago" + YYYY-MM-DD)diff --git a/docs/lifecycle-ops.md b/docs/lifecycle-ops.mdindex c9dfd5c6..7f31a3f0 100644--- a/docs/lifecycle-ops.md+++ b/docs/lifecycle-ops.md@@ -10,7 +10,7 @@ on a homelab box where mistakes are harder to undo.|---|---|---|---|---|| `purge-project --confirm` | ✅ yes | the one project's data | no | Atomic `rm -rf <project_root>` on the namespaced disk path; sibling projects untouched. || `rename-project --from --to` | ✅ yes | no | yes (rename back) | Column-only update on `projects.name`. The on-disk dir is keyed by `project_id` (UUID), so the rename never moves a file. |-| `move-project --confirm` | ✅ yes | source only in the merge case | no | Fresh destination → lossless **true move** (re-stamp `workspace_id`, keep `project_id`, rename the dir): sessions/observations/handoffs + history all survive. Destination with a same-named project → **copy+purge merge**: only latest pages migrate. |+| `move-project --confirm` | ✅ yes | source only in the merge case (a `Reject`-policy `purge_project` webhook can still abort the source teardown leaving everything intact) | no | Fresh destination → lossless **true move** (re-stamp `workspace_id`, keep `project_id`, rename the dir): sessions/observations/handoffs + history all survive. Destination with a same-named project → **copy+purge merge**: only latest pages migrate. || `backup --output-path` | ✅ yes | no | n/a | Streams a gzipped tarball from the server's online `sqlite3 .backup` plus the wiki tree. Safe alongside the live writer. || `restore --from <tarball>` | ❌ **stop the server first** | overwrites the data dir | no (without prior backup) | Refuses if any sibling `ai-memory` process is alive (sysinfo guard). || `reset --confirm` | ❌ **stop the server first** | yes, all data | no | Refuses if any sibling `ai-memory` process is alive (sysinfo guard). |@@ -201,13 +201,18 @@ at the same path):resolves the conflicts or re-runs with an explicit policy.- **`overwrite`** — the source page supersedes the destination page at thesame path (the destination's prior version becomes history).-- **`duplicate`** — keep both: the source page lands under a de-duplicated- path (`<stem>-from-<src_workspace>.md`, then `-2`, …). Note: wikilinks- pointing at the original path are not rewritten, so the lossless- `true-move` path remains the way to preserve paths and links.+- **`duplicate`** — keep both: the source page lands at+ `<stem>-from-<src_workspace_slug>.md`, then `-2`, `-3`, … on+ further collisions. The `-from-` literal is the `DEDUP_FROM_TOKEN`+ constant in `crates/ai-memory-mcp/src/admin.rs`; if you ever+ change one, change the other. Wikilinks pointing at the original+ path are not rewritten, so the lossless `true-move` path remains+ the way to preserve paths and links.Every conflict (overwrite/duplicate) is listed in the response `conflicts`-array (`path` → `moved_to`). Set the policy via `--on-conflict` on the CLI.+array (`path` → `moved_to`). Set the policy via `--on-conflict` on the CLI+or `"on_conflict": "block" | "overwrite" | "duplicate"` in the JSON body+for direct `/admin/move-project` callers.**What does NOT migrate (merge case only):** in the `copy-purge` path thesource's `sessions`, `observations`, and `handoffs` (the raw episodic@@ -228,9 +233,19 @@ Failure modes:- **Missing `--confirm`** → 400.- **`from_workspace == to_workspace`** → 422 (use `rename-project`).- **Source project not found** → 404.-- **True-move admission, rename, or SQL re-stamp failure** → 500 and no- committed move. If a rare rollback double-fault happens after the directory- moved but before SQL committed, the error includes the exact manual repair.+- **Destination workspace directory already exists** (true-move only)+ → 409 with `WikiError::DestinationExists` body — the destination+ has on-disk content for the same `(workspace, project)` UUID pair+ without a corresponding DB row; refuse and let the operator+ reconcile manually.+- **Block-policy same-path conflict** (copy-purge merge only) → 409+ with `{"error": "...", "conflicts": [paths...]}` listing every+ conflicting path. Re-run with `on_conflict=overwrite` or+ `on_conflict=duplicate` to proceed.+- **True-move admission or SQL re-stamp failure** → 500 and no+ committed move. If a rare rollback double-fault happens after the+ directory moved but before SQL committed, the error includes the+ exact manual repair.### `backup`diff --git a/docs/llm-provider-comparison.md b/docs/llm-provider-comparison.mdindex 20898cb4..4610a619 100644--- a/docs/llm-provider-comparison.md+++ b/docs/llm-provider-comparison.md@@ -164,11 +164,32 @@ support to enforce - will drift to whatever feels natural.Compounding this at the time of the run: openai-compat providers(Ollama, OpenRouter passthrough) were using ai-memory's tolerant-parser path, so the schema was descriptive, not coercive. ai-memory-now has opt-in `AI_MEMORY_LLM_COMPAT_STRICT=true` for compatible-engines that honour OpenAI-style `response_format=json_schema`, but-the prompt still has to do the load-bearing work when strict mode is-off or the strict raw call falls back.+parser path, so the schema was descriptive, not coercive. The+provider opt-in for coercive structured output is documented below.++#### Opting into strict structured output for openai-compat++Set `AI_MEMORY_LLM_COMPAT_STRICT=true` to make `openai-compat` send+`response_format={ type: "json_schema", strict: true }` first. On a+parse-shape failure (the upstream returned a response but it wasn't a+valid JSON object), ai-memory falls back to the tolerant parser the+default mode uses; on HTTP 4xx / 5xx / auth / transport errors the+strict call propagates without retry (a tolerant retry would just hit+the same wall and double cost — see `is_parse_shape_error` in+`crates/ai-memory-llm/src/openai_compat.rs`).++**Cost.** Strict mode is one HTTP call when the upstream honours+`response_format`. When it doesn't, you pay a second call for the+tolerant fallback. Pick by engine:++| Engine class | Setting |+|---|---|+| Modern Ollama / vLLM / LM Studio honouring `response_format=json_schema` | `AI_MEMORY_LLM_COMPAT_STRICT=true` (one call, schema-constrained) |+| Reasoning models with `<think>…</think>` inside `content` (DeepSeek-R1, Qwen3-Thinking, MiniMax M2) | Leave OFF. Strict-then-fallback burns a call per consolidation; the tolerant path already strips `<think>` before parsing |+| Older engines / proxies that ignore `response_format` | Leave OFF. Strict adds latency and recovers nothing |++The prompt still has to do the load-bearing work when strict mode is+off or the strict call falls back.## The fix@@ -682,8 +703,10 @@ Re-run this harness when any of the following changes:drops for Ollama)- A new fixture is added to `evals/fixtures/`- The home server hardware changes-- An OpenAI / Anthropic / Voyage strict-JSON-schema feature- becomes available through OpenRouter+- A new local engine ships first-class+ `response_format=json_schema` support, making+ `AI_MEMORY_LLM_COMPAT_STRICT=true` worth re-benchmarking against+ the tolerant-parser baseline## How to reproducediff --git a/docs/mcp-install.md b/docs/mcp-install.mdindex fb2db0b2..a1a8cf3d 100644--- a/docs/mcp-install.md+++ b/docs/mcp-install.md@@ -402,8 +402,8 @@ You: List the MCP tools you can call. Use one of them to checkModel (any client): I can call: memory_query, memory_recent,memory_status, memory_briefing, memory_explore,memory_handoff_accept, memory_handoff_begin, memory_consolidate,- memory_write_page, memory_lint, memory_forget_sweep,- memory_install_self_routing.+ memory_write_page, memory_read_page, memory_delete_page,+ memory_lint, memory_forget_sweep, memory_install_self_routing.memory_status reports: 0 pages, 0 observations, 0 sessions.```diff --git a/docs/usage.md b/docs/usage.mdindex a16c7794..9481b161 100644--- a/docs/usage.md+++ b/docs/usage.md@@ -140,10 +140,22 @@ breadcrumbs, rendered markdown, metadata, and FTS5 search. In renderedpages, `[[wiki links]]` become clickable links to the target page —`[[path]]`, `[[path|label]]`, `[[project:path]]`, and`[[workspace/project:path]]` are all supported (resolved against the-current page's project unless the target carries its own scope). If the-server has `AI_MEMORY_AUTH_TOKEN` set, the browser uses HTTP Basic auth:-leave the username blank and paste the token as the password. MCP and-hook clients continue to use `Authorization: Bearer <token>`.+current page's project unless the target carries its own scope).+`[[…]]` stays literal inside fenced code (` ``` ` and `~~~` close+only by their own glyph), inline `` `…` `` code, and 4-space-indented+code; external schemes inside the brackets (`http://`, `https://`,+`mailto:`, `data:`, `javascript:`, `vbscript:`, `tel:`, `file:`)+stay literal too. If the server has `AI_MEMORY_AUTH_TOKEN` set, the+browser uses HTTP Basic auth: leave the username blank and paste the+token as the password. MCP and hook clients continue to use+`Authorization: Bearer <token>`.++To host the web UI under a URL subpath behind a reverse proxy, the+`--base-path` / `--web-slug` flags do the work — see+[`docs/frontend-api.md`](frontend-api.md#6-custom-ui-hosting-and-base-paths)+for the flag semantics and+[`docs/https-via-proxy.md`](https-via-proxy.md#hosting-under-a-subpath)+for the proxy-side walk-through.