Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

docs: catch up with the v0.8.x feature surface

AkitaOnRailsfad242a4fd18
diff --git a/README.md b/README.md
index 3f14bdf8..352a6e95 100644
--- a/README.md
+++ b/README.md
@@ -10,7 +10,7 @@
> re-explaining the architecture, the failed approaches, or the open
> questions.
-[![status: v0.2 milestones complete](https://img.shields.io/badge/status-v0.2--complete-green)](docs/ARCHITECTURE.md)
+[![status: v0.8 multi-user](https://img.shields.io/badge/status-v0.8--multiuser-green)](docs/ARCHITECTURE.md)
[![Rust](https://img.shields.io/badge/rust-1.95+-blue)](rust-toolchain.toml)
[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
@@ -58,11 +58,15 @@ priors are at the [bottom](#influences-and-prior-art).
"where you left off" block before its first prompt.
- **Per-project isolation by construction.** Each project lives at
`<wiki_root>/<workspace_id>/<project_id>/…` keyed by stable UUIDs.
- By default `workspace = "default"` and `project = basename($cwd)`.
- Drop a [`.ai-memory.toml` marker file](docs/marker-file.md) in any
- ancestor directory to override either or opt into repo-root project
- identity — perfect for multi-client consultancies, work/personal split,
- mono-repos, or linked git worktrees.
+ Workspace defaults to `"default"`. Project is derived from `$cwd`:
+ CLI subcommands (`bootstrap`, `write-page`, `lint`, …) walk to the
+ main git repo root so all worktrees of the same repo share one
+ project identity; the hook router defaults to `basename($cwd)` and
+ can opt into the repo-root rule. Drop a
+ [`.ai-memory.toml` marker file](docs/marker-file.md) in any
+ ancestor directory to override either field explicitly — perfect for
+ multi-client consultancies, work/personal split, mono-repos, or
+ linked git worktrees.
Same page path can exist in two projects without collision; a
rename is one column update; a purge is one `rm -rf`.
- **Karpathy-style LLM wiki.** Pages are compiled from observations
@@ -104,8 +108,11 @@ priors are at the [bottom](#influences-and-prior-art).
on Postgres for X" or "annotate this as a project rule" and it calls
`memory_write_page` to write a durable, git-versioned wiki page. From
a terminal it's `ai-memory write-page --path decisions/0007-db.md
- --title "Standardised on Postgres" --body "..." --pinned` (`--pinned`
- exempts it from the decay sweep). Unlike a handoff (single-use) or an
+ --body $'# Standardised on Postgres\n\n...' --pinned`. `--pinned`
+ exempts it from the decay sweep; the H1 on the first line of
+ `--body` becomes the page title (omit `--title` — it's still
+ accepted, but LLM callers trip over JSON-escaping their way through
+ it, see issue #67). Unlike a handoff (single-use) or an
auto-synthesised session page (rewritten on consolidation), a
write-page note is yours: it shows up in `memory_query`, renders in
`/web`, and stays until you change it.
diff --git a/docs/admission-webhooks.md b/docs/admission-webhooks.md
index 8377a88e..d5508857 100644
--- a/docs/admission-webhooks.md
+++ b/docs/admission-webhooks.md
@@ -58,9 +58,18 @@ Webhooks fire on these `op` values today (extensible enum):
reject the project move.
`delete` / `purge_project` / `move_project` are notifications — there is no
-body to mutate; a `Reject`-policy webhook still aborts the operation. Each webhook opts
-into the ops it cares about via `events`; the chain checks the op against
-`WebhookConfig::events` before dispatching.
+body to mutate; a `Reject`-policy webhook still aborts the operation
+(admission fires BEFORE the SQL destruction in both the `/admin/purge-project`
+and `/admin/move-project` copy-purge paths, so reject leaves the source
+intact). Each webhook opts into the ops it cares about via `events`; the
+chain checks the op against `WebhookConfig::events` before dispatching.
+
+A copy-purge `/admin/move-project` fires **two** webhook events from
+one request: one or more `write_page` notifications as the pages copy
+into the destination, then one terminal `purge_project` notification
+when the source is torn down. The `purge_project` event carries
+`partial_failure: true` if the SQL purge committed but the on-disk
+dir removal failed afterwards.
### What does NOT fire the chain (by design)
@@ -107,7 +116,13 @@ X-Memory-Op: write_page | consolidate | delete | purge_project | move_project
"client": "72836f52-...", // DCR client UUID
"session_id": "019e6d-..."
},
- "op": "write_page" // write_page | consolidate | delete | purge_project | move_project
+ "op": "write_page", // write_page | consolidate | delete | purge_project | move_project
+ "partial_failure": true // purge_project only, and ONLY when set
+ // (skipped on the wire when false).
+ // true → the DB rows were purged but
+ // `remove_project_dir` failed afterwards;
+ // a filesystem-tracking mirror (git push)
+ // should refuse to drop its own copy.
}
}
```
diff --git a/docs/deploy.md b/docs/deploy.md
index 8fb7c540..ca0ec4a7 100644
--- a/docs/deploy.md
+++ b/docs/deploy.md
@@ -117,6 +117,7 @@ deployment guide, including:
- When to add TLS and when to skip it (the loopback + stdio cases honestly don't need it).
- Copy-paste docker compose templates in [`docker/compose.tls.caddy.yml`](../docker/compose.tls.caddy.yml) and [`docker/compose.tls.cloudflared.yml`](../docker/compose.tls.cloudflared.yml).
- Per-OS trust-store install for the internal-CA path (the load-bearing manual step).
+- [Hosting under a subpath](https-via-proxy.md#hosting-under-a-subpath) via `--base-path` / `AI_MEMORY_BASE_PATH` when ai-memory shares a hostname with other apps.
- The explicit "what can go wrong" sections so you don't ship security theatre by accident.
For the single-user-on-loopback Quick Start, the bearer token alone
diff --git a/docs/frontend-api.md b/docs/frontend-api.md
index 90637d63..a9e8844f 100644
--- a/docs/frontend-api.md
+++ b/docs/frontend-api.md
@@ -331,15 +331,46 @@ across all projects in the workspace:
> Note: `last_open_handoff` is **not** consumed by the read API — the
> handoff stays "open" and can still be accepted by the next agent.
+### 4.9 Cross-project graph
+
+```http
+GET /api/v1/graph
+```
+
+Returns every resolved wikilink whose endpoints sit in different
+projects, with both endpoints' workspace + project + path. Useful for
+rendering a project-level dependency view in the SPA.
+
+```json
+{
+ "edges": [
+ {
+ "from_workspace": "default",
+ "from_project": "ai-memory",
+ "from_path": "decisions/0014-storage.md",
+ "to_workspace": "default",
+ "to_project": "infra",
+ "to_path": "runbooks/sqlite-wal.md"
+ }
+ ]
+}
+```
+
+Global today (no workspace / project filter); narrower query params
+are a follow-up.
+
## 5. Limits and pagination
- All `limit` query params clamp to `1..=100`.
- `POST /api/v1/search`: at most **25 scopes** per request.
- HTTP body cap: **10 MB** (shared with the MCP body limit; you won't
hit this for normal API traffic).
-- **No `Cache-Control` / ETag headers yet.** A polling SPA hits the DB
- on every request. If your UI polls aggressively, consider client-side
- debouncing; server-side caching is a planned iteration.
+- **Cache-Control + ETag.** Idempotent read endpoints (workspaces,
+ projects, pages list, page read, recent, briefing, overview) send
+ `Cache-Control: private, max-age=N` with an N tuned per endpoint
+ and a SHA-256 `ETag` derived from the response body. Browsers that
+ echo back `If-None-Match` receive a `304 Not Modified` with no body.
+ Search responses are not cacheable (request body affects the result).
## 6. Custom UI hosting and base paths
@@ -385,6 +416,22 @@ With `--base-path /wiki`, the API lives at `/wiki/api/v1`, MCP at
default web UI at `/wiki/web`. Set `--web-slug /` to mount the web UI or custom
SPA at the base root (`/wiki`) instead of `/wiki/web`.
+**Base-path safety rules.** Both `--base-path` and `--web-slug` go
+through the same normaliser. Segments must be RFC 3986 unreserved
+characters (`[A-Za-z0-9-._~]`). Three things collapse the prefix to
+`""` (root mount) with a startup `WARN` so you can see the
+downgrade in the log:
+
+- `.` or `..` segments. Their characters are unreserved on their own,
+ but at the segment boundary they mean "current" and "parent" — one
+ typo and your prefix is a traversal vector.
+- Any character outside the unreserved set (spaces, `<`, `"`, etc.).
+- Empty / whitespace-only input.
+
+The trailing-slash redirect at `{base_path}{web_slug}/` →
+`{base_path}{web_slug}` keeps the query string. Fragments are
+client-side and never reach the server.
+
When `--web-ui-dir` is **absent**, the built-in server-side `/web`
browser is the default (read-only HTML rendering, FTS5 search,
project tree). No regression.
@@ -459,15 +506,27 @@ Read these:
| Auth + middleware layering | `crates/ai-memory-cli/src/commands/serve.rs` (`mount_web_router`, `apply_http_layers`) |
| Custom-UI dir validation | `crates/ai-memory-cli/src/commands/serve.rs` (`validate_web_ui_args`) |
-## 9. Known gaps (planned iterations, not blockers)
+## 9. CORS
+
+`/api/v1` accepts cross-origin requests when the operator configures
+the allow-list. The CORS layer is scoped to that router only — `/mcp`,
+`/hook`, `/admin/*`, and `/web` stay same-origin.
+
+Configure via either `--cors-allow-origin <origin>` (repeatable) on
+the `serve` subcommand or `AI_MEMORY_CORS_ALLOW_ORIGINS=<csv>` in the
+environment. The list is validated at startup:
+
+- Each entry must be a fully-qualified `scheme://host[:port]` URL.
+- No trailing slash, no path, no query, no wildcard (`*`).
+- Mixed `http://` + `https://` is fine; pick what your SPA serves.
+
+Invalid origins fail startup with a clear error rather than silently
+accepting wildcard. The layer allows `GET / POST / OPTIONS`,
+`Authorization` + `Content-Type` headers, and credentials, with a
+10-minute preflight cache.
+
+## 10. Known gaps (planned iterations, not blockers)
-- **`Cache-Control` + ETags** on idempotent reads. Right now every
- `/api/v1/overview` poll hits the DB. Adding `Cache-Control: private,
- max-age=N` + ETags is a quick win once polling patterns are observed.
-- **CORS.** No `CorsLayer` is mounted, so the API is reachable only
- same-origin or via a reverse proxy that adds the headers. A future
- `--cors-allow-origin` flag would let separately-hosted frontends talk
- to a remote `ai-memory` server directly.
- **Write surface.** Browsers can't mutate today (notes, consolidate,
lint, purge — all live under `/admin/*` for the CLI or under MCP
tools for agents). A thin authenticated write surface ("edit this
diff --git a/docs/https-via-proxy.md b/docs/https-via-proxy.md
index e8e6fce4..aab7a4e1 100644
--- a/docs/https-via-proxy.md
+++ b/docs/https-via-proxy.md
@@ -159,6 +159,16 @@ The built-in browser is then at `https://memory.example.com/wiki/web`; add
`AI_MEMORY_WEB_SLUG=/` if you want the browser or custom `--web-ui-dir` SPA at
`https://memory.example.com/wiki` itself.
+**Safety rules on both flags.** `AI_MEMORY_BASE_PATH` and
+`AI_MEMORY_WEB_SLUG` go through the same normaliser. Segments must be
+RFC 3986 unreserved characters (`[A-Za-z0-9-._~]`). Dot-segments
+(`.` / `..`) are rejected — they mean "current" and "parent" at a
+segment boundary, so accepting them would let a typo turn the prefix
+into traversal. Anything outside the unreserved set falls back to a
+root mount, and the startup log says why. The trailing-slash redirect
+at `{base_path}{web_slug}/` keeps the query string on its way to the
+canonical form.
+
### MCP client config (Claude Code shown — others follow the same shape)
```bash
diff --git a/docs/install.md b/docs/install.md
index 2ab70a0f..bc01648a 100644
--- a/docs/install.md
+++ b/docs/install.md
@@ -610,6 +610,18 @@ local-data directory on Windows, typically
To require bearer-token auth, set `AI_MEMORY_AUTH_TOKEN` in the
server's environment.
+#### Optional serve flags
+
+The `serve` subcommand also accepts:
+
+| Flag | Env var | What it does |
+|---|---|---|
+| `--enable-web` | `AI_MEMORY_ENABLE_WEB=true` | Mount the read-only web browser + `/api/v1` JSON API. |
+| `--base-path /wiki` | `AI_MEMORY_BASE_PATH` | Host the entire HTTP surface (`/mcp`, `/hook`, `/admin/*`, `/api/v1`, `/web`) under a configurable subpath — useful behind a reverse proxy sharing a hostname. `.` and `..` segments are rejected; unsafe chars cause a fallback to root with a warning. See [`docs/https-via-proxy.md`](https-via-proxy.md#hosting-under-a-subpath). |
+| `--web-slug /web` | `AI_MEMORY_WEB_SLUG` | Where the web UI mounts within the base-path. Default `/web`; set to `/` to mount the UI at the base-path root. |
+| `--web-ui-dir <path>` | `AI_MEMORY_WEB_UI_DIR` | Serve a custom SPA from `<path>` instead of the built-in browser. ai-memory injects `<base href>` and `<meta name="ai-memory-base-path">` so the SPA can build relative URLs and API calls under the configured prefix. |
+| `--cors-allow-origin <origin>` | `AI_MEMORY_CORS_ALLOW_ORIGINS` (CSV) | Allow listed origins to call `/api/v1`. Layer is scoped only to that route — `/mcp`, `/hook`, `/admin`, and `/web` remain origin-locked. |
+
On Windows, see [`docs/windows.md`](windows.md). The short version: run
the install commands from the same environment that launches the agent.
WSL2-launched agents need WSL paths and POSIX `.sh` hooks. Native Claude Code
@@ -903,7 +915,11 @@ remote or uses a custom host/port.
```
--repo-path <PATH> (default: git rev-parse --show-toplevel)
--workspace <NAME> (default: "default")
---project <NAME> (default: "scratch")
+--project <NAME> (default: derived from cwd — main repo root's
+ basename via `git rev-parse --show-toplevel`,
+ or basename(cwd) when no repo is found.
+ "scratch" only as a defensive fallback for
+ hook events with no usable cwd.)
--max-input-tokens N (default: 150000; total source budget after prune)
--chunk-input-tokens N (default: 24000; per LLM call; 0 = single call)
--since "30 days ago" (git log filter; supports "N days/months/years ago" + YYYY-MM-DD)
diff --git a/docs/lifecycle-ops.md b/docs/lifecycle-ops.md
index c9dfd5c6..7f31a3f0 100644
--- a/docs/lifecycle-ops.md
+++ b/docs/lifecycle-ops.md
@@ -10,7 +10,7 @@ on a homelab box where mistakes are harder to undo.
|---|---|---|---|---|
| `purge-project --confirm` | ✅ yes | the one project's data | no | Atomic `rm -rf <project_root>` on the namespaced disk path; sibling projects untouched. |
| `rename-project --from --to` | ✅ yes | no | yes (rename back) | Column-only update on `projects.name`. The on-disk dir is keyed by `project_id` (UUID), so the rename never moves a file. |
-| `move-project --confirm` | ✅ yes | source only in the merge case | no | Fresh destination → lossless **true move** (re-stamp `workspace_id`, keep `project_id`, rename the dir): sessions/observations/handoffs + history all survive. Destination with a same-named project → **copy+purge merge**: only latest pages migrate. |
+| `move-project --confirm` | ✅ yes | source only in the merge case (a `Reject`-policy `purge_project` webhook can still abort the source teardown leaving everything intact) | no | Fresh destination → lossless **true move** (re-stamp `workspace_id`, keep `project_id`, rename the dir): sessions/observations/handoffs + history all survive. Destination with a same-named project → **copy+purge merge**: only latest pages migrate. |
| `backup --output-path` | ✅ yes | no | n/a | Streams a gzipped tarball from the server's online `sqlite3 .backup` plus the wiki tree. Safe alongside the live writer. |
| `restore --from <tarball>` | ❌ **stop the server first** | overwrites the data dir | no (without prior backup) | Refuses if any sibling `ai-memory` process is alive (sysinfo guard). |
| `reset --confirm` | ❌ **stop the server first** | yes, all data | no | Refuses if any sibling `ai-memory` process is alive (sysinfo guard). |
@@ -201,13 +201,18 @@ at the same path):
resolves the conflicts or re-runs with an explicit policy.
- **`overwrite`** — the source page supersedes the destination page at the
same path (the destination's prior version becomes history).
-- **`duplicate`** — keep both: the source page lands under a de-duplicated
- path (`<stem>-from-<src_workspace>.md`, then `-2`, …). Note: wikilinks
- pointing at the original path are not rewritten, so the lossless
- `true-move` path remains the way to preserve paths and links.
+- **`duplicate`** — keep both: the source page lands at
+ `<stem>-from-<src_workspace_slug>.md`, then `-2`, `-3`, … on
+ further collisions. The `-from-` literal is the `DEDUP_FROM_TOKEN`
+ constant in `crates/ai-memory-mcp/src/admin.rs`; if you ever
+ change one, change the other. Wikilinks pointing at the original
+ path are not rewritten, so the lossless `true-move` path remains
+ the way to preserve paths and links.
Every conflict (overwrite/duplicate) is listed in the response `conflicts`
-array (`path` → `moved_to`). Set the policy via `--on-conflict` on the CLI.
+array (`path` → `moved_to`). Set the policy via `--on-conflict` on the CLI
+or `"on_conflict": "block" | "overwrite" | "duplicate"` in the JSON body
+for direct `/admin/move-project` callers.
**What does NOT migrate (merge case only):** in the `copy-purge` path the
source's `sessions`, `observations`, and `handoffs` (the raw episodic
@@ -228,9 +233,19 @@ Failure modes:
- **Missing `--confirm`** → 400.
- **`from_workspace == to_workspace`** → 422 (use `rename-project`).
- **Source project not found** → 404.
-- **True-move admission, rename, or SQL re-stamp failure** → 500 and no
- committed move. If a rare rollback double-fault happens after the directory
- moved but before SQL committed, the error includes the exact manual repair.
+- **Destination workspace directory already exists** (true-move only)
+ → 409 with `WikiError::DestinationExists` body — the destination
+ has on-disk content for the same `(workspace, project)` UUID pair
+ without a corresponding DB row; refuse and let the operator
+ reconcile manually.
+- **Block-policy same-path conflict** (copy-purge merge only) → 409
+ with `{"error": "...", "conflicts": [paths...]}` listing every
+ conflicting path. Re-run with `on_conflict=overwrite` or
+ `on_conflict=duplicate` to proceed.
+- **True-move admission or SQL re-stamp failure** → 500 and no
+ committed move. If a rare rollback double-fault happens after the
+ directory moved but before SQL committed, the error includes the
+ exact manual repair.
### `backup`
diff --git a/docs/llm-provider-comparison.md b/docs/llm-provider-comparison.md
index 20898cb4..4610a619 100644
--- a/docs/llm-provider-comparison.md
+++ b/docs/llm-provider-comparison.md
@@ -164,11 +164,32 @@ support to enforce - will drift to whatever feels natural.
Compounding this at the time of the run: openai-compat providers
(Ollama, OpenRouter passthrough) were using ai-memory's tolerant
-parser path, so the schema was descriptive, not coercive. ai-memory
-now has opt-in `AI_MEMORY_LLM_COMPAT_STRICT=true` for compatible
-engines that honour OpenAI-style `response_format=json_schema`, but
-the prompt still has to do the load-bearing work when strict mode is
-off or the strict raw call falls back.
+parser path, so the schema was descriptive, not coercive. The
+provider opt-in for coercive structured output is documented below.
+
+#### Opting into strict structured output for openai-compat
+
+Set `AI_MEMORY_LLM_COMPAT_STRICT=true` to make `openai-compat` send
+`response_format={ type: "json_schema", strict: true }` first. On a
+parse-shape failure (the upstream returned a response but it wasn't a
+valid JSON object), ai-memory falls back to the tolerant parser the
+default mode uses; on HTTP 4xx / 5xx / auth / transport errors the
+strict call propagates without retry (a tolerant retry would just hit
+the same wall and double cost — see `is_parse_shape_error` in
+`crates/ai-memory-llm/src/openai_compat.rs`).
+
+**Cost.** Strict mode is one HTTP call when the upstream honours
+`response_format`. When it doesn't, you pay a second call for the
+tolerant fallback. Pick by engine:
+
+| Engine class | Setting |
+|---|---|
+| Modern Ollama / vLLM / LM Studio honouring `response_format=json_schema` | `AI_MEMORY_LLM_COMPAT_STRICT=true` (one call, schema-constrained) |
+| Reasoning models with `<think>…</think>` inside `content` (DeepSeek-R1, Qwen3-Thinking, MiniMax M2) | Leave OFF. Strict-then-fallback burns a call per consolidation; the tolerant path already strips `<think>` before parsing |
+| Older engines / proxies that ignore `response_format` | Leave OFF. Strict adds latency and recovers nothing |
+
+The prompt still has to do the load-bearing work when strict mode is
+off or the strict call falls back.
## The fix
@@ -682,8 +703,10 @@ Re-run this harness when any of the following changes:
drops for Ollama)
- A new fixture is added to `evals/fixtures/`
- The home server hardware changes
-- An OpenAI / Anthropic / Voyage strict-JSON-schema feature
- becomes available through OpenRouter
+- A new local engine ships first-class
+ `response_format=json_schema` support, making
+ `AI_MEMORY_LLM_COMPAT_STRICT=true` worth re-benchmarking against
+ the tolerant-parser baseline
## How to reproduce
diff --git a/docs/mcp-install.md b/docs/mcp-install.md
index fb2db0b2..a1a8cf3d 100644
--- a/docs/mcp-install.md
+++ b/docs/mcp-install.md
@@ -402,8 +402,8 @@ You: List the MCP tools you can call. Use one of them to check
Model (any client): I can call: memory_query, memory_recent,
memory_status, memory_briefing, memory_explore,
memory_handoff_accept, memory_handoff_begin, memory_consolidate,
- memory_write_page, memory_lint, memory_forget_sweep,
- memory_install_self_routing.
+ memory_write_page, memory_read_page, memory_delete_page,
+ memory_lint, memory_forget_sweep, memory_install_self_routing.
memory_status reports: 0 pages, 0 observations, 0 sessions.
```
diff --git a/docs/usage.md b/docs/usage.md
index a16c7794..9481b161 100644
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -140,10 +140,22 @@ breadcrumbs, rendered markdown, metadata, and FTS5 search. In rendered
pages, `[[wiki links]]` become clickable links to the target page —
`[[path]]`, `[[path|label]]`, `[[project:path]]`, and
`[[workspace/project:path]]` are all supported (resolved against the
-current page's project unless the target carries its own scope). If the
-server has `AI_MEMORY_AUTH_TOKEN` set, the browser uses HTTP Basic auth:
-leave the username blank and paste the token as the password. MCP and
-hook clients continue to use `Authorization: Bearer <token>`.
+current page's project unless the target carries its own scope).
+`[[…]]` stays literal inside fenced code (` ``` ` and `~~~` close
+only by their own glyph), inline `` `…` `` code, and 4-space-indented
+code; external schemes inside the brackets (`http://`, `https://`,
+`mailto:`, `data:`, `javascript:`, `vbscript:`, `tel:`, `file:`)
+stay literal too. If the server has `AI_MEMORY_AUTH_TOKEN` set, the
+browser uses HTTP Basic auth: leave the username blank and paste the
+token as the password. MCP and hook clients continue to use
+`Authorization: Bearer <token>`.
+
+To host the web UI under a URL subpath behind a reverse proxy, the
+`--base-path` / `--web-slug` flags do the work — see
+[`docs/frontend-api.md`](frontend-api.md#6-custom-ui-hosting-and-base-paths)
+for the flag semantics and
+[`docs/https-via-proxy.md`](https-via-proxy.md#hosting-under-a-subpath)
+for the proxy-side walk-through.
![Project list homepage with four projects shown as cards with page counts and last activity.](web-projects-home.png)