Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
docs: add docs/frontend-api.md — integration guide for /api/v1
AkitaOnRails2227f8a678b3
diff --git a/README.md b/README.mdindex 1f2af7ca..fbdf6625 100644--- a/README.md+++ b/README.md@@ -327,6 +327,11 @@ Useful entry points:`overview` bundles the open handoff + briefing + memory-health for a workspaceor project in one call (the data a project overview screen needs).+ **Full integration guide:** see [`docs/frontend-api.md`](docs/frontend-api.md)+ for auth setup, response schemas, error model, limits/pagination,+ custom-UI hosting, a worked `fetch`/`curl` example, and the canonical+ source-of-truth files. Read that first if you're building a frontend.+To serve your own static frontend instead of the built-in UI, point`--web-ui-dir` at the frontend's build output (same-origin with`/api/v1`, `/mcp`, `/admin/*`, so the existing auth applies):diff --git a/docs/frontend-api.md b/docs/frontend-api.mdnew file mode 100644index 00000000..c6697370--- /dev/null+++ b/docs/frontend-api.md@@ -0,0 +1,456 @@+# Frontend integration: `/api/v1`++> Read-only JSON API and custom-UI hosting model for building third-party+> frontends against an `ai-memory` server. Added in **v0.6.0** (PR #7).+> Everything below is sourced from the actual route handlers in+> `crates/ai-memory-web/src/routes/api.rs` and the response structs in+> `crates/ai-memory-store/src/reader.rs` — keep them as the canonical+> reference if anything here drifts.++## 1. What this surface is (and isn't)++| | What you can do | What you can't do |+|---|---|---|+| `/api/v1/*` | Browse workspaces, projects, pages; read full page markdown + frontmatter + back-links; FTS5 search (global or scoped, single or multi-project); aggregate "overview" snapshots; drill into stale / duplicate / orphan pages. | Write, delete, rename, lint, consolidate, run sweeps, manage handoffs. The `/api/v1` surface is **read-only by construction** — the handlers contain zero writer calls. Writes still go through `/admin/*` (used by the CLI) or MCP tools. |+| `--web-ui-dir` | Host any SPA at `/web`, same-origin with the API, behind the same auth. The default built-in `/web` browser stays the fallback when the flag is absent. | Host the SPA on a *different* origin without a reverse proxy — there's no CORS layer yet (see §9). |++## 2. Auth model++Every `/api/v1/*` request goes through the same bearer + host-allowlist+middleware as `/mcp`, `/hook`, and `/admin/*` — they're all nested+*before* the auth layers are applied+(`crates/ai-memory-cli/src/commands/serve.rs`, see `mount_web_router` →+`apply_http_layers`). So:++- **Anonymous request → `401 Unauthorized`** (when the server is running+ with a bearer token configured).+- **Disallowed `Host` header → `403 Forbidden`** (DNS-rebinding guard).+- The same token protects everything; there is no per-user scoping.+ Single-tenant by design (see [`docs/design-decisions.md`](design-decisions.md) §13).++Pass the bearer in the standard header:++```http+Authorization: Bearer <token>+```++Get a token:++```bash+ai-memory generate-auth-token # writes to stdout+# then export AI_MEMORY_AUTH_TOKEN=<token> in the server's environment,+# or put it under [auth].bearer_token in config.toml+```++In a same-origin SPA, the token can come from:++- A user-pasted value in the UI (the simplest model — same as the+ built-in `/web` browser's HTTP Basic prompt).+- A platform-specific secret store, then injected into `fetch()` calls.++> **XSS note:** if your SPA stores the bearer in `localStorage` and ships+> with an XSS bug, the token is exfiltrable. That's the SPA's risk, not+> the API's. Consider read-only environment-injected tokens or HTTP-only+> cookie tunneling via a reverse proxy if you're hardening for that.++## 3. Error model++All errors return a JSON body of shape:++```json+{ "error": "human-readable message" }+```++with one of these statuses:++| Status | When |+|---|---|+| `400 Bad Request` | invalid query params, malformed `Authorization`, partial scope (workspace without project or vice versa), too many scopes in `POST /search` (>25), empty `q`. |+| `401 Unauthorized` | bearer missing or wrong. |+| `403 Forbidden` | Host header not in allowlist. |+| `404 Not Found` | workspace, project, or page doesn't exist; or page file missing on disk. |+| `500 Internal Server Error` | reader pool / SQLite failure. Body is `{"error":"<context>"}` with the source error chain. |++## 4. Endpoint reference++All endpoints are `GET` unless noted. Paths under `/api/v1/`.++### 4.1 Workspaces++```http+GET /api/v1/workspaces+```++**Response:** `{ "workspaces": [WorkspaceSummary, …] }`++```json+{+ "workspaces": [+ {+ "workspace_name": "default",+ "project_count": 3,+ "page_count": 412,+ "last_updated": "2026-05-28T14:02:11.123Z"+ }+ ]+}+```++`last_updated` is `null` for an empty workspace.++### 4.2 Projects++```http+GET /api/v1/projects # all projects across all workspaces+GET /api/v1/projects?workspace=NAME # projects in one workspace+```++**Response:** `{ "projects": [ProjectSummary, …] }`++```json+{+ "projects": [+ {+ "workspace_name": "default",+ "project_name": "ai-memory",+ "page_count": 138,+ "last_updated": "2026-05-28T14:02:11.123Z"+ }+ ]+}+```++### 4.3 Pages (list)++```http+GET /api/v1/workspaces/{workspace}/projects/{project}/pages+```++**Response:** `{ "pages": [PageSummary, …] }`++```json+{+ "pages": [+ {+ "path": "decisions/0007-db.md",+ "title": "Standardised on Postgres",+ "kind": "decision",+ "tier": "semantic",+ "updated_at": "2026-05-27T09:12:00.000Z"+ }+ ]+}+```++`404` if the workspace or project doesn't exist.++### 4.4 Page (read full)++```http+GET /api/v1/workspaces/{workspace}/projects/{project}/pages/{*path}+```++Wiki path is a wildcard: `decisions/0007-db.md`, `concepts/foo/bar.md`,+etc. Returns merged metadata + body markdown + frontmatter + resolved+links + back-links.++**Response (flat object):**++```json+{+ "project": "ai-memory",+ "path": "decisions/0007-db.md",+ "title": "Standardised on Postgres",+ "kind": "decision",+ "tier": "semantic",+ "pinned": true,+ "created_at": "2026-05-27T09:12:00.000Z",+ "updated_at": "2026-05-28T11:04:33.123Z",+ "supersedes": null,+ "frontmatter": { "tags": ["adr"], "pinned": true },+ "body": "# Standardised on Postgres\n\n…",+ "links": [ { "path": "concepts/db-rules.md", "title": "DB rules", "kind": "rule" } ],+ "backlinks": [ { "path": "sessions/2026-05-27.md", "title": "Session 2026-05-27", "kind": "fact" } ]+}+```++`404` for missing workspace/project, missing page row, or missing file+on disk (the body is read from the markdown file at request time).++### 4.5 Search++Two forms — query-string for the common single-scope or global case,+JSON body for multi-scope.++```http+GET /api/v1/search?q=karpathy&limit=20 # global+GET /api/v1/search?q=karpathy&workspace=default&project=ai-memory # one project+```++```http+POST /api/v1/search+Content-Type: application/json++{+ "q": "karpathy",+ "scopes": [+ { "workspace": "default", "project": "ai-memory" },+ { "workspace": "default", "project": "shared-notes" }+ ],+ "limit": 20+}+```++**Response:** `{ "hits": [PageHit, …] }`++```json+{+ "hits": [+ {+ "id": "01928d27-…",+ "path": "concepts/karpathy-wiki.md",+ "title": "Karpathy LLM Wiki pattern",+ "snippet": "Andrej <mark>Karpathy</mark>'s LLM wiki design …",+ "rank": -8.4+ }+ ]+}+```++Rules:++- `q` is required and non-empty (400 otherwise).+- `limit` is clamped to `1..=100`. Default `10`.+- Partial scope is **rejected** with `400` (passing only `workspace` or+ only `project` to keep scoping unambiguous).+- `scopes` (POST) is capped at `25` entries; can't be combined with+ top-level `workspace`/`project`.+- `snippet` contains FTS5 HTML markers (`<mark>…</mark>`) around the+ matched terms.+- `rank` is FTS5 rank — **lower is better** (closer to query terms).++### 4.6 Recent++```http+GET /api/v1/workspaces/{workspace}/projects/{project}/recent?limit=20+```++`is_latest = 1` pages ordered by `updated_at` DESC. `limit` clamped+`1..=100`, default `10`.++**Response:** `{ "pages": [BriefingPage, …] }`++```json+{+ "pages": [+ {+ "path": "sessions/2026-05-28.md",+ "title": "Session 2026-05-28",+ "kind": "fact",+ "updated_at": "2026-05-28T14:02:11.123Z"+ }+ ]+}+```++### 4.7 Briefing (structured snapshot)++```http+GET /api/v1/workspaces/{workspace}/projects/{project}/briefing?limit=10+```++Same payload `memory_briefing` returns — counts + activity windows ++last-observation + open handoffs + `_rules/` + `_slots/` + N most-recent+pages. No LLM, deterministic.++**Response:** `BriefingSnapshot`++```json+{+ "counts": {+ "pages_latest": 138,+ "pages_all": 162,+ "sessions": 27,+ "observations": 4198+ },+ "activity_7d": { "days": 7, "sessions": 6, "observations": 921, "pages_updated": 41 },+ "activity_30d": { "days": 30, "sessions": 24, "observations": 3712, "pages_updated": 102 },+ "last_observation_at": "2026-05-28T13:58:02.123Z",+ "pending_handoff_count": 0,+ "rules": [{ "path": "_rules/postgres.md", "title": "Postgres only", "kind": "rule", "updated_at": "…" }],+ "slots": [{ "path": "_slots/focus.md", "title": "Current focus", "kind": "fact", "updated_at": "…" }],+ "recent_pages": [+ { "path": "sessions/2026-05-28.md", "title": "Session 2026-05-28", "kind": "fact", "updated_at": "…" }+ ]+}+```++### 4.8 Overview (workspace + project aggregates)++```http+GET /api/v1/workspaces/{workspace}/overview?limit=10+GET /api/v1/workspaces/{workspace}/projects/{project}/overview?limit=10+```++Bundles what a frontend usually needs on its home view in one round-trip.++**Workspace overview** returns `briefing` + `memory_health` aggregated+across all projects in the workspace:++```json+{+ "briefing": { "counts": { … }, "activity_7d": { … }, "rules": [ … ], "recent_pages": [ … ] },+ "memory_health": { "stale_count": 4, "duplicate_count": 1, "orphan_count": 12,+ "stale_pages": [HealthPage, …], "duplicate_pages": [ … ], "orphan_pages": [ … ] }+}+```++**Project overview** additionally includes the latest open handoff (or+`null`):++```json+{+ "handoff": { "id": "01928d…", "from_agent": "claude-code", "summary": "…", "open_questions": [ … ], "next_steps": [ … ] },+ "briefing": { … },+ "memory_health": { … }+}+```++`HealthPage`:++```json+{+ "workspace": "default",+ "project": "ai-memory",+ "path": "concepts/old-thing.md",+ "title": "Old thing",+ "kind": "fact"+}+```++> Note: `last_open_handoff` is **not** consumed by the read API — the+> handoff stays "open" and can still be accepted by the next agent.++## 5. Limits and pagination++- All `limit` query params clamp to `1..=100`.+- `POST /api/v1/search`: at most **25 scopes** per request.+- HTTP body cap: **10 MB** (shared with the MCP body limit; you won't+ hit this for normal API traffic).+- **No `Cache-Control` / ETag headers yet.** A polling SPA hits the DB+ on every request. If your UI polls aggressively, consider client-side+ debouncing; server-side caching is a planned iteration.++## 6. Custom UI hosting (`--web-ui-dir`)++```bash+ai-memory serve \+ --transport http \+ --bind 127.0.0.1:49374 \+ --enable-web \+ --web-ui-dir /path/to/your-spa/dist+```++The static directory is served at `/web` via `tower-http::ServeDir`:++- **Same auth as `/api/v1`.** Mounted before the bearer middleware+ layer, so `/web/*` requests must carry the same `Authorization`+ header (browsers typically prompt via HTTP Basic when auth is on —+ the user pastes the token as the password).+- **SPA fallback.** Missing paths fall back to `index.html`, so a+ client-side router (React Router, SvelteKit, etc.) can own+ `/web/whatever` without 404s.+- **Path traversal is rejected** by `ServeDir`'s default safety.+- **Pre-startup validation:** the directory must exist *and* contain+ `index.html`, or `ai-memory serve` exits with a clear error before+ binding. Requires `--enable-web` to also be set.++When `--web-ui-dir` is **absent**, the built-in server-side `/web`+browser is the default (read-only HTML rendering, FTS5 search,+project tree). No regression.++## 7. Worked example: minimal SPA fetch++```js+// Resolve the API base from the SPA's own origin (same-origin model).+const API = `${location.origin}/api/v1`;+const TOKEN = localStorage.getItem("ai-memory-token"); // your storage choice++async function apiGet(path, params) {+ const url = new URL(`${API}${path}`, location.origin);+ if (params) Object.entries(params).forEach(([k, v]) =>+ v != null && url.searchParams.set(k, v));+ const resp = await fetch(url, {+ headers: {+ Accept: "application/json",+ ...(TOKEN ? { Authorization: `Bearer ${TOKEN}` } : {}),+ },+ });+ if (!resp.ok) {+ const { error } = await resp.json().catch(() => ({ error: resp.statusText }));+ throw new Error(`${resp.status}: ${error}`);+ }+ return resp.json();+}++// Top-level "home" view in one request.+const overview = await apiGet("/workspaces/default/projects/ai-memory/overview", { limit: 10 });+console.log(overview.briefing.counts.pages_latest, "pages");++// Multi-scope search.+const search = await fetch(`${API}/search`, {+ method: "POST",+ headers: {+ "Content-Type": "application/json",+ Authorization: `Bearer ${TOKEN}`,+ },+ body: JSON.stringify({+ q: "karpathy",+ scopes: [+ { workspace: "default", project: "ai-memory" },+ { workspace: "default", project: "shared-notes" },+ ],+ limit: 20,+ }),+}).then(r => r.json());+```++`curl` smoke test:++```bash+TOKEN=$(ai-memory generate-auth-token)+curl -fsS "http://127.0.0.1:49374/api/v1/workspaces" -H "Authorization: Bearer $TOKEN" | jq+```++## 8. Where to look in source (the canonical spec)++If a doc/response shape ever conflicts with the code, the code wins.+Read these:++| | Location |+|---|---|+| Route registration + handler bodies | `crates/ai-memory-web/src/routes/api.rs` |+| Response structs (`PageHit`, `WorkspaceSummary`, `BriefingSnapshot`, `HealthPage`, …) | `crates/ai-memory-store/src/reader.rs` |+| 27 integration tests covering every endpoint (auth, 400s, 404s, multi-scope correctness, SPA fallback) | `crates/ai-memory-web/tests/routes.rs` |+| Auth + middleware layering | `crates/ai-memory-cli/src/commands/serve.rs` (`mount_web_router`, `apply_http_layers`) |+| Custom-UI dir validation | `crates/ai-memory-cli/src/commands/serve.rs` (`validate_web_ui_args`) |++## 9. Known gaps (planned iterations, not blockers)++- **`Cache-Control` + ETags** on idempotent reads. Right now every+ `/api/v1/overview` poll hits the DB. Adding `Cache-Control: private,+ max-age=N` + ETags is a quick win once polling patterns are observed.+- **CORS.** No `CorsLayer` is mounted, so the API is reachable only+ same-origin or via a reverse proxy that adds the headers. A future+ `--cors-allow-origin` flag would let separately-hosted frontends talk+ to a remote `ai-memory` server directly.+- **Write surface.** Browsers can't mutate today (notes, consolidate,+ lint, purge — all live under `/admin/*` for the CLI or under MCP+ tools for agents). A thin authenticated write surface ("edit this+ page" from the browser) is a deliberate v2 conversation.+- **Rate limiting** is shared with `/mcp` + `/admin` (only the body+ cap is enforced today). A future global limiter would tighten the+ authenticated-misbehaviour case.++For status updates on any of these, the issue tracker is the source of+truth.