Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
docs(admission): integration guide for the webhook chain
Djalma Júnior909ef337d1c2
diff --git a/docs/admission-webhooks.md b/docs/admission-webhooks.mdnew file mode 100644index 00000000..20630d0f--- /dev/null+++ b/docs/admission-webhooks.md@@ -0,0 +1,272 @@+# Admission webhooks: pre-persistence HTTP hooks++> Operator-configured HTTP hooks invoked inside `Wiki::write_page`+> just before the page hits disk. Each hook can mutate the+> page (return a new frontmatter / body) or just observe and side-effect+> (return `204 No Content`). Sourced from+> `crates/ai-memory-wiki/src/admission.rs` and the wiring in+> `crates/ai-memory-cli/src/commands/serve.rs` — keep both as the+> canonical reference if anything here drifts.++## 1. What this is (and isn't)++| | What you can do | What you can't do |+|---|---|---|+| Chain | Add canonical frontmatter fields (e.g. `contributors`); mirror the write into an external system (git, search index, audit log); reject writes that fail policy (e.g. `validate-no-secrets`). | Talk back to the engine's writer or store directly — the chain only sees one page at a time and mutates it in place. |+| Engine | Stays closed for modification: every new behaviour ships as an independent HTTP service in any language. | The engine doesn't discover or auto-register webhooks — operators name them in config. |++If your idea doesn't fit "mutate the page or observe the write", it's+probably a different extension point (`/hook` ingress, `/admin/*` admin+surface, or out-of-band scheduled job).++## 2. Lifecycle++The chain fires inside `Wiki::write_page`, **after** the markdown is+parsed and sanitised but **before** the atomic write and store upsert.+A mutation applied by a webhook propagates to both the on-disk markdown+file and the SQLite row in a single atomic step (see+[ARCHITECTURE.md](ARCHITECTURE.md) on the writer actor / atomic write+invariants).++Webhooks run **sequentially**, in the order declared in config. Each one+sees the (possibly mutated) page from the previous webhook — so a+`contributors` hook that adds frontmatter is followed by a `git-mirror`+hook that mirrors the enriched page.++Webhooks fire on these `op` values today (extensible enum):++- `write_page` — direct writes via MCP `memory_write_page`, the CLI+ `write-page`, `/admin/write-page`, the lint rewriter, hook synthesis.+- `consolidate` — LLM consolidation writes from the consolidator+ (SessionEnd opt-in + PreCompact + manual `memory_consolidate`).++Each webhook opts into the ops it cares about via `events`. The chain+checks the op against `WebhookConfig::events` before dispatching.++## 3. Wire contract++### Request (engine → webhook)++```http+POST <webhook.url>+Content-Type: application/json+X-Memory-Op: write_page | consolidate+```++```jsonc+{+ "page": {+ "path": "gotchas/example.md", // relative wiki path (PagePath)+ "frontmatter": { "title": "...", ... }, // arbitrary JSON, may be null+ "body": "..." // markdown body, no frontmatter block+ },+ "ctx": {+ "workspace": "default", // resolved name (see §5)+ "project": "ai-memory-ops", // resolved name+ "actor": { // request-layer identity+ "agent": "claude-code", // claude-code | codex | opencode | hook | cli | …+ "user": "djalmajr", // null when unauthenticated+ "sub": "8f3a-...", // JWT sub+ "client": "72836f52-...", // DCR client UUID+ "session_id": "019e6d-..."+ },+ "op": "write_page" // write_page | consolidate+ }+}+```++The `WebhookRequestBody` / `WebhookPagePayload` / `ActorContext` /+`AdmissionContext` types in+`crates/ai-memory-wiki/src/admission.rs` are the authoritative+serialisation source.++### Response (webhook → engine)++| Status | Body | Behaviour |+|---|---|---|+| `200 OK` | `{ "page": { "frontmatter": ..., "body": ... } }` — both inner fields optional. Anything missing means "leave that field unchanged". | The engine swaps in the returned values before the next webhook (or the final atomic write). |+| `204 No Content` | (empty) | The engine treats the webhook as a pure observer / side-effect — no mutation, no parse. |+| `4xx` / `5xx` | (optional textual body, logged) | See **§4 Failure policy**. |++The engine bounds the response read at `MAX_RESPONSE_BYTES` (1 MiB).+Anything beyond that is treated as a no-op with a `warn` log — webhooks+have no legitimate reason to return more than the page envelope.++## 4. Failure policy++Each webhook picks one when the engine can't reach it or it returns+non-2xx:++- **`ignore` (default, recommended)** — Engine logs a `warn` and+ continues with the unmutated page. The page write still succeeds.+ This is the right choice for everything except safety-critical+ enforcers.+- **`reject`** — Engine aborts the write, propagating the error up to+ the caller. Use this **only** when the webhook is a hard precondition+ for persistence (e.g. a future `validate-no-secrets` enforcer).++A webhook that subscribes to multiple ops uses the same policy across+all of them.++## 5. Workspace / project names++The engine resolves `workspace_id` and `project_id` into the same+human-readable names the UI and on-disk wiki use, so webhooks can+address pages by name without re-implementing UUID lookup. Resolution+happens just before the chain fires; both fields are empty when the+wiki was built without [`Wiki::with_store_reader`] (e.g. legacy+embedders, tests that wire a chain without a reader).++External webhooks should treat the names as opaque strings (workspace /+project values use the same validation as `--workspace` / `--project`+CLI flags). They are stable for the lifetime of the workspace / project+— the engine doesn't rename them silently. `rename-project` is a manual+op that an operator runs and that will eventually trigger a fresh+webhook fan-out if you wire one.++## 6. Loop prevention++A webhook that turns around and writes back to the engine (e.g. via+`/admin/write-page` or `memory_write_page`) must include the header++```+X-Memory-Skip-Admission-Chain: <name>[,<name>...]+```++on its re-entrant call. The engine matches the CSV against+`WebhookConfig::name` and short-circuits those hooks for that write.+Without this, you get infinite recursion (engine → webhook → engine →+webhook → ...). The header propagates only for the single re-entrant+write; the next external write picks the chain back up normally.++## 7. Limits++Constants are exported from the `ai-memory-wiki` crate root:++| Constant | Value | What it caps |+|---|---|---|+| `MAX_ADMISSION_WEBHOOKS` | `16` | Chain length. `AdmissionChain::new` errors out beyond this — a misconfigured template (helm loop, duplicated block) can't push N hooks into the write-path. |+| `MAX_RESPONSE_BYTES` | `1 MiB` | Webhook response body. Beyond this the response is dropped (treated as no-op + `warn`). |+| Per-webhook `timeout_ms` | operator-set (default `2000`) | Single request. The chain is sequential, so total worst case ≈ `Σ timeout_ms`. |++## 8. Configuration++`config.toml`:++```toml+[[admission_webhooks]]+name = "contributors" # stable identifier (used by skip list + logs)+url = "http://contributors.memory.svc.cluster.local:8080/enrich"+timeout_ms = 2000 # per request+failure_policy = "ignore" # ignore | reject+events = ["write_page", "consolidate"]++[[admission_webhooks]]+name = "git-mirror"+url = "http://git-mirror.memory.svc.cluster.local:8080/sync"+timeout_ms = 2000+failure_policy = "ignore"+events = ["write_page", "consolidate"]+```++Env override (figment env layer):++```bash+AI_MEMORY_ADMISSION_WEBHOOKS__0__NAME=contributors+AI_MEMORY_ADMISSION_WEBHOOKS__0__URL=http://contributors.memory.svc.cluster.local:8080/enrich+AI_MEMORY_ADMISSION_WEBHOOKS__0__TIMEOUT_MS=2000+AI_MEMORY_ADMISSION_WEBHOOKS__0__FAILURE_POLICY=ignore+AI_MEMORY_ADMISSION_WEBHOOKS__0__EVENTS=write_page,consolidate+```++Empty config = no chain attached → zero overhead per write (no client+built, no per-write branch).++## 9. Worked examples++### Mutating: append the writer to `frontmatter.contributors`++```jsonc+// POST /enrich+{+ "page": { "path": "gotchas/x.md", "frontmatter": { "title": "X" }, "body": "..." },+ "ctx": { "workspace": "default", "project": "ai-memory-ops",+ "actor": { "agent": "claude-code", "user": "djalmajr", "client": "72836f52-..." }, ... }+}++// → 200 OK+{+ "page": {+ "frontmatter": {+ "title": "X",+ "contributors": [+ { "agent": "claude-code", "user": "djalmajr", "client": "72836f52-...",+ "first_seen": "...", "last_seen": "...", "writes": 1 }+ ]+ }+ }+}+```++The engine replaces `frontmatter` with the returned object before+persisting. `body` is left untouched (not in the response).++### Side-effect: mirror the write into an external git repo++```jsonc+// POST /sync (same request body)+// → 204 No Content+```++The webhook materialises the page into a local clone of an external+repo, batches commits, and pushes asynchronously. The engine doesn't+wait for the push — only the local enqueue runs inside the write path+under the webhook's `timeout_ms`.++## 10. Tests++`crates/ai-memory-wiki/tests/admission.rs` covers the wire contract end+to end against an axum loopback server. Categories:++- Mutating frontmatter and body propagates correctly.+- `204` is a no-op (frontmatter / body unchanged).+- `failure_policy=ignore` swallows errors; `failure_policy=reject`+ aborts.+- Multi-webhook chain runs in declared order; each sees the previous+ mutation.+- `X-Memory-Skip-Admission-Chain` short-circuits named hooks.+- `X-Memory-Op` header is set correctly per op.+- `op`-filtering: webhooks only fire on subscribed events.+- `MAX_ADMISSION_WEBHOOKS` rejection at construct time.+- `MAX_RESPONSE_BYTES` cap drops oversized responses.+- `workspace` / `project` resolution propagates into the payload.++`crates/ai-memory-wiki/src/wiki.rs::tests::write_page_resolves_workspace_and_project_names_for_chain`+covers the integrated path (`Wiki::write_page` → store reader resolution+→ chain → recorded payload).++## 11. Where to look in source (the canonical spec)++| Concept | File:line |+|---|---|+| `AdmissionContext` / `ActorContext` / wire structs | `crates/ai-memory-wiki/src/admission.rs` |+| `AdmissionChain::run` (the hot loop) | `crates/ai-memory-wiki/src/admission.rs` |+| Invocation inside `write_page` (resolution + chain call) | `crates/ai-memory-wiki/src/wiki.rs::Wiki::write_page` |+| Config schema (`[[admission_webhooks]]`) | `crates/ai-memory-cli/src/config.rs::Config::admission_webhooks` |+| Server wiring (`with_admission_chain` + `with_store_reader`) | `crates/ai-memory-cli/src/commands/serve.rs` |+| Header → `ActorContext` mapping (mcp-auth → engine) | `crates/ai-memory-mcp/src/actor.rs` |++## 12. Non-goals (planned iterations, not blockers)++- **Parallel fan-out.** The chain is sequential by design today —+ mutation composition is well-defined that way. A future+ `parallel = true` for side-effect-only webhooks (no body mutation+ expected) is possible but not in scope.+- **Webhook discovery / dynamic registration.** Hooks are operator-named+ in config. A future `/admin/admission-webhooks` POST surface is+ conceivable but explicitly out of scope here — single-tenant config+ is simpler and matches how the engine treats every other extension+ point (`[[etl_sources]]`, etc.).+- **Per-webhook metrics surface.** The chain logs via `tracing` today.+ Surfacing per-webhook counters via `/admin/status` is a natural+ follow-up but lives outside this contract.