Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

docs(admission): integration guide for the webhook chain

Djalma Júnior909ef337d1c2
diff --git a/docs/admission-webhooks.md b/docs/admission-webhooks.md
new file mode 100644
index 00000000..20630d0f
--- /dev/null
+++ b/docs/admission-webhooks.md
@@ -0,0 +1,272 @@
+# Admission webhooks: pre-persistence HTTP hooks
+
+> Operator-configured HTTP hooks invoked inside `Wiki::write_page`
+> just before the page hits disk. Each hook can mutate the
+> page (return a new frontmatter / body) or just observe and side-effect
+> (return `204 No Content`). Sourced from
+> `crates/ai-memory-wiki/src/admission.rs` and the wiring in
+> `crates/ai-memory-cli/src/commands/serve.rs` — keep both as the
+> canonical reference if anything here drifts.
+
+## 1. What this is (and isn't)
+
+| | What you can do | What you can't do |
+|---|---|---|
+| Chain | Add canonical frontmatter fields (e.g. `contributors`); mirror the write into an external system (git, search index, audit log); reject writes that fail policy (e.g. `validate-no-secrets`). | Talk back to the engine's writer or store directly — the chain only sees one page at a time and mutates it in place. |
+| Engine | Stays closed for modification: every new behaviour ships as an independent HTTP service in any language. | The engine doesn't discover or auto-register webhooks — operators name them in config. |
+
+If your idea doesn't fit "mutate the page or observe the write", it's
+probably a different extension point (`/hook` ingress, `/admin/*` admin
+surface, or out-of-band scheduled job).
+
+## 2. Lifecycle
+
+The chain fires inside `Wiki::write_page`, **after** the markdown is
+parsed and sanitised but **before** the atomic write and store upsert.
+A mutation applied by a webhook propagates to both the on-disk markdown
+file and the SQLite row in a single atomic step (see
+[ARCHITECTURE.md](ARCHITECTURE.md) on the writer actor / atomic write
+invariants).
+
+Webhooks run **sequentially**, in the order declared in config. Each one
+sees the (possibly mutated) page from the previous webhook — so a
+`contributors` hook that adds frontmatter is followed by a `git-mirror`
+hook that mirrors the enriched page.
+
+Webhooks fire on these `op` values today (extensible enum):
+
+- `write_page` — direct writes via MCP `memory_write_page`, the CLI
+ `write-page`, `/admin/write-page`, the lint rewriter, hook synthesis.
+- `consolidate` — LLM consolidation writes from the consolidator
+ (SessionEnd opt-in + PreCompact + manual `memory_consolidate`).
+
+Each webhook opts into the ops it cares about via `events`. The chain
+checks the op against `WebhookConfig::events` before dispatching.
+
+## 3. Wire contract
+
+### Request (engine → webhook)
+
+```http
+POST <webhook.url>
+Content-Type: application/json
+X-Memory-Op: write_page | consolidate
+```
+
+```jsonc
+{
+ "page": {
+ "path": "gotchas/example.md", // relative wiki path (PagePath)
+ "frontmatter": { "title": "...", ... }, // arbitrary JSON, may be null
+ "body": "..." // markdown body, no frontmatter block
+ },
+ "ctx": {
+ "workspace": "default", // resolved name (see §5)
+ "project": "ai-memory-ops", // resolved name
+ "actor": { // request-layer identity
+ "agent": "claude-code", // claude-code | codex | opencode | hook | cli | …
+ "user": "djalmajr", // null when unauthenticated
+ "sub": "8f3a-...", // JWT sub
+ "client": "72836f52-...", // DCR client UUID
+ "session_id": "019e6d-..."
+ },
+ "op": "write_page" // write_page | consolidate
+ }
+}
+```
+
+The `WebhookRequestBody` / `WebhookPagePayload` / `ActorContext` /
+`AdmissionContext` types in
+`crates/ai-memory-wiki/src/admission.rs` are the authoritative
+serialisation source.
+
+### Response (webhook → engine)
+
+| Status | Body | Behaviour |
+|---|---|---|
+| `200 OK` | `{ "page": { "frontmatter": ..., "body": ... } }` — both inner fields optional. Anything missing means "leave that field unchanged". | The engine swaps in the returned values before the next webhook (or the final atomic write). |
+| `204 No Content` | (empty) | The engine treats the webhook as a pure observer / side-effect — no mutation, no parse. |
+| `4xx` / `5xx` | (optional textual body, logged) | See **§4 Failure policy**. |
+
+The engine bounds the response read at `MAX_RESPONSE_BYTES` (1 MiB).
+Anything beyond that is treated as a no-op with a `warn` log — webhooks
+have no legitimate reason to return more than the page envelope.
+
+## 4. Failure policy
+
+Each webhook picks one when the engine can't reach it or it returns
+non-2xx:
+
+- **`ignore` (default, recommended)** — Engine logs a `warn` and
+ continues with the unmutated page. The page write still succeeds.
+ This is the right choice for everything except safety-critical
+ enforcers.
+- **`reject`** — Engine aborts the write, propagating the error up to
+ the caller. Use this **only** when the webhook is a hard precondition
+ for persistence (e.g. a future `validate-no-secrets` enforcer).
+
+A webhook that subscribes to multiple ops uses the same policy across
+all of them.
+
+## 5. Workspace / project names
+
+The engine resolves `workspace_id` and `project_id` into the same
+human-readable names the UI and on-disk wiki use, so webhooks can
+address pages by name without re-implementing UUID lookup. Resolution
+happens just before the chain fires; both fields are empty when the
+wiki was built without [`Wiki::with_store_reader`] (e.g. legacy
+embedders, tests that wire a chain without a reader).
+
+External webhooks should treat the names as opaque strings (workspace /
+project values use the same validation as `--workspace` / `--project`
+CLI flags). They are stable for the lifetime of the workspace / project
+— the engine doesn't rename them silently. `rename-project` is a manual
+op that an operator runs and that will eventually trigger a fresh
+webhook fan-out if you wire one.
+
+## 6. Loop prevention
+
+A webhook that turns around and writes back to the engine (e.g. via
+`/admin/write-page` or `memory_write_page`) must include the header
+
+```
+X-Memory-Skip-Admission-Chain: <name>[,<name>...]
+```
+
+on its re-entrant call. The engine matches the CSV against
+`WebhookConfig::name` and short-circuits those hooks for that write.
+Without this, you get infinite recursion (engine → webhook → engine →
+webhook → ...). The header propagates only for the single re-entrant
+write; the next external write picks the chain back up normally.
+
+## 7. Limits
+
+Constants are exported from the `ai-memory-wiki` crate root:
+
+| Constant | Value | What it caps |
+|---|---|---|
+| `MAX_ADMISSION_WEBHOOKS` | `16` | Chain length. `AdmissionChain::new` errors out beyond this — a misconfigured template (helm loop, duplicated block) can't push N hooks into the write-path. |
+| `MAX_RESPONSE_BYTES` | `1 MiB` | Webhook response body. Beyond this the response is dropped (treated as no-op + `warn`). |
+| Per-webhook `timeout_ms` | operator-set (default `2000`) | Single request. The chain is sequential, so total worst case ≈ `Σ timeout_ms`. |
+
+## 8. Configuration
+
+`config.toml`:
+
+```toml
+[[admission_webhooks]]
+name = "contributors" # stable identifier (used by skip list + logs)
+url = "http://contributors.memory.svc.cluster.local:8080/enrich"
+timeout_ms = 2000 # per request
+failure_policy = "ignore" # ignore | reject
+events = ["write_page", "consolidate"]
+
+[[admission_webhooks]]
+name = "git-mirror"
+url = "http://git-mirror.memory.svc.cluster.local:8080/sync"
+timeout_ms = 2000
+failure_policy = "ignore"
+events = ["write_page", "consolidate"]
+```
+
+Env override (figment env layer):
+
+```bash
+AI_MEMORY_ADMISSION_WEBHOOKS__0__NAME=contributors
+AI_MEMORY_ADMISSION_WEBHOOKS__0__URL=http://contributors.memory.svc.cluster.local:8080/enrich
+AI_MEMORY_ADMISSION_WEBHOOKS__0__TIMEOUT_MS=2000
+AI_MEMORY_ADMISSION_WEBHOOKS__0__FAILURE_POLICY=ignore
+AI_MEMORY_ADMISSION_WEBHOOKS__0__EVENTS=write_page,consolidate
+```
+
+Empty config = no chain attached → zero overhead per write (no client
+built, no per-write branch).
+
+## 9. Worked examples
+
+### Mutating: append the writer to `frontmatter.contributors`
+
+```jsonc
+// POST /enrich
+{
+ "page": { "path": "gotchas/x.md", "frontmatter": { "title": "X" }, "body": "..." },
+ "ctx": { "workspace": "default", "project": "ai-memory-ops",
+ "actor": { "agent": "claude-code", "user": "djalmajr", "client": "72836f52-..." }, ... }
+}
+
+// → 200 OK
+{
+ "page": {
+ "frontmatter": {
+ "title": "X",
+ "contributors": [
+ { "agent": "claude-code", "user": "djalmajr", "client": "72836f52-...",
+ "first_seen": "...", "last_seen": "...", "writes": 1 }
+ ]
+ }
+ }
+}
+```
+
+The engine replaces `frontmatter` with the returned object before
+persisting. `body` is left untouched (not in the response).
+
+### Side-effect: mirror the write into an external git repo
+
+```jsonc
+// POST /sync (same request body)
+// → 204 No Content
+```
+
+The webhook materialises the page into a local clone of an external
+repo, batches commits, and pushes asynchronously. The engine doesn't
+wait for the push — only the local enqueue runs inside the write path
+under the webhook's `timeout_ms`.
+
+## 10. Tests
+
+`crates/ai-memory-wiki/tests/admission.rs` covers the wire contract end
+to end against an axum loopback server. Categories:
+
+- Mutating frontmatter and body propagates correctly.
+- `204` is a no-op (frontmatter / body unchanged).
+- `failure_policy=ignore` swallows errors; `failure_policy=reject`
+ aborts.
+- Multi-webhook chain runs in declared order; each sees the previous
+ mutation.
+- `X-Memory-Skip-Admission-Chain` short-circuits named hooks.
+- `X-Memory-Op` header is set correctly per op.
+- `op`-filtering: webhooks only fire on subscribed events.
+- `MAX_ADMISSION_WEBHOOKS` rejection at construct time.
+- `MAX_RESPONSE_BYTES` cap drops oversized responses.
+- `workspace` / `project` resolution propagates into the payload.
+
+`crates/ai-memory-wiki/src/wiki.rs::tests::write_page_resolves_workspace_and_project_names_for_chain`
+covers the integrated path (`Wiki::write_page` → store reader resolution
+→ chain → recorded payload).
+
+## 11. Where to look in source (the canonical spec)
+
+| Concept | File:line |
+|---|---|
+| `AdmissionContext` / `ActorContext` / wire structs | `crates/ai-memory-wiki/src/admission.rs` |
+| `AdmissionChain::run` (the hot loop) | `crates/ai-memory-wiki/src/admission.rs` |
+| Invocation inside `write_page` (resolution + chain call) | `crates/ai-memory-wiki/src/wiki.rs::Wiki::write_page` |
+| Config schema (`[[admission_webhooks]]`) | `crates/ai-memory-cli/src/config.rs::Config::admission_webhooks` |
+| Server wiring (`with_admission_chain` + `with_store_reader`) | `crates/ai-memory-cli/src/commands/serve.rs` |
+| Header → `ActorContext` mapping (mcp-auth → engine) | `crates/ai-memory-mcp/src/actor.rs` |
+
+## 12. Non-goals (planned iterations, not blockers)
+
+- **Parallel fan-out.** The chain is sequential by design today —
+ mutation composition is well-defined that way. A future
+ `parallel = true` for side-effect-only webhooks (no body mutation
+ expected) is possible but not in scope.
+- **Webhook discovery / dynamic registration.** Hooks are operator-named
+ in config. A future `/admin/admission-webhooks` POST surface is
+ conceivable but explicitly out of scope here — single-tenant config
+ is simpler and matches how the engine treats every other extension
+ point (`[[etl_sources]]`, etc.).
+- **Per-webhook metrics surface.** The chain logs via `tracing` today.
+ Surfacing per-webhook counters via `/admin/status` is a natural
+ follow-up but lives outside this contract.