Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

perf: bound hook ingestion

AkitaOnRails9ba0c1d195b5
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b1b44904..7dfeb220 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -35,6 +35,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
one SQL query instead of issuing incoming/outgoing lookups per seed.
- Embedding backfill stores embeddings in chunks instead of one writer
command and SQLite transaction per page.
+- Hook ingestion now bounds in-flight processing and returns HTTP 429 when
+ saturated instead of spawning unbounded background tasks.
- Documented the vector backend policy and the measured criteria required
before adding `sqlite-vec`.
- Clarified Gemini CLI support docs: MCP registration, lifecycle hooks,
diff --git a/CLAUDE.md b/CLAUDE.md
index 56a473de..2db466c6 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -83,7 +83,8 @@ blocking issue:**
3. Indexes commit in the same transaction as the data. No
background-task-indexing-after-return.
4. Typed `(WorkspaceId, ProjectId, PagePath)` identity in every layer.
-5. Hooks are fire-and-forget. Sub-second timeouts. Return 202 immediately.
+5. Hooks are fire-and-forget. Sub-second timeouts. Return 202 immediately,
+ or 429 when saturated.
6. Privacy strip is a typed boundary (`RawHookPayload → Sanitized<Observation>`).
7. JSON-schema structured outputs only. No XML, no `instructor`-style wrapping.
8. `{provider, model, dim}` stored next to every embedding. Refuse on mismatch.
diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rs
index 3bf06035..e3872c09 100644
--- a/crates/ai-memory-cli/src/commands/serve.rs
+++ b/crates/ai-memory-cli/src/commands/serve.rs
@@ -4,7 +4,7 @@ use std::sync::Arc;
use ai_memory_consolidate::{Consolidator, run_lint, run_sweep};
use ai_memory_core::{ActiveProject, ProjectId, Sanitizer, WorkspaceId};
-use ai_memory_hooks::{HookState, hook_router};
+use ai_memory_hooks::{DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT, HookState, hook_router};
use ai_memory_llm::{Embedder, LlmProvider, build_embedder, build_provider};
use ai_memory_mcp::{AdminState, AiMemoryServer, admin_router};
use ai_memory_store::{EmbeddingWrite, ReaderPool, Store, WriterHandle, f32_vec_to_bytes};
@@ -168,6 +168,9 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {
std::collections::HashMap::new(),
)),
active_project: active_project.clone(),
+ ingest_semaphore: std::sync::Arc::new(tokio::sync::Semaphore::new(
+ DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT,
+ )),
});
let admin = admin_router(AdminState {
writer: store.writer.clone(),
diff --git a/crates/ai-memory-hooks/src/lib.rs b/crates/ai-memory-hooks/src/lib.rs
index f7f7398a..1f9d959f 100644
--- a/crates/ai-memory-hooks/src/lib.rs
+++ b/crates/ai-memory-hooks/src/lib.rs
@@ -27,5 +27,5 @@ pub mod synth;
// pointing at this crate's `sanitize` module keep working.
pub use ai_memory_core::{SanitizeConfig, Sanitized, Sanitizer};
pub use payload::{HookEnvelope, HookEvent};
-pub use router::{HookState, hook_router};
+pub use router::{DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT, HookState, hook_router};
pub use synth::synthesize_session_page;
diff --git a/crates/ai-memory-hooks/src/router.rs b/crates/ai-memory-hooks/src/router.rs
index f1df717f..4e8e7cfa 100644
--- a/crates/ai-memory-hooks/src/router.rs
+++ b/crates/ai-memory-hooks/src/router.rs
@@ -1,11 +1,12 @@
//! axum router exposing `POST /hook`.
//!
-//! Always returns 202 immediately. Heavy work (DB writes, session-page
-//! synthesis) happens *after* the response is sent — but we still
-//! `await` the writer ack to honour the cross-cutting invariant that
-//! "indexes commit in the same transaction as the data" (no
-//! background-task-indexing-after-return, basic-memory #763). The agent
-//! never blocks on us thanks to the fire-and-forget client side.
+//! Returns 202 immediately unless the in-flight hook limit is saturated,
+//! in which case it returns 429. Heavy work (DB writes, session-page
+//! synthesis) happens *after* the response is sent — but we still `await`
+//! the writer ack to honour the cross-cutting invariant that "indexes commit
+//! in the same transaction as the data" (no background-task-indexing-after-return,
+//! basic-memory #763). The agent never blocks on us thanks to the
+//! fire-and-forget client side.
use std::collections::HashMap;
use std::str::FromStr;
@@ -33,6 +34,13 @@ use crate::log;
use crate::payload::{HookEnvelope, HookEvent, HookQuery, parse_agent};
use crate::synth::synthesize_session_page;
+/// Default maximum number of hook events allowed to be processing at once.
+///
+/// This matches the writer queue order of magnitude and prevents unbounded
+/// background tasks during tool-heavy bursts. Saturated servers return 429 so
+/// callers can drop or retry instead of growing memory without bound.
+pub const DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT: usize = 1024;
+
/// Shared state passed to the hook handler.
#[derive(Clone)]
pub struct HookState {
@@ -65,6 +73,9 @@ pub struct HookState {
/// of their own) default to the project the user is actually in
/// rather than the server's static `--project` (issue #2).
pub active_project: ActiveProject,
+ /// In-flight hook processing limiter. Requests acquire one permit before
+ /// spawning work and return 429 immediately when saturated.
+ pub ingest_semaphore: Arc<tokio::sync::Semaphore>,
}
/// Build a router with `POST /hook` (event ingress) and `GET /handoff`
@@ -82,7 +93,14 @@ async fn handle_hook(
Json(body): Json<serde_json::Value>,
) -> impl IntoResponse {
let env = HookEnvelope::from_query_and_body(query, body);
- tokio::spawn(process_envelope(state.clone(), env));
+ let Ok(permit) = state.ingest_semaphore.clone().try_acquire_owned() else {
+ warn!("hook ingest saturated; dropping event with 429");
+ return (StatusCode::TOO_MANY_REQUESTS, "hook queue full");
+ };
+ tokio::spawn(async move {
+ let _permit = permit;
+ process_envelope(state, env).await;
+ });
(StatusCode::ACCEPTED, "queued")
}
@@ -576,6 +594,9 @@ mod tests {
sanitizer,
project_cache: Arc::new(tokio::sync::Mutex::new(HashMap::new())),
active_project: ActiveProject::new(),
+ ingest_semaphore: Arc::new(tokio::sync::Semaphore::new(
+ DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT,
+ )),
}
}
@@ -607,6 +628,26 @@ mod tests {
assert_eq!(state.active_project.get(), Some((ws_b, proj_b)));
}
+ #[tokio::test]
+ async fn handle_hook_returns_429_when_ingest_saturated() {
+ let tmp = TempDir::new().unwrap();
+ let mut state = make_state(&tmp).await;
+ state.ingest_semaphore = Arc::new(tokio::sync::Semaphore::new(0));
+
+ let response = handle_hook(
+ State(Arc::new(state)),
+ Query(HookQuery {
+ event: "session-start".into(),
+ agent: Some("claude-code".into()),
+ }),
+ Json(serde_json::json!({})),
+ )
+ .await
+ .into_response();
+
+ assert_eq!(response.status(), StatusCode::TOO_MANY_REQUESTS);
+ }
+
/// An event without a cwd must fall back to the server defaults.
#[tokio::test]
async fn process_with_missing_cwd_falls_back_to_state_defaults() {
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index 81f8c036..fae05296 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -76,7 +76,8 @@ markdown stays the source of truth.
1. Agent CLI emits a lifecycle hook (SessionStart, UserPromptSubmit,
PostToolUse, …) → vendored shell script `curl`s the event JSON to
- `POST /hook` with a sub-second timeout. Agent never blocks.
+ `POST /hook` with a sub-second timeout. Agent never blocks; saturated
+ servers return HTTP 429 instead of queueing unbounded work.
2. Server's hook router sanitises the payload (the only path from
untrusted text into the store), assigns an [`ObservationKind`], and
enqueues a `WriteCmd` to the writer actor. `log.md` gets an
@@ -210,7 +211,8 @@ that touch the relevant area.
4. **Typed 3-tuple identity** (`workspace_id`, `project_id`, path)
in every domain row from day one. (basic-memory #783 / #834.)
5. **Hooks are fire-and-forget.** Hook scripts hard-timeout at
- ≤200 ms; server returns 202 immediately. (agentmemory #221 / #143.)
+ ≤200 ms; server returns 202 immediately or 429 when saturated.
+ (agentmemory #221 / #143.)
6. **Privacy strip is a typed boundary.** `Sanitized<NewObservation>`
has no other constructor than `sanitize()`. (design-decisions §14.)
7. **JSON-schema structured outputs only.** Native provider JSON
diff --git a/docs/design-decisions.md b/docs/design-decisions.md
index 44845a9e..c8707ed1 100644
--- a/docs/design-decisions.md
+++ b/docs/design-decisions.md
@@ -98,7 +98,8 @@ Three capture surfaces, in priority order:
1. **Lifecycle hooks/extensions** (Claude Code, Codex, OpenCode, OMP). These are fast, reliable, structured. We ship hook scripts or generated TypeScript integrations the user installs once. Lessons from agentmemory:
- Hooks must be **fire-and-forget** (#221). No `await fetch()` blocking session start.
- Sub-second hard timeouts on the writer side (`tokio::time::timeout`).
- - All hooks → single HTTP/Unix-socket POST → server queues → returns 202 immediately.
+ - All hooks → single HTTP/Unix-socket POST → server queues → returns 202
+ immediately, or 429 when saturated.
- Privacy strip at the hook boundary, not later (agentmemory `stripPrivateData`).
2. **Transcript tail** (universal fallback). Watch `~/.claude/projects/`, `~/.codex/`, `~/.config/opencode/sessions/`. Lossier but works for any agent. Required for the basic-memory #669/#687/#730 demand the tracker has been asking for.
@@ -182,7 +183,10 @@ Project resolution chain: explicit param → server's default → cwd-based heur
- **Single binary**, statically-linked where possible. Distroless Docker image. **Absolute data path** by default (`dirs::data_local_dir().join("ai-memory")`); log it loudly on startup (agentmemory #303 lesson).
- **Atomic config**: one `Config::load()` → typed struct, every reader takes `&Config`. No `process.env` double-read paths (agentmemory #456/#469).
-- **Write durability**: every observation lands in SQLite *and* is appended to a `log.md` line *before* the hook gets its 202. No background-task indexing-after-return (basic-memory #763/#578/#839).
+- **Write durability**: accepted hook work awaits the SQLite write and appends a
+ `log.md` line before that background task finishes. Indexes still commit in
+ the same transaction as the data; no detached indexing task runs after the
+ write ack (basic-memory #763/#578/#839).
- **Migrations**: `sqlx::migrate!` runs on startup; never inline DDL (basic-memory #727).
- **Schema versioning**: one source of truth for the schema; derived clients/docs. No "update 7 files" checklists (agentmemory AGENTS.md smell).
- **Backup/move**: `ai-memory export <dir>` dumps wiki/ + sqlite snapshot. `ai-memory import <dir>` consumes. Default data dir is portable. Optional: `auto_git_commit = true` config flag → commits the wiki directory on every `memory_lint` run.
diff --git a/docs/prior-art-implementation-findings.md b/docs/prior-art-implementation-findings.md
index 419734fb..cb7f8ced 100644
--- a/docs/prior-art-implementation-findings.md
+++ b/docs/prior-art-implementation-findings.md
@@ -72,11 +72,12 @@ documented in basic-memory's tracker.
### Capture And Handoff
-The hook router accepts lifecycle events over HTTP, returns `202` fast,
-and does the writer work outside the agent's critical path. `SessionEnd`
-creates a deterministic session page, ends the session, opens a typed
-handoff, and commits the wiki. `PreCompact` checkpoints state, using the
-LLM consolidator if configured and deterministic synthesis otherwise.
+The hook router accepts lifecycle events over HTTP, returns `202` fast
+unless saturated (`429`), and does the writer work outside the agent's
+critical path. `SessionEnd` creates a deterministic session page, ends the
+session, opens a typed handoff, and commits the wiki. `PreCompact`
+checkpoints state, using the LLM consolidator if configured and deterministic
+synthesis otherwise.
The explicit `memory_handoff_begin` / `memory_handoff_accept` model is a
cleaner version of agentmemory's informal handoff behavior.