Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

feat(hooks): opt-in LLM consolidation on SessionEnd + align docs (#40)

AkitaOnRails188bc074b608
diff --git a/CLAUDE.md b/CLAUDE.md
index 5a186687..434861ef 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -234,10 +234,13 @@ Live progress is tracked via the TaskList tool inside Claude Code sessions.
This project uses [ai-memory](https://github.com/akitaonrails/ai-memory)
for cross-session continuity. **Lifecycle hooks already capture every
prompt + tool call automatically.** You never need to manually write
-routine notes; the SessionStart hook auto-fetches pending handoffs and
-the SessionEnd hook auto-consolidates. Only write a durable wiki page
-when the user explicitly asks to remember or annotate something
-permanently.
+routine notes; the SessionStart hook auto-fetches pending handoffs, and
+on session end ai-memory writes a session-summary page and a handoff.
+LLM consolidation (compiling observations into topical wiki pages) runs
+on PreCompact, on demand via `memory_consolidate`, and at session end
+only when the server sets `AI_MEMORY_CONSOLIDATE_ON_SESSION_END`. Only
+write a durable wiki page when the user explicitly asks to remember or
+annotate something permanently.
### When to reach for each tool
@@ -254,7 +257,7 @@ match the intent to the tool. They do not need to name the tool.
| "where did we leave off?" — and you see a `📥 ai-memory: pending handoff` block in your context | already done — answer from that block; do NOT re-call `memory_handoff_accept` |
| "where did we leave off?" — and no such block is visible | `memory_handoff_accept` (rare; the SessionStart hook usually got there first) |
| "save context for the next session" / wrapping up | `memory_handoff_begin` (single-use handoff; terse summary; put detail in `open_questions` + `next_steps` bullets) |
-| "consolidate this session" / "compile what we learned" (usually automatic) | `memory_consolidate` |
+| "consolidate this session" / "compile what we learned" (also runs on PreCompact; at session end only if `AI_MEMORY_CONSOLIDATE_ON_SESSION_END` is set) | `memory_consolidate` |
| "remember this permanently" / "save a note" / "add an annotation" / durable project knowledge | `memory_write_page` (write a wiki page; do **not** use handoff for permanent notes) |
| "audit the wiki" / "find contradictions" / "what rules should we add?" | `memory_lint` |
| "prune old pages" / "memory cleanup" | `memory_forget_sweep` |
diff --git a/README.md b/README.md
index 19ea0a05..a9079f96 100644
--- a/README.md
+++ b/README.md
@@ -306,7 +306,10 @@ inspection commands. CLI URL/auth configuration lives in
ai-memory runs without an LLM: hooks still capture sessions, search uses
FTS5, and summaries fall back to rule-based output. Add an LLM provider
-when you want session-end consolidation, richer linting, and bootstrap.
+when you want LLM consolidation (on PreCompact, on demand via
+`memory_consolidate`, or opt-in at session end with
+`AI_MEMORY_CONSOLIDATE_ON_SESSION_END`), richer linting, and bootstrap.
+Session end always writes a rule-based summary page + handoff either way.
Recommended defaults:
diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rs
index 9235ceeb..610e3f43 100644
--- a/crates/ai-memory-cli/src/commands/serve.rs
+++ b/crates/ai-memory-cli/src/commands/serve.rs
@@ -175,6 +175,7 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {
ingest_semaphore: std::sync::Arc::new(tokio::sync::Semaphore::new(
DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT,
)),
+ consolidate_on_session_end: config.consolidate_on_session_end,
});
let admin = admin_router(AdminState {
writer: store.writer.clone(),
diff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rs
index c5118f48..da339c79 100644
--- a/crates/ai-memory-cli/src/config.rs
+++ b/crates/ai-memory-cli/src/config.rs
@@ -59,6 +59,13 @@ pub struct Config {
pub llm_model: Option<String>,
/// Optional LLM base URL override.
pub llm_base_url: Option<String>,
+ /// Opt-in: run LLM consolidation on SessionEnd (in addition to the
+ /// always-written heuristic session page), when an LLM provider is
+ /// configured. Off by default — SessionEnd stays cheap and
+ /// fire-and-forget; the LLM checkpoint otherwise happens on PreCompact
+ /// and via manual `memory_consolidate`. Set with
+ /// `AI_MEMORY_CONSOLIDATE_ON_SESSION_END=true`.
+ pub consolidate_on_session_end: bool,
/// Optional embedding provider (`openai`, `voyage`, `google` / `gemini`).
pub embedding_provider: Option<String>,
/// Optional embedding model override.
@@ -215,6 +222,7 @@ impl Default for Config {
llm_provider: None,
llm_model: None,
llm_base_url: None,
+ consolidate_on_session_end: false,
embedding_provider: None,
embedding_model: None,
embedding_dim: None,
diff --git a/crates/ai-memory-core/src/routing_snippet.rs b/crates/ai-memory-core/src/routing_snippet.rs
index 1a32919d..0a291838 100644
--- a/crates/ai-memory-core/src/routing_snippet.rs
+++ b/crates/ai-memory-core/src/routing_snippet.rs
@@ -32,10 +32,13 @@ pub const SNIPPET_BODY: &str = r#"
This project uses [ai-memory](https://github.com/akitaonrails/ai-memory)
for cross-session continuity. **Lifecycle hooks already capture every
prompt + tool call automatically.** You never need to manually write
-routine notes; the SessionStart hook auto-fetches pending handoffs and
-the SessionEnd hook auto-consolidates. Only write a durable wiki page
-when the user explicitly asks to remember or annotate something
-permanently.
+routine notes; the SessionStart hook auto-fetches pending handoffs, and
+on session end ai-memory writes a session-summary page and a handoff.
+LLM consolidation (compiling observations into topical wiki pages) runs
+on PreCompact, on demand via `memory_consolidate`, and at session end
+only when the server sets `AI_MEMORY_CONSOLIDATE_ON_SESSION_END`. Only
+write a durable wiki page when the user explicitly asks to remember or
+annotate something permanently.
### When to reach for each tool
@@ -52,7 +55,7 @@ match the intent to the tool. They do not need to name the tool.
| "where did we leave off?" — and you see a `📥 ai-memory: pending handoff` block in your context | already done — answer from that block; do NOT re-call `memory_handoff_accept` |
| "where did we leave off?" — and no such block is visible | `memory_handoff_accept` (rare; the SessionStart hook usually got there first) |
| "save context for the next session" / wrapping up | `memory_handoff_begin` (single-use handoff; terse summary; put detail in `open_questions` + `next_steps` bullets) |
-| "consolidate this session" / "compile what we learned" (usually automatic) | `memory_consolidate` |
+| "consolidate this session" / "compile what we learned" (also runs on PreCompact; at session end only if `AI_MEMORY_CONSOLIDATE_ON_SESSION_END` is set) | `memory_consolidate` |
| "remember this permanently" / "save a note" / "add an annotation" / durable project knowledge | `memory_write_page` (write a wiki page; do **not** use handoff for permanent notes) |
| "audit the wiki" / "find contradictions" / "what rules should we add?" | `memory_lint` |
| "prune old pages" / "memory cleanup" | `memory_forget_sweep` |
diff --git a/crates/ai-memory-hooks/src/router.rs b/crates/ai-memory-hooks/src/router.rs
index 0f3fa809..02fa688e 100644
--- a/crates/ai-memory-hooks/src/router.rs
+++ b/crates/ai-memory-hooks/src/router.rs
@@ -83,6 +83,12 @@ pub struct HookState {
/// In-flight hook processing limiter. Requests acquire one permit before
/// spawning work and return 429 immediately when saturated.
pub ingest_semaphore: Arc<tokio::sync::Semaphore>,
+ /// Opt-in (`AI_MEMORY_CONSOLIDATE_ON_SESSION_END`): when true and a
+ /// `consolidator` is present, SessionEnd also runs LLM consolidation on
+ /// top of the always-written heuristic session page. Off by default so
+ /// session close stays cheap; the LLM checkpoint otherwise happens on
+ /// PreCompact and via manual `memory_consolidate`.
+ pub consolidate_on_session_end: bool,
}
/// Build a router with `POST /hook` (event ingress) and `GET /handoff`
@@ -512,6 +518,27 @@ async fn process(state: &HookState, env: HookEnvelope) -> anyhow::Result<()> {
&observations,
);
let handoff_id = state.writer.insert_handoff(handoff).await?;
+ // Opt-in (AI_MEMORY_CONSOLIDATE_ON_SESSION_END): additionally run LLM
+ // consolidation so the session's knowledge is compiled into topical
+ // pages, not just the heuristic session record. The heuristic page
+ // above is always written first, so an LLM failure here is non-fatal —
+ // warn and keep the deterministic result. Runs before the commit so
+ // the consolidated pages land in the same atomic snapshot.
+ if state.consolidate_on_session_end
+ && let Some(c) = state.consolidator.as_ref()
+ {
+ match c.consolidate_session(session_id, false).await {
+ Ok(outcome) => info!(
+ session = %session_id,
+ path = %outcome.path,
+ "SessionEnd: LLM consolidation written (opt-in)",
+ ),
+ Err(e) => warn!(
+ error = %e,
+ "SessionEnd LLM consolidation failed; heuristic page already written",
+ ),
+ }
+ }
// Auto-commit the wiki tree so the session/handoff/log.md
// changes land in git in one atomic snapshot.
let commit_msg = format!(
@@ -737,6 +764,7 @@ mod tests {
sanitizer,
project_cache: Arc::new(tokio::sync::Mutex::new(HashMap::new())),
active_project: ActiveProject::new(),
+ consolidate_on_session_end: false,
ingest_semaphore: Arc::new(tokio::sync::Semaphore::new(
DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT,
)),
@@ -1085,6 +1113,44 @@ mod tests {
);
}
+ /// SessionEnd must always write the heuristic `sessions/<id>.md` page,
+ /// even with `consolidate_on_session_end` enabled but no LLM provider:
+ /// the opt-in LLM pass is additive and guarded by a present
+ /// `consolidator`, so flag-on + no-provider degrades to today's
+ /// deterministic behavior (issue #40 — no regression).
+ #[tokio::test]
+ async fn session_end_writes_heuristic_page_even_with_consolidate_flag_on() {
+ let tmp = TempDir::new().unwrap();
+ let mut state = make_state(&tmp).await;
+ state.consolidate_on_session_end = true; // flag on; consolidator stays None
+
+ let sid = "11111111-1111-1111-1111-111111111111";
+ for event in ["session-start", "session-end"] {
+ let env = HookEnvelope::from_query_and_body(
+ HookQuery {
+ event: event.into(),
+ agent: Some("claude-code".into()),
+ ..Default::default()
+ },
+ serde_json::json!({ "session_id": sid }),
+ );
+ process(&state, env).await.unwrap();
+ }
+
+ let pages = state
+ .reader
+ .recent_pages_for_project(state.workspace_id, state.project_id, 20)
+ .await
+ .unwrap();
+ assert!(
+ pages
+ .iter()
+ .any(|p| p.path.as_str().starts_with("sessions/")),
+ "SessionEnd must write a heuristic sessions/<id>.md page regardless of the flag; got {:?}",
+ pages.iter().map(|p| p.path.as_str()).collect::<Vec<_>>()
+ );
+ }
+
#[tokio::test]
async fn process_accepts_prompt_before_session_start() {
let tmp = TempDir::new().unwrap();
diff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rs
index 0ab6cc66..80225478 100644
--- a/crates/ai-memory-mcp/src/server.rs
+++ b/crates/ai-memory-mcp/src/server.rs
@@ -57,7 +57,8 @@ them:\n\
also auto-captures this). Keep the summary terse (2-3 sentences); \
put detail in open_questions + next_steps bullets.\n\
- `memory_consolidate` — when the user asks to compile session \
- observations into wiki pages (usually automatic at session end).\n\
+ observations into wiki pages. Also runs on PreCompact, and at \
+ session end only when AI_MEMORY_CONSOLIDATE_ON_SESSION_END is set.\n\
- `memory_write_page` — when the user explicitly asks to remember, \
save, or annotate durable project knowledge. This writes a wiki page; \
do NOT use `memory_handoff_begin` for permanent annotations.\n\
diff --git a/docs/design-decisions.md b/docs/design-decisions.md
index 85b5fdfd..a554bf34 100644
--- a/docs/design-decisions.md
+++ b/docs/design-decisions.md
@@ -86,7 +86,7 @@ Why not LanceDB/Qdrant/Kuzu/CozoDB/SurrealDB?
**LLM for consolidation passes:**
- **Off by default**, behaves like agentmemory after #138's fix. Without a provider, the system still works: synthetic compression (rule-based), no LLM-generated summaries, no `memory_consolidate` page-rewrite.
-- With a provider, scheduled consolidation runs (1× per session-end + optional 6h timer).
+- With a provider, LLM consolidation runs on PreCompact, on demand via `memory_consolidate`, and at session end only when `AI_MEMORY_CONSOLIDATE_ON_SESSION_END=true` (off by default — session end always writes a rule-based summary page + handoff regardless). Optional 6h maintenance timer.
- Provider trait `LlmProvider { complete(...); complete_structured(...) }`. Implementations: `AnthropicProvider`, `OpenAIProvider`, `GeminiProvider`, `OpenAICompatProvider` (the latter covers Ollama / vLLM / LM Studio and supersedes the earlier `OllamaProvider`).
- **Native HTTP per provider** - no LiteLLM-equivalent. The cognee tracker (#2412/#2430/#2537/#2608/#2749/#2782/#2840/#2842) showed silent-kwarg-drop in a generic gateway is the #1 source of provider bugs. Each provider's typed JSON, errors on unknown fields. Hand-coded but correct.
- **Structured output via JSON schema, not XML, not Instructor-style wrapping.** Use each provider's native JSON-mode where available; for Anthropic, request a tool-use response with a typed schema. Validate with `serde_json` + `schemars`-derived schemas.
diff --git a/docs/usage.md b/docs/usage.md
index 9b93283e..2c9f6b7f 100644
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -47,7 +47,7 @@ the wiki without you naming tools explicitly.
| "Catch me up" / "I've been away" | `memory_explore` | Prose digest whose verbosity scales with time since last activity. |
| "Where did we leave off?" | Existing handoff block, or `memory_handoff_accept` if no block exists | Resumes from the latest pending handoff. |
| "Save context for the next session" | `memory_handoff_begin` | Writes a terse handoff with open questions and next steps. |
-| "Consolidate this session" | `memory_consolidate` | Manually runs what session-end normally runs automatically. |
+| "Consolidate this session" | `memory_consolidate` | Manually runs LLM consolidation. Also runs on PreCompact, and at session end only when `AI_MEMORY_CONSOLIDATE_ON_SESSION_END` is set (off by default; session end otherwise writes a rule-based summary page). |
| "Remember this permanently" / "add an annotation" | `memory_write_page` | Writes durable wiki knowledge; not a single-use handoff. |
| "Audit the wiki" / "any contradictions?" | `memory_lint` | Runs stale-page, contradiction, and rule-suggestion checks. |
| "How big is the wiki?" / "stats?" | `memory_status`, `memory_briefing` | Counts and recent activity windows. |