Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(hooks): queue PreCompact LLM work on the durable worker (#1229)
Victor Alencardc1ae89143c2
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6bf29b1a..b2b8a6a9 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]### Fixed+- PreCompact and PostCompaction write the heuristic `sessions/<id>.md`+ checkpoint on the `/hook` and `/hook/batch` request and complete the+ ingest key there. When a provider is configured, the LLM rewrite is+ queued on `session_consolidation_jobs` and handled by the same bounded+ worker SessionEnd already uses. The previous inline wait sat behind+ the client drain (3 s, or 250 ms incremental) for a 12-135 s provider+ call, left the ingest key incomplete, and restarted the LLM on every+ redelivery. SessionEnd enqueue stays opt-in via+ `AI_MEMORY_CONSOLIDATE_ON_SESSION_END`; the worker now starts whenever+ a consolidator exists so checkpoint jobs still run when that flag is+ off. The rewrite is attributed to the session owner recorded at session+ start. (#1229)- Replaced the unmaintained YAML library (`serde_yaml` and its`unsafe-libyaml` parser, RUSTSEC-2024-0449) with `serde-saphyr`. The serverreads existing page frontmatter to exactly the same values as before. Onediff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rsindex 58cef900..fd5421cb 100644--- a/crates/ai-memory-cli/src/commands/serve.rs+++ b/crates/ai-memory-cli/src/commands/serve.rs@@ -11,7 +11,7 @@ use ai_memory_consolidate::{ScheduledAutoImproveSettings, run_auto_improve_scheduler_tick, run_embedding_backfill,run_lint,};-use ai_memory_core::{ActiveProject, ProjectId, Sanitizer, WorkspaceId};+use ai_memory_core::{ActiveProject, IdentityKey, ProjectId, Sanitizer, WorkspaceId};use ai_memory_hooks::{CLIENT_MARKER_HEADER, DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT, HookState, ProjectCacheStore,WorkstreamState, hook_router, workstream_router,@@ -862,8 +862,35 @@ async fn run_profile_worker(}}+async fn session_consolidation_actor(+ reader: &ReaderPool,+ session_id: ai_memory_core::SessionId,+) -> ai_memory_core::ActorContext {+ match reader.session_actor_user(session_id).await {+ Ok(Some(owner)) => IdentityKey::from_storage_key(&owner)+ .map(|key| key.to_actor_context())+ .unwrap_or_else(|| {+ tracing::warn!(+ session = %session_id,+ "session owner is not a valid identity key; consolidating as anonymous"+ );+ ai_memory_core::ActorContext::anonymous()+ }),+ Ok(None) => ai_memory_core::ActorContext::anonymous(),+ Err(error) => {+ tracing::warn!(+ %error,+ session = %session_id,+ "could not read session owner; consolidating as anonymous"+ );+ ai_memory_core::ActorContext::anonymous()+ }+ }+}+async fn run_session_consolidation_worker(writer: WriterHandle,+ reader: ReaderPool,consolidator: Arc<Consolidator>,notify: Arc<tokio::sync::Notify>,cancel: CancellationToken,@@ -933,13 +960,8 @@ async fn run_session_consolidation_worker("could not read the session page before SessionEnd consolidation; consolidating",),}- let consolidation = consolidator.consolidate_session(- session_id,- false,- ai_memory_core::ActorContext::anonymous(),- None,- None,- );+ let actor = session_consolidation_actor(&reader, session_id).await;+ let consolidation = consolidator.consolidate_session(session_id, false, actor, None, None);tokio::pin!(consolidation);let result = tokio::select! {() = cancel.cancelled() => {@@ -959,7 +981,7 @@ async fn run_session_consolidation_worker(generation,attempts,path = %outcome.path,- "SessionEnd: queued LLM consolidation written (opt-in)",+ "queued LLM consolidation written",);#[cfg(test)]completed.notify_one();@@ -1406,25 +1428,22 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {cancel.child_token(),));let (session_consolidation_notify, session_consolidation_task) =- if config.consolidate_on_session_end {- if let Some(consolidator) = consolidator.clone() {- let notify = Arc::new(tokio::sync::Notify::new());- let task = tokio::spawn(run_session_consolidation_worker(- store.writer.clone(),- consolidator,- notify.clone(),- cancel.child_token(),- #[cfg(test)]- Arc::new(tokio::sync::Notify::new()),- ));- info!(- max_attempts = ai_memory_store::SESSION_CONSOLIDATION_MAX_ATTEMPTS,- "durable SessionEnd consolidation worker started"- );- (Some(notify), Some(task))- } else {- (None, None)- }+ if let Some(consolidator) = consolidator.clone() {+ let notify = Arc::new(tokio::sync::Notify::new());+ let task = tokio::spawn(run_session_consolidation_worker(+ store.writer.clone(),+ store.reader.clone(),+ consolidator,+ notify.clone(),+ cancel.child_token(),+ #[cfg(test)]+ Arc::new(tokio::sync::Notify::new()),+ ));+ info!(+ max_attempts = ai_memory_store::SESSION_CONSOLIDATION_MAX_ATTEMPTS,+ "durable session consolidation worker started"+ );+ (Some(notify), Some(task))} else {(None, None)};@@ -4426,6 +4445,7 @@ mod tests {let cancel = CancellationToken::new();let task = tokio::spawn(run_session_consolidation_worker(store.writer.clone(),+ store.reader.clone(),consolidator,notify.clone(),cancel.child_token(),@@ -4462,6 +4482,209 @@ mod tests {);}+ /// A PreCompact checkpoint job is an open session. The same worker must+ /// consume it without `end_session` and without the SessionEnd opt-in.+ #[tokio::test]+ async fn session_consolidation_worker_consumes_an_open_session_checkpoint() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let workspace_id = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let project_id = store+ .writer+ .get_or_create_project(workspace_id, "project", None)+ .await+ .unwrap();+ let session_id = SessionId::new();+ store+ .writer+ .begin_session(NewSession {+ occurred_at: None,+ id: session_id,+ workspace_id,+ project_id,+ agent_kind: AgentKind::Codex,+ cwd: None,+ actor_user: None,+ })+ .await+ .unwrap();+ store+ .writer+ .insert_observation(Sanitized::new(+ NewObservation {+ occurred_at: None,+ session_id,+ workspace_id,+ project_id,+ kind: ObservationKind::UserPrompt,+ extension: None,+ source_event: None,+ title: "checkpoint".into(),+ body: "open session still running".into(),+ importance: 8,+ },+ &Sanitizer::default(),+ ))+ .await+ .unwrap();+ store+ .writer+ .enqueue_session_checkpoint(workspace_id, project_id, session_id)+ .await+ .unwrap();++ let wiki = Wiki::new(tmp.path(), store.writer.clone()).unwrap();+ let consolidator = Arc::new(Consolidator::new(+ store.reader.clone(),+ store.writer.clone(),+ wiki,+ Arc::new(SuccessfulConsolidationLlm),+ workspace_id,+ project_id,+ ));+ let notify = Arc::new(tokio::sync::Notify::new());+ let completed = Arc::new(tokio::sync::Notify::new());+ let cancel = CancellationToken::new();+ let task = tokio::spawn(run_session_consolidation_worker(+ store.writer.clone(),+ store.reader.clone(),+ consolidator,+ notify.clone(),+ cancel.child_token(),+ completed.clone(),+ ));+ notify.notify_one();++ tokio::time::timeout(Duration::from_secs(5), completed.notified())+ .await+ .expect("worker should complete the open-session checkpoint");++ let path = format!("sessions/{session_id}.md");+ assert!(+ store+ .reader+ .page_body_by_ids(workspace_id, project_id, &path)+ .await+ .unwrap()+ .is_some_and(|page| page.body.contains("Durable worker completed")),+ "checkpoint completion must follow the durable wiki write"+ );++ cancel.cancel();+ task.await.unwrap();+ let now = jiff::Timestamp::now().as_microsecond();+ assert!(+ store+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .is_none(),+ "completed checkpoint must not be claimed again"+ );+ }++ /// The worker attributes the rewrite to the session owner, not anonymous.+ #[tokio::test]+ async fn session_consolidation_worker_attributes_checkpoint_to_session_owner() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let workspace_id = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let project_id = store+ .writer+ .get_or_create_project(workspace_id, "project", None)+ .await+ .unwrap();+ let session_id = SessionId::new();+ store+ .writer+ .begin_session(NewSession {+ occurred_at: None,+ id: session_id,+ workspace_id,+ project_id,+ agent_kind: AgentKind::Codex,+ cwd: None,+ actor_user: Some("user:alice".into()),+ })+ .await+ .unwrap();+ store+ .writer+ .insert_observation(Sanitized::new(+ NewObservation {+ occurred_at: None,+ session_id,+ workspace_id,+ project_id,+ kind: ObservationKind::UserPrompt,+ extension: None,+ source_event: None,+ title: "checkpoint".into(),+ body: "owned session still running".into(),+ importance: 8,+ },+ &Sanitizer::default(),+ ))+ .await+ .unwrap();+ store+ .writer+ .enqueue_session_checkpoint(workspace_id, project_id, session_id)+ .await+ .unwrap();++ let wiki = Wiki::new(tmp.path(), store.writer.clone()).unwrap();+ let consolidator = Arc::new(Consolidator::new(+ store.reader.clone(),+ store.writer.clone(),+ wiki,+ Arc::new(SuccessfulConsolidationLlm),+ workspace_id,+ project_id,+ ));+ let notify = Arc::new(tokio::sync::Notify::new());+ let completed = Arc::new(tokio::sync::Notify::new());+ let cancel = CancellationToken::new();+ let task = tokio::spawn(run_session_consolidation_worker(+ store.writer.clone(),+ store.reader.clone(),+ consolidator,+ notify.clone(),+ cancel.child_token(),+ completed.clone(),+ ));+ notify.notify_one();++ tokio::time::timeout(Duration::from_secs(5), completed.notified())+ .await+ .expect("worker should complete the owned checkpoint");+ cancel.cancel();+ task.await.unwrap();++ let path = format!("sessions/{session_id}.md");+ let page = store+ .reader+ .page_body_by_ids(workspace_id, project_id, &path)+ .await+ .unwrap()+ .expect("checkpoint page must exist");+ let frontmatter: serde_json::Value = serde_json::from_str(&page.frontmatter_json).unwrap();+ assert_eq!(+ frontmatter["last_modified_by"]["username"].as_str(),+ Some("alice"),+ "checkpoint rewrite must be attributed to the session owner: {frontmatter}"+ );+ }+// Mutation captured: persisting the error's `Display` writes the private// provider body into the queue row; the worker must persist only the// redacted class/status summary, and a failed first attempt keeps the@@ -4533,6 +4756,7 @@ mod tests {let cancel = CancellationToken::new();let task = tokio::spawn(run_session_consolidation_worker(store.writer.clone(),+ store.reader.clone(),consolidator,notify.clone(),cancel.child_token(),@@ -4695,6 +4919,7 @@ mod tests {let cancel = CancellationToken::new();let task = tokio::spawn(run_session_consolidation_worker(store.writer.clone(),+ store.reader.clone(),consolidator,notify.clone(),cancel.child_token(),diff --git a/crates/ai-memory-hooks/src/router.rs b/crates/ai-memory-hooks/src/router.rsindex e923a57b..d05c05d0 100644--- a/crates/ai-memory-hooks/src/router.rs+++ b/crates/ai-memory-hooks/src/router.rs@@ -456,10 +456,10 @@ pub struct HookState {pub reader: ai_memory_store::ReaderPool,/// Wiki handle — used to write the session-summary page.pub wiki: Wiki,- /// Optional LLM-driven consolidator. When set, PreCompact uses it- /// to refresh `sessions/<id>.md` before the agent loses its- /// working context. When `None`, falls back to the deterministic- /// rule-based synth (still useful, just lower-signal).+ /// Optional LLM-driven consolidator. When set, PreCompact /+ /// PostCompaction write a rule-based `sessions/<id>.md` on the+ /// request and enqueue the provider rewrite on the durable worker.+ /// When `None`, only the rule-based synth runs.pub consolidator: Option<Arc<Consolidator>>,/// Privacy strip applied to every observation before it lands in/// the store. Same handle is also held by the wiki and consolidator@@ -489,12 +489,13 @@ pub struct HookState {/// tokens. Disabled (pass-through) unless configured by the CLI.pub ingest_rate: Arc<tokio::sync::Mutex<IngestRateLimiter>>,/// Opt-in (`AI_MEMORY_CONSOLIDATE_ON_SESSION_END`): when true and a- /// `consolidator` is present, SessionEnd also runs LLM consolidation on- /// top of the always-written heuristic session page. Off by default so- /// session close stays cheap; the LLM checkpoint otherwise happens on- /// PreCompact and via manual `memory_consolidate`.+ /// `consolidator` is present, SessionEnd also enqueues LLM consolidation+ /// on top of the always-written heuristic session page. Off by default so+ /// session close stays cheap. PreCompact / PostCompaction enqueue+ /// whenever a consolidator is present, independent of this flag.pub consolidate_on_session_end: bool,- /// Coalescing wake-up for the durable SessionEnd consolidation worker.+ /// Coalescing wake-up for the durable session-consolidation worker+ /// (opt-in SessionEnd jobs and PreCompact / PostCompaction checkpoints)./// The database is the queue; notifications only reduce pickup latency.pub session_consolidation_notify: Option<Arc<tokio::sync::Notify>>,/// Coalescing wake-up for the cross-project profile pass: a session that@@ -3321,6 +3322,45 @@ async fn enqueue_session_end_consolidation(Ok(inserted)}+async fn enqueue_session_checkpoint_consolidation(+ state: &HookState,+ session_id: SessionId,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+) -> anyhow::Result<bool> {+ if state.consolidator.is_none() {+ return Ok(false);+ }+ let Some(notify) = state.session_consolidation_notify.as_ref() else {+ warn!(+ session = %session_id,+ "PreCompact/PostCompaction LLM consolidation has no queue worker"+ );+ return Ok(false);+ };+ let path = PagePath::new(format!("sessions/{session_id}.md"))?;+ if agent_session_page_id(state, workspace_id, project_id, session_id, &path)+ .await?+ .is_some()+ {+ return Ok(false);+ }+ let inserted = state+ .writer+ .enqueue_session_checkpoint(workspace_id, project_id, session_id)+ .await?;+ if inserted {+ state.ingest_metrics.record_persisted(now_unix_ms());+ }+ notify.notify_one();+ debug!(+ session = %session_id,+ inserted,+ "PreCompact/PostCompaction LLM consolidation queued"+ );+ Ok(inserted)+}+/// Only events that begin or actively advance work may move the legacy/// process-wide and identity-only fallbacks. Completion events can arrive/// after the operator has moved to another project, especially when a native@@ -3843,8 +3883,8 @@ async fn process_authorized(.as_ref().map(IdentityKey::to_actor_context).unwrap_or_default();- if let Some(checkpoint_label) = checkpoint_label- && let Err(e) = consolidate_or_synth(+ if let Some(checkpoint_label) = checkpoint_label {+ match consolidate_or_synth(state,session_id,ws,@@ -3853,12 +3893,27 @@ async fn process_authorized(HookCheckpoint {label: checkpoint_label,track_page_write: outcome == HookProcessingOutcome::Resumed,+ defer_llm: true,},session_actor.clone(),).await- {- warn!(error = %e, "PreCompact/PostCompaction consolidation failed; continuing");+ {+ Ok(checkpoint) => {+ if checkpoint.should_enqueue_llm+ && let Err(e) =+ enqueue_session_checkpoint_consolidation(state, session_id, ws, proj).await+ {+ warn!(+ error = %e,+ "PreCompact/PostCompaction consolidation enqueue failed; continuing"+ );+ }+ }+ Err(e) => {+ warn!(error = %e, "PreCompact/PostCompaction consolidation failed; continuing");+ }+ }}// On SessionEnd, close boundary-only sessions without generated artifacts.@@ -4520,6 +4575,7 @@ fn checkpoint_fallback_warning(struct HookCheckpoint {label: &'static str,track_page_write: bool,+ defer_llm: bool,}/// What one checkpoint run did. `fallback_reason` is set exactly when the@@ -4531,10 +4587,11 @@ struct HookCheckpoint {#[derive(Debug)]struct CheckpointOutcome {// Read by the checkpoint tests (compiled only under `cfg(test)`); the- // production caller deliberately ignores a successful checkpoint's- // outcome, so the lint must be allowed explicitly.+ // production caller uses `should_enqueue_llm` and ignores+ // `fallback_reason`.#[allow(dead_code)]fallback_reason: Option<String>,+ should_enqueue_llm: bool,}/// Write a fresh `sessions/<id>.md` for the current session without@@ -4551,6 +4608,7 @@ async fn consolidate_or_synth() -> anyhow::Result<CheckpointOutcome> {let checkpoint_label = checkpoint.label;let mut fallback_reason = None;+ let should_enqueue_llm = checkpoint.defer_llm && state.consolidator.is_some();// First deliveries already stamped their observation write. Only a retry// needs to distinguish a recovered page write from an unchanged checkpoint.let previous_page_id = if checkpoint.track_page_write {@@ -4573,10 +4631,15 @@ async fn consolidate_or_synth(.await?.is_some(){- return Ok(CheckpointOutcome { fallback_reason });+ return Ok(CheckpointOutcome {+ fallback_reason,+ should_enqueue_llm: false,+ });}- let fallback_from_llm = state.consolidator.is_some();- if let Some(c) = state.consolidator.as_ref() {+ let fallback_from_llm = state.consolidator.is_some() && !checkpoint.defer_llm;+ if !checkpoint.defer_llm+ && let Some(c) = state.consolidator.as_ref()+ {let result = c// The hook path has no per-call override, so `None` lets the// project's standing `_prompts/consolidation.md` preferences@@ -4615,12 +4678,18 @@ async fn consolidate_or_synth();e});- return Ok(CheckpointOutcome { fallback_reason });+ return Ok(CheckpointOutcome {+ fallback_reason,+ should_enqueue_llm: false,+ });}// Nothing to checkpoint. The rule-based path below no-ops on the// same condition, so this is success, not a failure worth logging.Err(ConsolidatorError::EmptySession(_)) => {- return Ok(CheckpointOutcome { fallback_reason });+ return Ok(CheckpointOutcome {+ fallback_reason,+ should_enqueue_llm: false,+ });}Err(e) if checkpoint_degrades_to_synth(&e) => {let (warning, error_summary) = checkpoint_fallback_warning(checkpoint_label, &e);@@ -4636,7 +4705,10 @@ async fn consolidate_or_synth(}let observations = state.reader.observations_for_session(session_id).await?;if is_ephemeral_session(&observations) {- return Ok(CheckpointOutcome { fallback_reason });+ return Ok(CheckpointOutcome {+ fallback_reason,+ should_enqueue_llm: false,+ });}let new_page = synthesize_session_page(workspace_id,@@ -4682,7 +4754,10 @@ async fn consolidate_or_synth(e});debug!(session = %session_id, "{}: rule-based checkpoint written", checkpoint_label);- Ok(CheckpointOutcome { fallback_reason })+ Ok(CheckpointOutcome {+ fallback_reason,+ should_enqueue_llm,+ })}/// The id of `sessions/<id>.md` when the agent wrote it itself through@@ -5082,6 +5157,36 @@ mod tests {}}+ /// Stalls, then panics if the compaction hook path still calls the provider.+ struct SlowHookLlm;++ #[async_trait::async_trait]+ impl LlmProvider for SlowHookLlm {+ fn name(&self) -> &'static str {+ "slow-hook"+ }+ fn model(&self) -> &str {+ "slow-hook"+ }+ async fn complete(&self, _request: ChatRequest) -> LlmResult<ChatResponse> {+ self.stall().await+ }+ async fn complete_structured_raw(+ &self,+ _request: ChatRequest,+ _schema: serde_json::Value,+ ) -> LlmResult<serde_json::Value> {+ self.stall().await+ }+ }++ impl SlowHookLlm {+ async fn stall<T>(&self) -> LlmResult<T> {+ tokio::time::sleep(Duration::from_millis(500)).await;+ panic!("compaction hook path must not call the provider");+ }+ }+/// Build a minimal `HookState` backed by a real on-disk store./// A capture is a write, so it needs `write`.///@@ -6145,6 +6250,7 @@ mod tests {HookCheckpoint {label: "pre-compact",track_page_write: true,+ defer_llm: false,},ai_memory_core::ActorContext::anonymous(),)@@ -6238,6 +6344,7 @@ mod tests {HookCheckpoint {label: checkpoint_label,track_page_write: true,+ defer_llm: false,},ai_memory_core::ActorContext::anonymous(),)@@ -6335,6 +6442,7 @@ mod tests {HookCheckpoint {label: "pre-compact",track_page_write: true,+ defer_llm: false,},ai_memory_core::ActorContext::anonymous(),)@@ -6375,6 +6483,7 @@ mod tests {HookCheckpoint {label: "pre-compact",track_page_write: true,+ defer_llm: false,},ai_memory_core::ActorContext::anonymous(),)@@ -7580,6 +7689,7 @@ mod tests {HookCheckpoint {label: "test",track_page_write: true,+ defer_llm: false,},ai_memory_core::ActorContext::anonymous(),)@@ -7597,6 +7707,7 @@ mod tests {HookCheckpoint {label: "test",track_page_write: true,+ defer_llm: false,},ai_memory_core::ActorContext::anonymous(),)@@ -11779,6 +11890,144 @@ mod tests {);}+ /// Compaction checkpoints write the heuristic page, enqueue the durable+ /// job, and return without calling the provider (#1229). Independent of+ /// the SessionEnd opt-in.+ #[tokio::test]+ async fn compaction_checkpoint_queues_llm_work_without_waiting_for_the_provider() {+ for event in ["pre-compact", "post-compaction"] {+ let tmp = TempDir::new().unwrap();+ let mut state = make_state(&tmp).await;+ state.consolidator = Some(Arc::new(Consolidator::new(+ state.reader.clone(),+ state.writer.clone(),+ state.wiki.clone(),+ Arc::new(SlowHookLlm),+ state.workspace_id,+ state.project_id,+ )));+ state.session_consolidation_notify = Some(Arc::new(tokio::sync::Notify::new()));+ let sid = if event == "pre-compact" {+ "c0c0c0c0-c0c0-c0c0-c0c0-c0c0c0c0c001"+ } else {+ "c0c0c0c0-c0c0-c0c0-c0c0-c0c0c0c0c002"+ };+ let fire = |name: &str| {+ HookEnvelope::from_query_and_body(+ HookQuery {+ event: name.into(),+ agent: Some("claude-code".into()),+ ..Default::default()+ },+ serde_json::json!({+ "session_id": sid,+ "prompt": "keep this working state"+ }),+ )+ };+ process(&state, fire("user-prompt-submit"), None, Vec::new())+ .await+ .unwrap();+ let started = std::time::Instant::now();+ process(&state, fire(event), None, Vec::new())+ .await+ .unwrap();+ assert!(+ started.elapsed() < Duration::from_millis(200),+ "{event} must not wait for the provider on the hook path"+ );++ let session_id: SessionId = sid.parse().unwrap();+ let path = format!("sessions/{session_id}.md");+ let page = state+ .reader+ .page_body_by_ids(state.workspace_id, state.project_id, &path)+ .await+ .unwrap()+ .unwrap_or_else(|| panic!("{event} must write a rule-based checkpoint"));+ assert!(+ page.body.contains("keep this working state"),+ "{event} heuristic page must carry the session: {}",+ page.body+ );++ let now = Timestamp::now().as_microsecond();+ let job = state+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .unwrap_or_else(|| panic!("{event} must enqueue a durable checkpoint job"));+ assert_eq!(job.session_id(), session_id);+ assert!(+ state+ .reader+ .latest_open_handoff(+ state.workspace_id,+ state.project_id,+ None,+ ai_memory_core::OwnerFilter::Any+ )+ .await+ .unwrap()+ .is_none(),+ "{event} must not create a handoff"+ );+ }+ }++ /// Zero-LLM PreCompact still writes the synth page and must not insert a+ /// consolidation job.+ #[tokio::test]+ async fn compaction_checkpoint_writes_synth_without_a_job_when_no_provider() {+ let tmp = TempDir::new().unwrap();+ let state = make_state(&tmp).await;+ let sid = "c0c0c0c0-c0c0-c0c0-c0c0-c0c0c0c0c010";+ let fire = |name: &str| {+ HookEnvelope::from_query_and_body(+ HookQuery {+ event: name.into(),+ agent: Some("claude-code".into()),+ ..Default::default()+ },+ serde_json::json!({+ "session_id": sid,+ "prompt": "keep this working state"+ }),+ )+ };+ process(&state, fire("user-prompt-submit"), None, Vec::new())+ .await+ .unwrap();+ process(&state, fire("pre-compact"), None, Vec::new())+ .await+ .unwrap();++ let session_id: SessionId = sid.parse().unwrap();+ let path = format!("sessions/{session_id}.md");+ let page = state+ .reader+ .page_body_by_ids(state.workspace_id, state.project_id, &path)+ .await+ .unwrap()+ .expect("zero-LLM PreCompact must still write the synth page");+ assert!(+ page.body.contains("keep this working state"),+ "synth page must carry the session: {}",+ page.body+ );+ let now = Timestamp::now().as_microsecond();+ assert!(+ state+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .is_none(),+ "zero-LLM PreCompact must not insert a consolidation job"+ );+ }+/// A compaction checkpoint keeps the agent's page as SessionEnd does,/// and still replaces a page no agent wrote (control).#[tokio::test]@@ -11802,6 +12051,86 @@ mod tests {}}+ /// An agent-written session page is left alone and must not enqueue a+ /// checkpoint job: the worker would otherwise still claim it.+ #[tokio::test]+ async fn compaction_checkpoint_skips_enqueue_when_the_agent_wrote_the_page() {+ let tmp = TempDir::new().unwrap();+ let mut state = make_state(&tmp).await;+ state.consolidator = Some(Arc::new(Consolidator::new(+ state.reader.clone(),+ state.writer.clone(),+ state.wiki.clone(),+ Arc::new(SlowHookLlm),+ state.workspace_id,+ state.project_id,+ )));+ state.session_consolidation_notify = Some(Arc::new(tokio::sync::Notify::new()));+ let sid = "c0c0c0c0-c0c0-c0c0-c0c0-c0c0c0c0c020";+ let deliver = |event: &'static str| {+ HookEnvelope::from_query_and_body(+ HookQuery {+ event: event.into(),+ agent: Some("claude-code".into()),+ ..Default::default()+ },+ serde_json::json!({ "session_id": sid, "prompt": "consolidate this session" }),+ )+ };+ for event in ["session-start", "user-prompt-submit"] {+ process(&state, deliver(event), None, Vec::new())+ .await+ .unwrap();+ }+ let path = format!("sessions/{sid}.md");+ state+ .wiki+ .write_page(ai_memory_wiki::WritePageRequest {+ workspace_id: state.workspace_id,+ project_id: state.project_id,+ path: ai_memory_core::PagePath::new(path.clone()).unwrap(),+ frontmatter: serde_json::json!({+ "consolidated": true,+ "consolidated_by": "agent"+ }),+ body: "# Agent page\n\nCompiled by the agent's own model.".into(),+ tier: ai_memory_core::Tier::Episodic,+ pinned: false,+ title: None,+ admission_ctx: None,+ author_id: None,+ actor: ai_memory_core::ActorContext::anonymous(),+ evidence: Vec::new(),+ })+ .await+ .unwrap();+ process(&state, deliver("pre-compact"), None, Vec::new())+ .await+ .unwrap();++ let body = state+ .reader+ .page_body_by_ids(state.workspace_id, state.project_id, &path)+ .await+ .unwrap()+ .expect("the agent page must still exist")+ .body;+ assert!(+ body.contains("Compiled by the agent's own model"),+ "PreCompact must keep the agent page: {body}"+ );+ let now = Timestamp::now().as_microsecond();+ assert!(+ state+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .is_none(),+ "an agent-written page must not enqueue a checkpoint job"+ );+ }+/// A substantive SessionEnd must write the heuristic `sessions/<id>.md`/// page even with `consolidate_on_session_end` enabled but no LLM provider.#[tokio::test]diff --git a/crates/ai-memory-store/src/session_consolidation.rs b/crates/ai-memory-store/src/session_consolidation.rsindex 767e1f8e..c91a5c70 100644--- a/crates/ai-memory-store/src/session_consolidation.rs+++ b/crates/ai-memory-store/src/session_consolidation.rs@@ -1,4 +1,4 @@-//! Durable queue state for opt-in SessionEnd LLM consolidation.+//! Durable queue state for SessionEnd and open-session checkpoint LLM work.use ai_memory_core::{ProjectId, SessionId, WorkspaceId};use jiff::Timestamp;@@ -54,6 +54,12 @@ impl SessionConsolidationJob {}}+/// Whether the session must already have ended, or must still be open.+enum EnqueueEligibility {+ EndedSession,+ OpenSession,+}+/// Enqueue the current observation generation for an ended session.////// Returns `true` only when a new generation was inserted. A duplicate end,@@ -65,9 +71,49 @@ pub fn enqueue(project_id: ProjectId,session_id: SessionId,) -> StoreResult<bool> {- let now = Timestamp::now().as_microsecond();- let tx = conn.transaction()?;- let inserted = tx.execute(+ enqueue_eligible(+ conn,+ workspace_id,+ project_id,+ session_id,+ EnqueueEligibility::EndedSession,+ )+}++/// Enqueue the current observation generation for an *open* session checkpoint+/// (PreCompact / PostCompaction).+///+/// Returns `true` only when a new generation was inserted. A duplicate+/// checkpoint of the same generation, an already-ended session, a missing or+/// mismatched scope, or an observation-less session returns `false` without+/// creating work. SessionEnd owns the page once `ended_at` is set.+pub fn enqueue_checkpoint(+ conn: &mut Connection,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ session_id: SessionId,+) -> StoreResult<bool> {+ enqueue_eligible(+ conn,+ workspace_id,+ project_id,+ session_id,+ EnqueueEligibility::OpenSession,+ )+}++fn enqueue_eligible(+ conn: &mut Connection,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ session_id: SessionId,+ eligibility: EnqueueEligibility,+) -> StoreResult<bool> {+ let ended_predicate = match eligibility {+ EnqueueEligibility::EndedSession => "AND s.ended_at IS NOT NULL",+ EnqueueEligibility::OpenSession => "AND s.ended_at IS NULL",+ };+ let sql = format!("INSERT OR IGNORE INTO session_consolidation_jobs \(session_id, workspace_id, project_id, generation, state, requested_at, \next_attempt_at, started_at, completed_at, attempts, claim_id, last_error) \@@ -76,9 +122,14 @@ pub fn enqueue(FROM sessions s \JOIN observations o ON o.session_id = s.id \WHERE s.id = ?1 AND s.workspace_id = ?2 AND s.project_id = ?3 \- AND s.ended_at IS NOT NULL \+ {ended_predicate} \GROUP BY s.id, s.workspace_id, s.project_id \- HAVING COUNT(o.id) > 0",+ HAVING COUNT(o.id) > 0"+ );+ let now = Timestamp::now().as_microsecond();+ let tx = conn.transaction()?;+ let inserted = tx.execute(+ &sql,params![session_id.as_bytes(),workspace_id.as_bytes(),@@ -351,7 +402,7 @@ mod tests {use super::*;use crate::Store;- async fn ended_session() -> (tempfile::TempDir, Store, WorkspaceId, ProjectId, SessionId) {+ async fn open_session() -> (tempfile::TempDir, Store, WorkspaceId, ProjectId, SessionId) {let tmp = tempfile::TempDir::new().unwrap();let store = Store::open(tmp.path()).unwrap();let workspace_id = store@@ -379,6 +430,11 @@ mod tests {.await.unwrap();insert_observation(&store, workspace_id, project_id, session_id).await;+ (tmp, store, workspace_id, project_id, session_id)+ }++ async fn ended_session() -> (tempfile::TempDir, Store, WorkspaceId, ProjectId, SessionId) {+ let (tmp, store, workspace_id, project_id, session_id) = open_session().await;store.writer.end_session(session_id, None).await.unwrap();(tmp, store, workspace_id, project_id, session_id)}@@ -410,6 +466,109 @@ mod tests {.unwrap();}+ #[tokio::test]+ async fn checkpoint_enqueue_inserts_a_job_for_an_open_session() {+ let (_tmp, store, workspace_id, project_id, session_id) = open_session().await;+ assert!(+ store+ .writer+ .enqueue_session_checkpoint(workspace_id, project_id, session_id)+ .await+ .unwrap()+ );+ assert!(+ !store+ .writer+ .enqueue_session_checkpoint(workspace_id, project_id, session_id)+ .await+ .unwrap(),+ "the same open-session generation must be idempotent"+ );+ let now = Timestamp::now().as_microsecond();+ let job = store+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .unwrap();+ assert_eq!(job.session_id(), session_id);+ assert_eq!(job.generation(), 1);+ store+ .writer+ .complete_session_consolidation(job)+ .await+ .unwrap();+ }++ #[tokio::test]+ async fn checkpoint_enqueue_refuses_an_ended_session() {+ let (_tmp, store, workspace_id, project_id, session_id) = ended_session().await;+ assert!(+ !store+ .writer+ .enqueue_session_checkpoint(workspace_id, project_id, session_id)+ .await+ .unwrap(),+ "SessionEnd owns the page once the session has ended"+ );+ }++ #[tokio::test]+ async fn session_end_enqueue_still_refuses_an_open_session() {+ let (_tmp, store, workspace_id, project_id, session_id) = open_session().await;+ assert!(+ !store+ .writer+ .enqueue_session_consolidation(workspace_id, project_id, session_id)+ .await+ .unwrap(),+ "SessionEnd enqueue must still require ended_at"+ );+ }++ #[tokio::test]+ async fn session_end_enqueue_supersedes_a_pending_checkpoint() {+ let (_tmp, store, workspace_id, project_id, session_id) = open_session().await;+ assert!(+ store+ .writer+ .enqueue_session_checkpoint(workspace_id, project_id, session_id)+ .await+ .unwrap()+ );+ insert_observation(&store, workspace_id, project_id, session_id).await;+ store.writer.end_session(session_id, None).await.unwrap();+ assert!(+ store+ .writer+ .enqueue_session_consolidation(workspace_id, project_id, session_id)+ .await+ .unwrap()+ );+ let now = Timestamp::now().as_microsecond();+ let job = store+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .unwrap();+ assert_eq!(job.generation(), 2, "the SessionEnd generation must run");+ store+ .writer+ .complete_session_consolidation(job)+ .await+ .unwrap();+ assert!(+ store+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .is_none(),+ "the pending checkpoint must have been superseded"+ );+ }+#[tokio::test]async fn queue_is_idempotent_per_observation_generation() {let (_tmp, store, workspace_id, project_id, session_id) = ended_session().await;diff --git a/crates/ai-memory-store/src/writer.rs b/crates/ai-memory-store/src/writer.rsindex 7801a9e5..1df105d8 100644--- a/crates/ai-memory-store/src/writer.rs+++ b/crates/ai-memory-store/src/writer.rs@@ -268,6 +268,12 @@ pub(crate) enum WriteCmd {session_id: SessionId,reply: oneshot::Sender<StoreResult<bool>>,},+ EnqueueSessionCheckpoint {+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ session_id: SessionId,+ reply: oneshot::Sender<StoreResult<bool>>,+ },ClaimSessionConsolidation {now: i64,stale_before: i64,@@ -1667,6 +1673,26 @@ impl WriterHandle {rx.await.map_err(|_| StoreError::WriterClosed)?}+ /// Persist one open-session checkpoint job (PreCompact / PostCompaction)+ /// for the current observation generation. Duplicate generations are+ /// idempotent. An already-ended session is refused.+ pub async fn enqueue_session_checkpoint(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ session_id: SessionId,+ ) -> StoreResult<bool> {+ let (tx, rx) = oneshot::channel();+ self.send(WriteCmd::EnqueueSessionCheckpoint {+ workspace_id,+ project_id,+ session_id,+ reply: tx,+ })+ .await?;+ rx.await.map_err(|_| StoreError::WriterClosed)?+ }+/// Atomically claim the oldest due SessionEnd consolidation job.pub async fn claim_session_consolidation(&self,@@ -3898,6 +3924,20 @@ fn worker_loop(mut conn: Connection, mut rx: mpsc::Receiver<WriteCmd>) {);send_or_warn(reply, result, "enqueue_session_consolidation");}+ WriteCmd::EnqueueSessionCheckpoint {+ workspace_id,+ project_id,+ session_id,+ reply,+ } => {+ let result = crate::session_consolidation::enqueue_checkpoint(+ &mut conn,+ workspace_id,+ project_id,+ session_id,+ );+ send_or_warn(reply, result, "enqueue_session_checkpoint");+ }WriteCmd::ClaimSessionConsolidation {now,stale_before,diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex 29fcdac7..88af2d9a 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -414,7 +414,7 @@ prunable rather than only after it ages in place.| `pages` | Versioned wiki pages with `is_latest` + `supersedes` chain. M8 columns: `last_accessed_at`, `access_count`, and decay-only tombstone marker `superseded_at`. M9 cols: `embedding_provider`, `embedding_model`, `embedding_dim`. V36: `expires_at` (frontmatter TTL). V37: `salience` (NULL = `salience_default`; derived from `page_feedback`). || `pages_fts` | FTS5 virtual table over `(title, body)`, auto-synced by triggers. || `sessions`, `observations` | Sanitized, bounded lifecycle-hook projections. `sessions.ended_observation_count` is the stable generation watermark for resumed-session re-end eligibility; wall clocks are not used for that decision. They are an operational audit trail, not a complete native transcript. |-| `session_consolidation_jobs` | Durable, observation-generation-idempotent queue for the *automatic* SessionEnd LLM consolidation worker. One bounded server worker leases jobs, retries provider failures with backoff, and recovers expired leases after restart. A manual `memory_consolidate` writes its page out-of-band and then reconciles this table (flipping a `failed`/`pending`/`superseded` row for the session to `completed`, never touching a live `running` lease), so the operator does not see a `failed` job for a session that is in fact consolidated. |+| `session_consolidation_jobs` | Durable, observation-generation-idempotent queue for automatic LLM consolidation. PreCompact / PostCompaction checkpoint jobs cover an *open* session; opt-in SessionEnd jobs cover an *ended* session. One bounded server worker leases jobs, retries provider failures with backoff, and recovers expired leases after restart. The worker starts whenever a consolidator is configured; SessionEnd enqueue stays behind `AI_MEMORY_CONSOLIDATE_ON_SESSION_END`. A manual `memory_consolidate` writes its page out-of-band and then reconciles this table (flipping a `failed`/`pending`/`superseded` row for the session to `completed`, never touching a live `running` lease), so the operator does not see a `failed` job for a session that is in fact consolidated. || `observations_fts` | FTS5 virtual table over raw observation `(title, body)`, used only as bounded fallback. || `workstreams`, `managed_runs`, `workstream_native_sessions` | Optional lease state plus per-harness native source and delivery cursors for `ai-memory run`. || `workstream_events`, `workstream_events_fts` | Append-only normalized visible transcript events and full-text search; immutable sanitized source batches also live under `raw/workstreams/`. |diff --git a/docs/design-decisions.md b/docs/design-decisions.mdindex 2207b959..b4e2e567 100644--- a/docs/design-decisions.md+++ b/docs/design-decisions.md@@ -96,7 +96,7 @@ Why not LanceDB/Qdrant/Kuzu/CozoDB/SurrealDB?**LLM for consolidation passes:**- **Off by default**, behaves like agentmemory after #138's fix. Without a provider, the system still works: synthetic compression (rule-based), no LLM-generated summaries, no `memory_consolidate` page-rewrite.-- With a provider, LLM consolidation runs on PreCompact, on demand via `memory_consolidate`, and at session end only when `AI_MEMORY_CONSOLIDATE_ON_SESSION_END=true` (off by default). A substantive session end always writes a rule-based summary page + handoff regardless, except that a session page the agent wrote itself through `memory_write_page` with the session's id (`consolidated_by: agent`) is kept in place of the summary and skipped by the SessionEnd worker; a session containing only `SessionStart` / `SessionEnd` boundaries closes without either artifact or provider work and releases any startup handoff bound to that receiver. SessionEnd provider work is persisted by observation generation and consumed outside the hook request by one bounded retrying worker, so client drain cancellation cannot lose it. The automatic handoff and completed-end watermark commit in one SQLite transaction; an already-ended keyed replay converges the remaining wiki commit, provider enqueue, and ingest-key completion. The completed end also stores the observation count it covered; a resumed session re-enters the end path only after that count advances, avoiding non-convergent wall-clock comparisons. Optional 6h maintenance timer.+- With a provider, LLM consolidation runs on PreCompact / PostCompaction, on demand via `memory_consolidate`, and at session end only when `AI_MEMORY_CONSOLIDATE_ON_SESSION_END=true` (off by default). A substantive session end always writes a rule-based summary page + handoff regardless, except that a session page the agent wrote itself through `memory_write_page` with the session's id (`consolidated_by: agent`) is kept in place of the summary and skipped by the consolidation worker; a session containing only `SessionStart` / `SessionEnd` boundaries closes without either artifact or provider work and releases any startup handoff bound to that receiver. PreCompact / PostCompaction write a rule-based `sessions/<id>.md` checkpoint on the hook request (and skip both the synth and the queue when the agent already wrote that page). Provider work for those checkpoints, and for opt-in SessionEnd, is persisted by observation generation and consumed outside the hook request by one bounded retrying worker, so client drain cancellation cannot lose it. The worker starts whenever a consolidator is configured; SessionEnd enqueue stays behind the opt-in flag. Checkpoint jobs require the session to still be open (`ended_at IS NULL`); SessionEnd jobs still require it to have ended. The automatic handoff and completed-end watermark commit in one SQLite transaction; an already-ended keyed replay converges the remaining wiki commit, provider enqueue, and ingest-key completion. The completed end also stores the observation count it covered; a resumed session re-enters the end path only after that count advances, avoiding non-convergent wall-clock comparisons. Optional 6h maintenance timer.- Providers implement `LlmProvider { complete(...); complete_structured(...) }`. The current provider and authentication matrix lives in [`ARCHITECTURE.md`](ARCHITECTURE.md); this design boundary also covers OpenAI-compatible endpoints such as Ollama, vLLM, and LM Studio.- **Native HTTP per provider**, no LiteLLM-equivalent. The cognee tracker (#2412/#2430/#2537/#2608/#2749/#2782/#2840/#2842) showed silent-kwarg-drop in a generic gateway is the #1 source of provider bugs. Each provider gets typed JSON that errors on unknown fields. It is hand-coded, but correct.- **Structured output via JSON schema, not XML, not Instructor-style wrapping.** Use each provider's native JSON-mode where available; for Anthropic, request a tool-use response with a typed schema. Validate with `serde_json` + `schemars`-derived schemas.diff --git a/docs/llm-providers.md b/docs/llm-providers.mdindex 2db90351..bcdf3291 100644--- a/docs/llm-providers.md+++ b/docs/llm-providers.md@@ -5,7 +5,7 @@ai-memory runs without an LLM: hooks still capture sessions, search usesFTS5 + declared entities + graph neighbors, and summaries fall back torule-based output. Add an LLM provider-when you want LLM consolidation (on PreCompact, on demand via+when you want LLM consolidation (on PreCompact / PostCompaction, on demand via`memory_consolidate`, or opt-in at session end with`AI_MEMORY_CONSOLIDATE_ON_SESSION_END`), richer linting, and bootstrap.Substantive session ends always write a rule-based summary page + handoff either@@ -13,10 +13,12 @@ way. A session containing only `SessionStart` / `SessionEnd` boundaries isclosed without a page, handoff, or provider job. When that empty session hadaccepted startup context, its session-bound handoff is returned to the openpool for the next receiver instead of being lost.-When the session-end opt-in is enabled, provider work is durably queued after-those deterministic writes and handled by one bounded server worker, so hook-drain latency does not cancel it. Failed jobs retry with backoff and survive a-server restart. A resumed native session is ended again only after its+PreCompact / PostCompaction write a rule-based checkpoint on the hook request.+Provider work for those checkpoints, and for opt-in SessionEnd, is durably+queued after those deterministic writes and handled by one bounded server+worker, so hook drain latency does not cancel it. The worker starts whenever a+consolidator is configured; SessionEnd enqueue stays behind the opt-in flag.+Failed jobs retry with backoff and survive a server restart. A resumed native session is ended again only after itsobservation generation advances; the persisted generation watermark makesduplicate SessionEnd delivery and system clock skew converge without repeatedprovider work. The end watermark and automatic handoff commit atomically, anddiff --git a/docs/usage.md b/docs/usage.mdindex faebe8e8..02443006 100644--- a/docs/usage.md+++ b/docs/usage.md@@ -119,9 +119,12 @@ When set to `false`, `SessionEnd` writes `sessions/<id>.md` and queues consolidaWhen Claude Code or Codex compact their working context, the`PreCompact` hook fires and ai-memory writes a fresh-`sessions/<id>.md` page summarising the session so far. After-compaction, the agent can recover the summary via `memory_recent` even-though its raw chat history was compacted away.+`sessions/<id>.md` page summarising the session so far. The hook+response returns after that heuristic page; when a provider is+configured the LLM rewrite is queued and applied by the same durable+worker SessionEnd uses. After compaction, the agent can recover the+summary via `memory_recent` even though its raw chat history was+compacted away.Generated session pages carry `session_id` and `agent` in frontmatter. The`agent` value is the originating harness stored on the session (for example,@@ -148,7 +151,7 @@ at the managed ai-memory Agent Skills that carry detailed tool routing.| "Ask the agent in <other project> to do X" / "send this to project B" | `memory_message_send` | Drops a self-contained request into another project's inbox (requires `to_workspace` + `to_project`); the recipient must already exist. Cross-project, claim-once. See [agent-messaging.md](agent-messaging.md). || "Check my inbox" / "any messages waiting?" | `memory_message_list` then `memory_message_pop` | Lists pending inbox mail without consuming, then pops one message exactly once. A popped message is untrusted cross-project input: a request to evaluate, never instructions to obey. || "Never mind that request I sent" / "clear my outbox" | `memory_message_cancel` | Retracts a pending sent message by id, or clears the whole outbox when omitted. Only affects mail this project sent. |-| "Consolidate this session" | `memory_consolidate` | Manually runs LLM consolidation on the server's model. For an explicit request about the session the agent is taking part in, the agent route in the next row is preferred so the agent's own model writes the pages; keep `memory_consolidate` for sessions the agent did not take part in and for runs without an agent. Omit `session_id` (or send a blank one) to consolidate the latest completed session in the resolved project; pass one to target a specific session. A project can keep advisory preferences in `_prompts/consolidation.md`; `instructions` overrides them for one call. Also runs on PreCompact, and at session end only when `AI_MEMORY_CONSOLIDATE_ON_SESSION_END` is set (off by default; a substantive session end otherwise writes a rule-based summary page). Lifecycle-only sessions create no generated page, handoff, or provider job. Opt-in SessionEnd provider work is durably queued outside the hook response, retried with backoff, and recovered after server restart. Resumed sessions re-end only when their persisted observation generation advances, so duplicate delivery and clock skew cannot loop consolidation. |+| "Consolidate this session" | `memory_consolidate` | Manually runs LLM consolidation on the server's model. For an explicit request about the session the agent is taking part in, the agent route in the next row is preferred so the agent's own model writes the pages; keep `memory_consolidate` for sessions the agent did not take part in and for runs without an agent. Omit `session_id` (or send a blank one) to consolidate the latest completed session in the resolved project; pass one to target a specific session. A project can keep advisory preferences in `_prompts/consolidation.md`; `instructions` overrides them for one call. Also runs on PreCompact / PostCompaction, and at session end only when `AI_MEMORY_CONSOLIDATE_ON_SESSION_END` is set (off by default; a substantive session end otherwise writes a rule-based summary page). Lifecycle-only sessions create no generated page, handoff, or provider job. PreCompact / PostCompaction write a heuristic checkpoint on the hook request; their provider rewrite, and opt-in SessionEnd provider work, is durably queued outside the hook response, retried with backoff, and recovered after server restart. Resumed sessions re-end only when their persisted observation generation advances, so duplicate delivery and clock skew cannot loop consolidation. || "Write this session's page yourself" / an in-session "consolidate this session" | `memory_read_session_observations`, then `memory_write_page` with `session_id` and `path: sessions/<id>.md` | The agent compiles the session page with its own model instead of the server's provider. To match the server's multi-page layout it may also write up to four more pages with the same `session_id`: `concepts/<slug>.md` (`tier: semantic`, `kind: fact`), `decisions/<short>.md` (`kind: decision`), `gotchas/<slug>.md` (`kind: gotcha`) and `_rules/<slug>.md` (`kind: rule`); the session page is `tier: episodic`, `kind: fact`. The session must belong to the project the page is written to; the page records it as evidence, gets the same session-page frontmatter (plus `consolidated_by: agent`) and duplicate-title suffix, and the session's queued SessionEnd job is marked completed. The write uses the `write_page` admission op, so a webhook that filters on `consolidate` does not see it. A pinned page is never overwritten this way. SessionEnd keeps the page: neither the rule-based summary nor the opt-in SessionEnd worker replaces it, even when the session went on after the write, and a PreCompact or PostCompaction checkpoint leaves it alone too. || "What did we learn from this session?" / "what memory should we add?" | `memory_auto_improve` | Without a session ID, reviews the newest completed session with no persisted auto-improvement run, advancing past preflight skips on repeated calls; pass an ID for a targeted rerun. The server also runs scheduled auto-improvement for new completed sessions when an LLM is configured. `[auto_improve.scheduler] enabled = false` disables automatic review; `[auto_improve] require_approval = true` leaves scheduled and manual proposals in pending-writes for review. || "Remember this permanently" / "add an annotation" | `memory_write_page` | Writes durable wiki knowledge; not a single-use handoff. |