Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(hooks): detach session-end spool drains
AkitaOnRails0332297b0cc5
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6b145a25..97a8cb9b 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]### Fixed+- Native `session-end` hooks now enqueue and return quickly, then drain the hook+ spool through a hidden detached `hook-drain` process guarded by a real+ single-flight file lock. Background drains use the new bounded+ `AI_MEMORY_HOOK_BACKGROUND_DRAIN_BUDGET_MINUTES` setting (default 5, max 60),+ while `session-start` cleanup remains synchronous and uses one shared+ `AI_MEMORY_HOOK_START_BUDGET_MINUTES` budget for lock wait plus cleanup drain.+ This supersedes the previous inline `session-end` deferred-drain note and+ `AI_MEMORY_HOOK_END_BUDGET_MINUTES` session-end flush budget.- Pi is now supported through a generated `~/.pi/agent/extensions/ai-memory.ts`TypeScript extension that combines lifecycle capture with an HTTP MCP bridge;`install-hooks --agent pi --apply` writes it, while `install-mcp --client pi`diff --git a/Cargo.lock b/Cargo.lockindex 2a8da5d7..a0c81c8c 100644--- a/Cargo.lock+++ b/Cargo.lock@@ -48,6 +48,7 @@ dependencies = ["dirs","figment","flate2",+ "fs2","getrandom 0.3.4","jiff","reqwest",@@ -948,6 +949,16 @@ dependencies = ["percent-encoding",]+[[package]]+name = "fs2"+version = "0.4.3"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "9564fc758e15025b46aa6643b1b77d047d1a56a1aea6e01002ac0c7026876213"+dependencies = [+ "libc",+ "winapi",+]+[[package]]name = "fsevent-sys"version = "4.1.0"diff --git a/Cargo.toml b/Cargo.tomlindex f6eda06f..91c55c61 100644--- a/Cargo.toml+++ b/Cargo.toml@@ -88,6 +88,7 @@ parking_lot = "0.12"notify = "8"notify-debouncer-full = "0.6"sysinfo = "0.32"+fs2 = "0.4"# Windows NTFS file-index (inode equivalent) for own-write detection.winapi-util = "0.1"diff --git a/crates/ai-memory-cli/Cargo.toml b/crates/ai-memory-cli/Cargo.tomlindex db8cc515..e428888d 100644--- a/crates/ai-memory-cli/Cargo.toml+++ b/crates/ai-memory-cli/Cargo.toml@@ -32,6 +32,7 @@ clap.workspace = truedirs.workspace = truefigment.workspace = trueflate2.workspace = true+fs2.workspace = truermcp.workspace = trueserde.workspace = trueserde_json.workspace = truediff --git a/crates/ai-memory-cli/src/cli.rs b/crates/ai-memory-cli/src/cli.rsindex 8cc3a2a4..da9c39ef 100644--- a/crates/ai-memory-cli/src/cli.rs+++ b/crates/ai-memory-cli/src/cli.rs@@ -67,6 +67,9 @@ pub enum Command {/// from stdin), avoiding a shell spawn. Used by the WindowsNative/// hook config; mirrors hooks/<agent>/<event>.sh.Hook(HookArgs),+ /// Hidden hook spool drainer used by native session-end hooks.+ #[command(hide = true, name = "hook-drain")]+ HookDrain(HookDrainArgs),/// Print MCP server registration snippets for any supported client/// (Claude Code, Codex, OpenCode, Cursor, Claude Desktop, Gemini/// CLI, OpenClaw, OMP, Pi). See docs/mcp-install.md for the full guide.@@ -1151,6 +1154,10 @@ pub struct HookArgs {pub project_strategy: Option<ProjectStrategyArg>,}+/// Arguments for hidden `hook-drain`.+#[derive(Debug, Args)]+pub struct HookDrainArgs {}+/// Arguments for `install-hooks`.#[derive(Debug, Args)]pub struct InstallHooksArgs {diff --git a/crates/ai-memory-cli/src/commands/hook.rs b/crates/ai-memory-cli/src/commands/hook.rsindex ec8ae22d..3beb871d 100644--- a/crates/ai-memory-cli/src/commands/hook.rs+++ b/crates/ai-memory-cli/src/commands/hook.rs@@ -3,10 +3,11 @@//! Reads the event payload from stdin. Instead of POSTing synchronously on the//! agent's hot path (which would block every tool call on the network and drop//! events against a slow/remote server), the event is **spooled** locally — an-//! instant write — and the spool is drained to the server at session-//! boundaries (a cleanup pass on `session-start`, the main flush on-//! `session-end`). The one synchronous request is the `session-start` handoff-//! GET, whose result is injected back into the agent as context.+//! instant write. `session-start` performs a short, lock-aware synchronous+//! cleanup pass before fetching handoff context. `session-end` returns quickly:+//! after enqueue it spawns a detached `hook-drain` process, whose stdout/stderr+//! are redirected away from the agent, and that process drains under an+//! exclusive spool lock with a longer bounded budget.//!//! See `docs/windows.md#native-hook-command-claude-code-on-windows`.@@ -20,6 +21,7 @@ use ai_memory_llm::OidcToken;use crate::cli::HookArgs;use super::hook_capture::{build_client, extract_cwd, get_handoff, marker_query_suffix};+use super::hook_drain_process;use super::hook_spool;use super::path_util::strip_windows_verbatim_prefix;@@ -31,13 +33,13 @@ use super::path_util::strip_windows_verbatim_prefix;const DEFAULT_DRAIN_TIMEOUT: Duration = Duration::from_secs(3);const DEFAULT_HANDOFF_TIMEOUT: Duration = Duration::from_secs(3);const DEFAULT_START_BUDGET: Duration = Duration::from_secs(3);-const DEFAULT_END_BUDGET: Duration = Duration::from_secs(10);+const DEFAULT_BACKGROUND_DRAIN_BUDGET: Duration = Duration::from_secs(5 * 60);const MAX_OVERRIDE_MINUTES: u64 = 60;const DRAIN_TIMEOUT_ENV: &str = "AI_MEMORY_HOOK_DRAIN_TIMEOUT_MINUTES";const HANDOFF_TIMEOUT_ENV: &str = "AI_MEMORY_HOOK_HANDOFF_TIMEOUT_MINUTES";const START_BUDGET_ENV: &str = "AI_MEMORY_HOOK_START_BUDGET_MINUTES";-const END_BUDGET_ENV: &str = "AI_MEMORY_HOOK_END_BUDGET_MINUTES";+const BACKGROUND_DRAIN_BUDGET_ENV: &str = "AI_MEMORY_HOOK_BACKGROUND_DRAIN_BUDGET_MINUTES";const INCREMENTAL_THRESHOLD_ENV: &str = "AI_MEMORY_HOOK_INCREMENTAL_THRESHOLD";/// Backlog size at which `post-tool-use` does a mid-session catch-up drain, so a@@ -62,10 +64,10 @@ fn handoff_timeout() -> Duration {fn start_drain_budget() -> Duration {start_drain_budget_from(env_lookup)}-/// Total budget for the `session-end` flush (the main delivery point; a session-/// boundary tolerates more). Env: `AI_MEMORY_HOOK_END_BUDGET_MINUTES`.-fn end_drain_budget() -> Duration {- end_drain_budget_from(env_lookup)+/// Total budget for detached background drains. Env:+/// `AI_MEMORY_HOOK_BACKGROUND_DRAIN_BUDGET_MINUTES`.+fn background_drain_budget() -> Duration {+ background_drain_budget_from(env_lookup)}fn drain_event_timeout_from(lookup: impl FnMut(&str) -> Option<String>) -> Duration {@@ -80,8 +82,12 @@ fn start_drain_budget_from(lookup: impl FnMut(&str) -> Option<String>) -> Duratienv_minutes(START_BUDGET_ENV, DEFAULT_START_BUDGET, lookup)}-fn end_drain_budget_from(lookup: impl FnMut(&str) -> Option<String>) -> Duration {- env_minutes(END_BUDGET_ENV, DEFAULT_END_BUDGET, lookup)+fn background_drain_budget_from(lookup: impl FnMut(&str) -> Option<String>) -> Duration {+ env_minutes(+ BACKGROUND_DRAIN_BUDGET_ENV,+ DEFAULT_BACKGROUND_DRAIN_BUDGET,+ lookup,+ )}/// Backlog size at which `post-tool-use` triggers a mid-session catch-up drain.@@ -105,36 +111,35 @@ fn should_incremental_drain(event: &str, spool_len: usize, threshold: usize) ->event == "post-tool-use" && spool_len >= threshold}-/// When the `session-end` drain cannot clear the spool, return a concise,-/// actionable note. `None` when nothing remains queued or dropped, so a normal-/// session stays silent. The caller writes this to **stderr** (never stdout,-/// which carries the hook's JSON protocol): mirroring the spool's at-capacity-/// warning, an expected-but-noteworthy backlog is surfaced rather than left-/// silent, and the message names the two knobs the operator can turn instead of-/// the bare, scary cancelled-hook symptom.-fn session_end_deferred_note(result: &hook_spool::DrainResult) -> Option<String> {- if result.remaining == 0 && result.dropped == 0 {- return None;- }+fn spawn_session_end_drainer(data_dir: &Path) -> std::io::Result<()> {+ hook_drain_process::spawn(data_dir)+}- let mut note = format!(- "ai-memory: session-end flushed {} event(s); {} still queued for a later \- session boundary.",- result.sent, result.remaining,- );- if result.dropped > 0 {- note.push_str(&format!(- " {} event(s) were dropped as undeliverable after exhausting the retry budget.",- result.dropped,- ));- } else {- note.push_str(" No queued data was lost.");+fn after_session_end_enqueue(+ data_dir: &Path,+ spawn: impl FnOnce(&Path) -> std::io::Result<()>,+) -> std::io::Result<()> {+ spawn(data_dir)+}++/// Hidden drain-only fast path. Reads no stdin and writes no stdout.+pub async fn run_drain(data_dir: Option<PathBuf>) -> anyhow::Result<()> {+ let dd = resolve_data_dir(data_dir.as_deref());+ let spool = hook_spool::spool_dir(&dd);+ match hook_spool::drain_until_quiescent(+ &spool,+ &dd,+ background_drain_budget(),+ drain_event_timeout(),+ hook_spool::DrainLockWait::Bounded(Duration::from_secs(30)),+ )+ .await+ {+ Ok(hook_spool::LockedDrainResult::Drained(_))+ | Ok(hook_spool::LockedDrainResult::LockBusy) => {}+ Err(err) => eprintln!("ai-memory hook-drain warning: failed to acquire drain lock: {err}"),}- note.push_str(&format!(- " Raise {END_BUDGET_ENV} (whole minutes), lower {INCREMENTAL_THRESHOLD_ENV}, \- or check server reachability to keep the backlog bounded."- ));- Some(note)+ Ok(())}fn env_lookup(name: &str) -> Option<String> {@@ -171,6 +176,28 @@ fn parse_minutes(raw: Option<String>, default: Duration) -> Duration {pub async fn run(data_dir: Option<PathBuf>, args: HookArgs) -> anyhow::Result<()> {let mut payload = String::new();std::io::stdin().read_to_string(&mut payload).ok();+ let mut stdout = std::io::stdout();+ run_with_payload(+ data_dir,+ args,+ payload,+ &mut stdout,+ spawn_session_end_drainer,+ )+ .await+}++async fn run_with_payload<W, S>(+ data_dir: Option<PathBuf>,+ args: HookArgs,+ payload: String,+ stdout: &mut W,+ spawn_session_end_drainer: S,+) -> anyhow::Result<()>+where+ W: std::io::Write,+ S: FnOnce(&Path) -> std::io::Result<()>,+{let json: serde_json::Value = serde_json::from_str(&payload).unwrap_or(serde_json::Value::Null);let qs = extract_cwd(&json)@@ -212,11 +239,12 @@ pub async fn run(data_dir: Option<PathBuf>, args: HookArgs) -> anyhow::Result<()hook_spool::spool_len(&spool),incremental_drain_threshold(),) {- let _ = hook_spool::drain(+ let _ = hook_spool::drain_exclusive(&spool,&dd,INCREMENTAL_DRAIN_BUDGET,INCREMENTAL_DRAIN_BUDGET,+ hook_spool::DrainLockWait::NoWait,).await;}@@ -224,7 +252,13 @@ pub async fn run(data_dir: Option<PathBuf>, args: HookArgs) -> anyhow::Result<()// session-start: drain any backlog (e.g. from a previous session that ended// abruptly), then fetch + inject the pending handoff for the resuming agent.if args.event == "session-start" {- let _ = hook_spool::drain(&spool, &dd, start_drain_budget(), drain_event_timeout()).await;+ let _ = hook_spool::drain_exclusive_within_budget(+ &spool,+ &dd,+ start_drain_budget(),+ drain_event_timeout(),+ )+ .await;// Only fetch the handoff for agents that inject the session-start// hook's stdout as context. Grok ignores it, so fetching here would// consume the handoff server-side (the GET is destructive) and then@@ -243,23 +277,23 @@ pub async fn run(data_dir: Option<PathBuf>, args: HookArgs) -> anyhow::Result<()"additionalContext": handoff,}});- println!("{envelope}");+ writeln!(stdout, "{envelope}")?;return Ok(());}}}- // session-end: the main delivery point — flush the session's spooled- // observations (oldest-first) so the server has them before it consolidates.- if args.event == "session-end" {- let result =- hook_spool::drain(&spool, &dd, end_drain_budget(), drain_event_timeout()).await;- if let Some(note) = session_end_deferred_note(&result) {- eprintln!("{note}");- }+ // session-end: enqueue and hand delivery to a detached native drainer so the+ // agent quit path is not cancelled while the server processes backlog.+ if args.event == "session-end"+ && let Err(err) = after_session_end_enqueue(&dd, spawn_session_end_drainer)+ {+ eprintln!(+ "ai-memory hook warning: failed to start background spool drainer; event remains queued: {err}"+ );}- println!("{{}}");+ writeln!(stdout, "{{}}")?;Ok(())}@@ -337,59 +371,6 @@ mod tests {);}- #[test]- fn session_end_note_is_silent_when_nothing_deferred() {- // A fully-drained boundary (the normal case) must stay quiet so a light- // session never prints a spurious warning.- let clean = hook_spool::DrainResult {- sent: 12,- remaining: 0,- dropped: 0,- };- assert!(session_end_deferred_note(&clean).is_none());- }-- #[test]- fn session_end_note_reports_deferred_backlog_and_knobs() {- // The heavy-session condition from issue #130: a boundary drain- // delivers some events and leaves the rest queued. The note must report- // both counts and name the two knobs the issue's own workaround used.- let backlog = hook_spool::DrainResult {- sent: 500,- remaining: 1384,- dropped: 0,- };- let note = session_end_deferred_note(&backlog).expect("a backlog must produce a note");- assert!(note.contains("500"), "reports how many were flushed");- assert!(note.contains("1384"), "reports how many were deferred");- assert!(- note.contains(END_BUDGET_ENV),- "points at the session-end budget knob"- );- assert!(- note.contains(INCREMENTAL_THRESHOLD_ENV),- "points at the mid-session threshold knob"- );- assert!(note.contains("No queued data was lost"));- }-- #[test]- fn session_end_note_reports_dropped_events_without_promising_no_loss() {- let result = hook_spool::DrainResult {- sent: 4,- remaining: 2,- dropped: 1,- };-- let note = session_end_deferred_note(&result).expect("drops must produce a note");-- assert!(note.contains("4"), "reports delivered count");- assert!(note.contains("2"), "reports queued count");- assert!(note.contains("1"), "reports dropped count");- assert!(note.contains("dropped as undeliverable"));- assert!(!note.contains("No queued data was lost"));- }-#[test]fn parse_minutes_falls_back_on_invalid() {assert_eq!(@@ -431,6 +412,111 @@ mod tests {);}+ #[test]+ fn background_drain_budget_defaults_and_clamps() {+ assert_eq!(+ background_drain_budget_from(|_| None),+ DEFAULT_BACKGROUND_DRAIN_BUDGET+ );+ assert_eq!(+ background_drain_budget_from(|_| Some("1".into())),+ Duration::from_secs(60)+ );+ assert_eq!(+ background_drain_budget_from(|_| Some("999".into())),+ Duration::from_secs(60 * 60)+ );+ }++ #[tokio::test]+ async fn session_end_run_enqueues_outputs_empty_json_and_spawns_after_enqueue() {+ let tmp = tempfile::tempdir().unwrap();+ let data_dir = tmp.path().to_path_buf();+ let spool = hook_spool::spool_dir(&data_dir);+ let called = std::cell::Cell::new(0);+ let mut stdout = Vec::new();+ let args = HookArgs {+ event: "session-end".into(),+ agent: "claude-code".into(),+ server_url: "http://127.0.0.1:1".into(),+ auth_token: None,+ project_strategy: None,+ };++ run_with_payload(+ Some(data_dir.clone()),+ args,+ r#"{"session_id":"s","cwd":"/tmp"}"#.into(),+ &mut stdout,+ |path| {+ assert_eq!(path, data_dir.as_path());+ assert_eq!(hook_spool::spool_len(&spool), 1, "spawn runs after enqueue");+ called.set(called.get() + 1);+ Ok(())+ },+ )+ .await+ .unwrap();++ assert_eq!(stdout, b"{}\n");+ assert_eq!(called.get(), 1);+ assert_eq!(+ hook_spool::spool_len(&spool),+ 1,+ "session-end must not drain inline"+ );+ }++ #[tokio::test]+ async fn session_end_run_spawn_failure_keeps_event_queued_and_stdout_clean() {+ let tmp = tempfile::tempdir().unwrap();+ let data_dir = tmp.path().to_path_buf();+ let spool = hook_spool::spool_dir(&data_dir);+ let mut stdout = Vec::new();+ let args = HookArgs {+ event: "session-end".into(),+ agent: "claude-code".into(),+ server_url: "http://127.0.0.1:1".into(),+ auth_token: None,+ project_strategy: None,+ };++ run_with_payload(Some(data_dir), args, "{}".into(), &mut stdout, |_path| {+ Err(std::io::Error::other("spawn failed"))+ })+ .await+ .unwrap();++ assert_eq!(stdout, b"{}\n");+ assert_eq!(hook_spool::spool_len(&spool), 1);+ }++ #[test]+ fn session_end_spawn_failure_is_returned_for_warning_only() {+ let tmp = tempfile::tempdir().unwrap();+ let err = after_session_end_enqueue(tmp.path(), |_path| {+ Err(std::io::Error::other("spawn failed"))+ })+ .unwrap_err();++ assert_eq!(err.kind(), std::io::ErrorKind::Other);+ }++ #[test]+ fn session_end_policy_spawns_without_inline_drain() {+ let tmp = tempfile::tempdir().unwrap();+ let called = std::cell::Cell::new(false);++ after_session_end_enqueue(tmp.path(), |path| {+ assert_eq!(path, tmp.path());+ called.set(true);+ Ok(())+ })+ .unwrap();++ assert!(called.get());+ }+#[test]fn timing_accessors_read_the_expected_env_vars() {fn one_minute_for(expected_name: &'static str) -> impl FnMut(&str) -> Option<String> {@@ -453,7 +539,7 @@ mod tests {Duration::from_secs(60));assert_eq!(- end_drain_budget_from(one_minute_for(END_BUDGET_ENV)),+ background_drain_budget_from(one_minute_for(BACKGROUND_DRAIN_BUDGET_ENV)),Duration::from_secs(60));}diff --git a/crates/ai-memory-cli/src/commands/hook_drain_process.rs b/crates/ai-memory-cli/src/commands/hook_drain_process.rsnew file mode 100644index 00000000..87a50dae--- /dev/null+++ b/crates/ai-memory-cli/src/commands/hook_drain_process.rs@@ -0,0 +1,241 @@+//! Detached native process launcher for background hook-spool drains.++use std::ffi::OsString;+use std::fs::OpenOptions;+use std::io::{self, Write as _};+use std::path::{Path, PathBuf};+use std::process::{Command, Stdio};++/// Unit-testable description of the hidden drainer command.+#[derive(Debug, Clone, PartialEq, Eq)]+pub struct DrainCommandSpec {+ /// Executable path.+ pub exe: PathBuf,+ /// Command arguments.+ pub args: Vec<OsString>,+ /// Stderr log file under `<data_dir>/logs`.+ pub stderr_log: PathBuf,+}++/// Unit-testable stdio/detach configuration for a drainer process.+#[derive(Debug, Clone, PartialEq, Eq)]+pub struct SpawnConfig {+ /// stdin is redirected to null.+ pub stdin_null: bool,+ /// stdout is redirected to null.+ pub stdout_null: bool,+ /// stderr is redirected to this file.+ pub stderr_file: PathBuf,+ /// Platform detach flags applied to the command.+ pub detach: DetachConfig,+}++/// Platform detach configuration.+#[derive(Debug, Clone, PartialEq, Eq)]+pub enum DetachConfig {+ /// Unix starts a new process group.+ #[cfg(unix)]+ UnixProcessGroupZero,+ /// Windows creation flags.+ #[cfg(windows)]+ WindowsCreationFlags(u32),+ /// No extra detach flags on unsupported platforms.+ #[cfg(not(any(unix, windows)))]+ None,+}++/// Build `ai-memory --data-dir <dir> hook-drain`.+pub fn command_spec(data_dir: &Path) -> io::Result<DrainCommandSpec> {+ Ok(DrainCommandSpec {+ exe: std::env::current_exe()?,+ args: vec![+ OsString::from("--data-dir"),+ data_dir.as_os_str().to_os_string(),+ OsString::from("hook-drain"),+ ],+ stderr_log: data_dir.join("logs").join("hook-drain.log"),+ })+}++/// Build the stdio/detach shape used when spawning.+#[must_use]+pub fn spawn_config(spec: &DrainCommandSpec, try_breakaway: bool) -> SpawnConfig {+ SpawnConfig {+ stdin_null: true,+ stdout_null: true,+ stderr_file: spec.stderr_log.clone(),+ detach: detach_config(try_breakaway),+ }+}++/// Spawn the hidden drainer without inheriting hook stdio.+pub fn spawn(data_dir: &Path) -> io::Result<()> {+ let spec = command_spec(data_dir)?;+ spawn_spec(&spec)+}++fn spawn_spec(spec: &DrainCommandSpec) -> io::Result<()> {+ if let Some(parent) = spec.stderr_log.parent() {+ std::fs::create_dir_all(parent)?;+ }+ let config = spawn_config(spec, true);+ let stderr = OpenOptions::new()+ .create(true)+ .append(true)+ .open(&config.stderr_file)?;++ let mut command = Command::new(&spec.exe);+ command+ .args(&spec.args)+ .stdin(Stdio::null())+ .stdout(Stdio::null())+ .stderr(Stdio::from(stderr));+ apply_detach_flags(&mut command, &config.detach);++ match command.spawn() {+ Ok(_child) => Ok(()),+ Err(err) if should_retry_without_breakaway(&err) => {+ let mut log = OpenOptions::new()+ .create(true)+ .append(true)+ .open(&spec.stderr_log)?;+ let _ = writeln!(+ log,+ "ai-memory hook-drain: breakaway spawn failed with access denied; retrying without breakaway"+ );+ let config = spawn_config(spec, false);+ let stderr = OpenOptions::new()+ .create(true)+ .append(true)+ .open(&config.stderr_file)?;+ let mut retry = Command::new(&spec.exe);+ retry+ .args(&spec.args)+ .stdin(Stdio::null())+ .stdout(Stdio::null())+ .stderr(Stdio::from(stderr));+ apply_detach_flags(&mut retry, &config.detach);+ retry.spawn().map(|_| ())+ }+ Err(err) => Err(err),+ }+}++#[cfg(unix)]+fn detach_config(_try_breakaway: bool) -> DetachConfig {+ DetachConfig::UnixProcessGroupZero+}++#[cfg(unix)]+fn apply_detach_flags(command: &mut Command, detach: &DetachConfig) {+ use std::os::unix::process::CommandExt as _;+ if matches!(detach, DetachConfig::UnixProcessGroupZero) {+ command.process_group(0);+ }+}++#[cfg(windows)]+fn detach_config(try_breakaway: bool) -> DetachConfig {+ const DETACHED_PROCESS: u32 = 0x0000_0008;+ const CREATE_NEW_PROCESS_GROUP: u32 = 0x0000_0200;+ const CREATE_NO_WINDOW: u32 = 0x0800_0000;+ const CREATE_BREAKAWAY_FROM_JOB: u32 = 0x0100_0000;++ let mut flags = DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP | CREATE_NO_WINDOW;+ if try_breakaway {+ flags |= CREATE_BREAKAWAY_FROM_JOB;+ }+ DetachConfig::WindowsCreationFlags(flags)+}++#[cfg(windows)]+fn apply_detach_flags(command: &mut Command, detach: &DetachConfig) {+ use std::os::windows::process::CommandExt as _;+ if let DetachConfig::WindowsCreationFlags(flags) = detach {+ command.creation_flags(*flags);+ }+}++#[cfg(not(any(unix, windows)))]+fn detach_config(_try_breakaway: bool) -> DetachConfig {+ DetachConfig::None+}++#[cfg(not(any(unix, windows)))]+fn apply_detach_flags(_command: &mut Command, _detach: &DetachConfig) {}++#[cfg(not(windows))]+fn should_retry_without_breakaway(_err: &io::Error) -> bool {+ false+}++#[cfg(windows)]+fn should_retry_without_breakaway(err: &io::Error) -> bool {+ err.raw_os_error() == Some(5)+}++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn command_spec_uses_data_dir_then_hidden_subcommand() {+ let tmp = tempfile::tempdir().unwrap();+ let spec = command_spec(tmp.path()).unwrap();++ assert_eq!(+ spec.args,+ vec![+ OsString::from("--data-dir"),+ tmp.path().as_os_str().to_os_string(),+ OsString::from("hook-drain"),+ ]+ );+ assert_eq!(+ spec.stderr_log,+ tmp.path().join("logs").join("hook-drain.log")+ );+ }++ #[test]+ fn spawn_config_redirects_stdio_and_logs_under_data_dir_logs() {+ let tmp = tempfile::tempdir().unwrap();+ let spec = command_spec(tmp.path()).unwrap();+ let config = spawn_config(&spec, true);++ assert!(config.stdin_null);+ assert!(config.stdout_null);+ assert_eq!(+ config.stderr_file,+ tmp.path().join("logs").join("hook-drain.log")+ );+ }++ #[cfg(unix)]+ #[test]+ fn unix_spawn_config_uses_process_group_zero() {+ let tmp = tempfile::tempdir().unwrap();+ let spec = command_spec(tmp.path()).unwrap();+ assert_eq!(+ spawn_config(&spec, true).detach,+ DetachConfig::UnixProcessGroupZero+ );+ }++ #[cfg(windows)]+ #[test]+ fn windows_spawn_config_uses_expected_flags_and_breakaway_toggle() {+ let tmp = tempfile::tempdir().unwrap();+ let spec = command_spec(tmp.path()).unwrap();+ let DetachConfig::WindowsCreationFlags(with_breakaway) = spawn_config(&spec, true).detach+ else {+ panic!("windows flags")+ };+ let DetachConfig::WindowsCreationFlags(without_breakaway) =+ spawn_config(&spec, false).detach+ else {+ panic!("windows flags")+ };+ assert_ne!(with_breakaway, without_breakaway);+ }+}diff --git a/crates/ai-memory-cli/src/commands/hook_spool.rs b/crates/ai-memory-cli/src/commands/hook_spool.rsindex da786b97..e2c02aad 100644--- a/crates/ai-memory-cli/src/commands/hook_spool.rs+++ b/crates/ai-memory-cli/src/commands/hook_spool.rs@@ -18,10 +18,12 @@//! `auth.json` at drain time (so a token that expired while the event waited is//! renewed rather than rejected).+use std::fs::File;use std::path::{Path, PathBuf};use std::sync::atomic::{AtomicU64, Ordering};use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};+use fs2::FileExt as _;use serde::{Deserialize, Serialize};use super::hook_capture::{BatchOutcome, PostOutcome, build_client, post_batch, post_hook};@@ -183,6 +185,160 @@ pub struct DrainResult {pub dropped: usize,}+/// How long to wait for the exclusive drain lock.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub enum DrainLockWait {+ /// Do not wait if another drainer is active.+ NoWait,+ /// Poll for the lock until this bounded budget expires.+ Bounded(Duration),+}++/// An exclusive hook-spool drain lock. The OS releases it when dropped.+pub struct DrainLock {+ _file: File,+}++/// Outcome for lock-aware drain wrappers.+#[derive(Debug, PartialEq, Eq)]+pub enum LockedDrainResult {+ /// The lock was acquired and a drain ran.+ Drained(DrainResult),+ /// Another process is draining; this is expected and not an error.+ LockBusy,+}++/// Try to acquire the single-flight drain lock.+pub fn acquire_drain_lock(spool: &Path, wait: DrainLockWait) -> std::io::Result<Option<DrainLock>> {+ std::fs::create_dir_all(spool)?;+ let path = spool.join(".drain.lock");+ let file = std::fs::OpenOptions::new()+ .create(true)+ .truncate(false)+ .read(true)+ .write(true)+ .open(path)?;+ let started = Instant::now();+ loop {+ match file.try_lock_exclusive() {+ Ok(()) => return Ok(Some(DrainLock { _file: file })),+ Err(err) if err.kind() == std::io::ErrorKind::WouldBlock => match wait {+ DrainLockWait::NoWait => return Ok(None),+ DrainLockWait::Bounded(limit) if started.elapsed() >= limit => return Ok(None),+ DrainLockWait::Bounded(limit) => {+ let sleep =+ Duration::from_millis(25).min(limit.saturating_sub(started.elapsed()));+ if sleep.is_zero() {+ return Ok(None);+ }+ std::thread::sleep(sleep);+ }+ },+ Err(err) => return Err(err),+ }+ }+}++/// Run one exclusive drain pass if the single-flight lock is available.+pub async fn drain_exclusive(+ spool: &Path,+ data_dir: &Path,+ total_budget: Duration,+ per_event_timeout: Duration,+ wait: DrainLockWait,+) -> Option<DrainResult> {+ match drain_exclusive_result(spool, data_dir, total_budget, per_event_timeout, wait).await {+ Ok(LockedDrainResult::Drained(result)) => Some(result),+ Ok(LockedDrainResult::LockBusy) | Err(_) => None,+ }+}++/// Run one exclusive drain pass, distinguishing lock contention from lock IO errors.+pub async fn drain_exclusive_result(+ spool: &Path,+ data_dir: &Path,+ total_budget: Duration,+ per_event_timeout: Duration,+ wait: DrainLockWait,+) -> std::io::Result<LockedDrainResult> {+ let Some(_lock) = acquire_drain_lock(spool, wait)? else {+ return Ok(LockedDrainResult::LockBusy);+ };+ Ok(LockedDrainResult::Drained(+ drain(spool, data_dir, total_budget, per_event_timeout).await,+ ))+}++/// Run one exclusive drain pass while treating lock wait + drain as one budget.+pub async fn drain_exclusive_within_budget(+ spool: &Path,+ data_dir: &Path,+ total_budget: Duration,+ per_event_timeout: Duration,+) -> std::io::Result<LockedDrainResult> {+ let started = Instant::now();+ let Some(_lock) = acquire_drain_lock(spool, DrainLockWait::Bounded(total_budget))? else {+ return Ok(LockedDrainResult::LockBusy);+ };+ let Some(remaining_budget) = total_budget.checked_sub(started.elapsed()) else {+ return Ok(LockedDrainResult::Drained(DrainResult {+ remaining: spool_len(spool),+ ..DrainResult::default()+ }));+ };+ if remaining_budget.is_zero() {+ return Ok(LockedDrainResult::Drained(DrainResult {+ remaining: spool_len(spool),+ ..DrainResult::default()+ }));+ }+ Ok(LockedDrainResult::Drained(+ drain(spool, data_dir, remaining_budget, per_event_timeout).await,+ ))+}++/// Hold the drain lock and keep draining until the spool is quiescent or budget expires.+pub async fn drain_until_quiescent(+ spool: &Path,+ data_dir: &Path,+ total_budget: Duration,+ per_event_timeout: Duration,+ wait: DrainLockWait,+) -> std::io::Result<LockedDrainResult> {+ let Some(_lock) = acquire_drain_lock(spool, wait)? else {+ return Ok(LockedDrainResult::LockBusy);+ };+ let started = Instant::now();+ let mut combined = DrainResult::default();++ loop {+ let Some(remaining_budget) = total_budget.checked_sub(started.elapsed()) else {+ combined.remaining = spool_len(spool);+ return Ok(LockedDrainResult::Drained(combined));+ };+ if remaining_budget.is_zero() {+ combined.remaining = spool_len(spool);+ return Ok(LockedDrainResult::Drained(combined));+ }++ let result = drain(spool, data_dir, remaining_budget, per_event_timeout).await;+ combined.sent += result.sent;+ combined.dropped += result.dropped;+ combined.remaining = result.remaining;++ if result.remaining > 0 {+ return Ok(LockedDrainResult::Drained(combined));+ }++ let queued = spool_len(spool);+ if queued == 0 {+ combined.remaining = 0;+ return Ok(LockedDrainResult::Drained(combined));+ }+ combined.remaining = queued;+ }+}+/// Drain the spool to the server, oldest-first, within `total_budget`.////// Events are delivered in **batches** via `POST /hook/batch`: one request@@ -708,6 +864,131 @@ mod tests {assert_eq!(r, DrainResult::default());}+ #[test]+ fn drain_lock_is_exclusive_and_released_on_drop() {+ let tmp = tempfile::tempdir().unwrap();+ let spool = spool_dir(tmp.path());++ let first = acquire_drain_lock(&spool, DrainLockWait::NoWait)+ .unwrap()+ .expect("first lock acquired");+ assert!(+ acquire_drain_lock(&spool, DrainLockWait::NoWait)+ .unwrap()+ .is_none(),+ "overlapping lock should not acquire"+ );++ drop(first);+ assert!(+ acquire_drain_lock(&spool, DrainLockWait::NoWait)+ .unwrap()+ .is_some(),+ "lock should release on drop"+ );+ }++ #[tokio::test]+ async fn drain_exclusive_skips_when_lock_busy() {+ let tmp = tempfile::tempdir().unwrap();+ let spool = spool_dir(tmp.path());+ let _held = acquire_drain_lock(&spool, DrainLockWait::NoWait)+ .unwrap()+ .expect("held lock");++ let result = drain_exclusive(+ &spool,+ tmp.path(),+ Duration::from_secs(1),+ Duration::from_millis(100),+ DrainLockWait::NoWait,+ )+ .await;++ assert!(result.is_none());+ }++ #[tokio::test]+ async fn drain_exclusive_within_budget_zero_budget_leaves_entries() {+ let tmp = tempfile::tempdir().unwrap();+ let spool = spool_dir(tmp.path());+ write_spool_entry(+ &spool,+ "evt-0.json",+ "http://127.0.0.1:1/hook?event=e0".into(),+ );++ let result = drain_exclusive_within_budget(+ &spool,+ tmp.path(),+ Duration::ZERO,+ Duration::from_millis(100),+ )+ .await+ .unwrap();+ let LockedDrainResult::Drained(result) = result else {+ panic!("lock should not be busy")+ };++ assert_eq!(result.sent, 0);+ assert_eq!(result.remaining, 1);+ assert_eq!(spool_len(&spool), 1);+ }++ #[tokio::test]+ async fn drain_until_quiescent_catches_new_entries() {+ let tmp = tempfile::tempdir().unwrap();+ let spool = spool_dir(tmp.path());+ let req_count = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));+ let addr = serve_counting_hook_and_enqueue_on_first(req_count.clone(), spool.clone()).await;++ write_spool_entry(+ &spool,+ "first.json",+ format!("http://{addr}/hook?event=first"),+ );++ let result = drain_until_quiescent(+ &spool,+ tmp.path(),+ Duration::from_secs(5),+ Duration::from_secs(2),+ DrainLockWait::NoWait,+ )+ .await+ .expect("lock acquired");+ let LockedDrainResult::Drained(result) = result else {+ panic!("lock should not be busy")+ };++ assert_eq!(result.sent, 2);+ assert_eq!(result.remaining, 0);+ assert_eq!(req_count.load(std::sync::atomic::Ordering::SeqCst), 2);+ assert!(list_entries(&spool).unwrap().is_empty());+ }++ #[tokio::test]+ async fn drain_until_quiescent_reports_lock_io_errors() {+ let tmp = tempfile::tempdir().unwrap();+ let not_a_dir = tmp.path().join("not-a-dir");+ std::fs::write(¬_a_dir, b"file").unwrap();++ let err = drain_until_quiescent(+ ¬_a_dir,+ tmp.path(),+ Duration::from_secs(1),+ Duration::from_millis(100),+ DrainLockWait::NoWait,+ )+ .await+ .unwrap_err();++ assert!(matches!(+ err.kind(),+ std::io::ErrorKind::AlreadyExists | std::io::ErrorKind::NotADirectory+ ));+ }+#[tokio::test]async fn drain_drops_event_after_max_attempts() {let tmp = tempfile::tempdir().unwrap();@@ -946,6 +1227,47 @@ mod tests {addr.to_string()}+ async fn serve_counting_hook_and_enqueue_on_first(+ req_count: std::sync::Arc<std::sync::atomic::AtomicUsize>,+ spool: PathBuf,+ ) -> String {+ use tokio::io::{AsyncReadExt as _, AsyncWriteExt as _};+ let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();+ let addr = listener.local_addr().unwrap();+ tokio::spawn(async move {+ while let Ok((mut s, _)) = listener.accept().await {+ let rc = req_count.clone();+ let spool = spool.clone();+ let addr = addr.to_string();+ tokio::spawn(async move {+ let mut buf = vec![0_u8; 65536];+ let n = s.read(&mut buf).await.unwrap_or(0);+ let previous = rc.fetch_add(1, std::sync::atomic::Ordering::SeqCst);+ if previous == 0 {+ write_spool_entry(+ &spool,+ "second.json",+ format!("http://{addr}/hook?event=second"),+ );+ }+ let req = String::from_utf8_lossy(&buf[..n]);+ let payload = req.split("\r\n\r\n").nth(1).unwrap_or("");+ let accepted = serde_json::from_str::<serde_json::Value>(payload)+ .ok()+ .and_then(|v| v.as_array().map(Vec::len))+ .unwrap_or(0);+ let body = format!("{{\"accepted\":{accepted}}}");+ let resp = format!(+ "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",+ body.len()+ );+ let _ = s.write_all(resp.as_bytes()).await;+ });+ }+ });+ addr.to_string()+ }+fn write_spool_entry(spool: &Path, name: &str, url: String) {write_spool_entry_with_body(spool, name, url, "{}".into());}diff --git a/crates/ai-memory-cli/src/commands/mod.rs b/crates/ai-memory-cli/src/commands/mod.rsindex 71493aa8..f14b1df0 100644--- a/crates/ai-memory-cli/src/commands/mod.rs+++ b/crates/ai-memory-cli/src/commands/mod.rs@@ -21,6 +21,7 @@ pub mod forget_sweep;pub mod generate_auth_token;pub mod hook;pub mod hook_capture;+pub mod hook_drain_process;pub mod hook_spool;pub mod init;pub mod install_hooks;diff --git a/crates/ai-memory-cli/src/main.rs b/crates/ai-memory-cli/src/main.rsindex 941b9747..14116b57 100644--- a/crates/ai-memory-cli/src/main.rs+++ b/crates/ai-memory-cli/src/main.rs@@ -41,6 +41,7 @@ async fn main() -> Result<()> {// the bare path rather than loading the full config.let command = match command {Command::Hook(args) => return commands::hook::run(data_dir, args).await,+ Command::HookDrain(_args) => return commands::hook::run_drain(data_dir).await,other => other,};@@ -72,6 +73,8 @@ async fn main() -> Result<()> {Command::InstallHooks(args) => commands::install_hooks::run(&config, args),// `Hook` is handled in the fast-path above (before config/tracing).Command::Hook(args) => commands::hook::run(Some(config.data_dir.clone()), args).await,+ // `HookDrain` is handled in the fast-path above (before config/tracing).+ Command::HookDrain(_args) => commands::hook::run_drain(Some(config.data_dir.clone())).await,Command::InstallMcp(args) => commands::install_mcp::run(&config, args),Command::Commit(args) => commands::commit::run(&config, args).await,Command::Checkpoints(args) => commands::checkpoints::run(&config, args).await,diff --git a/crates/ai-memory-cli/tests/hook_drain.rs b/crates/ai-memory-cli/tests/hook_drain.rsnew file mode 100644index 00000000..87b5f8b4--- /dev/null+++ b/crates/ai-memory-cli/tests/hook_drain.rs@@ -0,0 +1,34 @@+//! Smoke tests for the hidden hook spool drainer command.++use std::process::Command;++fn bin() -> &'static str {+ env!("CARGO_BIN_EXE_ai-memory")+}++#[test]+fn hidden_hook_drain_empty_spool_writes_no_stdout() {+ let tmp = tempfile::tempdir().expect("tempdir");++ let output = Command::new(bin())+ .args(["--data-dir"])+ .arg(tmp.path())+ .arg("hook-drain")+ .output()+ .expect("run hook-drain");++ assert!(output.status.success());+ assert!(output.stdout.is_empty());+}++#[test]+fn hook_drain_is_hidden_from_top_level_help() {+ let output = Command::new(bin())+ .arg("--help")+ .output()+ .expect("run help");++ assert!(output.status.success());+ let stdout = String::from_utf8(output.stdout).expect("help utf8");+ assert!(!stdout.contains("hook-drain"));+}diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex db511ed7..0329d733 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -46,8 +46,10 @@ from hook paths.1. Agent CLI emits a lifecycle hook (SessionStart, UserPromptSubmit,PostToolUse, …). Shell-script hooks `curl` event JSON to `POST /hook`with a short timeout. Native `ai-memory hook --event ...` commands spool- events locally and drain them at session boundaries; high-latency- operators can raise the drain/handoff caps with minute-based env vars.+ events locally, do a short bounded cleanup at session start, and hand+ session-end delivery to a detached lock-aware `hook-drain` helper;+ high-latency operators can raise the drain/handoff/background caps with+ minute-based env vars.Agent hot paths never block on the network; saturated servers return HTTP429 instead of queueing unbounded work.2. Server's hook router sanitises the payload (the only path fromdiff --git a/docs/install.md b/docs/install.mdindex e15d8501..eea7be7f 100644--- a/docs/install.md+++ b/docs/install.md@@ -368,17 +368,19 @@ then stages runnable copies under `~/.local/share/ai-memory/hooks/<agent>/` sothe agent can execute files owned by your user. Re-run `install-hooks --apply`after package upgrades to refresh those staged copies.-Native `ai-memory hook --event ...` commands spool events locally and drain them-at session boundaries. The built-in timings stay short by default, but-high-latency or large-backlog instances can raise them with whole-minute runtime-env vars in the agent's environment; no `install-hooks` rerun is needed:+Native `ai-memory hook --event ...` commands spool events locally. Session start+does a short bounded cleanup drain before fetching a handoff; session end starts+a detached `hook-drain` helper so agent quit paths are not blocked by a large+backlog. The built-in timings stay short on agent-facing paths, but high-latency+or large-backlog instances can raise them with whole-minute runtime env vars in+the agent's environment; no `install-hooks` rerun is needed:| Env var | Built-in default | Max override | What it caps ||---|---:|---:|---|| `AI_MEMORY_HOOK_DRAIN_TIMEOUT_MINUTES` | 3 seconds | 60 minutes | each event POST during a drain || `AI_MEMORY_HOOK_HANDOFF_TIMEOUT_MINUTES` | 3 seconds | 60 minutes | the synchronous `session-start` handoff GET |-| `AI_MEMORY_HOOK_START_BUDGET_MINUTES` | 3 seconds | 60 minutes | total time the `session-start` cleanup drain may spend |-| `AI_MEMORY_HOOK_END_BUDGET_MINUTES` | 10 seconds | 60 minutes | total time the `session-end` flush may spend |+| `AI_MEMORY_HOOK_START_BUDGET_MINUTES` | 3 seconds | 60 minutes | total time `session-start` may spend waiting for the drain lock and cleanup draining |+| `AI_MEMORY_HOOK_BACKGROUND_DRAIN_BUDGET_MINUTES` | 5 minutes | 60 minutes | total time the detached `hook-drain` helper may spend after `session-end` || `AI_MEMORY_HOOK_INCREMENTAL_THRESHOLD` | 32 events | positive integer | spool backlog size that triggers a 250 ms `post-tool-use` catch-up drain |Timing values must be positive whole minutes. Missing, empty, non-numeric, ordiff --git a/docs/macos.md b/docs/macos.mdindex 0ec422db..abaf2847 100644--- a/docs/macos.md+++ b/docs/macos.md@@ -124,9 +124,9 @@ Apple Silicon pulls the native arm64 image without `--platform linux/amd64`.two relevant values are `posix-native` (direct binary call; the native default)and `posix` (the bundled `.sh` scripts; the Docker-wrapper default). Set itbefore running `install-hooks` so the choice is baked into the rendered-commands. The native hook spools events locally and drains them at-session boundaries; the whole-minute spool-timing overrides are shared with-Windows and documented in+commands. The native hook spools events locally, does short session-start+cleanup, and starts a detached session-end `hook-drain` helper; the whole-minute+spool-timing overrides are shared with Windows and documented in[`docs/windows.md`](windows.md#tuning-the-spool-timings-high-latency-instances).## Troubleshooting on macOSdiff --git a/docs/windows.md b/docs/windows.mdindex d050ceb1..09731155 100644--- a/docs/windows.md+++ b/docs/windows.md@@ -232,8 +232,10 @@ boundary before startup healing or cwd-prefix matching.### Tuning the spool timings (high-latency instances)-The native hook spools events locally and drains them to the server at session-boundaries. The built-in timings stay short by default, but high-latency or+The native hook spools events locally. Session start does a short bounded cleanup+drain before fetching a handoff; session end starts a detached `hook-drain`+helper so Claude Code and other agents are not kept open by a large backlog. The+built-in timings stay short on agent-facing paths, but high-latency orlarge-backlog instances can raise them with whole-minute overrides. Unlike`AI_MEMORY_HOOK_PLATFORM`, these are read by the hook **at runtime**, so theyapply to the agent's environment (no re-`install-hooks` needed):@@ -242,15 +244,16 @@ apply to the agent's environment (no re-`install-hooks` needed):|---|---:|---:|---|| `AI_MEMORY_HOOK_DRAIN_TIMEOUT_MINUTES` | 3 seconds | 60 minutes | each event POST during a drain || `AI_MEMORY_HOOK_HANDOFF_TIMEOUT_MINUTES` | 3 seconds | 60 minutes | the synchronous `session-start` handoff GET |-| `AI_MEMORY_HOOK_START_BUDGET_MINUTES` | 3 seconds | 60 minutes | total time the `session-start` cleanup drain may spend |-| `AI_MEMORY_HOOK_END_BUDGET_MINUTES` | 10 seconds | 60 minutes | total time the `session-end` flush may spend |+| `AI_MEMORY_HOOK_START_BUDGET_MINUTES` | 3 seconds | 60 minutes | total time `session-start` may spend waiting for the drain lock and cleanup draining |+| `AI_MEMORY_HOOK_BACKGROUND_DRAIN_BUDGET_MINUTES` | 5 minutes | 60 minutes | total time the detached `hook-drain` helper may spend after `session-end` || `AI_MEMORY_HOOK_INCREMENTAL_THRESHOLD` | 32 events | positive integer | spool backlog size that triggers a 250 ms `post-tool-use` catch-up drain |Timing values must be positive whole minutes. Missing, empty, non-numeric, orzero values fall back to the built-in defaults; values above 60 are clamped. Theincremental threshold is a positive event count; invalid values fall back to 32.-The budgets cap how long a session boundary blocks, so leave headroom over the-per-request timeout for several events to drain per boundary.+The session-start budget caps how long the hook may block before handoff fetch;+the background budget caps detached cleanup after session-end and does not keep+the agent waiting.## Current Harness Caveats