Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(store): bound the SQLite WAL after VACUUM and close the store on shutdown
Djalma Júnior67e386e30737
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 53f77b46..73670613 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -129,6 +129,18 @@ version in [`docs/changelog/`](docs/changelog/):were not re-stamped into the target project, which distorted cross-projectprofile convergence and caused a subsequent `purge-project` of the sourceproject to destroy them via cascade. They now re-stamp in the same transaction. (#1265)+- Fixed the SQLite `-wal` file staying as large as the database after a+ `VACUUM`. `compact`, `reclaim-ledger-versions --compact` and the session,+ project and workspace purges with compaction now run `PRAGMA+ wal_checkpoint(TRUNCATE)` after the rewrite, and the store's writer+ connection sets `journal_size_limit` to 64 MiB. A checkpoint that a reader+ holds back is logged at `warn` and the operation still succeeds. A 29.5 GiB+ database that shrank to 6.3 GiB kept a 17.8 GB WAL, and the next restart+ spent 9 minutes replaying it. `serve` now also drops its read connections+ and checkpoints the WAL on SIGINT, SIGTERM, Ctrl-C or the end of stdio+ input, waiting at most 5 seconds; a read that arrives after that point is+ refused. A hard kill before the checkpoint still leaves the WAL to replay.+ (#PRNUM)## [3.0.0] - 2026-10-13diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rsindex 4dba5368..555e7b0d 100644--- a/crates/ai-memory-cli/src/commands/serve.rs+++ b/crates/ai-memory-cli/src/commands/serve.rs@@ -1453,6 +1453,7 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {}}}+ close_store(&store).await;}TransportKind::Http => {seed_active_project_fallback(&store.reader, &active_project).await;@@ -1975,6 +1976,7 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {),}}+ close_store(&store).await;if let Some(serve_result) = serve_result {serve_result?;}@@ -1983,6 +1985,32 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {Ok(())}+/// Checkpoint the WAL and close the store's connections on the way out.+///+/// `main` ends the process without joining the store writer (detached tasks+/// still hold clones of its handle), so the connection is never closed and the+/// whole `-wal` is left for the next start to replay: 9 minutes for a 17.8 GB+/// one after a `VACUUM`. Bounded by [`SHUTDOWN_GRACE`] because the close queues+/// behind whatever the writer is running, and a stuck query must not turn a+/// stop back into a SIGKILL.+async fn close_store(store: &Store) {+ close_store_within(store, SHUTDOWN_GRACE).await;+}++/// [`close_store`] with the deadline as a parameter, so a test can pin the+/// bound without waiting out the real one. `true` when the store closed in+/// time.+async fn close_store_within(store: &Store, grace: Duration) -> bool {+ if tokio::time::timeout(grace, store.close()).await.is_err() {+ tracing::warn!(+ grace_secs = grace.as_secs(),+ "store did not close within the shutdown grace period; exiting anyway"+ );+ return false;+ }+ true+}+#[allow(clippy::too_many_arguments)]async fn start_maintenance_scheduler(settings: MaintenanceSettings,@@ -3331,6 +3359,35 @@ mod tests {use tempfile::TempDir;use tower::ServiceExt;+ /// The shutdown close queues behind whatever the writer is running, so the+ /// deadline is what keeps a stuck query from turning a stop back into a+ /// SIGKILL. The writer is made busy for real: another connection holds the+ /// write lock, so the writer's next command (the close itself, or the+ /// write queued before it) waits in SQLite's 5s busy handler.+ #[tokio::test]+ async fn close_store_returns_within_its_grace_while_the_writer_is_busy() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let blocker = rusqlite::Connection::open(store.db_path()).unwrap();+ blocker.execute_batch("BEGIN IMMEDIATE").unwrap();+ let writer = store.writer.clone();+ let queued = tokio::spawn(async move { writer.get_or_create_workspace("blocked").await });++ let started = std::time::Instant::now();+ let closed = close_store_within(&store, Duration::from_millis(200)).await;+ let elapsed = started.elapsed();++ // Let the writer go before asserting, so a failure does not leave it+ // parked for the rest of its busy timeout.+ blocker.execute_batch("ROLLBACK").unwrap();+ let _ = queued.await;+ assert!(!closed, "the close cannot finish while the writer is stuck");+ assert!(+ elapsed < Duration::from_secs(3),+ "close_store waited {elapsed:?} for a stuck writer instead of giving up"+ );+ }+/// An owner-check failure of the startup wiki baseline checkpoint must be/// classified as an ERROR-worthy `OwnerFailure`, not the WARN-only/// `OtherFailure` that hid it before (#872). A generic error staysdiff --git a/crates/ai-memory-cli/tests/suite/serve_shutdown.rs b/crates/ai-memory-cli/tests/suite/serve_shutdown.rsindex 14e3cc1d..80ce2553 100644--- a/crates/ai-memory-cli/tests/suite/serve_shutdown.rs+++ b/crates/ai-memory-cli/tests/suite/serve_shutdown.rs@@ -1,8 +1,8 @@-//! `serve --transport stdio` must stop when the operator interrupts it (#699).--#![cfg(unix)]+//! `serve --transport stdio` must stop when the operator interrupts it (#699),+//! and any stop of `serve` must leave the SQLite WAL checkpointed./// Startup, one interrupt and one shutdown: seconds, not milliseconds.+#[cfg(unix)]mod slow {use std::io::{BufRead, BufReader};use std::process::{Child, Command, Stdio};@@ -136,3 +136,260 @@ mod slow {);}}++/// `serve` ends the process without joining the store writer, so the WAL a+/// server grew used to be left whole for the next start to replay (a 17.8 GB+/// one, 9 minutes). These tests go through the real shutdown path of the built+/// binary, not `Store::close`, and prove `close_store` ran: another SQLite+/// handle stays open for the whole run, so the automatic checkpoint-and-delete+/// SQLite does when the *last* connection closes cannot be what empties the+/// `-wal`; only the explicit `wal_checkpoint(TRUNCATE)` can.+mod slow_close {+ use std::io::{BufRead, BufReader, Write};+ use std::path::PathBuf;+ use std::process::{Child, ChildStdin, Command, Stdio};+ use std::sync::mpsc::{self, Receiver};+ use std::thread::{self, JoinHandle};+ use std::time::{Duration, Instant};++ const BIN: &str = env!("CARGO_BIN_EXE_ai-memory");+ const READY_TIMEOUT: Duration = Duration::from_secs(60);+ const EXIT_TIMEOUT: Duration = Duration::from_secs(30);+ const FRAME_TIMEOUT: Duration = Duration::from_secs(10);++ struct Served {+ child: Child,+ stdin: Option<ChildStdin>,+ frames: Option<Receiver<String>>,+ stderr: Option<JoinHandle<String>>,+ wal: PathBuf,+ /// The handle that must outlive the server: see the module docs.+ bystander: Option<rusqlite::Connection>,+ data_dir: tempfile::TempDir,+ }++ fn pump<R: std::io::Read + Send + 'static>(+ reader: R,+ tx: mpsc::Sender<String>,+ ) -> JoinHandle<String> {+ thread::spawn(move || {+ let mut all = String::new();+ for line in BufReader::new(reader).lines().map_while(Result::ok) {+ all.push_str(&line);+ all.push('\n');+ if tx.send(line).is_err() {+ break;+ }+ }+ all+ })+ }++ impl Served {+ /// Start `serve`, wait for its ready line, and (stdio) finish the MCP+ /// handshake, so the server is in the state a real harness leaves it+ /// in. Then open the bystander handle.+ fn start(transport: &str, ready: &str) -> Self {+ let data_dir = tempfile::TempDir::new().expect("tempdir for serve");+ let mut cmd = Command::new(BIN);+ cmd.args(["serve", "--transport", transport]);+ if transport == "http" {+ cmd.args(["--bind", "127.0.0.1:0"]);+ }+ cmd.arg("--no-watcher")+ .arg("--data-dir")+ .arg(data_dir.path())+ .env("AI_MEMORY_DATA_DIR", data_dir.path())+ .env("AI_MEMORY_EMBEDDING_PROVIDER", "none")+ .env("RUST_LOG", "info")+ .stdin(Stdio::piped())+ .stdout(if transport == "stdio" {+ Stdio::piped()+ } else {+ Stdio::null()+ })+ .stderr(Stdio::piped());+ let mut child = cmd.spawn().expect("spawn ai-memory serve");+ let stdin = child.stdin.take();+ let (err_tx, err_rx) = mpsc::channel();+ let stderr = pump(child.stderr.take().expect("stderr piped"), err_tx);+ let frames = child.stdout.take().map(|stdout| {+ let (tx, rx) = mpsc::channel();+ drop(pump(stdout, tx));+ rx+ });+ let wal = data_dir.path().join("db").join("memory.sqlite-wal");+ let mut served = Self {+ child,+ stdin,+ frames,+ stderr: Some(stderr),+ wal,+ bystander: None,+ data_dir,+ };++ let deadline = Instant::now() + READY_TIMEOUT;+ loop {+ let left = deadline.saturating_duration_since(Instant::now());+ match err_rx.recv_timeout(left) {+ Ok(line) if line.contains(ready) => break,+ Ok(_) => {}+ Err(_) => panic!(+ "serve never logged {ready:?}.\nstderr:\n{}",+ served.finish_stderr()+ ),+ }+ }+ if transport == "stdio" {+ served.handshake();+ }+ let db = served.data_dir.path().join("db").join("memory.sqlite");+ let conn = rusqlite::Connection::open(db).expect("open the bystander handle");+ conn.query_row("SELECT count(*) FROM pages", [], |r| r.get::<_, i64>(0))+ .expect("read through the bystander handle");+ served.bystander = Some(conn);+ served+ }++ /// `initialize`, `initialized`, then a `ping` the running service+ /// answers: only then is the stdio arm past its pre-client select, in+ /// the branch where end of input and signals both end up.+ fn handshake(&mut self) {+ const INITIALIZE: &str = concat!(+ r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"#,+ r#""protocolVersion":"2025-11-25","capabilities":{},"#,+ r#""clientInfo":{"name":"serve-close-test","version":"0"}}}"#,+ "\n",+ );+ const PING: &str = concat!(+ r#"{"jsonrpc":"2.0","method":"notifications/initialized"}"#,+ "\n",+ r#"{"jsonrpc":"2.0","id":2,"method":"ping"}"#,+ "\n",+ );+ let stdin = self.stdin.as_mut().expect("stdin piped");+ stdin+ .write_all(INITIALIZE.as_bytes())+ .expect("write initialize");+ stdin.flush().expect("flush initialize");+ self.next_frame();+ let stdin = self.stdin.as_mut().expect("stdin piped");+ stdin.write_all(PING.as_bytes()).expect("write ping");+ stdin.flush().expect("flush ping");+ let pong = self.next_frame();+ assert!(+ pong.contains(r#""id":2"#) && pong.contains(r#""result""#),+ "the ping must be answered by the running service: {pong}"+ );+ }++ fn next_frame(&mut self) -> String {+ let frame = self+ .frames+ .as_ref()+ .expect("stdout piped")+ .recv_timeout(FRAME_TIMEOUT);+ match frame {+ Ok(frame) => frame,+ Err(_) => panic!("no frame on stdout.\nstderr:\n{}", self.finish_stderr()),+ }+ }++ fn wal_len(&self) -> u64 {+ std::fs::metadata(&self.wal).map_or(0, |m| m.len())+ }++ fn finish_stderr(&mut self) -> String {+ let _ = self.child.kill();+ let _ = self.child.wait();+ self.stdin.take();+ self.stderr+ .take()+ .map(|pump| pump.join().unwrap_or_default())+ .unwrap_or_default()+ }++ /// Wait for the child's exit, then assert the stop was clean and the+ /// WAL was truncated while the bystander handle was still open.+ fn assert_stopped_with_the_wal_checkpointed(&mut self) {+ let deadline = Instant::now() + EXIT_TIMEOUT;+ let status = loop {+ match self.child.try_wait().expect("try_wait") {+ Some(status) => break status,+ None if Instant::now() >= deadline => {+ panic!(+ "serve still running after the stop.\nstderr:\n{}",+ self.finish_stderr()+ )+ }+ None => thread::sleep(Duration::from_millis(25)),+ }+ };+ let stderr = self.finish_stderr();+ assert!(+ status.success(),+ "a stop is a clean exit, got {status:?}\nstderr:\n{stderr}"+ );+ assert!(+ self.bystander.is_some(),+ "the bystander handle must still be open"+ );+ assert_eq!(+ self.wal_len(),+ 0,+ "serve exited without checkpointing the WAL.\nstderr:\n{stderr}"+ );+ assert!(+ !stderr.contains("did not close within"),+ "the close hit its deadline.\nstderr:\n{stderr}"+ );+ }+ }++ impl Drop for Served {+ fn drop(&mut self) {+ let _ = self.child.kill();+ let _ = self.child.wait();+ }+ }++ /// The stdio arm, ended by end of input after a real handshake: the route+ /// that runs on every platform, including Windows, which has no SIGTERM.+ /// The close sits after the arm's signal and EOF branches alike, so this+ /// covers `serve.rs`'s stdio `close_store` call.+ #[test]+ fn stdio_serve_checkpoints_the_wal_when_its_input_ends() {+ let mut served = Served::start("stdio", "MCP server ready on stdio");+ assert!(+ served.wal_len() > 0,+ "fixture: a running server has a WAL to truncate"+ );++ // End of input is how a harness that exits stops its stdio server.+ served.stdin.take();++ served.assert_stopped_with_the_wal_checkpointed();+ }++ /// The HTTP arm, ended by the signal `docker stop` and `systemctl stop`+ /// send. Unix only, like every signal test in this suite; it covers+ /// `serve.rs`'s HTTP `close_store` call, which no stdio route reaches.+ #[cfg(unix)]+ #[test]+ fn http_serve_checkpoints_the_wal_on_sigterm() {+ let mut served = Served::start("http", "MCP HTTP server ready");+ assert!(+ served.wal_len() > 0,+ "fixture: a running server has a WAL to truncate"+ );++ let killed = Command::new("kill")+ .args(["-TERM", &served.child.id().to_string()])+ .status()+ .expect("send SIGTERM");+ assert!(killed.success(), "kill -TERM failed");++ served.assert_stopped_with_the_wal_checkpointed();+ }+}diff --git a/crates/ai-memory-store/src/error.rs b/crates/ai-memory-store/src/error.rsindex c8586610..ebbac1b5 100644--- a/crates/ai-memory-store/src/error.rs+++ b/crates/ai-memory-store/src/error.rs@@ -104,6 +104,11 @@ pub enum StoreError {#[error("writer actor is no longer running")]WriterClosed,+ /// The read connection pool was closed at shutdown; no new read opens a+ /// connection.+ #[error("read connection pool is closed")]+ ReaderClosed,+/// A hook tried to reuse a session id that belongs to another immutable/// session tuple or operator. Deliberately carries no row details.#[error("session collision")]diff --git a/crates/ai-memory-store/src/lib.rs b/crates/ai-memory-store/src/lib.rsindex 419f45c8..2a759c36 100644--- a/crates/ai-memory-store/src/lib.rs+++ b/crates/ai-memory-store/src/lib.rs@@ -132,6 +132,14 @@ pub const CLIENT_ACTIVITY_OVERFLOW_CLIENT: &str = "other";/// Default soft cap for the read-only connection pool.const READER_POOL_SOFT_CAP: usize = 4;+/// Size the `-wal` file is truncated back to after a checkpoint (`PRAGMA+/// journal_size_limit`). SQLite's default keeps the file at its high-water+/// mark forever, so one large transaction (a `VACUUM`, a backfill, a bulk+/// import) leaves a `-wal` as big as the database, and a crash then replays it+/// in full on the next start. This does not cap a running transaction: the WAL+/// still grows as far as that transaction needs.+const WAL_JOURNAL_SIZE_LIMIT_BYTES: i64 = 64 * 1024 * 1024;+/// Open (and migrate) a [`Store`] rooted at the given data directory.pub struct Store {/// Cloneable handle to submit mutations.@@ -157,6 +165,7 @@ impl Store {let mut conn = Connection::open(&db_path)?;conn.pragma_update(None, "journal_mode", "WAL")?;+ conn.pragma_update(None, "journal_size_limit", WAL_JOURNAL_SIZE_LIMIT_BYTES)?;conn.pragma_update(None, "synchronous", "NORMAL")?;conn.pragma_update(None, "busy_timeout", 5_000)?; // ms@@ -215,6 +224,23 @@ impl Store {pub fn db_path(&self) -> &Path {&self.db_path}++ /// Release the database for process exit: close the read pool (idle+ /// connections dropped, later reads refused with+ /// [`StoreError::ReaderClosed`], a read still running dropping its+ /// connection on return), then have the writer run a final+ /// `wal_checkpoint(TRUNCATE)`, close its connection and stop.+ ///+ /// The server exits without joining the writer thread (detached tasks keep+ /// clones of [`WriterHandle`] alive), so without this a stop leaves the+ /// whole `-wal` behind for the next start to replay. Best effort: a+ /// checkpoint held back by a reader is logged, not returned, and commands+ /// sent afterwards fail with [`StoreError::WriterClosed`]. Wrap the call in+ /// a timeout; it waits behind whatever the writer is running.+ pub async fn close(&self) {+ self.reader.close();+ self.writer.checkpoint_and_close().await;+ }}/// Create missing database-directory components with owner-only access whilediff --git a/crates/ai-memory-store/src/ops.rs b/crates/ai-memory-store/src/ops.rsindex e771abc5..3f8b15a7 100644--- a/crates/ai-memory-store/src/ops.rs+++ b/crates/ai-memory-store/src/ops.rs@@ -5566,9 +5566,74 @@ pub(crate) fn reclaim_freed_pages(conn: &Connection) -> StoreResult<()> {}batch.push_str("VACUUM;");conn.execute_batch(&batch)?;+ truncate_wal_best_effort(conn, "vacuum");Ok(())}+/// The `(busy, log, checkpointed)` row `PRAGMA wal_checkpoint` returns.+///+/// `busy` is non-zero when a reader kept the checkpoint from finishing. `log`+/// and `checkpointed` are WAL frame counts, and both are `-1` when the+/// database is not in WAL mode.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub(crate) struct WalCheckpoint {+ pub(crate) busy: i64,+ pub(crate) log: i64,+ pub(crate) checkpointed: i64,+}++/// Checkpoint the whole WAL into the database file and truncate it to zero+/// bytes.+///+/// # Errors+/// Propagates the SQL error. A checkpoint that a reader holds back is not an+/// error: it is reported through [`WalCheckpoint::busy`].+pub(crate) fn wal_checkpoint_truncate(conn: &Connection) -> StoreResult<WalCheckpoint> {+ Ok(+ conn.query_row("PRAGMA wal_checkpoint(TRUNCATE)", [], |row| {+ Ok(WalCheckpoint {+ busy: row.get(0)?,+ log: row.get(1)?,+ checkpointed: row.get(2)?,+ })+ })?,+ )+}++/// Truncate the WAL, and log instead of failing when that cannot be done.+///+/// `VACUUM` rewrites every page through the WAL. SQLite's automatic checkpoint+/// copies those pages back but never shrinks the file, so a `VACUUM` on a+/// multi-gigabyte database leaves a `-wal` as large as the database until the+/// next clean close, and a crash or a `SIGKILL` makes the next start replay it+/// in full. Truncating here, right after the rewrite, is what stops that.+///+/// Best effort and deliberately not retried: the operation this follows has+/// already committed, so a reader that holds the checkpoint back (or a failed+/// checkpoint) must not turn it into an error. `journal_size_limit` and the+/// next checkpoint reclaim the space once the reader is gone.+pub(crate) fn truncate_wal_best_effort(conn: &Connection, after: &'static str) {+ match wal_checkpoint_truncate(conn) {+ Ok(WalCheckpoint { busy: 0, .. }) => {}+ Ok(WalCheckpoint {+ busy,+ log,+ checkpointed,+ }) => tracing::warn!(+ after,+ busy,+ log,+ checkpointed,+ "WAL checkpoint(TRUNCATE) was blocked by an active reader; the -wal file keeps its size until a later checkpoint"+ ),+ Err(error) => tracing::warn!(+ after,+ %error,+ "WAL checkpoint(TRUNCATE) failed; the -wal file keeps its size until a later checkpoint"+ ),+ }+}+/// What one [`compact`] run reclaimed.#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]pub struct CompactSummary {@@ -5847,6 +5912,7 @@ pub fn reclaim_ledger_versions(// `VACUUM` cannot run inside a transaction, and it is the whole-file// rewrite `compact` already documents the cost of.conn.execute_batch("VACUUM;")?;+ truncate_wal_best_effort(conn, "vacuum");summary.compacted = true;summary.bytes_after = database_bytes(conn)?;}@@ -10872,6 +10938,91 @@ pub(crate) mod tests {);}+ /// A checkpoint a reader holds back must not fail the operation it follows:+ /// the `VACUUM` has already committed. It is logged with the+ /// `(busy, log, checkpointed)` triple, left for a later checkpoint, and+ /// converges once the reader is gone.+ ///+ /// The test zeroes `busy_timeout` (rusqlite defaults to 5s) so the blocked+ /// checkpoint returns at once; the server's writer connection waits up to+ /// its own 5s timeout for the reader first.+ #[test]+ fn compact_survives_a_checkpoint_blocked_by_an_open_reader() {+ let (tmp, mut conn, ..) = fresh_db();+ conn.pragma_update(None, "journal_mode", "WAL").unwrap();+ conn.pragma_update(None, "busy_timeout", 0).unwrap();+ conn.execute_batch(+ "CREATE TABLE junk (b BLOB);+ INSERT INTO junk VALUES (zeroblob(500000));+ INSERT INTO junk VALUES (zeroblob(500000));+ DELETE FROM junk;",+ )+ .unwrap();+ let wal = tmp.path().join("test.sqlite-wal");++ // A read transaction pins a snapshot older than the VACUUM's frames.+ let reader = Connection::open(tmp.path().join("test.sqlite")).unwrap();+ reader+ .execute_batch("BEGIN; SELECT count(*) FROM junk;")+ .unwrap();++ let mut summary = None;+ let logs = capture_warnings(|| summary = Some(compact(&mut conn)));+ summary+ .unwrap()+ .expect("a blocked checkpoint must not fail the VACUUM that preceded it");+ // The subscriber colours field names; drop the escape sequences so+ // the `busy=1` below reads as plain text.+ let mut plain = String::new();+ let mut chars = logs.chars();+ while let Some(c) = chars.next() {+ if c == '\u{1b}' {+ chars.by_ref().find(|c| *c == 'm');+ } else {+ plain.push(c);+ }+ }+ assert!(+ plain.contains("blocked by an active reader")+ && plain.contains("busy=1")+ && plain.contains("log=")+ && plain.contains("checkpointed="),+ "the blocked checkpoint is logged with its triple: {plain:?}"+ );+ assert!(+ std::fs::metadata(&wal).unwrap().len() > 0,+ "control: the reader really did hold the checkpoint back"+ );++ // Convergence, no retry loop needed: the next checkpoint finishes.+ reader.execute_batch("COMMIT;").unwrap();+ let triple = wal_checkpoint_truncate(&conn).unwrap();+ assert_eq!(triple.busy, 0);+ assert_eq!(std::fs::metadata(&wal).unwrap().len(), 0);+ }++ /// Control for the warning above: an unblocked checkpoint is silent, so a+ /// log line means a reader really did hold it back.+ #[test]+ fn compact_truncates_the_wal_without_warning_when_no_reader_is_open() {+ let (tmp, mut conn, ..) = fresh_db();+ conn.pragma_update(None, "journal_mode", "WAL").unwrap();+ conn.execute_batch(+ "CREATE TABLE junk (b BLOB);+ INSERT INTO junk VALUES (zeroblob(500000));+ DELETE FROM junk;",+ )+ .unwrap();++ let mut summary = None;+ let logs = capture_warnings(|| summary = Some(compact(&mut conn)));+ summary.unwrap().unwrap();++ assert!(!logs.contains("checkpoint"), "unexpected warning: {logs:?}");+ let wal = tmp.path().join("test.sqlite-wal");+ assert_eq!(std::fs::metadata(wal).map_or(0, |m| m.len()), 0);+ }+#[test]fn page_feedback_is_scoped_rebuildable_and_audited() {let (_tmp, mut conn, ws, proj) = fresh_db();diff --git a/crates/ai-memory-store/src/reader.rs b/crates/ai-memory-store/src/reader.rsindex a5e48632..878a0354 100644--- a/crates/ai-memory-store/src/reader.rs+++ b/crates/ai-memory-store/src/reader.rs@@ -2179,10 +2179,18 @@ pub struct ReaderPool {struct Inner {db_path: PathBuf,- pool: Mutex<Vec<Connection>>,+ pool: Mutex<Pool>,soft_cap: usize,}+/// The idle connections and whether the pool still hands out new ones. One+/// lock covers both, so a checkout or a return cannot interleave with+/// [`ReaderPool::close`] between reading the flag and touching the idle list.+struct Pool {+ idle: Vec<Connection>,+ closed: bool,+}+impl ReaderPool {/// Initialise the pool. Connections are opened lazily on first use.///@@ -2193,7 +2201,10 @@ impl ReaderPool {Ok(Self {inner: Arc::new(Inner {db_path: db_path.to_path_buf(),- pool: Mutex::new(Vec::with_capacity(soft_cap.max(1))),+ pool: Mutex::new(Pool {+ idle: Vec::with_capacity(soft_cap.max(1)),+ closed: false,+ }),soft_cap: soft_cap.max(1),}),tuning: RetrievalTuning::default(),@@ -2201,6 +2212,22 @@ impl ReaderPool {})}+ /// Close the pool for shutdown: drop every idle connection, and make every+ /// later read fail with [`StoreError::ReaderClosed`] instead of opening a+ /// new one. Idempotent.+ ///+ /// A read already running is not interrupted or waited for: it finishes+ /// on its own connection, and that connection is dropped when it is+ /// returned rather than pooled again. Such a read can hold a snapshot that+ /// keeps the writer's final checkpoint from truncating the WAL; the+ /// checkpoint then logs the blocked attempt and the WAL is replayed on the+ /// next start, as after any other unclean stop.+ pub fn close(&self) {+ let mut pool = self.inner.pool.lock();+ pool.closed = true;+ pool.idle.clear();+ }+/// Configure the opt-in ranking signals used by [`Self::hybrid_search`]./// Only handles cloned from this one afterwards observe the change.pub fn set_retrieval_tuning(&mut self, tuning: RetrievalTuning) {@@ -12736,16 +12763,24 @@ fn has_trailing_path_separator(path: &str) -> bool {}fn checkout(inner: &Inner) -> StoreResult<Connection> {- if let Some(conn) = inner.pool.lock().pop() {- return Ok(conn);+ {+ let mut pool = inner.pool.lock();+ if pool.closed {+ return Err(StoreError::ReaderClosed);+ }+ if let Some(conn) = pool.idle.pop() {+ return Ok(conn);+ }}open_read_only(&inner.db_path)}fn checkin(inner: &Inner, conn: Connection) {let mut pool = inner.pool.lock();- if pool.len() < inner.soft_cap {- pool.push(conn);+ // A connection returned after `close` is dropped, not pooled: pooling it+ // would revive a reader the shutdown just released.+ if !pool.closed && pool.idle.len() < inner.soft_cap {+ pool.idle.push(conn);}}@@ -12761,6 +12796,39 @@ fn open_read_only(path: &Path) -> StoreResult<Connection> {#[cfg(test)]mod tests {+ /// `close` is a state, not a one-off sweep: a connection checked out+ /// before it is dropped on return, and nothing opens after it. The control+ /// half shows the same return pools the connection while the pool is open.+ #[test]+ fn a_closed_pool_drops_a_returned_connection_and_refuses_checkout() {+ use super::{ReaderPool, checkin, checkout};+ use crate::error::StoreError;++ let tmp = tempfile::TempDir::new().unwrap();+ let path = tmp.path().join("pool.sqlite");+ rusqlite::Connection::open(&path).unwrap();+ let pool = ReaderPool::new(&path, 4).unwrap();++ let open = checkout(&pool.inner).unwrap();+ checkin(&pool.inner, open);+ assert_eq!(pool.inner.pool.lock().idle.len(), 1, "control: pooled");++ let held = checkout(&pool.inner).unwrap();+ pool.close();+ assert_eq!(pool.inner.pool.lock().idle.len(), 0, "idle set released");+ checkin(&pool.inner, held);+ assert_eq!(+ pool.inner.pool.lock().idle.len(),+ 0,+ "a connection returned after close was pooled again"+ );+ assert!(matches!(+ checkout(&pool.inner),+ Err(StoreError::ReaderClosed)+ ));+ pool.close();+ }+/// The percentage is what an operator actually reads, and it divides by a/// figure that is zero on a fresh database.#[test]diff --git a/crates/ai-memory-store/src/writer.rs b/crates/ai-memory-store/src/writer.rsindex ab72c425..0aeb723f 100644--- a/crates/ai-memory-store/src/writer.rs+++ b/crates/ai-memory-store/src/writer.rs@@ -491,6 +491,17 @@ pub(crate) enum WriteCmd {mode: crate::ops::PurgeMode,reply: oneshot::Sender<StoreResult<crate::ops::PurgeSessionSummary>>,},+ /// Final `wal_checkpoint(TRUNCATE)`, then close the connection and stop+ /// the actor. See [`WriterHandle::checkpoint_and_close`].+ Close {+ reply: oneshot::Sender<()>,+ },+ /// Test seam: read `PRAGMA journal_size_limit` off the writer's own+ /// connection, the only place the pragma is set.+ #[cfg(test)]+ JournalSizeLimit {+ reply: oneshot::Sender<StoreResult<i64>>,+ },/// Reclaim free pages on demand: rebuild the FTS indexes and `VACUUM`,/// deleting nothing. See [`ops::compact`].Compact {@@ -2359,6 +2370,31 @@ impl WriterHandle {rx.await.map_err(|_| StoreError::WriterClosed)?}+ /// Truncate the WAL, close the writer's connection and stop the actor,+ /// waiting until that has happened. Later calls fail with+ /// [`StoreError::WriterClosed`]. Returns at once if the actor already+ /// stopped.+ ///+ /// Queued behind every command already accepted, so it also waits for a+ /// long-running one (a `VACUUM`); callers bound it with a timeout. A+ /// checkpoint a reader holds back is logged, never an error.+ pub async fn checkpoint_and_close(&self) {+ let (tx, rx) = oneshot::channel();+ if self.send(WriteCmd::Close { reply: tx }).await.is_ok() {+ // The reply follows the checkpoint and the connection close; a+ // dropped sender means the actor stopped without answering.+ let _ = rx.await;+ }+ }++ /// `PRAGMA journal_size_limit` as the writer's connection reports it.+ #[cfg(test)]+ pub(crate) async fn journal_size_limit(&self) -> StoreResult<i64> {+ let (tx, rx) = oneshot::channel();+ self.send(WriteCmd::JournalSizeLimit { reply: tx }).await?;+ rx.await.map_err(|_| StoreError::WriterClosed)?+ }+/// Reclaim free pages on demand, deleting nothing.////// Runs through the writer actor like every other exclusive operation, so@@ -4240,6 +4276,25 @@ fn worker_loop(mut conn: Connection, mut rx: mpsc::Receiver<WriteCmd>) {);send_or_warn(reply, result, "purge_session");}+ WriteCmd::Close { reply } => {+ ops::truncate_wal_best_effort(&conn, "shutdown");+ // Close before replying, so the caller's return means the+ // connection is released.+ drop(conn);+ send_or_warn(reply, (), "close");+ // Same drain as `Shutdown`: callers that already hold a+ // permit see `WriterClosed` instead of waiting forever.+ rx.close();+ while rx.blocking_recv().is_some() {}+ break;+ }+ #[cfg(test)]+ WriteCmd::JournalSizeLimit { reply } => {+ let result = conn+ .query_row("PRAGMA journal_size_limit", [], |row| row.get(0))+ .map_err(StoreError::from);+ send_or_warn(reply, result, "journal_size_limit");+ }WriteCmd::Compact { reply } => {let result = ops::compact(&mut conn);send_or_warn(reply, result, "compact");diff --git a/crates/ai-memory-store/tests/suite/mod.rs b/crates/ai-memory-store/tests/suite/mod.rsindex ac9f3d9f..7e48dc0a 100644--- a/crates/ai-memory-store/tests/suite/mod.rs+++ b/crates/ai-memory-store/tests/suite/mod.rs@@ -32,4 +32,5 @@ mod session_scope_from_observations;mod sessions_by_agent;mod slot_visibility;mod stress_writer_throughput;+mod wal_bounds;mod workstream_provenance;diff --git a/crates/ai-memory-store/tests/suite/wal_bounds.rs b/crates/ai-memory-store/tests/suite/wal_bounds.rsnew file mode 100644index 00000000..ca5fa251--- /dev/null+++ b/crates/ai-memory-store/tests/suite/wal_bounds.rs@@ -0,0 +1,283 @@+//! The `-wal` file must not outlive the operation or the process that grew it.+//!+//! A `VACUUM` rewrites every page through the WAL, and SQLite's automatic+//! checkpoint never shrinks the file, so a reclaim on a large store left a+//! `-wal` as big as the database. Killed before a clean close, the server made+//! the next start replay all of it. Each test here runs against a real WAL-mode+//! [`Store`] and reads the `-wal` file's length; a fixture on the in-memory+//! journal cannot see the defect.++use std::path::PathBuf;++use ai_memory_core::{NewPage, PagePath, Tier};+use ai_memory_store::{Compaction, PurgeMode, Store, StoreError};++/// Enough ledger versions that a rewrite of the database is megabytes, not+/// bytes: the assertions compare the `-wal` against zero, and a database whose+/// whole rewrite is a few pages would hide a missing checkpoint in noise.+const LEDGER_VERSIONS: usize = 24;++fn wal_path(store: &Store) -> PathBuf {+ store.db_path().with_extension("sqlite-wal")+}++/// Length of the `-wal`, zero when SQLite has already deleted it.+fn wal_len(store: &Store) -> u64 {+ std::fs::metadata(wal_path(store)).map_or(0, |m| m.len())+}++/// A ledger body of about 30 KiB in the shape the hooks append, so the+/// reclaim's content gate accepts every version.+fn ledger_body(version: usize) -> String {+ (0..600)+ .map(|line| {+ format!(+ "## [2026-09-05T{version:02}:00:{:02}Z] tool Bash: cargo test\n",+ line % 60+ )+ })+ .collect()+}++/// A store with a ledger path carrying [`LEDGER_VERSIONS`] versions.+async fn store_with_ledger_residue(+ tmp: &tempfile::TempDir,+) -> (+ Store,+ ai_memory_core::WorkspaceId,+ ai_memory_core::ProjectId,+) {+ let store = Store::open(tmp.path()).unwrap();+ let ws = store+ .writer+ .get_or_create_workspace("default".to_string())+ .await+ .unwrap();+ let proj = store+ .writer+ .get_or_create_project(ws, "app".to_string(), None)+ .await+ .unwrap();+ for version in 0..LEDGER_VERSIONS {+ store+ .writer+ .upsert_page(NewPage {+ workspace_id: ws,+ project_id: proj,+ path: PagePath::new("log-2026-09.md").unwrap(),+ title: "t".into(),+ body: ledger_body(version),+ tier: Tier::Episodic,+ frontmatter_json: serde_json::json!({}),+ pinned: false,+ links: Vec::new(),+ author_id: None,+ expires_at: None,+ entities: Vec::new(),+ evidence: Vec::new(),+ })+ .await+ .unwrap();+ }+ (store, ws, proj)+}++#[tokio::test]+async fn compact_truncates_the_wal_it_grew() {+ let tmp = tempfile::tempdir().unwrap();+ let (store, _, _) = store_with_ledger_residue(&tmp).await;+ assert!(+ wal_len(&store) > 0,+ "fixture: the writes must have left a WAL to truncate"+ );++ store.writer.compact().await.unwrap();++ assert_eq!(wal_len(&store), 0, "compact left its VACUUM in the -wal");+}++#[tokio::test]+async fn reclaiming_ledger_versions_truncates_the_wal_it_grew() {+ let tmp = tempfile::tempdir().unwrap();+ let (store, _, _) = store_with_ledger_residue(&tmp).await;++ let summary = store+ .writer+ .reclaim_ledger_versions(false, false, Compaction::Reclaim)+ .await+ .unwrap();++ assert!(summary.compacted, "fixture: the VACUUM must have run");+ assert_eq!(summary.pages_deleted, (LEDGER_VERSIONS - 1) as u64);+ assert_eq!(+ wal_len(&store),+ 0,+ "the reclaim left its VACUUM in the -wal"+ );+}++/// Control: without `--compact` there is no `VACUUM`, so the reclaim must not+/// start checkpointing on its own. Keeps the assertion above from passing+/// merely because every call truncates.+#[tokio::test]+async fn reclaiming_ledger_versions_without_compaction_does_not_vacuum() {+ let tmp = tempfile::tempdir().unwrap();+ let (store, _, _) = store_with_ledger_residue(&tmp).await;++ let summary = store+ .writer+ .reclaim_ledger_versions(false, false, Compaction::Skip)+ .await+ .unwrap();++ assert!(!summary.compacted);+ assert!(wal_len(&store) > 0, "a delete without VACUUM keeps its WAL");+}++#[tokio::test]+async fn purging_a_project_with_compaction_truncates_the_wal() {+ let tmp = tempfile::tempdir().unwrap();+ let (store, ws, proj) = store_with_ledger_residue(&tmp).await;++ store+ .writer+ .purge_project(+ ws,+ proj,+ "default/app",+ None,+ true,+ Compaction::Reclaim,+ PurgeMode::Commit,+ )+ .await+ .unwrap();++ assert_eq!(+ wal_len(&store),+ 0,+ "purge --compact left its VACUUM in the -wal"+ );+}++/// The shutdown path: `main` exits without joining the writer, so the+/// checkpoint has to happen before the process goes. A second connection stays+/// open, as the read pool's would, so SQLite's own "last connection closed"+/// checkpoint cannot be what empties the file.+#[tokio::test]+async fn close_checkpoints_the_wal_even_while_another_connection_is_open() {+ let tmp = tempfile::tempdir().unwrap();+ let (store, _, _) = store_with_ledger_residue(&tmp).await;+ let bystander = rusqlite::Connection::open(store.db_path()).unwrap();+ bystander+ .query_row("SELECT count(*) FROM pages", [], |r| r.get::<_, i64>(0))+ .unwrap();+ assert!(wal_len(&store) > 0, "fixture: pending WAL before close");++ store.close().await;++ assert_eq!(wal_len(&store), 0, "close left the WAL for the next start");+ drop(bystander);+}++/// Releasing the connections is observable: SQLite deletes the `-wal` only when+/// the last connection to the database closes. A writer or a pooled reader+/// that survives `close` keeps the file.+#[tokio::test]+async fn close_releases_the_writer_and_the_pooled_readers() {+ let tmp = tempfile::tempdir().unwrap();+ let (store, _, _) = store_with_ledger_residue(&tmp).await;+ // Park a read connection in the pool.+ store.reader.users_exist().await.unwrap();+ assert!(+ wal_path(&store).exists(),+ "fixture: a live WAL before close"+ );++ store.close().await;++ assert!(+ !wal_path(&store).exists(),+ "a connection survived close: SQLite keeps the -wal while one is open"+ );+}++#[tokio::test]+async fn the_writer_refuses_commands_after_close_and_close_is_repeatable() {+ let tmp = tempfile::tempdir().unwrap();+ let (store, _, _) = store_with_ledger_residue(&tmp).await;++ store.close().await;+ store.close().await;++ let err = store+ .writer+ .get_or_create_workspace("too-late".to_string())+ .await+ .unwrap_err();+ assert!(matches!(err, StoreError::WriterClosed), "got {err:?}");+}++/// A read that is running when `close` happens must neither be refused nor+/// revive the pool: it finishes on its connection, and that connection is+/// dropped on return. Deterministic: the read parks on a channel until the+/// test has closed the pool, so there is no sleep and no timing window.+#[tokio::test]+async fn a_read_in_flight_across_close_is_not_repooled_and_new_reads_are_refused() {+ let tmp = tempfile::tempdir().unwrap();+ let (store, _, _) = store_with_ledger_residue(&tmp).await;+ let (started_tx, started_rx) = std::sync::mpsc::channel::<()>();+ let (release_tx, release_rx) = std::sync::mpsc::channel::<()>();+ let reader = store.reader.clone();+ let held = tokio::spawn(async move {+ reader+ .with_conn(move |conn| {+ // Touch the database: a read-only connection that never read+ // has not joined the WAL, and so could not keep the file alive.+ conn.query_row("SELECT count(*) FROM pages", [], |r| r.get::<_, i64>(0))?;+ started_tx.send(()).unwrap();+ release_rx.recv().unwrap();+ Ok(())+ })+ .await+ });+ // The connection is checked out and held.+ tokio::task::spawn_blocking(move || started_rx.recv().unwrap())+ .await+ .unwrap();++ store.reader.close();++ let refused = store.reader.users_exist().await.unwrap_err();+ assert!(+ matches!(refused, StoreError::ReaderClosed),+ "a read after close opened a connection: {refused:?}"+ );+ release_tx.send(()).unwrap();+ held.await+ .unwrap()+ .expect("the read already in flight finishes");++ // With the writer closed too, no connection may be left: SQLite deletes+ // the `-wal` only when the last one closes, so a returned connection that+ // went back into the pool would keep it.+ store.writer.checkpoint_and_close().await;+ assert!(+ !wal_path(&store).exists(),+ "the connection returned after close was pooled again"+ );+}++/// The pragma lives on the writer's own connection and nothing else sets it,+/// so only a read of that connection can notice it going missing. The fixtures+/// above are far below the limit and could not.+#[tokio::test]+async fn the_writer_connection_caps_the_wal_it_keeps_after_a_checkpoint() {+ let tmp = tempfile::tempdir().unwrap();+ let store = Store::open(tmp.path()).unwrap();++ assert_eq!(+ store.writer.journal_size_limit().await.unwrap(),+ 64 * 1024 * 1024+ );+}diff --git a/docs/lifecycle-ops.md b/docs/lifecycle-ops.mdindex 17a7bb45..dd5c146c 100644--- a/docs/lifecycle-ops.md+++ b/docs/lifecycle-ops.md@@ -75,6 +75,23 @@ pages. The rebuild is needed because an ordinary delete leaves the tokens insidethe index segments, and a `VACUUM` alone does not clear them. It needs free disk space of roughly the database's own size and takesminutes on a large store, which is why it is opt-in.+`VACUUM` rewrites every page through the SQLite write-ahead log, so the `-wal`+file next to `memory.sqlite` grows to roughly the database's size while it runs.+`compact`, `reclaim-ledger-versions --compact` and the purges with `--compact`+therefore run `PRAGMA wal_checkpoint(TRUNCATE)` right after the `VACUUM`, and the+`-wal` goes back to zero bytes. If a reader holds an old snapshot the checkpoint+cannot finish: the command still succeeds, `ai-memory` logs a `warn` with the+`busy`, `log` and `checkpointed` counts, and the `-wal` keeps its size until a+later checkpoint after that reader is gone (the writer waits up to 5 seconds for+readers before giving up). The writer connection also sets `journal_size_limit`+to 64 MiB, which is the size the `-wal` is cut back to after any checkpoint. It+is not a cap: a running transaction, or a reader that blocks the checkpoint, can+push the file past it. On `SIGINT`, `SIGTERM`, Ctrl-C or the end of stdio input, `serve` closes its read+connections and runs one last checkpoint, waiting at most 5 seconds for it; a+read still running keeps its connection until it ends, and the checkpoint is+best effort. A `SIGKILL` leaves the `-wal` as it was and the next start replays+it.+**`--compact` is not forensic erasure.** The wiki git repository stores pagecontent in its objects *and its commit messages*, and any backup taken beforethe purge still contains everything. Removing the bytes from the live SQLite