Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix(store): bound the SQLite WAL after VACUUM and close the store on shutdown

Djalma Júnior67e386e30737
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 53f77b46..73670613 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -129,6 +129,18 @@ version in [`docs/changelog/`](docs/changelog/):
were not re-stamped into the target project, which distorted cross-project
profile convergence and caused a subsequent `purge-project` of the source
project to destroy them via cascade. They now re-stamp in the same transaction. (#1265)
+- Fixed the SQLite `-wal` file staying as large as the database after a
+ `VACUUM`. `compact`, `reclaim-ledger-versions --compact` and the session,
+ project and workspace purges with compaction now run `PRAGMA
+ wal_checkpoint(TRUNCATE)` after the rewrite, and the store's writer
+ connection sets `journal_size_limit` to 64 MiB. A checkpoint that a reader
+ holds back is logged at `warn` and the operation still succeeds. A 29.5 GiB
+ database that shrank to 6.3 GiB kept a 17.8 GB WAL, and the next restart
+ spent 9 minutes replaying it. `serve` now also drops its read connections
+ and checkpoints the WAL on SIGINT, SIGTERM, Ctrl-C or the end of stdio
+ input, waiting at most 5 seconds; a read that arrives after that point is
+ refused. A hard kill before the checkpoint still leaves the WAL to replay.
+ (#PRNUM)
## [3.0.0] - 2026-10-13
diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rs
index 4dba5368..555e7b0d 100644
--- a/crates/ai-memory-cli/src/commands/serve.rs
+++ b/crates/ai-memory-cli/src/commands/serve.rs
@@ -1453,6 +1453,7 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {
}
}
}
+ close_store(&store).await;
}
TransportKind::Http => {
seed_active_project_fallback(&store.reader, &active_project).await;
@@ -1975,6 +1976,7 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {
),
}
}
+ close_store(&store).await;
if let Some(serve_result) = serve_result {
serve_result?;
}
@@ -1983,6 +1985,32 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {
Ok(())
}
+/// Checkpoint the WAL and close the store's connections on the way out.
+///
+/// `main` ends the process without joining the store writer (detached tasks
+/// still hold clones of its handle), so the connection is never closed and the
+/// whole `-wal` is left for the next start to replay: 9 minutes for a 17.8 GB
+/// one after a `VACUUM`. Bounded by [`SHUTDOWN_GRACE`] because the close queues
+/// behind whatever the writer is running, and a stuck query must not turn a
+/// stop back into a SIGKILL.
+async fn close_store(store: &Store) {
+ close_store_within(store, SHUTDOWN_GRACE).await;
+}
+
+/// [`close_store`] with the deadline as a parameter, so a test can pin the
+/// bound without waiting out the real one. `true` when the store closed in
+/// time.
+async fn close_store_within(store: &Store, grace: Duration) -> bool {
+ if tokio::time::timeout(grace, store.close()).await.is_err() {
+ tracing::warn!(
+ grace_secs = grace.as_secs(),
+ "store did not close within the shutdown grace period; exiting anyway"
+ );
+ return false;
+ }
+ true
+}
+
#[allow(clippy::too_many_arguments)]
async fn start_maintenance_scheduler(
settings: MaintenanceSettings,
@@ -3331,6 +3359,35 @@ mod tests {
use tempfile::TempDir;
use tower::ServiceExt;
+ /// The shutdown close queues behind whatever the writer is running, so the
+ /// deadline is what keeps a stuck query from turning a stop back into a
+ /// SIGKILL. The writer is made busy for real: another connection holds the
+ /// write lock, so the writer's next command (the close itself, or the
+ /// write queued before it) waits in SQLite's 5s busy handler.
+ #[tokio::test]
+ async fn close_store_returns_within_its_grace_while_the_writer_is_busy() {
+ let tmp = TempDir::new().unwrap();
+ let store = Store::open(tmp.path()).unwrap();
+ let blocker = rusqlite::Connection::open(store.db_path()).unwrap();
+ blocker.execute_batch("BEGIN IMMEDIATE").unwrap();
+ let writer = store.writer.clone();
+ let queued = tokio::spawn(async move { writer.get_or_create_workspace("blocked").await });
+
+ let started = std::time::Instant::now();
+ let closed = close_store_within(&store, Duration::from_millis(200)).await;
+ let elapsed = started.elapsed();
+
+ // Let the writer go before asserting, so a failure does not leave it
+ // parked for the rest of its busy timeout.
+ blocker.execute_batch("ROLLBACK").unwrap();
+ let _ = queued.await;
+ assert!(!closed, "the close cannot finish while the writer is stuck");
+ assert!(
+ elapsed < Duration::from_secs(3),
+ "close_store waited {elapsed:?} for a stuck writer instead of giving up"
+ );
+ }
+
/// An owner-check failure of the startup wiki baseline checkpoint must be
/// classified as an ERROR-worthy `OwnerFailure`, not the WARN-only
/// `OtherFailure` that hid it before (#872). A generic error stays
diff --git a/crates/ai-memory-cli/tests/suite/serve_shutdown.rs b/crates/ai-memory-cli/tests/suite/serve_shutdown.rs
index 14e3cc1d..80ce2553 100644
--- a/crates/ai-memory-cli/tests/suite/serve_shutdown.rs
+++ b/crates/ai-memory-cli/tests/suite/serve_shutdown.rs
@@ -1,8 +1,8 @@
-//! `serve --transport stdio` must stop when the operator interrupts it (#699).
-
-#![cfg(unix)]
+//! `serve --transport stdio` must stop when the operator interrupts it (#699),
+//! and any stop of `serve` must leave the SQLite WAL checkpointed.
/// Startup, one interrupt and one shutdown: seconds, not milliseconds.
+#[cfg(unix)]
mod slow {
use std::io::{BufRead, BufReader};
use std::process::{Child, Command, Stdio};
@@ -136,3 +136,260 @@ mod slow {
);
}
}
+
+/// `serve` ends the process without joining the store writer, so the WAL a
+/// server grew used to be left whole for the next start to replay (a 17.8 GB
+/// one, 9 minutes). These tests go through the real shutdown path of the built
+/// binary, not `Store::close`, and prove `close_store` ran: another SQLite
+/// handle stays open for the whole run, so the automatic checkpoint-and-delete
+/// SQLite does when the *last* connection closes cannot be what empties the
+/// `-wal`; only the explicit `wal_checkpoint(TRUNCATE)` can.
+mod slow_close {
+ use std::io::{BufRead, BufReader, Write};
+ use std::path::PathBuf;
+ use std::process::{Child, ChildStdin, Command, Stdio};
+ use std::sync::mpsc::{self, Receiver};
+ use std::thread::{self, JoinHandle};
+ use std::time::{Duration, Instant};
+
+ const BIN: &str = env!("CARGO_BIN_EXE_ai-memory");
+ const READY_TIMEOUT: Duration = Duration::from_secs(60);
+ const EXIT_TIMEOUT: Duration = Duration::from_secs(30);
+ const FRAME_TIMEOUT: Duration = Duration::from_secs(10);
+
+ struct Served {
+ child: Child,
+ stdin: Option<ChildStdin>,
+ frames: Option<Receiver<String>>,
+ stderr: Option<JoinHandle<String>>,
+ wal: PathBuf,
+ /// The handle that must outlive the server: see the module docs.
+ bystander: Option<rusqlite::Connection>,
+ data_dir: tempfile::TempDir,
+ }
+
+ fn pump<R: std::io::Read + Send + 'static>(
+ reader: R,
+ tx: mpsc::Sender<String>,
+ ) -> JoinHandle<String> {
+ thread::spawn(move || {
+ let mut all = String::new();
+ for line in BufReader::new(reader).lines().map_while(Result::ok) {
+ all.push_str(&line);
+ all.push('\n');
+ if tx.send(line).is_err() {
+ break;
+ }
+ }
+ all
+ })
+ }
+
+ impl Served {
+ /// Start `serve`, wait for its ready line, and (stdio) finish the MCP
+ /// handshake, so the server is in the state a real harness leaves it
+ /// in. Then open the bystander handle.
+ fn start(transport: &str, ready: &str) -> Self {
+ let data_dir = tempfile::TempDir::new().expect("tempdir for serve");
+ let mut cmd = Command::new(BIN);
+ cmd.args(["serve", "--transport", transport]);
+ if transport == "http" {
+ cmd.args(["--bind", "127.0.0.1:0"]);
+ }
+ cmd.arg("--no-watcher")
+ .arg("--data-dir")
+ .arg(data_dir.path())
+ .env("AI_MEMORY_DATA_DIR", data_dir.path())
+ .env("AI_MEMORY_EMBEDDING_PROVIDER", "none")
+ .env("RUST_LOG", "info")
+ .stdin(Stdio::piped())
+ .stdout(if transport == "stdio" {
+ Stdio::piped()
+ } else {
+ Stdio::null()
+ })
+ .stderr(Stdio::piped());
+ let mut child = cmd.spawn().expect("spawn ai-memory serve");
+ let stdin = child.stdin.take();
+ let (err_tx, err_rx) = mpsc::channel();
+ let stderr = pump(child.stderr.take().expect("stderr piped"), err_tx);
+ let frames = child.stdout.take().map(|stdout| {
+ let (tx, rx) = mpsc::channel();
+ drop(pump(stdout, tx));
+ rx
+ });
+ let wal = data_dir.path().join("db").join("memory.sqlite-wal");
+ let mut served = Self {
+ child,
+ stdin,
+ frames,
+ stderr: Some(stderr),
+ wal,
+ bystander: None,
+ data_dir,
+ };
+
+ let deadline = Instant::now() + READY_TIMEOUT;
+ loop {
+ let left = deadline.saturating_duration_since(Instant::now());
+ match err_rx.recv_timeout(left) {
+ Ok(line) if line.contains(ready) => break,
+ Ok(_) => {}
+ Err(_) => panic!(
+ "serve never logged {ready:?}.\nstderr:\n{}",
+ served.finish_stderr()
+ ),
+ }
+ }
+ if transport == "stdio" {
+ served.handshake();
+ }
+ let db = served.data_dir.path().join("db").join("memory.sqlite");
+ let conn = rusqlite::Connection::open(db).expect("open the bystander handle");
+ conn.query_row("SELECT count(*) FROM pages", [], |r| r.get::<_, i64>(0))
+ .expect("read through the bystander handle");
+ served.bystander = Some(conn);
+ served
+ }
+
+ /// `initialize`, `initialized`, then a `ping` the running service
+ /// answers: only then is the stdio arm past its pre-client select, in
+ /// the branch where end of input and signals both end up.
+ fn handshake(&mut self) {
+ const INITIALIZE: &str = concat!(
+ r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"#,
+ r#""protocolVersion":"2025-11-25","capabilities":{},"#,
+ r#""clientInfo":{"name":"serve-close-test","version":"0"}}}"#,
+ "\n",
+ );
+ const PING: &str = concat!(
+ r#"{"jsonrpc":"2.0","method":"notifications/initialized"}"#,
+ "\n",
+ r#"{"jsonrpc":"2.0","id":2,"method":"ping"}"#,
+ "\n",
+ );
+ let stdin = self.stdin.as_mut().expect("stdin piped");
+ stdin
+ .write_all(INITIALIZE.as_bytes())
+ .expect("write initialize");
+ stdin.flush().expect("flush initialize");
+ self.next_frame();
+ let stdin = self.stdin.as_mut().expect("stdin piped");
+ stdin.write_all(PING.as_bytes()).expect("write ping");
+ stdin.flush().expect("flush ping");
+ let pong = self.next_frame();
+ assert!(
+ pong.contains(r#""id":2"#) && pong.contains(r#""result""#),
+ "the ping must be answered by the running service: {pong}"
+ );
+ }
+
+ fn next_frame(&mut self) -> String {
+ let frame = self
+ .frames
+ .as_ref()
+ .expect("stdout piped")
+ .recv_timeout(FRAME_TIMEOUT);
+ match frame {
+ Ok(frame) => frame,
+ Err(_) => panic!("no frame on stdout.\nstderr:\n{}", self.finish_stderr()),
+ }
+ }
+
+ fn wal_len(&self) -> u64 {
+ std::fs::metadata(&self.wal).map_or(0, |m| m.len())
+ }
+
+ fn finish_stderr(&mut self) -> String {
+ let _ = self.child.kill();
+ let _ = self.child.wait();
+ self.stdin.take();
+ self.stderr
+ .take()
+ .map(|pump| pump.join().unwrap_or_default())
+ .unwrap_or_default()
+ }
+
+ /// Wait for the child's exit, then assert the stop was clean and the
+ /// WAL was truncated while the bystander handle was still open.
+ fn assert_stopped_with_the_wal_checkpointed(&mut self) {
+ let deadline = Instant::now() + EXIT_TIMEOUT;
+ let status = loop {
+ match self.child.try_wait().expect("try_wait") {
+ Some(status) => break status,
+ None if Instant::now() >= deadline => {
+ panic!(
+ "serve still running after the stop.\nstderr:\n{}",
+ self.finish_stderr()
+ )
+ }
+ None => thread::sleep(Duration::from_millis(25)),
+ }
+ };
+ let stderr = self.finish_stderr();
+ assert!(
+ status.success(),
+ "a stop is a clean exit, got {status:?}\nstderr:\n{stderr}"
+ );
+ assert!(
+ self.bystander.is_some(),
+ "the bystander handle must still be open"
+ );
+ assert_eq!(
+ self.wal_len(),
+ 0,
+ "serve exited without checkpointing the WAL.\nstderr:\n{stderr}"
+ );
+ assert!(
+ !stderr.contains("did not close within"),
+ "the close hit its deadline.\nstderr:\n{stderr}"
+ );
+ }
+ }
+
+ impl Drop for Served {
+ fn drop(&mut self) {
+ let _ = self.child.kill();
+ let _ = self.child.wait();
+ }
+ }
+
+ /// The stdio arm, ended by end of input after a real handshake: the route
+ /// that runs on every platform, including Windows, which has no SIGTERM.
+ /// The close sits after the arm's signal and EOF branches alike, so this
+ /// covers `serve.rs`'s stdio `close_store` call.
+ #[test]
+ fn stdio_serve_checkpoints_the_wal_when_its_input_ends() {
+ let mut served = Served::start("stdio", "MCP server ready on stdio");
+ assert!(
+ served.wal_len() > 0,
+ "fixture: a running server has a WAL to truncate"
+ );
+
+ // End of input is how a harness that exits stops its stdio server.
+ served.stdin.take();
+
+ served.assert_stopped_with_the_wal_checkpointed();
+ }
+
+ /// The HTTP arm, ended by the signal `docker stop` and `systemctl stop`
+ /// send. Unix only, like every signal test in this suite; it covers
+ /// `serve.rs`'s HTTP `close_store` call, which no stdio route reaches.
+ #[cfg(unix)]
+ #[test]
+ fn http_serve_checkpoints_the_wal_on_sigterm() {
+ let mut served = Served::start("http", "MCP HTTP server ready");
+ assert!(
+ served.wal_len() > 0,
+ "fixture: a running server has a WAL to truncate"
+ );
+
+ let killed = Command::new("kill")
+ .args(["-TERM", &served.child.id().to_string()])
+ .status()
+ .expect("send SIGTERM");
+ assert!(killed.success(), "kill -TERM failed");
+
+ served.assert_stopped_with_the_wal_checkpointed();
+ }
+}
diff --git a/crates/ai-memory-store/src/error.rs b/crates/ai-memory-store/src/error.rs
index c8586610..ebbac1b5 100644
--- a/crates/ai-memory-store/src/error.rs
+++ b/crates/ai-memory-store/src/error.rs
@@ -104,6 +104,11 @@ pub enum StoreError {
#[error("writer actor is no longer running")]
WriterClosed,
+ /// The read connection pool was closed at shutdown; no new read opens a
+ /// connection.
+ #[error("read connection pool is closed")]
+ ReaderClosed,
+
/// A hook tried to reuse a session id that belongs to another immutable
/// session tuple or operator. Deliberately carries no row details.
#[error("session collision")]
diff --git a/crates/ai-memory-store/src/lib.rs b/crates/ai-memory-store/src/lib.rs
index 419f45c8..2a759c36 100644
--- a/crates/ai-memory-store/src/lib.rs
+++ b/crates/ai-memory-store/src/lib.rs
@@ -132,6 +132,14 @@ pub const CLIENT_ACTIVITY_OVERFLOW_CLIENT: &str = "other";
/// Default soft cap for the read-only connection pool.
const READER_POOL_SOFT_CAP: usize = 4;
+/// Size the `-wal` file is truncated back to after a checkpoint (`PRAGMA
+/// journal_size_limit`). SQLite's default keeps the file at its high-water
+/// mark forever, so one large transaction (a `VACUUM`, a backfill, a bulk
+/// import) leaves a `-wal` as big as the database, and a crash then replays it
+/// in full on the next start. This does not cap a running transaction: the WAL
+/// still grows as far as that transaction needs.
+const WAL_JOURNAL_SIZE_LIMIT_BYTES: i64 = 64 * 1024 * 1024;
+
/// Open (and migrate) a [`Store`] rooted at the given data directory.
pub struct Store {
/// Cloneable handle to submit mutations.
@@ -157,6 +165,7 @@ impl Store {
let mut conn = Connection::open(&db_path)?;
conn.pragma_update(None, "journal_mode", "WAL")?;
+ conn.pragma_update(None, "journal_size_limit", WAL_JOURNAL_SIZE_LIMIT_BYTES)?;
conn.pragma_update(None, "synchronous", "NORMAL")?;
conn.pragma_update(None, "busy_timeout", 5_000)?; // ms
@@ -215,6 +224,23 @@ impl Store {
pub fn db_path(&self) -> &Path {
&self.db_path
}
+
+ /// Release the database for process exit: close the read pool (idle
+ /// connections dropped, later reads refused with
+ /// [`StoreError::ReaderClosed`], a read still running dropping its
+ /// connection on return), then have the writer run a final
+ /// `wal_checkpoint(TRUNCATE)`, close its connection and stop.
+ ///
+ /// The server exits without joining the writer thread (detached tasks keep
+ /// clones of [`WriterHandle`] alive), so without this a stop leaves the
+ /// whole `-wal` behind for the next start to replay. Best effort: a
+ /// checkpoint held back by a reader is logged, not returned, and commands
+ /// sent afterwards fail with [`StoreError::WriterClosed`]. Wrap the call in
+ /// a timeout; it waits behind whatever the writer is running.
+ pub async fn close(&self) {
+ self.reader.close();
+ self.writer.checkpoint_and_close().await;
+ }
}
/// Create missing database-directory components with owner-only access while
diff --git a/crates/ai-memory-store/src/ops.rs b/crates/ai-memory-store/src/ops.rs
index e771abc5..3f8b15a7 100644
--- a/crates/ai-memory-store/src/ops.rs
+++ b/crates/ai-memory-store/src/ops.rs
@@ -5566,9 +5566,74 @@ pub(crate) fn reclaim_freed_pages(conn: &Connection) -> StoreResult<()> {
}
batch.push_str("VACUUM;");
conn.execute_batch(&batch)?;
+ truncate_wal_best_effort(conn, "vacuum");
Ok(())
}
+/// The `(busy, log, checkpointed)` row `PRAGMA wal_checkpoint` returns.
+///
+/// `busy` is non-zero when a reader kept the checkpoint from finishing. `log`
+/// and `checkpointed` are WAL frame counts, and both are `-1` when the
+/// database is not in WAL mode.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) struct WalCheckpoint {
+ pub(crate) busy: i64,
+ pub(crate) log: i64,
+ pub(crate) checkpointed: i64,
+}
+
+/// Checkpoint the whole WAL into the database file and truncate it to zero
+/// bytes.
+///
+/// # Errors
+/// Propagates the SQL error. A checkpoint that a reader holds back is not an
+/// error: it is reported through [`WalCheckpoint::busy`].
+pub(crate) fn wal_checkpoint_truncate(conn: &Connection) -> StoreResult<WalCheckpoint> {
+ Ok(
+ conn.query_row("PRAGMA wal_checkpoint(TRUNCATE)", [], |row| {
+ Ok(WalCheckpoint {
+ busy: row.get(0)?,
+ log: row.get(1)?,
+ checkpointed: row.get(2)?,
+ })
+ })?,
+ )
+}
+
+/// Truncate the WAL, and log instead of failing when that cannot be done.
+///
+/// `VACUUM` rewrites every page through the WAL. SQLite's automatic checkpoint
+/// copies those pages back but never shrinks the file, so a `VACUUM` on a
+/// multi-gigabyte database leaves a `-wal` as large as the database until the
+/// next clean close, and a crash or a `SIGKILL` makes the next start replay it
+/// in full. Truncating here, right after the rewrite, is what stops that.
+///
+/// Best effort and deliberately not retried: the operation this follows has
+/// already committed, so a reader that holds the checkpoint back (or a failed
+/// checkpoint) must not turn it into an error. `journal_size_limit` and the
+/// next checkpoint reclaim the space once the reader is gone.
+pub(crate) fn truncate_wal_best_effort(conn: &Connection, after: &'static str) {
+ match wal_checkpoint_truncate(conn) {
+ Ok(WalCheckpoint { busy: 0, .. }) => {}
+ Ok(WalCheckpoint {
+ busy,
+ log,
+ checkpointed,
+ }) => tracing::warn!(
+ after,
+ busy,
+ log,
+ checkpointed,
+ "WAL checkpoint(TRUNCATE) was blocked by an active reader; the -wal file keeps its size until a later checkpoint"
+ ),
+ Err(error) => tracing::warn!(
+ after,
+ %error,
+ "WAL checkpoint(TRUNCATE) failed; the -wal file keeps its size until a later checkpoint"
+ ),
+ }
+}
+
/// What one [`compact`] run reclaimed.
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
pub struct CompactSummary {
@@ -5847,6 +5912,7 @@ pub fn reclaim_ledger_versions(
// `VACUUM` cannot run inside a transaction, and it is the whole-file
// rewrite `compact` already documents the cost of.
conn.execute_batch("VACUUM;")?;
+ truncate_wal_best_effort(conn, "vacuum");
summary.compacted = true;
summary.bytes_after = database_bytes(conn)?;
}
@@ -10872,6 +10938,91 @@ pub(crate) mod tests {
);
}
+ /// A checkpoint a reader holds back must not fail the operation it follows:
+ /// the `VACUUM` has already committed. It is logged with the
+ /// `(busy, log, checkpointed)` triple, left for a later checkpoint, and
+ /// converges once the reader is gone.
+ ///
+ /// The test zeroes `busy_timeout` (rusqlite defaults to 5s) so the blocked
+ /// checkpoint returns at once; the server's writer connection waits up to
+ /// its own 5s timeout for the reader first.
+ #[test]
+ fn compact_survives_a_checkpoint_blocked_by_an_open_reader() {
+ let (tmp, mut conn, ..) = fresh_db();
+ conn.pragma_update(None, "journal_mode", "WAL").unwrap();
+ conn.pragma_update(None, "busy_timeout", 0).unwrap();
+ conn.execute_batch(
+ "CREATE TABLE junk (b BLOB);
+ INSERT INTO junk VALUES (zeroblob(500000));
+ INSERT INTO junk VALUES (zeroblob(500000));
+ DELETE FROM junk;",
+ )
+ .unwrap();
+ let wal = tmp.path().join("test.sqlite-wal");
+
+ // A read transaction pins a snapshot older than the VACUUM's frames.
+ let reader = Connection::open(tmp.path().join("test.sqlite")).unwrap();
+ reader
+ .execute_batch("BEGIN; SELECT count(*) FROM junk;")
+ .unwrap();
+
+ let mut summary = None;
+ let logs = capture_warnings(|| summary = Some(compact(&mut conn)));
+ summary
+ .unwrap()
+ .expect("a blocked checkpoint must not fail the VACUUM that preceded it");
+ // The subscriber colours field names; drop the escape sequences so
+ // the `busy=1` below reads as plain text.
+ let mut plain = String::new();
+ let mut chars = logs.chars();
+ while let Some(c) = chars.next() {
+ if c == '\u{1b}' {
+ chars.by_ref().find(|c| *c == 'm');
+ } else {
+ plain.push(c);
+ }
+ }
+ assert!(
+ plain.contains("blocked by an active reader")
+ && plain.contains("busy=1")
+ && plain.contains("log=")
+ && plain.contains("checkpointed="),
+ "the blocked checkpoint is logged with its triple: {plain:?}"
+ );
+ assert!(
+ std::fs::metadata(&wal).unwrap().len() > 0,
+ "control: the reader really did hold the checkpoint back"
+ );
+
+ // Convergence, no retry loop needed: the next checkpoint finishes.
+ reader.execute_batch("COMMIT;").unwrap();
+ let triple = wal_checkpoint_truncate(&conn).unwrap();
+ assert_eq!(triple.busy, 0);
+ assert_eq!(std::fs::metadata(&wal).unwrap().len(), 0);
+ }
+
+ /// Control for the warning above: an unblocked checkpoint is silent, so a
+ /// log line means a reader really did hold it back.
+ #[test]
+ fn compact_truncates_the_wal_without_warning_when_no_reader_is_open() {
+ let (tmp, mut conn, ..) = fresh_db();
+ conn.pragma_update(None, "journal_mode", "WAL").unwrap();
+ conn.execute_batch(
+ "CREATE TABLE junk (b BLOB);
+ INSERT INTO junk VALUES (zeroblob(500000));
+ DELETE FROM junk;",
+ )
+ .unwrap();
+
+ let mut summary = None;
+ let logs = capture_warnings(|| summary = Some(compact(&mut conn)));
+ summary.unwrap().unwrap();
+
+ assert!(!logs.contains("checkpoint"), "unexpected warning: {logs:?}");
+ let wal = tmp.path().join("test.sqlite-wal");
+ assert_eq!(std::fs::metadata(wal).map_or(0, |m| m.len()), 0);
+ }
+
#[test]
fn page_feedback_is_scoped_rebuildable_and_audited() {
let (_tmp, mut conn, ws, proj) = fresh_db();
diff --git a/crates/ai-memory-store/src/reader.rs b/crates/ai-memory-store/src/reader.rs
index a5e48632..878a0354 100644
--- a/crates/ai-memory-store/src/reader.rs
+++ b/crates/ai-memory-store/src/reader.rs
@@ -2179,10 +2179,18 @@ pub struct ReaderPool {
struct Inner {
db_path: PathBuf,
- pool: Mutex<Vec<Connection>>,
+ pool: Mutex<Pool>,
soft_cap: usize,
}
+/// The idle connections and whether the pool still hands out new ones. One
+/// lock covers both, so a checkout or a return cannot interleave with
+/// [`ReaderPool::close`] between reading the flag and touching the idle list.
+struct Pool {
+ idle: Vec<Connection>,
+ closed: bool,
+}
+
impl ReaderPool {
/// Initialise the pool. Connections are opened lazily on first use.
///
@@ -2193,7 +2201,10 @@ impl ReaderPool {
Ok(Self {
inner: Arc::new(Inner {
db_path: db_path.to_path_buf(),
- pool: Mutex::new(Vec::with_capacity(soft_cap.max(1))),
+ pool: Mutex::new(Pool {
+ idle: Vec::with_capacity(soft_cap.max(1)),
+ closed: false,
+ }),
soft_cap: soft_cap.max(1),
}),
tuning: RetrievalTuning::default(),
@@ -2201,6 +2212,22 @@ impl ReaderPool {
})
}
+ /// Close the pool for shutdown: drop every idle connection, and make every
+ /// later read fail with [`StoreError::ReaderClosed`] instead of opening a
+ /// new one. Idempotent.
+ ///
+ /// A read already running is not interrupted or waited for: it finishes
+ /// on its own connection, and that connection is dropped when it is
+ /// returned rather than pooled again. Such a read can hold a snapshot that
+ /// keeps the writer's final checkpoint from truncating the WAL; the
+ /// checkpoint then logs the blocked attempt and the WAL is replayed on the
+ /// next start, as after any other unclean stop.
+ pub fn close(&self) {
+ let mut pool = self.inner.pool.lock();
+ pool.closed = true;
+ pool.idle.clear();
+ }
+
/// Configure the opt-in ranking signals used by [`Self::hybrid_search`].
/// Only handles cloned from this one afterwards observe the change.
pub fn set_retrieval_tuning(&mut self, tuning: RetrievalTuning) {
@@ -12736,16 +12763,24 @@ fn has_trailing_path_separator(path: &str) -> bool {
}
fn checkout(inner: &Inner) -> StoreResult<Connection> {
- if let Some(conn) = inner.pool.lock().pop() {
- return Ok(conn);
+ {
+ let mut pool = inner.pool.lock();
+ if pool.closed {
+ return Err(StoreError::ReaderClosed);
+ }
+ if let Some(conn) = pool.idle.pop() {
+ return Ok(conn);
+ }
}
open_read_only(&inner.db_path)
}
fn checkin(inner: &Inner, conn: Connection) {
let mut pool = inner.pool.lock();
- if pool.len() < inner.soft_cap {
- pool.push(conn);
+ // A connection returned after `close` is dropped, not pooled: pooling it
+ // would revive a reader the shutdown just released.
+ if !pool.closed && pool.idle.len() < inner.soft_cap {
+ pool.idle.push(conn);
}
}
@@ -12761,6 +12796,39 @@ fn open_read_only(path: &Path) -> StoreResult<Connection> {
#[cfg(test)]
mod tests {
+ /// `close` is a state, not a one-off sweep: a connection checked out
+ /// before it is dropped on return, and nothing opens after it. The control
+ /// half shows the same return pools the connection while the pool is open.
+ #[test]
+ fn a_closed_pool_drops_a_returned_connection_and_refuses_checkout() {
+ use super::{ReaderPool, checkin, checkout};
+ use crate::error::StoreError;
+
+ let tmp = tempfile::TempDir::new().unwrap();
+ let path = tmp.path().join("pool.sqlite");
+ rusqlite::Connection::open(&path).unwrap();
+ let pool = ReaderPool::new(&path, 4).unwrap();
+
+ let open = checkout(&pool.inner).unwrap();
+ checkin(&pool.inner, open);
+ assert_eq!(pool.inner.pool.lock().idle.len(), 1, "control: pooled");
+
+ let held = checkout(&pool.inner).unwrap();
+ pool.close();
+ assert_eq!(pool.inner.pool.lock().idle.len(), 0, "idle set released");
+ checkin(&pool.inner, held);
+ assert_eq!(
+ pool.inner.pool.lock().idle.len(),
+ 0,
+ "a connection returned after close was pooled again"
+ );
+ assert!(matches!(
+ checkout(&pool.inner),
+ Err(StoreError::ReaderClosed)
+ ));
+ pool.close();
+ }
+
/// The percentage is what an operator actually reads, and it divides by a
/// figure that is zero on a fresh database.
#[test]
diff --git a/crates/ai-memory-store/src/writer.rs b/crates/ai-memory-store/src/writer.rs
index ab72c425..0aeb723f 100644
--- a/crates/ai-memory-store/src/writer.rs
+++ b/crates/ai-memory-store/src/writer.rs
@@ -491,6 +491,17 @@ pub(crate) enum WriteCmd {
mode: crate::ops::PurgeMode,
reply: oneshot::Sender<StoreResult<crate::ops::PurgeSessionSummary>>,
},
+ /// Final `wal_checkpoint(TRUNCATE)`, then close the connection and stop
+ /// the actor. See [`WriterHandle::checkpoint_and_close`].
+ Close {
+ reply: oneshot::Sender<()>,
+ },
+ /// Test seam: read `PRAGMA journal_size_limit` off the writer's own
+ /// connection, the only place the pragma is set.
+ #[cfg(test)]
+ JournalSizeLimit {
+ reply: oneshot::Sender<StoreResult<i64>>,
+ },
/// Reclaim free pages on demand: rebuild the FTS indexes and `VACUUM`,
/// deleting nothing. See [`ops::compact`].
Compact {
@@ -2359,6 +2370,31 @@ impl WriterHandle {
rx.await.map_err(|_| StoreError::WriterClosed)?
}
+ /// Truncate the WAL, close the writer's connection and stop the actor,
+ /// waiting until that has happened. Later calls fail with
+ /// [`StoreError::WriterClosed`]. Returns at once if the actor already
+ /// stopped.
+ ///
+ /// Queued behind every command already accepted, so it also waits for a
+ /// long-running one (a `VACUUM`); callers bound it with a timeout. A
+ /// checkpoint a reader holds back is logged, never an error.
+ pub async fn checkpoint_and_close(&self) {
+ let (tx, rx) = oneshot::channel();
+ if self.send(WriteCmd::Close { reply: tx }).await.is_ok() {
+ // The reply follows the checkpoint and the connection close; a
+ // dropped sender means the actor stopped without answering.
+ let _ = rx.await;
+ }
+ }
+
+ /// `PRAGMA journal_size_limit` as the writer's connection reports it.
+ #[cfg(test)]
+ pub(crate) async fn journal_size_limit(&self) -> StoreResult<i64> {
+ let (tx, rx) = oneshot::channel();
+ self.send(WriteCmd::JournalSizeLimit { reply: tx }).await?;
+ rx.await.map_err(|_| StoreError::WriterClosed)?
+ }
+
/// Reclaim free pages on demand, deleting nothing.
///
/// Runs through the writer actor like every other exclusive operation, so
@@ -4240,6 +4276,25 @@ fn worker_loop(mut conn: Connection, mut rx: mpsc::Receiver<WriteCmd>) {
);
send_or_warn(reply, result, "purge_session");
}
+ WriteCmd::Close { reply } => {
+ ops::truncate_wal_best_effort(&conn, "shutdown");
+ // Close before replying, so the caller's return means the
+ // connection is released.
+ drop(conn);
+ send_or_warn(reply, (), "close");
+ // Same drain as `Shutdown`: callers that already hold a
+ // permit see `WriterClosed` instead of waiting forever.
+ rx.close();
+ while rx.blocking_recv().is_some() {}
+ break;
+ }
+ #[cfg(test)]
+ WriteCmd::JournalSizeLimit { reply } => {
+ let result = conn
+ .query_row("PRAGMA journal_size_limit", [], |row| row.get(0))
+ .map_err(StoreError::from);
+ send_or_warn(reply, result, "journal_size_limit");
+ }
WriteCmd::Compact { reply } => {
let result = ops::compact(&mut conn);
send_or_warn(reply, result, "compact");
diff --git a/crates/ai-memory-store/tests/suite/mod.rs b/crates/ai-memory-store/tests/suite/mod.rs
index ac9f3d9f..7e48dc0a 100644
--- a/crates/ai-memory-store/tests/suite/mod.rs
+++ b/crates/ai-memory-store/tests/suite/mod.rs
@@ -32,4 +32,5 @@ mod session_scope_from_observations;
mod sessions_by_agent;
mod slot_visibility;
mod stress_writer_throughput;
+mod wal_bounds;
mod workstream_provenance;
diff --git a/crates/ai-memory-store/tests/suite/wal_bounds.rs b/crates/ai-memory-store/tests/suite/wal_bounds.rs
new file mode 100644
index 00000000..ca5fa251
--- /dev/null
+++ b/crates/ai-memory-store/tests/suite/wal_bounds.rs
@@ -0,0 +1,283 @@
+//! The `-wal` file must not outlive the operation or the process that grew it.
+//!
+//! A `VACUUM` rewrites every page through the WAL, and SQLite's automatic
+//! checkpoint never shrinks the file, so a reclaim on a large store left a
+//! `-wal` as big as the database. Killed before a clean close, the server made
+//! the next start replay all of it. Each test here runs against a real WAL-mode
+//! [`Store`] and reads the `-wal` file's length; a fixture on the in-memory
+//! journal cannot see the defect.
+
+use std::path::PathBuf;
+
+use ai_memory_core::{NewPage, PagePath, Tier};
+use ai_memory_store::{Compaction, PurgeMode, Store, StoreError};
+
+/// Enough ledger versions that a rewrite of the database is megabytes, not
+/// bytes: the assertions compare the `-wal` against zero, and a database whose
+/// whole rewrite is a few pages would hide a missing checkpoint in noise.
+const LEDGER_VERSIONS: usize = 24;
+
+fn wal_path(store: &Store) -> PathBuf {
+ store.db_path().with_extension("sqlite-wal")
+}
+
+/// Length of the `-wal`, zero when SQLite has already deleted it.
+fn wal_len(store: &Store) -> u64 {
+ std::fs::metadata(wal_path(store)).map_or(0, |m| m.len())
+}
+
+/// A ledger body of about 30 KiB in the shape the hooks append, so the
+/// reclaim's content gate accepts every version.
+fn ledger_body(version: usize) -> String {
+ (0..600)
+ .map(|line| {
+ format!(
+ "## [2026-09-05T{version:02}:00:{:02}Z] tool Bash: cargo test\n",
+ line % 60
+ )
+ })
+ .collect()
+}
+
+/// A store with a ledger path carrying [`LEDGER_VERSIONS`] versions.
+async fn store_with_ledger_residue(
+ tmp: &tempfile::TempDir,
+) -> (
+ Store,
+ ai_memory_core::WorkspaceId,
+ ai_memory_core::ProjectId,
+) {
+ let store = Store::open(tmp.path()).unwrap();
+ let ws = store
+ .writer
+ .get_or_create_workspace("default".to_string())
+ .await
+ .unwrap();
+ let proj = store
+ .writer
+ .get_or_create_project(ws, "app".to_string(), None)
+ .await
+ .unwrap();
+ for version in 0..LEDGER_VERSIONS {
+ store
+ .writer
+ .upsert_page(NewPage {
+ workspace_id: ws,
+ project_id: proj,
+ path: PagePath::new("log-2026-09.md").unwrap(),
+ title: "t".into(),
+ body: ledger_body(version),
+ tier: Tier::Episodic,
+ frontmatter_json: serde_json::json!({}),
+ pinned: false,
+ links: Vec::new(),
+ author_id: None,
+ expires_at: None,
+ entities: Vec::new(),
+ evidence: Vec::new(),
+ })
+ .await
+ .unwrap();
+ }
+ (store, ws, proj)
+}
+
+#[tokio::test]
+async fn compact_truncates_the_wal_it_grew() {
+ let tmp = tempfile::tempdir().unwrap();
+ let (store, _, _) = store_with_ledger_residue(&tmp).await;
+ assert!(
+ wal_len(&store) > 0,
+ "fixture: the writes must have left a WAL to truncate"
+ );
+
+ store.writer.compact().await.unwrap();
+
+ assert_eq!(wal_len(&store), 0, "compact left its VACUUM in the -wal");
+}
+
+#[tokio::test]
+async fn reclaiming_ledger_versions_truncates_the_wal_it_grew() {
+ let tmp = tempfile::tempdir().unwrap();
+ let (store, _, _) = store_with_ledger_residue(&tmp).await;
+
+ let summary = store
+ .writer
+ .reclaim_ledger_versions(false, false, Compaction::Reclaim)
+ .await
+ .unwrap();
+
+ assert!(summary.compacted, "fixture: the VACUUM must have run");
+ assert_eq!(summary.pages_deleted, (LEDGER_VERSIONS - 1) as u64);
+ assert_eq!(
+ wal_len(&store),
+ 0,
+ "the reclaim left its VACUUM in the -wal"
+ );
+}
+
+/// Control: without `--compact` there is no `VACUUM`, so the reclaim must not
+/// start checkpointing on its own. Keeps the assertion above from passing
+/// merely because every call truncates.
+#[tokio::test]
+async fn reclaiming_ledger_versions_without_compaction_does_not_vacuum() {
+ let tmp = tempfile::tempdir().unwrap();
+ let (store, _, _) = store_with_ledger_residue(&tmp).await;
+
+ let summary = store
+ .writer
+ .reclaim_ledger_versions(false, false, Compaction::Skip)
+ .await
+ .unwrap();
+
+ assert!(!summary.compacted);
+ assert!(wal_len(&store) > 0, "a delete without VACUUM keeps its WAL");
+}
+
+#[tokio::test]
+async fn purging_a_project_with_compaction_truncates_the_wal() {
+ let tmp = tempfile::tempdir().unwrap();
+ let (store, ws, proj) = store_with_ledger_residue(&tmp).await;
+
+ store
+ .writer
+ .purge_project(
+ ws,
+ proj,
+ "default/app",
+ None,
+ true,
+ Compaction::Reclaim,
+ PurgeMode::Commit,
+ )
+ .await
+ .unwrap();
+
+ assert_eq!(
+ wal_len(&store),
+ 0,
+ "purge --compact left its VACUUM in the -wal"
+ );
+}
+
+/// The shutdown path: `main` exits without joining the writer, so the
+/// checkpoint has to happen before the process goes. A second connection stays
+/// open, as the read pool's would, so SQLite's own "last connection closed"
+/// checkpoint cannot be what empties the file.
+#[tokio::test]
+async fn close_checkpoints_the_wal_even_while_another_connection_is_open() {
+ let tmp = tempfile::tempdir().unwrap();
+ let (store, _, _) = store_with_ledger_residue(&tmp).await;
+ let bystander = rusqlite::Connection::open(store.db_path()).unwrap();
+ bystander
+ .query_row("SELECT count(*) FROM pages", [], |r| r.get::<_, i64>(0))
+ .unwrap();
+ assert!(wal_len(&store) > 0, "fixture: pending WAL before close");
+
+ store.close().await;
+
+ assert_eq!(wal_len(&store), 0, "close left the WAL for the next start");
+ drop(bystander);
+}
+
+/// Releasing the connections is observable: SQLite deletes the `-wal` only when
+/// the last connection to the database closes. A writer or a pooled reader
+/// that survives `close` keeps the file.
+#[tokio::test]
+async fn close_releases_the_writer_and_the_pooled_readers() {
+ let tmp = tempfile::tempdir().unwrap();
+ let (store, _, _) = store_with_ledger_residue(&tmp).await;
+ // Park a read connection in the pool.
+ store.reader.users_exist().await.unwrap();
+ assert!(
+ wal_path(&store).exists(),
+ "fixture: a live WAL before close"
+ );
+
+ store.close().await;
+
+ assert!(
+ !wal_path(&store).exists(),
+ "a connection survived close: SQLite keeps the -wal while one is open"
+ );
+}
+
+#[tokio::test]
+async fn the_writer_refuses_commands_after_close_and_close_is_repeatable() {
+ let tmp = tempfile::tempdir().unwrap();
+ let (store, _, _) = store_with_ledger_residue(&tmp).await;
+
+ store.close().await;
+ store.close().await;
+
+ let err = store
+ .writer
+ .get_or_create_workspace("too-late".to_string())
+ .await
+ .unwrap_err();
+ assert!(matches!(err, StoreError::WriterClosed), "got {err:?}");
+}
+
+/// A read that is running when `close` happens must neither be refused nor
+/// revive the pool: it finishes on its connection, and that connection is
+/// dropped on return. Deterministic: the read parks on a channel until the
+/// test has closed the pool, so there is no sleep and no timing window.
+#[tokio::test]
+async fn a_read_in_flight_across_close_is_not_repooled_and_new_reads_are_refused() {
+ let tmp = tempfile::tempdir().unwrap();
+ let (store, _, _) = store_with_ledger_residue(&tmp).await;
+ let (started_tx, started_rx) = std::sync::mpsc::channel::<()>();
+ let (release_tx, release_rx) = std::sync::mpsc::channel::<()>();
+ let reader = store.reader.clone();
+ let held = tokio::spawn(async move {
+ reader
+ .with_conn(move |conn| {
+ // Touch the database: a read-only connection that never read
+ // has not joined the WAL, and so could not keep the file alive.
+ conn.query_row("SELECT count(*) FROM pages", [], |r| r.get::<_, i64>(0))?;
+ started_tx.send(()).unwrap();
+ release_rx.recv().unwrap();
+ Ok(())
+ })
+ .await
+ });
+ // The connection is checked out and held.
+ tokio::task::spawn_blocking(move || started_rx.recv().unwrap())
+ .await
+ .unwrap();
+
+ store.reader.close();
+
+ let refused = store.reader.users_exist().await.unwrap_err();
+ assert!(
+ matches!(refused, StoreError::ReaderClosed),
+ "a read after close opened a connection: {refused:?}"
+ );
+ release_tx.send(()).unwrap();
+ held.await
+ .unwrap()
+ .expect("the read already in flight finishes");
+
+ // With the writer closed too, no connection may be left: SQLite deletes
+ // the `-wal` only when the last one closes, so a returned connection that
+ // went back into the pool would keep it.
+ store.writer.checkpoint_and_close().await;
+ assert!(
+ !wal_path(&store).exists(),
+ "the connection returned after close was pooled again"
+ );
+}
+
+/// The pragma lives on the writer's own connection and nothing else sets it,
+/// so only a read of that connection can notice it going missing. The fixtures
+/// above are far below the limit and could not.
+#[tokio::test]
+async fn the_writer_connection_caps_the_wal_it_keeps_after_a_checkpoint() {
+ let tmp = tempfile::tempdir().unwrap();
+ let store = Store::open(tmp.path()).unwrap();
+
+ assert_eq!(
+ store.writer.journal_size_limit().await.unwrap(),
+ 64 * 1024 * 1024
+ );
+}
diff --git a/docs/lifecycle-ops.md b/docs/lifecycle-ops.md
index 17a7bb45..dd5c146c 100644
--- a/docs/lifecycle-ops.md
+++ b/docs/lifecycle-ops.md
@@ -75,6 +75,23 @@ pages. The rebuild is needed because an ordinary delete leaves the tokens inside
the index segments, and a `VACUUM` alone does not clear them. It needs free disk space of roughly the database's own size and takes
minutes on a large store, which is why it is opt-in.
+`VACUUM` rewrites every page through the SQLite write-ahead log, so the `-wal`
+file next to `memory.sqlite` grows to roughly the database's size while it runs.
+`compact`, `reclaim-ledger-versions --compact` and the purges with `--compact`
+therefore run `PRAGMA wal_checkpoint(TRUNCATE)` right after the `VACUUM`, and the
+`-wal` goes back to zero bytes. If a reader holds an old snapshot the checkpoint
+cannot finish: the command still succeeds, `ai-memory` logs a `warn` with the
+`busy`, `log` and `checkpointed` counts, and the `-wal` keeps its size until a
+later checkpoint after that reader is gone (the writer waits up to 5 seconds for
+readers before giving up). The writer connection also sets `journal_size_limit`
+to 64 MiB, which is the size the `-wal` is cut back to after any checkpoint. It
+is not a cap: a running transaction, or a reader that blocks the checkpoint, can
+push the file past it. On `SIGINT`, `SIGTERM`, Ctrl-C or the end of stdio input, `serve` closes its read
+connections and runs one last checkpoint, waiting at most 5 seconds for it; a
+read still running keeps its connection until it ends, and the checkpoint is
+best effort. A `SIGKILL` leaves the `-wal` as it was and the next start replays
+it.
+
**`--compact` is not forensic erasure.** The wiki git repository stores page
content in its objects *and its commit messages*, and any backup taken before
the purge still contains everything. Removing the bytes from the live SQLite