Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(purge): stop overstating what purge-project and delete-workspace delete (#540) (#548)
Fabio Akita3f15ae8294c7
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex dfb062b4..953423fb 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -70,6 +70,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0earlier backups still contain it. `docs/lifecycle-ops.md` states thatboundary in a table so it is not inferred from the command name.+- `ai-memory purge-project --compact` and `{"compact": true}` on+ `POST /admin/delete-workspace` reclaim the bytes a delete frees, reusing the+ `Compaction` opt-in that `purge-session` gained in the same cycle (#540). Both+ responses now report `compacted`.++ Compaction rebuilds **all three** FTS5 indexes rather than the two a session+ purge needs. A managed workstream's `workstream_events` rows leave through the+ `projects → workstreams → workstream_events` cascade, and a cascade does not+ fire the `AFTER DELETE` trigger that would drop them from+ `workstream_events_fts` — so the tokens survive an ordinary delete, and a+ `VACUUM` alone does not clear them. A test pins this: removing the third+ rebuild leaves a purged agent transcript's text in the database file.+### Fixed- A failed or skipped embed now leaves a durable record. `page_embeddings`stores only successes, and its upsert rewrites `created_at`, so a global@@ -126,6 +139,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0drain honouring `AI_MEMORY_SERVER_URL` would send captured events to whateveraddress happened to be exported into the process that ran it.+- `purge-project` and `delete-workspace` no longer overstate what they delete+ (#540). The help text promised "Permanently delete a project and ALL its+ data", which reads as byte-level removal neither command performed.++ The deletion was never the defect. Rows go and bytes stay in free pages until+ the file is rewritten — ordinary SQLite behaviour, and the same boundary+ `purge-session` already documented. Measured on a canary token: delete alone+ leaves the text on disk, delete + `VACUUM` still leaves it, and only+ delete + FTS `rebuild` + `VACUUM` removes it. The claim was what needed+ fixing, so both commands now describe the boundary they actually enforce and+ offer the same `--compact` opt-in, and `docs/lifecycle-ops.md` covers all+ three destructive commands in one place instead of documenting the limits of+ one and overstating the other two.+### Docs- `docs/windows.md` gains **Scenario E**, a persistent-server story for nativediff --git a/crates/ai-memory-cli/src/cli.rs b/crates/ai-memory-cli/src/cli.rsindex 1562595e..88973a72 100644--- a/crates/ai-memory-cli/src/cli.rs+++ b/crates/ai-memory-cli/src/cli.rs@@ -145,9 +145,15 @@ pub enum Command {/// `is_latest=false` (they were a multi-project mash-up) so the/// next consolidation can regenerate them per-project. Idempotent.Reorg(ReorgArgs),- /// Permanently delete a project and ALL its data (pages, sessions,- /// observations, handoffs, embeddings, on-disk wiki files).- /// This is irreversible — requires `--confirm`.+ /// Delete a project: its pages, sessions, observations, handoffs,+ /// embeddings, managed workstreams and on-disk wiki files.+ ///+ /// Irreversible — requires `--confirm`. By default this is a logical+ /// delete: the rows are gone and nothing can reach them through the API+ /// or search, but their bytes remain in free pages of the database file+ /// until it is rewritten, as with any SQLite delete. `--compact`+ /// reclaims them; neither mode is forensic erasure, because the wiki git+ /// history and any earlier backup still hold the content.PurgeProject(PurgeProjectArgs),/// Permanently delete ONE session and everything derived from it.///@@ -627,6 +633,19 @@ pub struct PurgeProjectArgs {/// out — purging is destructive and irreversible.#[arg(long)]pub confirm: bool,+ /// Also reclaim the freed bytes: rebuild the FTS indexes and VACUUM the+ /// database after the delete commits.+ ///+ /// Without this the purge is a logical delete — the project is gone from+ /// the API and from search, but its bytes stay in free pages of the+ /// database file until it is next rewritten, as with any SQLite delete.+ ///+ /// This rewrites the WHOLE database and needs free disk space of about+ /// its size, so it takes minutes on a large store. It also does NOT make+ /// the content forensically unrecoverable: the wiki git history and any+ /// backup taken before the purge still contain it.+ #[arg(long)]+ pub compact: bool,}/// Arguments for `purge-session`.diff --git a/crates/ai-memory-cli/src/commands/purge_project.rs b/crates/ai-memory-cli/src/commands/purge_project.rsindex 6ffca85a..9e3a9686 100644--- a/crates/ai-memory-cli/src/commands/purge_project.rs+++ b/crates/ai-memory-cli/src/commands/purge_project.rs@@ -15,6 +15,8 @@ struct PurgeProjectRequest {confirm: bool,/// Purge even when a managed workstream still holds a live run lease.force: bool,+ /// Rebuild the FTS indexes and VACUUM after the delete commits.+ compact: bool,}/// Run the `purge-project` subcommand.@@ -49,6 +51,7 @@ pub async fn run(config: &Config, args: PurgeProjectArgs) -> Result<()> {project: project.clone(),confirm: true,force: args.force,+ compact: args.compact,},).await?;@@ -90,5 +93,15 @@ pub async fn run(config: &Config, args: PurgeProjectArgs) -> Result<()> {failed.len());}+ if report["compacted"].as_bool().unwrap_or(false) {+ println!("Database compacted: freed bytes reclaimed.");+ } else {+ println!(+ "Note: this was a logical delete. The project is unreachable through \+ the API and search, but its bytes remain in the database file until \+ it is rewritten. Re-run with --compact to reclaim them (slow), and \+ see docs/lifecycle-ops.md for what that does and does not guarantee."+ );+ }Ok(())}diff --git a/crates/ai-memory-hooks/src/router.rs b/crates/ai-memory-hooks/src/router.rsindex 327517bd..386dfb96 100644--- a/crates/ai-memory-hooks/src/router.rs+++ b/crates/ai-memory-hooks/src/router.rs@@ -6841,7 +6841,14 @@ mod tests {// points at it (exactly the purge-on-live-server scenario).state.writer- .purge_project(ws, proj, "default/heal-project", None, false)+ .purge_project(+ ws,+ proj,+ "default/heal-project",+ None,+ false,+ ai_memory_store::Compaction::Skip,+ ).await.unwrap();assert!(@@ -7643,7 +7650,14 @@ mod tests {state.writer- .purge_project(ws, proj, "default/repo-root-project", None, false)+ .purge_project(+ ws,+ proj,+ "default/repo-root-project",+ None,+ false,+ ai_memory_store::Compaction::Skip,+ ).await.unwrap();diff --git a/crates/ai-memory-mcp/src/admin.rs b/crates/ai-memory-mcp/src/admin.rsindex 40f70c96..05e27f59 100644--- a/crates/ai-memory-mcp/src/admin.rs+++ b/crates/ai-memory-mcp/src/admin.rs@@ -21,11 +21,13 @@//! - `POST /admin/commit` — stage + commit the wiki tree via git.//! - `GET /admin/checkpoints` — list recent wiki git checkpoints.//! - `POST /admin/restore-page` — restore one page from a checkpoint.-//! - `POST /admin/purge-project` — delete a project and all its data.+//! - `POST /admin/purge-project` — delete a project's rows and wiki files+//! (logical unless `compact` is set; see `ai_memory_store::Compaction`).//! - `POST /admin/purge-session` — delete one session and what it derived.//! - `POST /admin/rename-project` — rename a project (column-only; no files move).//! - `POST /admin/rename-workspace` — rename a workspace and refresh scope manifests.-//! - `POST /admin/delete-workspace` — delete a workspace and all of its projects.+//! - `POST /admin/delete-workspace` — delete a workspace and its projects+//! (logical unless `compact` is set; see `ai_memory_store::Compaction`).//! - `POST /admin/merge-workspace` — fold every project of one workspace into//! another, then delete the emptied source workspace.//! - `POST /admin/move-project` — move a project into another workspace@@ -3468,6 +3470,11 @@ struct PurgeProjectRequest {/// out from under a running agent, which then cannot save its history.#[serde(default)]force: bool,+ /// Reclaim freed bytes: rebuild the FTS indexes and `VACUUM` after the+ /// delete commits. Off by default; see `ai_memory_store::Compaction` for+ /// the cost and for what it does not guarantee.+ #[serde(default)]+ compact: bool,}/// Wire-format summary returned by `POST /admin/purge-project`.@@ -3496,6 +3503,10 @@ pub struct PurgeProjectReport {pub files_deleted: Vec<String>,/// Paths that could not be removed from disk (non-fatal; DB rows are gone).pub files_failed: Vec<String>,+ /// Whether the freed bytes were reclaimed (`VACUUM` ran). False for a+ /// plain logical delete: the rows are gone from the API and from search,+ /// but their bytes stay in free pages until the file is next rewritten.+ pub compacted: bool,/// Pre-purge checkpoint, if the tree had uncommitted changes.#[serde(skip_serializing_if = "Option::is_none")]pub pre_checkpoint: Option<String>,@@ -3630,9 +3641,15 @@ async fn handle_purge_project(Err(e) => return e,};+ let compaction = if req.compact {+ ai_memory_store::Compaction::Reclaim+ } else {+ ai_memory_store::Compaction::Skip+ };+let summary = match state.writer- .purge_project(ws_id, proj_id, &label, author_id, req.force)+ .purge_project(ws_id, proj_id, &label, author_id, req.force, compaction).await{Ok(s) => s,@@ -3688,6 +3705,7 @@ async fn handle_purge_project(workstreams_deleted: summary.workstreams_deleted,managed_runs_deleted: summary.managed_runs_deleted,workstream_ids: summary.workstream_ids,+ compacted: summary.compacted,files_deleted,files_failed,pre_checkpoint,@@ -3785,6 +3803,11 @@ struct DeleteWorkspaceRequest {/// non-empty workspace is refused so a typo can't wipe live data.#[serde(default)]force: bool,+ /// Reclaim freed bytes: rebuild the FTS indexes and `VACUUM` after the+ /// delete commits. Off by default; see `ai_memory_store::Compaction` for+ /// the cost and for what it does not guarantee.+ #[serde(default)]+ compact: bool,}/// Wire-format summary returned by `POST /admin/delete-workspace`.@@ -3807,6 +3830,10 @@ pub struct DeleteWorkspaceResult {pub files_deleted: Vec<String>,/// Paths that could not be removed from disk (non-fatal; DB rows are gone).pub files_failed: Vec<String>,+ /// Whether the freed bytes were reclaimed (`VACUUM` ran). False for a+ /// plain logical delete: the rows are gone from the API and from search,+ /// but their bytes stay in free pages until the file is next rewritten.+ pub compacted: bool,/// Pre-delete checkpoint, if the tree had uncommitted changes.#[serde(skip_serializing_if = "Option::is_none")]pub pre_checkpoint: Option<String>,@@ -3826,7 +3853,12 @@ async fn handle_delete_workspace(let actor = actor_ext.map(|axum::Extension(a)| a).unwrap_or_else(ai_memory_core::ActorContext::anonymous);- match delete_workspace_core(&state, &req.workspace, req.force, actor).await {+ let compaction = if req.compact {+ ai_memory_store::Compaction::Reclaim+ } else {+ ai_memory_store::Compaction::Skip+ };+ match delete_workspace_core(&state, &req.workspace, req.force, actor, compaction).await {Ok(result) => (StatusCode::OK,Json(serde_json::to_value(&result).unwrap_or(serde_json::Value::Null)),@@ -3845,6 +3877,7 @@ async fn delete_workspace_core(workspace: &str,force: bool,actor: ai_memory_core::ActorContext,+ compaction: ai_memory_store::Compaction,) -> Result<DeleteWorkspaceResult, MoveErr> {let ws_id = lookup_ws_no_create(state, workspace).await?;@@ -3886,7 +3919,11 @@ async fn delete_workspace_core(let pre_checkpoint =checkpoint_or_500(&state.wiki, format!("pre-delete-workspace {workspace}"))?;- let summary = match state.writer.delete_workspace(ws_id, force).await {+ let summary = match state+ .writer+ .delete_workspace(ws_id, force, compaction)+ .await+ {Ok(s) => s,Err(e) => {let status = match &e {@@ -3936,6 +3973,7 @@ async fn delete_workspace_core(workstreams_deleted: summary.workstreams_deleted,managed_runs_deleted: summary.managed_runs_deleted,workstream_ids: summary.workstream_ids,+ compacted: summary.compacted,files_deleted,files_failed,pre_checkpoint,@@ -5691,9 +5729,20 @@ async fn copy_purge_merge(// The source purge is an internal step of move-project (a distinct op that// records its own move report); it is not attributed as a standalone// `purge_project` here, so the audit author is left NULL.+ // `Skip`: compaction is an operator's explicit choice on a destructive+ // command, not a side effect of moving a project. A `VACUUM` here would+ // rewrite the whole database in the middle of a move the caller asked to+ // be cheap.let summary = match state.writer- .purge_project(src_ws, src_proj, &label, None, false)+ .purge_project(+ src_ws,+ src_proj,+ &label,+ None,+ false,+ ai_memory_store::Compaction::Skip,+ ).await{Ok(s) => s,@@ -5914,21 +5963,29 @@ async fn handle_merge_workspace(// Every project moved → the source workspace is now empty. Delete the shell// with force=false: it must be empty, so a race that repopulated it aborts// safely without wiping fresh data.- let source_workspace_deleted =- match delete_workspace_core(&state, &req.from, false, actor).await {- Ok(_) => true,- Err((_status, body)) => {- // The moves succeeded; only the final empty-shell delete failed- // (e.g. a concurrent write recreated a project). No data was- // lost, so report success-with-caveat instead of a hard error.- warn!(- workspace = %req.from,- "merge-workspace: source drained but shell delete failed: {:?}",- body- );- false- }- };+ let source_workspace_deleted = match delete_workspace_core(+ &state,+ &req.from,+ false,+ actor,+ // The shell is empty by this point; there is nothing to reclaim.+ ai_memory_store::Compaction::Skip,+ )+ .await+ {+ Ok(_) => true,+ Err((_status, body)) => {+ // The moves succeeded; only the final empty-shell delete failed+ // (e.g. a concurrent write recreated a project). No data was+ // lost, so report success-with-caveat instead of a hard error.+ warn!(+ workspace = %req.from,+ "merge-workspace: source drained but shell delete failed: {:?}",+ body+ );+ false+ }+ };let report = MergeWorkspaceReport {from: req.from,diff --git a/crates/ai-memory-store/src/ops.rs b/crates/ai-memory-store/src/ops.rsindex bffb0ae1..db22fc82 100644--- a/crates/ai-memory-store/src/ops.rs+++ b/crates/ai-memory-store/src/ops.rs@@ -148,6 +148,9 @@ pub struct PurgeSummary {/// pre-delete identifiers to remove `raw/workstreams/<id>/` after the SQL/// transaction commits and to report any filesystem partial failure.pub workstream_ids: Vec<String>,+ /// Whether the freed bytes were reclaimed (`VACUUM` ran). False for a+ /// plain logical delete.+ pub compacted: bool,}use jiff::Timestamp;use rusqlite::{Connection, OptionalExtension, Transaction, params};@@ -2956,7 +2959,13 @@ pub fn insert_wiki_migration(Ok(())}-/// Delete a project and all its data inside one transaction.+/// Delete a project's rows inside one transaction.+///+/// This is a *logical* delete unless `compaction` is [`Compaction::Reclaim`].+/// The rows are gone and nothing reaches them through the API or search, but+/// their bytes remain in free pages of the database file until it is+/// rewritten — as with any SQLite delete. Neither mode is forensic erasure:+/// the wiki git history and any earlier backup still hold the content.////// Execution order:/// 1. Count rows in each dependent table (pages/all versions, sessions,@@ -2981,7 +2990,9 @@ pub fn insert_wiki_migration(/// Returns [`StoreError::ManagedRunActive`] when a managed run's lease is/// still live and `force` is false, or [`StoreError`] if any SQL statement/// fails. The transaction is rolled back automatically on error.-/// Whether [`purge_session`] should reclaim the freed bytes afterwards.+/// Whether a destructive scope operation should reclaim the freed bytes+/// afterwards. Shared by [`purge_session`], [`purge_project`] and+/// [`delete_workspace`] so the family makes one promise, not three.////// A purge is a logical delete: the rows are gone and nothing can reach them/// through the API or through search, but their bytes remain in free pages of@@ -3011,6 +3022,30 @@ pub enum Compaction {Reclaim,}+/// Rebuild every FTS5 index, then `VACUUM`. Runs after the caller's+/// transaction has committed: `VACUUM` cannot run inside one, and a rebuild is+/// a whole-index operation we do not want holding the write lock for the+/// duration of a delete.+///+/// All three indexes are rebuilt regardless of which scope was deleted. They+/// are `content=`-backed external-content tables, so an ordinary delete leaves+/// the tokens behind inside the index segments and only a `rebuild` clears+/// them — and a project or workspace delete cascades `workstream_events` just+/// as it does `pages` and `observations`. Rebuilding only the indexes a given+/// caller "should" have touched is exactly the reasoning that leaves text+/// searchable after a purge, and the cost is dominated by the `VACUUM` that+/// follows in every case. A `rebuild` is idempotent, so the extra work is a+/// no-op for a scope that deleted nothing from that table.+fn reclaim_freed_pages(conn: &Connection) -> StoreResult<()> {+ conn.execute_batch(+ "INSERT INTO observations_fts(observations_fts) VALUES('rebuild'); \+ INSERT INTO pages_fts(pages_fts) VALUES('rebuild'); \+ INSERT INTO workstream_events_fts(workstream_events_fts) VALUES('rebuild'); \+ VACUUM;",+ )?;+ Ok(())+}+/// What [`purge_session`] removed. All counts are rows actually deleted.#[derive(Debug, Default, Clone, PartialEq, Eq)]pub struct PurgeSessionSummary {@@ -3220,15 +3255,8 @@ pub fn purge_session(tx.commit()?;- // Outside the transaction: `VACUUM` cannot run inside one, and a rebuild- // is a whole-index operation we do not want holding the write lock for- // the duration of the delete.if compaction == Compaction::Reclaim {- conn.execute_batch(- "INSERT INTO observations_fts(observations_fts) VALUES('rebuild'); \- INSERT INTO pages_fts(pages_fts) VALUES('rebuild'); \- VACUUM;",- )?;+ reclaim_freed_pages(conn)?;}Ok(PurgeSessionSummary {@@ -3248,6 +3276,7 @@ pub fn purge_project(workspace_project_label: &str,author_id: Option<ai_memory_core::UserId>,force: bool,+ compaction: Compaction,) -> StoreResult<PurgeSummary> {let tx = conn.transaction()?;@@ -3376,6 +3405,11 @@ pub fn purge_project()?;tx.commit()?;++ if compaction == Compaction::Reclaim {+ reclaim_freed_pages(conn)?;+ }+Ok(PurgeSummary {label: workspace_project_label.to_string(),page_paths,@@ -3387,6 +3421,7 @@ pub fn purge_project(workstreams_deleted,managed_runs_deleted,workstream_ids,+ compacted: compaction == Compaction::Reclaim,})}@@ -3403,6 +3438,9 @@ pub struct DeleteWorkspaceSummary {pub managed_runs_deleted: u64,/// Pre-delete identifiers for post-commit raw-segment cleanup.pub workstream_ids: Vec<String>,+ /// Whether the freed bytes were reclaimed (`VACUUM` ran). False for a+ /// plain logical delete.+ pub compacted: bool,}/// Delete a workspace row. Refuses a workspace that still holds projects@@ -3419,6 +3457,7 @@ pub fn delete_workspace(conn: &mut Connection,workspace_id: &WorkspaceId,force: bool,+ compaction: Compaction,) -> StoreResult<DeleteWorkspaceSummary> {let tx = conn.transaction()?;let wid = workspace_id.as_bytes();@@ -3458,12 +3497,18 @@ pub fn delete_workspace(return Err(StoreError::NotFound("workspace".into()));}tx.commit()?;++ if compaction == Compaction::Reclaim {+ reclaim_freed_pages(conn)?;+ }+Ok(DeleteWorkspaceSummary {projects_deleted,pages_deleted,workstreams_deleted,managed_runs_deleted,workstream_ids,+ compacted: compaction == Compaction::Reclaim,})}@@ -4295,7 +4340,7 @@ pub(crate) mod tests {let prepared = open_managed_run(&mut conn, &ws, &proj);// Non-empty (holds the "scratch" project + a page) → refused w/o force.- let err = delete_workspace(&mut conn, &ws, false).unwrap_err();+ let err = delete_workspace(&mut conn, &ws, false, Compaction::Skip).unwrap_err();assert!(matches!(err, StoreError::WorkspaceNotEmpty(n) if n >= 1),"expected WorkspaceNotEmpty, got {err:?}"@@ -4310,7 +4355,7 @@ pub(crate) mod tests {assert_eq!(ws_still, 1, "refused delete must not touch the row");// Force cascades: project + page gone, workspace row gone.- let summary = delete_workspace(&mut conn, &ws, true).unwrap();+ let summary = delete_workspace(&mut conn, &ws, true, Compaction::Skip).unwrap();assert!(summary.projects_deleted >= 1 && summary.pages_deleted >= 1,"{summary:?}"@@ -4332,7 +4377,7 @@ pub(crate) mod tests {// Deleting again → NotFound.assert!(matches!(- delete_workspace(&mut conn, &ws, true).unwrap_err(),+ delete_workspace(&mut conn, &ws, true, Compaction::Skip).unwrap_err(),StoreError::NotFound(_)));}@@ -4341,7 +4386,7 @@ pub(crate) mod tests {fn delete_workspace_empty_succeeds_without_force() {let (_tmp, mut conn, _ws, _proj) = fresh_db();let empty = get_or_create_workspace(&mut conn, "orphan-ws").unwrap();- let summary = delete_workspace(&mut conn, &empty, false).unwrap();+ let summary = delete_workspace(&mut conn, &empty, false, Compaction::Skip).unwrap();assert_eq!(summary.projects_deleted, 0);assert_eq!(summary.pages_deleted, 0);assert_eq!(summary.workstreams_deleted, 0);@@ -4451,6 +4496,178 @@ pub(crate) mod tests {conn.query_row(sql, [], |r| r.get(0)).unwrap()}+ /// Insert one managed workstream event carrying a distinctive marker.+ /// `workstream_events` cascades out of `workstreams`, which cascades out+ /// of `projects` — so a project purge removes these rows without ever+ /// running a `DELETE` statement against the table itself.+ fn seed_workstream_event(+ conn: &Connection,+ workstream_id: &ai_memory_core::WorkstreamId,+ marker: &str,+ ) {+ conn.execute(+ "INSERT INTO workstream_events (workstream_id, sequence, event_id, agent_kind, \+ native_session_id, kind, role, content, created_at) \+ VALUES (?1, 1, ?2, 'claude-code', 'native-1', 'message', 'user', ?3, ?4)",+ rusqlite::params![+ workstream_id.as_bytes(),+ format!("evt-{marker}"),+ format!("evt-body-{marker}"),+ Timestamp::now().as_microsecond(),+ ],+ )+ .unwrap();+ }++ /// The default for a project purge is the same logical delete `#387`+ /// documented for a session purge. Pinned so the CLI help, the admin route+ /// docs and `docs/lifecycle-ops.md` cannot drift away from the behaviour:+ /// if this starts failing, byte-level removal became the default and all+ /// three need updating together.+ #[test]+ fn purge_project_is_a_logical_delete_by_default() {+ let (tmp, mut conn, ws, proj) = fresh_db();+ let db_path = tmp.path().join("test.sqlite");+ seed_session(&mut conn, ws, proj, "canaryproj");++ let summary = purge_project(+ &mut conn,+ &ws,+ &proj,+ "default/scratch",+ None,+ false,+ Compaction::Skip,+ )+ .unwrap();+ assert!(!summary.compacted, "the default does not VACUUM");++ conn.execute_batch("PRAGMA wal_checkpoint(TRUNCATE);").ok();+ let bytes = std::fs::read(&db_path).unwrap();+ assert!(+ bytes.windows(14).any(|w| w == b"obs-canaryproj"),+ "documented boundary: purge-project is a logical delete. If this now \+ fails, byte-level removal became the default and the #540 note, the \+ CLI help and docs/lifecycle-ops.md all need updating to match."+ );+ }++ /// The opt-in half. Paired with `purge_project_is_a_logical_delete_by_default`+ /// so the two together pin the difference between what the command promises+ /// and what it offers.+ #[test]+ fn purge_project_with_reclaim_removes_the_bytes_from_the_file() {+ let (tmp, mut conn, ws, proj) = fresh_db();+ let db_path = tmp.path().join("test.sqlite");+ seed_session(&mut conn, ws, proj, "canaryproj");+ // A second project in the same workspace must survive untouched.+ let other = get_or_create_project(&mut conn, &ws, "keeper", None).unwrap();+ seed_session(&mut conn, ws, other, "survivor");++ let summary = purge_project(+ &mut conn,+ &ws,+ &proj,+ "default/scratch",+ None,+ false,+ Compaction::Reclaim,+ )+ .unwrap();+ assert!(summary.compacted, "the summary reports that VACUUM ran");++ conn.execute_batch("PRAGMA wal_checkpoint(TRUNCATE);").ok();+ let bytes = std::fs::read(&db_path).unwrap();+ assert!(+ !bytes.windows(14).any(|w| w == b"obs-canaryproj"),+ "Reclaim must remove the purged project's text from the database file"+ );+ assert!(+ bytes.windows(12).any(|w| w == b"obs-survivor"),+ "the sibling project's text is untouched by the compaction"+ );+ }++ /// The reason `reclaim_freed_pages` rebuilds all three FTS indexes rather+ /// than the two a session purge needs.+ ///+ /// `workstream_events` is an external-content FTS5 table whose rows leave+ /// via the `projects` → `workstreams` → `workstream_events` cascade. A+ /// cascade does not fire the `AFTER DELETE` trigger that would remove the+ /// row from the index (`recursive_triggers` is off), so both the index+ /// entry and its tokens survive the delete. Rebuilding only `pages_fts`+ /// and `observations_fts` — the set a session purge needs — would leave a+ /// managed agent's transcript text in the file after an operator asked for+ /// it to be reclaimed.+ #[test]+ fn reclaim_clears_workstream_event_text_that_left_via_cascade() {+ let (tmp, mut conn, ws, proj) = fresh_db();+ let db_path = tmp.path().join("test.sqlite");+ let prepared = open_managed_run(&mut conn, &ws, &proj);+ seed_workstream_event(&conn, &prepared.workstream_id, "canaryevt");++ // Present before the purge, both in the file and in the index.+ let indexed: i64 = conn+ .query_row(+ "SELECT COUNT(*) FROM workstream_events_fts \+ WHERE workstream_events_fts MATCH '\"canaryevt\"'",+ [],+ |r| r.get(0),+ )+ .unwrap();+ assert_eq!(indexed, 1, "the event is searchable before the purge");++ purge_project(+ &mut conn,+ &ws,+ &proj,+ "default/scratch",+ None,+ true,+ Compaction::Reclaim,+ )+ .unwrap();++ conn.execute_batch("PRAGMA wal_checkpoint(TRUNCATE);").ok();+ let bytes = std::fs::read(&db_path).unwrap();+ // Assert on the *token*, not the whole hyphenated body. The body only+ // ever existed contiguously in the content table, which `VACUUM` clears+ // on its own; what survives an ordinary delete is the tokenized form+ // inside the FTS5 segments, and only a `rebuild` removes that. Asserting+ // on the full string would pass with or without the rebuild and prove+ // nothing.+ assert!(+ !bytes.windows(9).any(|w| w == b"canaryevt"),+ "Reclaim must clear workstream event tokens; if this fails, the \+ workstream_events_fts rebuild was dropped from reclaim_freed_pages \+ and a managed transcript's text survives a compacted purge"+ );+ }++ /// `delete_workspace` takes the same path and makes the same promise.+ #[test]+ fn delete_workspace_reclaim_removes_the_bytes_and_reports_it() {+ let (tmp, mut conn, ws, proj) = fresh_db();+ let db_path = tmp.path().join("test.sqlite");+ seed_session(&mut conn, ws, proj, "canaryws");++ let skipped = {+ let other = get_or_create_workspace(&mut conn, "doomed").unwrap();+ delete_workspace(&mut conn, &other, false, Compaction::Skip).unwrap()+ };+ assert!(!skipped.compacted, "the default does not VACUUM");++ let summary = delete_workspace(&mut conn, &ws, true, Compaction::Reclaim).unwrap();+ assert!(summary.compacted, "the summary reports that VACUUM ran");++ conn.execute_batch("PRAGMA wal_checkpoint(TRUNCATE);").ok();+ let bytes = std::fs::read(&db_path).unwrap();+ assert!(+ !bytes.windows(12).any(|w| w == b"obs-canaryws"),+ "Reclaim must remove the deleted workspace's text from the file"+ );+ }+/// #387, the reporter's reproduction: after purging one session by UUID,/// its rows are gone from every table that held them.#[test]@@ -8227,8 +8444,16 @@ pub(crate) mod tests {let (_tmp, mut conn, ws, proj) = fresh_db();// Simulate the post-purge state: project row gone.// `purge_project` drives the cascading deletes we want here.- let _ = purge_project(&mut conn, &ws, &proj, "default/scratch", None, false)- .expect("purge of fresh project should succeed");+ let _ = purge_project(+ &mut conn,+ &ws,+ &proj,+ "default/scratch",+ None,+ false,+ Compaction::Skip,+ )+ .expect("purge of fresh project should succeed");// Now try to rename the project that no longer exists. The// pre-fix code returned `Ok(())` because `UPDATE` affected// zero rows. The fix returns `NotFound` so admin handlers@@ -8289,6 +8514,7 @@ pub(crate) mod tests {"default/scratch",Some(author),false,+ Compaction::Skip,).expect("purge should succeed");@@ -8336,8 +8562,16 @@ pub(crate) mod tests {let (_tmp, mut conn, ws, proj) = fresh_db();open_managed_run(&mut conn, &ws, &proj);- let err = purge_project(&mut conn, &ws, &proj, "default/scratch", None, false)- .expect_err("an active managed run must block the purge");+ let err = purge_project(+ &mut conn,+ &ws,+ &proj,+ "default/scratch",+ None,+ false,+ Compaction::Skip,+ )+ .expect_err("an active managed run must block the purge");match err {StoreError::ManagedRunActive { count, workstreams } => {@@ -8369,8 +8603,16 @@ pub(crate) mod tests {let (_tmp, mut conn, ws, proj) = fresh_db();let prepared = open_managed_run(&mut conn, &ws, &proj);- let summary = purge_project(&mut conn, &ws, &proj, "default/scratch", None, true)- .expect("force purges regardless of the live lease");+ let summary = purge_project(+ &mut conn,+ &ws,+ &proj,+ "default/scratch",+ None,+ true,+ Compaction::Skip,+ )+ .expect("force purges regardless of the live lease");assert_eq!(summary.workstreams_deleted, 1);assert_eq!(summary.managed_runs_deleted, 1);@@ -8397,8 +8639,16 @@ pub(crate) mod tests {).unwrap();- let summary = purge_project(&mut conn, &ws, &proj, "default/scratch", None, false)- .expect("a lapsed lease is not a running agent");+ let summary = purge_project(+ &mut conn,+ &ws,+ &proj,+ "default/scratch",+ None,+ false,+ Compaction::Skip,+ )+ .expect("a lapsed lease is not a running agent");assert_eq!(summary.workstreams_deleted, 1);assert_eq!(summary.managed_runs_deleted, 1);diff --git a/crates/ai-memory-store/src/writer.rs b/crates/ai-memory-store/src/writer.rsindex 7ea49a05..433659dc 100644--- a/crates/ai-memory-store/src/writer.rs+++ b/crates/ai-memory-store/src/writer.rs@@ -295,9 +295,12 @@ pub(crate) enum WriteCmd {dim: u32,reply: oneshot::Sender<StoreResult<u64>>,},- /// Delete a project and all its data (pages, sessions, observations,- /// handoffs, embeddings) in one transaction. Returns the paths of- /// every page file that must be removed from disk by the caller.+ /// Delete a project's rows (pages, sessions, observations, handoffs,+ /// embeddings) in one transaction. Returns the paths of every page file+ /// that must be removed from disk by the caller.+ ///+ /// A logical delete unless `compaction` is [`ops::Compaction::Reclaim`]:+ /// freed bytes stay in the file until it is rewritten.PurgeProject {workspace_id: WorkspaceId,project_id: ProjectId,@@ -308,6 +311,8 @@ pub(crate) enum WriteCmd {author_id: Option<ai_memory_core::UserId>,/// Purge even when a managed workstream still holds a live run lease.force: bool,+ /// Whether to reclaim the freed bytes afterwards (`VACUUM`).+ compaction: crate::ops::Compaction,reply: oneshot::Sender<StoreResult<PurgeSummary>>,},/// Delete one session and everything derived from it, inside a single@@ -327,6 +332,8 @@ pub(crate) enum WriteCmd {DeleteWorkspace {workspace_id: WorkspaceId,force: bool,+ /// Whether to reclaim the freed bytes afterwards (`VACUUM`).+ compaction: crate::ops::Compaction,reply: oneshot::Sender<StoreResult<DeleteWorkspaceSummary>>,},/// Rename a workspace's `name` column (UUID-keyed dir doesn't move).@@ -1273,7 +1280,8 @@ impl WriterHandle {rx.await.map_err(|_| StoreError::WriterClosed)?}- /// Delete a project and all its data in one atomic transaction.+ /// Delete a project's rows in one atomic transaction. Logical unless+ /// `compaction` is [`ops::Compaction::Reclaim`].////// ON DELETE CASCADE propagates the delete through pages, sessions,/// observations, handoffs, and page_embeddings automatically. The@@ -1290,6 +1298,7 @@ impl WriterHandle {label: impl Into<String>,author_id: Option<ai_memory_core::UserId>,force: bool,+ compaction: crate::ops::Compaction,) -> StoreResult<PurgeSummary> {let (tx, rx) = oneshot::channel();self.send(WriteCmd::PurgeProject {@@ -1298,6 +1307,7 @@ impl WriterHandle {label: label.into(),author_id,force,+ compaction,reply: tx,}).await?;@@ -1347,11 +1357,13 @@ impl WriterHandle {&self,workspace_id: WorkspaceId,force: bool,+ compaction: crate::ops::Compaction,) -> StoreResult<DeleteWorkspaceSummary> {let (tx, rx) = oneshot::channel();self.send(WriteCmd::DeleteWorkspace {workspace_id,force,+ compaction,reply: tx,}).await?;@@ -2345,6 +2357,7 @@ fn worker_loop(mut conn: Connection, mut rx: mpsc::Receiver<WriteCmd>) {label,author_id,force,+ compaction,reply,} => {let result = ops::purge_project(@@ -2354,6 +2367,7 @@ fn worker_loop(mut conn: Connection, mut rx: mpsc::Receiver<WriteCmd>) {&label,author_id,force,+ compaction,);send_or_warn(reply, result, "purge_project");}@@ -2378,9 +2392,10 @@ fn worker_loop(mut conn: Connection, mut rx: mpsc::Receiver<WriteCmd>) {WriteCmd::DeleteWorkspace {workspace_id,force,+ compaction,reply,} => {- let result = ops::delete_workspace(&mut conn, &workspace_id, force);+ let result = ops::delete_workspace(&mut conn, &workspace_id, force, compaction);send_or_warn(reply, result, "delete_workspace");}WriteCmd::RenameWorkspace {diff --git a/docs/lifecycle-ops.md b/docs/lifecycle-ops.mdindex d3b0c73c..65161ed3 100644--- a/docs/lifecycle-ops.md+++ b/docs/lifecycle-ops.md@@ -8,12 +8,12 @@ on a homelab box where mistakes are harder to undo.| Command | Safe with server **running**? | Wipes data? | Reversible? | Notes ||---|---|---|---|---|-| `purge-project --confirm` | ✅ yes | the one project's data | no | Deletes the UUID-namespaced wiki root and raw workstream segments; sibling projects remain untouched. Refuses with `409` while a managed workstream under the project holds a live run lease — `--force` overrides. |+| `purge-project --confirm` | ✅ yes | the one project's data | no | Deletes the UUID-namespaced wiki root and raw workstream segments; sibling projects remain untouched. Refuses with `409` while a managed workstream under the project holds a live run lease — `--force` overrides. Logical delete by default; `--compact` additionally rebuilds the FTS indexes and `VACUUM`s (see below). || `purge-session --session-id --confirm` | ✅ yes | the one session's data | no | Deletes one session by UUID: its row, its observations, the handoffs it **authored**, its `sessions/<id>.md` page and every superseded version, their embeddings, and its auto-improve runs. Strictly scoped — a session that does not belong to the named workspace/project is a `404` and nothing is deleted. Handoffs the session only *accepted* are kept: that text belongs to the session that wrote it. Logical delete by default; `--compact` additionally rebuilds the FTS indexes and `VACUUM`s (see below). || `handoffs --expire-all --confirm` | ✅ yes | no (state change only) | no (but nothing is destroyed) | Marks every **open** handoff in the scope `expired` so it stops being offered to an agent. Rows, summaries and provenance are kept and stay visible in the audit log. Unlike the automatic sweep it does **not** spare manual handoffs or ones from another directory — those exemptions are exactly what a leftover backlog is made of, so honouring them would clear nothing. `--older-than-days N` keeps recent batons. Owner-scoped: never touches another user's baton. || `rename-project --from --to` | ✅ yes | no | yes (rename back) | Column-only update on `projects.name`. The on-disk dir is keyed by `project_id` (UUID), so the rename never moves a file. || `/admin/rename-workspace` | ✅ yes | no | yes (rename back) | Column-only update on `workspaces.name`; refreshes `_meta.md` scope manifests and checkpoints the wiki tree. |-| `/admin/delete-workspace` | ✅ yes | the workspace and every child project | no | Runs `purge_workspace` admission first, deletes SQLite rows in one cascade, removes the UUID-keyed workspace directory and managed-workstream raw segments, reports filesystem partial failures, and dispatches mirror notification after durable work. |+| `/admin/delete-workspace` | ✅ yes | the workspace and every child project | no | Runs `purge_workspace` admission first, deletes SQLite rows in one cascade, removes the UUID-keyed workspace directory and managed-workstream raw segments, reports filesystem partial failures, and dispatches mirror notification after durable work. Logical delete by default; `"compact": true` additionally rebuilds the FTS indexes and `VACUUM`s (see below). || `move-project --confirm` | ✅ yes | source only in the merge case (a `Reject`-policy `purge_project` webhook can still abort the source teardown leaving everything intact) | no | Fresh destination → lossless **true move** (re-stamp `workspace_id`, keep `project_id`, rename the dir): sessions/observations/handoffs + history all survive. Destination with a same-named project → **copy+purge merge**: only latest pages migrate. || `move-session <id> --to --confirm` | ✅ yes | no | yes (move it back) | Re-stamps one session (or every session touching `--from-project`) into another project: `sessions`, `observations`, its `handoffs`, consolidation jobs, auto-improve runs/claims and its `sessions/<id>.md` page, one transaction per session; the page file moves with it (`--pages move`, default) or is retired for regeneration. Without `--confirm` it is a real dry run (rolled back). Refuses with `409` an open session or a pending consolidation job unless `--force`. || `backup --to` | ✅ yes | no | n/a | Streams a gzipped tarball from the server's online `sqlite3 .backup` plus the wiki tree. Safe alongside the live writer. |@@ -29,7 +29,7 @@ fundamentally cannot run while another process holds the SQLite WAL writer. See[CLAUDE.md §16](../CLAUDE.md) for the invariant.-## `purge-session` and what "deleted" means+## What "deleted" means (`purge-session`, `purge-project`, `delete-workspace`)`purge-session` answers *"forget this conversation"*: after it runs, thesession is gone from the API, from `status` counts and from search.@@ -65,6 +65,29 @@ the purge still contains everything. Removing the bytes from the live SQLitefile is worth doing on its own terms; do not describe it to a user as aguarantee that the content is unrecoverable, because it is not.+### The same is true of `purge-project` and `delete-workspace`++Every row above applies unchanged to `ai-memory purge-project --compact` and to+`POST /admin/delete-workspace` with `{"compact": true}`. The table is a+property of *any* SQLite delete, not of one command: rows go, bytes stay in+free pages until the file is rewritten.++This is worth saying explicitly because the help text for those two commands+used to promise "ALL its data", which read as byte-level removal they never+performed (#540). Both now describe the same boundary and both accept the same+opt-in.++One extra detail applies at project and workspace scope. A managed+workstream's `workstream_events` rows leave through the+`projects → workstreams → workstream_events` cascade rather than through a+`DELETE` against that table, and a cascade does not fire the `AFTER DELETE`+trigger that would drop the row from `workstream_events_fts`. Compaction+therefore rebuilds **all three** FTS indexes — `pages_fts`,+`observations_fts` and `workstream_events_fts` — not just the two a session+purge needs. Rebuilding only the indexes a given caller "should" have touched+is what leaves a managed agent's transcript text in the file after an operator+asked for it to be reclaimed.+### Scope containmentThe session id is never authority on its own. Every statement is filtered on@@ -123,8 +146,14 @@ so a clean SQLite DB can be rebuilt from the UUID-keyed wiki tree alone.```bashai-memory purge-project --workspace default --project my-project --confirm++# …and to reclaim the freed bytes as well (slow; rewrites the whole database):+ai-memory purge-project --workspace default --project my-project --confirm --compact```+Like `purge-session`, this is a logical delete unless `--compact` is given —+see [What "deleted" means](#what-deleted-means-purge-session-purge-project-delete-workspace).+What happens, in order:1. Server looks up `(workspace_id, project_id)` by name. Returns 404@@ -238,7 +267,10 @@ Failure modes:Deletes a workspace row and all child projects/pages/sessions/managedworkstreams through the `workspace_id` cascade. The route is guarded by`force: true` for non-empty workspaces and follows the destructive-operation-ordering used by project purges:+ordering used by project purges. It accepts the same opt-in reclaim as the+other two destructive commands — `{"compact": true}` — and is otherwise a+logical delete; see+[What "deleted" means](#what-deleted-means-purge-session-purge-project-delete-workspace).1. Look up the workspace without creating missing scopes.2. Run blocking `op=purge_workspace` admission. A reject-policy webhook aborts