Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(store): scope decay tombstone cleanup
AkitaOnRails6ade07edb37d
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex e72fa72d..7d083276 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -78,6 +78,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0TTL-expired standing pages are ignored. (#316)### Fixed+- A project-scoped forget sweep now purges aged decay tombstones only from its+ resolved workspace/project instead of deleting eligible derived rows across+ every project. Entity-index rows orphaned by the scoped purge are removed in+ the same transaction, and `hard_deleted` reports only the target scope. (#323)- Zero-LLM `memory_query` now keeps graph-neighbour expansion active insteadof falling back to FTS5 alone when no query embedding exists. Equal adjustedhybrid and explicit multi-scope scores now use a deterministic pathdiff --git a/crates/ai-memory-consolidate/src/sweep.rs b/crates/ai-memory-consolidate/src/sweep.rsindex c3cbfa53..0c1a0160 100644--- a/crates/ai-memory-consolidate/src/sweep.rs+++ b/crates/ai-memory-consolidate/src/sweep.rs@@ -188,7 +188,7 @@ pub async fn run_sweep(writer.soft_delete_for_decay(to_evict_ids).await?;}hard_deleted = writer- .hard_delete_decayed(params.hard_delete_after_days)+ .hard_delete_decayed(workspace_id, project_id, params.hard_delete_after_days).await?;}diff --git a/crates/ai-memory-consolidate/tests/lifecycle.rs b/crates/ai-memory-consolidate/tests/lifecycle.rsindex ef53dfe7..2f12b9df 100644--- a/crates/ai-memory-consolidate/tests/lifecycle.rs+++ b/crates/ai-memory-consolidate/tests/lifecycle.rs@@ -661,6 +661,136 @@ async fn ttl_expiry_lifecycle_end_to_end() {assert!(after_paths.contains(&"notes/refreshed.md"));}+#[tokio::test]+async fn aged_decay_cleanup_stays_within_the_requested_scope() {+ let tmp = TempDir::new().expect("tempdir");+ let store = Store::open(tmp.path()).expect("open store");+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .expect("workspace");+ let target = store+ .writer+ .get_or_create_project(ws, "target", None)+ .await+ .expect("target project");+ let sibling = store+ .writer+ .get_or_create_project(ws, "sibling", None)+ .await+ .expect("sibling project");+ let other_ws = store+ .writer+ .get_or_create_workspace("other")+ .await+ .expect("other workspace");+ let other_project = store+ .writer+ .get_or_create_project(other_ws, "target", None)+ .await+ .expect("other workspace project");+ let wiki = Wiki::new(tmp.path(), store.writer.clone()).expect("wiki");++ for (workspace_id, project_id, path, entity) in [+ (ws, target, "sessions/target.md", "target-entity"),+ (ws, sibling, "sessions/sibling.md", "sibling-entity"),+ (+ other_ws,+ other_project,+ "sessions/other-workspace.md",+ "other-workspace-entity",+ ),+ ] {+ wiki.write_page(WritePageRequest {+ workspace_id,+ project_id,+ path: PagePath::new(path).unwrap(),+ frontmatter: serde_json::json!({"entities": [entity]}),+ body: format!("# Tombstone\n{path}"),+ tier: Tier::Episodic,+ pinned: false,+ title: Some("Tombstone".into()),+ admission_ctx: None,+ author_id: None,+ actor: ai_memory_core::ActorContext::anonymous(),+ })+ .await+ .expect("write tombstone fixture");+ }++ // All three rows represent sweep-evicted pages old enough for immediate+ // cleanup. The sweep below targets only `(default, target)`.+ let conn = rusqlite::Connection::open(store.db_path()).expect("open aux conn");+ conn.pragma_update(None, "busy_timeout", 5_000).unwrap();+ conn.execute(+ "UPDATE pages SET is_latest = 0, superseded_at = 1, access_count = 0",+ [],+ )+ .expect("age tombstone fixtures");+ drop(conn);++ let params = DecayParams {+ hard_delete_after_days: 0,+ ..DecayParams::default()+ };+ let report = run_sweep(+ &store.reader,+ &store.writer,+ None,+ ws,+ target,+ ¶ms,+ false,+ )+ .await+ .expect("targeted sweep");+ assert_eq!(+ report.hard_deleted, 1,+ "the report must count only tombstones deleted from the target scope"+ );++ let conn = rusqlite::Connection::open(store.db_path()).expect("reopen aux conn");+ let remaining = |workspace_id: WorkspaceId, project_id: ProjectId| -> i64 {+ conn.query_row(+ "SELECT count(*) FROM pages WHERE workspace_id = ?1 AND project_id = ?2",+ params![workspace_id.as_bytes(), project_id.as_bytes()],+ |row| row.get(0),+ )+ .unwrap()+ };+ assert_eq!(remaining(ws, target), 0, "target tombstone is deleted");+ assert_eq!(remaining(ws, sibling), 1, "sibling project is preserved");+ assert_eq!(+ remaining(other_ws, other_project),+ 1,+ "same-named project in another workspace is preserved"+ );+ let remaining_entities = |workspace_id: WorkspaceId, project_id: ProjectId| -> i64 {+ conn.query_row(+ "SELECT count(*) FROM entities WHERE workspace_id = ?1 AND project_id = ?2",+ params![workspace_id.as_bytes(), project_id.as_bytes()],+ |row| row.get(0),+ )+ .unwrap()+ };+ assert_eq!(+ remaining_entities(ws, target),+ 0,+ "the target entity orphan is removed in the page-delete transaction"+ );+ assert_eq!(+ remaining_entities(ws, sibling),+ 1,+ "sibling entity index is preserved"+ );+ assert_eq!(+ remaining_entities(other_ws, other_project),+ 1,+ "other workspace entity index is preserved"+ );+}+fn ws_dir_for(tmp: &TempDir, ws: WorkspaceId, proj: ProjectId) -> std::path::PathBuf {tmp.path().join("wiki")diff --git a/crates/ai-memory-store/src/ops.rs b/crates/ai-memory-store/src/ops.rsindex e847fa77..f652d824 100644--- a/crates/ai-memory-store/src/ops.rs+++ b/crates/ai-memory-store/src/ops.rs@@ -1328,24 +1328,41 @@ fn delete_page_inner(Ok(rows > 0)}-/// Hard-delete rows that were soft-deleted by an earlier sweep at-/// least `hard_delete_after_days` ago AND received zero subsequent-/// accesses. Safe: M7 supersedes-chain pages have a non-null-/// `supersedes` so they never match.+/// Hard-delete rows in one workspace/project that were soft-deleted by an+/// earlier sweep at least `hard_delete_after_days` ago AND received zero+/// subsequent accesses. Safe: M7 supersedes-chain pages have a non-null+/// `supersedes` so they never match. Orphaned entity-index rows from those+/// page deletions are removed in the same transaction.pub fn hard_delete_decayed_pages(conn: &mut Connection,+ workspace_id: WorkspaceId,+ project_id: ProjectId,hard_delete_after_days: i64,) -> StoreResult<usize> {let cutoff = Timestamp::now().as_microsecond() - hard_delete_after_days * 86_400_000_000;- let n = conn.execute(+ let tx = conn.transaction()?;+ let n = tx.execute("DELETE FROM pages \- WHERE is_latest = 0 \+ WHERE workspace_id = ?1 \+ AND project_id = ?2 \+ AND is_latest = 0 \AND supersedes IS NULL \AND superseded_at IS NOT NULL \- AND superseded_at < ?1 \+ AND superseded_at < ?3 \AND access_count = 0",- params![cutoff],+ params![workspace_id.as_bytes(), project_id.as_bytes(), cutoff],)?;+ if n > 0 {+ tx.execute(+ "DELETE FROM entities \+ WHERE workspace_id = ?1 AND project_id = ?2 \+ AND NOT EXISTS ( \+ SELECT 1 FROM entity_page_links l WHERE l.entity_id = entities.id \+ )",+ params![workspace_id.as_bytes(), project_id.as_bytes()],+ )?;+ }+ tx.commit()?;Ok(n)}diff --git a/crates/ai-memory-store/src/writer.rs b/crates/ai-memory-store/src/writer.rsindex 9a91f534..6111c0be 100644--- a/crates/ai-memory-store/src/writer.rs+++ b/crates/ai-memory-store/src/writer.rs@@ -200,6 +200,8 @@ pub(crate) enum WriteCmd {reply: oneshot::Sender<StoreResult<usize>>,},HardDeleteDecayed {+ workspace_id: WorkspaceId,+ project_id: ProjectId,hard_delete_after_days: i64,reply: oneshot::Sender<StoreResult<usize>>,},@@ -913,14 +915,21 @@ impl WriterHandle {rx.await.map_err(|_| StoreError::WriterClosed)?}- /// Hard-delete pages soft-deleted by the sweep more than- /// `hard_delete_after_days` ago.+ /// Hard-delete pages in one workspace/project that were soft-deleted by+ /// the sweep more than `hard_delete_after_days` ago.////// # Errors/// Returns [`StoreError::WriterClosed`] or propagates SQL errors.- pub async fn hard_delete_decayed(&self, hard_delete_after_days: i64) -> StoreResult<usize> {+ pub async fn hard_delete_decayed(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ hard_delete_after_days: i64,+ ) -> StoreResult<usize> {let (tx, rx) = oneshot::channel();self.send(WriteCmd::HardDeleteDecayed {+ workspace_id,+ project_id,hard_delete_after_days,reply: tx,})@@ -1749,10 +1758,17 @@ fn worker_loop(mut conn: Connection, mut rx: mpsc::Receiver<WriteCmd>) {send_or_warn(reply, result, "soft_delete_for_decay");}WriteCmd::HardDeleteDecayed {+ workspace_id,+ project_id,hard_delete_after_days,reply,} => {- let result = ops::hard_delete_decayed_pages(&mut conn, hard_delete_after_days);+ let result = ops::hard_delete_decayed_pages(+ &mut conn,+ workspace_id,+ project_id,+ hard_delete_after_days,+ );send_or_warn(reply, result, "hard_delete_decayed_pages");}WriteCmd::HealCatchAllRepoPaths { home, reply } => {diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex c1225934..8430704b 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -137,7 +137,9 @@ from hook paths.hard-deleted through the wiki layer (file + rows, pin or not);pages with `retention < cold_threshold` are soft-deleted;soft-deletions older than `hard_delete_after_days` with no subsequent- access get purged. Semantic / pinned / freshly-touched pages survive.+ access get purged only within that sweep's resolved workspace/project,+ together with entity-index rows orphaned by the purge. Semantic / pinned /+ freshly-touched pages survive.Scheduled sweep, rule-based lint, and opt-in embedding backfill ticksenumerate every existing workspace/project scope before doing per-projectwork, matching the auto-improvement scheduler's store-wide scope model. Adiff --git a/docs/admission-webhooks.md b/docs/admission-webhooks.mdindex afa1502b..e73d0d0d 100644--- a/docs/admission-webhooks.md+++ b/docs/admission-webhooks.md@@ -105,9 +105,13 @@ terminal `purge_project` notification is unchanged.per-event log is a local audit artifact; back it up out-of-band (batchedrsync), not per-line.- **Handoffs** — SQLite rows, transient cross-agent state, not wiki pages.-- **Forget-sweep soft/hard-delete** — DB-only (`is_latest=0` / row delete);- the markdown file stays on disk, so there is nothing for a file mirror to- do. (Only `purge_project` removes files in bulk.)+- **Forget-sweep decay soft-delete and aged-tombstone cleanup** —+ project-scoped and DB-only (`is_latest=0` / row delete); the markdown file+ stays on disk, so there is nothing for a file mirror to do. This exemption+ does not include frontmatter TTL expiry: that path uses the wiki's normal+ conditional page delete, removes the file and rows, and runs the+ `delete_page` admission chain. (`purge_project` remains the bulk-removal+ path.)- **`rename-project`** — a `projects.name` column update; the on-disk pathis the stable UUID, so no file moves and nothing to propagate.- **`rename-workspace`** — a `workspaces.name` column update plus refreshed