Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
feat(temporal): page-grain ingestion windows + version-filtered as_of FTS (#656)
enrell3c8bd08a84ae
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex ffce67e8..a596c9b4 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]### Added+- Page-grain ingestion windows (`pages.valid_from` / `valid_to`, V62) and+ a second `as_of` stream: `memory_query(as_of=T)` now fuses the entity+ timeline with version-filtered full-text search over the page versions+ alive at T, using the default path's RRF (k=60) plus the same bounded+ authority adjustment — relevance at T over knowledge valid at T. This+ answers entity-less pages and paraphrased audit questions ("which+ database were we on during the outage?") that the entity-only lookup+ missed. `explain=true` reports both streams (`streams_active:+ ["entity", "fts"]`) with per-stream ranks and RRF contributions.+ Vector, graph, and the raw-observation fallback stay out of audit mode;+ the default (no `as_of`) path is unchanged. Every retire path+ (supersede, decay, reorg graveyard, move-regenerate) closes both grains+ in the same transaction, and the V62 backfill closes successor-less+ retirements at `COALESCE(superseded_at, updated_at)` (the V58 rule).+ Phase B world-time stays deferred. (#656)- `[retrieval]` section with two opt-in ranking signals, both off by defaultso unconfigured stores rank exactly as before. `query_intent` routeslexically session-recall queries ("上次 / 之前那次…的会话 / last time /diff --git a/README.md b/README.mdindex 1c6ff921..cd9928d5 100644--- a/README.md+++ b/README.md@@ -351,7 +351,7 @@ diagram, crate breakdown, schema notes, and invariants.- [`docs/ROADMAP-2.0.md`](docs/ROADMAP-2.0.md) - the plan for the 2.0 release, one item at a time.- [`docs/okf.md`](docs/okf.md) - the wiki is natively an Open Knowledge Format (OKF v0.2) bundle; design and field mapping.- [`docs/typed-edges.md`](docs/typed-edges.md) - typed relation edges (`causes` / `fixes` / `contradicts`) and how lint uses them.-- [`docs/temporal.md`](docs/temporal.md) - ingestion-time validity on the entity index and `as_of` time-travel queries.+- [`docs/temporal.md`](docs/temporal.md) - ingestion-time validity on the entity index and page versions, and `as_of` time-travel queries (entity timeline + version-filtered FTS).- [`docs/local-embeddings.md`](docs/local-embeddings.md) - in-process embeddings with no API key (`embedding_provider = "local"`).- [`docs/experience.md`](docs/experience.md) - the opt-in cross-session abstraction pass: knowledge visible only across trajectories.- [`docs/MIGRATION-2.0.md`](docs/MIGRATION-2.0.md) - upgrading an existing store to 2.0: the backup-gated automatic migration and how to restore.diff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rsindex b07f8ef1..abb41e50 100644--- a/crates/ai-memory-mcp/src/server.rs+++ b/crates/ai-memory-mcp/src/server.rs@@ -337,7 +337,8 @@ project name. `global=true` cannot be combined with \`scopes`/`project`/`workspace`. Don't conclude 'we never recorded \it' after one project misses. For \"what did we know about X back \then\" questions, pass `as_of` (ISO-8601 instant) — the query becomes \-an entity-timeline lookup returning the page versions valid at that \+a time-travel lookup fusing the entity timeline with version-filtered \+ full-text search, returning the page versions valid at that \moment, including ones superseded since. Note also that `memory_query` returns \SNIPPETS, not full page bodies — an empty or short snippet does NOT \mean the page is empty (a large page can match outside the snippet \@@ -527,11 +528,12 @@ struct QueryArgs {#[serde(default)]explain: Option<bool>,/// Time-travel: an ISO-8601 instant (e.g. `2026-06-01T00:00:00Z`).- /// When set, the query becomes an ENTITY-TIMELINE lookup: it returns- /// the page versions whose entity-link validity windows contained- /// that instant — what the store knew about the named entities then,- /// including versions superseded since (docs/temporal.md). FTS /- /// vector / graph streams are skipped in this mode; cannot be+ /// When set, the query becomes a time-travel lookup: the entity+ /// timeline fused (default-path RRF) with version-filtered full-text+ /// search over the page versions whose ingestion windows contained+ /// that instant — what the store knew then, including versions+ /// superseded since (docs/temporal.md). Vector / graph streams and+ /// the raw-observation fallback are skipped in this mode; cannot be/// combined with `global` or `scopes`. Omit for a normal search.#[serde(default)]as_of: Option<String>,@@ -615,6 +617,7 @@ struct MemoryQueryResponse {/// the primary search. Project/scopes retrieval always runs `fts` and/// `entity`, and `graph`; `vector` is present only when an embedder/// produced a query vector. Cross-project `global=true` retrieval is FTS-only.+ /// An `as_of` query runs `entity` plus version-filtered `fts`.#[serde(skip_serializing_if = "Option::is_none")]streams_active: Option<Vec<&'static str>>,}@@ -1960,8 +1963,11 @@ impl AiMemoryServer {));}- // Time-travel entity lookup (docs/temporal.md): entity stream- // only, against the ingestion-time validity windows.+ // Time-travel lookup (docs/temporal.md, issue #656): the+ // entity-window stream plus version-filtered FTS over the page+ // ingestion windows alive at T, fused with the default path's+ // RRF. Vector, graph, and the raw-observation fallback stay out+ // of audit mode.if let Some(raw_as_of) = args.as_of.as_deref().filter(|s| !s.trim().is_empty()) {if args.global.unwrap_or(false) || !args.scopes.is_empty() {return Err(McpError::internal_error(@@ -1979,24 +1985,30 @@ impl AiMemoryServer {&aps_actor,).await?;- let hits = self+ let fused = self.reader- .entity_hits_for_project_at(+ .search_pages_for_project_at(ws,proj,- &args.query,+ args.query.clone(),limit,- None,- Some(instant.as_microsecond()),+ instant.as_microsecond(),+ explain,).await.map_err(|e| McpError::internal_error(e.to_string(), None))?;return ok_json(&MemoryQueryResponse {- hits: hits.into_iter().map(|h| QueryHit::from(h.hit)).collect(),+ hits: fused+ .into_iter()+ .map(|(hit, details)| QueryHit {+ hit,+ score_details: details,+ })+ .collect(),raw_hits: Vec::new(),global_hits: Vec::new(),global_scope_hits: Vec::new(),- streams_active: explain.then(|| vec!["entity"]),+ streams_active: explain.then(|| vec!["entity", "fts"]),});}@@ -6627,9 +6639,11 @@ mod tests {);}- /// `as_of` turns memory_query into an entity-timeline lookup- /// (docs/temporal.md): a superseded version answers for the instant- /// it was valid, the current version answers for now, and the mode+ /// `as_of` turns memory_query into a time-travel lookup+ /// (docs/temporal.md, issue #656): entity timeline fused with+ /// version-filtered FTS. A superseded version answers for the+ /// instant it was valid, the current version answers for now, an+ /// entity-less version answers via the FTS leg alone, and the mode/// refuses to combine with global/scopes.#[tokio::test]async fn memory_query_as_of_travels_the_entity_timeline() {@@ -6683,7 +6697,41 @@ mod tests {.text.clone();assert!(text.contains("notes/db.md"), "{text}");- assert!(text.contains("\"entity\""), "entity-only mode: {text}");+ assert!(text.contains("\"entity\""), "entity stream ran: {text}");+ assert!(text.contains("\"fts\""), "FTS stream ran: {text}");+ assert!(text.contains("entity_rank"), "entity provenance: {text}");+ assert!(text.contains("fts_rank"), "FTS provenance: {text}");++ // FTS-leg end to end: "migrated" matches no entity (v2 carries+ // `sqlite`), but the live version's body says "we migrated" —+ // audit mode at now still finds it through version-filtered FTS.+ let fts_now = server+ .memory_query(+ Parameters(QueryArgs {+ query: "migrated".into(),+ limit: Some(5),+ project: Some("scratch".into()),+ scopes: Vec::new(),+ workspace: Some("default".into()),+ global: None,+ include_expired: None,+ explain: Some(true),+ as_of: Some(jiff::Timestamp::now().to_string()),+ }),+ OptionalParts(test_parts_default()),+ )+ .await+ .unwrap();+ let fts_text = fts_now+ .content+ .first()+ .and_then(|c| c.as_text())+ .unwrap()+ .text+ .clone();+ assert!(fts_text.contains("notes/db.md"), "{fts_text}");+ assert!(fts_text.contains("fts_rank"), "{fts_text}");+ assert!(!fts_text.contains("entity_rank"), "{fts_text}");// Same query without as_of → no postgres hit anymore... the FTS// stream may still match old text? No: default searches latestdiff --git a/crates/ai-memory-store/migrations/V62__page_ingestion_windows.sql b/crates/ai-memory-store/migrations/V62__page_ingestion_windows.sqlnew file mode 100644index 00000000..7d43c31a--- /dev/null+++ b/crates/ai-memory-store/migrations/V62__page_ingestion_windows.sql@@ -0,0 +1,43 @@+-- Page-grain ingestion windows (issue #656,+-- docs/design-page-ingestion-windows.md): materialize each page version's+-- ingestion window so `as_of` can run version-filtered FTS alongside the+-- V56 entity-link windows. Same dimension, same predicate shape, page+-- grain: `valid_from` is the version's own `created_at`; `valid_to` is the+-- superseding version's `created_at` (NULL while the version is latest).+--+-- Additive like V56: two nullable columns + index + one-shot backfill. No+-- row is deleted or rewritten beyond the two new columns, and re-running+-- the UPDATEs converges on the same values. Like every refinery step, an+-- older binary fails closed on the migrated store (DataSchemaAhead) —+-- rollback is the boot-path pre-migration snapshot (serve.rs, #633), not+-- a downgrade read. No per-migration gate beyond the standard refinery+-- step, matching V56-V61.+--+-- `valid_to`, not `superseded_at`: `pages.superseded_at` already means+-- the V03 decay-tombstone eviction marker (written exactly when+-- `supersedes IS NULL`), so reusing the name would conflate "evicted by+-- the forget sweep" with "replaced by a newer version".++ALTER TABLE pages ADD COLUMN valid_from INTEGER;+ALTER TABLE pages ADD COLUMN valid_to INTEGER;++UPDATE pages SET valid_from = created_at;++-- Ordinary supersession: close at the earliest successor's birth.+UPDATE pages SET valid_to = (+ SELECT MIN(s.created_at) FROM pages s WHERE s.supersedes = pages.id+) WHERE EXISTS (SELECT 1 FROM pages s WHERE s.supersedes = pages.id);++-- Successor-less retirements (decay tombstones, graveyard merges,+-- move-regenerate rows — the V58 class): close at the decay eviction+-- marker when present, else the row's own updated_at — the best+-- available retirement instant, the same rule V58 used for the link+-- grain. Latest versions keep valid_to NULL (still current).+UPDATE pages SET valid_to = COALESCE(superseded_at, updated_at)+WHERE is_latest = 0 AND valid_to IS NULL+ AND NOT EXISTS (SELECT 1 FROM pages s WHERE s.supersedes = pages.id);++-- The as_of window scan over page versions: (scope, window) probes ride+-- this, mirroring idx_entity_page_links_validity at link grain.+CREATE INDEX idx_pages_validity+ ON pages(workspace_id, project_id, valid_from, valid_to);diff --git a/crates/ai-memory-store/src/api_credentials.rs b/crates/ai-memory-store/src/api_credentials.rsindex 662bb11a..73b6fc25 100644--- a/crates/ai-memory-store/src/api_credentials.rs+++ b/crates/ai-memory-store/src/api_credentials.rs@@ -411,7 +411,7 @@ mod tests {|row| row.get(0),).unwrap();- assert_eq!(version, 61, "update the pin when adding a migration");+ assert_eq!(version, 62, "update the pin when adding a migration");let cols: i64 = conn.query_row("SELECT COUNT(*) FROM pragma_table_info('users') WHERE name = 'token_hash'",diff --git a/crates/ai-memory-store/src/lib.rs b/crates/ai-memory-store/src/lib.rsindex 1f8c4d6a..15ff1e1c 100644--- a/crates/ai-memory-store/src/lib.rs+++ b/crates/ai-memory-store/src/lib.rs@@ -6424,6 +6424,365 @@ mod tests {assert!(before.is_empty(), "{before:?}");}+ /// Issue #656 (Phase A): every version row carries its ingestion+ /// window — `valid_from` is the version's own `created_at`,+ /// `valid_to` is the superseding version's `created_at`, NULL while+ /// the version is latest.+ #[tokio::test]+ async fn page_windows_follow_supersession() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let proj = store+ .writer+ .get_or_create_project(ws, "temporal", None)+ .await+ .unwrap();++ let v1_id = store+ .writer+ .upsert_page(sample_page(ws, proj, "notes/db.md", "we use postgres"))+ .await+ .unwrap();+ tokio::time::sleep(std::time::Duration::from_millis(3)).await;+ let v2_id = store+ .writer+ .upsert_page(sample_page(ws, proj, "notes/db.md", "we use sqlite"))+ .await+ .unwrap();++ let db = rusqlite::Connection::open(tmp.path().join("db").join("memory.sqlite")).unwrap();+ let window = |id: PageId| {+ db.query_row(+ "SELECT valid_from, valid_to, created_at FROM pages WHERE id = ?1",+ rusqlite::params![id.as_bytes()],+ |r| {+ Ok((+ r.get::<_, Option<i64>>(0)?,+ r.get::<_, Option<i64>>(1)?,+ r.get::<_, i64>(2)?,+ ))+ },+ )+ .unwrap()+ };+ let (v1_from, v1_to, v1_created) = window(v1_id);+ let (v2_from, v2_to, v2_created) = window(v2_id);+ assert_eq!(v1_from, Some(v1_created), "valid_from opens at birth");+ assert_eq!(+ v1_to,+ Some(v2_created),+ "supersede closes the outgoing window at the new version's birth"+ );+ assert_eq!(v2_from, Some(v2_created));+ assert_eq!(v2_to, None, "latest version's window stays open");+ }++ /// The V62 backfill reconstructs the same page windows the write+ /// path produces: `valid_from` from each version's `created_at`,+ /// `valid_to` from the earliest superseding version, NULL for+ /// latest; successor-less retirements close at+ /// `COALESCE(superseded_at, updated_at)` (the V58 rule at page+ /// grain). Mirrors `v56_backfill_reconstructs_the_windows`.+ #[tokio::test]+ async fn v62_backfill_reconstructs_page_windows() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let proj = store+ .writer+ .get_or_create_project(ws, "backfill", None)+ .await+ .unwrap();+ for body in ["v1", "v2", "v3"] {+ store+ .writer+ .upsert_page(sample_page(ws, proj, "notes/x.md", body))+ .await+ .unwrap();+ tokio::time::sleep(std::time::Duration::from_millis(3)).await;+ }+ // A successor-less retirement: decay-tombstoned, never replaced.+ let tomb_id = store+ .writer+ .upsert_page(sample_page(ws, proj, "notes/old.md", "stale"))+ .await+ .unwrap();+ assert!(+ store+ .writer+ .soft_delete_for_decay_if_latest(+ ws,+ proj,+ PagePath::new("notes/old.md").unwrap(),+ tomb_id+ )+ .await+ .unwrap()+ );++ let db = rusqlite::Connection::open(tmp.path().join("db").join("memory.sqlite")).unwrap();+ let live: Vec<(Vec<u8>, Option<i64>, Option<i64>)> = db+ .prepare("SELECT id, valid_from, valid_to FROM pages ORDER BY created_at")+ .unwrap()+ .query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)))+ .unwrap()+ .collect::<Result<_, _>>()+ .unwrap();+ // Three chain versions (two closed, one open) + one tombstone.+ assert_eq!(live.len(), 4);+ assert!(live[0].2.is_some() && live[1].2.is_some());+ assert!(live[2].2.is_none(), "latest stays open");+ assert!(live[3].2.is_some(), "tombstone is closed");++ // Fabricate the pre-V62 state, then replay the migration's+ // backfill statements verbatim.+ db.execute("UPDATE pages SET valid_from = NULL, valid_to = NULL", [])+ .unwrap();+ db.execute_batch(+ "UPDATE pages SET valid_from = created_at; \+ UPDATE pages SET valid_to = ( \+ SELECT MIN(s.created_at) FROM pages s WHERE s.supersedes = pages.id \+ ) WHERE EXISTS (SELECT 1 FROM pages s WHERE s.supersedes = pages.id); \+ UPDATE pages SET valid_to = COALESCE(superseded_at, updated_at) \+ WHERE is_latest = 0 AND valid_to IS NULL \+ AND NOT EXISTS (SELECT 1 FROM pages s WHERE s.supersedes = pages.id);",+ )+ .unwrap();+ let backfilled: Vec<(Vec<u8>, Option<i64>, Option<i64>)> = db+ .prepare("SELECT id, valid_from, valid_to FROM pages ORDER BY created_at")+ .unwrap()+ .query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)))+ .unwrap()+ .collect::<Result<_, _>>()+ .unwrap();+ assert_eq!(backfilled.len(), 4);+ for (i, (live_row, back_row)) in live.iter().zip(backfilled.iter()).enumerate() {+ assert_eq!(live_row.0, back_row.0, "row {i} page id");+ assert_eq!(live_row.1, back_row.1, "row {i} valid_from");+ assert_eq!(live_row.2, back_row.2, "row {i} valid_to");+ }+ }++ /// Issue #656 acceptance: `postgres → sqlite → postgres` (+ revert).+ /// `as_of` inside the middle window returns that version via the+ /// entity stream AND via version-filtered FTS — the middle version+ /// deliberately carries no entities, so the entity leg alone would+ /// miss it. The default query path is unchanged.+ #[tokio::test]+ async fn as_of_fuses_entity_and_fts_at_t() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let proj = store+ .writer+ .get_or_create_project(ws, "temporal", None)+ .await+ .unwrap();++ let mut v1 = sample_page(ws, proj, "notes/db.md", "we use postgres");+ v1.entities = vec!["postgres".into()];+ let v1_id = store.writer.upsert_page(v1).await.unwrap();+ tokio::time::sleep(std::time::Duration::from_millis(3)).await;+ let t_first = jiff::Timestamp::now().as_microsecond();+ tokio::time::sleep(std::time::Duration::from_millis(3)).await;++ // Entity-less on purpose: only the FTS leg can find this.+ let v2_id = store+ .writer+ .upsert_page(sample_page(+ ws,+ proj,+ "notes/db.md",+ "we migrated to sqlite",+ ))+ .await+ .unwrap();+ tokio::time::sleep(std::time::Duration::from_millis(3)).await;+ let t_middle = jiff::Timestamp::now().as_microsecond();+ tokio::time::sleep(std::time::Duration::from_millis(3)).await;++ let mut v3 = sample_page(ws, proj, "notes/db.md", "we moved back to postgres");+ v3.entities = vec!["postgres".into()];+ let v3_id = store.writer.upsert_page(v3).await.unwrap();+ assert_ne!(v1_id, v2_id);+ assert_ne!(v2_id, v3_id);++ // First window, entity-phrased: both streams agree on v1.+ let first = store+ .reader+ .search_pages_for_project_at(ws, proj, "postgres".into(), 10, t_first, true)+ .await+ .unwrap();+ assert_eq!(first.len(), 1, "{first:?}");+ assert_eq!(first[0].0.id, v1_id);+ let details = first[0].1.as_ref().expect("explain=true");+ assert!(details.entity_rank.is_some(), "entity leg: {details:?}");+ assert!(details.fts_rank.is_some(), "FTS leg: {details:?}");++ // Middle window, entity-less version: the FTS leg carries it.+ let middle = store+ .reader+ .search_pages_for_project_at(ws, proj, "sqlite".into(), 10, t_middle, true)+ .await+ .unwrap();+ assert_eq!(middle.len(), 1, "{middle:?}");+ assert_eq!(middle[0].0.id, v2_id);+ let details = middle[0].1.as_ref().expect("explain=true");+ assert_eq!(details.entity_rank, None, "no entities: {details:?}");+ assert!(details.fts_rank.is_some(), "FTS leg: {details:?}");++ // Middle window, retired entity: v1 is dead, v3 unborn.+ let gone = store+ .reader+ .search_pages_for_project_at(ws, proj, "postgres".into(), 10, t_middle, false)+ .await+ .unwrap();+ assert!(gone.is_empty(), "{gone:?}");++ // Default path unchanged: latest only.+ let now_default = store+ .reader+ .search_pages_for_project(ws, proj, "postgres".into(), 10, None)+ .await+ .unwrap();+ assert_eq!(now_default.len(), 1);+ assert_eq!(now_default[0].id, v3_id);+ let sqlite_default = store+ .reader+ .search_pages_for_project(ws, proj, "sqlite".into(), 10, None)+ .await+ .unwrap();+ assert!(sqlite_default.is_empty(), "{sqlite_default:?}");+ }++ /// V58-style control at page grain: the decay retire path closes+ /// the page window in the same transaction, so `as_of` after the+ /// eviction cannot resurrect the tombstone. Reopening the window+ /// (the bug V58 repaired at link grain) makes the same query find+ /// it again — proving the close is what holds the guarantee.+ #[tokio::test]+ async fn decay_close_keeps_as_of_from_resurrecting() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let proj = store+ .writer+ .get_or_create_project(ws, "temporal", None)+ .await+ .unwrap();++ let mut page = sample_page(ws, proj, "notes/legacy.md", "legacy token alpha");+ page.entities = vec!["alpha".into()];+ let page_id = store.writer.upsert_page(page).await.unwrap();+ tokio::time::sleep(std::time::Duration::from_millis(3)).await;+ assert!(+ store+ .writer+ .soft_delete_for_decay_if_latest(+ ws,+ proj,+ PagePath::new("notes/legacy.md").unwrap(),+ page_id+ )+ .await+ .unwrap()+ );+ let after = jiff::Timestamp::now().as_microsecond();++ let db = rusqlite::Connection::open(tmp.path().join("db").join("memory.sqlite")).unwrap();+ let valid_to: Option<i64> = db+ .query_row(+ "SELECT valid_to FROM pages WHERE id = ?1",+ rusqlite::params![page_id.as_bytes()],+ |r| r.get(0),+ )+ .unwrap();+ assert!(valid_to.is_some(), "retire path must close the page window");++ let buried = store+ .reader+ .search_pages_for_project_at(ws, proj, "alpha".into(), 10, after, false)+ .await+ .unwrap();+ assert!(buried.is_empty(), "{buried:?}");++ // Control: the V58-class bug (window left open) resurrects it.+ db.execute(+ "UPDATE pages SET valid_to = NULL WHERE id = ?1",+ rusqlite::params![page_id.as_bytes()],+ )+ .unwrap();+ let resurrected = store+ .reader+ .search_pages_for_project_at(ws, proj, "alpha".into(), 10, after, false)+ .await+ .unwrap();+ assert_eq!(resurrected.len(), 1, "open window must answer again");+ assert_eq!(resurrected[0].0.id, page_id);+ }++ /// Deletion stays deletion: a removed page leaves no rows behind,+ /// so `as_of` has nothing to return — the timeline does not survive+ /// an explicit delete (docs/temporal.md).+ #[tokio::test]+ async fn purge_destroys_page_windows() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let proj = store+ .writer+ .get_or_create_project(ws, "temporal", None)+ .await+ .unwrap();++ let mut page = sample_page(ws, proj, "notes/gone.md", "ephemeral token beta");+ page.entities = vec!["beta".into()];+ store.writer.upsert_page(page).await.unwrap();+ let before = jiff::Timestamp::now().as_microsecond();+ let found = store+ .reader+ .search_pages_for_project_at(ws, proj, "beta".into(), 10, before, false)+ .await+ .unwrap();+ assert_eq!(found.len(), 1);++ store+ .writer+ .delete_page(ws, proj, PagePath::new("notes/gone.md").unwrap(), None)+ .await+ .unwrap();+ let after = jiff::Timestamp::now().as_microsecond();+ for instant in [before, after] {+ let hits = store+ .reader+ .search_pages_for_project_at(ws, proj, "beta".into(), 10, instant, false)+ .await+ .unwrap();+ assert!(hits.is_empty(), "t={instant}: {hits:?}");+ }+ }+/// End to end: a page written with only `tags` (no explicit/// `entities`) — the shape of essentially every page on a mature/// store — has no entity index until the one-shot startup backfilldiff --git a/crates/ai-memory-store/src/ops.rs b/crates/ai-memory-store/src/ops.rsindex 6ee8b86f..a73728dd 100644--- a/crates/ai-memory-store/src/ops.rs+++ b/crates/ai-memory-store/src/ops.rs@@ -875,9 +875,11 @@ pub(crate) fn upsert_page_in_tx(}let frontmatter_str = stamped_frontmatter(conformed, now)?;let new_id = PageId::new();+ // The page-grain ingestion window closes in the same statement,+ // same instant (issue #656, docs/design-page-ingestion-windows.md).tx.execute(- "UPDATE pages SET is_latest = 0 WHERE id = ?1",- params![&existing.id],+ "UPDATE pages SET is_latest = 0, valid_to = ?2 WHERE id = ?1",+ params![&existing.id, now],)?;// Close the superseded version's entity-link windows at the new// version's birth instant (docs/temporal.md).@@ -890,8 +892,8 @@ pub(crate) fn upsert_page_in_tx("INSERT INTO pages \(id, workspace_id, project_id, path, path_search, title, tier, body, body_sha256, \frontmatter_json, is_latest, supersedes, pinned, author_id, \- created_at, updated_at, expires_at) \- VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, 1, ?11, ?12, ?13, ?14, ?14, ?15)",+ created_at, updated_at, expires_at, valid_from) \+ VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, 1, ?11, ?12, ?13, ?14, ?14, ?15, ?14)",params![new_id.as_bytes(),page.workspace_id.as_bytes(),@@ -931,8 +933,8 @@ pub(crate) fn upsert_page_in_tx(tx.execute("INSERT INTO pages \(id, workspace_id, project_id, path, path_search, title, tier, body, body_sha256, \- frontmatter_json, is_latest, pinned, author_id, created_at, updated_at, expires_at) \- VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, 1, ?11, ?12, ?13, ?13, ?14)",+ frontmatter_json, is_latest, pinned, author_id, created_at, updated_at, expires_at, valid_from) \+ VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, 1, ?11, ?12, ?13, ?13, ?14, ?13)",params![new_id.as_bytes(),page.workspace_id.as_bytes(),@@ -2125,7 +2127,7 @@ pub fn soft_delete_for_decay_if_latest(let tx = conn.transaction()?;let affected = tx.execute("UPDATE pages \- SET is_latest = 0, superseded_at = ?1 \+ SET is_latest = 0, superseded_at = ?1, valid_to = ?1 \WHERE id = ?2 \AND workspace_id = ?3 \AND project_id = ?4 \@@ -2142,7 +2144,8 @@ pub fn soft_delete_for_decay_if_latest(if affected != 0 {// Retirement is supersession for the entity timeline too: an// open window on a tombstoned page made `as_of` resurrect- // retired knowledge forever (post-audit finding).+ // retired knowledge forever (post-audit finding). The page-grain+ // window closes in the same statement above (issue #656).tx.execute("UPDATE entity_page_links SET superseded_at = ?1 \WHERE page_id = ?2 AND superseded_at IS NULL",@@ -3174,16 +3177,19 @@ pub fn reorg_sessions(observations_updated += obs_rows;}// Graveyard only this workspace's latest pages; sibling workspaces may- // have already-consolidated pages that must remain current.+ // have already-consolidated pages that must remain current. Both+ // grains close at one shared instant (issue #656).+ let retire_at = Timestamp::now().as_microsecond();tx.execute("UPDATE entity_page_links SET superseded_at = ?2 \WHERE superseded_at IS NULL AND page_id IN ( \SELECT id FROM pages WHERE workspace_id = ?1 AND is_latest = 1)",- params![workspace_id.as_bytes(), Timestamp::now().as_microsecond()],+ params![workspace_id.as_bytes(), retire_at],)?;let pages_graveyarded: usize = tx.execute(- "UPDATE pages SET is_latest = 0 WHERE workspace_id = ?1 AND is_latest = 1",- params![workspace_id.as_bytes()],+ "UPDATE pages SET is_latest = 0, valid_to = ?2 \+ WHERE workspace_id = ?1 AND is_latest = 1",+ params![workspace_id.as_bytes(), retire_at],)?;tx.commit()?;Ok(ReorgSummary {@@ -4528,7 +4534,10 @@ pub fn move_session(}PagesMode::Regenerate => {// Close the retiring page's entity windows first (the- // predicate needs is_latest = 1, flipped just below).+ // predicate needs is_latest = 1, flipped just below); the+ // page-grain window closes with the flip, same instant+ // (issue #656).+ let retire_at = Timestamp::now().as_microsecond();tx.execute(&format!("UPDATE entity_page_links SET superseded_at = ?4 \@@ -4540,15 +4549,20 @@ pub fn move_session(page_scope_params.0,page_scope_params.1,page_path.as_str(),- Timestamp::now().as_microsecond(),+ retire_at,],)?;summary.pages_regenerated = tx.execute(&format!(- "UPDATE pages SET is_latest = 0 \+ "UPDATE pages SET is_latest = 0, valid_to = ?4 \WHERE {page_scope_sql} AND path = ?3 AND is_latest = 1"),- params![page_scope_params.0, page_scope_params.1, page_path.as_str()],+ params![+ page_scope_params.0,+ page_scope_params.1,+ page_path.as_str(),+ retire_at+ ],)? as u64;// The session's summary pointer targeted the page just// retired; the next consolidation sets it again.diff --git a/crates/ai-memory-store/src/reader.rs b/crates/ai-memory-store/src/reader.rsindex 7b4f36de..2ee9a351 100644--- a/crates/ai-memory-store/src/reader.rs+++ b/crates/ai-memory-store/src/reader.rs@@ -1853,7 +1853,247 @@ impl ReaderPool {.await}- /// Run a full-text search against raw observations scoped to one project.+ /// Authority-adjusted full-text candidates over the page versions+ /// whose ingestion windows contain `as_of_us` (issue #656): "what+ /// would search have said at T". Same candidate shape as+ /// [`Self::search_page_candidates_for_project`], but the corpus is+ /// versions alive at `T` instead of latest versions. TTL expiry is+ /// evaluated at `T`: a page already expired then was already hidden+ /// from search then.+ async fn search_page_candidates_for_project_at(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ query: String,+ candidate_limit: usize,+ as_of_us: i64,+ ) -> StoreResult<Vec<(PageHit, PageAuthority)>> {+ let fts_query = normalize_fts_query(&query);+ if fts_query.is_empty() || candidate_limit == 0 {+ return Ok(Vec::new());+ }+ self.with_conn(move |conn| {+ let kind_expr = page_kind_expr("pages.path", "pages.frontmatter_json");+ let sql = format!(+ "SELECT pages.id, pages.path, pages.title, \+ snippet(pages_fts, 1, '<mark>', '</mark>', '…', 24) AS snip, \+ pages_fts.rank, pages.tier, pages.pinned, \+ pages.frontmatter_json, {kind_expr} AS kind \+ FROM pages_fts \+ JOIN pages ON pages.rowid = pages_fts.rowid \+ WHERE pages_fts MATCH ?1 \+ AND pages.workspace_id = ?2 \+ AND pages.project_id = ?3 \+ AND pages.valid_from <= ?4 \+ AND (pages.valid_to IS NULL OR pages.valid_to > ?5){not_expired} \+ ORDER BY pages_fts.rank \+ LIMIT ?7",+ not_expired = not_expired("pages", "?6"),+ );+ let mut stmt = conn.prepare(&sql)?;+ #[allow(clippy::cast_possible_wrap)]+ let rows = stmt.query_map(+ params![+ fts_query,+ workspace_id.as_bytes(),+ project_id.as_bytes(),+ as_of_us,+ as_of_us,+ as_of_us,+ candidate_limit as i64+ ],+ |row| {+ let id_bytes: Vec<u8> = row.get(0)?;+ let path: String = row.get(1)?;+ let title: String = row.get(2)?;+ let snippet: String = row.get(3)?;+ let rank: f64 = row.get(4)?;+ let tier: String = row.get(5)?;+ let pinned = row.get::<_, i64>(6)? != 0;+ let frontmatter_json: String = row.get(7)?;+ let kind: String = row.get(8)?;+ Ok((+ id_bytes,+ path,+ title,+ snippet,+ rank,+ tier,+ pinned,+ frontmatter_json,+ kind,+ ))+ },+ )?;++ let mut candidates = Vec::new();+ for row in rows {+ let (id_bytes, path, title, snippet, rank, tier, pinned, frontmatter_json, kind) =+ row?;+ let authority =+ PageAuthority::from_stored(&path, &kind, &tier, pinned, &frontmatter_json);+ candidates.push((+ PageHit {+ id: PageId::from_slice(&id_bytes)?,+ path: PagePath::new(path)?,+ title,+ snippet,+ rank,+ },+ authority,+ ));+ }+ Ok(candidates)+ })+ .await+ }++ /// Time-travel search backing `memory_query(as_of)`+ /// (docs/temporal.md, issue #656): the entity-window lookup+ /// unchanged, plus version-filtered FTS over the page ingestion+ /// windows alive at `as_of_us`, fused with the same RRF (k=60) the+ /// default path uses and the same bounded authority adjustment —+ /// relevance *at T* over knowledge *valid at T*. Vector, graph, and+ /// the raw-observation fallback stay out of audit mode: embeddings+ /// and links are present-tense artifacts with no version scope, and+ /// audit reads must not perturb access stats (no bump, no rerank).+ ///+ /// `explain` mirrors [`Self::hybrid_search`]: `false` drops the+ /// per-hit [`SearchExplain`]; `streams_active` reporting stays with+ /// the MCP caller.+ ///+ /// # Errors+ /// Propagates any SQL or pool error.+ pub async fn search_pages_for_project_at(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ query: String,+ limit: usize,+ as_of_us: i64,+ explain: bool,+ ) -> StoreResult<Vec<(PageHit, Option<SearchExplain>)>> {+ if limit == 0 {+ return Ok(Vec::new());+ }+ let candidate_limit = authority_candidate_limit(limit);+ let entity_hits = self+ .entity_hits_for_project_at(+ workspace_id,+ project_id,+ &query,+ candidate_limit,+ None,+ Some(as_of_us),+ )+ .await?;+ let fts_candidates = self+ .search_page_candidates_for_project_at(+ workspace_id,+ project_id,+ query,+ candidate_limit,+ as_of_us,+ )+ .await?;+ let mut authorities: std::collections::HashMap<PageId, PageAuthority> = fts_candidates+ .iter()+ .map(|(hit, authority)| (hit.id, *authority))+ .collect();+ // Entity-only hits never passed through an FTS candidate row;+ // read their authority off the version row itself (no latest+ // filter — audit mode ranks superseded versions too).+ let missing: Vec<PageId> = entity_hits+ .iter()+ .map(|e| e.hit.id)+ .filter(|id| !authorities.contains_key(id))+ .collect();+ authorities.extend(+ self.page_authorities_for_versions(workspace_id, project_id, missing)+ .await?,+ );++ // RRF fuse: score(d) = Σ 1/(k + rank_i(d)) over the two streams.+ let k = 60.0_f64;+ struct FusedAt {+ path: PagePath,+ title: String,+ snippet: String,+ score: f64,+ explain: Option<SearchExplain>,+ }+ let mut fused: std::collections::HashMap<PageId, FusedAt> =+ std::collections::HashMap::new();+ for (rank, h) in fts_candidates.iter().map(|(hit, _)| hit).enumerate() {+ let contrib = 1.0 / (k + (rank + 1) as f64);+ let entry = fused.entry(h.id).or_insert_with(|| FusedAt {+ path: h.path.clone(),+ title: h.title.clone(),+ snippet: h.snippet.clone(),+ score: 0.0,+ explain: explain.then(SearchExplain::default),+ });+ entry.score += contrib;+ if let Some(details) = &mut entry.explain {+ details.fts_rank = Some(rank + 1);+ details.fts_score = Some(h.rank);+ details.rrf.fts = contrib;+ details.fused += contrib;+ }+ }+ for (rank, e) in entity_hits.iter().enumerate() {+ let contrib = 1.0 / (k + (rank + 1) as f64);+ let entry = fused.entry(e.hit.id).or_insert_with(|| FusedAt {+ path: e.hit.path.clone(),+ title: e.hit.title.clone(),+ snippet: e.hit.snippet.clone(),+ score: 0.0,+ explain: explain.then(SearchExplain::default),+ });+ entry.score += contrib;+ if let Some(details) = &mut entry.explain {+ details.entity_rank = Some(rank + 1);+ details.entity_weight = Some(e.weight);+ details.matched_entities = e.matched.clone();+ details.rrf.entity = contrib;+ details.fused += contrib;+ }+ }++ let mut out: Vec<(PageHit, Option<SearchExplain>)> = fused+ .into_iter()+ .map(|(id, entry)| {+ (+ PageHit {+ id,+ path: entry.path,+ title: entry.title,+ snippet: entry.snippet,+ rank: -entry.score, // lower = better (matches FTS5 convention)+ },+ entry.explain,+ )+ })+ .collect();+ // No session-recall routing in audit mode; the plain authority+ // factor keeps maintained pages' bounded advantage at T.+ for (hit, details) in &mut out {+ if let Some(authority) = authorities.get(&hit.id) {+ hit.rank = authority.adjust_rank(hit.rank);+ if let Some(explain) = details {+ explain.authority = Some(authority.factor);+ }+ }+ }+ out.sort_by(|a, b| {+ a.0.rank+ .partial_cmp(&b.0.rank)+ .unwrap_or(std::cmp::Ordering::Equal)+ .then_with(|| a.0.path.as_str().cmp(b.0.path.as_str()))+ });+ out.truncate(limit);+ Ok(out)+ }////// # Errors/// Propagates any SQL or pool error.@@ -4117,6 +4357,30 @@ impl ReaderPool {workspace_id: WorkspaceId,project_id: ProjectId,page_ids: Vec<PageId>,+ ) -> StoreResult<std::collections::HashMap<PageId, PageAuthority>> {+ self.page_authorities_for_ids(workspace_id, project_id, page_ids, true)+ .await+ }++ /// Authority inputs for specific page versions, without the+ /// latest-only filter: audit mode ranks superseded versions too+ /// (issue #656).+ async fn page_authorities_for_versions(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ page_ids: Vec<PageId>,+ ) -> StoreResult<std::collections::HashMap<PageId, PageAuthority>> {+ self.page_authorities_for_ids(workspace_id, project_id, page_ids, false)+ .await+ }++ async fn page_authorities_for_ids(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ page_ids: Vec<PageId>,+ latest_only: bool,) -> StoreResult<std::collections::HashMap<PageId, PageAuthority>> {if page_ids.is_empty() {return Ok(std::collections::HashMap::new());@@ -4135,6 +4399,11 @@ impl ReaderPool {sql_params.push(Value::Blob(project_id.as_bytes().to_vec()));let kind_expr = page_kind_expr("pages.path", "pages.frontmatter_json");+ let latest_filter = if latest_only {+ "AND pages.is_latest = 1"+ } else {+ ""+ };let sql = format!("WITH requested(id) AS (VALUES {values_clause}) \SELECT pages.id, pages.path, pages.tier, pages.pinned, \@@ -4143,7 +4412,7 @@ impl ReaderPool {JOIN pages ON pages.id = requested.id \WHERE pages.workspace_id = ? \AND pages.project_id = ? \- AND pages.is_latest = 1"+ {latest_filter}");let mut stmt = conn.prepare(&sql)?;let rows = stmt.query_map(params_from_iter(sql_params.iter()), |row| {diff --git a/docs/design-page-ingestion-windows.md b/docs/design-page-ingestion-windows.mdindex 703ffc1e..667b4658 100644--- a/docs/design-page-ingestion-windows.md+++ b/docs/design-page-ingestion-windows.md@@ -1,12 +1,16 @@# Design: Page-grain ingestion windows + version-filtered `as_of` (issue #656)-*Status: design, targeting the `release/2.2` line as an additive 2.2.0-feature (v2.1.0 shipped; new features now ride the 2.2 train). Not implemented yet — this settles naming, the `as_of`+FTS-reversal, and materialized-vs-view **before any schema change**, per the-maintainer's review on #656. Scope here is Phase A only; Phase B stays-deferred (§7). Note: v2.1.0 has since cut on this line, so the-implementation PR lands as 2.1.x or rides the next minor at the-maintainer's discretion (open question 1).*+*Status: implemented on the `release/2.2` line (2.2.0) — V62 page-grain+windows, version-filtered `as_of` FTS fused via the default path's RRF,+`docs/temporal.md` updated. Scope stayed Phase A only; Phase B remains+deferred (§7). Open question 1 resolved by the v2.1.0 cut (new features+ride 2.2). Open question 2 resolved for the materialized pair+(`valid_from` + `valid_to`): the symmetry with the V56 link windows was+worth the duplicated column. Open question 3 resolved for RRF merge+rather than bare fallback. One deliberate deviation from §3: the FTS leg+evaluates TTL expiry at T while the entity leg ignores it (see+`docs/temporal.md` "one asymmetry"); changing the entity leg's rule+would alter existing behaviour and is out of scope.*## 1. The problem, and why entity-link windows are not enoughdiff --git a/docs/temporal.md b/docs/temporal.mdindex 51c5e386..1c1dc2c9 100644--- a/docs/temporal.md+++ b/docs/temporal.md@@ -5,25 +5,33 @@ window, so "what did we know about X as of June" is answerable fromthe store — the one mainstream graph-camp mechanism (Zep/Graphiti)worth having at our scale.+*2.2, issue #656.* Page versions carry the same window at page grain+(`pages.valid_from` / `valid_to`, V62), and `as_of` fuses the entity+timeline with version-filtered full-text search. Vocabulary is pinned+in `docs/design-page-ingestion-windows.md`: this is **ingestion time**+("the version the store treated as current"), never world-time+validity — "valid-time" stays reserved for the deferred Phase B.+## Honest scope-- **Ingestion time only.** `valid_from` / `superseded_at` record when- ai-memory *learned* and *replaced* a fact, not when it was true in- the world. A world-time split (Graphiti's `valid_at`/`invalid_at`- extracted by an LLM) is deliberately out of scope: it requires- trusting an LLM to date facts, and our zero-LLM default path could- never populate it.-- **Entity links only.** Page versions already form a timeline- (`supersedes` + `created_at`); entity links now inherit it. Typed- relation edges (item 3) die with their page version, which is- already a timeline — no extra columns needed there yet.+- **Ingestion time only.** `valid_from` / `superseded_at` (link grain)+ and `valid_from` / `valid_to` (page grain) record when ai-memory+ *learned* and *replaced* a fact, not when it was true in the world. A+ world-time split (Graphiti's `valid_at`/`invalid_at` extracted by an+ LLM) is deliberately out of scope: it requires trusting an LLM to+ date facts, and our zero-LLM default path could never populate it.+- **Two grains, one semantic.** Entity links inherit their page+ version's timeline (V56, repaired by V58); page versions now+ materialize it directly (V62). Typed relation edges (item 3) die with+ their page version, which is already a timeline — no extra columns+ needed there yet.- **Deletion is deletion.** Purged pages cascade their entity linksaway; the timeline does not survive an explicit purge (that is whatpurge means here — see the purge docs).## Schema-`entity_page_links` gains two additive columns:+`entity_page_links` carries two additive columns:| column | meaning ||---|---|@@ -38,6 +46,23 @@ insert, and the supersede path closes the outgoing version's windowsin the same transaction — a link's window can never be open-ended ina superseded version.+`pages` carries the same window at page grain (V62, issue #656):++| column | meaning |+|---|---|+| `valid_from` | the version's own `created_at` |+| `valid_to` | `created_at` of the superseding version; the retirement instant for successor-less retirements (decay tombstones, graveyard merges, move-regenerate rows — `COALESCE(superseded_at, updated_at)`, the V58 rule); `NULL` while the version is latest |++The end is named `valid_to`, not `superseded_at`, because+`pages.superseded_at` already exists with a different meaning: the V03+decay-tombstone eviction marker, written exactly when `supersedes IS+NULL`. Every retire path closes both grains in the same transaction+(supersede, decay, reorg graveyard, move-regenerate); the V62 backfill+covers existing stores with the same rules. No FTS rebuild is needed:+the `pages_fts_*` triggers already index every version row, superseded+ones included — version-filtered search is a join-predicate change,+not an index change.+## When to reach for itOrdinary `memory_query` answers "what do we know NOW" and is the right@@ -62,17 +87,44 @@ memory_query { "query": "postgres", "as_of": "2026-07-01T00:00:00Z" }// superseded```+The entity leg needs the entity: it only answers when the query names+a tracked entity the version carried. The FTS leg covers the rest — a+version with no `entities:`/`tags:` frontmatter, or a question phrased+without the exact retired term ("which database were we on during the+outage?"), still resolves through the text that was live at T.+## Query`memory_query` accepts `as_of` (ISO-8601). When present the query is-a **time-travel entity lookup**: the entity stream runs alone against-links whose window contains the instant-(`valid_from <= T AND (superseded_at IS NULL OR superseded_at > T)`),-returning the page versions that carried the matched entities at that-time. FTS / vector / graph streams are deliberately skipped — mixing-"current text relevance" with "historical entity validity" in one-ranked list would answer neither question honestly. Omit `as_of` and-nothing changes.+a **time-travel lookup over two streams, both past-tense at T**:++- the **entity timeline**: links whose window contains the instant+ (`valid_from <= T AND (superseded_at IS NULL OR superseded_at > T)`),+ returning the page versions that carried the matched entities then;+- **version-filtered FTS**: page versions whose page-grain window+ contains the instant+ (`valid_from <= T AND (valid_to IS NULL OR valid_to > T)`),+ answering entity-less pages and paraphrased questions — "what would+ search have said in July".++The two streams merge with the default path's RRF (k=60) plus the same+bounded authority adjustment, and `explain=true` reports both in+`streams_active` (`["entity", "fts"]`) with per-stream ranks and RRF+contributions. Constraining the FTS corpus to versions alive at T is+what answers the original objection to FTS in `as_of` (that mixing+present-tense relevance with past validity answers neither honestly):+both signals are past-tense here.++Still out of `as_of`: vector (embeddings are present-tense artifacts+of the current text; version-scoped vectors are a separate project),+graph neighbours (latest-scoped by construction), and the+raw-observation fallback. Audit reads take no access bump and no+rerank. Omit `as_of` and nothing changes.++One asymmetry to know: the FTS leg hides pages already TTL-expired at+T (the usual `not_expired` predicate evaluated at T); the entity leg+ignores TTL, on the grounds that a page valid at T was still what we+knew at T even if it expired since.## Where entities come from