Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
feat(query): report per-stream timings with explain=true
AkitaOnRailsd0dcbc88d962
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex d2ee7896..6337dde6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -42,6 +42,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0`explain=true` reports an `observation` stream with the matched observationids. Hits are still pages and nothing is written; sessions without a pagekeep the `raw_hits` fallback, and `global=true` search is unchanged.+- Added a `timings` object to `memory_query` responses with `explain=true`:+ each retrieval stream's wall time in milliseconds (`fts_ms`, `entity_ms`,+ `vector_ms`, `graph_ms`, `observation_ms`, `rerank_ms`), summed over `scopes`.+ Streams that did not run are left out, and a query without `explain` reads+ no clock for them.- Added the end reason to session-end observations. When a harness's`SessionEnd` payload carries a `reason` (Claude Code sends `clear`,`logout`, `prompt_input_exit` or `other`), its first line is stored,diff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rsindex 5189d35e..c223ca97 100644--- a/crates/ai-memory-mcp/src/server.rs+++ b/crates/ai-memory-mcp/src/server.rs@@ -654,9 +654,10 @@ struct QueryArgs {#[serde(default)]pin_first: Option<bool>,/// Attach `score_details` to project/scopes hits: per-stream ranks- /// (FTS5, entity, vector, graph), raw scores, and RRF contributions, plus a- /// top-level `streams_active` list. A `global=true` query uses a- /// different FTS-only ranker, so it reports only `streams_active`.+ /// (FTS5, entity, vector, graph, observation), raw scores, and RRF+ /// contributions, plus top-level `streams_active` and per-stream+ /// `timings` (milliseconds). A `global=true` query uses a different+ /// FTS-only ranker, so it reports only `streams_active` and its FTS time./// Default false.#[serde(default)]explain: Option<bool>,@@ -854,6 +855,13 @@ struct MemoryQueryResponse {/// An `as_of` query runs `entity` plus version-filtered `fts`.#[serde(skip_serializing_if = "Option::is_none")]streams_active: Option<Vec<&'static str>>,+ /// Present only when `explain=true`: wall time per retrieval stream for+ /// the primary search (`fts_ms`, `entity_ms`, `vector_ms`, `graph_ms`,+ /// `observation_ms`, `rerank_ms`), summed over `scopes`; a stream that+ /// did not run is omitted. `global=true` reports `fts_ms` only. Not+ /// measured at all without `explain`.+ #[serde(skip_serializing_if = "Option::is_none")]+ timings: Option<ai_memory_store::StreamTimings>,}/// Response for `memory_recent`. `hits` carries the current project's recent@@ -2373,12 +2381,17 @@ impl AiMemoryServer {}}+ /// One project's fused hits, plus per-stream timings when+ /// `options.explain` asked for them.async fn search_project(&self,workspace_id: WorkspaceId,project_id: ProjectId,options: ProjectSearchOptions<'_>,- ) -> ai_memory_store::StoreResult<Vec<(PageHit, Option<ai_memory_store::SearchExplain>)>> {+ ) -> ai_memory_store::StoreResult<(+ Vec<(PageHit, Option<ai_memory_store::SearchExplain>)>,+ Option<ai_memory_store::StreamTimings>,+ )> {// `i64::MIN` as the expiry cutoff makes every stored TTL pass// the `expires_at > cutoff` guard, i.e. expired pages stay// searchable when the caller opted in.@@ -2394,9 +2407,10 @@ impl AiMemoryServer {(Some(e), Some(_)) => (e.provider().to_string(), e.model_identity(), e.dim()),_ => (String::new(), String::new(), 0),};- let fused: Vec<(PageHit, Option<ai_memory_store::SearchExplain>)> = if options.explain {- self.reader- .hybrid_search_explained(+ if options.explain {+ let (hits, timings) = self+ .reader+ .hybrid_search_explained_timed(workspace_id,project_id,options.query.to_owned(),@@ -2408,12 +2422,16 @@ impl AiMemoryServer {expiry_cutoff,options.include_superseded,)- .await?- .into_iter()- .map(|(hit, details)| (hit, Some(details)))- .collect()+ .await?;+ Ok((+ hits.into_iter()+ .map(|(hit, details)| (hit, Some(details)))+ .collect(),+ Some(timings),+ ))} else {- self.reader+ let hits = self+ .reader.hybrid_search(workspace_id,project_id,@@ -2426,12 +2444,9 @@ impl AiMemoryServer {expiry_cutoff,options.include_superseded,)- .await?- .into_iter()- .map(|hit| (hit, None))- .collect()- };- Ok(fused)+ .await?;+ Ok((hits.into_iter().map(|hit| (hit, None)).collect(), None))+ }}fn rerank_fetch_limit(&self, limit: usize) -> usize {@@ -2695,7 +2710,8 @@ impl AiMemoryServer {score (lower rank = better match). Only latest page versions. \Set `explain=true` to attach per-stream ranks, matched entities, raw \scores, RRF contributions, graph provenance, and the authority multiplier to \- project/scopes hits; it also returns `streams_active`. \+ project/scopes hits; it also returns `streams_active` and per-stream \+ `timings` in milliseconds. \Cross-project `global=true` search is FTS-only and therefore reports \`streams_active` without per-hit RRF details. \If compiled wiki search misses in default/project/`scopes` mode, \@@ -2747,6 +2763,7 @@ impl AiMemoryServer {None,));}+ let fts_started = explain.then(std::time::Instant::now);let global_hits = self.reader.search_pages_with_meta(@@ -2757,6 +2774,10 @@ impl AiMemoryServer {).await.map_err(|e| McpError::internal_error(e.to_string(), None))?;+ let timings = fts_started.map(|started| ai_memory_store::StreamTimings {+ fts_ms: Some(started.elapsed().as_secs_f64() * 1000.0),+ ..Default::default()+ });return ok_json(&MemoryQueryResponse {hits: Vec::new(),answer: None,@@ -2768,6 +2789,7 @@ impl AiMemoryServer {global_hits,global_scope_hits: Vec::new(),streams_active: explain.then(|| vec!["fts"]),+ timings,});}if !args.scopes.is_empty()@@ -2837,6 +2859,8 @@ impl AiMemoryServer {global_hits: Vec::new(),global_scope_hits: Vec::new(),streams_active: explain.then(|| vec!["entity", "fts"]),+ // Time travel fuses inside one store call; no per-stream split.+ timings: None,});}@@ -2851,11 +2875,14 @@ impl AiMemoryServer {.await?,)};+ // Explain-only: per-stream timings of the primary search, summed over+ // `scopes`. `None` on the normal path, which reads no clocks for it.+ let mut timings: Option<ai_memory_store::StreamTimings> = None;let hits = if let Some(scopes) = &resolved_scopes {let mut hits_by_id: HashMap<PageId, (PageHit, Option<ai_memory_store::SearchExplain>)> =HashMap::new();for &(ws, proj) in scopes {- let hits = self+ let (hits, scope_timings) = self.search_project(ws,proj,@@ -2870,6 +2897,11 @@ impl AiMemoryServer {).await.map_err(|e| McpError::internal_error(e.to_string(), None))?;+ if let Some(scope_timings) = scope_timings {+ timings+ .get_or_insert_with(Default::default)+ .accumulate(&scope_timings);+ }for hit in hits {hits_by_id.entry(hit.0.id)@@ -2914,9 +2946,17 @@ impl AiMemoryServer {},).await+ .map(|(hits, project_timings)| {+ timings = project_timings;+ hits+ })};let hits = hits.map_err(|e| McpError::internal_error(e.to_string(), None))?;+ let rerank_started = (explain && self.reranker.is_some()).then(std::time::Instant::now);let hits = self.rerank_hits(&args.query, hits, limit).await;+ if let (Some(timings), Some(started)) = (timings.as_mut(), rerank_started) {+ timings.rerank_ms = Some(started.elapsed().as_secs_f64() * 1000.0);+ }let bump_actor = Self::bump_actor_from_parts(&parts);self.spawn_access_bump(hits.iter().map(|(h, _)| h.id).collect(),@@ -2994,7 +3034,7 @@ impl AiMemoryServer {if current == scope.as_tuple() {Vec::new()} else {- let hits = self+ let (hits, _) = self.search_project(scope.workspace_id,scope.project_id,@@ -3063,7 +3103,7 @@ impl AiMemoryServer {.await{Ok(Some(scope)) if scope.as_tuple() != current => {- let hits = self+ let (hits, _) = self.search_project(scope.workspace_id,scope.project_id,@@ -3219,6 +3259,7 @@ impl AiMemoryServer {global_hits: Vec::new(),global_scope_hits,streams_active,+ timings,};ok_json(&response)}@@ -10400,6 +10441,7 @@ mod tests {assert!(text.contains("foo.md"), "expected hit; got {text}");assert!(!text.contains("score_details"));assert!(!text.contains("streams_active"));+ assert!(!text.contains("timings"), "{text}");let explained = server.memory_query(@@ -10442,6 +10484,97 @@ mod tests {let fused = details["fused"].as_f64().unwrap();let authority = details["authority"].as_f64().unwrap();assert!((rank + fused * authority).abs() < f64::EPSILON);+ // Explain times exactly the streams it reports as active: no+ // embedder means no vector timing, no reranker means no rerank one.+ let timings = value["timings"].as_object().expect("explain timings");+ let mut keys: Vec<&str> = timings.keys().map(String::as_str).collect();+ keys.sort_unstable();+ assert_eq!(+ keys,+ vec!["entity_ms", "fts_ms", "graph_ms", "observation_ms"],+ "{value}"+ );+ assert!(+ timings+ .values()+ .all(|ms| ms.as_f64().is_some_and(|ms| ms >= 0.0)),+ "{value}"+ );+ }++ /// `rerank_ms` appears with a reranker attached, and only on an explained+ /// query; global search reports its single FTS pass.+ #[tokio::test]+ async fn explain_timings_cover_the_reranker_and_global_search() {+ let (_tmp, store, server, ws, pj) = setup_server().await;+ // The reranker needs two candidates to be called at all.+ store+ .writer+ .upsert_page(NewPage {+ workspace_id: ws,+ project_id: pj,+ path: PagePath::new("bar.md").unwrap(),+ title: "Bar".into(),+ body: "karpathy compiled wiki notes".into(),+ tier: Tier::Semantic,+ frontmatter_json: serde_json::json!({}),+ pinned: false,+ links: Vec::new(),+ author_id: None,+ expires_at: None,+ entities: Vec::new(),+ evidence: Vec::new(),+ })+ .await+ .unwrap();+ let (reranker, calls, _) = stub_reranker(StubRerankOutcome::Reverse, Duration::ZERO);+ let server = server.with_reranker(reranker);+ let ask = async |explain: bool, global: bool| -> serde_json::Value {+ call_tool_json(+ server+ .memory_query(+ Parameters(QueryArgs {+ query: "karpathy".into(),+ limit: Some(5),+ project: None,+ scopes: Vec::new(),+ workspace: None,+ global: global.then_some(true),+ include_expired: None,+ include_superseded: None,+ pin_first: None,+ explain: Some(explain),+ as_of: None,+ answer: None,+ reasoning: None,+ }),+ OptionalParts(test_parts_default()),+ )+ .await+ .unwrap(),+ )+ };++ let explained = ask(true, false).await;+ assert_eq!(calls.load(std::sync::atomic::Ordering::SeqCst), 1);+ assert!(+ explained["timings"]["rerank_ms"]+ .as_f64()+ .is_some_and(|ms| ms >= 0.0),+ "{explained}"+ );+ let plain = ask(false, false).await;+ assert_eq!(calls.load(std::sync::atomic::Ordering::SeqCst), 2);+ assert!(plain.get("timings").is_none(), "{plain}");++ let global = ask(true, true).await;+ let timings = global["timings"].as_object().expect("global timings");+ assert_eq!(+ timings.keys().map(String::as_str).collect::<Vec<_>>(),+ vec!["fts_ms"],+ "{global}"+ );+ assert!(ask(false, true).await.get("timings").is_none());}/// With NO embedder configured — the default deployment —diff --git a/crates/ai-memory-mcp/tests/suite/query_observation_stream.rs b/crates/ai-memory-mcp/tests/suite/query_observation_stream.rsindex ee7008ec..a4f2eee5 100644--- a/crates/ai-memory-mcp/tests/suite/query_observation_stream.rs+++ b/crates/ai-memory-mcp/tests/suite/query_observation_stream.rs@@ -299,3 +299,27 @@ async fn a_live_session_without_a_page_still_reaches_the_raw_fallback() {assert_eq!(raw.len(), 1, "{payload}");assert_eq!(raw[0]["id"], seeded.observation_id.as_str());}++/// `explain=true` times the observation stream exactly when it runs, and a+/// query without explain carries no timings at all.+#[tokio::test]+async fn explain_timings_follow_the_observation_stream() {+ let on = seeded(true).await;+ let payload = query(&on.router, args()).await;+ assert!(+ payload["timings"]["observation_ms"]+ .as_f64()+ .is_some_and(|ms| ms >= 0.0),+ "{payload}"+ );+ let mut quiet = args();+ quiet["explain"] = json!(false);+ let payload = query(&on.router, quiet).await;+ assert!(payload.get("timings").is_none(), "{payload}");++ let off = seeded(false).await;+ let payload = query(&off.router, args()).await;+ let timings = payload["timings"].as_object().expect("timings");+ assert!(timings.contains_key("fts_ms"), "{payload}");+ assert!(!timings.contains_key("observation_ms"), "{payload}");+}diff --git a/crates/ai-memory-store/src/lib.rs b/crates/ai-memory-store/src/lib.rsindex 7a05c6d0..f38a17b2 100644--- a/crates/ai-memory-store/src/lib.rs+++ b/crates/ai-memory-store/src/lib.rs@@ -94,7 +94,8 @@ pub use reader::{RELATED_WALK_MAX_NODES, ReaderPool, ReindexTargetStatus, RelatedNode, RelatedPage,RrfContributions, ScopeRow, SearchExplain, SessionConsolidationSummary, SessionDependentRows,SessionEndDisposition, SessionSummary, SettledPage, StatusCounts, StorageStatus,- StoredEmbedding, StoredPageBody, WorkspaceScopeRow, WorkspaceSummary, f32_vec_to_bytes,+ StoredEmbedding, StoredPageBody, StreamTimings, WorkspaceScopeRow, WorkspaceSummary,+ f32_vec_to_bytes,};pub use retrieval_tuning::{RetrievalTuning, is_session_recall_query};pub use scope::{diff --git a/crates/ai-memory-store/src/reader.rs b/crates/ai-memory-store/src/reader.rsindex 7324855b..791e8e24 100644--- a/crates/ai-memory-store/src/reader.rs+++ b/crates/ai-memory-store/src/reader.rs@@ -729,6 +729,59 @@ fn is_zero(v: &f64) -> bool {*v == 0.0}+/// Per-query wall time of each retrieval stream, in milliseconds, for an+/// `explain=true` query. Measured only on the explained path, so a normal+/// query takes no clock readings for it. A `None` field is a stream that did+/// not run (no query vector, stream turned off, no reranker). Summed across+/// projects when one query searches several scopes.+#[derive(Debug, Clone, Copy, Default, PartialEq, Serialize)]+pub struct StreamTimings {+ /// Page FTS5 candidate search.+ #[serde(skip_serializing_if = "Option::is_none")]+ pub fts_ms: Option<f64>,+ /// Entity-match stream.+ #[serde(skip_serializing_if = "Option::is_none")]+ pub entity_ms: Option<f64>,+ /// Stored-vector scans (the page vectors plus, when enabled, the abstract+ /// vectors). Embedding the query itself is not included.+ #[serde(skip_serializing_if = "Option::is_none")]+ pub vector_ms: Option<f64>,+ /// Graph-neighbour expansion.+ #[serde(skip_serializing_if = "Option::is_none")]+ pub graph_ms: Option<f64>,+ /// Observation-evidence stream (`[retrieval] observation_stream`).+ #[serde(skip_serializing_if = "Option::is_none")]+ pub observation_ms: Option<f64>,+ /// Post-fusion reranker call. Filled by the MCP layer, which owns the+ /// reranker; the store never sets it.+ #[serde(skip_serializing_if = "Option::is_none")]+ pub rerank_ms: Option<f64>,+}++impl StreamTimings {+ /// Add `other` into `self`, field by field. A stream that ran in either+ /// counts as having run.+ pub fn accumulate(&mut self, other: &Self) {+ fn add(into: &mut Option<f64>, from: Option<f64>) {+ if let Some(ms) = from {+ *into = Some(into.unwrap_or(0.0) + ms);+ }+ }+ add(&mut self.fts_ms, other.fts_ms);+ add(&mut self.entity_ms, other.entity_ms);+ add(&mut self.vector_ms, other.vector_ms);+ add(&mut self.graph_ms, other.graph_ms);+ add(&mut self.observation_ms, other.observation_ms);+ add(&mut self.rerank_ms, other.rerank_ms);+ }+}++/// Milliseconds since `started`, or `None` when the clock was never read+/// (the non-explained path passes `None`).+fn elapsed_ms(started: Option<std::time::Instant>) -> Option<f64> {+ started.map(|t| t.elapsed().as_secs_f64() * 1000.0)+}+/// Score transparency for one `memory_query` hit: where the hit ranked/// in each retrieval stream, the raw per-stream scores, and how the/// RRF fusion combined them. All ranks are 1-based; a `None` rank@@ -5801,6 +5854,7 @@ impl ReaderPool {include_superseded,).await?+ .0.into_iter().map(|(hit, _)| hit).collect())@@ -5827,7 +5881,42 @@ impl ReaderPool {expiry_cutoff_us: Option<i64>,include_superseded: bool,) -> StoreResult<Vec<(PageHit, SearchExplain)>> {- let hits = self+ Ok(self+ .hybrid_search_explained_timed(+ workspace_id,+ project_id,+ query,+ query_vec,+ provider,+ model,+ dim,+ limit,+ expiry_cutoff_us,+ include_superseded,+ )+ .await?+ .0)+ }++ /// [`Self::hybrid_search_explained`] plus the wall time of each stream.+ ///+ /// # Errors+ /// Propagates any SQL or pool error.+ #[allow(clippy::too_many_arguments)]+ pub async fn hybrid_search_explained_timed(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ query: String,+ query_vec: Option<Vec<f32>>,+ provider: String,+ model: String,+ dim: u32,+ limit: usize,+ expiry_cutoff_us: Option<i64>,+ include_superseded: bool,+ ) -> StoreResult<(Vec<(PageHit, SearchExplain)>, StreamTimings)> {+ let (hits, timings) = self.hybrid_search_inner(workspace_id,project_id,@@ -5846,10 +5935,12 @@ impl ReaderPool {// `belief_factor`; P2, docs/design-hindsight-borrowings.md §3) are// populated inside the explained path of `hybrid_search_inner` from one// batched query — never per-hit — so no second pass is needed here.- Ok(hits- .into_iter()- .map(|(hit, explain)| (hit, explain.unwrap_or_default()))- .collect())+ Ok((+ hits.into_iter()+ .map(|(hit, explain)| (hit, explain.unwrap_or_default()))+ .collect(),+ timings,+ ))}#[allow(clippy::too_many_arguments)]@@ -5866,8 +5957,11 @@ impl ReaderPool {expiry_cutoff_us: Option<i64>,explain: bool,include_superseded: bool,- ) -> StoreResult<Vec<(PageHit, Option<SearchExplain>)>> {+ ) -> StoreResult<(Vec<(PageHit, Option<SearchExplain>)>, StreamTimings)> {let cutoff = expiry_cutoff_us.unwrap_or_else(now_us);+ // Stream timings exist only for explain: `explain.then(Instant::now)`+ // reads no clock on the normal path.+ let mut timings = StreamTimings::default();// The routing decision is read off the query before the FTS call// consumes it; with routing disabled this is a no-op `false`.let tuning = self.tuning;@@ -5881,6 +5975,7 @@ impl ReaderPool {// it. This avoids cloning a caller-controlled query on the hot path.// The default path uses the latest-only entry; the opt-in path reaches// superseded versions through the `_at` entry with no `as_of` instant.+ let started = explain.then(std::time::Instant::now);let entity_hits = if include_superseded {self.entity_hits_for_project_at(workspace_id,@@ -5902,12 +5997,14 @@ impl ReaderPool {).await?};+ timings.entity_ms = elapsed_ms(started);// The stream-off hot path keeps moving the query without a clone.let observation_query = if tuning.observation_stream {query.clone()} else {String::new()};+ let started = explain.then(std::time::Instant::now);let fts_candidates = self.search_page_candidates_for_project(workspace_id,@@ -5918,6 +6015,7 @@ impl ReaderPool {include_superseded,).await?;+ timings.fts_ms = elapsed_ms(started);let mut authorities: std::collections::HashMap<PageId, PageAuthority> = fts_candidates.iter().map(|(hit, authority)| (hit.id, *authority))@@ -5929,6 +6027,7 @@ impl ReaderPool {let mut vec_hits: Vec<(PageId, PagePath, f32)> = Vec::new();let mut abstract_hits: Vec<(PageId, PagePath, f32)> = Vec::new();if let Some(qv) = query_vec {+ let started = explain.then(std::time::Instant::now);if tuning.abstract_vectors {abstract_hits = self.top_embedding_hits_in_table(@@ -5958,6 +6057,7 @@ impl ReaderPool {include_superseded,).await?;+ timings.vector_ms = elapsed_ms(started);}let mut seed_seen = std::collections::HashSet::new();@@ -5987,6 +6087,7 @@ impl ReaderPool {}}}+ let started = explain.then(std::time::Instant::now);let graph_hits = self.graph_neighbors_for_project_explained(workspace_id,@@ -5997,17 +6098,22 @@ impl ReaderPool {include_superseded,).await?;- // R1: the observation-evidence stream, off by default. It votes only- // for latest session pages, so it does not seed graph expansion.+ timings.graph_ms = elapsed_ms(started);+ // R1: the observation-evidence stream. It votes only for latest+ // session pages, so it does not seed graph expansion.let observation_votes = if tuning.observation_stream {- self.observation_stream_votes(- workspace_id,- project_id,- &observation_query,- candidate_limit,- cutoff,- )- .await?+ let started = explain.then(std::time::Instant::now);+ let votes = self+ .observation_stream_votes(+ workspace_id,+ project_id,+ &observation_query,+ candidate_limit,+ cutoff,+ )+ .await?;+ timings.observation_ms = elapsed_ms(started);+ votes} else {Vec::new()};@@ -6317,7 +6423,7 @@ impl ReaderPool {hit.superseded = superseded.contains(&hit.id);}}- Ok(out)+ Ok((out, timings))}/// The subset of `page_ids` that are superseded (non-latest) versions in@@ -12167,6 +12273,67 @@ mod tests {assert!(free > 0);}+ /// Stream timings are an explain-only surface: the normal path returns+ /// none (it reads no clock for them), the explained path times every+ /// stream that ran and leaves the rest out.+ #[tokio::test]+ async fn stream_timings_exist_only_on_the_explained_path() {+ let tmp = tempfile::TempDir::new().unwrap();+ let store = crate::Store::open(tmp.path()).unwrap();+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let proj = store+ .writer+ .get_or_create_project(ws, "app", None)+ .await+ .unwrap();+ let search = |explain: bool| {+ store.reader.hybrid_search_inner(+ ws,+ proj,+ "anything".into(),+ None,+ String::new(),+ String::new(),+ 0,+ 5,+ None,+ explain,+ false,+ )+ };+ let (_, off) = search(false).await.unwrap();+ assert_eq!(off, super::StreamTimings::default());+ let (_, on) = search(true).await.unwrap();+ assert!(on.fts_ms.is_some() && on.entity_ms.is_some() && on.graph_ms.is_some());+ assert!(on.observation_ms.is_some(), "the stream is on by default");+ assert!(on.vector_ms.is_none(), "no query vector, no vector scan");+ assert!(on.rerank_ms.is_none(), "the store never reranks");+ }++ #[test]+ fn stream_timings_accumulate_across_scopes() {+ let mut total = super::StreamTimings::default();+ total.accumulate(&super::StreamTimings {+ fts_ms: Some(1.5),+ ..Default::default()+ });+ total.accumulate(&super::StreamTimings {+ fts_ms: Some(2.0),+ graph_ms: Some(0.25),+ ..Default::default()+ });+ assert_eq!(total.fts_ms, Some(3.5));+ assert_eq!(total.graph_ms, Some(0.25));+ assert_eq!(+ total.vector_ms, None,+ "a stream that never ran stays absent"+ );+ }+/// The single-scope manifest lookup the wiki resolves a new scope's/// `_meta.md` names from. It carries `repo_path`, and it is keyed by the/// full pair: a project id offered under the wrong workspace resolves todiff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex e3c9daca..fde0f6bf 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -592,7 +592,7 @@ does not need to ride on the public protocol change.| Tool | Hint | Purpose ||---|---|---|-| `memory_query` | read-only | FTS5 + entity-match + graph RRF + optional vector RRF search, followed by bounded kind/tier/pinned/tag authority adjustment and raw fallback. Bumps access counters for page hits. Defaults to the current project; single-project calls (project implicit or named with `workspace`+`project`) also union the reserved `_global` preferences scope as `global_scope_hits`, and only an explicit multi-`scopes` set opts out (#930); `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). With `AI_MEMORY_RERANKER=llm`, project/scopes candidate pools are fused before at most one final LLM relevance pass; query/title/snippet data is bounded and JSON-encoded, and any timeout, provider error, invalid/incomplete score set, or four-call concurrency saturation preserves the adjusted order. The distinct `global=true` FTS-only ranker and supplemental global-preference hits are not reranked. `explain=true` attaches per-hit `score_details` (per-stream ranks, matched entities, raw FTS/cosine/entity inverse-frequency scores, the matched observation ids when the observation stream voted, RRF contributions, graph provenance including the typed edge kind (`causes`/`fixes`/`contradicts`) a neighbour was reached by, the page's evidence count, authority multiplier, and optional rerank score) to project/scopes hits plus a top-level `streams_active` list. The global FTS-only ranker reports its active stream without per-hit details. `include_expired=true` also returns TTL-expired pages. `include_superseded=true` also returns superseded (non-latest) page versions across the FTS/entity/vector/graph streams, each hit labelled `superseded: true` (the current version is never marked); default-off is byte-identical to the latest-only behaviour, and `global=true` / `as_of` are unaffected. `pin_first=true` prepends the project's bounded pinned latest pages (`ReaderPool::list_pinned_pages`, cap 10) ahead of the fused hits, deduped by page id (a pinned page that also matches appears once, marked `pinned: true`) and re-truncated to the requested limit; it applies to single-project searches (default or `workspace`+`project`), is ignored on `scopes`/`global`/`as_of`, and default-off is byte-identical. `answer=true` (opt-in, off by default) additionally synthesizes a cited natural-language answer over the top hits via the configured LLM provider (`complete_structured`, JSON-schema `{ answer, citations }`), attached as `answer: { text, citations }`; with no provider configured it returns the hits plus an `answer_unavailable` note instead of erroring, and with `answer` unset/`false` no provider is accessed and the response is byte-identical (invariant #13). It applies to the normal single-project/`scopes` path; `global`/`as_of` ignore it. Answer quality is not yet eval-validated. An optional `reasoning` tier (`minimal` (default) / `low` / `medium` / `high` / `max`) tunes the synthesis effort: `ChatRequest` carries no per-request reasoning field (the provider-level `reasoning_effort` is fixed at construction from config), so the tier maps to a per-tier max-token budget scaled off the path's base (answer base 2 000; `minimal` = 1x = byte-identical, `low` 1.5x, `medium` 2x, `high` 3x, `max` 4x). The tier is inert unless the `answer` LLM path runs (invariant #13); an unknown value is rejected by the schema (invariant #7). |+| `memory_query` | read-only | FTS5 + entity-match + graph RRF + optional vector RRF search, followed by bounded kind/tier/pinned/tag authority adjustment and raw fallback. Bumps access counters for page hits. Defaults to the current project; single-project calls (project implicit or named with `workspace`+`project`) also union the reserved `_global` preferences scope as `global_scope_hits`, and only an explicit multi-`scopes` set opts out (#930); `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). With `AI_MEMORY_RERANKER=llm`, project/scopes candidate pools are fused before at most one final LLM relevance pass; query/title/snippet data is bounded and JSON-encoded, and any timeout, provider error, invalid/incomplete score set, or four-call concurrency saturation preserves the adjusted order. The distinct `global=true` FTS-only ranker and supplemental global-preference hits are not reranked. `explain=true` attaches per-hit `score_details` (per-stream ranks, matched entities, raw FTS/cosine/entity inverse-frequency scores, the matched observation ids when the observation stream voted, RRF contributions, graph provenance including the typed edge kind (`causes`/`fixes`/`contradicts`) a neighbour was reached by, the page's evidence count, authority multiplier, and optional rerank score) to project/scopes hits plus a top-level `streams_active` list and a `timings` object with each stream's wall time in milliseconds (`fts_ms`, `entity_ms`, `vector_ms` for the stored-vector scans, `graph_ms`, `observation_ms`, `rerank_ms`; a stream that did not run is left out, and `scopes` searches sum per stream). Timings are read only on an explained query; the normal path takes no clock reading for them. The global FTS-only ranker reports its active stream and `fts_ms` without per-hit details; `as_of` reports no timings. `include_expired=true` also returns TTL-expired pages. `include_superseded=true` also returns superseded (non-latest) page versions across the FTS/entity/vector/graph streams, each hit labelled `superseded: true` (the current version is never marked); default-off is byte-identical to the latest-only behaviour, and `global=true` / `as_of` are unaffected. `pin_first=true` prepends the project's bounded pinned latest pages (`ReaderPool::list_pinned_pages`, cap 10) ahead of the fused hits, deduped by page id (a pinned page that also matches appears once, marked `pinned: true`) and re-truncated to the requested limit; it applies to single-project searches (default or `workspace`+`project`), is ignored on `scopes`/`global`/`as_of`, and default-off is byte-identical. `answer=true` (opt-in, off by default) additionally synthesizes a cited natural-language answer over the top hits via the configured LLM provider (`complete_structured`, JSON-schema `{ answer, citations }`), attached as `answer: { text, citations }`; with no provider configured it returns the hits plus an `answer_unavailable` note instead of erroring, and with `answer` unset/`false` no provider is accessed and the response is byte-identical (invariant #13). It applies to the normal single-project/`scopes` path; `global`/`as_of` ignore it. Answer quality is not yet eval-validated. An optional `reasoning` tier (`minimal` (default) / `low` / `medium` / `high` / `max`) tunes the synthesis effort: `ChatRequest` carries no per-request reasoning field (the provider-level `reasoning_effort` is fixed at construction from config), so the tier maps to a per-tier max-token budget scaled off the path's base (answer base 2 000; `minimal` = 1x = byte-identical, `low` 1.5x, `medium` 2x, `high` 3x, `max` 4x). The tier is inert unless the `answer` LLM path runs (invariant #13); an unknown value is rejected by the schema (invariant #7). || `memory_recent` | read-only | Most-recently-updated `is_latest=1` pages. || `memory_read_page` | read-only | Fetch the FULL body of a single wiki page by `path` or by top FTS5 hit for a `query`; optional `workspace` + `project` targets a named sibling workspace/project. Use when an agent needs more than the 24-word snippets from `memory_query`. `include_related=true` also walks the link graph outward from the page (bounded BFS reusing the `page_links` primitive per node: default 1 hop, hard cap 3, global visited set for dedup/cycle-safety, total-node cap 50, cross-project aware) and returns a `related` array of reachable pages, each labelled with its `depth` (hop distance) and `direction` (`link`/`backlink`); default-off is byte-identical (no `related` field). || `memory_read_session_observations` | read-only | Page through ONE session's raw hook observations (`ObservationRecord` with full sanitized body, capped per row by `body_max_chars`), restricted to the rows that landed in the resolved scope and to sessions the caller may see; `total` and `elided_other_scope` report the in-scope count and the rows the session left in another project. `session_id` omitted reads the latest completed visible session. |diff --git a/docs/usage.md b/docs/usage.mdindex 48bd2edb..2a9ab14d 100644--- a/docs/usage.md+++ b/docs/usage.md@@ -154,7 +154,7 @@ at the managed ai-memory Agent Skills that carry detailed tool routing.| "Remember this permanently" / "add an annotation" | `memory_write_page` | Writes durable wiki knowledge; not a single-use handoff. || "Remember this until Friday" / "expire this after the migration" | `memory_write_page` with `expires_at` | Writes a time-bounded page. Use RFC3339 or `YYYY-MM-DD` (end of day UTC); normal retrieval hides it after expiry and the next forget sweep deletes it. TTL outranks `pinned`. || "Search expired notes for X" | `memory_query` with `include_expired: true` | Opts an explicit project, sibling-scope, or global search into expired historical pages; ordinary searches exclude them. |-| "Why did this page rank here?" | `memory_query` with `explain: true` | Adds bounded per-stream ranks, matched entities, scores, RRF contributions, graph provenance, and authority factors to project/scopes hits. A global query reports only its distinct FTS stream. |+| "Why did this page rank here?" | `memory_query` with `explain: true` | Adds bounded per-stream ranks, matched entities, scores, RRF contributions, graph provenance, and authority factors to project/scopes hits, plus a top-level `timings` object with each stream's wall time in milliseconds (`fts_ms`, `entity_ms`, `vector_ms`, `graph_ms`, `observation_ms`, `rerank_ms`; streams that did not run are left out). A global query reports only its distinct FTS stream and `fts_ms`. || Improve top project/scopes search relevance | Set `AI_MEMORY_RERANKER=llm` on a server with an LLM provider | Sends the bounded query plus up to 30 bounded titles/snippets to the provider for at most one final relevance pass. Invalid, partial, failed, timed-out, or concurrency-saturated requests preserve the normal order; `global=true` and supplemental global-preference hits are unchanged. || "Delete this page" / "remove the note about X" | `memory_delete_page` | Removes a page by exact path. Pass `workspace` + `project` together when the page lives in a sibling workspace, so a project name shared between workspaces never silently routes the delete to the wrong slot. || "That recalled page helped" / "this page is stale" | `memory_feedback` | Records `helpful`, `not_helpful`, `stale`, or `wrong` for the exact path. Retention weight affects sweep-eligible episodic pages; stale/wrong also flag any current page for lint review. Retrieved content never authorizes feedback by itself. |