Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
feat(store): entity matches as a fourth RRF stream (V38)
Samir Hanna Verza0a02b3ab79ea
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 474b3525..39d47792 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,91 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]### Added+- Entity-match retrieval as a fourth RRF stream (V38 `entities` ++ `entity_page_links`). Consolidation now emits an `entities` list per+ page — the specific technologies, components, and domain nouns the+ content names — which lands in the page's frontmatter (markdown stays+ the source of truth; a reindex rebuilds the index) and is matched+ lexically at query time: exact name, name-prefix, or a word inside a+ multi-word entity, weighted by inverse entity frequency so a rare noun+ outranks a ubiquitous one. This finds pages whose *wording* misses the+ query — a note whose body says "the streaming broker" now answers a+ query for "jetstream". No spaCy, no NLP dependency, and no LLM call at+ query time: extraction happens only where an LLM already runs, so a+ project that never consolidated has an empty table and the stream+ contributes nothing — the zero-LLM default path is unchanged. With+ `explain: true` each hit reports its `entity_rank` and+ `matched_entities`. `entities.id` is deliberately shaped as the stable+ anchor a future temporal-triples table could reference; that P2 item+ stays deferred, and no graph database is involved either way.+- Optional post-RRF reranking for `memory_query`, off by default. Set+ `AI_MEMORY_RERANKER=llm` (requires `AI_MEMORY_LLM_PROVIDER`) and the+ query over-fetches `limit * 3` candidates (capped at 30) from the RRF+ stage, then has the configured LLM provider score each one against the+ query through JSON-schema structured output — no new provider dialect,+ and it works across all eight existing providers. Scored candidates+ sort by relevance and any the model skipped follow in RRF order.+ Degrades rather than fails: a 20-second timeout, a provider error, or a+ response covering fewer than half the candidates all fall back to plain+ RRF order, matching the embedder's contract. With `explain: true` each+ hit's `score_details` carries its `rerank_score`. An unknown+ `AI_MEMORY_RERANKER` value, or `llm` without a provider, fails at+ startup instead of silently disabling the feature.+- New MCP tool `memory_feedback` (17th tool) — the "finer-grained+ reinforcement beyond access counts" P2 item. Record how useful a+ recalled page actually was by exact path: `helpful` / `not_helpful`+ step the page's new `pages.salience` column (V37, bounded to+ `[0.25, 2.0]` in 0.25 steps), which now scales the retention formula's+ time term instead of a single global `salience_default`; `stale` /+ `wrong` floor the salience AND surface the page as a+ `feedback_flagged` finding in the next `memory_lint` report. Signals+ land in a new append-only `page_feedback` table with an optional+ sanitized reason and full audit-log entry. Nothing is ever deleted by+ feedback. Signals attach to the page *version* that was read, so+ rewriting a flagged page retires both its salience and its lint+ findings — there is no separate dismissal state. Pages without+ feedback keep `salience = NULL`, which reads as exactly the previous+ behaviour.+- `memory_query` explain mode: pass `explain: true` and every hit carries+ `score_details` — its 1-based rank and raw score in each retrieval+ stream (FTS5 bm25 score, embedding cosine, graph-neighbour provenance+ naming the seed page and link direction), the per-stream RRF+ contributions, the fused score, and the bounded page-authority+ multiplier applied after fusion — plus a top-level `streams_active`+ list that makes degradation visible (an embedder failure shows up as+ `["fts"]` instead of `["fts","vector","graph"]`). Reporting the+ authority factor alongside `fused` is what keeps the surface honest:+ the returned `rank` is the fused score *after* that multiplier, so+ `fused` alone would not account for the ordering. Costs nothing extra:+ the data was already computed and discarded. Hybrid ranking also gained+ a deterministic path tiebreak for equal adjusted scores, which+ previously fell back to hash-map iteration order.+- Per-page TTL via a frontmatter `expires_at:` key (RFC3339, or a bare+ `YYYY-MM-DD` meaning end of that day UTC), mirrored into a new+ `pages.expires_at` column (V36) and settable through a new optional+ `expires_at` parameter on `memory_write_page`. Expired pages are hidden+ from `memory_query`/`memory_recent`/briefing/session-brief surfaces —+ `memory_query` gains `include_expired: true` to still see them — while+ exact-path reads still return the page, annotated `expired: true`,+ because an explicit read is not a search. The forget sweep hard-deletes+ them through the wiki layer, so the markdown file goes too, not just+ the rows. An explicit TTL outranks `pinned` (a pin means "don't decay+ this", not "keep it past the date its author set"); `memory_lint`+ flags pinned+expiring pages so the combination is visible rather than+ silent ([#309]).+- New `openai-compat` embedding provider for self-hosted engines+ (Ollama, LM Studio, vLLM). Set+ `AI_MEMORY_EMBEDDING_PROVIDER=openai-compat` together with explicit+ `AI_MEMORY_EMBEDDING_BASE_URL`, `AI_MEMORY_EMBEDDING_MODEL`, and+ `AI_MEMORY_EMBEDDING_DIM` — there is no safe default model or+ dimensionality for a self-hosted engine, so each is required rather+ than guessed. Unlike the other providers it is keyless: a bearer+ token is sent only when `LLM_API_KEY` is present, for gateways that+ want one. Embeddings are stored under their own+ `provider="openai-compat"` identity, so switching an existing+ `openai`+base-URL setup over changes the stored+ `{provider, model, dim}` triple — run `ai-memory embed --force` to+ re-embed. (#300)- Optional post-RRF reranking for `memory_query`, off by default. Set`AI_MEMORY_RERANKER=llm` (requires `AI_MEMORY_LLM_PROVIDER`) and thequery over-fetches `limit * 3` candidates (capped at 30) from the RRFdiff --git a/crates/ai-memory-cli/src/commands/reindex.rs b/crates/ai-memory-cli/src/commands/reindex.rsindex d2c7afa8..973e59cf 100644--- a/crates/ai-memory-cli/src/commands/reindex.rs+++ b/crates/ai-memory-cli/src/commands/reindex.rs@@ -121,6 +121,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();diff --git a/crates/ai-memory-consolidate/src/consolidator.rs b/crates/ai-memory-consolidate/src/consolidator.rsindex 9013e966..ea4bf373 100644--- a/crates/ai-memory-consolidate/src/consolidator.rs+++ b/crates/ai-memory-consolidate/src/consolidator.rs@@ -521,6 +521,20 @@ fn build_update(),);}+ // Entities land in frontmatter (markdown stays the source of truth);+ // the store derives its index from there, so a reindex rebuilds them.+ let entities = ai_memory_core::normalize_entities(&upd.entities);+ if !entities.is_empty() {+ fm.insert(+ "entities".into(),+ serde_json::Value::Array(+ entities+ .into_iter()+ .map(serde_json::Value::String)+ .collect(),+ ),+ );+ }if is_slot_path(&path) {fm.insert("slot_kind".into(),@@ -725,9 +739,22 @@ fn build_batch_request_with_slots(- \"tier\" (string) required — one of: working | episodic | semantic | procedural\n\- \"kind\" (string) required — one of: decision | gotcha | rule | fact\n\- \"tags\" (array of string) required — may be empty `[]`, but the key must be present\n\+ - \"entities\" (array of string) required — may be empty `[]`, but the key must be present; see below\n\- \"slot_kind\" (string) optional — ONLY for `_slots/*`; one of \"state\" or \"invariant\"; this is the SLOT WRITE REGIME, NOT a tier value\n\No other keys except optional `slot_kind` on `_slots/*`. No `body`, no `content`, no `summary`. Field names \are case-sensitive and the `_markdown` suffix matters.\n\+ \n## `entities` field — the specific nouns the page is about\n\+ Up to 10 short names (max 64 chars each), lowercase, taken from \+ what the page actually names: technologies (`sqlite`, `tokio`), \+ components (`writer actor`, `hook router`), services, crates, \+ file or module names, and product/domain nouns. They power a \+ retrieval stream, so a later query naming one of them finds this \+ page even when the wording differs.\n\+ Do NOT include: generic words (`code`, `bug`, `change`, \+ `refactor`), the tier or kind values, whole sentences, or \+ restatements of the title. Prefer fewer, more specific entries \+ over padding the list. `[]` is correct for a page with no \+ specific nouns.\n\\n## Output format (read this carefully)\n\Reply with ONE JSON object matching the ConsolidatedBatch schema, \and nothing else. NO prose preamble, NO trailing commentary, NO \@@ -1115,6 +1142,7 @@ mod tests {body_markdown: "Ship the slot-kind PR.".into(),tags: Vec::new(),slot_kind: SlotKind::State,+ entities: Vec::new(),};let (req, _) = build_update(WorkspaceId::new(),@@ -1138,6 +1166,7 @@ mod tests {body_markdown: "body".into(),tags: Vec::new(),slot_kind: SlotKind::State,+ entities: Vec::new(),};let actor = ai_memory_core::ActorContext {user: Some("djalmajr".into()),@@ -1174,6 +1203,7 @@ mod tests {body_markdown: "This repo uses a markdown wiki as source of truth.".into(),tags: Vec::new(),slot_kind: SlotKind::Invariant,+ entities: Vec::new(),};let (req, _) = build_update(WorkspaceId::new(),diff --git a/crates/ai-memory-consolidate/src/curator.rs b/crates/ai-memory-consolidate/src/curator.rsindex 250567ba..990e3896 100644--- a/crates/ai-memory-consolidate/src/curator.rs+++ b/crates/ai-memory-consolidate/src/curator.rs@@ -508,6 +508,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();diff --git a/crates/ai-memory-consolidate/src/lib.rs b/crates/ai-memory-consolidate/src/lib.rsindex 482beef3..d668a9ce 100644--- a/crates/ai-memory-consolidate/src/lib.rs+++ b/crates/ai-memory-consolidate/src/lib.rs@@ -62,7 +62,7 @@ pub use embed::{EmbedBackfillCounts, EmbedBackfillError, EmbedBackfillOptions, run_embedding_backfill,};pub use lint::{LintError, LintFinding, LintReport, run_lint};-pub use sweep::{EvictedPage, SweepError, SweepReport, run_sweep};+pub use sweep::{EvictedPage, ExpiredPage, SweepError, SweepReport, run_sweep};pub use types::{ConsolidatedBatch, ConsolidatedPage, ConsolidatedPageUpdate, ConsolidationOutcome, PageKind,SlotKind,diff --git a/crates/ai-memory-consolidate/src/types.rs b/crates/ai-memory-consolidate/src/types.rsindex 2825741b..74c5fbf2 100644--- a/crates/ai-memory-consolidate/src/types.rs+++ b/crates/ai-memory-consolidate/src/types.rs@@ -125,6 +125,14 @@ pub struct ConsolidatedPageUpdate {/// current behaviour.#[serde(default)]pub slot_kind: SlotKind,+ /// Salient nouns this page is about — the specific technologies,+ /// components, services, and files the content names. Indexed as a+ /// retrieval stream so a query naming one of them finds the page+ /// even when the wording differs. Normalised and capped+ /// (`ai_memory_core::normalize_entities`) before storage; defaults+ /// to empty so older structured outputs still deserialise.+ #[serde(default)]+ pub entities: Vec<String>,}/// Batch produced by [`ConsolidatorMulti`].diff --git a/crates/ai-memory-consolidate/tests/recall_eval.rs b/crates/ai-memory-consolidate/tests/recall_eval.rsindex 32ddc9ea..4ff11098 100644--- a/crates/ai-memory-consolidate/tests/recall_eval.rs+++ b/crates/ai-memory-consolidate/tests/recall_eval.rs@@ -250,6 +250,84 @@ async fn graph_neighbor_expansion_recovers_linked_page() {);}+/// The entity stream recovers a probe that BOTH pure FTS and the graph+/// stream miss: the page's body never carries the query's wording, and it+/// has no links. The only bridge is the frontmatter `entities:` list the+/// consolidator writes.+#[tokio::test]+async fn entity_stream_recovers_a_probe_fts_and_graph_both_miss() {+ let tmp = TempDir::new().expect("tempdir");+ let store = Store::open(tmp.path()).expect("open store");+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .expect("ws");+ let proj = store+ .writer+ .get_or_create_project(ws, "eval", None)+ .await+ .expect("proj");+ let wiki = Wiki::new(tmp.path(), store.writer.clone()).expect("wiki");++ wiki.write_page(WritePageRequest {+ workspace_id: ws,+ project_id: proj,+ path: PagePath::new("decisions/queue-choice.md").expect("path"),+ // Body deliberately avoids the probe term and carries no links.+ frontmatter: serde_json::json!({+ "title": "Queue choice",+ "entities": ["nats jetstream", "delivery guarantees"],+ }),+ body: "We picked the streaming broker for at-least-once delivery.".into(),+ tier: Tier::Semantic,+ pinned: false,+ title: None,+ admission_ctx: None,+ author_id: None,+ actor: ai_memory_core::ActorContext::anonymous(),+ })+ .await+ .expect("write page");++ let probe = "jetstream";+ let fts_hits = store+ .reader+ .search_pages_for_project(ws, proj, probe.into(), 5, None)+ .await+ .expect("fts search");+ assert!(+ fts_hits.is_empty(),+ "pure FTS must miss: the body never says {probe}"+ );++ let hybrid_hits = store+ .reader+ .hybrid_search(+ ws,+ proj,+ probe.into(),+ None,+ String::new(),+ String::new(),+ 0,+ 5,+ None,+ )+ .await+ .expect("hybrid search");+ assert!(+ hybrid_hits+ .iter()+ .any(|hit| hit.path.as_str() == "decisions/queue-choice.md"),+ "the entity stream should recover the page: {:?}",+ hybrid_hits+ .iter()+ .map(|h| h.path.as_str())+ .collect::<Vec<_>>(),+ );+}+#[tokio::test]async fn raw_observation_fallback_recovers_detail_when_wiki_misses() {let tmp = TempDir::new().expect("tempdir");diff --git a/crates/ai-memory-core/src/lib.rs b/crates/ai-memory-core/src/lib.rsindex 2ec9aaef..40ab32b3 100644--- a/crates/ai-memory-core/src/lib.rs+++ b/crates/ai-memory-core/src/lib.rs@@ -45,7 +45,10 @@ pub use ids::{PageFeedbackId, PageId, PagePath, ProjectId, SessionId, UserId, WorkspaceId, WorkstreamId,};pub use observation::{NewObservation, NewSession, Observation, ObservationKind};-pub use page::{FeedbackKind, LinkTarget, NewPage, Page, Tier};+pub use page::{+ FeedbackKind, LinkTarget, MAX_ENTITIES_PER_PAGE, MAX_ENTITY_LEN, NewPage, Page, Tier,+ normalize_entities, normalize_entity,+};pub use routing_snippet::{MARKER_END, MARKER_START, SNIPPET_BODY, find_marker_line, full_block};pub use sanitize::{OBSERVATION_BODY_MAX_BYTES, SanitizeConfig, Sanitized, Sanitizer, truncate_utf8_bytes,diff --git a/crates/ai-memory-core/src/page.rs b/crates/ai-memory-core/src/page.rsindex d93df0a6..8d4310ef 100644--- a/crates/ai-memory-core/src/page.rs+++ b/crates/ai-memory-core/src/page.rs@@ -74,6 +74,61 @@ pub struct NewPage {/// search/recent/briefing and hard-deleted by the retention sweep.#[serde(default)]pub expires_at: Option<Timestamp>,+ /// Salient nouns for this page, from the frontmatter `entities:`+ /// list the consolidator writes. Normalised by+ /// [`normalize_entity`] before it reaches the store, which indexes+ /// them as a retrieval stream (V38). Empty for hand-written pages+ /// that don't declare any.+ #[serde(default)]+ pub entities: Vec<String>,+}++/// Longest accepted entity name; longer values are rejected rather than+/// truncated (a 100-char "entity" is a sentence, not a noun).+pub const MAX_ENTITY_LEN: usize = 64;+/// Cap on entities indexed per page. Beyond this the list stops being a+/// salience signal and starts being a second copy of the body.+pub const MAX_ENTITIES_PER_PAGE: usize = 10;++/// Normalise one entity name for storage and matching: trim, collapse+/// internal whitespace, lowercase. Returns `None` when the result is+/// empty or longer than [`MAX_ENTITY_LEN`].+///+/// Lowercasing is what makes query-time matching lexical rather than a+/// second LLM call: `Postgres`, `postgres`, and `POSTGRES` all index and+/// match as one entity.+#[must_use]+pub fn normalize_entity(raw: &str) -> Option<String> {+ let collapsed = raw.split_whitespace().collect::<Vec<_>>().join(" ");+ let normalized = collapsed.to_lowercase();+ if normalized.is_empty() || normalized.chars().count() > MAX_ENTITY_LEN {+ return None;+ }+ Some(normalized)+}++/// Normalise a whole entity list: drop invalid entries, de-duplicate+/// while preserving order, and cap at [`MAX_ENTITIES_PER_PAGE`].+#[must_use]+pub fn normalize_entities<I, S>(raw: I) -> Vec<String>+where+ I: IntoIterator<Item = S>,+ S: AsRef<str>,+{+ let mut seen = std::collections::BTreeSet::new();+ let mut out = Vec::new();+ for item in raw {+ let Some(name) = normalize_entity(item.as_ref()) else {+ continue;+ };+ if seen.insert(name.clone()) {+ out.push(name);+ if out.len() >= MAX_ENTITIES_PER_PAGE {+ break;+ }+ }+ }+ out}/// A link target discovered in a page body.@@ -269,6 +324,7 @@ mod tests {"\"procedural\"");}+#[test]fn feedback_kind_round_trips_and_flags_lint_routing() {for k in [@@ -285,4 +341,35 @@ mod tests {assert!(FeedbackKind::Stale.routes_to_lint());assert!(FeedbackKind::Wrong.routes_to_lint());}++ #[test]+ fn normalize_entity_lowercases_trims_and_collapses() {+ assert_eq!(+ normalize_entity(" Postgres ").as_deref(),+ Some("postgres")+ );+ assert_eq!(+ normalize_entity("Writer\n\tActor").as_deref(),+ Some("writer actor")+ );+ assert_eq!(normalize_entity(" ").as_deref(), None);+ assert_eq!(normalize_entity("").as_deref(), None);+ assert_eq!(+ normalize_entity(&"x".repeat(MAX_ENTITY_LEN)).as_deref(),+ Some("x".repeat(MAX_ENTITY_LEN).as_str()),+ );+ assert_eq!(normalize_entity(&"x".repeat(MAX_ENTITY_LEN + 1)), None);+ }++ #[test]+ fn normalize_entities_dedupes_preserves_order_and_caps() {+ assert_eq!(+ normalize_entities(["SQLite", "sqlite", " FTS5 ", ""]),+ vec!["sqlite".to_string(), "fts5".to_string()],+ );+ let many: Vec<String> = (0..MAX_ENTITIES_PER_PAGE + 5)+ .map(|i| format!("entity{i}"))+ .collect();+ assert_eq!(normalize_entities(&many).len(), MAX_ENTITIES_PER_PAGE);+ }}diff --git a/crates/ai-memory-hooks/src/router.rs b/crates/ai-memory-hooks/src/router.rsindex 1679c6d0..ee29d966 100644--- a/crates/ai-memory-hooks/src/router.rs+++ b/crates/ai-memory-hooks/src/router.rs@@ -6258,6 +6258,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}}diff --git a/crates/ai-memory-hooks/src/synth.rs b/crates/ai-memory-hooks/src/synth.rsindex d3145a16..ca52447d 100644--- a/crates/ai-memory-hooks/src/synth.rs+++ b/crates/ai-memory-hooks/src/synth.rs@@ -48,6 +48,7 @@ pub fn synthesize_session_page(links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}}diff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rsindex bd3c791e..9daa9c0e 100644--- a/crates/ai-memory-mcp/src/server.rs+++ b/crates/ai-memory-mcp/src/server.rs@@ -3377,6 +3377,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -4507,12 +4508,416 @@ mod tests {assert_eq!(details["fts_rank"], 1);assert!(details.get("vector_rank").is_none());assert_eq!(details["rrf"]["vector"], 0.0);+ // This project never consolidated, so no entity rows exist and the+ // fourth stream contributes nothing — the same way the vector stream+ // stays silent without an embedder.+ assert!(details.get("entity_rank").is_none());+ assert_eq!(details["rrf"]["entity"], 0.0);let rank = value["hits"][0]["rank"].as_f64().unwrap();let fused = details["fused"].as_f64().unwrap();let authority = details["authority"].as_f64().unwrap();assert!((rank + fused * authority).abs() < f64::EPSILON);}+ /// Regression: with NO embedder configured — the default deployment —+ /// `memory_query` must still run the entity and graph streams, not+ /// just FTS. This path used to short-circuit to a bare FTS query,+ /// which silently disabled both for every zero-LLM install.+ #[tokio::test]+ async fn query_without_embedder_still_runs_entity_and_graph_streams() {+ let (_tmp, store, server, ws, proj) = setup_server().await;+ assert!(+ server.embedder.is_none(),+ "this test is specifically the no-embedder path"+ );++ // A page findable ONLY via its entities (body avoids the query+ // word), and a page findable ONLY via a link from an FTS hit.+ let mut entity_only = NewPage {+ workspace_id: ws,+ project_id: proj,+ path: PagePath::new("concepts/broker.md").unwrap(),+ title: "Broker".into(),+ body: "The chosen transport gives at-least-once semantics.".into(),+ tier: Tier::Semantic,+ frontmatter_json: serde_json::json!({}),+ pinned: false,+ links: Vec::new(),+ author_id: None,+ expires_at: None,+ entities: vec!["nats jetstream".into()],+ };+ store.writer.upsert_page(entity_only.clone()).await.unwrap();+ entity_only.path = PagePath::new("concepts/linked.md").unwrap();+ entity_only.title = "Linked".into();+ entity_only.body = "Nothing quotable here.".into();+ entity_only.entities = Vec::new();+ store.writer.upsert_page(entity_only).await.unwrap();+ store+ .writer+ .upsert_page(NewPage {+ workspace_id: ws,+ project_id: proj,+ path: PagePath::new("concepts/seed.md").unwrap(),+ title: "Seed".into(),+ body: "graphseed points at [[concepts/linked.md]].".into(),+ tier: Tier::Semantic,+ frontmatter_json: serde_json::json!({}),+ pinned: false,+ links: vec![PagePath::new("concepts/linked.md").unwrap().into()],+ author_id: None,+ expires_at: None,+ entities: Vec::new(),+ })+ .await+ .unwrap();++ let query = async |q: &str| -> serde_json::Value {+ let result = server+ .memory_query(+ Parameters(QueryArgs {+ query: q.into(),+ limit: Some(10),+ project: None,+ scopes: Vec::new(),+ workspace: None,+ global: None,+ include_expired: None,+ explain: Some(true),+ }),+ OptionalParts(test_parts_default()),+ )+ .await+ .unwrap();+ let text = result.content.first().and_then(|c| c.as_text()).unwrap();+ serde_json::from_str(&text.text).unwrap()+ };++ let entity_hit = query("jetstream").await;+ let paths: Vec<&str> = entity_hit["hits"]+ .as_array()+ .unwrap()+ .iter()+ .map(|h| h["path"].as_str().unwrap())+ .collect();+ assert!(+ paths.contains(&"concepts/broker.md"),+ "entity stream must run without an embedder: {paths:?}"+ );+ assert_eq!(+ entity_hit["hits"][0]["score_details"]["matched_entities"],+ serde_json::json!(["nats jetstream"]),+ "{entity_hit}"+ );++ let graph_hit = query("graphseed").await;+ let paths: Vec<&str> = graph_hit["hits"]+ .as_array()+ .unwrap()+ .iter()+ .map(|h| h["path"].as_str().unwrap())+ .collect();+ assert!(+ paths.contains(&"concepts/linked.md"),+ "graph stream must run without an embedder: {paths:?}"+ );+ }++ /// A configured reranker reorders the RRF result; a failing or+ /// hanging one degrades to RRF order instead of erroring the query.+ #[tokio::test]+ async fn reranker_reorders_hits_and_degrades_on_failure() {+ use ai_memory_llm::{RerankCandidate, RerankScore, Reranker};++ /// Scores candidates by a fixed path→relevance table; paths not+ /// in the table are skipped (the "reranker had no opinion" case).+ struct TableReranker(Vec<(&'static str, f32)>);+ #[async_trait::async_trait]+ impl Reranker for TableReranker {+ fn name(&self) -> &'static str {+ "table"+ }+ fn model(&self) -> &str {+ "test"+ }+ async fn rerank(+ &self,+ _query: &str,+ candidates: &[RerankCandidate],+ ) -> ai_memory_llm::error::LlmResult<Vec<RerankScore>> {+ Ok(candidates+ .iter()+ .filter_map(|c| {+ self.0+ .iter()+ .find(|(path, _)| *path == c.id)+ .map(|(_, rel)| RerankScore {+ id: c.id.clone(),+ relevance: *rel,+ })+ })+ .collect())+ }+ }++ struct FailingReranker;+ #[async_trait::async_trait]+ impl Reranker for FailingReranker {+ fn name(&self) -> &'static str {+ "failing"+ }+ fn model(&self) -> &str {+ "test"+ }+ async fn rerank(+ &self,+ _query: &str,+ _candidates: &[RerankCandidate],+ ) -> ai_memory_llm::error::LlmResult<Vec<RerankScore>> {+ Err(ai_memory_llm::LlmError::Provider {+ status: 500,+ body: "boom".into(),+ })+ }+ }++ let (_tmp, store, server, ws, proj) = setup_server().await;+ // Three pages all matching "karpathy"; `foo.md` comes from the+ // fixture. FTS decides the baseline order.+ for (path, body) in [+ ("second.md", "Karpathy wiki second page about compiling"),+ ("third.md", "Karpathy wiki third page about compiling"),+ ] {+ store+ .writer+ .upsert_page(NewPage {+ workspace_id: ws,+ project_id: proj,+ path: PagePath::new(path).unwrap(),+ title: path.into(),+ body: body.into(),+ tier: Tier::Semantic,+ frontmatter_json: serde_json::json!({}),+ pinned: false,+ links: Vec::new(),+ author_id: None,+ expires_at: None,+ entities: Vec::new(),+ })+ .await+ .unwrap();+ }+ let paths_of = |json: &serde_json::Value| -> Vec<String> {+ json["hits"]+ .as_array()+ .unwrap()+ .iter()+ .map(|h| h["path"].as_str().unwrap().to_string())+ .collect()+ };+ async fn query(server: &AiMemoryServer) -> serde_json::Value {+ let args = QueryArgs {+ query: "karpathy".into(),+ limit: Some(3),+ project: None,+ scopes: Vec::new(),+ workspace: None,+ global: None,+ include_expired: None,+ explain: Some(true),+ };+ let result = server+ .memory_query(Parameters(args), OptionalParts(test_parts_default()))+ .await+ .unwrap();+ let text = result.content.first().and_then(|c| c.as_text()).unwrap();+ serde_json::from_str::<serde_json::Value>(&text.text).unwrap()+ }++ let baseline = paths_of(&query(&server).await);+ assert_eq!(baseline.len(), 3, "{baseline:?}");++ // Rerank promoting the LAST baseline hit to the front, and+ // scoring only two of the three (the third keeps its position).+ let promoted = baseline[2].clone();+ let demoted = baseline[0].clone();+ let table: Vec<(&'static str, f32)> = vec![+ (+ Box::leak(promoted.clone().into_boxed_str()) as &'static str,+ 0.95,+ ),+ (+ Box::leak(demoted.clone().into_boxed_str()) as &'static str,+ 0.10,+ ),+ ];+ let reranked_server =+ AiMemoryServer::new(store.reader.clone(), store.writer.clone(), ws, proj)+ .with_reranker(Arc::new(TableReranker(table)));+ let reranked = query(&reranked_server).await;+ let order = paths_of(&reranked);+ assert_eq!(order[0], promoted, "top score must lead: {order:?}");+ assert_eq!(+ order[1], demoted,+ "scored candidates sort by relevance ahead of unjudged ones: {order:?}"+ );+ assert_eq!(+ order[2], baseline[1],+ "the skipped candidate follows in RRF order: {order:?}"+ );+ assert!(+ (reranked["hits"][0]["score_details"]["rerank_score"]+ .as_f64()+ .unwrap()+ - 0.95)+ .abs()+ < 1e-6,+ "explain must surface the rerank score: {reranked}"+ );+ assert!(+ reranked["hits"][2]["score_details"]+ .get("rerank_score")+ .is_none(),+ "an unscored candidate carries no rerank_score: {reranked}"+ );++ // Thin coverage (1 of 3 scored) is treated as a degradation.+ let thin_server = AiMemoryServer::new(store.reader.clone(), store.writer.clone(), ws, proj)+ .with_reranker(Arc::new(TableReranker(vec![(+ Box::leak(promoted.clone().into_boxed_str()) as &'static str,+ 0.99,+ )])));+ assert_eq!(+ paths_of(&query(&thin_server).await),+ baseline,+ "a reranker that scored under half the candidates must degrade to RRF order"+ );++ // A failing reranker must not fail the query.+ let failing_server =+ AiMemoryServer::new(store.reader.clone(), store.writer.clone(), ws, proj)+ .with_reranker(Arc::new(FailingReranker));+ assert_eq!(+ paths_of(&query(&failing_server).await),+ baseline,+ "a failing reranker must degrade to RRF order"+ );+ }++ /// Feedback moves the page's salience, drives the decay formula, and+ /// routes stale/wrong signals to lint — without deleting anything.+ #[tokio::test]+ async fn memory_feedback_moves_salience_and_flags_for_lint() {+ let (_tmp, store, server, ws, proj) = setup_server().await;+ let feedback = |signal: &str, reason: Option<&str>| FeedbackArgs {+ path: "foo.md".into(),+ signal: signal.into(),+ reason: reason.map(str::to_string),+ project: None,+ workspace: None,+ };+ let call = async |args: FeedbackArgs| {+ server+ .memory_feedback(Parameters(args), OptionalParts(test_parts_default()))+ .await+ };+ let json_of = |result: &CallToolResult| -> serde_json::Value {+ let text = result.content.first().and_then(|c| c.as_text()).unwrap();+ serde_json::from_str(&text.text).unwrap()+ };++ // Baseline: no feedback yet → NULL salience → decay uses the default.+ let baseline = store.reader.decay_candidates(ws, proj).await.unwrap();+ assert_eq!(baseline.len(), 1);+ assert!(baseline[0].salience.is_none());++ let helpful = json_of(&call(feedback("helpful", None)).await.unwrap());+ assert_eq!(helpful["recorded"], true);+ assert_eq!(helpful["signal"], "helpful");+ assert_eq!(helpful["routed_to_lint"], false);+ let boosted = helpful["salience"].as_f64().unwrap();+ assert!(+ boosted > ai_memory_store::DecayParams::default().salience_default,+ "helpful must raise salience, got {boosted}",+ );+ let after = store.reader.decay_candidates(ws, proj).await.unwrap();+ assert_eq!(after[0].salience, Some(boosted));++ // stale drops to the floor and flags the page for lint.+ let stale = json_of(+ &call(feedback("stale", Some("we moved off Postgres in March")))+ .await+ .unwrap(),+ );+ assert_eq!(stale["routed_to_lint"], true);+ assert!(+ (stale["salience"].as_f64().unwrap() - ai_memory_store::SALIENCE_MIN).abs() < 1e-9,+ "stale must floor salience: {stale}",+ );+ let flagged = store.reader.open_feedback_findings(ws, proj).await.unwrap();+ assert_eq!(flagged.len(), 1, "{flagged:?}");+ assert_eq!(flagged[0].path, "foo.md");+ assert_eq!(flagged[0].kind, "stale");+ assert_eq!(+ flagged[0].reason.as_deref(),+ Some("we moved off Postgres in March")+ );++ // Nothing was deleted: the page is still searchable.+ assert!(+ !store+ .reader+ .search_pages_for_project(ws, proj, "karpathy".into(), 5, None)+ .await+ .unwrap()+ .is_empty(),+ "feedback must never delete or hide a page",+ );++ // Rewriting the page retires the flag (feedback is per version).+ store+ .writer+ .upsert_page(NewPage {+ workspace_id: ws,+ project_id: proj,+ path: PagePath::new("foo.md").unwrap(),+ title: "Foo".into(),+ body: "Karpathy says compile, not retrieve. Now on SQLite.".into(),+ tier: Tier::Semantic,+ frontmatter_json: serde_json::json!({}),+ pinned: false,+ links: Vec::new(),+ author_id: None,+ expires_at: None,+ entities: Vec::new(),+ })+ .await+ .unwrap();+ assert!(+ store+ .reader+ .open_feedback_findings(ws, proj)+ .await+ .unwrap()+ .is_empty(),+ "rewriting the page must clear its findings",+ );++ // Unknown signal and unknown path both fail closed.+ assert!(call(feedback("meh", None)).await.is_err());+ assert!(+ call(FeedbackArgs {+ path: "does/not/exist.md".into(),+ signal: "helpful".into(),+ reason: None,+ project: None,+ workspace: None,+ })+ .await+ .is_err(),+ );+ }+// Issue #154: default-scoped queries union the reserved `_global`// preferences scope; explicitly scoped queries do not.#[tokio::test]@@ -4535,6 +4940,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -5066,6 +5472,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -5147,6 +5554,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -5361,6 +5769,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -5480,6 +5889,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -5497,6 +5907,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -5514,6 +5925,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -5606,6 +6018,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -5624,6 +6037,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: Some("2020-01-01T23:59:59Z".parse().unwrap()),+ entities: Vec::new(),}).await.unwrap();@@ -5739,6 +6153,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -6023,6 +6438,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();diff --git a/crates/ai-memory-mcp/tests/admin_phase3.rs b/crates/ai-memory-mcp/tests/admin_phase3.rsindex a6d30002..6c654e0e 100644--- a/crates/ai-memory-mcp/tests/admin_phase3.rs+++ b/crates/ai-memory-mcp/tests/admin_phase3.rs@@ -497,6 +497,7 @@ async fn reorg_live_graveyards_only_default_workspace_pages() {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -525,6 +526,7 @@ async fn reorg_live_graveyards_only_default_workspace_pages() {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -604,6 +606,7 @@ async fn lint_dry_run_returns_lint_report_shape() {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();diff --git a/crates/ai-memory-mcp/tests/admin_read_page.rs b/crates/ai-memory-mcp/tests/admin_read_page.rsindex f5067d4d..b41dabd4 100644--- a/crates/ai-memory-mcp/tests/admin_read_page.rs+++ b/crates/ai-memory-mcp/tests/admin_read_page.rs@@ -88,6 +88,7 @@ async fn read_page_falls_back_to_db_when_file_missing() {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();diff --git a/crates/ai-memory-mcp/tests/admin_status_search.rs b/crates/ai-memory-mcp/tests/admin_status_search.rsindex 816b0b53..bf3fbb32 100644--- a/crates/ai-memory-mcp/tests/admin_status_search.rs+++ b/crates/ai-memory-mcp/tests/admin_status_search.rs@@ -62,6 +62,7 @@ async fn seed_page(store: &Store, title: &str, path: &str, body: &str) {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),};store.writer.upsert_page(page).await.unwrap();}@@ -152,6 +153,7 @@ async fn list_projects_returns_workspace_project_pairs() {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();diff --git a/crates/ai-memory-mcp/tests/autoscope_multiuser.rs b/crates/ai-memory-mcp/tests/autoscope_multiuser.rsindex bafe9489..8fa9e59e 100644--- a/crates/ai-memory-mcp/tests/autoscope_multiuser.rs+++ b/crates/ai-memory-mcp/tests/autoscope_multiuser.rs@@ -114,6 +114,7 @@ impl MultiUserHarness {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.expect("seed page");diff --git a/crates/ai-memory-mcp/tests/autoscope_stress.rs b/crates/ai-memory-mcp/tests/autoscope_stress.rsindex 3ae0874b..b35bd7ab 100644--- a/crates/ai-memory-mcp/tests/autoscope_stress.rs+++ b/crates/ai-memory-mcp/tests/autoscope_stress.rs@@ -122,6 +122,7 @@ impl Harness {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.expect("seed page");diff --git a/crates/ai-memory-store/migrations/V38__entities.sql b/crates/ai-memory-store/migrations/V38__entities.sqlnew file mode 100644index 00000000..fce00b8f--- /dev/null+++ b/crates/ai-memory-store/migrations/V38__entities.sql@@ -0,0 +1,41 @@+-- Entity index: the noun layer of the wiki, used as a fourth retrieval+-- stream alongside FTS5, vector cosine, and page-link neighbours.+--+-- Entities come from the frontmatter `entities:` list that the+-- consolidator writes (extraction happens where an LLM already runs, so+-- the zero-LLM default path is untouched — an empty table simply+-- contributes nothing, exactly like the vector stream with no embedder).+-- Markdown stays the source of truth: `reindex` rebuilds both tables+-- from the files, same contract as `links`.+--+-- Query-time matching is lexical (exact + prefix over `name`), never an+-- LLM call.+--+-- Deliberately a plain noun table, not a triple store: `entities.id` is+-- the stable anchor a future temporal-triples table+-- (`subject_entity_id`, `predicate`, `object_entity_id`, `valid_from`,+-- `valid_to`, `source_page_id`) can reference. That P2 item stays+-- deferred — this builds only the foundation it would need, and no+-- graph database is involved either way.+CREATE TABLE entities (+ id BLOB PRIMARY KEY,+ workspace_id BLOB NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,+ project_id BLOB NOT NULL REFERENCES projects(id) ON DELETE CASCADE,+ -- Lowercase-normalized surface form; uniqueness is per project so+ -- two projects can each have their own `postgres` entity.+ name TEXT NOT NULL,+ created_at INTEGER NOT NULL,+ UNIQUE (workspace_id, project_id, name)+);++CREATE TABLE entity_page_links (+ entity_id BLOB NOT NULL REFERENCES entities(id) ON DELETE CASCADE,+ page_id BLOB NOT NULL REFERENCES pages(id) ON DELETE CASCADE,+ PRIMARY KEY (entity_id, page_id)+);++-- Reverse lookup for the retrieval stream (page → entities) and for+-- replacing a page's entity set on rewrite.+CREATE INDEX idx_entity_page_links_page ON entity_page_links(page_id);+-- Prefix matching (`name LIKE 'postg%'`) rides this index.+CREATE INDEX idx_entities_name ON entities(workspace_id, project_id, name);diff --git a/crates/ai-memory-store/src/lib.rs b/crates/ai-memory-store/src/lib.rsindex f220dd06..86924b83 100644--- a/crates/ai-memory-store/src/lib.rs+++ b/crates/ai-memory-store/src/lib.rs@@ -35,7 +35,10 @@ pub use auto_improve::{AutoImproveTelemetryCount, FailAutoImproveProposal, NewAutoImproveProposal,RejectAutoImproveProposal, StageAutoImproveRun, StagedAutoImproveRun, artifact_path_for,};-pub use decay::{DecayParams, retention_score};+pub use decay::{+ DecayParams, SALIENCE_MAX, SALIENCE_MIN, SALIENCE_STEP, retention_score,+ salience_after_feedback,+};pub use error::{StoreError, StoreResult};pub use maintenance::MaintenanceJob;pub use ops::{@@ -146,6 +149,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}}@@ -3835,6 +3839,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),};store.writer.upsert_page(page).await.unwrap();@@ -5386,4 +5391,169 @@ mod tests {"active");}+ #[tokio::test]+ async fn entity_stream_finds_and_weights_pages() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let proj = store+ .writer+ .get_or_create_project(ws, "entities", None)+ .await+ .unwrap();++ let with_entities = |path: &str, body: &str, entities: Vec<&str>| {+ let mut page = sample_page(ws, proj, path, body);+ page.entities = entities.into_iter().map(str::to_string).collect();+ page+ };+ // `turbopuffer` is rare (one page); `sqlite` is common (three).+ store+ .writer+ .upsert_page(with_entities(+ "rare.md",+ "no query words in this body at all",+ vec!["turbopuffer", "sqlite"],+ ))+ .await+ .unwrap();+ for path in ["common1.md", "common2.md"] {+ store+ .writer+ .upsert_page(with_entities(path, "unrelated prose", vec!["sqlite"]))+ .await+ .unwrap();+ }++ // A query naming the entity finds the page whose body lacks the term.+ let hits = store+ .reader+ .entity_hits_for_project(ws, proj, "how do we use Turbopuffer?", 10, None)+ .await+ .unwrap();+ assert_eq!(hits.len(), 1, "{hits:?}");+ assert_eq!(hits[0].hit.path.as_str(), "rare.md");+ assert_eq!(hits[0].matched, vec!["turbopuffer".to_string()]);++ // Inverse frequency: the page carrying the rare entity outranks+ // pages carrying only the common one.+ let hits = store+ .reader+ .entity_hits_for_project(ws, proj, "turbopuffer sqlite", 10, None)+ .await+ .unwrap();+ assert_eq!(hits.len(), 3, "{hits:?}");+ assert_eq!(hits[0].hit.path.as_str(), "rare.md");+ assert!(+ hits[0].weight > hits[1].weight,+ "rare entity must weigh more: {:?}",+ hits.iter()+ .map(|h| (h.hit.path.as_str(), h.weight))+ .collect::<Vec<_>>(),+ );++ // Prefix matching, and short tokens are ignored as noise.+ assert_eq!(+ store+ .reader+ .entity_hits_for_project(ws, proj, "turbopuff", 10, None)+ .await+ .unwrap()+ .len(),+ 1,+ "prefix should match the entity",+ );+ assert!(+ store+ .reader+ .entity_hits_for_project(ws, proj, "a of", 10, None)+ .await+ .unwrap()+ .is_empty(),+ "sub-3-char tokens must not match",+ );++ // Hybrid search surfaces the entity-only hit and explains it.+ let explained = store+ .reader+ .hybrid_search_explained(+ ws,+ proj,+ "turbopuffer".into(),+ None,+ String::new(),+ String::new(),+ 0,+ 10,+ None,+ )+ .await+ .unwrap();+ let (hit, explain) = explained+ .iter()+ .find(|(h, _)| h.path.as_str() == "rare.md")+ .expect("entity stream must feed hybrid search");+ assert_eq!(explain.entity_rank, Some(1));+ assert_eq!(explain.matched_entities, vec!["turbopuffer".to_string()]);+ assert!(explain.rrf.entity > 0.0);+ assert!(+ explain.fts_rank.is_none(),+ "the body has no query term, so FTS must miss it",+ );+ // `rank` is the fused score after the bounded authority multiplier,+ // so it tracks `fused` only up to that factor — which is exactly why+ // the explain reports the factor alongside it.+ let authority = explain.authority.unwrap_or(1.0);+ assert!((hit.rank + explain.fused * authority).abs() < 1e-12);++ // Rewriting the page replaces its entity set.+ store+ .writer+ .upsert_page(with_entities("rare.md", "rewritten body", vec!["lancedb"]))+ .await+ .unwrap();+ assert!(+ store+ .reader+ .entity_hits_for_project(ws, proj, "turbopuffer", 10, None)+ .await+ .unwrap()+ .is_empty(),+ "a rewrite must drop the old entity links",+ );+ assert_eq!(+ store+ .reader+ .entity_hits_for_project(ws, proj, "lancedb", 10, None)+ .await+ .unwrap()+ .len(),+ 1,+ );++ // A project with no declared entities contributes nothing.+ let bare = store+ .writer+ .get_or_create_project(ws, "bare", None)+ .await+ .unwrap();+ store+ .writer+ .upsert_page(sample_page(ws, bare, "plain.md", "sqlite and turbopuffer"))+ .await+ .unwrap();+ assert!(+ store+ .reader+ .entity_hits_for_project(ws, bare, "turbopuffer", 10, None)+ .await+ .unwrap()+ .is_empty(),+ "no entities declared → the stream stays silent",+ );+ }}diff --git a/crates/ai-memory-store/src/ops.rs b/crates/ai-memory-store/src/ops.rsindex 7ced4d17..36e221ca 100644--- a/crates/ai-memory-store/src/ops.rs+++ b/crates/ai-memory-store/src/ops.rs@@ -588,6 +588,7 @@ pub(crate) fn upsert_page_in_tx(],)?;replace_links_in_tx(tx, &new_id, page)?;+ replace_entities_in_tx(tx, &new_id, page)?;refresh_incoming_links_for_path(tx, page, &new_id)?;audit(tx,@@ -626,6 +627,7 @@ pub(crate) fn upsert_page_in_tx(],)?;replace_links_in_tx(tx, &new_id, page)?;+ replace_entities_in_tx(tx, &new_id, page)?;refresh_incoming_links_for_path(tx, page, &new_id)?;audit(tx,@@ -641,6 +643,58 @@ pub(crate) fn upsert_page_in_tx(Ok(new_id)}+/// Re-point a page version's entity set (V38). Entity *rows* are shared+/// per project and intentionally survive: a page dropping its last+/// mention of `postgres` should not invalidate every other page's link+/// to the same noun, and the orphan row costs one short string. The+/// links are what get replaced.+fn replace_entities_in_tx(+ tx: &rusqlite::Transaction<'_>,+ page_id: &PageId,+ page: &NewPage,+) -> StoreResult<()> {+ tx.execute(+ "DELETE FROM entity_page_links WHERE page_id = ?1",+ params![page_id.as_bytes()],+ )?;+ if page.entities.is_empty() {+ return Ok(());+ }+ let now = Timestamp::now().as_microsecond();+ // Defence in depth: the wiki layer normalises on the way in, but the+ // store is the last gate before the UNIQUE(name) constraint.+ for name in ai_memory_core::normalize_entities(&page.entities) {+ tx.execute(+ "INSERT INTO entities (id, workspace_id, project_id, name, created_at) \+ VALUES (?1, ?2, ?3, ?4, ?5) \+ ON CONFLICT (workspace_id, project_id, name) DO NOTHING",+ params![+ ai_memory_core::PageId::new().as_bytes(),+ page.workspace_id.as_bytes(),+ page.project_id.as_bytes(),+ name,+ now,+ ],+ )?;+ let entity_id: Vec<u8> = tx.query_row(+ "SELECT id FROM entities \+ WHERE workspace_id = ?1 AND project_id = ?2 AND name = ?3",+ params![+ page.workspace_id.as_bytes(),+ page.project_id.as_bytes(),+ name+ ],+ |row| row.get(0),+ )?;+ tx.execute(+ "INSERT INTO entity_page_links (entity_id, page_id) VALUES (?1, ?2) \+ ON CONFLICT (entity_id, page_id) DO NOTHING",+ params![entity_id, page_id.as_bytes()],+ )?;+ }+ Ok(())+}+fn replace_links_in_tx(tx: &rusqlite::Transaction<'_>,from_page_id: &PageId,@@ -2620,6 +2674,7 @@ pub(crate) mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}}diff --git a/crates/ai-memory-store/src/reader.rs b/crates/ai-memory-store/src/reader.rsindex a89acf2a..f3634565 100644--- a/crates/ai-memory-store/src/reader.rs+++ b/crates/ai-memory-store/src/reader.rs@@ -236,6 +236,74 @@ pub struct FeedbackFinding {pub reason: Option<String>,}+/// A page matched by the entity stream, with the inverse-frequency+/// weight that ranked it and the entity names that matched.+#[derive(Debug, Clone)]+pub struct EntityHit {+ /// The matched page.+ pub hit: PageHit,+ /// Sum of `1 / pages_carrying_entity` over the matched entities.+ pub weight: f64,+ /// Entity names that matched the query.+ pub matched: Vec<String>,+}++/// Escape a literal for use inside a SQL `LIKE` pattern with+/// `ESCAPE '\'`. Entity tokens legitimately contain `_`, which `LIKE`+/// would otherwise treat as "any single character" — so `foo_bar` would+/// match `fooxbar`.+fn like_escape(literal: &str) -> String {+ literal+ .replace('\\', "\\\\")+ .replace('%', "\\%")+ .replace('_', "\\_")+}++/// Split a query into entity-match tokens: lowercase, alphanumeric-ish+/// runs, 3+ characters, de-duplicated, bounded. Short tokens are+/// dropped because a 1-2 character prefix match is noise, not a signal.+///+/// A hyphenated/underscored run yields BOTH the whole token and its+/// parts — the same both-forms trick [`crate::ops::path_search_text`]+/// uses on paths, and for the same reason: an entity may be written+/// `ai-memory`, `ai_memory`, or `ai memory`, and the query may pick+/// any of them.+fn entity_query_tokens(query: &str) -> Vec<String> {+ /// Beyond this the query is prose, not a set of nouns; the FTS+ /// stream already handles prose better than prefix matching does.+ const MAX_TOKENS: usize = 12;+ const MIN_TOKEN_CHARS: usize = 3;+ let mut seen = std::collections::BTreeSet::new();+ let mut out = Vec::new();+ let mut push = |token: String| -> bool {+ if token.chars().count() < MIN_TOKEN_CHARS {+ return true;+ }+ if seen.insert(token.clone()) {+ out.push(token);+ }+ out.len() < MAX_TOKENS+ };+ 'outer: for raw in query.split(|c: char| !(c.is_alphanumeric() || c == '-' || c == '_')) {+ let token = raw.trim_matches(['-', '_']).to_lowercase();+ if token.is_empty() {+ continue;+ }+ let compound = token.contains(['-', '_']);+ if !push(token.clone()) {+ break;+ }+ if compound {+ for part in token.split(['-', '_']) {+ if !push(part.to_string()) {+ break 'outer;+ }+ }+ }+ }+ out+}+/// A graph-expansion neighbour with provenance (which seed and which/// link direction produced it). Internal to hybrid search + explain.struct GraphNeighbor {@@ -266,6 +334,8 @@ pub struct RrfContributions {pub vector: f64,/// Graph-neighbour stream contribution.pub graph: f64,+ /// Entity-match stream contribution.+ pub entity: f64,}/// Score transparency for one `memory_query` hit: where the hit ranked@@ -295,6 +365,12 @@ pub struct SearchExplain {/// Which seed page and link direction pulled the hit in.#[serde(skip_serializing_if = "Option::is_none")]pub graph_via: Option<GraphVia>,+ /// 1-based rank in the entity-match stream.+ #[serde(skip_serializing_if = "Option::is_none")]+ pub entity_rank: Option<usize>,+ /// Entity names on this page that matched the query.+ #[serde(skip_serializing_if = "Vec::is_empty")]+ pub matched_entities: Vec<String>,/// Per-stream RRF contributions.pub rrf: RrfContributions,/// Total fused score (sum of the RRF contributions) — higher is@@ -2118,6 +2194,129 @@ impl ReaderPool {.await}+ /// Rank pages by how many of the query's tokens match their indexed+ /// entities, weighting each match by inverse entity frequency — a+ /// query token matching an entity that appears on 2 pages is a much+ /// stronger signal than one matching an entity on 200.+ ///+ /// Matching is lexical (exact name, or entity starting with the+ /// token) so this stays a plain SQL stream with no LLM call. Empty+ /// result when the project has no entities indexed, which is the+ /// zero-LLM default: entities only exist where consolidation ran.+ ///+ /// `expiry_cutoff_us`: see [`Self::search_pages_for_project`].+ ///+ /// # Errors+ /// Propagates any SQL or pool error.+ pub async fn entity_hits_for_project(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ query: &str,+ limit: usize,+ expiry_cutoff_us: Option<i64>,+ ) -> StoreResult<Vec<EntityHit>> {+ let tokens = entity_query_tokens(query);+ if tokens.is_empty() || limit == 0 {+ return Ok(Vec::new());+ }+ let cutoff = expiry_cutoff_us.unwrap_or_else(now_us);+ self.with_conn(move |conn| {+ let mut placeholders = String::with_capacity(tokens.len() * 48);+ let mut sql_params: Vec<Value> = Vec::with_capacity(tokens.len() * 3 + 4);+ for (idx, token) in tokens.iter().enumerate() {+ if idx > 0 {+ placeholders.push_str(" OR ");+ }+ // Three ways a token can match an entity name, in+ // increasing looseness: exact; the name starts with the+ // token (`postgres` → `postgresql`); or a *word* inside a+ // multi-word name starts with the token (`jetstream` →+ // `nats jetstream`). The last one is why this is a scan+ // rather than a pure index seek — acceptable because the+ // table is bounded at 10 entities per page.+ //+ // `?` is positional-by-order, same as the graph query.+ placeholders.push_str(+ "(e.name = ? \+ OR e.name LIKE ? || '%' ESCAPE '\\' \+ OR e.name LIKE '% ' || ? || '%' ESCAPE '\\')",+ );+ sql_params.push(Value::Text(token.clone()));+ let escaped = like_escape(token);+ sql_params.push(Value::Text(escaped.clone()));+ sql_params.push(Value::Text(escaped));+ }+ sql_params.push(Value::Blob(workspace_id.as_bytes().to_vec()));+ sql_params.push(Value::Blob(project_id.as_bytes().to_vec()));+ sql_params.push(Value::Integer(cutoff));+ #[allow(clippy::cast_possible_wrap)]+ sql_params.push(Value::Integer(limit as i64));++ let mut sql = String::with_capacity(placeholders.len() + 1_200);+ write!(+ &mut sql,+ // `freq` is how many current pages in this project carry+ // the entity; 1/freq is the inverse-frequency weight.+ "WITH matched AS ( \+ SELECT e.id AS entity_id, e.name AS name \+ FROM entities e \+ WHERE ({placeholders}) \+ ), \+ freq AS ( \+ SELECT m.entity_id, m.name, COUNT(*) AS pages \+ FROM matched m \+ JOIN entity_page_links l ON l.entity_id = m.entity_id \+ JOIN pages p ON p.id = l.page_id AND p.is_latest = 1 \+ GROUP BY m.entity_id, m.name \+ ) \+ SELECT pg.id, pg.path, pg.title, substr(pg.body, 1, 240) AS snippet, \+ SUM(1.0 / f.pages) AS weight, \+ COUNT(*) AS matches, \+ group_concat(f.name, ', ') AS names \+ FROM freq f \+ JOIN entity_page_links l ON l.entity_id = f.entity_id \+ JOIN pages pg ON pg.id = l.page_id \+ WHERE pg.workspace_id = ? AND pg.project_id = ? AND pg.is_latest = 1{not_expired} \+ GROUP BY pg.id, pg.path, pg.title \+ ORDER BY weight DESC, matches DESC, pg.path ASC \+ LIMIT ?",+ not_expired = not_expired("pg", "?"),+ )+ .expect("writing SQL into String cannot fail");++ let mut stmt = conn.prepare(&sql)?;+ let rows = stmt.query_map(params_from_iter(sql_params.iter()), |row| {+ let id_bytes: Vec<u8> = row.get(0)?;+ let path: String = row.get(1)?;+ let title: String = row.get(2)?;+ let snippet: String = row.get(3)?;+ let weight: f64 = row.get(4)?;+ let names: Option<String> = row.get(6)?;+ Ok((id_bytes, path, title, snippet, weight, names))+ })?;+ let mut out = Vec::new();+ for row in rows {+ let (id_bytes, path, title, snippet, weight, names) = row?;+ out.push(EntityHit {+ hit: PageHit {+ id: PageId::from_slice(&id_bytes)?,+ path: PagePath::new(path)?,+ title,+ snippet,+ rank: 0.0,+ },+ weight,+ matched: names+ .map(|n| n.split(", ").map(str::to_string).collect())+ .unwrap_or_default(),+ });+ }+ Ok(out)+ })+ .await+ }+/// Return `stale` / `wrong` feedback still attached to a *current*/// page version in one project, grouped by (page, kind) — the input/// for the lint pass's `feedback_flagged` findings. Rewriting a@@ -2175,6 +2374,9 @@ impl ReaderPool {.await}+ /// Return pages linked to or from the seed pages, scoped to latest pages+ /// in the same project.+ ////// `expiry_cutoff_us`: see [`Self::search_pages_for_project`].////// # Errors@@ -2366,13 +2568,14 @@ impl ReaderPool {/// Hybrid search: RRF-fuse FTS5 results with cosine-similarity/// over the stored embeddings of the matching `(provider, model,- /// dim)`, then add link-neighbour expansion as a third RRF stream.- /// Applies the bounded page-authority adjustment after fusion and returns- /// the top-`limit` pages by adjusted fused score.+ /// dim)`, entity matches, and link-neighbour expansion — four RRF+ /// streams. Applies the bounded page-authority adjustment after+ /// fusion and returns the top-`limit` pages by adjusted fused score.////// Each stream degrades independently to contributing nothing: no- /// `query_vec` skips the vector stream, while graph expansion still- /// runs from whatever seeds the other streams produced.+ /// `query_vec` skips the vector stream, an empty `entities` table+ /// skips the entity stream, and graph expansion still runs from+ /// whatever seeds the other streams produced.////// `expiry_cutoff_us`: see [`Self::search_pages_for_project`]. The/// cutoff resolves once here so all streams agree on it.@@ -2472,6 +2675,9 @@ impl ReaderPool {// window was too narrow at small limits for a canonical page to enter// the fused pool and receive its bounded promotion.let candidate_limit = authority_candidate_limit(limit);+ // The FTS candidate call consumes the query; the entity stream+ // tokenizes the same text separately.+ let entity_query = query.clone();// Fetch FTS5 hits first.let fts_candidates = self@@ -2491,6 +2697,15 @@ impl ReaderPool {.into_iter().map(|(hit, _authority)| hit).collect();+ let entity_hits = self+ .entity_hits_for_project(+ workspace_id,+ project_id,+ &entity_query,+ candidate_limit,+ Some(cutoff),+ )+ .await?;let mut vec_hits: Vec<(PageId, PagePath, f32)> = Vec::new();if let Some(qv) = query_vec {vec_hits = self@@ -2526,6 +2741,14 @@ impl ReaderPool {}}}+ for e in &entity_hits {+ if seed_seen.insert(e.hit.id) {+ seed_ids.push(e.hit.id);+ if let Some(paths) = &mut seed_paths {+ paths.push(e.hit.path.as_str().to_string());+ }+ }+ }let graph_hits = self.graph_neighbors_for_project_explained(workspace_id,@@ -2581,6 +2804,23 @@ impl ReaderPool {details.fused += contrib;}}+ for (rank, e) in entity_hits.iter().enumerate() {+ let contrib = 1.0 / (k + (rank + 1) as f64);+ let entry = fused.entry(e.hit.id).or_insert_with(|| Fused {+ path: e.hit.path.clone(),+ title: e.hit.title.clone(),+ snippet: e.hit.snippet.clone(),+ score: 0.0,+ explain: explain.then(SearchExplain::default),+ });+ entry.score += contrib;+ if let Some(details) = &mut entry.explain {+ details.entity_rank = Some(rank + 1);+ details.matched_entities = e.matched.clone();+ details.rrf.entity = contrib;+ details.fused += contrib;+ }+ }for (rank, n) in graph_hits.iter().enumerate() {let contrib = 1.0 / (k + (rank + 1) as f64);let entry = fused.entry(n.hit.id).or_insert_with(|| Fused {@@ -5954,6 +6194,7 @@ fn open_read_only(path: &Path) -> StoreResult<Connection> {#[cfg(test)]mod tests {+ use super::{entity_query_tokens, like_escape};use crate::Store;use ai_memory_core::{@@ -6535,4 +6776,43 @@ mod tests {assert_eq!(matched, None, "legacy trailing separator must not match");}++ #[test]+ fn entity_query_tokens_lowercases_filters_short_and_bounds() {+ // A compound token yields the whole form AND its parts, so a+ // query can spell an entity `writer-actor` or `writer actor`.+ assert_eq!(+ entity_query_tokens("How does the Writer-Actor use SQLite?"),+ vec![+ "how".to_string(),+ "does".to_string(),+ "the".to_string(),+ "writer-actor".to_string(),+ "writer".to_string(),+ "actor".to_string(),+ "use".to_string(),+ "sqlite".to_string(),+ ],+ );+ assert!(+ entity_query_tokens("a of an X").is_empty(),+ "sub-3-char tokens are noise for prefix matching",+ );+ assert_eq!(+ entity_query_tokens("dup dup dup"),+ vec!["dup".to_string()],+ "tokens de-duplicate",+ );+ assert!(entity_query_tokens(&"word ".repeat(50)).len() <= 12);+ }++ #[test]+ fn like_escape_neutralises_wildcards() {+ // `_` is LIKE's single-char wildcard and appears in real entity+ // names (`writer_actor`), so it must not match `writerxactor`.+ assert_eq!(like_escape("writer_actor"), "writer\\_actor");+ assert_eq!(like_escape("100%"), "100\\%");+ assert_eq!(like_escape("back\\slash"), "back\\\\slash");+ assert_eq!(like_escape("plain-name"), "plain-name");+ }}diff --git a/crates/ai-memory-store/src/writer.rs b/crates/ai-memory-store/src/writer.rsindex e62f7415..9a91f534 100644--- a/crates/ai-memory-store/src/writer.rs+++ b/crates/ai-memory-store/src/writer.rs@@ -2044,6 +2044,7 @@ mod tests {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}}diff --git a/crates/ai-memory-web/tests/routes.rs b/crates/ai-memory-web/tests/routes.rsindex 7154f022..ba6cee7a 100644--- a/crates/ai-memory-web/tests/routes.rs+++ b/crates/ai-memory-web/tests/routes.rs@@ -39,6 +39,7 @@ fn new_page(links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}}@@ -623,6 +624,7 @@ async fn api_pages_derives_kind_from_path_when_frontmatter_absent() {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();@@ -643,6 +645,7 @@ async fn api_pages_derives_kind_from_path_when_frontmatter_absent() {links: Vec::new(),author_id: None,expires_at: None,+ entities: Vec::new(),}).await.unwrap();diff --git a/crates/ai-memory-wiki/src/wiki.rs b/crates/ai-memory-wiki/src/wiki.rsindex e21dca65..1f0b2910 100644--- a/crates/ai-memory-wiki/src/wiki.rs+++ b/crates/ai-memory-wiki/src/wiki.rs@@ -409,6 +409,7 @@ impl Wiki {links,author_id: None,expires_at: meta.expires_at,+ entities: meta.entities,}).await?;Ok(id)@@ -843,8 +844,9 @@ impl Wiki {scrub_frontmatter_strings(&mut markdown.frontmatter, &self.sanitizer);let title = self.sanitizer.scrub(&detail.summary.title);let links = extract_links(&markdown.body, &path);- let emitted = emit(&markdown)?;let expires_at = parse_expires_at(&path, &markdown.frontmatter)?;+ let entities = parse_entities(&path, &markdown.frontmatter)?;+ let emitted = emit(&markdown)?;let page = NewPage {workspace_id,project_id,@@ -857,6 +859,7 @@ impl Wiki {links,author_id,expires_at,+ entities,};let result = {@@ -949,8 +952,9 @@ impl Wiki {frontmatter_json: md.frontmatter,pinned: meta.pinned,links,- expires_at: meta.expires_at,author_id: None,+ expires_at: meta.expires_at,+ entities: meta.entities,}).await?;Ok(id)@@ -1228,6 +1232,7 @@ impl Wiki {links: extract_links(&req.body, &req.path),author_id: req.author_id,expires_at: parse_expires_at(&req.path, &req.frontmatter)?,+ entities: parse_entities(&req.path, &req.frontmatter)?,})}).collect::<WikiResult<Vec<_>>>()?;@@ -1353,6 +1358,7 @@ impl Wiki {.unwrap_or_else(|| derive_title(&markdown.frontmatter, &markdown.body, &path));let links = extract_links(&markdown.body, &path);let expires_at = parse_expires_at(&path, &markdown.frontmatter)?;+ let entities = parse_entities(&path, &markdown.frontmatter)?;let Markdown {frontmatter: final_frontmatter,@@ -1389,6 +1395,7 @@ impl Wiki {links,author_id,expires_at,+ entities,}).await{@@ -1517,10 +1524,12 @@ fn derive_index_metadata(.and_then(|v| v.as_bool()).unwrap_or(false);let expires_at = parse_expires_at(path, frontmatter)?;+ let entities = parse_entities(path, frontmatter)?;Ok(IndexMetadata {tier,pinned,expires_at,+ entities,})}@@ -1530,14 +1539,39 @@ struct IndexMetadata {tier: Tier,pinned: bool,expires_at: Option<jiff::Timestamp>,+ entities: Vec<String>,}-/// Parse the frontmatter `expires_at:` key into a TTL instant.+/// Parse the optional frontmatter `entities:` list — salient nouns the+/// consolidator extracted, indexed by the store as a retrieval stream.///-/// Accepts RFC3339 (`2026-08-01T12:00:00Z`) or a bare `YYYY-MM-DD`,-/// which means the *end* of that day in UTC — a date-only TTL that-/// expired at midnight would surprise anyone who wrote "expires-/// 2026-08-01" meaning "good through the 1st".+/// Unlike `expires_at`, malformed entries are *dropped* rather than+/// rejected: entities are a soft ranking signal, and refusing a whole+/// page write because one hand-edited entity is 80 characters long+/// would trade a real page for a marginal signal. A non-array value is+/// still an error — that's a structural mistake, not a bad item.+pub(crate) fn parse_entities(+ path: &PagePath,+ frontmatter: &serde_json::Value,+) -> WikiResult<Vec<String>> {+ let raw = match frontmatter.get("entities") {+ None | Some(serde_json::Value::Null) => return Ok(Vec::new()),+ Some(serde_json::Value::Array(items)) => items,+ Some(_) => {+ return Err(ai_memory_wiki_error(&format!(+ "invalid non-array entities in frontmatter for {}",+ path.as_str()+ )));+ }+ };+ let strings = raw.iter().filter_map(|v| v.as_str());+ Ok(ai_memory_core::normalize_entities(strings))+}++/// Parse the optional frontmatter `expires_at:` key. Accepts RFC3339+/// (`2026-09-01T12:00:00Z`) or a bare date (`2026-09-01` = end of that+/// day, UTC). Anything else is rejected in the same style as an+/// invalid `tier`, so a typo can't silently mean "never expires".pub(crate) fn parse_expires_at(path: &PagePath,frontmatter: &serde_json::Value,@@ -3574,4 +3608,41 @@ mod tests {"reindex must reject symlinked manifests, got {err:#}");}++ /// Frontmatter `entities:` round-trips through the write path and is+ /// normalised on the way into the index. A non-array value is a+ /// structural error; bad *items* are dropped, because entities are a+ /// ranking signal and one long hand-typed entry must not cost the page.+ #[test]+ fn parse_entities_normalises_and_rejects_non_arrays() {+ let path = PagePath::new("concepts/x.md").unwrap();++ assert!(+ parse_entities(&path, &serde_json::json!({}))+ .unwrap()+ .is_empty(),+ "absent key means no entities",+ );+ assert_eq!(+ parse_entities(+ &path,+ &serde_json::json!({"entities": ["SQLite", " sqlite ", "Writer\nActor", 42]}),+ )+ .unwrap(),+ vec!["sqlite".to_string(), "writer actor".to_string()],+ "lowercased, whitespace-collapsed, de-duplicated, non-strings dropped",+ );+ assert!(+ parse_entities(+ &path,+ &serde_json::json!({"entities": ["ok", "x".repeat(200)]}),+ )+ .unwrap()+ == vec!["ok".to_string()],+ "over-long items are dropped, not fatal",+ );+ let err = parse_entities(&path, &serde_json::json!({"entities": "sqlite"}))+ .expect_err("a string instead of a list is a structural error");+ assert!(err.to_string().contains("non-array entities"), "{err}");+ }}diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex b270f6a6..50e61edf 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -110,20 +110,26 @@ from hook paths.trust boundary and delimiters precede automatically injected handoffs,project briefs, and managed-workstream packets; current instructions andcheckout state remain authoritative.-6. `memory_query` answers via FTS5 + link-neighbour RRF; when an- embedder is configured, vector cosine over `page_embeddings` joins- the same RRF. Before final truncation, a bounded authority multiplier- adjusts the relevance score using the canonical page kind, tier,- `pinned`, and explicit positive/negative frontmatter tags. It favors- maintained rules, decisions, procedures, and gotchas in close contests- while keeping episodic, historical, lint, and test evidence searchable.- No query-intent regex or hard exclusion participates. If compiled wiki pages miss entirely in default,- explicit project, or explicit `scopes` mode, bounded raw observation- FTS returns fallback `raw_hits`; `global=true` searches compiled wiki- pages across projects only. Page hits bump `access_count` +- `last_accessed_at` - the M8 reinforcement term. That bump is throttled- to at most once per page per minute, so a burst of overlapping searches- does not flood the writer actor with redundant reinforcement writes.+6. `memory_query` answers via FTS5 + entity-match + link-neighbour RRF;+ when an embedder is configured, vector cosine over `page_embeddings`+ joins the same RRF. Each stream contributes nothing when its input is+ absent, so the zero-LLM default is FTS5 + entity + graph. Before final+ truncation, a bounded authority multiplier adjusts the relevance score+ using the canonical page kind, tier, `pinned`, and explicit+ positive/negative frontmatter tags. It favors maintained rules,+ decisions, procedures, and gotchas in close contests while keeping+ episodic, historical, lint, and test evidence searchable. No+ query-intent regex or hard exclusion participates. When+ `AI_MEMORY_RERANKER=llm` is set the adjusted candidates get a final LLM+ relevance pass (degrading to the adjusted order on error). If compiled+ wiki pages miss entirely in default, explicit project, or explicit+ `scopes` mode, bounded raw observation FTS returns fallback `raw_hits`;+ `global=true` searches compiled wiki pages across projects only. Page+ hits bump `access_count` + `last_accessed_at` - the M8 reinforcement+ term, which `memory_feedback` complements with explicit per-page+ salience. That bump is throttled to at most once per page per minute,+ so a burst of overlapping searches does not flood the writer actor+ with redundant reinforcement writes.7. The forget sweep runs on demand and on the server's `[maintenance]`schedule: pages past their frontmatter `expires_at:` TTL arehard-deleted through the wiki layer (file + rows, pin or not);@@ -250,6 +256,7 @@ separately gated Claude Code assistant/Stop excerpt remains capped at 2 KB.| `handoffs` | Typed cross-agent handoff records (open / accepted / expired). || `page_embeddings` | Optional vector rows for latest pages, with `(provider, model, dim)` denormalised so hybrid search can ignore stale vectors after an embedding config change and report missing-embedding diagnostics. || `page_feedback` | Append-only `memory_feedback` signals (`helpful` / `not_helpful` / `stale` / `wrong`) keyed by page *version*, with an optional reason. Source of truth for the derived `pages.salience`; the lint pass reads unresolved stale/wrong rows joined against `is_latest = 1`, so a rewrite retires the finding. |+| `entities`, `entity_page_links` | V38 noun index derived from the frontmatter `entities:` list the consolidator writes. Names are lowercase and unique per project; links are per page *version* and replaced on rewrite. Powers the 4th retrieval stream. `entities.id` is deliberately the stable anchor a future temporal-triples table could reference — that P2 item stays deferred. || `audit_log` | Every mutation, addressable by `at DESC`. |**Memory tiers (M8 policy):**@@ -355,8 +362,7 @@ must re-write its own routing rules into a project's `CLAUDE.md` /safe default-on learning review through the same approval/write path aspending writes, and `memory_delete_page` is the exact-path destructive pairneeded by admission-aware mirrors. `memory_handoff_cancel` is the safety valve-for mistaken handoff creation. The narrow-surface discipline still holds —-every new tool has to earn its slot — but the v1 count is 16, not 10. `memory_feedback` implements the+for mistaken handoff creation. `memory_feedback` implements the"finer-grained reinforcement beyond access counts" P2 item from`prior-art-implementation-findings.md`: it cannot ride on a read toolwithout conflating read and write semantics, and the access counter it@@ -518,6 +524,19 @@ GitHub token through `/copilot_internal/v2/token`, and calls Copilot Chat withthe `vscode-chat` integration headers. The raw GitHub token is not sent to theCopilot chat endpoint.+**Reranker env** (opt-in, requires an LLM provider):+```+AI_MEMORY_RERANKER llm (only supported value; unknown values fail at startup)+```+`memory_query` then over-fetches (`limit * 3`, capped at 30 candidates)+from the RRF stage and has the configured LLM provider score each+candidate against the query via JSON-schema structured output. Scored+candidates sort by relevance; any the model skipped follow in RRF order.+On timeout (20 s), provider error, or a response covering fewer than half+the candidates, the query answers from plain RRF order — the same+degrade-don't-fail contract as the embedder. Off by default: it puts an+LLM call on the search hot path.+**Embedder env** (opt-in):```AI_MEMORY_EMBEDDING_PROVIDER openai | voyage | google | gemini | openai-compat