Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix: expose entity ranking weight
AkitaOnRailse506db221de4
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 48b751ab..6288ad0e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -17,7 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0inverse entity frequency before RRF fusion and the existing authority andoptional LLM reranking stages. Empty entity indexes contribute no candidatesor score, and entity matching makes no LLM call. `explain: true` reports the- entity stream's rank, contribution, and matched names. (#320)+ entity stream's rank, raw inverse-frequency weight, contribution, and matched+ names. (#320)- Optional post-RRF reranking for project and explicit-scope`memory_query`, off by default. Set `AI_MEMORY_RERANKER=llm` (requires`AI_MEMORY_LLM_PROVIDER`) to over-fetch candidates, fuse scopes, anddiff --git a/README.md b/README.mdindex c399afda..ff31b6d1 100644--- a/README.md+++ b/README.md@@ -717,7 +717,8 @@ Names are lowercased, whitespace-normalized, de-duplicated, capped at 10 perpage and 64 characters each, and rebuilt from Markdown during a clean-store`ai-memory reindex`.Entity lookup is project-scoped, ignores expired pages by default, and reports-`entity_rank`, `matched_entities`, and its RRF contribution under+`entity_rank`, its raw inverse-frequency `entity_weight`, `matched_entities`,+and its RRF contribution under`memory_query(..., explain: true)`.## LLM Providersdiff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rsindex 916ade11..0afbdce1 100644--- a/crates/ai-memory-mcp/src/server.rs+++ b/crates/ai-memory-mcp/src/server.rs@@ -5124,6 +5124,7 @@ mod tests {// fourth stream contributes nothing — the same way the vector stream// stays silent without an embedder.assert!(details.get("entity_rank").is_none());+ assert!(details.get("entity_weight").is_none());assert_eq!(details["rrf"]["entity"], 0.0);let rank = value["hits"][0]["rank"].as_f64().unwrap();let fused = details["fused"].as_f64().unwrap();@@ -5219,6 +5220,12 @@ mod tests {serde_json::json!(["nats jetstream"]),"{entity_hit}");+ assert!(+ entity_hit["hits"][0]["score_details"]["entity_weight"]+ .as_f64()+ .is_some_and(|weight| weight > 0.0),+ "{entity_hit}"+ );assert_eq!(entity_hit["streams_active"],serde_json::json!(["fts", "entity", "graph"])diff --git a/crates/ai-memory-store/src/lib.rs b/crates/ai-memory-store/src/lib.rsindex 0fc3c68d..eb60ac20 100644--- a/crates/ai-memory-store/src/lib.rs+++ b/crates/ai-memory-store/src/lib.rs@@ -5526,6 +5526,7 @@ mod tests {.find(|(h, _)| h.path.as_str() == "rare.md").expect("entity stream must feed hybrid search");assert_eq!(explain.entity_rank, Some(1));+ assert!(explain.entity_weight.is_some_and(|weight| weight > 0.0));assert_eq!(explain.matched_entities, vec!["turbopuffer".to_string()]);assert!(explain.rrf.entity > 0.0);assert!(diff --git a/crates/ai-memory-store/src/reader.rs b/crates/ai-memory-store/src/reader.rsindex 029746d1..4d3fe907 100644--- a/crates/ai-memory-store/src/reader.rs+++ b/crates/ai-memory-store/src/reader.rs@@ -370,6 +370,9 @@ pub struct SearchExplain {/// 1-based rank in the entity-match stream.#[serde(skip_serializing_if = "Option::is_none")]pub entity_rank: Option<usize>,+ /// Raw inverse-frequency weight used to order the entity stream.+ #[serde(skip_serializing_if = "Option::is_none")]+ pub entity_weight: Option<f64>,/// Entity names on this page that matched the query.#[serde(skip_serializing_if = "Vec::is_empty")]pub matched_entities: Vec<String>,@@ -2819,6 +2822,7 @@ impl ReaderPool {entry.score += contrib;if let Some(details) = &mut entry.explain {details.entity_rank = Some(rank + 1);+ details.entity_weight = Some(e.weight);details.matched_entities = e.matched.clone();details.rrf.entity = contrib;details.fused += contrib;diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex b5ede7ce..c1225934 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -330,7 +330,7 @@ invariants below.| Tool | Hint | Purpose ||---|---|---|-| `memory_query` | read-only | FTS5 + entity-match + graph RRF + optional vector RRF search, followed by bounded kind/tier/pinned/tag authority adjustment and raw fallback. Bumps access counters for page hits. Defaults to the current project; default-scoped calls also union the reserved `_global` preferences scope as `global_scope_hits`; `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). With `AI_MEMORY_RERANKER=llm`, project/scopes candidate pools are fused before at most one final LLM relevance pass; query/title/snippet data is bounded and JSON-encoded, and any timeout, provider error, invalid/incomplete score set, or four-call concurrency saturation preserves the adjusted order. The distinct `global=true` FTS-only ranker and supplemental global-preference hits are not reranked. `explain=true` attaches per-hit `score_details` (per-stream ranks, matched entities, raw FTS/cosine scores, RRF contributions, graph provenance, authority multiplier, and optional rerank score) to project/scopes hits plus a top-level `streams_active` list. The global FTS-only ranker reports its active stream without per-hit details. `include_expired=true` also returns TTL-expired pages. |+| `memory_query` | read-only | FTS5 + entity-match + graph RRF + optional vector RRF search, followed by bounded kind/tier/pinned/tag authority adjustment and raw fallback. Bumps access counters for page hits. Defaults to the current project; default-scoped calls also union the reserved `_global` preferences scope as `global_scope_hits`; `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). With `AI_MEMORY_RERANKER=llm`, project/scopes candidate pools are fused before at most one final LLM relevance pass; query/title/snippet data is bounded and JSON-encoded, and any timeout, provider error, invalid/incomplete score set, or four-call concurrency saturation preserves the adjusted order. The distinct `global=true` FTS-only ranker and supplemental global-preference hits are not reranked. `explain=true` attaches per-hit `score_details` (per-stream ranks, matched entities, raw FTS/cosine/entity inverse-frequency scores, RRF contributions, graph provenance, authority multiplier, and optional rerank score) to project/scopes hits plus a top-level `streams_active` list. The global FTS-only ranker reports its active stream without per-hit details. `include_expired=true` also returns TTL-expired pages. || `memory_recent` | read-only | Most-recently-updated `is_latest=1` pages. || `memory_read_page` | read-only | Fetch the FULL body of a single wiki page by `path` or by top FTS5 hit for a `query`; optional `workspace` + `project` targets a named sibling workspace/project. Use when an agent needs more than the 24-word snippets from `memory_query`. || `memory_status` | read-only | Counts, paths, version. |diff --git a/docs/usage.md b/docs/usage.mdindex 5e7c4878..2fbfefb0 100644--- a/docs/usage.md+++ b/docs/usage.md@@ -109,9 +109,9 @@ name prefixes, and word prefixes after spaces, hyphens, or underscores matchwithout a query-time LLM call. Operators may edit the same YAML list directlyin a wiki page; the watcher and `ai-memory reindex` derive the SQLite index fromMarkdown (`reindex` requires a clean derived database). `explain: true` exposes-`entity_rank`, `matched_entities`, and the entity RRF contribution. Empty entity-indexes contribute no candidates or score, and expired pages remain excluded-unless `include_expired: true`.+`entity_rank`, its raw inverse-frequency `entity_weight`, `matched_entities`,+and the entity RRF contribution. Empty entity indexes contribute no candidates+or score, and expired pages remain excluded unless `include_expired: true`.## Install the routing snippet and Agent Skills