Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix: expose entity ranking weight

AkitaOnRailse506db221de4
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 48b751ab..6288ad0e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -17,7 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
inverse entity frequency before RRF fusion and the existing authority and
optional LLM reranking stages. Empty entity indexes contribute no candidates
or score, and entity matching makes no LLM call. `explain: true` reports the
- entity stream's rank, contribution, and matched names. (#320)
+ entity stream's rank, raw inverse-frequency weight, contribution, and matched
+ names. (#320)
- Optional post-RRF reranking for project and explicit-scope
`memory_query`, off by default. Set `AI_MEMORY_RERANKER=llm` (requires
`AI_MEMORY_LLM_PROVIDER`) to over-fetch candidates, fuse scopes, and
diff --git a/README.md b/README.md
index c399afda..ff31b6d1 100644
--- a/README.md
+++ b/README.md
@@ -717,7 +717,8 @@ Names are lowercased, whitespace-normalized, de-duplicated, capped at 10 per
page and 64 characters each, and rebuilt from Markdown during a clean-store
`ai-memory reindex`.
Entity lookup is project-scoped, ignores expired pages by default, and reports
-`entity_rank`, `matched_entities`, and its RRF contribution under
+`entity_rank`, its raw inverse-frequency `entity_weight`, `matched_entities`,
+and its RRF contribution under
`memory_query(..., explain: true)`.
## LLM Providers
diff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rs
index 916ade11..0afbdce1 100644
--- a/crates/ai-memory-mcp/src/server.rs
+++ b/crates/ai-memory-mcp/src/server.rs
@@ -5124,6 +5124,7 @@ mod tests {
// fourth stream contributes nothing — the same way the vector stream
// stays silent without an embedder.
assert!(details.get("entity_rank").is_none());
+ assert!(details.get("entity_weight").is_none());
assert_eq!(details["rrf"]["entity"], 0.0);
let rank = value["hits"][0]["rank"].as_f64().unwrap();
let fused = details["fused"].as_f64().unwrap();
@@ -5219,6 +5220,12 @@ mod tests {
serde_json::json!(["nats jetstream"]),
"{entity_hit}"
);
+ assert!(
+ entity_hit["hits"][0]["score_details"]["entity_weight"]
+ .as_f64()
+ .is_some_and(|weight| weight > 0.0),
+ "{entity_hit}"
+ );
assert_eq!(
entity_hit["streams_active"],
serde_json::json!(["fts", "entity", "graph"])
diff --git a/crates/ai-memory-store/src/lib.rs b/crates/ai-memory-store/src/lib.rs
index 0fc3c68d..eb60ac20 100644
--- a/crates/ai-memory-store/src/lib.rs
+++ b/crates/ai-memory-store/src/lib.rs
@@ -5526,6 +5526,7 @@ mod tests {
.find(|(h, _)| h.path.as_str() == "rare.md")
.expect("entity stream must feed hybrid search");
assert_eq!(explain.entity_rank, Some(1));
+ assert!(explain.entity_weight.is_some_and(|weight| weight > 0.0));
assert_eq!(explain.matched_entities, vec!["turbopuffer".to_string()]);
assert!(explain.rrf.entity > 0.0);
assert!(
diff --git a/crates/ai-memory-store/src/reader.rs b/crates/ai-memory-store/src/reader.rs
index 029746d1..4d3fe907 100644
--- a/crates/ai-memory-store/src/reader.rs
+++ b/crates/ai-memory-store/src/reader.rs
@@ -370,6 +370,9 @@ pub struct SearchExplain {
/// 1-based rank in the entity-match stream.
#[serde(skip_serializing_if = "Option::is_none")]
pub entity_rank: Option<usize>,
+ /// Raw inverse-frequency weight used to order the entity stream.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub entity_weight: Option<f64>,
/// Entity names on this page that matched the query.
#[serde(skip_serializing_if = "Vec::is_empty")]
pub matched_entities: Vec<String>,
@@ -2819,6 +2822,7 @@ impl ReaderPool {
entry.score += contrib;
if let Some(details) = &mut entry.explain {
details.entity_rank = Some(rank + 1);
+ details.entity_weight = Some(e.weight);
details.matched_entities = e.matched.clone();
details.rrf.entity = contrib;
details.fused += contrib;
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index b5ede7ce..c1225934 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -330,7 +330,7 @@ invariants below.
| Tool | Hint | Purpose |
|---|---|---|
-| `memory_query` | read-only | FTS5 + entity-match + graph RRF + optional vector RRF search, followed by bounded kind/tier/pinned/tag authority adjustment and raw fallback. Bumps access counters for page hits. Defaults to the current project; default-scoped calls also union the reserved `_global` preferences scope as `global_scope_hits`; `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). With `AI_MEMORY_RERANKER=llm`, project/scopes candidate pools are fused before at most one final LLM relevance pass; query/title/snippet data is bounded and JSON-encoded, and any timeout, provider error, invalid/incomplete score set, or four-call concurrency saturation preserves the adjusted order. The distinct `global=true` FTS-only ranker and supplemental global-preference hits are not reranked. `explain=true` attaches per-hit `score_details` (per-stream ranks, matched entities, raw FTS/cosine scores, RRF contributions, graph provenance, authority multiplier, and optional rerank score) to project/scopes hits plus a top-level `streams_active` list. The global FTS-only ranker reports its active stream without per-hit details. `include_expired=true` also returns TTL-expired pages. |
+| `memory_query` | read-only | FTS5 + entity-match + graph RRF + optional vector RRF search, followed by bounded kind/tier/pinned/tag authority adjustment and raw fallback. Bumps access counters for page hits. Defaults to the current project; default-scoped calls also union the reserved `_global` preferences scope as `global_scope_hits`; `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). With `AI_MEMORY_RERANKER=llm`, project/scopes candidate pools are fused before at most one final LLM relevance pass; query/title/snippet data is bounded and JSON-encoded, and any timeout, provider error, invalid/incomplete score set, or four-call concurrency saturation preserves the adjusted order. The distinct `global=true` FTS-only ranker and supplemental global-preference hits are not reranked. `explain=true` attaches per-hit `score_details` (per-stream ranks, matched entities, raw FTS/cosine/entity inverse-frequency scores, RRF contributions, graph provenance, authority multiplier, and optional rerank score) to project/scopes hits plus a top-level `streams_active` list. The global FTS-only ranker reports its active stream without per-hit details. `include_expired=true` also returns TTL-expired pages. |
| `memory_recent` | read-only | Most-recently-updated `is_latest=1` pages. |
| `memory_read_page` | read-only | Fetch the FULL body of a single wiki page by `path` or by top FTS5 hit for a `query`; optional `workspace` + `project` targets a named sibling workspace/project. Use when an agent needs more than the 24-word snippets from `memory_query`. |
| `memory_status` | read-only | Counts, paths, version. |
diff --git a/docs/usage.md b/docs/usage.md
index 5e7c4878..2fbfefb0 100644
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -109,9 +109,9 @@ name prefixes, and word prefixes after spaces, hyphens, or underscores match
without a query-time LLM call. Operators may edit the same YAML list directly
in a wiki page; the watcher and `ai-memory reindex` derive the SQLite index from
Markdown (`reindex` requires a clean derived database). `explain: true` exposes
-`entity_rank`, `matched_entities`, and the entity RRF contribution. Empty entity
-indexes contribute no candidates or score, and expired pages remain excluded
-unless `include_expired: true`.
+`entity_rank`, its raw inverse-frequency `entity_weight`, `matched_entities`,
+and the entity RRF contribution. Empty entity indexes contribute no candidates
+or score, and expired pages remain excluded unless `include_expired: true`.
## Install the routing snippet and Agent Skills