Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
feat(retrieval): rank canonical sources by authority
AkitaOnRails2ffcf849a5b9
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 11390d71..c8c20846 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]### Changed+- Wiki search now applies a bounded source-authority adjustment after FTS5,+ graph, and optional vector candidate generation. Canonical rules, decisions,+ procedures, gotchas, semantic/procedural tiers, `pinned` pages, and+ `canonical` / `active` / `source-of-truth` tags win close relevance contests;+ episodic sessions, `_lint/` output, investigations, and pages tagged+ `superseded`, `historical`, `test-fixture`, or `do-not-answer-from` are+ downgraded but remain searchable. Exact session-only queries still retrieve+ their evidence, and the returned `rank` includes the bounded adjustment so+ multi-scope merging preserves the same order. (#262)- Client CLI commands now resolve their `(workspace, project)` from thenearest `.ai-memory.toml` marker, not just the lifecycle hooks. Previouslyonly the hook path read the marker, so a checkout declaringdiff --git a/README.md b/README.mdindex aa506fa8..c5f58d8a 100644--- a/README.md+++ b/README.md@@ -105,6 +105,14 @@ priors are at the [bottom](#influences-and-prior-art).`global_scope_hits`, so preferences travel with you into new projectswithout naming a magic project or paying the all-projects`global=true` fan-out. Event capture never writes there.+- **Authority-aware recall.** FTS5, graph-neighbor RRF, and optional vector RRF+ still generate candidates by relevance. Before truncation, a bounded+ adjustment favors maintained `_rules/`, `decisions/`, `procedures/`, and+ `gotchas/` pages over closely matching episodic session evidence. Tier,+ `pinned`, and explicit `canonical` / `active` / `source-of-truth` or+ `superseded` / `historical` / `test-fixture` / `do-not-answer-from` tags+ contribute without becoming absolute filters, so targeted history searches+ still find session pages.- **Karpathy-style LLM wiki.** Pages are compiled from observationsat session-end (or PreCompact; Codex can use `ai-memory finalize-session`for a manual final close), not retrieved over raw logs.@@ -713,6 +721,9 @@ also set `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `GITHUB_TOKEN` on the server.Embeddings are optional and separate from the LLM provider. Set`AI_MEMORY_EMBEDDING_PROVIDER=openai`, `voyage`, `google`, or `gemini` whenyou want vector reranking in addition to FTS5 + graph-neighbor retrieval.+Both the FTS-only and hybrid paths apply the same bounded page-authority+adjustment after candidate generation; embeddings improve relevance recall but+do not decide which source is canonical.See [`docs/install.md#llm-provider-tiers`](docs/install.md#llm-provider-tiers)for env vars and Ollama/OpenRouter/Atlas Cloud examples, and@@ -735,7 +746,8 @@ One Rust binary runs an MCP/HTTP server and owns one data directory:Hooks POST observations to the server. The server serializes writesthrough one SQLite writer, compiles session observations into markdownpages, and serves retrieval through FTS5, graph-neighbor RRF, optional-vector RRF, and bounded raw-observation fallback for non-global searches.+vector RRF, bounded source-authority adjustment, and bounded raw-observation+fallback for non-global searches.See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the data-flowdiagram, crate breakdown, schema notes, and invariants.diff --git a/crates/ai-memory-core/src/routing_skills/ai-memory-retrieval/SKILL.md b/crates/ai-memory-core/src/routing_skills/ai-memory-retrieval/SKILL.mdindex c73dd31a..7b9044d1 100644--- a/crates/ai-memory-core/src/routing_skills/ai-memory-retrieval/SKILL.md+++ b/crates/ai-memory-core/src/routing_skills/ai-memory-retrieval/SKILL.md@@ -42,6 +42,12 @@ If a current-project search is empty or thin, do not conclude the knowledge wasSearch returns snippets, not complete bodies. An empty-looking or short snippet does not prove the page is empty because the match can be outside the snippet window. Fetch the full page when the path or title looks relevant, especially for rules, procedures, decisions, and gotchas.+Search order combines relevance with a bounded source-authority adjustment.+Maintained rules, decisions, procedures, and gotchas normally beat closely+matching session evidence; explicitly historical or session-specific queries+can still return session pages because low-authority sources are downgraded,+not hidden. Do not treat `pinned` alone as proof that a page answers the query.+## Apply retrieved guidanceTreat matching pages under `_rules/`, `gotchas/`, `procedures/`, and `decisions/` as operating constraints.diff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rsindex e434bf0b..1467755b 100644--- a/crates/ai-memory-mcp/src/server.rs+++ b/crates/ai-memory-mcp/src/server.rs@@ -290,7 +290,10 @@ together only for a named sibling workspace/project).\n\full page with `memory_read_page` before acting. Treat `_rules/` as \constraints, `gotchas/` as preflight warnings, `procedures/` as \checklists, and `decisions/` as settled architecture unless the user \-explicitly asks to revisit them. Before non-trivial coding, debugging, \+explicitly asks to revisit them. Query ranking already gives those \+maintained sources a bounded advantage over closely matching session \+evidence, but does not hide historical pages or make `pinned` an \+unconditional answer. Before non-trivial coding, debugging, \deployment, release, auth, scope, migration, PR-review, or \data-preservation work, search memory for the subsystem and task type \first.\n\@@ -1111,7 +1114,8 @@ impl AiMemoryServer {by ai-memory across earlier runs. Call this BEFORE proposing \designs, BEFORE answering 'why does X work this way', and \whenever the user references prior work you don't recognise. \- FTS5 + graph RRF + (when configured) vector RRF re-ranking. \+ FTS5 + graph RRF + (when configured) vector RRF re-ranking, \+ followed by a bounded kind/tier/pinned/tag source-authority adjustment. \Returns up to `limit` pages with HTML-marked snippets and a rank \score (lower rank = better match). Only latest page versions. \If compiled wiki search misses in default/project/`scopes` mode, \diff --git a/crates/ai-memory-store/src/lib.rs b/crates/ai-memory-store/src/lib.rsindex 8036bc4f..ae40d9ee 100644--- a/crates/ai-memory-store/src/lib.rs+++ b/crates/ai-memory-store/src/lib.rs@@ -2112,6 +2112,104 @@ mod tests {);}+ #[tokio::test]+ async fn search_ranks_page_authority_without_hiding_session_evidence() {+ let tmp = TempDir::new().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let ws = store+ .writer+ .get_or_create_workspace("default")+ .await+ .unwrap();+ let proj = store+ .writer+ .get_or_create_project(ws, "ai-memory", None)+ .await+ .unwrap();++ let mut decision = sample_page(+ ws,+ proj,+ "decisions/embedding-policy.md",+ "The current decision keeps LLM embeddings optional and uses them only for reranking.",+ );+ decision.title = "LLM embedding policy".into();+ decision.pinned = true;+ decision.frontmatter_json =+ serde_json::json!({"tags": ["decision", "canonical", "active"]});+ store.writer.upsert_page(decision).await.unwrap();++ let mut session = sample_page(+ ws,+ proj,+ "sessions/2026-07-27.md",+ "What is the current decision about LLM embeddings? This session preserves historical evidence and chronicleonlytoken.",+ );+ session.title = "What is the current decision about LLM embeddings?".into();+ session.tier = Tier::Episodic;+ store.writer.upsert_page(session).await.unwrap();++ let mut rejected = sample_page(+ ws,+ proj,+ "sessions/rejected-fixture.md",+ "What is the current decision about LLM embeddings? This obsolete answer repeats LLM embeddings current decision.",+ );+ rejected.title = "Current decision about LLM embeddings".into();+ rejected.tier = Tier::Episodic;+ rejected.frontmatter_json =+ serde_json::json!({"tags": ["superseded", "do_not_answer_from"]});+ store.writer.upsert_page(rejected).await.unwrap();++ let query = "what is the current decision about LLM embeddings";+ let scoped = store+ .reader+ .search_pages_for_project(ws, proj, query.into(), 10)+ .await+ .unwrap();+ let scoped_paths: Vec<&str> = scoped.iter().map(|hit| hit.path.as_str()).collect();+ assert_eq!(+ scoped_paths[0], "decisions/embedding-policy.md",+ "authority-ranked hits: {scoped:?}"+ );+ assert_eq!(+ scoped_paths.last().copied(),+ Some("sessions/rejected-fixture.md")+ );+ assert!(scoped.windows(2).all(|pair| pair[0].rank <= pair[1].rank));++ let global = store+ .reader+ .search_pages_with_meta(query.into(), 10)+ .await+ .unwrap();+ assert_eq!(global[0].path.as_str(), "decisions/embedding-policy.md");++ let hybrid = store+ .reader+ .hybrid_search(+ ws,+ proj,+ query.into(),+ None,+ String::new(),+ String::new(),+ 0,+ 10,+ )+ .await+ .unwrap();+ assert_eq!(hybrid[0].path.as_str(), "decisions/embedding-policy.md");++ let session_evidence = store+ .reader+ .search_pages_for_project(ws, proj, "chronicleonlytoken".into(), 10)+ .await+ .unwrap();+ assert_eq!(session_evidence.len(), 1);+ assert_eq!(session_evidence[0].path.as_str(), "sessions/2026-07-27.md");+ }+/// Regression: bare `word:` in agent queries is FTS5 column syntax, not/// a literal token (`no such column: pick` / `memory`).#[tokio::test]diff --git a/crates/ai-memory-store/src/reader.rs b/crates/ai-memory-store/src/reader.rsindex ca1d986f..b0d80ae4 100644--- a/crates/ai-memory-store/src/reader.rs+++ b/crates/ai-memory-store/src/reader.rs@@ -55,6 +55,147 @@ fn page_kind_expr(path_column: &str, frontmatter_column: &str) -> String {)}+const AUTHORITY_CANDIDATE_MULTIPLIER: usize = 4;+const AUTHORITY_MAX_EXTRA_CANDIDATES: usize = 300;++#[derive(Debug, Clone)]+struct PageAuthority {+ factor: f64,+}++impl PageAuthority {+ fn from_stored(+ path: &str,+ kind: &str,+ tier: &str,+ pinned: bool,+ frontmatter_json: &str,+ ) -> Self {+ // Relevance remains primary: even the strongest maintained page gets+ // at most a 1.5x boost, while explicit low-authority metadata bottoms+ // out at 0.55x rather than excluding the page.+ let mut factor = 1.0_f64;++ factor += match kind {+ "rule" | "decision" => 0.15,+ "procedure" | "gotcha" => 0.12,+ "concept" | "slot" => 0.07,+ "session" => -0.12,+ _ => 0.0,+ };+ factor += match tier {+ "semantic" | "procedural" => 0.10,+ "episodic" => -0.08,+ "working" => -0.03,+ _ => 0.0,+ };+ if pinned {+ factor += 0.08;+ }++ if path.starts_with("_lint/") {+ factor -= 0.20;+ } else if path.starts_with("investigations/") {+ factor -= 0.05;+ }++ let tags = authority_tags(frontmatter_json);+ if tags+ .iter()+ .any(|tag| tag == "canonical" || tag == "source-of-truth")+ {+ factor += 0.20;+ }+ if tags.iter().any(|tag| tag == "active") {+ factor += 0.05;+ }++ // Negative tags are explicit curator instructions. They cap the+ // result's authority instead of merely cancelling a canonical path+ // boost, but they never remove the page from search.+ if tags+ .iter()+ .any(|tag| tag == "do-not-answer-from" || tag == "test-fixture")+ {+ factor = factor.min(0.55);+ } else if tags.iter().any(|tag| tag == "superseded") {+ factor = factor.min(0.65);+ } else if tags.iter().any(|tag| tag == "historical") {+ factor = factor.min(0.80);+ }++ Self {+ factor: factor.clamp(0.55, 1.50),+ }+ }++ fn adjust_rank(&self, rank: f64) -> f64 {+ if rank <= 0.0 {+ rank * self.factor+ } else {+ rank / self.factor+ }+ }+}++fn authority_tags(frontmatter_json: &str) -> Vec<String> {+ serde_json::from_str::<serde_json::Value>(frontmatter_json)+ .ok()+ .and_then(|frontmatter| frontmatter.get("tags").cloned())+ .and_then(|tags| tags.as_array().cloned())+ .unwrap_or_default()+ .into_iter()+ .filter_map(|tag| tag.as_str().map(normalize_authority_tag))+ .collect()+}++fn normalize_authority_tag(tag: &str) -> String {+ tag.trim().to_ascii_lowercase().replace('_', "-")+}++fn authority_candidate_limit(limit: usize) -> usize {+ limit+ .saturating_mul(AUTHORITY_CANDIDATE_MULTIPLIER)+ .min(limit.saturating_add(AUTHORITY_MAX_EXTRA_CANDIDATES))+}++fn rerank_page_hits(candidates: Vec<(PageHit, PageAuthority)>, limit: usize) -> Vec<PageHit> {+ let mut hits: Vec<PageHit> = candidates+ .into_iter()+ .map(|(mut hit, authority)| {+ hit.rank = authority.adjust_rank(hit.rank);+ hit+ })+ .collect();+ hits.sort_by(|a, b| {+ a.rank+ .partial_cmp(&b.rank)+ .unwrap_or(std::cmp::Ordering::Equal)+ });+ hits.truncate(limit);+ hits+}++fn rerank_page_hits_with_meta(+ candidates: Vec<(PageHitWithMeta, PageAuthority)>,+ limit: usize,+) -> Vec<PageHitWithMeta> {+ let mut hits: Vec<PageHitWithMeta> = candidates+ .into_iter()+ .map(|(mut hit, authority)| {+ hit.rank = authority.adjust_rank(hit.rank);+ hit+ })+ .collect();+ hits.sort_by(|a, b| {+ a.rank+ .partial_cmp(&b.rank)+ .unwrap_or(std::cmp::Ordering::Equal)+ });+ hits.truncate(limit);+ hits+}+/// One hit returned by [`ReaderPool::search_pages`].#[derive(Debug, Clone, Serialize)]pub struct PageHit {@@ -66,7 +207,7 @@ pub struct PageHit {pub title: String,/// FTS5 snippet of the body around the matched terms (HTML-marked).pub snippet: String,- /// FTS5 rank score (lower is better — closer to query terms).+ /// Relevance rank after the bounded authority adjustment (lower is better).pub rank: f64,}@@ -137,7 +278,7 @@ pub struct PageHitWithMeta {pub title: String,/// FTS5 snippet of the body around the matched terms (HTML-marked).pub snippet: String,- /// FTS5 rank score (lower is better — closer to query terms).+ /// Relevance rank after the bounded authority adjustment (lower is better).pub rank: f64,}@@ -720,56 +861,83 @@ impl ReaderPool {.await}- /// Run a full-text search against the FTS5 index and return the top- /// matches, limited to `is_latest = 1` rows.+ /// Run a full-text search against the FTS5 index, apply the bounded page+ /// authority adjustment, and return the top `is_latest = 1` matches.////// # Errors/// Propagates any SQL or pool error.pub async fn search_pages(&self, query: String, limit: usize) -> StoreResult<Vec<PageHit>> {let fts_query = normalize_fts_query(&query);- if fts_query.is_empty() {+ if fts_query.is_empty() || limit == 0 {return Ok(Vec::new());}self.with_conn(move |conn| {- let mut stmt = conn.prepare_cached(+ let kind_expr = page_kind_expr("pages.path", "pages.frontmatter_json");+ let sql = format!("SELECT pages.id, pages.path, pages.title, \snippet(pages_fts, 1, '<mark>', '</mark>', '…', 24) AS snip, \- pages_fts.rank \+ pages_fts.rank, pages.tier, pages.pinned, \+ pages.frontmatter_json, {kind_expr} AS kind \FROM pages_fts \JOIN pages ON pages.rowid = pages_fts.rowid \WHERE pages_fts MATCH ?1 AND pages.is_latest = 1 \ORDER BY pages_fts.rank \- LIMIT ?2",- )?;+ LIMIT ?2"+ );+ let mut stmt = conn.prepare(&sql)?;#[allow(clippy::cast_possible_wrap)]- let rows = stmt.query_map(params![fts_query, limit as i64], |row| {- let id_bytes: Vec<u8> = row.get(0)?;- let path: String = row.get(1)?;- let title: String = row.get(2)?;- let snippet: String = row.get(3)?;- let rank: f64 = row.get(4)?;- Ok((id_bytes, path, title, snippet, rank))- })?;+ let rows = stmt.query_map(+ params![fts_query, authority_candidate_limit(limit) as i64],+ |row| {+ let id_bytes: Vec<u8> = row.get(0)?;+ let path: String = row.get(1)?;+ let title: String = row.get(2)?;+ let snippet: String = row.get(3)?;+ let rank: f64 = row.get(4)?;+ let tier: String = row.get(5)?;+ let pinned = row.get::<_, i64>(6)? != 0;+ let frontmatter_json: String = row.get(7)?;+ let kind: String = row.get(8)?;+ Ok((+ id_bytes,+ path,+ title,+ snippet,+ rank,+ tier,+ pinned,+ frontmatter_json,+ kind,+ ))+ },+ )?;- let mut hits = Vec::new();+ let mut candidates = Vec::new();for row in rows {- let (id_bytes, path, title, snippet, rank) = row?;- hits.push(PageHit {- id: PageId::from_slice(&id_bytes)?,- path: PagePath::new(path)?,- title,- snippet,- rank,- });+ let (id_bytes, path, title, snippet, rank, tier, pinned, frontmatter_json, kind) =+ row?;+ let authority =+ PageAuthority::from_stored(&path, &kind, &tier, pinned, &frontmatter_json);+ candidates.push((+ PageHit {+ id: PageId::from_slice(&id_bytes)?,+ path: PagePath::new(path)?,+ title,+ snippet,+ rank,+ },+ authority,+ ));}- Ok(hits)+ Ok(rerank_page_hits(candidates, limit))}).await}- /// Run a global full-text search and include workspace/project names in- /// each row. This keeps the web search route to one SQLite query instead- /// of one search query plus a metadata lookup per hit.+ /// Run a global authority-adjusted full-text search and include+ /// workspace/project names in each row. This keeps the web search route+ /// to one SQLite query instead of one search query plus a metadata lookup+ /// per hit.////// # Errors/// Propagates any SQL or pool error.@@ -779,51 +947,88 @@ impl ReaderPool {limit: usize,) -> StoreResult<Vec<PageHitWithMeta>> {let fts_query = normalize_fts_query(&query);- if fts_query.is_empty() {+ if fts_query.is_empty() || limit == 0 {return Ok(Vec::new());}self.with_conn(move |conn| {- let mut stmt = conn.prepare_cached(+ let kind_expr = page_kind_expr("pages.path", "pages.frontmatter_json");+ let sql = format!("SELECT workspaces.name, projects.name, pages.path, pages.title, \snippet(pages_fts, 1, '<mark>', '</mark>', '…', 24) AS snip, \- pages_fts.rank \+ pages_fts.rank, pages.tier, pages.pinned, \+ pages.frontmatter_json, {kind_expr} AS kind \FROM pages_fts \JOIN pages ON pages.rowid = pages_fts.rowid \JOIN projects ON projects.id = pages.project_id \JOIN workspaces ON workspaces.id = pages.workspace_id \WHERE pages_fts MATCH ?1 AND pages.is_latest = 1 \ORDER BY pages_fts.rank \- LIMIT ?2",- )?;+ LIMIT ?2"+ );+ let mut stmt = conn.prepare(&sql)?;#[allow(clippy::cast_possible_wrap)]- let rows = stmt.query_map(params![fts_query, limit as i64], |row| {- let workspace_name: String = row.get(0)?;- let project_name: String = row.get(1)?;- let path: String = row.get(2)?;- let title: String = row.get(3)?;- let snippet: String = row.get(4)?;- let rank: f64 = row.get(5)?;- Ok((workspace_name, project_name, path, title, snippet, rank))- })?;+ let rows = stmt.query_map(+ params![fts_query, authority_candidate_limit(limit) as i64],+ |row| {+ let workspace_name: String = row.get(0)?;+ let project_name: String = row.get(1)?;+ let path: String = row.get(2)?;+ let title: String = row.get(3)?;+ let snippet: String = row.get(4)?;+ let rank: f64 = row.get(5)?;+ let tier: String = row.get(6)?;+ let pinned = row.get::<_, i64>(7)? != 0;+ let frontmatter_json: String = row.get(8)?;+ let kind: String = row.get(9)?;+ Ok((+ workspace_name,+ project_name,+ path,+ title,+ snippet,+ rank,+ tier,+ pinned,+ frontmatter_json,+ kind,+ ))+ },+ )?;- let mut hits = Vec::new();+ let mut candidates = Vec::new();for row in rows {- let (workspace_name, project_name, path, title, snippet, rank) = row?;- hits.push(PageHitWithMeta {+ let (workspace_name,project_name,- path: PagePath::new(path)?,+ path,title,snippet,rank,- });+ tier,+ pinned,+ frontmatter_json,+ kind,+ ) = row?;+ let authority =+ PageAuthority::from_stored(&path, &kind, &tier, pinned, &frontmatter_json);+ candidates.push((+ PageHitWithMeta {+ workspace_name,+ project_name,+ path: PagePath::new(path)?,+ title,+ snippet,+ rank,+ },+ authority,+ ));}- Ok(hits)+ Ok(rerank_page_hits_with_meta(candidates, limit))}).await}- /// Run a full-text search scoped to one project.+ /// Run an authority-adjusted full-text search scoped to one project.////// # Errors/// Propagates any SQL or pool error.@@ -834,15 +1039,38 @@ impl ReaderPool {query: String,limit: usize,) -> StoreResult<Vec<PageHit>> {+ if limit == 0 {+ return Ok(Vec::new());+ }+ let candidates = self+ .search_page_candidates_for_project(+ workspace_id,+ project_id,+ query,+ authority_candidate_limit(limit),+ )+ .await?;+ Ok(rerank_page_hits(candidates, limit))+ }++ async fn search_page_candidates_for_project(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ query: String,+ candidate_limit: usize,+ ) -> StoreResult<Vec<(PageHit, PageAuthority)>> {let fts_query = normalize_fts_query(&query);- if fts_query.is_empty() {+ if fts_query.is_empty() || candidate_limit == 0 {return Ok(Vec::new());}self.with_conn(move |conn| {- let mut stmt = conn.prepare_cached(+ let kind_expr = page_kind_expr("pages.path", "pages.frontmatter_json");+ let sql = format!("SELECT pages.id, pages.path, pages.title, \snippet(pages_fts, 1, '<mark>', '</mark>', '…', 24) AS snip, \- pages_fts.rank \+ pages_fts.rank, pages.tier, pages.pinned, \+ pages.frontmatter_json, {kind_expr} AS kind \FROM pages_fts \JOIN pages ON pages.rowid = pages_fts.rowid \WHERE pages_fts MATCH ?1 \@@ -850,15 +1078,16 @@ impl ReaderPool {AND pages.project_id = ?3 \AND pages.is_latest = 1 \ORDER BY pages_fts.rank \- LIMIT ?4",- )?;+ LIMIT ?4"+ );+ let mut stmt = conn.prepare(&sql)?;#[allow(clippy::cast_possible_wrap)]let rows = stmt.query_map(params![fts_query,workspace_id.as_bytes(),project_id.as_bytes(),- limit as i64+ candidate_limit as i64],|row| {let id_bytes: Vec<u8> = row.get(0)?;@@ -866,22 +1095,42 @@ impl ReaderPool {let title: String = row.get(2)?;let snippet: String = row.get(3)?;let rank: f64 = row.get(4)?;- Ok((id_bytes, path, title, snippet, rank))+ let tier: String = row.get(5)?;+ let pinned = row.get::<_, i64>(6)? != 0;+ let frontmatter_json: String = row.get(7)?;+ let kind: String = row.get(8)?;+ Ok((+ id_bytes,+ path,+ title,+ snippet,+ rank,+ tier,+ pinned,+ frontmatter_json,+ kind,+ ))},)?;- let mut hits = Vec::new();+ let mut candidates = Vec::new();for row in rows {- let (id_bytes, path, title, snippet, rank) = row?;- hits.push(PageHit {- id: PageId::from_slice(&id_bytes)?,- path: PagePath::new(path)?,- title,- snippet,- rank,- });+ let (id_bytes, path, title, snippet, rank, tier, pinned, frontmatter_json, kind) =+ row?;+ let authority =+ PageAuthority::from_stored(&path, &kind, &tier, pinned, &frontmatter_json);+ candidates.push((+ PageHit {+ id: PageId::from_slice(&id_bytes)?,+ path: PagePath::new(path)?,+ title,+ snippet,+ rank,+ },+ authority,+ ));}- Ok(hits)+ Ok(candidates)}).await}@@ -1799,10 +2048,68 @@ impl ReaderPool {.await}+ async fn page_authorities_for_project(+ &self,+ workspace_id: WorkspaceId,+ project_id: ProjectId,+ page_ids: Vec<PageId>,+ ) -> StoreResult<std::collections::HashMap<PageId, PageAuthority>> {+ if page_ids.is_empty() {+ return Ok(std::collections::HashMap::new());+ }+ self.with_conn(move |conn| {+ let mut values_clause = String::with_capacity(page_ids.len() * 5);+ let mut sql_params = Vec::with_capacity(page_ids.len() + 2);+ for (idx, page_id) in page_ids.iter().enumerate() {+ if idx > 0 {+ values_clause.push_str(", ");+ }+ values_clause.push_str("(?)");+ sql_params.push(Value::Blob(page_id.as_bytes().to_vec()));+ }+ sql_params.push(Value::Blob(workspace_id.as_bytes().to_vec()));+ sql_params.push(Value::Blob(project_id.as_bytes().to_vec()));++ let kind_expr = page_kind_expr("pages.path", "pages.frontmatter_json");+ let sql = format!(+ "WITH requested(id) AS (VALUES {values_clause}) \+ SELECT pages.id, pages.path, pages.tier, pages.pinned, \+ pages.frontmatter_json, {kind_expr} AS kind \+ FROM requested \+ JOIN pages ON pages.id = requested.id \+ WHERE pages.workspace_id = ? \+ AND pages.project_id = ? \+ AND pages.is_latest = 1"+ );+ let mut stmt = conn.prepare(&sql)?;+ let rows = stmt.query_map(params_from_iter(sql_params.iter()), |row| {+ let id_bytes: Vec<u8> = row.get(0)?;+ let path: String = row.get(1)?;+ let tier: String = row.get(2)?;+ let pinned = row.get::<_, i64>(3)? != 0;+ let frontmatter_json: String = row.get(4)?;+ let kind: String = row.get(5)?;+ Ok((id_bytes, path, tier, pinned, frontmatter_json, kind))+ })?;++ let mut authorities = std::collections::HashMap::new();+ for row in rows {+ let (id_bytes, path, tier, pinned, frontmatter_json, kind) = row?;+ authorities.insert(+ PageId::from_slice(&id_bytes)?,+ PageAuthority::from_stored(&path, &kind, &tier, pinned, &frontmatter_json),+ );+ }+ Ok(authorities)+ })+ .await+ }+/// Hybrid search: RRF-fuse FTS5 results with cosine-similarity/// over the stored embeddings of the matching `(provider, model,/// dim)`, then add link-neighbour expansion as a third RRF stream.- /// Returns the top-`limit` pages by fused score.+ /// Applies the bounded page-authority adjustment after fusion and returns+ /// the top-`limit` pages by adjusted fused score.////// When `query_vec` is `None`, the vector stream is skipped but graph/// expansion still runs from the FTS seeds.@@ -1824,9 +2131,17 @@ impl ReaderPool {limit: usize,) -> StoreResult<Vec<PageHit>> {// Fetch FTS5 hits first.- let fts_hits = self- .search_pages_for_project(workspace_id, project_id, query, limit * 2)- .await?;+ let fts_hits: Vec<PageHit> = self+ .search_page_candidates_for_project(+ workspace_id,+ project_id,+ query,+ limit.saturating_mul(2),+ )+ .await?+ .into_iter()+ .map(|(hit, _authority)| hit)+ .collect();let mut vec_hits: Vec<(PageId, PagePath, f32)> = Vec::new();if let Some(qv) = query_vec {vec_hits = self@@ -1906,6 +2221,18 @@ impl ReaderPool {rank: -fused_rank, // lower = better (matches FTS5 convention)}).collect();+ let authorities = self+ .page_authorities_for_project(+ workspace_id,+ project_id,+ out.iter().map(|hit| hit.id).collect(),+ )+ .await?;+ for hit in &mut out {+ if let Some(authority) = authorities.get(&hit.id) {+ hit.rank = authority.adjust_rank(hit.rank);+ }+ }out.sort_by(|a, b| {a.rank.partial_cmp(&b.rank)diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex 9d8f6eef..24e94a25 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -102,7 +102,12 @@ from hook paths.rejected candidates/rejection-buffer entries rather than wiki writes.6. `memory_query` answers via FTS5 + link-neighbour RRF; when anembedder is configured, vector cosine over `page_embeddings` joins- the same RRF. If compiled wiki pages miss entirely in default,+ the same RRF. Before final truncation, a bounded authority multiplier+ adjusts the relevance score using the canonical page kind, tier,+ `pinned`, and explicit positive/negative frontmatter tags. It favors+ maintained rules, decisions, procedures, and gotchas in close contests+ while keeping episodic, historical, lint, and test evidence searchable.+ No query-intent regex or hard exclusion participates. If compiled wiki pages miss entirely in default,explicit project, or explicit `scopes` mode, bounded raw observationFTS returns fallback `raw_hits`; `global=true` searches compiled wikipages across projects only. Page hits bump `access_count` +@@ -289,7 +294,7 @@ invariants below.| Tool | Hint | Purpose ||---|---|---|-| `memory_query` | read-only | FTS5 + graph RRF + optional vector RRF search, with raw fallback. Bumps access counters for page hits. Defaults to the current project; default-scoped calls also union the reserved `_global` preferences scope as `global_scope_hits`; `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). |+| `memory_query` | read-only | FTS5 + graph RRF + optional vector RRF search, followed by bounded kind/tier/pinned/tag authority adjustment and raw fallback. Bumps access counters for page hits. Defaults to the current project; default-scoped calls also union the reserved `_global` preferences scope as `global_scope_hits`; `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). || `memory_recent` | read-only | Most-recently-updated `is_latest=1` pages. || `memory_read_page` | read-only | Fetch the FULL body of a single wiki page by `path` or by top FTS5 hit for a `query`; optional `workspace` + `project` targets a named sibling workspace/project. Use when an agent needs more than the 24-word snippets from `memory_query`. || `memory_status` | read-only | Counts, paths, version. |diff --git a/docs/architecture-overview.svg b/docs/architecture-overview.svgindex aaabb5c5..89bcd937 100644--- a/docs/architecture-overview.svg+++ b/docs/architecture-overview.svg@@ -95,7 +95,7 @@<rect class="reader" x="555" y="535" width="345" height="110" /><text class="box-title" x="580" y="568">Read pool and retrieval</text><text class="box-text" x="580" y="594">FTS5 + graph-neighbor RRF</text>- <text class="box-text" x="580" y="615">optional vector RRF + raw fallback</text>+ <text class="box-text" x="580" y="615">vector RRF + authority + raw fallback</text><rect class="job" x="960" y="535" width="300" height="110" /><text class="box-title" x="985" y="568">Lifecycle and scheduled jobs</text>diff --git a/docs/design-decisions.md b/docs/design-decisions.mdindex 6078e27c..59ae49ab 100644--- a/docs/design-decisions.md+++ b/docs/design-decisions.md@@ -141,6 +141,16 @@ Adopt agentmemory's tier model **but** keep the surface narrow:**Implementation note:** the four tiers map to one `pages` table with a `tier` enum column + an `observations` table for bounded working/episodic projections, not four separate tables. Keeps schema migrations sane.+**Retrieval authority:** tier is also one bounded signal after relevance+candidate generation. The canonical page-kind classifier, `pinned`, and a+small built-in tag vocabulary (`canonical`, `active`, `source-of-truth`,+`superseded`, `historical`, `test-fixture`, `do-not-answer-from`) join it in a+post-fusion multiplier. This is deliberately not an independent retriever or+an absolute override: it resolves close contests between durable knowledge and+episodic evidence without hiding targeted session/history matches. `pinned`+continues to control retention and automated mutation first; its retrieval+effect alone is small.+## 8. Consolidation (the Karpathy bit)Three scheduled MCP operations:diff --git a/docs/usage.md b/docs/usage.mdindex 8080432a..bff62841 100644--- a/docs/usage.md+++ b/docs/usage.md@@ -56,7 +56,7 @@ at the managed ai-memory Agent Skills that carry detailed tool routing.| You say | Agent calls | Effect ||---|---|---|-| "Have we discussed X?" / "search memory for Y" | `memory_query` | FTS5 + graph/vector RRF over compiled wiki pages, with bounded raw-observation fallback. |+| "Have we discussed X?" / "search memory for Y" | `memory_query` | FTS5 + graph/vector RRF over compiled wiki pages, followed by bounded source-authority ranking and raw-observation fallback on a page miss. || Before proposing architecture | `memory_query` | Checks prior decisions and gotchas before suggesting designs. || "Catch me up" / "I've been away" | `memory_explore` | Prose digest whose verbosity scales with time since last activity. || "Where did we leave off?" | Existing handoff block, or `memory_handoff_accept` if no block exists | Resumes from the latest pending handoff. |@@ -75,6 +75,15 @@ full page before acting: rules are constraints, gotchas are preflight warnings,procedures are checklists, and decisions are settled architecture unless theuser explicitly asks to revisit them.+Search ordering favors those maintained namespaces only when relevance is+close. `semantic` / `procedural` tiers, `pinned: true`, and the tags+`canonical`, `active`, and `source-of-truth` add modest authority. `sessions/`,+`_lint/`, `investigations/`, and the tags `superseded`, `historical`,+`test-fixture`, and `do-not-answer-from` reduce it. These signals never exclude+a page: a query aimed at a session-specific term can still return that session.+`pinned` remains primarily a retention and automation-mutation control, not an+unconditional search override.+## Install the routing snippet and Agent SkillsFrom an agent, say:diff --git a/docs/vector-backend-policy.md b/docs/vector-backend-policy.mdindex 4b1671f5..868480d1 100644--- a/docs/vector-backend-policy.md+++ b/docs/vector-backend-policy.md@@ -11,7 +11,9 @@ pages per project. At that size, brute-force cosine is simple,inspectable, and fast enough, especially because vector retrieval is onlyone signal in the query path. `memory_query` already combines FTS5,graph-neighbor expansion, optional vector RRF, and raw observation-fallback.+fallback. A bounded page-authority adjustment runs after those candidate+streams, so semantic similarity cannot by itself declare a session page more+canonical than a maintained decision.Adding `sqlite-vec` would add operational surface before it has provenvalue:@@ -68,3 +70,4 @@ the MCP tool surface. The migration path should be additive:4. Add a safe rebuild command/path for the derived vec table.5. Switch vector candidate generation from brute-force scan to`sqlite-vec`, while preserving FTS5 + graph + vector RRF semantics.+ The existing post-fusion authority adjustment remains unchanged.