Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
refactor(retrieval): 改为全局召回并统一候选精排 (#5450)
Qin Haojie69e71e4ff589
diff --git a/docs/en/api/06-retrieval.md b/docs/en/api/06-retrieval.mdindex 71f58aa83..8f67428aa 100644--- a/docs/en/api/06-retrieval.md+++ b/docs/en/api/06-retrieval.md@@ -17,12 +17,12 @@ OpenViking provides multiple retrieval methods, including simple vector similariThe core retrieval pipeline is as follows:```-Query → Intent Analysis (search only) → Vector Search (L0) → Rerank (L1) → Results+Query → Intent Analysis (search only, optional) → Global Vector Search → Rerank (search only, optional) → Results```1. **Intent Analysis** (search only): Understand query intent, expand queries2. **Vector Search**: Find candidates using embeddings-3. **Rerank**: Re-score using content for better accuracy+3. **Rerank**: THINKING with a usable reranker reranks `2 × limit` recalled candidates once; otherwise recall is limited to `limit` hits4. **Results**: Return top-k contexts## API Reference@@ -33,14 +33,12 @@ Basic vector similarity search without session context.#### 1. API Implementation Introduction-The `find()` method performs pure vector similarity search for simple query scenarios. It uses hierarchical retrieval to search at the L0 summary level first, then matches in detail at L1/L2 levels.+The `find()` method runs one global vector similarity search in QUICK mode for simple query scenarios. It recalls `limit` candidates and supports filtering by L0/L1/L2 through `level`.**Processing Pipeline**:1. Convert query text to vector2. Perform global vector search within specified target URI-3. Use hierarchical retrieval strategy to recursively search relevant directories and files-4. Optional: Use rerank model to optimize result ordering-5. Return matched context list+3. Apply the score threshold and return matched contexts without reranking**Code Entry Points**:- `openviking_cli/client/sync_http.py:SyncHTTPClient.find()` - Python SDK entry (HTTP)@@ -374,7 +372,7 @@ The `search()` method adds session context understanding and intent analysis cap1. Load session context (if session_id is provided)2. Analyze query intent, understand actual needs combined with conversation history3. Expand queries to improve recall rate-4. Execute same hierarchical retrieval pipeline as `find()`+4. Run one global search per query; with a usable reranker, recall `2 × limit` candidates and rerank once to return at most `limit` results, otherwise recall `limit` hits directly5. Return search results with query plan**Code Entry Points**:diff --git a/docs/en/concepts/01-architecture.md b/docs/en/concepts/01-architecture.mdindex 054d7ee1f..9527a1c77 100644--- a/docs/en/concepts/01-architecture.md+++ b/docs/en/concepts/01-architecture.md@@ -55,7 +55,7 @@ OpenViking is a context database designed for AI Agents, unifying all context ty|--------|----------------|------------------|| **Client** | Unified entry | Provides all operation interfaces, delegates to Service layer || **Service** | Business logic | FSService, SearchService, SessionService, ResourceService, PackService, DebugService |-| **Retrieve** | Context retrieval | Intent analysis (IntentAnalyzer), hierarchical retrieval (HierarchicalRetriever), Rerank |+| **Retrieve** | Context retrieval | Intent analysis (IntentAnalyzer), global retrieval (HierarchicalRetriever), Rerank || **Session** | Session management | Message recording, usage tracking, session compression, memory commit || **Parse** | Context extraction | Document parsing (PDF/MD/HTML), tree building (TreeBuilder), async semantic generation || **Compressor** | Memory compression | Schema-driven memory extraction and LLM deduplication decisions |@@ -99,12 +99,12 @@ Input → Parser → TreeBuilder → AGFS → SemanticQueue → Vector Index### Retrieving Context```-Query → Intent Analysis → Hierarchical Retrieval → Rerank → Results+Query → Intent Analysis → Global Retrieval → Rerank → Results```1. **Intent Analysis**: Analyze query intent, generate 0-5 typed queries-2. **Hierarchical Retrieval**: Directory-level recursive search using priority queue-3. **Rerank**: Scalar filtering + model reranking+2. **Global Retrieval**: One vector search per query within the permitted scope+3. **Rerank**: Optional single pass over the recalled candidates4. **Results**: Return contexts sorted by relevance### Session Commitdiff --git a/docs/en/concepts/07-retrieval.md b/docs/en/concepts/07-retrieval.mdindex 8c8b804a8..58b976341 100644--- a/docs/en/concepts/07-retrieval.md+++ b/docs/en/concepts/07-retrieval.md@@ -1,13 +1,11 @@# Retrieval Mechanism-OpenViking uses two-stage retrieval: intent analysis + hierarchical retrieval + rerank.+OpenViking uses global vector retrieval with optional reranking after recall. Session intent analysis is a separate preceding stage.## Overview```-Query → Intent Analysis → Hierarchical Retrieval → Rerank → Results- ↓ ↓ ↓- TypedQuery Directory Recursion Refined Scoring+Query → Intent Analysis (optional) → Global Vector Search → Rerank (optional) → Results```## find() vs search()@@ -71,93 +69,32 @@ class TypedQuery:- **0 queries**: Chitchat, greetings that don't need retrieval- **Multiple queries**: Complex tasks may need skill + resource + memory-## Hierarchical Retrieval+## Global Retrieval-HierarchicalRetriever uses priority queue to recursively search directory structure.+`HierarchicalRetriever` runs one global vector search per query. Target directories, context type, permissions, metadata filters, and `level` constrain the search scope. Without an explicit `level`, text queries can directly match L0/L1/L2 records; retrieval does not navigate directories recursively.-### Flow+### Retrieval Modes-```-Step 1: Determine root directories by context_type- ↓-Step 2: Global vector search to locate starting directories- ↓-Step 3: Merge starting points + Rerank scoring- ↓-Step 4: Recursive search (priority queue)- ↓-Step 5: Convert to MatchedContext-```--### Root Directory Mapping+| Mode | Vector candidates | Rerank |+|------|-------------------|--------|+| QUICK | `limit` | Disabled |+| THINKING with a usable rerank configuration | `2 × limit` | One pass over the recalled candidates, returning at most `limit` results |+| THINKING without a usable rerank configuration | `limit` | Disabled |-| context_type | Root Directories |-|--------------|------------------|-| MEMORY | `viking://~/memories` |-| RESOURCE | `viking://resources` |-| SKILL | `viking://~/skills` and `viking://agent/skills` |--### Recursive Search Algorithm--```python-while dir_queue:- current_uri, parent_score = heapq.heappop(dir_queue)+`find()` uses QUICK. `search()` automatically selects THINKING when a reranker is configured, otherwise QUICK. This internal retrieval mode is separate from LLM reasoning parameters and session intent analysis. Queries generated by intent analysis continue to be retrieved and ranked separately, then aggregated as before.- # Search children- results = await search(parent_uri=current_uri)-- for r in results:- # Score propagation- final_score = score_propagation_alpha * embedding_score + (1 - score_propagation_alpha) * parent_score-- if final_score > threshold:- collected.append(r)-- if not r.is_leaf: # Directory continues recursion- heapq.heappush(dir_queue, (r.uri, final_score))-- # Convergence detection- if topk_unchanged_for_3_rounds:- break-```--### Key Parameters--| Parameter | Value | Description |-|-----------|-------|-------------|-| `retrieval.score_propagation_alpha` | 1.0 | Child-score weight in the propagation blend; `1.0` uses only the child's own score and ignores the parent score |-| `MAX_CONVERGENCE_ROUNDS` | 3 | Convergence detection rounds |-| `GLOBAL_SEARCH_TOPK` | 10 | Global search candidates |+Image queries skip text reranking and recall `limit` candidates. They default to L2 unless `level` is specified.## Rerank Strategy-Rerank refines candidate results in THINKING mode.--### Trigger Conditions--- Rerank AK/SK configured-- Using THINKING mode (default for search())-- If rerank returns an invalid result or the API call fails, retrieval falls back to vector scores--### Scoring Method--```python-if rerank_client and mode == THINKING:- scores = rerank_client.rerank_batch(query, documents)-else:- scores = [r["_score"] for r in results] # Vector scores-```--### Usage Points--1. **Starting point evaluation**: Evaluate global search candidate directories-2. **Recursive search**: Evaluate children at each level+Rerank processes only the candidates from the completed global search; it never starts another search. For example, with `limit=10`, reranking recalls the top 20 vector hits and returns at most 10 results after reranking. Without reranking, recall is limited to 10 hits.-### Backend Support+- Candidate index records supply the `abstract` text for reranking.+- Score thresholds apply after reranking, or to vector scores when reranking is disabled.+- Failed requests or invalid rerank results retain the existing fallback to vector scores.+- Directory priority queues, parent-to-child score propagation, and convergence rounds are removed.-| Backend | Model |-|---------|-------|-| Volcengine | doubao-seed-rerank |+See [rerank configuration](../guides/01-configuration.md#rerank) for supported providers and settings.## Retrieval Resultsdiff --git a/docs/en/configuration/01-server.md b/docs/en/configuration/01-server.mdindex 8aec2919d..01f4cdd03 100644--- a/docs/en/configuration/01-server.md+++ b/docs/en/configuration/01-server.md@@ -158,7 +158,6 @@ Rerank has no separate `enabled` field. It becomes available when the required p{"retrieval": {"hotness_alpha": 0,- "score_propagation_alpha": 1,"enable_intent": true}}@@ -169,7 +168,6 @@ Rerank has no separate `enabled` field. It becomes available when the required p| Field | Type / values | Default | Purpose ||---|---|---|---|| `hotness_alpha` | number, `0`–`1` | `0` | Hotness score weight; `0` disables it |-| `score_propagation_alpha` | number, `0`–`1` | `1` | Child-result score weight in hierarchical retrieval || `enable_intent` | boolean | `true` | Run intent analysis/query planning when `session_id` is present |Search and Find requests default to `limit: 10`; override the limit on each API or SDK request. `retrieval.enable_intent` controls LLM query planning for session-aware Search, while result reranking is enabled only when `rerank` has a usable provider configuration.diff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.mdindex f03c043a3..f2633d783 100644--- a/docs/en/guides/01-configuration.md+++ b/docs/en/guides/01-configuration.md@@ -1105,7 +1105,6 @@ Retrieval ranking configuration for final search scores.{"retrieval": {"hotness_alpha": 0.0,- "score_propagation_alpha": 1.0,"recall_intent_timeout_s": 5.0,"recall_rewrite_timeout_s": 30.0}@@ -1115,7 +1114,6 @@ Retrieval ranking configuration for final search scores.| Parameter | Type | Description | Default ||-----------|------|-------------|---------|| `hotness_alpha` | float | Weight for blending hotness into final retrieval scores. `0.0` disables the hotness boost and keeps scores equal to semantic similarity; `1.0` uses only hotness. Valid range: `0.0` to `1.0`. | `0.0` |-| `score_propagation_alpha` | float | Weight for each child result's own score when blending with its parent score during hierarchical retrieval. `1.0` ignores the parent score (semantic similarity only); `0.5` is an equal blend with the parent score; `0.0` uses only the parent score. Valid range: `0.0` to `1.0`. | `1.0` |Keep `hotness_alpha` at `0.0` when you need scores to reflect pure vector similarity. Set it above `0.0` only when frequently accessed or recently updated contexts should receive a ranking boost.@@ -2066,8 +2064,7 @@ For detailed encryption explanations, see [Data Encryption](../concepts/10-encry"extra_headers": {}},"retrieval": {- "hotness_alpha": 0.0,- "score_propagation_alpha": 1.0+ "hotness_alpha": 0.0},"encryption": {"enabled": false,diff --git a/docs/en/guides/10-prompt-guide.md b/docs/en/guides/10-prompt-guide.mdindex 6c131b063..fd9c7cca3 100644--- a/docs/en/guides/10-prompt-guide.md+++ b/docs/en/guides/10-prompt-guide.md@@ -344,7 +344,7 @@ These prompts are mainly used to generate file-level and directory-level summari- `semantic.overview_generation`- Effective stage: directory overview generation stage- - Affects: directory overviews, hierarchical retrieval, and navigation experience+ - Affects: directory overviews, global retrieval, and navigation experience- Purpose: generates a directory-level overview from file summaries and child directory abstracts- Key inputs: `dir_name`, `file_summaries`, `children_abstracts`, `output_language`diff --git a/docs/zh/api/06-retrieval.md b/docs/zh/api/06-retrieval.mdindex 1dbecb960..89fa70a9c 100644--- a/docs/zh/api/06-retrieval.md+++ b/docs/zh/api/06-retrieval.md@@ -17,12 +17,12 @@ OpenViking 提供多种检索方法,包括简单的向量相似度搜索、带检索的核心流程如下:```-查询 → 意图分析(仅search)→ 向量搜索(L0)→ 重排序(L1)→ 结果+查询 → 意图分析(仅 search,可选)→ 全局向量搜索 → 重排序(仅 search,可选)→ 结果```1. **意图分析**(仅 search):理解查询意图,扩展查询2. **向量搜索**:使用 Embedding 查找候选项-3. **重排序**:使用内容重新评分以提高准确性+3. **重排序**:THINKING 模式且配置了可用的 Rerank 时,对全局召回的 `2 × limit` 个候选统一精排一次;否则只召回 `limit` 条4. **结果**:返回 top-k 上下文## API 参考@@ -33,14 +33,12 @@ OpenViking 提供多种检索方法,包括简单的向量相似度搜索、带#### 1. API 实现介绍-`find()` 方法执行纯向量相似度搜索,适用于简单的查询场景。它使用分层检索器(HierarchicalRetriever)在 L0 摘要层进行初步搜索,然后在 L1/L2 层进行详细匹配。+`find()` 方法使用 QUICK 模式执行一次全局向量相似度搜索,适用于简单的查询场景。候选数为 `limit`,可通过 `level` 限定 L0/L1/L2。**处理流程**:1. 将查询文本转换为向量2. 在指定的目标 URI 范围内执行全局向量搜索-3. 使用分层检索策略递归搜索相关目录和文件-4. 可选:使用重排序模型优化结果排序-5. 返回匹配的上下文列表+3. 按分数阈值筛选,返回匹配的上下文列表;不执行 Rerank**代码入口**:- `openviking_cli/client/sync_http.py:SyncHTTPClient.find()` - Python SDK 入口(HTTP)@@ -375,7 +373,7 @@ openviking find "红色海报风格" --image ./poster.png --uri "viking://resour1. 加载会话上下文(如果提供了 session_id)2. 分析查询意图,结合对话历史理解真实需求3. 扩展查询以提高召回率-4. 执行与 `find()` 相同的分层检索流程+4. 每条查询分别执行一次全局检索;配置了可用的 Rerank 时召回 `2 × limit` 个候选,统一精排后返回最多 `limit` 条,否则直接召回 `limit` 条5. 返回带查询计划的搜索结果**代码入口**:diff --git a/docs/zh/concepts/01-architecture.md b/docs/zh/concepts/01-architecture.mdindex 34efcc3b6..d8fb19d9b 100644--- a/docs/zh/concepts/01-architecture.md+++ b/docs/zh/concepts/01-architecture.md@@ -54,7 +54,7 @@ OpenViking 是为 AI Agent 设计的上下文数据库,将所有上下文(Me|------|------|---------|| **Client** | 统一入口 | 提供所有操作接口,委托给 Service 层 || **Service** | 业务逻辑 | FSService、SearchService、SessionService、ResourceService、PackService、DebugService |-| **Retrieve** | 上下文检索 | 意图分析(IntentAnalyzer)、层级检索(HierarchicalRetriever)、Rerank 精排 |+| **Retrieve** | 上下文检索 | 意图分析(IntentAnalyzer)、全局检索(HierarchicalRetriever)、Rerank 精排 || **Session** | 会话管理 | 消息记录、使用追踪、会话压缩、记忆提交 || **Parse** | 上下文提取 | 文档解析(PDF/MD/HTML)、树构建(TreeBuilder)、异步语义生成 || **Compressor** | 记忆压缩 | Schema 驱动的记忆提取、LLM 去重决策 |@@ -98,12 +98,12 @@ OpenViking 采用双层存储架构,实现内容与索引分离(详见 [存### 检索上下文```-查询 → 意图分析 → 层级检索 → Rerank → 结果+查询 → 意图分析 → 全局检索 → Rerank → 结果```1. **意图分析**:分析查询意图,生成 0-5 个类型化查询-2. **层级检索**:目录级递归搜索,使用优先队列-3. **Rerank**:标量过滤 + 模型重排+2. **全局检索**:每条查询在权限范围内执行一次向量搜索+3. **Rerank**:可选,对召回候选统一精排一次4. **结果**:返回按相关性排序的上下文### 会话提交diff --git a/docs/zh/concepts/07-retrieval.md b/docs/zh/concepts/07-retrieval.mdindex 0d38d48b2..575ca71c7 100644--- a/docs/zh/concepts/07-retrieval.md+++ b/docs/zh/concepts/07-retrieval.md@@ -1,13 +1,11 @@# 检索机制-OpenViking 采用两阶段检索:意图分析 + 层级检索 + Rerank。+OpenViking 使用全局向量检索,并可在召回完成后对候选结果执行一次 Rerank。会话意图分析是独立的前置阶段。## 概览```-查询 → 意图分析 → 层级检索 → Rerank → 结果- ↓ ↓ ↓- TypedQuery 目录递归 精排评分+查询 → 意图分析(可选)→ 全局向量检索 → Rerank(可选)→ 结果```## find() vs search()@@ -71,93 +69,32 @@ class TypedQuery:- **0 个查询**:闲聊、问候等不需要检索的场景- **多个查询**:复杂任务可能需要技能 + 资源 + 记忆-## 层级检索+## 全局检索-HierarchicalRetriever 使用优先队列递归搜索目录结构。+`HierarchicalRetriever` 对每条查询执行一次全局向量检索。目标目录、上下文类型、权限、元数据过滤和 `level` 一起限定搜索范围。未指定 `level` 时,文本查询可直接命中 L0/L1/L2;不再先定位目录再逐层搜索。-### 流程+### 检索模式-```-Step 1: 根据 context_type 确定根目录- ↓-Step 2: 全局向量搜索定位起始目录- ↓-Step 3: 合并起始点 + Rerank 评分- ↓-Step 4: 递归搜索(优先队列)- ↓-Step 5: 转换为 MatchedContext-```--### 根目录映射+| 模式 | 向量候选数 | Rerank |+|------|------------|--------|+| QUICK | `limit` | 不执行 |+| THINKING,且配置了可用的 Rerank | `2 × limit` | 对召回候选统一执行一次,返回最多 `limit` 条 |+| THINKING,未配置可用的 Rerank | `limit` | 不执行 |-| context_type | 根目录 |-|--------------|--------|-| MEMORY | `viking://~/memories` |-| RESOURCE | `viking://resources` |-| SKILL | `viking://~/skills` 与 `viking://agent/skills` |--### 递归搜索算法--```python-while dir_queue:- current_uri, parent_score = heapq.heappop(dir_queue)+`find()` 使用 QUICK。`search()` 配置了可用的 Rerank 时自动使用 THINKING,否则使用 QUICK。这是内部检索模式,不是 LLM 的思考参数,也不控制会话意图分析。意图分析生成的多条查询仍分别检索和排序,再按现有方式汇总。- # 搜索子节点- results = await search(parent_uri=current_uri)-- for r in results:- # 分数传播- final_score = score_propagation_alpha * embedding_score + (1 - score_propagation_alpha) * parent_score-- if final_score > threshold:- collected.append(r)-- if not r.is_leaf: # 目录继续递归- heapq.heappush(dir_queue, (r.uri, final_score))-- # 收敛检测- if topk_unchanged_for_3_rounds:- break-```--### 关键参数--| 参数 | 值 | 说明 |-|------|-----|------|-| `retrieval.score_propagation_alpha` | 1.0 | 分数传播混合中子节点自身分数的权重;`1.0` 表示仅使用子节点自身分数,忽略父节点分数 |-| `MAX_CONVERGENCE_ROUNDS` | 3 | 收敛检测轮数 |-| `GLOBAL_SEARCH_TOPK` | 10 | 全局搜索候选数 |+图像查询跳过文本 Rerank,候选数为 `limit`;未指定 `level` 时默认搜索 L2。## Rerank 策略-Rerank 在 THINKING 模式下对候选结果精排。--### 触发条件--- 配置了 Rerank AK/SK-- 使用 THINKING 模式(search() 默认)-- 如果 rerank 返回无效结果或 API 调用失败,会回退到向量分数--### 评分方式--```python-if rerank_client and mode == THINKING:- scores = rerank_client.rerank_batch(query, documents)-else:- scores = [r["_score"] for r in results] # 向量分数-```--### 使用位置--1. **起始点评估**:评估全局搜索的候选目录-2. **递归搜索**:评估每层的子节点+Rerank 只处理本次全局召回的候选,不会触发下一轮检索。例如 `limit=10`,启用 Rerank 时先召回向量分数最高的 20 条,再按 Rerank 结果返回最多 10 条;未启用时直接召回 10 条。-### 后端支持+- 使用候选索引记录的 `abstract` 字段作为 Rerank 文本。+- 分数阈值在 Rerank 后应用;未启用 Rerank 时使用向量分数。+- Rerank 请求失败或返回无效结果时,保留已有的向量分数回退行为。+- 不再使用目录优先队列、父子分数传播或多轮收敛判断。-| 后端 | 模型 |-|------|------|-| Volcengine | doubao-seed-rerank |+支持的 provider 和配置见 [Rerank 配置](../guides/01-configuration.md#rerank)。## 检索结果diff --git a/docs/zh/concepts/15-acl.md b/docs/zh/concepts/15-acl.mdindex 403c91bf1..327a39f8c 100644--- a/docs/zh/concepts/15-acl.md+++ b/docs/zh/concepts/15-acl.md@@ -166,4 +166,4 @@ ov acl set viking://resources/project-a --acl-mode restricted- [ACL API](../api/12-acl.md) - HTTP、SDK 和 CLI 接口- [多租户](./11-multi-tenant.md) - account、user 和角色边界- [Viking URI](./04-viking-uri.md) - URI namespace-- [检索](./07-retrieval.md) - 分层检索流程+- [检索](./07-retrieval.md) - 全局检索流程diff --git a/docs/zh/configuration/01-server.md b/docs/zh/configuration/01-server.mdindex 8f43f42a0..d0631fe66 100644--- a/docs/zh/configuration/01-server.md+++ b/docs/zh/configuration/01-server.md@@ -158,7 +158,6 @@ Rerank 没有单独的 `enabled` 字段;配置了对应 provider 所需的凭{"retrieval": {"hotness_alpha": 0,- "score_propagation_alpha": 1,"enable_intent": true}}@@ -169,7 +168,6 @@ Rerank 没有单独的 `enabled` 字段;配置了对应 provider 所需的凭| 字段 | 类型 / 可选值 | 默认值 | 作用 ||---|---|---|---|| `hotness_alpha` | number,`0`–`1` | `0` | 热度分数权重;`0` 表示关闭热度加权 |-| `score_propagation_alpha` | number,`0`–`1` | `1` | 层级检索时子结果自身分数的权重 || `enable_intent` | boolean | `true` | 有 `session_id` 时是否进行意图分析和查询规划 |Search 和 Find 请求的默认 `limit` 为 `10`,可以在每次 API 或 SDK 请求中覆盖。`retrieval.enable_intent` 控制带 Session 的 Search 是否执行 LLM 查询规划;只有配置了可用的 `rerank` provider 时才会执行结果重排。diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.mdindex c68d2bd8f..5a292a595 100644--- a/docs/zh/guides/01-configuration.md+++ b/docs/zh/guides/01-configuration.md@@ -1068,7 +1068,6 @@ Jev 适配器将 query 和候选文档作为结构化 System One `state`,并{"retrieval": {"hotness_alpha": 0.0,- "score_propagation_alpha": 1.0,"recall_intent_timeout_s": 5.0,"recall_rewrite_timeout_s": 30.0}@@ -1078,7 +1077,6 @@ Jev 适配器将 query 和候选文档作为结构化 System One `state`,并| 参数 | 类型 | 说明 | 默认值 ||------|------|------|--------|| `hotness_alpha` | float | hotness 分数在最终召回分数中的混合权重。`0.0` 表示关闭 hotness boost,最终分数等于语义相似度;`1.0` 表示只使用 hotness。有效范围:`0.0` 到 `1.0`。 | `0.0` |-| `score_propagation_alpha` | float | 层级检索中,子节点自身分数与父节点传播分数混合时,子节点自身分数的权重。`1.0` 表示忽略父节点分数(仅使用语义相似度);`0.5` 表示与父节点分数等权混合;`0.0` 表示只使用父节点分数。有效范围:`0.0` 到 `1.0`。 | `1.0` |如果需要分数严格反映向量相似度,保持 `hotness_alpha` 为 `0.0`。只有当希望高频访问或最近更新的上下文获得排序提升时,才将它设置为大于 `0.0`。@@ -2032,8 +2030,7 @@ Task 记录文件位于所属账号的系统目录:"extra_headers": {}},"retrieval": {- "hotness_alpha": 0.0,- "score_propagation_alpha": 1.0+ "hotness_alpha": 0.0},"encryption": {"enabled": false,diff --git a/docs/zh/guides/10-prompt-guide.md b/docs/zh/guides/10-prompt-guide.mdindex 574a978b5..589541ee2 100644--- a/docs/zh/guides/10-prompt-guide.md+++ b/docs/zh/guides/10-prompt-guide.md@@ -344,7 +344,7 @@ peer_enabled: true- `semantic.overview_generation`- 生效环节:目录级概览生成阶段- - 影响能力:目录 overview、层级检索与导航体验+ - 影响能力:目录 overview、全局检索与导航体验- 作用:根据文件摘要和子目录 abstract 生成目录级 overview- 关键输入:`dir_name`、`file_summaries`、`children_abstracts`、`output_language`diff --git a/examples/ov.conf.example b/examples/ov.conf.exampleindex a9194a273..23402435a 100644--- a/examples/ov.conf.example+++ b/examples/ov.conf.example@@ -244,7 +244,6 @@},"retrieval": {"hotness_alpha": 0.0,- "score_propagation_alpha": 1.0,"recall_intent_timeout_s": 5.0,"recall_rewrite_timeout_s": 30.0,},diff --git a/openviking/eval/datasets/local_doc_example_glm5.jsonl b/openviking/eval/datasets/local_doc_example_glm5.jsonlindex 62aced17c..003c61a7a 100644--- a/openviking/eval/datasets/local_doc_example_glm5.jsonl+++ b/openviking/eval/datasets/local_doc_example_glm5.jsonl@@ -12,7 +12,7 @@{"question": "`client.add_resource` 支持哪些输入格式?如何处理本地目录的递归添加和子目录过滤?", "files": ["OpenViking/docs/en/api/02-resources.md", "OpenViking/openviking/parse/directory_scan.py:1-80"], "answer": "`add_resource` 支持多种输入格式,目录处理通过 directory_scan 模块实现。\n\n**支持的输入格式**:\n\n| 格式 | 扩展名 | 处理方式 |\n|------|--------|----------|\n| PDF | `.pdf` | 文本和图片提取 |\n| Markdown | `.md` | 原生支持 |\n| HTML | `.html`, `.htm` | 清理文本提取 |\n| 纯文本 | `.txt` | 直接导入 |\n| JSON/YAML | `.json`, `.yaml` | 结构化解析 |\n| 代码 | `.py`, `.js`, `.go` 等 | 语法感知解析 |\n| 图片 | `.png`, `.jpg` 等 | VLM 描述 |\n| 视频/音频 | `.mp4`, `.mp3` 等 | 转录 |\n\n**目录递归添加流程**:\n\n1. **DirectoryScan 预扫描**:\n - 遍历目录树\n - 分类文件为 processable / unsupported\n - 跳过:点文件、符号链接、空文件\n\n2. **目录过滤机制**:\n```python\ndef _should_skip_directory(dir_path, root, ignore_dirs):\n # 跳过:点目录、符号链接、IGNORE_DIRS\n```\n\n3. **内置忽略目录**:\n - `.git`、`node_modules`、`__pycache__` 等\n - 通过 `IGNORE_DIRS` 常量定义\n\n4. **API 参数**:\n```python\nclient.add_resource(\n \"./documents\", # 本地目录路径\n to=\"viking://resources/docs\", # 目标 URI\n reason=\"文档说明\", # 添加原因\n wait=True, # 等待处理完成\n timeout=300 # 超时时间\n)\n```\n\n**处理流水线**:\nInput → Parser → TreeBuilder → AGFS → SemanticQueue → Vector Index"}{"question": "向量索引是如何构建和维护的?增量更新时如何避免全量重建?", "files": ["OpenViking/openviking/storage/viking_vector_index_backend.py", "OpenViking/docs/en/concepts/05-storage.md"], "answer": "向量索引通过 VikingVectorIndexBackend 实现,支持增量更新。\n\n**索引构建流程**:\n\n1. **初始化配置**:\n```python\nconfig = VectorDBBackendConfig(\n backend=\"local\", # local/http/volcengine\n path=\"./data/vectordb\",\n dimension=1024,\n distance_metric=\"cosine\"\n)\nbackend = VikingVectorIndexBackend(config=config)\n```\n\n2. **支持的后端模式**:\n - **local**:本地持久化存储\n - **http**:远程 HTTP 服务\n - **volcengine**:火山引擎 VikingDB\n\n3. **索引特性**:\n - 使用 BruteForce 索引进行向量相似度搜索\n - 支持标量过滤和多操作符\n - 每个集合自动管理索引\n\n**增量更新机制**:\n\n1. **URI 级别操作**:\n - VikingFS 的 `rm()`、`mv()` 操作同步更新向量索引\n - 新增内容自动触发向量化并写入索引\n\n2. **避免全量重建**:\n - 通过 URI 唯一标识每个向量条目\n - 更新时只删除旧向量、插入新向量\n - 不影响其他条目的索引\n\n3. **双层存储优势**:\n - 向量索引只存储 URI 引用和元数据\n - 内容变更不影响索引结构\n - 独立扩展存储和索引层\n\n**核心 API**:\n- `upsert()`:插入或更新向量\n- `delete()`:删除向量\n- `search()`:向量相似度搜索"}{"question": "配置文件 `ov.conf` 的必填项有哪些?如何配置不同的 VLM 后端(OpenAI、Volcengine 等)?", "files": ["OpenViking/docs/en/guides/01-configuration.md:1-150"], "answer": "配置文件 `ov.conf` 包含 embedding、vlm、rerank、storage 等配置项。\n\n**必填配置项**:\n\n```json\n{\n \"embedding\": {\n \"dense\": {\n \"provider\": \"volcengine\", // 必填:volcengine/openai/vikingdb\n \"api_key\": \"your-api-key\", // 必填\n \"model\": \"doubao-embedding-vision-251215\",\n \"dimension\": 1024\n }\n },\n \"vlm\": {\n \"provider\": \"volcengine\", // 必填\n \"api_key\": \"your-api-key\", // 必填\n \"model\": \"doubao-seed-2-0-pro-260215\"\n }\n}\n```\n\n**VLM 后端配置**:\n\n1. **Volcengine(豆包模型)**:\n```json\n{\n \"vlm\": {\n \"api_base\": \"https://ark.cn-beijing.volces.com/api/v3\",\n \"api_key\": \"your-volcengine-api-key\",\n \"provider\": \"volcengine\",\n \"model\": \"doubao-seed-2-0-pro-260215\"\n }\n}\n```\n\n2. **OpenAI**:\n```json\n{\n \"vlm\": {\n \"api_base\": \"https://api.openai.com/v1\",\n \"api_key\": \"your-openai-api-key\",\n \"provider\": \"openai\",\n \"model\": \"gpt-4-vision-preview\"\n }\n}\n```\n\n**Embedding 配置**:\n\n| 参数 | 说明 |\n|------|------|\n| `provider` | volcengine/openai/vikingdb |\n| `api_key` | API 密钥 |\n| `model` | 模型名称 |\n| `dimension` | 向量维度 |\n| `input` | 输入类型:text/multimodal |\n\n**可选配置**:\n- `rerank`:重排序模型配置\n- `storage`:存储后端配置"}-{"question": "`HierarchicalRetriever` 的检索流程是怎样的?如何实现从 L0 到 L2 的层级穿透检索?", "files": ["OpenViking/openviking/retrieve/hierarchical_retriever.py"], "answer": "HierarchicalRetriever 实现目录递归检索和层级穿透。\n\n**检索流程**:\n\n1. **初始化**:\n```python\nclass HierarchicalRetriever:\n MAX_CONVERGENCE_ROUNDS = 3 # 收敛轮次\n SCORE_PROPAGATION_ALPHA = 0.5 # 分数传播系数\n DIRECTORY_DOMINANCE_RATIO = 1.2 # 目录分数阈值\n GLOBAL_SEARCH_TOPK = 3 # 全局检索数量\n```\n\n2. **检索模式**:\n - **THINKING**:深度思考模式\n - **QUICK**:快速检索模式\n\n3. **核心方法**:\n```python\nasync def retrieve(\n self,\n query: TypedQuery,\n limit: int = 5,\n mode: RetrieverMode = RetrieverMode.THINKING,\n score_threshold: Optional[float] = None,\n metadata_filter: Optional[Dict[str, Any]] = None,\n) -> QueryResult:\n```\n\n**层级穿透机制**:\n\n1. **L0 检索**:\n - 向量相似度搜索 `.abstract.md` 内容\n - 快速过滤不相关内容\n\n2. **L1 确认**:\n - 读取 `.overview.md` 进行重排序\n - 确认内容相关性\n\n3. **L2 加载**:\n - 按需读取完整内容\n - 仅在确认需要时加载\n\n**关键特性**:\n- 分数传播:目录分数向子节点传播\n- 目录优势:目录分数需超过最大子节点分数"}+{"question": "`HierarchicalRetriever` 的全局检索流程是怎样的?THINKING 和 QUICK 如何影响召回数量与 Rerank?", "files": ["OpenViking/openviking/retrieve/hierarchical_retriever.py"], "answer": "HierarchicalRetriever 每条查询只执行一次全局向量检索,目标目录、上下文类型、权限、元数据和 level 共同限定搜索范围。QUICK 直接召回 limit 条,不执行 Rerank。THINKING 配置了可用的 Rerank 时召回 2 × limit 条候选,对候选统一精排一次,应用分数阈值后返回最多 limit 条;没有可用的 Rerank 时只召回 limit 条。未显式指定模式时,有可用 Rerank 选择 THINKING,否则选择 QUICK。图像查询跳过文本 Rerank。Rerank 失败或返回无效结果时沿用向量分数。检索不再逐层遍历目录,也没有父子分数传播或多轮收敛判断。"}{"question": "语义处理队列的工作机制是什么?如何监控和调试异步处理状态?", "files": ["OpenViking/docs/en/concepts/01-architecture.md", "OpenViking/openviking/service/resource_service.py:1-80"], "answer": "语义处理队列通过 SemanticQueue 异步生成 L0/L1 内容。\n\n**工作机制**:\n\n1. **处理流水线**:\n```\nInput → Parser → TreeBuilder → AGFS → SemanticQueue → Vector Index\n```\n\n2. **队列处理**:\n - TreeBuilder 将文件移动到 AGFS 后入队\n - SemanticQueue 异步处理队列项\n - 自底向上生成 L0/L1\n\n3. **异步处理优势**:\n - 不阻塞资源添加操作\n - 支持大规模文档处理\n - LLM 调用并行化\n\n**监控和调试**:\n\n1. **等待处理完成**:\n```python\nresult = client.add_resource(\"./docs\", wait=False)\nclient.wait_processed() # 等待队列处理完成\n```\n\n2. **超时控制**:\n```python\nresult = client.add_resource(\n \"./docs\",\n wait=True,\n timeout=300 # 超时时间(秒)\n)\n```\n\n3. **DebugService**:\n - 提供 ObserverService 用于调试\n - 可视化检索轨迹\n\n4. **日志监控**:\n - 通过 logger 获取处理状态\n - 监控队列长度和处理延迟\n\n**Service 层支持**:\n- ResourceService 管理 add_resource 操作\n- wait_processed() 方法等待语义处理完成"}{"question": "如何通过代码动态创建和管理 Session?Session 与 Memory 的关系是什么?", "files": ["OpenViking/docs/en/concepts/08-session.md", "OpenViking/openviking/session/compressor_v2.py"], "answer": "Session 通过 client API 动态创建,commit 后自动提取 Memory。\n\n**Session 创建和管理**:\n\n1. **创建 Session**:\n```python\n# 创建新会话\nsession = client.session(session_id=\"chat_001\")\n\n# 或使用 create_session\nresult = client.create_session()\n```\n\n2. **添加消息**:\n```python\nsession.add_message(\n \"user\",\n [TextPart(\"How to configure embedding?\")]\n)\n\nsession.add_message(\n \"assistant\",\n [\n TextPart(\"Here's how...\"),\n ContextPart(uri=\"viking://~/memories/profile.md\"),\n ]\n)\n```\n\n3. **提交会话**:\n```python\nresult = session.commit()\n# {\"status\": \"committed\", \"memories_extracted\": 5}\n```\n\n**Session 与 Memory 的关系**:\n\n1. **Memory 来源**:\n - Memory 从 Session 消息中提取\n - SessionCompressorV2 在 commit 时触发\n\n2. **提取流程**:\n - 格式化会话消息\n - 根据当前记忆策略调用 LLM 提取已启用的记忆类型\n - 写入当前用户或相关 Peer 的记忆目录并索引\n\n3. **内置记忆类型与默认位置**:\n - profile → `user/memories/profile.md`\n - preferences → `user/memories/preferences/`\n - entities → `user/memories/entities/`\n - events → `user/memories/events/`\n - identity → `user/memories/identity.md`\n - soul → `user/memories/soul.md`\n - cases → `user/memories/cases/`\n - trajectories → `user/memories/trajectories/`\n - experiences → `user/memories/experiences/`\n - tools → `user/memories/tools/`\n - skills → `user/memories/skills/`\n\n这些 `user/...` 短路径会按当前身份解析为 `viking://user/{user_id}/...`,应用也可以通过自定义模板扩展或调整记忆类型。"}{"question": "OpenViking Server 模式与本地模式的区别是什么?如何选择部署方式?", "files": ["OpenViking/openviking/server/bootstrap.py:1-61", "OpenViking/docs/en/concepts/01-architecture.md"], "answer": "Server 模式提供 HTTP API,本地模式直接嵌入应用。\n\n**模式对比**:\n\n| 维度 | 本地模式 | Server 模式 |\n|------|----------|-------------|\n| 访问方式 | Python SDK | HTTP API |\n| 部署复杂度 | 简单 | 需启动服务 |\n| 多客户端 | 不支持 | 支持 |\n| 资源共享 | 单进程 | 多进程共享 |\n| 适用场景 | 单机应用 | 生产环境 |\n\n**Server 模式启动**:\n\n1. **命令行启动**:\n```bash\nopenviking-server --host 0.0.0.0 --port 1933 --config ./ov.conf\n```\n\n2. **参数说明**:\n- `--host`:绑定地址\n- `--port`:端口号\n- `--config`:配置文件路径\n\n3. **HTTP API 示例**:\n```bash\ncurl -X POST http://localhost:1933/api/v1/resources \\\n -H \"Content-Type: application/json\" \\\n -H \"X-API-Key: your-key\" \\\n -d '{\"path\": \"./documents/guide.md\"}'\n```\n\n**部署方式选择**:\n\n1. **选择本地模式**:\n - 单机应用开发\n - 快速原型验证\n - 无需多客户端访问\n\n2. **选择 Server 模式**:\n - 生产环境部署\n - 多客户端/多服务访问\n - 需要集中式上下文管理\n - 微服务架构集成\n\n**Service 层复用**:\n两种模式共享 Service 层逻辑,支持 HTTP Server 和 CLI 复用。"}diff --git a/openviking/retrieve/__init__.py b/openviking/retrieve/__init__.pyindex fd7817ab5..205877a3d 100644--- a/openviking/retrieve/__init__.py+++ b/openviking/retrieve/__init__.py@@ -3,7 +3,7 @@"""Retrieval module for OpenViking.-Provides intent-driven hierarchical context retrieval.+Provides global context retrieval with optional intent analysis and reranking."""from openviking.retrieve.hierarchical_retriever import HierarchicalRetrieverdiff --git a/openviking/retrieve/hierarchical_retriever.py b/openviking/retrieve/hierarchical_retriever.pyindex 1ec3c0111..7085013b9 100644--- a/openviking/retrieve/hierarchical_retriever.py+++ b/openviking/retrieve/hierarchical_retriever.py@@ -1,19 +1,14 @@# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.# SPDX-License-Identifier: AGPL-3.0"""-Hierarchical retriever for OpenViking.--Implements directory-based hierarchical retrieval with recursive search-and rerank-based relevance scoring.+Global vector retrieval with optional reranking of the recalled candidates."""import asyncio-import heapq-import loggingimport mathimport timefrom datetime import datetime-from typing import Any, Dict, List, Optional, Tuple+from typing import Any, Dict, List, Optionalfrom openviking.core.context import ContextLevelfrom openviking.core.retrieval_targets import default_target_directories@@ -51,12 +46,9 @@ class RetrieverMode(str):class HierarchicalRetriever:- """Hierarchical retriever with dense and sparse vector support."""+ """Global retriever with dense and sparse vector support."""- MAX_CONVERGENCE_ROUNDS = 3 # Stop after multiple rounds with unchanged topk- DIRECTORY_DOMINANCE_RATIO = 1.2 # Directory score must exceed max child score- GLOBAL_SEARCH_TOPK = 10 # Global retrieval count (more candidates = better rerank precision)- MAX_PARALLEL_CHILD_SEARCHES = 4 # Limit per-request fan-out against remote vector stores+ RERANK_CANDIDATE_MULTIPLIER = 2LEVEL_URI_SUFFIX = {0: ".abstract.md", 1: ".overview.md"}def __init__(@@ -66,7 +58,7 @@ class HierarchicalRetriever:rerank_config: Optional[RerankConfig] = None,retrieval_config: Optional[RetrievalConfig] = None,):- """Initialize hierarchical retriever with rerank_config.+ """Initialize retriever with rerank_config.Args:storage: VikingVectorIndexBackend instance@@ -80,7 +72,6 @@ class HierarchicalRetriever:self.rerank_max_input_tokens = rerank_config.max_input_tokens if rerank_config else 0self.retrieval_config = retrieval_config or RetrievalConfig()self.hotness_alpha = self.retrieval_config.hotness_alpha- self.score_propagation_alpha = self.retrieval_config.score_propagation_alpha# Use rerank threshold if available, otherwise use a defaultself.threshold = rerank_config.threshold if rerank_config else 0@@ -110,10 +101,12 @@ class HierarchicalRetriever:level: Optional[List[int]] = None,) -> QueryResult:"""- Execute hierarchical retrieval.+ Run one global vector search, then optionally rerank its candidates.Args:ctx: Request context used for tenant and permission filtering+ mode: QUICK uses vector scores; THINKING optionally reranks the recalled+ candidates. None selects THINKING when a reranker is configured.score_threshold: Custom score threshold (overrides config)score_gte: True uses >=, False uses >scope_dsl: Additional scope constraints passed from public find/search filter@@ -122,13 +115,14 @@ class HierarchicalRetriever:t0 = time.monotonic()telemetry = get_current_telemetry()effective_threshold = self._resolve_threshold(score_threshold)- image_query = bool(getattr(query, "image_query", False))+ image_query = query.image_queryif mode is None:- mode = RetrieverMode.QUICK if not self._rerank_client else RetrieverMode.THINKING- if image_query:- mode = RetrieverMode.QUICK- if level is None:- level = [2]+ mode = RetrieverMode.THINKING if self._rerank_client else RetrieverMode.QUICK+ use_rerank = (+ mode == RetrieverMode.THINKING and self._rerank_client is not None and not image_query+ )+ if image_query and level is None:+ level = [2]# 创建 proxy 包装器,绑定当前 ctxvector_proxy = VikingDBManagerProxy(self.vector_store, ctx)@@ -137,7 +131,7 @@ class HierarchicalRetriever:if not await vector_proxy.collection_exists_bound():logger.warning(- "[RecursiveSearch] Collection %s does not exist",+ "[HierarchicalRetriever] Collection %s does not exist",vector_proxy.collection_name,)return QueryResult(@@ -162,7 +156,7 @@ class HierarchicalRetriever:query_vector = result.dense_vectorsparse_query_vector = result.sparse_vector- # Step 1: Determine starting directories based on explicit target dirs.+ # Report the effective search scope in the query result.if target_dirs:root_uris = target_dirselse:@@ -172,166 +166,55 @@ class HierarchicalRetriever:if image_query and context_type is None:context_type = ContextType.RESOURCE.value- if mode == RetrieverMode.QUICK:- search_limit = (- max(limit * 5, 50) if image_query else max(limit, self.GLOBAL_SEARCH_TOPK)- )- with telemetry.measure("search.vector_retrieval"):- quick_results = await vector_proxy.search_in_tenant(- query_vector=query_vector,- sparse_query_vector=sparse_query_vector,- context_type=context_type,- target_directories=target_dirs,- extra_filter=scope_dsl,- level=level,- limit=search_limit,- )- telemetry.count("vector.searches", 1)- telemetry.count("vector.scored", len(quick_results))- telemetry.count("vector.scanned", len(quick_results))-- collected_by_uri: Dict[str, Dict[str, Any]] = {}- for result in quick_results:- uri = result.get("uri", "")- if not uri:- continue-- score = self._finite_score(result.get("_score", 0.0))- if not self._passes_threshold(score, effective_threshold, score_gte):- continue-- candidate = dict(result)- candidate["_score"] = score- candidate["_final_score"] = score-- previous = collected_by_uri.get(uri)- if previous is None or score > previous.get("_final_score", 0.0):- collected_by_uri[uri] = candidate-- candidates = sorted(- collected_by_uri.values(),- key=lambda x: x.get("_final_score", 0.0),- reverse=True,+ search_limit = limit * self.RERANK_CANDIDATE_MULTIPLIER if use_rerank else limit+ with telemetry.measure("search.vector_retrieval"):+ vector_results = await vector_proxy.search_in_tenant(+ query_vector=query_vector,+ sparse_query_vector=sparse_query_vector,+ context_type=context_type,+ target_directories=target_dirs,+ extra_filter=scope_dsl,+ level=level,+ limit=search_limit,)- apply_hotness = False- rerank_used = False- else:- # Step 2: Global vector search to supplement starting points- with telemetry.measure("search.vector_retrieval"):- global_results = await vector_proxy.search_in_tenant(- query_vector=query_vector,- sparse_query_vector=sparse_query_vector,- context_type=context_type,- target_directories=target_dirs,- extra_filter=scope_dsl,- level=[0, 1],- limit=max(limit, self.GLOBAL_SEARCH_TOPK),- )- telemetry.count("vector.searches", 1)- telemetry.count("vector.scored", len(global_results))- telemetry.count("vector.scanned", len(global_results))-- leaf_results: List[Dict[str, Any]] = []- if await self.vector_store._acl_enabled(ctx) and (level is None or 2 in level):- leaf_results = await vector_proxy.search_in_tenant(- query_vector=query_vector,- sparse_query_vector=sparse_query_vector,- context_type=context_type,- target_directories=target_dirs,- extra_filter=scope_dsl,- level=[2],- limit=max(limit, self.GLOBAL_SEARCH_TOPK),- )- telemetry.count("vector.searches", 1)- telemetry.count("vector.scored", len(leaf_results))- telemetry.count("vector.scanned", len(leaf_results))- if self._rerank_client and mode == RetrieverMode.THINKING and leaf_results:- leaf_scores = await self._rerank_scores(- query.query,- [str(result.get("abstract", "")) for result in leaf_results],- [self._finite_score(result.get("_score", 0.0)) for result in leaf_results],- )- leaf_results = [- {**result, "_score": score}- for result, score in zip(leaf_results, leaf_scores, strict=True)- ]-- # Debug: Print all URIs in global_results- if logger.isEnabledFor(logging.DEBUG):- logger.debug(f"[retrieve] target_dirs: {target_dirs}")- logger.debug(f"[retrieve] root_uris: {root_uris}")- logger.debug(f"[retrieve] scope_dsl: {scope_dsl}")- logger.debug(- f"[retrieve] Step 2 completed, global_results contains {len(global_results)} items:"- )- for i, r in enumerate(global_results):- uri = r.get("uri", "UNKNOWN_URI")- score = r.get("_score", 0.0)- result_level = r.get("level", "UNKNOWN_LEVEL")- account_id = r.get("account_id", "UNKNOWN_ACCOUNT_ID")- logger.debug(- f" [{i}] URI: {uri}, score: {score:.4f}, level: {result_level}, account_id: {account_id}"- )-- # Step 3: Pick recursive entry points from directory hits and explicit roots.- directory_scores = [self._finite_score(r.get("_score", 0.0)) for r in global_results]- if self._rerank_client and mode == RetrieverMode.THINKING:- directory_scores = await self._rerank_scores(- query.query,- [str(r.get("abstract", "")) for r in global_results],- directory_scores,- )+ telemetry.count("vector.searches", 1)+ telemetry.count("vector.scored", len(vector_results))+ telemetry.count("vector.scanned", len(vector_results))- starting_points = []- seen_starting_uris = set()- for result, score in zip(global_results, directory_scores, strict=True):- uri = result.get("uri", "")- if not uri or uri in seen_starting_uris:- continue- starting_points.append((uri, score))- seen_starting_uris.add(uri)-- for uri in root_uris:- if uri not in seen_starting_uris:- starting_points.append((uri, 0.0))- seen_starting_uris.add(uri)-- # Add directory hits to the result pool only when explicitly requested.- initial_candidates = list(leaf_results)- if level is not None:- for result, score in zip(global_results, directory_scores, strict=True):- if result.get("level", 2) not in level:- continue- candidate = dict(result)- candidate["_score"] = score- initial_candidates.append(candidate)-- # Step 4: Recursive search- with telemetry.measure("search.vector_retrieval"):- candidates = await self._recursive_search(- vector_proxy=vector_proxy,- query=query.query,- query_vector=query_vector,- sparse_query_vector=sparse_query_vector,- starting_points=starting_points,- limit=limit,- mode=mode,- threshold=effective_threshold,- score_gte=score_gte,- context_type=context_type,- target_dirs=target_dirs,- scope_dsl=scope_dsl,- initial_candidates=initial_candidates,- level=level,- )- apply_hotness = True- rerank_used = self._rerank_client is not None and mode == RetrieverMode.THINKING+ # Keep the highest vector-scored hit for each URI before reranking.+ collected_by_uri: Dict[str, Dict[str, Any]] = {}+ for result in vector_results:+ uri = result.get("uri", "")+ if not uri:+ continue+ score = self._finite_score(result.get("_score", 0.0))+ previous = collected_by_uri.get(uri)+ if previous is None or score > previous["_score"]:+ collected_by_uri[uri] = {**result, "_score": score}++ candidates = sorted(+ collected_by_uri.values(), key=lambda candidate: candidate["_score"], reverse=True+ )+ scores = [candidate["_score"] for candidate in candidates]+ rerank_used = use_rerank and bool(candidates)+ if rerank_used:+ scores = await self._rerank_scores(+ query.query,+ [str(candidate.get("abstract", "")) for candidate in candidates],+ scores,+ )- # Step 6: Convert results+ # A low vector score can still rerank highly, so filter only after reranking.+ candidates = [+ {**candidate, "_final_score": score}+ for candidate, score in zip(candidates, scores, strict=True)+ if self._passes_threshold(score, effective_threshold, score_gte)+ ]+ telemetry.count("vector.passed", len(candidates))matched = await self._convert_to_matched_contexts(candidates,ctx=ctx,- apply_hotness=apply_hotness,+ apply_hotness=use_rerank,)final = matched[:limit]@@ -418,175 +301,6 @@ class HierarchicalRetriever:normalized_scores[index] = self._finite_score(score, fallback_scores[index])return normalized_scores- async def _recursive_search(- self,- vector_proxy: VikingDBManagerProxy,- query: str,- query_vector: Optional[List[float]],- sparse_query_vector: Optional[Dict[str, float]],- starting_points: List[Tuple[str, float]],- limit: int,- mode: str,- threshold: Optional[float] = None,- score_gte: bool = False,- context_type: Optional[str] = None,- target_dirs: Optional[List[str]] = None,- scope_dsl: Optional[FilterExpr | Dict[str, Any]] = None,- initial_candidates: Optional[List[Dict[str, Any]]] = None,- level: Optional[List[int]] = None,- ) -> List[Dict[str, Any]]:- """- Recursive search with directory priority return and score propagation.-- Args:- threshold: Score threshold- score_gte: True uses >=, False uses >- grep_patterns: Keyword match patterns- scope_dsl: Additional scope constraints from public find/search filter- """- effective_threshold = self._resolve_threshold(threshold)-- sparse_query_vector = sparse_query_vector or None-- collected_by_uri: Dict[str, Dict[str, Any]] = {}- dir_queue: List[tuple] = [] # Priority queue: (-score, uri)- visited: set = set()- prev_topk_uris: set = set()- prev_pool_size = 0- convergence_rounds = 0- stagnant_rounds = 0-- # Add initial candidates that match the requested level.- if initial_candidates:- for r in initial_candidates:- uri = r.get("uri", "")- if not uri:- continue- if level is None or r.get("level", 2) in level:- score = self._finite_score(r.get("_score", 0.0))- if not self._passes_threshold(score, effective_threshold, score_gte):- logger.debug(- f"[RecursiveSearch] Initial candidate URI {uri} score {score:.4f} did not pass threshold {effective_threshold}"- )- continue- r["_final_score"] = score- collected_by_uri[uri] = r- logger.debug(- f"[RecursiveSearch] Added initial candidate: {uri} (score: {score:.4f})"- )-- alpha = self.score_propagation_alpha-- # Initialize: process starting points- for uri, score in starting_points:- heapq.heappush(dir_queue, (-score, uri))-- async def search_children(current_uri: str) -> List[Dict[str, Any]]:- return await vector_proxy.search_children_in_tenant(- parent_uri=current_uri,- query_vector=query_vector,- sparse_query_vector=sparse_query_vector, # Pass sparse vector- context_type=context_type,- target_directories=target_dirs,- extra_filter=scope_dsl,- limit=max(limit * 2, 20),- )-- parallelism = max(1, self.MAX_PARALLEL_CHILD_SEARCHES)-- while dir_queue:- batch: List[Tuple[str, float]] = []- while dir_queue and len(batch) < parallelism:- temp_score, current_uri = heapq.heappop(dir_queue)- current_score = -temp_score- if current_uri in visited:- continue- visited.add(current_uri)- logger.info(f"[RecursiveSearch] Entering URI: {current_uri}")- batch.append((current_uri, current_score))-- if not batch:- continue-- batch_results = await asyncio.gather(- *(search_children(current_uri) for current_uri, _ in batch)- )-- telemetry = get_current_telemetry()- for (_, current_score), results in zip(batch, batch_results, strict=True):- telemetry.count("vector.searches", 1)- telemetry.count("vector.scored", len(results))- telemetry.count("vector.scanned", len(results))-- if not results:- continue-- query_scores = [self._finite_score(r.get("_score", 0.0)) for r in results]- if self._rerank_client and mode == RetrieverMode.THINKING:- documents = [str(r.get("abstract", "")) for r in results]- query_scores = await self._rerank_scores(query, documents, query_scores)-- for r, score in zip(results, query_scores, strict=True):- uri = r.get("uri", "")- final_score = (- alpha * score + (1 - alpha) * current_score if current_score else score- )-- if not self._passes_threshold(final_score, effective_threshold, score_gte):- logger.debug(- f"[RecursiveSearch] URI {uri} score {final_score} did not pass threshold {effective_threshold}"- )- continue-- telemetry.count("vector.passed", 1)- if level is None or r.get("level", 2) in level:- # Deduplicate by URI and keep the highest-scored candidate.- previous = collected_by_uri.get(uri)- if previous is None or final_score > previous.get("_final_score", 0):- r["_final_score"] = final_score- collected_by_uri[uri] = r- logger.debug(- "[RecursiveSearch] Updated URI: %s candidate score to %.4f",- uri,- final_score,- )-- # Only recurse into directories (L0/L1). L2 files are terminal hits.- if uri not in visited and r.get("level", 2) != 2:- heapq.heappush(dir_queue, (-final_score, uri))-- # Convergence check after each parallel expansion round.- current_topk = sorted(- collected_by_uri.values(),- key=lambda x: x.get("_final_score", 0),- reverse=True,- )[:limit]- current_topk_uris = {c.get("uri", "") for c in current_topk}- current_pool_size = len(collected_by_uri)-- if current_topk_uris == prev_topk_uris and len(current_topk_uris) >= limit:- convergence_rounds += 1-- if convergence_rounds >= self.MAX_CONVERGENCE_ROUNDS:- break- elif current_pool_size == prev_pool_size:- stagnant_rounds += 1-- if stagnant_rounds >= self.MAX_CONVERGENCE_ROUNDS:- break- else:- convergence_rounds = 0- stagnant_rounds = 0- prev_topk_uris = current_topk_uris- prev_pool_size = current_pool_size-- collected = sorted(- collected_by_uri.values(),- key=lambda x: x.get("_final_score", 0),- reverse=True,- )- return collected[:limit]-async def _convert_to_matched_contexts(self,candidates: List[Dict[str, Any]],@@ -651,9 +365,7 @@ class HierarchicalRetriever:abstract=abstract,category=c.get("category", ""),score=final_score,- search_tags=normalize_search_tags(- c.get("search_tags"), discard_invalid=True- ),+ search_tags=normalize_search_tags(c.get("search_tags"), discard_invalid=True),))diff --git a/openviking/retrieve/skill_package_retriever.py b/openviking/retrieve/skill_package_retriever.pyindex 257fba6af..c90b0802c 100644--- a/openviking/retrieve/skill_package_retriever.py+++ b/openviking/retrieve/skill_package_retriever.py@@ -19,6 +19,8 @@ from openviking_cli.retrieve.types import QueryResult, TypedQueryclass SkillPackageRetriever(HierarchicalRetriever):"""Reuse hit conversion and thresholds; keep package pagination out of general search."""+ MIN_PAGE_SIZE = 10+async def retrieve_skills(self,query: TypedQuery,@@ -45,7 +47,7 @@ class SkillPackageRetriever(HierarchicalRetriever):embedded = await embed_compat(self.embedder, query.query, is_query=True)query_vector, sparse_vector = embedded.dense_vector, embedded.sparse_vector- page_size = max(limit, self.GLOBAL_SEARCH_TOPK)+ page_size = max(limit, self.MIN_PAGE_SIZE)offset = 0seen = set()candidates = {}diff --git a/openviking/storage/queuefs/embedding_msg_converter.py b/openviking/storage/queuefs/embedding_msg_converter.pyindex b3b5869d6..7ed5f279f 100644--- a/openviking/storage/queuefs/embedding_msg_converter.py+++ b/openviking/storage/queuefs/embedding_msg_converter.py@@ -58,7 +58,7 @@ class EmbeddingMsgConverter:if owner_fields is not None:context_data["owner_user_id"] = owner_fields["owner_user_id"]- # Derive level field for hierarchical retrieval.+ # Derive the level field used by retrieval filters.uri = context_data.get("uri", "")context_level = context.levelif context_level is not None:diff --git a/openviking/storage/viking_vector_index_backend.py b/openviking/storage/viking_vector_index_backend.pyindex 166cca13b..b977ce506 100644--- a/openviking/storage/viking_vector_index_backend.py+++ b/openviking/storage/viking_vector_index_backend.py@@ -1928,53 +1928,6 @@ class VikingVectorIndexBackend:ctx=ctx,)- async def search_children_in_tenant(- self,- ctx: RequestContext,- parent_uri: str,- query_vector: Optional[List[float]],- sparse_query_vector: Optional[Dict[str, float]] = None,- context_type: Optional[str] = None,- target_directories: Optional[List[str]] = None,- extra_filter: Optional[FilterExpr | Dict[str, Any]] = None,- limit: int = 10,- ) -> List[Dict[str, Any]]:- # TODO:Better Alternative to Current Temporary Fix-- # If parent_uri is already under the requested target_directories,- # adding a redundant scope prefix filter can slow down the backend.- # Keep tenant/context filters but skip target_directories in that case.- effective_target_directories = target_directories- if target_directories:- parent_norm = parent_uri.rstrip("/")- for target_dir in target_directories:- if not target_dir:- continue- target_norm = target_dir.rstrip("/")- if parent_norm == target_norm or parent_norm.startswith(target_norm + "/"):- effective_target_directories = None- break-- acl_enabled = await self._acl_enabled(ctx)- merged_filter = self._merge_filters(- PathScope("uri", parent_uri, depth=1),- self._build_scope_filter(- ctx=ctx,- context_type=context_type,- target_directories=effective_target_directories,- extra_filter=extra_filter,- acl_enabled=acl_enabled,- ),- )- return await self.search(- query_vector=query_vector,- sparse_query_vector=sparse_query_vector,- filter=merged_filter,- limit=limit,- output_fields=RETRIEVAL_OUTPUT_FIELDS,- ctx=ctx,- )-async def get_context_by_uri(self,uri: str,diff --git a/openviking/storage/vikingdb_manager.py b/openviking/storage/vikingdb_manager.pyindex db9b170cc..8e601fbf4 100644--- a/openviking/storage/vikingdb_manager.py+++ b/openviking/storage/vikingdb_manager.py@@ -465,27 +465,6 @@ class VikingDBManagerProxy:offset=offset,)- async def search_children_in_tenant(- self,- parent_uri: str,- query_vector: Optional[List[float]],- sparse_query_vector: Optional[Dict[str, float]] = None,- context_type: Optional[str] = None,- target_directories: Optional[List[str]] = None,- extra_filter: Optional[FilterExpr | Dict[str, Any]] = None,- limit: int = 10,- ) -> List[Dict[str, Any]]:- return await self._manager.search_children_in_tenant(- self._ctx,- parent_uri=parent_uri,- query_vector=query_vector,- sparse_query_vector=sparse_query_vector,- context_type=context_type,- target_directories=target_directories,- extra_filter=extra_filter,- limit=limit,- )-async def get_context_by_uri(self,uri: str,diff --git a/openviking_cli/utils/config/retrieval_config.py b/openviking_cli/utils/config/retrieval_config.pyindex 055a44ac7..2188a67a0 100644--- a/openviking_cli/utils/config/retrieval_config.py+++ b/openviking_cli/utils/config/retrieval_config.py@@ -16,16 +16,6 @@ class RetrievalConfig(BaseModel):"0 disables hotness boost; 1 uses only hotness."),)- score_propagation_alpha: float = Field(- default=1.0,- ge=0.0,- le=1.0,- description=(- "Weight for each child result's own score when blending with its parent score "- "during hierarchical retrieval. 0 uses only the parent score; "- "1 uses only the child score."- ),- )recall_intent_timeout_s: float = Field(default=5.0,gt=0.0,diff --git a/tests/misc/test_retrieval_enable_intent.py b/tests/misc/test_retrieval_enable_intent.pyindex 944a38cc6..bb1a03bf2 100644--- a/tests/misc/test_retrieval_enable_intent.py+++ b/tests/misc/test_retrieval_enable_intent.py@@ -9,8 +9,8 @@ import pytestfrom openviking.server.identity import RequestContext, Rolefrom openviking.storage.viking_fs import VikingFSfrom openviking_cli.retrieve.types import QueryResult-from openviking_cli.utils.config.retrieval_config import RetrievalConfigfrom openviking_cli.session.user_id import UserIdentifier+from openviking_cli.utils.config.retrieval_config import RetrievalConfigdef _ctx() -> RequestContext:@@ -20,12 +20,13 @@ def _ctx() -> RequestContext:def _make_viking_fs(*, enable_intent: bool) -> VikingFS:fs = VikingFS.__new__(VikingFS)fs.agfs = MagicMock()+ fs.acl_manager = Nonefs.query_embedder = MagicMock(name="embedder")fs.rerank_config = Nonefs.retrieval_config = RetrievalConfig(enable_intent=enable_intent)fs.vector_store = MagicMock(name="vector_store")fs._bound_ctx = contextvars.ContextVar("vikingfs_bound_ctx_intent_test", default=None)- fs._ensure_access = MagicMock()+ fs._ensure_access = AsyncMock()fs._get_vector_store = MagicMock(return_value=fs.vector_store)fs._get_embedder = MagicMock(return_value=fs.query_embedder)fs._ctx_or_default = MagicMock(return_value=_ctx())diff --git a/tests/retrieve/test_hierarchical_retriever_image.py b/tests/retrieve/test_hierarchical_retriever_image.pyindex 5dd5e2547..1c3591c8f 100644--- a/tests/retrieve/test_hierarchical_retriever_image.py+++ b/tests/retrieve/test_hierarchical_retriever_image.py@@ -71,6 +71,7 @@ async def test_image_query_uses_multimodal_input_without_filtering_non_images(mo)embedder = MultimodalEmbedder()retriever = HierarchicalRetriever(storage=object(), embedder=embedder)+ retriever._rerank_client = object()query_input = [{"type": "image_url", "image_url": {"url": "data:image/png;base64,abc"}}]result = await retriever.retrieve(@@ -92,7 +93,7 @@ async def test_image_query_uses_multimodal_input_without_filtering_non_images(mo]assert FakeProxy.captured["context_type"] == "resource"assert FakeProxy.captured["level"] == [2]- assert FakeProxy.captured["limit"] == 50+ assert FakeProxy.captured["limit"] == 2@pytest.mark.asynciodiff --git a/tests/retrieve/test_hierarchical_retriever_rerank.py b/tests/retrieve/test_hierarchical_retriever_rerank.pyindex 059c014f3..4a9f9806b 100644--- a/tests/retrieve/test_hierarchical_retriever_rerank.py+++ b/tests/retrieve/test_hierarchical_retriever_rerank.py@@ -1,12 +1,11 @@# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.# SPDX-License-Identifier: AGPL-3.0-"""Hierarchical retriever rerank behavior tests."""+"""Global retrieval and final rerank behavior tests."""import asyncioimport threadingimport time-from types import SimpleNamespaceimport pytest@@ -50,160 +49,29 @@ class DummyEmbedder:class DummyStorage:- def __init__(self) -> None:- self.collection_name = "context"- self.acl_manager = None- self.search_calls = []- self.child_search_calls = []+ collection_name = "context"- async def _acl_enabled(self, ctx: RequestContext) -> bool:- return self.acl_manager is not None and await self.acl_manager.is_enabled(ctx.account_id)+ def __init__(self, results=()):+ self.results = list(results)+ self.search_calls = []async def get_account_backend(self, account_id):assert account_idreturn self- async def collection_exists(self) -> bool:+ async def collection_exists(self):return True- async def search_in_tenant(- self,- ctx,- query_vector=None,- sparse_query_vector=None,- context_type=None,- target_directories=None,- extra_filter=None,- level=None,- limit: int = 10,- offset: int = 0,- ):- self.search_calls.append(- {- "ctx": ctx,- "query_vector": query_vector,- "sparse_query_vector": sparse_query_vector,- "context_type": context_type,- "target_directories": target_directories,- "extra_filter": extra_filter,- "level": level,- "limit": limit,- "offset": offset,- }- )- return [- _result("viking://resources/root-a", 0.2, level=1, abstract="root A"),- _result("viking://resources/root-b", 0.8, level=1, abstract="root B"),- ]-- async def search_children_in_tenant(- self,- ctx,- parent_uri: str,- query_vector=None,- sparse_query_vector=None,- context_type=None,- target_directories=None,- extra_filter=None,- limit: int = 10,- ):- self.child_search_calls.append(- {- "ctx": ctx,- "parent_uri": parent_uri,- "query_vector": query_vector,- "sparse_query_vector": sparse_query_vector,- "context_type": context_type,- "target_directories": target_directories,- "extra_filter": extra_filter,- "limit": limit,- }- )- if parent_uri == "viking://resources":- return [- _result("viking://resources/file-a", 0.2, abstract="child A", category="doc"),- _result("viking://resources/file-b", 0.8, abstract="child B", category="doc"),- ]- return []---class QuickSearchStorage(DummyStorage):- def __init__(self, results):- super().__init__()- self.results = list(results)-- async def search_in_tenant(- self,- ctx,- query_vector=None,- sparse_query_vector=None,- context_type=None,- target_directories=None,- extra_filter=None,- level=None,- limit: int = 10,- offset: int = 0,- ):- self.search_calls.append(- {- "ctx": ctx,- "query_vector": query_vector,- "sparse_query_vector": sparse_query_vector,- "context_type": context_type,- "target_directories": target_directories,- "extra_filter": extra_filter,- "level": level,- "limit": limit,- "offset": offset,- }- )- return [+ async def search_in_tenant(self, ctx, **kwargs):+ self.search_calls.append({"ctx": ctx, **kwargs})+ level = kwargs.get("level")+ results = [dict(result)for result in self.resultsif level is None or result.get("level", 2) in level]-- async def search_children_in_tenant(- self,- ctx,- parent_uri: str,- query_vector=None,- sparse_query_vector=None,- context_type=None,- target_directories=None,- extra_filter=None,- limit: int = 10,- ):- self.child_search_calls.append(- {- "ctx": ctx,- "parent_uri": parent_uri,- "query_vector": query_vector,- "sparse_query_vector": sparse_query_vector,- "context_type": context_type,- "target_directories": target_directories,- "extra_filter": extra_filter,- "limit": limit,- }- )- return [_result(f"{parent_uri}/should-not-be-returned", 1.0, abstract="child")]---class DirectChildProxy:- async def search_children_in_tenant(- self,- parent_uri: str,- query_vector=None,- sparse_query_vector=None,- context_type=None,- target_directories=None,- extra_filter=None,- limit: int = 10,- ):- return [- _result(f"{parent_uri}/file-a", 0.2, abstract="child A"),- _result(f"{parent_uri}/file-b", 0.8, abstract="child B"),- ]+ results.sort(key=lambda result: result["_score"], reverse=True)+ return results[: kwargs["limit"]]class FakeRerankClient:@@ -232,24 +100,6 @@ def _config() -> RerankConfig:return RerankConfig(ak="ak", sk="sk", threshold=0.1)-def test_retriever_initializes_rerank_client(monkeypatch):- fake_client = FakeRerankClient([0.9, 0.1])-- monkeypatch.setattr(- "openviking.retrieve.hierarchical_retriever.RerankClient.from_config",- lambda config: fake_client,- )-- storage = DummyStorage()- retriever = HierarchicalRetriever(- storage=storage,- embedder=DummyEmbedder(),- rerank_config=_config(),- )-- assert retriever._rerank_client is fake_client--def test_rerank_max_input_tokens_accepts_zero_or_at_least_128():assert RerankConfig(max_input_tokens=0).max_input_tokens == 0with pytest.raises(ValueError, match="max_input_tokens"):@@ -257,29 +107,43 @@ def test_rerank_max_input_tokens_accepts_zero_or_at_least_128():@pytest.mark.asyncio-async def test_retrieve_uses_rerank_scores_in_thinking_mode(monkeypatch):- fake_client = FakeRerankClient([0.95, 0.05, 0.11, 0.95])[email protected]("mode", [None, RetrieverMode.THINKING])+async def test_retrieve_reranks_global_candidates_once(monkeypatch, mode):+ fake_client = FakeRerankClient([0.1, 0.2, 0.95, 0.99])monkeypatch.setattr("openviking.retrieve.hierarchical_retriever.RerankClient.from_config",lambda config: fake_client,)-- storage = DummyStorage()- retriever = HierarchicalRetriever(- storage=storage,- embedder=DummyEmbedder(),- rerank_config=_config(),+ storage = DummyStorage(+ [+ _result("viking://resources/root", 0.95, level=0, abstract="root abstract"),+ _result("viking://resources/dir", 0.85, level=1, abstract="dir overview"),+ _result("viking://resources/file-c", 0.2, abstract="file C"),+ _result("viking://resources/file-d", 0.05, abstract="file D"),+ _result("viking://resources/outside-pool", 0.01, abstract="outside pool"),+ ])+ retriever = HierarchicalRetriever(storage, DummyEmbedder(), rerank_config=_config())+ query = _query()+ query.target_directories = ["viking://resources"]+ scope = {"op": "must", "field": "category", "conds": ["doc"]}- result = await retriever.retrieve(_query(), ctx=_ctx(), limit=2, mode=RetrieverMode.THINKING)+ result = await retriever.retrieve(query, ctx=_ctx(), limit=2, mode=mode, scope_dsl=scope)assert [ctx.uri for ctx in result.matched_contexts] == [- "viking://resources/file-b",- "viking://resources/file-a",+ "viking://resources/file-d",+ "viking://resources/file-c",]- assert fake_client.calls[0] == ("hello", ["root A", "root B"])- assert fake_client.calls[1] == ("hello", ["child A", "child B"])- assert storage.search_calls[0]["level"] == [0, 1]+ assert [ctx.score for ctx in result.matched_contexts] == [0.99, 0.95]+ assert fake_client.calls == [("hello", ["root abstract", "dir overview", "file C", "file D"])]+ assert len(storage.search_calls) == 1+ assert storage.search_calls[0]["limit"] == 4+ assert storage.search_calls[0]["level"] is None+ assert storage.search_calls[0]["target_directories"] == ["viking://resources"]+ assert storage.search_calls[0]["extra_filter"] == scope+ assert storage.search_calls[0]["ctx"] == _ctx()+ assert storage.search_calls[0]["query_vector"] == [1.0]+ assert storage.search_calls[0]["sparse_query_vector"] == {"hello": 1.0}@pytest.mark.asyncio@@ -362,45 +226,39 @@ async def test_rerank_scores_bounds_oversized_documents_and_preserves_tail(@pytest.mark.asyncio-async def test_retrieve_falls_back_to_vector_scores_when_rerank_returns_none(monkeypatch):- class NoneRerankClient(FakeRerankClient):- def rerank_batch(self, query: str, documents: list[str]):[email protected]("failure", [None, [0.9], RuntimeError("provider unavailable")])+async def test_retrieve_falls_back_to_vector_scores_when_rerank_fails(monkeypatch, failure):+ class FailedRerankClient(FakeRerankClient):+ def rerank_batch(self, query, documents):self.calls.append((query, list(documents)))- return None+ if isinstance(failure, Exception):+ raise failure+ return failure- fake_client = NoneRerankClient([])+ fake_client = FailedRerankClient([])monkeypatch.setattr("openviking.retrieve.hierarchical_retriever.RerankClient.from_config",lambda config: fake_client,)-- storage = QuickSearchStorage([- _result("viking://resources/a/deep-a.md", 0.2, abstract="deep A"),- _result("viking://resources/b/deep-b.md", 0.8, abstract="deep B"),- ])- async def acl_enabled(_account_id):- return True-- storage.acl_manager = SimpleNamespace(is_enabled=acl_enabled)-- async def no_hierarchical_children(*_args, **_kwargs):- return []-- storage.search_children_in_tenant = no_hierarchical_children- retriever = HierarchicalRetriever(- storage=storage,- embedder=DummyEmbedder(),- rerank_config=_config(),+ storage = DummyStorage(+ [+ _result("viking://resources/a/deep-a.md", 0.2, abstract="deep A"),+ _result("viking://resources/b/deep-b.md", 0.8, abstract="deep B"),+ _result("viking://resources/b/deep-c.md", 0.05, abstract="deep C"),+ ])+ retriever = HierarchicalRetriever(storage, DummyEmbedder(), rerank_config=_config())- result = await retriever.retrieve(_query(), ctx=_ctx(), limit=2, mode=RetrieverMode.THINKING)+ result = await retriever.retrieve(_query(), ctx=_ctx(), limit=2)assert [ctx.uri for ctx in result.matched_contexts] == ["viking://resources/b/deep-b.md","viking://resources/a/deep-a.md",]- assert [call["level"] for call in storage.search_calls] == [[0, 1], [2]]- assert fake_client.calls+ assert [ctx.score for ctx in result.matched_contexts] == [0.8, 0.2]+ assert len(storage.search_calls) == 1+ assert storage.search_calls[0]["limit"] == 4+ assert fake_client.calls == [("hello", ["deep B", "deep A", "deep C"])]@pytest.mark.asyncio@@ -436,50 +294,44 @@ async def test_rerank_scores_runs_blocking_client_off_event_loop():@pytest.mark.asyncio-async def test_quick_mode_uses_single_vector_search_without_rerank_or_recursion(monkeypatch):- fake_client = FakeRerankClient([0.05, 0.95, 0.95])+ "rerank_config, mode",+ [(None, RetrieverMode.THINKING), (RerankConfig(), None), (_config(), RetrieverMode.QUICK)],+)+async def test_retrieve_without_rerank_does_not_expand_candidates(monkeypatch, rerank_config, mode):+ fake_client = FakeRerankClient([])monkeypatch.setattr("openviking.retrieve.hierarchical_retriever.RerankClient.from_config",lambda config: fake_client,)- storage = QuickSearchStorage(+ storage = DummyStorage([_result("viking://resources/root", 0.95, level=0, abstract="root abstract"),_result("viking://resources/file", 0.9, abstract="file abstract"),_result("viking://resources/dir", 0.85, level=1, abstract="dir overview"),+ _result("viking://resources/extra", 0.8, abstract="extra"),])+ retriever = HierarchicalRetriever(storage, DummyEmbedder(), rerank_config=rerank_config)- retriever = HierarchicalRetriever(- storage=storage,- embedder=DummyEmbedder(),- rerank_config=_config(),- )-- result = await retriever.retrieve(_query(), ctx=_ctx(), limit=3, mode=RetrieverMode.QUICK)+ result = await retriever.retrieve(_query(), ctx=_ctx(), limit=3, mode=mode)assert [ctx.uri for ctx in result.matched_contexts] == ["viking://resources/root/.abstract.md","viking://resources/file","viking://resources/dir/.overview.md",]+ assert fake_client.calls == []assert [ctx.level for ctx in result.matched_contexts] == [0, 2, 1]- assert [ctx.score for ctx in result.matched_contexts] == [- pytest.approx(0.95),- pytest.approx(0.9),- pytest.approx(0.85),- ]+ assert [ctx.score for ctx in result.matched_contexts] == [0.95, 0.9, 0.85]assert len(storage.search_calls) == 1- assert storage.search_calls[0]["limit"] == retriever.GLOBAL_SEARCH_TOPK- assert storage.search_calls[0]["extra_filter"] is None+ assert storage.search_calls[0]["limit"] == 3assert storage.search_calls[0]["level"] is None- assert storage.child_search_calls == []- assert fake_client.calls == []@pytest.mark.asyncio-async def test_quick_mode_pushes_explicit_level_filter_to_vector_search():- storage = QuickSearchStorage(+async def test_retrieve_pushes_explicit_level_filter_to_vector_search():+ storage = DummyStorage([_result("viking://resources/root", 0.99, level=0, abstract="root abstract"),_result("viking://resources/dir", 0.98, level=1, abstract="dir overview"),@@ -497,7 +349,6 @@ async def test_quick_mode_pushes_explicit_level_filter_to_vector_search():_query(),ctx=_ctx(),limit=3,- mode=RetrieverMode.QUICK,scope_dsl={"op": "must", "field": "category", "conds": ["doc"]},level=[2],)@@ -507,19 +358,18 @@ async def test_quick_mode_pushes_explicit_level_filter_to_vector_search():"viking://resources/file-a",]assert len(storage.search_calls) == 1- assert storage.search_calls[0]["limit"] == retriever.GLOBAL_SEARCH_TOPK+ assert storage.search_calls[0]["limit"] == 3assert storage.search_calls[0]["extra_filter"] == {"op": "must","field": "category","conds": ["doc"],}assert storage.search_calls[0]["level"] == [2]- assert storage.child_search_calls == []@pytest.mark.asyncio-async def test_quick_mode_threshold_uses_raw_vector_score():- storage = QuickSearchStorage(+async def test_retrieve_without_rerank_filters_by_vector_score():+ storage = DummyStorage([_result("viking://resources/high", 0.91, abstract="high"),_result("viking://resources/exact", 0.9, abstract="exact"),@@ -535,14 +385,12 @@ async def test_quick_mode_threshold_uses_raw_vector_score():_query(),ctx=_ctx(),limit=2,- mode=RetrieverMode.QUICK,score_threshold=0.9,)inclusive_result = await retriever.retrieve(_query(),ctx=_ctx(),limit=2,- mode=RetrieverMode.QUICK,score_threshold=0.9,score_gte=True,)@@ -555,12 +403,12 @@ async def test_quick_mode_threshold_uses_raw_vector_score():@pytest.mark.asyncio-async def test_quick_mode_keeps_scores_pure_when_hotness_and_propagation_configured(monkeypatch):+async def test_retrieve_without_rerank_keeps_vector_scores_when_hotness_configured(monkeypatch):monkeypatch.setattr("openviking.retrieve.hierarchical_retriever.hotness_score",- lambda *args, **kwargs: pytest.fail("hotness_score should not be called in QUICK mode"),+ lambda *args, **kwargs: pytest.fail("hotness_score should not be called without rerank"),)- storage = QuickSearchStorage(+ storage = DummyStorage([_result("viking://resources/file-a",@@ -575,36 +423,12 @@ async def test_quick_mode_keeps_scores_pure_when_hotness_and_propagation_configustorage=storage,embedder=DummyEmbedder(),rerank_config=None,- retrieval_config=RetrievalConfig(hotness_alpha=0.5, score_propagation_alpha=0.1),+ retrieval_config=RetrievalConfig(hotness_alpha=0.5),)- result = await retriever.retrieve(_query(), ctx=_ctx(), limit=1, mode=RetrieverMode.QUICK)+ result = await retriever.retrieve(_query(), ctx=_ctx(), limit=1)assert result.matched_contexts[0].score == pytest.approx(0.8)- assert storage.child_search_calls == []---async def test_score_propagation_alpha_uses_configured_weight():- retriever = HierarchicalRetriever(- storage=DummyStorage(),- embedder=None,- rerank_config=None,- retrieval_config=RetrievalConfig(score_propagation_alpha=1.0),- )-- candidates = await retriever._recursive_search(- vector_proxy=DirectChildProxy(),- query="hello",- query_vector=None,- sparse_query_vector=None,- starting_points=[("viking://resources", 0.4)],- limit=1,- mode=RetrieverMode.QUICK,- )-- assert candidates[0]["uri"] == "viking://resources/file-b"- assert candidates[0]["_final_score"] == pytest.approx(0.8)@pytest.mark.asynciodiff --git a/tests/retrieve/test_skill_mixed_pagination.py b/tests/retrieve/test_skill_mixed_pagination.pyindex cb144a379..be876eee8 100644--- a/tests/retrieve/test_skill_mixed_pagination.py+++ b/tests/retrieve/test_skill_mixed_pagination.py@@ -25,8 +25,6 @@ async def test_general_search_preserves_multiple_hits_from_one_skill(route):)matches = result.skillselse:- # Thinking obtains files through the ACL-aware global leaf query.- store._acl_enabled = AsyncMock(return_value=True)result = await HierarchicalRetriever(store, None).retrieve(TypedQuery("hello", None, ""),ctx(),diff --git a/tests/retrieve/test_skill_package_results.py b/tests/retrieve/test_skill_package_results.pyindex c3be9cb2d..88c761329 100644--- a/tests/retrieve/test_skill_package_results.py+++ b/tests/retrieve/test_skill_package_results.py@@ -54,11 +54,9 @@ class Files:class PagedStore:collection_name = "context"- def __init__(self, records=(), children=None):+ def __init__(self, records=()):self.records = list(records)- self.children = children or {}self.calls = []- self.child_calls = []async def _acl_enabled(self, ctx):return False@@ -98,10 +96,6 @@ class PagedStore:self.calls.append(kwargs)return self._page(self.records, kwargs)- async def search_children_in_tenant(self, ctx, parent_uri, **kwargs):- self.child_calls.append({"parent_uri": parent_uri, **kwargs})- return self._page(self.children.get(parent_uri, []), kwargs)-def query(target=SKILLS):return TypedQuery("backup recovery", ContextType.SKILL, "", target_directories=[target])diff --git a/tests/test_config_loader.py b/tests/test_config_loader.pyindex bf5042242..0f73728f3 100644--- a/tests/test_config_loader.py+++ b/tests/test_config_loader.py@@ -450,7 +450,6 @@ def test_openviking_config_retrieval_hotness_alpha_defaults_to_zero(monkeypatch)config = OpenVikingConfig.from_dict({})assert config.retrieval.hotness_alpha == 0.0- assert config.retrieval.score_propagation_alpha == 1.0assert config.storage.transaction.redo_recovery_enabled is TrueOpenVikingConfigSingleton.reset_instance()@@ -473,8 +472,7 @@ def test_openviking_config_transaction_redo_recovery_enabled_can_be_disabled(monOpenVikingConfigSingleton.reset_instance()[email protected]("field_name", ["hotness_alpha", "score_propagation_alpha"])-def test_openviking_config_retrieval_alpha_validates_range(monkeypatch, field_name):+def test_openviking_config_retrieval_alpha_validates_range(monkeypatch):monkeypatch.setenv(OPENVIKING_CONFIG_ENV, "/tmp/codex-no-config.json")from openviking_cli.utils.config.open_viking_config import (@@ -483,7 +481,7 @@ def test_openviking_config_retrieval_alpha_validates_range(monkeypatch, field_na)with pytest.raises(ValueError):- OpenVikingConfig.from_dict({"retrieval": {field_name: 1.5}})+ OpenVikingConfig.from_dict({"retrieval": {"hotness_alpha": 1.5}})OpenVikingConfigSingleton.reset_instance()