Skip to content

Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.

refactor(agent): consolidate knowledge retrieval tools into search_knowledge, read_document, list_documents (#3381)

lyingbug92e7c0ab37bb
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 26049574b..2ffc17b54 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,14 @@
All notable changes to this project will be documented in this file.
+## [Unreleased]
+
+### Improvements
+
+- **IMPROVED**: **Agent knowledge-retrieval tools consolidated** — `knowledge_search` and `grep_chunks` are replaced by a single `search_knowledge` tool (`query`, `mode` = `hybrid` | `semantic` | `keyword`, optional `knowledge_base_ids`, `limit` default 10 / max 30); keyword mode is served by the keyword index (BM25 / engine keyword retrieval) instead of an unindexed regular-expression scan of the chunks table. `list_knowledge_chunks`, `get_document_info` and `wiki_read_source_doc` are replaced by `read_document` (`id` = a `dN` document or `cN` chunk handle, `offset` / `limit` paging, in-document `query` with optional `regex`, `context` neighbours around a chunk), which always returns the document metadata header together with the chunks. New `list_documents` (`knowledge_base_id`, `keyword`, `page`, `page_size`) pages through one knowledge base. `wiki_search` takes a single `query` (a case-insensitive regular expression, as before) plus optional `regex`, `knowledge_base_ids` and `limit`; the old `queries` / `knowledge_base_id` parameters are still accepted. `query_knowledge_graph` is only offered when a graph-enabled knowledge base is in scope. New agents default to `search_knowledge`, `read_document`, `list_documents` and `search_conversations`. The built-in MCP server keeps its tool names: `search_knowledge` gains `mode` and `limit`, `grep_chunks` keeps its regular-expression contract on index-backed retrieval and gains `limit`, `read_document` gains `query`, and `wiki_search` gains `regex`.
+- **IMPROVED**: **Retrieval tool compatibility fixes** — `search_knowledge` no longer refuses a mode that some bases cannot serve: FAQ and vector-only bases answer `keyword` requests semantically and keyword-only bases answer `semantic` requests by keyword, with `requested_mode` / `mode_fallbacks` in the result and `<mode_fallback>` in the model rendering, which now shows the mode actually used instead of always `semantic`. `read_document` and `list_documents` are offered on wiki-only scopes (chunks are always stored) without widening RAG agents' derived KB filter; `list_documents` applies the @file / @tag scope before paging so totals and pages only count readable documents; chunk context is resolved by chunk index so parent, summary and image chunks no longer shift the neighbourhood. `wiki_search` (agent and MCP) keeps its regular-expression semantics by default and only falls back to a literal match for text that is not a valid expression; MCP `grep_chunks` keeps grep semantics by fetching candidates with the pattern's literal terms and verifying each chunk against the pattern.
+- **IMPROVED**: **Backward-compatible tool aliasing** — old tool names stored in agent configurations, presets and API calls are mapped to their successors at run time (`legacyToolSuccessors` / `NormalizeAllowedTools`); no data migration is needed, and stored chat history that references the old tool names still renders.
+
## [0.8.0] - 2026-09-03
### New Features
diff --git a/config/agent_type_presets.yaml b/config/agent_type_presets.yaml
index f1715dd84..3bb68addb 100644
--- a/config/agent_type_presets.yaml
+++ b/config/agent_type_presets.yaml
@@ -55,14 +55,14 @@ agent_type_presets:
# users don't need them for single-turn Q&A and they noticeably bloat
# token usage. Enable them manually in the tool list when needed.
allowed_tools:
- - "knowledge_search"
- - "grep_chunks"
- - "list_knowledge_chunks"
- - "get_document_info"
+ - "search_knowledge"
+ - "read_document"
+ - "list_documents"
retain_retrieval_history: false
faq_priority_enabled: true
kb_selection_mode: "all"
# kb_filter derived from allowed_tools: any_of = [vector, keyword]
+ # (read_document / list_documents are auxiliary and do not widen it)
# --------------------------------------------------------------------------
# Wiki Q&A — graph-navigated wiki answering.
@@ -92,7 +92,7 @@ agent_type_presets:
allowed_tools:
- "wiki_search"
- "wiki_read_page"
- - "wiki_read_source_doc"
+ - "read_document"
- "wiki_flag_issue"
retain_retrieval_history: false
kb_selection_mode: "all"
@@ -126,10 +126,9 @@ agent_type_presets:
allowed_tools:
- "wiki_search"
- "wiki_read_page"
- - "knowledge_search"
- - "grep_chunks"
- - "list_knowledge_chunks"
- - "get_document_info"
+ - "search_knowledge"
+ - "read_document"
+ - "list_documents"
- "wiki_flag_issue"
retain_retrieval_history: false
faq_priority_enabled: true
@@ -139,7 +138,7 @@ agent_type_presets:
# - KB has wiki + chunks → full "wiki-first, RAG-verify" workflow
# - KB has only chunks → fallback to pure RAG (see prompt's "Fallback rules")
# - KB has only wiki → fallback to wiki-only answering
- # - FAQ KB → allowed; knowledge_search still works on FAQ chunks
+ # - FAQ KB → allowed; search_knowledge still works on FAQ chunks
# kb_filter derived from allowed_tools: any_of = [vector, keyword, wiki].
# We only require that the KB expose AT LEAST ONE retrievable surface so
# the agent has SOMETHING to call. KBs with zero indexing (e.g. graph-only
diff --git a/config/builtin_agents.yaml b/config/builtin_agents.yaml
index 6716fba71..7f8cb94b3 100644
--- a/config/builtin_agents.yaml
+++ b/config/builtin_agents.yaml
@@ -75,11 +75,10 @@ builtin_agents:
kb_selection_mode: "all"
retrieve_kb_only_when_mentioned: false
allowed_tools:
- - "knowledge_search"
- - "grep_chunks"
- - "list_knowledge_chunks"
+ - "search_knowledge"
+ - "read_document"
+ - "list_documents"
- "query_knowledge_graph"
- - "get_document_info"
web_search_enabled: true
web_search_max_results: 5
reflection_enabled: false
@@ -170,7 +169,7 @@ builtin_agents:
allowed_tools:
- "wiki_search"
- "wiki_read_page"
- - "wiki_read_source_doc"
+ - "read_document"
- "wiki_flag_issue"
web_search_enabled: false
web_search_max_results: 0
@@ -215,7 +214,7 @@ builtin_agents:
allowed_tools:
- "wiki_search"
- "wiki_read_page"
- - "wiki_read_source_doc"
+ - "read_document"
- "wiki_write_page"
- "wiki_replace_text"
- "wiki_rename_page"
diff --git a/config/prompt_templates/agent_system_prompt.yaml b/config/prompt_templates/agent_system_prompt.yaml
index 9cd443936..871dd8ca2 100644
--- a/config/prompt_templates/agent_system_prompt.yaml
+++ b/config/prompt_templates/agent_system_prompt.yaml
@@ -45,8 +45,8 @@ templates:
You are WeKnora, an assistant developed by Tencent, specializing in questions and work involving the user's knowledge bases.
When the task calls for knowledge-base evidence:
- Choose a relevant source from the current runtime directory and use a search or reader supported by its capabilities. A knowledge base's <description> states what it is for and its <generated_profile> (gist, topics, typical questions) states what it actually contains; when several are bound, prefer the ones whose profile matches the question and pass their ids to the search tool instead of searching all of them.
- - Use semantic search for concepts and paraphrases, and keyword search for exact terms, error messages, or identifiers.
- - Read additional context when returned passages are incomplete or insufficient. Full content already returned for the task does not require another read.
+ - Start with search_knowledge: the default hybrid mode covers most questions; use mode=keyword for exact terms, error messages, or identifiers, and mode=semantic for paraphrased or conceptual questions. Rephrase or switch mode rather than repeating an unproductive query.
+ - Use read_document(id=dN) when a returned passage is incomplete: page with offset, or pass query to find a passage inside the document. Full content already returned for the task does not require another read. list_documents shows what a knowledge base contains when the search surface is unclear.
- For exact quotes, numbers, or code, verify the underlying source passage rather than relying on document metadata or a generated overview.
Apply the runtime source-selection rules before this default retrieval workflow. Other requested sources and deliverables are part of the user's task.
@@ -99,7 +99,7 @@ templates:
For a task that needs Wiki evidence:
- Find an entry point with wiki_search, or read the index when the user requests an overview. Restrict the lookup to the relevant bound knowledge base.
- Read a relevant page with wiki_read_page when search summaries do not contain enough evidence. Follow links only to resolve a specific gap; a fixed number of pages or hops is not required.
- - For exact quotes, detailed code, or disputed facts, inspect the original documents referenced by the page using an available reader. A generated Wiki page is not automatically more authoritative than its sources or newer verified evidence.
+ - For exact quotes, detailed code, or disputed facts, inspect the original documents referenced by the page with read_document(id=dN) using the source handles the page lists. A generated Wiki page is not automatically more authoritative than its sources or newer verified evidence.
- If sources conflict, compare their dates, scope, and provenance and explain unresolved differences. Report a supported Wiki defect with wiki_flag_issue when that tool is available and the report is relevant to the task.
- Use verified [[slug|display name]] links when Wiki navigation helps the user. Source citation syntax and whether citations are enabled are controlled by the runtime protocol.
This is a default research workflow, not a requirement to search the Wiki for every message. Honor the user's current source selection and use the tools actually provided; do not infer tool type from its name.
@@ -152,7 +152,7 @@ templates:
You are WeKnora, an assistant that uses Wiki pages and original document chunks to answer questions and complete knowledge-based work.
When knowledge-base retrieval is appropriate:
- Use the runtime directory's capabilities to choose an available surface. A missing capability is unknown, not permission to assume a chunk or Wiki index exists.
- - Prefer Wiki for conceptual overviews and relationships; prefer original chunks for exact quotes, numbers, code, and details. Explicit source or document selections take precedence over this default.
+ - Prefer Wiki (wiki_search, then wiki_read_page) for conceptual overviews and relationships; prefer original chunks (search_knowledge, then read_document) for exact quotes, numbers, code, and details. Explicit source or document selections take precedence over this default.
- Search the relevant scope. Read additional pages or neighboring chunks only if the returned material leaves a gap; a complete passage need not be fetched again merely because it arrived through search.
- If a lookup fails to find relevant material, change the query or try another relevant available surface when it is likely to help. Stop once evidence is sufficient or further searches are unlikely to resolve the gap.
- When Wiki and source documents disagree, compare their provenance, dates, and scope. Do not assume either surface is always correct. Report a supported Wiki defect through wiki_flag_issue if available and relevant.
diff --git a/frontend/src/i18n/locales/en-US.ts b/frontend/src/i18n/locales/en-US.ts
index ae9eaec0d..7b7262305 100755
--- a/frontend/src/i18n/locales/en-US.ts
+++ b/frontend/src/i18n/locales/en-US.ts
@@ -5696,6 +5696,9 @@ export default {
},
tools: {
multiKbSearch: 'Cross-KB Search',
+ searchKnowledge: 'Search knowledge',
+ readDocument: 'Read document',
+ listDocuments: 'List documents',
knowledgeSearch: 'Knowledge Search',
grepChunks: 'Text Pattern Search',
getChunkDetail: 'Get Chunk Detail',
@@ -5847,6 +5850,8 @@ export default {
getRelatedDocuments: 'Find Related Documents',
getDocumentContent: 'Get Document Content',
wikiReadSourceDoc: 'Deep-read Source Doc',
+ readDocument: 'Read document',
+ listDocuments: 'List documents',
todoWrite: 'Plan Management',
knowledgeGraphExtract: 'Knowledge Graph Extraction',
thinking: 'Thinking',
@@ -6122,6 +6127,12 @@ export default {
thinkingDesc: 'Dynamic and reflective problem-solving thinking tool',
todoWrite: 'Plan',
todoWriteDesc: 'Create structured research plans',
+ searchKnowledge: 'Search knowledge',
+ searchKnowledgeDesc: 'Semantic, keyword or hybrid search over knowledge-base chunks',
+ readDocument: 'Read document',
+ readDocumentDesc: 'Read a document\'s metadata and chunks, with paging and in-document search',
+ listDocuments: 'List documents',
+ listDocumentsDesc: 'Page through the documents of a knowledge base',
grepChunks: 'Keyword Search',
grepChunksDesc: 'Quickly locate documents and chunks containing specific keywords',
knowledgeSearch: 'Semantic Search',
diff --git a/frontend/src/i18n/locales/ja-JP.ts b/frontend/src/i18n/locales/ja-JP.ts
index 866e9cc63..4ce7e273a 100644
--- a/frontend/src/i18n/locales/ja-JP.ts
+++ b/frontend/src/i18n/locales/ja-JP.ts
@@ -5696,6 +5696,9 @@ export default {
},
tools: {
multiKbSearch: 'ナレッジベース横断検索',
+ searchKnowledge: 'ナレッジベースを検索',
+ readDocument: 'ドキュメントを読む',
+ listDocuments: 'ドキュメント一覧',
knowledgeSearch: 'ナレッジベース検索',
grepChunks: 'テキストパターン検索',
getChunkDetail: 'チャンク詳細の取得',
@@ -5847,6 +5850,8 @@ export default {
getRelatedDocuments: '関連ドキュメントを検索',
getDocumentContent: 'ドキュメント内容を取得',
wikiReadSourceDoc: 'ソースドキュメントを精読',
+ readDocument: 'ドキュメントを読む',
+ listDocuments: 'ドキュメント一覧',
todoWrite: '計画管理',
knowledgeGraphExtract: 'ナレッジグラフ抽出',
thinking: '思考',
@@ -6122,6 +6127,12 @@ export default {
thinkingDesc: '動的かつ内省的に問題を解決する思考ツール',
todoWrite: '計画',
todoWriteDesc: '構造化された調査計画を作成します',
+ searchKnowledge: 'ナレッジベースを検索',
+ searchKnowledgeDesc: 'セマンティック・キーワード・ハイブリッド検索でナレッジベースのチャンクを検索します',
+ readDocument: 'ドキュメントを読む',
+ readDocumentDesc: 'ドキュメントのメタデータとチャンク内容を読み取ります。ページングと文書内検索に対応',
+ listDocuments: 'ドキュメント一覧',
+ listDocumentsDesc: 'ナレッジベース内のドキュメントをページ単位で一覧表示します',
grepChunks: 'キーワード検索',
grepChunksDesc: '特定のキーワードを含むドキュメントとチャンクを素早く特定します',
knowledgeSearch: 'セマンティック検索',
diff --git a/frontend/src/i18n/locales/ko-KR.ts b/frontend/src/i18n/locales/ko-KR.ts
index cc9f917a4..b39ddb7ff 100755
--- a/frontend/src/i18n/locales/ko-KR.ts
+++ b/frontend/src/i18n/locales/ko-KR.ts
@@ -1302,6 +1302,12 @@ export default {
thinkingDesc: '동적이고 반성적인 문제 해결 사고 도구',
todoWrite: '계획 수립',
todoWriteDesc: '구조화된 연구 계획 생성',
+ searchKnowledge: '지식베이스 검색',
+ searchKnowledgeDesc: '의미, 키워드 또는 하이브리드 방식으로 지식베이스 청크 검색',
+ readDocument: '문서 읽기',
+ readDocumentDesc: '문서 메타데이터와 청크 내용을 읽으며 페이징과 문서 내 검색 지원',
+ listDocuments: '문서 목록 보기',
+ listDocumentsDesc: '지식베이스의 문서를 페이지 단위로 나열',
grepChunks: '키워드 검색',
grepChunksDesc: '특정 키워드를 포함하는 문서와 청크를 빠르게 찾기',
knowledgeSearch: '의미 검색',
@@ -1595,6 +1601,8 @@ export default {
getRelatedDocuments: '관련 문서 찾기',
getDocumentContent: '문서 내용 가져오기',
wikiReadSourceDoc: '원본 문서 읽기',
+ readDocument: '문서 읽기',
+ listDocuments: '문서 목록 보기',
todoWrite: '계획 관리',
knowledgeGraphExtract: '지식 그래프 추출',
thinking: '사고',
@@ -1771,6 +1779,9 @@ export default {
},
tools: {
multiKbSearch: '크로스 KB 검색',
+ searchKnowledge: '지식베이스 검색',
+ readDocument: '문서 읽기',
+ listDocuments: '문서 목록 보기',
knowledgeSearch: '지식베이스 검색',
grepChunks: '텍스트 패턴 검색',
getChunkDetail: '청크 상세 조회',
diff --git a/frontend/src/i18n/locales/ru-RU.ts b/frontend/src/i18n/locales/ru-RU.ts
index 654e2bf0b..93062d939 100755
--- a/frontend/src/i18n/locales/ru-RU.ts
+++ b/frontend/src/i18n/locales/ru-RU.ts
@@ -1302,6 +1302,12 @@ export default {
thinkingDesc: 'Динамический инструмент рефлексивного решения проблем',
todoWrite: 'Планирование',
todoWriteDesc: 'Создание структурированных исследовательских планов',
+ searchKnowledge: 'Поиск по базе знаний',
+ searchKnowledgeDesc: 'Семантический, ключевой или гибридный поиск по фрагментам базы знаний',
+ readDocument: 'Чтение документа',
+ readDocumentDesc: 'Чтение метаданных и фрагментов документа с постраничной навигацией и поиском внутри документа',
+ listDocuments: 'Список документов',
+ listDocumentsDesc: 'Постраничный просмотр документов базы знаний',
grepChunks: 'Поиск по ключевым словам',
grepChunksDesc: 'Быстрый поиск документов и фрагментов с определёнными ключевыми словами',
knowledgeSearch: 'Семантический поиск',
@@ -1595,6 +1601,8 @@ export default {
getRelatedDocuments: 'Поиск связанных документов',
getDocumentContent: 'Получение содержимого документа',
wikiReadSourceDoc: 'Читать исходный документ',
+ readDocument: 'Чтение документа',
+ listDocuments: 'Список документов',
todoWrite: 'Управление планами',
knowledgeGraphExtract: 'Извлечение графа знаний',
thinking: 'Размышление',
@@ -1771,6 +1779,9 @@ export default {
},
tools: {
multiKbSearch: 'Кросс-КБ поиск',
+ searchKnowledge: 'Поиск по базе знаний',
+ readDocument: 'Чтение документа',
+ listDocuments: 'Список документов',
knowledgeSearch: 'Поиск по базе знаний',
grepChunks: 'Поиск по текстовому шаблону',
getChunkDetail: 'Получить детали фрагмента',
diff --git a/frontend/src/i18n/locales/zh-CN.ts b/frontend/src/i18n/locales/zh-CN.ts
index 5aacce250..644ef8fbb 100755
--- a/frontend/src/i18n/locales/zh-CN.ts
+++ b/frontend/src/i18n/locales/zh-CN.ts
@@ -1304,6 +1304,12 @@ export default {
thinkingDesc: '动态和反思性的问题解决思考工具',
todoWrite: '制定计划',
todoWriteDesc: '创建结构化的研究计划',
+ searchKnowledge: '检索知识库',
+ searchKnowledgeDesc: '语义、关键词或混合检索知识库分块',
+ readDocument: '阅读文档',
+ readDocumentDesc: '读取文档元数据与分块内容,支持分页和文内查找',
+ listDocuments: '浏览文档列表',
+ listDocumentsDesc: '分页列出知识库中的文档',
grepChunks: '关键词搜索',
grepChunksDesc: '快速定位包含特定关键词的文档和分块',
knowledgeSearch: '语义搜索',
@@ -1597,6 +1603,8 @@ export default {
getRelatedDocuments: '查找相关文档',
getDocumentContent: '获取文档内容',
wikiReadSourceDoc: '精读源文档',
+ readDocument: '阅读文档',
+ listDocuments: '浏览文档列表',
todoWrite: '计划管理',
knowledgeGraphExtract: '知识图谱抽取',
thinking: '思考',
@@ -1773,6 +1781,9 @@ export default {
},
tools: {
multiKbSearch: '跨库搜索',
+ searchKnowledge: '检索知识库',
+ readDocument: '阅读文档',
+ listDocuments: '浏览文档列表',
knowledgeSearch: '知识库搜索',
grepChunks: '搜索关键词',
getChunkDetail: '获取片段详情',
diff --git a/frontend/src/types/tool-results.ts b/frontend/src/types/tool-results.ts
index 142fda049..2dac12514 100644
--- a/frontend/src/types/tool-results.ts
+++ b/frontend/src/types/tool-results.ts
@@ -135,13 +135,20 @@ export interface DocumentInfoDocument {
type_icon?: string;
}
+// Emitted by list_documents (paged) and by the retired get_document_info tool
+// still present in stored chat history.
export interface DocumentInfoData {
display_type: 'document_info';
documents?: DocumentInfoDocument[];
- total_docs: number;
- requested: number;
+ total_docs?: number;
+ requested?: number;
errors?: string[];
title?: string;
+ // list_documents pagination
+ page?: number;
+ page_size?: number;
+ next_page?: number;
+ knowledge_base_id?: string;
}
// Graph query results data
@@ -289,15 +296,55 @@ export interface GrepResultsData {
max_results: number;
}
-// Knowledge chunks list data (list_knowledge_chunks tool)
+// Chunk row inside a knowledge_chunks_list payload. `role` / `match_snippet`
+// are only set by read_document in query mode (in-document search).
+export interface KnowledgeChunksListChunk {
+ index?: number;
+ chunk_id?: string;
+ chunk_index?: number;
+ content?: string;
+ knowledge_id?: string;
+ knowledge_base_id?: string;
+ chunk_type?: string;
+ role?: 'match' | 'context_before' | 'context_after' | 'focus';
+ match_snippet?: string;
+}
+
+// Document metadata attached to a read_document result.
+export interface KnowledgeChunksListDocument {
+ knowledge_id?: string;
+ title?: string;
+ type?: string;
+ source?: string;
+ file_name?: string;
+ file_type?: string;
+ file_size?: number;
+ parse_status?: string;
+ chunk_count?: number;
+ description?: string;
+ metadata?: Record<string, any>;
+}
+
+// Knowledge chunks list data (read_document tool; also emitted by the retired
+// list_knowledge_chunks / wiki_read_source_doc tools still present in stored
+// chat history)
export interface KnowledgeChunksListData {
display_type: 'knowledge_chunks_list';
knowledge_id?: string;
knowledge_title?: string;
total_chunks?: number;
fetched_chunks?: number;
+ chunks?: KnowledgeChunksListChunk[];
+ document?: KnowledgeChunksListDocument;
+ // Pagination (offset-based for read_document, page-based for legacy tools)
+ offset?: number;
+ next_offset?: number;
page?: number;
page_size?: number;
+ // read_document query mode (in-document search)
+ query?: string;
+ match_count?: number;
+ truncated?: boolean;
faq_question?: string;
faq_id?: string;
single_chunk?: boolean;
diff --git a/frontend/src/utils/agent-readiness.test.mjs b/frontend/src/utils/agent-readiness.test.mjs
index e5fafb382..d8a4f3660 100644
--- a/frontend/src/utils/agent-readiness.test.mjs
+++ b/frontend/src/utils/agent-readiness.test.mjs
@@ -30,14 +30,36 @@ test('rejects a configured chat model that no longer exists', () => {
), false)
})
-test('requires rerank when knowledge_search has a knowledge-base scope', () => {
+test('requires rerank when search_knowledge has a knowledge-base scope', () => {
+ assert.equal(agentRequiresRerankModel({
+ kb_selection_mode: 'all',
+ allowed_tools: ['search_knowledge'],
+ }), true)
+})
+
+test('still requires rerank for configs saved with the retired knowledge_search name', () => {
assert.equal(agentRequiresRerankModel({
kb_selection_mode: 'all',
allowed_tools: ['knowledge_search'],
}), true)
+ assert.equal(agentRequiresRerankModel({
+ kb_selection_mode: 'selected',
+ allowed_tools: ['thinking', 'grep_chunks'],
+ }), true)
+})
+
+test('does not require rerank for read-only document tools', () => {
+ assert.equal(agentRequiresRerankModel({
+ kb_selection_mode: 'all',
+ allowed_tools: ['read_document', 'list_documents'],
+ }), false)
})
test('does not require rerank when knowledge bases are disabled', () => {
+ assert.equal(agentRequiresRerankModel({
+ kb_selection_mode: 'none',
+ allowed_tools: ['search_knowledge'],
+ }), false)
assert.equal(agentRequiresRerankModel({
kb_selection_mode: 'none',
allowed_tools: ['knowledge_search'],
@@ -87,6 +109,11 @@ test('getAgentNotReadyReasonKeys flags missing chat model', () => {
})
test('getAgentNotReadyReasonKeys requires rerank only in agent mode with KB search', () => {
+ assert.deepEqual(getAgentNotReadyReasonKeys(
+ { kb_selection_mode: 'all', allowed_tools: ['search_knowledge'] },
+ [{ id: 'chat-1', type: 'KnowledgeQA' }, { id: 'rerank-1', type: 'Rerank' }],
+ { isAgentMode: true, isSharedAgent: false },
+ ), ['summary_model', 'rerank_model'])
assert.deepEqual(getAgentNotReadyReasonKeys(
{ kb_selection_mode: 'all', allowed_tools: ['knowledge_search'] },
[{ id: 'chat-1', type: 'KnowledgeQA' }, { id: 'rerank-1', type: 'Rerank' }],
diff --git a/frontend/src/utils/agent-readiness.ts b/frontend/src/utils/agent-readiness.ts
index 6b6bebccf..955e57f69 100644
--- a/frontend/src/utils/agent-readiness.ts
+++ b/frontend/src/utils/agent-readiness.ts
@@ -1,5 +1,6 @@
import type { CustomAgentConfig } from '@/api/agent'
import type { ModelConfig } from '@/api/model'
+import { allowedToolsInclude } from './legacy-tool-names'
export type AgentNotReadyReasonKey = 'summary_model' | 'rerank_model' | 'allowed_tools'
@@ -25,9 +26,12 @@ export function agentHasConfiguredChatModel(
/**
* Keep this aligned with backend agentRequiresRerankModel.
*
- * Rerank is needed only when knowledge_search can actually run. Explicitly
+ * Rerank is needed only when search_knowledge can actually run. Explicitly
* disabling the knowledge-base scope makes KB tools ineffective, even if an
- * older agent configuration still contains knowledge_search in allowed_tools.
+ * older agent configuration still contains the tool in allowed_tools.
+ *
+ * Stored configs may still carry the retired `knowledge_search` name (and
+ * `grep_chunks`, which was folded into search_knowledge); those count too.
*/
export function agentRequiresRerankModel(
config: Pick<CustomAgentConfig, 'kb_selection_mode' | 'allowed_tools'> | undefined,
@@ -36,10 +40,10 @@ export function agentRequiresRerankModel(
const allowedTools = config.allowed_tools || []
// The backend falls back to DefaultAllowedTools when the list is empty,
- // and that default includes knowledge_search.
+ // and that default includes search_knowledge.
if (allowedTools.length === 0) return true
- return allowedTools.includes('knowledge_search')
+ return allowedToolsInclude(allowedTools, 'search_knowledge')
}
export function getAgentNotReadyReasonKeys(
diff --git a/frontend/src/utils/agent-tool-display.test.mjs b/frontend/src/utils/agent-tool-display.test.mjs
index d8cbabb94..f482ec84b 100644
--- a/frontend/src/utils/agent-tool-display.test.mjs
+++ b/frontend/src/utils/agent-tool-display.test.mjs
@@ -26,6 +26,15 @@ test('getAgentToolIconName maps rag pipeline tools', () => {
assert.equal(getAgentToolIconName('knowledge_search'), 'data-search')
})
+test('getAgentToolIconName maps the consolidated knowledge tools and their retired names alike', () => {
+ assert.equal(getAgentToolIconName('search_knowledge'), 'data-search')
+ assert.equal(getAgentToolIconName('search_knowledge', 'web'), 'internet')
+ assert.equal(getAgentToolIconName('read_document'), 'file-search')
+ assert.equal(getAgentToolIconName('list_documents'), 'file-search')
+ assert.equal(getAgentToolIconName('list_knowledge_chunks'), 'file-search')
+ assert.equal(getAgentToolIconName('get_document_info'), 'file-search')
+})
+
test('getAgentToolIconName maps sandbox shell tools to the terminal icon', () => {
assert.equal(getAgentToolIconName('shell_exec'), 'terminal')
})
diff --git a/frontend/src/utils/agent-tool-icons.ts b/frontend/src/utils/agent-tool-icons.ts
index 8af7b0371..5bdc8f005 100644
--- a/frontend/src/utils/agent-tool-icons.ts
+++ b/frontend/src/utils/agent-tool-icons.ts
@@ -22,6 +22,10 @@ export function getAgentToolIconName(
if (toolName === 'web_search') {
return 'internet'
}
+ if (toolName === 'read_document' || toolName === 'list_documents') {
+ return 'file-search'
+ }
+ // Retired names still present in stored chat history
if (toolName === 'get_document_info' || toolName === 'list_knowledge_chunks') {
return 'file-search'
}
diff --git a/frontend/src/utils/legacy-tool-names.test.mjs b/frontend/src/utils/legacy-tool-names.test.mjs
new file mode 100644
index 000000000..fbcf0e6d8
--- /dev/null
+++ b/frontend/src/utils/legacy-tool-names.test.mjs
@@ -0,0 +1,56 @@
+import assert from 'node:assert/strict'
+import test from 'node:test'
+import {
+ allowedToolsInclude,
+ normalizeLegacyToolName,
+ normalizeLegacyToolNames,
+} from './legacy-tool-names.ts'
+
+test('normalizeLegacyToolName maps retired retrieval tools to their successors', () => {
+ assert.equal(normalizeLegacyToolName('knowledge_search'), 'search_knowledge')
+ assert.equal(normalizeLegacyToolName('grep_chunks'), 'search_knowledge')
+ assert.equal(normalizeLegacyToolName('list_knowledge_chunks'), 'read_document')
+ assert.equal(normalizeLegacyToolName('get_document_info'), 'read_document')
+ assert.equal(normalizeLegacyToolName('wiki_read_source_doc'), 'read_document')
+})
+
+test('normalizeLegacyToolName leaves current and unknown names untouched', () => {
+ assert.equal(normalizeLegacyToolName('search_knowledge'), 'search_knowledge')
+ assert.equal(normalizeLegacyToolName('list_documents'), 'list_documents')
+ assert.equal(normalizeLegacyToolName('wiki_read_page'), 'wiki_read_page')
+ assert.equal(normalizeLegacyToolName('mcp_foo_bar'), 'mcp_foo_bar')
+})
+
+test('normalizeLegacyToolNames rewrites an old default RAG tool list and dedupes', () => {
+ assert.deepEqual(
+ normalizeLegacyToolNames(['thinking', 'grep_chunks', 'knowledge_search', 'list_knowledge_chunks', 'get_document_info', 'query_knowledge_graph']),
+ ['thinking', 'search_knowledge', 'read_document', 'query_knowledge_graph'],
+ )
+})
+
+test('normalizeLegacyToolNames keeps order and does not duplicate already-migrated names', () => {
+ assert.deepEqual(
+ normalizeLegacyToolNames(['search_knowledge', 'knowledge_search', 'wiki_search', 'wiki_read_source_doc', 'read_document']),
+ ['search_knowledge', 'wiki_search', 'read_document'],
+ )
+})
+
+test('normalizeLegacyToolNames returns a fresh array and tolerates junk input', () => {
+ const input = ['thinking']
+ const out = normalizeLegacyToolNames(input)
+ assert.deepEqual(out, ['thinking'])
+ assert.notEqual(out, input)
+ assert.deepEqual(normalizeLegacyToolNames(undefined), [])
+ assert.deepEqual(normalizeLegacyToolNames(null), [])
+ assert.deepEqual(normalizeLegacyToolNames([]), [])
+ assert.deepEqual(normalizeLegacyToolNames(['', ' ', 42, null, ' todo_write ']), ['todo_write'])
+})
+
+test('allowedToolsInclude matches by current or retired name', () => {
+ assert.equal(allowedToolsInclude(['knowledge_search'], 'search_knowledge'), true)
+ assert.equal(allowedToolsInclude(['search_knowledge'], 'knowledge_search'), true)
+ assert.equal(allowedToolsInclude(['grep_chunks'], 'search_knowledge'), true)
+ assert.equal(allowedToolsInclude(['wiki_search', 'wiki_read_page'], 'search_knowledge'), false)
+ assert.equal(allowedToolsInclude([], 'search_knowledge'), false)
+ assert.equal(allowedToolsInclude(undefined, 'search_knowledge'), false)
+})
diff --git a/frontend/src/utils/legacy-tool-names.ts b/frontend/src/utils/legacy-tool-names.ts
new file mode 100644
index 000000000..834168ce7
--- /dev/null
+++ b/frontend/src/utils/legacy-tool-names.ts
@@ -0,0 +1,57 @@
+/**
+ * Legacy agent tool name normalization.
+ *
+ * The backend consolidated the knowledge-retrieval tools:
+ * - knowledge_search / grep_chunks → search_knowledge
+ * - list_knowledge_chunks / get_document_info /
+ * wiki_read_source_doc → read_document
+ *
+ * Agent configs saved before that change still carry the retired names in
+ * `allowed_tools`. The editor maps them to the new names on load so the tool
+ * checkboxes render correctly; chat history rendering keeps accepting the old
+ * names as-is (see `AgentStreamDisplay.vue`), so this helper is ONLY for
+ * config-shaped tool lists, never for tool-call events.
+ */
+export const LEGACY_TOOL_NAME_MAP: Readonly<Record<string, string>> = Object.freeze({
+ knowledge_search: 'search_knowledge',
+ grep_chunks: 'search_knowledge',
+ list_knowledge_chunks: 'read_document',
+ get_document_info: 'read_document',
+ wiki_read_source_doc: 'read_document',
+})
+
+/** Map a single tool name to its current name (identity for non-legacy names). */
+export function normalizeLegacyToolName(toolName: string): string {
+ return LEGACY_TOOL_NAME_MAP[toolName] ?? toolName
+}
+
+/**
+ * Normalize an `allowed_tools`-style list: retired names are replaced by their
+ * successors, duplicates are dropped (first occurrence wins), order otherwise
+ * preserved. Non-string / empty entries are dropped. Always returns a new array.
+ */
+export function normalizeLegacyToolNames(
+ tools: readonly unknown[] | undefined | null,
+): string[] {
+ if (!Array.isArray(tools) || tools.length === 0) return []
+ const seen = new Set<string>()
+ const out: string[] = []
+ for (const raw of tools) {
+ if (typeof raw !== 'string') continue
+ const name = normalizeLegacyToolName(raw.trim())
+ if (!name || seen.has(name)) continue
+ seen.add(name)
+ out.push(name)
+ }
+ return out
+}
+
+/** True iff the list contains the tool, by current OR retired name. */
+export function allowedToolsInclude(
+ tools: readonly string[] | undefined | null,
+ toolName: string,
+): boolean {
+ if (!tools || tools.length === 0) return false
+ const target = normalizeLegacyToolName(toolName)
+ return tools.some(t => normalizeLegacyToolName(t) === target)
+}
diff --git a/frontend/src/utils/tool-capabilities.test.mjs b/frontend/src/utils/tool-capabilities.test.mjs
new file mode 100644
index 000000000..ce0bce627
--- /dev/null
+++ b/frontend/src/utils/tool-capabilities.test.mjs
@@ -0,0 +1,25 @@
+import assert from 'node:assert/strict'
+import test from 'node:test'
+import { deriveKbFilterFromTools, evaluateToolRequirement } from './tool-capabilities.ts'
+
+const wikiOnly = { vector: false, keyword: false, wiki: true, graph: false, faq: false }
+
+test('document readers are usable on a wiki-only knowledge base', () => {
+ for (const tool of ['read_document', 'list_documents']) {
+ assert.deepEqual(evaluateToolRequirement(tool, wikiOnly, true), { ok: true, missKind: 'none' })
+ }
+ assert.equal(evaluateToolRequirement('search_knowledge', wikiOnly, true).ok, false)
+})
+
+test('document readers do not widen a RAG agent KB filter to wiki-only bases', () => {
+ const rag = deriveKbFilterFromTools(['search_knowledge', 'read_document', 'list_documents'])
+ assert.deepEqual([...rag.any_of].sort(), ['keyword', 'vector'])
+
+ const wiki = deriveKbFilterFromTools(['wiki_search', 'wiki_read_page', 'read_document'])
+ assert.deepEqual(wiki.any_of, ['wiki'])
+
+ const readerOnly = deriveKbFilterFromTools(['read_document'])
+ assert.deepEqual([...readerOnly.any_of].sort(), ['keyword', 'vector', 'wiki'])
+
+ assert.equal(deriveKbFilterFromTools(['thinking']), null)
+})
diff --git a/frontend/src/utils/tool-capabilities.ts b/frontend/src/utils/tool-capabilities.ts
index 7470019f3..6f7ff91a9 100644
--- a/frontend/src/utils/tool-capabilities.ts
+++ b/frontend/src/utils/tool-capabilities.ts
@@ -22,8 +22,9 @@
* - empty : tool has no KB requirement (always available).
*
* IMPORTANT: keep this list aligned with backend tool definitions
- * (`internal/agent/tools/`). A tool missing from this map defaults to
- * "always available" so new tools don't silently start disabled.
+ * (`internal/agent/tools/`; this map mirrors
+ * `internal/agent/tools/capabilities.go`). A tool missing from this map
+ * defaults to "always available" so new tools don't silently start disabled.
*/
export type KBCapability = 'vector' | 'keyword' | 'wiki' | 'graph' | 'faq';
@@ -31,6 +32,14 @@ export type KBCapability = 'vector' | 'keyword' | 'wiki' | 'graph' | 'faq';
export interface ToolRequirement {
anyOf?: KBCapability[];
allOf?: KBCapability[];
+ /**
+ * Auxiliary tools work on the listed KBs but do not by themselves make a KB
+ * worth selecting (a document reader can open a wiki-only KB's sources, yet
+ * must not pull wiki-only KBs into a RAG agent's scope). They only shape
+ * the derived KB filter when no primary tool has a requirement. Mirrors
+ * `ToolRequirement.Auxiliary` in `internal/agent/tools/capabilities.go`.
+ */
+ auxiliary?: boolean;
/**
* Whether this tool can use user-provided file references (via @ 提及) as
* an additional retrieval scope. Tools with `consumesFiles: false` ignore
@@ -49,18 +58,27 @@ export const TOOL_CAPABILITY_REQUIREMENTS: Record<string, ToolRequirement> = {
// We use vector|keyword as the canonical "has RAG chunks" signal. FAQ KBs
// also expose chunks, but the current UX message bucket is "RAG KB"; once
// we add a dedicated `requiresFaqKb` i18n key we can include `faq` here.
+ search_knowledge: { anyOf: ['vector', 'keyword'], consumesFiles: true },
+ // Document readers work on stored chunks, which wiki-only KBs have too.
+ read_document: { anyOf: ['vector', 'keyword', 'wiki'], consumesFiles: true, auxiliary: true },
+ list_documents: { anyOf: ['vector', 'keyword', 'wiki'], consumesFiles: true, auxiliary: true },
+ database_query: { anyOf: ['vector', 'keyword'], consumesFiles: true },
+ // Retired names kept so stale agent configs / stored history still resolve.
+ // The editor normalizes them to the new names on load (see
+ // `@/utils/legacy-tool-names`).
knowledge_search: { anyOf: ['vector', 'keyword'], consumesFiles: true },
grep_chunks: { anyOf: ['vector', 'keyword'], consumesFiles: true },
list_knowledge_chunks: { anyOf: ['vector', 'keyword'], consumesFiles: true },
- query_knowledge_graph: { anyOf: ['vector', 'keyword'], consumesFiles: true },
get_document_info: { anyOf: ['vector', 'keyword'], consumesFiles: true },
- database_query: { anyOf: ['vector', 'keyword'], consumesFiles: true },
+ wiki_read_source_doc: { anyOf: ['vector', 'keyword', 'wiki'], consumesFiles: true, auxiliary: true },
+
+ // ---- Knowledge graph (needs a graph-indexed KB) ----
+ query_knowledge_graph: { allOf: ['graph'], consumesFiles: true },
// ---- Wiki (operates on wiki pages referenced by the wiki machinery;
// arbitrary user-picked file IDs aren't meaningful here) ----
wiki_search: { allOf: ['wiki'] },
wiki_read_page: { allOf: ['wiki'] },
- wiki_read_source_doc: { allOf: ['wiki'] },
wiki_flag_issue: { allOf: ['wiki'] },
wiki_write_page: { allOf: ['wiki'] },
wiki_replace_text: { allOf: ['wiki'] },
@@ -153,13 +171,16 @@ function primaryMissKind(c: KBCapability): RequirementMissKind {
export function deriveKbFilterFromTools(
tools: string[],
): { any_of: KBCapability[] } | null {
- const caps = new Set<KBCapability>();
+ const primary = new Set<KBCapability>();
+ const auxiliary = new Set<KBCapability>();
for (const t of tools) {
const req = TOOL_CAPABILITY_REQUIREMENTS[t];
if (!req) continue;
- req.anyOf?.forEach(c => caps.add(c));
- req.allOf?.forEach(c => caps.add(c));
+ const target = req.auxiliary ? auxiliary : primary;
+ req.anyOf?.forEach(c => target.add(c));
+ req.allOf?.forEach(c => target.add(c));
}
+ const caps = primary.size > 0 ? primary : auxiliary;
if (caps.size === 0) return null;
return { any_of: Array.from(caps) };
}
diff --git a/frontend/src/utils/tool-icons.ts b/frontend/src/utils/tool-icons.ts
index 1d22b126b..d7b99d9d0 100644
--- a/frontend/src/utils/tool-icons.ts
+++ b/frontend/src/utils/tool-icons.ts
@@ -9,15 +9,19 @@ const t = (key: string) => i18n.global.t(key)
// Tool name to icon mapping
export const toolIcons: Record<string, string> = {
multi_kb_search: '🔍',
- knowledge_search: '📚',
- grep_chunks: '🔎',
+ search_knowledge: '📚',
+ read_document: '🧩',
+ list_documents: 'ℹ️',
get_chunk_detail: '📄',
list_knowledge_bases: '📂',
- list_knowledge_chunks: '🧩',
- get_document_info: 'ℹ️',
query_knowledge_graph: '🕸️',
think: '💭',
todo_write: '📋',
+ // Retired names (still present in stored chat history)
+ knowledge_search: '📚',
+ grep_chunks: '🔎',
+ list_knowledge_chunks: '🧩',
+ get_document_info: 'ℹ️',
};
// Match type internal keys for icon mapping
@@ -55,15 +59,19 @@ export function getMatchTypeIcon(matchType: string): string {
// Tool name to i18n key mapping
const toolDisplayNameKeys: Record<string, string> = {
multi_kb_search: 'tools.multiKbSearch',
- knowledge_search: 'tools.knowledgeSearch',
- grep_chunks: 'tools.grepChunks',
+ search_knowledge: 'tools.searchKnowledge',
+ read_document: 'tools.readDocument',
+ list_documents: 'tools.listDocuments',
get_chunk_detail: 'tools.getChunkDetail',
- list_knowledge_chunks: 'tools.listKnowledgeChunks',
list_knowledge_bases: 'tools.listKnowledgeBases',
- get_document_info: 'tools.getDocumentInfo',
query_knowledge_graph: 'tools.queryKnowledgeGraph',
think: 'tools.think',
todo_write: 'tools.todoWrite',
+ // Retired names (still present in stored chat history)
+ knowledge_search: 'tools.knowledgeSearch',
+ grep_chunks: 'tools.grepChunks',
+ list_knowledge_chunks: 'tools.listKnowledgeChunks',
+ get_document_info: 'tools.getDocumentInfo',
};
// Get tool display name (user-friendly, localized)
diff --git a/frontend/src/views/agent/AgentEditorModal.vue b/frontend/src/views/agent/AgentEditorModal.vue
index 4b8d4daec..2226485c5 100644
--- a/frontend/src/views/agent/AgentEditorModal.vue
+++ b/frontend/src/views/agent/AgentEditorModal.vue
@@ -624,7 +624,7 @@
</div>
</div>
- <!-- ReRank 模型(启用知识库或 knowledge_search 工具时显示) -->
+ <!-- ReRank 模型(启用知识库或 search_knowledge 工具时显示) -->
<div
v-if="showRerankModelField"
class="setting-row"
@@ -1810,6 +1810,7 @@ import {
} from '@/api/agent';
import { type ModelConfig } from '@/api/model';
import { type AgentNotReadyReasonKey, agentRequiresRerankModel } from '@/utils/agent-readiness';
+import { normalizeLegacyToolNames } from '@/utils/legacy-tool-names';
import { installSkillCatalog, type SkillCatalogItem } from '@/api/skill';
import { type WebSearchProviderEntity } from '@/api/web-search-provider';
import {
@@ -2320,10 +2321,10 @@ const defaultMaxCompletionTokensFor = (mode: string, sandboxConfigId?: string) =
};
// 知识库相关工具列表(用于 watch(hasKnowledgeBase) 从"无"变"有"时 seed 默认工具)
-const knowledgeBaseTools = ['grep_chunks', 'knowledge_search', 'list_knowledge_chunks', 'get_document_info'];
+const knowledgeBaseTools = ['search_knowledge', 'read_document', 'list_documents'];
// Wiki 读取类工具(用于 watch(agentMode) 切到 smart-reasoning 时 seed 默认工具)
-const wikiReadTools = ['wiki_search', 'wiki_read_page', 'wiki_read_source_doc', 'wiki_flag_issue'];
+const wikiReadTools = ['wiki_search', 'wiki_read_page', 'read_document', 'wiki_flag_issue'];
// 初始化标志,防止初始化时触发 watch 自动添加工具
const isInitializing = ref(false);
@@ -2343,17 +2344,16 @@ const allTools = computed(() => [
// 基础思考类
{ value: 'thinking', label: t('agentEditor.tools.thinking'), description: t('agentEditor.tools.thinkingDesc'), group: 'base' },
{ value: 'todo_write', label: t('agentEditor.tools.todoWrite'), description: t('agentEditor.tools.todoWriteDesc'), group: 'base' },
- // 知识库语义/关键词检索
- { value: 'grep_chunks', label: t('agentEditor.tools.grepChunks'), description: t('agentEditor.tools.grepChunksDesc'), group: 'rag' },
- { value: 'knowledge_search', label: t('agentEditor.tools.knowledgeSearch'), description: t('agentEditor.tools.knowledgeSearchDesc'), group: 'rag' },
- { value: 'list_knowledge_chunks', label: t('agentEditor.tools.listChunks'), description: t('agentEditor.tools.listChunksDesc'), group: 'rag' },
+ // 知识库检索 / 文档阅读(旧的 grep_chunks / knowledge_search / list_knowledge_chunks /
+ // get_document_info 已合并,旧配置加载时由 normalizeLegacyToolNames 映射到新名字)
+ { value: 'search_knowledge', label: t('agentEditor.tools.searchKnowledge'), description: t('agentEditor.tools.searchKnowledgeDesc'), group: 'rag' },
+ { value: 'read_document', label: t('agentEditor.tools.readDocument'), description: t('agentEditor.tools.readDocumentDesc'), group: 'rag' },
+ { value: 'list_documents', label: t('agentEditor.tools.listDocuments'), description: t('agentEditor.tools.listDocumentsDesc'), group: 'rag' },
{ value: 'query_knowledge_graph', label: t('agentEditor.tools.queryGraph'), description: t('agentEditor.tools.queryGraphDesc'), group: 'rag' },
- { value: 'get_document_info', label: t('agentEditor.tools.getDocInfo'), description: t('agentEditor.tools.getDocInfoDesc'), group: 'rag' },
{ value: 'database_query', label: t('agentEditor.tools.dbQuery'), description: t('agentEditor.tools.dbQueryDesc'), group: 'rag' },
// Wiki 读取类(阅读、搜索、标记问题)
{ value: 'wiki_search', label: t('agentEditor.tools.wikiSearch'), description: t('agentEditor.tools.wikiSearchDesc'), group: 'wiki_read' },
{ value: 'wiki_read_page', label: t('agentEditor.tools.wikiReadPage'), description: t('agentEditor.tools.wikiReadPageDesc'), group: 'wiki_read' },
- { value: 'wiki_read_source_doc', label: t('agentEditor.tools.wikiReadSourceDoc'), description: t('agentEditor.tools.wikiReadSourceDocDesc'), group: 'wiki_read' },
{ value: 'wiki_flag_issue', label: t('agentEditor.tools.wikiFlagIssue'), description: t('agentEditor.tools.wikiFlagIssueDesc'), group: 'wiki_read' },
// Wiki 编辑类(会直接修改 Wiki 内容)
{ value: 'wiki_write_page', label: t('agentEditor.tools.wikiWritePage'), description: t('agentEditor.tools.wikiWritePageDesc'), group: 'wiki_edit', danger: true },
@@ -3263,7 +3263,7 @@ const applyAgentTypePreset = (preset: AgentTypePreset | null) => {
}
if (typeof c.temperature === 'number') target.temperature = c.temperature;
if (typeof c.max_iterations === 'number') target.max_iterations = c.max_iterations;
- if (Array.isArray(c.allowed_tools)) target.allowed_tools = [...c.allowed_tools];
+ if (Array.isArray(c.allowed_tools)) target.allowed_tools = normalizeLegacyToolNames(c.allowed_tools);
if (typeof c.retain_retrieval_history === 'boolean') target.retain_retrieval_history = c.retain_retrieval_history;
if (typeof c.faq_priority_enabled === 'boolean') target.faq_priority_enabled = c.faq_priority_enabled;
if (typeof c.web_search_enabled === 'boolean') target.web_search_enabled = c.web_search_enabled;
@@ -3391,7 +3391,10 @@ watch(() => props.visible, async (val) => {
};
// 确保数组字段存在
if (!agentData.config.knowledge_bases) agentData.config.knowledge_bases = [];
- if (!agentData.config.allowed_tools) agentData.config.allowed_tools = [];
+ // 旧配置里可能还带着已合并的工具名(knowledge_search / grep_chunks /
+ // list_knowledge_chunks / get_document_info / wiki_read_source_doc),
+ // 映射到新名字并去重,否则复选框对不上 allTools。
+ agentData.config.allowed_tools = normalizeLegacyToolNames(agentData.config.allowed_tools);
if (!agentData.config.mcp_services) agentData.config.mcp_services = [];
// 授权等待超时:旧数据缺省时用默认 600 秒
if (agentData.config.mcp_auth_wait_timeout == null || agentData.config.mcp_auth_wait_timeout <= 0) {
@@ -4774,7 +4777,7 @@ const handleSave = async () => {
}
// ReRank 模型按运行范围按需使用:知识库范围为 none,或未启用
- // knowledge_search 时不需要;其余情况由对话入口在使用前给出明确提示。
+ // search_knowledge 时不需要;其余情况由对话入口在使用前给出明确提示。
formData.value.config.question_suggestions.starters.items =
formData.value.config.question_suggestions.starters.items
diff --git a/frontend/src/views/chat/components/AgentStreamDisplay.vue b/frontend/src/views/chat/components/AgentStreamDisplay.vue
index 57cb6581b..8fac74f00 100644
--- a/frontend/src/views/chat/components/AgentStreamDisplay.vue
+++ b/frontend/src/views/chat/components/AgentStreamDisplay.vue
@@ -199,7 +199,7 @@
<div class="results-summary-text" v-html="getGrepResultsSummary(event.tool_data)"></div>
</div>
- <div v-if="!event.pending && event.tool_name === 'list_knowledge_chunks' && event.tool_data"
+ <div v-if="!event.pending && (event.tool_name === 'read_document' || event.tool_name === 'list_knowledge_chunks') && event.tool_data"
class="search-results-summary-fixed knowledge-chunks-summary">
<div class="results-summary-text" v-html="getKnowledgeChunksSummary(event.tool_data)"></div>
</div>
@@ -500,7 +500,7 @@
<div class="results-summary-text" v-html="getGrepResultsSummary(event.tool_data)"></div>
</div>
- <div v-if="!event.pending && event.tool_name === 'list_knowledge_chunks' && event.tool_data"
+ <div v-if="!event.pending && (event.tool_name === 'read_document' || event.tool_name === 'list_knowledge_chunks') && event.tool_data"
class="search-results-summary-fixed knowledge-chunks-summary">
<div class="results-summary-text" v-html="getKnowledgeChunksSummary(event.tool_data)"></div>
</div>
@@ -707,6 +707,9 @@ const TOOL_NAME_KEYS: Record<string, string> = {
discover_mcp_tools: 'agentStream.mcp.discoverTools',
call_mcp_tool: 'agentStream.mcp.callTool',
search_knowledge: 'agentStream.tools.searchKnowledge',
+ read_document: 'agentStream.tools.readDocument',
+ list_documents: 'agentStream.tools.listDocuments',
+ // Retired names still present in stored chat history
knowledge_search: 'agentStream.tools.searchKnowledge',
grep_chunks: 'agentStream.tools.grepChunks',
web_search: 'agentStream.tools.webSearch',
@@ -1421,7 +1424,7 @@ function getToolReferenceItems(event: any): KnowledgeReferenceLike[] {
})));
}
- if (toolName === 'list_knowledge_chunks' || toolName === 'wiki_read_source_doc') {
+ if (toolName === 'read_document' || toolName === 'list_knowledge_chunks' || toolName === 'wiki_read_source_doc') {
const chunks = Array.isArray(toolData.chunks) ? toolData.chunks : [];
if (chunks.length) {
return mergeDocumentReferences(chunks
@@ -2214,6 +2217,7 @@ const isReferenceDrawerTool = (toolName?: string | null): boolean =>
toolName === 'web_search' ||
toolName === 'web_fetch' ||
toolName === 'grep_chunks' ||
+ toolName === 'read_document' ||
toolName === 'list_knowledge_chunks' ||
toolName === 'wiki_search' ||
toolName === 'wiki_read_page' ||
@@ -2251,8 +2255,9 @@ const hasResults = (event: any): boolean => {
return totalMatches > 0 || resultCount > 0;
}
- // list_knowledge_chunks: summary is inline below the header (no expandable body)
- if (toolName === 'list_knowledge_chunks') {
+ // read_document (and legacy list_knowledge_chunks): summary is inline below
+ // the header (no expandable body)
+ if (toolName === 'read_document' || toolName === 'list_knowledge_chunks') {
return false;
}
@@ -2640,11 +2645,11 @@ const getToolSummary = (event: any): string => {
// For search tools, don't return summary here - it will be displayed in SearchResults component
if (toolName === 'search_knowledge' || toolName === 'knowledge_search') {
return '';
- } else if (toolName === 'get_document_info') {
+ } else if (toolName === 'get_document_info' || toolName === 'list_documents') {
if (toolData?.title) {
return t('agentStream.toolSummary.getDocument', { title: toolData.title });
}
- } else if (toolName === 'list_knowledge_chunks') {
+ } else if (toolName === 'read_document' || toolName === 'list_knowledge_chunks') {
if (toolData?.faq_question) {
return t('agentStream.toolSummary.listFaqEntry', { question: toolData.faq_question });
}
@@ -3039,9 +3044,9 @@ const getToolDescription = (event: any): string => {
return success ? t('agentStream.toolStatus.webSearch') : t('agentStream.toolStatus.webSearchFailed');
} else if (toolName === 'grep_chunks') {
return success ? t('agentStream.toolStatus.grepSearch') : t('agentStream.toolStatus.grepSearchFailed');
- } else if (toolName === 'get_document_info') {
+ } else if (toolName === 'get_document_info' || toolName === 'list_documents') {
return success ? t('agentStream.toolStatus.getDocInfo') : t('agentStream.toolStatus.getDocInfoFailed');
- } else if (toolName === 'get_document_content' || toolName === 'wiki_read_source_doc') {
+ } else if (toolName === 'read_document' || toolName === 'get_document_content' || toolName === 'wiki_read_source_doc') {
return success ? t('agentStream.toolStatus.viewDocument') : t('agentStream.toolStatus.calledFailed', { name: t('agentStream.toolStatus.viewDocument') });
} else if (toolName === 'thinking') {
return success ? t('agentStream.toolStatus.thinkingDone') : t('agentStream.toolStatus.thinkingFailed');
diff --git a/internal/agent/act.go b/internal/agent/act.go
index b80d5f11f..6933c4fd3 100644
--- a/internal/agent/act.go
+++ b/internal/agent/act.go
@@ -162,30 +162,33 @@ func dataKeys(data map[string]interface{}) []string {
// toolDisplayNames maps internal tool names to user-friendly display labels.
var toolDisplayNames = map[string]string{
- agenttools.ToolDiscoverMCPTools: "查看外部工具",
- agenttools.ToolCallMCPTool: "调用外部工具",
- agenttools.ToolThinking: "深度思考",
- agenttools.ToolTodoWrite: "制定计划",
- agenttools.ToolGrepChunks: "关键词搜索",
- agenttools.ToolKnowledgeSearch: "知识搜索",
- agenttools.ToolListKnowledgeChunks: "查看文档分块",
- agenttools.ToolQueryKnowledgeGraph: "查询知识图谱",
- agenttools.ToolGetDocumentInfo: "获取文档信息",
- agenttools.ToolSearchConversations: "回顾历史对话",
- agenttools.ToolSearchMemory: "查询长期记忆",
- agenttools.ToolDatabaseQuery: "查询数据",
- agenttools.ToolDataAnalysis: "数据分析",
- agenttools.ToolDataSchema: "查看数据结构",
- agenttools.ToolWebSearch: "搜索网页",
- agenttools.ToolWebFetch: "获取网页",
- agenttools.LegacyToolExecuteSkillScript: "执行技能脚本",
- agenttools.LegacyToolReadSkill: "读取技能",
- agenttools.ToolReadFile: "读取文件",
- agenttools.ToolListSandboxFiles: "列出沙箱文件",
- agenttools.LegacyToolReadSandboxFile: "读取沙箱文件",
- agenttools.ToolWriteSandboxFile: "写入沙箱文件",
- agenttools.ToolEditSandboxFile: "编辑沙箱文件",
- agenttools.ToolShellExec: "执行沙箱命令",
+ agenttools.ToolDiscoverMCPTools: "查看外部工具",
+ agenttools.ToolCallMCPTool: "调用外部工具",
+ agenttools.ToolThinking: "深度思考",
+ agenttools.ToolTodoWrite: "制定计划",
+ agenttools.ToolSearchKnowledge: "检索知识库",
+ agenttools.ToolReadDocument: "阅读文档",
+ agenttools.ToolListDocuments: "浏览文档列表",
+ agenttools.ToolQueryKnowledgeGraph: "查询知识图谱",
+ agenttools.LegacyToolGrepChunks: "关键词搜索",
+ agenttools.LegacyToolKnowledgeSearch: "知识搜索",
+ agenttools.LegacyToolListKnowledgeChunks: "查看文档分块",
+ agenttools.LegacyToolGetDocumentInfo: "获取文档信息",
+ agenttools.ToolSearchConversations: "回顾历史对话",
+ agenttools.ToolSearchMemory: "查询长期记忆",
+ agenttools.ToolDatabaseQuery: "查询数据",
+ agenttools.ToolDataAnalysis: "数据分析",
+ agenttools.ToolDataSchema: "查看数据结构",
+ agenttools.ToolWebSearch: "搜索网页",
+ agenttools.ToolWebFetch: "获取网页",
+ agenttools.LegacyToolExecuteSkillScript: "执行技能脚本",
+ agenttools.LegacyToolReadSkill: "读取技能",
+ agenttools.ToolReadFile: "读取文件",
+ agenttools.ToolListSandboxFiles: "列出沙箱文件",
+ agenttools.LegacyToolReadSandboxFile: "读取沙箱文件",
+ agenttools.ToolWriteSandboxFile: "写入沙箱文件",
+ agenttools.ToolEditSandboxFile: "编辑沙箱文件",
+ agenttools.ToolShellExec: "执行沙箱命令",
}
// toolHintSensitiveArgs lists tools whose arguments should NOT be shown in hints
diff --git a/internal/agent/act_order_test.go b/internal/agent/act_order_test.go
index 74f7a6861..11205c6c1 100644
--- a/internal/agent/act_order_test.go
+++ b/internal/agent/act_order_test.go
@@ -40,7 +40,7 @@ func TestParallelReadsRespectMutationBarriers(t *testing.T) {
},
})
}
- for _, name := range []string{tools.ToolKnowledgeSearch, tools.ToolGrepChunks} {
+ for _, name := range []string{tools.ToolSearchKnowledge, tools.ToolReadDocument} {
register(name, func(ctx context.Context) {
readStarts <- struct{}{}
select {
@@ -53,7 +53,10 @@ func TestParallelReadsRespectMutationBarriers(t *testing.T) {
register(tools.ToolShellExec, func(context.Context) {})
register(tools.ToolReadFile, func(context.Context) {})
register("mcp_unknown_mutation", func(context.Context) {})
- names := []string{tools.ToolKnowledgeSearch, tools.ToolGrepChunks, tools.ToolWriteSandboxFile, tools.ToolShellExec, tools.ToolReadFile, "mcp_unknown_mutation"}
+ names := []string{
+ tools.ToolSearchKnowledge, tools.ToolReadDocument, tools.ToolWriteSandboxFile,
+ tools.ToolShellExec, tools.ToolReadFile, "mcp_unknown_mutation",
+ }
response := &types.ChatResponse{}
for _, name := range names {
response.ToolCalls = append(response.ToolCalls, types.LLMToolCall{ID: name, Function: types.FunctionCall{Name: name, Arguments: `{}`}})
diff --git a/internal/agent/engine_test.go b/internal/agent/engine_test.go
index e0aa7350c..b7a4fe290 100644
--- a/internal/agent/engine_test.go
+++ b/internal/agent/engine_test.go
@@ -240,9 +240,9 @@ func TestRunToolCallRejectsUnresolvedHandlesBeforeExecution(t *testing.T) {
ID: "call-1",
Function: types.FunctionCall{
Name: tool.Name(),
- Arguments: `{"knowledge_id":"d99"}`,
+ Arguments: `{"id":"d99"}`,
},
- ModelArguments: `{"knowledge_id":"d99"}`,
+ ModelArguments: `{"id":"d99"}`,
ArgumentResolution: modelcontext.ArgumentResolutionUnresolved,
UnresolvedHandles: []string{"d99"},
},
@@ -258,7 +258,7 @@ func TestRunToolCallDecodesHandlesAfterJSONRepair(t *testing.T) {
newEngine := func() (*AgentEngine, *countingTool) {
engine := newTestEngine(t, &mockChat{})
engine.toolRegistry = agenttools.NewToolRegistry()
- tool := newCountingTool(agenttools.ToolListKnowledgeChunks)
+ tool := newCountingTool(agenttools.ToolReadDocument)
engine.toolRegistry.RegisterTool(tool)
return engine, tool
}
@@ -268,8 +268,8 @@ func TestRunToolCallDecodesHandlesAfterJSONRepair(t *testing.T) {
context.Background(),
types.LLMToolCall{
ID: "call-unknown",
- Function: types.FunctionCall{Name: unknownTool.Name(), Arguments: `{"knowledge_id":"d99",}`},
- ModelArguments: `{"knowledge_id":"d99",}`,
+ Function: types.FunctionCall{Name: unknownTool.Name(), Arguments: `{"id":"d99",}`},
+ ModelArguments: `{"id":"d99",}`,
},
0, 0, 1, "session", "message",
)
@@ -283,14 +283,14 @@ func TestRunToolCallDecodesHandlesAfterJSONRepair(t *testing.T) {
context.Background(),
types.LLMToolCall{
ID: "call-known",
- Function: types.FunctionCall{Name: knownTool.Name(), Arguments: `{"knowledge_id":"d1",}`},
- ModelArguments: `{"knowledge_id":"d1",}`,
+ Function: types.FunctionCall{Name: knownTool.Name(), Arguments: `{"id":"d1",}`},
+ ModelArguments: `{"id":"d1",}`,
},
0, 0, 1, "session", "message",
)
require.Equal(t, 1, knownTool.calls)
require.True(t, known.Result.Success)
- require.Equal(t, "doc-real", known.Args["knowledge_id"])
+ require.Equal(t, "doc-real", known.Args["id"])
}
func (m *mockChat) Chat(_ context.Context, _ []chat.Message, _ *chat.ChatOptions) (*types.ChatResponse, error) {
diff --git a/internal/agent/grounding_prompt.go b/internal/agent/grounding_prompt.go
index f07b8e142..a36d4614e 100644
--- a/internal/agent/grounding_prompt.go
+++ b/internal/agent/grounding_prompt.go
@@ -33,9 +33,8 @@ func formatGroundingGuidance(names []string) string {
}
var kbTools []string
for _, name := range []string{
- tools.ToolKnowledgeSearch, tools.ToolGrepChunks, tools.ToolListKnowledgeChunks,
- tools.ToolGetDocumentInfo, tools.ToolWikiSearch, tools.ToolWikiReadPage,
- tools.ToolWikiReadSourceDoc, tools.ToolQueryKnowledgeGraph,
+ tools.ToolSearchKnowledge, tools.ToolReadDocument, tools.ToolListDocuments,
+ tools.ToolWikiSearch, tools.ToolWikiReadPage, tools.ToolQueryKnowledgeGraph,
tools.ToolDataSchema, tools.ToolDataAnalysis, tools.ToolDatabaseQuery,
} {
if slices.Contains(names, name) {
diff --git a/internal/agent/grounding_prompt_test.go b/internal/agent/grounding_prompt_test.go
index e57b7af03..3eade98ab 100644
--- a/internal/agent/grounding_prompt_test.go
+++ b/internal/agent/grounding_prompt_test.go
@@ -53,21 +53,43 @@ func TestGroundingUsesRegistryInsteadOfConfiguration(t *testing.T) {
want []string
absent []string
}{
- {"skill only with stale flags", []string{tools.ToolReadFile, tools.ToolShellExec}, true,
- nil, []string{"Available knowledge tools:", "web_search is available", "web_fetch is available"}},
- {"rag", []string{tools.ToolKnowledgeSearch, tools.ToolListKnowledgeChunks}, false,
- []string{"Available knowledge tools: knowledge_search, list_knowledge_chunks"}, []string{"wiki_search", "web_search is available"}},
- {"wiki", []string{tools.ToolWikiReadPage, tools.ToolWikiSearch}, false,
- []string{"Available knowledge tools: wiki_search, wiki_read_page"}, []string{"knowledge_search", "web_search is available"}},
- {"registered web", []string{tools.ToolWebSearch, tools.ToolWebFetch}, false,
- []string{"web_search is available", "web_fetch is available"}, []string{"Available knowledge tools:"}},
- {"no tools", nil, false, nil,
- []string{"Available knowledge tools:", "web_search is available", "web_fetch is available"}},
+ {
+ "skill only with stale flags",
+ []string{tools.ToolReadFile, tools.ToolShellExec},
+ true,
+ nil,
+ []string{"Available knowledge tools:", "web_search is available", "web_fetch is available"},
+ },
+ {
+ "rag",
+ []string{tools.ToolSearchKnowledge, tools.ToolReadDocument},
+ false,
+ []string{"Available knowledge tools: search_knowledge, read_document"},
+ []string{"wiki_search", "web_search is available"},
+ },
+ {
+ "wiki",
+ []string{tools.ToolWikiReadPage, tools.ToolWikiSearch},
+ false,
+ []string{"Available knowledge tools: wiki_search, wiki_read_page"},
+ []string{"search_knowledge", "web_search is available"},
+ },
+ {
+ "registered web",
+ []string{tools.ToolWebSearch, tools.ToolWebFetch},
+ false,
+ []string{"web_search is available", "web_fetch is available"},
+ []string{"Available knowledge tools:"},
+ },
+ {
+ "no tools", nil, false, nil,
+ []string{"Available knowledge tools:", "web_search is available", "web_fetch is available"},
+ },
} {
t.Run(tc.name, func(t *testing.T) {
engine := newTestEngine(t, &mockChat{})
engine.config.WebSearchEnabled = tc.webFlag
- engine.config.AllowedTools = []string{tools.ToolKnowledgeSearch, tools.ToolWebSearch}
+ engine.config.AllowedTools = []string{tools.ToolSearchKnowledge, tools.ToolWebSearch}
engine.toolRegistry = tools.NewToolRegistry()
for _, name := range tc.registered {
engine.toolRegistry.RegisterTool(newCountingTool(name))
diff --git a/internal/agent/observe.go b/internal/agent/observe.go
index 6316887f0..f1ede12f7 100644
--- a/internal/agent/observe.go
+++ b/internal/agent/observe.go
@@ -856,14 +856,18 @@ func countTotalToolCalls(steps []types.AgentStep) int {
// may become stale across turns (KB can be switched, updated, or deleted).
// Historical results from these tools are redacted to force fresh retrieval.
var kbToolNames = map[string]bool{
- agenttools.ToolKnowledgeSearch: true,
- agenttools.ToolGrepChunks: true,
- agenttools.ToolListKnowledgeChunks: true,
+ agenttools.ToolSearchKnowledge: true,
+ agenttools.ToolReadDocument: true,
+ agenttools.ToolListDocuments: true,
agenttools.ToolQueryKnowledgeGraph: true,
- agenttools.ToolGetDocumentInfo: true,
agenttools.ToolWikiSearch: true,
agenttools.ToolWikiReadPage: true,
- agenttools.ToolWikiReadSourceDoc: true,
+ // Retired names still appear in stored histories.
+ agenttools.LegacyToolKnowledgeSearch: true,
+ agenttools.LegacyToolGrepChunks: true,
+ agenttools.LegacyToolListKnowledgeChunks: true,
+ agenttools.LegacyToolGetDocumentInfo: true,
+ agenttools.LegacyToolWikiReadSourceDoc: true,
}
// redactHistoryKBResults replaces full KB tool results in historical context
diff --git a/internal/agent/observe_test.go b/internal/agent/observe_test.go
index 553f49447..4ae42d472 100644
--- a/internal/agent/observe_test.go
+++ b/internal/agent/observe_test.go
@@ -81,7 +81,7 @@ func TestAnalyzeResponse_ToolCall_DoesNotTerminate(t *testing.T) {
ID: "call-1",
Type: "function",
Function: types.FunctionCall{
- Name: agenttools.ToolKnowledgeSearch,
+ Name: agenttools.ToolSearchKnowledge,
Arguments: `{"query": "hi"}`,
},
},
diff --git a/internal/agent/prompt_composition_test.go b/internal/agent/prompt_composition_test.go
index bd135ec26..ed0e518c4 100644
--- a/internal/agent/prompt_composition_test.go
+++ b/internal/agent/prompt_composition_test.go
@@ -20,7 +20,7 @@ import (
func TestPromptSectionsUseActualSourcesAndKeepCustomBase(t *testing.T) {
for _, browser := range []bool{false, true} {
for _, web := range []bool{false, true} {
- names := []string{tools.ToolKnowledgeSearch, tools.ToolWikiSearch, tools.ToolReadFile}
+ names := []string{tools.ToolSearchKnowledge, tools.ToolWikiSearch, tools.ToolReadFile}
if browser {
names = append(names, "local_browser")
}
@@ -176,7 +176,7 @@ func TestDefaultTemplatesComposeWithBrowserCitationsAndOutputPolicy(t *testing.T
for _, citations := range []bool{false, true} {
t.Run(template.ID+"/browser="+fmtBool(browser)+"/citations="+fmtBool(citations), func(t *testing.T) {
names := []string{
- "knowledge_search", "wiki_search", "wiki_read_page",
+ "search_knowledge", "wiki_search", "wiki_read_page",
"read_file", "shell_exec", "discover_mcp_tools",
}
if browser {
diff --git a/internal/agent/tools/capabilities.go b/internal/agent/tools/capabilities.go
index e648e2cab..273924883 100644
--- a/internal/agent/tools/capabilities.go
+++ b/internal/agent/tools/capabilities.go
@@ -42,6 +42,12 @@ type ToolRequirement struct {
AnyOf []KBCapability
AllOf []KBCapability
ConsumesFiles bool
+ // Auxiliary tools work on any KB listed in AnyOf/AllOf but do not by
+ // themselves make a KB worth selecting: a document reader can open a
+ // wiki-only KB's sources, yet it should not pull wiki-only KBs into a
+ // RAG agent's "all knowledge bases" scope. They contribute to the derived
+ // filter only when no primary tool has a requirement.
+ Auxiliary bool
}
// ToolCapabilityRequirements maps tool names to their capability needs.
@@ -56,17 +62,22 @@ var ToolCapabilityRequirements = map[string]ToolRequirement{
"todo_write": {},
// ---- RAG / chunk retrieval (need at least one chunk-indexed KB) ----
+ "search_knowledge": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
+ "read_document": documentReaderRequirement,
+ "list_documents": documentReaderRequirement,
+ "query_knowledge_graph": {AllOf: []KBCapability{CapGraph}, ConsumesFiles: true},
+ "database_query": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
+ // Retired names keep their requirement so a stored allowlist that has not
+ // been normalized yet still derives the same KB filter.
"knowledge_search": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
"grep_chunks": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
"list_knowledge_chunks": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
- "query_knowledge_graph": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
"get_document_info": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
- "database_query": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
// ---- Wiki (operates on wiki pages; doesn't consume arbitrary file IDs) ----
"wiki_search": {AllOf: []KBCapability{CapWiki}},
"wiki_read_page": {AllOf: []KBCapability{CapWiki}},
- "wiki_read_source_doc": {AllOf: []KBCapability{CapWiki}},
+ "wiki_read_source_doc": documentReaderRequirement,
"wiki_flag_issue": {AllOf: []KBCapability{CapWiki}},
"wiki_write_page": {AllOf: []KBCapability{CapWiki}},
"wiki_replace_text": {AllOf: []KBCapability{CapWiki}},
@@ -80,6 +91,14 @@ var ToolCapabilityRequirements = map[string]ToolRequirement{
"data_schema": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
}
+// documentReaderRequirement: document readers work on stored chunks, which
+// every KB writes, wiki-only ones included.
+var documentReaderRequirement = ToolRequirement{
+ AnyOf: []KBCapability{CapVector, CapKeyword, CapWiki},
+ ConsumesFiles: true,
+ Auxiliary: true,
+}
+
func hasCap(caps types.KBCapabilities, c KBCapability) bool {
switch c {
case CapVector:
@@ -112,18 +131,26 @@ func (f KBFilter) IsEmpty() bool { return len(f.AnyOf) == 0 }
// list contains only such tools, the returned filter is empty (accept all).
func DeriveKBFilterFromTools(allowedTools []string) KBFilter {
seen := make(map[KBCapability]struct{})
+ auxiliary := make(map[KBCapability]struct{})
for _, t := range allowedTools {
req, ok := ToolCapabilityRequirements[t]
if !ok {
continue
}
+ target := seen
+ if req.Auxiliary {
+ target = auxiliary
+ }
for _, c := range req.AnyOf {
- seen[c] = struct{}{}
+ target[c] = struct{}{}
}
for _, c := range req.AllOf {
- seen[c] = struct{}{}
+ target[c] = struct{}{}
}
}
+ if len(seen) == 0 {
+ seen = auxiliary
+ }
if len(seen) == 0 {
return KBFilter{}
}
diff --git a/internal/agent/tools/chunk_snippet.go b/internal/agent/tools/chunk_snippet.go
new file mode 100644
index 000000000..4220b5c53
--- /dev/null
+++ b/internal/agent/tools/chunk_snippet.go
@@ -0,0 +1,354 @@
+package tools
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "regexp"
+ "strings"
+
+ "github.com/Tencent/WeKnora/internal/searchutil"
+ "github.com/Tencent/WeKnora/internal/types"
+ "github.com/Tencent/WeKnora/internal/types/interfaces"
+)
+
+// Shared helpers for the knowledge retrieval tools (search_knowledge,
+// read_document, list_documents): snippet extraction, FAQ display, image
+// enrichment and the minimal XML escaping used by tool output.
+
+// xmlEscape replaces characters that would break simple XML attribute /
+// element values. It is intentionally minimal because the rendered output is
+// consumed by the LLM (forgiving parser) rather than a strict XML processor.
+func xmlEscape(s string) string {
+ replacer := strings.NewReplacer(
+ "&", "&amp;",
+ "<", "&lt;",
+ ">", "&gt;",
+ "\"", "&quot;",
+ "'", "&apos;",
+ )
+ return replacer.Replace(s)
+}
+
+func dedupNonEmptyStrings(values []string) []string {
+ seen := make(map[string]bool, len(values))
+ out := make([]string, 0, len(values))
+ for _, value := range values {
+ if value == "" || seen[value] {
+ continue
+ }
+ seen[value] = true
+ out = append(out, value)
+ }
+ return out
+}
+
+// regexMatchesAny reports whether text matches at least one of the compiled
+// patterns.
+func regexMatchesAny(text string, compiled []*regexp.Regexp) bool {
+ if text == "" || len(compiled) == 0 {
+ return false
+ }
+ for _, re := range compiled {
+ if re != nil && re.MatchString(text) {
+ return true
+ }
+ }
+ return false
+}
+
+// extractChunkMatchSnippet returns a preview for tool output. FAQ chunks only
+// surface the matched question plus answers from metadata (answers are not
+// stored in chunk content for question_only index mode). Other chunk types
+// use regex context around the first body match.
+func extractChunkMatchSnippet(chunk *types.Chunk, compiled []*regexp.Regexp) string {
+ if chunk != nil && chunk.ChunkType == types.ChunkTypeFAQ {
+ if s := faqMatchSnippet(chunk, compiled); s != "" {
+ return s
+ }
+ }
+ if chunk == nil {
+ return ""
+ }
+ return extractSnippetRegex(chunk.Content, compiled)
+}
+
+// extractSnippetRegex returns a short context snippet around the earliest
+// regex match across any of the provided compiled patterns. Result is
+// compressed to a single line and bounded in length on both sides of the
+// match to keep the XML output concise.
+func extractSnippetRegex(content string, compiled []*regexp.Regexp) string {
+ if content == "" || len(compiled) == 0 {
+ return ""
+ }
+
+ earliest := -1
+ earliestEnd := -1
+ for _, re := range compiled {
+ if re == nil {
+ continue
+ }
+ loc := re.FindStringIndex(content)
+ if loc == nil {
+ continue
+ }
+ if earliest < 0 || loc[0] < earliest {
+ earliest = loc[0]
+ earliestEnd = loc[1]
+ }
+ }
+ if earliest < 0 {
+ return ""
+ }
+
+ matchStr := content[earliest:earliestEnd]
+ before := content[:earliest]
+ after := content[earliestEnd:]
+
+ beforeRunes := []rune(before)
+ if len(beforeRunes) > snippetContextRunes {
+ beforeRunes = beforeRunes[len(beforeRunes)-snippetContextRunes:]
+ }
+ afterRunes := []rune(after)
+ if len(afterRunes) > snippetContextRunes {
+ afterRunes = afterRunes[:snippetContextRunes]
+ }
+ matchRunes := []rune(matchStr)
+ if len(matchRunes) > snippetMaxMatchRunes {
+ matchRunes = append(matchRunes[:snippetMaxMatchRunes], []rune("...")...)
+ }
+
+ snippet := string(beforeRunes) + string(matchRunes) + string(afterRunes)
+ snippet = collapseWhitespace(snippet)
+ if len([]rune(snippet)) > snippetMaxTotalRunes {
+ snippet = string([]rune(snippet)[:snippetMaxTotalRunes]) + "..."
+ }
+ return "... " + snippet + " ..."
+}
+
+// extractSnippetForQueries tries to produce a short contextual snippet around
+// the first occurrence of any token extracted from the provided queries.
+// When no token matches (common for fully paraphrased semantic queries) it
+// falls back to the leading runes of content so callers always get something
+// to scan. The snippet is single-lined and bounded in length.
+func extractSnippetForQueries(content string, queries []string) string {
+ content = strings.TrimSpace(content)
+ if content == "" {
+ return ""
+ }
+
+ tokens := searchQueryTokens(queries)
+
+ lowered := strings.ToLower(content)
+ earliest := -1
+ earliestEnd := -1
+ for _, tok := range tokens {
+ idx := strings.Index(lowered, tok)
+ if idx < 0 {
+ continue
+ }
+ end := idx + len(tok)
+ if earliest < 0 || idx < earliest {
+ earliest = idx
+ earliestEnd = end
+ }
+ }
+
+ if earliest < 0 {
+ runes := []rune(content)
+ if len(runes) > snippetContextRunes*2 {
+ return strings.TrimSpace(string(runes[:snippetContextRunes*2])) + " ..."
+ }
+ return content
+ }
+
+ matchStr := content[earliest:earliestEnd]
+ before := content[:earliest]
+ after := content[earliestEnd:]
+
+ beforeRunes := []rune(before)
+ if len(beforeRunes) > snippetContextRunes {
+ beforeRunes = beforeRunes[len(beforeRunes)-snippetContextRunes:]
+ }
+ afterRunes := []rune(after)
+ if len(afterRunes) > snippetContextRunes {
+ afterRunes = afterRunes[:snippetContextRunes]
+ }
+
+ snippet := collapseWhitespace(string(beforeRunes) + matchStr + string(afterRunes))
+ return "... " + snippet + " ..."
+}
+
+func collapseWhitespace(s string) string {
+ s = strings.ReplaceAll(s, "\n", " ")
+ for strings.Contains(s, " ") {
+ s = strings.ReplaceAll(s, " ", " ")
+ }
+ return strings.TrimSpace(s)
+}
+
+// faqStandardQuestion returns the FAQ standard question for an FAQ-type chunk,
+// or "" for non-FAQ chunks (or when metadata is missing/unparseable). All FAQ
+// entries inside one knowledge share the same knowledge title, so surfacing the
+// standard question gives each entry a distinct, human-readable identity in
+// tool output that would otherwise look like duplicate same-titled chunks.
+func faqStandardQuestion(c *types.Chunk) string {
+ if c == nil || c.ChunkType != types.ChunkTypeFAQ {
+ return ""
+ }
+ meta, err := c.FAQMetadata()
+ if err != nil || meta == nil {
+ return ""
+ }
+ return strings.TrimSpace(meta.StandardQuestion)
+}
+
+// enrichChunkImageInfo populates chunk.ImageInfo for a batch of parent text
+// chunks by looking up their image_ocr / image_caption children. Chunks that
+// already have a non-empty ImageInfo are left untouched.
+func enrichChunkImageInfo(
+ ctx context.Context,
+ chunkRepo interfaces.ChunkRepository,
+ tenantID uint64,
+ chunks []*types.Chunk,
+) {
+ if len(chunks) == 0 || chunkRepo == nil {
+ return
+ }
+ ids := make([]string, 0, len(chunks))
+ for _, c := range chunks {
+ if c.ImageInfo == "" && c.ID != "" {
+ ids = append(ids, c.ID)
+ }
+ }
+ if len(ids) == 0 {
+ return
+ }
+ infoMap := searchutil.CollectImageInfoByChunkIDs(ctx, chunkRepo, tenantID, ids)
+ if len(infoMap) == 0 {
+ return
+ }
+ for _, c := range chunks {
+ if c.ImageInfo != "" {
+ continue
+ }
+ if merged, ok := infoMap[c.ID]; ok && merged != "" {
+ c.ImageInfo = merged
+ }
+ }
+}
+
+// enrichChunkContent appends Markdown image references (caption + OCR text)
+// carried by chunk.ImageInfo to the chunk body.
+func enrichChunkContent(c *types.Chunk) string {
+ content := c.Content
+ if c.ImageInfo != "" {
+ var imgInfos []types.ImageInfo
+ if err := json.Unmarshal([]byte(c.ImageInfo), &imgInfos); err == nil && len(imgInfos) > 0 {
+ var imgBuilder strings.Builder
+ for _, img := range imgInfos {
+ if imageMarkdown := searchutil.BuildImageInfoMarkdownWithURL(img.URL, &img); imageMarkdown != "" {
+ imgBuilder.WriteString("\n")
+ imgBuilder.WriteString(imageMarkdown)
+ }
+ }
+ content += imgBuilder.String()
+ }
+ }
+ return content
+}
+
+// chunkImageList converts chunk.ImageInfo into the structured image list used
+// by tool Data payloads (url / caption / ocr_text per image).
+func chunkImageList(imageInfo string) []map[string]string {
+ if imageInfo == "" {
+ return nil
+ }
+ var imageInfos []types.ImageInfo
+ if err := json.Unmarshal([]byte(imageInfo), &imageInfos); err != nil || len(imageInfos) == 0 {
+ return nil
+ }
+ imageList := make([]map[string]string, 0, len(imageInfos))
+ for _, img := range imageInfos {
+ imgData := make(map[string]string)
+ if img.URL != "" {
+ imgData["url"] = img.URL
+ }
+ if img.Caption != "" {
+ imgData["caption"] = img.Caption
+ }
+ if img.OCRText != "" {
+ imgData["ocr_text"] = img.OCRText
+ }
+ if len(imgData) > 0 {
+ imageList = append(imageList, imgData)
+ }
+ }
+ if len(imageList) == 0 {
+ return nil
+ }
+ return imageList
+}
+
+// writeChunkImagesMarkdown appends one Markdown image line per image carried
+// by the chunk to the XML output builder.
+func writeChunkImagesMarkdown(b *strings.Builder, c *types.Chunk) {
+ if c == nil || c.ImageInfo == "" {
+ return
+ }
+ var imageInfos []types.ImageInfo
+ if err := json.Unmarshal([]byte(c.ImageInfo), &imageInfos); err != nil || len(imageInfos) == 0 {
+ return
+ }
+ for _, img := range imageInfos {
+ if imageMarkdown := searchutil.BuildImageInfoMarkdownWithURL(img.URL, &img); imageMarkdown != "" {
+ b.WriteString(imageMarkdown)
+ b.WriteString("\n")
+ }
+ }
+}
+
+// chunkDataMap builds the structured row shared by read_document Data
+// payloads and the modelcontext renderer.
+func chunkDataMap(seq int, c *types.Chunk) map[string]interface{} {
+ chunkData := map[string]interface{}{
+ "seq": seq,
+ "chunk_id": c.ID,
+ "chunk_index": c.ChunkIndex,
+ "content": c.Content,
+ "chunk_type": c.ChunkType,
+ "knowledge_id": c.KnowledgeID,
+ "knowledge_base": c.KnowledgeBaseID,
+ "start_at": c.StartAt,
+ "end_at": c.EndAt,
+ "parent_chunk_id": c.ParentChunkID,
+ }
+ appendFAQChunkData(chunkData, c)
+ normalizeFAQChunkDataMap(chunkData, c)
+ if images := chunkImageList(c.ImageInfo); len(images) > 0 {
+ chunkData["images"] = images
+ }
+ return chunkData
+}
+
+// writeChunkXML renders one chunk (FAQ entries included) for the UI/log
+// Output of read_document.
+func writeChunkXML(b *strings.Builder, c *types.Chunk, extraAttrs string) {
+ if c == nil {
+ return
+ }
+ if c.ChunkType == types.ChunkTypeFAQ {
+ writeFAQEntryXML(b, c)
+ writeChunkImagesMarkdown(b, c)
+ return
+ }
+ fmt.Fprintf(b, "<chunk chunk_id=\"%s\" chunk_index=\"%d\" type=\"%s\"%s>\n",
+ xmlEscape(c.ID), c.ChunkIndex, c.ChunkType, extraAttrs)
+ content := strings.TrimSpace(c.Content)
+ if content == "" {
+ content = "(empty)"
+ }
+ fmt.Fprintf(b, "<content>%s</content>\n", content)
+ writeChunkImagesMarkdown(b, c)
+ b.WriteString("</chunk>\n")
+}
diff --git a/internal/agent/tools/grep_chunks_snippet_test.go b/internal/agent/tools/chunk_snippet_test.go
similarity index 100%
rename from internal/agent/tools/grep_chunks_snippet_test.go
rename to internal/agent/tools/chunk_snippet_test.go
diff --git a/internal/agent/tools/definitions.go b/internal/agent/tools/definitions.go
index b6f6d30ac..45e87bce1 100644
--- a/internal/agent/tools/definitions.go
+++ b/internal/agent/tools/definitions.go
@@ -7,15 +7,18 @@ const maxFunctionNameLength = 64
// Tool names constants
const (
// Capability-scoped MCP discovery and invocation; not tenant-selectable builtins.
- ToolDiscoverMCPTools = "discover_mcp_tools"
- ToolCallMCPTool = "call_mcp_tool"
- ToolThinking = "thinking"
- ToolTodoWrite = "todo_write"
- ToolGrepChunks = "grep_chunks"
- ToolKnowledgeSearch = "knowledge_search"
- ToolListKnowledgeChunks = "list_knowledge_chunks"
+ ToolDiscoverMCPTools = "discover_mcp_tools"
+ ToolCallMCPTool = "call_mcp_tool"
+ ToolThinking = "thinking"
+ ToolTodoWrite = "todo_write"
+ // Knowledge retrieval surface. search_knowledge covers semantic, keyword
+ // and hybrid retrieval over chunk-indexed knowledge bases; read_document
+ // reads a document's metadata and chunks (by page, by chunk handle, or by
+ // an in-document text search); list_documents browses one knowledge base.
+ ToolSearchKnowledge = "search_knowledge"
+ ToolReadDocument = "read_document"
+ ToolListDocuments = "list_documents"
ToolQueryKnowledgeGraph = "query_knowledge_graph"
- ToolGetDocumentInfo = "get_document_info"
ToolSearchConversations = "search_conversations"
ToolSearchMemory = "search_memory"
ToolDatabaseQuery = "database_query"
@@ -61,16 +64,15 @@ const (
// skills switch and not a per-agent tool checkbox.
ToolShellExec = "shell_exec"
// Wiki-related tools (only available when wiki KBs are in scope)
- ToolWikiReadPage = "wiki_read_page"
- ToolWikiWritePage = "wiki_write_page"
- ToolWikiReplaceText = "wiki_replace_text"
- ToolWikiRenamePage = "wiki_rename_page"
- ToolWikiDeletePage = "wiki_delete_page"
- ToolWikiSearch = "wiki_search"
- ToolWikiReadSourceDoc = "wiki_read_source_doc"
- ToolWikiFlagIssue = "wiki_flag_issue"
- ToolWikiReadIssue = "wiki_read_issue"
- ToolWikiUpdateIssue = "wiki_update_issue"
+ ToolWikiReadPage = "wiki_read_page"
+ ToolWikiWritePage = "wiki_write_page"
+ ToolWikiReplaceText = "wiki_replace_text"
+ ToolWikiRenamePage = "wiki_rename_page"
+ ToolWikiDeletePage = "wiki_delete_page"
+ ToolWikiSearch = "wiki_search"
+ ToolWikiFlagIssue = "wiki_flag_issue"
+ ToolWikiReadIssue = "wiki_read_issue"
+ ToolWikiUpdateIssue = "wiki_update_issue"
)
// AvailableTool defines a simple tool metadata used by settings APIs.
@@ -86,11 +88,10 @@ func AvailableToolDefinitions() []AvailableTool {
return []AvailableTool{
{Name: ToolThinking, Label: "思考", Description: "动态和反思性的问题解决思考工具"},
{Name: ToolTodoWrite, Label: "制定计划", Description: "创建结构化的研究计划"},
- {Name: ToolGrepChunks, Label: "关键词搜索", Description: "快速定位包含特定关键词的文档和分块"},
- {Name: ToolKnowledgeSearch, Label: "语义搜索", Description: "理解问题并查找语义相关内容"},
- {Name: ToolListKnowledgeChunks, Label: "查看文档分块", Description: "获取文档完整分块内容"},
+ {Name: ToolSearchKnowledge, Label: "检索知识库", Description: "语义、关键词或混合检索知识库分块"},
+ {Name: ToolReadDocument, Label: "阅读文档", Description: "读取文档元数据与分块内容,支持分页和文内查找"},
+ {Name: ToolListDocuments, Label: "浏览文档列表", Description: "分页列出知识库中的文档"},
{Name: ToolQueryKnowledgeGraph, Label: "查询知识图谱", Description: "从知识图谱中查询关系"},
- {Name: ToolGetDocumentInfo, Label: "获取文档信息", Description: "查看文档元数据"},
{
Name: ToolSearchConversations,
Label: "回顾历史对话",
@@ -101,7 +102,6 @@ func AvailableToolDefinitions() []AvailableTool {
{Name: ToolDataSchema, Label: "查看数据元信息", Description: "获取表格文件的元信息"},
{Name: ToolWikiReadPage, Label: "读取Wiki页面", Description: "读取指定的Wiki页面内容"},
{Name: ToolWikiSearch, Label: "搜索Wiki", Description: "在Wiki中搜索页面"},
- {Name: ToolWikiReadSourceDoc, Label: "精读源文档", Description: "使用知识点深入阅读特定原始文档"},
{Name: ToolWikiFlagIssue, Label: "标记Wiki问题", Description: "标记页面中存在的事实错误或合并冲突问题"},
{Name: ToolWikiWritePage, Label: "创建/覆盖Wiki", Description: "创建新页面或完全覆盖已有页面"},
{Name: ToolWikiReplaceText, Label: "局部替换Wiki", Description: "替换Wiki页面中的特定文本"},
@@ -115,10 +115,9 @@ func AvailableToolDefinitions() []AvailableTool {
// DefaultAllowedTools returns the default allowed tools list.
func DefaultAllowedTools() []string {
return []string{
- ToolKnowledgeSearch,
- ToolGrepChunks,
- ToolListKnowledgeChunks,
- ToolGetDocumentInfo,
+ ToolSearchKnowledge,
+ ToolReadDocument,
+ ToolListDocuments,
// Looking up what this user asked before is only ever a read of their
// own history, and it is what lets "上次你给我的那个配置" resolve at all
// without stuffing every past conversation into the context window.
@@ -141,8 +140,67 @@ const (
LegacyToolExecuteSkillScript = "execute_skill_script"
LegacyToolReadSkill = "read_skill"
LegacyToolReadSandboxFile = "read_sandbox_file"
+ // The pre-consolidation knowledge retrieval surface. knowledge_search and
+ // grep_chunks were folded into search_knowledge (mode=semantic|keyword|
+ // hybrid); list_knowledge_chunks, get_document_info and
+ // wiki_read_source_doc were folded into read_document.
+ LegacyToolKnowledgeSearch = "knowledge_search"
+ LegacyToolGrepChunks = "grep_chunks"
+ LegacyToolListKnowledgeChunks = "list_knowledge_chunks"
+ LegacyToolGetDocumentInfo = "get_document_info"
+ LegacyToolWikiReadSourceDoc = "wiki_read_source_doc"
)
+// legacyToolSuccessors maps retired allowlist entries to the tool that now
+// provides the capability. Stored agent configurations, presets and API
+// callers keep working without a data migration: NormalizeAllowedTools maps
+// them at registration time.
+var legacyToolSuccessors = map[string]string{
+ LegacyToolKnowledgeSearch: ToolSearchKnowledge,
+ LegacyToolGrepChunks: ToolSearchKnowledge,
+ LegacyToolListKnowledgeChunks: ToolReadDocument,
+ LegacyToolGetDocumentInfo: ToolReadDocument,
+ LegacyToolWikiReadSourceDoc: ToolReadDocument,
+}
+
+// SuccessorToolName returns the current tool that replaces a retired
+// allowlist entry, or name itself when it is not retired.
+func SuccessorToolName(name string) string {
+ if successor, ok := legacyToolSuccessors[name]; ok {
+ return successor
+ }
+ return name
+}
+
+// IsLegacyRetrievalTool reports whether name is a retired knowledge retrieval
+// tool that NormalizeAllowedTools rewrites to its successor.
+func IsLegacyRetrievalTool(name string) bool {
+ _, ok := legacyToolSuccessors[name]
+ return ok
+}
+
+// NormalizeAllowedTools rewrites retired tool names in an allowlist to their
+// successors and drops duplicates while preserving first-seen order.
+func NormalizeAllowedTools(allowed []string) []string {
+ if len(allowed) == 0 {
+ return allowed
+ }
+ seen := make(map[string]struct{}, len(allowed))
+ out := make([]string, 0, len(allowed))
+ for _, name := range allowed {
+ name = SuccessorToolName(name)
+ if name == "" {
+ continue
+ }
+ if _, ok := seen[name]; ok {
+ continue
+ }
+ seen[name] = struct{}{}
+ out = append(out, name)
+ }
+ return out
+}
+
// RetiredToolReplacement tells the model how to replace a removed tool.
// Empty when name was never a WeKnora tool.
func RetiredToolReplacement(name string) string {
@@ -153,6 +211,18 @@ func RetiredToolReplacement(name string) string {
return `read_skill is no longer available; use read_file(path="skill://<name>/<file_path or SKILL.md>")`
case LegacyToolReadSandboxFile:
return "read_sandbox_file is no longer available; use read_file(path=...)"
+ case LegacyToolKnowledgeSearch:
+ return "knowledge_search is no longer available; use search_knowledge(query=..., mode=\"semantic\"|\"hybrid\")"
+ case LegacyToolGrepChunks:
+ return "grep_chunks is no longer available; use search_knowledge(query=..., mode=\"keyword\") for exact " +
+ "terms, or read_document(id=dN, query=...) to search inside one document"
+ case LegacyToolListKnowledgeChunks:
+ return "list_knowledge_chunks is no longer available; use read_document(id=dN or cN, offset=..., limit=...)"
+ case LegacyToolGetDocumentInfo:
+ return "get_document_info is no longer available; read_document(id=dN) returns the document metadata " +
+ "with its first page"
+ case LegacyToolWikiReadSourceDoc:
+ return "wiki_read_source_doc is no longer available; use read_document(id=dN, query=... or offset=...)"
default:
return ""
}
diff --git a/internal/agent/tools/definitions_test.go b/internal/agent/tools/definitions_test.go
new file mode 100644
index 000000000..d4d5b7e94
--- /dev/null
+++ b/internal/agent/tools/definitions_test.go
@@ -0,0 +1,130 @@
+package tools
+
+import (
+ "reflect"
+ "strings"
+ "testing"
+
+ "github.com/Tencent/WeKnora/internal/types"
+)
+
+func TestNormalizeAllowedToolsMapsRetiredRetrievalTools(t *testing.T) {
+ got := NormalizeAllowedTools([]string{
+ ToolThinking,
+ LegacyToolKnowledgeSearch,
+ LegacyToolGrepChunks,
+ LegacyToolListKnowledgeChunks,
+ LegacyToolGetDocumentInfo,
+ ToolWikiSearch,
+ LegacyToolWikiReadSourceDoc,
+ ToolSearchKnowledge, // duplicate after mapping
+ "",
+ })
+ want := []string{ToolThinking, ToolSearchKnowledge, ToolReadDocument, ToolWikiSearch}
+ if !reflect.DeepEqual(got, want) {
+ t.Fatalf("NormalizeAllowedTools() = %v, want %v", got, want)
+ }
+}
+
+func TestNormalizeAllowedToolsKeepsUnknownNamesAndOrder(t *testing.T) {
+ in := []string{"mcp_custom_tool", ToolReadDocument, ToolWebSearch}
+ got := NormalizeAllowedTools(in)
+ if !reflect.DeepEqual(got, in) {
+ t.Fatalf("NormalizeAllowedTools() = %v, want unchanged %v", got, in)
+ }
+ if NormalizeAllowedTools(nil) != nil {
+ t.Fatal("nil allowlist must stay nil")
+ }
+}
+
+func TestSuccessorToolName(t *testing.T) {
+ cases := map[string]string{
+ LegacyToolKnowledgeSearch: ToolSearchKnowledge,
+ LegacyToolGrepChunks: ToolSearchKnowledge,
+ LegacyToolListKnowledgeChunks: ToolReadDocument,
+ LegacyToolGetDocumentInfo: ToolReadDocument,
+ LegacyToolWikiReadSourceDoc: ToolReadDocument,
+ ToolSearchKnowledge: ToolSearchKnowledge,
+ ToolWikiReadPage: ToolWikiReadPage,
+ }
+ for in, want := range cases {
+ if got := SuccessorToolName(in); got != want {
+ t.Errorf("SuccessorToolName(%q) = %q, want %q", in, got, want)
+ }
+ }
+ if !IsLegacyRetrievalTool(LegacyToolGrepChunks) || IsLegacyRetrievalTool(ToolSearchKnowledge) {
+ t.Fatal("IsLegacyRetrievalTool must flag only retired names")
+ }
+}
+
+func TestRetiredRetrievalToolsExplainTheirReplacement(t *testing.T) {
+ for name, want := range map[string]string{
+ LegacyToolKnowledgeSearch: ToolSearchKnowledge,
+ LegacyToolGrepChunks: "mode=\"keyword\"",
+ LegacyToolListKnowledgeChunks: ToolReadDocument,
+ LegacyToolGetDocumentInfo: ToolReadDocument,
+ LegacyToolWikiReadSourceDoc: ToolReadDocument,
+ } {
+ msg := RetiredToolReplacement(name)
+ if !strings.Contains(msg, want) {
+ t.Errorf("RetiredToolReplacement(%q) = %q, want mention of %q", name, msg, want)
+ }
+ }
+ if RetiredToolReplacement(ToolSearchKnowledge) != "" {
+ t.Fatal("active tools have no retirement message")
+ }
+}
+
+func TestDefaultAllowedToolsUseTheConsolidatedSurface(t *testing.T) {
+ defaults := DefaultAllowedTools()
+ for _, name := range []string{ToolSearchKnowledge, ToolReadDocument, ToolListDocuments} {
+ found := false
+ for _, d := range defaults {
+ if d == name {
+ found = true
+ }
+ }
+ if !found {
+ t.Errorf("default allowlist is missing %s: %v", name, defaults)
+ }
+ }
+ for _, d := range defaults {
+ if IsLegacyRetrievalTool(d) {
+ t.Errorf("default allowlist still names retired tool %s", d)
+ }
+ }
+ for _, def := range AvailableToolDefinitions() {
+ if IsLegacyRetrievalTool(def.Name) {
+ t.Errorf("UI tool list still exposes retired tool %s", def.Name)
+ }
+ }
+}
+
+func TestDocumentReadersAcceptWikiOnlyKnowledgeBases(t *testing.T) {
+ wikiOnly := types.KBCapabilities{Wiki: true}
+ for _, name := range []string{ToolReadDocument, ToolListDocuments, LegacyToolWikiReadSourceDoc} {
+ if !KBSatisfiesToolRequirements(wikiOnly, []string{name}) {
+ t.Errorf("%s must be usable on a wiki-only knowledge base", name)
+ }
+ }
+ if KBSatisfiesToolRequirements(wikiOnly, []string{ToolSearchKnowledge}) {
+ t.Error("search_knowledge needs a chunk index")
+ }
+}
+
+// Document readers must not widen a RAG agent's derived KB filter to
+// wiki-only bases, while still counting when they are the only KB tools.
+func TestDocumentReadersDoNotWidenTheDerivedKBFilter(t *testing.T) {
+ wikiOnly := types.KBCapabilities{Wiki: true}
+ rag := []string{ToolSearchKnowledge, ToolReadDocument, ToolListDocuments}
+ if KBSatisfiesToolRequirements(wikiOnly, rag) {
+ t.Fatal("a RAG tool set must not accept wiki-only knowledge bases")
+ }
+ wiki := []string{ToolWikiSearch, ToolWikiReadPage, ToolReadDocument}
+ if !KBSatisfiesToolRequirements(wikiOnly, wiki) {
+ t.Fatal("a wiki tool set must accept wiki-only knowledge bases")
+ }
+ if !KBSatisfiesToolRequirements(types.KBCapabilities{Vector: true}, []string{ToolReadDocument}) {
+ t.Fatal("a reader-only tool set still derives its own filter")
+ }
+}
diff --git a/internal/agent/tools/execution_policy.go b/internal/agent/tools/execution_policy.go
index 40686ece6..6ef89e416 100644
--- a/internal/agent/tools/execution_policy.go
+++ b/internal/agent/tools/execution_policy.go
@@ -5,11 +5,11 @@ package tools
// a file write must finish before the next shell or skill call starts.
func CanRunConcurrently(name string) bool {
switch name {
- case ToolKnowledgeSearch, ToolGrepChunks, ToolListKnowledgeChunks,
- ToolQueryKnowledgeGraph, ToolGetDocumentInfo, ToolSearchConversations,
+ case ToolSearchKnowledge, ToolReadDocument, ToolListDocuments,
+ ToolQueryKnowledgeGraph, ToolSearchConversations,
ToolSearchMemory, ToolDataSchema, ToolWebSearch, ToolWebFetch,
ToolReadFile, ToolListSandboxFiles,
- ToolWikiSearch, ToolWikiReadPage, ToolWikiReadSourceDoc, ToolWikiReadIssue:
+ ToolWikiSearch, ToolWikiReadPage, ToolWikiReadIssue:
return true
default:
return false
diff --git a/internal/agent/tools/faq_snippet.go b/internal/agent/tools/faq_snippet.go
index a97a5302a..7a9ab3cad 100644
--- a/internal/agent/tools/faq_snippet.go
+++ b/internal/agent/tools/faq_snippet.go
@@ -66,7 +66,7 @@ func faqFieldsEmpty(meta *types.FAQChunkMetadata) bool {
return meta.StandardQuestion == "" && len(meta.SimilarQuestions) == 0 && len(meta.Answers) == 0
}
-// writeFAQMetadataXML emits a nested <faq> block (used inside knowledge_search <chunk>).
+// writeFAQMetadataXML emits a nested <faq> block (used inside search_knowledge <chunk>).
func writeFAQMetadataXML(b *strings.Builder, meta *types.FAQChunkMetadata) {
if faqFieldsEmpty(meta) {
return
@@ -76,7 +76,7 @@ func writeFAQMetadataXML(b *strings.Builder, meta *types.FAQChunkMetadata) {
b.WriteString("</faq>\n")
}
-// writeFAQEntryXML emits a top-level FAQ entry for list_knowledge_chunks (not wrapped in <chunk>).
+// writeFAQEntryXML emits a top-level FAQ entry for read_document (not wrapped in <chunk>).
func writeFAQEntryXML(b *strings.Builder, c *types.Chunk) {
if c == nil || c.ChunkType != types.ChunkTypeFAQ {
return
@@ -135,7 +135,7 @@ func appendFAQChunkData(chunkData map[string]interface{}, c *types.Chunk) {
}
}
-// Bounds for retrieval-tool match snippets (grep_chunks, knowledge_search).
+// Bounds for retrieval-tool match snippets (search_knowledge, read_document).
const (
snippetContextRunes = 200
snippetMaxMatchRunes = 200
@@ -143,7 +143,7 @@ const (
snippetMaxAnswerRunes = 600
)
-// faqMatchSnippetFromQueries builds "Q: … | A: …" for knowledge_search hits.
+// faqMatchSnippetFromQueries builds "Q: … | A: …" for search_knowledge hits.
func faqMatchSnippetFromQueries(meta *types.FAQChunkMetadata, queries []string) string {
if meta == nil {
return ""
@@ -155,7 +155,7 @@ func faqMatchSnippetFromQueries(meta *types.FAQChunkMetadata, queries []string)
return formatFAQMatchSnippet(question, meta.Answers)
}
-// faqMatchSnippet builds "Q: … | A: …" for grep_chunks regex hits.
+// faqMatchSnippet builds "Q: … | A: …" for read_document regex hits.
func faqMatchSnippet(chunk *types.Chunk, compiled []*regexp.Regexp) string {
if chunk == nil {
return ""
diff --git a/internal/agent/tools/get_document_info.go b/internal/agent/tools/get_document_info.go
deleted file mode 100644
index 27b31f498..000000000
--- a/internal/agent/tools/get_document_info.go
+++ /dev/null
@@ -1,420 +0,0 @@
-package tools
-
-import (
- "context"
- "encoding/json"
- "fmt"
- "strings"
- "sync"
-
- "github.com/Tencent/WeKnora/internal/types"
- "github.com/Tencent/WeKnora/internal/types/interfaces"
-)
-
-var getDocumentInfoTool = BaseTool{
- name: ToolGetDocumentInfo,
- description: `Retrieve detailed metadata information about documents.
-
-## When to Use
-
-Use this tool when:
-- Need to understand document basic information (title, type, size, etc.)
-- Check if document exists and is available
-- Batch query metadata for multiple documents
-- Understand document processing status
-
-Do not use when:
-- Need document content (use knowledge_search)
-- Need specific text chunks (search results already contain full content)
-
-
-## Returned Information
-
-- Basic info: title, description, source type
-- File info: filename, type, size
-- Processing status: whether processed, chunk count
-- Metadata: custom tags and properties
-
-
-## Notes
-
-- Concurrent query for multiple documents provides better performance
-- Returns complete document metadata, not just title
-- Can check document processing status (parse_status)
-
-## IDs
-- knowledge_ids: regular documents, using the short dN IDs from retrieval results
-- faq_ids: individual FAQ entries, using the short cN chunk IDs. Returns the standard question and answers, not the container title.`,
- schema: json.RawMessage(`{
- "type": "object",
- "properties": {
- "knowledge_ids": {
- "type": "array",
- "items": { "type": "string" },
- "description": "Short dN document IDs for regular documents"
- },
- "faq_ids": {
- "type": "array",
- "items": { "type": "string" },
- "description": "Short cN FAQ chunk IDs from retrieval results. Use instead of knowledge_ids for a single FAQ Q&A."
- }
- }
-}`),
-}
-
-// GetDocumentInfoInput defines the input parameters for get document info tool.
-// Either knowledge_ids or faq_ids may be provided (at least one); both are optional in the schema.
-type GetDocumentInfoInput struct {
- KnowledgeIDs []string `json:"knowledge_ids,omitempty"`
- FAQIDs []string `json:"faq_ids,omitempty"`
-}
-
-// GetDocumentInfoTool retrieves detailed information about a document/knowledge
-type GetDocumentInfoTool struct {
- BaseTool
- knowledgeService interfaces.KnowledgeService
- chunkService interfaces.ChunkService
- searchTargets types.SearchTargets // Pre-computed unified search targets with KB-tenant mapping
-}
-
-// NewGetDocumentInfoTool creates a new get document info tool
-func NewGetDocumentInfoTool(
- knowledgeService interfaces.KnowledgeService,
- chunkService interfaces.ChunkService,
- searchTargets types.SearchTargets,
-) *GetDocumentInfoTool {
- return &GetDocumentInfoTool{
- BaseTool: getDocumentInfoTool,
- knowledgeService: knowledgeService,
- chunkService: chunkService,
- searchTargets: searchTargets,
- }
-}
-
-// Execute retrieves document information with concurrent processing
-func (t *GetDocumentInfoTool) Execute(ctx context.Context, args json.RawMessage) (*types.ToolResult, error) {
- // Parse args from json.RawMessage
- var input GetDocumentInfoInput
- if err := json.Unmarshal(args, &input); err != nil {
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("Failed to parse args: %v", err),
- }, err
- }
-
- knowledgeIDs := input.KnowledgeIDs
- faqIDs := input.FAQIDs
- if len(knowledgeIDs) == 0 && len(faqIDs) == 0 {
- return &types.ToolResult{
- Success: false,
- Error: "knowledge_ids or faq_ids is required (non-empty array)",
- }, fmt.Errorf("missing ids")
- }
-
- type docInfo struct {
- knowledge *types.Knowledge
- chunk *types.Chunk
- faqMeta *types.FAQChunkMetadata
- chunkCount int
- err error
- }
-
- var wg sync.WaitGroup
- var mu sync.Mutex
- results := make(map[string]*docInfo)
- enabled := true
-
- for _, faqID := range faqIDs {
- faqID = strings.TrimSpace(faqID)
- if faqID == "" {
- continue
- }
- wg.Add(1)
- go func(id string) {
- defer wg.Done()
- chunk, err := authorizeChunkInSearchTargets(
- ctx, t.searchTargets, id, t.chunkService, t.knowledgeService,
- )
- if err != nil {
- mu.Lock()
- results["faq:"+id] = &docInfo{err: fmt.Errorf("FAQ entry is not accessible: %v", err)}
- mu.Unlock()
- return
- }
- var meta *types.FAQChunkMetadata
- if chunk.ChunkType == types.ChunkTypeFAQ {
- meta, _ = chunk.FAQMetadata()
- }
- mu.Lock()
- results["faq:"+id] = &docInfo{chunk: chunk, faqMeta: meta, chunkCount: 1}
- mu.Unlock()
- }(faqID)
- }
-
- for _, knowledgeID := range knowledgeIDs {
- wg.Add(1)
- go func(id string) {
- defer wg.Done()
-
- knowledge, err := authorizeKnowledgeInSearchTargets(
- ctx, t.searchTargets, id, t.knowledgeService,
- )
- if err != nil {
- mu.Lock()
- results[id] = &docInfo{
- err: fmt.Errorf("failed to get document info: %v", err),
- }
- mu.Unlock()
- return
- }
-
- // Use knowledge's actual tenant_id for chunk query (supports cross-tenant shared KB).
- // Keep chunk-type filter aligned with list_knowledge_chunks so the
- // "chunk_count" reported here matches what that tool can page over.
- _, total, err := t.chunkService.GetRepository().
- ListPagedChunksByKnowledgeID(ctx, knowledge.TenantID, id, &types.Pagination{
- Page: 1,
- PageSize: 1,
- }, []types.ChunkType{types.ChunkTypeText, types.ChunkTypeFAQ}, nil, "", "", "", "", &enabled)
- if err != nil {
- mu.Lock()
- results[id] = &docInfo{
- err: fmt.Errorf("failed to get document info: %v", err),
- }
- mu.Unlock()
- return
- }
- chunkCount := int(total)
-
- mu.Lock()
- results[id] = &docInfo{
- knowledge: knowledge,
- chunkCount: chunkCount,
- }
- mu.Unlock()
- }(knowledgeID)
- }
-
- wg.Wait()
-
- requested := len(knowledgeIDs) + len(faqIDs)
- successDocs := make([]*docInfo, 0)
- var errors []string
-
- for _, knowledgeID := range knowledgeIDs {
- result := results[knowledgeID]
- if result == nil {
- errors = append(errors, fmt.Sprintf("%s: not found", knowledgeID))
- continue
- }
- if result.err != nil {
- errors = append(errors, fmt.Sprintf("%s: %v", knowledgeID, result.err))
- } else if result.knowledge != nil {
- successDocs = append(successDocs, result)
- }
- }
- for _, faqID := range faqIDs {
- faqID = strings.TrimSpace(faqID)
- if faqID == "" {
- continue
- }
- result := results["faq:"+faqID]
- if result == nil {
- errors = append(errors, fmt.Sprintf("faq:%s: not found", faqID))
- continue
- }
- if result.err != nil {
- errors = append(errors, fmt.Sprintf("faq:%s: %v", faqID, result.err))
- } else if result.chunk != nil {
- successDocs = append(successDocs, result)
- }
- }
-
- if len(successDocs) == 0 {
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("Failed to retrieve any document info. Errors: %v", errors),
- }, fmt.Errorf("all document retrievals failed")
- }
-
- output := "=== Document Info ===\n\n"
- output += fmt.Sprintf("Successfully retrieved %d / %d entries\n\n", len(successDocs), requested)
-
- if len(errors) > 0 {
- output += "=== Partial Failures ===\n"
- for _, errMsg := range errors {
- output += fmt.Sprintf(" - %s\n", errMsg)
- }
- output += "\n"
- }
-
- formattedDocs := make([]map[string]interface{}, 0, len(successDocs))
- for i, doc := range successDocs {
- output += fmt.Sprintf("[Entry #%d]\n", i+1)
-
- if doc.chunk != nil {
- formatted := formatFAQEntryInfo(&output, doc.chunk, doc.faqMeta)
- formattedDocs = append(formattedDocs, formatted)
- continue
- }
-
- k := doc.knowledge
- output += fmt.Sprintf(" ID: %s\n", k.ID)
- output += fmt.Sprintf(" Title: %s\n", k.Title)
-
- if k.Description != "" {
- output += fmt.Sprintf(" Description: %s\n", k.Description)
- }
-
- output += fmt.Sprintf(" Source: %s\n", formatSource(k.Type, k.Source))
-
- if k.FileName != "" {
- output += fmt.Sprintf(" File Name: %s\n", k.FileName)
- output += fmt.Sprintf(" File Type: %s\n", k.FileType)
- output += fmt.Sprintf(" File Size: %s\n", formatFileSize(k.FileSize))
- }
-
- output += fmt.Sprintf(" Parse Status: %s\n", formatParseStatus(k.ParseStatus))
- output += fmt.Sprintf(" Chunk Count: %d\n", doc.chunkCount)
-
- if k.Metadata != nil {
- if metadata, err := k.Metadata.Map(); err == nil && len(metadata) > 0 {
- output += " Metadata:\n"
- for key, value := range metadata {
- output += fmt.Sprintf(" - %s: %v\n", key, value)
- }
- }
- }
-
- output += "\n"
-
- formattedDocs = append(formattedDocs, map[string]interface{}{
- "knowledge_id": k.ID,
- "title": k.Title,
- "description": k.Description,
- "type": k.Type,
- "source": k.Source,
- "file_name": k.FileName,
- "file_type": k.FileType,
- "file_size": k.FileSize,
- "parse_status": k.ParseStatus,
- "chunk_count": doc.chunkCount,
- "metadata": k.GetMetadata(),
- "is_faq": false,
- })
- }
-
- var firstTitle string
- if len(formattedDocs) > 0 {
- if t, ok := formattedDocs[0]["title"].(string); ok {
- firstTitle = t
- }
- }
-
- return &types.ToolResult{
- Success: true,
- Output: output,
- Data: map[string]interface{}{
- "documents": formattedDocs,
- "total_docs": len(successDocs),
- "requested": requested,
- "errors": errors,
- "display_type": "document_info",
- "title": firstTitle,
- },
- }, nil
-}
-
-func formatFAQEntryInfo(output *string, chunk *types.Chunk, meta *types.FAQChunkMetadata) map[string]interface{} {
- title := faqStandardQuestion(chunk)
- if title == "" && meta != nil {
- title = strings.TrimSpace(meta.StandardQuestion)
- }
- if title == "" {
- title = "FAQ Entry"
- }
-
- *output += fmt.Sprintf(" FAQ ID: %s\n", chunk.ID)
- *output += fmt.Sprintf(" Question: %s\n", title)
- if chunk.KnowledgeID != "" {
- *output += fmt.Sprintf(" Container ID: %s\n", chunk.KnowledgeID)
- }
- if meta != nil && len(meta.Answers) > 0 {
- *output += " Answers:\n"
- for _, ans := range meta.Answers {
- *output += fmt.Sprintf(" - %s\n", ans)
- }
- }
- if meta != nil && len(meta.SimilarQuestions) > 0 {
- display, omitted := truncateSimilarQuestionsForDisplay(meta.SimilarQuestions)
- *output += " Similar Questions:\n"
- for _, sq := range display {
- *output += fmt.Sprintf(" - %s\n", sq)
- }
- if omitted > 0 {
- *output += fmt.Sprintf(" ... and %d more omitted\n", omitted)
- }
- }
- *output += "\n"
-
- entry := map[string]interface{}{
- "faq_id": chunk.ID,
- "knowledge_id": chunk.KnowledgeID,
- "title": title,
- "faq_question": title,
- "type": "faq",
- "is_faq": true,
- "chunk_count": 1,
- }
- if meta != nil {
- if len(meta.Answers) > 0 {
- entry["faq_answers"] = meta.Answers
- }
- appendSimilarQuestionsToChunkData(entry, meta.SimilarQuestions)
- }
- return entry
-}
-
-func formatSource(knowledgeType, source string) string {
- switch knowledgeType {
- case "file":
- return "File Upload"
- case "url":
- return fmt.Sprintf("URL: %s", source)
- case "passage":
- return "Text Input"
- default:
- return knowledgeType
- }
-}
-
-func formatFileSize(size int64) string {
- if size == 0 {
- return "Unknown"
- }
- const unit = 1024
- if size < unit {
- return fmt.Sprintf("%d B", size)
- }
- div, exp := int64(unit), 0
- for n := size / unit; n >= unit; n /= unit {
- div *= unit
- exp++
- }
- return fmt.Sprintf("%.1f %cB", float64(size)/float64(div), "KMGTPE"[exp])
-}
-
-func formatParseStatus(status string) string {
- switch status {
- case "pending":
- return "Pending"
- case "processing":
- return "Processing"
- case "completed", "success":
- return "Completed"
- case "failed":
- return "Failed"
- default:
- return status
- }
-}
diff --git a/internal/agent/tools/grep_chunks.go b/internal/agent/tools/grep_chunks.go
deleted file mode 100644
index 9f5dc05e4..000000000
--- a/internal/agent/tools/grep_chunks.go
+++ /dev/null
@@ -1,1021 +0,0 @@
-package tools
-
-import (
- "context"
- "encoding/json"
- "fmt"
- "math"
- "regexp"
- "sort"
- "strings"
- "sync"
-
- "github.com/Tencent/WeKnora/internal/logger"
- "github.com/Tencent/WeKnora/internal/searchutil"
- "github.com/Tencent/WeKnora/internal/types"
- "gorm.io/gorm"
-)
-
-var grepChunksTool = BaseTool{
- name: ToolGrepChunks,
- description: `Search knowledge base chunk content with a single POSIX regular expression, applied directly in the database (PostgreSQL ~* / MySQL/SQLite REGEXP, case-insensitive). Behaves like ` + "`grep -E -i`" + `.
-Pack multiple concepts into ONE regex using ` + "`|`" + ` alternation — do not call this tool repeatedly for synonyms.
-Returns matching chunks with a short cN chunk source ID, a parent dN document ID, and a <match> snippet around the first match.
-Examples:
-- Alternation (RECOMMENDED): "stardust|skyvault|psionic" (matches any of the words)
-- Multiple terms in order: "psionic.*engine" (matches both words in order)
-- Word boundary / anchor: "\\brag\\b" or "^chapter\\s+\\d+"
-- Plain text: "engine" (matches literal substring anywhere in chunk content)
-IMPORTANT — JSON escaping: every backslash in a regex MUST be written as \\ inside the JSON tool arguments (e.g. to search for literal "C++" write "C\\+\\+", NOT "C\+\+"; for "\d+" write "\\d+"). Plain "\+" / "\d" etc. are invalid JSON escapes and will fail to parse.
-Use this to locate candidate chunks by exact identifiers, error codes, product names, or recurring terms.
-
-## Deep read after grep:
-- **FAQ hit** (chunk type faq): call list_knowledge_chunks with **faq_id=cN** from the grep result (NOT the parent dN document ID).
-- **Document hit**: call list_knowledge_chunks with **knowledge_id=dN**, or get_document_info with **knowledge_ids=[dN]**.`,
- schema: json.RawMessage(`{
- "type": "object",
- "properties": {
- "query": {
- "type": "string",
- "description": "A single POSIX regex applied directly to chunk content (case-insensitive). Combine multiple concepts with \"|\" alternation in ONE regex (e.g. \"stardust|skyvault|psionic\") — do not split into multiple calls.",
- "minLength": 1
- }
- },
- "required": ["query"]
-}`),
-}
-
-// GrepChunksInput defines the input parameters for grep chunks tool.
-// The canonical parameter is a single `query` string (a regex with optional
-// `|` alternation), matching real `grep -E` semantics. Legacy array forms
-// (`queries`, `patterns`) and the singular `pattern` alias remain accepted
-// so older model outputs or external callers don't break silently — they
-// are joined together into a single alternation regex before execution.
-type GrepChunksInput struct {
- Query string `json:"query,omitempty"`
-}
-
-// GrepChunksTool performs regex pattern matching across knowledge base chunks.
-// PostgreSQL: uses the case-insensitive POSIX operator ~*.
-// MySQL/SQLite: falls back to REGEXP.
-//
-// The tool tracks previously-returned chunk IDs per-instance (one instance per
-// agent session) so that a subsequent search hitting the same chunk can be
-// rendered compactly with an `already_seen="true"` marker instead of replaying
-// the snippet, mirroring the UX of wiki_search.
-type GrepChunksTool struct {
- BaseTool
- db *gorm.DB
- searchTargets types.SearchTargets
-
- mu sync.Mutex
- seenChunks map[string]bool
-}
-
-// NewGrepChunksTool creates a new grep chunks tool
-func NewGrepChunksTool(db *gorm.DB, searchTargets types.SearchTargets) *GrepChunksTool {
- return &GrepChunksTool{
- BaseTool: grepChunksTool,
- db: db,
- searchTargets: searchTargets,
- seenChunks: make(map[string]bool),
- }
-}
-
-// Execute executes the grep chunks tool
-func (t *GrepChunksTool) Execute(ctx context.Context, args json.RawMessage) (*types.ToolResult, error) {
- logger.Infof(ctx, "[Tool][GrepChunks] Execute started")
-
- var input GrepChunksInput
- if err := json.Unmarshal(args, &input); err != nil {
- logger.Errorf(ctx, "[Tool][GrepChunks] Failed to parse args: %v", err)
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("Failed to parse args: %v", err),
- }, err
- }
-
- // Resolve the canonical single-string `query`, falling back to legacy
- // aliases. Legacy array inputs are joined with `|` so they degrade into
- // a single alternation regex — preserving the previous "match ANY"
- // semantics without requiring multiple DB scans.
- query := strings.TrimSpace(input.Query)
-
- if query == "" {
- logger.Errorf(ctx, "[Tool][GrepChunks] Missing or empty query parameter")
- return &types.ToolResult{
- Success: false,
- Error: "query parameter is required and must be a non-empty regex string",
- }, fmt.Errorf("missing query parameter")
- }
-
- // Compile with (?i) prefix for case-insensitive Go-side matching.
- // Compilation also validates the regex syntax before we send it to the DB.
- re, err := regexp.Compile("(?i)" + query)
- if err != nil {
- logger.Errorf(ctx, "[Tool][GrepChunks] Invalid regex %q: %v", query, err)
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("invalid regex query %q: %v", query, err),
- }, err
- }
- queries := []string{query}
- compiled := []*regexp.Regexp{re}
-
- // Result count is controlled by the backend, not the caller — keep it
- // bounded so the LLM context stays small regardless of regex breadth.
- const limit = 30
-
- kbTenantMap := t.searchTargets.GetKBTenantMap()
- fullKBIDs, knowledgeIDs, tagTargets := t.resolveGrepScope()
- kbIDsForMeta := fullKBIDs
- if len(kbIDsForMeta) == 0 {
- kbIDsForMeta = t.searchTargets.GetAllKnowledgeBaseIDs()
- }
-
- logger.Infof(ctx, "[Tool][GrepChunks] Queries: %v, Limit: %d, fullKBs: %d, knowledgeIDs: %d, tagScopes: %d",
- queries, limit, len(fullKBIDs), len(knowledgeIDs), len(tagTargets))
-
- results, err := t.searchChunks(ctx, queries, fullKBIDs, knowledgeIDs, tagTargets, kbTenantMap)
- if err != nil {
- logger.Errorf(ctx, "[Tool][GrepChunks] Search failed: %v", err)
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("Search failed: %v", err),
- }, err
- }
-
- logger.Infof(ctx, "[Tool][GrepChunks] Found %d matching chunks", len(results))
-
- deduplicatedResults := t.deduplicateChunks(ctx, results)
- logger.Infof(ctx, "[Tool][GrepChunks] After deduplication: %d chunks (from %d)",
- len(deduplicatedResults), len(results))
-
- // Score chunks using compiled regex (counts + earliest-position boost).
- scoredResults := t.scoreChunks(ctx, deduplicatedResults, compiled)
-
- finalResults := scoredResults
- if len(scoredResults) > 10 {
- mmrK := len(scoredResults)
- if limit > 0 && mmrK > limit {
- mmrK = limit
- }
- logger.Debugf(ctx, "[Tool][GrepChunks] Applying MMR: k=%d, lambda=0.7, input=%d results",
- mmrK, len(scoredResults))
- mmrResults := t.applyMMR(ctx, scoredResults, mmrK, 0.7)
- if len(mmrResults) > 0 {
- finalResults = mmrResults
- logger.Infof(ctx, "[Tool][GrepChunks] MMR completed: %d results selected", len(finalResults))
- }
- }
-
- sort.Slice(finalResults, func(i, j int) bool {
- // Title matches rank above everything else (see chunkWithTitle.TitleMatch).
- if finalResults[i].TitleMatch != finalResults[j].TitleMatch {
- return finalResults[i].TitleMatch
- }
- if finalResults[i].MatchedPatterns != finalResults[j].MatchedPatterns {
- return finalResults[i].MatchedPatterns > finalResults[j].MatchedPatterns
- }
- if finalResults[i].MatchScore != finalResults[j].MatchScore {
- return finalResults[i].MatchScore > finalResults[j].MatchScore
- }
- return finalResults[i].ChunkIndex < finalResults[j].ChunkIndex
- })
-
- if len(finalResults) > limit {
- finalResults = finalResults[:limit]
- }
-
- // chunk_results: per-chunk hits for UI detail (grouped by knowledge_id on the
- // frontend). knowledge_results: pre-aggregated per document for summaries;
- // document_count is the full distinct-document total even when the knowledge
- // list is truncated for payload size.
- chunkResults := buildGrepChunkResults(finalResults, compiled)
- aggregatedResults := t.aggregateByKnowledge(finalResults, queries, compiled)
- documentCount := len(aggregatedResults)
- knowledgeResultsForUI := aggregatedResults
- const maxKnowledgeRows = 20
- if len(knowledgeResultsForUI) > maxKnowledgeRows {
- knowledgeResultsForUI = knowledgeResultsForUI[:maxKnowledgeRows]
- }
-
- output := t.formatOutput(ctx, finalResults, queries, compiled)
-
- return &types.ToolResult{
- Success: true,
- Output: output,
- Data: map[string]interface{}{
- "query": query,
- "queries": queries, // legacy alias for older frontends
- "patterns": queries, // legacy alias for older frontends
- "chunk_results": chunkResults,
- "knowledge_results": knowledgeResultsForUI,
- "result_count": len(chunkResults),
- "document_count": documentCount,
- "total_matches": len(finalResults),
- "knowledge_base_ids": kbIDsForMeta,
- "limit": limit,
- "max_results": limit, // legacy alias
- "display_type": "grep_results",
- },
- }, nil
-}
-
-type chunkWithTitle struct {
- types.Chunk
- KnowledgeTitle string `json:"knowledge_title" gorm:"column:knowledge_title"`
- MatchScore float64 `json:"match_score" gorm:"column:match_score"`
- MatchedPatterns int `json:"matched_patterns"`
- // TitleMatch is true when the query regex matches the owning knowledge's
- // TITLE (not just chunk body). A doc literally titled "图片素材" is the most
- // on-topic hit for the query "图片素材", yet its body may mention the term far
- // less than long FAQ docs that repeat it — so title hits are floated to the
- // very top of both the per-chunk and per-knowledge ordering.
- TitleMatch bool `json:"title_match"`
- TotalChunkCount int `json:"total_chunk_count" gorm:"column:total_chunk_count"`
-}
-
-// regexOperatorForDialect returns the SQL operator used to apply a POSIX
-// regular expression to a text column for the current dialect.
-// PostgreSQL ~* is case-insensitive by default; MySQL/SQLite REGEXP relies on
-// collation / driver extensions.
-func (t *GrepChunksTool) regexOperatorForDialect() string {
- switch t.db.Dialector.Name() {
- case "postgres":
- return "~*"
- default:
- // MySQL, SQLite (with the go-sqlite3 REGEXP extension), or anything else
- // that understands the REGEXP keyword.
- return "REGEXP"
- }
-}
-
-// resolveGrepScope splits search targets into full-KB, specific-knowledge, and
-// tag-constrained KB scopes. Tag scopes stay as tag IDs so the chunk query can
-// use the knowledge_tag_relations index instead of expanding a large tag into a
-// huge knowledge_id IN list.
-func (t *GrepChunksTool) resolveGrepScope() (fullKBIDs, knowledgeIDs []string, tagTargets []*types.SearchTarget) {
- seenKB := make(map[string]bool)
- seenKnowledge := make(map[string]bool)
- seenTagScope := make(map[string]bool)
- for _, target := range t.searchTargets {
- if target == nil || target.KnowledgeBaseID == "" {
- continue
- }
- // A target that carries a resolved document whitelist is already the
- // intersection of its mention and any tag scope, so it must be grepped
- // by knowledge ID. Falling back to the tag branch would widen the
- // search to every document carrying the tag.
- targetKnowledgeIDs, targetTagIDs := searchTargetScope(target)
- switch {
- case len(targetTagIDs) > 0:
- tenantID := target.TenantID
- if tenantID == 0 {
- tenantID = t.searchTargets.GetTenantIDForKB(target.KnowledgeBaseID)
- }
- tagIDs := targetTagIDs
- if len(tagIDs) == 0 || tenantID == 0 {
- continue
- }
- scopeKey := fmt.Sprintf("%s:%d:%s", target.KnowledgeBaseID, tenantID, strings.Join(tagIDs, "\x00"))
- if seenTagScope[scopeKey] {
- continue
- }
- seenTagScope[scopeKey] = true
- tagTargets = append(tagTargets, &types.SearchTarget{
- Type: types.SearchTargetTypeKnowledgeBase,
- KnowledgeBaseID: target.KnowledgeBaseID,
- TenantID: tenantID,
- TagIDs: tagIDs,
- })
- case len(targetKnowledgeIDs) > 0:
- for _, kid := range targetKnowledgeIDs {
- if !seenKnowledge[kid] {
- seenKnowledge[kid] = true
- knowledgeIDs = append(knowledgeIDs, kid)
- }
- }
- default:
- if !seenKB[target.KnowledgeBaseID] {
- seenKB[target.KnowledgeBaseID] = true
- fullKBIDs = append(fullKBIDs, target.KnowledgeBaseID)
- }
- }
- }
- return fullKBIDs, knowledgeIDs, tagTargets
-}
-
-func dedupNonEmptyStrings(values []string) []string {
- seen := make(map[string]bool, len(values))
- out := make([]string, 0, len(values))
- for _, value := range values {
- if value == "" || seen[value] {
- continue
- }
- seen[value] = true
- out = append(out, value)
- }
- return out
-}
-
-// scopeClause builds the SQL predicate restricting chunks to the active scope.
-// Specific knowledge IDs, tag-constrained KB scopes, and full-KB (kb, tenant)
-// pairs are OR'd together so a turn that mentions @KB + @tag + @file searches
-// every requested scope. Returns ("", nil) when no usable scope exists.
-func scopeClause(
- kbIDs, knowledgeIDs []string,
- tagTargets []*types.SearchTarget,
- kbTenantMap map[string]uint64,
-) (string, []interface{}) {
- var clauses []string
- var args []interface{}
- if len(knowledgeIDs) > 0 {
- clauses = append(clauses, "chunks.knowledge_id IN ?")
- args = append(args, knowledgeIDs)
- }
- for _, target := range tagTargets {
- if target == nil || target.KnowledgeBaseID == "" || len(target.TagIDs) == 0 {
- continue
- }
- tenantID := target.TenantID
- if tenantID == 0 {
- tenantID = kbTenantMap[target.KnowledgeBaseID]
- }
- if tenantID == 0 {
- continue
- }
- clauses = append(clauses,
- "(chunks.knowledge_base_id = ? AND chunks.tenant_id = ? AND EXISTS ("+
- "SELECT 1 FROM knowledge_tag_relations ktr "+
- "WHERE ktr.knowledge_id = chunks.knowledge_id AND ktr.tag_id IN ?"+
- "))")
- args = append(args, target.KnowledgeBaseID, tenantID, target.TagIDs)
- }
- for _, kbID := range kbIDs {
- tenantID := kbTenantMap[kbID]
- if tenantID == 0 {
- continue
- }
- clauses = append(clauses, "(chunks.knowledge_base_id = ? AND chunks.tenant_id = ?)")
- args = append(args, kbID, tenantID)
- }
- if len(clauses) == 0 {
- return "", nil
- }
- return "(" + strings.Join(clauses, " OR ") + ")", args
-}
-
-// searchChunks performs the database search using regex queries.
-func (t *GrepChunksTool) searchChunks(
- ctx context.Context,
- queries []string,
- kbIDs []string,
- knowledgeIDs []string,
- tagTargets []*types.SearchTarget,
- kbTenantMap map[string]uint64,
-) ([]chunkWithTitle, error) {
- if len(kbIDs) == 0 && len(knowledgeIDs) == 0 && len(tagTargets) == 0 {
- logger.Warnf(ctx, "[Tool][GrepChunks] No kbIDs, knowledgeIDs, or tag scopes specified, returning empty results")
- return nil, nil
- }
-
- regexOp := t.regexOperatorForDialect()
-
- query := t.db.WithContext(ctx).Table("chunks").
- Select("chunks.id, chunks.content, chunks.chunk_index, chunks.knowledge_id, "+
- "chunks.knowledge_base_id, chunks.chunk_type, chunks.metadata, chunks.created_at, "+
- "knowledges.title as knowledge_title").
- Joins("JOIN knowledges ON chunks.knowledge_id = knowledges.id").
- Where("chunks.is_enabled = ?", true).
- Where("chunks.deleted_at IS NULL").
- Where("knowledges.deleted_at IS NULL")
-
- // Combine specific knowledge IDs, tag scopes, and full-KB scopes with OR so
- // that mixing @KB with @tag/@file searches BOTH, mirroring knowledge_search's
- // concurrent fan-out instead of dropping one side.
- scopeSQL, scopeArgs := scopeClause(kbIDs, knowledgeIDs, tagTargets, kbTenantMap)
- if scopeSQL == "" {
- logger.Warnf(ctx, "[Tool][GrepChunks] No valid scope (kb-tenant pairs / knowledge IDs)")
- return nil, nil
- }
- logger.Infof(ctx, "[Tool][GrepChunks] Scope: %d knowledge IDs, %d tag scopes, %d KBs",
- len(knowledgeIDs), len(tagTargets), len(kbIDs))
- query = query.Where(scopeSQL, scopeArgs...)
-
- // For MySQL/SQLite REGEXP case-insensitivity we rely on the column's default
- // collation (utf8mb4_general_ci etc.) OR the driver's REGEXP implementation,
- // which mirrors what wiki_search already ships in this codebase.
- var regexConditions []string
- var regexArgs []interface{}
- for _, q := range queries {
- // Match the regex against either the chunk body OR the owning
- // knowledge's title, so a doc whose title matches (e.g. titled
- // "图片素材") surfaces even when its body rarely repeats the term.
- regexConditions = append(regexConditions,
- fmt.Sprintf("(chunks.content %s ? OR knowledges.title %s ?)", regexOp, regexOp))
- regexArgs = append(regexArgs, q, q)
- }
- query = query.Where("("+strings.Join(regexConditions, " OR ")+")", regexArgs...)
-
- const maxFetchLimit = 500
-
- var results []chunkWithTitle
- if err := query.Order("chunks.created_at DESC").Limit(maxFetchLimit).Find(&results).Error; err != nil {
- logger.Errorf(ctx, "[Tool][GrepChunks] Failed to fetch results: %v", err)
- return nil, err
- }
-
- if len(results) > 0 {
- knowledgeIDSet := make(map[string]struct{})
- for _, r := range results {
- if r.KnowledgeID != "" {
- knowledgeIDSet[r.KnowledgeID] = struct{}{}
- }
- }
- uniqueKnowledgeIDs := make([]string, 0, len(knowledgeIDSet))
- for kid := range knowledgeIDSet {
- uniqueKnowledgeIDs = append(uniqueKnowledgeIDs, kid)
- }
-
- type countRow struct {
- KnowledgeID string `gorm:"column:knowledge_id"`
- Count int `gorm:"column:cnt"`
- }
- var counts []countRow
- if err := t.db.WithContext(ctx).Table("chunks").
- Select("knowledge_id, COUNT(*) AS cnt").
- Where("knowledge_id IN ?", uniqueKnowledgeIDs).
- Where("is_enabled = ?", true).
- Where("deleted_at IS NULL").
- Group("knowledge_id").
- Find(&counts).Error; err != nil {
- logger.Warnf(ctx, "[Tool][GrepChunks] Failed to fetch chunk counts, skipping: %v", err)
- } else {
- countMap := make(map[string]int, len(counts))
- for _, c := range counts {
- countMap[c.KnowledgeID] = c.Count
- }
- for i := range results {
- results[i].TotalChunkCount = countMap[results[i].KnowledgeID]
- }
- }
- }
-
- return results, nil
-}
-
-// formatOutput emits per-chunk XML with <match_snippet> and <query_hit>
-// elements, mirroring the wiki_search output shape. Chunks that were already
-// surfaced by a previous call to this tool in the same session are rendered
-// compactly with `already_seen="true"` so the LLM doesn't waste context
-// re-reading the same snippet.
-func (t *GrepChunksTool) formatOutput(
- ctx context.Context,
- results []chunkWithTitle,
- queries []string,
- compiled []*regexp.Regexp,
-) string {
- var b strings.Builder
-
- b.WriteString(fmt.Sprintf("<grep_results chunk_count=\"%d\">\n", len(results)))
- for _, q := range queries {
- b.WriteString(fmt.Sprintf("<query>%s</query>\n", xmlEscape(q)))
- }
-
- if len(results) == 0 {
- b.WriteString("</grep_results>")
- return b.String()
- }
-
- for _, r := range results {
- counts := countRegexHits(r.Content, compiled, queries)
- snippet := extractChunkMatchSnippet(&r.Chunk, compiled)
-
- extraAttr := ""
- if q := faqStandardQuestion(&r.Chunk); q != "" {
- extraAttr = fmt.Sprintf(" faq_question=\"%s\"", xmlEscape(q))
- }
- isFAQ := r.ChunkType == types.ChunkTypeFAQ
-
- t.mu.Lock()
- seen := t.seenChunks[r.ID]
- t.seenChunks[r.ID] = true
- t.mu.Unlock()
-
- if isFAQ {
- if seen {
- fmt.Fprintf(&b,
- "<faq faq_id=\"%s\" knowledge_title=\"%s\"%s index=\"%d\" score=\"%.3f\" already_seen=\"true\">\n",
- xmlEscape(r.ID), xmlEscape(r.KnowledgeTitle),
- extraAttr, r.ChunkIndex, r.MatchScore)
- } else {
- fmt.Fprintf(&b,
- "<faq faq_id=\"%s\" knowledge_title=\"%s\"%s index=\"%d\" score=\"%.3f\">\n",
- xmlEscape(r.ID), xmlEscape(r.KnowledgeTitle),
- extraAttr, r.ChunkIndex, r.MatchScore)
- }
- } else if seen {
- fmt.Fprintf(&b,
- "<chunk chunk_id=\"%s\" knowledge_id=\"%s\" knowledge_title=\"%s\"%s chunk_index=\"%d\" score=\"%.3f\" already_seen=\"true\">\n",
- xmlEscape(r.ID), xmlEscape(r.KnowledgeID), xmlEscape(r.KnowledgeTitle),
- extraAttr, r.ChunkIndex, r.MatchScore)
- } else {
- fmt.Fprintf(&b,
- "<chunk chunk_id=\"%s\" knowledge_id=\"%s\" knowledge_title=\"%s\"%s chunk_index=\"%d\" score=\"%.3f\">\n",
- xmlEscape(r.ID), xmlEscape(r.KnowledgeID), xmlEscape(r.KnowledgeTitle),
- extraAttr, r.ChunkIndex, r.MatchScore)
- }
-
- for _, q := range queries {
- if c := counts[q]; c > 0 {
- b.WriteString(fmt.Sprintf("<query_hit query=\"%s\" count=\"%d\" />\n",
- xmlEscape(q), c))
- }
- }
- if seen {
- b.WriteString("<note>(snippet omitted, already returned in a previous grep_chunks call this session)</note>\n")
- } else if snippet != "" {
- b.WriteString(fmt.Sprintf("<match_snippet>%s</match_snippet>\n", xmlEscape(snippet)))
- }
- if isFAQ {
- b.WriteString("</faq>\n")
- } else {
- b.WriteString("</chunk>\n")
- }
- }
-
- b.WriteString("</grep_results>")
- _ = ctx
- return b.String()
-}
-
-type knowledgeAggregation struct {
- KnowledgeID string `json:"knowledge_id"`
- KnowledgeBaseID string `json:"knowledge_base_id"`
- KnowledgeTitle string `json:"knowledge_title"`
- // FAQQuestion is the standard question of the first matched FAQ entry in
- // this knowledge. FAQ entries share the owning knowledge's title, so the
- // frontend uses this to give the row a distinct, human-readable label.
- FAQQuestion string `json:"faq_question,omitempty"`
- TitleMatch bool `json:"title_match"`
- ChunkHitCount int `json:"chunk_hit_count"`
- TotalChunkCount int `json:"total_chunk_count"`
- PatternCounts map[string]int `json:"pattern_counts"`
- TotalPatternHits int `json:"total_pattern_hits"`
- DistinctPatterns int `json:"distinct_patterns"`
- MatchSnippet string `json:"match_snippet,omitempty"`
-}
-
-func (t *GrepChunksTool) aggregateByKnowledge(
- results []chunkWithTitle,
- queries []string,
- compiled []*regexp.Regexp,
-) []knowledgeAggregation {
- if len(results) == 0 {
- return nil
- }
-
- queryKeys := make([]string, 0, len(queries))
- for _, q := range queries {
- if strings.TrimSpace(q) == "" {
- continue
- }
- queryKeys = append(queryKeys, q)
- }
-
- aggregated := make(map[string]*knowledgeAggregation)
- for _, chunk := range results {
- knowledgeID := chunk.KnowledgeID
- if knowledgeID == "" {
- knowledgeID = fmt.Sprintf("chunk-%s", chunk.ID)
- }
-
- if _, ok := aggregated[knowledgeID]; !ok {
- title := chunk.KnowledgeTitle
- if strings.TrimSpace(title) == "" {
- title = "Untitled"
- }
- aggregated[knowledgeID] = &knowledgeAggregation{
- KnowledgeID: knowledgeID,
- KnowledgeBaseID: chunk.KnowledgeBaseID,
- KnowledgeTitle: title,
- TotalChunkCount: chunk.TotalChunkCount,
- PatternCounts: make(map[string]int, len(queryKeys)),
- }
- for _, qKey := range queryKeys {
- aggregated[knowledgeID].PatternCounts[qKey] = 0
- }
- }
-
- entry := aggregated[knowledgeID]
- entry.ChunkHitCount++
- if chunk.TitleMatch {
- entry.TitleMatch = true
- }
- if entry.FAQQuestion == "" {
- if q := faqStandardQuestion(&chunk.Chunk); q != "" {
- entry.FAQQuestion = q
- }
- }
- if entry.MatchSnippet == "" {
- if snippet := extractChunkMatchSnippet(&chunk.Chunk, compiled); snippet != "" {
- entry.MatchSnippet = snippet
- }
- }
-
- occurrences := countRegexHits(chunk.Content, compiled, queryKeys)
- for _, q := range queryKeys {
- count := occurrences[q]
- if count == 0 {
- continue
- }
- entry.PatternCounts[q] += count
- entry.TotalPatternHits += count
- }
- }
-
- resultSlice := make([]knowledgeAggregation, 0, len(aggregated))
- for _, entry := range aggregated {
- distinct := 0
- for _, count := range entry.PatternCounts {
- if count > 0 {
- distinct++
- }
- }
- entry.DistinctPatterns = distinct
- resultSlice = append(resultSlice, *entry)
- }
-
- sort.Slice(resultSlice, func(i, j int) bool {
- // A knowledge whose TITLE matches the query is the most on-topic hit
- // and always ranks first, regardless of body keyword frequency.
- if resultSlice[i].TitleMatch != resultSlice[j].TitleMatch {
- return resultSlice[i].TitleMatch
- }
- if resultSlice[i].DistinctPatterns != resultSlice[j].DistinctPatterns {
- return resultSlice[i].DistinctPatterns > resultSlice[j].DistinctPatterns
- }
- if resultSlice[i].TotalPatternHits != resultSlice[j].TotalPatternHits {
- return resultSlice[i].TotalPatternHits > resultSlice[j].TotalPatternHits
- }
- if resultSlice[i].ChunkHitCount != resultSlice[j].ChunkHitCount {
- return resultSlice[i].ChunkHitCount > resultSlice[j].ChunkHitCount
- }
- return resultSlice[i].KnowledgeTitle < resultSlice[j].KnowledgeTitle
- })
- return resultSlice
-}
-
-type grepChunkResult struct {
- ChunkID string `json:"chunk_id,omitempty"`
- FAQID string `json:"faq_id,omitempty"`
- KnowledgeID string `json:"knowledge_id"`
- KnowledgeBaseID string `json:"knowledge_base_id"`
- KnowledgeTitle string `json:"knowledge_title"`
- ChunkType string `json:"chunk_type"`
- Index int `json:"index,omitempty"`
- ChunkIndex int `json:"chunk_index,omitempty"`
- FAQQuestion string `json:"faq_question,omitempty"`
- TitleMatch bool `json:"title_match,omitempty"`
- MatchSnippet string `json:"match_snippet,omitempty"`
- Score float64 `json:"score"`
-}
-
-func buildGrepChunkResults(results []chunkWithTitle, compiled []*regexp.Regexp) []grepChunkResult {
- if len(results) == 0 {
- return nil
- }
- out := make([]grepChunkResult, 0, len(results))
- for _, r := range results {
- item := grepChunkResult{
- KnowledgeID: r.KnowledgeID,
- KnowledgeBaseID: r.KnowledgeBaseID,
- KnowledgeTitle: r.KnowledgeTitle,
- ChunkType: string(r.ChunkType),
- TitleMatch: r.TitleMatch,
- MatchSnippet: extractChunkMatchSnippet(&r.Chunk, compiled),
- Score: r.MatchScore,
- }
- if r.ChunkType == types.ChunkTypeFAQ {
- item.FAQID = r.ID
- item.Index = r.ChunkIndex
- item.FAQQuestion = faqStandardQuestion(&r.Chunk)
- } else {
- item.ChunkID = r.ID
- item.ChunkIndex = r.ChunkIndex
- }
- out = append(out, item)
- }
- return out
-}
-
-// regexMatchesAny reports whether text matches at least one of the compiled
-// patterns. Used to flag title hits without counting occurrences.
-func regexMatchesAny(text string, compiled []*regexp.Regexp) bool {
- if text == "" || len(compiled) == 0 {
- return false
- }
- for _, re := range compiled {
- if re != nil && re.MatchString(text) {
- return true
- }
- }
- return false
-}
-
-// countRegexHits returns the total number of matches per (compiled) pattern
-// within content, keyed by the original (uncompiled) pattern string.
-func countRegexHits(content string, compiled []*regexp.Regexp, patterns []string) map[string]int {
- counts := make(map[string]int, len(patterns))
- if content == "" || len(compiled) == 0 {
- return counts
- }
- for i, re := range compiled {
- if re == nil {
- continue
- }
- matches := re.FindAllStringIndex(content, -1)
- counts[patterns[i]] = len(matches)
- }
- return counts
-}
-
-// extractChunkMatchSnippet returns a preview for tool output. FAQ chunks only
-// surface the matched question plus answers from metadata (answers are not
-// stored in chunk content for question_only index mode). Other chunk types
-// use regex context around the first body match.
-func extractChunkMatchSnippet(chunk *types.Chunk, compiled []*regexp.Regexp) string {
- if chunk != nil && chunk.ChunkType == types.ChunkTypeFAQ {
- if s := faqMatchSnippet(chunk, compiled); s != "" {
- return s
- }
- }
- if chunk == nil {
- return ""
- }
- return extractSnippetRegex(chunk.Content, compiled)
-}
-
-// extractSnippetRegex returns a short context snippet around the earliest
-// regex match across any of the provided compiled patterns. Result is
-// compressed to a single line and bounded in length on both sides of the
-// match to keep the XML output concise.
-func extractSnippetRegex(content string, compiled []*regexp.Regexp) string {
- if content == "" || len(compiled) == 0 {
- return ""
- }
-
- earliest := -1
- earliestEnd := -1
- for _, re := range compiled {
- if re == nil {
- continue
- }
- loc := re.FindStringIndex(content)
- if loc == nil {
- continue
- }
- if earliest < 0 || loc[0] < earliest {
- earliest = loc[0]
- earliestEnd = loc[1]
- }
- }
- if earliest < 0 {
- return ""
- }
-
- matchStr := content[earliest:earliestEnd]
- before := content[:earliest]
- after := content[earliestEnd:]
-
- beforeRunes := []rune(before)
- if len(beforeRunes) > snippetContextRunes {
- beforeRunes = beforeRunes[len(beforeRunes)-snippetContextRunes:]
- }
- afterRunes := []rune(after)
- if len(afterRunes) > snippetContextRunes {
- afterRunes = afterRunes[:snippetContextRunes]
- }
- matchRunes := []rune(matchStr)
- if len(matchRunes) > snippetMaxMatchRunes {
- matchRunes = append(matchRunes[:snippetMaxMatchRunes], []rune("...")...)
- }
-
- snippet := string(beforeRunes) + string(matchRunes) + string(afterRunes)
- snippet = strings.ReplaceAll(snippet, "\n", " ")
- for strings.Contains(snippet, " ") {
- snippet = strings.ReplaceAll(snippet, " ", " ")
- }
- snippet = strings.TrimSpace(snippet)
- if len([]rune(snippet)) > snippetMaxTotalRunes {
- snippet = string([]rune(snippet)[:snippetMaxTotalRunes]) + "..."
- }
- return "... " + snippet + " ..."
-}
-
-// xmlEscape replaces characters that would break simple XML attribute /
-// element values. It is intentionally minimal because the rendered output is
-// consumed by the LLM (forgiving parser) rather than a strict XML processor.
-func xmlEscape(s string) string {
- replacer := strings.NewReplacer(
- "&", "&amp;",
- "<", "&lt;",
- ">", "&gt;",
- "\"", "&quot;",
- "'", "&apos;",
- )
- return replacer.Replace(s)
-}
-
-// deduplicateChunks removes duplicate or near-duplicate chunks using content signature
-func (t *GrepChunksTool) deduplicateChunks(ctx context.Context, results []chunkWithTitle) []chunkWithTitle {
- seen := make(map[string]bool)
- contentSig := make(map[string]bool)
- uniqueResults := make([]chunkWithTitle, 0)
-
- for _, r := range results {
- keys := []string{r.ID}
- if r.ParentChunkID != "" {
- keys = append(keys, "parent:"+r.ParentChunkID)
- }
- if r.KnowledgeID != "" {
- keys = append(keys, fmt.Sprintf("kb:%s#%d", r.KnowledgeID, r.ChunkIndex))
- }
-
- dup := false
- for _, k := range keys {
- if seen[k] {
- dup = true
- break
- }
- }
- if dup {
- continue
- }
-
- sig := t.buildContentSignature(r.Content)
- if sig != "" {
- if contentSig[sig] {
- continue
- }
- contentSig[sig] = true
- }
-
- for _, k := range keys {
- seen[k] = true
- }
- uniqueResults = append(uniqueResults, r)
- }
-
- seenByID := make(map[string]bool)
- deduplicated := make([]chunkWithTitle, 0)
- for _, r := range uniqueResults {
- if !seenByID[r.ID] {
- seenByID[r.ID] = true
- deduplicated = append(deduplicated, r)
- }
- }
- _ = ctx
- return deduplicated
-}
-
-// buildContentSignature creates a normalized signature for content to detect near-duplicates
-func (t *GrepChunksTool) buildContentSignature(content string) string {
- return searchutil.BuildContentSignature(content)
-}
-
-// scoreChunks calculates match scores for chunks based on regex matches.
-func (t *GrepChunksTool) scoreChunks(
- ctx context.Context,
- results []chunkWithTitle,
- compiled []*regexp.Regexp,
-) []chunkWithTitle {
- scored := make([]chunkWithTitle, len(results))
- for i := range results {
- scored[i] = results[i]
- score, patternCount := t.calculateMatchScore(results[i].Content, compiled)
- // Title-aware boost: when the owning knowledge's TITLE matches the
- // query, treat the chunk as highly relevant regardless of how often
- // the body repeats the term. The boost keeps such chunks alive through
- // MMR selection; TitleMatch is the primary sort key downstream so they
- // also land at the very top of the final ordering.
- if regexMatchesAny(results[i].KnowledgeTitle, compiled) {
- scored[i].TitleMatch = true
- score = math.Min(score+0.5, 1.0)
- if patternCount == 0 {
- // Title-only recall (body never matched the regex) still counts
- // as one matched pattern so it isn't sorted below true zeros.
- patternCount = 1
- }
- }
- scored[i].MatchScore = score
- scored[i].MatchedPatterns = patternCount
- }
- _ = ctx
- return scored
-}
-
-// calculateMatchScore counts how many regex patterns match the content and
-// applies a small boost for earlier match positions.
-func (t *GrepChunksTool) calculateMatchScore(content string, compiled []*regexp.Regexp) (float64, int) {
- if content == "" || len(compiled) == 0 {
- return 0.0, 0
- }
-
- matchCount := 0
- earliestPos := len(content)
-
- for _, re := range compiled {
- if re == nil {
- continue
- }
- loc := re.FindStringIndex(content)
- if loc == nil {
- continue
- }
- matchCount++
- if loc[0] < earliestPos {
- earliestPos = loc[0]
- }
- }
-
- if matchCount == 0 {
- return 0.0, 0
- }
-
- baseScore := float64(matchCount) / float64(len(compiled))
-
- positionBonus := 0.0
- if earliestPos < len(content) {
- positionRatio := 1.0 - float64(earliestPos)/float64(len(content))
- positionBonus = positionRatio * 0.1
- }
-
- return math.Min(baseScore+positionBonus, 1.0), matchCount
-}
-
-// applyMMR applies Maximal Marginal Relevance algorithm to reduce redundancy
-func (t *GrepChunksTool) applyMMR(
- ctx context.Context,
- results []chunkWithTitle,
- k int,
- lambda float64,
-) []chunkWithTitle {
- if k <= 0 || len(results) == 0 {
- return nil
- }
-
- logger.Debugf(ctx, "[Tool][GrepChunks] Applying MMR: lambda=%.2f, k=%d, candidates=%d",
- lambda, k, len(results))
-
- selected := make([]chunkWithTitle, 0, k)
- selectedTokenSets := make([]map[string]struct{}, 0, k)
-
- candidates := make([]chunkWithTitle, len(results))
- copy(candidates, results)
-
- tokenSets := make([]map[string]struct{}, len(candidates))
- for i, r := range candidates {
- tokenSets[i] = t.tokenizeSimple(r.Content)
- }
-
- for len(selected) < k && len(candidates) > 0 {
- bestIdx := 0
- bestScore := -1.0
-
- for i, r := range candidates {
- relevance := r.MatchScore
- redundancy := 0.0
- for _, selectedTS := range selectedTokenSets {
- redundancy = math.Max(redundancy, t.jaccard(tokenSets[i], selectedTS))
- }
- mmr := lambda*relevance - (1.0-lambda)*redundancy
- if mmr > bestScore {
- bestScore = mmr
- bestIdx = i
- }
- }
-
- selected = append(selected, candidates[bestIdx])
- selectedTokenSets = append(selectedTokenSets, tokenSets[bestIdx])
-
- last := len(candidates) - 1
- candidates[bestIdx] = candidates[last]
- tokenSets[bestIdx] = tokenSets[last]
- candidates = candidates[:last]
- tokenSets = tokenSets[:last]
- }
-
- return selected
-}
-
-// tokenizeSimple tokenizes text into a set of words (simple whitespace-based)
-func (t *GrepChunksTool) tokenizeSimple(text string) map[string]struct{} {
- return searchutil.TokenizeSimple(text)
-}
-
-// jaccard calculates Jaccard similarity between two token sets
-func (t *GrepChunksTool) jaccard(a, b map[string]struct{}) float64 {
- return searchutil.Jaccard(a, b)
-}
diff --git a/internal/agent/tools/grep_chunks_aggregate_test.go b/internal/agent/tools/grep_chunks_aggregate_test.go
deleted file mode 100644
index a3a425db5..000000000
--- a/internal/agent/tools/grep_chunks_aggregate_test.go
+++ /dev/null
@@ -1,58 +0,0 @@
-package tools
-
-import (
- "regexp"
- "testing"
-
- "github.com/Tencent/WeKnora/internal/types"
-)
-
-func TestAggregateByKnowledge_mergesChunksFromSameDocument(t *testing.T) {
- tool := &GrepChunksTool{}
- compiled := []*regexp.Regexp{regexp.MustCompile(`(?i)sample`)}
- results := []chunkWithTitle{
- {
- Chunk: types.Chunk{
- ID: "chunk-a",
- KnowledgeID: "doc-1",
- KnowledgeBaseID: "kb-1",
- Content: "sample hit one",
- },
- KnowledgeTitle: "report-a.pdf",
- },
- {
- Chunk: types.Chunk{
- ID: "chunk-b",
- KnowledgeID: "doc-1",
- KnowledgeBaseID: "kb-1",
- Content: "sample hit two",
- },
- KnowledgeTitle: "report-a.pdf",
- },
- {
- Chunk: types.Chunk{
- ID: "chunk-c",
- KnowledgeID: "doc-2",
- KnowledgeBaseID: "kb-1",
- Content: "sample other doc",
- },
- KnowledgeTitle: "report-b.pdf",
- },
- }
-
- aggregated := tool.aggregateByKnowledge(results, []string{"sample"}, compiled)
- if len(aggregated) != 2 {
- t.Fatalf("want 2 documents, got %d", len(aggregated))
- }
-
- byID := make(map[string]knowledgeAggregation, len(aggregated))
- for _, row := range aggregated {
- byID[row.KnowledgeID] = row
- }
- if byID["doc-1"].ChunkHitCount != 2 {
- t.Fatalf("doc-1 chunk hits = %d, want 2", byID["doc-1"].ChunkHitCount)
- }
- if byID["doc-2"].ChunkHitCount != 1 {
- t.Fatalf("doc-2 chunk hits = %d, want 1", byID["doc-2"].ChunkHitCount)
- }
-}
diff --git a/internal/agent/tools/grep_chunks_scope_test.go b/internal/agent/tools/grep_chunks_scope_test.go
deleted file mode 100644
index 4c57d69fa..000000000
--- a/internal/agent/tools/grep_chunks_scope_test.go
+++ /dev/null
@@ -1,79 +0,0 @@
-package tools
-
-import (
- "strings"
- "testing"
-
- "github.com/Tencent/WeKnora/internal/types"
-)
-
-func TestScopeClause_KnowledgeAndFullKBOredTogether(t *testing.T) {
- sql, args := scopeClause(
- []string{"kb-1"},
- []string{"doc-9"},
- nil,
- map[string]uint64{"kb-1": 7},
- )
-
- if !strings.Contains(sql, "chunks.knowledge_id IN ?") {
- t.Fatalf("expected knowledge-id predicate, got %q", sql)
- }
- if !strings.Contains(sql, "chunks.knowledge_base_id = ? AND chunks.tenant_id = ?") {
- t.Fatalf("expected kb-tenant predicate, got %q", sql)
- }
- if !strings.Contains(sql, " OR ") {
- t.Fatalf("expected the two scopes to be OR'd, got %q", sql)
- }
- // args: knowledgeIDs slice + kbID + tenantID
- if len(args) != 3 {
- t.Fatalf("expected 3 args, got %d: %v", len(args), args)
- }
-}
-
-func TestScopeClause_TagScopeUsesExistsInsteadOfExpandedKnowledgeIDs(t *testing.T) {
- sql, args := scopeClause(
- nil,
- nil,
- []*types.SearchTarget{{
- KnowledgeBaseID: "kb-1",
- TenantID: 7,
- TagIDs: []string{"tag-a", "tag-b"},
- }},
- map[string]uint64{"kb-1": 7},
- )
-
- if strings.Contains(sql, "chunks.knowledge_id IN ?") {
- t.Fatalf("tag scope must not expand to knowledge-id IN predicate, got %q", sql)
- }
- if !strings.Contains(sql, "EXISTS") || !strings.Contains(sql, "knowledge_tag_relations") {
- t.Fatalf("expected tag relation EXISTS predicate, got %q", sql)
- }
- if !strings.Contains(sql, "ktr.tag_id IN ?") {
- t.Fatalf("expected tag-id predicate, got %q", sql)
- }
- if len(args) != 3 {
- t.Fatalf("expected kbID, tenantID, tagIDs args, got %d: %v", len(args), args)
- }
-}
-
-func TestScopeClause_SkipsKBWithoutTenant(t *testing.T) {
- sql, args := scopeClause(
- []string{"kb-1", "kb-2"},
- nil,
- nil,
- map[string]uint64{"kb-1": 7}, // kb-2 has no tenant mapping
- )
- if strings.Contains(sql, "kb-2") {
- t.Fatalf("kb-2 has no tenant and must be skipped, got %q", sql)
- }
- if len(args) != 2 {
- t.Fatalf("expected only kb-1 (2 args), got %d: %v", len(args), args)
- }
-}
-
-func TestScopeClause_EmptyWhenNoUsableScope(t *testing.T) {
- sql, args := scopeClause([]string{"kb-1"}, nil, nil, map[string]uint64{})
- if sql != "" || args != nil {
- t.Fatalf("expected empty scope, got sql=%q args=%v", sql, args)
- }
-}
diff --git a/internal/agent/tools/image_markdown_output_test.go b/internal/agent/tools/image_markdown_output_test.go
index 8e431161c..a1c73b180 100644
--- a/internal/agent/tools/image_markdown_output_test.go
+++ b/internal/agent/tools/image_markdown_output_test.go
@@ -28,11 +28,13 @@ func TestLLMToolOutputsUseMarkdownImages(t *testing.T) {
ImageInfo: string(imageInfo),
}
- listOutput := (&ListKnowledgeChunksTool{}).buildOutput("knowledge-1", "测试文档", 1, 1, []*types.Chunk{chunk})
- wikiOutput := enrichChunkContent(chunk)
+ readOutput := (&ReadDocumentTool{}).buildOutput(
+ &types.Knowledge{ID: "knowledge-1", Title: "测试文档"}, 1, []readChunkRow{{chunk: chunk}}, "",
+ )
+ enrichedOutput := enrichChunkContent(chunk)
for name, output := range map[string]string{
- "list_knowledge_chunks": listOutput,
- "wiki_read_source_doc": wikiOutput,
+ "read_document": readOutput,
+ "enrich_chunk_content": enrichedOutput,
} {
t.Run(name, func(t *testing.T) {
if !strings.Contains(output, "![目标说话人提取流程图](resource://AbCdEfGhIjKlMnOpQrStUv)") {
diff --git a/internal/agent/tools/knowledge_search.go b/internal/agent/tools/knowledge_search.go
deleted file mode 100644
index a89f8cbdc..000000000
--- a/internal/agent/tools/knowledge_search.go
+++ /dev/null
@@ -1,1395 +0,0 @@
-package tools
-
-import (
- "context"
- "encoding/json"
- "fmt"
- "math"
- "sort"
- "strings"
- "sync"
-
- "github.com/Tencent/WeKnora/internal/config"
- "github.com/Tencent/WeKnora/internal/logger"
- "github.com/Tencent/WeKnora/internal/models/rerank"
- "github.com/Tencent/WeKnora/internal/searchutil"
- "github.com/Tencent/WeKnora/internal/types"
- "github.com/Tencent/WeKnora/internal/types/interfaces"
-)
-
-var knowledgeSearchTool = BaseTool{
- name: ToolKnowledgeSearch,
- description: `Semantic/vector search tool for retrieving knowledge by meaning, intent, and conceptual relevance.
-
-This tool uses embeddings to understand the user's query and find semantically similar content across knowledge base chunks.
-
-## Purpose
-Designed for high-level understanding tasks, such as:
-- conceptual explanations
-- topic overviews
-- reasoning-based information needs
-- contextual or intent-driven retrieval
-- queries that cannot be answered with literal keyword matching
-
-The tool searches by MEANING rather than exact text. It identifies chunks that are conceptually relevant even when the wording differs.
-
-## What the Tool Does NOT Do
-- Does NOT perform exact keyword matching
-- Does NOT search for specific named entities
-- Should NOT be used for literal lookup tasks
-- Should NOT receive long raw text or user messages as queries
-- Should NOT be used to locate specific strings or error codes
-
-For literal/keyword/entity search, another tool should be used.
-
-## Required Input Behavior
-"queries" must contain **1–5 short, well-formed semantic questions or conceptual statements** that clearly express the meaning the model is trying to retrieve.
-
-Each query should represent a **concept, idea, topic, explanation, or intent**, such as:
-- abstract topics
-- definitions
-- mechanisms
-- best practices
-- comparisons
-- how/why questions
-
-Avoid:
-- keyword lists
-- raw text from user messages
-- full paragraphs
-- unprocessed input
-
-## Examples of valid query shapes (not content):
-- "What is the main idea of..."
-- "How does X work in general?"
-- "Explain the purpose of..."
-- "What are the key principles behind..."
-- "Overview of ..."
-
-## Parameters
-- queries (required): 1–5 semantic questions or conceptual statements.
- These should reflect the meaning or topic you want embeddings to capture.
-- knowledge_base_ids (optional): limit the search scope.
-
-## Output
-Returns chunks ranked by semantic similarity, reranked when applicable.
-Each chunk has a short cN source ID and belongs to a dN document ID. Results represent conceptual relevance, not literal keyword overlap. Use dN for document-level follow-up tool calls.`,
- schema: json.RawMessage(`{
- "type": "object",
- "properties": {
- "queries": {
- "type": "array",
- "description": "REQUIRED: 1-5 semantic questions/topics (e.g., ['What is RAG?', 'RAG benefits'])",
- "items": {
- "type": "string"
- },
- "minItems": 1,
- "maxItems": 5
- },
- "knowledge_base_ids": {
- "type": "array",
- "description": "Optional: bound knowledge-base IDs (the short bN values shown in runtime context)",
- "items": {
- "type": "string"
- },
- "minItems": 0,
- "maxItems": 10
- }
- },
- "required": ["queries"]
-}`),
-}
-
-// KnowledgeSearchInput defines the input parameters for knowledge search tool
-type KnowledgeSearchInput struct {
- Queries []string `json:"queries"`
- KnowledgeBaseIDs []string `json:"knowledge_base_ids,omitempty"`
-}
-
-// searchResultWithMeta wraps search result with metadata about which query matched it
-type searchResultWithMeta struct {
- *types.SearchResult
- SourceQuery string
- QueryType string // "vector" or "keyword"
- KnowledgeBaseID string // ID of the knowledge base this result came from
- KnowledgeBaseType string // Type of the knowledge base (document, faq, etc.)
-}
-
-// KnowledgeSearchTool searches knowledge bases with flexible query modes.
-// seenChunks lets repeated calls in the same session surface previously-
-// returned chunks in a compact form (mirroring wiki_search's de-duping UX)
-// so the LLM doesn't burn tokens re-reading identical content.
-type KnowledgeSearchTool struct {
- BaseTool
- knowledgeBaseService interfaces.KnowledgeBaseService
- knowledgeService interfaces.KnowledgeService
- chunkService interfaces.ChunkService
- searchTargets types.SearchTargets // Pre-computed unified search targets
- rerankModel rerank.Reranker
- config *config.Config // Global config for fallback values
-
- seenMu sync.Mutex
- seenChunks map[string]bool
-}
-
-// NewKnowledgeSearchTool creates a new knowledge search tool
-func NewKnowledgeSearchTool(
- knowledgeBaseService interfaces.KnowledgeBaseService,
- knowledgeService interfaces.KnowledgeService,
- chunkService interfaces.ChunkService,
- searchTargets types.SearchTargets,
- rerankModel rerank.Reranker,
- cfg *config.Config,
-) *KnowledgeSearchTool {
- return &KnowledgeSearchTool{
- BaseTool: knowledgeSearchTool,
- knowledgeBaseService: knowledgeBaseService,
- knowledgeService: knowledgeService,
- chunkService: chunkService,
- searchTargets: searchTargets,
- rerankModel: rerankModel,
- config: cfg,
- seenChunks: make(map[string]bool),
- }
-}
-
-// Execute executes the knowledge search tool
-func (t *KnowledgeSearchTool) Execute(ctx context.Context, args json.RawMessage) (*types.ToolResult, error) {
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Execute started")
-
- // Parse args from json.RawMessage
- var input KnowledgeSearchInput
- if err := json.Unmarshal(args, &input); err != nil {
- logger.Errorf(ctx, "[Tool][KnowledgeSearch] Failed to parse args: %v", err)
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("Failed to parse args: %v", err),
- }, err
- }
-
- // Log input arguments
- argsJSON, _ := json.MarshalIndent(input, "", " ")
- logger.Debugf(ctx, "[Tool][KnowledgeSearch] Input args:\n%s", string(argsJSON))
-
- // Determine which KBs to search - user can optionally filter to specific KBs
- var userSpecifiedKBs []string
- if len(input.KnowledgeBaseIDs) > 0 {
- userSpecifiedKBs = input.KnowledgeBaseIDs
- if err := validateKnowledgeBaseIDsInSearchTargets(t.searchTargets, userSpecifiedKBs); err != nil {
- return &types.ToolResult{Success: false, Error: err.Error()}, err
- }
- logger.Infof(ctx, "[Tool][KnowledgeSearch] User specified %d knowledge bases: %v", len(userSpecifiedKBs), userSpecifiedKBs)
- }
-
- // Use pre-computed search targets, optionally filtered by user-specified KBs
- searchTargets := t.searchTargets
- if len(userSpecifiedKBs) > 0 {
- // Filter search targets to only include user-specified KBs
- userKBSet := make(map[string]bool)
- for _, kbID := range userSpecifiedKBs {
- userKBSet[kbID] = true
- }
- var filteredTargets types.SearchTargets
- for _, target := range t.searchTargets {
- if target == nil {
- continue
- }
- if userKBSet[target.KnowledgeBaseID] {
- filteredTargets = append(filteredTargets, target)
- }
- }
- searchTargets = filteredTargets
- }
-
- // Validate search targets
- if len(searchTargets) == 0 {
- logger.Errorf(ctx, "[Tool][KnowledgeSearch] No search targets available")
- return &types.ToolResult{
- Success: false,
- Error: "no knowledge bases specified and no search targets configured",
- }, fmt.Errorf("no search targets available")
- }
-
- kbIDs := searchTargets.GetAllKnowledgeBaseIDs()
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Using %d search targets across %d KBs", len(searchTargets), len(kbIDs))
-
- // Parse query parameter
- queries := input.Queries
-
- // Validate: query must be provided
- if len(queries) == 0 {
- logger.Errorf(ctx, "[Tool][KnowledgeSearch] No queries provided")
- return &types.ToolResult{
- Success: false,
- Error: "queries parameter is required",
- }, fmt.Errorf("no queries provided")
- }
-
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Queries: %v", queries)
-
- // Search parameters: fall back to global config, then to hardcoded defaults.
- // We used to read tenant.ConversationConfig here as the first source of
- // truth, but that field was removed when the chat pipeline moved to
- // CustomAgent — tenant-level KV settings now live on the agent itself.
- var topK int
- var vectorThreshold, keywordThreshold, minScore float64
-
- // Fallback to global config if not set
- if topK == 0 && t.config != nil {
- topK = t.config.Conversation.EmbeddingTopK
- }
- if vectorThreshold == 0 && t.config != nil {
- vectorThreshold = t.config.Conversation.VectorThreshold
- }
- if keywordThreshold == 0 && t.config != nil {
- keywordThreshold = t.config.Conversation.KeywordThreshold
- }
-
- // Final fallback to hardcoded defaults if config is not available
- if topK == 0 {
- topK = 5
- }
- if vectorThreshold == 0 {
- vectorThreshold = 0.6
- }
- if keywordThreshold == 0 {
- keywordThreshold = 0.5
- }
- if minScore == 0 {
- minScore = 0.3
- }
-
- logger.Infof(
- ctx,
- "[Tool][KnowledgeSearch] Search params: top_k=%d, vector_threshold=%.2f, keyword_threshold=%.2f, min_score=%.2f",
- topK,
- vectorThreshold,
- keywordThreshold,
- minScore,
- )
-
- // Execute concurrent search using pre-computed search targets
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Starting concurrent search with %d search targets",
- len(searchTargets))
- kbTypeMap := t.getKnowledgeBaseTypes(ctx, kbIDs)
-
- allResults := t.concurrentSearchByTargets(ctx, queries, searchTargets,
- topK, vectorThreshold, keywordThreshold, kbTypeMap)
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Concurrent search completed: %d raw results", len(allResults))
-
- // Note: HybridSearch now uses RRF (Reciprocal Rank Fusion) which produces normalized scores
- // RRF scores are in range [0, ~0.033] (max when rank=1 on both sides: 2/(60+1))
- // Threshold filtering is already done inside HybridSearch before RRF, so we skip it here
-
- // Deduplicate before reranking to reduce processing overhead
- deduplicatedBeforeRerank := t.deduplicateResults(allResults)
-
- // Apply ReRank if model is configured.
- // Use first query for reranking (or combine all queries if needed)
- rerankQuery := ""
- if len(queries) > 0 {
- rerankQuery = queries[0]
- if len(queries) > 1 {
- // Combine multiple queries for reranking
- rerankQuery = strings.Join(queries, " ")
- }
- }
-
- // Variable to hold results through reranking and MMR stages
- var filteredResults []*searchResultWithMeta
-
- if t.rerankModel != nil && len(deduplicatedBeforeRerank) > 0 && rerankQuery != "" {
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Applying rerank, input: %d results, threshold: %.2f, queries: %v",
- len(deduplicatedBeforeRerank), t.rerankThreshold(), queries)
- rerankedResults, err := t.rerankResults(ctx, rerankQuery, deduplicatedBeforeRerank)
- if err != nil {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] Rerank failed, using original results: %v", err)
- filteredResults = deduplicatedBeforeRerank
- } else {
- filteredResults = rerankedResults
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Rerank completed successfully: %d results",
- len(filteredResults))
- }
- } else {
- // No reranking model available, use deduplicated results
- filteredResults = deduplicatedBeforeRerank
- }
-
- // Apply MMR (Maximal Marginal Relevance) to reduce redundancy and improve diversity
- // Note: composite scoring is already applied inside rerankResults
- if len(filteredResults) > 0 {
- // Calculate k for MMR: use min(len(results), max(1, topK))
- mmrK := len(filteredResults)
- if topK > 0 && mmrK > topK {
- mmrK = topK
- }
- if mmrK < 1 {
- mmrK = 1
- }
- // Apply MMR with lambda=0.7 (balance between relevance and diversity)
- logger.Debugf(
- ctx,
- "[Tool][KnowledgeSearch] Applying MMR: k=%d, lambda=0.7, input=%d results",
- mmrK,
- len(filteredResults),
- )
- mmrResults := t.applyMMR(ctx, filteredResults, mmrK, 0.7)
- if len(mmrResults) > 0 {
- filteredResults = mmrResults
- logger.Infof(ctx, "[Tool][KnowledgeSearch] MMR completed: %d results selected", len(filteredResults))
- } else {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] MMR returned no results, using original results")
- }
- }
-
- // Note: minScore filter is skipped because HybridSearch now uses RRF scores
- // RRF scores are in range [0, ~0.033], not [0, 1], so old thresholds don't apply
- // Threshold filtering is already done inside HybridSearch before RRF fusion
-
- // Final deduplication after rerank (in case rerank changed scores/order but duplicates remain)
- logger.Debugf(ctx, "[Tool][KnowledgeSearch] Final deduplication after rerank...")
- deduplicatedResults := t.deduplicateResults(filteredResults)
- logger.Infof(ctx, "[Tool][KnowledgeSearch] After final deduplication: %d results (from %d)",
- len(deduplicatedResults), len(filteredResults))
-
- // Sort results by score (descending)
- sort.Slice(deduplicatedResults, func(i, j int) bool {
- if deduplicatedResults[i].Score != deduplicatedResults[j].Score {
- return deduplicatedResults[i].Score > deduplicatedResults[j].Score
- }
- // If scores are equal, sort by knowledge ID for consistency
- return deduplicatedResults[i].KnowledgeID < deduplicatedResults[j].KnowledgeID
- })
-
- // Log all ranked results (including lower ranks for rerank debugging)
- if len(deduplicatedResults) > 0 {
- total := len(deduplicatedResults)
- for i, r := range deduplicatedResults {
- logger.Infof(ctx, "[Tool][KnowledgeSearch][Rank %d/%d] score=%.3f, type=%s, kb=%s, chunk_id=%s",
- i+1, total, r.Score, r.QueryType, r.KnowledgeID, r.ID)
- }
- }
-
- // Enrich image info for search results (lazy-loaded from child image chunks)
- if t.chunkService != nil && len(deduplicatedResults) > 0 {
- byTenant := make(map[uint64][]*types.SearchResult)
- for _, r := range deduplicatedResults {
- tid := t.searchTargets.GetTenantIDForKB(r.KnowledgeBaseID)
- if tid == 0 {
- continue
- }
- byTenant[tid] = append(byTenant[tid], r.SearchResult)
- }
- for tid, batch := range byTenant {
- searchutil.EnrichSearchResultsImageInfo(ctx, t.chunkService.GetRepository(), tid, batch)
- }
- }
-
- // Build output
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Formatting output with %d final results", len(deduplicatedResults))
- result, err := t.formatOutput(ctx, deduplicatedResults, kbIDs, queries)
- if err != nil {
- logger.Errorf(ctx, "[Tool][KnowledgeSearch] Failed to format output: %v", err)
- return result, err
- }
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Output: %s", result.Output)
- return result, nil
-}
-
-// getKnowledgeBaseTypes fetches knowledge base types for the given IDs
-func (t *KnowledgeSearchTool) getKnowledgeBaseTypes(ctx context.Context, kbIDs []string) map[string]string {
- kbTypeMap := make(map[string]string, len(kbIDs))
-
- for _, kbID := range kbIDs {
- if kbID == "" {
- continue
- }
- if _, exists := kbTypeMap[kbID]; exists {
- continue
- }
-
- kb, err := t.knowledgeBaseService.GetKnowledgeBaseByID(ctx, kbID)
- if err != nil {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] Failed to fetch knowledge base %s info: %v", kbID, err)
- continue
- }
-
- kbTypeMap[kbID] = kb.Type
- }
-
- return kbTypeMap
-}
-
-// concurrentSearchByTargets executes hybrid search using pre-computed search targets.
-// Targets sharing the same underlying embedding model (identified by model name + endpoint)
-// are grouped so the query embedding is computed once per (model, query) pair, and all
-// full-KB targets in a group are combined into a single retrieval call.
-func (t *KnowledgeSearchTool) concurrentSearchByTargets(
- ctx context.Context,
- queries []string,
- searchTargets types.SearchTargets,
- topK int,
- vectorThreshold, keywordThreshold float64,
- kbTypeMap map[string]string,
-) []*searchResultWithMeta {
- // Batch-fetch KB records for embedding model grouping
- kbIDs := searchTargets.GetAllKnowledgeBaseIDs()
- var kbList []*types.KnowledgeBase
- if kbs, err := t.knowledgeBaseService.GetKnowledgeBasesByIDsOnly(ctx, kbIDs); err == nil {
- kbList = kbs
- }
-
- // Filter out non-searchable KBs (wiki-only / graph-only). knowledge_search
- // can only serve KBs with vector or keyword indexing; feeding a wiki-only
- // KB into HybridSearch causes spurious "model ID cannot be empty" errors
- // because such KBs have no EmbeddingModelID configured. Such scopes
- // should be queried via wiki_search / graph tools instead.
- //
- // KBs that we couldn't fetch from the repo (not in kbList) are kept so
- // the downstream HybridSearch path can still surface the real error.
- searchableKBs := make(map[string]bool, len(kbList))
- knownKBs := make(map[string]bool, len(kbList))
- for _, kb := range kbList {
- if kb == nil {
- continue
- }
- knownKBs[kb.ID] = true
- if kb.IsVectorEnabled() || kb.IsKeywordEnabled() {
- searchableKBs[kb.ID] = true
- }
- }
- filteredTargets := make(types.SearchTargets, 0, len(searchTargets))
- for _, st := range searchTargets {
- if st == nil || st.KnowledgeBaseID == "" {
- continue
- }
- if searchableKBs[st.KnowledgeBaseID] {
- filteredTargets = append(filteredTargets, st)
- continue
- }
- if knownKBs[st.KnowledgeBaseID] {
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Skipping non-searchable KB %s (no vector/keyword index, likely wiki/graph-only)", st.KnowledgeBaseID)
- continue
- }
- // KB record unavailable; keep so downstream can surface real errors.
- filteredTargets = append(filteredTargets, st)
- }
- if len(filteredTargets) == 0 {
- logger.Infof(ctx, "[Tool][KnowledgeSearch] No searchable KBs in scope (all wiki/graph-only); skipping retrieval")
- return nil
- }
- searchTargets = filteredTargets
-
- // Resolve actual model identities (name + endpoint) for cross-tenant grouping
- modelKeyMap := t.knowledgeBaseService.ResolveEmbeddingModelKeys(ctx, kbList)
-
- groups := make(map[string][]*types.SearchTarget)
- for _, st := range searchTargets {
- if st == nil || st.KnowledgeBaseID == "" {
- continue
- }
- key := modelKeyMap[st.KnowledgeBaseID]
- groups[key] = append(groups[key], st)
- }
-
- var wg sync.WaitGroup
- var mu sync.Mutex
- allResults := make([]*searchResultWithMeta, 0)
-
- for _, query := range queries {
- q := query
- for modelKey, targets := range groups {
- wg.Add(1)
- go func(q string, modelKey string, targets []*types.SearchTarget) {
- defer wg.Done()
-
- // Compute embedding once for this (model, query) pair
- var queryEmbedding []float32
- if modelKey != "" {
- emb, err := t.knowledgeBaseService.GetQueryEmbedding(ctx, targets[0].KnowledgeBaseID, q)
- if err != nil {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] Failed to pre-compute embedding for model %s: %v", modelKey, err)
- } else {
- queryEmbedding = emb
- }
- }
-
- // Separate full-KB targets (combinable) from specific-knowledge targets
- var fullKBIDs []string
- var knowledgeTargets []*types.SearchTarget
- for _, st := range targets {
- if st.Type == types.SearchTargetTypeKnowledgeBase && len(st.TagIDs) == 0 {
- fullKBIDs = append(fullKBIDs, st.KnowledgeBaseID)
- } else {
- knowledgeTargets = append(knowledgeTargets, st)
- }
- }
-
- var innerWg sync.WaitGroup
-
- // Combined retrieval for all full-KB targets in this group
- if len(fullKBIDs) > 0 {
- innerWg.Add(1)
- go func() {
- defer innerWg.Done()
- searchParams := types.SearchParams{
- QueryText: q,
- QueryEmbedding: queryEmbedding,
- KnowledgeBaseIDs: fullKBIDs,
- MatchCount: topK,
- VectorThreshold: vectorThreshold,
- KeywordThreshold: keywordThreshold,
- }
- kbResults, err := t.knowledgeBaseService.HybridSearch(ctx, fullKBIDs[0], searchParams)
- if err != nil {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] Combined search failed for KBs %v: %v", fullKBIDs, err)
- return
- }
- mu.Lock()
- for _, r := range kbResults {
- allResults = append(allResults, &searchResultWithMeta{
- SearchResult: r,
- SourceQuery: q,
- QueryType: "hybrid",
- KnowledgeBaseID: r.KnowledgeBaseID,
- KnowledgeBaseType: kbTypeMap[r.KnowledgeBaseID],
- })
- }
- mu.Unlock()
- }()
- }
-
- // Individual retrieval for specific-knowledge targets
- for _, target := range knowledgeTargets {
- st := target
- innerWg.Add(1)
- go func() {
- defer innerWg.Done()
- stVectorThreshold, stKeywordThreshold := st.RecallThresholds(
- vectorThreshold,
- keywordThreshold,
- )
- searchParams := types.SearchParams{
- QueryText: q,
- QueryEmbedding: queryEmbedding,
- MatchCount: topK,
- VectorThreshold: stVectorThreshold,
- KeywordThreshold: stKeywordThreshold,
- KnowledgeIDs: st.KnowledgeIDs,
- TagIDs: st.TagIDs,
- ScopeTagIDs: st.ScopeTagIDs,
- }
- kbResults, err := t.knowledgeBaseService.HybridSearch(ctx, st.KnowledgeBaseID, searchParams)
- if err != nil {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] Failed to search KB %s: %v", st.KnowledgeBaseID, err)
- return
- }
- mu.Lock()
- for _, r := range kbResults {
- allResults = append(allResults, &searchResultWithMeta{
- SearchResult: r,
- SourceQuery: q,
- QueryType: "hybrid",
- KnowledgeBaseID: r.KnowledgeBaseID,
- KnowledgeBaseType: kbTypeMap[r.KnowledgeBaseID],
- })
- }
- mu.Unlock()
- }()
- }
-
- innerWg.Wait()
- }(q, modelKey, targets)
- }
- }
- wg.Wait()
- return allResults
-}
-
-// rerankResults applies reranking to all search results (including FAQ entries)
-// using the configured rerank model, then filters by threshold and applies
-// composite scoring so MMR/sorting uses a single score scale.
-//
-// A failed rerank call degrades to the raw retrieval order, mirroring the chat
-// pipeline's api_error_fallback. An empty result after threshold filtering is
-// kept empty: filterRerankRankResults already preserves the top candidate down
-// to agentRerankFallbackMinScore, so reaching zero means even the best match is
-// below that floor. There is deliberately no chat-model re-scoring path here —
-// it mixed the reranker's [0,1] scale with raw RRF scores and could resurrect
-// candidates the reranker had already rejected.
-func (t *KnowledgeSearchTool) rerankResults(
- ctx context.Context,
- query string,
- results []*searchResultWithMeta,
-) ([]*searchResultWithMeta, error) {
- if len(results) == 0 || t.rerankModel == nil {
- return results, nil
- }
-
- rankResults, err := t.rerankScores(ctx, query, results)
- if err != nil {
- logger.Warnf(ctx,
- "[Tool][KnowledgeSearch] Rerank model failed, using raw retrieval results: %v", err)
- return results, nil
- }
-
- threshold := t.rerankThreshold()
- reranked := t.applyModelRerankScores(
- results,
- rankResults,
- threshold,
- t.searchTargets.HasRecallThresholdOverride(),
- )
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Reranked %d/%d results above threshold %.2f",
- len(reranked), len(results), threshold)
- return reranked, nil
-}
-
-func (t *KnowledgeSearchTool) getFAQMetadata(
- ctx context.Context,
- chunkID string,
- cache map[string]*types.FAQChunkMetadata,
-) (*types.FAQChunkMetadata, error) {
- if chunkID == "" || t.chunkService == nil {
- return nil, nil
- }
-
- if meta, ok := cache[chunkID]; ok {
- return meta, nil
- }
-
- chunk, err := t.chunkService.GetChunkByID(ctx, chunkID)
- if err != nil {
- cache[chunkID] = nil
- return nil, err
- }
- if chunk == nil {
- cache[chunkID] = nil
- return nil, nil
- }
-
- meta, err := chunk.FAQMetadata()
- if err != nil {
- cache[chunkID] = nil
- return nil, err
- }
- cache[chunkID] = meta
- return meta, nil
-}
-
-// rerankScores scores the candidates with the configured rerank model and
-// returns the raw relevance scores, leaving threshold filtering and composite
-// scoring to the caller.
-func (t *KnowledgeSearchTool) rerankScores(
- ctx context.Context,
- query string,
- results []*searchResultWithMeta,
-) ([]rerank.RankResult, error) {
- passages := make([]string, len(results))
- for i, result := range results {
- passages[i] = t.getEnrichedPassage(ctx, result.SearchResult)
- }
-
- rerankResp, err := t.rerankModel.Rerank(ctx, query, passages)
- if err != nil {
- return nil, fmt.Errorf("rerank call failed: %w", err)
- }
- return rerankResp, nil
-}
-
-func (t *KnowledgeSearchTool) rerankThreshold() float64 {
- if t.config != nil && t.config.Conversation != nil && t.config.Conversation.RerankThreshold > 0 {
- return t.config.Conversation.RerankThreshold
- }
- return 0.3
-}
-
-const agentRerankFallbackMinScore = 0.15
-
-func filterRerankRankResults(
- rankResults []rerank.RankResult,
- threshold float64,
- preserveTop bool,
-) []rerank.RankResult {
- if len(rankResults) == 0 {
- return nil
- }
- filtered := make([]rerank.RankResult, 0, len(rankResults))
- for _, r := range rankResults {
- if r.RelevanceScore >= threshold {
- filtered = append(filtered, r)
- }
- }
- if len(filtered) == 0 {
- top := rankResults[0]
- for _, r := range rankResults[1:] {
- if r.RelevanceScore > top.RelevanceScore {
- top = r
- }
- }
- if preserveTop || top.RelevanceScore >= agentRerankFallbackMinScore {
- return []rerank.RankResult{top}
- }
- }
- return filtered
-}
-
-func (t *KnowledgeSearchTool) applyModelRerankScores(
- originals []*searchResultWithMeta,
- rankResults []rerank.RankResult,
- threshold float64,
- preserveTop bool,
-) []*searchResultWithMeta {
- filtered := filterRerankRankResults(rankResults, threshold, preserveTop)
- out := make([]*searchResultWithMeta, 0, len(filtered))
- for _, rr := range filtered {
- if rr.Index < 0 || rr.Index >= len(originals) {
- continue
- }
- newResult := *originals[rr.Index]
- baseScore := newResult.Score
- modelScore := rr.RelevanceScore
- newResult.Score = t.compositeScore(&newResult, modelScore, baseScore)
- out = append(out, &newResult)
- }
- sort.Slice(out, func(i, j int) bool {
- return out[i].Score > out[j].Score
- })
- return out
-}
-
-// deduplicateResults removes duplicate chunks, keeping the highest score
-// Uses multiple keys (ID, parent chunk ID, knowledge+index) and content signature for deduplication
-func (t *KnowledgeSearchTool) deduplicateResults(results []*searchResultWithMeta) []*searchResultWithMeta {
- seen := make(map[string]bool)
- contentSig := make(map[string]bool)
- uniqueResults := make([]*searchResultWithMeta, 0)
-
- for _, r := range results {
- // Build multiple keys for deduplication
- keys := []string{r.ID}
- if r.ParentChunkID != "" {
- keys = append(keys, "parent:"+r.ParentChunkID)
- }
- if r.KnowledgeID != "" {
- keys = append(keys, fmt.Sprintf("kb:%s#%d", r.KnowledgeID, r.ChunkIndex))
- }
-
- // Check if any key is already seen
- dup := false
- for _, k := range keys {
- if seen[k] {
- dup = true
- break
- }
- }
- if dup {
- continue
- }
-
- // Check content signature for near-duplicate content
- sig := t.buildContentSignature(r.Content)
- if sig != "" {
- if contentSig[sig] {
- continue
- }
- contentSig[sig] = true
- }
-
- // Mark all keys as seen
- for _, k := range keys {
- seen[k] = true
- }
-
- uniqueResults = append(uniqueResults, r)
- }
-
- // If we have duplicates by ID but different scores, keep the highest score
- // This handles cases where the same chunk appears multiple times with different scores
- seenByID := make(map[string]*searchResultWithMeta)
- for _, r := range uniqueResults {
- if existing, ok := seenByID[r.ID]; ok {
- // Keep the result with higher score
- if r.Score > existing.Score {
- seenByID[r.ID] = r
- }
- } else {
- seenByID[r.ID] = r
- }
- }
-
- // Convert back to slice
- deduplicated := make([]*searchResultWithMeta, 0, len(seenByID))
- for _, r := range seenByID {
- deduplicated = append(deduplicated, r)
- }
-
- return deduplicated
-}
-
-// buildContentSignature creates a normalized signature for content to detect near-duplicates
-func (t *KnowledgeSearchTool) buildContentSignature(content string) string {
- return searchutil.BuildContentSignature(content)
-}
-
-// writeKnowledgeMetadataHeader emits document-scoped metadata once per
-// knowledge item. Chunk entries keep only chunk-specific content so repeated
-// results from the same document do not waste model context.
-func writeKnowledgeMetadataHeader(ob *strings.Builder, results []*searchResultWithMeta) {
- seen := make(map[string]struct{}, len(results))
- hasMetadata := false
- var documents strings.Builder
- for _, result := range results {
- if result == nil || result.SearchResult == nil || result.KnowledgeID == "" || result.KnowledgeCustomMetadata == "" {
- continue
- }
- if _, ok := seen[result.KnowledgeID]; ok {
- continue
- }
- seen[result.KnowledgeID] = struct{}{}
- hasMetadata = true
- documents.WriteString(fmt.Sprintf(
- "<document knowledge_id=\"%s\" knowledge_base_id=\"%s\" title=\"%s\">\n",
- xmlEscape(result.KnowledgeID),
- xmlEscape(result.KnowledgeBaseID),
- xmlEscape(result.KnowledgeTitle),
- ))
- documents.WriteString(fmt.Sprintf("<metadata>%s</metadata>\n", xmlEscape(result.KnowledgeCustomMetadata)))
- documents.WriteString("</document>\n")
- }
- if !hasMetadata {
- return
- }
- ob.WriteString("<documents>\n")
- ob.WriteString(documents.String())
- ob.WriteString("</documents>\n")
-}
-
-// formatOutput formats the search results for display
-func (t *KnowledgeSearchTool) formatOutput(
- ctx context.Context,
- results []*searchResultWithMeta,
- kbsToSearch []string,
- queries []string,
-) (*types.ToolResult, error) {
- if len(results) == 0 {
- data := map[string]interface{}{
- "knowledge_base_ids": kbsToSearch,
- "results": []interface{}{},
- "count": 0,
- }
- if len(queries) > 0 {
- data["queries"] = queries
- }
- output := fmt.Sprintf("No relevant content found in %d knowledge base(s).\n\n", len(kbsToSearch))
- output += "=== ⚠️ CRITICAL - Next Steps ===\n"
- output += "- ❌ DO NOT use training data or general knowledge to answer\n"
- output += "- ✅ If web_search is enabled: You MUST use web_search to find information\n"
- output += "- ✅ If web_search is disabled: State 'I couldn't find relevant information in the knowledge base'\n"
- output += "- NEVER fabricate or infer answers - ONLY use retrieved content\n"
-
- return &types.ToolResult{
- Success: true,
- Output: output,
- Data: data,
- }, nil
- }
-
- // Count results by KB
- kbCounts := make(map[string]int)
- for _, r := range results {
- kbCounts[r.KnowledgeBaseID]++
- }
-
- // Format individual results as XML. Tag names are kept in sync with
- // wiki_search (`<search_results>`, per-entry element, `<query>`) so that
- // agents and downstream consumers see a single consistent shape across
- // all retrieval tools.
- var ob strings.Builder
- ob.WriteString(fmt.Sprintf("<search_results count=\"%d\">\n", len(results)))
- for _, q := range queries {
- ob.WriteString(fmt.Sprintf("<query>%s</query>\n", xmlEscape(q)))
- }
- writeKnowledgeMetadataHeader(&ob, results)
-
- formattedResults := make([]map[string]interface{}, 0, len(results))
- enabled := true
-
- faqMetadataCache := make(map[string]*types.FAQChunkMetadata)
-
- knowledgeChunkMap := make(map[string]map[int]bool)
- knowledgeTotalMap := make(map[string]int64)
- knowledgeTitleMap := make(map[string]string)
-
- for i, result := range results {
- var faqMeta *types.FAQChunkMetadata
- if result.KnowledgeBaseType == types.KnowledgeBaseTypeFAQ {
- meta, err := t.getFAQMetadata(ctx, result.ID, faqMetadataCache)
- if err != nil {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] Failed to load FAQ metadata for chunk %s: %v", result.ID, err)
- } else {
- faqMeta = meta
- }
- }
-
- if knowledgeChunkMap[result.KnowledgeID] == nil {
- knowledgeChunkMap[result.KnowledgeID] = make(map[int]bool)
- }
- knowledgeChunkMap[result.KnowledgeID][result.ChunkIndex] = true
- knowledgeTitleMap[result.KnowledgeID] = result.KnowledgeTitle
-
- // Cache total chunk count per knowledge
- if _, exists := knowledgeTotalMap[result.KnowledgeID]; !exists {
- effectiveTenantID := t.searchTargets.GetTenantIDForKB(result.KnowledgeBaseID)
- if effectiveTenantID == 0 {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] KB %s not found in searchTargets, skipping chunk count", result.KnowledgeBaseID)
- knowledgeTotalMap[result.KnowledgeID] = 0
- } else {
- // Use the same chunk-type filter as list_knowledge_chunks so the
- // total reported here matches what list_knowledge_chunks can page
- // over. Mismatched filters previously let LLMs compute offsets
- // against an inflated/deflated total and page past the end.
- _, total, err := t.chunkService.GetRepository().ListPagedChunksByKnowledgeID(ctx,
- effectiveTenantID, result.KnowledgeID,
- &types.Pagination{Page: 1, PageSize: 1},
- []types.ChunkType{types.ChunkTypeText, types.ChunkTypeFAQ}, nil, "", "", "", "",
- &enabled,
- )
- if err != nil {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] Failed to get total chunks for knowledge %s: %v", result.KnowledgeID, err)
- knowledgeTotalMap[result.KnowledgeID] = 0
- } else {
- knowledgeTotalMap[result.KnowledgeID] = total
- }
- }
- }
-
- t.seenMu.Lock()
- seen := t.seenChunks[result.ID]
- t.seenChunks[result.ID] = true
- t.seenMu.Unlock()
-
- isFAQ := faqMeta != nil
- if seen {
- // Compact rendering for chunks we already returned in a previous
- // knowledge_search call during this session. The model has the
- // content in context already, so re-emitting it only burns tokens.
- if isFAQ {
- ob.WriteString(fmt.Sprintf(
- "<faq rank=\"%d\" faq_id=\"%s\" index=\"%d\" knowledge_base_id=\"%s\" knowledge_title=\"%s\" score=\"%.3f\" source_query=\"%s\" already_seen=\"true\">\n",
- i+1,
- xmlEscape(result.ID),
- result.ChunkIndex,
- xmlEscape(result.KnowledgeBaseID),
- xmlEscape(result.KnowledgeTitle),
- result.Score,
- xmlEscape(result.SourceQuery),
- ))
- } else {
- ob.WriteString(fmt.Sprintf(
- "<chunk rank=\"%d\" chunk_id=\"%s\" chunk_index=\"%d\" knowledge_id=\"%s\" knowledge_base_id=\"%s\" knowledge_title=\"%s\" score=\"%.3f\" source_query=\"%s\" already_seen=\"true\">\n",
- i+1,
- xmlEscape(result.ID),
- result.ChunkIndex,
- xmlEscape(result.KnowledgeID),
- xmlEscape(result.KnowledgeBaseID),
- xmlEscape(result.KnowledgeTitle),
- result.Score,
- xmlEscape(result.SourceQuery),
- ))
- }
- ob.WriteString("<note>(content omitted, already returned in a previous knowledge_search call this session)</note>\n")
- if isFAQ {
- ob.WriteString("</faq>\n")
- } else {
- ob.WriteString("</chunk>\n")
- }
- } else {
- if isFAQ {
- ob.WriteString(fmt.Sprintf(
- "<faq rank=\"%d\" faq_id=\"%s\" index=\"%d\" knowledge_base_id=\"%s\" knowledge_title=\"%s\" score=\"%.3f\" source_query=\"%s\">\n",
- i+1,
- xmlEscape(result.ID),
- result.ChunkIndex,
- xmlEscape(result.KnowledgeBaseID),
- xmlEscape(result.KnowledgeTitle),
- result.Score,
- xmlEscape(result.SourceQuery),
- ))
- } else {
- ob.WriteString(fmt.Sprintf(
- "<chunk rank=\"%d\" chunk_id=\"%s\" chunk_index=\"%d\" knowledge_id=\"%s\" knowledge_base_id=\"%s\" knowledge_title=\"%s\" score=\"%.3f\" source_query=\"%s\">\n",
- i+1,
- xmlEscape(result.ID),
- result.ChunkIndex,
- xmlEscape(result.KnowledgeID),
- xmlEscape(result.KnowledgeBaseID),
- xmlEscape(result.KnowledgeTitle),
- result.Score,
- xmlEscape(result.SourceQuery),
- ))
- }
- snippet := ""
- if faqMeta != nil {
- snippet = faqMatchSnippetFromQueries(faqMeta, queries)
- }
- if snippet == "" {
- snippet = extractSnippetForQueries(result.Content, queries)
- }
- if snippet != "" {
- ob.WriteString(fmt.Sprintf("<match_snippet>%s</match_snippet>\n", xmlEscape(snippet)))
- }
- ob.WriteString(fmt.Sprintf("<content>%s</content>\n", result.Content))
-
- if result.ImageInfo != "" {
- var imageInfos []types.ImageInfo
- if err := json.Unmarshal([]byte(result.ImageInfo), &imageInfos); err == nil && len(imageInfos) > 0 {
- for _, img := range imageInfos {
- if imageMarkdown := searchutil.BuildImageInfoMarkdownWithURL(img.URL, &img); imageMarkdown != "" {
- ob.WriteString(imageMarkdown)
- ob.WriteString("\n")
- }
- }
- }
- }
-
- if isFAQ {
- writeFAQFieldsXML(&ob, faqMeta)
- ob.WriteString("</faq>\n")
- } else {
- ob.WriteString("</chunk>\n")
- }
- }
-
- formattedResults = append(formattedResults, map[string]interface{}{
- "result_index": i + 1,
- "content": result.Content,
- "knowledge_id": result.KnowledgeID,
- "knowledge_base_id": result.KnowledgeBaseID,
- "knowledge_title": result.KnowledgeTitle,
- "knowledge_metadata": result.KnowledgeCustomMetadata,
- "match_type": result.MatchType,
- "source_query": result.SourceQuery,
- "query_type": result.QueryType,
- "knowledge_base_type": result.KnowledgeBaseType,
- })
-
- last := formattedResults[len(formattedResults)-1]
-
- if result.ImageInfo != "" {
- var imageInfos []types.ImageInfo
- if err := json.Unmarshal([]byte(result.ImageInfo), &imageInfos); err == nil && len(imageInfos) > 0 {
- imageList := make([]map[string]string, 0, len(imageInfos))
- for _, img := range imageInfos {
- imgData := make(map[string]string)
- if img.URL != "" {
- imgData["url"] = img.URL
- }
- if img.Caption != "" {
- imgData["caption"] = img.Caption
- }
- if img.OCRText != "" {
- imgData["ocr_text"] = img.OCRText
- }
- if len(imgData) > 0 {
- imageList = append(imageList, imgData)
- }
- }
- if len(imageList) > 0 {
- last["images"] = imageList
- }
- }
- }
-
- if faqMeta != nil {
- last["faq_id"] = result.ID
- last["index"] = result.ChunkIndex
- if faqMeta.StandardQuestion != "" {
- last["faq_standard_question"] = faqMeta.StandardQuestion
- }
- appendSimilarQuestionsToChunkData(last, faqMeta.SimilarQuestions)
- if len(faqMeta.Answers) > 0 {
- last["faq_answers"] = faqMeta.Answers
- }
- } else {
- last["chunk_id"] = result.ID
- last["chunk_index"] = result.ChunkIndex
- }
- }
-
- // Retrieval statistics
- ob.WriteString("<retrieval_statistics>\n")
- for knowledgeID, retrievedChunks := range knowledgeChunkMap {
- totalChunks := knowledgeTotalMap[knowledgeID]
- retrievedCount := len(retrievedChunks)
- title := knowledgeTitleMap[knowledgeID]
- if totalChunks > 0 {
- remaining := totalChunks - int64(retrievedCount)
- percentage := float64(retrievedCount) / float64(totalChunks) * 100
- ob.WriteString(fmt.Sprintf("<document_stat knowledge_id=\"%s\" title=\"%s\" total_chunks=\"%d\" retrieved=\"%d\" remaining=\"%d\" coverage=\"%.1f%%\" />\n",
- xmlEscape(knowledgeID), xmlEscape(title), totalChunks, retrievedCount, remaining, percentage))
- }
- }
- ob.WriteString("</retrieval_statistics>\n")
- ob.WriteString("</search_results>")
-
- output := ob.String()
-
- data := map[string]interface{}{
- "knowledge_base_ids": kbsToSearch,
- "results": formattedResults,
- "count": len(formattedResults),
- "kb_counts": kbCounts,
- "display_type": "search_results",
- }
-
- if len(queries) > 0 {
- data["queries"] = queries
- }
-
- return &types.ToolResult{
- Success: true,
- Output: output,
- Data: data,
- }, nil
-}
-
-// chunkRange represents a continuous range of chunk indices
-type chunkRange struct {
- start int
- end int
-}
-
-// getEnrichedPassage 合并Content和ImageInfo的文本内容
-func (t *KnowledgeSearchTool) getEnrichedPassage(ctx context.Context, result *types.SearchResult) string {
- if result.ImageInfo == "" {
- return result.Content
- }
-
- // 解析ImageInfo
- var imageInfos []types.ImageInfo
- err := json.Unmarshal([]byte(result.ImageInfo), &imageInfos)
- if err != nil {
- logger.Warnf(ctx, "[Tool][KnowledgeSearch] Failed to parse image info: %v", err)
- return result.Content
- }
-
- if len(imageInfos) == 0 {
- return result.Content
- }
-
- // 提取所有图片的描述和OCR文本
- var imageTexts []string
- for _, img := range imageInfos {
- if img.Caption != "" {
- imageTexts = append(imageTexts, fmt.Sprintf("Image Caption: %s", img.Caption))
- }
- if img.OCRText != "" {
- imageTexts = append(imageTexts, fmt.Sprintf("Image Text: %s", img.OCRText))
- }
- }
-
- if len(imageTexts) == 0 {
- return result.Content
- }
-
- // 组合内容和图片信息
- combinedText := result.Content
- if combinedText != "" {
- combinedText += "\n\n"
- }
- combinedText += strings.Join(imageTexts, "\n")
-
- logger.Debugf(ctx, "[Tool][KnowledgeSearch] Enriched passage: content_len=%d, image_texts=%d",
- len(result.Content), len(imageTexts))
-
- return combinedText
-}
-
-// compositeScore calculates a composite score considering multiple factors
-func (t *KnowledgeSearchTool) compositeScore(
- result *searchResultWithMeta,
- modelScore, baseScore float64,
-) float64 {
- // Source weight: web_search results get slightly lower weight
- sourceWeight := 1.0
- if strings.ToLower(result.KnowledgeSource) == "web_search" {
- sourceWeight = 0.95
- }
-
- // Position prior: slightly favor chunks earlier in the document
- positionPrior := 1.0
- if result.StartAt >= 0 && result.EndAt > result.StartAt {
- // Calculate position ratio and apply small boost for earlier positions
- positionRatio := 1.0 - float64(result.StartAt)/float64(result.EndAt+1)
- positionPrior += t.clampFloat(positionRatio, -0.05, 0.05)
- }
-
- // Composite formula: weighted combination of model score, base score, and source weight
- composite := 0.6*modelScore + 0.3*baseScore + 0.1*sourceWeight
- composite *= positionPrior
-
- // Clamp to [0, 1]
- if composite < 0 {
- composite = 0
- }
- if composite > 1 {
- composite = 1
- }
-
- return composite
-}
-
-// clampFloat clamps a float value to the specified range
-func (t *KnowledgeSearchTool) clampFloat(v, minV, maxV float64) float64 {
- return searchutil.ClampFloat(v, minV, maxV)
-}
-
-// applyMMR applies Maximal Marginal Relevance algorithm to reduce redundancy
-func (t *KnowledgeSearchTool) applyMMR(
- ctx context.Context,
- results []*searchResultWithMeta,
- k int,
- lambda float64,
-) []*searchResultWithMeta {
- if k <= 0 || len(results) == 0 {
- return nil
- }
-
- logger.Infof(ctx, "[Tool][KnowledgeSearch] Applying MMR: lambda=%.2f, k=%d, candidates=%d",
- lambda, k, len(results))
-
- selected := make([]*searchResultWithMeta, 0, k)
- candidates := make([]*searchResultWithMeta, len(results))
- copy(candidates, results)
-
- // Pre-compute token sets for all candidates
- tokenSets := make([]map[string]struct{}, len(candidates))
- for i, r := range candidates {
- tokenSets[i] = t.tokenizeSimple(t.getEnrichedPassage(ctx, r.SearchResult))
- }
-
- // MMR selection loop, incremental form: maxRedundancy[i] caches candidate i's
- // maximum jaccard against everything selected so far, so each round only needs
- // one comparison per remaining candidate instead of one per (candidate, selected)
- // pair. Selection output is identical to the naive form, including tie-breaking,
- // because the candidate iteration order is unchanged.
- selectedTokenSets := make([]map[string]struct{}, 0, k)
- maxRedundancy := make([]float64, len(candidates))
- for len(selected) < k && len(candidates) > 0 {
- bestIdx := 0
- bestScore := -1.0
-
- for i, r := range candidates {
- // MMR score: balance relevance and diversity
- mmr := lambda*r.Score - (1.0-lambda)*maxRedundancy[i]
- if mmr > bestScore {
- bestScore = mmr
- bestIdx = i
- }
- }
-
- // Add best candidate to selected and remove from candidates
- selected = append(selected, candidates[bestIdx])
- chosenTokens := tokenSets[bestIdx]
- selectedTokenSets = append(selectedTokenSets, chosenTokens)
- candidates = append(candidates[:bestIdx], candidates[bestIdx+1:]...)
- // Remove corresponding token set and cached redundancy
- tokenSets = append(tokenSets[:bestIdx], tokenSets[bestIdx+1:]...)
- maxRedundancy = append(maxRedundancy[:bestIdx], maxRedundancy[bestIdx+1:]...)
-
- // Fold the freshly selected result into every remaining candidate's cache
- for i := range candidates {
- maxRedundancy[i] = math.Max(maxRedundancy[i], t.jaccard(tokenSets[i], chosenTokens))
- }
- }
-
- // Compute average redundancy among selected results, reusing the cached token
- // sets instead of re-tokenizing every pair
- avgRed := 0.0
- if len(selectedTokenSets) > 1 {
- pairs := 0
- for i := 0; i < len(selectedTokenSets); i++ {
- for j := i + 1; j < len(selectedTokenSets); j++ {
- avgRed += t.jaccard(selectedTokenSets[i], selectedTokenSets[j])
- pairs++
- }
- }
- if pairs > 0 {
- avgRed /= float64(pairs)
- }
- }
-
- logger.Infof(ctx, "[Tool][KnowledgeSearch] MMR completed: selected=%d, avg_redundancy=%.4f",
- len(selected), avgRed)
-
- return selected
-}
-
-// tokenizeSimple tokenizes text into a set of words (simple whitespace-based)
-func (t *KnowledgeSearchTool) tokenizeSimple(text string) map[string]struct{} {
- return searchutil.TokenizeSimple(text)
-}
-
-// extractSnippetForQueries tries to produce a short contextual snippet around
-// the first occurrence of any token extracted from the provided queries.
-// When no token matches (common for fully paraphrased semantic queries) it
-// falls back to the leading 160 runes of content so callers always get
-// something to scan. The snippet is single-lined and bounded in length to
-// keep the rendered XML compact.
-func extractSnippetForQueries(content string, queries []string) string {
- content = strings.TrimSpace(content)
- if content == "" {
- return ""
- }
-
- tokens := searchQueryTokens(queries)
-
- lowered := strings.ToLower(content)
- earliest := -1
- earliestEnd := -1
- for _, tok := range tokens {
- idx := strings.Index(lowered, tok)
- if idx < 0 {
- continue
- }
- end := idx + len(tok)
- if earliest < 0 || idx < earliest {
- earliest = idx
- earliestEnd = end
- }
- }
-
- if earliest < 0 {
- runes := []rune(content)
- if len(runes) > snippetContextRunes*2 {
- return strings.TrimSpace(string(runes[:snippetContextRunes*2])) + " ..."
- }
- return content
- }
-
- matchStr := content[earliest:earliestEnd]
- before := content[:earliest]
- after := content[earliestEnd:]
-
- beforeRunes := []rune(before)
- if len(beforeRunes) > snippetContextRunes {
- beforeRunes = beforeRunes[len(beforeRunes)-snippetContextRunes:]
- }
- afterRunes := []rune(after)
- if len(afterRunes) > snippetContextRunes {
- afterRunes = afterRunes[:snippetContextRunes]
- }
-
- snippet := string(beforeRunes) + matchStr + string(afterRunes)
- snippet = strings.ReplaceAll(snippet, "\n", " ")
- for strings.Contains(snippet, " ") {
- snippet = strings.ReplaceAll(snippet, " ", " ")
- }
- return "... " + strings.TrimSpace(snippet) + " ..."
-}
-
-// jaccard calculates Jaccard similarity between two token sets
-func (t *KnowledgeSearchTool) jaccard(a, b map[string]struct{}) float64 {
- return searchutil.Jaccard(a, b)
-}
diff --git a/internal/agent/tools/list_documents.go b/internal/agent/tools/list_documents.go
new file mode 100644
index 000000000..24b2bedee
--- /dev/null
+++ b/internal/agent/tools/list_documents.go
@@ -0,0 +1,290 @@
+package tools
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "sort"
+ "strings"
+
+ "github.com/Tencent/WeKnora/internal/types"
+ "github.com/Tencent/WeKnora/internal/types/interfaces"
+)
+
+const (
+ listDocumentsDefaultPageSize = 20
+ listDocumentsMaxPageSize = 100
+)
+
+var listDocumentsTool = BaseTool{
+ name: ToolListDocuments,
+ description: "List the documents inside one knowledge base, newest first, with pagination and an optional " +
+ "title filter.\nUse it to see what a knowledge base contains or to find a document by name; each entry " +
+ "carries the dN handle that read_document accepts.",
+ schema: json.RawMessage(`{
+ "type": "object",
+ "properties": {
+ "knowledge_base_id": {
+ "type": "string",
+ "description": "bN knowledge-base handle from the runtime context",
+ "minLength": 1
+ },
+ "keyword": {
+ "type": "string",
+ "description": "Optional substring to match against document titles and file names"
+ },
+ "page": {
+ "type": "integer",
+ "description": "1-based page number (default 1)",
+ "minimum": 1
+ },
+ "page_size": {
+ "type": "integer",
+ "description": "Documents per page (default 20, max 100)",
+ "minimum": 1,
+ "maximum": 100
+ }
+ },
+ "required": ["knowledge_base_id"]
+}`),
+}
+
+// ListDocumentsInput defines the input parameters for list_documents.
+type ListDocumentsInput struct {
+ KnowledgeBaseID string `json:"knowledge_base_id"`
+ Keyword string `json:"keyword,omitempty"`
+ Page int `json:"page,omitempty"`
+ PageSize int `json:"page_size,omitempty"`
+}
+
+// ListDocumentsTool pages through the documents of one knowledge base.
+type ListDocumentsTool struct {
+ BaseTool
+ knowledgeService interfaces.KnowledgeService
+ searchTargets types.SearchTargets
+}
+
+// NewListDocumentsTool creates a new list_documents tool.
+func NewListDocumentsTool(
+ knowledgeService interfaces.KnowledgeService, searchTargets types.SearchTargets,
+) *ListDocumentsTool {
+ return &ListDocumentsTool{
+ BaseTool: listDocumentsTool,
+ knowledgeService: knowledgeService,
+ searchTargets: searchTargets,
+ }
+}
+
+// Execute lists documents, honouring any document/tag scope pinned on the KB.
+func (t *ListDocumentsTool) Execute(ctx context.Context, args json.RawMessage) (*types.ToolResult, error) {
+ var input ListDocumentsInput
+ if err := json.Unmarshal(args, &input); err != nil {
+ return &types.ToolResult{Success: false, Error: fmt.Sprintf("Failed to parse args: %v", err)}, err
+ }
+ kbID := strings.TrimSpace(input.KnowledgeBaseID)
+ if kbID == "" {
+ return &types.ToolResult{Success: false, Error: "knowledge_base_id is required"},
+ fmt.Errorf("missing knowledge_base_id")
+ }
+ if err := validateKnowledgeBaseIDsInSearchTargets(t.searchTargets, []string{kbID}); err != nil {
+ return &types.ToolResult{Success: false, Error: err.Error()}, err
+ }
+ if t.knowledgeService == nil {
+ return &types.ToolResult{Success: false, Error: "knowledge service is unavailable"},
+ fmt.Errorf("service unavailable")
+ }
+
+ page := input.Page
+ if page < 1 {
+ page = 1
+ }
+ pageSize := input.PageSize
+ if pageSize < 1 {
+ pageSize = listDocumentsDefaultPageSize
+ }
+ if pageSize > listDocumentsMaxPageSize {
+ pageSize = listDocumentsMaxPageSize
+ }
+ keyword := strings.TrimSpace(input.Keyword)
+
+ filtered, total, err := t.listScoped(ctx, kbID, keyword, page, pageSize)
+ if err != nil {
+ return &types.ToolResult{Success: false, Error: fmt.Sprintf("failed to list documents: %v", err)}, err
+ }
+
+ documents := make([]map[string]interface{}, 0, len(filtered))
+ var b strings.Builder
+ fmt.Fprintf(&b, "<documents knowledge_base_id=\"%s\" total=\"%d\" page=\"%d\" page_size=\"%d\"",
+ xmlEscape(kbID), total, page, pageSize)
+ if keyword != "" {
+ fmt.Fprintf(&b, " keyword=\"%s\"", xmlEscape(keyword))
+ }
+ b.WriteString(">\n")
+ for _, k := range filtered {
+ fmt.Fprintf(&b, "<document knowledge_id=\"%s\" title=\"%s\"", xmlEscape(k.ID), xmlEscape(k.Title))
+ if k.FileType != "" {
+ fmt.Fprintf(&b, " file_type=\"%s\"", xmlEscape(k.FileType))
+ }
+ if k.ParseStatus != "" {
+ fmt.Fprintf(&b, " parse_status=\"%s\"", xmlEscape(formatParseStatus(k.ParseStatus)))
+ }
+ if !k.UpdatedAt.IsZero() {
+ fmt.Fprintf(&b, " updated_at=\"%s\"", k.UpdatedAt.Format("2006-01-02"))
+ }
+ if k.Description != "" {
+ fmt.Fprintf(&b, ">%s</document>\n", xmlEscape(k.Description))
+ } else {
+ b.WriteString(" />\n")
+ }
+ doc := map[string]interface{}{
+ "knowledge_id": k.ID,
+ "title": k.Title,
+ "description": k.Description,
+ "type": k.Type,
+ "source": k.Source,
+ "file_name": k.FileName,
+ "file_type": k.FileType,
+ "file_size": k.FileSize,
+ "parse_status": k.ParseStatus,
+ "is_faq": false,
+ }
+ if !k.UpdatedAt.IsZero() {
+ doc["updated_at"] = k.UpdatedAt.Format("2006-01-02T15:04:05Z07:00")
+ }
+ documents = append(documents, doc)
+ }
+ b.WriteString("</documents>")
+
+ data := map[string]interface{}{
+ "display_type": "document_info",
+ "knowledge_base_id": kbID,
+ "documents": documents,
+ "total_docs": total,
+ "page": page,
+ "page_size": pageSize,
+ }
+ if keyword != "" {
+ data["keyword"] = keyword
+ }
+ if int64(page*pageSize) < total {
+ data["next_page"] = page + 1
+ }
+ output := b.String()
+ if len(filtered) == 0 {
+ if keyword != "" {
+ output = fmt.Sprintf("No documents matching %q in knowledge base %s.", keyword, kbID)
+ } else if total == 0 {
+ output = fmt.Sprintf("Knowledge base %s has no documents.", kbID)
+ }
+ }
+ return &types.ToolResult{Success: true, Output: output, Data: data}, nil
+}
+
+// listDocumentsScopeCap bounds how many tagged documents are loaded when a
+// turn pins both explicit documents and tags on the same knowledge base, the
+// one case where the union has to be paged in memory.
+const listDocumentsScopeCap = 1000
+
+// listScoped returns one page of the documents the turn may read, with a
+// total that counts only those documents. A pinned @tag scope is pushed into
+// the database filter; pinned @file documents are loaded directly. Filtering
+// after paging would report totals for the whole base and leave holes in the
+// page.
+func (t *ListDocumentsTool) listScoped(
+ ctx context.Context, kbID, keyword string, page, pageSize int,
+) ([]*types.Knowledge, int64, error) {
+ tenantID := t.searchTargets.GetTenantIDForKB(kbID)
+ // Documents live under the knowledge base owner, which may differ from the
+ // caller's tenant for organization-shared bases.
+ listCtx := ctx
+ if tenantID != 0 {
+ listCtx = context.WithValue(ctx, types.TenantIDContextKey, tenantID)
+ }
+ listPage := func(filter types.KnowledgeListFilter, p, size int) ([]*types.Knowledge, int64, error) {
+ filter.Keyword = keyword
+ result, err := t.knowledgeService.ListPagedKnowledgeByKnowledgeBaseID(
+ listCtx, kbID, &types.Pagination{Page: p, PageSize: size}, filter,
+ )
+ if err != nil || result == nil {
+ return nil, 0, err
+ }
+ rows, _ := result.Data.([]*types.Knowledge)
+ return rows, result.Total, nil
+ }
+
+ var explicitIDs, tagIDs []string
+ for _, target := range t.searchTargets {
+ if target == nil || target.KnowledgeBaseID != kbID {
+ continue
+ }
+ if searchTargetIsWholeKB(target) {
+ return listPage(types.KnowledgeListFilter{}, page, pageSize)
+ }
+ ids, tags := searchTargetScope(target)
+ explicitIDs = append(explicitIDs, ids...)
+ tagIDs = append(tagIDs, tags...)
+ }
+ explicitIDs = dedupNonEmptyStrings(explicitIDs)
+ tagIDs = dedupNonEmptyStrings(tagIDs)
+
+ if len(explicitIDs) == 0 {
+ if len(tagIDs) == 0 {
+ return nil, 0, nil
+ }
+ return listPage(types.KnowledgeListFilter{TagIDs: tagIDs}, page, pageSize)
+ }
+
+ // Explicit documents (plus any tagged ones) form a small union that is
+ // paged in memory with the same newest-first order as the database.
+ union := make(map[string]*types.Knowledge)
+ explicit, err := t.knowledgeService.GetKnowledgeBatch(listCtx, tenantID, explicitIDs)
+ if err != nil {
+ return nil, 0, err
+ }
+ for _, k := range explicit {
+ if k != nil && k.KnowledgeBaseID == kbID && knowledgeMatchesKeyword(k, keyword) {
+ union[k.ID] = k
+ }
+ }
+ if len(tagIDs) > 0 {
+ tagged, _, err := listPage(types.KnowledgeListFilter{TagIDs: tagIDs}, 1, listDocumentsScopeCap)
+ if err != nil {
+ return nil, 0, err
+ }
+ for _, k := range tagged {
+ if k != nil {
+ union[k.ID] = k
+ }
+ }
+ }
+ all := make([]*types.Knowledge, 0, len(union))
+ for _, k := range union {
+ all = append(all, k)
+ }
+ sort.Slice(all, func(i, j int) bool {
+ if !all[i].CreatedAt.Equal(all[j].CreatedAt) {
+ return all[i].CreatedAt.After(all[j].CreatedAt)
+ }
+ return all[i].ID < all[j].ID
+ })
+ total := int64(len(all))
+ start := (page - 1) * pageSize
+ if start >= len(all) {
+ return nil, total, nil
+ }
+ end := start + pageSize
+ if end > len(all) {
+ end = len(all)
+ }
+ return all[start:end], total, nil
+}
+
+// knowledgeMatchesKeyword mirrors the list filter's title / file name match.
+func knowledgeMatchesKeyword(k *types.Knowledge, keyword string) bool {
+ if keyword == "" {
+ return true
+ }
+ needle := strings.ToLower(keyword)
+ return strings.Contains(strings.ToLower(k.Title), needle) ||
+ strings.Contains(strings.ToLower(k.FileName), needle)
+}
diff --git a/internal/agent/tools/list_documents_test.go b/internal/agent/tools/list_documents_test.go
new file mode 100644
index 000000000..55d45ae89
--- /dev/null
+++ b/internal/agent/tools/list_documents_test.go
@@ -0,0 +1,166 @@
+package tools
+
+import (
+ "context"
+ "encoding/json"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/Tencent/WeKnora/internal/types"
+)
+
+func newListDocumentsFixture() *ListDocumentsTool {
+ updated := time.Date(2026, 3, 4, 5, 6, 7, 0, time.UTC)
+ docs := []*types.Knowledge{
+ {
+ ID: "doc-a", KnowledgeBaseID: "kb-1", Title: "Alpha Guide", FileType: "pdf",
+ ParseStatus: "completed", UpdatedAt: updated, Description: "First",
+ },
+ {
+ ID: "doc-b", KnowledgeBaseID: "kb-1", Title: "Beta Notes", FileType: "md",
+ ParseStatus: "completed", UpdatedAt: updated,
+ },
+ {
+ ID: "doc-c", KnowledgeBaseID: "kb-1", Title: "Alpha Errata", FileType: "txt",
+ ParseStatus: "failed", UpdatedAt: updated,
+ },
+ }
+ service := &readDocKnowledgeService{
+ docs: map[string]*types.Knowledge{"doc-a": docs[0], "doc-b": docs[1], "doc-c": docs[2]},
+ pages: map[string]*types.PageResult{"kb-1": {Data: docs}},
+ }
+ return NewListDocumentsTool(service, types.SearchTargets{{
+ Type: types.SearchTargetTypeKnowledgeBase, KnowledgeBaseID: "kb-1", TenantID: 7,
+ }})
+}
+
+func TestListDocumentsRequiresKnowledgeBaseInScope(t *testing.T) {
+ tool := newListDocumentsFixture()
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{}`))
+ if err == nil || res.Success {
+ t.Fatalf("missing kb: res=%+v err=%v", res, err)
+ }
+ res, err = tool.Execute(context.Background(), json.RawMessage(`{"knowledge_base_id":"kb-9"}`))
+ if err == nil || res.Success {
+ t.Fatalf("out-of-scope kb: res=%+v err=%v", res, err)
+ }
+}
+
+func TestListDocumentsRendersRowsForModelAndUI(t *testing.T) {
+ tool := newListDocumentsFixture()
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"knowledge_base_id":"kb-1","page_size":2}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ rows, _ := res.Data["documents"].([]map[string]interface{})
+ if len(rows) != 2 || rows[0]["knowledge_id"] != "doc-a" || rows[0]["is_faq"] != false {
+ t.Fatalf("rows = %+v", rows)
+ }
+ if res.Data["display_type"] != "document_info" || res.Data["total_docs"] != int64(3) || res.Data["page"] != 1 {
+ t.Fatalf("data = %+v", res.Data)
+ }
+ if res.Data["next_page"] != 2 {
+ t.Fatalf("3 documents at page_size 2 must advertise page 2: %+v", res.Data)
+ }
+ if !strings.Contains(res.Output, `<documents knowledge_base_id="kb-1" total="3" page="1" page_size="2">`) ||
+ !strings.Contains(res.Output, `updated_at="2026-03-04"`) ||
+ !strings.Contains(res.Output, ">First</document>") {
+ t.Fatalf("output = %s", res.Output)
+ }
+}
+
+func TestListDocumentsKeywordFilterAndEmptyStatement(t *testing.T) {
+ tool := newListDocumentsFixture()
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"knowledge_base_id":"kb-1","keyword":"alpha"}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ rows, _ := res.Data["documents"].([]map[string]interface{})
+ if len(rows) != 2 || res.Data["keyword"] != "alpha" {
+ t.Fatalf("keyword filter rows = %+v data=%+v", rows, res.Data)
+ }
+ res, err = tool.Execute(context.Background(), json.RawMessage(`{"knowledge_base_id":"kb-1","keyword":"zzz"}`))
+ if err != nil || !res.Success || !strings.Contains(res.Output, `No documents matching "zzz"`) {
+ t.Fatalf("empty keyword result: res=%+v err=%v", res, err)
+ }
+}
+
+func TestListDocumentsPinnedDocumentsPageAndCountOnlyInScope(t *testing.T) {
+ tool := newListDocumentsFixture()
+ tool.searchTargets = types.SearchTargets{{
+ Type: types.SearchTargetTypeKnowledge, KnowledgeBaseID: "kb-1", TenantID: 7,
+ KnowledgeIDs: []string{"doc-b", "doc-c"},
+ }}
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"knowledge_base_id":"kb-1","page_size":1}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ rows, _ := res.Data["documents"].([]map[string]interface{})
+ if len(rows) != 1 || res.Data["total_docs"] != int64(2) || res.Data["next_page"] != 2 {
+ t.Fatalf("pinned scope must be counted and paged by itself: rows=%+v data=%+v", rows, res.Data)
+ }
+
+ res, _ = tool.Execute(context.Background(), json.RawMessage(`{"knowledge_base_id":"kb-1","page_size":1,"page":2}`))
+ second, _ := res.Data["documents"].([]map[string]interface{})
+ if len(second) != 1 || second[0]["knowledge_id"] == rows[0]["knowledge_id"] {
+ t.Fatalf("page 2 must hold the other pinned document: %+v", second)
+ }
+ if _, ok := res.Data["next_page"]; ok {
+ t.Fatalf("no page after the last pinned document: %+v", res.Data)
+ }
+
+ res, _ = tool.Execute(context.Background(), json.RawMessage(`{"knowledge_base_id":"kb-1","keyword":"beta"}`))
+ filtered, _ := res.Data["documents"].([]map[string]interface{})
+ if len(filtered) != 1 || filtered[0]["knowledge_id"] != "doc-b" || res.Data["total_docs"] != int64(1) {
+ t.Fatalf("keyword applies to pinned documents: %+v data=%+v", filtered, res.Data)
+ }
+}
+
+func TestListDocumentsTagScopeIsPushedIntoTheQuery(t *testing.T) {
+ tool := newListDocumentsFixture()
+ service := tool.knowledgeService.(*readDocKnowledgeService)
+ service.tags = map[string][]*types.KnowledgeTag{
+ "doc-a": {testKnowledgeTag("tag-x")},
+ "doc-c": {testKnowledgeTag("tag-x")},
+ }
+ tool.searchTargets = types.SearchTargets{{
+ Type: types.SearchTargetTypeKnowledgeBase, KnowledgeBaseID: "kb-1", TenantID: 7, TagIDs: []string{"tag-x"},
+ }}
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"knowledge_base_id":"kb-1","page_size":1}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if res.Data["total_docs"] != int64(2) || res.Data["next_page"] != 2 {
+ t.Fatalf("tag scope total must count tagged documents only: %+v", res.Data)
+ }
+ last := service.listFilters[len(service.listFilters)-1]
+ if len(last.TagIDs) != 1 || last.TagIDs[0] != "tag-x" {
+ t.Fatalf("tag scope must reach the database filter: %+v", last)
+ }
+}
+
+func TestListDocumentsUnionOfPinnedDocumentsAndTags(t *testing.T) {
+ tool := newListDocumentsFixture()
+ service := tool.knowledgeService.(*readDocKnowledgeService)
+ service.tags = map[string][]*types.KnowledgeTag{"doc-c": {testKnowledgeTag("tag-x")}}
+ tool.searchTargets = types.SearchTargets{
+ {Type: types.SearchTargetTypeKnowledge, KnowledgeBaseID: "kb-1", TenantID: 7, KnowledgeIDs: []string{"doc-a"}},
+ {Type: types.SearchTargetTypeKnowledgeBase, KnowledgeBaseID: "kb-1", TenantID: 7, TagIDs: []string{"tag-x"}},
+ }
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"knowledge_base_id":"kb-1"}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ rows, _ := res.Data["documents"].([]map[string]interface{})
+ got := make([]string, 0, len(rows))
+ for _, r := range rows {
+ got = append(got, r["knowledge_id"].(string))
+ }
+ if strings.Join(got, ",") != "doc-a,doc-c" && strings.Join(got, ",") != "doc-c,doc-a" {
+ t.Fatalf("union = %v", got)
+ }
+ if res.Data["total_docs"] != int64(2) {
+ t.Fatalf("union total = %+v", res.Data)
+ }
+}
diff --git a/internal/agent/tools/list_knowledge_chunks.go b/internal/agent/tools/list_knowledge_chunks.go
deleted file mode 100644
index b192ee26a..000000000
--- a/internal/agent/tools/list_knowledge_chunks.go
+++ /dev/null
@@ -1,441 +0,0 @@
-package tools
-
-import (
- "context"
- "encoding/json"
- "fmt"
- "strings"
-
- "github.com/Tencent/WeKnora/internal/searchutil"
- "github.com/Tencent/WeKnora/internal/types"
- "github.com/Tencent/WeKnora/internal/types/interfaces"
-)
-
-var listKnowledgeChunksTool = BaseTool{
- name: ToolListKnowledgeChunks,
- description: `Retrieve full chunk content for a document or a single FAQ entry.
-
-## Use After grep_chunks or knowledge_search:
-- **FAQ hit** (type faq): list_knowledge_chunks(faq_id="cN") — reads that one FAQ chunk with answers from metadata.
-- **Document hit**: list_knowledge_chunks(knowledge_id="dN") — pages through all chunks.
-
-## Parameters (provide exactly one id target):
-- faq_id (optional): Short cN ID for an FAQ chunk from grep_chunks / knowledge_search.
-- chunk_id (optional): Short cN ID for a single non-FAQ chunk.
-- knowledge_id (optional): Short dN document ID to page through all chunks.
-- limit / offset: Only for knowledge_id paging (default limit 20, max 100).
-
-## Output:
-Full chunk content. FAQ entries include <faq> with <answer> from metadata.`,
- schema: json.RawMessage(`{
- "type": "object",
- "properties": {
- "faq_id": {
- "type": "string",
- "description": "Short cN FAQ chunk ID. Use for FAQ hits instead of the parent dN document ID."
- },
- "chunk_id": {
- "type": "string",
- "description": "Short cN ID for one non-FAQ chunk"
- },
- "knowledge_id": {
- "type": "string",
- "description": "Short dN document ID to list all chunks"
- },
- "limit": {
- "type": "integer",
- "description": "Chunks per page when using knowledge_id (default 20, max 100)",
- "default": 20,
- "minimum": 1,
- "maximum": 100
- },
- "offset": {
- "type": "integer",
- "description": "Start position when using knowledge_id (default 0)",
- "default": 0,
- "minimum": 0
- }
- }
-}`),
-}
-
-// ListKnowledgeChunksInput defines the input parameters for list knowledge chunks tool
-type ListKnowledgeChunksInput struct {
- KnowledgeID string `json:"knowledge_id,omitempty"`
- FAQID string `json:"faq_id,omitempty"`
- ChunkID string `json:"chunk_id,omitempty"`
- Limit int `json:"limit"`
- Offset int `json:"offset"`
-}
-
-// ListKnowledgeChunksTool retrieves chunk snapshots for a specific knowledge document.
-type ListKnowledgeChunksTool struct {
- BaseTool
- chunkService interfaces.ChunkService
- knowledgeService interfaces.KnowledgeService
- searchTargets types.SearchTargets // Pre-computed unified search targets with KB-tenant mapping
-}
-
-// NewListKnowledgeChunksTool creates a new tool instance.
-func NewListKnowledgeChunksTool(
- knowledgeService interfaces.KnowledgeService,
- chunkService interfaces.ChunkService,
- searchTargets types.SearchTargets,
-) *ListKnowledgeChunksTool {
- return &ListKnowledgeChunksTool{
- BaseTool: listKnowledgeChunksTool,
- chunkService: chunkService,
- knowledgeService: knowledgeService,
- searchTargets: searchTargets,
- }
-}
-
-// Execute performs the chunk fetch against the chunk service.
-func (t *ListKnowledgeChunksTool) Execute(ctx context.Context, args json.RawMessage) (*types.ToolResult, error) {
- // Parse args from json.RawMessage
- var input ListKnowledgeChunksInput
- if err := json.Unmarshal(args, &input); err != nil {
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("Failed to parse args: %v", err),
- }, err
- }
-
- chunkID := strings.TrimSpace(input.FAQID)
- if chunkID == "" {
- chunkID = strings.TrimSpace(input.ChunkID)
- }
- if chunkID != "" {
- return t.executeByChunkID(ctx, chunkID)
- }
-
- knowledgeID := strings.TrimSpace(input.KnowledgeID)
- if knowledgeID == "" {
- return &types.ToolResult{
- Success: false,
- Error: "one of faq_id, chunk_id, or knowledge_id is required",
- }, fmt.Errorf("missing id parameter")
- }
-
- knowledge, err := authorizeKnowledgeInSearchTargets(ctx, t.searchTargets, knowledgeID, t.knowledgeService)
- if err != nil {
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("Knowledge is not accessible: %v", err),
- }, err
- }
-
- // Use the knowledge's actual tenant_id for chunk query (supports cross-tenant shared KB)
- effectiveTenantID := knowledge.TenantID
-
- chunkLimit := 20
- if input.Limit > 0 {
- chunkLimit = input.Limit
- }
- offset := 0
- if input.Offset > 0 {
- offset = input.Offset
- }
- if offset < 0 {
- offset = 0
- }
-
- pagination := &types.Pagination{
- Page: offset/chunkLimit + 1,
- PageSize: chunkLimit,
- }
-
- enabled := true
- chunks, total, err := t.chunkService.GetRepository().ListPagedChunksByKnowledgeID(ctx,
- effectiveTenantID, knowledgeID, pagination, []types.ChunkType{types.ChunkTypeText, types.ChunkTypeFAQ}, nil, "", "", "", "", &enabled)
- if err != nil {
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("failed to list chunks: %v", err),
- }, err
- }
- if chunks == nil {
- return &types.ToolResult{
- Success: false,
- Error: "chunk query returned no data",
- }, fmt.Errorf("chunk query returned no data")
- }
-
- totalChunks := total
- fetched := len(chunks)
-
- // Explicit out-of-range guidance: when the caller paged past the end
- // (offset >= total with total > 0), silently returning fetched=0 is
- // confusing for LLMs that just saw the document in search results. Tell
- // them exactly what happened and what offset would be valid so the next
- // call lands on a real page.
- if fetched == 0 && totalChunks > 0 && int64(offset) >= totalChunks {
- suggestedOffset := totalChunks - int64(chunkLimit)
- if suggestedOffset < 0 {
- suggestedOffset = 0
- }
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf(
- "offset %d is out of range: document has only %d chunks (valid offset range: 0..%d). Retry with offset=%d (or any value < %d).",
- offset, totalChunks, totalChunks-1, suggestedOffset, totalChunks,
- ),
- Data: map[string]interface{}{
- "knowledge_id": knowledgeID,
- "total_chunks": totalChunks,
- "requested_offset": offset,
- "requested_limit": chunkLimit,
- "suggested_offset": suggestedOffset,
- },
- }, nil
- }
-
- // Enrich image info from child image chunks (lazy loading)
- if fetched > 0 {
- chunkIDs := make([]string, 0, fetched)
- for _, c := range chunks {
- chunkIDs = append(chunkIDs, c.ID)
- }
- infoMap := searchutil.CollectImageInfoByChunkIDs(ctx, t.chunkService.GetRepository(), effectiveTenantID, chunkIDs)
- for _, c := range chunks {
- if c.ImageInfo == "" {
- if merged, ok := infoMap[c.ID]; ok {
- c.ImageInfo = merged
- }
- }
- }
- }
-
- knowledgeTitle := t.lookupKnowledgeTitle(ctx, knowledgeID)
-
- output := t.buildOutput(knowledgeID, knowledgeTitle, totalChunks, fetched, chunks)
-
- formattedChunks := make([]map[string]interface{}, 0, len(chunks))
- for idx, c := range chunks {
- chunkData := map[string]interface{}{
- "seq": idx + 1,
- "chunk_id": c.ID,
- "chunk_index": c.ChunkIndex,
- "content": c.Content,
- "chunk_type": c.ChunkType,
- "knowledge_id": c.KnowledgeID,
- "knowledge_base": c.KnowledgeBaseID,
- "start_at": c.StartAt,
- "end_at": c.EndAt,
- "parent_chunk_id": c.ParentChunkID,
- }
-
- appendFAQChunkData(chunkData, c)
- normalizeFAQChunkDataMap(chunkData, c)
-
- // 添加图片信息
- if c.ImageInfo != "" {
- var imageInfos []types.ImageInfo
- if err := json.Unmarshal([]byte(c.ImageInfo), &imageInfos); err == nil && len(imageInfos) > 0 {
- imageList := make([]map[string]string, 0, len(imageInfos))
- for _, img := range imageInfos {
- imgData := make(map[string]string)
- if img.URL != "" {
- imgData["url"] = img.URL
- }
- if img.Caption != "" {
- imgData["caption"] = img.Caption
- }
- if img.OCRText != "" {
- imgData["ocr_text"] = img.OCRText
- }
- if len(imgData) > 0 {
- imageList = append(imageList, imgData)
- }
- }
- if len(imageList) > 0 {
- chunkData["images"] = imageList
- }
- }
- }
-
- formattedChunks = append(formattedChunks, chunkData)
- }
-
- return &types.ToolResult{
- Success: true,
- Output: output,
- Data: map[string]interface{}{
- "display_type": "knowledge_chunks_list",
- "knowledge_id": knowledgeID,
- "knowledge_title": knowledgeTitle,
- "total_chunks": totalChunks,
- "fetched_chunks": fetched,
- "page": pagination.Page,
- "page_size": pagination.PageSize,
- "chunks": formattedChunks,
- },
- }, nil
-}
-
-// executeByChunkID loads one chunk by faq_id / chunk_id (FAQ entry or any chunk).
-func (t *ListKnowledgeChunksTool) executeByChunkID(ctx context.Context, chunkID string) (*types.ToolResult, error) {
- chunk, err := authorizeChunkInSearchTargets(
- ctx, t.searchTargets, chunkID, t.chunkService, t.knowledgeService,
- )
- if err != nil {
- return &types.ToolResult{
- Success: false,
- Error: fmt.Sprintf("chunk is not accessible: %v", err),
- }, err
- }
-
- chunks := []*types.Chunk{chunk}
- if chunk.ImageInfo == "" {
- effectiveTenantID := t.searchTargets.GetTenantIDForKB(chunk.KnowledgeBaseID)
- if effectiveTenantID > 0 {
- infoMap := searchutil.CollectImageInfoByChunkIDs(ctx, t.chunkService.GetRepository(), effectiveTenantID, []string{chunk.ID})
- if merged, ok := infoMap[chunk.ID]; ok {
- chunk.ImageInfo = merged
- }
- }
- }
-
- knowledgeTitle := t.lookupKnowledgeTitle(ctx, chunk.KnowledgeID)
- output := t.buildOutput(chunk.KnowledgeID, knowledgeTitle, 1, 1, chunks)
-
- formattedChunks := []map[string]interface{}{
- {
- "seq": 1,
- "chunk_id": chunk.ID,
- "chunk_index": chunk.ChunkIndex,
- "content": chunk.Content,
- "chunk_type": chunk.ChunkType,
- "knowledge_id": chunk.KnowledgeID,
- "knowledge_base": chunk.KnowledgeBaseID,
- },
- }
- appendFAQChunkData(formattedChunks[0], chunk)
- normalizeFAQChunkDataMap(formattedChunks[0], chunk)
-
- data := map[string]interface{}{
- "display_type": "knowledge_chunks_list",
- "knowledge_id": chunk.KnowledgeID,
- "knowledge_title": knowledgeTitle,
- "total_chunks": int64(1),
- "fetched_chunks": 1,
- "page": 1,
- "page_size": 1,
- "chunks": formattedChunks,
- "faq_id": chunk.ID,
- "single_chunk": true,
- }
- if q := faqStandardQuestion(chunk); q != "" {
- data["faq_question"] = q
- }
-
- return &types.ToolResult{
- Success: true,
- Output: output,
- Data: data,
- }, nil
-}
-
-// lookupKnowledgeTitle looks up the title of a knowledge document
-// Uses GetKnowledgeByIDOnly to support cross-tenant shared KB
-func (t *ListKnowledgeChunksTool) lookupKnowledgeTitle(ctx context.Context, knowledgeID string) string {
- if t.knowledgeService == nil {
- return ""
- }
- knowledge, err := t.knowledgeService.GetKnowledgeByIDOnly(ctx, knowledgeID)
- if err != nil || knowledge == nil {
- return ""
- }
- return strings.TrimSpace(knowledge.Title)
-}
-
-// buildOutput builds the output as XML for the list knowledge chunks tool
-func (t *ListKnowledgeChunksTool) buildOutput(
- knowledgeID string,
- knowledgeTitle string,
- total int64,
- fetched int,
- chunks []*types.Chunk,
-) string {
- var b strings.Builder
-
- titleAttr := ""
- if knowledgeTitle != "" {
- titleAttr = fmt.Sprintf(" title=\"%s\"", knowledgeTitle)
- }
- fmt.Fprintf(&b, "<knowledge_chunks knowledge_id=\"%s\"%s total=\"%d\" fetched=\"%d\">\n",
- knowledgeID, titleAttr, total, fetched)
-
- if fetched == 0 {
- b.WriteString("</knowledge_chunks>")
- return b.String()
- }
-
- for _, c := range chunks {
- if c.ChunkType == types.ChunkTypeFAQ {
- writeFAQEntryXML(&b, c)
- writeChunkImagesMarkdown(&b, c)
- continue
- }
-
- if q := faqStandardQuestion(c); q != "" {
- fmt.Fprintf(&b, "<chunk chunk_id=\"%s\" chunk_index=\"%d\" type=\"%s\" question=\"%s\">\n",
- c.ID, c.ChunkIndex, c.ChunkType, xmlEscape(q))
- } else {
- fmt.Fprintf(&b, "<chunk chunk_id=\"%s\" chunk_index=\"%d\" type=\"%s\">\n",
- c.ID, c.ChunkIndex, c.ChunkType)
- }
- fmt.Fprintf(&b, "<content>%s</content>\n", summarizeContent(c.Content))
- writeChunkImagesMarkdown(&b, c)
- b.WriteString("</chunk>\n")
- }
-
- if int64(fetched) < total {
- fmt.Fprintf(&b, "<pagination remaining=\"%d\" />\n", int64(total)-int64(fetched))
- }
-
- b.WriteString("</knowledge_chunks>")
- return b.String()
-}
-
-func writeChunkImagesMarkdown(b *strings.Builder, c *types.Chunk) {
- if c == nil || c.ImageInfo == "" {
- return
- }
- var imageInfos []types.ImageInfo
- if err := json.Unmarshal([]byte(c.ImageInfo), &imageInfos); err != nil || len(imageInfos) == 0 {
- return
- }
- for _, img := range imageInfos {
- if imageMarkdown := searchutil.BuildImageInfoMarkdownWithURL(img.URL, &img); imageMarkdown != "" {
- b.WriteString(imageMarkdown)
- b.WriteString("\n")
- }
- }
-}
-
-// faqStandardQuestion returns the FAQ standard question for an FAQ-type chunk,
-// or "" for non-FAQ chunks (or when metadata is missing/unparseable). All FAQ
-// entries inside one knowledge share the same knowledge title, so surfacing the
-// standard question gives each entry a distinct, human-readable identity in
-// tool output that would otherwise look like duplicate same-titled chunks.
-func faqStandardQuestion(c *types.Chunk) string {
- if c == nil || c.ChunkType != types.ChunkTypeFAQ {
- return ""
- }
- meta, err := c.FAQMetadata()
- if err != nil || meta == nil {
- return ""
- }
- return strings.TrimSpace(meta.StandardQuestion)
-}
-
-// summarizeContent summarizes the content of a chunk
-func summarizeContent(content string) string {
- cleaned := strings.TrimSpace(content)
- if cleaned == "" {
- return "(empty)"
- }
-
- return strings.TrimSpace(string(cleaned))
-}
diff --git a/internal/agent/tools/persist.go b/internal/agent/tools/persist.go
index 3144c7fd2..eed6eee6c 100644
--- a/internal/agent/tools/persist.go
+++ b/internal/agent/tools/persist.go
@@ -327,7 +327,7 @@ func compactToolSummary(success bool, errMsg string, data map[string]interface{}
count = intField(data, "count")
}
if count > 0 {
- return fmt.Sprintf("Semantic search returned %d result(s) (details omitted from history)", count)
+ return fmt.Sprintf("Knowledge search returned %d result(s) (details omitted from history)", count)
}
case "shell_exec":
if rebuilt := rebuildShellExecHistory(data); rebuilt != "" {
diff --git a/internal/agent/tools/persist_test.go b/internal/agent/tools/persist_test.go
index a3da5d24b..60c5b9c13 100644
--- a/internal/agent/tools/persist_test.go
+++ b/internal/agent/tools/persist_test.go
@@ -8,10 +8,10 @@ import (
)
func TestShouldOmitRawToolOutput(t *testing.T) {
- if !ShouldOmitRawToolOutput(ToolListKnowledgeChunks, map[string]interface{}{"display_type": "knowledge_chunks_list"}) {
- t.Fatal("structured list_knowledge_chunks output should be omitted")
+ if !ShouldOmitRawToolOutput(ToolReadDocument, map[string]interface{}{"display_type": "knowledge_chunks_list"}) {
+ t.Fatal("structured read_document output should be omitted")
}
- if !ShouldOmitRawToolOutput(ToolGrepChunks, map[string]interface{}{"display_type": "grep_results"}) {
+ if !ShouldOmitRawToolOutput(LegacyToolGrepChunks, map[string]interface{}{"display_type": "grep_results"}) {
t.Fatal("structured grep output should be omitted")
}
if ShouldOmitRawToolOutput("custom_tool", nil) {
@@ -45,7 +45,7 @@ func TestSanitizeToolDataForPersist_knowledgeChunksList(t *testing.T) {
"total_chunks": 282,
"chunks": []map[string]interface{}{{"content": "secret"}},
}
- out := SanitizeToolDataForPersist(ToolListKnowledgeChunks, data)
+ out := SanitizeToolDataForPersist(ToolReadDocument, data)
if _, ok := out["chunks"]; ok {
t.Fatal("chunk bodies should be stripped from persisted tool data")
}
@@ -59,7 +59,7 @@ func TestSanitizeAgentStepsForStorage_stripsLargeOutput(t *testing.T) {
Iteration: 1,
ToolCalls: []types.ToolCall{{
ID: "call-1",
- Name: ToolListKnowledgeChunks,
+ Name: ToolReadDocument,
Result: &types.ToolResult{
Success: true,
Output: strings.Repeat("x", 10000),
@@ -88,7 +88,7 @@ func TestSanitizeAgentStepsForStorage_stripsLargeOutput(t *testing.T) {
}
func TestSanitizeToolResultForClient_omitsOutput(t *testing.T) {
- meta := SanitizeToolResultForClient(ToolListKnowledgeChunks, &types.ToolResult{
+ meta := SanitizeToolResultForClient(ToolReadDocument, &types.ToolResult{
Success: true,
Output: "<knowledge_chunks>very large</knowledge_chunks>",
Data: map[string]interface{}{
diff --git a/internal/agent/tools/query_knowledge_graph.go b/internal/agent/tools/query_knowledge_graph.go
index 313ead57f..44f424223 100644
--- a/internal/agent/tools/query_knowledge_graph.go
+++ b/internal/agent/tools/query_knowledge_graph.go
@@ -19,43 +19,10 @@ type graphConfigSummary struct {
var queryKnowledgeGraphTool = BaseTool{
name: ToolQueryKnowledgeGraph,
- description: `Query knowledge graph to explore entity relationships and knowledge networks.
-
-## Core Function
-Explores relationships between entities in knowledge bases that have graph extraction configured.
-
-## When to Use
-✅ **Use for**:
-- Understanding relationships between entities (e.g., "relationship between Docker and Kubernetes")
-- Exploring knowledge networks and concept associations
-- Finding related information about specific entities
-- Understanding technical architecture and system relationships
-
-❌ **Don't use for**:
-- General text search → use knowledge_search
-- Knowledge base without graph extraction configured
-- Need exact document content → use knowledge_search
-
-## Parameters
-- **knowledge_base_ids** (required): Array of short bN knowledge base IDs (1-10). Only KBs with graph extraction configured will be effective.
-- **query** (required): Query content - can be entity name, relationship query, or concept search.
-
-## Graph Configuration
-Knowledge graph must be pre-configured in knowledge bases:
-- **Entity types** (Nodes): e.g., "Technology", "Tool", "Concept"
-- **Relationship types** (Relations): e.g., "depends_on", "uses", "contains"
-
-If KB is not configured with graph, tool will return regular search results.
-
-## Workflow
-1. **Relationship exploration**: query_knowledge_graph → list_knowledge_chunks (for detailed content)
-2. **Network analysis**: query_knowledge_graph → knowledge_search (for comprehensive understanding)
-3. **Topic research**: knowledge_search → query_knowledge_graph (for deep entity relationships)
-
-## Notes
-- Results indicate graph configuration status
-- Cross-KB results are automatically deduplicated
-- Results are sorted by relevance`,
+ description: "Query the knowledge graph of graph-enabled knowledge bases to explore how entities relate " +
+ "(for example \"relationship between Docker and Kubernetes\"). Returns the chunks that carry those " +
+ "relationships with cN handles.\nUse search_knowledge for ordinary text retrieval and " +
+ "read_document(id=dN) to read a source in full.",
schema: utils.GenerateSchema[QueryKnowledgeGraphInput](),
}
@@ -295,8 +262,7 @@ func (t *QueryKnowledgeGraphTool) Execute(ctx context.Context, args json.RawMess
}
if !hasGraphConfig {
- output += "⚠️ None of the queried knowledge bases have graph extraction configured\n"
- output += "💡 Hint: Configure entity and relationship types in knowledge base settings\n\n"
+ output += "⚠️ None of the queried knowledge bases have graph extraction configured\n\n"
}
// Display result counts by KB
@@ -310,11 +276,6 @@ func (t *QueryKnowledgeGraphTool) Execute(ctx context.Context, args json.RawMess
// Display search results
output += "=== 🔍 Query Results ===\n\n"
- if !hasGraphConfig {
- output += "💡 Returning relevant document chunks (knowledge base has no graph configuration)\n\n"
- } else {
- output += "💡 Content retrieval based on graph configuration\n\n"
- }
formattedResults := make([]map[string]interface{}, 0, len(allResults))
currentKB := ""
@@ -352,15 +313,6 @@ func (t *QueryKnowledgeGraphTool) Execute(ctx context.Context, args json.RawMess
})
}
- output += "=== 💡 Tips ===\n"
- output += "- ✓ Results are deduplicated across knowledge bases and sorted by relevance\n"
- output += "- ✓ Use get_chunk_detail to get full content\n"
- output += "- ✓ Use list_knowledge_chunks to explore context\n"
- if !hasGraphConfig {
- output += "- ⚠️ Configure graph extraction for more precise entity-relationship results\n"
- }
- output += "- ⏳ Full graph query language (Cypher) support is under development\n"
-
// Build structured graph data for frontend visualization
graphData := buildGraphVisualizationData(allResults)
diff --git a/internal/agent/tools/read_document.go b/internal/agent/tools/read_document.go
new file mode 100644
index 000000000..50d43f227
--- /dev/null
+++ b/internal/agent/tools/read_document.go
@@ -0,0 +1,614 @@
+package tools
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "regexp"
+ "strings"
+ "unicode/utf8"
+
+ "github.com/Tencent/WeKnora/internal/types"
+ "github.com/Tencent/WeKnora/internal/types/interfaces"
+)
+
+const (
+ readDocumentDefaultLimit = 20
+ readDocumentMaxLimit = 100
+ readDocumentMaxContext = 5
+ readDocumentMaxMatches = 20
+ readDocumentScanPageSize = 100
+)
+
+var readDocumentTool = BaseTool{
+ name: ToolReadDocument,
+ description: "Read a document from the knowledge bases in scope: its metadata plus chunks in document order.\n" +
+ "id is a dN document handle (reads a page of chunks starting at offset) or a cN chunk handle (reads that " +
+ "chunk; set context to include neighbouring chunks on each side).\n" +
+ "query finds passages inside the document: matching chunks come back with one chunk of context on each " +
+ "side. query is a case-insensitive literal by default; set regex=true for a POSIX regular expression.\n" +
+ "Page through long documents with offset and limit: offset counts chunks in reading order from 0 and is " +
+ "not a chunk index, so continue with the returned next_offset, and use id=cN with context to read around " +
+ "a specific chunk. Use search_knowledge first when you do not yet know which document holds the answer.",
+ schema: json.RawMessage(`{
+ "type": "object",
+ "properties": {
+ "id": {
+ "type": "string",
+ "description": "dN document handle or cN chunk handle from retrieval results or the runtime context",
+ "minLength": 1
+ },
+ "offset": {
+ "type": "integer",
+ "description": "Position in reading order to start from (default 0); continue with next_offset",
+ "minimum": 0
+ },
+ "limit": {
+ "type": "integer",
+ "description": "Chunks per page (default 20, max 100)",
+ "minimum": 1,
+ "maximum": 100
+ },
+ "query": {
+ "type": "string",
+ "description": "Text to find inside the document; returns matching chunks with context instead of a page"
+ },
+ "regex": {
+ "type": "boolean",
+ "description": "Interpret query as a POSIX regular expression (default false: literal match)"
+ },
+ "context": {
+ "type": "integer",
+ "description": "Neighbouring chunks to include on each side of a cN chunk (default 0, max 5)",
+ "minimum": 0,
+ "maximum": 5
+ }
+ },
+ "required": ["id"]
+}`),
+}
+
+// ReadDocumentInput defines the input parameters for read_document.
+type ReadDocumentInput struct {
+ ID string `json:"id"`
+ Offset int `json:"offset,omitempty"`
+ Limit int `json:"limit,omitempty"`
+ Query string `json:"query,omitempty"`
+ Regex bool `json:"regex,omitempty"`
+ Context int `json:"context,omitempty"`
+}
+
+// ReadDocumentTool reads document metadata and chunks by page, by chunk
+// handle, or by an in-document text search.
+type ReadDocumentTool struct {
+ BaseTool
+ knowledgeService interfaces.KnowledgeService
+ chunkService interfaces.ChunkService
+ searchTargets types.SearchTargets
+}
+
+// NewReadDocumentTool creates a new read_document tool.
+func NewReadDocumentTool(
+ knowledgeService interfaces.KnowledgeService,
+ chunkService interfaces.ChunkService,
+ searchTargets types.SearchTargets,
+) *ReadDocumentTool {
+ return &ReadDocumentTool{
+ BaseTool: readDocumentTool,
+ knowledgeService: knowledgeService,
+ chunkService: chunkService,
+ searchTargets: searchTargets,
+ }
+}
+
+// readChunkRow is one chunk in the rendered result together with its role
+// (page entry, search match, or context around a match).
+type readChunkRow struct {
+ chunk *types.Chunk
+ role string // "", "match", "context_before", "context_after", "focus"
+ snippet string
+}
+
+// Execute resolves the id, then dispatches to page, chunk or query reading.
+func (t *ReadDocumentTool) Execute(ctx context.Context, args json.RawMessage) (*types.ToolResult, error) {
+ var input ReadDocumentInput
+ if err := json.Unmarshal(args, &input); err != nil {
+ return &types.ToolResult{Success: false, Error: fmt.Sprintf("Failed to parse args: %v", err)}, err
+ }
+ id := strings.TrimSpace(input.ID)
+ if id == "" {
+ return &types.ToolResult{Success: false, Error: "id is required (a dN document handle or a cN chunk handle)"},
+ fmt.Errorf("missing id")
+ }
+ if t.knowledgeService == nil || t.chunkService == nil {
+ return &types.ToolResult{Success: false, Error: "document services are unavailable"},
+ fmt.Errorf("services unavailable")
+ }
+
+ limit := input.Limit
+ if limit <= 0 {
+ limit = readDocumentDefaultLimit
+ }
+ if limit > readDocumentMaxLimit {
+ limit = readDocumentMaxLimit
+ }
+ offset := input.Offset
+ if offset < 0 {
+ offset = 0
+ }
+ contextChunks := input.Context
+ if contextChunks < 0 {
+ contextChunks = 0
+ }
+ if contextChunks > readDocumentMaxContext {
+ contextChunks = readDocumentMaxContext
+ }
+ query := strings.TrimSpace(input.Query)
+
+ knowledge, chunk, err := t.resolveTarget(ctx, id)
+ if err != nil {
+ return &types.ToolResult{Success: false, Error: err.Error()}, err
+ }
+
+ var matcher *regexp.Regexp
+ if query != "" {
+ pattern := query
+ if !input.Regex {
+ pattern = regexp.QuoteMeta(query)
+ }
+ compiled, cerr := regexp.Compile("(?i)" + pattern)
+ if cerr != nil {
+ return &types.ToolResult{
+ Success: false, Error: fmt.Sprintf("invalid regex query %q: %v", query, cerr),
+ }, cerr
+ }
+ matcher = compiled
+ }
+
+ // A query always searches the owning document, even when id named a
+ // chunk: silently returning the single chunk would read as "no match".
+ switch {
+ case matcher != nil:
+ return t.readByQuery(ctx, knowledge, query, matcher)
+ case chunk != nil:
+ return t.readAroundChunk(ctx, knowledge, chunk, contextChunks)
+ default:
+ return t.readPage(ctx, knowledge, offset, limit)
+ }
+}
+
+// resolveTarget accepts either a document id or a chunk id. Documents are
+// tried first because they are the common case; a chunk id falls through to
+// the chunk lookup. Scope violations are reported as such rather than being
+// masked as "not found".
+func (t *ReadDocumentTool) resolveTarget(ctx context.Context, id string) (*types.Knowledge, *types.Chunk, error) {
+ if knowledge, err := t.knowledgeService.GetKnowledgeByIDOnly(ctx, id); err == nil && knowledge != nil {
+ authorized, authErr := authorizeLoadedKnowledge(ctx, t.searchTargets, knowledge, t.knowledgeService)
+ if authErr != nil {
+ return nil, nil, fmt.Errorf("document is not accessible: %w", authErr)
+ }
+ return authorized, nil, nil
+ }
+ chunk, err := t.chunkService.GetChunkByIDOnly(ctx, id)
+ if err != nil || chunk == nil {
+ return nil, nil, fmt.Errorf("id %s is neither a document (dN) nor a chunk (cN) in the current scope", id)
+ }
+ authorizedChunk, authErr := authorizeChunkInSearchTargets(
+ ctx, t.searchTargets, id, t.chunkService, t.knowledgeService,
+ )
+ if authErr != nil {
+ return nil, nil, fmt.Errorf("chunk is not accessible: %w", authErr)
+ }
+ knowledge, err := t.knowledgeService.GetKnowledgeByIDOnly(ctx, authorizedChunk.KnowledgeID)
+ if err != nil || knowledge == nil {
+ // The chunk is authorized; a missing parent row only costs the header.
+ knowledge = &types.Knowledge{
+ ID: authorizedChunk.KnowledgeID,
+ TenantID: authorizedChunk.TenantID,
+ KnowledgeBaseID: authorizedChunk.KnowledgeBaseID,
+ }
+ }
+ return knowledge, authorizedChunk, nil
+}
+
+func (t *ReadDocumentTool) tenantFor(knowledge *types.Knowledge) uint64 {
+ if knowledge.TenantID != 0 {
+ return knowledge.TenantID
+ }
+ return t.searchTargets.GetTenantIDForKB(knowledge.KnowledgeBaseID)
+}
+
+// listChunks fetches one page of text/FAQ chunks in document order.
+func (t *ReadDocumentTool) listChunks(
+ ctx context.Context, knowledge *types.Knowledge, page, pageSize int,
+) ([]*types.Chunk, int64, error) {
+ enabled := true
+ chunks, total, err := t.chunkService.GetRepository().ListPagedChunksByKnowledgeID(
+ ctx, t.tenantFor(knowledge), knowledge.ID,
+ &types.Pagination{Page: page, PageSize: pageSize},
+ []types.ChunkType{types.ChunkTypeText, types.ChunkTypeFAQ},
+ nil, "", "", "", "", &enabled,
+ )
+ if err != nil {
+ return nil, 0, fmt.Errorf("failed to list chunks: %w", err)
+ }
+ return chunks, total, nil
+}
+
+// countChunks returns the number of readable chunks without loading them.
+func (t *ReadDocumentTool) countChunks(ctx context.Context, knowledge *types.Knowledge) int64 {
+ _, total, err := t.listChunks(ctx, knowledge, 1, 1)
+ if err != nil {
+ return 0
+ }
+ return total
+}
+
+// fetchWindow returns chunks [offset, offset+limit) in document order. The
+// repository pages by page number, so a window that does not start on a page
+// boundary is assembled from the two pages that cover it.
+func (t *ReadDocumentTool) fetchWindow(
+ ctx context.Context, knowledge *types.Knowledge, offset, limit int,
+) ([]*types.Chunk, int64, error) {
+ if limit <= 0 {
+ return nil, 0, nil
+ }
+ firstPage := offset/limit + 1
+ skip := offset % limit
+ chunks, total, err := t.listChunks(ctx, knowledge, firstPage, limit)
+ if err != nil {
+ return nil, 0, err
+ }
+ if skip > 0 {
+ if skip < len(chunks) {
+ chunks = chunks[skip:]
+ } else {
+ chunks = nil
+ }
+ // The next page exists whenever the first page did not already reach
+ // the end of the document, regardless of how far the window extends.
+ if int64(firstPage*limit) < total {
+ next, _, err := t.listChunks(ctx, knowledge, firstPage+1, limit)
+ if err != nil {
+ return nil, 0, err
+ }
+ chunks = append(chunks, next...)
+ }
+ }
+ if len(chunks) > limit {
+ chunks = chunks[:limit]
+ }
+ return chunks, total, nil
+}
+
+// readPage renders metadata plus chunks [offset, offset+limit).
+func (t *ReadDocumentTool) readPage(
+ ctx context.Context, knowledge *types.Knowledge, offset, limit int,
+) (*types.ToolResult, error) {
+ chunks, total, err := t.fetchWindow(ctx, knowledge, offset, limit)
+ if err != nil {
+ return &types.ToolResult{Success: false, Error: err.Error()}, err
+ }
+ if len(chunks) == 0 && total > 0 && int64(offset) >= total {
+ suggested := int(total) - limit
+ if suggested < 0 {
+ suggested = 0
+ }
+ return &types.ToolResult{
+ Success: false,
+ Error: fmt.Sprintf(
+ "offset %d is out of range: document has %d chunks (valid offsets 0..%d). Retry with offset=%d.",
+ offset, total, total-1, suggested),
+ Data: map[string]interface{}{
+ "knowledge_id": knowledge.ID,
+ "total_chunks": total,
+ "requested_offset": offset,
+ "requested_limit": limit,
+ "suggested_offset": suggested,
+ },
+ }, nil
+ }
+ enrichChunkImageInfo(ctx, t.chunkService.GetRepository(), t.tenantFor(knowledge), chunks)
+
+ rows := make([]readChunkRow, 0, len(chunks))
+ for _, c := range chunks {
+ rows = append(rows, readChunkRow{chunk: c})
+ }
+ data := t.buildData(knowledge, total, rows)
+ data["offset"] = offset
+ data["page_size"] = limit
+ data["page"] = offset/limit + 1
+ if next := offset + len(chunks); int64(next) < total && len(chunks) > 0 {
+ data["next_offset"] = next
+ }
+ return &types.ToolResult{
+ Success: true,
+ Output: t.buildOutput(knowledge, total, rows, ""),
+ Data: data,
+ }, nil
+}
+
+// readAroundChunk renders a single chunk with optional neighbours.
+func (t *ReadDocumentTool) readAroundChunk(
+ ctx context.Context, knowledge *types.Knowledge, focus *types.Chunk, contextChunks int,
+) (*types.ToolResult, error) {
+ total := t.countChunks(ctx, knowledge)
+ rows := []readChunkRow{{chunk: focus, role: "focus"}}
+ if contextChunks > 0 && focus.ChunkType != types.ChunkTypeFAQ {
+ // Neighbours are resolved by chunk_index, not by list position:
+ // parent, summary and image chunks share the index sequence, so a
+ // position computed from ChunkIndex would drift away from the focus.
+ neighbours, err := t.chunkService.GetRepository().ListChunkNeighbors(
+ ctx, t.tenantFor(knowledge), knowledge.ID, focus.ChunkIndex, contextChunks, contextChunks,
+ []types.ChunkType{types.ChunkTypeText, types.ChunkTypeFAQ},
+ )
+ if err != nil {
+ return &types.ToolResult{
+ Success: false, Error: fmt.Sprintf("failed to list neighbouring chunks: %v", err),
+ }, err
+ }
+ rows = rows[:0]
+ placed := false
+ for _, c := range neighbours {
+ if !placed && c.ChunkIndex > focus.ChunkIndex {
+ rows = append(rows, readChunkRow{chunk: focus, role: "focus"})
+ placed = true
+ }
+ role := "context_after"
+ if c.ChunkIndex < focus.ChunkIndex {
+ role = "context_before"
+ }
+ rows = append(rows, readChunkRow{chunk: c, role: role})
+ }
+ if !placed {
+ rows = append(rows, readChunkRow{chunk: focus, role: "focus"})
+ }
+ }
+ chunks := make([]*types.Chunk, 0, len(rows))
+ for _, r := range rows {
+ chunks = append(chunks, r.chunk)
+ }
+ enrichChunkImageInfo(ctx, t.chunkService.GetRepository(), t.tenantFor(knowledge), chunks)
+
+ data := t.buildData(knowledge, total, rows)
+ data["single_chunk"] = len(rows) == 1
+ data["focus_chunk_id"] = focus.ID
+ if q := faqStandardQuestion(focus); q != "" {
+ data["faq_id"] = focus.ID
+ data["faq_question"] = q
+ }
+ return &types.ToolResult{
+ Success: true,
+ Output: t.buildOutput(knowledge, total, rows, ""),
+ Data: data,
+ }, nil
+}
+
+// readByQuery scans the document in order and returns matching chunks with
+// one chunk of context on each side, capped at readDocumentMaxMatches.
+func (t *ReadDocumentTool) readByQuery(
+ ctx context.Context, knowledge *types.Knowledge, query string, matcher *regexp.Regexp,
+) (*types.ToolResult, error) {
+ var rows []readChunkRow
+ emitted := make(map[string]bool)
+ matchCount := 0
+ truncated := false
+ var total int64
+ var prev *types.Chunk
+ forceNext := false
+ page := 1
+ tenantID := t.tenantFor(knowledge)
+ compiled := []*regexp.Regexp{matcher}
+ // Stop collecting before the registry's head/tail truncation would cut
+ // rows in the middle; the header and tags take the remaining share.
+ budget := OutputBudget(ctx) * 4 / 5
+ used := 0
+ emit := func(row readChunkRow) {
+ rows = append(rows, row)
+ emitted[row.chunk.ID] = true
+ used += utf8.RuneCountInString(row.chunk.Content)
+ }
+
+scan:
+ for {
+ chunks, pageTotal, err := t.listChunks(ctx, knowledge, page, readDocumentScanPageSize)
+ if err != nil {
+ return &types.ToolResult{Success: false, Error: err.Error()}, err
+ }
+ if page == 1 {
+ total = pageTotal
+ }
+ if len(chunks) == 0 {
+ break
+ }
+ enrichChunkImageInfo(ctx, t.chunkService.GetRepository(), tenantID, chunks)
+ for _, c := range chunks {
+ haystack := enrichChunkContent(c)
+ if q := faqStandardQuestion(c); q != "" {
+ haystack += "\n" + q
+ }
+ if matcher.MatchString(haystack) {
+ if matchCount >= readDocumentMaxMatches || used >= budget {
+ truncated = true
+ break scan
+ }
+ matchCount++
+ if prev != nil && !emitted[prev.ID] {
+ emit(readChunkRow{chunk: prev, role: "context_before"})
+ }
+ if !emitted[c.ID] {
+ emit(readChunkRow{chunk: c, role: "match", snippet: extractChunkMatchSnippet(c, compiled)})
+ }
+ forceNext = true
+ } else if forceNext {
+ if !emitted[c.ID] && used < budget {
+ emit(readChunkRow{chunk: c, role: "context_after"})
+ }
+ forceNext = false
+ }
+ prev = c
+ }
+ if int64(page*readDocumentScanPageSize) >= total {
+ break
+ }
+ page++
+ }
+
+ data := t.buildData(knowledge, total, rows)
+ data["query"] = query
+ data["match_count"] = matchCount
+ data["truncated"] = truncated
+ return &types.ToolResult{
+ Success: true,
+ Output: t.buildOutput(knowledge, total, rows, query),
+ Data: data,
+ }, nil
+}
+
+// documentInfo is the metadata header shared by every read mode.
+func documentInfo(knowledge *types.Knowledge, total int64) map[string]interface{} {
+ info := map[string]interface{}{
+ "knowledge_id": knowledge.ID,
+ "title": knowledge.Title,
+ "type": knowledge.Type,
+ "source": formatSource(knowledge.Type, knowledge.Source),
+ "parse_status": knowledge.ParseStatus,
+ "chunk_count": total,
+ }
+ if knowledge.Description != "" {
+ info["description"] = knowledge.Description
+ }
+ if knowledge.FileName != "" {
+ info["file_name"] = knowledge.FileName
+ info["file_type"] = knowledge.FileType
+ info["file_size"] = formatFileSize(knowledge.FileSize)
+ }
+ if knowledge.Metadata != nil {
+ if metadata, err := knowledge.Metadata.Map(); err == nil && len(metadata) > 0 {
+ info["metadata"] = metadata
+ }
+ }
+ return info
+}
+
+func (t *ReadDocumentTool) buildData(
+ knowledge *types.Knowledge, total int64, rows []readChunkRow,
+) map[string]interface{} {
+ formatted := make([]map[string]interface{}, 0, len(rows))
+ for i, r := range rows {
+ row := chunkDataMap(i+1, r.chunk)
+ if row["knowledge_id"] == "" {
+ row["knowledge_id"] = knowledge.ID
+ }
+ if r.role != "" {
+ row["role"] = r.role
+ }
+ if r.snippet != "" {
+ row["match_snippet"] = r.snippet
+ }
+ formatted = append(formatted, row)
+ }
+ return map[string]interface{}{
+ "display_type": "knowledge_chunks_list",
+ "knowledge_id": knowledge.ID,
+ "knowledge_title": strings.TrimSpace(knowledge.Title),
+ "total_chunks": total,
+ "fetched_chunks": len(formatted),
+ "chunks": formatted,
+ "document": documentInfo(knowledge, total),
+ }
+}
+
+func (t *ReadDocumentTool) buildOutput(
+ knowledge *types.Knowledge, total int64, rows []readChunkRow, query string,
+) string {
+ var b strings.Builder
+ fmt.Fprintf(&b, "<document knowledge_id=\"%s\" title=\"%s\" total_chunks=\"%d\" fetched=\"%d\"",
+ xmlEscape(knowledge.ID), xmlEscape(strings.TrimSpace(knowledge.Title)), total, len(rows))
+ if knowledge.Type != "" {
+ fmt.Fprintf(&b, " source=\"%s\"", xmlEscape(formatSource(knowledge.Type, knowledge.Source)))
+ }
+ if knowledge.FileType != "" {
+ fmt.Fprintf(&b, " file_type=\"%s\"", xmlEscape(knowledge.FileType))
+ }
+ if knowledge.ParseStatus != "" {
+ fmt.Fprintf(&b, " parse_status=\"%s\"", xmlEscape(formatParseStatus(knowledge.ParseStatus)))
+ }
+ b.WriteString(">\n")
+ if knowledge.Description != "" {
+ fmt.Fprintf(&b, "<description>%s</description>\n", xmlEscape(knowledge.Description))
+ }
+ if knowledge.Metadata != nil {
+ if metadata, err := knowledge.Metadata.Map(); err == nil && len(metadata) > 0 {
+ b.WriteString("<metadata>")
+ first := true
+ for key, value := range metadata {
+ if !first {
+ b.WriteString("; ")
+ }
+ first = false
+ fmt.Fprintf(&b, "%s: %v", xmlEscape(key), value)
+ }
+ b.WriteString("</metadata>\n")
+ }
+ }
+ if query != "" {
+ fmt.Fprintf(&b, "<query>%s</query>\n", xmlEscape(query))
+ }
+ for _, r := range rows {
+ extra := ""
+ if r.role != "" {
+ extra = fmt.Sprintf(" role=\"%s\"", r.role)
+ }
+ if r.snippet != "" {
+ fmt.Fprintf(&b, "<match_snippet>%s</match_snippet>\n", xmlEscape(r.snippet))
+ }
+ writeChunkXML(&b, r.chunk, extra)
+ }
+ b.WriteString("</document>")
+ return b.String()
+}
+
+func formatSource(knowledgeType, source string) string {
+ switch knowledgeType {
+ case "file":
+ return "File Upload"
+ case "url":
+ return fmt.Sprintf("URL: %s", source)
+ case "passage":
+ return "Text Input"
+ default:
+ return knowledgeType
+ }
+}
+
+func formatFileSize(size int64) string {
+ if size == 0 {
+ return "Unknown"
+ }
+ const unit = 1024
+ if size < unit {
+ return fmt.Sprintf("%d B", size)
+ }
+ div, exp := int64(unit), 0
+ for n := size / unit; n >= unit; n /= unit {
+ div *= unit
+ exp++
+ }
+ return fmt.Sprintf("%.1f %cB", float64(size)/float64(div), "KMGTPE"[exp])
+}
+
+func formatParseStatus(status string) string {
+ switch status {
+ case "pending":
+ return "Pending"
+ case "processing":
+ return "Processing"
+ case "completed", "success":
+ return "Completed"
+ case "failed":
+ return "Failed"
+ default:
+ return status
+ }
+}
diff --git a/internal/agent/tools/read_document_test.go b/internal/agent/tools/read_document_test.go
new file mode 100644
index 000000000..d6ff85794
--- /dev/null
+++ b/internal/agent/tools/read_document_test.go
@@ -0,0 +1,486 @@
+package tools
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "strings"
+ "testing"
+
+ "github.com/Tencent/WeKnora/internal/types"
+ "github.com/Tencent/WeKnora/internal/types/interfaces"
+)
+
+// ---- fakes -----------------------------------------------------------------
+
+type readDocKnowledgeService struct {
+ interfaces.KnowledgeService
+ docs map[string]*types.Knowledge
+ pages map[string]*types.PageResult // keyed by kb id
+ tags map[string][]*types.KnowledgeTag
+ listFilters []types.KnowledgeListFilter
+}
+
+func (f *readDocKnowledgeService) GetKnowledgeBatch(
+ _ context.Context, _ uint64, ids []string,
+) ([]*types.Knowledge, error) {
+ out := make([]*types.Knowledge, 0, len(ids))
+ for _, id := range ids {
+ if k, ok := f.docs[id]; ok {
+ out = append(out, k)
+ }
+ }
+ return out, nil
+}
+
+func (f *readDocKnowledgeService) GetKnowledgeByIDOnly(_ context.Context, id string) (*types.Knowledge, error) {
+ if k, ok := f.docs[id]; ok {
+ return k, nil
+ }
+ return nil, errors.New("record not found")
+}
+
+func (f *readDocKnowledgeService) GetKnowledgeTags(
+ _ context.Context, ids []string,
+) (map[string][]*types.KnowledgeTag, error) {
+ out := make(map[string][]*types.KnowledgeTag, len(ids))
+ for _, id := range ids {
+ if tags, ok := f.tags[id]; ok {
+ out[id] = tags
+ }
+ }
+ return out, nil
+}
+
+func (f *readDocKnowledgeService) ListPagedKnowledgeByKnowledgeBaseID(
+ _ context.Context, kbID string, page *types.Pagination, filter types.KnowledgeListFilter,
+) (*types.PageResult, error) {
+ f.listFilters = append(f.listFilters, filter)
+ result, ok := f.pages[kbID]
+ if !ok {
+ return &types.PageResult{Page: page.Page, PageSize: page.PageSize, Data: []*types.Knowledge{}}, nil
+ }
+ rows, _ := result.Data.([]*types.Knowledge)
+ if len(filter.TagIDs) > 0 {
+ wanted := make(map[string]bool, len(filter.TagIDs))
+ for _, id := range filter.TagIDs {
+ wanted[id] = true
+ }
+ tagged := make([]*types.Knowledge, 0, len(rows))
+ for _, k := range rows {
+ for _, tag := range f.tags[k.ID] {
+ if wanted[tag.ID] {
+ tagged = append(tagged, k)
+ break
+ }
+ }
+ }
+ rows = tagged
+ }
+ if filter.Keyword != "" {
+ filtered := make([]*types.Knowledge, 0, len(rows))
+ for _, k := range rows {
+ if strings.Contains(strings.ToLower(k.Title), strings.ToLower(filter.Keyword)) {
+ filtered = append(filtered, k)
+ }
+ }
+ rows = filtered
+ }
+ total := int64(len(rows))
+ start := (page.Page - 1) * page.PageSize
+ if start > len(rows) {
+ start = len(rows)
+ }
+ end := start + page.PageSize
+ if end > len(rows) {
+ end = len(rows)
+ }
+ return &types.PageResult{Total: total, Page: page.Page, PageSize: page.PageSize, Data: rows[start:end]}, nil
+}
+
+type readDocChunkRepo struct {
+ interfaces.ChunkRepository
+ ordered []*types.Chunk // document order
+ listErr error
+ requests []types.Pagination
+}
+
+func (r *readDocChunkRepo) ListPagedChunksByKnowledgeID(
+ _ context.Context, _ uint64, _ string, page *types.Pagination, _ []types.ChunkType,
+ _ []string, _ string, _ string, _ string, _ string, _ *bool,
+) ([]*types.Chunk, int64, error) {
+ if r.listErr != nil {
+ return nil, 0, r.listErr
+ }
+ r.requests = append(r.requests, *page)
+ start := (page.Page - 1) * page.PageSize
+ end := start + page.PageSize
+ total := int64(len(r.ordered))
+ if start >= len(r.ordered) {
+ return nil, total, nil
+ }
+ if end > len(r.ordered) {
+ end = len(r.ordered)
+ }
+ out := make([]*types.Chunk, end-start)
+ copy(out, r.ordered[start:end])
+ return out, total, nil
+}
+
+func (r *readDocChunkRepo) ListChunksByParentIDs(context.Context, uint64, []string) ([]*types.Chunk, error) {
+ return nil, nil
+}
+
+func (r *readDocChunkRepo) ListChunkNeighbors(
+ _ context.Context, _ uint64, _ string, chunkIndex, before, after int, _ []types.ChunkType,
+) ([]*types.Chunk, error) {
+ var preceding, following []*types.Chunk
+ for _, c := range r.ordered {
+ switch {
+ case c.ChunkIndex < chunkIndex:
+ preceding = append(preceding, c)
+ case c.ChunkIndex > chunkIndex && len(following) < after:
+ following = append(following, c)
+ }
+ }
+ if len(preceding) > before {
+ preceding = preceding[len(preceding)-before:]
+ }
+ return append(preceding, following...), nil
+}
+
+type readDocChunkService struct {
+ interfaces.ChunkService
+ repo *readDocChunkRepo
+}
+
+func (s *readDocChunkService) GetChunkByIDOnly(_ context.Context, id string) (*types.Chunk, error) {
+ for _, c := range s.repo.ordered {
+ if c.ID == id {
+ return c, nil
+ }
+ }
+ return nil, errors.New("chunk not found")
+}
+
+func (s *readDocChunkService) GetRepository() interfaces.ChunkRepository { return s.repo }
+
+func testDocumentChunks(knowledgeID string, n int) []*types.Chunk {
+ chunks := make([]*types.Chunk, 0, n)
+ for i := 0; i < n; i++ {
+ chunks = append(chunks, &types.Chunk{
+ ID: fmt.Sprintf("chunk-%d", i), TenantID: 7, KnowledgeID: knowledgeID, KnowledgeBaseID: "kb-1",
+ ChunkIndex: i, ChunkType: types.ChunkTypeText, IsEnabled: true,
+ Content: fmt.Sprintf("Section %d body. %s", i, engineSentence(i == 4 || i == 9)),
+ })
+ }
+ return chunks
+}
+
+func engineSentence(present bool) string {
+ if present {
+ return "The psionic engine is here."
+ }
+ return ""
+}
+
+func newReadDocumentFixture(chunkCount int) (*ReadDocumentTool, *readDocChunkRepo) {
+ doc := &types.Knowledge{
+ ID: "doc-1", TenantID: 7, KnowledgeBaseID: "kb-1", Title: "Engine Manual", Type: "file",
+ FileName: "manual.pdf", FileType: "pdf", FileSize: 2048, ParseStatus: "completed",
+ Description: "How the engine works",
+ }
+ repo := &readDocChunkRepo{ordered: testDocumentChunks(doc.ID, chunkCount)}
+ tool := NewReadDocumentTool(
+ &readDocKnowledgeService{docs: map[string]*types.Knowledge{doc.ID: doc}},
+ &readDocChunkService{repo: repo},
+ types.SearchTargets{{Type: types.SearchTargetTypeKnowledgeBase, KnowledgeBaseID: "kb-1", TenantID: 7}},
+ )
+ return tool, repo
+}
+
+func chunkIDsFromData(t *testing.T, data map[string]interface{}) []string {
+ t.Helper()
+ rows, ok := data["chunks"].([]map[string]interface{})
+ if !ok {
+ t.Fatalf("chunks payload = %#v", data["chunks"])
+ }
+ ids := make([]string, 0, len(rows))
+ for _, r := range rows {
+ ids = append(ids, fmt.Sprint(r["chunk_id"]))
+ }
+ return ids
+}
+
+// ---- tests -----------------------------------------------------------------
+
+func TestReadDocumentRequiresID(t *testing.T) {
+ tool, _ := newReadDocumentFixture(3)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{}`))
+ if err == nil || res.Success || !strings.Contains(res.Error, "id is required") {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+}
+
+func TestReadDocumentPagesFromExactOffset(t *testing.T) {
+ tool, repo := newReadDocumentFixture(12)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-1","offset":5,"limit":4}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if got := chunkIDsFromData(t, res.Data); strings.Join(got, ",") != "chunk-5,chunk-6,chunk-7,chunk-8" {
+ t.Fatalf("window = %v", got)
+ }
+ if res.Data["next_offset"] != 9 || res.Data["total_chunks"] != int64(12) || res.Data["fetched_chunks"] != 4 {
+ t.Fatalf("pagination data = %+v", res.Data)
+ }
+ if len(repo.requests) != 2 {
+ t.Fatalf("an unaligned window needs exactly two page fetches, got %v", repo.requests)
+ }
+ info, _ := res.Data["document"].(map[string]interface{})
+ if info["title"] != "Engine Manual" || info["file_type"] != "pdf" || info["chunk_count"] != int64(12) {
+ t.Fatalf("document header = %+v", info)
+ }
+ if !strings.Contains(res.Output, `title="Engine Manual"`) ||
+ !strings.Contains(res.Output, "<description>How the engine works</description>") {
+ t.Fatalf("output header missing: %s", res.Output)
+ }
+}
+
+func TestReadDocumentUnalignedWindowReachesDocumentEnd(t *testing.T) {
+ tool, repo := newReadDocumentFixture(9)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-1","offset":6,"limit":4}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if got := chunkIDsFromData(t, res.Data); strings.Join(got, ",") != "chunk-6,chunk-7,chunk-8" {
+ t.Fatalf("window past the document end = %v, want every remaining chunk", got)
+ }
+ if _, ok := res.Data["next_offset"]; ok {
+ t.Fatalf("nothing remains after chunk-8: %+v", res.Data)
+ }
+ if len(repo.requests) != 2 {
+ t.Fatalf("expected the second page to be fetched, requests=%v", repo.requests)
+ }
+}
+
+func TestReadDocumentChunkContextNearDocumentEnd(t *testing.T) {
+ tool, _ := newReadDocumentFixture(9)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"chunk-7","context":2}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if got := chunkIDsFromData(t, res.Data); strings.Join(got, ",") != "chunk-5,chunk-6,chunk-7,chunk-8" {
+ t.Fatalf("context near the end = %v", got)
+ }
+ if res.Data["total_chunks"] != int64(9) {
+ t.Fatalf("total = %+v", res.Data)
+ }
+}
+
+// Parent, summary and image chunks share the chunk_index sequence but are not
+// readable text, so readable chunks have gaps in their indexes. Neighbours
+// must follow the index order, not list positions derived from it.
+func TestReadDocumentChunkContextFollowsIndexesAcrossGaps(t *testing.T) {
+ tool, repo := newReadDocumentFixture(6)
+ for i, c := range repo.ordered {
+ c.ChunkIndex = i * 3 // 0, 3, 6, 9, 12, 15
+ }
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"chunk-3","context":1}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if got := chunkIDsFromData(t, res.Data); strings.Join(got, ",") != "chunk-2,chunk-3,chunk-4" {
+ t.Fatalf("neighbours across index gaps = %v", got)
+ }
+ rows := res.Data["chunks"].([]map[string]interface{})
+ if rows[0]["role"] != "context_before" || rows[1]["role"] != "focus" || rows[2]["role"] != "context_after" {
+ t.Fatalf("roles = %v %v %v", rows[0]["role"], rows[1]["role"], rows[2]["role"])
+ }
+}
+
+func TestReadDocumentQueryWithChunkHandleSearchesOwningDocument(t *testing.T) {
+ tool, _ := newReadDocumentFixture(12)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"chunk-1","query":"psionic"}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if res.Data["match_count"] != 2 || res.Data["query"] != "psionic" {
+ t.Fatalf("query must run against the owning document, data=%+v", res.Data)
+ }
+}
+
+func TestReadDocumentQueryStopsAtOutputBudget(t *testing.T) {
+ tool, repo := newReadDocumentFixture(12)
+ for _, c := range repo.ordered {
+ c.Content = "needle " + strings.Repeat("x", 400)
+ }
+ ctx := WithOutputBudget(context.Background(), 1500)
+ res, err := tool.Execute(ctx, json.RawMessage(`{"id":"doc-1","query":"needle"}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if res.Data["truncated"] != true {
+ t.Fatalf("budget exhaustion must be reported: %+v", res.Data)
+ }
+ if fetched := res.Data["fetched_chunks"].(int); fetched == 0 || fetched >= 12 {
+ t.Fatalf("expected a partial, budget-bound result, fetched=%d", fetched)
+ }
+}
+
+func TestReadDocumentLastPageHasNoNextOffset(t *testing.T) {
+ tool, _ := newReadDocumentFixture(5)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-1","offset":0,"limit":20}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if _, ok := res.Data["next_offset"]; ok {
+ t.Fatalf("complete read must not advertise a next page: %+v", res.Data)
+ }
+ if got := chunkIDsFromData(t, res.Data); len(got) != 5 {
+ t.Fatalf("expected all 5 chunks, got %v", got)
+ }
+}
+
+func TestReadDocumentOffsetOutOfRangeSuggestsValidOffset(t *testing.T) {
+ tool, _ := newReadDocumentFixture(5)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-1","offset":40,"limit":20}`))
+ if err != nil {
+ t.Fatalf("out-of-range is a tool-level error, not a Go error: %v", err)
+ }
+ if res.Success || !strings.Contains(res.Error, "offset 40 is out of range") ||
+ !strings.Contains(res.Error, "offset=0") {
+ t.Fatalf("res=%+v", res)
+ }
+}
+
+func TestReadDocumentByChunkHandleWithContext(t *testing.T) {
+ tool, _ := newReadDocumentFixture(10)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"chunk-4","context":1}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if got := chunkIDsFromData(t, res.Data); strings.Join(got, ",") != "chunk-3,chunk-4,chunk-5" {
+ t.Fatalf("context window = %v", got)
+ }
+ rows := res.Data["chunks"].([]map[string]interface{})
+ if rows[0]["role"] != "context_before" || rows[1]["role"] != "focus" || rows[2]["role"] != "context_after" {
+ t.Fatalf("roles = %v %v %v", rows[0]["role"], rows[1]["role"], rows[2]["role"])
+ }
+ if res.Data["focus_chunk_id"] != "chunk-4" || res.Data["single_chunk"] != false {
+ t.Fatalf("data = %+v", res.Data)
+ }
+}
+
+func TestReadDocumentByChunkHandleAloneIsSingleChunk(t *testing.T) {
+ tool, _ := newReadDocumentFixture(3)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"chunk-2"}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if got := chunkIDsFromData(t, res.Data); strings.Join(got, ",") != "chunk-2" {
+ t.Fatalf("chunks = %v", got)
+ }
+ if res.Data["single_chunk"] != true || res.Data["knowledge_title"] != "Engine Manual" {
+ t.Fatalf("data = %+v", res.Data)
+ }
+}
+
+func TestReadDocumentQueryReturnsMatchesWithContext(t *testing.T) {
+ tool, _ := newReadDocumentFixture(12)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-1","query":"PSIONIC engine"}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ want := "chunk-3,chunk-4,chunk-5,chunk-8,chunk-9,chunk-10"
+ if got := chunkIDsFromData(t, res.Data); strings.Join(got, ",") != want {
+ t.Fatalf("matches with context = %v", got)
+ }
+ if res.Data["match_count"] != 2 || res.Data["truncated"] != false || res.Data["query"] != "PSIONIC engine" {
+ t.Fatalf("data = %+v", res.Data)
+ }
+ rows := res.Data["chunks"].([]map[string]interface{})
+ if rows[1]["role"] != "match" || rows[1]["match_snippet"] == nil {
+ t.Fatalf("match row = %+v", rows[1])
+ }
+ if !strings.Contains(res.Output, "<query>PSIONIC engine</query>") {
+ t.Fatalf("output = %s", res.Output)
+ }
+}
+
+func TestReadDocumentQueryIsLiteralUnlessRegex(t *testing.T) {
+ tool, repo := newReadDocumentFixture(3)
+ repo.ordered[1].Content = "Compile with C++ and check v1.2"
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-1","query":"C++"}`))
+ if err != nil || !res.Success || res.Data["match_count"] != 1 {
+ t.Fatalf("literal C++ should match once: res=%+v err=%v", res, err)
+ }
+ res, err = tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-1","query":"C++","regex":true}`))
+ if err == nil || res.Success || !strings.Contains(res.Error, "invalid regex") {
+ t.Fatalf("C++ is not a valid regex: res=%+v err=%v", res, err)
+ }
+ alternation := json.RawMessage(`{"id":"doc-1","query":"v1\\.2|nothing","regex":true}`)
+ res, err = tool.Execute(context.Background(), alternation)
+ if err != nil || !res.Success || res.Data["match_count"] != 1 {
+ t.Fatalf("regex alternation: res=%+v err=%v", res, err)
+ }
+}
+
+func TestReadDocumentQueryWithoutMatchesKeepsHeader(t *testing.T) {
+ tool, _ := newReadDocumentFixture(3)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-1","query":"absent"}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if res.Data["match_count"] != 0 || res.Data["fetched_chunks"] != 0 {
+ t.Fatalf("data = %+v", res.Data)
+ }
+ if info, _ := res.Data["document"].(map[string]interface{}); info["title"] != "Engine Manual" {
+ t.Fatalf("header must survive an empty match set: %+v", res.Data)
+ }
+}
+
+func TestReadDocumentRejectsUnknownIDAndOutOfScopeDocument(t *testing.T) {
+ tool, _ := newReadDocumentFixture(3)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"nope"}`))
+ if err == nil || res.Success || !strings.Contains(res.Error, "neither a document") {
+ t.Fatalf("unknown id: res=%+v err=%v", res, err)
+ }
+
+ foreign := &types.Knowledge{ID: "doc-2", TenantID: 9, KnowledgeBaseID: "kb-9", Title: "Other"}
+ tool.knowledgeService = &readDocKnowledgeService{docs: map[string]*types.Knowledge{"doc-2": foreign}}
+ res, err = tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-2"}`))
+ if err == nil || res.Success || !strings.Contains(res.Error, "not accessible") {
+ t.Fatalf("out-of-scope document: res=%+v err=%v", res, err)
+ }
+}
+
+func TestReadDocumentSurfacesRepositoryFailure(t *testing.T) {
+ tool, repo := newReadDocumentFixture(3)
+ repo.listErr = errors.New("db down")
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"doc-1"}`))
+ if err == nil || res.Success || !strings.Contains(res.Error, "db down") {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+}
+
+func TestReadDocumentFAQChunkExposesQuestion(t *testing.T) {
+ tool, repo := newReadDocumentFixture(2)
+ faq := repo.ordered[0]
+ faq.ChunkType = types.ChunkTypeFAQ
+ meta := &types.FAQChunkMetadata{StandardQuestion: "How to reset?", Answers: []string{"Hold the button."}}
+ if err := faq.SetFAQMetadata(meta); err != nil {
+ t.Fatal(err)
+ }
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"id":"chunk-0"}`))
+ if err != nil || !res.Success {
+ t.Fatalf("res=%+v err=%v", res, err)
+ }
+ if res.Data["faq_question"] != "How to reset?" || res.Data["faq_id"] != "chunk-0" {
+ t.Fatalf("data = %+v", res.Data)
+ }
+ if !strings.Contains(res.Output, "<answer>Hold the button.</answer>") {
+ t.Fatalf("output = %s", res.Output)
+ }
+}
diff --git a/internal/agent/tools/schema_validity_test.go b/internal/agent/tools/schema_validity_test.go
new file mode 100644
index 000000000..666b44f9d
--- /dev/null
+++ b/internal/agent/tools/schema_validity_test.go
@@ -0,0 +1,50 @@
+package tools
+
+import (
+ "encoding/json"
+ "testing"
+)
+
+// Static tool schemas are hand-written JSON spliced with Go string
+// constants; a stray quote would only surface as a provider-side 400.
+func TestBuiltInToolSchemasAreValidJSON(t *testing.T) {
+ static := map[string]json.RawMessage{
+ ToolSearchKnowledge: searchKnowledgeTool.schema,
+ ToolReadDocument: readDocumentTool.schema,
+ ToolListDocuments: listDocumentsTool.schema,
+ ToolQueryKnowledgeGraph: queryKnowledgeGraphTool.schema,
+ ToolThinking: sequentialThinkingTool.schema,
+ }
+ wikiSearch := NewWikiSearchTool(nil, nil, NewWikiScopesFromKBIDs([]string{"kb"}), nil)
+ wikiRead := NewWikiReadPageTool(nil, nil, NewWikiScopesFromKBIDs([]string{"kb"}), nil)
+ static[wikiSearch.Name()] = wikiSearch.Parameters()
+ static[wikiRead.Name()] = wikiRead.Parameters()
+
+ for name, schema := range static {
+ var parsed map[string]interface{}
+ if err := json.Unmarshal(schema, &parsed); err != nil {
+ t.Errorf("%s schema is not valid JSON: %v", name, err)
+ continue
+ }
+ if parsed["type"] != "object" {
+ t.Errorf("%s schema must describe an object, got %v", name, parsed["type"])
+ }
+ props, _ := parsed["properties"].(map[string]interface{})
+ if len(props) == 0 {
+ t.Errorf("%s schema declares no properties", name)
+ }
+ }
+ if desc, _ := func() (string, error) {
+ var parsed struct {
+ Properties struct {
+ Thought struct {
+ Description string `json:"description"`
+ } `json:"thought"`
+ } `json:"properties"`
+ }
+ err := json.Unmarshal(sequentialThinkingTool.schema, &parsed)
+ return parsed.Properties.Thought.Description, err
+ }(); desc == "" || !json.Valid(sequentialThinkingTool.schema) {
+ t.Fatalf("thinking schema lost its thought description: %q", desc)
+ }
+}
diff --git a/internal/agent/tools/scope_authorization.go b/internal/agent/tools/scope_authorization.go
index e8c3162d6..9b4124b81 100644
--- a/internal/agent/tools/scope_authorization.go
+++ b/internal/agent/tools/scope_authorization.go
@@ -76,6 +76,20 @@ func authorizeKnowledgeInSearchTargets(
}
return nil, fmt.Errorf("document %s not found: %w", knowledgeID, err)
}
+ return authorizeLoadedKnowledge(ctx, searchTargets, knowledge, knowledgeService)
+}
+
+// authorizeLoadedKnowledge is the scope check behind
+// authorizeKnowledgeInSearchTargets for callers that already hold the row.
+func authorizeLoadedKnowledge(
+ ctx context.Context,
+ searchTargets types.SearchTargets,
+ knowledge *types.Knowledge,
+ knowledgeService interfaces.KnowledgeService,
+) (*types.Knowledge, error) {
+ if knowledge == nil {
+ return nil, fmt.Errorf("knowledge_id is required")
+ }
if !searchTargets.ContainsKB(knowledge.KnowledgeBaseID) {
return nil, fmt.Errorf("knowledge base %s is not within the current Agent scope", knowledge.KnowledgeBaseID)
}
diff --git a/internal/agent/tools/search_conversations.go b/internal/agent/tools/search_conversations.go
index a6d36f56f..a3e57cded 100644
--- a/internal/agent/tools/search_conversations.go
+++ b/internal/agent/tools/search_conversations.go
@@ -31,7 +31,7 @@ not in the current conversation:
- The user assumes shared context that this session does not contain
Do not use when:
-- The answer is in documents (use knowledge_search — that is the knowledge base)
+- The answer is in documents (use search_knowledge — that is the knowledge base)
- The information is in the current conversation already
- The user is asking a general question with no reference to the past
diff --git a/internal/agent/tools/search_knowledge.go b/internal/agent/tools/search_knowledge.go
new file mode 100644
index 000000000..a0d0354b1
--- /dev/null
+++ b/internal/agent/tools/search_knowledge.go
@@ -0,0 +1,1060 @@
+package tools
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "math"
+ "regexp"
+ "sort"
+ "strings"
+ "sync"
+
+ "github.com/Tencent/WeKnora/internal/config"
+ "github.com/Tencent/WeKnora/internal/logger"
+ "github.com/Tencent/WeKnora/internal/models/rerank"
+ "github.com/Tencent/WeKnora/internal/searchutil"
+ "github.com/Tencent/WeKnora/internal/types"
+ "github.com/Tencent/WeKnora/internal/types/interfaces"
+)
+
+// Retrieval modes accepted by search_knowledge.
+const (
+ SearchModeHybrid = "hybrid"
+ SearchModeSemantic = "semantic"
+ SearchModeKeyword = "keyword"
+)
+
+const (
+ searchKnowledgeDefaultLimit = 10
+ searchKnowledgeMaxLimit = 30
+)
+
+var searchKnowledgeTool = BaseTool{
+ name: ToolSearchKnowledge,
+ description: "Search the knowledge bases in scope and return the most relevant chunks with their content.\n" +
+ "mode \"hybrid\" (default) combines semantic and keyword retrieval. \"semantic\" ranks by meaning and " +
+ "tolerates paraphrase. \"keyword\" matches the literal terms and is the right choice for identifiers, " +
+ "error codes, product names and exact phrases. A base without the requested index is searched with the " +
+ "index it has; the result reports that as a mode fallback.\n" +
+ "Write query as one natural-language question or a short phrase; in keyword mode write the exact terms. " +
+ "Pass knowledge_base_ids to focus on the bases whose profile fits the question. Run the tool again with a " +
+ "different query or mode when the results are thin.\n" +
+ "Every chunk carries a cN handle and belongs to a dN document. Use read_document(id=dN) to read the " +
+ "surrounding context or the whole document. Searches match chunk text only; to find a document by its " +
+ "title or file name use list_documents(keyword=...).",
+ schema: json.RawMessage(`{
+ "type": "object",
+ "properties": {
+ "query": {
+ "type": "string",
+ "description": "A natural-language question or short phrase; exact terms when mode is keyword",
+ "minLength": 1
+ },
+ "mode": {
+ "type": "string",
+ "enum": ["hybrid", "semantic", "keyword"],
+ "description": "Retrieval mode (default hybrid)"
+ },
+ "knowledge_base_ids": {
+ "type": "array",
+ "description": "Optional bN knowledge-base handles from the runtime context to restrict the search",
+ "items": { "type": "string" },
+ "maxItems": 10
+ },
+ "limit": {
+ "type": "integer",
+ "description": "Maximum chunks to return (default 10, max 30)",
+ "minimum": 1,
+ "maximum": 30
+ }
+ },
+ "required": ["query"]
+}`),
+}
+
+// SearchKnowledgeInput defines the input parameters for search_knowledge.
+type SearchKnowledgeInput struct {
+ Query string `json:"query"`
+ Mode string `json:"mode,omitempty"`
+ KnowledgeBaseIDs []string `json:"knowledge_base_ids,omitempty"`
+ Limit int `json:"limit,omitempty"`
+}
+
+// searchResultWithMeta wraps search result with metadata about which query matched it
+type searchResultWithMeta struct {
+ *types.SearchResult
+ SourceQuery string
+ QueryType string // retrieval path that produced the row
+ KnowledgeBaseID string // ID of the knowledge base this result came from
+ KnowledgeBaseType string // Type of the knowledge base (document, faq, etc.)
+}
+
+// SearchKnowledgeTool searches chunk-indexed knowledge bases in semantic,
+// keyword or hybrid mode and returns reranked, de-duplicated chunks.
+type SearchKnowledgeTool struct {
+ BaseTool
+ knowledgeBaseService interfaces.KnowledgeBaseService
+ knowledgeService interfaces.KnowledgeService
+ chunkService interfaces.ChunkService
+ searchTargets types.SearchTargets // Pre-computed unified search targets
+ rerankModel rerank.Reranker
+ config *config.Config // Global config for fallback values
+ // patternFilter, when set, keeps only results whose text matches it.
+ // It is not exposed to the model; the MCP grep_chunks endpoint uses it
+ // to keep grep semantics on top of index-backed retrieval.
+ patternFilter *regexp.Regexp
+}
+
+// WithPatternFilter restricts results to chunks whose text matches re. The
+// candidate pool is widened to the maximum limit before filtering.
+func (t *SearchKnowledgeTool) WithPatternFilter(re *regexp.Regexp) *SearchKnowledgeTool {
+ t.patternFilter = re
+ return t
+}
+
+// NewSearchKnowledgeTool creates a new search_knowledge tool.
+func NewSearchKnowledgeTool(
+ knowledgeBaseService interfaces.KnowledgeBaseService,
+ knowledgeService interfaces.KnowledgeService,
+ chunkService interfaces.ChunkService,
+ searchTargets types.SearchTargets,
+ rerankModel rerank.Reranker,
+ cfg *config.Config,
+) *SearchKnowledgeTool {
+ return &SearchKnowledgeTool{
+ BaseTool: searchKnowledgeTool,
+ knowledgeBaseService: knowledgeBaseService,
+ knowledgeService: knowledgeService,
+ chunkService: chunkService,
+ searchTargets: searchTargets,
+ rerankModel: rerankModel,
+ config: cfg,
+ }
+}
+
+// normalizeSearchMode validates the mode argument and applies the default.
+func normalizeSearchMode(mode string) (string, error) {
+ mode = strings.ToLower(strings.TrimSpace(mode))
+ switch mode {
+ case "":
+ return SearchModeHybrid, nil
+ case SearchModeHybrid, SearchModeSemantic, SearchModeKeyword:
+ return mode, nil
+ default:
+ return "", fmt.Errorf("mode must be one of hybrid, semantic, keyword (got %q)", mode)
+ }
+}
+
+// clampSearchLimit applies the default and upper bound for limit.
+func clampSearchLimit(limit int) int {
+ if limit <= 0 {
+ return searchKnowledgeDefaultLimit
+ }
+ if limit > searchKnowledgeMaxLimit {
+ return searchKnowledgeMaxLimit
+ }
+ return limit
+}
+
+// Execute executes the search.
+func (t *SearchKnowledgeTool) Execute(ctx context.Context, args json.RawMessage) (*types.ToolResult, error) {
+ var input SearchKnowledgeInput
+ if err := json.Unmarshal(args, &input); err != nil {
+ return &types.ToolResult{Success: false, Error: fmt.Sprintf("Failed to parse args: %v", err)}, err
+ }
+
+ query := strings.TrimSpace(input.Query)
+ if query == "" {
+ return &types.ToolResult{Success: false, Error: "query is required"}, fmt.Errorf("no query provided")
+ }
+ mode, err := normalizeSearchMode(input.Mode)
+ if err != nil {
+ return &types.ToolResult{Success: false, Error: err.Error()}, err
+ }
+ limit := clampSearchLimit(input.Limit)
+
+ // Optional KB filter: reject hallucinated or out-of-scope handles.
+ searchTargets := t.searchTargets
+ if len(input.KnowledgeBaseIDs) > 0 {
+ if err := validateKnowledgeBaseIDsInSearchTargets(t.searchTargets, input.KnowledgeBaseIDs); err != nil {
+ return &types.ToolResult{Success: false, Error: err.Error()}, err
+ }
+ wanted := make(map[string]bool, len(input.KnowledgeBaseIDs))
+ for _, kbID := range input.KnowledgeBaseIDs {
+ wanted[kbID] = true
+ }
+ var filtered types.SearchTargets
+ for _, target := range t.searchTargets {
+ if target != nil && wanted[target.KnowledgeBaseID] {
+ filtered = append(filtered, target)
+ }
+ }
+ searchTargets = filtered
+ }
+ if len(searchTargets) == 0 {
+ return &types.ToolResult{
+ Success: false,
+ Error: "no knowledge base is in scope for this search",
+ }, fmt.Errorf("no search targets available")
+ }
+
+ kbIDs := searchTargets.GetAllKnowledgeBaseIDs()
+ kbList, err := t.knowledgeBaseService.GetKnowledgeBasesByIDsOnly(ctx, kbIDs)
+ if err != nil {
+ logger.Warnf(ctx, "[Tool][SearchKnowledge] Failed to load knowledge bases %v: %v", kbIDs, err)
+ }
+ kbModes, msg := resolveKBSearchModes(mode, kbList)
+ if msg != "" {
+ return &types.ToolResult{Success: false, Error: msg}, fmt.Errorf("%s", msg)
+ }
+ kbTypeMap := make(map[string]string, len(kbList))
+ for _, kb := range kbList {
+ if kb != nil {
+ kbTypeMap[kb.ID] = kb.Type
+ }
+ }
+
+ retrievalLimit := limit
+ if t.patternFilter != nil {
+ retrievalLimit = searchKnowledgeMaxLimit
+ }
+ topK, vectorThreshold, keywordThreshold := t.retrievalParams(retrievalLimit)
+ logger.Infof(ctx,
+ "[Tool][SearchKnowledge] query=%q mode=%s limit=%d top_k=%d targets=%d kbs=%d",
+ query, mode, limit, topK, len(searchTargets), len(kbIDs))
+
+ allResults := t.concurrentSearchByTargets(ctx, query, mode, kbModes, searchTargets, kbList,
+ topK, vectorThreshold, keywordThreshold, kbTypeMap)
+
+ deduplicated := t.deduplicateResults(allResults)
+
+ ranked := deduplicated
+ if t.rerankModel != nil && len(deduplicated) > 0 {
+ reranked, err := t.rerankResults(ctx, query, deduplicated)
+ if err != nil {
+ logger.Warnf(ctx, "[Tool][SearchKnowledge] Rerank failed, using retrieval order: %v", err)
+ } else {
+ ranked = reranked
+ }
+ }
+
+ if len(ranked) > 0 {
+ mmrK := len(ranked)
+ if mmrK > retrievalLimit {
+ mmrK = retrievalLimit
+ }
+ if selected := t.applyMMR(ctx, ranked, mmrK, 0.7); len(selected) > 0 {
+ ranked = selected
+ }
+ }
+
+ final := t.deduplicateResults(ranked)
+ sort.Slice(final, func(i, j int) bool {
+ if final[i].Score != final[j].Score {
+ return final[i].Score > final[j].Score
+ }
+ return final[i].KnowledgeID < final[j].KnowledgeID
+ })
+ if t.patternFilter != nil {
+ matched := final[:0]
+ for _, r := range final {
+ if t.patternFilter.MatchString(t.getEnrichedPassage(ctx, r.SearchResult)) {
+ matched = append(matched, r)
+ }
+ }
+ final = matched
+ }
+ if len(final) > limit {
+ final = final[:limit]
+ }
+
+ // Enrich image info for search results (lazy-loaded from child image chunks)
+ if t.chunkService != nil && len(final) > 0 {
+ byTenant := make(map[uint64][]*types.SearchResult)
+ for _, r := range final {
+ tid := t.searchTargets.GetTenantIDForKB(r.KnowledgeBaseID)
+ if tid == 0 {
+ continue
+ }
+ byTenant[tid] = append(byTenant[tid], r.SearchResult)
+ }
+ for tid, batch := range byTenant {
+ searchutil.EnrichSearchResultsImageInfo(ctx, t.chunkService.GetRepository(), tid, batch)
+ }
+ }
+
+ result := t.formatOutput(ctx, final, kbIDs, query, mode)
+ annotateModeFallback(result.Data, mode, kbModes)
+ if len(final) == 0 {
+ result.Output = emptySearchStatement(query, result.Data, len(kbIDs))
+ }
+ return result, nil
+}
+
+// emptySearchStatement describes an empty result, including any mode
+// fallback, so the model does not read "no semantic neighbours" as "the
+// exact term does not occur".
+func emptySearchStatement(query string, data map[string]interface{}, kbCount int) string {
+ mode, _ := data["mode"].(string)
+ msg := fmt.Sprintf("No matching chunks for %q (mode=%s) in %d knowledge base(s).", query, mode, kbCount)
+ fallbacks, _ := data["mode_fallbacks"].([]map[string]interface{})
+ for _, fb := range fallbacks {
+ msg += fmt.Sprintf(" Knowledge base %v was searched with mode=%v (%v).",
+ fb["knowledge_base_id"], fb["mode"], fb["reason"])
+ }
+ return msg
+}
+
+// kbSearchMode is the retrieval path used for one knowledge base.
+type kbSearchMode struct {
+ mode string
+ fallback bool // mode differs from the requested one
+ reason string // why the requested mode could not be served
+}
+
+// resolveKBSearchModes decides, per knowledge base, which retrieval path
+// serves the requested mode. A base that lacks the requested index falls
+// back to the one it has (FAQ bases have no keyword index; vector-only bases
+// have no keyword index; keyword-only bases have no vector index) instead of
+// failing the whole call. Bases with no chunk index at all (wiki-only /
+// graph-only) are left out. The call is refused only when no base in scope
+// can be searched. An empty kbs list (lookup failed) resolves to nil, which
+// callers treat as "use the requested mode everywhere".
+func resolveKBSearchModes(mode string, kbs []*types.KnowledgeBase) (map[string]kbSearchMode, string) {
+ if len(kbs) == 0 {
+ return nil, ""
+ }
+ modes := make(map[string]kbSearchMode, len(kbs))
+ for _, kb := range kbs {
+ if kb == nil {
+ continue
+ }
+ hasVector := kb.IsVectorEnabled()
+ hasKeyword := kb.IsKeywordEnabled() && kb.Type != types.KnowledgeBaseTypeFAQ
+ switch {
+ case !hasVector && !hasKeyword:
+ continue
+ case mode == SearchModeKeyword && !hasKeyword:
+ reason := "no keyword index"
+ if kb.Type == types.KnowledgeBaseTypeFAQ {
+ reason = "FAQ bases are indexed for semantic search only"
+ }
+ modes[kb.ID] = kbSearchMode{mode: SearchModeSemantic, fallback: true, reason: reason}
+ case mode == SearchModeSemantic && !hasVector:
+ modes[kb.ID] = kbSearchMode{mode: SearchModeKeyword, fallback: true, reason: "no vector index"}
+ default:
+ modes[kb.ID] = kbSearchMode{mode: mode}
+ }
+ }
+ if len(modes) == 0 {
+ return nil, "none of the selected knowledge bases has a chunk index; use a wiki or graph tool for this scope"
+ }
+ return modes, ""
+}
+
+// annotateModeFallback records in the result which bases were searched with
+// a different mode than requested, so neither the model nor the UI mistakes
+// a semantic fallback for literal keyword matches.
+func annotateModeFallback(data map[string]interface{}, requested string, kbModes map[string]kbSearchMode) {
+ if data == nil {
+ return
+ }
+ data["requested_mode"] = requested
+ var fallbacks []map[string]interface{}
+ used := make(map[string]bool)
+ ids := make([]string, 0, len(kbModes))
+ for kbID := range kbModes {
+ ids = append(ids, kbID)
+ }
+ sort.Strings(ids)
+ for _, kbID := range ids {
+ m := kbModes[kbID]
+ used[m.mode] = true
+ if m.fallback {
+ fallbacks = append(fallbacks, map[string]interface{}{
+ "knowledge_base_id": kbID, "mode": m.mode, "reason": m.reason,
+ })
+ }
+ }
+ if len(fallbacks) == 0 {
+ return
+ }
+ data["mode_fallbacks"] = fallbacks
+ if len(used) == 1 {
+ for effective := range used {
+ data["mode"] = effective
+ }
+ }
+}
+
+// retrievalParams resolves the candidate pool size and recall thresholds from
+// the global configuration, with hard-coded fallbacks.
+func (t *SearchKnowledgeTool) retrievalParams(limit int) (topK int, vectorThreshold, keywordThreshold float64) {
+ if t.config != nil && t.config.Conversation != nil {
+ topK = t.config.Conversation.EmbeddingTopK
+ vectorThreshold = t.config.Conversation.VectorThreshold
+ keywordThreshold = t.config.Conversation.KeywordThreshold
+ }
+ if topK <= 0 {
+ topK = searchKnowledgeMaxLimit
+ }
+ if topK < limit {
+ topK = limit
+ }
+ if vectorThreshold == 0 {
+ vectorThreshold = 0.6
+ }
+ if keywordThreshold == 0 {
+ keywordThreshold = 0.5
+ }
+ return topK, vectorThreshold, keywordThreshold
+}
+
+// concurrentSearchByTargets executes retrieval using pre-computed search targets.
+// Targets sharing the same underlying embedding model (identified by model name + endpoint)
+// are grouped so the query embedding is computed once per model, and all
+// full-KB targets in a group are combined into a single retrieval call.
+func (t *SearchKnowledgeTool) concurrentSearchByTargets(
+ ctx context.Context,
+ query string,
+ mode string,
+ kbModes map[string]kbSearchMode,
+ searchTargets types.SearchTargets,
+ kbList []*types.KnowledgeBase,
+ topK int,
+ vectorThreshold, keywordThreshold float64,
+ kbTypeMap map[string]string,
+) []*searchResultWithMeta {
+ // Filter out non-searchable KBs (wiki-only / graph-only). Feeding a
+ // wiki-only KB into HybridSearch causes spurious "model ID cannot be
+ // empty" errors because such KBs have no EmbeddingModelID configured.
+ // KBs that we couldn't fetch (not in kbList) are kept so the downstream
+ // HybridSearch path can still surface the real error.
+ knownKBs := make(map[string]bool, len(kbList))
+ for _, kb := range kbList {
+ if kb != nil {
+ knownKBs[kb.ID] = true
+ }
+ }
+ modeFor := func(kbID string) (string, bool) {
+ if m, ok := kbModes[kbID]; ok {
+ return m.mode, true
+ }
+ // Unknown to the lookup: keep it so HybridSearch surfaces the real error.
+ return mode, !knownKBs[kbID]
+ }
+ filteredTargets := make(types.SearchTargets, 0, len(searchTargets))
+ for _, st := range searchTargets {
+ if st == nil || st.KnowledgeBaseID == "" {
+ continue
+ }
+ if _, ok := modeFor(st.KnowledgeBaseID); ok {
+ filteredTargets = append(filteredTargets, st)
+ continue
+ }
+ logger.Infof(ctx, "[Tool][SearchKnowledge] Skipping non-searchable KB %s (no vector/keyword index)",
+ st.KnowledgeBaseID)
+ }
+ if len(filteredTargets) == 0 {
+ return nil
+ }
+ searchTargets = filteredTargets
+
+ // Resolve actual model identities (name + endpoint) for cross-tenant grouping
+ modelKeyMap := t.knowledgeBaseService.ResolveEmbeddingModelKeys(ctx, kbList)
+
+ groups := make(map[string][]*types.SearchTarget)
+ for _, st := range searchTargets {
+ key := modelKeyMap[st.KnowledgeBaseID]
+ groups[key] = append(groups[key], st)
+ }
+
+ var wg sync.WaitGroup
+ var mu sync.Mutex
+ allResults := make([]*searchResultWithMeta, 0)
+ collect := func(rows []*types.SearchResult, usedMode string) {
+ mu.Lock()
+ defer mu.Unlock()
+ for _, r := range rows {
+ allResults = append(allResults, &searchResultWithMeta{
+ SearchResult: r,
+ SourceQuery: query,
+ QueryType: usedMode,
+ KnowledgeBaseID: r.KnowledgeBaseID,
+ KnowledgeBaseType: kbTypeMap[r.KnowledgeBaseID],
+ })
+ }
+ }
+
+ for modelKey, targets := range groups {
+ wg.Add(1)
+ go func(modelKey string, targets []*types.SearchTarget) {
+ defer wg.Done()
+
+ needsEmbedding := false
+ for _, st := range targets {
+ if m, _ := modeFor(st.KnowledgeBaseID); m != SearchModeKeyword {
+ needsEmbedding = true
+ break
+ }
+ }
+ // Compute embedding once for this (model, query) pair
+ var queryEmbedding []float32
+ if modelKey != "" && needsEmbedding {
+ emb, err := t.knowledgeBaseService.GetQueryEmbedding(ctx, targets[0].KnowledgeBaseID, query)
+ if err != nil {
+ logger.Warnf(ctx, "[Tool][SearchKnowledge] Failed to pre-compute embedding for model %s: %v",
+ modelKey, err)
+ } else {
+ queryEmbedding = emb
+ }
+ }
+
+ // Separate full-KB targets (combinable per retrieval mode) from
+ // specific-knowledge targets.
+ fullKBIDsByMode := make(map[string][]string)
+ var knowledgeTargets []*types.SearchTarget
+ for _, st := range targets {
+ if st.Type == types.SearchTargetTypeKnowledgeBase && len(st.TagIDs) == 0 {
+ m, _ := modeFor(st.KnowledgeBaseID)
+ fullKBIDsByMode[m] = append(fullKBIDsByMode[m], st.KnowledgeBaseID)
+ } else {
+ knowledgeTargets = append(knowledgeTargets, st)
+ }
+ }
+
+ var innerWg sync.WaitGroup
+ for usedMode, fullKBIDs := range fullKBIDsByMode {
+ innerWg.Add(1)
+ go func(usedMode string, fullKBIDs []string) {
+ defer innerWg.Done()
+ kbResults, err := t.knowledgeBaseService.HybridSearch(ctx, fullKBIDs[0], types.SearchParams{
+ QueryText: query,
+ QueryEmbedding: queryEmbedding,
+ KnowledgeBaseIDs: fullKBIDs,
+ MatchCount: topK,
+ VectorThreshold: vectorThreshold,
+ KeywordThreshold: keywordThreshold,
+ DisableVectorMatch: usedMode == SearchModeKeyword,
+ DisableKeywordsMatch: usedMode == SearchModeSemantic,
+ })
+ if err != nil {
+ logger.Warnf(ctx, "[Tool][SearchKnowledge] Combined search failed for KBs %v: %v",
+ fullKBIDs, err)
+ return
+ }
+ collect(kbResults, usedMode)
+ }(usedMode, fullKBIDs)
+ }
+
+ for _, target := range knowledgeTargets {
+ st := target
+ innerWg.Add(1)
+ go func() {
+ defer innerWg.Done()
+ usedMode, _ := modeFor(st.KnowledgeBaseID)
+ stVectorThreshold, stKeywordThreshold := st.RecallThresholds(vectorThreshold, keywordThreshold)
+ kbResults, err := t.knowledgeBaseService.HybridSearch(ctx, st.KnowledgeBaseID, types.SearchParams{
+ QueryText: query,
+ QueryEmbedding: queryEmbedding,
+ MatchCount: topK,
+ VectorThreshold: stVectorThreshold,
+ KeywordThreshold: stKeywordThreshold,
+ KnowledgeIDs: st.KnowledgeIDs,
+ TagIDs: st.TagIDs,
+ ScopeTagIDs: st.ScopeTagIDs,
+ DisableVectorMatch: usedMode == SearchModeKeyword,
+ DisableKeywordsMatch: usedMode == SearchModeSemantic,
+ })
+ if err != nil {
+ logger.Warnf(ctx, "[Tool][SearchKnowledge] Failed to search KB %s: %v", st.KnowledgeBaseID, err)
+ return
+ }
+ collect(kbResults, usedMode)
+ }()
+ }
+ innerWg.Wait()
+ }(modelKey, targets)
+ }
+ wg.Wait()
+ return allResults
+}
+
+// rerankResults applies reranking to all search results (including FAQ entries)
+// using the configured rerank model, then filters by threshold and applies
+// composite scoring so MMR/sorting uses a single score scale.
+//
+// A failed rerank call degrades to the raw retrieval order. An empty result
+// after threshold filtering is kept empty: filterRerankRankResults already
+// preserves the top candidate down to agentRerankFallbackMinScore, so reaching
+// zero means even the best match is below that floor.
+func (t *SearchKnowledgeTool) rerankResults(
+ ctx context.Context,
+ query string,
+ results []*searchResultWithMeta,
+) ([]*searchResultWithMeta, error) {
+ if len(results) == 0 || t.rerankModel == nil {
+ return results, nil
+ }
+
+ rankResults, err := t.rerankScores(ctx, query, results)
+ if err != nil {
+ logger.Warnf(ctx, "[Tool][SearchKnowledge] Rerank model failed, using raw retrieval results: %v", err)
+ return results, nil
+ }
+
+ threshold := t.rerankThreshold()
+ reranked := t.applyModelRerankScores(
+ results,
+ rankResults,
+ threshold,
+ t.searchTargets.HasRecallThresholdOverride(),
+ )
+ logger.Infof(ctx, "[Tool][SearchKnowledge] Reranked %d/%d results above threshold %.2f",
+ len(reranked), len(results), threshold)
+ return reranked, nil
+}
+
+func (t *SearchKnowledgeTool) getFAQMetadata(
+ ctx context.Context,
+ chunkID string,
+ cache map[string]*types.FAQChunkMetadata,
+) (*types.FAQChunkMetadata, error) {
+ if chunkID == "" || t.chunkService == nil {
+ return nil, nil
+ }
+ if meta, ok := cache[chunkID]; ok {
+ return meta, nil
+ }
+ chunk, err := t.chunkService.GetChunkByID(ctx, chunkID)
+ if err != nil {
+ cache[chunkID] = nil
+ return nil, err
+ }
+ if chunk == nil {
+ cache[chunkID] = nil
+ return nil, nil
+ }
+ meta, err := chunk.FAQMetadata()
+ if err != nil {
+ cache[chunkID] = nil
+ return nil, err
+ }
+ cache[chunkID] = meta
+ return meta, nil
+}
+
+// rerankScores scores the candidates with the configured rerank model and
+// returns the raw relevance scores, leaving threshold filtering and composite
+// scoring to the caller.
+func (t *SearchKnowledgeTool) rerankScores(
+ ctx context.Context,
+ query string,
+ results []*searchResultWithMeta,
+) ([]rerank.RankResult, error) {
+ passages := make([]string, len(results))
+ for i, result := range results {
+ passages[i] = t.getEnrichedPassage(ctx, result.SearchResult)
+ }
+ rerankResp, err := t.rerankModel.Rerank(ctx, query, passages)
+ if err != nil {
+ return nil, fmt.Errorf("rerank call failed: %w", err)
+ }
+ return rerankResp, nil
+}
+
+func (t *SearchKnowledgeTool) rerankThreshold() float64 {
+ if t.config != nil && t.config.Conversation != nil && t.config.Conversation.RerankThreshold > 0 {
+ return t.config.Conversation.RerankThreshold
+ }
+ return 0.3
+}
+
+const agentRerankFallbackMinScore = 0.15
+
+func filterRerankRankResults(
+ rankResults []rerank.RankResult,
+ threshold float64,
+ preserveTop bool,
+) []rerank.RankResult {
+ if len(rankResults) == 0 {
+ return nil
+ }
+ filtered := make([]rerank.RankResult, 0, len(rankResults))
+ for _, r := range rankResults {
+ if r.RelevanceScore >= threshold {
+ filtered = append(filtered, r)
+ }
+ }
+ if len(filtered) == 0 {
+ top := rankResults[0]
+ for _, r := range rankResults[1:] {
+ if r.RelevanceScore > top.RelevanceScore {
+ top = r
+ }
+ }
+ if preserveTop || top.RelevanceScore >= agentRerankFallbackMinScore {
+ return []rerank.RankResult{top}
+ }
+ }
+ return filtered
+}
+
+func (t *SearchKnowledgeTool) applyModelRerankScores(
+ originals []*searchResultWithMeta,
+ rankResults []rerank.RankResult,
+ threshold float64,
+ preserveTop bool,
+) []*searchResultWithMeta {
+ filtered := filterRerankRankResults(rankResults, threshold, preserveTop)
+ out := make([]*searchResultWithMeta, 0, len(filtered))
+ for _, rr := range filtered {
+ if rr.Index < 0 || rr.Index >= len(originals) {
+ continue
+ }
+ newResult := *originals[rr.Index]
+ baseScore := newResult.Score
+ modelScore := rr.RelevanceScore
+ newResult.Score = t.compositeScore(&newResult, modelScore, baseScore)
+ out = append(out, &newResult)
+ }
+ sort.Slice(out, func(i, j int) bool {
+ return out[i].Score > out[j].Score
+ })
+ return out
+}
+
+// deduplicateResults removes duplicate chunks, keeping the first occurrence.
+// Uses multiple keys (ID, parent chunk ID, knowledge+index) and a content
+// signature for near-duplicate detection. Input order is preserved.
+func (t *SearchKnowledgeTool) deduplicateResults(results []*searchResultWithMeta) []*searchResultWithMeta {
+ seen := make(map[string]bool)
+ contentSig := make(map[string]bool)
+ uniqueResults := make([]*searchResultWithMeta, 0, len(results))
+
+ for _, r := range results {
+ if r == nil || r.SearchResult == nil {
+ continue
+ }
+ keys := []string{r.ID}
+ if r.ParentChunkID != "" {
+ keys = append(keys, "parent:"+r.ParentChunkID)
+ }
+ if r.KnowledgeID != "" {
+ keys = append(keys, fmt.Sprintf("kb:%s#%d", r.KnowledgeID, r.ChunkIndex))
+ }
+ dup := false
+ for _, k := range keys {
+ if seen[k] {
+ dup = true
+ break
+ }
+ }
+ if dup {
+ continue
+ }
+ sig := searchutil.BuildContentSignature(r.Content)
+ if sig != "" {
+ if contentSig[sig] {
+ continue
+ }
+ contentSig[sig] = true
+ }
+ for _, k := range keys {
+ seen[k] = true
+ }
+ uniqueResults = append(uniqueResults, r)
+ }
+ return uniqueResults
+}
+
+// writeKnowledgeMetadataHeader emits document-scoped metadata once per
+// knowledge item. Chunk entries keep only chunk-specific content so repeated
+// results from the same document do not waste context.
+func writeKnowledgeMetadataHeader(ob *strings.Builder, results []*searchResultWithMeta) {
+ seen := make(map[string]struct{}, len(results))
+ hasMetadata := false
+ var documents strings.Builder
+ for _, result := range results {
+ if result == nil || result.SearchResult == nil || result.KnowledgeID == "" ||
+ result.KnowledgeCustomMetadata == "" {
+ continue
+ }
+ if _, ok := seen[result.KnowledgeID]; ok {
+ continue
+ }
+ seen[result.KnowledgeID] = struct{}{}
+ hasMetadata = true
+ fmt.Fprintf(&documents,
+ "<document knowledge_id=\"%s\" knowledge_base_id=\"%s\" title=\"%s\">\n",
+ xmlEscape(result.KnowledgeID),
+ xmlEscape(result.KnowledgeBaseID),
+ xmlEscape(result.KnowledgeTitle),
+ )
+ fmt.Fprintf(&documents, "<metadata>%s</metadata>\n", xmlEscape(result.KnowledgeCustomMetadata))
+ documents.WriteString("</document>\n")
+ }
+ if !hasMetadata {
+ return
+ }
+ ob.WriteString("<documents>\n")
+ ob.WriteString(documents.String())
+ ob.WriteString("</documents>\n")
+}
+
+// formatOutput builds the structured Data (the source of truth for the UI and
+// the model rendering) and a compact XML Output for logs.
+func (t *SearchKnowledgeTool) formatOutput(
+ ctx context.Context,
+ results []*searchResultWithMeta,
+ kbsToSearch []string,
+ query string,
+ mode string,
+) *types.ToolResult {
+ data := map[string]interface{}{
+ "knowledge_base_ids": kbsToSearch,
+ "query": query,
+ "queries": []string{query}, // legacy alias for older frontends
+ "mode": mode,
+ "results": []interface{}{},
+ "count": 0,
+ "display_type": "search_results",
+ }
+ if len(results) == 0 {
+ return &types.ToolResult{
+ Success: true,
+ Output: fmt.Sprintf("No matching chunks for %q (mode=%s) in %d knowledge base(s).",
+ query, mode, len(kbsToSearch)),
+ Data: data,
+ }
+ }
+
+ kbCounts := make(map[string]int)
+ for _, r := range results {
+ kbCounts[r.KnowledgeBaseID]++
+ }
+
+ var ob strings.Builder
+ fmt.Fprintf(&ob, "<search_results count=\"%d\" mode=\"%s\">\n", len(results), xmlEscape(mode))
+ fmt.Fprintf(&ob, "<query>%s</query>\n", xmlEscape(query))
+ writeKnowledgeMetadataHeader(&ob, results)
+
+ formattedResults := make([]map[string]interface{}, 0, len(results))
+ faqMetadataCache := make(map[string]*types.FAQChunkMetadata)
+ queries := []string{query}
+
+ for i, result := range results {
+ var faqMeta *types.FAQChunkMetadata
+ if result.KnowledgeBaseType == types.KnowledgeBaseTypeFAQ {
+ meta, err := t.getFAQMetadata(ctx, result.ID, faqMetadataCache)
+ if err != nil {
+ logger.Warnf(ctx, "[Tool][SearchKnowledge] Failed to load FAQ metadata for chunk %s: %v",
+ result.ID, err)
+ } else {
+ faqMeta = meta
+ }
+ }
+ isFAQ := faqMeta != nil
+
+ snippet := ""
+ if isFAQ {
+ snippet = faqMatchSnippetFromQueries(faqMeta, queries)
+ }
+ if snippet == "" && t.patternFilter != nil {
+ snippet = extractSnippetRegex(result.Content, []*regexp.Regexp{t.patternFilter})
+ }
+ if snippet == "" {
+ snippet = extractSnippetForQueries(result.Content, queries)
+ }
+
+ tag := "chunk"
+ idAttr := fmt.Sprintf("chunk_id=\"%s\" chunk_index=\"%d\" knowledge_id=\"%s\"",
+ xmlEscape(result.ID), result.ChunkIndex, xmlEscape(result.KnowledgeID))
+ if isFAQ {
+ tag = "faq"
+ idAttr = fmt.Sprintf("faq_id=\"%s\" index=\"%d\"", xmlEscape(result.ID), result.ChunkIndex)
+ }
+ fmt.Fprintf(&ob, "<%s rank=\"%d\" %s knowledge_base_id=\"%s\" knowledge_title=\"%s\" score=\"%.3f\">\n",
+ tag, i+1, idAttr, xmlEscape(result.KnowledgeBaseID), xmlEscape(result.KnowledgeTitle), result.Score)
+ if snippet != "" {
+ fmt.Fprintf(&ob, "<match_snippet>%s</match_snippet>\n", xmlEscape(snippet))
+ }
+ fmt.Fprintf(&ob, "<content>%s</content>\n", result.Content)
+ if result.ImageInfo != "" {
+ var imageInfos []types.ImageInfo
+ if err := json.Unmarshal([]byte(result.ImageInfo), &imageInfos); err == nil {
+ for _, img := range imageInfos {
+ if md := searchutil.BuildImageInfoMarkdownWithURL(img.URL, &img); md != "" {
+ ob.WriteString(md)
+ ob.WriteString("\n")
+ }
+ }
+ }
+ }
+ if isFAQ {
+ writeFAQFieldsXML(&ob, faqMeta)
+ }
+ fmt.Fprintf(&ob, "</%s>\n", tag)
+
+ row := map[string]interface{}{
+ "result_index": i + 1,
+ "content": result.Content,
+ "knowledge_id": result.KnowledgeID,
+ "knowledge_base_id": result.KnowledgeBaseID,
+ "knowledge_title": result.KnowledgeTitle,
+ "knowledge_metadata": result.KnowledgeCustomMetadata,
+ "match_type": result.MatchType,
+ "match_snippet": snippet,
+ "score": result.Score,
+ "source_query": result.SourceQuery,
+ "query_type": result.QueryType,
+ "knowledge_base_type": result.KnowledgeBaseType,
+ }
+ if images := chunkImageList(result.ImageInfo); len(images) > 0 {
+ row["images"] = images
+ }
+ if isFAQ {
+ row["faq_id"] = result.ID
+ row["index"] = result.ChunkIndex
+ if faqMeta.StandardQuestion != "" {
+ row["faq_standard_question"] = faqMeta.StandardQuestion
+ }
+ appendSimilarQuestionsToChunkData(row, faqMeta.SimilarQuestions)
+ if len(faqMeta.Answers) > 0 {
+ row["faq_answers"] = faqMeta.Answers
+ }
+ } else {
+ row["chunk_id"] = result.ID
+ row["chunk_index"] = result.ChunkIndex
+ }
+ formattedResults = append(formattedResults, row)
+ }
+ ob.WriteString("</search_results>")
+
+ data["results"] = formattedResults
+ data["count"] = len(formattedResults)
+ data["kb_counts"] = kbCounts
+ return &types.ToolResult{Success: true, Output: ob.String(), Data: data}
+}
+
+// getEnrichedPassage merges chunk content with image captions / OCR text so
+// the reranker and MMR see the same evidence the model will.
+func (t *SearchKnowledgeTool) getEnrichedPassage(ctx context.Context, result *types.SearchResult) string {
+ if result.ImageInfo == "" {
+ return result.Content
+ }
+ var imageInfos []types.ImageInfo
+ if err := json.Unmarshal([]byte(result.ImageInfo), &imageInfos); err != nil {
+ logger.Warnf(ctx, "[Tool][SearchKnowledge] Failed to parse image info: %v", err)
+ return result.Content
+ }
+ if len(imageInfos) == 0 {
+ return result.Content
+ }
+ var imageTexts []string
+ for _, img := range imageInfos {
+ if img.Caption != "" {
+ imageTexts = append(imageTexts, fmt.Sprintf("Image Caption: %s", img.Caption))
+ }
+ if img.OCRText != "" {
+ imageTexts = append(imageTexts, fmt.Sprintf("Image Text: %s", img.OCRText))
+ }
+ }
+ if len(imageTexts) == 0 {
+ return result.Content
+ }
+ combinedText := result.Content
+ if combinedText != "" {
+ combinedText += "\n\n"
+ }
+ return combinedText + strings.Join(imageTexts, "\n")
+}
+
+// compositeScore calculates a composite score considering multiple factors
+func (t *SearchKnowledgeTool) compositeScore(
+ result *searchResultWithMeta,
+ modelScore, baseScore float64,
+) float64 {
+ // Source weight: web_search results get slightly lower weight
+ sourceWeight := 1.0
+ if strings.ToLower(result.KnowledgeSource) == "web_search" {
+ sourceWeight = 0.95
+ }
+
+ // Position prior: slightly favor chunks earlier in the document
+ positionPrior := 1.0
+ if result.StartAt >= 0 && result.EndAt > result.StartAt {
+ positionRatio := 1.0 - float64(result.StartAt)/float64(result.EndAt+1)
+ positionPrior += t.clampFloat(positionRatio, -0.05, 0.05)
+ }
+
+ composite := 0.6*modelScore + 0.3*baseScore + 0.1*sourceWeight
+ composite *= positionPrior
+ if composite < 0 {
+ composite = 0
+ }
+ if composite > 1 {
+ composite = 1
+ }
+ return composite
+}
+
+func (t *SearchKnowledgeTool) clampFloat(v, minV, maxV float64) float64 {
+ return searchutil.ClampFloat(v, minV, maxV)
+}
+
+// applyMMR applies Maximal Marginal Relevance to reduce redundancy.
+func (t *SearchKnowledgeTool) applyMMR(
+ ctx context.Context,
+ results []*searchResultWithMeta,
+ k int,
+ lambda float64,
+) []*searchResultWithMeta {
+ if k <= 0 || len(results) == 0 {
+ return nil
+ }
+
+ selected := make([]*searchResultWithMeta, 0, k)
+ candidates := make([]*searchResultWithMeta, len(results))
+ copy(candidates, results)
+
+ tokenSets := make([]map[string]struct{}, len(candidates))
+ for i, r := range candidates {
+ tokenSets[i] = t.tokenizeSimple(t.getEnrichedPassage(ctx, r.SearchResult))
+ }
+
+ // Incremental form: maxRedundancy[i] caches candidate i's maximum jaccard
+ // against everything selected so far, so each round only needs one
+ // comparison per remaining candidate. Selection output is identical to
+ // the naive form, including tie-breaking.
+ maxRedundancy := make([]float64, len(candidates))
+ for len(selected) < k && len(candidates) > 0 {
+ bestIdx := 0
+ bestScore := -1.0
+ for i, r := range candidates {
+ mmr := lambda*r.Score - (1.0-lambda)*maxRedundancy[i]
+ if mmr > bestScore {
+ bestScore = mmr
+ bestIdx = i
+ }
+ }
+ selected = append(selected, candidates[bestIdx])
+ chosenTokens := tokenSets[bestIdx]
+ candidates = append(candidates[:bestIdx], candidates[bestIdx+1:]...)
+ tokenSets = append(tokenSets[:bestIdx], tokenSets[bestIdx+1:]...)
+ maxRedundancy = append(maxRedundancy[:bestIdx], maxRedundancy[bestIdx+1:]...)
+ for i := range candidates {
+ maxRedundancy[i] = math.Max(maxRedundancy[i], t.jaccard(tokenSets[i], chosenTokens))
+ }
+ }
+ return selected
+}
+
+func (t *SearchKnowledgeTool) tokenizeSimple(text string) map[string]struct{} {
+ return searchutil.TokenizeSimple(text)
+}
+
+func (t *SearchKnowledgeTool) jaccard(a, b map[string]struct{}) float64 {
+ return searchutil.Jaccard(a, b)
+}
diff --git a/internal/agent/tools/knowledge_search_metadata_test.go b/internal/agent/tools/search_knowledge_metadata_test.go
similarity index 100%
rename from internal/agent/tools/knowledge_search_metadata_test.go
rename to internal/agent/tools/search_knowledge_metadata_test.go
diff --git a/internal/agent/tools/knowledge_search_mmr_test.go b/internal/agent/tools/search_knowledge_mmr_test.go
similarity index 96%
rename from internal/agent/tools/knowledge_search_mmr_test.go
rename to internal/agent/tools/search_knowledge_mmr_test.go
index c0f3516c8..8491d7167 100644
--- a/internal/agent/tools/knowledge_search_mmr_test.go
+++ b/internal/agent/tools/search_knowledge_mmr_test.go
@@ -13,7 +13,7 @@ import (
// redundancy of every candidate against every already-selected result on each
// round, re-tokenizing the selected passages every time. It is kept here purely
// as the reference oracle for the incremental implementation.
-func (t *KnowledgeSearchTool) applyMMRNaive(
+func (t *SearchKnowledgeTool) applyMMRNaive(
ctx context.Context,
results []*searchResultWithMeta,
k int,
@@ -102,7 +102,7 @@ func mmrTestCorpus(n int) []*searchResultWithMeta {
func TestApplyMMR_matchesNaiveSelection(t *testing.T) {
t.Parallel()
- tool := &KnowledgeSearchTool{}
+ tool := &SearchKnowledgeTool{}
ctx := context.Background()
results := mmrTestCorpus(40)
@@ -136,7 +136,7 @@ func TestApplyMMR_matchesNaiveSelection(t *testing.T) {
func TestApplyMMR_emptyAndNonPositiveK(t *testing.T) {
t.Parallel()
- tool := &KnowledgeSearchTool{}
+ tool := &SearchKnowledgeTool{}
ctx := context.Background()
if got := tool.applyMMR(ctx, mmrTestCorpus(3), 0, 0.7); got != nil {
@@ -148,7 +148,7 @@ func TestApplyMMR_emptyAndNonPositiveK(t *testing.T) {
}
func BenchmarkApplyMMR(b *testing.B) {
- tool := &KnowledgeSearchTool{}
+ tool := &SearchKnowledgeTool{}
ctx := context.Background()
results := mmrTestCorpus(250)
diff --git a/internal/agent/tools/knowledge_search_rerank_test.go b/internal/agent/tools/search_knowledge_rerank_test.go
similarity index 97%
rename from internal/agent/tools/knowledge_search_rerank_test.go
rename to internal/agent/tools/search_knowledge_rerank_test.go
index c64fccce5..d601a8acd 100644
--- a/internal/agent/tools/knowledge_search_rerank_test.go
+++ b/internal/agent/tools/search_knowledge_rerank_test.go
@@ -38,8 +38,8 @@ func (s *stubReranker) Rerank(
func (s *stubReranker) GetModelName() string { return "stub-rerank" }
func (s *stubReranker) GetModelID() string { return "stub-rerank-id" }
-func newRerankTestTool(model rerank.Reranker) *KnowledgeSearchTool {
- return &KnowledgeSearchTool{
+func newRerankTestTool(model rerank.Reranker) *SearchKnowledgeTool {
+ return &SearchKnowledgeTool{
rerankModel: model,
config: &config.Config{
Conversation: &config.ConversationConfig{RerankThreshold: 0.3},
@@ -96,7 +96,7 @@ func TestFilterRerankRankResults_thresholdAndFallback(t *testing.T) {
func TestApplyModelRerankScores_faqUsesCompositeScale(t *testing.T) {
t.Parallel()
- tool := &KnowledgeSearchTool{
+ tool := &SearchKnowledgeTool{
config: &config.Config{
Conversation: &config.ConversationConfig{RerankThreshold: 0.3},
},
@@ -192,7 +192,7 @@ func TestRerankResults_withoutModelIsPassthrough(t *testing.T) {
func TestRerankThreshold_default(t *testing.T) {
t.Parallel()
- tool := &KnowledgeSearchTool{}
+ tool := &SearchKnowledgeTool{}
if got := tool.rerankThreshold(); got != 0.3 {
t.Fatalf("default threshold = %v, want 0.3", got)
}
diff --git a/internal/agent/tools/search_knowledge_test.go b/internal/agent/tools/search_knowledge_test.go
new file mode 100644
index 000000000..6576c6eae
--- /dev/null
+++ b/internal/agent/tools/search_knowledge_test.go
@@ -0,0 +1,236 @@
+package tools
+
+import (
+ "context"
+ "encoding/json"
+ "strings"
+ "testing"
+
+ "github.com/Tencent/WeKnora/internal/config"
+ "github.com/Tencent/WeKnora/internal/types"
+)
+
+func TestNormalizeSearchMode(t *testing.T) {
+ for in, want := range map[string]string{
+ "": SearchModeHybrid, "hybrid": SearchModeHybrid,
+ " Semantic ": SearchModeSemantic, "KEYWORD": SearchModeKeyword,
+ } {
+ got, err := normalizeSearchMode(in)
+ if err != nil || got != want {
+ t.Errorf("normalizeSearchMode(%q) = %q, %v; want %q", in, got, err, want)
+ }
+ }
+ if _, err := normalizeSearchMode("regex"); err == nil {
+ t.Fatal("unknown mode must be rejected")
+ }
+}
+
+func TestClampSearchLimit(t *testing.T) {
+ if got := clampSearchLimit(0); got != searchKnowledgeDefaultLimit {
+ t.Fatalf("default limit = %d", got)
+ }
+ if got := clampSearchLimit(500); got != searchKnowledgeMaxLimit {
+ t.Fatalf("max limit = %d", got)
+ }
+ if got := clampSearchLimit(7); got != 7 {
+ t.Fatalf("explicit limit = %d", got)
+ }
+}
+
+func kbWithIndexes(id string, vector, keyword bool, kbType string) *types.KnowledgeBase {
+ kb := &types.KnowledgeBase{ID: id, Type: kbType}
+ kb.IndexingStrategy.VectorEnabled = vector
+ kb.IndexingStrategy.KeywordEnabled = keyword
+ return kb
+}
+
+func TestResolveKBSearchModesFallsBackPerKnowledgeBase(t *testing.T) {
+ vectorOnly := kbWithIndexes("kb-v", true, false, "")
+ keywordOnly := kbWithIndexes("kb-k", false, true, "")
+ both := kbWithIndexes("kb-b", true, true, "")
+ faq := kbWithIndexes("kb-faq", true, true, types.KnowledgeBaseTypeFAQ)
+ wikiOnly := kbWithIndexes("kb-w", false, false, "")
+
+ modes, msg := resolveKBSearchModes(SearchModeKeyword, []*types.KnowledgeBase{vectorOnly, both, faq, wikiOnly})
+ if msg != "" {
+ t.Fatalf("keyword must not be refused while some base can serve it: %q", msg)
+ }
+ if m := modes["kb-b"]; m.mode != SearchModeKeyword || m.fallback {
+ t.Fatalf("keyword-capable base = %+v", m)
+ }
+ if m := modes["kb-v"]; m.mode != SearchModeSemantic || !m.fallback || m.reason != "no keyword index" {
+ t.Fatalf("vector-only base must fall back to semantic: %+v", m)
+ }
+ if m := modes["kb-faq"]; m.mode != SearchModeSemantic || !m.fallback || !strings.Contains(m.reason, "FAQ") {
+ t.Fatalf("FAQ base must fall back to semantic: %+v", m)
+ }
+ if _, ok := modes["kb-w"]; ok {
+ t.Fatal("a wiki-only base has no chunk index and must be left out")
+ }
+
+ modes, msg = resolveKBSearchModes(SearchModeKeyword, []*types.KnowledgeBase{faq})
+ if msg != "" || modes["kb-faq"].mode != SearchModeSemantic {
+ t.Fatalf("an FAQ-only scope must still be searchable: modes=%+v msg=%q", modes, msg)
+ }
+ modes, _ = resolveKBSearchModes(SearchModeSemantic, []*types.KnowledgeBase{keywordOnly})
+ if m := modes["kb-k"]; m.mode != SearchModeKeyword || !m.fallback {
+ t.Fatalf("keyword-only base must serve semantic requests with keyword: %+v", m)
+ }
+ modes, _ = resolveKBSearchModes(SearchModeHybrid, []*types.KnowledgeBase{keywordOnly})
+ if m := modes["kb-k"]; m.mode != SearchModeHybrid || m.fallback {
+ t.Fatalf("hybrid already adapts per index: %+v", m)
+ }
+ if _, msg := resolveKBSearchModes(SearchModeHybrid, []*types.KnowledgeBase{wikiOnly}); msg == "" {
+ t.Fatal("a scope with no chunk index must be refused")
+ }
+ if modes, msg := resolveKBSearchModes(SearchModeKeyword, nil); modes != nil || msg != "" {
+ t.Fatalf("an unknown KB list must not refuse the call: %+v %q", modes, msg)
+ }
+}
+
+func TestAnnotateModeFallbackReportsEffectiveMode(t *testing.T) {
+ all := map[string]kbSearchMode{"kb-faq": {mode: SearchModeSemantic, fallback: true, reason: "FAQ"}}
+ data := map[string]interface{}{"mode": SearchModeKeyword}
+ annotateModeFallback(data, SearchModeKeyword, all)
+ if data["mode"] != SearchModeSemantic || data["requested_mode"] != SearchModeKeyword {
+ t.Fatalf("a scope that fell back entirely must report the mode it used: %+v", data)
+ }
+ fallbacks, _ := data["mode_fallbacks"].([]map[string]interface{})
+ if len(fallbacks) != 1 || fallbacks[0]["knowledge_base_id"] != "kb-faq" {
+ t.Fatalf("fallbacks = %+v", data["mode_fallbacks"])
+ }
+ if msg := emptySearchStatement("E1001", data, 1); !strings.Contains(msg, "mode=semantic") ||
+ !strings.Contains(msg, "kb-faq was searched with mode=semantic (FAQ)") {
+ t.Fatalf("empty statement must explain the fallback: %q", msg)
+ }
+
+ partial := map[string]kbSearchMode{
+ "kb-b": {mode: SearchModeKeyword},
+ "kb-v": {mode: SearchModeSemantic, fallback: true, reason: "no keyword index"},
+ }
+ data = map[string]interface{}{"mode": SearchModeKeyword}
+ annotateModeFallback(data, SearchModeKeyword, partial)
+ if data["mode"] != SearchModeKeyword {
+ t.Fatalf("a partial fallback keeps the requested mode: %+v", data)
+ }
+ data = map[string]interface{}{"mode": SearchModeKeyword}
+ annotateModeFallback(data, SearchModeKeyword, map[string]kbSearchMode{"kb-b": {mode: SearchModeKeyword}})
+ if _, ok := data["mode_fallbacks"]; ok {
+ t.Fatalf("no fallback, no annotation: %+v", data)
+ }
+}
+
+func TestSearchKnowledgeRejectsMissingQueryAndBadMode(t *testing.T) {
+ tool := &SearchKnowledgeTool{BaseTool: searchKnowledgeTool}
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"query":" "}`))
+ if err == nil || res.Success || !strings.Contains(res.Error, "query is required") {
+ t.Fatalf("blank query: res=%+v err=%v", res, err)
+ }
+ res, err = tool.Execute(context.Background(), json.RawMessage(`{"query":"x","mode":"grep"}`))
+ if err == nil || res.Success || !strings.Contains(res.Error, "mode must be one of") {
+ t.Fatalf("bad mode: res=%+v err=%v", res, err)
+ }
+}
+
+func TestSearchKnowledgeRejectsOutOfScopeKnowledgeBase(t *testing.T) {
+ tool := NewSearchKnowledgeTool(nil, nil, nil, types.SearchTargets{{
+ Type: types.SearchTargetTypeKnowledgeBase, KnowledgeBaseID: "kb-1", TenantID: 1,
+ }}, nil, nil)
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"query":"x","knowledge_base_ids":["kb-9"]}`))
+ if err == nil || res.Success {
+ t.Fatalf("out-of-scope KB must be rejected: res=%+v err=%v", res, err)
+ }
+}
+
+func TestRetrievalParamsFallBackAndNeverUndercutLimit(t *testing.T) {
+ tool := &SearchKnowledgeTool{config: &config.Config{Conversation: &config.ConversationConfig{EmbeddingTopK: 5}}}
+ topK, v, k := tool.retrievalParams(12)
+ if topK != 12 || v != 0.6 || k != 0.5 {
+ t.Fatalf("retrievalParams = %d %.2f %.2f", topK, v, k)
+ }
+ bare := &SearchKnowledgeTool{}
+ if topK, _, _ := bare.retrievalParams(3); topK != searchKnowledgeMaxLimit {
+ t.Fatalf("missing config should fall back to %d, got %d", searchKnowledgeMaxLimit, topK)
+ }
+}
+
+func TestDeduplicateResultsKeepsFirstOccurrenceInOrder(t *testing.T) {
+ tool := &SearchKnowledgeTool{}
+ in := []*searchResultWithMeta{
+ {SearchResult: &types.SearchResult{ID: "c1", KnowledgeID: "d1", ChunkIndex: 0, Content: "alpha", Score: 0.9}},
+ // same document + index as c1
+ {SearchResult: &types.SearchResult{ID: "c2", KnowledgeID: "d1", ChunkIndex: 0, Content: "other", Score: 0.8}},
+ // same content as c1
+ {SearchResult: &types.SearchResult{ID: "c3", KnowledgeID: "d2", ChunkIndex: 1, Content: "alpha", Score: 0.7}},
+ {SearchResult: &types.SearchResult{ID: "c4", KnowledgeID: "d3", ChunkIndex: 2, Content: "gamma", Score: 0.6}},
+ nil,
+ }
+ out := tool.deduplicateResults(in)
+ if len(out) != 2 || out[0].ID != "c1" || out[1].ID != "c4" {
+ ids := make([]string, 0, len(out))
+ for _, r := range out {
+ ids = append(ids, r.ID)
+ }
+ t.Fatalf("deduplicateResults = %v, want [c1 c4]", ids)
+ }
+}
+
+func TestFormatOutputEmptyResultIsPlainStatement(t *testing.T) {
+ tool := &SearchKnowledgeTool{}
+ res := tool.formatOutput(
+ context.Background(), nil, []string{"kb-1", "kb-2"}, "why is the sky blue", SearchModeSemantic,
+ )
+ if !res.Success {
+ t.Fatal("empty search is not an error")
+ }
+ if strings.Contains(res.Output, "CRITICAL") || strings.Contains(res.Output, "DO NOT") {
+ t.Fatalf("empty result must not carry behavioural instructions: %q", res.Output)
+ }
+ if !strings.Contains(res.Output, "No matching chunks") || !strings.Contains(res.Output, "mode=semantic") {
+ t.Fatalf("empty result statement = %q", res.Output)
+ }
+ if res.Data["display_type"] != "search_results" || res.Data["count"] != 0 ||
+ res.Data["mode"] != SearchModeSemantic {
+ t.Fatalf("empty result data = %+v", res.Data)
+ }
+}
+
+func TestFormatOutputRowsCarryHandlesAndSnippets(t *testing.T) {
+ tool := &SearchKnowledgeTool{}
+ results := []*searchResultWithMeta{
+ {
+ SearchResult: &types.SearchResult{
+ ID: "chunk-1", KnowledgeID: "doc-1", ChunkIndex: 3, KnowledgeTitle: "Guide <v2>",
+ Content: "Install the engine, then configure psionic drive parameters.", Score: 0.91,
+ KnowledgeCustomMetadata: "region: EU",
+ },
+ SourceQuery: "psionic drive", QueryType: SearchModeHybrid, KnowledgeBaseID: "kb-1",
+ },
+ }
+ res := tool.formatOutput(context.Background(), results, []string{"kb-1"}, "psionic drive", SearchModeHybrid)
+ if !res.Success {
+ t.Fatal("format failed")
+ }
+ rows, ok := res.Data["results"].([]map[string]interface{})
+ if !ok || len(rows) != 1 {
+ t.Fatalf("results rows = %#v", res.Data["results"])
+ }
+ row := rows[0]
+ if row["chunk_id"] != "chunk-1" || row["knowledge_id"] != "doc-1" || row["chunk_index"] != 3 {
+ t.Fatalf("row identity = %+v", row)
+ }
+ if snippet, _ := row["match_snippet"].(string); !strings.Contains(snippet, "psionic") {
+ t.Fatalf("snippet should surface the query term: %q", snippet)
+ }
+ if res.Data["mode"] != SearchModeHybrid || res.Data["query"] != "psionic drive" {
+ t.Fatalf("data = %+v", res.Data)
+ }
+ if !strings.Contains(res.Output, `<search_results count="1" mode="hybrid">`) ||
+ !strings.Contains(res.Output, `title="Guide &lt;v2&gt;"`) ||
+ !strings.Contains(res.Output, "<metadata>region: EU</metadata>") {
+ t.Fatalf("output = %s", res.Output)
+ }
+ if strings.Contains(res.Output, "already_seen") || strings.Contains(res.Output, "retrieval_statistics") {
+ t.Fatalf("legacy annotations must be gone: %s", res.Output)
+ }
+}
diff --git a/internal/agent/tools/sequentialthinking.go b/internal/agent/tools/sequentialthinking.go
index 6e290a983..209d17e9f 100644
--- a/internal/agent/tools/sequentialthinking.go
+++ b/internal/agent/tools/sequentialthinking.go
@@ -9,6 +9,14 @@ import (
"github.com/Tencent/WeKnora/internal/types"
)
+// thinkingThoughtDescription is the schema description of the `thought`
+// argument; it is JSON-escaped already because it is spliced into the raw
+// schema literal below.
+const thinkingThoughtDescription = "Your current thinking step. Write in natural, user-friendly language. " +
+ "NEVER mention tool names (like \\\"search_knowledge\\\", \\\"read_document\\\", \\\"web_search\\\", etc.). " +
+ "Instead, describe actions in plain language (e.g., \\\"I'll search for key terms\\\" instead of " +
+ "\\\"I'll use search_knowledge\\\"). Focus on WHAT you're trying to find and WHY, not HOW (which tools you'll use)."
+
var sequentialThinkingTool = BaseTool{
name: ToolThinking,
description: `A detailed tool for dynamic and reflective problem-solving through thoughts.
@@ -50,10 +58,12 @@ Each thought can build on, question, or revise previous insights as understandin
* Hypothesis generation
* Hypothesis verification
- **CRITICAL - User-Friendly Thinking**: Write your thoughts in natural, user-friendly language. NEVER mention tool names (like "grep_chunks", "knowledge_search", "web_search", etc.) in your thinking process. Instead, describe your actions in plain language:
- - ❌ BAD: "I'll use grep_chunks to search for keywords, then knowledge_search for semantic understanding"
+ **CRITICAL - User-Friendly Thinking**: Write your thoughts in natural, user-friendly language.
+ NEVER mention tool names (like "search_knowledge", "read_document", "web_search", etc.) in your thinking process.
+ Instead, describe your actions in plain language:
+ - ❌ BAD: "I'll use search_knowledge in keyword mode, then read_document for the details"
- ✅ GOOD: "I'll start by searching for key terms in the knowledge base, then explore related concepts"
- - ❌ BAD: "After grep_chunks returns results, I'll use knowledge_search"
+ - ❌ BAD: "After search_knowledge returns results, I'll call read_document"
- ✅ GOOD: "After finding relevant documents, I'll search for semantically related content"
Write thinking as if explaining your reasoning to a user, not documenting technical steps. Focus on WHAT you're trying to find and WHY, not HOW (which tools you'll use).
@@ -85,7 +95,7 @@ Each thought can build on, question, or revise previous insights as understandin
"properties": {
"thought": {
"type": "string",
- "description": "Your current thinking step. Write in natural, user-friendly language. NEVER mention tool names (like \"grep_chunks\", \"knowledge_search\", \"web_search\", etc.). Instead, describe actions in plain language (e.g., \"I'll search for key terms\" instead of \"I'll use grep_chunks\"). Focus on WHAT you're trying to find and WHY, not HOW (which tools you'll use)."
+ "description": "` + thinkingThoughtDescription + `"
},
"next_thought_needed": {
"type": "boolean",
diff --git a/internal/agent/tools/todo_write.go b/internal/agent/tools/todo_write.go
index 0e12d6fd2..8923ed544 100644
--- a/internal/agent/tools/todo_write.go
+++ b/internal/agent/tools/todo_write.go
@@ -207,10 +207,9 @@ func generatePlanOutput(task string, steps []PlanStep) string {
if len(steps) == 0 {
output += "Note: No specific steps provided. It is recommended to create 3-7 retrieval tasks for systematic research.\n\n"
output += "Suggested retrieval workflow (focused on retrieval tasks, excluding summarization):\n"
- output += "1. Use grep_chunks to search keywords and locate relevant documents\n"
- output += "2. Use knowledge_search for semantic search to retrieve relevant content\n"
- output += "3. Use list_knowledge_chunks to get the full content of key documents\n"
- output += "4. Use web_search to get supplementary information (if needed)\n"
+ output += "1. Use search_knowledge to locate the relevant documents and passages\n"
+ output += "2. Use read_document to read the key documents in full\n"
+ output += "3. Use web_search to get supplementary information (if needed)\n"
output += "\nNote: Summarization and synthesis are handled by the thinking tool. Do not add summarization tasks here.\n"
return output
}
diff --git a/internal/agent/tools/tool_policy_coverage_test.go b/internal/agent/tools/tool_policy_coverage_test.go
index 0b37bff2a..b36526137 100644
--- a/internal/agent/tools/tool_policy_coverage_test.go
+++ b/internal/agent/tools/tool_policy_coverage_test.go
@@ -18,11 +18,10 @@ func TestEveryBuiltInToolDeclaresAModelHandlePolicy(t *testing.T) {
ToolCallMCPTool,
ToolThinking,
ToolTodoWrite,
- ToolGrepChunks,
- ToolKnowledgeSearch,
- ToolListKnowledgeChunks,
+ ToolSearchKnowledge,
+ ToolReadDocument,
+ ToolListDocuments,
ToolQueryKnowledgeGraph,
- ToolGetDocumentInfo,
ToolSearchConversations,
ToolSearchMemory,
ToolDatabaseQuery,
@@ -43,7 +42,6 @@ func TestEveryBuiltInToolDeclaresAModelHandlePolicy(t *testing.T) {
ToolWikiRenamePage,
ToolWikiDeletePage,
ToolWikiSearch,
- ToolWikiReadSourceDoc,
ToolWikiFlagIssue,
ToolWikiReadIssue,
ToolWikiUpdateIssue,
diff --git a/internal/agent/tools/wiki_read_source_doc.go b/internal/agent/tools/wiki_read_source_doc.go
deleted file mode 100644
index 461e9f76a..000000000
--- a/internal/agent/tools/wiki_read_source_doc.go
+++ /dev/null
@@ -1,369 +0,0 @@
-package tools
-
-import (
- "context"
- "encoding/json"
- "fmt"
- "regexp"
- "strings"
-
- "github.com/Tencent/WeKnora/internal/searchutil"
- "github.com/Tencent/WeKnora/internal/types"
- "github.com/Tencent/WeKnora/internal/types/interfaces"
-)
-
-type wikiReadSourceDocTool struct {
- BaseTool
- knowledgeService interfaces.KnowledgeService
- chunkService interfaces.ChunkService
- searchTargets types.SearchTargets
- scopeEnforced bool
-}
-
-func NewWikiReadSourceDocTool(
- knowledgeService interfaces.KnowledgeService,
- chunkService interfaces.ChunkService,
- searchTargets ...types.SearchTargets,
-) types.Tool {
- tool := &wikiReadSourceDocTool{
- BaseTool: NewBaseTool(
- ToolWikiReadSourceDoc,
- `Read or search within a specific source document to drill down for details omitted from the wiki.
-Provide the knowledge_id from the <sources> block.
-You can EITHER search using a regex query OR fetch a specific contiguous range of chunks using start_chunk_index and end_chunk_index (useful for expanding context around a known chunk).
-If neither query nor range is provided, it returns the beginning of the document.`,
- json.RawMessage(`{
- "type": "object",
- "properties": {
- "knowledge_id": {
- "type": "string",
- "description": "The short dN source document ID from the <sources> block"
- },
- "query": {
- "type": "string",
- "description": "Optional: A regex query to filter the document chunks. Use this to find specific quotes or details efficiently. Remember to double-escape backslashes for JSON: write \"C\\\\+\\\\+\" (NOT \"C\\+\\+\") and \"\\\\d+\" (NOT \"\\d+\")."
- },
- "start_chunk_index": {
- "type": "integer",
- "description": "Optional: The starting chunk index (1-based) to read a specific range."
- },
- "end_chunk_index": {
- "type": "integer",
- "description": "Optional: The ending chunk index (1-based) to read a specific range. Must be >= start_chunk_index."
- }
- },
- "required": ["knowledge_id"]
-}`),
- ),
- knowledgeService: knowledgeService,
- chunkService: chunkService,
- }
- // Presence of the variadic argument — not its length — enables the Agent
- // authorization boundary, so an empty scope fails closed.
- if len(searchTargets) > 0 {
- tool.searchTargets = searchTargets[0]
- tool.scopeEnforced = true
- }
- return tool
-}
-
-// enrichChunkImageInfo populates chunk.ImageInfo for a batch of parent text
-// chunks by looking up their image_ocr / image_caption children. Chunks that
-// already have a non-empty ImageInfo are left untouched.
-func enrichChunkImageInfo(
- ctx context.Context,
- chunkRepo interfaces.ChunkRepository,
- tenantID uint64,
- chunks []*types.Chunk,
-) {
- if len(chunks) == 0 || chunkRepo == nil {
- return
- }
- ids := make([]string, 0, len(chunks))
- for _, c := range chunks {
- if c.ImageInfo == "" && c.ID != "" {
- ids = append(ids, c.ID)
- }
- }
- if len(ids) == 0 {
- return
- }
- infoMap := searchutil.CollectImageInfoByChunkIDs(ctx, chunkRepo, tenantID, ids)
- if len(infoMap) == 0 {
- return
- }
- for _, c := range chunks {
- if c.ImageInfo != "" {
- continue
- }
- if merged, ok := infoMap[c.ID]; ok && merged != "" {
- c.ImageInfo = merged
- }
- }
-}
-
-func enrichChunkContent(c *types.Chunk) string {
- content := c.Content
- if c.ImageInfo != "" {
- var imgInfos []types.ImageInfo
- if err := json.Unmarshal([]byte(c.ImageInfo), &imgInfos); err == nil && len(imgInfos) > 0 {
- var imgBuilder strings.Builder
- for _, img := range imgInfos {
- if imageMarkdown := searchutil.BuildImageInfoMarkdownWithURL(img.URL, &img); imageMarkdown != "" {
- imgBuilder.WriteString("\n")
- imgBuilder.WriteString(imageMarkdown)
- }
- }
- content += imgBuilder.String()
- }
- }
- return content
-}
-
-func (t *wikiReadSourceDocTool) Execute(ctx context.Context, args json.RawMessage) (*types.ToolResult, error) {
- var params struct {
- KnowledgeID string `json:"knowledge_id"`
- Query string `json:"query"`
- StartChunkIndex int `json:"start_chunk_index"`
- EndChunkIndex int `json:"end_chunk_index"`
- }
- if err := json.Unmarshal(args, &params); err != nil {
- return &types.ToolResult{Success: false, Error: "Invalid parameters: " + err.Error()}, nil
- }
-
- knowledgeID := strings.TrimSpace(params.KnowledgeID)
- if knowledgeID == "" {
- return &types.ToolResult{Success: false, Error: "knowledge_id is required"}, nil
- }
-
- var knowledge *types.Knowledge
- var err error
- if t.scopeEnforced {
- knowledge, err = authorizeKnowledgeInSearchTargets(ctx, t.searchTargets, knowledgeID, t.knowledgeService)
- } else {
- knowledge, err = t.knowledgeService.GetKnowledgeByIDOnly(ctx, knowledgeID)
- }
- if err != nil || knowledge == nil {
- if err == nil {
- err = fmt.Errorf("knowledge service returned an empty result")
- }
- return &types.ToolResult{Success: false, Error: fmt.Sprintf("Document not found: %v", err)}, nil
- }
-
- var sb strings.Builder
- sb.WriteString("<source_document>\n<metadata>\n")
- sb.WriteString(fmt.Sprintf("<title>%s</title>\n", knowledge.Title))
- sb.WriteString(fmt.Sprintf("<knowledge_id>%s</knowledge_id>\n", knowledgeID))
-
- hasRange := params.StartChunkIndex > 0
- var re *regexp.Regexp
-
- if hasRange {
- if params.EndChunkIndex < params.StartChunkIndex {
- params.EndChunkIndex = params.StartChunkIndex + 10 // Default window
- }
- if params.EndChunkIndex-params.StartChunkIndex > 50 {
- params.EndChunkIndex = params.StartChunkIndex + 50 // Max 50 chunks to prevent bloat
- }
- sb.WriteString(fmt.Sprintf("<chunk_range start=\"%d\" end=\"%d\"/>\n", params.StartChunkIndex, params.EndChunkIndex))
- } else if params.Query != "" {
- compiled, err := regexp.Compile("(?i)" + params.Query)
- if err != nil {
- return &types.ToolResult{Success: false, Error: fmt.Sprintf("Invalid regex query '%s': %v", params.Query, err)}, nil
- }
- re = compiled
- sb.WriteString(fmt.Sprintf("<query>%s</query>\n", params.Query))
- }
-
- matchCount := 0
- pageSize := 100
- page := 1
- if hasRange {
- page = (params.StartChunkIndex-1)/pageSize + 1
- }
-
- var chunksOutput strings.Builder
- formattedChunks := make([]map[string]interface{}, 0)
- totalChunks := int64(0)
- reachedMax := false
-
- var prevChunk *types.Chunk
- var forceOutputNext bool
- outputtedIndices := make(map[int]bool)
- appendFormattedChunk := func(chunk *types.Chunk, content string) {
- if chunk == nil {
- return
- }
- formattedChunks = append(formattedChunks, map[string]interface{}{
- "chunk_id": chunk.ID,
- "chunk_index": chunk.ChunkIndex,
- "chunk_type": chunk.ChunkType,
- "content": content,
- "knowledge_id": knowledgeID,
- "knowledge_base": knowledge.KnowledgeBaseID,
- "knowledge_title": knowledge.Title,
- })
- }
-
- for {
- enabled := true
- pagination := &types.Pagination{
- Page: page,
- PageSize: pageSize,
- }
-
- chunks, total, err := t.chunkService.GetRepository().ListPagedChunksByKnowledgeID(
- ctx,
- knowledge.TenantID,
- knowledgeID,
- pagination,
- []types.ChunkType{types.ChunkTypeText, types.ChunkTypeFAQ},
- nil, "", "", "", "", &enabled,
- )
- if err != nil {
- return &types.ToolResult{Success: false, Error: fmt.Sprintf("Failed to list chunks: %v", err)}, nil
- }
-
- if page == 1 {
- totalChunks = total
- }
-
- if len(chunks) == 0 {
- break
- }
-
- // Lazily enrich ImageInfo from image_ocr / image_caption child chunks.
- // Parent text chunks don't carry image_info themselves (see
- // image_multimodal.go); without this the source-doc reader would
- // silently drop OCR text and captions for image-heavy documents.
- enrichChunkImageInfo(ctx, t.chunkService.GetRepository(), knowledge.TenantID, chunks)
-
- for _, c := range chunks {
- chunkNum := c.ChunkIndex + 1
- chunkContent := enrichChunkContent(c)
-
- if hasRange {
- if chunkNum < params.StartChunkIndex {
- continue
- }
- if chunkNum > params.EndChunkIndex {
- reachedMax = true
- break
- }
- fmt.Fprintf(&chunksOutput, "<chunk index=\"%d\" type=\"range\">\n%s\n</chunk>\n", chunkNum, chunkContent)
- appendFormattedChunk(c, chunkContent)
- matchCount++
- continue
- }
-
- isMatch := false
- if re != nil {
- isMatch = re.MatchString(chunkContent)
- } else {
- isMatch = true
- }
-
- if isMatch {
- matchCount++
-
- if re != nil {
- // Output previous chunk for context
- if prevChunk != nil && !outputtedIndices[prevChunk.ChunkIndex] {
- prevContent := enrichChunkContent(prevChunk)
- fmt.Fprintf(&chunksOutput, "<chunk index=\"%d\" type=\"context_before\">\n%s\n</chunk>\n", prevChunk.ChunkIndex+1, prevContent)
- appendFormattedChunk(prevChunk, prevContent)
- outputtedIndices[prevChunk.ChunkIndex] = true
- }
- }
-
- if !outputtedIndices[c.ChunkIndex] {
- matchAttr := ""
- if re != nil {
- matchAttr = ` type="match"`
- }
- fmt.Fprintf(&chunksOutput, "<chunk index=\"%d\"%s>\n%s\n</chunk>\n", c.ChunkIndex+1, matchAttr, chunkContent)
- appendFormattedChunk(c, chunkContent)
- outputtedIndices[c.ChunkIndex] = true
- }
-
- if re != nil {
- forceOutputNext = true
- }
- } else if forceOutputNext {
- if !outputtedIndices[c.ChunkIndex] {
- fmt.Fprintf(&chunksOutput, "<chunk index=\"%d\" type=\"context_after\">\n%s\n</chunk>\n", c.ChunkIndex+1, chunkContent)
- appendFormattedChunk(c, chunkContent)
- outputtedIndices[c.ChunkIndex] = true
- }
- forceOutputNext = false
- }
-
- prevChunk = c
-
- if re == nil && matchCount >= 10 {
- break
- }
- if re != nil && matchCount >= 20 {
- break
- }
- }
-
- if hasRange && reachedMax {
- break
- }
-
- if !hasRange {
- if re == nil && matchCount >= 10 {
- break
- }
- if re != nil && matchCount >= 20 {
- reachedMax = true
- break
- }
- }
-
- if int64(page*pageSize) >= total {
- break
- }
- page++
- }
-
- sb.WriteString(fmt.Sprintf("<total_chunks>%d</total_chunks>\n</metadata>\n", totalChunks))
-
- if matchCount > 0 {
- sb.WriteString(fmt.Sprintf("<chunks count=\"%d\">\n", matchCount))
- sb.WriteString(chunksOutput.String())
- sb.WriteString("</chunks>\n")
- } else {
- sb.WriteString("<chunks count=\"0\" />\n")
- }
-
- if reachedMax {
- sb.WriteString("<message>Reached maximum limit for fetching chunks in a single call. Please refine your query or range if needed.</message>\n")
- } else if matchCount == 0 {
- if hasRange {
- sb.WriteString("<message>No chunks found in the specified range.</message>\n")
- } else if re != nil {
- sb.WriteString("<message>No chunks matched your query in this document.</message>\n")
- } else {
- sb.WriteString("<message>Document has no text chunks available.</message>\n")
- }
- } else if !hasRange && re == nil {
- sb.WriteString("<message>No query or range provided. Showing the first 10 chunks as a preview.</message>\n")
- }
-
- sb.WriteString("</source_document>")
-
- return &types.ToolResult{
- Success: true,
- Output: sb.String(),
- Data: map[string]interface{}{
- "display_type": "knowledge_chunks_list",
- "knowledge_id": knowledgeID,
- "knowledge_title": knowledge.Title,
- "total_chunks": totalChunks,
- "fetched_chunks": len(formattedChunks),
- "chunks": formattedChunks,
- },
- }, nil
-}
diff --git a/internal/agent/tools/wiki_tools.go b/internal/agent/tools/wiki_tools.go
index f7cedefad..400be4c25 100644
--- a/internal/agent/tools/wiki_tools.go
+++ b/internal/agent/tools/wiki_tools.go
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"regexp"
+ "sort"
"strings"
"sync"
"unicode/utf8"
@@ -749,34 +750,36 @@ func NewWikiSearchTool(
return &wikiSearchTool{
BaseTool: NewBaseTool(
ToolWikiSearch,
- `Search wiki pages using PostgreSQL POSIX regular expressions (~* operator, case-insensitive).
-STRONGLY PREFER using regex to search for multiple concepts at once rather than simple plain text queries.
-Returns matching pages with titles, slugs, and summaries (each tagged with its short bN knowledge_base_id).
-Examples:
-- Alternation (RECOMMENDED): "stardust|skyvault" (matches either word)
-- Multiple terms (RECOMMENDED): "psionic.*engine" (matches both words in order)
-- Prefix matching: "^entity/.*" (finds all entities)
-- Plain text: "engine" (matches anywhere in title/content/slug/summary)
-IMPORTANT — JSON escaping: every backslash in a regex MUST be written as \\ inside the JSON tool arguments (e.g. to search for literal "C++" write "C\\+\\+", NOT "C\+\+"; for "\d+" write "\\d+"). Plain "\+" / "\d" etc. are invalid JSON escapes and will fail to parse.
-Use this to find relevant wiki pages when you don't know the exact slug.`,
+ "Search the wiki pages of the knowledge bases in scope by title, slug, alias, summary and content. "+
+ "Returns page slugs with summaries; read a page with wiki_read_page.\n"+
+ "query is a case-insensitive POSIX regular expression (for example \"stardust|skyvault\" matches "+
+ "either term); text that is not a valid regular expression, such as \"C++\", is matched literally. "+
+ "Set regex=false to force a literal match. Pass knowledge_base_ids to restrict the search.",
json.RawMessage(`{
"type": "object",
"properties": {
- "queries": {
+ "query": {
+ "type": "string",
+ "description": "Case-insensitive POSIX regular expression; invalid patterns are matched literally",
+ "minLength": 1
+ },
+ "regex": {
+ "type": "boolean",
+ "description": "false forces a literal match; true requires a valid regular expression (default: auto)"
+ },
+ "knowledge_base_ids": {
"type": "array",
"items": { "type": "string" },
- "description": "List of regex search queries to run"
+ "description": "Optional bN knowledge-base handles to restrict the search"
},
"limit": {
"type": "integer",
- "description": "Max results to return per query (default 10)"
- },
- "knowledge_base_id": {
- "type": "string",
- "description": "Optional: restrict search to a single short bN knowledge base ID in scope."
+ "description": "Maximum pages to return per knowledge base (default 10, max 50)",
+ "minimum": 1,
+ "maximum": 50
}
},
- "required": ["queries"]
+ "required": ["query"]
}`),
),
wikiService: wikiService,
@@ -789,41 +792,59 @@ Use this to find relevant wiki pages when you don't know the exact slug.`,
func (t *wikiSearchTool) Execute(ctx context.Context, args json.RawMessage) (*types.ToolResult, error) {
var params struct {
- Query any `json:"query"`
- Queries any `json:"queries"`
- Limit int `json:"limit"`
- KnowledgeBaseID string `json:"knowledge_base_id"`
+ Query any `json:"query"`
+ Queries any `json:"queries"` // legacy alias
+ Regex *bool `json:"regex"`
+ Limit int `json:"limit"`
+ KnowledgeBaseIDs []string `json:"knowledge_base_ids"`
+ KnowledgeBaseID string `json:"knowledge_base_id"` // legacy alias
}
if err := json.Unmarshal(args, &params); err != nil {
return &types.ToolResult{Success: false, Error: "Invalid parameters: " + err.Error()}, nil
}
var queriesToRun []string
- queriesToRun = append(queriesToRun, parseStringOrArray(params.Queries)...)
queriesToRun = append(queriesToRun, parseStringOrArray(params.Query)...)
+ queriesToRun = append(queriesToRun, parseStringOrArray(params.Queries)...)
+ queriesToRun = dedupNonEmptyStrings(queriesToRun)
if len(queriesToRun) == 0 {
- return &types.ToolResult{Success: false, Error: "Missing 'queries' parameter"}, nil
+ return &types.ToolResult{Success: false, Error: "Missing 'query' parameter"}, nil
}
if params.Limit <= 0 {
params.Limit = 10
}
+ if params.Limit > 50 {
+ params.Limit = 50
+ }
- // Restrict scopes by knowledge_base_id arg if provided.
+ // Restrict scopes by knowledge_base_ids if provided.
+ requestedKBs := dedupNonEmptyStrings(append(append([]string{}, params.KnowledgeBaseIDs...), params.KnowledgeBaseID))
effectiveScopes := t.scopes
- if params.KnowledgeBaseID != "" {
- filtered := make([]WikiScope, 0, 1)
+ if len(requestedKBs) > 0 {
+ wanted := make(map[string]bool, len(requestedKBs))
+ for _, kbID := range requestedKBs {
+ wanted[kbID] = true
+ }
+ filtered := make([]WikiScope, 0, len(requestedKBs))
for _, sc := range t.scopes {
- if sc.KnowledgeBaseID == params.KnowledgeBaseID {
+ if wanted[sc.KnowledgeBaseID] {
filtered = append(filtered, sc)
- break
+ delete(wanted, sc.KnowledgeBaseID)
}
}
- if len(filtered) == 0 {
+ if len(wanted) > 0 || len(filtered) == 0 {
+ missing := make([]string, 0, len(wanted))
+ for kbID := range wanted {
+ missing = append(missing, kbID)
+ }
+ sort.Strings(missing)
return &types.ToolResult{
Success: false,
- Error: "knowledge_base_id is not within the current wiki scope",
+ Error: fmt.Sprintf(
+ "knowledge base %s is not within the current wiki scope", strings.Join(missing, ", "),
+ ),
}, nil
}
effectiveScopes = filtered
@@ -848,12 +869,16 @@ func (t *wikiSearchTool) Execute(ctx context.Context, args json.RawMessage) (*ty
for _, query := range queriesToRun {
var allHits []searchHit
filteredCount := 0
+ pattern, perr := wikiSearchPattern(query, params.Regex)
+ if perr != nil {
+ return &types.ToolResult{Success: false, Error: perr.Error()}, nil
+ }
for _, sc := range effectiveScopes {
kbID := sc.KnowledgeBaseID
if kbID == "" {
continue
}
- pages, err := t.wikiService.SearchPages(ctx, kbID, query, params.Limit)
+ pages, err := t.wikiService.SearchPages(ctx, kbID, pattern, params.Limit)
if err != nil {
searchErrors = append(searchErrors, fmt.Sprintf("Wiki search %q failed in KB %s: %v", query, kbID, err))
continue
@@ -907,7 +932,7 @@ func (t *wikiSearchTool) Execute(ctx context.Context, args json.RawMessage) (*ty
t.seenSlugs[key] = true
t.mu.Unlock()
- snippet := extractSnippet(p.Content, query)
+ snippet := extractSnippet(p.Content, pattern)
snippetTag := ""
if snippet != "" {
snippetTag = fmt.Sprintf("\n<match_snippet>%s</match_snippet>", snippet)
@@ -947,6 +972,25 @@ func (t *wikiSearchTool) Execute(ctx context.Context, args json.RawMessage) (*ty
}, nil
}
+// wikiSearchPattern turns the query into the POSIX pattern the repository
+// runs with ~*. Without an explicit regex flag the query keeps its historical
+// meaning as a regular expression, and only text that does not compile (for
+// example "C++") is escaped and matched literally, so every query that worked
+// before still behaves the same. regex=false always escapes; regex=true
+// rejects an invalid pattern instead of silently changing its meaning.
+func wikiSearchPattern(query string, regex *bool) (string, error) {
+ if regex != nil && !*regex {
+ return regexp.QuoteMeta(query), nil
+ }
+ if _, err := regexp.Compile("(?i)" + query); err != nil {
+ if regex != nil && *regex {
+ return "", fmt.Errorf("invalid regular expression %q: %v", query, err)
+ }
+ return regexp.QuoteMeta(query), nil
+ }
+ return query, nil
+}
+
// --- Helper ---
func truncateForSummary(content string, maxLen int) string {
diff --git a/internal/agent/tools/wiki_tools_test.go b/internal/agent/tools/wiki_tools_test.go
index 998a3817a..512aea3af 100644
--- a/internal/agent/tools/wiki_tools_test.go
+++ b/internal/agent/tools/wiki_tools_test.go
@@ -24,6 +24,7 @@ type fakeWikiPageService struct {
searchErrors map[string]error
getCalls []string
searchCalls []string
+ searchQueries map[string][]string
}
func wikiPageKey(kbID, slug string) string {
@@ -40,10 +41,13 @@ func (f *fakeWikiPageService) GetPageBySlug(_ context.Context, kbID, slug string
return f.pages[wikiPageKey(kbID, slug)], nil
}
-func (f *fakeWikiPageService) SearchPages(_ context.Context, kbID, _ string, _ int) ([]*types.WikiPage, error) {
+func (f *fakeWikiPageService) SearchPages(_ context.Context, kbID, query string, _ int) ([]*types.WikiPage, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.searchCalls = append(f.searchCalls, kbID)
+ if f.searchQueries != nil {
+ f.searchQueries[kbID] = append(f.searchQueries[kbID], query)
+ }
if err := f.searchErrors[kbID]; err != nil {
return nil, err
}
@@ -262,7 +266,7 @@ func TestWikiSearchSharesRoutesWithWikiReadPage(t *testing.T) {
searchTool := NewWikiSearchTool(service, nil, scopes, routes)
readTool := NewWikiReadPageTool(service, nil, scopes, routes)
- searchResult, err := searchTool.Execute(context.Background(), json.RawMessage(`{"queries":["target"]}`))
+ searchResult, err := searchTool.Execute(context.Background(), json.RawMessage(`{"query":"target"}`))
if err != nil || searchResult == nil || !searchResult.Success {
t.Fatalf("wiki_search failed: result=%+v err=%v", searchResult, err)
}
@@ -331,7 +335,9 @@ func TestWikiSearchRejectsKnowledgeBaseOutsideScope(t *testing.T) {
service, nil, NewWikiScopesFromKBIDs([]string{"kb-1"}), NewWikiRouteResolver(),
)
- result, err := tool.Execute(context.Background(), json.RawMessage(`{"queries":["target"],"knowledge_base_id":"kb-outside"}`))
+ result, err := tool.Execute(
+ context.Background(), json.RawMessage(`{"query":"target","knowledge_base_ids":["kb-outside"]}`),
+ )
if err != nil {
t.Fatalf("wiki_search returned unexpected error: %v", err)
}
@@ -366,7 +372,7 @@ func TestWikiSearchFailsWhenEveryAllowedBackendLookupFails(t *testing.T) {
tool := NewWikiSearchTool(
service, nil, NewWikiScopesFromKBIDs([]string{"kb-1"}), NewWikiRouteResolver(),
)
- result, err := tool.Execute(context.Background(), json.RawMessage(`{"queries":["target"]}`))
+ result, err := tool.Execute(context.Background(), json.RawMessage(`{"query":"target"}`))
if err != nil || result == nil || result.Success || !strings.Contains(result.Error, "search backend unavailable") {
t.Fatalf("search backend failure must be preserved: result=%+v err=%v", result, err)
}
@@ -449,3 +455,52 @@ func TestResolveWikiCreateKBRequiresUnambiguousServerContext(t *testing.T) {
t.Fatalf("cached provenance should choose kb-2: kb=%s err=%v", kbID, err)
}
}
+
+// wiki_search keeps its historical regular-expression semantics by default;
+// only text that does not compile falls back to a literal match.
+func TestWikiSearchKeepsRegexDefaultWithLiteralFallback(t *testing.T) {
+ page := newTestWikiPage("kb-1", "concept/cpp")
+ service := &fakeWikiPageService{
+ searchResults: map[string][]*types.WikiPage{"kb-1": {page}},
+ }
+ service.searchQueries = map[string][]string{}
+ tool := NewWikiSearchTool(service, nil, NewWikiScopesFromKBIDs([]string{"kb-1"}), NewWikiRouteResolver())
+
+ for _, raw := range []string{
+ `{"query":"stardust|skyvault"}`, // default: regex kept as-is
+ `{"query":"C++ v1.2"}`, // default: invalid regex, matched literally
+ `{"query":"a.b","regex":false}`, // explicit literal
+ `{"query":"^entity/.*","regex":true}`, // explicit regex
+ } {
+ if res, err := tool.Execute(context.Background(), json.RawMessage(raw)); err != nil || !res.Success {
+ t.Fatalf("%s: res=%+v err=%v", raw, res, err)
+ }
+ }
+ got := service.searchQueries["kb-1"]
+ want := []string{`stardust|skyvault`, `C\+\+ v1\.2`, `a\.b`, `^entity/.*`}
+ if !reflect.DeepEqual(got, want) {
+ t.Fatalf("backend patterns = %v, want %v", got, want)
+ }
+
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"query":"C++","regex":true}`))
+ if err != nil || res.Success || !strings.Contains(res.Error, "invalid regular expression") {
+ t.Fatalf("regex=true must reject an invalid pattern: res=%+v err=%v", res, err)
+ }
+}
+
+func TestWikiSearchAcceptsLegacyArrayAndSingleKBArguments(t *testing.T) {
+ page := newTestWikiPage("kb-2", "concept/legacy")
+ service := &fakeWikiPageService{searchResults: map[string][]*types.WikiPage{"kb-2": {page}}}
+ tool := NewWikiSearchTool(service, nil, NewWikiScopesFromKBIDs([]string{"kb-1", "kb-2"}), NewWikiRouteResolver())
+
+ res, err := tool.Execute(context.Background(), json.RawMessage(`{"queries":["legacy"],"knowledge_base_id":"kb-2"}`))
+ if err != nil || res == nil || !res.Success {
+ t.Fatalf("legacy arguments: res=%+v err=%v", res, err)
+ }
+ if !reflect.DeepEqual(service.searchCalls, []string{"kb-2"}) {
+ t.Fatalf("legacy knowledge_base_id must narrow the scope, calls=%v", service.searchCalls)
+ }
+ if !strings.Contains(res.Output, "[[concept/legacy|concept/legacy]]") {
+ t.Fatalf("output = %s", res.Output)
+ }
+}
diff --git a/internal/application/repository/chunk.go b/internal/application/repository/chunk.go
index 78eb77471..943bcb069 100644
--- a/internal/application/repository/chunk.go
+++ b/internal/application/repository/chunk.go
@@ -323,6 +323,42 @@ func (r *chunkRepository) ListChunkByParentID(
return chunks, nil
}
+// ListChunkNeighbors implements interfaces.ChunkRepository.
+func (r *chunkRepository) ListChunkNeighbors(
+ ctx context.Context,
+ tenantID uint64,
+ knowledgeID string,
+ chunkIndex int,
+ before int,
+ after int,
+ chunkTypes []types.ChunkType,
+) ([]*types.Chunk, error) {
+ base := func() *gorm.DB {
+ return r.db.WithContext(ctx).
+ Where("tenant_id = ? AND knowledge_id = ? AND chunk_type IN (?) AND status in (?) AND is_enabled = ?",
+ tenantID, knowledgeID, chunkTypes,
+ []int{int(types.ChunkStatusIndexed), int(types.ChunkStatusDefault)}, true)
+ }
+ var preceding, following []*types.Chunk
+ if before > 0 {
+ if err := base().Where("chunk_index < ?", chunkIndex).
+ Order("chunk_index DESC").Limit(before).Find(&preceding).Error; err != nil {
+ return nil, err
+ }
+ }
+ if after > 0 {
+ if err := base().Where("chunk_index > ?", chunkIndex).
+ Order("chunk_index ASC").Limit(after).Find(&following).Error; err != nil {
+ return nil, err
+ }
+ }
+ out := make([]*types.Chunk, 0, len(preceding)+len(following))
+ for i := len(preceding) - 1; i >= 0; i-- {
+ out = append(out, preceding[i])
+ }
+ return append(out, following...), nil
+}
+
func (r *chunkRepository) ListChunksByParentIDs(
ctx context.Context,
tenantID uint64,
diff --git a/internal/application/service/agent_browser_source_test.go b/internal/application/service/agent_browser_source_test.go
index 03186e210..21e9fe08d 100644
--- a/internal/application/service/agent_browser_source_test.go
+++ b/internal/application/service/agent_browser_source_test.go
@@ -41,7 +41,7 @@ func TestBrowserSourceKeepsOtherConfiguredTools(t *testing.T) {
SandboxConfigID: "sandbox", MCPSelectionMode: "all", KnowledgeBases: []string{"kb"},
SearchTargets: types.SearchTargets{&types.SearchTarget{KnowledgeBaseID: "kb", TenantID: 7}},
SkillDirs: []string{t.TempDir()},
- AllowedTools: []string{tools.ToolKnowledgeSearch, tools.ToolShellExec, tools.ToolThinking},
+ AllowedTools: []string{tools.ToolSearchKnowledge, tools.ToolShellExec, tools.ToolThinking},
}
engine, err := svc.CreateAgentEngine(ctx, cfg, model, nil, nil, "session", "message")
require.NoError(t, err)
@@ -49,7 +49,7 @@ func TestBrowserSourceKeepsOtherConfiguredTools(t *testing.T) {
_, err = engine.Execute(ctx, "session", "message", "use my browser and other sources", nil)
require.NoError(t, err)
for _, name := range []string{
- tools.ToolKnowledgeSearch, tools.ToolWebSearch, tools.ToolWebFetch, tools.ToolShellExec,
+ tools.ToolSearchKnowledge, tools.ToolWebSearch, tools.ToolWebFetch, tools.ToolShellExec,
} {
require.Contains(t, model.lastToolNames, name)
}
diff --git a/internal/application/service/agent_history_test.go b/internal/application/service/agent_history_test.go
index 0e5c463d5..1c0451bcf 100644
--- a/internal/application/service/agent_history_test.go
+++ b/internal/application/service/agent_history_test.go
@@ -153,7 +153,7 @@ func TestBuildAssistantHistoryMessages_ToolCallsExpandIntoOpenAIShape(t *testing
ToolCalls: []types.ToolCall{
{
ID: "call_1",
- Name: agenttools.ToolKnowledgeSearch,
+ Name: agenttools.ToolSearchKnowledge,
Args: map[string]interface{}{"query": "foo"},
Result: &types.ToolResult{
Success: true,
@@ -187,7 +187,7 @@ func TestBuildAssistantHistoryMessages_ToolCallsExpandIntoOpenAIShape(t *testing
assert.Equal(t, "Let me search.", got[0].Content)
if assert.Len(t, got[0].ToolCalls, 1) {
assert.Equal(t, "call_1", got[0].ToolCalls[0].ID)
- assert.Equal(t, agenttools.ToolKnowledgeSearch, got[0].ToolCalls[0].Function.Name)
+ assert.Equal(t, agenttools.ToolSearchKnowledge, got[0].ToolCalls[0].Function.Name)
assert.Contains(t, got[0].ToolCalls[0].Function.Arguments, "foo")
}
// 2. tool result paired with the call ID
@@ -214,7 +214,7 @@ func TestBuildAssistantHistoryMessages_SkipsPipelineTimelineToolCalls(t *testing
ToolCalls: []types.ToolCall{
{
ID: types.PipelineToolCallIDPrefix + "abc",
- Name: agenttools.ToolKnowledgeSearch,
+ Name: agenttools.ToolSearchKnowledge,
Args: map[string]interface{}{"query": "你好"},
Result: &types.ToolResult{Success: true, Output: "未检索到相关内容"},
},
@@ -245,7 +245,7 @@ func TestBuildAssistantHistoryMessages_ToolFailureSurfacesAsError(t *testing.T)
ToolCalls: []types.ToolCall{
{
ID: "call_err",
- Name: agenttools.ToolKnowledgeSearch,
+ Name: agenttools.ToolSearchKnowledge,
Args: map[string]interface{}{"query": "x"},
Result: &types.ToolResult{
Success: false,
@@ -264,7 +264,7 @@ func TestBuildAssistantHistoryMessages_ToolFailureSurfacesAsError(t *testing.T)
Role: "tool",
Content: "Error: kb unreachable",
ToolCallID: "call_err",
- Name: agenttools.ToolKnowledgeSearch,
+ Name: agenttools.ToolSearchKnowledge,
}, got[1])
}
@@ -323,7 +323,7 @@ func TestBuildTurnBodyMessages_KeepsMidRunUsersInPlace(t *testing.T) {
Timestamp: firstStep,
ToolCalls: []types.ToolCall{{
ID: "call_a",
- Name: agenttools.ToolKnowledgeSearch,
+ Name: agenttools.ToolSearchKnowledge,
Args: map[string]interface{}{"query": "A"},
Result: &types.ToolResult{Success: true, Output: "found A"},
}},
@@ -334,7 +334,7 @@ func TestBuildTurnBodyMessages_KeepsMidRunUsersInPlace(t *testing.T) {
Timestamp: secondStep,
ToolCalls: []types.ToolCall{{
ID: "call_b",
- Name: agenttools.ToolKnowledgeSearch,
+ Name: agenttools.ToolSearchKnowledge,
Args: map[string]interface{}{"query": "B"},
Result: &types.ToolResult{Success: true, Output: "found B"},
}},
@@ -377,7 +377,7 @@ func TestBuildTurnBodyMessages_LateMidRunUserPrecedesAnswer(t *testing.T) {
Timestamp: step,
ToolCalls: []types.ToolCall{{
ID: "call_a",
- Name: agenttools.ToolKnowledgeSearch,
+ Name: agenttools.ToolSearchKnowledge,
Result: &types.ToolResult{Success: true, Output: "found"},
}},
}},
@@ -398,13 +398,13 @@ func TestBuildTurnBodyMessages_LateMidRunUserPrecedesAnswer(t *testing.T) {
// dropped — every other tool (KB search, web search, MCP tools…) must survive.
func TestFilterNonTerminalToolCalls(t *testing.T) {
in := []types.ToolCall{
- {Name: agenttools.ToolKnowledgeSearch},
+ {Name: agenttools.ToolSearchKnowledge},
{Name: "final_answer"},
{Name: agenttools.ToolWebSearch},
}
out := filterNonTerminalToolCalls(in)
if assert.Len(t, out, 2) {
- assert.Equal(t, agenttools.ToolKnowledgeSearch, out[0].Name)
+ assert.Equal(t, agenttools.ToolSearchKnowledge, out[0].Name)
assert.Equal(t, agenttools.ToolWebSearch, out[1].Name)
}
}
@@ -424,7 +424,7 @@ func TestBuildAssistantHistoryMessages_ReplaysReasoningContent(t *testing.T) {
ReasoningContent: "model's chain of thought",
ToolCalls: []types.ToolCall{{
ID: "call_1",
- Name: agenttools.ToolKnowledgeSearch,
+ Name: agenttools.ToolSearchKnowledge,
Args: map[string]interface{}{"query": "foo"},
ProviderMetadata: types.ToolCallMetadata{"google": json.RawMessage(`{"thought_signature":"gemini-history-signature"}`)},
Result: &types.ToolResult{
diff --git a/internal/application/service/agent_rerank_requirement_test.go b/internal/application/service/agent_rerank_requirement_test.go
index ffa68e122..06d096156 100644
--- a/internal/application/service/agent_rerank_requirement_test.go
+++ b/internal/application/service/agent_rerank_requirement_test.go
@@ -18,7 +18,7 @@ func TestAgentRequiresRerankModel(t *testing.T) {
name: "knowledge search with all knowledge bases",
agent: &types.CustomAgent{Config: types.CustomAgentConfig{
KBSelectionMode: "all",
- AllowedTools: []string{tools.ToolKnowledgeSearch},
+ AllowedTools: []string{tools.ToolSearchKnowledge},
}},
want: true,
},
@@ -26,7 +26,7 @@ func TestAgentRequiresRerankModel(t *testing.T) {
name: "knowledge search with selected knowledge bases",
agent: &types.CustomAgent{Config: types.CustomAgentConfig{
KBSelectionMode: "selected",
- AllowedTools: []string{tools.ToolKnowledgeSearch},
+ AllowedTools: []string{tools.ToolSearchKnowledge},
}},
want: true,
},
@@ -34,7 +34,7 @@ func TestAgentRequiresRerankModel(t *testing.T) {
name: "knowledge search with knowledge bases disabled",
agent: &types.CustomAgent{Config: types.CustomAgentConfig{
KBSelectionMode: "none",
- AllowedTools: []string{tools.ToolKnowledgeSearch},
+ AllowedTools: []string{tools.ToolSearchKnowledge},
}},
want: false,
},
diff --git a/internal/application/service/agent_service.go b/internal/application/service/agent_service.go
index 51b2f109f..a59ccb0f4 100644
--- a/internal/application/service/agent_service.go
+++ b/internal/application/service/agent_service.go
@@ -853,8 +853,9 @@ func (s *agentService) registerTools(
// - Legacy agents without AllowedTools fall back to DefaultAllowedTools().
var allowedTools []string
if len(config.AllowedTools) > 0 {
- allowedTools = make([]string, len(config.AllowedTools))
- copy(allowedTools, config.AllowedTools)
+ // Retired retrieval tool names in stored configs map onto their
+ // successors here, so no data migration is needed.
+ allowedTools = tools.NormalizeAllowedTools(config.AllowedTools)
logger.Infof(ctx, "Using custom allowed tools from config: %v", allowedTools)
} else {
allowedTools = tools.DefaultAllowedTools()
@@ -866,6 +867,7 @@ func (s *agentService) registerTools(
// ---- Capability detection from SearchTargets ----
var hasVectorKB bool
+ var hasGraphKB bool
var wikiKBIDs []string
wikiRoutes := tools.NewWikiRouteResolver()
for _, target := range config.SearchTargets {
@@ -879,6 +881,9 @@ func (s *agentService) registerTools(
if kb.IsVectorEnabled() || kb.IsKeywordEnabled() {
hasVectorKB = true
}
+ if kb.IsGraphEnabled() {
+ hasGraphKB = true
+ }
if kb.IsWikiEnabled() {
wikiKBIDs = append(wikiKBIDs, kb.ID)
}
@@ -900,25 +905,23 @@ func (s *agentService) registerTools(
if !hasKnowledge {
filteredTools := make([]string, 0)
kbTools := map[string]bool{
- tools.ToolKnowledgeSearch: true,
- tools.ToolGrepChunks: true,
- tools.ToolListKnowledgeChunks: true,
+ tools.ToolSearchKnowledge: true,
+ tools.ToolReadDocument: true,
+ tools.ToolListDocuments: true,
tools.ToolQueryKnowledgeGraph: true,
- tools.ToolGetDocumentInfo: true,
tools.ToolDatabaseQuery: true,
tools.ToolDataAnalysis: true,
tools.ToolDataSchema: true,
// Wiki tools also require at least one KB in scope.
- tools.ToolWikiReadPage: true,
- tools.ToolWikiSearch: true,
- tools.ToolWikiReadSourceDoc: true,
- tools.ToolWikiFlagIssue: true,
- tools.ToolWikiWritePage: true,
- tools.ToolWikiReplaceText: true,
- tools.ToolWikiRenamePage: true,
- tools.ToolWikiDeletePage: true,
- tools.ToolWikiReadIssue: true,
- tools.ToolWikiUpdateIssue: true,
+ tools.ToolWikiReadPage: true,
+ tools.ToolWikiSearch: true,
+ tools.ToolWikiFlagIssue: true,
+ tools.ToolWikiWritePage: true,
+ tools.ToolWikiReplaceText: true,
+ tools.ToolWikiRenamePage: true,
+ tools.ToolWikiDeletePage: true,
+ tools.ToolWikiReadIssue: true,
+ tools.ToolWikiUpdateIssue: true,
}
// If no knowledge and no web search, also disable todo_write (not useful for simple chat)
@@ -968,24 +971,27 @@ func (s *agentService) registerTools(
// in AgentEditorModal.vue. These are *all* tools that retrieve/inspect
// content from RAG-style knowledge bases.
ragToolSet := map[string]bool{
- tools.ToolKnowledgeSearch: true,
- tools.ToolGrepChunks: true,
- tools.ToolListKnowledgeChunks: true,
+ tools.ToolSearchKnowledge: true,
tools.ToolQueryKnowledgeGraph: true,
- tools.ToolGetDocumentInfo: true,
tools.ToolDatabaseQuery: true,
}
+ // Document readers work on stored chunks, which every KB writes whatever
+ // its indexing strategy, so a wiki-only scope keeps them: they are how a
+ // wiki reader checks the source documents a page cites.
+ documentToolSet := map[string]bool{
+ tools.ToolReadDocument: true,
+ tools.ToolListDocuments: true,
+ }
allWikiToolSet := map[string]bool{
- tools.ToolWikiReadPage: true,
- tools.ToolWikiSearch: true,
- tools.ToolWikiReadSourceDoc: true,
- tools.ToolWikiFlagIssue: true,
- tools.ToolWikiWritePage: true,
- tools.ToolWikiReplaceText: true,
- tools.ToolWikiRenamePage: true,
- tools.ToolWikiDeletePage: true,
- tools.ToolWikiReadIssue: true,
- tools.ToolWikiUpdateIssue: true,
+ tools.ToolWikiReadPage: true,
+ tools.ToolWikiSearch: true,
+ tools.ToolWikiFlagIssue: true,
+ tools.ToolWikiWritePage: true,
+ tools.ToolWikiReplaceText: true,
+ tools.ToolWikiRenamePage: true,
+ tools.ToolWikiDeletePage: true,
+ tools.ToolWikiReadIssue: true,
+ tools.ToolWikiUpdateIssue: true,
}
// Hard safety nets: drop tools whose runtime prerequisite is missing.
@@ -1010,7 +1016,7 @@ func (s *agentService) registerTools(
filtered := make([]string, 0, len(allowedTools))
dropped := make([]string, 0)
for _, t := range allowedTools {
- if ragToolSet[t] {
+ if ragToolSet[t] || (documentToolSet[t] && !hasWikiKB) {
dropped = append(dropped, t)
continue
}
@@ -1021,6 +1027,15 @@ func (s *agentService) registerTools(
logger.Warnf(ctx, "Dropped RAG tools %v because no RAG-capable KB is in scope", dropped)
}
}
+ // The graph tool only answers on graph-enabled bases; offering it on a
+ // scope without one produced degraded plain-search results and a tool
+ // the model kept trying. It follows the graph capability instead.
+ if !hasGraphKB {
+ if trimmed := withoutString(allowedTools, tools.ToolQueryKnowledgeGraph); len(trimmed) != len(allowedTools) {
+ allowedTools = trimmed
+ logger.Infof(ctx, "Dropped query_knowledge_graph because no graph-enabled KB is in scope")
+ }
+ }
// Deduplicate while preserving original order.
allowedTools = dedupStrings(allowedTools)
@@ -1035,8 +1050,8 @@ func (s *agentService) registerTools(
toolToRegister = tools.NewSequentialThinkingTool()
case tools.ToolTodoWrite:
toolToRegister = tools.NewTodoWriteTool()
- case tools.ToolKnowledgeSearch:
- toolToRegister = tools.NewKnowledgeSearchTool(
+ case tools.ToolSearchKnowledge:
+ toolToRegister = tools.NewSearchKnowledgeTool(
s.knowledgeBaseService,
s.knowledgeService,
s.chunkService,
@@ -1044,16 +1059,13 @@ func (s *agentService) registerTools(
rerankModel,
s.cfg,
)
- case tools.ToolGrepChunks:
- toolToRegister = tools.NewGrepChunksTool(s.db, config.SearchTargets)
- logger.Infof(ctx, "Registered grep_chunks tool with searchTargets: %d targets", len(config.SearchTargets))
- case tools.ToolListKnowledgeChunks:
- toolToRegister = tools.NewListKnowledgeChunksTool(s.knowledgeService, s.chunkService, config.SearchTargets)
+ case tools.ToolReadDocument:
+ toolToRegister = tools.NewReadDocumentTool(s.knowledgeService, s.chunkService, config.SearchTargets)
+ case tools.ToolListDocuments:
+ toolToRegister = tools.NewListDocumentsTool(s.knowledgeService, config.SearchTargets)
case tools.ToolQueryKnowledgeGraph:
toolToRegister = tools.NewQueryKnowledgeGraphTool(s.knowledgeBaseService, config.SearchTargets).
WithKnowledgeScope(s.knowledgeService)
- case tools.ToolGetDocumentInfo:
- toolToRegister = tools.NewGetDocumentInfoTool(s.knowledgeService, s.chunkService, config.SearchTargets)
case tools.ToolSearchConversations:
// The owner is captured from the caller's identity here, not read
// from the model's arguments, so no prompt can redirect the search
@@ -1095,8 +1107,6 @@ func (s *agentService) registerTools(
toolToRegister = tools.NewWikiReadPageTool(s.wikiPageService, s.knowledgeService, wikiScopes, wikiRoutes)
case tools.ToolWikiSearch:
toolToRegister = tools.NewWikiSearchTool(s.wikiPageService, s.knowledgeService, wikiScopes, wikiRoutes)
- case tools.ToolWikiReadSourceDoc:
- toolToRegister = tools.NewWikiReadSourceDocTool(s.knowledgeService, s.chunkService, config.SearchTargets)
case tools.ToolWikiFlagIssue:
toolToRegister = tools.NewWikiFlagIssueTool(s.wikiPageService, wikiKBIDs, wikiRoutes).
WithKnowledgeScope(s.knowledgeService, config.SearchTargets)
diff --git a/internal/application/service/agent_share.go b/internal/application/service/agent_share.go
index 9e30ed295..6284689d2 100644
--- a/internal/application/service/agent_share.go
+++ b/internal/application/service/agent_share.go
@@ -20,22 +20,25 @@ var (
ErrAgentNotFoundForShare = errors.New("agent not found")
ErrNotAgentOwner = errors.New("only agent owner can share")
ErrOrgRoleCannotShareAgent = errors.New("only editors and admins can share agents to this organization")
- ErrAgentNotConfigured = errors.New("agent is not fully configured (missing required chat model, or rerank model when the knowledge_search tool is enabled)")
+ ErrAgentNotConfigured = errors.New(
+ "agent is not fully configured (missing required chat model, or rerank model when the search_knowledge " +
+ "tool is enabled)",
+ )
)
// agentRequiresRerankModel returns true when the agent's configured scope and
// tools will actually invoke the reranker at runtime. An agent whose knowledge
-// base scope is explicitly disabled cannot run knowledge_search, so it does not
+// base scope is explicitly disabled cannot run search_knowledge, so it does not
// need a rerank model even if that tool remains in AllowedTools.
//
// This mirrors the runtime check in session_agent_qa.go: only
-// `knowledge_search` with an enabled knowledge-base scope uses the reranker.
+// `search_knowledge` with an enabled knowledge-base scope uses the reranker.
// Wiki-first agents (wiki_search / wiki_read_page / …) never call it and
// therefore don't need a rerank model configured, even when knowledge bases
// are attached.
//
// When AllowedTools is empty the runtime falls back to
-// tools.DefaultAllowedTools(), which includes knowledge_search, so we treat
+// tools.DefaultAllowedTools(), which includes search_knowledge, so we treat
// that case as requiring the reranker.
func agentRequiresRerankModel(agent *types.CustomAgent) bool {
if agent == nil {
@@ -49,7 +52,7 @@ func agentRequiresRerankModel(agent *types.CustomAgent) bool {
allowed = tools.DefaultAllowedTools()
}
for _, t := range allowed {
- if t == tools.ToolKnowledgeSearch {
+ if tools.SuccessorToolName(t) == tools.ToolSearchKnowledge {
return true
}
}
diff --git a/internal/application/service/agent_share_runtime_test.go b/internal/application/service/agent_share_runtime_test.go
index 582428ede..d8f5f9135 100644
--- a/internal/application/service/agent_share_runtime_test.go
+++ b/internal/application/service/agent_share_runtime_test.go
@@ -93,10 +93,10 @@ func TestFilterSharedAgentWriteTools(t *testing.T) {
func TestFilterSharedAgentWriteToolsCoversAllWikiMutations(t *testing.T) {
readOnlyWikiTools := map[string]bool{
- tools.ToolWikiReadPage: true,
- tools.ToolWikiSearch: true,
- tools.ToolWikiReadSourceDoc: true,
- tools.ToolWikiReadIssue: true,
+ tools.ToolWikiReadPage: true,
+ tools.ToolWikiSearch: true,
+ tools.ToolReadDocument: true,
+ tools.ToolWikiReadIssue: true,
}
for _, definition := range tools.AvailableToolDefinitions() {
diff --git a/internal/application/service/agent_tool_surface_test.go b/internal/application/service/agent_tool_surface_test.go
new file mode 100644
index 000000000..ae87ca55d
--- /dev/null
+++ b/internal/application/service/agent_tool_surface_test.go
@@ -0,0 +1,132 @@
+package service
+
+import (
+ "context"
+ "testing"
+
+ "github.com/Tencent/WeKnora/internal/agent/tools"
+ "github.com/Tencent/WeKnora/internal/types"
+ "github.com/stretchr/testify/require"
+)
+
+type toolSurfaceKBService struct {
+ fakeAgentKnowledgeBaseService
+}
+
+func (s *toolSurfaceKBService) GetKnowledgeBaseByIDOnly(context.Context, string) (*types.KnowledgeBase, error) {
+ return s.kb, nil
+}
+
+func newToolSurfaceService(kb *types.KnowledgeBase) *agentService {
+ return &agentService{
+ knowledgeBaseService: &toolSurfaceKBService{fakeAgentKnowledgeBaseService{kb: kb}},
+ knowledgeService: &fakeAgentKnowledgeService{},
+ }
+}
+
+func toolSurfaceConfig(allowed ...string) *types.AgentConfig {
+ return &types.AgentConfig{
+ KnowledgeBases: []string{"kb"},
+ SearchTargets: types.SearchTargets{&types.SearchTarget{
+ Type: types.SearchTargetTypeKnowledgeBase, KnowledgeBaseID: "kb", TenantID: 7,
+ }},
+ AllowedTools: allowed,
+ }
+}
+
+// A stored allowlist that still names the retired retrieval tools registers
+// their successors, so agents saved before the consolidation keep working
+// without a data migration.
+func TestRegisterToolsMapsRetiredRetrievalToolsToSuccessors(t *testing.T) {
+ kb := &types.KnowledgeBase{ID: "kb"}
+ kb.IndexingStrategy.VectorEnabled = true
+ svc := newToolSurfaceService(kb)
+ registry := tools.NewToolRegistry()
+ cfg := toolSurfaceConfig(
+ tools.LegacyToolKnowledgeSearch, tools.LegacyToolGrepChunks,
+ tools.LegacyToolListKnowledgeChunks, tools.LegacyToolGetDocumentInfo,
+ tools.LegacyToolWikiReadSourceDoc, tools.ToolThinking,
+ )
+ require.NoError(t, svc.registerTools(t.Context(), registry, cfg, nil, nil, "session"))
+
+ names := registry.ListTools()
+ require.Contains(t, names, tools.ToolSearchKnowledge)
+ require.Contains(t, names, tools.ToolReadDocument)
+ require.Contains(t, names, tools.ToolThinking)
+ for _, retired := range []string{
+ tools.LegacyToolKnowledgeSearch, tools.LegacyToolGrepChunks, tools.LegacyToolListKnowledgeChunks,
+ tools.LegacyToolGetDocumentInfo, tools.LegacyToolWikiReadSourceDoc,
+ } {
+ require.NotContains(t, names, retired)
+ }
+ require.Equal(t, 1, countName(names, tools.ToolSearchKnowledge), "duplicates collapse to one registration")
+ require.Equal(t, 1, countName(names, tools.ToolReadDocument))
+}
+
+// The graph tool follows the graph capability of the scope, not the checkbox.
+func TestRegisterToolsOffersGraphToolOnlyWithGraphEnabledKB(t *testing.T) {
+ plain := &types.KnowledgeBase{ID: "kb"}
+ plain.IndexingStrategy.VectorEnabled = true
+ registry := tools.NewToolRegistry()
+ require.NoError(t, newToolSurfaceService(plain).registerTools(
+ t.Context(), registry, toolSurfaceConfig(tools.ToolSearchKnowledge, tools.ToolQueryKnowledgeGraph),
+ nil, nil, "s",
+ ))
+ require.Contains(t, registry.ListTools(), tools.ToolSearchKnowledge)
+ require.NotContains(t, registry.ListTools(), tools.ToolQueryKnowledgeGraph)
+
+ graph := &types.KnowledgeBase{ID: "kb", ExtractConfig: &types.ExtractConfig{Enabled: true}}
+ graph.IndexingStrategy.VectorEnabled = true
+ graph.IndexingStrategy.GraphEnabled = true
+ registry = tools.NewToolRegistry()
+ require.NoError(t, newToolSurfaceService(graph).registerTools(
+ t.Context(), registry, toolSurfaceConfig(tools.ToolSearchKnowledge, tools.ToolQueryKnowledgeGraph),
+ nil, nil, "s",
+ ))
+ require.Contains(t, registry.ListTools(), tools.ToolQueryKnowledgeGraph)
+}
+
+// Default agents get the consolidated read surface.
+func TestRegisterToolsDefaultSurface(t *testing.T) {
+ kb := &types.KnowledgeBase{ID: "kb"}
+ kb.IndexingStrategy.KeywordEnabled = true
+ registry := tools.NewToolRegistry()
+ require.NoError(t, newToolSurfaceService(kb).registerTools(
+ t.Context(), registry, toolSurfaceConfig(), nil, nil, "s",
+ ))
+ names := registry.ListTools()
+ for _, want := range []string{tools.ToolSearchKnowledge, tools.ToolReadDocument, tools.ToolListDocuments} {
+ require.Contains(t, names, want)
+ }
+}
+
+func countName(names []string, want string) int {
+ n := 0
+ for _, name := range names {
+ if name == want {
+ n++
+ }
+ }
+ return n
+}
+
+// A wiki-only knowledge base has no chunk index but still stores chunks, so
+// the document readers the wiki agents use to check sources stay available
+// while chunk search is dropped.
+func TestRegisterToolsKeepsDocumentReadersForWikiOnlyScope(t *testing.T) {
+ kb := &types.KnowledgeBase{ID: "kb"}
+ kb.IndexingStrategy.WikiEnabled = true
+ registry := tools.NewToolRegistry()
+ cfg := toolSurfaceConfig(
+ tools.ToolWikiSearch, tools.ToolWikiReadPage, tools.ToolReadDocument,
+ tools.ToolListDocuments, tools.ToolSearchKnowledge, tools.LegacyToolWikiReadSourceDoc,
+ )
+ require.NoError(t, newToolSurfaceService(kb).registerTools(t.Context(), registry, cfg, nil, nil, "s"))
+ names := registry.ListTools()
+ for _, want := range []string{
+ tools.ToolWikiSearch, tools.ToolWikiReadPage, tools.ToolReadDocument, tools.ToolListDocuments,
+ } {
+ require.Contains(t, names, want)
+ }
+ require.NotContains(t, names, tools.ToolSearchKnowledge, "chunk search needs a vector or keyword index")
+}
diff --git a/internal/application/service/agent_web_tools_test.go b/internal/application/service/agent_web_tools_test.go
index 08831f082..503fb5860 100644
--- a/internal/application/service/agent_web_tools_test.go
+++ b/internal/application/service/agent_web_tools_test.go
@@ -21,7 +21,7 @@ func TestAgentWebToolsFollowRuntimeSwitch(t *testing.T) {
_, err := registry.GetTool(name)
assert.Equal(t, enabled, err == nil, name)
}
- assert.NotContains(t, registry.ListTools(), tools.ToolKnowledgeSearch)
- assert.NotContains(t, registry.ListTools(), tools.ToolGrepChunks)
+ assert.NotContains(t, registry.ListTools(), tools.ToolSearchKnowledge)
+ assert.NotContains(t, registry.ListTools(), tools.ToolReadDocument)
}
}
diff --git a/internal/im/tool_display.go b/internal/im/tool_display.go
index 158015e7e..ff33d989f 100644
--- a/internal/im/tool_display.go
+++ b/internal/im/tool_display.go
@@ -32,6 +32,8 @@ var imToolNameLabels = map[string]string{
"search_knowledge": "知识库检索",
"knowledge_search": "知识库检索",
"grep_chunks": "搜索关键词",
+ "read_document": "阅读文档",
+ "list_documents": "浏览文档列表",
"web_search": "网络搜索",
"web_fetch": "网页抓取",
"get_document_info": "获取文档信息",
@@ -452,11 +454,16 @@ func imToolStatusDescription(step IMToolStep) string {
return "获取文档信息"
}
return "获取文档信息失败"
- case "get_document_content", "wiki_read_source_doc":
+ case "get_document_content", "wiki_read_source_doc", "read_document":
if success {
- return "获取文档内容"
+ return "阅读文档"
}
- return "获取文档内容失败"
+ return "阅读文档失败"
+ case "list_documents":
+ if success {
+ return "浏览文档列表"
+ }
+ return "浏览文档列表失败"
case "thinking":
if success {
return "完成思考"
@@ -502,7 +509,7 @@ func imToolHeaderSummary(step IMToolStep) string {
return fmt.Sprintf("获取文档:%s", strings.TrimSpace(title))
}
}
- case "list_knowledge_chunks":
+ case "list_knowledge_chunks", "read_document":
if data != nil {
if question, ok := data["faq_question"].(string); ok && strings.TrimSpace(question) != "" {
return fmt.Sprintf("查看 FAQ:%s", strings.TrimSpace(question))
@@ -532,8 +539,10 @@ func imToolResultSummary(step IMToolStep) string {
return imWebSearchSummary(step.Data)
case "grep_chunks":
return imGrepSearchSummary(step.Data)
- case "list_knowledge_chunks":
+ case "list_knowledge_chunks", "read_document":
return imKnowledgeChunksSummary(step.Data)
+ case "list_documents":
+ return imDocumentListSummary(step.Data)
default:
return briefToolSummary(step.Output)
}
@@ -670,6 +679,21 @@ func imGrepDocumentCount(data map[string]interface{}) int {
return 0
}
+func imDocumentListSummary(data map[string]interface{}) string {
+ if data == nil {
+ return ""
+ }
+ total := imNumericValue(data["total_docs"])
+ listed := 0
+ if docs, ok := data["documents"].([]interface{}); ok {
+ listed = len(docs)
+ }
+ if total == 0 && listed == 0 {
+ return "知识库中没有文档"
+ }
+ return fmt.Sprintf("列出 %d / %d 个文档", listed, total)
+}
+
func imKnowledgeChunksSummary(data map[string]interface{}) string {
if data == nil {
return ""
diff --git a/internal/mcpserver/grep_pattern_test.go b/internal/mcpserver/grep_pattern_test.go
new file mode 100644
index 000000000..c47dfc191
--- /dev/null
+++ b/internal/mcpserver/grep_pattern_test.go
@@ -0,0 +1,50 @@
+package mcpserver
+
+import "testing"
+
+func TestGrepPatternAndTermsKeepsRegexSemantics(t *testing.T) {
+ cases := []struct {
+ query string
+ terms string
+ matches []string
+ rejects []string
+ }{
+ {
+ query: "stardust|skyvault", terms: "stardust skyvault",
+ matches: []string{"the Stardust drive", "SKYVAULT"}, rejects: []string{"star dust"},
+ },
+ {
+ query: `\bE1001\b`, terms: "E1001",
+ matches: []string{"code E1001 raised"}, rejects: []string{"E10012"},
+ },
+ {
+ query: "psionic.*engine", terms: "psionic engine",
+ matches: []string{"psionic warp engine"}, rejects: []string{"engine psionic"},
+ },
+ {
+ // Not a valid expression: matched literally, as it used to fail.
+ query: "C++", terms: "C",
+ matches: []string{"written in C++"}, rejects: []string{"written in C"},
+ },
+ {query: "图片素材", terms: "图片素材", matches: []string{"关于图片素材的说明"}},
+ }
+ for _, tc := range cases {
+ re, terms := grepPatternAndTerms(tc.query)
+ if terms != tc.terms {
+ t.Errorf("%q terms = %q, want %q", tc.query, terms, tc.terms)
+ }
+ for _, s := range tc.matches {
+ if !re.MatchString(s) {
+ t.Errorf("%q should match %q", tc.query, s)
+ }
+ }
+ for _, s := range tc.rejects {
+ if re.MatchString(s) {
+ t.Errorf("%q should not match %q", tc.query, s)
+ }
+ }
+ }
+ if _, terms := grepPatternAndTerms(`^\d+$`); terms != "" {
+ t.Fatalf("a pattern without literal text has no index terms, got %q", terms)
+ }
+}
diff --git a/internal/mcpserver/tools_retrieve.go b/internal/mcpserver/tools_retrieve.go
index bf4aa1242..5a2fd0bb8 100644
--- a/internal/mcpserver/tools_retrieve.go
+++ b/internal/mcpserver/tools_retrieve.go
@@ -3,8 +3,9 @@ package mcpserver
import (
"context"
"encoding/json"
- "fmt"
+ "regexp"
"strings"
+ "unicode"
"github.com/Tencent/WeKnora/internal/agent/tools"
"github.com/Tencent/WeKnora/internal/types"
@@ -29,10 +30,14 @@ func listKnowledgeBasesTool() mcp.Tool {
func searchKnowledgeTool() mcp.Tool {
return mcp.NewTool(types.MCPEndpointToolSearchKnowledge,
- mcp.WithDescription("Semantic search across the knowledge bases in scope. Returns the most relevant "+
- "passages with their document ids and titles. Use a natural-language question or topic; use grep_chunks "+
- "instead for exact keywords, codes or names."),
- mcp.WithString("query", mcp.Required(), mcp.Description("A natural-language question or topic to search for")),
+ mcp.WithDescription("Search the knowledge bases in scope and return the most relevant passages with their "+
+ "document ids and titles. mode hybrid (default) combines semantic and keyword retrieval; semantic "+
+ "ranks by meaning; keyword matches exact terms, identifiers, codes or names."),
+ mcp.WithString("query", mcp.Required(), mcp.Description("A natural-language question, topic, or the exact "+
+ "terms to match")),
+ mcp.WithString("mode", mcp.Description("hybrid (default), semantic, or keyword"),
+ mcp.Enum(tools.SearchModeHybrid, tools.SearchModeSemantic, tools.SearchModeKeyword)),
+ mcp.WithNumber("limit", mcp.Description("Maximum passages to return, default 10, max 30")),
mcp.WithArray("knowledge_base_ids", mcp.WithStringItems(),
mcp.Description("Optional knowledge base ids or names to restrict the search; defaults to every "+
"knowledge base in scope")),
@@ -42,11 +47,13 @@ func searchKnowledgeTool() mcp.Tool {
func grepChunksTool() mcp.Tool {
return mcp.NewTool(types.MCPEndpointToolGrepChunks,
- mcp.WithDescription("Case-insensitive regular-expression search over the raw text chunks of the knowledge "+
+ mcp.WithDescription("Case-insensitive regular-expression search over the text chunks of the knowledge "+
"bases in scope. Best for exact terms, identifiers, error codes or product names that semantic search "+
- "may miss."),
+ "may miss. Candidates come from the keyword index (the semantic index for bases without one) using the "+
+ "literal terms in the pattern, and every returned chunk matches the pattern."),
mcp.WithString("query", mcp.Required(), mcp.Description("Regular expression (POSIX, case-insensitive) to "+
- "match against chunk text")),
+ "match against chunk text; text that is not a valid regular expression is matched literally")),
+ mcp.WithNumber("limit", mcp.Description("Maximum passages to return, default 10, max 30")),
mcp.WithArray("knowledge_base_ids", mcp.WithStringItems(),
mcp.Description("Optional knowledge base ids or names to restrict the search")),
mcp.WithReadOnlyHintAnnotation(true),
@@ -68,11 +75,13 @@ func listDocumentsTool() mcp.Tool {
func readDocumentTool() mcp.Tool {
return mcp.NewTool(types.MCPEndpointToolReadDocument,
mcp.WithDescription("Read a document's metadata and its text chunks in order. Page through long documents "+
- "with offset and limit."),
+ "with offset and limit, or pass query to return only the chunks containing a phrase (with one chunk "+
+ "of context on each side)."),
mcp.WithString("knowledge_id", mcp.Required(),
mcp.Description("Document id from search results or list_documents")),
mcp.WithNumber("offset", mcp.Description("Chunk offset to start from, default 0")),
mcp.WithNumber("limit", mcp.Description("Number of chunks to return, default 20, max 100")),
+ mcp.WithString("query", mcp.Description("Optional case-insensitive phrase to find inside the document")),
mcp.WithReadOnlyHintAnnotation(true),
)
}
@@ -139,13 +148,36 @@ func (s *Server) handleSearchKnowledge(ctx context.Context, req mcp.CallToolRequ
return mcp.NewToolResultError(
"none of the selected knowledge bases supports semantic or keyword retrieval"), nil
}
- tool := tools.NewKnowledgeSearchTool(
+ return s.runSearchKnowledge(ctx, kbs, strings.TrimSpace(query),
+ req.GetString("mode", tools.SearchModeHybrid), req.GetInt("limit", 0))
+}
+
+// runSearchKnowledge executes the shared search_knowledge agent tool for
+// both the search_knowledge and grep_chunks endpoint tools.
+func (s *Server) runSearchKnowledge(
+ ctx context.Context, kbs []*types.KnowledgeBase, query, mode string, limit int,
+) (*mcp.CallToolResult, error) {
+ return s.runSearchKnowledgeWithFilter(ctx, kbs, query, mode, limit, nil)
+}
+
+func (s *Server) runSearchKnowledgeWithFilter(
+ ctx context.Context, kbs []*types.KnowledgeBase, query, mode string, limit int, pattern *regexp.Regexp,
+) (*mcp.CallToolResult, error) {
+ tool := tools.NewSearchKnowledgeTool(
s.kbService, s.knowledgeService, s.chunkService, searchTargetsFor(kbs), nil, s.cfg,
)
- args, _ := json.Marshal(map[string]any{
- "queries": []string{strings.TrimSpace(query)},
+ if pattern != nil {
+ tool.WithPatternFilter(pattern)
+ }
+ callArgs := map[string]any{
+ "query": query,
+ "mode": mode,
"knowledge_base_ids": knowledgeBaseIDs(kbs),
- })
+ }
+ if limit > 0 {
+ callArgs["limit"] = limit
+ }
+ args, _ := json.Marshal(callArgs)
res, execErr := tool.Execute(ctx, args)
return toolResultFromAgentTool(res, execErr), nil
}
@@ -163,10 +195,52 @@ func (s *Server) handleGrepChunks(ctx context.Context, req mcp.CallToolRequest)
if err != nil {
return mcp.NewToolResultError(err.Error()), nil
}
- tool := tools.NewGrepChunksTool(s.db, searchTargetsFor(kbs))
- args, _ := json.Marshal(map[string]any{"query": strings.TrimSpace(query)})
- res, execErr := tool.Execute(ctx, args)
- return toolResultFromAgentTool(res, execErr), nil
+ kbs = retrievableKnowledgeBases(kbs)
+ if len(kbs) == 0 {
+ return mcp.NewToolResultError(
+ "none of the selected knowledge bases has a chunk index (vector or keyword)"), nil
+ }
+ pattern, terms := grepPatternAndTerms(strings.TrimSpace(query))
+ if terms == "" {
+ return mcp.NewToolResultError("the pattern contains no literal text to look up; include at least one " +
+ "word, identifier or phrase (for example \"timeout|deadline\" instead of \"^\\d+$\")"), nil
+ }
+ return s.runSearchKnowledgeWithFilter(
+ ctx, kbs, terms, tools.SearchModeKeyword, req.GetInt("limit", 0), pattern,
+ )
+}
+
+var grepEscapeRE = regexp.MustCompile(`\\[A-Za-z]`)
+
+// grepPatternAndTerms keeps grep_chunks' regular-expression contract on top
+// of index-backed retrieval. The pattern compiles case-insensitively (text
+// that is not a valid expression is escaped and matched literally, as it
+// would have failed in the database before). The literal runs inside the
+// pattern become the keyword query that fetches candidates; results are then
+// verified against the pattern itself.
+func grepPatternAndTerms(query string) (*regexp.Regexp, string) {
+ re, err := regexp.Compile("(?i)" + query)
+ source := query
+ if err != nil {
+ re = regexp.MustCompile("(?i)" + regexp.QuoteMeta(query))
+ } else {
+ // Escapes such as \d, \b or \s carry no literal text.
+ source = grepEscapeRE.ReplaceAllString(query, " ")
+ }
+ fields := strings.FieldsFunc(source, func(r rune) bool {
+ return !unicode.IsLetter(r) && !unicode.IsDigit(r) && r != '_' && r != '-'
+ })
+ seen := make(map[string]bool, len(fields))
+ terms := make([]string, 0, len(fields))
+ for _, f := range fields {
+ key := strings.ToLower(f)
+ if seen[key] {
+ continue
+ }
+ seen[key] = true
+ terms = append(terms, f)
+ }
+ return re, strings.Join(terms, " ")
}
type documentSummary struct {
@@ -276,28 +350,27 @@ func (s *Server) handleReadDocument(ctx context.Context, req mcp.CallToolRequest
if offset < 0 {
offset = 0
}
- tool := tools.NewListKnowledgeChunksTool(
+ tool := tools.NewReadDocumentTool(
s.knowledgeService, s.chunkService, searchTargetsFor([]*types.KnowledgeBase{kb}),
)
- args, _ := json.Marshal(map[string]any{
- "knowledge_id": k.ID,
- "limit": limit,
- "offset": offset,
- })
+ callArgs := map[string]any{
+ "id": k.ID,
+ "limit": limit,
+ "offset": offset,
+ }
+ if query := strings.TrimSpace(req.GetString("query", "")); query != "" {
+ callArgs["query"] = query
+ }
+ args, _ := json.Marshal(callArgs)
res, execErr := tool.Execute(ctx, args)
if execErr != nil || res == nil || !res.Success {
return toolResultFromAgentTool(res, execErr), nil
}
- header := fmt.Sprintf("Document: %s\nID: %s\nKnowledge base: %s (%s)\nType: %s\nStatus: %s\n",
- k.Title, k.ID, kb.Name, kb.ID, k.FileType, k.ParseStatus)
- if k.Description != "" {
- header += "Summary: " + k.Description + "\n"
- }
- res.Output = header + "\n" + res.Output
if res.Data == nil {
res.Data = map[string]interface{}{}
}
res.Data["document"] = summarizeKnowledge(k)
+ res.Data["knowledge_base"] = map[string]any{"id": kb.ID, "name": kb.Name}
return toolResultFromAgentTool(res, nil), nil
}
diff --git a/internal/mcpserver/tools_wiki.go b/internal/mcpserver/tools_wiki.go
index 109f40b9d..1b3a182c3 100644
--- a/internal/mcpserver/tools_wiki.go
+++ b/internal/mcpserver/tools_wiki.go
@@ -22,7 +22,10 @@ func wikiSearchTool() mcp.Tool {
mcp.WithDescription("Search the generated wiki pages of the knowledge bases in scope by title, alias and "+
"content. Returns page slugs that wiki_read_page accepts. Only knowledge bases with a wiki are "+
"searched."),
- mcp.WithString("query", mcp.Required(), mcp.Description("Search terms or a regular expression")),
+ mcp.WithString("query", mcp.Required(), mcp.Description("Search terms or a regular expression "+
+ "(case-insensitive POSIX); text that is not a valid regular expression is matched literally")),
+ mcp.WithBoolean("regex", mcp.Description("false forces a literal match; true requires a valid regular "+
+ "expression; omit for the default behaviour")),
mcp.WithArray("knowledge_base_ids", mcp.WithStringItems(),
mcp.Description("Optional knowledge base ids or names to restrict the search")),
mcp.WithNumber("limit", mcp.Description("Maximum pages to return, default 10, max 50")),
@@ -93,10 +96,16 @@ func (s *Server) handleWikiSearch(ctx context.Context, req mcp.CallToolRequest)
limit = maxWikiSearchLimit
}
tool := tools.NewWikiSearchTool(s.wikiService, s.knowledgeService, wikiScopesFor(kbs), tools.NewWikiRouteResolver())
- args, _ := json.Marshal(map[string]any{
- "queries": []string{strings.TrimSpace(query)},
- "limit": limit,
- })
+ callArgs := map[string]any{
+ "query": strings.TrimSpace(query),
+ "limit": limit,
+ }
+ // Only forward regex when the client set it, so the default keeps the
+ // endpoint's historical regular-expression semantics.
+ if v, ok := req.GetArguments()["regex"].(bool); ok {
+ callArgs["regex"] = v
+ }
+ args, _ := json.Marshal(callArgs)
res, execErr := tool.Execute(ctx, args)
return toolResultFromAgentTool(res, execErr), nil
}
diff --git a/internal/modelcontext/model_output.go b/internal/modelcontext/model_output.go
index b0344395f..a07b80468 100644
--- a/internal/modelcontext/model_output.go
+++ b/internal/modelcontext/model_output.go
@@ -42,11 +42,18 @@ func (r *sourceRegistry) ModelOutput(result *types.ToolResult) string {
case "grep_results":
return r.modelKnowledgeOutput("keyword", mapsValue(result.Data["chunk_results"]), result.Output)
case "search_results":
- return r.modelKnowledgeOutput("semantic", mapsValue(result.Data["results"]), result.Output)
+ // search_knowledge reports the mode it actually used; legacy
+ // knowledge_search payloads carry none and were always semantic.
+ mode := stringValue(result.Data, "mode")
+ if mode == "" {
+ mode = "semantic"
+ }
+ output := r.modelKnowledgeOutput(mode, mapsValue(result.Data["results"]), result.Output)
+ return r.annotateModeFallbacks(output, result.Data)
case "knowledge_chunks_list":
return r.modelKnowledgeChunksOutput(result.Data, result.Output)
case "document_info":
- return r.modelDocumentInfoOutput(mapsValue(result.Data["documents"]), result.Output)
+ return r.modelDocumentInfoOutput(result.Data, result.Output)
case "graph_query_results":
return r.modelKnowledgeOutput("graph", mapsValue(result.Data["results"]), result.Output)
case "web_search_results":
@@ -97,12 +104,26 @@ func (r *sourceRegistry) modelDatabaseQueryOutput(rows []map[string]interface{},
return r.CompactKnownText(fallback)
}
-func (r *sourceRegistry) modelDocumentInfoOutput(rows []map[string]interface{}, fallback string) string {
+func (r *sourceRegistry) modelDocumentInfoOutput(data map[string]interface{}, fallback string) string {
+ rows := mapsValue(data["documents"])
if len(rows) == 0 {
return r.CompactKnownText(fallback)
}
var b strings.Builder
- b.WriteString("<documents>\n")
+ b.WriteString("<documents")
+ if kbHandle := r.RegisterKnowledgeBase(stringValue(data, "knowledge_base_id")); kbHandle != "" {
+ fmt.Fprintf(&b, " kb=\"%s\"", escapeAttr(kbHandle))
+ }
+ if total := intValue(data, "total_docs"); total > 0 {
+ fmt.Fprintf(&b, " total=\"%d\"", total)
+ }
+ if page := intValue(data, "page"); page > 0 {
+ fmt.Fprintf(&b, " page=\"%d\"", page)
+ }
+ if next := intValue(data, "next_page"); next > 0 {
+ fmt.Fprintf(&b, " next_page=\"%d\"", next)
+ }
+ b.WriteString(">\n")
count := 0
for _, row := range rows {
knowledgeID := stringValue(row, "knowledge_id")
@@ -145,7 +166,16 @@ func (r *sourceRegistry) modelDocumentInfoOutput(rows []map[string]interface{},
if fileType := stringValue(row, "file_type"); fileType != "" {
fmt.Fprintf(&b, " file_type=\"%s\"", escapeAttr(fileType))
}
- fmt.Fprintf(&b, " chunk_count=\"%d\">\n", intValue(row, "chunk_count"))
+ if chunkCount := intValue(row, "chunk_count"); chunkCount > 0 {
+ fmt.Fprintf(&b, " chunk_count=\"%d\"", chunkCount)
+ }
+ if status := stringValue(row, "parse_status"); status != "" {
+ fmt.Fprintf(&b, " parse_status=\"%s\"", escapeAttr(status))
+ }
+ if updated := stringValue(row, "updated_at"); len(updated) >= 10 {
+ fmt.Fprintf(&b, " updated_at=\"%s\"", escapeAttr(updated[:10]))
+ }
+ b.WriteString(">\n")
if description := stringValue(row, "description"); description != "" {
fmt.Fprintf(&b, " <description>%s</description>\n", escapeText(description))
}
@@ -168,6 +198,7 @@ type modelChunk struct {
chunkType string
index int
view string
+ role string
match string
content string
question string
@@ -180,6 +211,16 @@ type modelChunk struct {
}
func (r *sourceRegistry) modelKnowledgeOutput(mode string, rows []map[string]interface{}, fallback string) string {
+ chunks := r.modelChunksFromRows(mode, rows)
+ if len(chunks) == 0 {
+ return r.CompactKnownText(fallback)
+ }
+ return renderKnowledgeChunks(mode, chunks, nil)
+}
+
+// modelChunksFromRows registers every row's handles and converts it into the
+// renderer's chunk model.
+func (r *sourceRegistry) modelChunksFromRows(mode string, rows []map[string]interface{}) []modelChunk {
chunks := make([]modelChunk, 0, len(rows))
for idx, row := range rows {
chunkID := firstNonEmpty(stringValue(row, "chunk_id"), stringValue(row, "faq_id"), stringValue(row, "id"))
@@ -214,6 +255,7 @@ func (r *sourceRegistry) modelKnowledgeOutput(mode string, rows []map[string]int
chunkType: chunkType,
index: chunkIndex,
view: viewForRow(row, mode),
+ role: stringValue(row, "role"),
match: firstNonEmpty(stringValue(row, "match_snippet"), stringValue(row, "matched_content")),
content: stringValue(row, "content"),
question: firstNonEmpty(stringValue(row, "faq_question"), stringValue(row, "faq_standard_question")),
@@ -225,10 +267,28 @@ func (r *sourceRegistry) modelKnowledgeOutput(mode string, rows []map[string]int
inputOrder: idx,
})
}
- if len(chunks) == 0 {
- return r.CompactKnownText(fallback)
+ return chunks
+}
+
+// annotateModeFallbacks adds the requested mode and one <mode_fallback> per
+// knowledge base that was searched with a different retrieval path.
+func (r *sourceRegistry) annotateModeFallbacks(output string, data map[string]interface{}) string {
+ fallbacks := mapsValue(data["mode_fallbacks"])
+ requested := stringValue(data, "requested_mode")
+ if len(fallbacks) == 0 || !strings.HasSuffix(output, "</retrieval>") {
+ return output
+ }
+ if requested != "" {
+ output = strings.Replace(output, "<retrieval ",
+ fmt.Sprintf("<retrieval requested_mode=\"%s\" ", escapeAttr(requested)), 1)
}
- return renderKnowledgeChunks(mode, chunks)
+ var b strings.Builder
+ for _, fb := range fallbacks {
+ fmt.Fprintf(&b, " <mode_fallback kb=\"%s\" mode=\"%s\" reason=\"%s\" />\n",
+ escapeAttr(r.RegisterKnowledgeBase(stringValue(fb, "knowledge_base_id"))),
+ escapeAttr(stringValue(fb, "mode")), escapeAttr(stringValue(fb, "reason")))
+ }
+ return strings.TrimSuffix(output, "</retrieval>") + b.String() + "</retrieval>"
}
func viewForRow(row map[string]interface{}, mode string) string {
@@ -253,22 +313,110 @@ func (r *sourceRegistry) modelKnowledgeChunksOutput(data map[string]interface{},
row["knowledge_title"] = title
}
}
- output := r.modelKnowledgeOutput("deep_read", rows, fallback)
+ var info map[string]interface{}
+ if raw, ok := data["document"].(map[string]interface{}); ok {
+ info = raw
+ }
if len(rows) == 0 {
+ // A document that has no chunks (or a query with no matches) still
+ // deserves its header so the model learns what it read.
+ if info == nil {
+ return r.CompactKnownText(fallback)
+ }
+ docHandle := r.RegisterDocument(knowledgeID)
+ var b strings.Builder
+ b.WriteString("<retrieval type=\"knowledge\" mode=\"deep_read\">\n")
+ fmt.Fprintf(&b, " <document id=\"%s\" title=\"%s\">\n", escapeAttr(docHandle), escapeAttr(title))
+ writeDocumentInfo(&b, info)
+ if query := stringValue(data, "query"); query != "" {
+ fmt.Fprintf(&b, " <matches query=\"%s\" count=\"0\" />\n", escapeAttr(query))
+ }
+ b.WriteString(" </document>\n</retrieval>")
+ return b.String()
+ }
+ chunks := r.modelChunksFromRows("deep_read", rows)
+ if len(chunks) == 0 {
+ return r.CompactKnownText(fallback)
+ }
+ output := renderKnowledgeChunks("deep_read", chunks, info)
+ if info == nil {
+ // Legacy list_knowledge_chunks payloads carry no document header.
+ remaining := intValue(data, "total_chunks") - intValue(data, "fetched_chunks")
+ if remaining > 0 {
+ output = strings.TrimSuffix(output, "</retrieval>")
+ output += fmt.Sprintf(" <pagination remaining=\"%d\" page=\"%d\" page_size=\"%d\" />\n</retrieval>",
+ remaining, intValue(data, "page"), intValue(data, "page_size"))
+ }
return output
}
- remaining := intValue(data, "total_chunks") - intValue(data, "fetched_chunks")
- if remaining > 0 {
- output = strings.TrimSuffix(output, "</retrieval>")
- output += fmt.Sprintf(" <pagination remaining=\"%d\" page=\"%d\" page_size=\"%d\" />\n</retrieval>",
- remaining, intValue(data, "page"), intValue(data, "page_size"))
+ var footer strings.Builder
+ if query := stringValue(data, "query"); query != "" {
+ fmt.Fprintf(&footer, " <matches query=\"%s\" count=\"%d\"", escapeAttr(query), intValue(data, "match_count"))
+ if boolValue(data, "truncated") {
+ footer.WriteString(" truncated=\"true\"")
+ }
+ footer.WriteString(" />\n")
+ } else if _, ok := data["next_offset"]; ok {
+ remaining := intValue(data, "total_chunks") - (intValue(data, "offset") + intValue(data, "fetched_chunks"))
+ if remaining < 0 {
+ remaining = 0
+ }
+ fmt.Fprintf(&footer, " <pagination next_offset=\"%d\" remaining=\"%d\" />\n",
+ intValue(data, "next_offset"), remaining)
+ }
+ if footer.Len() > 0 {
+ output = strings.TrimSuffix(output, "</retrieval>") + footer.String() + "</retrieval>"
}
return output
}
-func renderKnowledgeChunks(mode string, chunks []modelChunk) string {
+// writeDocumentInfo renders the read_document metadata header as one
+// <info> element under the document.
+func writeDocumentInfo(b *strings.Builder, info map[string]interface{}) {
+ if len(info) == 0 {
+ return
+ }
+ b.WriteString(" <info")
+ for _, key := range []string{"source", "file_name", "file_type", "file_size", "parse_status"} {
+ if value := stringValue(info, key); value != "" {
+ fmt.Fprintf(b, " %s=\"%s\"", key, escapeAttr(value))
+ }
+ }
+ if count := intValue(info, "chunk_count"); count > 0 {
+ fmt.Fprintf(b, " chunk_count=\"%d\"", count)
+ }
+ description := stringValue(info, "description")
+ metadata, _ := info["metadata"].(map[string]interface{})
+ if description == "" && len(metadata) == 0 {
+ b.WriteString(" />\n")
+ return
+ }
+ b.WriteString(">\n")
+ if description != "" {
+ fmt.Fprintf(b, " <description>%s</description>\n", escapeText(description))
+ }
+ if len(metadata) > 0 {
+ keys := make([]string, 0, len(metadata))
+ for key := range metadata {
+ keys = append(keys, key)
+ }
+ sort.Strings(keys)
+ b.WriteString(" <metadata>")
+ for i, key := range keys {
+ if i > 0 {
+ b.WriteString("; ")
+ }
+ fmt.Fprintf(b, "%s: %v", escapeText(key), metadata[key])
+ }
+ b.WriteString("</metadata>\n")
+ }
+ b.WriteString(" </info>\n")
+}
+
+func renderKnowledgeChunks(mode string, chunks []modelChunk, info map[string]interface{}) string {
type docGroup struct {
handle string
+ realID string
kbHandle string
title string
metadata string
@@ -285,7 +433,7 @@ func renderKnowledgeChunks(mode string, chunks []modelChunk) string {
group := groupsByKey[key]
if group == nil {
group = &docGroup{
- handle: chunk.docHandle, kbHandle: chunk.kbHandle, title: chunk.title,
+ handle: chunk.docHandle, realID: chunk.docRealID, kbHandle: chunk.kbHandle, title: chunk.title,
metadata: chunk.metadata, order: chunk.inputOrder,
}
groupsByKey[key] = group
@@ -314,11 +462,17 @@ func renderKnowledgeChunks(mode string, chunks []modelChunk) string {
if group.metadata != "" {
fmt.Fprintf(&b, " <metadata>%s</metadata>\n", escapeText(group.metadata))
}
+ if info != nil && stringValue(info, "knowledge_id") == group.realID {
+ writeDocumentInfo(&b, info)
+ }
for _, chunk := range group.chunks {
fmt.Fprintf(&b, " <chunk id=\"%s\" index=\"%d\" view=\"%s\"", chunk.handle, chunk.index, chunk.view)
if chunk.chunkType != "" {
fmt.Fprintf(&b, " type=\"%s\"", escapeAttr(chunk.chunkType))
}
+ if chunk.role != "" {
+ fmt.Fprintf(&b, " role=\"%s\"", escapeAttr(chunk.role))
+ }
b.WriteString(">\n")
if chunk.question != "" {
fmt.Fprintf(&b, " <question>%s</question>\n", escapeText(chunk.question))
diff --git a/internal/modelcontext/registry_test.go b/internal/modelcontext/registry_test.go
index 47887a2d6..3527f3a63 100644
--- a/internal/modelcontext/registry_test.go
+++ b/internal/modelcontext/registry_test.go
@@ -498,3 +498,183 @@ func TestCitationPolicyDoesNotSuppressRequestedResourcesOrBreakOutputFormat(t *t
enabled := NewRegistry(true).ProtocolPrompt()
require.Contains(t, enabled, "do not break a required schema")
}
+
+func TestRegistryDecodesReadDocumentIDAgainstBothHandleTables(t *testing.T) {
+ registry := NewRegistry(true)
+ registry.RegisterDocument("doc-real")
+ registry.RegisterChunk(ChunkReference{ChunkID: "chunk-real", KnowledgeID: "doc-real", KnowledgeBaseID: "kb-real"})
+ registry.RegisterKnowledgeBase("kb-real")
+
+ for _, tc := range []struct{ name, tool, raw, want string }{
+ {
+ "read document by dN", "read_document",
+ `{"id":"d1","offset":20}`, `{"id":"doc-real","offset":20}`,
+ },
+ {
+ "read document by cN", "read_document",
+ `{"id":"c1","context":2}`, `{"id":"chunk-real","context":2}`,
+ },
+ {
+ "read document query stays literal", "read_document",
+ `{"id":"d1","query":"d1 c1"}`, `{"id":"doc-real","query":"d1 c1"}`,
+ },
+ {
+ "search knowledge KB", "search_knowledge",
+ `{"query":"b1","knowledge_base_ids":["b1"]}`, `{"query":"b1","knowledge_base_ids":["kb-real"]}`,
+ },
+ {
+ "list documents KB", "list_documents",
+ `{"knowledge_base_id":"b1","keyword":"b1"}`, `{"knowledge_base_id":"kb-real","keyword":"b1"}`,
+ },
+ {
+ "wiki search KBs", "wiki_search",
+ `{"query":"topic","knowledge_base_ids":["b1"]}`, `{"query":"topic","knowledge_base_ids":["kb-real"]}`,
+ },
+ } {
+ t.Run(tc.name, func(t *testing.T) {
+ calls := []types.LLMToolCall{{Function: types.FunctionCall{Name: tc.tool, Arguments: tc.raw}}}
+ registry.DecodeToolCalls(calls)
+ require.JSONEq(t, tc.want, calls[0].Function.Arguments)
+ require.Equal(t, ArgumentResolutionResolved, calls[0].ArgumentResolution)
+ })
+ }
+
+ // A bare "id" on a tool without the read_document contract is opaque.
+ calls := []types.LLMToolCall{{Function: types.FunctionCall{Name: "thinking", Arguments: `{"id":"d1"}`}}}
+ registry.DecodeToolCalls(calls)
+ require.JSONEq(t, `{"id":"d1"}`, calls[0].Function.Arguments)
+
+ unknown := []types.LLMToolCall{{Function: types.FunctionCall{Name: "read_document", Arguments: `{"id":"d9"}`}}}
+ registry.DecodeToolCalls(unknown)
+ require.Equal(t, ArgumentResolutionUnresolved, unknown[0].ArgumentResolution)
+ require.Equal(t, []string{"d9"}, unknown[0].UnresolvedHandles)
+}
+
+func TestModelOutputRendersReadDocumentHeaderAndNavigation(t *testing.T) {
+ registry := NewRegistry(true)
+ paged := registry.ModelToolResultForTool("read_document", &types.ToolResult{Success: true, Data: map[string]any{
+ "display_type": "knowledge_chunks_list",
+ "knowledge_id": "doc-real",
+ "knowledge_title": "Engine Manual",
+ "total_chunks": 12,
+ "fetched_chunks": 2,
+ "offset": 4,
+ "next_offset": 6,
+ "document": map[string]interface{}{
+ "knowledge_id": "doc-real", "title": "Engine Manual", "source": "File Upload", "file_type": "pdf",
+ "parse_status": "completed", "chunk_count": 12, "description": "How the engine works",
+ "metadata": map[string]interface{}{"region": "EU", "author": "Ada"},
+ },
+ "chunks": []map[string]interface{}{
+ {"chunk_id": "chunk-4", "chunk_index": 4, "content": "four", "knowledge_base": "kb-real"},
+ {"chunk_id": "chunk-5", "chunk_index": 5, "content": "five", "knowledge_base": "kb-real"},
+ },
+ }})
+ require.Contains(t, paged, `<retrieval type="knowledge" mode="deep_read">`)
+ require.Contains(t, paged, `<document id="d1" kb="b1" title="Engine Manual">`)
+ require.Contains(t, paged, `<info source="File Upload" file_type="pdf" parse_status="completed" chunk_count="12">`)
+ require.Contains(t, paged, `<description>How the engine works</description>`)
+ require.Contains(t, paged, `<metadata>author: Ada; region: EU</metadata>`)
+ require.Contains(t, paged, `<chunk id="c1" index="4" view="full">`)
+ require.Contains(t, paged, `<pagination next_offset="6" remaining="6" />`)
+ require.NotContains(t, paged, "doc-real")
+
+ matched := registry.ModelToolResultForTool("read_document", &types.ToolResult{Success: true, Data: map[string]any{
+ "display_type": "knowledge_chunks_list",
+ "knowledge_id": "doc-real",
+ "knowledge_title": "Engine Manual",
+ "total_chunks": 12,
+ "fetched_chunks": 3,
+ "query": "psionic",
+ "match_count": 1,
+ "truncated": false,
+ "document": map[string]any{"knowledge_id": "doc-real", "title": "Engine Manual", "chunk_count": 12},
+ "chunks": []map[string]interface{}{
+ {"chunk_id": "chunk-3", "chunk_index": 3, "content": "three", "role": "context_before"},
+ {
+ "chunk_id": "chunk-4", "chunk_index": 4, "content": "psionic four",
+ "role": "match", "match_snippet": "... psionic ...",
+ },
+ {"chunk_id": "chunk-5", "chunk_index": 5, "content": "five", "role": "context_after"},
+ },
+ }})
+ require.Contains(t, matched, `role="match"`)
+ require.Contains(t, matched, `role="context_before"`)
+ require.Contains(t, matched, `<match>... psionic ...</match>`)
+ require.Contains(t, matched, `<matches query="psionic" count="1" />`)
+ require.NotContains(t, matched, "<pagination")
+
+ empty := registry.ModelToolResultForTool("read_document", &types.ToolResult{Success: true, Data: map[string]any{
+ "display_type": "knowledge_chunks_list",
+ "knowledge_id": "doc-real",
+ "knowledge_title": "Engine Manual",
+ "total_chunks": 12,
+ "fetched_chunks": 0,
+ "query": "absent",
+ "match_count": 0,
+ "document": map[string]any{"knowledge_id": "doc-real", "title": "Engine Manual", "chunk_count": 12},
+ "chunks": []map[string]interface{}{},
+ }})
+ require.Contains(t, empty, `<document id="d1" title="Engine Manual">`)
+ require.Contains(t, empty, `<matches query="absent" count="0" />`)
+}
+
+func TestModelOutputRendersDocumentListWithPagination(t *testing.T) {
+ registry := NewRegistry(true)
+ out := registry.ModelToolResultForTool("list_documents", &types.ToolResult{Success: true, Data: map[string]any{
+ "display_type": "document_info",
+ "knowledge_base_id": "kb-real",
+ "total_docs": int64(3),
+ "page": 1,
+ "page_size": 2,
+ "next_page": 2,
+ "documents": []map[string]interface{}{
+ {
+ "knowledge_id": "doc-a", "title": "Alpha", "file_type": "pdf", "parse_status": "completed",
+ "updated_at": "2026-03-04T05:06:07Z", "description": "First",
+ },
+ {"knowledge_id": "doc-b", "title": "Beta", "file_type": "md", "parse_status": "completed"},
+ },
+ }})
+ require.Contains(t, out, `<documents kb="b1" total="3" page="1" next_page="2">`)
+ require.Contains(t, out,
+ `<document id="d1" title="Alpha" file_type="pdf" parse_status="completed" updated_at="2026-03-04">`)
+ require.Contains(t, out, `<description>First</description>`)
+ require.Contains(t, out, `<document id="d2" title="Beta"`)
+ require.NotContains(t, out, "doc-a")
+ require.NotContains(t, out, "kb-real")
+}
+
+func TestModelOutputReportsSearchModeAndFallbacks(t *testing.T) {
+ registry := NewRegistry(true)
+ row := map[string]interface{}{
+ "chunk_id": "chunk-1", "knowledge_id": "doc-1", "knowledge_base_id": "kb-faq",
+ "knowledge_title": "FAQ", "content": "reset the device",
+ }
+ keyword := registry.ModelToolResultForTool("search_knowledge", &types.ToolResult{
+ Success: true, Data: map[string]any{
+ "display_type": "search_results", "mode": "keyword", "results": []map[string]interface{}{row},
+ },
+ })
+ require.Contains(t, keyword, `<retrieval type="knowledge" mode="keyword">`)
+
+ const faqReason = "FAQ bases are indexed for semantic search only"
+ fallback := registry.ModelToolResultForTool("search_knowledge", &types.ToolResult{
+ Success: true, Data: map[string]any{
+ "display_type": "search_results", "mode": "semantic", "requested_mode": "keyword",
+ "results": []map[string]interface{}{row},
+ "mode_fallbacks": []map[string]interface{}{
+ {"knowledge_base_id": "kb-faq", "mode": "semantic", "reason": faqReason},
+ },
+ },
+ })
+ require.Contains(t, fallback, `<retrieval requested_mode="keyword" type="knowledge" mode="semantic">`)
+ require.Contains(t, fallback,
+ `<mode_fallback kb="b1" mode="semantic" reason="FAQ bases are indexed for semantic search only" />`)
+ require.NotContains(t, fallback, "kb-faq")
+
+ legacy := registry.ModelToolResultForTool("knowledge_search", &types.ToolResult{Success: true, Data: map[string]any{
+ "display_type": "search_results", "results": []map[string]interface{}{row},
+ }})
+ require.Contains(t, legacy, `mode="semantic"`, "legacy knowledge_search payloads were semantic")
+}
diff --git a/internal/modelcontext/sources.go b/internal/modelcontext/sources.go
index 669397d36..4a048844d 100644
--- a/internal/modelcontext/sources.go
+++ b/internal/modelcontext/sources.go
@@ -490,6 +490,8 @@ func (r *sourceRegistry) registerSourceIDByKey(key, value string, evidence bool)
if parsed, err := url.Parse(value); err == nil && (parsed.Scheme == "http" || parsed.Scheme == "https") {
r.registerWeb(value, "", evidence)
}
+ case spaceAnySource:
+ // Decode-only: a bare "id" never registers a new source.
}
}
diff --git a/internal/modelcontext/tool_policy.go b/internal/modelcontext/tool_policy.go
index 72980bdee..38c9699ed 100644
--- a/internal/modelcontext/tool_policy.go
+++ b/internal/modelcontext/tool_policy.go
@@ -28,6 +28,10 @@ const (
spaceDocumentRef // "knowledgeID|title" stored refs; only the ID is durable
spaceKnowledgeBase
spaceWeb
+ // spaceAnySource accepts a chunk or a document handle in one argument
+ // (read_document's id). It is decode-only: registration never learns a
+ // source from a bare "id" key, whose meaning differs from tool to tool.
+ spaceAnySource
)
// sourceKeySpaces is the single table of ID-bearing keys the source codec
@@ -42,6 +46,7 @@ var sourceKeySpaces = map[string]sourceKeySpace{
"knowledge_base": spaceKnowledgeBase, "knowledge_base_id": spaceKnowledgeBase,
"knowledge_base_ids": spaceKnowledgeBase, "kb_id": spaceKnowledgeBase, "kb_ids": spaceKnowledgeBase,
"url": spaceWeb, "urls": spaceWeb,
+ "id": spaceAnySource,
}
type toolHandlePolicy struct {
@@ -64,6 +69,22 @@ var toolHandlePolicies = map[string]toolHandlePolicy{
"discover_mcp_tools": {opaqueOutput: true, mcpRoutingKey: "server_id", mcpDirectoryOutput: true},
"call_mcp_tool": {opaqueOutput: true, mcpRoutingKey: "tool_ref"},
"read_file": {},
+ "search_knowledge": {
+ sourceIDKeys: map[string]struct{}{"knowledge_base_ids": {}},
+ sourceOutput: true,
+ },
+ // read_document takes one id that may name a document (dN) or a chunk
+ // (cN); the "id" key resolves against both handle tables.
+ "read_document": {
+ sourceIDKeys: map[string]struct{}{"id": {}},
+ sourceOutput: true,
+ },
+ "list_documents": {
+ sourceIDKeys: map[string]struct{}{"knowledge_base_id": {}},
+ sourceOutput: true,
+ },
+ // Retired retrieval tools keep their policies so stored histories that
+ // still carry their calls decode and compact exactly as they did.
"knowledge_search": {
sourceIDKeys: map[string]struct{}{"knowledge_base_ids": {}},
sourceOutput: true,
@@ -126,7 +147,7 @@ var toolHandlePolicies = map[string]toolHandlePolicy{
sourceIDKeys: map[string]struct{}{"suspected_knowledge_ids": {}},
},
"wiki_search": {
- sourceIDKeys: map[string]struct{}{"knowledge_base_id": {}},
+ sourceIDKeys: map[string]struct{}{"knowledge_base_id": {}, "knowledge_base_ids": {}},
sourceOutput: true,
},
toolWikiReadIssue: {
diff --git a/internal/types/interfaces/chunk.go b/internal/types/interfaces/chunk.go
index 90675ad86..c3d865df6 100644
--- a/internal/types/interfaces/chunk.go
+++ b/internal/types/interfaces/chunk.go
@@ -64,6 +64,20 @@ type ChunkRepository interface {
isEnabled *bool,
) ([]*types.Chunk, int64, error)
ListChunkByParentID(ctx context.Context, tenantID uint64, parentID string) ([]*types.Chunk, error)
+ // ListChunkNeighbors returns up to `before` enabled chunks immediately
+ // preceding chunkIndex and up to `after` immediately following it, in
+ // document order, restricted to chunkTypes. It walks chunk_index rather
+ // than list positions, so gaps left by other chunk types (parents,
+ // summaries, images) do not shift the neighbourhood.
+ ListChunkNeighbors(
+ ctx context.Context,
+ tenantID uint64,
+ knowledgeID string,
+ chunkIndex int,
+ before int,
+ after int,
+ chunkTypes []types.ChunkType,
+ ) ([]*types.Chunk, error)
// ListChunksByParentIDs lists chunks whose parent_chunk_id is in the given list
ListChunksByParentIDs(ctx context.Context, tenantID uint64, parentIDs []string) ([]*types.Chunk, error)
// UpdateChunk updates a chunk
diff --git a/website-docs/01-getting-started/01-introduction.md b/website-docs/01-getting-started/01-introduction.md
index b76b98448..7285545a6 100644
--- a/website-docs/01-getting-started/01-introduction.md
+++ b/website-docs/01-getting-started/01-introduction.md
@@ -111,7 +111,7 @@ flowchart TB
- **文档理解**:版式分析、扫描件 OCR、表格抽取、图片多模态描述(VLM)、音频转写(ASR)、按文件类型选择解析引擎(`ParserEngineRules`,可接 MinerU / OpenDataLoader)。
- **索引管道**:可配置分块(含父子分块与自适应策略)、向量索引、关键词全文索引、FAQ 索引、Wiki 生成、知识图谱抽取、预生成问题(question generation)。
- **检索**:向量 + BM25 混合检索、RRF 融合、Rerank 重排、查询改写与扩展、意图识别(greeting/chitchat/web_search 等,见 `config/prompt_templates/intent_prompts.yaml`)。
-- **问答与 Agent**:流式 SSE 问答、多轮上下文压缩、引用溯源;ReAct Agent(工具:`knowledge_search`、`grep_chunks`、`wiki_search`、`data_analysis` 等)、MCP 外部工具、Agent Skills(Docker、Cube 或 E2B 沙箱执行脚本)、Web 搜索。
+- **问答与 Agent**:流式 SSE 问答、多轮上下文压缩、引用溯源;ReAct Agent(工具:`search_knowledge`、`read_document`、`list_documents`、`wiki_search`、`data_analysis` 等)、MCP 外部工具、Agent Skills(Docker、Cube 或 E2B 沙箱执行脚本)、Web 搜索。
- **多租户与安全**:RBAC 角色鉴权(默认开启,`WEKNORA_TENANT_ENABLE_RBAC`)、审计日志(默认保留 90 天)、邀请制注册(`auth.registration_mode=invite_only`,也可用旧变量 `DISABLE_REGISTRATION=true`)、OIDC 单点登录、SSRF 防护、敏感字段 AES-256 加密。
- **可观测性**:Langfuse 全链路追踪(LLM/Embedding/Rerank/VLM/ASR 调用与 token 统计)、健康检查、Swagger API 文档(`GIN_MODE=debug` 时)。
- **生态**:REST API(`/api/v1`)+ API Key、独立 MCP Server(把 WeKnora 作为工具暴露给其他 Agent)、CLI(`cli/`)、微信小程序(`miniprogram/`)、浏览器插件渠道。
diff --git a/website-docs/01-getting-started/04-configuration.md b/website-docs/01-getting-started/04-configuration.md
index 0bfb43eaa..711e655d9 100644
--- a/website-docs/01-getting-started/04-configuration.md
+++ b/website-docs/01-getting-started/04-configuration.md
@@ -336,12 +336,14 @@ AWS S3 的 `S3_ACCESS_KEY` / `S3_SECRET_KEY` 可以**同时留空**,此时走
| id | 系统 Prompt | 工具白名单 | 备注 |
| --- | --- | --- | --- |
-| `rag-qa` | `progressive_rag_agent` | knowledge_search、grep_chunks、list_knowledge_chunks、get_document_info | temperature 0.7、max_iterations 30、FAQ 优先 |
-| `wiki-qa` | `wiki_researcher` | wiki_search、wiki_read_page、wiki_read_source_doc、wiki_flag_issue | 需 Wiki 已启用的知识库 |
+| `rag-qa` | `progressive_rag_agent` | search_knowledge、read_document、list_documents | temperature 0.7、max_iterations 30、FAQ 优先 |
+| `wiki-qa` | `wiki_researcher` | wiki_search、wiki_read_page、read_document、wiki_flag_issue | 需 Wiki 已启用的知识库 |
| `hybrid-rag-wiki` | `hybrid_rag_wiki_agent` | Wiki + RAG 工具全集 | max_iterations 40,最灵活的预设 |
| `data-analysis` | `data_analyst` | data_schema、data_analysis | temperature 0.3;`kb_filter: none_of: [faq]`;支持 csv/xlsx |
| `custom` | 无 | 无预填 | 完全手动配置 |
+已保存配置里出现的旧工具名(`knowledge_search`、`grep_chunks` → `search_knowledge`;`list_knowledge_chunks`、`get_document_info`、`wiki_read_source_doc` → `read_document`)会在运行时自动映射为新工具,无需手动改写。
+
## config/builtin_agents.yaml:内置 Agent
定义随系统分发、对所有租户可见的 Agent(`BuiltinAgentEntry`,`internal/types/builtin_agent_config.go`)。每条含 `id`、`avatar`、`is_builtin: true`、`i18n`(default/zh-CN/zh-TW/ja-JP/ko-KR 的名称与描述)与完整 `config`(`CustomAgentConfig`)。文件内置五个 Agent:
diff --git a/website-docs/03-features/07-agent.md b/website-docs/03-features/07-agent.md
index 209f34782..11ad95c7f 100644
--- a/website-docs/03-features/07-agent.md
+++ b/website-docs/03-features/07-agent.md
@@ -65,9 +65,9 @@ smart-reasoning 下还可选**类型预设**(`Config.AgentType`,定义在 `c
| 预设 ID | 系统提示词模板 | 温度 | 最大迭代 | 预填工具 | KB 过滤 |
| --- | --- | --- | --- | --- | --- |
-| `rag-qa` | `progressive_rag_agent` | 0.7 | 30 | knowledge_search、grep_chunks、list_knowledge_chunks、get_document_info | 由工具派生:any_of vector/keyword |
-| `wiki-qa` | `wiki_researcher` | 0.7 | 30 | wiki_search、wiki_read_page、wiki_read_source_doc、wiki_flag_issue | 由工具派生:any_of wiki |
-| `hybrid-rag-wiki` | `hybrid_rag_wiki_agent` | 0.7 | 40 | wiki_search、wiki_read_page、knowledge_search、grep_chunks、list_knowledge_chunks、get_document_info、wiki_flag_issue | any_of vector/keyword/wiki |
+| `rag-qa` | `progressive_rag_agent` | 0.7 | 30 | search_knowledge、read_document、list_documents | 由工具派生:any_of vector/keyword |
+| `wiki-qa` | `wiki_researcher` | 0.7 | 30 | wiki_search、wiki_read_page、read_document、wiki_flag_issue | 由工具派生:any_of wiki |
+| `hybrid-rag-wiki` | `hybrid_rag_wiki_agent` | 0.7 | 40 | wiki_search、wiki_read_page、search_knowledge、read_document、list_documents、wiki_flag_issue | any_of vector/keyword/wiki |
| `data-analysis` | `data_analyst` | 0.3 | 30 | data_schema、data_analysis;关闭 web 搜索;限定文件类型 csv/xlsx | 显式 `none_of: [faq]` |
| `custom` | 无 | — | — | 不预填 | 不限制 |
@@ -102,14 +102,14 @@ smart-reasoning 下还可选**类型预设**(`Config.AgentType`,定义在 `c
Handler 层(`internal/handler/custom_agent.go`)提供 `CreateAgent`、`GetAgent`、`ListAgents`、`UpdateAgent`、`DeleteAgent`、`CopyAgent`、`GetPlaceholders`(返回 `types.PlaceholdersByField(PromptFieldAgentSystemPrompt)` 的占位符清单)、`GetAgentTypePresets`(带 i18n 的预设列表)、`GetSuggestedQuestions`。创建/更新时经 `authorizeAgentKnowledgeScope` 校验受限 API Key 的 KB 范围:`kb_selection_mode: all` 对 KB 受限 key 直接 403,`selected` 逐一鉴权。
-运行时映射:`buildAgentConfig`(`session_agent_qa.go`)把 `CustomAgentConfig` 转换为引擎的 `types.AgentConfig`(`internal/types/agent.go`),并叠加:web 搜索需 Agent 与请求同时开启(`customAgent.Config.WebSearchEnabled && req.WebSearchEnabled`)、web provider 回退租户默认、`SearchTargets` 由 KB/@文档/@标签 scope 统一构建、`MaxContextTokens` 兜底 200000、`@Skill` 的每轮优先提示与 `@MCP` 的每轮范围收窄(共享 Agent 的 @MCP 只能落在 Agent 预设集合内)。另外只有当 `knowledge_search` 实际可用时才要求配置 rerank 模型(`agentRequiresRerankModel`)。
+运行时映射:`buildAgentConfig`(`session_agent_qa.go`)把 `CustomAgentConfig` 转换为引擎的 `types.AgentConfig`(`internal/types/agent.go`),并叠加:web 搜索需 Agent 与请求同时开启(`customAgent.Config.WebSearchEnabled && req.WebSearchEnabled`)、web provider 回退租户默认、`SearchTargets` 由 KB/@文档/@标签 scope 统一构建、`MaxContextTokens` 兜底 200000、`@Skill` 的每轮优先提示与 `@MCP` 的每轮范围收窄(共享 Agent 的 @MCP 只能落在 Agent 预设集合内)。另外只有当 `search_knowledge` 实际可用时才要求配置 rerank 模型(`agentRequiresRerankModel`,旧名 `knowledge_search` / `grep_chunks` 经 `SuccessorToolName` 归一后同样计入)。
#### 分享机制(agent_share) {#_7-3-分享机制-agent-share}
`internal/application/service/agent_share.go`:Agent 可分享给**组织(Organization)**:
- 仅 Agent 属主租户可分享(`ErrNotAgentOwner`);分享者所在租户须为组织 Editor+ 成员;
-- 分享前校验 Agent 配置完整:必须有 `model_id`;若 `knowledge_search` 在其工具集内(或工具集为空回退默认集)且 KB scope 未禁用,还必须有 `rerank_model_id`,否则 `ErrAgentNotConfigured`;
+- 分享前校验 Agent 配置完整:必须有 `model_id`;若 `search_knowledge` 在其工具集内(或工具集为空回退默认集)且 KB scope 未禁用,还必须有 `rerank_model_id`,否则 `ErrAgentNotConfigured`;
- **权限强制为只读**:`permission = types.OrgRoleViewer`(跨租户编辑不在 v1 范围);重复分享则幂等更新;
- 接收方租户可通过 `TenantDisabledSharedAgentRepository` 把某个共享 Agent 在本租户禁用;
- 使用共享 Agent 对话时(`session_agent_qa.go`),检索与模型 scope 切到 **Agent 属主租户**(`resolveRetrievalTenantID`),因此共享方的 KB 对使用方可用,而使用方自己的 MCP @提及会被限制在 Agent 预设内。
@@ -121,9 +121,9 @@ Handler 层(`internal/handler/custom_agent.go`)提供 `CreateAgent`、`GetAg
| ID | 名称(zh-CN) | agent_mode / agent_type | 关键配置 |
| --- | --- | --- | --- |
| `builtin-quick-answer` | 快速问答 | `quick-answer` | 模板 `default_kb` + `default_context`;temperature 0.7;FAQ 优先(直接回答阈值 0.9、加权 1.2);query expansion + rewrite;web 搜索开、5 条;不进 Agent 引擎 |
-| `builtin-smart-reasoning` | 智能推理 | `smart-reasoning` / `rag-qa` | `max_iterations: 50`;工具:knowledge_search、grep_chunks、list_knowledge_chunks、query_knowledge_graph、get_document_info;web 搜索开;多轮 5 轮 |
+| `builtin-smart-reasoning` | 智能推理 | `smart-reasoning` / `rag-qa` | `max_iterations: 50`;工具:search_knowledge、read_document、list_documents、query_knowledge_graph;web 搜索开;多轮 5 轮 |
| `builtin-data-analyst` | 数据分析师 | `smart-reasoning` / `data-analysis` | 模板 `data_analyst`;temperature 0.3;`max_iterations: 30`;工具仅 data_schema + data_analysis;限定 csv/xlsx;关闭 web 搜索;历史 10 轮 |
-| `builtin-wiki-researcher` | 维基问答 | `smart-reasoning` / `wiki-qa` | 模板 `wiki_researcher`;`max_iterations: 30`;工具:wiki_search、wiki_read_page、wiki_read_source_doc、wiki_flag_issue(只读 + 报障);关闭 web 搜索 |
+| `builtin-wiki-researcher` | 维基问答 | `smart-reasoning` / `wiki-qa` | 模板 `wiki_researcher`;`max_iterations: 30`;工具:wiki_search、wiki_read_page、read_document、wiki_flag_issue(只读 + 报障);关闭 web 搜索 |
| `builtin-wiki-fixer` | 维基修订 | `smart-reasoning` / `custom` | 模板 `wiki_fixer`;`retain_retrieval_history: true`(修订需要跨轮记住页面内容);工具含全部 wiki 写操作(wiki_write_page、wiki_replace_text、wiki_rename_page、wiki_delete_page、wiki_read_issue、wiki_update_issue 等 9 个);`kb_selection_mode: selected` |
补充两点(来自 `internal/types/custom_agent.go`):
@@ -241,7 +241,7 @@ flowchart TB
REG["tools.ToolRegistry"]
end
subgraph Tools["工具集"]
- KB["KB 检索工具<br/>knowledge_search / grep_chunks / ..."]
+ KB["KB 检索工具<br/>search_knowledge / read_document / list_documents"]
WIKI["Wiki 工具 x10"]
WEB["web_search / web_fetch"]
DATA["data_schema / data_analysis(DuckDB)"]
@@ -289,7 +289,7 @@ flowchart TB
**运行时上下文(runtime_context)**:与 system prompt 不同,绑定 KB 的完整详情(capabilities、最近文档/FAQ 列表)、@提及的固定文档(pinned_documents)、当前时间、会话 ID,是以 XML 块 `<runtime_context scope="this_turn">` 注入到**当前轮用户消息**里的(`internal/agent/observe.go` 的 `buildRuntimeContextBlock`),且**不持久化**到历史,避免过期 scope 干扰后续轮次。块内还固定携带两条指令:
-- `<communication_instruction>`:禁止在答案/思考中出现内部工具名和内部 ID(要求说"关键词检索"而非 `grep_chunks` 等);
+- `<communication_instruction>`:禁止在答案/思考中出现内部工具名和内部 ID(要求说"知识库检索"而非 `search_knowledge` 等);
- `<answer_instruction>`:信息足够后直接以纯文本写出完整答案并停止(不要再发起工具调用)——这就是 Agent 的终止协议。
当用户 @提及了 MCP 服务或技能时,`buildMustUseBlock` 会额外注入 `<must_use>` 块,强制模型使用对应前缀的 MCP 工具或先用 `read_file` 读取技能说明。
@@ -411,11 +411,10 @@ flowchart TD
| --- | --- | --- |
| `thinking` | `thought`\*、`next_thought_needed`\*、`thought_number`\*、`total_thoughts`\*、`is_revision`、`revises_thought`、`branch_from_thought`、`branch_id`、`needs_more_thoughts` | Sequential Thinking:记录/修订/分支思考步骤;返回思考进度(含 `incomplete_steps`),提示禁止在思考里出现工具名和最终答案 |
| `todo_write` | `task`、`steps[]`\*(`id`/`description`/`status`:pending/in_progress/completed) | 创建/更新检索类任务计划,仅限检索任务(总结交给 thinking);返回格式化计划,`display_type: "plan"` |
-| `knowledge_search` | `queries[]`\*(1–5 条语义问题)、`knowledge_base_ids[]` | 语义/向量检索,可选 rerank;默认 topK=5、vector 阈值 0.6、keyword 阈值 0.5;`minScore` 参数虽然仍可传入且默认 0.3,但**后置过滤已被跳过**——`HybridSearch` 改用 RRF 融合后分数落在 [0, ~0.033] 区间,旧的 [0,1] 阈值不再适用,阈值过滤在 RRF 之前就已由各引擎完成,重排阶段另有 `rerankThreshold()`(优先取全局配置);结果带 `cN`/`dN` 短 ID;会话内已见 chunk 去重压缩 |
-| `grep_chunks` | `query`\*(单条 POSIX 正则,支持 `\|` 交替) | 直接在 DB 做大小写不敏感正则匹配(PostgreSQL `~*` / MySQL `REGEXP`);上限 30 条,>10 条时做 MMR(λ=0.7)去冗;返回 `<match>` 片段、按文档聚合摘要(最多 20 行);已见 chunk 标 `already_seen` |
-| `list_knowledge_chunks` | `faq_id` / `chunk_id` / `knowledge_id`(三选一)、`limit`(默认 20 上限 100)、`offset` | 读取单个 FAQ/chunk 或分页遍历某文档全部分块;校验 KB 在 searchTargets 内及 @mention 范围 |
-| `query_knowledge_graph` | `knowledge_base_ids[]`\*(1–10 个 `bN`)、`query`\* | 并发查询各 KB 知识图谱的实体与关系;未配置图谱的 KB 退化为普通检索结果 |
-| `get_document_info` | `knowledge_ids[]`(`dN`)、`faq_ids[]`(`cN`)(至少一个) | 并发批量返回文档元数据(标题、类型、大小、parse_status、分块数)或 FAQ 标准问/答案 |
+| `search_knowledge` | `query`\*(一条自然语言问题或短语;keyword 模式下写精确词)、`mode`(`hybrid` 默认 / `semantic` / `keyword`)、`knowledge_base_ids[]`(`bN`)、`limit`(默认 10,上限 30) | 唯一的知识库检索入口:`hybrid` 走向量 + 关键词的 RRF 融合,`semantic` 只走向量,`keyword` 由关键词索引(BM25 / 引擎关键词检索)提供,不再对 chunks 表做无索引的正则扫描;默认 vector 阈值 0.6、keyword 阈值 0.5,阈值过滤在 RRF 之前由各引擎完成;有 rerank 模型时重排(`rerankThreshold()` 优先取全局配置),候选超过 `limit` 时做 MMR(λ=0.7)去冗;结果带 `cN`/`dN` 短 ID,会话内已见 chunk 去重压缩;所选 KB 没有对应索引时按库降级而不是报错:`keyword` 遇到 FAQ 库或纯向量库改走语义检索,`semantic` 遇到纯关键词库改走关键词检索,结果里以 `requested_mode` 和 `mode_fallbacks` 标明哪些库降级及原因;只有作用域内没有任何分块索引(如全是仅 Wiki 的库)时才报错 |
+| `read_document` | `id`\*(`dN` 文档句柄或 `cN` 分块句柄)、`offset`(阅读顺序中的位置,从 0 开始,不是 chunk 下标;翻页用返回的 `next_offset`)、`limit`(默认 20,上限 100)、`query`(文档内查找,默认字面量、大小写不敏感)、`regex`(把 `query` 当 POSIX 正则)、`context`(`cN` 前后各带几个相邻分块,上限 5) | 始终先返回文档元数据头(标题、类型、parse_status、分块数、metadata),再按需返回分块:`dN` 从 `offset` 起分页遍历;`cN` 读取该分块并可带前后 `context`(邻居按 chunk_index 顺序取,不受父分块、摘要、图片分块占用的下标影响);带 `query` 时即使 `id` 是 `cN` 也在其所属文档内查找;`query` 返回命中分块及前后各一块上下文(最多 20 处命中);FAQ 条目按同样方式读取;校验 KB 在 searchTargets 内及 @mention 范围 |
+| `list_documents` | `knowledge_base_id`\*(`bN`)、`keyword`(标题子串过滤)、`page`(默认 1)、`page_size`(默认 20,上限 100) | 分页列出单个知识库的文档,返回可直接交给 `read_document` 的 `dN` 句柄 |
+| `query_knowledge_graph` | `knowledge_base_ids[]`\*(1–10 个 `bN`)、`query`\* | 并发查询各 KB 知识图谱的实体与关系;只有当作用域内存在启用图谱的 KB 时才会提供给模型(`agent_service.go` 装配白名单时移除),能力要求 `all_of: [graph]` |
| `database_query` | SQL(SELECT-only) | 只读查询白名单表(`knowledge_bases`/`knowledges`/`chunks`),自动注入 tenant_id 过滤与 `deleted_at IS NULL`;SQL 参数在 UI/Langfuse 中脱敏 |
| `data_schema` | `knowledge_id`\*(`dN`) | 读取 CSV/Excel 文件的 `table_summary` + `table_column` 类型分块,返回列信息与行数;并告知模型该文档在 `data_analysis` 中固定以表名 `dataset` 访问 |
| `data_analysis` | `knowledge_id`\*、`sql`\* | 把 CSV/Excel 载入 DuckDB 后执行 SQL。文档由 `knowledge_id` 选定,SQL 中固定以表名 `dataset` 引用它(每次查询在独立连接上建临时视图映射到物理表,模型永远不需要在 SQL 里写文档 ID);多 Sheet Excel 合并为一张表并暴露 `__sheet_name` 列;自动纠正列名大小写/空格差异;会话结束 Cleanup 时 DROP 所建表 |
@@ -428,9 +427,8 @@ flowchart TD
| `edit_sandbox_file` | path、edits | 基于原版本批量精确替换 |
| `search_memory` | query、limit | 按当前调用者作用域查长期记忆 |
| `search_conversations` | 查询与范围 | 检索当前调用者可用的历史对话 |
-| `wiki_search` | `queries[]`\*(正则)、`limit`(默认 10)、`knowledge_base_id` | 在 Wiki 页面(标题/内容/slug/摘要)上做 POSIX 正则搜索,返回带 `bN` 标记的页面与摘要;已见 slug 去重 |
+| `wiki_search` | `query`\*(大小写不敏感的 POSIX 正则,如 `stardust\|skyvault`;不是合法正则的文本如 `C++` 按字面匹配)、`regex`(`false` 强制字面匹配,`true` 要求合法正则)、`knowledge_base_ids[]`(`bN`)、`limit`(每个 KB 默认 10,上限 50);旧参数 `queries[]`、`knowledge_base_id` 仍接受 | 在 Wiki 页面(标题/slug/别名/摘要/内容)上搜索,返回带 `bN` 标记的页面 slug 与摘要;已见 slug 去重 |
| `wiki_read_page` | `slugs[]`\*、`knowledge_base_id` | 按 slug 读取 Wiki 页面全文、元数据、出入链(链接附摘要,已见的省略);`index` slug 返回按类型分组的目录概览(每类 top 20) |
-| `wiki_read_source_doc` | `knowledge_id`\*(`dN`)、`query`(正则)、`start_chunk_index`、`end_chunk_index` | 深入阅读 Wiki 页面的源文档:正则过滤或按 chunk 区间取连续内容;都不传则返回文档开头 |
| `wiki_write_page` | `slug`\*、`title`\*、`summary`\*、`content`\*、`page_type`\*、`aliases[]`、`source_refs[]` | 新建或整页覆盖 Wiki 页面;写入前规范化并校验 slug;自动处理出链 |
| `wiki_replace_text` | `slug`\*、`old_text`\*、`new_text`\*、`source_refs[]` | 精确文本替换,适合小修订 |
| `wiki_rename_page` | `slug`\*、`new_slug`\* | 重命名 slug 并级联更新所有引用它的页面链接 |
@@ -440,7 +438,13 @@ flowchart TD
| `wiki_update_issue` | `issue_id`\*、`status`\*(resolved/ignored/pending) | 更新 issue 状态 |
| `mcp_{service}_{tool}`(动态) | 由 MCP 服务的 InputSchema 决定 | 包装外部 MCP 工具;描述前缀 `[MCP Service: X (external)]` 提示不可信来源;可挂人工审批与会话内 OAuth |
-默认工具白名单 `DefaultAllowedTools()`(旧 Agent 未配置 `allowed_tools` 时的回退):`thinking`、`todo_write`、`knowledge_search`、`grep_chunks`、`list_knowledge_chunks`、`query_knowledge_graph`、`get_document_info`、`database_query`、`data_analysis`、`data_schema`。
+默认工具白名单 `DefaultAllowedTools()`(新建 Agent 的默认值,也是旧 Agent 未配置 `allowed_tools` 时的回退):`search_knowledge`、`read_document`、`list_documents`、`search_conversations`(`search_memory` 与 `web_search` 一样不在此列表里,由记忆 / 联网开关在注册工具时注入或剔除)。
+
+**旧工具名的兼容**:`knowledge_search`、`grep_chunks` 已合并为 `search_knowledge`;`list_knowledge_chunks`、`get_document_info`、`wiki_read_source_doc` 已合并为 `read_document`。`definitions.go` 的 `legacyToolSuccessors` 记录这组映射,`NormalizeAllowedTools` 在注册工具时把已保存 Agent 配置、预设与 API 调用里的旧名字自动改写为新工具,无需数据迁移;历史消息中记录的旧工具名仍能正常渲染。
+
+**文档阅读工具的可用范围**:`read_document`、`list_documents` 读取的是落库的分块,所有知识库无论索引策略都会写入分块,因此仅 Wiki 的知识库上它们照样注册(`agent_service.go` 的 `documentToolSet`),供 Wiki 智能体回读原文;`search_knowledge` 仍要求向量或关键词索引。能力表里这两个工具标为 `Auxiliary`:它们能用于仅 Wiki 的库,但不会把仅 Wiki 的库拉进 RAG 智能体「全部知识库」的范围,派生 KB 过滤器时只有在没有其他知识库工具时才计入。`list_documents` 的 @文件 / @标签 范围在分页之前生效:标签下推到数据库过滤条件,指定文档直接按 ID 读取,`total_docs` 与 `next_page` 只统计范围内的文档。
+
+**推荐的检索工作流**:`search_knowledge`(按问题选择 `mode`:默认 `hybrid`,精确词 / 报错信息 / 标识符用 `keyword`,改写或概念性问题用 `semantic`)→ `read_document`(按 `dN` 分页阅读上下文,或用 `query` 在文档内定位)→ 在答案里以 `cN` 句柄引用。Wiki 知识库则是 `wiki_search` → `wiki_read_page` → `read_document` 回读原始来源。
#### 工具注册表(ToolRegistry) {#_3-2-工具注册表-toolregistry}
@@ -461,8 +465,12 @@ flowchart TD
var ToolCapabilityRequirements = map[string]ToolRequirement{
"thinking": {},
"todo_write": {},
- "knowledge_search": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
- "grep_chunks": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
+ "search_knowledge": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
+ "read_document": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
+ "list_documents": {AnyOf: []KBCapability{CapVector, CapKeyword}, ConsumesFiles: true},
+ "query_knowledge_graph": {AllOf: []KBCapability{CapGraph}, ConsumesFiles: true},
+ // 旧名 knowledge_search / grep_chunks / list_knowledge_chunks / get_document_info / wiki_read_source_doc
+ // 保留同样的声明,以便旧配置在归一化之前也能通过能力校验
// ...
"wiki_search": {AllOf: []KBCapability{CapWiki}},
// ...
@@ -516,7 +524,7 @@ var ToolCapabilityRequirements = map[string]ToolRequirement{
- 每轮展开为:user 消息(含图片 caption 与附件 prompt;忽略 `RenderedContent` 快照,避免将旧渲染协议带入上下文)→ 每个含工具调用的 `AgentStep` 展开为 assistant(with tool_calls) + 若干 tool 消息 → 末尾一条规范化最终答案 assistant 消息(剥离 `<think>` 块);
- 历史中的 tool 消息内容用 `CompactToolOutputForHistory`(`internal/agent/tools/persist.go`)压缩:带 `display_type` 的大载荷(如 `knowledge_chunks_list` 的 chunks、`grep_results` 的 chunk_results)替换为一行摘要(如 `"Listed 20/87 chunks from X (content omitted from history)"`)。
-进入引擎后,`buildMessagesWithLLMContext` 还会做**历史 KB 结果脱敏**(`redactHistoryKBResults`):除非 Agent 开启 `RetainRetrievalHistory`,历史轮次中 KB 类工具(`knowledge_search`、`grep_chunks`、`list_knowledge_chunks`、`query_knowledge_graph`、`get_document_info`、`wiki_search`、`wiki_read_page`、`wiki_read_source_doc`)的结果一律替换为 `"[Previous retrieval result omitted — knowledge base may have changed. Please perform a fresh search.]"`,强制模型对可能已变更的知识库做新鲜检索。
+进入引擎后,`buildMessagesWithLLMContext` 还会做**历史 KB 结果脱敏**(`redactHistoryKBResults`):除非 Agent 开启 `RetainRetrievalHistory`,历史轮次中 KB 类工具(`search_knowledge`、`read_document`、`list_documents`、`query_knowledge_graph`、`wiki_search`、`wiki_read_page`,以及历史里可能残留的旧名 `knowledge_search`、`grep_chunks`、`list_knowledge_chunks`、`get_document_info`、`wiki_read_source_doc`)的结果一律替换为 `"[Previous retrieval result omitted — knowledge base may have changed. Please perform a fresh search.]"`,强制模型对可能已变更的知识库做新鲜检索。
持久化侧,`SanitizeAgentStepsForStorage` 在把 `AgentSteps` 写入 DB / SSE 重放前剥离 LLM-only 大载荷,只留紧凑摘要。
diff --git a/website-docs/03-features/08-mcp.md b/website-docs/03-features/08-mcp.md
index ac4864b2e..0177d62fd 100644
--- a/website-docs/03-features/08-mcp.md
+++ b/website-docs/03-features/08-mcp.md
@@ -34,11 +34,13 @@ OAuth 服务按调用者分别授权。工具需要审批时,在对话中检
| 组 | 工具 | 说明 |
|---|---|---|
-| 检索与阅读 | `list_knowledge_bases`、`search_knowledge`、`grep_chunks`、`list_documents`、`read_document` | 知识库参数同时接受 ID 或名称;语义搜索与关键词正则搜索分开 |
+| 检索与阅读 | `list_knowledge_bases`、`search_knowledge`、`grep_chunks`、`list_documents`、`read_document` | 知识库参数同时接受 ID 或名称;`search_knowledge` 用 `mode`(hybrid / semantic / keyword)选择检索方式并可设 `limit`(默认 10,上限 30);`grep_chunks` 保持大小写不敏感的正则语义:从模式里提取字面词作为关键词索引的检索词(没有关键词索引的库改用语义索引取候选),再逐条用正则校验,返回的分块都匹配该模式;不含任何字面词的模式(如 `^\d+$`)会被拒绝;`read_document` 按 `offset` / `limit` 翻页,或用 `query` 在文档内查找短语 |
| 问答 | `ask` | 只运行端点配置的默认 Agent(客户端不能自选 Agent),服务端自动建会话,返回带引用的完整回答和 `session_id`,续聊时传回即可;不开启联网搜索 |
-| Wiki | `wiki_search`、`wiki_read_page`、`wiki_index` | 只对开启了 Wiki 的知识库生效 |
+| Wiki | `wiki_search`、`wiki_read_page`、`wiki_index` | 只对开启了 Wiki 的知识库生效;`wiki_search` 的 `query` 保持原有的正则语义(大小写不敏感),不是合法正则的文本按字面匹配;`regex=false` 强制字面匹配,`regex=true` 要求合法正则 |
| 写入 | `add_document`、`update_document`、`delete_document` | 默认关闭;支持 Markdown 文本或 URL 导入 |
+> **`grep_chunks` 的召回有上限。** 它基于索引取候选,每次最多 30 条,再用正则筛选,所以返回的每条都匹配模式,但不保证穷尽:库里存在的匹配也可能没进候选池。容易漏的情况有三类:`foo.*bar` 这类组合模式,候选按 foo、bar 的相关度排序,真正相邻出现的分块可能排不进前 30;`C++` 这类几乎只剩符号的模式,抽出的字面词只有 `C`,在索引里几乎没有区分度;没有关键词索引的库改用语义索引取候选,字面匹配更依赖运气。旧实现对 chunks 表做全表正则扫描,能保证"有就能找到",但数据量大时代价过高,已经移除。需要在某篇文档里穷尽查找时,用 `read_document` 的 `query`,它会顺序扫完整篇文档。
+
工具实现直接复用 Agent 的原生工具(`internal/agent/tools/`),鉴权复用 API Key 的作用域模型:端点被换算成一把仅含 retrieve / chat / ingest 等能力、限定知识库范围的作用域,所以后端各服务对它的检查与对受限 API Key 完全一致。实现细节见下方内置 MCP Server 参考。
## 接入方式对照 {#两个方向的对照速览}
@@ -387,7 +389,7 @@ flowchart LR
A -->|"注入 tenant / principal /<br/>TenantAPIKeyScope / *MCPEndpoint"| S["mcp-go StreamableHTTPServer<br/>(internal/mcpserver)"]
S -->|"tools/list"| F["ToolFilter:按端点白名单过滤"]
S -->|"tools/call"| G["Guard:白名单 + 限流 + last_used"]
- G --> T["工具处理器:复用 internal/agent/tools<br/>KnowledgeSearch / GrepChunks / ListChunks / Wiki…"]
+ G --> T["工具处理器:复用 internal/agent/tools<br/>SearchKnowledge / ReadDocument / ListDocuments / Wiki…"]
G --> Q["ask:SessionService.AgentQA / KnowledgeQA<br/>同步收集 final_answer + references"]
```
diff --git a/website-docs/03-features/09-knowledge-graph.md b/website-docs/03-features/09-knowledge-graph.md
index acf970ac3..b40435b6f 100644
--- a/website-docs/03-features/09-knowledge-graph.md
+++ b/website-docs/03-features/09-knowledge-graph.md
@@ -122,7 +122,7 @@ SET node.chunks = apoc.coll.union(node.chunks, row.chunks)
1. **PluginExtractEntity**(`extract_entity.go`,挂在 `QUERY_UNDERSTAND` 事件):`NEO4J_ENABLE=true` 时,先筛出 `ExtractConfig.Enabled` 的知识库(存入 `chatManage.EntityKBIDs` / `EntityKnowledge`),再用 `ExtractManager.ExtractEntity` 模板 + Chat 模型从**用户查询**里抽取实体名,存入 `chatManage.Entity`。
2. **PluginSearchEntity**(`search_entity.go`,挂在 `ENTITY_SEARCH` 事件):对每个启用图谱的知识库 / 文件并行调用 `graphRepo.SearchNode`——Cypher 用 `n.name CONTAINS nodeText` 模糊匹配实体并返回一跳邻居与关系,合并为 `chatManage.GraphResult`;随后 `filterSeenChunk` 取出图谱节点携带的 `chunks`(去掉向量检索已命中的),从 `chunkRepo` 拉取原文并转换为 `SearchResult` 并入候选集,实现"实体 → 关联 chunk"的图谱补充召回。
-Agent 模式则提供 `query_knowledge_graph` 工具(`internal/agent/tools/query_knowledge_graph.go`):校验各知识库是否配置了图谱(`ExtractConfig.Nodes/Relations` 非空),并发对多库执行检索、按 chunk 去重排序,输出中附带各库的图谱配置状态(实体类型 / 关系类型清单);未配置图谱的库回落为普通混合检索结果。
+Agent 模式则提供 `query_knowledge_graph` 工具(`internal/agent/tools/query_knowledge_graph.go`):校验各知识库是否配置了图谱(`ExtractConfig.Nodes/Relations` 非空),并发对多库执行检索、按 chunk 去重排序,输出中附带各库的图谱配置状态(实体类型 / 关系类型清单);未配置图谱的库回落为普通混合检索结果。该工具的能力要求是 `all_of: [graph]`,并且只有当 Agent 作用域内存在启用图谱的知识库时才会提供给模型——`agent_service.go` 装配工具白名单时会把它从没有图谱库的作用域中移除,避免模型反复调用一个只能返回退化结果的工具。
## 流程图
diff --git a/website-docs/03-features/11-web-search.md b/website-docs/03-features/11-web-search.md
index 5f0b36d8a..01f7c4e22 100644
--- a/website-docs/03-features/11-web-search.md
+++ b/website-docs/03-features/11-web-search.md
@@ -70,7 +70,7 @@ CRUD 路由(`RegisterWebSearchProviderRoutes`,`internal/router/router.go`)
## Agent 搜索与读页
-`web_search` 负责发现来源,`web_fetch` 负责读取选中的页面。用户指定网页时可直接读页;用户要求外部或实时信息时可直接搜索。知识库是否需要检索取决于任务相关性与当前可用工具,不再强制先调用 `grep_chunks` 和 `knowledge_search`。
+`web_search` 负责发现来源,`web_fetch` 负责读取选中的页面。用户指定网页时可直接读页;用户要求外部或实时信息时可直接搜索。知识库是否需要检索取决于任务相关性与当前可用工具,不再强制先调用 `search_knowledge`。
```mermaid
flowchart TD
diff --git a/website-docs/03-features/14-wiki.md b/website-docs/03-features/14-wiki.md
index 36231a42e..9b337f193 100644
--- a/website-docs/03-features/14-wiki.md
+++ b/website-docs/03-features/14-wiki.md
@@ -166,22 +166,22 @@ migration `000061_wiki_page_hierarchy.up.sql` 引入独立的 `wiki_folders` 表
### 与 Agent 的关系
-智能体可通过以下 10 个 Wiki 工具读取、修改页面并处理问题,工具定义位于 `internal/agent/tools/definitions.go`:
+智能体可通过以下 9 个 Wiki 工具读取、修改页面并处理问题(回读源文档则复用通用的 `read_document`),工具定义位于 `internal/agent/tools/definitions.go`:
| 工具 | 作用 | 关键参数 |
| --- | --- | --- |
| `wiki_read_page` | 按 slug 批量读取页面全文 | `slugs: string[]` |
-| `wiki_search` | 正则搜索页面 | `queries`、`limit?`、`knowledge_base_id?` |
+| `wiki_search` | 搜索页面(标题 / slug / 别名 / 摘要 / 内容;`query` 按大小写不敏感的 POSIX 正则解释,不是合法正则的文本如 `C++` 按字面匹配;`regex=false` 强制字面匹配) | `query`、`regex?`、`knowledge_base_ids?`、`limit?`(旧参数 `queries`、`knowledge_base_id` 仍接受) |
| `wiki_write_page` | 创建/整页覆盖(`synthesis`、`comparison` 页只能由此创建) | `slug`、`title`、`summary`、`content`、`page_type`、`aliases?`、`source_refs?` |
| `wiki_replace_text` | 页内精确文本替换 | `slug`、`old_text`、`new_text` |
| `wiki_rename_page` | 重命名 slug,自动更新反向链接 | `slug`、`new_slug` |
| `wiki_delete_page` | 删除页面并清理死链 | `slug` |
-| `wiki_read_source_doc` | 回读源文档原文(带上下文) | 文档 ID |
+| `read_document` | 回读源文档原文:元数据头 + 分块,可分页或用 `query` 在文档内定位(替代原 `wiki_read_source_doc`;仅 Wiki 的知识库同样可用,因为分块总会落库) | `id`(`dN` / `cN`)、`offset?`、`limit?`、`query?`、`regex?`、`context?` |
| `wiki_flag_issue` | 标记页面问题 | `slug`、`issue_type ∈ {mixed_entities, contradictory_facts, out_of_date, other}`、`description` |
| `wiki_read_issue` | 查看问题详情 | 问题 ID |
| `wiki_update_issue` | 更新问题状态 | 问题 ID、`status ∈ {pending, ignored, resolved}` |
-工具输出为 XML-like 结构(`<wiki_page><metadata>...<summary>...<content>...`),前端用 `frontend/src/utils/wikiToolReferences.ts` 的 `parseWikiToolReferences()` 解析成引用卡片渲染在对话中。
+推荐的阅读顺序是 `wiki_search` → `wiki_read_page` → `read_document`:先定位页面,再读整页,需要精确引用时回到原始来源的 `cN` 分块。工具输出为 XML-like 结构(`<wiki_page><metadata>...<summary>...<content>...`),前端用 `frontend/src/utils/wikiToolReferences.ts` 的 `parseWikiToolReferences()` 解析成引用卡片渲染在对话中。
配套机制:
diff --git a/website-docs/06-development/03-extension-points.md b/website-docs/06-development/03-extension-points.md
index a677baea2..404ef480c 100644
--- a/website-docs/06-development/03-extension-points.md
+++ b/website-docs/06-development/03-extension-points.md
@@ -604,14 +604,16 @@ func (r *ToolRegistry) ListTools() []string
### 现有实现
-工具名常量集中在 `internal/agent/tools/definitions.go`:`thinking`、`todo_write`、`grep_chunks`、`knowledge_search`、`list_knowledge_chunks`、`query_knowledge_graph`、`get_document_info`、`database_query`、`data_analysis`、`data_schema`、`web_search`、`web_fetch`、沙箱/技能工具(`shell_exec`、`read_file`、`list_sandbox_files`、`write_sandbox_file`、`edit_sandbox_file`),记忆工具(`search_memory`、`search_conversations`)、wiki 工具(`wiki_read_page`、`wiki_write_page`、`wiki_replace_text`、`wiki_rename_page`、`wiki_delete_page`、`wiki_search`、`wiki_read_source_doc`、`wiki_flag_issue`、`wiki_read_issue`、`wiki_update_issue`)。实现文件与工具同名(如 `grep_chunks.go`、`knowledge_search.go`、`data_analysis.go`、`mcp_tool.go`——后者把 MCP 服务的远程工具包装成 `types.Tool`)。
+工具名常量集中在 `internal/agent/tools/definitions.go`:`thinking`、`todo_write`、知识检索工具(`search_knowledge`、`read_document`、`list_documents`、`query_knowledge_graph`)、`database_query`、`data_analysis`、`data_schema`、`web_search`、`web_fetch`、沙箱/技能工具(`shell_exec`、`read_file`、`list_sandbox_files`、`write_sandbox_file`、`edit_sandbox_file`),记忆工具(`search_memory`、`search_conversations`)、wiki 工具(`wiki_read_page`、`wiki_write_page`、`wiki_replace_text`、`wiki_rename_page`、`wiki_delete_page`、`wiki_search`、`wiki_flag_issue`、`wiki_read_issue`、`wiki_update_issue`)。实现文件与工具同名(如 `search_knowledge.go`、`read_document.go`、`list_documents.go`、`data_analysis.go`、`mcp_tool.go`——后者把 MCP 服务的远程工具包装成 `types.Tool`)。
+
+已退役的检索工具名(`knowledge_search`、`grep_chunks`、`list_knowledge_chunks`、`get_document_info`、`wiki_read_source_doc`)仍以 `LegacyTool*` 常量保留:`legacyToolSuccessors` 把它们映射到 `search_knowledge` / `read_document`,`NormalizeAllowedTools` 在注册工具时自动改写已保存 Agent 配置里的旧名字,`SuccessorToolName` / `IsLegacyRetrievalTool` 供其他服务判定。重命名或合并工具时请沿用这一机制,而不是做数据迁移。
### 新增步骤
1. 在 `internal/agent/tools/` 新建 `my_tool.go`,实现 `types.Tool` 四个方法(`Parameters()` 返回 JSON Schema;注意工具名 ≤ 64 字符的 OpenAI 限制,见 `definitions.go` 的 `maxFunctionNameLength`);
2. **注册点一:`internal/agent/tools/definitions.go`** — 增加 `ToolMyTool = "my_tool"` 常量,并把工具加进 `AvailableToolDefinitions()`(UI 的可选工具列表,注释明确要求与已注册工具保持同步);
3. **注册点二:Agent 引擎的工具装配处** — 在构建 `ToolRegistry` 的服务逻辑(Agent 会话初始化,按 Agent 配置的允许工具列表实例化并 `RegisterTool`)中加入新工具的构造;带资源清理需求时实现 `Cleanup`(`types.Cleanable`);
-4. 输出体量大的工具注意 `ToolRegistry` 的 `maxToolOutputSize` 截断行为;为工具编写 `_test.go`(同目录有大量参考,如 `grep_chunks_scope_test.go`)。
+4. 输出体量大的工具注意 `ToolRegistry` 的 `maxToolOutputSize` 截断行为;为工具编写 `_test.go`(同目录有大量参考,如 `search_knowledge_test.go`、`scope_authorization_test.go`)。
---