Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
feat: add typed tool result stubs (#2248)
Eurakaxun7b52d8fcd0cc
diff --git a/docs/design/tool-stub-design.md b/docs/design/tool-stub-design.mdnew file mode 100644index 000000000..3fb2b7895--- /dev/null+++ b/docs/design/tool-stub-design.md@@ -0,0 +1,347 @@+# Tool Stub 设计文档++**范围**: OpenViking 当前 `tool stub` 能力的实现说明,覆盖类型识别、规则化摘要、原文外置、回溯读取与测试边界。+**状态**: 已实现,本文档描述当前代码行为,不额外引入新需求。++---++## 概述++OpenViking 原生已经支持 tool result preview。原有链路能够在 session 写入阶段把过大的 tool output externalize,并在 `ToolPart` 中留下一个 preview stub,同时保留 ref 供后续回溯。++这次工作的重点不是新建 externalize 机制,而是在现有能力上优化 preview 的生成方式:从偏 `head + tail` 的直接截断,升级为按内容类型输出更稳定、更可读的规则化摘要。++换句话说,本次改动保持下面这些基础能力不变:++1. 哪些 tool output 需要 externalize,仍由 session 写入阶段决定。+2. 原始内容仍写入 `ToolResultStore`。+3. `ToolPart` 仍保留 stub 和 `tool_output_ref`。+4. 原文回溯方式仍是 `read/search/list`。++这次变化主要集中在 preview 生成层:++1. 在 session 写入阶段识别哪些 tool output 需要 externalize。+2. 原始输出写入 session 下的 tool result store。+3. preview 从简单截断优化为基于内容和 MIME 的 deterministic synopsis。+4. 把原始 `ToolPart.tool_output` 替换成 stub 文本,并保留 `tool_output_ref`。+5. 后续通过 `read/search/list` 工具按 ref 回溯原文。++`text` 类型只做规则化摘要,不接 LLM。++---++## 设计目标++1. 在保留 OV 原生 externalize 和 ref 回溯链路的前提下,优化 preview 的可读性。+2. 减少大 tool output 对上下文窗口的占用。+3. 把原有偏 `head + tail` 的截断 preview,升级为按类型输出的规则化摘要。+4. 对常见文本型输出给出稳定、可读的 deterministic synopsis。+5. 保留原始输出,支持按 ref 精确回溯。+6. 当前版本不做 LLM 摘要,只做 deterministic 的规则化摘要。++---++## 端到端流程++### 1. 选择哪些输出需要 externalize++入口在 [session.py](/F:/code/OpenViking/openviking/session/session.py:685) 的 `_externalize_large_tool_output_group()`。++当前按两类条件触发:++1. 单个 tool output 超过 `threshold_chars`。+2. 同一个 assistant turn 中多个 tool output 的总 inline 体积超过 `assistant_turn_inline_budget_chars`。++命中后会进入 externalization 流程。触发阈值仍沿用 OV 原有配置;规则化摘要本身不改变“什么时候 externalize”。++### 2. 外置原始结果并替换 ToolPart++入口在 [session.py](/F:/code/OpenViking/openviking/session/session.py:606) 的 `_externalize_tool_part()`。++这一步会:++1. 把原始 `tool_output` 写入 `ToolResultStore`。+2. 调用 preview 生成逻辑产出 stub 文本。+3. 用 stub 替换 `ToolPart.tool_output`。+4. 把原始结果的 ref 写入 `ToolPart.tool_output_ref`。++因此被 stub 后,消息里保留的是 preview,不再是完整原始 tool result;原始结果仍在外置存储里。++### 3. 生成 synopsis 和 stub++入口在 [tool_result_store.py](/F:/code/OpenViking/openviking/session/tool_result_store.py:38) 的 `make_preview()`:++1. 原生 preview/stub 能力保留,但内容生成逻辑从简单截断演进为 typed synopsis。+2. `generate_tool_result_synopsis()` 负责类型识别和摘要生成。+3. `render_tool_result_stub()` 负责把摘要渲染成最终 stub 文本。++核心实现位于 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:363)。++当前原则:++1. externalize 触发阈值沿用 OV 原有配置。+2. 常见类型使用固定规则上限生成 synopsis,不依赖 `preview_chars` 控制摘要长度。+3. `preview_chars` 只作为无法规则化时的 fallback head/tail 采样预算,并作为兼容字段保留在 stub header / metadata 中。++### 4. 原文回溯++当前回溯能力由 session 暴露三类工具:++1. [session.py](/F:/code/OpenViking/openviking/session/session.py:932) `read_tool_result()`:按 `offset/limit` 读取原始内容片段。+2. [session.py](/F:/code/OpenViking/openviking/session/session.py:952) `search_tool_result()`:在原始内容中做关键字搜索。+3. [session.py](/F:/code/OpenViking/openviking/session/session.py:972) `list_tool_results()`:列出当前 session 已 externalize 的结果。++对应存储层实现位于 [tool_result_store.py](/F:/code/OpenViking/openviking/session/tool_result_store.py:77)。++---++## 支持的数据类型++当前支持的 synopsis kind 定义在 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:18):++`json` / `csv` / `tsv` / `yaml` / `xml` / `code` / `text` / `unknown`++### 类型对照表++| 类型 | 识别方式 | 处理办法 | stub 中保留内容 |+|---|---|---|---|+| `json` | MIME 含 `json`,或内容以 `{` / `[` 开头且能被 JSON decoder 解析 | 解析 top-level shape,提取 keys、array length、标量示例 | `summary + structure + notable_items` |+| `csv` | 含逗号,且能按 CSV 读成规则表格 | 统计行列数、列名,保留首条数据样例 | `summary + structure` |+| `tsv` | 含制表符,且能按 TSV 读成规则表格 | 统计行列数、列名,保留首条数据样例 | `summary + structure` |+| `yaml` | 满足 YAML 启发式并能 `yaml.safe_load()` 成 dict/list | 提取 top-level keys 和 child type | `summary + structure` |+| `xml` | MIME 含 `xml`,或内容以 `<` 开头且可解析 | 提取 root tag、属性数、子标签计数 | `summary + structure` |+| `code` | 命中代码模式正则 | 提取 imports、symbols、line_count | `summary + structure + notable_items` |+| `text` | 作为最终 fallback | 规则化文本摘要,不保留全文 sample | `summary` |+| `unknown` | 空内容、binary-like 内容,或带明确 MIME 但解析失败的结构化内容 | 无法规则化时使用 fallback head/tail sample | `summary + sample` |++---++## 类型识别顺序++识别顺序定义在 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:363)。++当前顺序如下:++1. 空内容:直接标为 `unknown`。+2. binary-like 内容:如果出现 NUL 或控制字符比例过高,标为 `unknown`。+3. `json`+4. `xml`+5. `tsv`+6. `csv`+7. `yaml`+8. `code`+9. 最终 fallback 为 `text`++这个顺序的目的是优先识别结构化格式,再识别代码,最后才把剩余内容视作普通文本。日志样式输出不再作为独立类型处理,会走 `text`,与 lossless-claw 的 large-file exploration 行为保持一致。++---++## 各类型处理策略++### JSON++实现位于 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:150)。++输出重点:++1. 顶层类型是 object 还是 array。+2. top-level keys,最多 10 个。+3. 子字段是 object / array / scalar。+4. 最多若干条标量示例。+5. 若第一个 JSON value 后还有额外字符,会记入 `trailing_chars_after_first_json_value`。+6. 不额外保留原始 JSON sample。++### CSV / TSV++实现位于 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:218)。++输出重点:++1. 列数与数据行数。+2. 首行列名。+3. 首条数据样例,最多 180 字符。++只接受“列数基本一致”的表格;不规则分隔文本不会被误判成表格。++### YAML++实现位于 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:184) 与 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:268)。++输出重点:++1. 顶层是 object 还是 array。+2. top-level keys,最多 30 个。+3. 每个 key 对应的 child type。+4. 不额外保留 YAML sample。++### XML++实现位于 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:204)。++输出重点:++1. 根标签名。+2. 根节点属性数量。+3. 一级子标签频次,最多 30 个。+4. 不额外保留 XML sample。++### Code++实现位于 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:280)。++输出重点:++1. 总行数。+2. import 语句,最多 12 条,单条最多 180 字符。+3. 顶层 symbol,如 `class Foo`、`def bar`、`fn baz`,最多 24 条,单条最多 200 字符。+4. 不额外保留 head/tail sample。++当前是轻量规则识别,不做 AST 级代码摘要。++### Text++实现位于 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:319)。++`text` 类型明确不接 LLM,只做 deterministic fallback。摘录采用固定上限,不受 `preview_chars` 影响。++日志样式输出也归入 `text`。如果需要定位错误/警告行,优先通过 stub 中的 ref 使用 `openviking_tool_result_search` 搜索原始 payload,避免仅靠关键字把普通文档误判成日志。++输出重点:++1. `Characters`+2. `Words`+3. `Lines`+4. `Detected section headers`+5. `Opening excerpt`+6. `Closing excerpt`++标题提取规则:++1. Markdown 标题,如 `# Heading`+2. 全大写风格标题行,如 `SYSTEM STATUS`++摘录规则:++1. opening excerpt 固定最多取前 500 字符。+2. closing excerpt 固定最多取后 500 字符。+3. 先压缩空白,再写入摘要。+4. 不额外保留 `sample` 字段,避免把完整原文重新带回上下文。++### Unknown++`unknown` 是当前实现里的保守分类,不是富类型支持。++当前会落到 `unknown` 的场景包括:++1. 输出为空。+2. 文本中存在明显二进制控制字符。+3. MIME 明确标成 JSON/XML,但内容解析失败。++对这些内容,stub 保留基础说明;非空内容会使用 `preview_chars` 生成 head/tail fallback sample。原始 payload 仍通过 ref 回溯。++---++## Stub 文本结构++渲染逻辑位于 [tool_result_synopsis.py](/F:/code/OpenViking/openviking/session/tool_result_synopsis.py:436)。++当前 stub 由两部分组成:++### Header++包含:++1. `tool_name`+2. `kind`+3. `original_chars`+4. `preview_chars`+5. `ref`+6. `sha256`+7. `reason`++### Body++按 synopsis 内容选择性渲染:++1. `Synopsis`+2. `Structure`+3. `Notable items`+4. `Sample`+5. `Explore`++其中 `Explore` 会提示模型使用:++1. `openviking_tool_result_search`+2. `openviking_tool_result_read`+3. `openviking_tool_result_list`++---++## 原始内容存储与回溯++实现位于 [tool_result_store.py](/F:/code/OpenViking/openviking/session/tool_result_store.py:101)。++每个 externalized tool result 会写入两类文件:++1. `output.txt`:原始输出内容。+2. `metadata.json`:元数据和 synopsis。++元数据包括:++1. `tool_result_id`+2. `session_id`+3. `message_id`+4. `tool_id`+5. `tool_name`+6. `agent_id`+7. `created_at`+8. `original_chars`+9. `preview_chars`+10. `sha256`+11. `mime_type`+12. `synopsis_kind`+13. `synopsis`+14. `storage_uri`+15. `output_uri`+16. `offset_unit=unicode_code_point`++### 读取方式++1. `read()`:按 `offset/limit` 读取原始文本片段,适合长文本逐段展开。+2. `search()`:在原始文本中查关键词,并返回带 offset 的 snippet。+3. `list()`:按 session 列出已有外置结果,便于发现 ref。++当前读取模型是“面向长文本”的;它适合回看日志、代码、表格文本、普通文本。++---++## 当前边界与取舍++1. `text` 不接 LLM,原因是我们当前只需要稳定、低成本、可测试的规则化 stub。+2. 当前回溯接口是 `read/search/list`,更适合大文本原文回看。+3. `offset/limit` 对“顺序文本展开”很合适,但对图片、二进制、复杂多模态结果并不理想。++---++## 测试覆盖++当前相关测试包括:++1. [test_tool_result_synopsis.py](/F:/code/OpenViking/tests/session/test_tool_result_synopsis.py:1):覆盖类型识别和 synopsis 生成,包括固定 caps、text 的 500/500 deterministic fallback,以及 unknown 的 head/tail fallback。+2. [test_tool_result_externalization.py](/F:/code/OpenViking/tests/session/test_tool_result_externalization.py:1):覆盖 externalization、stub 替换、阈值边界、aggregate budget、ref 回溯等端到端流程。+3. [test_api_sessions.py](/F:/code/OpenViking/tests/server/test_api_sessions.py:190):覆盖 HTTP API 层的 tool result externalization、stub 文案、`read/list/search` 回溯,以及 `synopsis_kind` / `synopsis.kind` 元数据透出。++当前相关测试共 29 个用例通过,可作为后续继续补齐真实输出回归用例的基础。++---++## 结论++OpenViking 当前的 `tool stub` 已经具备一版完整闭环:++1. OV 原生的 externalize、preview stub、ref 回溯链路继续保留。+2. preview 生成方式已从偏 `head + tail` 的截断,升级为按类型的规则化摘要。+3. `text` 类型采用 deterministic 规则摘要,不接 LLM。+4. 能通过 `read/search/list` 对外置原文进行回溯。++后续优先级应放在更深的回归测试、更多真实输出样本,以及是否需要继续优化 `read/search/list` 的原文回溯体验,而不是先扩展 LLM 摘要或媒体解析。diff --git a/openviking/session/session.py b/openviking/session/session.pyindex 183a983f2..193d05ad1 100644--- a/openviking/session/session.py+++ b/openviking/session/session.py@@ -18,7 +18,14 @@ from openviking.message import Message, Partfrom openviking.message.part import ContextPart, TextPart, ToolPartfrom openviking.server.config import ToolOutputExternalizationConfigfrom openviking.server.identity import RequestContext, Role-from openviking.session.tool_result_store import ToolResultStore, make_preview, sha256_text+from openviking.session.tool_result_synopsis import ToolResultSynopsis, generate_tool_result_synopsis+from openviking.session.tool_result_store import (+ ToolResultStore,+ build_tool_result_id,+ make_preview,+ render_preview_from_synopsis,+ sha256_text,+)from openviking.telemetry import get_current_telemetry, tracerfrom openviking.telemetry.request_wait_tracker import get_request_wait_trackerfrom openviking.utils.time_utils import get_current_timestamp@@ -581,6 +588,7 @@ class Session:sha256=sha256_text(output) if output else "",reason="source_read",original_chars=len(output),+ mime_type=part.tool_output_mime_type or "text/plain",)part.tool_output = previewpart.tool_output_ref = source_ref@@ -608,6 +616,7 @@ class Session:reason: str,group_id: str,group_original_chars: int,+ synopsis: Optional[ToolResultSynopsis] = None,) -> None:store = self._tool_result_store()original_output = part.tool_output or ""@@ -626,6 +635,7 @@ class Session:created_at=msg.created_at,preview_chars=preview_chars,mime_type=part.tool_output_mime_type or "text/plain",+ synopsis=synopsis,))except Exception as exc:@@ -644,6 +654,7 @@ class Session:sha256=digest,reason=f"{reason}:externalization_failed",original_chars=len(original_output),+ mime_type=part.tool_output_mime_type or "text/plain",)part.tool_output_ref = ""part.tool_output_truncated = True@@ -654,14 +665,14 @@ class Session:returnref = stored.storage_uri- part.tool_output = make_preview(- original_output,- preview_chars=preview_chars,+ part.tool_output = render_preview_from_synopsis(+ stored.synopsis,ref=ref,tool_name=part.tool_name,sha256=digest,reason=reason,original_chars=len(original_output),+ preview_chars=min(len(original_output), max(preview_chars, 0)),)part.tool_output_ref = refpart.tool_output_truncated = True@@ -693,6 +704,9 @@ class Session:group_original_chars = sum(len(p.tool_output or "") for _, p in tool_parts)normal_indices: List[int] = []selected: set[int] = set()+ externalized_preview_cache: Dict[+ tuple[int, int, str], tuple[ToolResultSynopsis, int]+ ] = {}for idx, (_msg, part) in enumerate(tool_parts):part.tool_output_group_id = group_id@@ -711,13 +725,45 @@ class Session:if len(part.tool_output or "") > cfg.threshold_chars:selected.add(idx)+ def prepared_externalized_preview(+ idx: int, part: ToolPart, preview_chars: int+ ) -> tuple[ToolResultSynopsis, int]:+ content = part.tool_output or ""+ reason = "single_threshold" if len(content) > cfg.threshold_chars else "turn_budget"+ cache_key = (idx, preview_chars, reason)+ cached = externalized_preview_cache.get(cache_key)+ if cached is not None:+ return cached++ synopsis = generate_tool_result_synopsis(+ content,+ preview_chars=preview_chars,+ tool_name=part.tool_name,+ mime_type=part.tool_output_mime_type or "text/plain",+ )+ digest = sha256_text(content)+ ref = f"{self._session_uri}/tool-results/{build_tool_result_id(part.tool_id, digest)}"+ rendered = render_preview_from_synopsis(+ synopsis,+ ref=ref,+ tool_name=part.tool_name,+ sha256=digest,+ reason=reason,+ original_chars=len(content),+ preview_chars=min(len(content), max(preview_chars, 0)),+ )+ prepared = (synopsis, len(rendered))+ externalized_preview_cache[cache_key] = prepared+ return prepared+def projected_inline_chars(selected_indices: set[int]) -> int:preview_chars = self._effective_tool_preview_chars(cfg, len(selected_indices))total = 0for idx, (_, part) in enumerate(tool_parts):output_len = len(part.tool_output or "")if idx in selected_indices:- total += min(output_len, preview_chars)+ _synopsis, rendered_len = prepared_externalized_preview(idx, part, preview_chars)+ total += rendered_lenelse:total += output_lenreturn total@@ -730,7 +776,17 @@ class Session:while (projected_inline_chars(selected) > cfg.assistant_turn_inline_budget_chars and remaining):- selected.add(remaining.pop(0))+ baseline = projected_inline_chars(selected)+ chosen_pos = None+ for pos, idx in enumerate(remaining):+ candidate = set(selected)+ candidate.add(idx)+ if projected_inline_chars(candidate) < baseline:+ chosen_pos = pos+ break+ if chosen_pos is None:+ break+ selected.add(remaining.pop(chosen_pos))preview_chars = self._effective_tool_preview_chars(cfg, len(selected))for idx in sorted(selected):@@ -740,6 +796,7 @@ class Session:if len(part.tool_output or "") > cfg.threshold_charselse "turn_budget")+ synopsis, _rendered_len = prepared_externalized_preview(idx, part, preview_chars)self._externalize_tool_part(msg,part,@@ -748,6 +805,7 @@ class Session:reason=reason,group_id=group_id,group_original_chars=group_original_chars,+ synopsis=synopsis,)def _externalize_large_tool_outputs(self, msg: Message) -> None:diff --git a/openviking/session/tool_result_store.py b/openviking/session/tool_result_store.pyindex ec304f0f9..1eb302ab4 100644--- a/openviking/session/tool_result_store.py+++ b/openviking/session/tool_result_store.py@@ -10,6 +10,11 @@ import refrom dataclasses import dataclassfrom typing import Any, Dict, List, Optional+from openviking.session.tool_result_synopsis import (+ ToolResultSynopsis,+ generate_tool_result_synopsis,+ render_tool_result_stub,+)from openviking.storage.viking_fs import VikingFSfrom openviking_cli.exceptions import InvalidArgumentError, NotFoundError@@ -40,35 +45,46 @@ def make_preview(sha256: str = "",reason: str = "",original_chars: Optional[int] = None,+ mime_type: str = "text/plain",) -> str:- """Build a deterministic head+tail preview."""+ """Build a deterministic, typed synopsis stub."""original = len(content) if original_chars is None else original_chars- if preview_chars <= 0 or len(content) <= preview_chars:- body = content- else:- half = max(1, preview_chars // 2)- body = (- "--- BEGIN PREVIEW HEAD ---\n"- f"{content[:half]}\n"- "--- END PREVIEW HEAD ---\n\n"- "--- BEGIN PREVIEW TAIL ---\n"- f"{content[-half:]}\n"- "--- END PREVIEW TAIL ---"- )+ synopsis = generate_tool_result_synopsis(+ content,+ preview_chars=preview_chars,+ tool_name=tool_name,+ mime_type=mime_type,+ )+ return render_preview_from_synopsis(+ synopsis,+ ref=ref,+ tool_name=tool_name,+ sha256=sha256,+ reason=reason,+ original_chars=original,+ preview_chars=min(len(content), max(preview_chars, 0)),+ )- header = [- "[OpenViking tool result externalized]",- f"tool_name: {tool_name or 'tool'}",- f"original_chars: {original}",- f"preview_chars: {min(len(content), max(preview_chars, 0))}",- ]- if ref:- header.append(f"ref: {ref}")- if sha256:- header.append(f"sha256: {sha256[:16]}")- if reason:- header.append(f"reason: {reason}")- return "\n".join(header) + "\n\n" + body++def render_preview_from_synopsis(+ synopsis: ToolResultSynopsis,+ *,+ ref: str = "",+ tool_name: str = "",+ sha256: str = "",+ reason: str = "",+ original_chars: int,+ preview_chars: int,+) -> str:+ return render_tool_result_stub(+ synopsis,+ ref=ref,+ tool_name=tool_name,+ sha256=sha256,+ reason=reason,+ original_chars=original_chars,+ preview_chars=max(preview_chars, 0),+ )@dataclass@@ -78,6 +94,7 @@ class StoredToolResult:output_uri: strmetadata_uri: strmetadata: Dict[str, Any]+ synopsis: ToolResultSynopsisclass ToolResultStore:@@ -115,6 +132,7 @@ class ToolResultStore:created_at: Optional[str],preview_chars: int,mime_type: str = "text/plain",+ synopsis: Optional[ToolResultSynopsis] = None,) -> StoredToolResult:digest = sha256_text(content)tool_result_id = build_tool_result_id(tool_id, digest)@@ -125,16 +143,34 @@ class ToolResultStore:try:existing_metadata = await self.read_metadata(tool_result_id)if existing_metadata.get("sha256") == digest:+ synopsis_data = existing_metadata.get("synopsis")+ synopsis = (+ ToolResultSynopsis.from_dict(synopsis_data)+ if isinstance(synopsis_data, dict)+ else generate_tool_result_synopsis(+ content,+ preview_chars=preview_chars,+ tool_name=tool_name,+ mime_type=mime_type,+ )+ )return StoredToolResult(tool_result_id=tool_result_id,storage_uri=storage_uri,output_uri=output_uri,metadata_uri=metadata_uri,metadata=existing_metadata,+ synopsis=synopsis,)except NotFoundError:pass+ synopsis = synopsis or generate_tool_result_synopsis(+ content,+ preview_chars=preview_chars,+ tool_name=tool_name,+ mime_type=mime_type,+ )metadata = {"tool_result_id": tool_result_id,"session_id": self._session_id,@@ -147,6 +183,8 @@ class ToolResultStore:"preview_chars": min(len(content), max(preview_chars, 0)),"sha256": digest,"mime_type": mime_type,+ "synopsis_kind": synopsis.kind,+ "synopsis": synopsis.to_dict(),"storage_uri": storage_uri,"output_uri": output_uri,"offset_unit": "unicode_code_point",@@ -163,6 +201,7 @@ class ToolResultStore:output_uri=output_uri,metadata_uri=metadata_uri,metadata=metadata,+ synopsis=synopsis,)async def read_metadata(self, tool_result_id: str) -> Dict[str, Any]:diff --git a/openviking/session/tool_result_synopsis.py b/openviking/session/tool_result_synopsis.pynew file mode 100644index 000000000..af5724ebb--- /dev/null+++ b/openviking/session/tool_result_synopsis.py@@ -0,0 +1,484 @@+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.+# SPDX-License-Identifier: AGPL-3.0+"""Deterministic synopsis generation for externalized tool results."""++from __future__ import annotations++import csv+import io+import json+import re+import xml.etree.ElementTree as ET+from collections import Counter+from dataclasses import asdict, dataclass+from typing import Any, Dict, List, Literal, Optional++import yaml++ToolResultKind = Literal["json", "csv", "tsv", "yaml", "xml", "code", "text", "unknown"]++_CODE_PATTERNS = [+ re.compile(r"^\s*(from\s+\S+\s+import|import\s+\S+)", re.MULTILINE),+ re.compile(+ r"^\s*(class|def|async\s+def|function|export\s+function|const|let|var)\s+\w+",+ re.MULTILINE,+ ),+ re.compile(r"^\s*(package|use|pub\s+fn|fn|public\s+class)\s+\w+", re.MULTILINE),+]+_TEXT_HEADER_LIMIT = 18+_TEXT_EXCERPT_CHARS = 500+_JSON_MAX_DEPTH = 2+_JSON_ARRAY_SAMPLE_LIMIT = 3+_JSON_OBJECT_KEY_LIMIT = 10+_YAML_KEY_LIMIT = 30+_XML_CHILD_TAG_LIMIT = 30+_TABLE_FIRST_ROW_SAMPLE_CHARS = 180+_CODE_IMPORT_LIMIT = 12+_CODE_SYMBOL_LIMIT = 24+_CODE_IMPORT_LINE_CHARS = 180+_CODE_SYMBOL_LINE_CHARS = 200+@dataclass(frozen=True)+class ToolResultSynopsis:+ kind: ToolResultKind+ title: str+ summary: List[str]+ structure: List[str]+ notable_items: List[str]+ sample: str = ""++ def to_dict(self) -> Dict[str, Any]:+ return asdict(self)++ @classmethod+ def from_dict(cls, data: Dict[str, Any]) -> "ToolResultSynopsis":+ return cls(+ kind=data.get("kind", "unknown"),+ title=str(data.get("title", "")),+ summary=[str(item) for item in data.get("summary", [])],+ structure=[str(item) for item in data.get("structure", [])],+ notable_items=[str(item) for item in data.get("notable_items", [])],+ sample=str(data.get("sample", "")),+ )+++def _clip(value: str, limit: int) -> str:+ if limit <= 0:+ return ""+ if len(value) <= limit:+ return value+ return value[: max(0, limit - 3)] + "..."+++def _normalize_text_for_line(text: str, max_len: int) -> str:+ compact = re.sub(r"\s+", " ", text).strip()+ return _clip(compact, max_len)+++def _head_tail_sample(content: str, limit: int) -> str:+ if limit <= 0:+ return ""+ if len(content) <= limit:+ return content+ half = max(1, limit // 2)+ return (+ "--- BEGIN SAMPLE HEAD ---\n"+ f"{content[:half]}\n"+ "--- END SAMPLE HEAD ---\n\n"+ "--- BEGIN SAMPLE TAIL ---\n"+ f"{content[-half:]}\n"+ "--- END SAMPLE TAIL ---"+ )+++def _looks_binary(content: str) -> bool:+ if not content:+ return False+ if "\x00" in content:+ return True+ control_chars = sum(+ 1 for ch in content[:1000] if ord(ch) < 32 and ch not in {"\n", "\r", "\t"}+ )+ return control_chars / max(1, min(len(content), 1000)) > 0.05+++def _type_name(value: Any) -> str:+ if isinstance(value, dict):+ return "object"+ if isinstance(value, list):+ return "array"+ if isinstance(value, bool):+ return "boolean"+ if value is None:+ return "null"+ if isinstance(value, (int, float)):+ return "number"+ return "string"+++def _json_scalar_examples(value: Any, *, prefix: str = "", limit: int = 5) -> List[str]:+ examples: List[str] = []+ if len(examples) >= limit:+ return examples+ if isinstance(value, dict):+ for key, child in value.items():+ child_prefix = f"{prefix}.{key}" if prefix else str(key)+ examples.extend(_json_scalar_examples(child, prefix=child_prefix, limit=limit))+ if len(examples) >= limit:+ break+ elif isinstance(value, list):+ if value:+ examples.extend(_json_scalar_examples(value[0], prefix=f"{prefix}[0]", limit=limit))+ else:+ examples.append(f"{prefix}: {_clip(repr(value), 80)}")+ return examples[:limit]+++def _json_shape(value: Any, depth: int = 0) -> str:+ if depth >= _JSON_MAX_DEPTH:+ return "..."+ if isinstance(value, list):+ samples = [_json_shape(item, depth + 1) for item in value[:_JSON_ARRAY_SAMPLE_LIMIT]]+ sample_text = f", sample=[{', '.join(samples)}]" if samples else ""+ return f"array(len={len(value)}{sample_text})"+ if isinstance(value, dict):+ keys = [str(key) for key in list(value.keys())[:_JSON_OBJECT_KEY_LIMIT]]+ key_text = f": {', '.join(keys)}" if keys else ""+ return f"object(keys={len(value)}{key_text})"+ return _type_name(value)+++def _summarize_json(value: Any, *, trailing_chars: int = 0) -> ToolResultSynopsis:+ summary = [f"JSON {_type_name(value)}."]+ structure: List[str] = []+ notable_items: List[str] = []+ if isinstance(value, dict):+ keys = [str(key) for key in value.keys()]+ summary.append(f"top-level keys: {', '.join(keys[:_JSON_OBJECT_KEY_LIMIT])}")+ structure.append(f"shape: {_json_shape(value)}")+ for key, child in list(value.items())[:_JSON_OBJECT_KEY_LIMIT]:+ if isinstance(child, list):+ structure.append(f"{key}: array length: {len(child)}")+ elif isinstance(child, dict):+ child_keys = ", ".join(str(k) for k in list(child.keys())[:_JSON_OBJECT_KEY_LIMIT])+ structure.append(f"{key}: object keys: {child_keys}")+ else:+ structure.append(f"{key}: {_type_name(child)}")+ elif isinstance(value, list):+ summary.append(f"array length: {len(value)}")+ structure.append(f"shape: {_json_shape(value)}")+ if value:+ structure.append(f"first item type: {_type_name(value[0])}")+ notable_items.extend(_json_scalar_examples(value))+ if trailing_chars:+ notable_items.append(f"trailing_chars_after_first_json_value: {trailing_chars}")+ return ToolResultSynopsis(+ kind="json",+ title="JSON output",+ summary=summary,+ structure=structure,+ notable_items=notable_items,+ sample="",+ )+++def _summarize_yaml(value: Any) -> ToolResultSynopsis:+ summary = [f"YAML {_type_name(value)}."]+ structure: List[str] = []+ if isinstance(value, dict):+ keys = [str(key) for key in value.keys()]+ summary.append(f"top-level keys: {', '.join(keys[:_YAML_KEY_LIMIT])}")+ for key, child in list(value.items())[:_YAML_KEY_LIMIT]:+ structure.append(f"{key}: {_type_name(child)}")+ elif isinstance(value, list):+ summary.append(f"array length: {len(value)}")+ return ToolResultSynopsis(+ kind="yaml",+ title="YAML output",+ summary=summary,+ structure=structure,+ notable_items=[],+ sample="",+ )+++def _summarize_xml(root: ET.Element) -> ToolResultSynopsis:+ child_counts = Counter(child.tag for child in list(root))+ structure = [f"root: {root.tag}", f"attributes: {len(root.attrib)}"]+ structure.extend(f"{tag}: {count}" for tag, count in child_counts.most_common(_XML_CHILD_TAG_LIMIT))+ return ToolResultSynopsis(+ kind="xml",+ title="XML output",+ summary=[f"XML document with root: {root.tag}."],+ structure=structure,+ notable_items=[],+ sample="",+ )+++def _try_table(+ content: str,+ delimiter: str,+ kind: ToolResultKind,+ preview_chars: int,+) -> Optional[ToolResultSynopsis]:+ try:+ rows = list(csv.reader(io.StringIO(content), delimiter=delimiter))+ except csv.Error:+ return None+ rows = [row for row in rows if any(cell.strip() for cell in row)]+ if len(rows) < 2 or len(rows[0]) < 2:+ return None+ column_count = len(rows[0])+ if any(len(row) != column_count for row in rows[1 : min(len(rows), 10)]):+ return None+ columns = [cell.strip() or f"column_{idx + 1}" for idx, cell in enumerate(rows[0])]+ data_rows = len(rows) - 1+ first_data = (+ _normalize_text_for_line(delimiter.join(rows[1]), _TABLE_FIRST_ROW_SAMPLE_CHARS)+ if len(rows) > 1+ else "(no data rows)"+ )+ return ToolResultSynopsis(+ kind=kind,+ title=f"{kind.upper()} table output",+ summary=[f"{kind.upper()} table with rows: {data_rows} and columns: {column_count}."],+ structure=[+ f"columns: {', '.join(columns)}",+ f"rows: {data_rows}",+ f"first row sample: {first_data}",+ ],+ notable_items=[],+ sample="",+ )+++def _looks_yaml(content: str) -> bool:+ stripped = content.lstrip()+ if stripped.startswith("---"):+ return True+ if any(pattern.search(content) for pattern in _CODE_PATTERNS):+ return False+ lines = [line for line in content.splitlines() if line.strip()]+ for idx, line in enumerate(lines[:-1]):+ if re.match(r"^[A-Za-z0-9_.-]+:\s*(?:#.*)?$", line):+ return lines[idx + 1].startswith((" ", "\t"))+ return False+++def _try_yaml(content: str) -> Optional[Any]:+ if not _looks_yaml(content):+ return None+ try:+ value = yaml.safe_load(content)+ except Exception:+ return None+ if isinstance(value, (dict, list)):+ return value+ return None+++def _summarize_code(content: str, preview_chars: int) -> ToolResultSynopsis:+ lines = content.splitlines()+ imports = []+ symbols = []+ for line in lines:+ stripped = line.strip()+ import_match = re.match(r"^(?:from\s+(\S+)\s+import|import\s+(\S+))", stripped)+ if import_match:+ imports.append(+ _normalize_text_for_line(+ import_match.group(1) or import_match.group(2) or "",+ _CODE_IMPORT_LINE_CHARS,+ )+ )+ continue+ symbol_match = re.match(+ r"^(class|def|async\s+def|function|export\s+function|pub\s+fn|fn)\s+([A-Za-z_]\w*)",+ stripped,+ )+ if symbol_match:+ symbols.append(+ _normalize_text_for_line(+ f"{symbol_match.group(1)} {symbol_match.group(2)}",+ _CODE_SYMBOL_LINE_CHARS,+ )+ )+ structure = [f"line_count: {len(lines)}"]+ if imports:+ structure.append(f"imports: {', '.join(imports[:_CODE_IMPORT_LIMIT])}")+ return ToolResultSynopsis(+ kind="code",+ title="Code output",+ summary=[f"Code-like output with {len(lines)} lines."],+ structure=structure,+ notable_items=symbols[:_CODE_SYMBOL_LIMIT],+ sample="",+ )+++def _summarize_text(content: str, preview_chars: int) -> ToolResultSynopsis:+ lines = content.splitlines()+ normalized = re.sub(r"\s+", " ", content).strip()+ headers = []+ seen_headers = set()+ for line in lines:+ stripped = line.strip()+ if len(stripped) <= 1:+ continue+ if not (+ re.match(r"^#{1,6}\s+", stripped)+ or re.match(r"^[A-Z0-9][A-Z0-9\s:_-]{6,}$", stripped)+ ):+ continue+ header = _normalize_text_for_line(stripped, 160)+ if header in seen_headers:+ continue+ seen_headers.add(header)+ headers.append(header)+ if len(headers) >= _TEXT_HEADER_LIMIT:+ break++ word_count = len(normalized.split()) if normalized else 0+ excerpt_chars = _TEXT_EXCERPT_CHARS+ first = _normalize_text_for_line(content[:excerpt_chars], excerpt_chars)+ last = _normalize_text_for_line(content[-excerpt_chars:], excerpt_chars) if excerpt_chars else ""+ return ToolResultSynopsis(+ kind="text",+ title="Text output",+ summary=[+ "Text exploration summary:",+ f"Characters: {len(content)}.",+ f"Words: {word_count}.",+ f"Lines: {len(lines)}.",+ f"Detected section headers: {' | '.join(headers) if headers else 'none detected'}.",+ f"Opening excerpt: {first or '(empty)'}.",+ f"Closing excerpt: {last or '(empty)'}.",+ ],+ structure=[],+ notable_items=[],+ sample="",+ )+++def generate_tool_result_synopsis(+ content: str,+ *,+ preview_chars: int,+ tool_name: str = "",+ mime_type: str = "text/plain",+) -> ToolResultSynopsis:+ if not content:+ return ToolResultSynopsis(+ kind="unknown",+ title="Empty output",+ summary=["Output is empty."],+ structure=[],+ notable_items=[],+ sample="",+ )+ if _looks_binary(content):+ return ToolResultSynopsis(+ kind="unknown",+ title="Binary-like output",+ summary=[f"Contains {len(content)} characters and non-text control bytes."],+ structure=[],+ notable_items=[],+ sample=_head_tail_sample(content, preview_chars),+ )++ stripped = content.strip()+ lower_mime = (mime_type or "").lower()+ if "json" in lower_mime or stripped.startswith(("{", "[")):+ try:+ decoder = json.JSONDecoder()+ value, end = decoder.raw_decode(stripped)+ trailing_chars = len(stripped[end:].strip())+ return _summarize_json(value, trailing_chars=trailing_chars)+ except Exception:+ if "json" in lower_mime:+ return ToolResultSynopsis(+ kind="unknown",+ title="Unparsed JSON-like output",+ summary=["JSON-like output failed to parse."],+ structure=[],+ notable_items=[],+ sample=_head_tail_sample(content, preview_chars),+ )+ if "xml" in lower_mime or stripped.startswith("<"):+ try:+ return _summarize_xml(ET.fromstring(content))+ except Exception:+ if "xml" in lower_mime:+ return ToolResultSynopsis(+ kind="unknown",+ title="Unparsed XML-like output",+ summary=["XML-like output failed to parse."],+ structure=[],+ notable_items=[],+ sample=_head_tail_sample(content, preview_chars),+ )+ if "\t" in content:+ table = _try_table(content, "\t", "tsv", preview_chars)+ if table:+ return table+ if "," in content:+ table = _try_table(content, ",", "csv", preview_chars)+ if table:+ return table+ yaml_value = _try_yaml(content)+ if yaml_value is not None:+ return _summarize_yaml(yaml_value)+ if any(pattern.search(content) for pattern in _CODE_PATTERNS):+ return _summarize_code(content, preview_chars)+ return _summarize_text(content, preview_chars)+++def render_tool_result_stub(+ synopsis: ToolResultSynopsis,+ *,+ ref: str = "",+ tool_name: str = "",+ sha256: str = "",+ reason: str = "",+ original_chars: int,+ preview_chars: int,+) -> str:+ header = [+ "[OpenViking tool result externalized]",+ f"tool_name: {tool_name or 'tool'}",+ f"kind: {synopsis.kind}",+ f"original_chars: {original_chars}",+ f"preview_chars: {max(preview_chars, 0)}",+ ]+ if ref:+ header.append(f"ref: {ref}")+ if sha256:+ header.append(f"sha256: {sha256[:16]}")+ if reason:+ header.append(f"reason: {reason}")++ body = ["Synopsis:"]+ body.extend(f"- {line}" for line in synopsis.summary)+ if synopsis.structure:+ body.append("")+ body.append("Structure:")+ body.extend(f"- {line}" for line in synopsis.structure)+ if synopsis.notable_items:+ body.append("")+ body.append("Notable items:")+ body.extend(f"- {line}" for line in synopsis.notable_items)+ if synopsis.sample:+ body.append("")+ body.append("Sample:")+ body.append(synopsis.sample)+ if ref:+ body.append("")+ body.append("Explore:")+ body.append(+ f"- Use openviking_tool_result_search with ref={ref} to find relevant raw snippets."+ )+ body.append(+ f"- Use openviking_tool_result_read with ref={ref} and offset/limit to inspect raw output."+ )+ body.append("- Use openviking_tool_result_list to discover other externalized outputs in this session.")+ return "\n".join(header) + "\n\n" + "\n".join(body)diff --git a/tests/server/test_api_sessions.py b/tests/server/test_api_sessions.pyindex 6cb35f4fb..1d662d494 100644--- a/tests/server/test_api_sessions.py+++ b/tests/server/test_api_sessions.py@@ -214,6 +214,8 @@ async def test_tool_result_externalization_read_and_search(client: httpx.AsyncClassert part["tool_output_truncated"] is Trueassert part["tool_output_ref"].startswith(f"viking://session/{session_id}/tool-results/")assert raw not in part["tool_output"]+ assert "kind: text" in part["tool_output"]+ assert "openviking_tool_result_search" in part["tool_output"]tool_result_id = part["tool_output_ref"].rsplit("/", 1)[-1]read_resp = await client.get(@@ -223,6 +225,16 @@ async def test_tool_result_externalization_read_and_search(client: httpx.AsyncClread_body = read_resp.json()["result"]assert read_body["content"] == rawassert read_body["offset_unit"] == "unicode_code_point"+ assert read_body["metadata"]["synopsis_kind"] == "text"+ assert read_body["metadata"]["synopsis"]["kind"] == "text"++ list_resp = await client.get(f"/api/v1/sessions/{session_id}/tool-results")+ assert list_resp.status_code == 200+ listed = list_resp.json()["result"]["tool_results"]+ assert len(listed) == 1+ assert listed[0]["tool_result_id"] == tool_result_id+ assert listed[0]["synopsis_kind"] == "text"+ assert listed[0]["synopsis"]["kind"] == "text"search_resp = await client.get(f"/api/v1/sessions/{session_id}/tool-results/{tool_result_id}/search",diff --git a/tests/session/test_tool_result_externalization.py b/tests/session/test_tool_result_externalization.pyindex b99208643..d9c51c74f 100644--- a/tests/session/test_tool_result_externalization.py+++ b/tests/session/test_tool_result_externalization.py@@ -3,10 +3,66 @@import json-from openviking.message import ToolPart+import pytest++import openviking.session.session as session_module+import openviking.session.tool_result_store as tool_result_store+from openviking.message import Message, ToolPartfrom openviking.server.config import ToolOutputExternalizationConfigfrom openviking.session import Sessionfrom openviking.session.tool_result_store import ToolResultStore+from openviking_cli.exceptions import NotFoundError+++class MemoryVikingFS:+ def __init__(self):+ self.files = {}++ async def write_file(self, uri, content, *, ctx=None): # noqa: ANN001+ self.files[uri] = content++ async def append_file(self, uri, content, *, ctx=None): # noqa: ANN001+ self.files[uri] = self.files.get(uri, "") + content++ async def read_file(self, uri, *, ctx=None): # noqa: ANN001+ if uri not in self.files:+ raise NotFoundError(uri, "file")+ return self.files[uri]++ async def ls(self, uri, *, output="original", node_limit=1000, ctx=None): # noqa: ANN001+ prefix = uri.rstrip("/") + "/"+ names = set()+ for file_uri in self.files:+ if not file_uri.startswith(prefix):+ continue+ rest = file_uri[len(prefix) :]+ first = rest.split("/", 1)[0]+ if first:+ names.add(first)+ return [{"name": name, "isDir": True} for name in sorted(names)][:node_limit]++[email protected](autouse=True)+def _drain_background_tasks():+ yield+++def session():+ return Session(MemoryVikingFS(), session_id="test_session_tool_results")+++def session_with_tool_call(session):+ tool_id = "test_tool_001"+ tool_part = ToolPart(+ tool_id=tool_id,+ tool_name="test_tool",+ tool_input={"param": "value"},+ tool_status="running",+ )+ msg = session.add_message("assistant", [tool_part])+ return session, msg.id, tool_iddef _small_config(**overrides):@@ -21,9 +77,14 @@ def _small_config(**overrides):return ToolOutputExternalizationConfig(**values)+def _json_items_payload(count: int) -> str:+ items = ",".join(f'{{"id":{idx},"name":"item{idx}"}}' for idx in range(count))+ return f'{{"items":[{items}]}}'++async def test_add_message_externalizes_large_tool_output(session: Session):session._tool_output_externalization_config = _small_config()- raw = "alpha-" * 20+ raw = "alpha-" * 300msg = session.add_message("user",@@ -52,7 +113,7 @@ async def test_add_message_externalizes_large_tool_output(session: Session):async def test_hydrate_tool_outputs_for_extraction_uses_memory_copy(session: Session):session._tool_output_externalization_config = _small_config()- raw = "alpha-" * 20+ raw = "alpha-" * 300msg = session.add_message("user",@@ -77,12 +138,170 @@ async def test_hydrate_tool_outputs_for_extraction_uses_memory_copy(session: Sesassert raw not in compressed_part.tool_output+async def test_externalized_tool_output_uses_typed_synopsis_stub(session: Session):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=20,+ preview_chars=200,+ )+ raw = '{"users":[{"id":1,"name":"Ada"},{"id":2,"name":"Lin"}],"meta":{"count":2}}' * 3++ msg = session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_json_synopsis",+ tool_name="fetch_json",+ tool_output=raw,+ tool_status="completed",+ )+ ],+ )++ part = msg.get_tool_parts()[0]+ assert part.tool_output_truncated is True+ assert "kind: json" in part.tool_output+ assert "Synopsis:" in part.tool_output+ assert "users" in part.tool_output+ assert "openviking_tool_result_search" in part.tool_output+ assert raw not in part.tool_output++ tool_result_id = part.tool_output_ref.rsplit("/", 1)[-1]+ stored = await session.read_tool_result(tool_result_id, limit=-1)+ assert stored["content"] == raw+ assert stored["metadata"]["synopsis"]["kind"] == "json"+ assert stored["metadata"]["synopsis_kind"] == "json"+++async def test_externalized_tool_output_generates_synopsis_once(session: Session, monkeypatch):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=20,+ preview_chars=200,+ )+ raw = '{"users":[{"id":1,"name":"Ada"},{"id":2,"name":"Lin"}],"meta":{"count":2}}' * 3++ calls = 0+ original = tool_result_store.generate_tool_result_synopsis++ def wrapped(*args, **kwargs): # noqa: ANN002, ANN003+ nonlocal calls+ calls += 1+ return original(*args, **kwargs)++ monkeypatch.setattr(tool_result_store, "generate_tool_result_synopsis", wrapped)+ monkeypatch.setattr(session_module, "generate_tool_result_synopsis", wrapped)++ session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_json_synopsis_once",+ tool_name="fetch_json",+ tool_output=raw,+ tool_status="completed",+ )+ ],+ )++ assert calls == 1+++async def test_preview_contract_fields_remain_present(session: Session):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=20,+ preview_chars=40,+ )+ raw = "alpha-" * 20++ part = session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_contract",+ tool_name="read_file",+ tool_output=raw,+ tool_status="completed",+ )+ ],+ ).get_tool_parts()[0]++ assert "[OpenViking tool result externalized]" in part.tool_output+ assert "tool_name: read_file" in part.tool_output+ assert "original_chars:" in part.tool_output+ assert "preview_chars:" in part.tool_output+ assert "ref:" in part.tool_output+ assert "sha256:" in part.tool_output+ assert "reason: single_threshold" in part.tool_output+++async def test_externalized_tool_output_uses_mime_type_for_xml(session: Session):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=20,+ preview_chars=200,+ )+ raw = "\ufeff<root><item id='1'>A</item><item id='2'>B</item></root>"++ part = session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_xml",+ tool_name="fetch_xml",+ tool_output=raw,+ tool_output_mime_type="application/xml",+ tool_status="completed",+ )+ ],+ ).get_tool_parts()[0]++ assert "kind: xml" in part.tool_output+ assert "root: root" in part.tool_output+++async def test_tool_output_at_or_below_threshold_remains_inline_when_budget_allows(+ session: Session,+):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=20,+ assistant_turn_inline_budget_chars=1000,+ )++ below = session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_below_threshold",+ tool_name="echo",+ tool_output="x" * 19,+ tool_status="completed",+ )+ ],+ ).get_tool_parts()[0]+ equal = session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_equal_threshold",+ tool_name="echo",+ tool_output="y" * 20,+ tool_status="completed",+ )+ ],+ ).get_tool_parts()[0]++ assert below.tool_output_truncated is False+ assert below.tool_output_ref == ""+ assert below.tool_output == "x" * 19+ assert equal.tool_output_truncated is False+ assert equal.tool_output_ref == ""+ assert equal.tool_output == "y" * 20++async def test_assistant_turn_budget_splits_aggregate_and_externalizes_largest_output(session: Session,):session._tool_output_externalization_config = _small_config(- threshold_chars=100,- assistant_turn_inline_budget_chars=25,+ threshold_chars=1200,+ assistant_turn_inline_budget_chars=800,)start_count = len(session.messages)@@ -92,13 +311,13 @@ async def test_assistant_turn_budget_splits_aggregate_and_externalizes_largest_oToolPart(tool_id="call_a",tool_name="tool_a",- tool_output="a" * 18,+ tool_output=_json_items_payload(40),tool_status="completed",),ToolPart(tool_id="call_b",tool_name="tool_b",- tool_output="b" * 12,+ tool_output=_json_items_payload(1),tool_status="completed",),],@@ -114,10 +333,152 @@ async def test_assistant_turn_budget_splits_aggregate_and_externalizes_largest_oassert len(externalized) == 1assert externalized[0].tool_id == "call_a"assert externalized[0].tool_output_externalized_reason == "turn_budget"- assert all(p.tool_output_group_original_chars == 30 for p in parts)+ assert all(+ p.tool_output_group_original_chars+ == len(_json_items_payload(40)) + len(_json_items_payload(1))+ for p in parts+ )assert parts[0].tool_output_group_id == parts[1].tool_output_group_id+async def test_assistant_turn_budget_externalizes_multiple_largest_outputs(+ session: Session,+):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=1200,+ assistant_turn_inline_budget_chars=1600,+ assistant_turn_preview_budget_chars=120,+ preview_chars=12,+ )++ start_count = len(session.messages)+ session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_a",+ tool_name="tool_a",+ tool_output=_json_items_payload(40),+ tool_status="completed",+ ),+ ToolPart(+ tool_id="call_b",+ tool_name="tool_b",+ tool_output=_json_items_payload(40),+ tool_status="completed",+ ),+ ToolPart(+ tool_id="call_c",+ tool_name="tool_c",+ tool_output=_json_items_payload(1),+ tool_status="completed",+ ),+ ],+ )++ parts = [msg.get_tool_parts()[0] for msg in session.messages[start_count:]]+ by_id = {part.tool_id: part for part in parts}++ assert by_id["call_a"].tool_output_truncated is True+ assert by_id["call_b"].tool_output_truncated is True+ assert by_id["call_c"].tool_output_truncated is False+ assert by_id["call_a"].tool_output_externalized_reason == "turn_budget"+ assert by_id["call_b"].tool_output_externalized_reason == "turn_budget"+++async def test_assistant_turn_budget_skips_externalization_when_stub_is_larger(session: Session):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=100,+ assistant_turn_inline_budget_chars=25,+ assistant_turn_preview_budget_chars=20,+ preview_chars=12,+ )++ start_count = len(session.messages)+ session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_a",+ tool_name="tool_a",+ tool_output="a" * 18,+ tool_status="completed",+ ),+ ToolPart(+ tool_id="call_b",+ tool_name="tool_b",+ tool_output="b" * 12,+ tool_status="completed",+ ),+ ],+ )++ parts = [msg.get_tool_parts()[0] for msg in session.messages[start_count:]]+ assert [part.tool_output_truncated for part in parts] == [False, False]+ assert sum(len(part.tool_output or "") for part in parts) == 30+++async def test_assistant_turn_budget_uses_rendered_stub_length(session: Session):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=1200,+ assistant_turn_inline_budget_chars=1600,+ assistant_turn_preview_budget_chars=120,+ preview_chars=12,+ )++ start_count = len(session.messages)+ session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_a",+ tool_name="tool_a",+ tool_output=_json_items_payload(40),+ tool_status="completed",+ ),+ ToolPart(+ tool_id="call_b",+ tool_name="tool_b",+ tool_output=_json_items_payload(40),+ tool_status="completed",+ ),+ ToolPart(+ tool_id="call_c",+ tool_name="tool_c",+ tool_output=_json_items_payload(1),+ tool_status="completed",+ ),+ ],+ )++ parts = [msg.get_tool_parts()[0] for msg in session.messages[start_count:]]+ by_id = {part.tool_id: part for part in parts}+ raw_a = _json_items_payload(40)+ raw_b = _json_items_payload(40)+ raw_c = _json_items_payload(1)+ ref_a = (+ f"viking://session/{session.session_id}/tool-results/"+ f"{tool_result_store.build_tool_result_id('call_a', tool_result_store.sha256_text(raw_a))}"+ )+ stub_a = tool_result_store.make_preview(+ raw_a,+ preview_chars=12,+ ref=ref_a,+ tool_name="tool_a",+ sha256=tool_result_store.sha256_text(raw_a),+ reason="turn_budget",+ original_chars=len(raw_a),+ mime_type="text/plain",+ )+ single_externalized_total = len(stub_a) + len(raw_b) + len(raw_c)++ assert by_id["call_a"].tool_output_truncated is True+ assert by_id["call_b"].tool_output_truncated is True+ assert by_id["call_c"].tool_output_truncated is False+ assert single_externalized_total > 1600+ assert sum(len(part.tool_output or "") for part in parts) < single_externalized_total++async def test_tool_result_aggregate_splits_when_externalization_disabled(session: Session):session._tool_output_externalization_config = _small_config(enabled=False)start_count = len(session.messages)@@ -218,11 +579,48 @@ async def test_read_back_tool_result_preview_honors_min_preview_chars(session: S)read_part = read_msg.get_tool_parts()[0]+ assert "kind:" in read_part.tool_outputassert "preview_chars: 12" in read_part.tool_output+ assert "openviking_tool_result_read" in read_part.tool_outputassert "abcdef" in read_part.tool_outputassert "efghij" in read_part.tool_output+async def test_tool_result_pairing_survives_synopsis_stubbing(session: Session):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=20,+ assistant_turn_inline_budget_chars=25,+ preview_chars=100,+ )++ returned = session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_big",+ tool_name="fetch_json",+ tool_output='{"items":[{"id":1},{"id":2}]}' * 5,+ tool_status="completed",+ ),+ ToolPart(+ tool_id="call_small",+ tool_name="echo",+ tool_output="ok",+ tool_status="completed",+ ),+ ],+ )++ assert returned.get_tool_parts()[0].tool_id in {"call_big", "call_small"}+ all_parts = [msg.get_tool_parts()[0] for msg in session.messages[-2:]]+ assert {part.tool_id for part in all_parts} == {"call_big", "call_small"}+ big = next(part for part in all_parts if part.tool_id == "call_big")+ small = next(part for part in all_parts if part.tool_id == "call_small")+ assert big.tool_output_truncated is True+ assert "kind: json" in big.tool_output+ assert small.tool_output == "ok"++async def test_update_tool_part_externalizes_large_output(session_with_tool_call):session, message_id, tool_id = session_with_tool_callsession._tool_output_externalization_config = _small_config()@@ -268,3 +666,34 @@ async def test_list_tool_results_filters_tool_name_before_limit():result = await store.list(tool_name="target", limit=1)assert result["tool_results"] == [{"tool_result_id": "tr_target", "tool_name": "target"}]+++async def test_tool_result_search_uses_raw_payload_not_stub_helper_text(session: Session):+ session._tool_output_externalization_config = _small_config(+ threshold_chars=20,+ preview_chars=60,+ )+ raw = "needle\n" + ("ordinary payload\n" * 40)++ part = session.add_message(+ "user",+ [+ ToolPart(+ tool_id="call_search_raw",+ tool_name="read_log",+ tool_output=raw,+ tool_status="completed",+ )+ ],+ ).get_tool_parts()[0]+ tool_result_id = part.tool_output_ref.rsplit("/", 1)[-1]++ assert "openviking_tool_result_read" in part.tool_output+ raw_match = await session.search_tool_result(tool_result_id, query="needle")+ helper_match = await session.search_tool_result(+ tool_result_id,+ query="openviking_tool_result_read",+ )++ assert len(raw_match["matches"]) == 1+ assert helper_match["matches"] == []diff --git a/tests/session/test_tool_result_synopsis.py b/tests/session/test_tool_result_synopsis.pynew file mode 100644index 000000000..3dcf24f4c--- /dev/null+++ b/tests/session/test_tool_result_synopsis.py@@ -0,0 +1,183 @@+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.+# SPDX-License-Identifier: AGPL-3.0++import pytest++from openviking.session.tool_result_synopsis import (+ ToolResultSynopsis,+ generate_tool_result_synopsis,+ render_tool_result_stub,+)++[email protected](autouse=True)+def _drain_background_tasks():+ yield+++def test_render_stub_preserves_navigation_contract():+ synopsis = ToolResultSynopsis(+ kind="text",+ title="Text output",+ summary=["Contains 120 characters across 3 lines."],+ structure=["line_count: 3"],+ notable_items=["first line: alpha"],+ sample="alpha\nbeta\ngamma",+ )++ rendered = render_tool_result_stub(+ synopsis,+ ref="viking://session/s1/tool-results/tr_123",+ tool_name="read_file",+ sha256="0123456789abcdef0123",+ reason="single_threshold",+ original_chars=120,+ preview_chars=80,+ )++ assert "[OpenViking tool result externalized]" in rendered+ assert "kind: text" in rendered+ assert "tool_name: read_file" in rendered+ assert "original_chars: 120" in rendered+ assert "preview_chars: 80" in rendered+ assert "ref: viking://session/s1/tool-results/tr_123" in rendered+ assert "openviking_tool_result_search" in rendered+ assert "openviking_tool_result_read" in rendered+++def test_json_synopsis_reports_shape():+ content = '{"users":[{"id":1,"name":"Ada"},{"id":2,"name":"Lin"}],"meta":{"count":2}}'+ synopsis = generate_tool_result_synopsis(content, preview_chars=200)++ assert synopsis.kind == "json"+ joined = "\n".join(synopsis.summary + synopsis.structure + synopsis.notable_items)+ assert "top-level keys" in joined+ assert "users" in joined+ assert "array length: 2" in joined+++def test_csv_synopsis_reports_columns_and_rows():+ content = "name,score\nAda,99\nLin,98\n"+ synopsis = generate_tool_result_synopsis(content, preview_chars=200)++ assert synopsis.kind == "csv"+ joined = "\n".join(synopsis.summary + synopsis.structure)+ assert "columns: name, score" in joined+ assert "rows: 2" in joined+++def test_tsv_synopsis_reports_columns_and_rows():+ content = "name\tscore\nAda\t99\nLin\t98\n"+ synopsis = generate_tool_result_synopsis(content, preview_chars=200)++ assert synopsis.kind == "tsv"+ joined = "\n".join(synopsis.summary + synopsis.structure)+ assert "columns: name, score" in joined+ assert "rows: 2" in joined+++def test_yaml_synopsis_reports_shape():+ content = "services:\n api:\n image: ov:latest\n replicas: 2\n"+ synopsis = generate_tool_result_synopsis(content, preview_chars=200)++ assert synopsis.kind == "yaml"+ joined = "\n".join(synopsis.summary + synopsis.structure)+ assert "services" in joined+ assert "top-level keys" in joined+++def test_xml_synopsis_reports_root_and_children():+ content = "<root><item id='1'>A</item><item id='2'>B</item></root>"+ synopsis = generate_tool_result_synopsis(content, preview_chars=200)++ assert synopsis.kind == "xml"+ joined = "\n".join(synopsis.summary + synopsis.structure)+ assert "root: root" in joined+ assert "item: 2" in joined+++def test_code_synopsis_reports_symbols():+ content = "import os\n\nclass Runner:\n pass\n\ndef main():\n return os.getcwd()\n"+ synopsis = generate_tool_result_synopsis(content, preview_chars=200)++ assert synopsis.kind == "code"+ joined = "\n".join(synopsis.summary + synopsis.structure + synopsis.notable_items)+ assert "imports: os" in joined+ assert "class Runner" in joined+ assert "def main" in joined+++def test_log_like_output_uses_text_synopsis_for_lossless_parity():+ content = "INFO start\nWARN retry once\nERROR failed to open file\nINFO done\n"+ synopsis = generate_tool_result_synopsis(content, preview_chars=200)++ assert synopsis.kind == "text"+ joined = "\n".join(synopsis.summary + synopsis.notable_items)+ assert "Text exploration summary" in joined+ assert "Log-like output" not in joined+ assert "ERROR failed" in joined+++def test_text_synopsis_uses_deterministic_summary_shape():+ content = "# Heading\n\nSYSTEM STATUS\n\nThis is a paragraph.\n\nThis is another paragraph."+ synopsis = generate_tool_result_synopsis(content, preview_chars=200)++ assert synopsis.kind == "text"+ joined = "\n".join(+ synopsis.summary + synopsis.structure + synopsis.notable_items + [synopsis.sample]+ )+ assert "Text exploration summary" in joined+ assert "Characters:" in joined+ assert "Words:" in joined+ assert "Lines:" in joined+ assert "Detected section headers: # Heading | SYSTEM STATUS" in joined+ assert "Opening excerpt:" in joined+ assert "Closing excerpt:" in joined+++def test_text_synopsis_uses_fixed_500_char_excerpts_independent_of_preview_chars():+ opening = "A" * 600+ closing = "Z" * 600+ content = f"{opening}\n\nmiddle section\n\n{closing}"++ synopsis = generate_tool_result_synopsis(content, preview_chars=10)+ joined = "\n".join(synopsis.summary)++ assert f"Opening excerpt: {'A' * 500}" in joined+ assert f"Closing excerpt: {'Z' * 500}" in joined+++def test_json_synopsis_uses_fixed_shape_limits_independent_of_preview_chars():+ content = "{" + ",".join(f'"key{i}":{i}' for i in range(15)) + "}"++ synopsis = generate_tool_result_synopsis(content, preview_chars=1)+ joined = "\n".join(synopsis.summary + synopsis.structure + synopsis.notable_items)++ assert "key0" in joined+ assert "key9" in joined+ assert "key10" not in joined+ assert synopsis.sample == ""+++def test_code_synopsis_uses_fixed_symbol_limits_independent_of_preview_chars():+ imports = "\n".join(f"import module_{idx}" for idx in range(15))+ defs = "\n".join(f"def function_{idx}():\n pass" for idx in range(30))+ content = f"{imports}\n\n{defs}"++ synopsis = generate_tool_result_synopsis(content, preview_chars=1)+ joined = "\n".join(synopsis.summary + synopsis.structure + synopsis.notable_items)++ assert "module_0" in joined+ assert "module_11" in joined+ assert "module_12" not in joined+ assert "def function_23" in joined+ assert "def function_24" not in joined+ assert synopsis.sample == ""+++def test_binary_like_text_uses_unknown_fallback():+ content = "\x00\x01\x02" + ("x" * 80)+ synopsis = generate_tool_result_synopsis(content, preview_chars=30)++ assert synopsis.kind == "unknown"+ assert synopsis.sample