Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

add e2e test under tests/ut/e2e (#1154)

Eurakaxun27d48bd7b7e7
diff --git a/examples/openclaw-plugin/tests/e2e/test-archive-expand.py b/examples/openclaw-plugin/tests/e2e/test-archive-expand.py
index e05f21117..bba147b49 100644
--- a/examples/openclaw-plugin/tests/e2e/test-archive-expand.py
+++ b/examples/openclaw-plugin/tests/e2e/test-archive-expand.py
@@ -66,6 +66,16 @@ ov_archive_expand 归档展开端到端测试 — 用户: 小杰(后端开发
3. LLM 后端可达(Gateway 需要调用 LLM 生成回复和触发工具调用)
4. 有效的 Gateway auth token(通过 --token 传入或自动发现)
+ 关键 openclaw.json 配置:
+ - plugins.slots.contextEngine = "openviking"
+ - plugins.entries.openviking.enabled = true
+ - plugins.entries.openviking.config.autoCapture = true
+ - plugins.entries.openviking.config.commitTokenThreshold = 50
+ ↑ 此值控制 auto-commit 时机。32 轮对话需要多次 auto-commit 产生归档,
+ 若值过大可能导致归档数不足。建议 <= 5000。
+ - agents.defaults.alsoAllow = ["ov_archive_expand"]
+ ↑ 必须显式允许 ov_archive_expand 工具,否则 LLM 无法调用
+
服务部署参考:
- OpenViking: openviking-server(HTTP 默认 2934,AGFS 默认 2833)
- Gateway: openclaw gateway(HTTP 默认 19789)
@@ -132,7 +142,33 @@ ov_archive_expand 归档展开端到端测试 — 用户: 小杰(后端开发
在 Phase 4 前重启 Gateway 清除工作记忆,强制走归档展开路径。
================================================================================
-六、预期结果
+六、已知限制
+================================================================================
+
+ 1. LLM 是否调用 ov_archive_expand:
+ 不同模型对工具调用的倾向性不同。如果模型直接从 archive overview 摘要
+ 中推测答案而不展开归档,关键词可能命中(摘要恰好包含)也可能不命中。
+ 使用 --gateway-restart-cmd 可强制清除工作记忆,迫使走归档展开路径。
+
+ 2. 关键词精确匹配:
+ 数字格式差异可能导致匹配失败(如 "12000" vs "12,000" vs "1.2万")。
+ Q4 的 "12000" 在实际测试中因 LLM 输出 "12,000" 而未命中,但整体命中率
+ 仍达 67% 超过 50% 阈值。
+
+ 3. 测试耗时:
+ 完整测试需要 32 轮对话 + 验证 + 追问,约 10-15 分钟。如需快速验证,可
+ 使用 --phase expand 单独跑追问阶段(前提是已有归档数据)。
+
+ 4. 对话顺序依赖:
+ 4 批对话必须按顺序执行(Phase 1 → 2a → 2b → 2c),因为后续批次的归档
+ 编号依赖前序批次。不能单独跑 chat2 而跳过 chat1。
+
+ 5. 环境要求:
+ Gateway 必须配置 OpenViking 插件且启用 ov_archive_expand 工具定义,
+ 否则 LLM 无法调用归档展开。
+
+================================================================================
+七、预期结果
================================================================================
15/15 断言全部通过:
diff --git a/examples/openclaw-plugin/tests/e2e/test-memory-chain.py b/examples/openclaw-plugin/tests/e2e/test-memory-chain.py
new file mode 100644
index 000000000..f990a696c
--- /dev/null
+++ b/examples/openclaw-plugin/tests/e2e/test-memory-chain.py
@@ -0,0 +1,1176 @@
+#!/usr/bin/env python3
+"""
+OpenClaw 记忆链路完整端到端测试
+
+================================================================================
+一、用例设计思路
+================================================================================
+
+验证 OpenViking 记忆插件的完整链路,覆盖消息写入到记忆召回的每个环节:
+
+ afterTurn → commit → assemble → sessionId 一致性 → 新用户记忆召回
+
+各环节验证目标:
+ 1. afterTurn: 本轮消息无损写入 OV session,sessionId 一致
+ 2. commit: 归档消息 + 提取长期记忆(cards/events/profile 等)
+ 3. assemble: 同用户继续对话时,从 latest_archive_overview + active
+ messages 正确重组上下文
+ 4. budget trimming: 小 token budget 下 archive overview 被合理裁剪
+ 5. sessionId 一致性: 整条链路使用统一的 OV sessionId,无 sessionKey 残留
+ 6. 新用户记忆召回: 不同用户发问时,auto-recall 注入相关记忆
+
+对话数据设计:
+ 12 轮对话涵盖:个人背景、技术栈、项目细节、缓存方案讨论、团队信息、
+ 消息队列选型、监控体系、偏好设置等,确保记忆提取能覆盖多种类别。
+
+断言策略:
+ - 确定性检查 (hard): afterTurn 写入、commit_count、sessionId、overview 存在性
+ - LLM 依赖检查 (soft): Assemble/Recall 的关键词命中率 >= 50%
+
+================================================================================
+二、测试流程
+================================================================================
+
+ Phase 1: 多轮对话 (12 轮) — 写入对话数据
+ Phase 2: afterTurn 验证 — 检查 OV session 存在性和消息内容
+ Phase 3: Commit 验证 — 触发 commit, 检查归档结构和记忆提取
+ Phase 4: Assemble 验证 — 同用户继续对话, 验证上下文重组 + budget trimming
+ Phase 5: SessionId 一致性验证 — 无 sessionKey 残留
+ Phase 6: 新用户记忆召回 (3 问) — 验证 auto-recall
+
+================================================================================
+三、环境前提
+================================================================================
+
+ 1. OpenViking 服务已启动
+ 2. OpenClaw Gateway 已启动并配置了 OpenViking 插件
+ 3. LLM 后端可达(Gateway 需要调用 LLM 生成回复)
+ 4. 有效的 Gateway auth token
+
+ 关键 openclaw.json 配置(影响测试行为):
+ - plugins.slots.contextEngine = "openviking" # 使用 OV 作为 context engine
+ - plugins.slots.memory = "none" # 不使用内置 memory-core
+ - plugins.entries.openviking.enabled = true # 启用 OV 插件
+ - plugins.entries.openviking.config.autoCapture = true # afterTurn 自动捕获
+ - plugins.entries.openviking.config.autoRecall = true # 新用户自动召回记忆
+ - plugins.entries.openviking.config.commitTokenThreshold = 50
+ ↑ 此值控制 auto-commit 触发时机。设为 50(token)时几乎每轮触发;
+ 若值较大(如 10000),测试中 auto-commit 不会提前发生,
+ Phase 3 的 commit 验证行为会不同。脚本已兼容两种场景。
+
+================================================================================
+四、使用方法
+================================================================================
+
+ 安装依赖:
+ pip install requests rich
+
+ 完整测试:
+ python test-memory-chain.py --phase all \\
+ --gateway http://127.0.0.1:19789 \\
+ --openviking http://127.0.0.1:2934 \\
+ --token <your_gateway_token>
+
+ 分阶段执行:
+ python test-memory-chain.py --phase chat # 仅多轮对话
+ python test-memory-chain.py --phase afterTurn # 仅 afterTurn 验证
+ python test-memory-chain.py --phase commit # 仅 commit 验证
+ python test-memory-chain.py --phase assemble # 仅 assemble 验证
+ python test-memory-chain.py --phase session-id # 仅 sessionId 检查
+ python test-memory-chain.py --phase recall # 仅记忆召回
+
+ 其他选项:
+ --user-id <id> 固定用户 ID(默认随机生成)
+ --delay <seconds> 轮次间等待秒数(默认 3s)
+ --verbose / -v 详细输出
+
+ 注意:
+ - 完整测试约需 8-12 分钟
+ - 首次运行前建议清理 OV 数据和 session 数据
+
+================================================================================
+五、已知限制
+================================================================================
+
+ 1. LLM 回复非确定性:
+ Assemble 和 Recall 阶段的关键词命中依赖 LLM 回复内容。不同模型、不同
+ temperature 设置下可能产生不同结果。关键词命中率阈值已设为 50% 以容忍
+ 表述差异,但仍可能偶发失败。
+
+ 2. auto-commit 时序:
+ Gateway 的 afterTurn 可能触发 auto-commit,导致手动 commit 时无新内容。
+ 脚本已处理此场景(auto_committed=True 时条件性通过)。
+
+ 3. 记忆提取质量:
+ 不同 LLM 对同一对话提取的记忆类别和内容可能不同,影响 Recall 阶段的
+ 关键词匹配。建议使用支持中文的高质量模型。
+
+ 4. sessionId 映射:
+ Gateway 内部使用 UUID 作为 OV sessionId,不等于传入的 user_id。脚本
+ 通过 OV sessions 列表接口自动发现实际 sessionId。
+
+================================================================================
+六、预期结果
+================================================================================
+
+ 29/29 断言全部通过:
+ - Phase 1: 12 轮对话全部成功
+ - Phase 2~3: afterTurn 写入正确, commit 归档正常
+ - Phase 4: Assemble Q1/Q2 关键词命中率 >= 50%
+ - Phase 5: sessionId 一致,无残留
+ - Phase 6: Recall Q1/Q2/Q3 关键词命中率 >= 50%
+"""
+
+import argparse
+import io
+import json
+import os
+import sys
+import time
+import uuid
+from datetime import datetime
+from typing import Any
+
+import requests
+from rich.console import Console
+from rich.markdown import Markdown
+from rich.panel import Panel
+from rich.table import Table
+from rich.tree import Tree
+
+if sys.platform == "win32":
+ sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="replace")
+ sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8", errors="replace")
+
+# ── 常量 ───────────────────────────────────────────────────────────────────
+
+USER_ID = f"test-chain-{uuid.uuid4().hex[:8]}"
+DISPLAY_NAME = "测试用户"
+DEFAULT_GATEWAY = "http://127.0.0.1:19789"
+DEFAULT_OPENVIKING = "http://127.0.0.1:2934"
+AGENT_ID = "main"
+
+console = Console(force_terminal=True)
+
+# ── 测试结果收集 ──────────────────────────────────────────────────────────
+
+assertions: list[dict] = []
+
+
+def check(label: str, condition: bool, detail: str = ""):
+ """记录一个断言结果。"""
+ assertions.append({"label": label, "ok": condition, "detail": detail})
+ icon = "[green]PASS[/green]" if condition else "[red]FAIL[/red]"
+ msg = f" {icon} {label}"
+ if detail:
+ msg += f" [dim]({detail})[/dim]"
+ console.print(msg)
+
+
+# ── 对话数据 ──────────────────────────────────────────────────────────────
+
+CHAT_MESSAGES = [
+ "你好,我是一个软件工程师,我叫张明,在一家科技公司工作。我主要负责后端服务开发,使用的技术栈是 Python 和 Go。最近我们在重构一个订单系统,遇到了不少挑战。",
+ "关于订单系统的问题,主要是性能瓶颈。我们发现在高峰期,数据库连接池经常被耗尽。目前用的是 PostgreSQL,连接池大小设置的是100,但每秒峰值请求量有5000。你有什么建议吗?",
+ "谢谢你的建议。我还想问一下,我们目前的缓存策略用的是 Redis,但缓存击穿的问题很严重。热点数据过期后,大量请求直接打到数据库。我们尝试过加互斥锁,但性能下降很多。",
+ "对了,关于代码风格,我们团队更倾向于使用函数式编程的思想,尽量避免副作用。变量命名用 snake_case,文档用中文写。代码审查很严格,每个 PR 至少需要两人 review。",
+ "说到工作流程,我们每天早上9点站会,周三下午技术分享会。我一般上午写代码,下午处理 code review 和会议。晚上如果不加班,会看看技术书籍或者写写博客。",
+ "我最近在学习分布式系统的设计,正在看《数据密集型应用系统设计》这本书。之前看完了《深入理解计算机系统》,收获很大。你有什么好的分布式系统学习资料推荐吗?",
+ "目前订单系统重构的进度大概完成了60%,还剩下支付模块和库存同步模块。支付模块比较复杂,需要对接多个支付渠道。我们打算用消息队列来解耦库存同步。",
+ "消息队列我们在 Kafka 和 RabbitMQ 之间犹豫。Kafka 吞吐量高,但运维复杂;RabbitMQ 功能丰富,但性能稍差。我们的消息量大概每天1000万条,你觉得选哪个好?",
+ "我们团队有8个人,3个后端、2个前端、1个测试、1个运维,还有1个产品经理。后端老王经验最丰富,遇到难题都找他。测试小李很细心,bug检出率很高。",
+ "对了,跟我聊天的时候注意几点:我喜欢简洁直接的回答,不要太啰嗦;技术问题最好带代码示例;如果不确定的问题要说明,不要瞎编。谢谢!",
+ "补充一下,我们的监控用的是 Prometheus + Grafana,日志用 ELK Stack。最近在考虑引入链路追踪,OpenTelemetry 看起来不错,但不知道跟现有系统集成麻不麻烦。",
+ "昨天线上出了个诡异的 bug,某个接口偶发超时,但日志里看不出什么问题。后来发现是下游服务的连接数满了,但监控指标没配好,没报警。这种问题怎么预防比较好?",
+]
+
+# assemble 阶段: 同用户继续对话,用于验证 assemble 是否携带了摘要上下文
+ASSEMBLE_FOLLOWUP_MESSAGES = [
+ {
+ "question": "对了,我之前提到的订单系统重构进展到哪了?支付模块开始了吗?",
+ "anchor_keywords": ["订单系统", "支付模块", "60%"],
+ "hook": "assemble — latest_archive_overview 重组",
+ },
+ {
+ "question": "我之前跟你说过选消息队列的事,Kafka 和 RabbitMQ 各有什么优缺点来着?",
+ "anchor_keywords": ["Kafka", "RabbitMQ"],
+ "hook": "assemble — latest_archive_overview 重组",
+ },
+]
+
+# 新用户记忆召回
+RECALL_QUESTIONS = [
+ {
+ "question": "我是做什么工作的?用什么技术栈?请简洁回答",
+ "expected_keywords": ["软件工程师", "Python", "Go"],
+ },
+ {
+ "question": "我最近在做什么项目?遇到了什么技术挑战?请简洁回答",
+ "expected_keywords": ["订单系统", "性能瓶颈", "缓存"],
+ },
+ {
+ "question": "我们团队有多少人?团队里谁经验最丰富?请简洁回答",
+ "expected_keywords": ["8", "老王"],
+ },
+]
+
+
+# ── Token 自动发现 ────────────────────────────────────────────────────────
+
+_gateway_token: str = ""
+
+
+def discover_gateway_token() -> str:
+ """从常见路径自动发现 gateway auth token。"""
+ import pathlib
+
+ candidates = [
+ pathlib.Path.cwd() / "config" / ".openclaw" / "openclaw.json",
+ pathlib.Path.home() / ".openclaw" / "openclaw.json",
+ ]
+ state_dir = os.environ.get("OPENCLAW_STATE_DIR")
+ if state_dir:
+ candidates.insert(0, pathlib.Path(state_dir) / "openclaw.json")
+
+ for p in candidates:
+ try:
+ cfg = json.loads(p.read_text(encoding="utf-8"))
+ token = cfg.get("gateway", {}).get("auth", {}).get("token", "")
+ if token:
+ return token
+ except Exception:
+ continue
+ return ""
+
+
+def set_gateway_token(token: str):
+ global _gateway_token
+ _gateway_token = token
+
+
+# ── Gateway / OpenViking API ─────────────────────────────────────────────
+
+
+def send_message(gateway_url: str, message: str, user_id: str) -> dict:
+ """通过 OpenClaw Responses API 发送消息。"""
+ headers = {"Content-Type": "application/json"}
+ if _gateway_token:
+ headers["Authorization"] = f"Bearer {_gateway_token}"
+ resp = requests.post(
+ f"{gateway_url}/v1/responses",
+ headers=headers,
+ json={"model": "openclaw", "input": message, "user": user_id},
+ timeout=300,
+ )
+ resp.raise_for_status()
+ return resp.json()
+
+
+def extract_reply_text(data: dict) -> str:
+ """从 Responses API 响应中提取助手回复文本。"""
+ for item in data.get("output", []):
+ if item.get("type") == "message" and item.get("role") == "assistant":
+ for part in item.get("content", []):
+ if part.get("type") in ("text", "output_text"):
+ return part.get("text", "")
+ return "(无回复)"
+
+
+class OpenVikingInspector:
+ """OpenViking 内部状态检查器。"""
+
+ def __init__(self, base_url: str, api_key: str = "", agent_id: str = AGENT_ID):
+ self.base_url = base_url.rstrip("/")
+ self.api_key = api_key
+ self.agent_id = agent_id
+
+ def _headers(self) -> dict:
+ h: dict[str, str] = {"Content-Type": "application/json"}
+ if self.api_key:
+ h["X-API-Key"] = self.api_key
+ if self.agent_id:
+ h["X-OpenViking-Agent"] = self.agent_id
+ return h
+
+ def _get(self, path: str, timeout: int = 10) -> dict | None:
+ try:
+ resp = requests.get(f"{self.base_url}{path}", headers=self._headers(), timeout=timeout)
+ if resp.status_code == 200:
+ data = resp.json()
+ return data.get("result", data)
+ return None
+ except Exception as e:
+ console.print(f"[dim]GET {path} 失败: {e}[/dim]")
+ return None
+
+ def _post(self, path: str, body: dict | None = None, timeout: int = 30) -> dict | None:
+ try:
+ resp = requests.post(
+ f"{self.base_url}{path}",
+ headers=self._headers(),
+ json=body or {},
+ timeout=timeout,
+ )
+ if resp.status_code == 200:
+ data = resp.json()
+ return data.get("result", data)
+ return None
+ except Exception as e:
+ console.print(f"[dim]POST {path} 失败: {e}[/dim]")
+ return None
+
+ def health_check(self) -> bool:
+ try:
+ resp = requests.get(f"{self.base_url}/health", timeout=5)
+ return resp.status_code == 200
+ except Exception:
+ return False
+
+ def get_session(self, session_id: str) -> dict | None:
+ return self._get(f"/api/v1/sessions/{session_id}")
+
+ def get_session_messages(self, session_id: str) -> list | None:
+ result = self._get(f"/api/v1/sessions/{session_id}/messages")
+ if isinstance(result, list):
+ return result
+ if isinstance(result, dict):
+ return result.get("messages", [])
+ return None
+
+ def get_session_context(self, session_id: str, token_budget: int = 128000) -> dict | None:
+ return self._get(f"/api/v1/sessions/{session_id}/context?token_budget={token_budget}")
+
+ def commit_session(self, session_id: str, wait: bool = True) -> dict | None:
+ result = self._post(f"/api/v1/sessions/{session_id}/commit", timeout=120)
+ if not result:
+ return None
+
+ if wait and result.get("task_id"):
+ task_id = result["task_id"]
+ deadline = time.time() + 120
+ while time.time() < deadline:
+ time.sleep(0.5)
+ task = self._get(f"/api/v1/tasks/{task_id}")
+ if not task:
+ continue
+ if task.get("status") == "completed":
+ result["status"] = "completed"
+ result["memories_extracted"] = task.get("result", {}).get(
+ "memories_extracted", {}
+ )
+ return result
+ if task.get("status") == "failed":
+ result["status"] = "failed"
+ result["error"] = task.get("error")
+ return result
+
+ return result
+
+ def search_memories(
+ self, query: str, target_uri: str = "viking://user/memories", limit: int = 10
+ ) -> list:
+ result = self._post(
+ "/api/v1/search/find",
+ {"query": query, "target_uri": target_uri, "limit": limit},
+ )
+ if isinstance(result, dict):
+ return result.get("memories", [])
+ return []
+
+ def list_sessions(self) -> list:
+ result = self._get("/api/v1/sessions")
+ if isinstance(result, list):
+ return result
+ return []
+
+ def find_session_for_user(self, user_hint: str) -> str | None:
+ """通过遍历 OV session 列表找到与当前测试关联的 session。
+ Gateway 可能使用内部 UUID 而非 user_id 作为 OV session_id,
+ 因此需要检查 session 内容或选择最近更新的 session。"""
+ sessions = self.list_sessions()
+ real_sessions = [
+ s for s in sessions
+ if isinstance(s, dict) and not s.get("session_id", "").startswith("memory-store-")
+ ]
+ if not real_sessions:
+ return None
+
+ best_id = None
+ best_time = ""
+ for s in real_sessions:
+ sid = s.get("session_id", "")
+ if not sid:
+ continue
+ if sid == user_hint:
+ return sid
+ detail = self.get_session(sid)
+ if not detail:
+ continue
+ updated = detail.get("updated_at", "")
+ if updated > best_time:
+ best_time = updated
+ best_id = sid
+
+ return best_id or (real_sessions[-1].get("session_id") if real_sessions else None)
+
+ def list_fs(self, uri: str) -> list:
+ result = self._get(f"/api/v1/fs/ls?uri={uri}&output=original")
+ return result if isinstance(result, list) else []
+
+ def read_fs(self, uri: str) -> str | None:
+ """读取 fs 中某个文件的内容。"""
+ result = self._get(f"/api/v1/content/read?uri={uri}")
+ if isinstance(result, str):
+ return result
+ if isinstance(result, dict):
+ return result.get("content")
+ return None
+
+
+# ── 渲染函数 ──────────────────────────────────────────────────────────────
+
+
+def render_reply(text: str, title: str = "回复"):
+ lines = text.split("\n")
+ if len(lines) > 25:
+ text = "\n".join(lines[:25]) + f"\n\n... (共 {len(lines)} 行,已截断)"
+ console.print(Panel(Markdown(text), title=f"[green]{title}[/green]", border_style="green"))
+
+
+def render_json(data: Any, title: str = "JSON"):
+ console.print(
+ Panel(json.dumps(data, indent=2, ensure_ascii=False, default=str)[:2000], title=title)
+ )
+
+
+def render_session_info(info: dict, title: str = "Session 信息"):
+ table = Table(title=title, show_header=True)
+ table.add_column("属性", style="cyan")
+ table.add_column("值", style="green")
+ for key, value in info.items():
+ if isinstance(value, (dict, list)):
+ value = json.dumps(value, ensure_ascii=False)
+ table.add_row(str(key), str(value)[:120])
+ console.print(table)
+
+
+# ── Phase 1: 多轮对话 ────────────────────────────────────────────────────
+
+
+def run_phase_chat(gateway_url: str, user_id: str, delay: float, verbose: bool) -> tuple[int, int]:
+ """Phase 1: 多轮对话 — 测试 afterTurn 写入。"""
+ console.print()
+ console.rule(f"[bold]Phase 1: 多轮对话 ({len(CHAT_MESSAGES)} 轮) — afterTurn 写入[/bold]")
+ console.print(f"[yellow]用户ID:[/yellow] {user_id}")
+ console.print(f"[yellow]Gateway:[/yellow] {gateway_url}")
+ console.print()
+
+ total = len(CHAT_MESSAGES)
+ ok = fail = 0
+
+ for i, msg in enumerate(CHAT_MESSAGES, 1):
+ console.rule(f"[dim]Turn {i}/{total}[/dim]", style="dim")
+ console.print(
+ Panel(
+ msg[:200] + ("..." if len(msg) > 200 else ""),
+ title=f"[bold cyan]用户 [{i}/{total}][/bold cyan]",
+ border_style="cyan",
+ )
+ )
+ try:
+ data = send_message(gateway_url, msg, user_id)
+ reply = extract_reply_text(data)
+ render_reply(reply[:500] + ("..." if len(reply) > 500 else ""))
+ ok += 1
+ except Exception as e:
+ console.print(f"[red][ERROR][/red] {e}")
+ fail += 1
+
+ if i < total:
+ time.sleep(delay)
+
+ console.print()
+ console.print(f"[yellow]对话完成:[/yellow] {ok} 成功, {fail} 失败")
+
+ wait = max(delay * 2, 5)
+ console.print(f"[yellow]等待 {wait:.0f}s 让 afterTurn 处理完成...[/yellow]")
+ time.sleep(wait)
+
+ return ok, fail
+
+
+# ── Phase 2: afterTurn 验证 ──────────────────────────────────────────────
+
+
+def run_phase_after_turn(openviking_url: str, user_id: str, verbose: bool) -> tuple[bool, str]:
+ """Phase 2: afterTurn 验证 — 检查 OV session 内部状态确认消息已写入。
+ 返回 (success, resolved_session_id)。"""
+ console.print()
+ console.rule("[bold]Phase 2: afterTurn 验证 — 检查 OV session 消息写入[/bold]")
+ console.print()
+ console.print("[dim]验证点:[/dim]")
+ console.print("[dim]- afterTurn 应将每轮消息写入 OV session[/dim]")
+ console.print("[dim]- session.message_count > 0[/dim]")
+ console.print("[dim]- pending_tokens > 0 (消息尚未 commit)[/dim]")
+ console.print()
+
+ inspector = OpenVikingInspector(openviking_url)
+
+ # 2.1 健康检查
+ console.print("[bold]2.1 OpenViking 健康检查[/bold]")
+ healthy = inspector.health_check()
+ check("OpenViking 服务可达", healthy)
+ if not healthy:
+ return False, user_id
+
+ # 2.2 Session 发现: Gateway 可能使用内部 UUID 而非 user_id
+ console.print("\n[bold]2.2 Session 发现 & 消息计数[/bold]")
+ resolved_id = inspector.find_session_for_user(user_id)
+
+ if resolved_id and resolved_id != user_id:
+ console.print(
+ f" [yellow]Gateway 使用内部 session_id: {resolved_id} (非 user_id)[/yellow]"
+ )
+
+ check("Session 存在", resolved_id is not None, f"resolved_id={resolved_id}")
+
+ if not resolved_id:
+ console.print("[red]OV 中没有找到任何相关 session,无法继续验证[/red]")
+ return False, user_id
+
+ session_info = inspector.get_session(resolved_id)
+ if not session_info:
+ console.print("[red]Session 详情获取失败[/red]")
+ return False, resolved_id
+
+ if verbose:
+ render_session_info(session_info, f"Session: {resolved_id}")
+
+ msg_count = session_info.get("message_count", 0)
+ commit_count = session_info.get("commit_count", 0)
+ # Gateway 可能在对话过程中 auto-commit,所以 message_count 可能为 0
+ # 但 commit_count > 0 表明 afterTurn 已经处理并提交了消息
+ has_activity = msg_count > 0 or commit_count > 0
+ check(
+ "afterTurn 已处理 (message_count > 0 或 commit_count > 0)",
+ has_activity,
+ f"message_count={msg_count}, commit_count={commit_count}",
+ )
+
+ pending = session_info.get("pending_tokens", 0)
+ if commit_count == 0:
+ check(
+ "pending_tokens > 0 (有待 commit 的内容)",
+ pending > 0,
+ f"pending_tokens={pending}",
+ )
+ else:
+ console.print(
+ f" [dim]auto-commit 已触发 (commit_count={commit_count}),"
+ f"pending_tokens={pending} 属正常[/dim]"
+ )
+
+ auto_committed = commit_count > 0
+
+ # 2.3 检查消息内容: 通过 context API 检查(兼容 auto-commit 场景)
+ console.print("\n[bold]2.3 消息内容抽样校验[/bold]")
+ ctx = inspector.get_session_context(resolved_id)
+ if ctx:
+ ctx_messages = ctx.get("messages", [])
+ overview = ctx.get("latest_archive_overview", "")
+ all_text = json.dumps(ctx_messages, ensure_ascii=False) + "\n" + overview
+
+ check(
+ "context 返回内容 (messages 或 overview)",
+ len(ctx_messages) > 0 or bool(overview),
+ f"messages={len(ctx_messages)}, overview_len={len(overview)}",
+ )
+
+ sample_text = "张明"
+ # 如果已 auto-commit, 信息可能在 overview 里而非 messages 中
+ check(
+ f"内容包含特征文本「{sample_text}」",
+ sample_text in all_text,
+ "验证 afterTurn 写入的内容与发送一致",
+ )
+
+ sample_text_2 = "PostgreSQL"
+ check(
+ f"内容包含特征文本「{sample_text_2}」",
+ sample_text_2 in all_text,
+ "验证多轮消息写入",
+ )
+
+ if verbose and ctx.get("stats"):
+ console.print(f" [dim]stats: {ctx['stats']}[/dim]")
+ else:
+ # 回退到 messages API
+ messages = inspector.get_session_messages(resolved_id)
+ if messages is not None:
+ check("能获取到 session 消息列表", True, f"共 {len(messages)} 条消息")
+ else:
+ check("能获取到 session 消息列表", False, "GET messages 返回 None")
+
+ return True, resolved_id, auto_committed
+
+
+# ── Phase 3: Commit 验证 ─────────────────────────────────────────────────
+
+
+def run_phase_commit(
+ openviking_url: str, session_id: str, verbose: bool, auto_committed: bool = False,
+) -> bool:
+ """Phase 3: Commit 验证 — 触发 commit, 检查归档结构和记忆提取。"""
+ console.print()
+ console.rule("[bold]Phase 3: Commit 验证 — 触发 session.commit()[/bold]")
+ console.print()
+ console.print("[dim]验证点:[/dim]")
+ console.print("[dim]- commit 返回 status=completed/accepted[/dim]")
+ console.print("[dim]- 消息被归档或已经归档 (auto-commit)[/dim]")
+ console.print("[dim]- 提取出记忆 (memories_extracted > 0)[/dim]")
+ console.print(f"[dim]- 使用 session_id: {session_id}[/dim]")
+ if auto_committed:
+ console.print("[yellow]注: Gateway 已触发 auto-commit, 手动 commit 可能无新内容[/yellow]")
+ console.print()
+
+ inspector = OpenVikingInspector(openviking_url)
+
+ # 3.1 执行 commit
+ console.print("[bold]3.1 执行 session.commit()[/bold]")
+ console.print("[dim]正在等待 commit 完成 (可能需要 1-2 分钟)...[/dim]")
+
+ commit_result = inspector.commit_session(session_id, wait=True)
+ if auto_committed and not commit_result:
+ check(
+ "commit 返回结果",
+ True,
+ "auto-commit 已处理, 手动 commit 无新内容属正常",
+ )
+ else:
+ check("commit 返回结果", commit_result is not None)
+
+ if not commit_result:
+ if auto_committed:
+ console.print("[yellow]auto-commit 已处理, 手动 commit 无新内容属正常[/yellow]")
+ else:
+ console.print("[red]Commit 失败,无法继续[/red]")
+ return False
+
+ if commit_result:
+ if verbose:
+ render_json(commit_result, "Commit 结果")
+
+ status = commit_result.get("status", "unknown")
+ check(
+ "commit status 为 completed 或 accepted",
+ status in ("completed", "accepted"),
+ f"status={status}",
+ )
+
+ archived = commit_result.get("archived", False)
+ if auto_committed and not archived:
+ check(
+ "归档状态合理 (auto-commit 已处理)",
+ True,
+ "auto-commit 已归档, 本次 commit 无新内容",
+ )
+ else:
+ check("archived=true (消息已归档)", archived is True, f"archived={archived}")
+
+ memories = commit_result.get("memories_extracted", {})
+ total_mem = sum(memories.values()) if memories else 0
+ if auto_committed and total_mem == 0:
+ check(
+ "记忆提取状态合理 (auto-commit 已处理)",
+ True,
+ "auto-commit 已提取记忆",
+ )
+ else:
+ check(
+ "memories_extracted > 0 (提取出记忆)",
+ total_mem > 0,
+ f"total={total_mem}, categories={memories}",
+ )
+
+ # 3.2 commit 后 session 状态
+ console.print("\n[bold]3.2 Session 归档状态[/bold]")
+ post_session = inspector.get_session(session_id)
+ if post_session:
+ commit_count = post_session.get("commit_count", 0)
+ check(
+ "commit_count >= 1",
+ commit_count >= 1,
+ f"commit_count={commit_count}",
+ )
+
+ total_memories = post_session.get("memories_extracted", {})
+ total_mem_count = sum(total_memories.values()) if isinstance(total_memories, dict) else 0
+ check(
+ "累计提取记忆 > 0",
+ total_mem_count > 0,
+ f"total={total_mem_count}, categories={total_memories}",
+ )
+
+ post_pending = post_session.get("pending_tokens", 0)
+ console.print(f" [dim]commit 后 pending_tokens={post_pending}[/dim]")
+
+ # 3.3 检查归档目录结构
+ console.print("\n[bold]3.3 归档目录结构检查[/bold]")
+ ctx_after = inspector.get_session_context(session_id)
+ if ctx_after:
+ has_summary_archive = bool(ctx_after.get("latest_archive_overview"))
+ check(
+ "context 返回 latest_archive_overview",
+ has_summary_archive,
+ f"overview={'有' if has_summary_archive else '无'}",
+ )
+
+ if has_summary_archive:
+ overview = ctx_after.get("latest_archive_overview", "")
+ check(
+ "latest_archive_overview 非空 (摘要已生成)",
+ len(overview) > 10,
+ f"overview 长度={len(overview)} chars",
+ )
+ if verbose:
+ console.print(f" [dim]overview 前 200 字: {overview[:200]}...[/dim]")
+ else:
+ check("context 可调用", False)
+
+ # 3.4 检查 estimatedTokens 合理性
+ if ctx_after:
+ stats = ctx_after.get("stats", {})
+ archive_tokens = stats.get("archiveTokens", 0)
+ check(
+ "archiveTokens > 0 (归档 token 计数合理)",
+ archive_tokens > 0,
+ f"archiveTokens={archive_tokens}",
+ )
+
+ return True
+
+
+# ── Phase 4: Assemble 验证 ───────────────────────────────────────────────
+
+
+def run_phase_assemble(
+ gateway_url: str, openviking_url: str, user_id: str, session_id: str,
+ delay: float, verbose: bool,
+) -> bool:
+ """Phase 4: Assemble 验证 — 同用户继续对话,验证上下文从 latest archive overview 重组。"""
+ console.print()
+ console.rule("[bold]Phase 4: Assemble 验证 — 同用户继续对话[/bold]")
+ console.print()
+ console.print("[dim]验证点:[/dim]")
+ console.print(
+ "[dim]- 同用户对话触发 assemble(): 从 OV latest_archive_overview + active messages 重组上下文[/dim]"
+ )
+ console.print("[dim]- 回复应能引用 Phase 1 中已被归档的信息[/dim]")
+ console.print("[dim]- context 应返回 latest_archive_overview (证明 assemble 有数据源)[/dim]")
+ console.print(f"[dim]- OV session_id: {session_id}[/dim]")
+ console.print()
+
+ inspector = OpenVikingInspector(openviking_url)
+
+ # 4.1 确认 assemble 的数据源 (latest_archive_overview) 就绪
+ console.print("[bold]4.1 确认 assemble 数据源[/bold]")
+ ctx = inspector.get_session_context(session_id)
+ if ctx:
+ has_summary_archive = bool(ctx.get("latest_archive_overview"))
+ check(
+ "context 返回 latest_archive_overview",
+ has_summary_archive,
+ f"latest_archive_overview={'有' if has_summary_archive else '无'}",
+ )
+ else:
+ check("context 可用", False)
+
+ # 4.2 assemble budget trimming: 用极小 budget 验证裁剪
+ console.print("\n[bold]4.2 Assemble budget trimming[/bold]")
+ tiny_ctx = inspector.get_session_context(session_id, token_budget=1)
+ if tiny_ctx:
+ stats = tiny_ctx.get("stats", {})
+ total_archives = stats.get("totalArchives", 0)
+ included = stats.get("includedArchives", 0)
+ dropped = stats.get("droppedArchives", 0)
+ check(
+ "budget=1 时 latest_archive_overview 被裁剪",
+ included == 0 or dropped > 0,
+ f"total={total_archives}, included={included}, dropped={dropped}",
+ )
+ active_tokens = stats.get("activeTokens", 0)
+ console.print(
+ f" [dim]activeTokens={active_tokens}, archiveTokens={stats.get('archiveTokens', 0)}[/dim]"
+ )
+ else:
+ check("tiny budget context 可调用", False)
+
+ # 4.3 同用户继续对话 — assemble 应重组归档上下文
+ console.print("\n[bold]4.3 同用户继续对话 — 验证 assemble 重组归档内容[/bold]")
+ console.print(f"[yellow]用户ID:[/yellow] {user_id} (同一用户,继续对话)")
+ console.print()
+
+ total = len(ASSEMBLE_FOLLOWUP_MESSAGES)
+ for i, item in enumerate(ASSEMBLE_FOLLOWUP_MESSAGES, 1):
+ q = item["question"]
+ keywords = item["anchor_keywords"]
+
+ console.rule(f"[dim]Assemble 验证 {i}/{total}[/dim]", style="dim")
+ console.print(
+ Panel(
+ f"{q}\n\n[dim]锚点关键词: {', '.join(keywords)}[/dim]\n[dim]Hook: {item['hook']}[/dim]",
+ title=f"[bold cyan]Assemble Q{i}[/bold cyan]",
+ border_style="cyan",
+ )
+ )
+
+ try:
+ data = send_message(gateway_url, q, user_id)
+ reply = extract_reply_text(data)
+ render_reply(reply)
+
+ reply_lower = reply.lower()
+ hits = [kw for kw in keywords if kw.lower() in reply_lower]
+ hit_rate = len(hits) / len(keywords) if keywords else 0
+ check(
+ f"Assemble Q{i}: 回复包含归档内容 (命中率 >= 50%)",
+ hit_rate >= 0.5,
+ f"命中={hits}, 未命中={[k for k in keywords if k not in hits]}, rate={hit_rate:.0%}",
+ )
+ except Exception as e:
+ check(f"Assemble Q{i}: 发送成功", False, str(e))
+
+ if i < total:
+ time.sleep(delay)
+
+ # 4.4 对话后验证 afterTurn 继续写入 (新消息进入 active messages)
+ console.print("\n[bold]4.4 Assemble 后 afterTurn 继续写入[/bold]")
+ time.sleep(3)
+ post_ctx = inspector.get_session_context(session_id)
+ if post_ctx:
+ post_msg_count = len(post_ctx.get("messages", []))
+ check(
+ "继续对话后 active messages 增加",
+ post_msg_count > 0,
+ f"active messages={post_msg_count}",
+ )
+
+ return True
+
+
+# ── Phase 5: SessionId 一致性验证 ────────────────────────────────────────
+
+
+def run_phase_session_id(
+ openviking_url: str, user_id: str, session_id: str, verbose: bool,
+) -> bool:
+ """Phase 5: SessionId 一致性验证 — 确认整条链路使用统一的 sessionId。"""
+ console.print()
+ console.rule("[bold]Phase 5: SessionId 一致性验证[/bold]")
+ console.print()
+ console.print("[dim]验证点:[/dim]")
+ console.print("[dim]- 整条链路使用同一个 OV session_id[/dim]")
+ console.print("[dim]- 不存在以 sessionKey 变体为 ID 的残留 session[/dim]")
+ console.print("[dim]- context 用 session_id 可查到数据[/dim]")
+ console.print(f"[dim]- resolved session_id: {session_id}[/dim]")
+ console.print()
+
+ inspector = OpenVikingInspector(openviking_url)
+
+ # 5.1 resolved session_id 可查到
+ console.print("[bold]5.1 OV session 可查到[/bold]")
+ session = inspector.get_session(session_id)
+ check(
+ f"OV session (id={session_id[:24]}...) 存在",
+ session is not None,
+ f"user_id={user_id}",
+ )
+
+ # 5.2 不存在以 sessionKey 变体为 ID 的 session
+ console.print("\n[bold]5.2 无 sessionKey 残留[/bold]")
+ stale_variants = [
+ f"sk:{user_id}",
+ f"sessionKey:{user_id}",
+ f"key:{user_id}",
+ ]
+ for variant in stale_variants:
+ stale = inspector.get_session(variant)
+ is_absent = stale is None or stale.get("message_count", 0) == 0
+ check(
+ f"不存在残留 session「{variant}」",
+ is_absent,
+ "旧 sessionKey 映射应已移除" if is_absent else f"发现残留: {stale}",
+ )
+
+ # 5.3 context 用 session_id 能查到数据
+ console.print("\n[bold]5.3 同一 sessionId 查询归档[/bold]")
+ ctx = inspector.get_session_context(session_id)
+ if ctx:
+ has_data = bool(ctx.get("latest_archive_overview")) or len(ctx.get("messages", [])) > 0
+ check(
+ "context(session_id) 返回数据",
+ has_data,
+ f"overview={'有' if ctx.get('latest_archive_overview') else '无'}, messages={len(ctx.get('messages', []))}",
+ )
+ else:
+ check("context(session_id) 可调用", False)
+
+ # 5.4 验证 commit 也是用同一 sessionId (session 有 commit_count > 0)
+ console.print("\n[bold]5.4 Commit 使用同一 sessionId[/bold]")
+ if session:
+ cc = session.get("commit_count", 0)
+ check(
+ "session 有 commit 记录",
+ cc > 0,
+ f"commit_count={cc}",
+ )
+
+ return True
+
+
+# ── Phase 6: 新用户记忆召回 ──────────────────────────────────────────────
+
+
+def run_phase_recall(gateway_url: str, user_id: str, delay: float, verbose: bool) -> list:
+ """Phase 6: 新用户记忆召回 — 验证 before_prompt_build auto-recall。"""
+ console.print()
+ console.rule(f"[bold]Phase 6: 新用户记忆召回 ({len(RECALL_QUESTIONS)} 轮) — auto-recall[/bold]")
+ console.print()
+ console.print("[dim]验证点:[/dim]")
+ console.print("[dim]- 新用户 (新 session) 发送问题[/dim]")
+ console.print("[dim]- before_prompt_build 通过 memory search 注入相关记忆[/dim]")
+ console.print("[dim]- 回复应包含 Phase 1 对话中的关键信息[/dim]")
+ console.print()
+
+ verify_user = f"{user_id}-recall-{uuid.uuid4().hex[:4]}"
+ console.print(f"[yellow]验证用户:[/yellow] {verify_user} (新 session)")
+ console.print()
+
+ results = []
+ total = len(RECALL_QUESTIONS)
+
+ for i, item in enumerate(RECALL_QUESTIONS, 1):
+ q = item["question"]
+ expected = item["expected_keywords"]
+
+ console.rule(f"[dim]Recall {i}/{total}[/dim]", style="dim")
+ console.print(
+ Panel(
+ f"{q}\n\n[dim]期望关键词: {', '.join(expected)}[/dim]",
+ title=f"[bold cyan]Recall Q{i}[/bold cyan]",
+ border_style="cyan",
+ )
+ )
+
+ try:
+ data = send_message(gateway_url, q, verify_user)
+ reply = extract_reply_text(data)
+ render_reply(reply)
+
+ reply_lower = reply.lower()
+ hits = [kw for kw in expected if kw.lower() in reply_lower]
+ hit_rate = len(hits) / len(expected) if expected else 0
+ success = hit_rate >= 0.5
+
+ check(
+ f"Recall Q{i}: 关键词命中率 >= 50%",
+ success,
+ f"命中={hits}, rate={hit_rate:.0%}",
+ )
+ results.append({"question": q, "hits": hits, "hit_rate": hit_rate, "success": success})
+ except Exception as e:
+ check(f"Recall Q{i}: 发送成功", False, str(e))
+ results.append({"question": q, "hits": [], "hit_rate": 0, "success": False})
+
+ if i < total:
+ time.sleep(delay)
+
+ return results
+
+
+# ── 完整测试 ──────────────────────────────────────────────────────────────
+
+
+def run_full_test(gateway_url: str, openviking_url: str, user_id: str, delay: float, verbose: bool):
+ console.print()
+ console.print(
+ Panel.fit(
+ f"[bold]OpenClaw 记忆链路完整测试[/bold]\n\n"
+ f"Gateway: {gateway_url}\n"
+ f"OpenViking: {openviking_url}\n"
+ f"User ID: {user_id}\n"
+ f"时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}",
+ title="测试信息",
+ )
+ )
+
+ # Phase 1: Chat
+ chat_ok, chat_fail = run_phase_chat(gateway_url, user_id, delay, verbose)
+
+ # Phase 2: afterTurn — 返回实际的 OV session_id
+ _, resolved_session_id, auto_committed = run_phase_after_turn(openviking_url, user_id, verbose)
+
+ # Phase 3: Commit
+ run_phase_commit(openviking_url, resolved_session_id, verbose, auto_committed)
+
+ console.print("\n[yellow]等待 10s 让记忆提取完成...[/yellow]")
+ time.sleep(10)
+
+ # Phase 4: Assemble (同用户继续)
+ run_phase_assemble(gateway_url, openviking_url, user_id, resolved_session_id, delay, verbose)
+
+ # Phase 5: SessionId 一致性
+ run_phase_session_id(openviking_url, user_id, resolved_session_id, verbose)
+
+ # Phase 6: 新用户召回
+ run_phase_recall(gateway_url, user_id, delay, verbose)
+
+ # ── 汇总报告 ──────────────────────────────────────────────────────────
+ console.print()
+ console.rule("[bold]测试报告[/bold]")
+
+ passed = sum(1 for a in assertions if a["ok"])
+ failed = sum(1 for a in assertions if not a["ok"])
+ total = len(assertions)
+
+ table = Table(title=f"断言结果: {passed}/{total} 通过")
+ table.add_column("#", style="bold", width=4)
+ table.add_column("状态", width=6)
+ table.add_column("断言", max_width=60)
+ table.add_column("详情", style="dim", max_width=50)
+
+ for i, a in enumerate(assertions, 1):
+ status = "[green]PASS[/green]" if a["ok"] else "[red]FAIL[/red]"
+ table.add_row(str(i), status, a["label"][:60], (a.get("detail") or "")[:50])
+
+ console.print(table)
+
+ # 按阶段汇总
+ tree = Tree(f"[bold]通过: {passed}/{total}, 失败: {failed}[/bold]")
+ tree.add(f"Phase 1: 多轮对话 — {chat_ok} 成功 / {chat_fail} 失败")
+
+ fail_list = [a for a in assertions if not a["ok"]]
+ if fail_list:
+ fail_branch = tree.add(f"[red]失败断言 ({len(fail_list)})[/red]")
+ for a in fail_list:
+ fail_branch.add(f"[red]FAIL[/red] {a['label']}")
+
+ console.print(tree)
+
+ if failed == 0:
+ console.print("\n[green bold]全部通过!端到端链路验证成功。[/green bold]")
+ else:
+ console.print(f"\n[red bold]有 {failed} 个断言失败,请检查上方详情。[/red bold]")
+
+
+# ── 入口 ───────────────────────────────────────────────────────────────────
+
+
+def main():
+ global AGENT_ID
+ parser = argparse.ArgumentParser(
+ description="OpenClaw 记忆链路完整测试",
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ epilog="""
+示例:
+ python test-memory-chain.py
+ python test-memory-chain.py --gateway http://127.0.0.1:18790
+ python test-memory-chain.py --phase chat
+ python test-memory-chain.py --phase afterTurn --user-id test-chain-abc123
+ python test-memory-chain.py --phase assemble --user-id test-chain-abc123
+ python test-memory-chain.py --verbose
+ """,
+ )
+ parser.add_argument(
+ "--gateway",
+ default=DEFAULT_GATEWAY,
+ help=f"OpenClaw Gateway 地址 (默认: {DEFAULT_GATEWAY})",
+ )
+ parser.add_argument(
+ "--openviking",
+ default=DEFAULT_OPENVIKING,
+ help=f"OpenViking 服务地址 (默认: {DEFAULT_OPENVIKING})",
+ )
+ parser.add_argument(
+ "--user-id",
+ default=USER_ID,
+ help="测试用户ID (默认: 随机生成)",
+ )
+ parser.add_argument(
+ "--phase",
+ choices=["all", "chat", "afterTurn", "commit", "assemble", "session-id", "recall"],
+ default="all",
+ help="运行阶段 (默认: all)",
+ )
+ parser.add_argument(
+ "--delay",
+ type=float,
+ default=2.0,
+ help="轮次间等待秒数 (默认: 2)",
+ )
+ parser.add_argument(
+ "--token",
+ default="",
+ help="Gateway auth token (默认: 自动从 openclaw.json 发现)",
+ )
+ parser.add_argument(
+ "--agent-id",
+ default=AGENT_ID,
+ help=f"OpenViking agent ID (默认: {AGENT_ID})",
+ )
+ parser.add_argument(
+ "--verbose",
+ "-v",
+ action="store_true",
+ help="详细输出",
+ )
+ args = parser.parse_args()
+
+ gateway_url = args.gateway.rstrip("/")
+ openviking_url = args.openviking.rstrip("/")
+ user_id = args.user_id
+
+ token = args.token or discover_gateway_token()
+ set_gateway_token(token)
+
+ AGENT_ID = args.agent_id
+
+ console.print("[bold]OpenClaw 记忆链路测试[/bold]")
+ console.print(f"[yellow]Gateway:[/yellow] {gateway_url}")
+ console.print(f"[yellow]OpenViking:[/yellow] {openviking_url}")
+ console.print(f"[yellow]User ID:[/yellow] {user_id}")
+
+ if args.phase == "all":
+ run_full_test(gateway_url, openviking_url, user_id, args.delay, args.verbose)
+ elif args.phase == "chat":
+ run_phase_chat(gateway_url, user_id, args.delay, args.verbose)
+ elif args.phase == "afterTurn":
+ _, _, _ = run_phase_after_turn(openviking_url, user_id, args.verbose)
+ elif args.phase == "commit":
+ inspector = OpenVikingInspector(openviking_url, agent_id=AGENT_ID)
+ sid = inspector.find_session_for_user(user_id) or user_id
+ session_info = inspector.get_session(sid)
+ ac = (session_info.get("commit_count", 0) > 0) if session_info else False
+ run_phase_commit(openviking_url, sid, args.verbose, ac)
+ elif args.phase == "assemble":
+ inspector = OpenVikingInspector(openviking_url, agent_id=AGENT_ID)
+ sid = inspector.find_session_for_user(user_id) or user_id
+ run_phase_assemble(gateway_url, openviking_url, user_id, sid, args.delay, args.verbose)
+ elif args.phase == "session-id":
+ inspector = OpenVikingInspector(openviking_url, agent_id=AGENT_ID)
+ sid = inspector.find_session_for_user(user_id) or user_id
+ run_phase_session_id(openviking_url, user_id, sid, args.verbose)
+ elif args.phase == "recall":
+ run_phase_recall(gateway_url, user_id, args.delay, args.verbose)
+
+ # 打印最终断言统计
+ if assertions:
+ passed = sum(1 for a in assertions if a["ok"])
+ total = len(assertions)
+ console.print(f"\n[yellow]断言统计: {passed}/{total} 通过[/yellow]")
+
+ console.print("\n[yellow]测试结束。[/yellow]")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/openclaw-plugin/tests/e2e/test-tool-capture.py b/examples/openclaw-plugin/tests/e2e/test-tool-capture.py
new file mode 100644
index 000000000..6d417acd2
--- /dev/null
+++ b/examples/openclaw-plugin/tests/e2e/test-tool-capture.py
@@ -0,0 +1,555 @@
+#!/usr/bin/env python3
+"""
+extractNewTurnTexts 工具调用捕获端到端测试
+
+================================================================================
+一、用例设计思路
+================================================================================
+
+核心验证点:
+ 当模型在回复中调用工具(如 code_execution、native_tool 等)时,Gateway 的
+ extractNewTurnTexts 需要将 toolUse(工具调用)和 toolResult(工具结果)的
+ 内容正确捕获并写入 OV session,确保后续归档和记忆提取不会丢失工具调用信息。
+
+测试策略:
+ 1. 发送消息触发模型使用工具(如计算阶乘、写代码等)
+ 2. 等待 afterTurn 完成
+ 3. 从 OV session 中读取已存储的消息
+ 4. 断言存储的消息中包含 [toolUse:] 和 [toolResult:] 标记
+ 5. 验证关键词可在 Gateway 响应和 OV 存储中追溯
+
+================================================================================
+二、测试流程
+================================================================================
+
+ Phase 1: 发送 3 条消息,设计为触发工具调用
+ Phase 2: 检查 OV session 存在且有内容
+ Phase 3: 验证 toolUse/toolResult 标记和关键词可追溯
+ Phase 4: 验证改动前后对比(tool 相关行数 > 0)
+
+================================================================================
+三、环境前提
+================================================================================
+
+ 1. OpenViking 服务已启动
+ 2. OpenClaw Gateway 已启动并配置了 OpenViking 插件
+ 3. LLM 后端可达且支持工具调用(function calling / tool use)
+ 4. 有效的 Gateway auth token
+
+ 关键 openclaw.json 配置:
+ - plugins.slots.contextEngine = "openviking"
+ - plugins.entries.openviking.enabled = true
+ - plugins.entries.openviking.config.autoCapture = true # afterTurn 自动捕获
+
+================================================================================
+四、使用方法
+================================================================================
+
+ 安装依赖:
+ pip install requests rich
+
+ 运行测试:
+ python test-tool-capture.py \\
+ --gateway http://127.0.0.1:19789 \\
+ --openviking http://127.0.0.1:2934 \\
+ --token <your_gateway_token>
+
+ 其他选项:
+ --verbose / -v 详细输出(显示完整 JSON 响应)
+ --delay <sec> 消息间等待秒数(默认 5s)
+
+ 注意:
+ - 测试约需 2-3 分钟
+ - 首次运行前建议清理 OV 数据和 session 数据
+
+================================================================================
+五、已知限制
+================================================================================
+
+ 1. 模型工具调用行为不确定:
+ 不同 LLM 模型对同一输入是否调用工具的行为不同。有些模型可能选择直接
+ 回答而不调用工具。脚本对 [toolUse:] 标记做了条件性检查(模型未调用工具
+ 时跳过强断言)。
+
+ 2. 工具调用格式差异:
+ 不同 LLM provider 的 tool_use/tool_result 输出格式可能不同(如 Anthropic
+ vs OpenAI),extractNewTurnTexts 需要正确处理各种格式。
+
+ 3. 关键词追溯:
+ 脚本通过关键词(如"5040"、"factorial"、"斐波那契")验证工具结果是否被
+ 正确存储。如果模型未执行预期的计算,关键词可能无法匹配。
+
+================================================================================
+六、预期结果
+================================================================================
+
+ 15/15 断言全部通过:
+ - Phase 1: 3 条消息发送成功
+ - Phase 2: OV session 存在且有内容
+ - Phase 3: toolResult 标记存在, 关键词可追溯
+ - Phase 4: tool 相关行数 > 0
+"""
+
+import argparse
+import io
+import json
+import re
+import sys
+import time
+import uuid
+from datetime import datetime
+
+import requests
+from rich.console import Console
+from rich.panel import Panel
+from rich.table import Table
+
+if sys.platform == "win32":
+ sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="replace")
+ sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8", errors="replace")
+
+# ── 常量 ──────────────────────────────────────────────────────────────────
+
+GATEWAY_URL = "http://127.0.0.1:19789"
+OPENVIKING_URL = "http://127.0.0.1:2934"
+AGENT_ID = "main"
+
+console = Console(force_terminal=True)
+assertions: list[dict] = []
+
+
+def check(label: str, condition: bool, detail: str = ""):
+ assertions.append({"label": label, "ok": condition, "detail": detail})
+ icon = "[green]PASS[/green]" if condition else "[red]FAIL[/red]"
+ msg = f" {icon} {label}"
+ if detail:
+ msg += f" [dim]({detail})[/dim]"
+ console.print(msg)
+
+
+def load_gateway_token() -> str:
+ """从常见路径自动发现 gateway auth token。"""
+ import os
+ import pathlib
+
+ candidates = [
+ pathlib.Path.cwd() / "config" / ".openclaw" / "openclaw.json",
+ pathlib.Path.home() / ".openclaw" / "openclaw.json",
+ ]
+ state_dir = os.environ.get("OPENCLAW_STATE_DIR")
+ if state_dir:
+ candidates.insert(0, pathlib.Path(state_dir) / "openclaw.json")
+
+ for p in candidates:
+ try:
+ cfg = json.loads(p.read_text(encoding="utf-8"))
+ token = cfg.get("gateway", {}).get("auth", {}).get("token", "")
+ if token:
+ return token
+ except Exception:
+ continue
+ return ""
+
+
+# ── API helpers ──────────────────────────────────────────────────────────
+
+
+def send_message(gateway_url: str, message: str, user_id: str, token: str) -> dict:
+ """通过 OpenClaw Responses API 发送消息。"""
+ headers = {"Content-Type": "application/json"}
+ if token:
+ headers["Authorization"] = f"Bearer {token}"
+ resp = requests.post(
+ f"{gateway_url}/v1/responses",
+ headers=headers,
+ json={"model": "openclaw", "input": message, "user": user_id},
+ timeout=300,
+ )
+ resp.raise_for_status()
+ return resp.json()
+
+
+def extract_reply_text(data: dict) -> str:
+ for item in data.get("output", []):
+ if item.get("type") == "message" and item.get("role") == "assistant":
+ for part in item.get("content", []):
+ if part.get("type") in ("text", "output_text"):
+ return part.get("text", "")
+ return "(无回复)"
+
+
+def has_tool_use_in_output(data: dict) -> bool:
+ """检查 Responses API 返回中是否有 tool_use / function_call。"""
+ for item in data.get("output", []):
+ item_type = item.get("type", "")
+ if item_type in ("function_call", "tool_use", "computer_call"):
+ return True
+ if item.get("role") == "assistant":
+ for part in item.get("content", []):
+ if part.get("type") in ("tool_use", "toolUse"):
+ return True
+ return False
+
+
+class OVInspector:
+ def __init__(self, base_url: str, agent_id: str = AGENT_ID):
+ self.base_url = base_url.rstrip("/")
+ self.agent_id = agent_id
+
+ def _headers(self) -> dict:
+ h: dict[str, str] = {"Content-Type": "application/json"}
+ if self.agent_id:
+ h["X-OpenViking-Agent"] = self.agent_id
+ return h
+
+ def _get(self, path: str, timeout: int = 10):
+ try:
+ resp = requests.get(f"{self.base_url}{path}", headers=self._headers(), timeout=timeout)
+ if resp.status_code == 200:
+ data = resp.json()
+ return data.get("result", data)
+ return None
+ except Exception as e:
+ console.print(f"[dim]GET {path} 失败: {e}[/dim]")
+ return None
+
+ def list_sessions(self) -> list:
+ result = self._get("/api/v1/sessions")
+ if isinstance(result, list):
+ return result
+ return []
+
+ def get_session(self, session_id: str):
+ return self._get(f"/api/v1/sessions/{session_id}")
+
+ def get_session_context(self, session_id: str, token_budget: int = 128000):
+ return self._get(f"/api/v1/sessions/{session_id}/context?token_budget={token_budget}")
+
+ def find_latest_session(self) -> str | None:
+ """找到最近更新的 session ID(gateway 内部使用 UUID,非 user_id)。
+ 通过检查每个 session 的 updated_at 来找到最新的。"""
+ sessions = self.list_sessions()
+ real_sessions = [
+ s for s in sessions
+ if isinstance(s, dict) and not s.get("session_id", "").startswith("memory-store-")
+ ]
+ if not real_sessions:
+ return None
+
+ best_id = None
+ best_time = ""
+ for s in real_sessions:
+ sid = s.get("session_id", "")
+ if not sid:
+ continue
+ detail = self.get_session(sid)
+ if not detail:
+ continue
+ updated = detail.get("updated_at", "")
+ if updated > best_time:
+ best_time = updated
+ best_id = sid
+
+ return best_id or real_sessions[-1].get("session_id")
+
+
+# ── 核心测试 ──────────────────────────────────────────────────────────────
+
+
+TOOL_TRIGGER_MESSAGES = [
+ {
+ "input": "请帮我计算 factorial(7) 的结果,用代码算一下",
+ "description": "触发代码执行工具",
+ "expect_keywords": ["5040", "factorial"],
+ },
+ {
+ "input": "我叫李明,记住我是一名数据工程师,擅长 Spark 和 Flink,偏好用 Scala 写代码。请同时告诉我今天星期几。",
+ "description": "信息存储 + 可能触发工具",
+ "expect_keywords": ["李明", "数据工程师"],
+ },
+ {
+ "input": "帮我写一段 Python 代码计算斐波那契数列前10个数,并运行它告诉我结果",
+ "description": "触发代码执行并返回结果",
+ "expect_keywords": ["斐波那契"],
+ },
+]
+
+
+def run_test(
+ gateway_url: str,
+ openviking_url: str,
+ user_id: str,
+ delay: float,
+ verbose: bool,
+ token: str = "",
+ agent_id: str = "",
+):
+ if not token:
+ token = load_gateway_token()
+ inspector = OVInspector(openviking_url, agent_id=agent_id or AGENT_ID)
+
+ console.print(
+ Panel(
+ f"[bold]Tool Capture 测试[/bold]\n\n"
+ f"Gateway: {gateway_url}\n"
+ f"OpenViking: {openviking_url}\n"
+ f"User ID: {user_id}\n"
+ f"时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}",
+ title="测试信息",
+ )
+ )
+
+ # ── Phase 1: 发送消息 ────────────────────────────────────────────────
+
+ console.rule("[bold]Phase 1: 发送消息触发 afterTurn[/bold]")
+
+ gateway_responses = []
+ for i, msg_cfg in enumerate(TOOL_TRIGGER_MESSAGES):
+ console.print(f"\n[cyan]消息 {i + 1}/{len(TOOL_TRIGGER_MESSAGES)}:[/cyan] {msg_cfg['description']}")
+ console.print(f" [dim]> {msg_cfg['input'][:80]}...[/dim]")
+
+ try:
+ data = send_message(gateway_url, msg_cfg["input"], user_id, token)
+ reply = extract_reply_text(data)
+ has_tool = has_tool_use_in_output(data)
+
+ console.print(f" [green]回复:[/green] {reply[:120]}...")
+ if has_tool:
+ console.print(" [yellow]检测到 tool_use 在响应中[/yellow]")
+
+ if verbose:
+ console.print(f" [dim]完整响应: {json.dumps(data, ensure_ascii=False)[:500]}[/dim]")
+
+ gateway_responses.append({
+ "index": i,
+ "msg": msg_cfg,
+ "response": data,
+ "reply": reply,
+ "has_tool": has_tool,
+ })
+
+ check(
+ f"消息 {i + 1} 发送成功",
+ True,
+ f"reply_len={len(reply)}",
+ )
+ except Exception as e:
+ console.print(f" [red]发送失败: {e}[/red]")
+ check(f"消息 {i + 1} 发送成功", False, str(e))
+
+ if i < len(TOOL_TRIGGER_MESSAGES) - 1:
+ time.sleep(delay)
+
+ # ── Phase 2: 等待 afterTurn 写入 ───────────────────────────────────
+
+ console.rule("[bold]Phase 2: 检查 OV session 中的存储内容[/bold]")
+ console.print("[yellow]等待 afterTurn 写入 OV session...[/yellow]")
+ time.sleep(8)
+
+ # Gateway 使用内部 UUID 作为 session ID,需要从 OV 列表中找到最新的
+ ov_session_id = inspector.find_latest_session()
+ if not ov_session_id:
+ console.print("[red] OV 中没有找到任何 session[/red]")
+ check("OV session 存在", False, "no sessions found")
+ print_summary()
+ return
+
+ console.print(f" [cyan]OV session ID: {ov_session_id}[/cyan]")
+
+ session_info = inspector.get_session(ov_session_id)
+ if session_info:
+ msg_count = session_info.get("message_count", "?")
+ console.print(f" Session found: message_count={msg_count}")
+ check("OV session 存在", True, f"id={ov_session_id[:16]}...")
+ else:
+ console.print("[red] OV session 详情获取失败[/red]")
+ check("OV session 存在", False, "session detail failed")
+ print_summary()
+ return
+
+ # 通过 context API 获取全量上下文(活跃消息 + 归档摘要)
+ ctx = inspector.get_session_context(ov_session_id)
+ messages = ctx.get("messages", []) if ctx else []
+
+ # 同时获取归档概要文本(低 commit 阈值下消息可能已归档)
+ archive_overview = ""
+ if ctx:
+ for msg in messages:
+ if isinstance(msg, dict):
+ for part in msg.get("parts", []):
+ if isinstance(part, dict) and part.get("type") == "text":
+ archive_overview += (part.get("text", "") or "") + "\n"
+
+ if not messages and not archive_overview:
+ console.print("[red] OV session 消息为空[/red]")
+ check("OV session 有消息", False, "context messages empty")
+ print_summary()
+ return
+
+ console.print(f" [green]OV session context 消息数: {len(messages)}[/green]")
+ check("OV session 有内容", len(messages) > 0 or bool(archive_overview), f"messages={len(messages)}")
+
+ # ── Phase 3: 分析存储的内容是否包含 tool 信息 ──────────────────────
+
+ console.rule("[bold]Phase 3: 验证 toolUse/toolResult 内容被捕获[/bold]")
+
+ all_stored_text = ""
+ for msg in messages:
+ if not isinstance(msg, dict):
+ continue
+ parts = msg.get("parts", [])
+ for part in parts:
+ if isinstance(part, dict) and part.get("type") == "text":
+ all_stored_text += (part.get("text", "") or "") + "\n"
+
+ if verbose:
+ console.print(Panel(
+ all_stored_text[:3000] + ("..." if len(all_stored_text) > 3000 else ""),
+ title="OV 存储的全部文本",
+ ))
+
+ any_tool_in_gateway = any(r.get("has_tool") for r in gateway_responses)
+
+ # 检查 toolUse 标记(仅在 gateway 响应确实含 tool_use 时才必须)
+ has_tool_use_marker = bool(re.search(r'\[toolUse:', all_stored_text, re.IGNORECASE))
+ if any_tool_in_gateway:
+ check(
+ "存储文本包含 [toolUse:] 标记",
+ has_tool_use_marker,
+ f"found={has_tool_use_marker}",
+ )
+ else:
+ check(
+ "[toolUse:] 标记(模型未调用工具,跳过强断言)",
+ True,
+ f"no tool_use in gateway response, marker={has_tool_use_marker}",
+ )
+
+ # 检查 toolResult 标记
+ has_tool_result_marker = bool(re.search(r'result\]:', all_stored_text, re.IGNORECASE))
+ check(
+ "存储文本包含 tool result 标记",
+ has_tool_result_marker or not any_tool_in_gateway,
+ f"found={has_tool_result_marker} tool_in_gateway={any_tool_in_gateway}",
+ )
+
+ # 检查 assistant 标记
+ has_assistant = bool(re.search(r'\[assistant\]:', all_stored_text, re.IGNORECASE))
+ check(
+ "存储文本包含 [assistant] 标记",
+ has_assistant,
+ f"found={has_assistant}",
+ )
+
+ # 检查 user 标记
+ has_user = bool(re.search(r'\[user\]:', all_stored_text, re.IGNORECASE))
+ check(
+ "存储文本包含 [user] 标记",
+ has_user,
+ f"found={has_user}",
+ )
+
+ # 检查关键内容是否保留(检查活跃上下文 + 归档摘要 + gateway 响应)
+ all_gateway_text = "\n".join(r.get("reply", "") for r in gateway_responses)
+ combined_text = all_stored_text + "\n" + all_gateway_text
+
+ for msg_cfg in TOOL_TRIGGER_MESSAGES:
+ for kw in msg_cfg.get("expect_keywords", []):
+ in_stored = kw.lower() in all_stored_text.lower()
+ in_gateway = kw.lower() in all_gateway_text.lower()
+ detail = f"keyword='{kw}' stored={in_stored} gateway={in_gateway}"
+ check(
+ f"关键词 {kw} 可追溯",
+ in_stored or in_gateway,
+ detail,
+ )
+
+ # ── Phase 4: 对比改动前后的行为 ──────────────────────────────────────
+
+ console.rule("[bold]Phase 4: 改动前后对比分析[/bold]")
+
+ # 旧版本:只有 [user] 和 [assistant] 的文本
+ # 新版本:应该额外包含 [toolUse: xxx] 和 [xxx result] 的内容
+ tool_related_lines = []
+ for line in all_stored_text.split("\n"):
+ stripped = line.strip()
+ if re.search(r'\[toolUse:', stripped, re.IGNORECASE):
+ tool_related_lines.append(("toolUse", stripped[:150]))
+ elif re.search(r'result\]:', stripped, re.IGNORECASE):
+ tool_related_lines.append(("toolResult", stripped[:150]))
+
+ if tool_related_lines:
+ table = Table(title="捕获到的 Tool 相关内容")
+ table.add_column("类型", style="cyan", width=12)
+ table.add_column("内容预览", max_width=120)
+ for kind, preview in tool_related_lines:
+ table.add_row(kind, preview)
+ console.print(table)
+
+ check(
+ "tool 相关行数 > 0(新逻辑生效)",
+ len(tool_related_lines) > 0,
+ f"tool_lines={len(tool_related_lines)}",
+ )
+
+ # ── 汇总 ─────────────────────────────────────────────────────────────
+
+ print_summary()
+
+
+def print_summary():
+ console.print()
+ console.rule("[bold]测试汇总[/bold]")
+
+ passed = sum(1 for a in assertions if a["ok"])
+ failed = sum(1 for a in assertions if not a["ok"])
+ total = len(assertions)
+
+ table = Table(title=f"断言结果: {passed}/{total} 通过")
+ table.add_column("#", style="bold", width=4)
+ table.add_column("状态", width=6)
+ table.add_column("断言", max_width=60)
+ table.add_column("详情", style="dim", max_width=50)
+
+ for i, a in enumerate(assertions, 1):
+ status = "[green]PASS[/green]" if a["ok"] else "[red]FAIL[/red]"
+ table.add_row(str(i), status, a["label"][:60], (a.get("detail") or "")[:50])
+
+ console.print(table)
+
+ if failed == 0:
+ console.print("\n[green bold]全部通过!toolUse/toolResult 捕获验证成功。[/green bold]")
+ else:
+ console.print(f"\n[red bold]有 {failed} 个断言失败。[/red bold]")
+ console.print("[yellow]注: 如果模型没有调用工具,toolUse/toolResult 标记可能不存在 — 这不代表代码有 bug。[/yellow]")
+ console.print("[yellow]可以在 gateway 日志中确认 afterTurn 的存储内容。[/yellow]")
+
+
+# ── 入口 ──────────────────────────────────────────────────────────────────
+
+
+def main():
+ parser = argparse.ArgumentParser(description="测试 toolUse/toolResult 捕获")
+ parser.add_argument("--gateway", default=GATEWAY_URL, help="Gateway 地址")
+ parser.add_argument("--openviking", default=OPENVIKING_URL, help="OpenViking 地址")
+ parser.add_argument("--token", default="", help="Gateway auth token (默认: 自动发现)")
+ parser.add_argument("--agent-id", default=AGENT_ID, help=f"OpenViking agent ID (默认: {AGENT_ID})")
+ parser.add_argument("--delay", type=float, default=3.0, help="消息间延迟秒数")
+ parser.add_argument("--verbose", "-v", action="store_true", help="详细输出")
+ args = parser.parse_args()
+
+ user_id = f"test-tool-{uuid.uuid4().hex[:8]}"
+
+ run_test(
+ gateway_url=args.gateway.rstrip("/"),
+ openviking_url=args.openviking.rstrip("/"),
+ user_id=user_id,
+ delay=args.delay,
+ verbose=args.verbose,
+ token=args.token,
+ agent_id=args.agent_id,
+ )
+
+
+if __name__ == "__main__":
+ main()