"Vibe-Trading: Your Personal Trading Agent"
Python ai-agentalgorithmic-tradingbacktestingfintechllmmcpmulti-agentpythonquantitative-financetrading
Merge PR #2074: reuse prior backtests with manifest-verified and distinct evidence
Haozhe Wu0a3c27f2ea64
diff --cc CHANGELOG.mdindex 0d831572f,39f3f7229..2ad4f94a5--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -5,29 -5,21 +5,42 @@@ This project adheres to [Semantic Versi## [Unreleased]+ ### Added++ - **`list_backtest_runs` finds the backtests earlier turns and sessions already+ ran**, so a follow-up turn reads their artifacts with `read_run_artifact`+ instead of re-running them. It lists runs from `req.json` and `run_card.json`+ (paged, current session by default, also over MCP) with each backtest's+ absolute `run_dir`. Assistant replies in the history now carry+ `[prev_run: <id>]` from the message metadata, and the grounding ledger adopts+ a prior run read through `read_run_artifact` (run card and `metrics.csv` must+ agree, tables must match the run card manifest). The session filter is not an+ access control: runs have no owner. A run forged by the model in an earlier+ turn is not detected.+### Fixed+- **Forex symbol detection** no longer depends on AKShare's module layout:+ `akshare.forex` was deleted in 1.20.0 and the symbol map moved to+ `akshare.fx.cons_forex`, so the single hard-coded import turned every FX pair+ into an A-share lookup. Both layouts are now probed, and a wholly missing map+ degrades to "not a forex pair" instead of silently matching nothing.+- **Bare US ETF and fund symbols** such as `QQQ` and `SPY` now resolve to+ `QQQ.US` / market `us` instead of a suffix-less `global` candidate. The fetch+ layer already routed bare US tickers as US equities, so the resolver was the+ only place that disagreed: the identity lock then rejected the follow-up call+ ("consumer symbol/venue differs from the locked resolver identity"). ETFs and+ mutual funds on non-US venues keep their native form (#2084).+- **`etf_holdings` accepts the `.US` market suffix** on US symbols, so the+ identity locked as `QQQ.US` still reaches the SEC series/class ticker `QQQ`+ instead of being reported as an unknown share class.+- **JSONP-wrapped provider bodies** decode instead of raising: the throttled+ JSON GET helper unwraps a `callback(...)` envelope when `response.json()`+ cannot parse the body, so A-share news search keeps working when Eastmoney+ switches to JSONP (#2081).+- **Degraded Eastmoney news indexes** now raise an actionable error instead of+ reading as an empty article list: the Guba user index (`passportWeb`) can no+ longer masquerade as "this stock has no news" (#2081).- **Global futures routing and pricing**: digit-bearing contracts such as`M2K=F` and `6EZ2026.CME`, including `local:` inputs, reach the globalfutures engine with their full product code. OJ and ZR use quote-unitdiff --cc README.mdindex b026f5c05,ea9630d63..032c8a1ad--- a/README.md+++ b/README.md@@@ -2364,7 -2314,7 +2364,7 @@@ Vibe-Trading│ │ ├── memory/ # Cross-session persistent memory│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)│ │ │- │ │ ├── tools/ # 116 auto-discovered agent tools-│ │ ├── tools/ # 114 auto-discovered agent tools++│ │ ├── tools/ # 117 auto-discovered agent tools│ │ │ ├── backtest_tool.py # run backtests│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)diff --cc README_ar.mdindex 0b79c90ac,c83f03e5c..b32303c94--- a/README_ar.md+++ b/README_ar.md@@@ -1757,7 -1751,7 +1757,7 @@@ Vibe-Trading│ │ ├── memory/ # Cross-session persistent memory│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)│ │ │- │ │ ├── tools/ # 116 auto-discovered agent tools-│ │ ├── tools/ # 114 auto-discovered agent tools++│ │ ├── tools/ # 117 auto-discovered agent tools│ │ │ ├── backtest_tool.py # run backtests│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)diff --cc README_es.mdindex 085552b58,6a07caa11..d04b3153e--- a/README_es.md+++ b/README_es.md@@@ -1845,7 -1839,7 +1845,7 @@@ Vibe-Trading│ │ ├── memory/ # Memoria persistente entre sesiones│ │ │ └── persistent.py # memoria basada en archivos (~/.vibe-trading/memory/)│ │ │- │ │ ├── tools/ # 116 herramientas de agente autodescubiertas-│ │ ├── tools/ # 114 herramientas de agente autodescubiertas++│ │ ├── tools/ # 117 herramientas de agente autodescubiertas│ │ │ ├── backtest_tool.py # ejecuta backtests│ │ │ ├── remember_tool.py # memoria entre sesiones (save/recall/forget)│ │ │ ├── skill_writer_tool.py # CRUD de skills (save/patch/delete/file)diff --cc README_id.mdindex e9c543c18,64109efe1..f7abe551a--- a/README_id.md+++ b/README_id.md@@@ -2090,7 -2084,7 +2090,7 @@@ Vibe-Trading│ │ ├── memory/ # Cross-session persistent memory│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)│ │ │- │ │ ├── tools/ # 116 auto-discovered agent tools-│ │ ├── tools/ # 114 auto-discovered agent tools++│ │ ├── tools/ # 117 auto-discovered agent tools│ │ │ ├── backtest_tool.py # run backtests│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)diff --cc README_ja.mdindex 4a5307d1b,7428958d8..13a3d1987--- a/README_ja.md+++ b/README_ja.md@@@ -1765,7 -1759,7 +1765,7 @@@ Vibe-Trading│ │ ├── memory/ # クロスセッション永続メモリ│ │ │ └── persistent.py # ファイルベースメモリ (~/.vibe-trading/memory/)│ │ │- │ │ ├── tools/ # 116 個の自動検出エージェントツール-│ │ ├── tools/ # 114 個の自動検出エージェントツール++│ │ ├── tools/ # 117 個の自動検出エージェントツール│ │ │ ├── backtest_tool.py # バックテスト実行│ │ │ ├── remember_tool.py # クロスセッションメモリ (save/recall/forget)│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)diff --cc README_ko.mdindex 8b80fe810,111cba108..42d11b4b2--- a/README_ko.md+++ b/README_ko.md@@@ -1762,7 -1756,7 +1762,7 @@@ Vibe-Trading│ │ ├── memory/ # Cross-session persistent memory│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)│ │ │- │ │ ├── tools/ # 116 auto-discovered agent tools-│ │ ├── tools/ # 114 auto-discovered agent tools++│ │ ├── tools/ # 117 auto-discovered agent tools│ │ │ ├── backtest_tool.py # run backtests│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)diff --cc README_zh.mdindex 6c6f6f7eb,3f1a61f1b..d226b48ab--- a/README_zh.md+++ b/README_zh.md@@@ -1769,7 -1754,7 +1769,7 @@@ Vibe-Trading│ │ ├── memory/ # 跨 session 持久记忆│ │ │ └── persistent.py # 基于文件的记忆(~/.vibe-trading/memory/)│ │ │- │ │ ├── tools/ # 116 个自动发现的 agent 工具-│ │ ├── tools/ # 114 个自动发现的 agent 工具++│ │ ├── tools/ # 117 个自动发现的 agent 工具│ │ │ ├── backtest_tool.py # 运行回测│ │ │ ├── remember_tool.py # 跨 session 记忆(save/recall/forget)│ │ │ ├── skill_writer_tool.py # skill CRUD(save/patch/delete/file)diff --cc agent/src/agent/grounding/evidence.pyindex ca3223780,c60af1740..0d22297f8--- a/agent/src/agent/grounding/evidence.py+++ b/agent/src/agent/grounding/evidence.py@@@ -239,6 -237,19 +240,19 @@@ def _relative_posix(path: Path, root: Preturn "" if relative == "." else relative+ def _artifact_ref(path: Path, root: Path) -> str:+ """``path`` as a ref names it: below the active run, else below its parent.++ A backtest an earlier turn left is a sibling of the active run, so+ ``<run id>/<backtest>/artifacts/metrics.csv`` names it exactly.+ """+ relative = _relative_posix(path, root)- if relative.startswith("/"):++ if Path(relative).is_absolute():+ with contextlib.suppress(OSError, ValueError):+ return Path(path).resolve().relative_to(root.parent).as_posix()+ return relative++def _file_sha256(path: Path) -> str | None:"""Hex SHA-256 of a file's bytes, or None when it cannot be read."""try:@@@ -934,9 -949,9 +952,9 @@@ class _EvidenceMixinor self._is_model_written(path)):continue-- if name not in _MANIFEST_EXEMPT and manifest.get(name) != _file_sha256(path):++ if (name not in _MANIFEST_EXEMPT or (verify_tables and name != "run_card.json")) and manifest.get(name) != _file_sha256(path):continue- artifact = _relative_posix(path, root)+ artifact = _artifact_ref(path, root)self._evidence = [record for record in self._evidence if record.artifact != artifact]for field_name, value in _summary_scalars(path):if room <= 0:@@@ -968,9 -983,47 +986,49 @@@):continuedigest = _file_sha256(path)- if digest is not None:+ if digest is not None and (+ not verify_tables or manifest.get(f"artifacts/{path.name}") == digest+ ):self._engine_tables[str(path.resolve())] = (digest, scope)+ def _adopt_prior_backtest(self, raw_dir: Any, call_id: str) -> None:+ """Ground figures read from a backtest an earlier turn finished.++ The ledger only watches backtests launched under the active run. A run+ reused through ``list_backtest_runs`` + ``read_run_artifact`` is adopted+ only if it sits in an allowed run root, its run card and ``metrics.csv``+ agree, no file is a symlink or one the model wrote, and each table it+ registers is listed in the run card manifest with the same hash. It is+ named in a figure ref as ``<run id>/<backtest>`` (or ``backtest::<run id>/<backtest>``).+ """+ if not raw_dir:+ return+ from src.tools.path_utils import safe_run_dir++ try:+ directory = safe_run_dir(str(raw_dir))+ except ValueError:+ return+ card, metrics = directory / "run_card.json", directory / "artifacts" / "metrics.csv"+ if directory.is_relative_to(self.run_dir.resolve()) or any(+ path.is_symlink() or self._is_model_written(path) for path in (card, metrics)+ ):+ return++ if _run_card_manifest(directory).get("artifacts/metrics.csv") != _file_sha256(metrics):++ return+ recorded = dict(_summary_scalars(card))+ shared = [(name, value) for name, value in _summary_scalars(metrics) if f"metrics.{name}" in recorded]+ if not shared or any(+ not math.isclose(value, recorded[f"metrics.{name}"], rel_tol=1e-9, abs_tol=1e-12)+ for name, value in shared+ ):+ return- scope = directory.name if (directory / "req.json").is_file() else f"{directory.parent.name}/{directory.name}"++ scope = _artifact_ref(directory, self.run_dir.resolve())+ self._record_metrics_file(metrics, call_id)+ self._backtest_scopes[call_id] = scope+ self._scope_latest[scope] = call_id+ self._record_backtest_outputs(directory, call_id, scope, verify_tables=True)+def _note_model_write(self, tool_name: str, arguments: Mapping[str, Any], payload: dict[str, Any] | None) -> None:diff --cc agent/src/agent/grounding/policies.pyindex a2ad289cb,4ef7edd4e..1fd306b6e--- a/agent/src/agent/grounding/policies.py+++ b/agent/src/agent/grounding/policies.py@@@ -2611,10 -2579,14 +2611,17 @@@ def _records_atpath = Path(token)if not path.is_absolute():path = root / path+ normalized = Path(os.path.normpath(path))try:- relative = Path(os.path.normpath(path)).relative_to(root).as_posix()+ relative = normalized.relative_to(root).as_posix()except ValueError:- return []+ try: # a backtest an earlier turn left sits beside the active run+ relative = normalized.relative_to(root.parent).as_posix()+ except ValueError:- return []++ # A prior run in another approved run root keeps an absolute++ # artifact ref. This matches observed records only; it authorizes++ # no filesystem read and admits no new evidence.++ relative = normalized.as_posix()if relative in ("", "."):return []diff --cc agent/src/tools/backtest_runs_tool.pyindex 000000000,f941d1b3f..8d283d5d0mode 000000,100644..100644--- a/agent/src/tools/backtest_runs_tool.py+++ b/agent/src/tools/backtest_runs_tool.py@@@ -1,0 -1,233 +1,245 @@@+ """Backtest run discovery: list earlier runs so their artifacts can be reused.++ Each user turn gets its own run directory, so a follow-up turn cannot reach the+ artifacts an earlier turn produced unless something names them. This tool lists+ runs from the files the runtime already writes (``req.json``, ``run_card.json``;+ no new index) and returns the absolute ``run_dir`` of every backtest for+ ``read_run_artifact``.++ The session filter is a convenience, not an access control: runs have no owner,+ ``scope="all"`` lists what ``GET /runs`` already shows, and ``read_run_artifact``+ and ``session_search`` reach the same data. Directories are confined to the+ allowed run roots and symlinks are never followed.+ """++ from __future__ import annotations++ import json+ from pathlib import Path+ from typing import Any++ from src.agent.tools import BaseTool+ from src.config.limits import TOOL_RESULT_LIMIT+ from src.config.paths import get_runs_dir+ from src.tools.path_utils import safe_run_dir, safe_run_id++ _SCOPES = ("session", "all")+ _MAX_LIMIT = 20+ _MAX_SCAN = 5000+ _MAX_BACKTESTS = 20+ _MAX_JSON_BYTES = 8 * 1024 * 1024+ _PROMPT_CHARS = 160+ _METRIC_KEYS = ("total_return", "sharpe", "max_drawdown", "unfilled_plan_rejections")+++ def _is_plain_dir(path: Path) -> bool:+ return path.is_dir() and not path.is_symlink()+++ def _as_dict(value: Any) -> dict[str, Any]:+ return value if isinstance(value, dict) else {}+++ def _read_json(path: Path, run_root: Path) -> dict[str, Any]:+ """A JSON object from a regular file inside ``run_root``, else ``{}``."""+ try:+ if path.is_symlink() or not path.resolve().is_relative_to(run_root):+ return {}+ if path.stat().st_size > _MAX_JSON_BYTES:+ return {}+ return _as_dict(json.loads(path.read_text(encoding="utf-8")))+ except (OSError, ValueError):+ return {}+++ def _session_of(run_root: Path) -> str | None:+ """Session that created a run, from the ``req.json`` the loop wrote."""+ return _as_dict(_read_json(run_root / "req.json", run_root).get("context")).get(+ "session_id"+ )+++ def _backtests_of(run_root: Path) -> list[dict[str, Any]]:+ """Backtests a turn left: the run itself and its direct sub-directories.++ A turn root that holds an archive manifest is a copy of one of its+ sub-directories, so it is not listed twice.+ """+ dirs = [] if (run_root / ".archived_backtest.json").exists() else [run_root]+ try:+ dirs += sorted(d for d in run_root.iterdir() if _is_plain_dir(d))+ except OSError:+ pass+ found = []+ for directory in dirs:+ card = _read_json(directory / "run_card.json", run_root)+ if not card:+ continue+ metrics = _as_dict(card.get("metrics"))+ config = _as_dict(card.get("backtest"))+ entry = {+ "name": directory.name if directory != run_root else ".",+ "run_dir": str(directory),+ "start_date": config.get("start_date"),+ "end_date": config.get("end_date"),+ "metrics": {key: metrics[key] for key in _METRIC_KEYS if key in metrics},+ }+ rejected = _as_dict(card.get("structured_metrics")).get(+ "unfilled_plan_rejections_by_symbol"+ )+ if rejected:- entry["unfilled_plan_rejections_by_symbol"] = rejected++ if len(json.dumps(rejected)) <= TOOL_RESULT_LIMIT // 8:++ entry["unfilled_plan_rejections_by_symbol"] = rejected++ else:++ entry["unfilled_plan_rejections_by_symbol_omitted"] = True+ found.append(entry)+ return found[:_MAX_BACKTESTS]+++ def _run_summary(run_root: Path) -> dict[str, Any]:+ prompt = " ".join(+ str(_read_json(run_root / "req.json", run_root).get("prompt") or "").split()+ )- return {++ result = {+ "run_id": run_root.name,+ "run_dir": str(run_root),+ "prompt": prompt[:_PROMPT_CHARS],+ "backtests": _backtests_of(run_root),+ }++ omitted = 0++ while len(result["backtests"]) > 1 and len(json.dumps(result)) > TOOL_RESULT_LIMIT // 2:++ result["backtests"].pop()++ omitted += 1++ if omitted:++ result["omitted_backtests"] = omitted++ return result+++ def _candidates(runs_root: Path):+ """Allowed run directories, newest first."""+ try:+ names = sorted(+ (d.name for d in runs_root.iterdir() if _is_plain_dir(d)), reverse=True+ )+ except OSError:+ return+ for name in names[:_MAX_SCAN]:+ try:+ yield safe_run_dir(str(runs_root / name))+ except ValueError:+ continue+++ def list_backtest_runs(+ *,+ current_run_dir: str | None = None,+ scope: str = "session",+ run_id: str | None = None,+ limit: int = 5,+ offset: int = 0,+ ) -> str:+ """List earlier runs, newest first, as a JSON envelope.++ Args:+ current_run_dir: The active turn's run directory (injected by the+ loop). It defines the session for ``scope="session"`` and is never+ listed.+ scope: ``session`` for the current session's runs, ``all`` for every run.+ run_id: Only this bare run directory name (validated by ``safe_run_id``).+ limit: Runs per page, 1..20.+ offset: Runs to skip (the previous page's ``next_offset``).++ Returns:+ JSON with ``runs`` (each with its ``run_dir`` and ``backtests``) and+ ``next_offset`` (``null`` on the last page).+ """+ if scope not in _SCOPES:+ return json.dumps(+ {"status": "error", "error": f"scope must be one of {list(_SCOPES)}"}+ )+ try:+ limit = min(max(int(limit), 1), _MAX_LIMIT)+ offset = max(int(offset), 0)+ runs = (+ iter([safe_run_id(run_id)])+ if run_id is not None+ else _candidates(get_runs_dir())+ )+ except (TypeError, ValueError) as exc:+ return json.dumps({"status": "error", "error": str(exc)})+ current = Path(current_run_dir).resolve() if current_run_dir else None+ session = _session_of(current) if current and scope == "session" else None++ page: list[dict[str, Any]] = []+ skipped, more = 0, False+ for run_root in runs:+ if run_root == current or (session and _session_of(run_root) != session):+ continue++ entry = _run_summary(run_root)++ if not entry["backtests"]:++ continue+ if skipped < offset:+ skipped += 1+ continue- entry = _run_summary(run_root)+ if page and (+ len(page) == limit+ or len(json.dumps([*page, entry])) > TOOL_RESULT_LIMIT // 2+ ):+ more = True+ break+ page.append(entry)+ return json.dumps(+ {+ "status": "ok",+ "runs": page,+ "next_offset": offset + len(page) if more else None,+ },+ ensure_ascii=False,+ default=str,+ )+++ class BacktestRunsTool(BaseTool):+ """Discover the backtests earlier turns and sessions already ran."""++ name = "list_backtest_runs"+ description = (+ "List earlier backtest runs (newest first, paged) with the absolute run_dir of each "+ "backtest, its dates and headline metrics. Use it in a follow-up turn, or for a "+ "'[prev_run: ...]' marker, then read artifacts with read_run_artifact(run_dir=...) "+ "instead of re-running the backtest. Defaults to the current session; scope='all' lists every run."+ )+ parameters = {+ "type": "object",+ "properties": {+ "scope": {+ "type": "string",+ "enum": list(_SCOPES),+ "description": "session (default) or all",+ },+ "run_id": {+ "type": "string",+ "description": "Only this run id (e.g. from a [prev_run: ...] marker)",+ },+ "limit": {+ "type": "integer",+ "description": f"Runs per page [1, {_MAX_LIMIT}] (default 5)",+ },+ "offset": {+ "type": "integer",+ "description": "Runs to skip; use next_offset from the previous page",+ },+ },+ "required": [],+ }+ repeatable = True+ is_readonly = True++ def execute(self, **kwargs: Any) -> str:+ return list_backtest_runs(+ current_run_dir=kwargs.get("run_dir"),+ scope=kwargs.get("scope") or "session",+ run_id=kwargs.get("run_id"),+ limit=kwargs.get("limit", 5),+ offset=kwargs.get("offset", 0),+ )diff --cc agent/tests/test_backtest_runs_tool.pyindex 000000000,108dfbb4d..384f23708mode 000000,100644..100644--- a/agent/tests/test_backtest_runs_tool.py+++ b/agent/tests/test_backtest_runs_tool.py@@@ -1,0 -1,223 +1,231 @@@+ """``list_backtest_runs``: find earlier runs and feed ``read_run_artifact``. All values are synthetic."""++ from __future__ import annotations++ import hashlib+ import json+ from pathlib import Path++ import pytest++ from src.tools import backtest_runs_tool+ from src.tools.backtest_runs_tool import BacktestRunsTool, list_backtest_runs+ from src.tools.run_artifact_tool import read_run_artifact++ SESSION = "sessionaaaa1"+++ def _write_backtest(directory: Path, rejections: dict | None = None) -> None:+ artifacts = directory / "artifacts"+ artifacts.mkdir(parents=True)+ (artifacts / "metrics.csv").write_text(+ "total_return,sharpe\n0.1,1.25\n", encoding="utf-8"+ )+ card = {+ "backtest": {"start_date": "2024-01-01", "end_date": "2024-12-31"},+ "metrics": {"total_return": 0.1, "sharpe": 1.25},+ "structured_metrics": {"unfilled_plan_rejections_by_symbol": rejections or {}},+ "artifacts": [+ {+ "path": "artifacts/metrics.csv",+ "sha256": hashlib.sha256(+ (artifacts / "metrics.csv").read_bytes()+ ).hexdigest(),+ }+ ],+ }+ (directory / "run_card.json").write_text(json.dumps(card), encoding="utf-8")+++ def _turn(+ runs: Path, name: str, session: str | None = SESSION, subs=("a", "b"), **kwargs+ ) -> Path:+ turn = runs / name+ turn.mkdir(parents=True)+ context = {"session_id": session} if session else {}+ (turn / "req.json").write_text(+ json.dumps({"prompt": "compare", "context": context}), encoding="utf-8"+ )+ for sub in subs:+ _write_backtest(turn / sub, **(kwargs if sub == "a" else {}))+ return turn+++ @pytest.fixture()+ def runs(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:+ monkeypatch.setenv("VIBE_TRADING_HOME", str(tmp_path))+ monkeypatch.delenv("VIBE_TRADING_ALLOWED_RUN_ROOTS", raising=False)+ (tmp_path / "runs").mkdir()+ return tmp_path / "runs"+++ def _list(current: Path | None = None, **kwargs) -> dict:+ return json.loads(+ list_backtest_runs(current_run_dir=str(current) if current else None, **kwargs)+ )+++ def test_previous_turn_is_found_and_readable_without_rerun(runs: Path) -> None:+ first = _turn(+ runs, "20261026_230021_aaaaaa", rejections={"ZZZ": {"execution_blocked": 15}}+ )+ current = _turn(runs, "20261026_231424_bbbbbb", subs=())++ result = _list(current)++ run = result["runs"][0]+ assert [r["run_dir"] for r in result["runs"]] == [str(first)] and result[+ "next_offset"+ ] is None+ by_name = {b["name"]: b for b in run["backtests"]}+ assert set(by_name) == {"a", "b"}+ assert by_name["a"]["run_dir"] == str(first / "a")+ assert by_name["a"]["metrics"] == {"total_return": 0.1, "sharpe": 1.25}+ assert by_name["a"]["unfilled_plan_rejections_by_symbol"] == {+ "ZZZ": {"execution_blocked": 15}+ }+ assert "unfilled_plan_rejections_by_symbol" not in by_name["b"]+ # The returned run_dir is what read_run_artifact takes.+ assert json.loads(read_run_artifact(by_name["a"]["run_dir"], "metrics"))[+ "rows"+ ] == [[0.1, 1.25]]+++ def test_archived_copy_at_the_turn_root_is_not_listed_twice(runs: Path) -> None:+ turn = _turn(runs, "20261026_100000_aaaaaa", subs=("a",))+ (turn / "run_card.json").write_text(+ (turn / "a" / "run_card.json").read_text(encoding="utf-8"), encoding="utf-8"+ )+ assert {b["name"] for b in _list(scope="all")["runs"][0]["backtests"]} == {".", "a"}+ (turn / ".archived_backtest.json").write_text("{}", encoding="utf-8")+ assert {b["name"] for b in _list(scope="all")["runs"][0]["backtests"]} == {"a"}+++ def test_session_scope_excludes_other_sessions_and_the_active_run(runs: Path) -> None:+ mine = _turn(runs, "20261026_100000_aaaaaa")+ other = _turn(runs, "20261026_110000_bbbbbb", session="sessionbbbb2")+ current = _turn(runs, "20261026_120000_cccccc", subs=())++ assert [r["run_id"] for r in _list(current)["runs"]] == [mine.name]+ assert [r["run_id"] for r in _list(current, scope="all")["runs"]] == [+ other.name,+ mine.name,+ ]+ assert (- len(_list(None)["runs"]) == 3- ) # no active run and no session: nothing is excluded++ len(_list(None)["runs"]) == 2++ ) # No session filter, but a turn with no backtest is not a backtest run.+++ def _walk(**kwargs) -> tuple[list[str], list[int]]:+ seen, sizes, offset = [], [], 0+ while offset is not None:+ page = _list(scope="all", offset=offset, **kwargs)+ assert page["runs"], "a page must make progress"+ seen += [r["run_id"] for r in page["runs"]]+ sizes.append(len(page["runs"]))+ offset = page["next_offset"]+ return seen, sizes+++ def test_pagination_never_skips_or_repeats_a_run(runs: Path, monkeypatch) -> None:+ names = [f"20261026_{hour:02d}0000_{hour:06d}" for hour in range(1, 8)]+ for name in names:+ _turn(runs, name, subs=("a",))+ expected = sorted(names, reverse=True)++ seen, sizes = _walk(limit=3) # cut by the page limit+ assert seen == expected and sizes == [3, 3, 1]++ monkeypatch.setattr(+ backtest_runs_tool, "TOOL_RESULT_LIMIT", 2500+ ) # cut by the response budget instead+ seen, sizes = _walk(limit=20)+ assert seen == expected and len(sizes) > 1 and max(sizes) < len(names)++ monkeypatch.setattr(+ backtest_runs_tool, "TOOL_RESULT_LIMIT", 1+ ) # one oversized run per page still advances+ seen, sizes = _walk(limit=20)+ assert seen == expected and sizes == [1] * len(names)+++ def test_pagination_with_the_session_filter_counts_only_matching_runs(+ runs: Path, monkeypatch+ ) -> None:+ mine = [f"20261026_{h:02d}0000_{h:06d}" for h in (1, 3, 5, 7)]+ for h in range(1, 9):+ name = f"20261026_{h:02d}0000_{h:06d}"+ _turn(+ runs, name, session=SESSION if name in mine else "sessionother", subs=("a",)+ )+ current = _turn(runs, "20261026_235900_ffffff", subs=())+ monkeypatch.setattr(backtest_runs_tool, "TOOL_RESULT_LIMIT", 2500)+ seen, offset = [], 0+ while offset is not None:+ page = _list(current, offset=offset)+ seen += [r["run_id"] for r in page["runs"]]+ offset = page["next_offset"]+ assert seen == sorted(mine, reverse=True)+++ def test_run_id_filter_and_bad_arguments(runs: Path) -> None:+ target = _turn(runs, "20261026_110000_bbbbbb")+ _turn(runs, "20261026_100000_aaaaaa")+ assert [r["run_id"] for r in _list(scope="all", run_id=target.name)["runs"]] == [+ target.name+ ]+ for bad in ("does_not_exist", "../etc", "a/b", "..", "", "x\x00y"):+ assert _list(scope="all", run_id=bad)["status"] == "error"+ assert _list(scope="everything")["status"] == "error"+ assert _list(scope="all", limit="many")["status"] == "error"+ assert _list(scope="all", limit=10_000)["status"] == "ok"+++ def test_symlinks_and_unsafe_files_are_never_followed(+ runs: Path, tmp_path_factory+ ) -> None:+ foreign = _turn(tmp_path_factory.mktemp("outside"), "20261026_090000_evil00")+ (runs / "20261026_090000_evil00").symlink_to(foreign, target_is_directory=True)+ turn = _turn(runs, "20261026_100000_aaaaaa", subs=("ok", "broken", "huge"))+ (turn / "linked").symlink_to(foreign / "a", target_is_directory=True)+ (turn / "stolen").mkdir()+ (turn / "stolen" / "run_card.json").symlink_to(foreign / "a" / "run_card.json")+ (turn / "broken" / "run_card.json").write_text("{not json", encoding="utf-8")+ (turn / "huge" / "run_card.json").write_text(+ " " * (backtest_runs_tool._MAX_JSON_BYTES + 1), encoding="utf-8"+ )++ listed = _list(scope="all")["runs"]++ assert [r["run_id"] for r in listed] == [turn.name]+ assert [b["name"] for b in listed[0]["backtests"]] == ["ok"]+++ def test_run_outside_allowed_roots_is_skipped(runs: Path, monkeypatch) -> None:+ _turn(runs, "20261026_100000_aaaaaa")+ monkeypatch.setattr(+ backtest_runs_tool,+ "safe_run_dir",+ lambda p: (_ for _ in ()).throw(ValueError("outside")),+ )+ assert _list(scope="all")["runs"] == []+++ def test_tool_contract(runs: Path) -> None:+ tool = BacktestRunsTool()+ assert (+ tool.name == "list_backtest_runs"+ and tool.is_readonly+ and tool.parameters["required"] == []+ )+ _turn(runs, "20261026_100000_aaaaaa")+ current = _turn(runs, "20261026_110000_bbbbbb", subs=())+ assert len(json.loads(tool.execute(run_dir=str(current)))["runs"]) == 1++++++def test_a_large_rejection_map_cannot_overflow_the_first_page(runs: Path) -> None:++ _turn(runs, "20261026_100000_aaaaaa", rejections={f"SYMBOL{i}": {"blocked": 15} for i in range(2000)})++ output = list_backtest_runs(scope="all")++ assert len(output) <= backtest_runs_tool.TOOL_RESULT_LIMIT++ entry = json.loads(output)["runs"][0]["backtests"][0]++ assert entry["unfilled_plan_rejections_by_symbol_omitted"] is Truediff --cc agent/tests/test_grounding_prior_run_reuse.pyindex 000000000,45ce05d32..d94fb3ed6mode 000000,100644..100644--- a/agent/tests/test_grounding_prior_run_reuse.py+++ b/agent/tests/test_grounding_prior_run_reuse.py@@@ -1,0 -1,179 +1,209 @@@+ """Figures read back from a backtest an earlier turn finished are grounded; forged or edited ones are not.++ Every value is synthetic.+ """++ from __future__ import annotations++ import json+ from pathlib import Path++ import pytest++ from src.agent.grounding import GroundingLedger+ from src.tools.run_artifact_tool import read_run_artifact+ from tests.test_grounding_backtest_outputs import EW, RP, _declared, _write_backtest++ PRIOR = "20261026_230021_46_a7d6f2"+ CURRENT = "20261026_231424_12_64857f"+ REF = f"{PRIOR}/rp"+++ @pytest.fixture()+ def reuse(tmp_path: Path, monkeypatch: pytest.MonkeyPatch):+ """A finished earlier turn with two backtests, and a ledger for the next turn."""+ monkeypatch.setenv("VIBE_TRADING_HOME", str(tmp_path))+ monkeypatch.delenv("VIBE_TRADING_ALLOWED_RUN_ROOTS", raising=False)+ prior = tmp_path / "runs" / PRIOR+ prior.mkdir(parents=True)+ (prior / "req.json").write_text(+ json.dumps({"prompt": "first", "context": {}}), encoding="utf-8"+ )+ _write_backtest(prior / "rp", RP)+ _write_backtest(+ prior / "ew", EW, weights=[("2024-01-02", 0.333333, 0.333333, 0.333334)]+ )+ current = tmp_path / "runs" / CURRENT+ current.mkdir()+ return (+ GroundingLedger(+ run_dir=current, user_message="profundizá el backtest anterior"+ ),+ prior,+ )+++ def _read(+ ledger: GroundingLedger, run_dir: Path, artifact: str, call_id: str, **kwargs+ ) -> dict:+ result = read_run_artifact(str(run_dir), artifact, **kwargs)+ ledger.ingest_tool_result(+ tool_name="read_run_artifact",+ arguments={"run_dir": str(run_dir), "artifact": artifact},+ result=result,+ call_id=call_id,+ success=True,+ )+ return json.loads(result)+++ def _observed(ledger: GroundingLedger, call_id: str) -> bool:+ return any(r.call_id == call_id and r.tool == "backtest" for r in ledger._evidence)+++ @pytest.mark.parametrize(+ "ref", [REF, f"{REF}/artifacts/metrics.csv", f"backtest::{REF}"]+ )+ def test_metrics_of_a_prior_backtest_ground_only_its_own_values(+ reuse, ref: str+ ) -> None:+ ledger, prior = reuse+ _read(ledger, prior / "rp", "metrics", "read-rp")+ _read(ledger, prior / "ew", "metrics", "read-ew")++ own = ledger.validate_final_answer(+ _declared("Sortino 1.133.", f"1.133 | observed | Sortino | {ref}")+ )+ invented = ledger.validate_final_answer(+ _declared("Sortino 1.190.", f"1.190 | observed | Sortino | {ref}")+ )+ other_run = ledger.validate_final_answer(+ _declared("Sortino 1.212.", f"1.212 | observed | Sortino | {ref}")+ )++ assert own.valid, own.issues+ assert not invented.valid and not other_run.valid+++ def test_unread_prior_run_grounds_nothing(reuse) -> None:+ ledger, _ = reuse+ assert not ledger.validate_final_answer(+ _declared("Sortino 1.133.", f"1.133 | observed | Sortino | {REF}")+ ).valid+++ def test_rows_read_from_a_prior_table_ground_only_what_was_shown(reuse) -> None:+ ledger, prior = reuse+ page = _read(+ ledger,+ prior / "rp",+ "target_positions",+ "read-w",+ max_rows=2,+ columns=["timestamp", "000001.SZ"],+ )+ assert page["returned_rows"] == 2+ table = f"{REF}/artifacts/target_positions.csv"+ shown = _declared("Peso inicial 38.97%.", f"38.97% | observed | peso | {table}")+ unseen = _declared("Peso 29.79%.", f"29.79% | observed | peso | {table}")+ assert ledger.validate_final_answer(shown).valid+ assert not ledger.validate_final_answer(unseen).valid+++ def test_edited_metrics_or_card_are_refused(reuse) -> None:+ ledger, prior = reuse+ metrics = prior / "rp" / "artifacts" / "metrics.csv"+ metrics.write_text(+ metrics.read_text(encoding="utf-8").replace("1.1329", "9.9999"),+ encoding="utf-8",+ )+ _read(ledger, prior / "rp", "metrics", "read-edited")+ assert not _observed(ledger, "read-edited")+ assert not ledger.validate_final_answer(+ _declared("Sortino 9.9999.", f"9.9999 | observed | Sortino | {REF}")+ ).valid++++def test_extra_unmanifested_metric_cannot_enter_prior_run_evidence(reuse) -> None:++ ledger, prior = reuse++ metrics = prior / "rp" / "artifacts" / "metrics.csv"++ lines = metrics.read_text().splitlines()++ metrics.write_text(lines[0] + ",invented_metric\n" + lines[1] + ",7.654321\n")++ _read(ledger, prior / "rp", "metrics", "extra-metric")++ assert not _observed(ledger, "extra-metric")++ assert not ledger.validate_final_answer(++ _declared("Metric 7.654321.", f"7.654321 | observed | metric | {REF}")++ ).valid++++++def test_identically_named_runs_in_other_allowed_roots_keep_distinct_scopes(reuse, tmp_path, monkeypatch):++ ledger, prior = reuse++ other = tmp_path / "other" / PRIOR / "rp"++ _write_backtest(other, EW)++ monkeypatch.setenv("VIBE_TRADING_ALLOWED_RUN_ROOTS", str(tmp_path / "other"))++ _read(ledger, prior / "rp", "metrics", "original-root")++ _read(ledger, other, "metrics", "other-root")++ assert ledger.validate_final_answer(++ _declared("Sortino 1.133.", f"1.133 | observed | Sortino | {REF}")++ ).valid++ assert not ledger.validate_final_answer(++ _declared("Sortino 1.212.", f"1.212 | observed | Sortino | {REF}")++ ).valid++ assert ledger.validate_final_answer(++ _declared("Sortino 1.212.", f"1.212 | observed | Sortino | {other}")++ ).valid+++++ def test_table_changed_or_unlisted_in_the_manifest_is_not_registered(reuse) -> None:+ ledger, prior = reuse+ changed = prior / "rp" / "artifacts" / "target_positions.csv"+ changed.write_text("timestamp,000001.SZ\n2024-01-02,0.4567\n", encoding="utf-8")+ card = json.loads((prior / "ew" / "run_card.json").read_text(encoding="utf-8"))+ card["artifacts"] = [+ e for e in card["artifacts"] if e["path"] != "artifacts/target_positions.csv"+ ]+ (prior / "ew" / "run_card.json").write_text(json.dumps(card), encoding="utf-8")+ _read(ledger, prior / "rp", "target_positions", "read-changed")+ _read(ledger, prior / "ew", "target_positions", "read-unlisted")+ assert not any("target_positions" in path for path in ledger._engine_tables)+++ def test_run_without_card_symlinked_card_or_outside_the_roots_is_not_adopted(+ reuse, tmp_path_factory: pytest.TempPathFactory+ ) -> None:+ ledger, prior = reuse+ (prior / "ew" / "run_card.json").unlink()+ foreign = tmp_path_factory.mktemp("foreign")+ _write_backtest(foreign / "rp", RP)+ (prior / "rp" / "run_card.json").unlink()+ (prior / "rp" / "run_card.json").symlink_to(foreign / "rp" / "run_card.json")+ _read(ledger, prior / "ew", "metrics", "no-card")+ _read(ledger, prior / "rp", "metrics", "linked-card")+ ledger._adopt_prior_backtest(str(foreign / "rp"), "outside-roots")+ assert not any(+ _observed(ledger, call) for call in ("no-card", "linked-card", "outside-roots")+ )+++ @pytest.mark.parametrize(+ "target",+ ["run_card.json", "artifacts/metrics.csv", "artifacts/target_positions.csv"],+ )+ @pytest.mark.parametrize("tool", ["write_file", "edit_file"])+ def test_a_file_the_model_wrote_in_the_prior_run_is_never_evidence(+ reuse, tool: str, target: str+ ) -> None:+ ledger, prior = reuse+ path = prior / "rp" / target+ ledger.ingest_tool_result(+ tool_name=tool,+ arguments={"path": str(path)},+ result=json.dumps({"status": "ok", "path": str(path)}),+ call_id="model-wrote",+ success=True,+ )+ _read(ledger, prior / "rp", "target_positions", "read-after-write")+ if target.endswith("target_positions.csv"):+ assert str(path.resolve()) not in ledger._engine_tables+ else:+ assert not _observed(ledger, "read-after-write")