Skip to content

"Vibe-Trading: Your Personal Trading Agent"

Merge PR #2074: reuse prior backtests with manifest-verified and distinct evidence

Haozhe Wu0a3c27f2ea64
diff --cc CHANGELOG.md
index 0d831572f,39f3f7229..2ad4f94a5
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@@ -5,29 -5,21 +5,42 @@@ This project adheres to [Semantic Versi
## [Unreleased]
+ ### Added
+
+ - **`list_backtest_runs` finds the backtests earlier turns and sessions already
+ ran**, so a follow-up turn reads their artifacts with `read_run_artifact`
+ instead of re-running them. It lists runs from `req.json` and `run_card.json`
+ (paged, current session by default, also over MCP) with each backtest's
+ absolute `run_dir`. Assistant replies in the history now carry
+ `[prev_run: <id>]` from the message metadata, and the grounding ledger adopts
+ a prior run read through `read_run_artifact` (run card and `metrics.csv` must
+ agree, tables must match the run card manifest). The session filter is not an
+ access control: runs have no owner. A run forged by the model in an earlier
+ turn is not detected.
+
### Fixed
+- **Forex symbol detection** no longer depends on AKShare's module layout:
+ `akshare.forex` was deleted in 1.20.0 and the symbol map moved to
+ `akshare.fx.cons_forex`, so the single hard-coded import turned every FX pair
+ into an A-share lookup. Both layouts are now probed, and a wholly missing map
+ degrades to "not a forex pair" instead of silently matching nothing.
+- **Bare US ETF and fund symbols** such as `QQQ` and `SPY` now resolve to
+ `QQQ.US` / market `us` instead of a suffix-less `global` candidate. The fetch
+ layer already routed bare US tickers as US equities, so the resolver was the
+ only place that disagreed: the identity lock then rejected the follow-up call
+ ("consumer symbol/venue differs from the locked resolver identity"). ETFs and
+ mutual funds on non-US venues keep their native form (#2084).
+- **`etf_holdings` accepts the `.US` market suffix** on US symbols, so the
+ identity locked as `QQQ.US` still reaches the SEC series/class ticker `QQQ`
+ instead of being reported as an unknown share class.
+- **JSONP-wrapped provider bodies** decode instead of raising: the throttled
+ JSON GET helper unwraps a `callback(...)` envelope when `response.json()`
+ cannot parse the body, so A-share news search keeps working when Eastmoney
+ switches to JSONP (#2081).
+- **Degraded Eastmoney news indexes** now raise an actionable error instead of
+ reading as an empty article list: the Guba user index (`passportWeb`) can no
+ longer masquerade as "this stock has no news" (#2081).
- **Global futures routing and pricing**: digit-bearing contracts such as
`M2K=F` and `6EZ2026.CME`, including `local:` inputs, reach the global
futures engine with their full product code. OJ and ZR use quote-unit
diff --cc README.md
index b026f5c05,ea9630d63..032c8a1ad
--- a/README.md
+++ b/README.md
@@@ -2364,7 -2314,7 +2364,7 @@@ Vibe-Trading
│ │ ├── memory/ # Cross-session persistent memory
│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)
│ │ │
- │ │ ├── tools/ # 116 auto-discovered agent tools
-│ │ ├── tools/ # 114 auto-discovered agent tools
++│ │ ├── tools/ # 117 auto-discovered agent tools
│ │ │ ├── backtest_tool.py # run backtests
│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
diff --cc README_ar.md
index 0b79c90ac,c83f03e5c..b32303c94
--- a/README_ar.md
+++ b/README_ar.md
@@@ -1757,7 -1751,7 +1757,7 @@@ Vibe-Trading
│ │ ├── memory/ # Cross-session persistent memory
│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)
│ │ │
- │ │ ├── tools/ # 116 auto-discovered agent tools
-│ │ ├── tools/ # 114 auto-discovered agent tools
++│ │ ├── tools/ # 117 auto-discovered agent tools
│ │ │ ├── backtest_tool.py # run backtests
│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
diff --cc README_es.md
index 085552b58,6a07caa11..d04b3153e
--- a/README_es.md
+++ b/README_es.md
@@@ -1845,7 -1839,7 +1845,7 @@@ Vibe-Trading
│ │ ├── memory/ # Memoria persistente entre sesiones
│ │ │ └── persistent.py # memoria basada en archivos (~/.vibe-trading/memory/)
│ │ │
- │ │ ├── tools/ # 116 herramientas de agente autodescubiertas
-│ │ ├── tools/ # 114 herramientas de agente autodescubiertas
++│ │ ├── tools/ # 117 herramientas de agente autodescubiertas
│ │ │ ├── backtest_tool.py # ejecuta backtests
│ │ │ ├── remember_tool.py # memoria entre sesiones (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # CRUD de skills (save/patch/delete/file)
diff --cc README_id.md
index e9c543c18,64109efe1..f7abe551a
--- a/README_id.md
+++ b/README_id.md
@@@ -2090,7 -2084,7 +2090,7 @@@ Vibe-Trading
│ │ ├── memory/ # Cross-session persistent memory
│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)
│ │ │
- │ │ ├── tools/ # 116 auto-discovered agent tools
-│ │ ├── tools/ # 114 auto-discovered agent tools
++│ │ ├── tools/ # 117 auto-discovered agent tools
│ │ │ ├── backtest_tool.py # run backtests
│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
diff --cc README_ja.md
index 4a5307d1b,7428958d8..13a3d1987
--- a/README_ja.md
+++ b/README_ja.md
@@@ -1765,7 -1759,7 +1765,7 @@@ Vibe-Trading
│ │ ├── memory/ # クロスセッション永続メモリ
│ │ │ └── persistent.py # ファイルベースメモリ (~/.vibe-trading/memory/)
│ │ │
- │ │ ├── tools/ # 116 個の自動検出エージェントツール
-│ │ ├── tools/ # 114 個の自動検出エージェントツール
++│ │ ├── tools/ # 117 個の自動検出エージェントツール
│ │ │ ├── backtest_tool.py # バックテスト実行
│ │ │ ├── remember_tool.py # クロスセッションメモリ (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
diff --cc README_ko.md
index 8b80fe810,111cba108..42d11b4b2
--- a/README_ko.md
+++ b/README_ko.md
@@@ -1762,7 -1756,7 +1762,7 @@@ Vibe-Trading
│ │ ├── memory/ # Cross-session persistent memory
│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)
│ │ │
- │ │ ├── tools/ # 116 auto-discovered agent tools
-│ │ ├── tools/ # 114 auto-discovered agent tools
++│ │ ├── tools/ # 117 auto-discovered agent tools
│ │ │ ├── backtest_tool.py # run backtests
│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
diff --cc README_zh.md
index 6c6f6f7eb,3f1a61f1b..d226b48ab
--- a/README_zh.md
+++ b/README_zh.md
@@@ -1769,7 -1754,7 +1769,7 @@@ Vibe-Trading
│ │ ├── memory/ # 跨 session 持久记忆
│ │ │ └── persistent.py # 基于文件的记忆(~/.vibe-trading/memory/)
│ │ │
- │ │ ├── tools/ # 116 个自动发现的 agent 工具
-│ │ ├── tools/ # 114 个自动发现的 agent 工具
++│ │ ├── tools/ # 117 个自动发现的 agent 工具
│ │ │ ├── backtest_tool.py # 运行回测
│ │ │ ├── remember_tool.py # 跨 session 记忆(save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD(save/patch/delete/file)
diff --cc agent/src/agent/grounding/evidence.py
index ca3223780,c60af1740..0d22297f8
--- a/agent/src/agent/grounding/evidence.py
+++ b/agent/src/agent/grounding/evidence.py
@@@ -239,6 -237,19 +240,19 @@@ def _relative_posix(path: Path, root: P
return "" if relative == "." else relative
+ def _artifact_ref(path: Path, root: Path) -> str:
+ """``path`` as a ref names it: below the active run, else below its parent.
+
+ A backtest an earlier turn left is a sibling of the active run, so
+ ``<run id>/<backtest>/artifacts/metrics.csv`` names it exactly.
+ """
+ relative = _relative_posix(path, root)
- if relative.startswith("/"):
++ if Path(relative).is_absolute():
+ with contextlib.suppress(OSError, ValueError):
+ return Path(path).resolve().relative_to(root.parent).as_posix()
+ return relative
+
+
def _file_sha256(path: Path) -> str | None:
"""Hex SHA-256 of a file's bytes, or None when it cannot be read."""
try:
@@@ -934,9 -949,9 +952,9 @@@ class _EvidenceMixin
or self._is_model_written(path)
):
continue
-- if name not in _MANIFEST_EXEMPT and manifest.get(name) != _file_sha256(path):
++ if (name not in _MANIFEST_EXEMPT or (verify_tables and name != "run_card.json")) and manifest.get(name) != _file_sha256(path):
continue
- artifact = _relative_posix(path, root)
+ artifact = _artifact_ref(path, root)
self._evidence = [record for record in self._evidence if record.artifact != artifact]
for field_name, value in _summary_scalars(path):
if room <= 0:
@@@ -968,9 -983,47 +986,49 @@@
):
continue
digest = _file_sha256(path)
- if digest is not None:
+ if digest is not None and (
+ not verify_tables or manifest.get(f"artifacts/{path.name}") == digest
+ ):
self._engine_tables[str(path.resolve())] = (digest, scope)
+ def _adopt_prior_backtest(self, raw_dir: Any, call_id: str) -> None:
+ """Ground figures read from a backtest an earlier turn finished.
+
+ The ledger only watches backtests launched under the active run. A run
+ reused through ``list_backtest_runs`` + ``read_run_artifact`` is adopted
+ only if it sits in an allowed run root, its run card and ``metrics.csv``
+ agree, no file is a symlink or one the model wrote, and each table it
+ registers is listed in the run card manifest with the same hash. It is
+ named in a figure ref as ``<run id>/<backtest>`` (or ``backtest::<run id>/<backtest>``).
+ """
+ if not raw_dir:
+ return
+ from src.tools.path_utils import safe_run_dir
+
+ try:
+ directory = safe_run_dir(str(raw_dir))
+ except ValueError:
+ return
+ card, metrics = directory / "run_card.json", directory / "artifacts" / "metrics.csv"
+ if directory.is_relative_to(self.run_dir.resolve()) or any(
+ path.is_symlink() or self._is_model_written(path) for path in (card, metrics)
+ ):
+ return
++ if _run_card_manifest(directory).get("artifacts/metrics.csv") != _file_sha256(metrics):
++ return
+ recorded = dict(_summary_scalars(card))
+ shared = [(name, value) for name, value in _summary_scalars(metrics) if f"metrics.{name}" in recorded]
+ if not shared or any(
+ not math.isclose(value, recorded[f"metrics.{name}"], rel_tol=1e-9, abs_tol=1e-12)
+ for name, value in shared
+ ):
+ return
- scope = directory.name if (directory / "req.json").is_file() else f"{directory.parent.name}/{directory.name}"
++ scope = _artifact_ref(directory, self.run_dir.resolve())
+ self._record_metrics_file(metrics, call_id)
+ self._backtest_scopes[call_id] = scope
+ self._scope_latest[scope] = call_id
+ self._record_backtest_outputs(directory, call_id, scope, verify_tables=True)
+
def _note_model_write(
self, tool_name: str, arguments: Mapping[str, Any], payload: dict[str, Any] | None
) -> None:
diff --cc agent/src/agent/grounding/policies.py
index a2ad289cb,4ef7edd4e..1fd306b6e
--- a/agent/src/agent/grounding/policies.py
+++ b/agent/src/agent/grounding/policies.py
@@@ -2611,10 -2579,14 +2611,17 @@@ def _records_at
path = Path(token)
if not path.is_absolute():
path = root / path
+ normalized = Path(os.path.normpath(path))
try:
- relative = Path(os.path.normpath(path)).relative_to(root).as_posix()
+ relative = normalized.relative_to(root).as_posix()
except ValueError:
- return []
+ try: # a backtest an earlier turn left sits beside the active run
+ relative = normalized.relative_to(root.parent).as_posix()
+ except ValueError:
- return []
++ # A prior run in another approved run root keeps an absolute
++ # artifact ref. This matches observed records only; it authorizes
++ # no filesystem read and admits no new evidence.
++ relative = normalized.as_posix()
if relative in ("", "."):
return []
diff --cc agent/src/tools/backtest_runs_tool.py
index 000000000,f941d1b3f..8d283d5d0
mode 000000,100644..100644
--- a/agent/src/tools/backtest_runs_tool.py
+++ b/agent/src/tools/backtest_runs_tool.py
@@@ -1,0 -1,233 +1,245 @@@
+ """Backtest run discovery: list earlier runs so their artifacts can be reused.
+
+ Each user turn gets its own run directory, so a follow-up turn cannot reach the
+ artifacts an earlier turn produced unless something names them. This tool lists
+ runs from the files the runtime already writes (``req.json``, ``run_card.json``;
+ no new index) and returns the absolute ``run_dir`` of every backtest for
+ ``read_run_artifact``.
+
+ The session filter is a convenience, not an access control: runs have no owner,
+ ``scope="all"`` lists what ``GET /runs`` already shows, and ``read_run_artifact``
+ and ``session_search`` reach the same data. Directories are confined to the
+ allowed run roots and symlinks are never followed.
+ """
+
+ from __future__ import annotations
+
+ import json
+ from pathlib import Path
+ from typing import Any
+
+ from src.agent.tools import BaseTool
+ from src.config.limits import TOOL_RESULT_LIMIT
+ from src.config.paths import get_runs_dir
+ from src.tools.path_utils import safe_run_dir, safe_run_id
+
+ _SCOPES = ("session", "all")
+ _MAX_LIMIT = 20
+ _MAX_SCAN = 5000
+ _MAX_BACKTESTS = 20
+ _MAX_JSON_BYTES = 8 * 1024 * 1024
+ _PROMPT_CHARS = 160
+ _METRIC_KEYS = ("total_return", "sharpe", "max_drawdown", "unfilled_plan_rejections")
+
+
+ def _is_plain_dir(path: Path) -> bool:
+ return path.is_dir() and not path.is_symlink()
+
+
+ def _as_dict(value: Any) -> dict[str, Any]:
+ return value if isinstance(value, dict) else {}
+
+
+ def _read_json(path: Path, run_root: Path) -> dict[str, Any]:
+ """A JSON object from a regular file inside ``run_root``, else ``{}``."""
+ try:
+ if path.is_symlink() or not path.resolve().is_relative_to(run_root):
+ return {}
+ if path.stat().st_size > _MAX_JSON_BYTES:
+ return {}
+ return _as_dict(json.loads(path.read_text(encoding="utf-8")))
+ except (OSError, ValueError):
+ return {}
+
+
+ def _session_of(run_root: Path) -> str | None:
+ """Session that created a run, from the ``req.json`` the loop wrote."""
+ return _as_dict(_read_json(run_root / "req.json", run_root).get("context")).get(
+ "session_id"
+ )
+
+
+ def _backtests_of(run_root: Path) -> list[dict[str, Any]]:
+ """Backtests a turn left: the run itself and its direct sub-directories.
+
+ A turn root that holds an archive manifest is a copy of one of its
+ sub-directories, so it is not listed twice.
+ """
+ dirs = [] if (run_root / ".archived_backtest.json").exists() else [run_root]
+ try:
+ dirs += sorted(d for d in run_root.iterdir() if _is_plain_dir(d))
+ except OSError:
+ pass
+ found = []
+ for directory in dirs:
+ card = _read_json(directory / "run_card.json", run_root)
+ if not card:
+ continue
+ metrics = _as_dict(card.get("metrics"))
+ config = _as_dict(card.get("backtest"))
+ entry = {
+ "name": directory.name if directory != run_root else ".",
+ "run_dir": str(directory),
+ "start_date": config.get("start_date"),
+ "end_date": config.get("end_date"),
+ "metrics": {key: metrics[key] for key in _METRIC_KEYS if key in metrics},
+ }
+ rejected = _as_dict(card.get("structured_metrics")).get(
+ "unfilled_plan_rejections_by_symbol"
+ )
+ if rejected:
- entry["unfilled_plan_rejections_by_symbol"] = rejected
++ if len(json.dumps(rejected)) <= TOOL_RESULT_LIMIT // 8:
++ entry["unfilled_plan_rejections_by_symbol"] = rejected
++ else:
++ entry["unfilled_plan_rejections_by_symbol_omitted"] = True
+ found.append(entry)
+ return found[:_MAX_BACKTESTS]
+
+
+ def _run_summary(run_root: Path) -> dict[str, Any]:
+ prompt = " ".join(
+ str(_read_json(run_root / "req.json", run_root).get("prompt") or "").split()
+ )
- return {
++ result = {
+ "run_id": run_root.name,
+ "run_dir": str(run_root),
+ "prompt": prompt[:_PROMPT_CHARS],
+ "backtests": _backtests_of(run_root),
+ }
++ omitted = 0
++ while len(result["backtests"]) > 1 and len(json.dumps(result)) > TOOL_RESULT_LIMIT // 2:
++ result["backtests"].pop()
++ omitted += 1
++ if omitted:
++ result["omitted_backtests"] = omitted
++ return result
+
+
+ def _candidates(runs_root: Path):
+ """Allowed run directories, newest first."""
+ try:
+ names = sorted(
+ (d.name for d in runs_root.iterdir() if _is_plain_dir(d)), reverse=True
+ )
+ except OSError:
+ return
+ for name in names[:_MAX_SCAN]:
+ try:
+ yield safe_run_dir(str(runs_root / name))
+ except ValueError:
+ continue
+
+
+ def list_backtest_runs(
+ *,
+ current_run_dir: str | None = None,
+ scope: str = "session",
+ run_id: str | None = None,
+ limit: int = 5,
+ offset: int = 0,
+ ) -> str:
+ """List earlier runs, newest first, as a JSON envelope.
+
+ Args:
+ current_run_dir: The active turn's run directory (injected by the
+ loop). It defines the session for ``scope="session"`` and is never
+ listed.
+ scope: ``session`` for the current session's runs, ``all`` for every run.
+ run_id: Only this bare run directory name (validated by ``safe_run_id``).
+ limit: Runs per page, 1..20.
+ offset: Runs to skip (the previous page's ``next_offset``).
+
+ Returns:
+ JSON with ``runs`` (each with its ``run_dir`` and ``backtests``) and
+ ``next_offset`` (``null`` on the last page).
+ """
+ if scope not in _SCOPES:
+ return json.dumps(
+ {"status": "error", "error": f"scope must be one of {list(_SCOPES)}"}
+ )
+ try:
+ limit = min(max(int(limit), 1), _MAX_LIMIT)
+ offset = max(int(offset), 0)
+ runs = (
+ iter([safe_run_id(run_id)])
+ if run_id is not None
+ else _candidates(get_runs_dir())
+ )
+ except (TypeError, ValueError) as exc:
+ return json.dumps({"status": "error", "error": str(exc)})
+ current = Path(current_run_dir).resolve() if current_run_dir else None
+ session = _session_of(current) if current and scope == "session" else None
+
+ page: list[dict[str, Any]] = []
+ skipped, more = 0, False
+ for run_root in runs:
+ if run_root == current or (session and _session_of(run_root) != session):
+ continue
++ entry = _run_summary(run_root)
++ if not entry["backtests"]:
++ continue
+ if skipped < offset:
+ skipped += 1
+ continue
- entry = _run_summary(run_root)
+ if page and (
+ len(page) == limit
+ or len(json.dumps([*page, entry])) > TOOL_RESULT_LIMIT // 2
+ ):
+ more = True
+ break
+ page.append(entry)
+ return json.dumps(
+ {
+ "status": "ok",
+ "runs": page,
+ "next_offset": offset + len(page) if more else None,
+ },
+ ensure_ascii=False,
+ default=str,
+ )
+
+
+ class BacktestRunsTool(BaseTool):
+ """Discover the backtests earlier turns and sessions already ran."""
+
+ name = "list_backtest_runs"
+ description = (
+ "List earlier backtest runs (newest first, paged) with the absolute run_dir of each "
+ "backtest, its dates and headline metrics. Use it in a follow-up turn, or for a "
+ "'[prev_run: ...]' marker, then read artifacts with read_run_artifact(run_dir=...) "
+ "instead of re-running the backtest. Defaults to the current session; scope='all' lists every run."
+ )
+ parameters = {
+ "type": "object",
+ "properties": {
+ "scope": {
+ "type": "string",
+ "enum": list(_SCOPES),
+ "description": "session (default) or all",
+ },
+ "run_id": {
+ "type": "string",
+ "description": "Only this run id (e.g. from a [prev_run: ...] marker)",
+ },
+ "limit": {
+ "type": "integer",
+ "description": f"Runs per page [1, {_MAX_LIMIT}] (default 5)",
+ },
+ "offset": {
+ "type": "integer",
+ "description": "Runs to skip; use next_offset from the previous page",
+ },
+ },
+ "required": [],
+ }
+ repeatable = True
+ is_readonly = True
+
+ def execute(self, **kwargs: Any) -> str:
+ return list_backtest_runs(
+ current_run_dir=kwargs.get("run_dir"),
+ scope=kwargs.get("scope") or "session",
+ run_id=kwargs.get("run_id"),
+ limit=kwargs.get("limit", 5),
+ offset=kwargs.get("offset", 0),
+ )
diff --cc agent/tests/test_backtest_runs_tool.py
index 000000000,108dfbb4d..384f23708
mode 000000,100644..100644
--- a/agent/tests/test_backtest_runs_tool.py
+++ b/agent/tests/test_backtest_runs_tool.py
@@@ -1,0 -1,223 +1,231 @@@
+ """``list_backtest_runs``: find earlier runs and feed ``read_run_artifact``. All values are synthetic."""
+
+ from __future__ import annotations
+
+ import hashlib
+ import json
+ from pathlib import Path
+
+ import pytest
+
+ from src.tools import backtest_runs_tool
+ from src.tools.backtest_runs_tool import BacktestRunsTool, list_backtest_runs
+ from src.tools.run_artifact_tool import read_run_artifact
+
+ SESSION = "sessionaaaa1"
+
+
+ def _write_backtest(directory: Path, rejections: dict | None = None) -> None:
+ artifacts = directory / "artifacts"
+ artifacts.mkdir(parents=True)
+ (artifacts / "metrics.csv").write_text(
+ "total_return,sharpe\n0.1,1.25\n", encoding="utf-8"
+ )
+ card = {
+ "backtest": {"start_date": "2024-01-01", "end_date": "2024-12-31"},
+ "metrics": {"total_return": 0.1, "sharpe": 1.25},
+ "structured_metrics": {"unfilled_plan_rejections_by_symbol": rejections or {}},
+ "artifacts": [
+ {
+ "path": "artifacts/metrics.csv",
+ "sha256": hashlib.sha256(
+ (artifacts / "metrics.csv").read_bytes()
+ ).hexdigest(),
+ }
+ ],
+ }
+ (directory / "run_card.json").write_text(json.dumps(card), encoding="utf-8")
+
+
+ def _turn(
+ runs: Path, name: str, session: str | None = SESSION, subs=("a", "b"), **kwargs
+ ) -> Path:
+ turn = runs / name
+ turn.mkdir(parents=True)
+ context = {"session_id": session} if session else {}
+ (turn / "req.json").write_text(
+ json.dumps({"prompt": "compare", "context": context}), encoding="utf-8"
+ )
+ for sub in subs:
+ _write_backtest(turn / sub, **(kwargs if sub == "a" else {}))
+ return turn
+
+
+ @pytest.fixture()
+ def runs(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
+ monkeypatch.setenv("VIBE_TRADING_HOME", str(tmp_path))
+ monkeypatch.delenv("VIBE_TRADING_ALLOWED_RUN_ROOTS", raising=False)
+ (tmp_path / "runs").mkdir()
+ return tmp_path / "runs"
+
+
+ def _list(current: Path | None = None, **kwargs) -> dict:
+ return json.loads(
+ list_backtest_runs(current_run_dir=str(current) if current else None, **kwargs)
+ )
+
+
+ def test_previous_turn_is_found_and_readable_without_rerun(runs: Path) -> None:
+ first = _turn(
+ runs, "20261026_230021_aaaaaa", rejections={"ZZZ": {"execution_blocked": 15}}
+ )
+ current = _turn(runs, "20261026_231424_bbbbbb", subs=())
+
+ result = _list(current)
+
+ run = result["runs"][0]
+ assert [r["run_dir"] for r in result["runs"]] == [str(first)] and result[
+ "next_offset"
+ ] is None
+ by_name = {b["name"]: b for b in run["backtests"]}
+ assert set(by_name) == {"a", "b"}
+ assert by_name["a"]["run_dir"] == str(first / "a")
+ assert by_name["a"]["metrics"] == {"total_return": 0.1, "sharpe": 1.25}
+ assert by_name["a"]["unfilled_plan_rejections_by_symbol"] == {
+ "ZZZ": {"execution_blocked": 15}
+ }
+ assert "unfilled_plan_rejections_by_symbol" not in by_name["b"]
+ # The returned run_dir is what read_run_artifact takes.
+ assert json.loads(read_run_artifact(by_name["a"]["run_dir"], "metrics"))[
+ "rows"
+ ] == [[0.1, 1.25]]
+
+
+ def test_archived_copy_at_the_turn_root_is_not_listed_twice(runs: Path) -> None:
+ turn = _turn(runs, "20261026_100000_aaaaaa", subs=("a",))
+ (turn / "run_card.json").write_text(
+ (turn / "a" / "run_card.json").read_text(encoding="utf-8"), encoding="utf-8"
+ )
+ assert {b["name"] for b in _list(scope="all")["runs"][0]["backtests"]} == {".", "a"}
+ (turn / ".archived_backtest.json").write_text("{}", encoding="utf-8")
+ assert {b["name"] for b in _list(scope="all")["runs"][0]["backtests"]} == {"a"}
+
+
+ def test_session_scope_excludes_other_sessions_and_the_active_run(runs: Path) -> None:
+ mine = _turn(runs, "20261026_100000_aaaaaa")
+ other = _turn(runs, "20261026_110000_bbbbbb", session="sessionbbbb2")
+ current = _turn(runs, "20261026_120000_cccccc", subs=())
+
+ assert [r["run_id"] for r in _list(current)["runs"]] == [mine.name]
+ assert [r["run_id"] for r in _list(current, scope="all")["runs"]] == [
+ other.name,
+ mine.name,
+ ]
+ assert (
- len(_list(None)["runs"]) == 3
- ) # no active run and no session: nothing is excluded
++ len(_list(None)["runs"]) == 2
++ ) # No session filter, but a turn with no backtest is not a backtest run.
+
+
+ def _walk(**kwargs) -> tuple[list[str], list[int]]:
+ seen, sizes, offset = [], [], 0
+ while offset is not None:
+ page = _list(scope="all", offset=offset, **kwargs)
+ assert page["runs"], "a page must make progress"
+ seen += [r["run_id"] for r in page["runs"]]
+ sizes.append(len(page["runs"]))
+ offset = page["next_offset"]
+ return seen, sizes
+
+
+ def test_pagination_never_skips_or_repeats_a_run(runs: Path, monkeypatch) -> None:
+ names = [f"20261026_{hour:02d}0000_{hour:06d}" for hour in range(1, 8)]
+ for name in names:
+ _turn(runs, name, subs=("a",))
+ expected = sorted(names, reverse=True)
+
+ seen, sizes = _walk(limit=3) # cut by the page limit
+ assert seen == expected and sizes == [3, 3, 1]
+
+ monkeypatch.setattr(
+ backtest_runs_tool, "TOOL_RESULT_LIMIT", 2500
+ ) # cut by the response budget instead
+ seen, sizes = _walk(limit=20)
+ assert seen == expected and len(sizes) > 1 and max(sizes) < len(names)
+
+ monkeypatch.setattr(
+ backtest_runs_tool, "TOOL_RESULT_LIMIT", 1
+ ) # one oversized run per page still advances
+ seen, sizes = _walk(limit=20)
+ assert seen == expected and sizes == [1] * len(names)
+
+
+ def test_pagination_with_the_session_filter_counts_only_matching_runs(
+ runs: Path, monkeypatch
+ ) -> None:
+ mine = [f"20261026_{h:02d}0000_{h:06d}" for h in (1, 3, 5, 7)]
+ for h in range(1, 9):
+ name = f"20261026_{h:02d}0000_{h:06d}"
+ _turn(
+ runs, name, session=SESSION if name in mine else "sessionother", subs=("a",)
+ )
+ current = _turn(runs, "20261026_235900_ffffff", subs=())
+ monkeypatch.setattr(backtest_runs_tool, "TOOL_RESULT_LIMIT", 2500)
+ seen, offset = [], 0
+ while offset is not None:
+ page = _list(current, offset=offset)
+ seen += [r["run_id"] for r in page["runs"]]
+ offset = page["next_offset"]
+ assert seen == sorted(mine, reverse=True)
+
+
+ def test_run_id_filter_and_bad_arguments(runs: Path) -> None:
+ target = _turn(runs, "20261026_110000_bbbbbb")
+ _turn(runs, "20261026_100000_aaaaaa")
+ assert [r["run_id"] for r in _list(scope="all", run_id=target.name)["runs"]] == [
+ target.name
+ ]
+ for bad in ("does_not_exist", "../etc", "a/b", "..", "", "x\x00y"):
+ assert _list(scope="all", run_id=bad)["status"] == "error"
+ assert _list(scope="everything")["status"] == "error"
+ assert _list(scope="all", limit="many")["status"] == "error"
+ assert _list(scope="all", limit=10_000)["status"] == "ok"
+
+
+ def test_symlinks_and_unsafe_files_are_never_followed(
+ runs: Path, tmp_path_factory
+ ) -> None:
+ foreign = _turn(tmp_path_factory.mktemp("outside"), "20261026_090000_evil00")
+ (runs / "20261026_090000_evil00").symlink_to(foreign, target_is_directory=True)
+ turn = _turn(runs, "20261026_100000_aaaaaa", subs=("ok", "broken", "huge"))
+ (turn / "linked").symlink_to(foreign / "a", target_is_directory=True)
+ (turn / "stolen").mkdir()
+ (turn / "stolen" / "run_card.json").symlink_to(foreign / "a" / "run_card.json")
+ (turn / "broken" / "run_card.json").write_text("{not json", encoding="utf-8")
+ (turn / "huge" / "run_card.json").write_text(
+ " " * (backtest_runs_tool._MAX_JSON_BYTES + 1), encoding="utf-8"
+ )
+
+ listed = _list(scope="all")["runs"]
+
+ assert [r["run_id"] for r in listed] == [turn.name]
+ assert [b["name"] for b in listed[0]["backtests"]] == ["ok"]
+
+
+ def test_run_outside_allowed_roots_is_skipped(runs: Path, monkeypatch) -> None:
+ _turn(runs, "20261026_100000_aaaaaa")
+ monkeypatch.setattr(
+ backtest_runs_tool,
+ "safe_run_dir",
+ lambda p: (_ for _ in ()).throw(ValueError("outside")),
+ )
+ assert _list(scope="all")["runs"] == []
+
+
+ def test_tool_contract(runs: Path) -> None:
+ tool = BacktestRunsTool()
+ assert (
+ tool.name == "list_backtest_runs"
+ and tool.is_readonly
+ and tool.parameters["required"] == []
+ )
+ _turn(runs, "20261026_100000_aaaaaa")
+ current = _turn(runs, "20261026_110000_bbbbbb", subs=())
+ assert len(json.loads(tool.execute(run_dir=str(current)))["runs"]) == 1
++
++
++def test_a_large_rejection_map_cannot_overflow_the_first_page(runs: Path) -> None:
++ _turn(runs, "20261026_100000_aaaaaa", rejections={f"SYMBOL{i}": {"blocked": 15} for i in range(2000)})
++ output = list_backtest_runs(scope="all")
++ assert len(output) <= backtest_runs_tool.TOOL_RESULT_LIMIT
++ entry = json.loads(output)["runs"][0]["backtests"][0]
++ assert entry["unfilled_plan_rejections_by_symbol_omitted"] is True
diff --cc agent/tests/test_grounding_prior_run_reuse.py
index 000000000,45ce05d32..d94fb3ed6
mode 000000,100644..100644
--- a/agent/tests/test_grounding_prior_run_reuse.py
+++ b/agent/tests/test_grounding_prior_run_reuse.py
@@@ -1,0 -1,179 +1,209 @@@
+ """Figures read back from a backtest an earlier turn finished are grounded; forged or edited ones are not.
+
+ Every value is synthetic.
+ """
+
+ from __future__ import annotations
+
+ import json
+ from pathlib import Path
+
+ import pytest
+
+ from src.agent.grounding import GroundingLedger
+ from src.tools.run_artifact_tool import read_run_artifact
+ from tests.test_grounding_backtest_outputs import EW, RP, _declared, _write_backtest
+
+ PRIOR = "20261026_230021_46_a7d6f2"
+ CURRENT = "20261026_231424_12_64857f"
+ REF = f"{PRIOR}/rp"
+
+
+ @pytest.fixture()
+ def reuse(tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
+ """A finished earlier turn with two backtests, and a ledger for the next turn."""
+ monkeypatch.setenv("VIBE_TRADING_HOME", str(tmp_path))
+ monkeypatch.delenv("VIBE_TRADING_ALLOWED_RUN_ROOTS", raising=False)
+ prior = tmp_path / "runs" / PRIOR
+ prior.mkdir(parents=True)
+ (prior / "req.json").write_text(
+ json.dumps({"prompt": "first", "context": {}}), encoding="utf-8"
+ )
+ _write_backtest(prior / "rp", RP)
+ _write_backtest(
+ prior / "ew", EW, weights=[("2024-01-02", 0.333333, 0.333333, 0.333334)]
+ )
+ current = tmp_path / "runs" / CURRENT
+ current.mkdir()
+ return (
+ GroundingLedger(
+ run_dir=current, user_message="profundizá el backtest anterior"
+ ),
+ prior,
+ )
+
+
+ def _read(
+ ledger: GroundingLedger, run_dir: Path, artifact: str, call_id: str, **kwargs
+ ) -> dict:
+ result = read_run_artifact(str(run_dir), artifact, **kwargs)
+ ledger.ingest_tool_result(
+ tool_name="read_run_artifact",
+ arguments={"run_dir": str(run_dir), "artifact": artifact},
+ result=result,
+ call_id=call_id,
+ success=True,
+ )
+ return json.loads(result)
+
+
+ def _observed(ledger: GroundingLedger, call_id: str) -> bool:
+ return any(r.call_id == call_id and r.tool == "backtest" for r in ledger._evidence)
+
+
+ @pytest.mark.parametrize(
+ "ref", [REF, f"{REF}/artifacts/metrics.csv", f"backtest::{REF}"]
+ )
+ def test_metrics_of_a_prior_backtest_ground_only_its_own_values(
+ reuse, ref: str
+ ) -> None:
+ ledger, prior = reuse
+ _read(ledger, prior / "rp", "metrics", "read-rp")
+ _read(ledger, prior / "ew", "metrics", "read-ew")
+
+ own = ledger.validate_final_answer(
+ _declared("Sortino 1.133.", f"1.133 | observed | Sortino | {ref}")
+ )
+ invented = ledger.validate_final_answer(
+ _declared("Sortino 1.190.", f"1.190 | observed | Sortino | {ref}")
+ )
+ other_run = ledger.validate_final_answer(
+ _declared("Sortino 1.212.", f"1.212 | observed | Sortino | {ref}")
+ )
+
+ assert own.valid, own.issues
+ assert not invented.valid and not other_run.valid
+
+
+ def test_unread_prior_run_grounds_nothing(reuse) -> None:
+ ledger, _ = reuse
+ assert not ledger.validate_final_answer(
+ _declared("Sortino 1.133.", f"1.133 | observed | Sortino | {REF}")
+ ).valid
+
+
+ def test_rows_read_from_a_prior_table_ground_only_what_was_shown(reuse) -> None:
+ ledger, prior = reuse
+ page = _read(
+ ledger,
+ prior / "rp",
+ "target_positions",
+ "read-w",
+ max_rows=2,
+ columns=["timestamp", "000001.SZ"],
+ )
+ assert page["returned_rows"] == 2
+ table = f"{REF}/artifacts/target_positions.csv"
+ shown = _declared("Peso inicial 38.97%.", f"38.97% | observed | peso | {table}")
+ unseen = _declared("Peso 29.79%.", f"29.79% | observed | peso | {table}")
+ assert ledger.validate_final_answer(shown).valid
+ assert not ledger.validate_final_answer(unseen).valid
+
+
+ def test_edited_metrics_or_card_are_refused(reuse) -> None:
+ ledger, prior = reuse
+ metrics = prior / "rp" / "artifacts" / "metrics.csv"
+ metrics.write_text(
+ metrics.read_text(encoding="utf-8").replace("1.1329", "9.9999"),
+ encoding="utf-8",
+ )
+ _read(ledger, prior / "rp", "metrics", "read-edited")
+ assert not _observed(ledger, "read-edited")
+ assert not ledger.validate_final_answer(
+ _declared("Sortino 9.9999.", f"9.9999 | observed | Sortino | {REF}")
+ ).valid
+
+
++def test_extra_unmanifested_metric_cannot_enter_prior_run_evidence(reuse) -> None:
++ ledger, prior = reuse
++ metrics = prior / "rp" / "artifacts" / "metrics.csv"
++ lines = metrics.read_text().splitlines()
++ metrics.write_text(lines[0] + ",invented_metric\n" + lines[1] + ",7.654321\n")
++ _read(ledger, prior / "rp", "metrics", "extra-metric")
++ assert not _observed(ledger, "extra-metric")
++ assert not ledger.validate_final_answer(
++ _declared("Metric 7.654321.", f"7.654321 | observed | metric | {REF}")
++ ).valid
++
++
++def test_identically_named_runs_in_other_allowed_roots_keep_distinct_scopes(reuse, tmp_path, monkeypatch):
++ ledger, prior = reuse
++ other = tmp_path / "other" / PRIOR / "rp"
++ _write_backtest(other, EW)
++ monkeypatch.setenv("VIBE_TRADING_ALLOWED_RUN_ROOTS", str(tmp_path / "other"))
++ _read(ledger, prior / "rp", "metrics", "original-root")
++ _read(ledger, other, "metrics", "other-root")
++ assert ledger.validate_final_answer(
++ _declared("Sortino 1.133.", f"1.133 | observed | Sortino | {REF}")
++ ).valid
++ assert not ledger.validate_final_answer(
++ _declared("Sortino 1.212.", f"1.212 | observed | Sortino | {REF}")
++ ).valid
++ assert ledger.validate_final_answer(
++ _declared("Sortino 1.212.", f"1.212 | observed | Sortino | {other}")
++ ).valid
++
++
+ def test_table_changed_or_unlisted_in_the_manifest_is_not_registered(reuse) -> None:
+ ledger, prior = reuse
+ changed = prior / "rp" / "artifacts" / "target_positions.csv"
+ changed.write_text("timestamp,000001.SZ\n2024-01-02,0.4567\n", encoding="utf-8")
+ card = json.loads((prior / "ew" / "run_card.json").read_text(encoding="utf-8"))
+ card["artifacts"] = [
+ e for e in card["artifacts"] if e["path"] != "artifacts/target_positions.csv"
+ ]
+ (prior / "ew" / "run_card.json").write_text(json.dumps(card), encoding="utf-8")
+ _read(ledger, prior / "rp", "target_positions", "read-changed")
+ _read(ledger, prior / "ew", "target_positions", "read-unlisted")
+ assert not any("target_positions" in path for path in ledger._engine_tables)
+
+
+ def test_run_without_card_symlinked_card_or_outside_the_roots_is_not_adopted(
+ reuse, tmp_path_factory: pytest.TempPathFactory
+ ) -> None:
+ ledger, prior = reuse
+ (prior / "ew" / "run_card.json").unlink()
+ foreign = tmp_path_factory.mktemp("foreign")
+ _write_backtest(foreign / "rp", RP)
+ (prior / "rp" / "run_card.json").unlink()
+ (prior / "rp" / "run_card.json").symlink_to(foreign / "rp" / "run_card.json")
+ _read(ledger, prior / "ew", "metrics", "no-card")
+ _read(ledger, prior / "rp", "metrics", "linked-card")
+ ledger._adopt_prior_backtest(str(foreign / "rp"), "outside-roots")
+ assert not any(
+ _observed(ledger, call) for call in ("no-card", "linked-card", "outside-roots")
+ )
+
+
+ @pytest.mark.parametrize(
+ "target",
+ ["run_card.json", "artifacts/metrics.csv", "artifacts/target_positions.csv"],
+ )
+ @pytest.mark.parametrize("tool", ["write_file", "edit_file"])
+ def test_a_file_the_model_wrote_in_the_prior_run_is_never_evidence(
+ reuse, tool: str, target: str
+ ) -> None:
+ ledger, prior = reuse
+ path = prior / "rp" / target
+ ledger.ingest_tool_result(
+ tool_name=tool,
+ arguments={"path": str(path)},
+ result=json.dumps({"status": "ok", "path": str(path)}),
+ call_id="model-wrote",
+ success=True,
+ )
+ _read(ledger, prior / "rp", "target_positions", "read-after-write")
+ if target.endswith("target_positions.csv"):
+ assert str(path.resolve()) not in ledger._engine_tables
+ else:
+ assert not _observed(ledger, "read-after-write")