Skip to content

"Vibe-Trading: Your Personal Trading Agent"

perf(runtime): ship lean schema loading and preserve research observations

Haozhe Wu9c8a869fa943
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8006f8ed9..e1340a9c5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,24 @@ This project adheres to [Semantic Versioning](https://semver.org/).
## [Unreleased]
+### Changed
+
+- Reduce agent request overhead with on-demand tool schemas while keeping the
+ complete tool catalog and existing execution permissions. Ordinary backtests
+ no longer request extra attribution automatically; chat titles reuse the prompt.
+- Preserve oversized tool results in readable run artifacts and stop Swarm from
+ clearing new observations before the model sees them. Context pressure uses
+ the model window and includes tool schemas.
+
+### Fixed
+
+- Include compaction, cancelled responses and failed streams in usage accounting;
+ stop further work when a research goal exhausts its token budget. Forward the
+ configured output cap to both DeepSeek adapters.
+- Redact credentials before tool results reach model context or replay caches,
+ and reject incomplete compatible-provider streams instead of releasing a
+ truncated answer or tool call.
+
## [0.1.17] — 2026-10-21
Rolls up 638 commits / 228 merged pull requests since 0.1.16, plus 41 pull
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 97afce16a..4532b101b 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -21,6 +21,38 @@ agent run or its SSE stream. Factor submissions are the exception to the write
retry rule: the Web UI reuses a request ID, and the server returns the existing
job for an identical submission instead of starting another worker.
+## Runtime cost and evidence handling
+
+The agent exposes tool schemas on demand for registries of at least 16 tools.
+The system prompt retains the complete catalog; `load_tools` only reveals exact
+schemas and never executes a domain tool or changes its permissions. Schema
+state belongs to one question and contributes to measured context usage.
+`VIBE_TRADING_TOOL_LOADING=eager` restores full schemas for provider compatibility.
+The discovery policy is packaged in `src/agent/tool_loading.json`.
+
+A direct lookup uses its tool contract. A normal backtest reports results,
+validation warnings and limitations; additional attribution or optimization
+follows the user's request. Existing numerical grounding and live order gates
+remain in place. Web chat uses the prompt-derived session title without an
+additional automatic model request.
+
+Large tool results are redacted and saved under the current run's `tool-results/`
+when its tools can read the files. The bounded receipt names the original file
+and readable parts; it does not imply that a sampled upstream response is complete.
+Read failures retain explicit truncation rather than advertising an unavailable
+file. Swarm uses the same model-window/context ceiling as the main agent and
+saves old observations under pressure only after a successful request has seen
+them. It stops if the next request still cannot fit. These files are run artifacts,
+not a persistent market-data cache.
+
+`llm_usage.json` includes provider-reported compaction, cancelled-response and
+failed-stream usage. Auxiliary calls carry a `purpose`; `call_index` distinguishes
+several calls in one iteration. Unavailable usage is emitted explicitly rather
+than reported as zero. A goal's token budget stops further model/tool work once
+reported usage reaches the limit; the last in-flight response can exceed it.
+`LANGCHAIN_MAX_TOKENS` remains an optional output cap and is forwarded by both
+DeepSeek adapters without changing the model or reasoning settings.
+
## Specialized analysis
In Factor Research, select one factor to evaluate only that factor, or several
diff --git a/MANIFEST.in b/MANIFEST.in
index e0d6b8a4b..c08d513db 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -14,6 +14,9 @@ include docker-compose.yml
# Developer entry-point template
include agent/.env.example
+# Runtime schema-loading policy
+recursive-include agent/src/agent *.json
+
# Alpha Zoo factor metadata and per-zoo LICENSE / NOTICE files
recursive-include agent/src/factors *.yaml *.md
diff --git a/agent/.env.example b/agent/.env.example
index 52de6ec32..748f6ec92 100644
--- a/agent/.env.example
+++ b/agent/.env.example
@@ -340,6 +340,12 @@ TUSHARE_TOKEN=your-tushare-token
# ============================================================================
# Agent Tuning (AgentTuningConfig, optional, defaults are sensible)
# ============================================================================
+# Large agent registries expose exact schemas on demand; every tool remains
+# discoverable. Set eager for providers that need a fixed full schema list.
+# VIBE_TRADING_TOOL_LOADING=lazy
+# LANGCHAIN_MAX_TOKENS is an optional output cap, forwarded to both DeepSeek
+# adapters. Unset preserves the provider default; this is not a context limit.
+# LANGCHAIN_MAX_TOKENS=8192
# SWARM_WORKER_TIMEOUT=300
# SWARM_WORKER_MAX_ITER=50
# SWARM_MAX_WORKERS=4
@@ -354,6 +360,9 @@ TUSHARE_TOKEN=your-tushare-token
# provider's context-length error teaches the real limit mid-run).
# Override the window only if the catalog is wrong for your model:
# VIBE_TRADING_CONTEXT_WINDOW=272000
+# Main agent and Swarm share the model window and context ceiling. Swarm saves
+# older results as readable workspace files only under pressure, after they
+# have reached the model at least once. It stops if the request still cannot fit.
# Cost ceiling on how large a run's prompt may grow before it is compacted,
# whatever the window (real tokens):
# VIBE_TRADING_CONTEXT_MAX_TOKENS=200000
diff --git a/agent/src/agent/context.py b/agent/src/agent/context.py
index 3b18250a0..f1f00abd6 100644
--- a/agent/src/agent/context.py
+++ b/agent/src/agent/context.py
@@ -87,8 +87,8 @@ Decide which workflow to use based on the request:
1. `load_skill("strategy-generate")` — read the SignalEngine contract
2. `write_file("config.json", ...)` — source, codes, dates, parameters. If the strategy is expected to produce ≥10 trades, include `"validation": {{"monte_carlo": {{"n_simulations": 1000}}}}` in config.json for Monte Carlo testing
3. `write_file("code/signal_engine.py", ...)` — SignalEngine class
-4. Syntax check → `backtest(run_dir=...)` → `read_file("artifacts/metrics.csv")`
-5. Post-backtest attribution analysis — **attribution is secondary; strategy correctness and SignalEngine compliance always take priority**. Run each layer whose condition is met. If a layer is skipped, append one line: `ℹ️ Layer N (name): skipped — [reason]`. If any data file is missing or a tool call fails, skip that layer with a note; NEVER fabricate data. Present all results as markdown pipe tables.
+4. Syntax check → `backtest(run_dir=...)`. Use its structured summary, metrics and validation directly. Read `artifacts/metrics.csv` or `read_run_artifact` only for evidence needed by the request that the summary does not contain. Fix implementation errors; a valid negative result is still a completed backtest. Change strategy parameters or run another candidate only when the user requests optimization or comparison.
+5. Post-backtest attribution — run the following layers only when the user requests attribution, robustness analysis or a comprehensive research report. A request to run a backtest ends with its results, validation warnings and limitations; it does not automatically request more analysis. Within a requested attribution report, run each applicable layer below and explain missing inputs without fabricating data. Strategy correctness and SignalEngine compliance always take priority.
**Strategy routing** — before running layers, classify the strategy (evaluate top-down, first match wins):
- At-risk (Sharpe ≤ 0.5 or MaxDD ≥ 40%): run Layer 1 + Layer 4, focus on failure diagnosis
@@ -139,7 +139,7 @@ Decide which workflow to use based on the request:
- Do NOT use swarm unless the user specifically asks for team-based or committee analysis.
**Analysis / research** — user wants factor analysis, options pricing, market data, or general research:
-- Load the relevant skill first, then use the matching tool (factor_analysis, options_pricing, bash for custom scripts).
+- For a direct lookup or calculation whose contract is in the tool schema, use the matching tool directly. Load a skill when the requested methodology or implementation requires its guidance. Fetch only the evidence needed to answer the request; independent reads can share a tool-call turn, except resolver-dependent reads below.
**Document / web** — user provides a PDF or URL:
- `read_document(path=...)` for PDFs, `read_url(url=...)` for web pages.
@@ -262,11 +262,11 @@ Decide which workflow to use based on the request:
observed evidence, that quotes a figure no tool produced, or that attaches
figures to a symbol this session never handled, is rejected and handed back to
you with the specific conflict — it never reaches the user.
-- Load the relevant skill BEFORE starting any task, after any required identity resolution. Skills contain the exact API contracts and examples.
+- Load skills for workflows that require their contracts or methodology (including strategy generation and Shadow Account above). A simple lookup does not require a preliminary skill read. Required identity resolution still comes first.
- Ask the user if critical info is missing (assets, dates, strategy type). Never guess.
-- Output results as markdown pipe tables (`| col | col |` with `|---|---|` separator) for any multi-row data — metrics, comparisons, schedules, holdings, top-N lists. Renderers upgrade these to native tables. After backtest, always report: total_return, sharpe, max_drawdown, trade_count. Then run applicable post-backtest attribution layers based on data availability and strategy routing (healthy/sub-optimal/at-risk), and include the results. Attribution is secondary — strategy correctness always comes first.
+- Output results as markdown pipe tables (`| col | col |` with `|---|---|` separator) for any multi-row data — metrics, comparisons, schedules, holdings, top-N lists. Renderers upgrade these to native tables. After backtest, always report: total_return, sharpe, max_drawdown, trade_count, plus validation warnings and limitations. Additional attribution follows the user's requested scope as described above.
- Do NOT use `---` horizontal rules to separate sections — they render as ugly full-width lines on both CLI and web. Use `##` / `###` markdown headings instead.
-- All file paths are relative to run_dir (auto-injected).
+- All file paths are relative to run_dir (auto-injected). A tool result with `_result_file` has been saved in full in this workspace: read its listed parts or load the complete JSON in a calculation script instead of refetching it. The preview is incomplete; the source's own sampling and coverage limits still apply.
- Respond in the same language the user used.
- You have persistent cross-session memory (`remember` tool). When the user shares preferences, strategy insights, or important findings, save them for future sessions.
- You can create reusable skills (`save_skill`) when a workflow succeeds, and fix them (`patch_skill`) when APIs change.
@@ -343,11 +343,22 @@ class ContextBuilder:
except Exception: # noqa: BLE001
routing = ""
+ from src.agent.tool_loading import DeferredToolRegistry
+
+ tool_discovery = "Available tools and their full argument contracts are supplied with this request's tool definitions."
+ if isinstance(self.registry, DeferredToolRegistry):
+ tool_discovery = (
+ "The complete available-tool catalog follows. Only loaded tools have full schemas "
+ "in this request. Use load_tools(names=[...]) to load the exact names you need, "
+ "preferably together, then call them directly. Loading does not execute a tool. "
+ "No catalog entry is hidden by search or ranking.\n\n" + self.registry.catalog()
+ )
+
return _SYSTEM_PROMPT.format(
+ tool_descriptions=tool_discovery,
tool_count=len(self.registry._tools),
skill_count=len(self.skills_loader.skills),
data_source_count=self._count_data_sources(),
- tool_descriptions=self._format_tool_descriptions(),
skill_descriptions=self.skills_loader.get_descriptions(),
memory_section=memory_section,
strategy_discovery_routing=routing,
@@ -423,33 +434,6 @@ class ContextBuilder:
messages.append({"role": "user", "content": enriched})
return messages
- # Prose tool section is deliberately compact: the full tool schema (name,
- # description, parameter JSON schema) is sent to the model via the API
- # ``tools`` array (``ToolRegistry.get_definitions()`` -> ``bind_tools``), so
- # repeating every description + parameter here as prose roughly doubled the
- # per-call input tokens for zero model benefit. Keep one short discovery
- # hint per tool; the authoritative spec arrives with every request. This is
- # a pure token-cost change: no tool is removed, and the grounding/identity
- # gate (which validates tool results, not the prompt) is untouched.
- _TOOL_PROSE_DESC_MAX = 80
-
- def _format_tool_descriptions(self) -> str:
- """Format a compact one-line tool list for the system prompt.
-
- Returns one line per registered tool: ``- <name>: <short hint>``. The
- full parameter schema is deliberately NOT repeated here — it is supplied
- through the API ``tools`` parameter on every call (see
- ``ToolRegistry.get_definitions``), so the model always has the exact
- spec even though the prose stays small.
- """
- lines = []
- for tool in self.registry._tools.values():
- desc = (tool.description or "").strip().replace("\n", " ")
- if len(desc) > self._TOOL_PROSE_DESC_MAX:
- desc = desc[: self._TOOL_PROSE_DESC_MAX].rstrip() + "…"
- lines.append(f"- {tool.name}: {desc}" if desc else f"- {tool.name}")
- return "\n".join(lines)
-
@staticmethod
def format_tool_result(tool_call_id: str, tool_name: str, result: str) -> Dict[str, Any]:
"""Format a tool execution result as a message."""
diff --git a/agent/src/agent/context_budget.py b/agent/src/agent/context_budget.py
index 16cb5de1f..22334d5ce 100644
--- a/agent/src/agent/context_budget.py
+++ b/agent/src/agent/context_budget.py
@@ -174,30 +174,37 @@ class ContextMeter:
self._estimate = estimator
self._anchor: Optional[tuple[int, int]] = None
self.static_tokens: Optional[int] = None
+ self._anchor_schema_tokens: Optional[int] = None
+ self._static_schema_tokens: Optional[int] = None
- def observe(self, input_tokens: Optional[int], messages: list) -> None:
+ def observe(self, input_tokens: Optional[int], messages: list, *, tool_schema_tokens: Optional[int] = None) -> None:
"""Anchor on the real input size of the request just answered.
Args:
input_tokens: Provider-reported prompt tokens of that request.
messages: Exactly the messages that request sent.
+ tool_schema_tokens: Estimate of the schemas sent with that request.
"""
if not isinstance(input_tokens, int) or isinstance(input_tokens, bool) or input_tokens <= 0:
return
self._anchor = (input_tokens, self._estimate(messages))
+ self._anchor_schema_tokens = tool_schema_tokens
if self.static_tokens is None:
# The first request is the static prompt plus the opening turn(s).
self.static_tokens = max(0, input_tokens - self._estimate(messages[1:]))
+ self._static_schema_tokens = tool_schema_tokens
def tokens(self, messages: list, tool_schema_tokens: int) -> int:
"""Real-token size the next request would have."""
if self._anchor is None:
return self._estimate(messages) + tool_schema_tokens
anchor_input, anchor_estimate = self._anchor
- return max(0, anchor_input + self._estimate(messages) - anchor_estimate)
+ schema_delta = 0 if self._anchor_schema_tokens is None else tool_schema_tokens - self._anchor_schema_tokens
+ return max(0, anchor_input + self._estimate(messages) - anchor_estimate + schema_delta)
def static(self, messages: list, tool_schema_tokens: int) -> int:
"""Static prompt size: measured once usage arrives, else estimated."""
if self.static_tokens is not None:
- return self.static_tokens
+ schema_delta = 0 if self._static_schema_tokens is None else tool_schema_tokens - self._static_schema_tokens
+ return max(0, self.static_tokens + schema_delta)
return self._estimate(messages[:1]) + tool_schema_tokens
diff --git a/agent/src/agent/llm_usage.py b/agent/src/agent/llm_usage.py
index 78286ea12..7cf7b4bb5 100644
--- a/agent/src/agent/llm_usage.py
+++ b/agent/src/agent/llm_usage.py
@@ -7,8 +7,8 @@ provider payload shapes, the run-scoped accumulator, and the atomic write of
``llm_usage.json`` beside a run. ``loop.py`` re-exports every moved name, so
existing imports keep working.
-Behavior is unchanged: every function body, docstring, and comment moved
-byte-for-byte.
+Auxiliary calls share the accumulator, with their purpose recorded separately.
+Reported reasoning tokens are retained as a subset of output tokens.
"""
from __future__ import annotations
@@ -63,6 +63,9 @@ def _normalize_llm_usage(usage: Any) -> dict[str, int] | None:
normalized["cache_creation_tokens"] = sum(
_coerce_usage_int(details.get(key)) for key in creation_keys
)
+ output_details = usage.get("output_token_details")
+ if isinstance(output_details, dict) and output_details.get("reasoning") is not None:
+ normalized["reasoning_tokens"] = _coerce_usage_int(output_details["reasoning"])
return normalized
@@ -90,6 +93,8 @@ def _record_llm_usage(
summary: dict[str, Any],
usage: Any,
iteration: int,
+ *,
+ purpose: str | None = None,
) -> dict[str, int] | None:
"""Accumulate and persist one provider-reported usage event."""
normalized = _normalize_llm_usage(usage)
@@ -101,10 +106,13 @@ def _record_llm_usage(
totals["output_tokens"] = int(totals.get("output_tokens") or 0) + normalized["output_tokens"]
totals["total_tokens"] = int(totals.get("total_tokens") or 0) + normalized["total_tokens"]
totals["calls"] = int(totals.get("calls") or 0) + 1
- for key in ("cache_read_tokens", "cache_creation_tokens"):
+ for key in ("cache_read_tokens", "cache_creation_tokens", "reasoning_tokens"):
if key in normalized:
totals[key] = int(totals.get(key) or 0) + normalized[key]
- summary.setdefault("per_iteration", []).append({"iter": iteration, **normalized})
+ entry = {"iter": iteration, **normalized}
+ if purpose is not None:
+ entry["purpose"] = purpose
+ summary.setdefault("per_iteration", []).append(entry)
summary["updated_at"] = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
try:
diff --git a/agent/src/agent/loop.py b/agent/src/agent/loop.py
index a463e9b13..8983fe10d 100644
--- a/agent/src/agent/loop.py
+++ b/agent/src/agent/loop.py
@@ -39,6 +39,7 @@ from src.agent.tuning import (
from src.agent.loop_grounding import validate_and_release
from src.agent.loop_streaming import TurnStream
from src.agent.context import ContextBuilder
+from src.agent.tool_loading import prepare_registry, request_definitions
from src.agent.grounding import GroundingLedger
from src.agent.grounding.release import MAX_GROUNDING_REVISIONS
from src.agent.memory import WorkspaceMemory
@@ -194,7 +195,7 @@ class AgentLoop(ToolCallLifecycle):
max_iterations: Maximum number of loop iterations.
persistent_memory: PersistentMemory for cross-session recall.
"""
- self.registry = registry
+ self.registry = prepare_registry(registry)
self.llm = llm
runtime_snapshot = getattr(llm, "runtime_snapshot", None)
if not isinstance(runtime_snapshot, LLMRuntimeSnapshot):
@@ -240,6 +241,7 @@ class AgentLoop(ToolCallLifecycle):
self._released_fallback_reason: str | None = None
self._written_files: set[str] = set()
self._stall_reason: str | None = None
+ self._budget_stop_reason: str | None = None
self._last_activity_wall: float = 0.0
self._run_done = threading.Event()
# Identical deterministic tool calls (e.g. financial_rigor calc with
@@ -265,6 +267,8 @@ class AgentLoop(ToolCallLifecycle):
self._learned_context_window: int | None = None
self._compacted_to_tokens: int | None = None
self._tool_schema_tokens = 0
+ self._llm_usage_summary = _new_llm_usage_summary(self.llm)
+ self._goal_usage_callback = None
def cancel(self) -> None:
"""Cancel the current loop.
@@ -378,6 +382,7 @@ class AgentLoop(ToolCallLifecycle):
self._released_fallback_reason = None
self._written_files = set()
self._stall_reason = None
+ self._budget_stop_reason = None
self._last_activity_wall = _time.time()
self._run_done = threading.Event()
self._called_identical = {}
@@ -390,10 +395,10 @@ class AgentLoop(ToolCallLifecycle):
self._context_meter = ContextMeter()
self._learned_context_window: int | None = None
self._compacted_to_tokens: int | None = None
+ self.registry = prepare_registry(self.registry)
# Sent with every request and never compacted, so they count against
# the window exactly like the system prompt.
- definitions = getattr(self.registry, "get_definitions", None)
- self._tool_schema_tokens = estimate_tokens(definitions()) if callable(definitions) else 0
+ self._tool_schema_tokens = estimate_tokens(request_definitions(self.registry))
run_started_wall = _time.time()
state_store = RunStateStore()
@@ -465,7 +470,52 @@ class AgentLoop(ToolCallLifecycle):
# keeps tools available; ordinary correction turns do not.
grounding_correction_text_only = False
pending_grounding_draft = ""
- llm_usage_summary = _new_llm_usage_summary(self.llm)
+ self._llm_usage_summary = _new_llm_usage_summary(self.llm)
+
+ def check_goal_token_budget(snapshot: dict[str, Any] | None) -> None:
+ goal = (snapshot or {}).get("goal") or {}
+ limit = goal.get("token_budget")
+ if limit is not None and int(goal.get("tokens_used") or 0) >= limit:
+ self._budget_stop_reason = (
+ "goal_token_budget_exhausted: the research goal's token budget "
+ "has been reached. Increase its budget or end the goal before continuing."
+ )
+
+ if active_goal_id and session_id:
+ from src.goal import GoalStore
+
+ goal_store = GoalStore()
+ check_goal_token_budget(goal_store.get_goal_snapshot(active_goal_id))
+
+ def account_goal_usage(usage_delta: dict[str, int] | None) -> None:
+ nonlocal goal_store, goal_turn_accounted
+ if not (active_goal_id and session_id):
+ return
+ token_delta = int(usage_delta.get("total_tokens") or 0) if usage_delta else 0
+ turn_delta = 0 if goal_turn_accounted else 1
+ if not (token_delta or turn_delta):
+ return
+ try:
+ if goal_store is None:
+ from src.goal import GoalStore
+
+ goal_store = GoalStore()
+ goal_store.account_usage(
+ session_id=session_id,
+ goal_id=active_goal_id,
+ expected_goal_id=active_goal_id,
+ token_delta=token_delta,
+ turn_delta=turn_delta,
+ )
+ goal_turn_accounted = True
+ snapshot = goal_store.get_goal_snapshot(active_goal_id)
+ check_goal_token_budget(snapshot)
+ if snapshot is not None:
+ self._emit("goal.updated", {"goal": snapshot["goal"], "snapshot": snapshot})
+ except Exception as exc: # noqa: BLE001
+ logger.debug("Goal usage accounting skipped: %s", exc)
+
+ self._goal_usage_callback = account_goal_usage
last_response_model: str | None = None
goal_continuations = 0
goal_last_progress: tuple[int, int] | None = None
@@ -488,6 +538,9 @@ class AgentLoop(ToolCallLifecycle):
stream_failure_streak = 0
try:
while iteration < self.max_iterations:
+ self._tool_schema_tokens = estimate_tokens(request_definitions(self.registry))
+ if self._budget_stop_reason is not None:
+ break
if self._cancel_event.is_set():
trace.write({"type": "cancelled", "iter": self._run_iteration + 1})
logger.info("AgentLoop cancelled by user")
@@ -527,6 +580,11 @@ class AgentLoop(ToolCallLifecycle):
else:
self._compact_to_budget(messages, run_dir, trace, iteration, current_iter)
+ # A compaction response also costs tokens. Its usage callback
+ # can exhaust the caller's budget before the next agent call.
+ if self._cancel_event.is_set() or self._budget_stop_reason is not None:
+ break
+
logger.info(f"ReAct iteration {iteration}/{self.max_iterations}")
# Inject wrap-up nudge when approaching iteration limit.
@@ -591,7 +649,7 @@ class AgentLoop(ToolCallLifecycle):
tool_defs = (
None
if is_last_iteration or correction_text_only
- else self.registry.get_definitions()
+ else request_definitions(self.registry)
)
if is_last_iteration:
trace.write({"type": "forced_text_only", "iter": current_iter})
@@ -606,17 +664,28 @@ class AgentLoop(ToolCallLifecycle):
_llm_timeout_s = _llm_timeout_seconds()
llm_timeout = _llm_timeout_s if _llm_timeout_s > 0 else None
+ def stream_request():
+ """Count usage from a failed request before any retry."""
+ try:
+ return self.llm.stream_chat(
+ messages,
+ tools=tool_defs,
+ on_text_chunk=stream.on_text,
+ on_reasoning_chunk=stream.on_reasoning,
+ on_waiting_chunk=stream.on_waiting,
+ timeout=llm_timeout,
+ idle_timeout_s=llm_timeout,
+ should_cancel=self._cancel_event.is_set,
+ )
+ except ProviderStreamError as error:
+ self._account_llm_usage(
+ run_dir, getattr(error, "usage_metadata", None), current_iter,
+ purpose="failed_stream",
+ )
+ raise
+
try:
- response = self.llm.stream_chat(
- messages,
- tools=tool_defs,
- on_text_chunk=stream.on_text,
- on_reasoning_chunk=stream.on_reasoning,
- on_waiting_chunk=stream.on_waiting,
- timeout=llm_timeout,
- idle_timeout_s=llm_timeout,
- should_cancel=self._cancel_event.is_set,
- )
+ response = stream_request()
except ProviderStreamError as exc:
# One retry for transient mid-stream failures (connection
# reset, relay hiccup) — mirrors the swarm worker policy.
@@ -667,28 +736,14 @@ class AgentLoop(ToolCallLifecycle):
# failure. A blocking sleep would make Stop take that long
# to be observed; the event returns the moment it is set.
self._cancel_event.wait(retry_delay_s)
- if self._cancel_event.is_set():
+ if self._cancel_event.is_set() or self._budget_stop_reason is not None:
break
- response = self.llm.stream_chat(
- messages,
- tools=tool_defs,
- on_text_chunk=stream.on_text,
- on_reasoning_chunk=stream.on_reasoning,
- on_waiting_chunk=stream.on_waiting,
- timeout=llm_timeout,
- idle_timeout_s=llm_timeout,
- should_cancel=self._cancel_event.is_set,
- )
+ response = stream_request()
else:
stream_failure_streak = 0
self._consume_readonly_replay_visibility(messages, trace, current_iter)
- # Cancelled mid-stream: discard this turn's partial response and
- # end the run now, without executing any of its tool calls.
- if self._cancel_event.is_set():
- break
-
# An LLM response arrived - real progress for the stall watchdog.
self._last_activity_wall = _time.time()
@@ -698,55 +753,15 @@ class AgentLoop(ToolCallLifecycle):
self._context_meter.observe(
usage.get("input_tokens") if isinstance(usage, dict) else None,
messages,
+ tool_schema_tokens=estimate_tokens(tool_defs) if tool_defs else 0,
)
if current_response_model:
last_response_model = current_response_model
- usage_delta = _record_llm_usage(
- run_dir,
- llm_usage_summary,
- usage,
- current_iter,
- )
- if usage_delta:
- self._emit(
- "llm_usage",
- {
- **usage_delta,
- "iter": current_iter,
- },
- )
- else:
- self._emit("llm_usage_unavailable", {"iter": current_iter})
+ self._account_llm_usage(run_dir, usage, current_iter)
# A caller's spending monitor can cancel synchronously on the
# usage event. Do not execute tools from that paid response.
- if self._cancel_event.is_set():
+ if self._cancel_event.is_set() or self._budget_stop_reason is not None:
break
- if active_goal_id and session_id:
- token_delta = int(usage_delta.get("total_tokens") or 0) if usage_delta else 0
- turn_delta = 0 if goal_turn_accounted else 1
- if token_delta or turn_delta:
- try:
- if goal_store is None:
- from src.goal import GoalStore
-
- goal_store = GoalStore()
- goal_store.account_usage(
- session_id=session_id,
- goal_id=active_goal_id,
- expected_goal_id=active_goal_id,
- token_delta=token_delta,
- turn_delta=turn_delta,
- )
- goal_turn_accounted = True
- snapshot = goal_store.get_goal_snapshot(active_goal_id)
- if snapshot is not None:
- self._emit(
- "goal.updated",
- {"goal": snapshot["goal"], "snapshot": snapshot},
- )
- except Exception as exc: # noqa: BLE001
- logger.debug("Goal usage accounting skipped: %s", exc)
-
thinking_text = "".join(stream.thinking_chunks)
if thinking_text:
trace.write_text_entry(
@@ -1123,6 +1138,7 @@ class AgentLoop(ToolCallLifecycle):
empty_response_provider=self._llm_runtime.provider,
empty_response_model=self._llm_runtime.configured_model or "(unset)",
max_iterations=self.max_iterations,
+ budget_reason=self._budget_stop_reason,
)
)
final_reason = outcome.reason
@@ -1405,6 +1421,33 @@ class AgentLoop(ToolCallLifecycle):
return unreadable_tools
+ def _account_llm_usage(
+ self, run_dir: Path, usage: Any, iteration: int, *, purpose: str | None = None,
+ ) -> None:
+ """Persist reported usage and notify spending monitors before more work.
+
+ Args:
+ run_dir: Active run's artifact directory.
+ usage: Provider metadata, or None when unavailable.
+ iteration: Parent agent iteration, also for auxiliary calls.
+ purpose: Auxiliary call kind; omitted for ordinary agent responses.
+ """
+ delta = _record_llm_usage(
+ run_dir, self._llm_usage_summary, usage, iteration, purpose=purpose,
+ )
+ if self._goal_usage_callback is not None:
+ self._goal_usage_callback(delta)
+ metadata = {"iter": iteration}
+ if purpose is not None:
+ metadata["purpose"] = purpose
+ if delta:
+ self._emit("llm_usage", {
+ **delta, **metadata,
+ "call_index": self._llm_usage_summary["totals"]["calls"],
+ })
+ else:
+ self._emit("llm_usage_unavailable", metadata)
+
# -- Context compression ---------------------------------------------------
def _auto_compact(
@@ -1431,7 +1474,8 @@ class AgentLoop(ToolCallLifecycle):
focus_topic: Optional topic to prioritize in the summary.
iteration: Current trace iteration.
"""
- del run_dir
+ if self._cancel_event.is_set() or self._budget_stop_reason is not None:
+ return
# Save full transcript before compressing next to the active trace.
transcript_path = trace.dir_path / f"transcript_{int(_time.time())}.jsonl"
with open(transcript_path, "w", encoding="utf-8") as f:
@@ -1485,6 +1529,8 @@ class AgentLoop(ToolCallLifecycle):
degraded_compact = False
_compact_timeout = _llm_timeout_seconds()
for conv_text in chunks:
+ if self._cancel_event.is_set() or self._budget_stop_reason is not None:
+ return
# Structured template while there is still nothing to update — that
# covers a fresh session's first chunk and the corner case where
# every fold so far returned empty content.
@@ -1535,6 +1581,7 @@ class AgentLoop(ToolCallLifecycle):
_compact_timeout,
)
degraded_compact = True
+ self._account_llm_usage(run_dir, None, iteration, purpose="compaction")
break
if _compact_error:
logger.warning(
@@ -1542,10 +1589,21 @@ class AgentLoop(ToolCallLifecycle):
_compact_error[0],
)
degraded_compact = True
+ self._account_llm_usage(run_dir, None, iteration, purpose="compaction")
break
summary_resp = _compact_result[0]
else:
- summary_resp = self.llm.chat([{"role": "user", "content": prompt}])
+ try:
+ summary_resp = self.llm.chat([{"role": "user", "content": prompt}])
+ except Exception:
+ self._account_llm_usage(run_dir, None, iteration, purpose="compaction")
+ raise
+ self._account_llm_usage(
+ run_dir, getattr(summary_resp, "usage_metadata", None), iteration,
+ purpose="compaction",
+ )
+ if self._cancel_event.is_set() or self._budget_stop_reason is not None:
+ return
if summary_resp.content:
summary = summary_resp.content
if degraded_compact and not summary:
diff --git a/agent/src/agent/result_artifacts.py b/agent/src/agent/result_artifacts.py
new file mode 100644
index 000000000..2376ba31c
--- /dev/null
+++ b/agent/src/agent/result_artifacts.py
@@ -0,0 +1,140 @@
+"""Deliver oversized observations without losing their original result bytes."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import logging
+from pathlib import Path
+
+from src.config.limits import TOOL_RESULT_LIMIT, truncate_tool_result
+from src.tools.redaction import redact_tool_result
+
+logger = logging.getLogger(__name__)
+
+
+def _write_original(path: Path, content: str, root: Path) -> None:
+ """Save immutable content inside the workspace, or refuse a changed path."""
+ if not path.resolve().is_relative_to(root):
+ raise ValueError('Result artifact escapes workspace')
+ path.parent.mkdir(parents=True, exist_ok=True)
+ try:
+ with path.open('x', encoding='utf-8') as stream:
+ stream.write(content)
+ except FileExistsError:
+ if path.read_text(encoding='utf-8') != content:
+ raise ValueError('Existing result artifact was modified')
+
+
+def save_result_parts(content: str, folder: Path, root: Path) -> list[str]:
+ """Save immutable parts that fit through the actual read_file JSON envelope.
+
+ Args:
+ content: Complete, already redacted result text.
+ folder: Directory for sequentially numbered parts.
+ root: Workspace boundary that every file must stay within.
+
+ Returns:
+ Absolute paths to parts, in their original order.
+ """
+ offset, part_count = 0, 0
+ while offset < len(content):
+ part = folder / f'{part_count + 1}.txt'
+ low, high = 0, min(len(content) - offset, TOOL_RESULT_LIMIT)
+ while low < high:
+ size = (low + high + 1) // 2
+ envelope = json.dumps({'status': 'ok', 'path': str(part),
+ 'content': content[offset:offset + size]}, ensure_ascii=False)
+ if len(envelope) <= TOOL_RESULT_LIMIT:
+ low = size
+ else:
+ high = size - 1
+ if not low:
+ raise ValueError('Artifact path leaves no room for readable content')
+ _write_original(part, content[offset:offset + low], root)
+ part_count += 1
+ offset += low
+ return [str(folder / f"{index + 1}.txt") for index in range(part_count)]
+
+
+def deliver_tool_result(result: str, run_dir: str | Path | None, registry) -> str:
+ """Keep a large result in full, with a bounded valid JSON receipt.
+
+ Args:
+ result: Complete tool result; credentials are scrubbed before saving.
+ run_dir: Current task workspace, never a cross-task data cache.
+ registry: Actual registry, used to check for a local file reader.
+
+ Returns:
+ The original bounded result or a receipt naming its full file and
+ readable parts. If saving or local reading is unavailable, retains
+ the existing explicit truncation behavior without claiming a file.
+ """
+ result = redact_tool_result(result)
+ if len(result) <= TOOL_RESULT_LIMIT or not run_dir:
+ return truncate_tool_result(result)
+ from src.tools.bash_tool import BashTool
+ from src.tools.read_file_tool import ReadFileTool
+ from src.tools.path_utils import safe_run_dir
+
+ try:
+ readable = isinstance(registry.get('bash'), BashTool)
+ if not readable and isinstance(registry.get('read_file'), ReadFileTool):
+ safe_run_dir(str(run_dir))
+ readable = True
+ except (AttributeError, KeyError, ValueError):
+ readable = False
+ if not readable:
+ return truncate_tool_result(result)
+ try:
+ payload = json.loads(result)
+ is_json = True
+ except (ValueError, TypeError):
+ payload = None
+ is_json = False
+ root = Path(run_dir).resolve()
+ digest = hashlib.sha256(result.encode()).hexdigest()
+ folder = root / 'tool-results' / digest
+ path = folder / ('result.json' if is_json else 'result.txt')
+ try:
+ _write_original(path, result, root)
+ parts = save_result_parts(result, folder / 'parts', root)
+ part_count = len(parts)
+ except (OSError, UnicodeError, ValueError):
+ logger.warning('Could not preserve complete tool result', exc_info=True)
+ return truncate_tool_result(result)
+
+ receipt = {
+ '_result_file': {'path': str(path), 'sha256': digest,
+ 'format': 'json' if is_json else 'text',
+ 'characters': len(result), 'parts_dir': str(folder / 'parts'),
+ 'part_count': part_count},
+ '_delivery': (
+ 'The complete tool result is saved in this task workspace. Load the JSON '
+ 'file directly in a calculation script, or use read_file on parts/1.txt '
+ 'through part_count in order. Do not refetch data because this preview '
+ 'is shortened. Source row limits/sampling still apply: inspect the '
+ 'original result coverage before calculating; saving does not fill missing rows.'
+ ),
+ }
+ if isinstance(payload, dict):
+ for key in ('status', 'ok', '_provenance', '_unresolved', '_diagnostics'):
+ if key in payload:
+ receipt[key] = payload[key]
+ # Bound the receipt too. Source metadata stays in the complete artifact if
+ # it alone is larger than the delivery budget; no false absence is implied.
+ if len(json.dumps(receipt, ensure_ascii=False)) > TOOL_RESULT_LIMIT // 2:
+ for key in ('_diagnostics', '_provenance', '_unresolved'):
+ receipt.pop(key, None)
+ receipt['_metadata_location'] = 'Source metadata is in the complete result file.'
+ room = max(0, TOOL_RESULT_LIMIT // 4 - len(json.dumps(receipt, ensure_ascii=False)))
+ # A text preview inside JSON cannot break the surrounding JSON envelope.
+ while room:
+ receipt['_preview'] = result[:room // 2] + '\n... preview ...\n' + result[-room // 2:]
+ encoded = json.dumps(receipt, ensure_ascii=False)
+ if len(encoded) <= TOOL_RESULT_LIMIT:
+ return encoded
+ room //= 2
+ receipt.pop('_preview', None)
+ encoded = json.dumps(receipt, ensure_ascii=False)
+ return encoded if len(encoded) <= TOOL_RESULT_LIMIT else truncate_tool_result(result)
diff --git a/agent/src/agent/run_outcome.py b/agent/src/agent/run_outcome.py
index 234820d7a..6b98ee861 100644
--- a/agent/src/agent/run_outcome.py
+++ b/agent/src/agent/run_outcome.py
@@ -2,7 +2,7 @@
Extracted from the tail of ``AgentLoop._run_bound`` (issue #1624, roadmap
item 3: split ``loop.py`` along its existing seams). This module owns the
-decision ladder that maps how a run ended — stall, cancellation,
+decision ladder that maps how a run ended — stall, cancellation, token budget,
no-progress, content-filter circuit breaker, success, empty model
response, max iterations — onto ``(status, reason)`` plus the ``end``
trace event and the caller-facing result dict.
@@ -12,8 +12,8 @@ side-effectful calls (``state_store.mark_*``, ``trace.write/close``) stay
in ``loop.py`` right after ``resolve_final_status`` returns, in the same
order they executed before the extraction.
-Behavior is unchanged: every branch condition and message string moved
-verbatim.
+The extraction preserved the existing branch conditions and messages. Token
+budget exhaustion now precedes success so partial artifacts cannot hide it.
One deliberate semantic delta, flagged in review: pre-extraction the
``degraded`` flag on the end event and result dict keyed off
@@ -74,6 +74,7 @@ class OutcomeInputs:
empty-response failure message.
max_iterations: The run's iteration budget (for the failure
message).
+ budget_reason: An exhausted research-goal token budget, if any.
"""
stall_reason: Optional[str]
@@ -89,6 +90,7 @@ class OutcomeInputs:
empty_response_provider: str
empty_response_model: str
max_iterations: int
+ budget_reason: Optional[str] = None
@dataclass
@@ -117,8 +119,8 @@ class RunOutcome:
def resolve_final_status(inputs: OutcomeInputs) -> RunOutcome:
"""Run the terminal-status decision ladder.
- The branch order is load-bearing and unchanged from the pre-extraction
- ``_run_bound`` tail: stall (already recorded) → cancelled → no
+ The branch order is load-bearing: stall (already recorded) → cancelled →
+ exhausted token budget → no
progress → content-filter breaker → success → empty response → max
iterations.
"""
@@ -128,6 +130,8 @@ def resolve_final_status(inputs: OutcomeInputs) -> RunOutcome:
return RunOutcome("failed", inputs.stall_reason, None, False)
if inputs.cancelled:
return RunOutcome("cancelled", "cancelled by user", "cancelled", False)
+ if inputs.budget_reason is not None:
+ return RunOutcome("failed", inputs.budget_reason, "failure", False)
if inputs.no_progress_reason is not None:
return RunOutcome("failed", inputs.no_progress_reason, "failure", False)
if inputs.content_filter_circuit_breaker:
diff --git a/agent/src/agent/tool_lifecycle.py b/agent/src/agent/tool_lifecycle.py
index dcbeb9c23..486e21559 100644
--- a/agent/src/agent/tool_lifecycle.py
+++ b/agent/src/agent/tool_lifecycle.py
@@ -22,7 +22,7 @@ from src.agent.tuning import _heartbeat_interval_s, _override, _tool_timeout_sec
from src.agent.tool_results import (
_archive_backtest_result, _failure_code, _is_tool_success, _normalize_tool_run_dir,
)
-from src.config.limits import truncate_tool_result
+from src.agent.result_artifacts import deliver_tool_result
from src.tools.redaction import redact_payload, redact_tool_result
logger = logging.getLogger("src.agent.loop")
@@ -183,7 +183,7 @@ class ToolCallLifecycle:
restored = _replay_context_result(cached)
messages.append(
context.format_tool_result(
- tc.id, tc.name, truncate_tool_result(restored)
+ tc.id, tc.name, deliver_tool_result(restored, self.memory.run_dir, self.registry)
)
)
self._readonly_replay_visibility_pending.add(tc.id)
@@ -258,7 +258,7 @@ class ToolCallLifecycle:
cache_key = self._identical_call_key(tc.name, tc.arguments)
if cache_key is not None and cache_key in self._called_identical:
cached = self._called_identical[cache_key]
- messages.append(context.format_tool_result(tc.id, tc.name, cached))
+ messages.append(context.format_tool_result(tc.id, tc.name, deliver_tool_result(cached, self.memory.run_dir, self.registry)))
self._successful_call_keys[tc.id] = cache_key
self._called_ok.add(cache_key)
trace.write({
@@ -819,6 +819,9 @@ class ToolCallLifecycle:
iteration: Current iteration.
update_memory: Whether this call reached the tool implementation.
"""
+ # Scrub before caching, truncation and model context, not just the
+ # downstream trace/event preview. Shell stdout may hold credentials.
+ result = redact_tool_result(result)
if update_memory:
self._update_memory(tc.name)
@@ -895,7 +898,7 @@ class ToolCallLifecycle:
self._readonly_replay_protected.clear()
status = "ok" if success else "error"
- truncated = truncate_tool_result(result)
+ truncated = deliver_tool_result(result, self.memory.run_dir, self.registry)
messages.append(context.format_tool_result(tc.id, tc.name, truncated))
# One redaction feeds every subscriber below: the persisted trace
diff --git a/agent/src/agent/tool_loading.json b/agent/src/agent/tool_loading.json
new file mode 100644
index 000000000..72d630f75
--- /dev/null
+++ b/agent/src/agent/tool_loading.json
@@ -0,0 +1,5 @@
+{
+ "minimum_tools": 16,
+ "initial_tools": ["search_symbol", "load_skill", "read_file", "write_file", "edit_file", "bash", "compact"],
+ "catalog_description_chars": 80
+}
diff --git a/agent/src/agent/tool_loading.py b/agent/src/agent/tool_loading.py
new file mode 100644
index 000000000..5a1d4fe72
--- /dev/null
+++ b/agent/src/agent/tool_loading.py
@@ -0,0 +1,136 @@
+"""Expose exact tool schemas on demand while keeping the full catalog visible."""
+
+from __future__ import annotations
+
+import json
+from functools import lru_cache
+from pathlib import Path
+from threading import RLock
+from typing import Any
+
+from src.agent.tools import BaseTool, ToolRegistry
+
+
+@lru_cache(maxsize=1)
+def loading_policy() -> dict[str, Any]:
+ """Read the shipped tool-loading policy."""
+ return json.loads(Path(__file__).with_suffix(".json").read_text(encoding="utf-8"))
+
+
+class DeferredToolRegistry(ToolRegistry):
+ """Run-local schema exposure; execution permissions stay with the source tools.
+
+ All registered tools remain in the catalog and full manifest. No retrieval
+ ranking, keyword routing, subagent, or alternate tool dispatcher is involved.
+ """
+
+ def __init__(self, source: ToolRegistry) -> None:
+ super().__init__()
+ self._tools = dict(source._tools)
+ self._import_failures = dict(source.import_failures)
+ self._registration_failures = dict(source.registration_failures)
+ self._lock = RLock()
+ self.register(LoadToolsTool(self))
+ self.reset_exposure()
+
+ def reset_exposure(self) -> None:
+ """Start a new question without carrying unrelated schemas forward."""
+ with self._lock:
+ self._loaded = dict.fromkeys([
+ *(name for name in loading_policy()["initial_tools"] if name in self._tools),
+ "load_tools",
+ ])
+
+ def request_definitions(self) -> list[dict[str, Any]]:
+ """Return unmodified schemas for the tools exposed on this question."""
+ with self._lock:
+ return [self._tools[name].to_openai_schema() for name in self._loaded]
+
+ def catalog(self) -> str:
+ """List every available name with a short discovery hint, never a top-k."""
+ limit = loading_policy()["catalog_description_chars"]
+ lines = []
+ for name, tool in self._tools.items():
+ if name == "load_tools":
+ continue
+ hint = " ".join((tool.description or "").split())
+ if len(hint) > limit:
+ hint = hint[:limit].rsplit(" ", 1)[0] + "…"
+ lines.append(f"- {name}: {hint}")
+ return "\n".join(lines)
+
+ def expose(self, names: list[str]) -> dict[str, Any]:
+ """Load known names atomically without adding tools absent from the registry.
+
+ Args:
+ names: Exact tool names selected from the complete catalog.
+
+ Returns:
+ Acknowledgement of the loaded schemas.
+ """
+ if not isinstance(names, list) or not names or any(not isinstance(n, str) for n in names):
+ return {"status": "error", "error": "names must be a non-empty list of exact tool names"}
+ unknown = [name for name in names if name not in self._tools]
+ if unknown:
+ return {"status": "error", "unknown_tools": unknown, "available_tools": self.tool_names}
+ with self._lock:
+ # Append new schemas so previously exposed definitions retain their
+ # prefix/order for providers that cache common prompt prefixes.
+ self._loaded.update(dict.fromkeys(names))
+ result: dict[str, Any] = {
+ "status": "ok", "loaded": list(dict.fromkeys(names)),
+ "message": "Full schemas are available in the next request. Call the tools directly; loading has not executed them.",
+ }
+ return result
+
+
+class LoadToolsTool(BaseTool):
+ """Select exact schemas from the current run's complete tool catalog."""
+
+ name = "load_tools"
+ description = (
+ "Load full argument schemas for tools named in the system catalog. "
+ "Select all tools needed for the next steps in one call, then call them "
+ "directly in the next turn. This only exposes schemas; it executes no tool."
+ )
+ parameters = {
+ "type": "object", "properties": {
+ "names": {"type": "array", "items": {"type": "string"}, "minItems": 1},
+ }, "required": ["names"],
+ }
+ repeatable = True
+
+ @classmethod
+ def check_available(cls) -> bool:
+ """Installed explicitly per run, never by global subclass discovery."""
+ return False
+
+ def __init__(self, registry: DeferredToolRegistry) -> None:
+ self.registry = registry
+
+ def execute(self, names: list[str], **kwargs: Any) -> str:
+ """Expose schemas without invoking a tool or changing its permissions."""
+ return json.dumps(self.registry.expose(names), ensure_ascii=False)
+
+
+def prepare_registry(registry: ToolRegistry) -> ToolRegistry:
+ """Select deferred exposure for large registries when enabled in config."""
+ from src.config.accessor import get_env_config
+
+ if isinstance(registry, DeferredToolRegistry):
+ return DeferredToolRegistry(registry)
+ if (
+ isinstance(registry, ToolRegistry)
+ and get_env_config().agent_tuning.vibe_trading_tool_loading == "lazy"
+ and len(registry) >= loading_policy()["minimum_tools"]
+ ):
+ return DeferredToolRegistry(registry)
+ return registry
+
+
+def request_definitions(registry: ToolRegistry) -> list[dict[str, Any]]:
+ """Return this request's schemas without changing the full registry API."""
+ if isinstance(registry, DeferredToolRegistry):
+ return registry.request_definitions()
+ definitions = getattr(registry, "get_definitions", None)
+ return definitions() if callable(definitions) else []
diff --git a/agent/src/config/env_schema.py b/agent/src/config/env_schema.py
index efb4f1a76..c63e85d59 100644
--- a/agent/src/config/env_schema.py
+++ b/agent/src/config/env_schema.py
@@ -23,7 +23,7 @@ import os
import types
from enum import Enum
-from typing import Annotated, Any, Union, get_args, get_origin
+from typing import Annotated, Any, Literal, Union, get_args, get_origin
from pydantic import BaseModel, BeforeValidator, ConfigDict, Field, model_validator
@@ -471,6 +471,9 @@ class AgentTuningConfig(_EnvBase):
vibe_trading_agent_max_iterations: int = Field(
alias="VIBE_TRADING_AGENT_MAX_ITERATIONS", default=50, ge=1,
)
+ vibe_trading_tool_loading: Literal["eager", "lazy"] = Field(
+ alias="VIBE_TRADING_TOOL_LOADING", default="lazy",
+ )
vibe_trading_context_window: int | None = Field(
alias="VIBE_TRADING_CONTEXT_WINDOW", default=None, gt=0,
)
diff --git a/agent/src/live/runtime/decision_budget.py b/agent/src/live/runtime/decision_budget.py
index 4fc9ac7a0..b437d659f 100644
--- a/agent/src/live/runtime/decision_budget.py
+++ b/agent/src/live/runtime/decision_budget.py
@@ -165,7 +165,12 @@ class DecisionBudget:
Dollar figures use operator-supplied prices, without cache discounts.
Set the input rate to the highest applicable input/cache-write rate.
"""
- values = [usage.get(k) for k in ("iter", "input_tokens", "output_tokens", "total_tokens")]
+ # One agent iteration may include several compaction calls plus its
+ # ordinary response. Deduplicate the individual calls, not that parent
+ # iteration. Legacy events without a call index keep their old key.
+ values = [usage.get("call_index", usage.get("iter")), *[
+ usage.get(k) for k in ("input_tokens", "output_tokens", "total_tokens")
+ ]]
if any(type(v) is not int or v < 0 for v in values) or not max(values[3], values[1] + values[2]):
with self._connect() as db:
db.execute("UPDATE decisions SET accounting_error=1 WHERE id=?", (decision,))
diff --git a/agent/src/providers/chat.py b/agent/src/providers/chat.py
index 1884fa5fa..74a9ee744 100644
--- a/agent/src/providers/chat.py
+++ b/agent/src/providers/chat.py
@@ -167,13 +167,14 @@ def _extract_retry_after_s(original: Exception) -> Optional[float]:
class ProviderStreamError(RuntimeError):
"""Raised when provider streaming fails before a complete response."""
- def __init__(self, *, provider: str, model: str, original: Exception) -> None:
+ def __init__(self, *, provider: str, model: str, original: Exception, usage_metadata: dict | None = None) -> None:
"""Initialize a provider-contextual stream error.
Args:
provider: Effective provider name.
model: Effective model name.
original: Original exception from the stream path.
+ usage_metadata: Usage already reported before stream failure.
Attributes:
retry_after_s: Provider-suggested Retry-After delay in seconds,
@@ -183,6 +184,7 @@ class ProviderStreamError(RuntimeError):
self.provider = provider
self.model = model
self.original = original
+ self.usage_metadata = usage_metadata
self.status_code: Optional[int] = getattr(original, "status_code", None)
self.retry_after_s: Optional[float] = _extract_retry_after_s(original)
safe_message = _redact_provider_error(str(original))
@@ -507,6 +509,7 @@ class ChatLLM:
Returns:
Parsed ``LLMResponse``.
"""
+ accumulated = None
try:
llm = self._llm.bind_tools(tools) if tools else self._llm
config = {"timeout": timeout} if timeout else {}
@@ -515,6 +518,7 @@ class ChatLLM:
pending_text = ""
possible_dsml_text = True
cancelled = False
+ cancelled_usage = None
last_chunk_ts = _time.monotonic()
for chunk in llm.stream(messages, config=config, **call_kwargs):
now = _time.monotonic()
@@ -528,6 +532,11 @@ class ChatLLM:
last_chunk_ts = now
if should_cancel and should_cancel():
cancelled = True
+ # This chunk already arrived. Preserve any reported usage
+ # while still dropping its text and tool-call deltas.
+ if getattr(chunk, "usage_metadata", None):
+ usage_chunk = chunk if accumulated is None else accumulated + chunk
+ cancelled_usage = usage_chunk.usage_metadata
break
if getattr(chunk, "additional_kwargs", {}).get("provider_waiting") is True and on_waiting_chunk:
on_waiting_chunk()
@@ -549,7 +558,10 @@ class ChatLLM:
accumulated = chunk if accumulated is None else accumulated + chunk
if accumulated is None:
if cancelled:
- return LLMResponse(content="", tool_calls=[], finish_reason="stop")
+ return LLMResponse(
+ content="", tool_calls=[], finish_reason="stop",
+ usage_metadata=cancelled_usage,
+ )
# Endpoint streamed nothing (no SSE support); a non-streaming
# invoke still works — fall back rather than return empty.
logger.warning(
@@ -557,7 +569,17 @@ class ChatLLM:
"non-streaming invoke."
)
return self._parse_response(llm.invoke(messages, config=config, **call_kwargs))
+ # Chat Completions must explicitly finish a choice; a clean socket
+ # EOF is not a final answer. LangChain otherwise defaults missing
+ # metadata to "stop" and releases a truncated report/tool call.
+ metadata = getattr(accumulated, "response_metadata", {}) or {}
+ if (not cancelled and getattr(self._llm, "_llm_type", None) == "openai-chat"
+ and not (metadata.get("finish_reason") or metadata.get("stop_reason")
+ or metadata.get("status") in {"completed", "incomplete", "failed"})):
+ raise ConnectionError("Provider stream ended without a terminal finish reason")
response = self._parse_response(accumulated)
+ if cancelled_usage is not None:
+ response.usage_metadata = cancelled_usage
if pending_text and not (response.has_tool_calls and response.content == ""):
on_text_chunk(pending_text)
return response
@@ -574,7 +596,10 @@ class ChatLLM:
_cfg = get_env_config()
provider = _cfg.llm.langchain_provider.strip().lower() or "openai"
model = self.model_name or _cfg.llm.langchain_model_name.strip() or "(unset)"
- raise ProviderStreamError(provider=provider, model=model, original=exc) from exc
+ raise ProviderStreamError(
+ provider=provider, model=model, original=exc,
+ usage_metadata=getattr(accumulated, "usage_metadata", None),
+ ) from exc
@staticmethod
def _tool_call_thought_signature_maps(ai_message: Any) -> tuple[dict[str, str], dict[int, str]]:
diff --git a/agent/src/providers/llm.py b/agent/src/providers/llm.py
index 65f0df99a..87898c9bd 100644
--- a/agent/src/providers/llm.py
+++ b/agent/src/providers/llm.py
@@ -793,6 +793,12 @@ if ChatOpenAI is not None:
is absent, breaking ReAct continuations after a tool call (#39).
"""
payload = super()._get_request_payload(input_, stop=stop, **kwargs)
+ # LangChain rewrites max_tokens to OpenAI's newer spelling.
+ # DeepSeek documents max_tokens and silently ignores the other
+ # field, leaving its much larger thinking-mode default in force.
+ if "messages" in payload and self._capabilities().name == "deepseek":
+ if "max_completion_tokens" in payload:
+ payload["max_tokens"] = payload.pop("max_completion_tokens")
# Models that rejected `temperature` before omit it up front (#1223).
if str(self.model_name) in _ANTHROPIC_TEMPERATURE_UNSUPPORTED:
payload.pop("temperature", None)
@@ -1018,6 +1024,7 @@ def _build_native_deepseek(
creds = get_llm_credentials("deepseek", model)
api_key = creds["api_key"]
base_url = creds["base_url"]
+ output_limit = get_env_config().llm.langchain_max_tokens
return chat_deepseek(
model=model,
temperature=temperature,
@@ -1026,6 +1033,7 @@ def _build_native_deepseek(
callbacks=callbacks,
api_key=api_key or None,
base_url=base_url or None,
+ **({"max_tokens": output_limit} if output_limit is not None else {}),
)
diff --git a/agent/src/skills/strategy-generate/SKILL.md b/agent/src/skills/strategy-generate/SKILL.md
index ea584a84b..cdeddbba1 100644
--- a/agent/src/skills/strategy-generate/SKILL.md
+++ b/agent/src/skills/strategy-generate/SKILL.md
@@ -11,8 +11,8 @@ category: strategy
3. **Strategy coding**: write `code/signal_engine.py` (following the `SignalEngine` contract)
4. **Syntax check**: `bash("python -c \"import ast; ast.parse(open('code/signal_engine.py').read()); print('OK')\"")`
5. **Run backtest**: call the `backtest` tool (built into the engine; no need to write `run_backtest.py`)
-6. **Evaluate results**: read `artifacts/metrics.csv` and judge by the review criteria
-7. **Iterative fixing**: if results are poor, modify with `edit_file` → run `backtest` → re-evaluate
+6. **Evaluate results**: use the backtest tool's structured metrics, validation and artifact summary. Read `artifacts/metrics.csv` or targeted rows with `read_run_artifact` only when the requested evidence is missing from that summary. Judge by the review criteria and report failures honestly.
+7. **Fix implementation errors**: repair code/contract errors and rerun to verify. Poor returns in a correctly implemented strategy are a valid result: report them and stop. Change the strategy or test additional candidates only when the user requested optimization or comparison, within their specified scope. Deeper attribution and robustness research are performed when requested, not added to every backtest.
**You only need to write `signal_engine.py` and `config.json`. The `backtest` tool automatically handles data loading and backtest execution.**
diff --git a/agent/src/swarm/context.py b/agent/src/swarm/context.py
new file mode 100644
index 000000000..a4e66325b
--- /dev/null
+++ b/agent/src/swarm/context.py
@@ -0,0 +1,110 @@
+"""Budget worker context without discarding a fresh batch of observations."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import logging
+from pathlib import Path
+from typing import Any
+
+from src.agent.context_budget import CompactionBudget, ContextMeter, estimate_tokens, resolve_window
+from src.agent.result_artifacts import save_result_parts
+from src.config.accessor import get_env_config
+from src.config.limits import TOOL_RESULT_LIMIT
+from src.tools.redaction import redact_tool_result
+
+logger = logging.getLogger(__name__)
+_OFFLOADED = "[Worker context result saved: "
+
+
+class WorkerContext:
+ """Track real prompt size and save old observations before removing them.
+
+ A result stays visible for at least one successful model request. Under
+ pressure, older results may move to readable files inside this worker's
+ workspace. Neither a tool count nor a batch position implies obsolescence.
+ """
+
+ def __init__(self, *, model: str, artifact_dir: Path, can_read_files: bool) -> None:
+ cfg = get_env_config()
+ self.window, self.window_source = resolve_window(
+ cfg.llm.langchain_provider, model or cfg.llm.langchain_model_name,
+ env_window=cfg.agent_tuning.vibe_trading_context_window,
+ )
+ self.max_tokens = cfg.agent_tuning.vibe_trading_context_max_tokens
+ self.meter = ContextMeter()
+ self.artifact_dir = artifact_dir.resolve()
+ self.can_read_files = can_read_files
+ self.pending: set[str] = set()
+
+ def budget(self, messages: list[dict], tools: list[dict] | None) -> CompactionBudget:
+ """Return the same window/cost ceiling used by the main agent."""
+ return CompactionBudget(
+ window=self.window, source=self.window_source, max_tokens=self.max_tokens,
+ static_tokens=self.meter.static(messages, estimate_tokens(tools or [])),
+ )
+
+ def tokens(self, messages: list[dict], tools: list[dict] | None) -> int:
+ """Estimate the next request from the last provider input-token count."""
+ return self.meter.tokens(messages, estimate_tokens(tools or []))
+
+ def observe(self, messages: list[dict], tools: list[dict] | None, usage: dict | None) -> None:
+ """Record a successful request; its pending results have now been seen."""
+ self.meter.observe((usage or {}).get('input_tokens'), messages,
+ tool_schema_tokens=estimate_tokens(tools or []))
+ self.pending.clear()
+
+ def compact(self, messages: list[dict], tools: list[dict] | None) -> list[dict[str, Any]]:
+ """Offload only old results needed to get below the pressure threshold.
+
+ Args:
+ messages: Mutable current worker conversation.
+ tools: Exact schemas for the next model request.
+
+ Returns:
+ Receipts for results successfully saved to disk. An unavailable
+ reader or failed write leaves the original result in context.
+ """
+ budget = self.budget(messages, tools)
+ target = min(budget.micro_at, budget.total)
+ if not self.can_read_files or self.tokens(messages, tools) <= target:
+ return []
+ receipts = []
+ for message in messages:
+ if self.tokens(messages, tools) <= target:
+ break
+ content = message.get('content')
+ call_id = message.get('tool_call_id')
+ if (message.get('role') != 'tool' or call_id in self.pending
+ or not isinstance(content, str) or content.startswith(_OFFLOADED)):
+ continue
+ digest = hashlib.sha256(content.encode()).hexdigest()
+ if len(content) < TOOL_RESULT_LIMIT // 10:
+ continue
+ try:
+ paths = self._save_result(redact_tool_result(content), digest)
+ except (OSError, ValueError):
+ logger.warning('Could not preserve worker context result', exc_info=True)
+ continue
+ marker = (
+ f'{_OFFLOADED}{json.dumps(paths)}] This result was returned earlier, not missing. '
+ 'Use read_file on these parts in order to recover it; do not refetch external '
+ 'data just because its earlier result was moved out of context.'
+ )
+ if len(content) <= len(marker):
+ continue
+ message['content'] = marker
+ receipts.append({'tool_call_id': call_id, 'paths': paths, 'characters': len(content)})
+ return receipts
+
+ def _save_result(self, content: str, digest: str) -> list[str]:
+ """Save parts that fit through the real read_file result envelope.
+
+ JSON escaping and the path count toward the shared result cap. Splitting
+ prevents a successfully saved observation from being truncated again
+ when the model reads it back. Existing symlinks cannot redirect writes.
+ """
+ return save_result_parts(
+ content, self.artifact_dir / "context-results" / digest, self.artifact_dir,
+ )
diff --git a/agent/src/swarm/worker.py b/agent/src/swarm/worker.py
index 1c7f101db..6dd78139a 100644
--- a/agent/src/swarm/worker.py
+++ b/agent/src/swarm/worker.py
@@ -20,7 +20,7 @@ from src.agent.llm_usage import _normalize_llm_usage
from src.agent.progress import HeartbeatTimer
from src.agent.skills import SkillsLoader
from src.agent.tools import BaseTool, ToolRegistry
-from src.config.limits import truncate_tool_result
+from src.agent.result_artifacts import deliver_tool_result
from src.config.schema import AgentConfig
from src.providers.chat import ChatLLM, LLMResponse, ProviderStreamError
from src.providers.content_filter import (
@@ -34,6 +34,7 @@ from src.swarm.models import (
SwarmTask,
WorkerResult,
)
+from src.swarm.context import WorkerContext
from src.tools import build_swarm_registry
from src.tools.mcp import MCPRemoteTool
from src.tools.redaction import redact_payload, redact_tool_result
@@ -116,7 +117,8 @@ def _escalated_stream_retry_delay_s(streak: int) -> float:
_HEARTBEAT_INTERVAL_S = _heartbeat_interval_s()
_STREAM_RETRY_DELAY_S = _stream_retry_delay_s()
_STREAM_RETRY_MAX_DELAY_S = _stream_retry_max_delay_s()
-_MAX_TOKEN_ESTIMATE = 60_000
+# Test override; production uses the model window and configured cost ceiling.
+_MAX_TOKEN_ESTIMATE: int | None = None
def _emit(
@@ -397,13 +399,19 @@ def build_worker_prompt(
# paragraphs in under markdown decoration instead of leading with it
# as their own role instructions require.
has_code_tools = bool({"write_file", "bash", "edit_file"} & set(agent_spec.tools or []))
+ iteration_budget = agent_spec.max_iterations or _default_max_iterations()
+ budget_rule = (
+ f"You have at most {iteration_budget} model iterations. Each iteration can "
+ "request multiple independent tools. Reuse available results and finish "
+ "once the requested deliverable is complete.\n\n"
+ )
if has_code_tools:
prompt_parts.append(
"## Execution Rules\n\n"
- "You have a HARD LIMIT of 20 tool calls. After that you will be cut off. Work efficiently.\n\n"
+ + budget_rule +
"**Phase 1 — Plan (0 tool calls):** Before calling any tool, state your plan in 3-5 bullet points.\n\n"
- "**Phase 2 — Execute (≤15 tool calls):**\n"
- "- `load_skill` first to get data access methods and analysis patterns.\n"
+ "**Phase 2 — Execute:**\n"
+ "- Load a relevant skill when its method or API contract is needed; direct tool lookups do not require a preliminary skill read.\n"
"- Write ONE focused Python script via `write_file`, then run it with `bash python script.py`.\n"
"- Do NOT write long Python code inside bash. Use write_file + bash.\n"
"- Do NOT fetch data with curl/requests. Use the patterns from load_skill (yfinance, OKX API via Python).\n"
@@ -418,7 +426,7 @@ def build_worker_prompt(
else:
prompt_parts.append(
"## Execution Rules\n\n"
- "You have a HARD LIMIT of 20 tool calls. After that you will be cut off. Work efficiently.\n\n"
+ + budget_rule +
"**Plan (0 tool calls):** Before calling any tool, state your plan in 3-5 bullet points.\n\n"
"**Execute:** You do not have `write_file`/`bash`/`edit_file` in this role -- call your "
"assigned data/analysis tools directly to gather what your role needs. Do not attempt to "
@@ -707,6 +715,27 @@ def _run_worker_impl(
# 6. ReAct loop
artifact_dir = agent_artifact_dir(run_dir, agent_id, task_id)
artifact_dir.mkdir(parents=True, exist_ok=True)
+ get_tool = getattr(registry, "get", None)
+ worker_context = WorkerContext(
+ model=agent_spec.model_name or getattr(llm, "model_name", "") or "",
+ artifact_dir=artifact_dir,
+ can_read_files=callable(get_tool) and get_tool("read_file") is not None,
+ )
+
+ def account_response(response: LLMResponse) -> None:
+ """Count each attempt, retaining cache fields and estimated provenance."""
+ nonlocal total_input_tokens, total_output_tokens
+ nonlocal total_cache_read_tokens, total_cache_creation_tokens, token_source
+ measured = _measure_usage(messages, response)
+ total_input_tokens += measured.input_tokens
+ total_output_tokens += measured.output_tokens
+ total_cache_read_tokens += measured.cache_read_tokens
+ total_cache_creation_tokens += measured.cache_creation_tokens
+ token_source = (
+ "estimated"
+ if token_source == "estimated" or measured.token_source == "estimated"
+ else "provider"
+ )
t0 = time.monotonic()
iteration = 0
@@ -716,21 +745,12 @@ def _run_worker_impl(
wrap_up_at = max(1, int(max_iterations * 0.8))
last_assistant_content = ""
- _KEEP_RECENT_TOOLS = 3
data_tool_calls = 0
content_filter_count = 0
consecutive_content_filter_count = 0
stream_failure_streak = 0
for iteration in range(max_iterations):
- # Microcompact: clear old tool results to prevent token bloat
- tool_msgs = [m for m in messages if m.get("role") == "tool"]
- if len(tool_msgs) > _KEEP_RECENT_TOOLS:
- for msg in tool_msgs[:-_KEEP_RECENT_TOOLS]:
- content = msg.get("content", "")
- if isinstance(content, str) and len(content) > 100:
- msg["content"] = "[cleared]"
-
# Check timeout
elapsed = time.monotonic() - t0
if elapsed > timeout:
@@ -775,28 +795,6 @@ def _run_worker_impl(
),
)
- # Check token estimate
- token_estimate = len(json.dumps(messages, ensure_ascii=False)) // 4
- if token_estimate > _MAX_TOKEN_ESTIMATE:
- summary = last_assistant_content or f"Worker context too large (~{token_estimate} tokens, {iteration} iterations)"
- summary = _resolve_summary(artifact_dir, summary)
- _emit(event_callback, "worker_token_limit", agent_id, task_id, {"tokens": token_estimate})
- _finalize_run(artifact_dir, summary, messages)
- return WorkerResult(
- status="token_limit",
- summary=summary,
- artifact_paths=_collect_artifacts(run_dir, artifact_dir),
- iterations=iteration,
- input_tokens=total_input_tokens,
- output_tokens=total_output_tokens,
- cache_read_tokens=total_cache_read_tokens,
- cache_creation_tokens=total_cache_creation_tokens,
- token_source=token_source,
- content_filter_warnings=compute_content_filter_warnings(
- content_filter_count, iteration + 1,
- ),
- )
-
# Inject wrap-up nudge when approaching iteration limit
if iteration == wrap_up_at:
remaining = max_iterations - iteration
@@ -823,6 +821,33 @@ def _run_worker_impl(
# On last iteration, call LLM without tool definitions to force text output
is_last_iteration = iteration == max_iterations - 1
tool_defs = None if is_last_iteration else registry.get_definitions()
+ compacted = worker_context.compact(messages, tool_defs)
+ if compacted:
+ _emit(event_callback, "worker_context_offloaded", agent_id, task_id,
+ {"iteration": iteration, "results": compacted})
+ token_estimate = worker_context.tokens(messages, tool_defs)
+ token_limit = (worker_context.budget(messages, tool_defs).total
+ if _MAX_TOKEN_ESTIMATE is None else _MAX_TOKEN_ESTIMATE)
+ if token_estimate > token_limit:
+ summary = last_assistant_content or f"Worker context too large (~{token_estimate} tokens, {iteration} iterations)"
+ summary = _resolve_summary(artifact_dir, summary)
+ _emit(event_callback, "worker_token_limit", agent_id, task_id, {"tokens": token_estimate})
+ _finalize_run(artifact_dir, summary, messages)
+ return WorkerResult(
+ status="token_limit",
+ summary=summary,
+ artifact_paths=_collect_artifacts(run_dir, artifact_dir),
+ iterations=iteration,
+ input_tokens=total_input_tokens,
+ output_tokens=total_output_tokens,
+ cache_read_tokens=total_cache_read_tokens,
+ cache_creation_tokens=total_cache_creation_tokens,
+ token_source=token_source,
+ content_filter_warnings=compute_content_filter_warnings(
+ content_filter_count, iteration + 1,
+ ),
+ )
+
# Stream the LLM — moonshot/kimi non-streaming invoke is unreliable
# (issue #42), and streaming also feeds dashboard live progress.
@@ -870,13 +895,22 @@ def _run_worker_impl(
interval=_HEARTBEAT_INTERVAL_S,
emit=_on_llm_heartbeat,
):
- return llm.stream_chat(
- messages,
- tools=tool_defs,
- timeout=remaining_timeout,
- on_text_chunk=_on_text_chunk,
- **stream_kwargs,
- )
+ try:
+ return llm.stream_chat(
+ messages,
+ tools=tool_defs,
+ timeout=remaining_timeout,
+ on_text_chunk=_on_text_chunk,
+ **stream_kwargs,
+ )
+ except ProviderStreamError as error:
+ usage = getattr(error, "usage_metadata", None)
+ account_response(LLMResponse(usage_metadata=usage))
+ _emit(event_callback, "worker_stream_usage", agent_id, task_id,
+ {"iteration": iteration, "usage": usage,
+ "status": "failed_stream"})
+ raise
+
# A transient mid-stream hiccup (connection reset) used to be
# absorbed by ChatLLM's silent non-streaming fallback; it now
@@ -969,16 +1003,8 @@ def _run_worker_impl(
# cache breakdown). The provenance label is sticky-estimated, mirroring
# runtime._merge_token_source: one heuristic iteration contaminates the
# cumulative total, so a partly-estimated sum never reads as billed.
- measured = _measure_usage(messages, response)
- total_input_tokens += measured.input_tokens
- total_output_tokens += measured.output_tokens
- total_cache_read_tokens += measured.cache_read_tokens
- total_cache_creation_tokens += measured.cache_creation_tokens
- token_source = (
- "estimated"
- if token_source == "estimated" or measured.token_source == "estimated"
- else "provider"
- )
+ worker_context.observe(messages, tool_defs, getattr(response, "usage_metadata", None))
+ account_response(response)
# Track last meaningful assistant content
if response.content and len(response.content.strip()) > 20:
@@ -1127,6 +1153,9 @@ def _run_worker_impl(
)
else:
result = registry.execute(tc.name, args)
+ # The model and persisted conversation need the same protection as
+ # event previews, before truncation can cut a credential envelope.
+ result = redact_tool_result(result)
result_is_error = _is_error_result(result)
if tc.name != "load_skill" and not result_is_error:
data_tool_calls += 1
@@ -1148,9 +1177,10 @@ def _run_worker_impl(
)
messages.append(
ContextBuilder.format_tool_result(
- tc.id, tc.name, truncate_tool_result(result)
+ tc.id, tc.name, deliver_tool_result(result, artifact_dir, registry)
)
)
+ worker_context.pending.add(tc.id)
# Content filter ratio tracking
content_filter_warnings = compute_content_filter_warnings(
diff --git a/agent/tests/test_agent_loop_trace.py b/agent/tests/test_agent_loop_trace.py
index 8dffdf1fe..2de540d7a 100644
--- a/agent/tests/test_agent_loop_trace.py
+++ b/agent/tests/test_agent_loop_trace.py
@@ -79,7 +79,8 @@ def test_tool_call_trace_redacts_args_and_structured_results(tmp_path: Path) ->
result_payload = json.loads(tool_result["result"])
assert result_payload["authorization"] == "[redacted]"
assert result_payload["nested"]["api_key"] == "[redacted]"
- assert messages[0]["content"].count("secret-token") == 1
+ assert "secret-token" not in messages[0]["content"]
+ assert json.loads(messages[0]["content"])["symbol"] == "AAPL"
class _LongAnswerResponse:
diff --git a/agent/tests/test_chat_llm_streaming.py b/agent/tests/test_chat_llm_streaming.py
index 4ca23df97..f632bcb45 100644
--- a/agent/tests/test_chat_llm_streaming.py
+++ b/agent/tests/test_chat_llm_streaming.py
@@ -403,3 +403,70 @@ def test_only_explicit_provider_waiting_reaches_activity_callback():
])).stream_chat([{"role": "user", "content": "q"}], on_waiting_chunk=lambda: waiting.append(True))
assert waiting == [True]
assert response.content == "final"
+
+
[email protected]("terminal", [True, False])
[email protected]("tool_call", [True, False])
+def test_actual_chat_stream_requires_terminal_metadata(terminal, tool_call):
+ """Replay a clean EOF through the real SDK; truncated JSON is not a call."""
+ import json
+ import httpx
+ from src.providers.llm import ChatOpenAIWithReasoning
+
+ requests = []
+ delta = ({"tool_calls": [{"index": 0, "id": "call-partial", "type": "function",
+ "function": {"name": "write_file", "arguments": '{"path":"a.txt"}'}}]}
+ if tool_call else {"content": "I will now calculate the answer."})
+
+ def respond(request):
+ requests.append(request)
+ packets = [{"id": "offline", "object": "chat.completion.chunk", "created": 0,
+ "model": "deepseek-v4-pro", "choices": [{"index": 0, "delta": delta,
+ "finish_reason": None}]}]
+ if terminal:
+ packets.append({**packets[0], "choices": [{"index": 0, "delta": {},
+ "finish_reason": "tool_calls" if tool_call else "stop"}]})
+ packets.append({**packets[0], "choices": [], "usage": {
+ "prompt_tokens": 20, "completion_tokens": 10, "total_tokens": 30}})
+ # EOF without [DONE] is intentionally legal only if choice termination
+ # was already observed. An empty chunk/usage packet alone is not proof.
+ body = "".join("data: " + json.dumps(item) + "\n\n" for item in packets)
+ return httpx.Response(200, text=body, headers={"Content-Type": "text/event-stream"})
+
+ with httpx.Client(transport=httpx.MockTransport(respond)) as transport:
+ adapter = ChatOpenAIWithReasoning(model="deepseek-v4-pro", api_key="offline",
+ base_url="https://example.invalid/v1", http_client=transport,
+ stream_usage=True, max_retries=0)
+ adapter._vibe_provider = "deepseek"
+ client = _client(adapter)
+ if terminal:
+ result = client.stream_chat([{"role": "user", "content": "Calculate"}])
+ assert result.has_tool_calls is tool_call
+ assert result.usage_metadata["total_tokens"] == 30
+ else:
+ with pytest.raises(ProviderStreamError, match="terminal finish reason") as error:
+ client.stream_chat([{"role": "user", "content": "Calculate"}])
+ assert error.value.retryable
+ assert error.value.usage_metadata["total_tokens"] == 30
+ assert len(requests) == 1 # No hidden invoke replay after emitted chunks.
+
+
[email protected]("status", ["completed", "incomplete"])
+def test_responses_terminal_status_remains_valid(status):
+ """Responses has terminal status rather than Chat Completions finish_reason."""
+ chunk = _FakeChunk(content="Answer")
+ chunk.response_metadata = {"status": status}
+ fake = _FakeStreamingLLM([chunk])
+ fake._llm_type = "openai-chat"
+ assert _client(fake).stream_chat([{"role": "user", "content": "Hi"}]).content == "Answer"
+
+
+def test_cancelled_openai_stream_does_not_require_terminal_metadata():
+ chunk = _FakeChunk(content="Partial")
+ chunk.response_metadata = {}
+ fake = _FakeStreamingLLM([chunk])
+ fake._llm_type = "openai-chat"
+ response = _client(fake).stream_chat(
+ [{"role": "user", "content": "Hi"}], should_cancel=lambda: True)
+ assert response.content == ""
+ assert not fake.invoke_called
diff --git a/agent/tests/test_deferred_tool_loading.py b/agent/tests/test_deferred_tool_loading.py
new file mode 100644
index 000000000..bdb6cae75
--- /dev/null
+++ b/agent/tests/test_deferred_tool_loading.py
@@ -0,0 +1,133 @@
+"""Deferred schemas retain discoverability, exact contracts, and execution gates."""
+
+import json
+from types import SimpleNamespace
+from unittest.mock import Mock
+
+from src.agent.context import ContextBuilder, _SYSTEM_PROMPT
+from src.agent.context_budget import ContextMeter
+from src.agent.loop import AgentLoop
+from src.agent.memory import WorkspaceMemory
+from src.agent.tool_loading import DeferredToolRegistry, prepare_registry, request_definitions
+from src.agent.tools import BaseTool, ToolRegistry
+from src.providers.chat import LLMResponse, ToolCallRequest
+
+
+class ExampleTool(BaseTool):
+ description = "Read a named test observation."
+ parameters = {"type": "object", "properties": {"label": {"type": "string"}}, "required": ["label"]}
+
+ def __init__(self, name):
+ self.name = name
+ self.calls = []
+
+ def execute(self, label, **kwargs):
+ self.calls.append(label)
+ return json.dumps({"status": "ok", "observation": label})
+
+
+def registry():
+ source = ToolRegistry()
+ for index in range(18):
+ source.register(ExampleTool(f"read_observation_{index}"))
+ return source
+
+
+def test_every_tool_is_visible_and_loading_preserves_exact_schema():
+ source = registry()
+ deferred = DeferredToolRegistry(source)
+ assert [d["function"]["name"] for d in request_definitions(deferred)] == ["load_tools"]
+ assert all(f"- {name}:" in deferred.catalog() for name in source.tool_names)
+ assert deferred.expose(["read_observation_17", "read_observation_2"])["status"] == "ok"
+ offered = {d["function"]["name"]: d for d in request_definitions(deferred)}
+ assert offered["read_observation_17"] == source.get("read_observation_17").to_openai_schema()
+ assert offered["read_observation_2"] == source.get("read_observation_2").to_openai_schema()
+ assert source.get("read_observation_17").calls == []
+ assert "load_tools" not in source
+ assert len(deferred.get_definitions()) == len(source) + 1 # Full manifest.
+
+
+def test_loading_cannot_add_a_tool_outside_the_current_registry():
+ deferred = DeferredToolRegistry(registry())
+ before = request_definitions(deferred)
+ rejected = deferred.expose(["read_observation_2", "trading_place_order"])
+ assert rejected["status"] == "error"
+ assert request_definitions(deferred) == before
+ assert "trading_place_order" not in deferred
+
+
+def test_independent_runs_do_not_share_exposed_schemas():
+ source = registry()
+ first, second = DeferredToolRegistry(source), DeferredToolRegistry(source)
+ first.expose(["read_observation_2"])
+ assert len(request_definitions(first)) == 2
+ assert len(request_definitions(second)) == 1
+ first.reset_exposure()
+ assert len(request_definitions(first)) == 1
+
+
+def test_new_schemas_append_without_reordering_the_existing_prefix():
+ deferred = DeferredToolRegistry(registry())
+ deferred.expose(["read_observation_17"])
+ prefix = request_definitions(deferred)
+ deferred.expose(["read_observation_0", "read_observation_17"])
+ assert request_definitions(deferred)[:len(prefix)] == prefix
+
+
+def test_eager_escape_hatch_and_small_registries_stay_unchanged(monkeypatch):
+ source = registry()
+ monkeypatch.setenv("VIBE_TRADING_TOOL_LOADING", "eager")
+ assert prepare_registry(source) is source
+
+
+def test_deferred_prompt_retains_grounding_and_every_skill_name():
+ eager = ContextBuilder(registry(), WorkspaceMemory())
+ lazy = ContextBuilder(DeferredToolRegistry(registry()), WorkspaceMemory())
+ original, compact = eager.build_system_prompt(), lazy.build_system_prompt()
+ assert original.split("## Guidelines", 1)[1].split("## Current Date", 1)[0] == compact.split("## Guidelines", 1)[1].split("## Current Date", 1)[0]
+ assert original.split("## Output Principles", 1)[1].split("## Tools", 1)[0] == compact.split("## Output Principles", 1)[1].split("## Tools", 1)[0]
+ assert all(skill.name in compact for skill in lazy.skills_loader.skills)
+ assert "No catalog entry is hidden" in compact
+ assert "Layer 2" in compact
+ assert "MUST" in _SYSTEM_PROMPT
+
+
+def test_new_schemas_change_measured_context_size_without_double_counting():
+ messages = [{"role": "system", "content": "fixed"}, {"role": "user", "content": "question"}]
+ meter = ContextMeter()
+ meter.observe(8_000, messages, tool_schema_tokens=600)
+ assert meter.tokens(messages, 600) == 8_000
+ assert meter.tokens(messages, 1_600) == 9_000
+ assert meter.static(messages, 1_600) - meter.static(messages, 600) == 1_000
+ meter.observe(9_000, messages, tool_schema_tokens=1_600)
+ assert meter.tokens(messages, 1_600) == 9_000
+ assert meter.tokens(messages, 0) == 7_400
+
+
+def test_real_loop_loads_then_calls_the_original_tool(tmp_path, monkeypatch):
+ monkeypatch.setenv("VIBE_TRADING_TOOL_LOADING", "lazy")
+ source = registry()
+ responses = [
+ LLMResponse(tool_calls=[ToolCallRequest(id="load", name="load_tools", arguments={"names": ["read_observation_17"]})]),
+ LLMResponse(tool_calls=[ToolCallRequest(id="read", name="read_observation_17", arguments={"label": "verified"})]),
+ LLMResponse(content="The observation is verified."),
+ ]
+ llm = SimpleNamespace(model_name="offline", stream_chat=Mock(side_effect=responses))
+ agent = AgentLoop(source, llm, max_iterations=5)
+ agent.memory.run_dir = str(tmp_path)
+ result = agent.run("Read the observation and summarize it.")
+ assert result["status"] == "success"
+ assert source.get("read_observation_17").calls == ["verified"]
+ requests = llm.stream_chat.call_args_list
+ assert [t["function"]["name"] for t in requests[0].kwargs["tools"]] == ["load_tools"]
+ assert "read_observation_17" in [t["function"]["name"] for t in requests[1].kwargs["tools"]]
+
+
+def test_loops_do_not_share_schema_state_when_given_the_same_wrapper():
+ shared = DeferredToolRegistry(registry())
+ first = AgentLoop(shared, SimpleNamespace())
+ second = AgentLoop(shared, SimpleNamespace())
+ first.registry.expose(["read_observation_2"])
+ assert len(request_definitions(first.registry)) == 2
+ assert len(request_definitions(second.registry)) == 1
+ assert len(request_definitions(shared)) == 1
diff --git a/agent/tests/test_live_decision_budget.py b/agent/tests/test_live_decision_budget.py
index c716f13ae..5cf4d4769 100644
--- a/agent/tests/test_live_decision_budget.py
+++ b/agent/tests/test_live_decision_budget.py
@@ -69,6 +69,20 @@ def test_missing_usage_and_crash_do_not_reset_budget(tmp_path):
_store(tmp_path).begin(settings, NOW + timedelta(hours=1))
+def test_compaction_calls_share_iteration_without_being_deduplicated(tmp_path):
+ budget = _store(tmp_path)
+ settings = DecisionSettings(max_daily_tokens=300)
+ decision = budget.begin(settings, NOW)
+ first = {**_usage(), "purpose": "compaction", "call_index": 1}
+ second = {**_usage(), "purpose": "compaction", "call_index": 2}
+ budget.record(decision, first, settings, NOW)
+ budget.record(decision, first, settings, NOW) # SSE replay is still idempotent.
+ budget.record(decision, second, settings, NOW)
+ with pytest.raises(DecisionBudgetError, match="token budget"):
+ budget.record(decision, {**_usage(), "call_index": 3}, settings, NOW)
+ assert budget.snapshot(NOW)["usage"]["tokens"] == 330
+
+
def test_daily_schedule_follows_dst_and_fires_once(tmp_path):
budget = _store(tmp_path)
settings = DecisionSettings(daily_at="10:00")
diff --git a/agent/tests/test_llm_cost_accounting.py b/agent/tests/test_llm_cost_accounting.py
new file mode 100644
index 000000000..2a8487c55
--- /dev/null
+++ b/agent/tests/test_llm_cost_accounting.py
@@ -0,0 +1,293 @@
+"""Paid responses remain accounted for when compaction or Stop interrupts work."""
+
+from types import SimpleNamespace
+from unittest.mock import Mock
+import json
+
+import pytest
+from langchain_core.messages import AIMessageChunk
+
+from src.agent.llm_usage import _normalize_llm_usage
+from src.agent.loop import AgentLoop
+from src.agent.tools import ToolRegistry
+from src.agent.trace import TraceWriter
+from src.config.accessor import reset_env_config
+from src.providers.chat import ChatLLM, LLMResponse, ToolCallRequest
+
+
+USAGE = {"input_tokens": 1000, "output_tokens": 100, "total_tokens": 1100}
+
+
+def _messages():
+ return [{"role": "system", "content": "system"}, *[
+ {"role": "user", "content": f"part {index} " + "x" * 29900}
+ for index in range(8)
+ ]]
+
+
+def _agent(tmp_path, callback=None):
+ llm = SimpleNamespace(
+ model_name="offline-test",
+ chat=Mock(return_value=LLMResponse(content="Summary retained.", usage_metadata=USAGE)),
+ stream_chat=Mock(return_value=LLMResponse(content="Done.", usage_metadata=USAGE)),
+ )
+ agent = AgentLoop(registry=ToolRegistry(), llm=llm, event_callback=callback)
+ agent.memory.run_dir = str(tmp_path)
+ return agent, llm
+
+
+def test_every_summary_chunk_is_persisted_and_reported(tmp_path):
+ events = []
+ agent, llm = _agent(tmp_path, lambda kind, data: events.append((kind, data)))
+ trace = TraceWriter(tmp_path)
+ try:
+ agent._auto_compact(_messages(), tmp_path, trace, iteration=7)
+ finally:
+ trace.close()
+ assert llm.chat.call_count == 3
+ report = json.loads((tmp_path / "llm_usage.json").read_text())
+ assert report["totals"]["calls"] == 3
+ assert report["totals"]["total_tokens"] == 3300
+ assert all(row["purpose"] == "compaction" for row in report["per_iteration"])
+ assert [data["total_tokens"] for kind, data in events if kind == "llm_usage"] == [1100] * 3
+
+
+def test_summary_budget_cancellation_prevents_more_summaries_and_agent_call(tmp_path, monkeypatch):
+ def monitor(kind, data):
+ if kind == "llm_usage":
+ agent.cancel()
+
+ agent, llm = _agent(tmp_path, monitor)
+
+ def compact(messages, run_dir, trace, iteration, current_iter):
+ messages[:] = _messages()
+ agent._auto_compact(messages, run_dir, trace, iteration=current_iter)
+
+ monkeypatch.setattr(agent, "_compact_to_budget", compact)
+ result = agent.run("Summarize the earlier work.")
+ assert result["status"] == "cancelled"
+ assert llm.chat.call_count == 1
+ llm.stream_chat.assert_not_called()
+ report = json.loads((tmp_path / "llm_usage.json").read_text())
+ assert report["totals"]["total_tokens"] == 1100
+
+
[email protected]("budget,summary_calls,agent_calls,status", [
+ (1000, 1, 0, "failed"), (5000, 3, 1, "success"),
+])
+def test_summary_usage_obeys_goal_budget_without_stopping_funded_work(
+ tmp_path, monkeypatch, budget, summary_calls, agent_calls, status,
+):
+ from src.goal import GoalStore
+
+ monkeypatch.setenv("VIBE_TRADING_GOAL_DB_PATH", str(tmp_path / "goals.db"))
+ monkeypatch.setattr("src.agent.tuning.GOAL_MAX_CONTINUATIONS", 0, raising=False)
+ store = GoalStore()
+ goal = store.replace_goal(
+ session_id="summary-cost", objective="Summarize the earlier work.",
+ criteria=["Summarize"], token_budget=budget,
+ )
+ agent, llm = _agent(tmp_path)
+
+ def compact(messages, run_dir, trace, iteration, current_iter):
+ messages[:] = _messages()
+ agent._auto_compact(messages, run_dir, trace, iteration=current_iter)
+
+ monkeypatch.setattr(agent, "_compact_to_budget", compact)
+ result = agent.run("Summarize the earlier work.", session_id="summary-cost")
+ assert result["status"] == status
+ assert llm.chat.call_count == summary_calls
+ assert llm.stream_chat.call_count == agent_calls
+ updated = store.get_goal(goal.goal_id)
+ assert updated.tokens_used == (summary_calls + agent_calls) * USAGE["total_tokens"]
+ assert updated.turns_used == 1
+
+
+def test_failed_summary_without_timeout_reports_unknown_usage(tmp_path, monkeypatch):
+ events = []
+ agent, llm = _agent(tmp_path, lambda kind, data: events.append((kind, data)))
+ llm.chat.side_effect = RuntimeError("Provider disconnected")
+ monkeypatch.setattr("src.agent.loop._llm_timeout_seconds", lambda: 0)
+ trace = TraceWriter(tmp_path)
+ try:
+ with pytest.raises(RuntimeError, match="Provider disconnected"):
+ agent._auto_compact(_messages(), tmp_path, trace, iteration=7)
+ finally:
+ trace.close()
+ assert ("llm_usage_unavailable", {"iter": 7, "purpose": "compaction"}) in events
+ assert not (tmp_path / "llm_usage.json").exists()
+
+
+def test_cancelled_agent_response_keeps_reported_usage(tmp_path):
+ events = []
+ agent, llm = _agent(tmp_path, lambda kind, data: events.append((kind, data)))
+
+ def cancel_response(*args, **kwargs):
+ agent.cancel()
+ return LLMResponse(content="Do not release this partial text.", usage_metadata=USAGE)
+
+ llm.stream_chat.side_effect = cancel_response
+ result = agent.run("Say hello.")
+ assert result["status"] == "cancelled"
+ report = json.loads((tmp_path / "llm_usage.json").read_text())
+ assert report["totals"]["total_tokens"] == 1100
+ assert len([event for event in events if event[0] == "llm_usage"]) == 1
+ llm.stream_chat.assert_called_once()
+
+
+def test_cancelled_stream_keeps_usage_from_already_received_chunk():
+ client = object.__new__(ChatLLM)
+ client._llm = SimpleNamespace(stream=lambda *args, **kwargs: iter([
+ AIMessageChunk(content="", usage_metadata=USAGE),
+ ]))
+ client._prompt_cache_request = lambda messages: (messages, {})
+ on_text = Mock()
+ response = client.stream_chat(
+ [{"role": "user", "content": "hello"}], should_cancel=lambda: True,
+ on_text_chunk=on_text,
+ )
+ assert response.usage_metadata == USAGE
+ on_text.assert_not_called()
+
+
+def test_reasoning_is_a_reported_subset_of_output_not_added_again():
+ normalized = _normalize_llm_usage({**USAGE, "output_token_details": {"reasoning": 80}})
+ assert normalized["reasoning_tokens"] == 80
+ assert normalized["output_tokens"] == 100
+ assert normalized["total_tokens"] == 1100
+ assert "reasoning_tokens" not in _normalize_llm_usage(USAGE)
+
+
+def test_goal_token_budget_stops_current_turn_and_blocks_unfunded_retry(tmp_path, monkeypatch):
+ from src.goal import GoalStore
+
+ monkeypatch.setenv("VIBE_TRADING_GOAL_DB_PATH", str(tmp_path / "goals.db"))
+ store = GoalStore()
+ goal = store.replace_goal(
+ session_id="cost-test", objective="Summarize the project files.",
+ criteria=["Read the file"], token_budget=1000,
+ )
+ agent, llm = _agent(tmp_path)
+ llm.stream_chat.side_effect = [
+ LLMResponse(content="", tool_calls=[ToolCallRequest(
+ id="should-not-run", name="read_file", arguments={"path": "report.txt"},
+ )], usage_metadata=USAGE),
+ LLMResponse(content="Done.", usage_metadata=USAGE),
+ ]
+ result = agent.run("Continue the goal.", session_id="cost-test")
+ assert result["status"] == "failed"
+ assert "goal_token_budget_exhausted" in result["reason"]
+ llm.stream_chat.assert_called_once()
+ assert result["react_trace"] == []
+ assert store.get_goal(goal.goal_id).tokens_used == 1100
+
+ # An ordinary retry cannot silently restart spending on the exhausted goal.
+ agent.run("Continue.", session_id="cost-test")
+ llm.stream_chat.assert_called_once()
+
+
[email protected]("limit", [None, 2048])
+def test_native_deepseek_forwards_configured_output_limit(monkeypatch, limit):
+ from src.providers import llm as provider
+
+ factory = Mock(return_value=object())
+ monkeypatch.setattr(provider, "import_module", lambda name: SimpleNamespace(ChatDeepSeek=factory))
+ monkeypatch.setattr(provider, "get_llm_credentials", lambda *args: {
+ "api_key": "test-key", "base_url": "https://example.invalid",
+ })
+ if limit is None:
+ monkeypatch.delenv("LANGCHAIN_MAX_TOKENS", raising=False)
+ else:
+ monkeypatch.setenv("LANGCHAIN_MAX_TOKENS", str(limit))
+ reset_env_config()
+ provider._build_native_deepseek(model="configured-model", temperature=0)
+ kwargs = factory.call_args.kwargs
+ assert kwargs.get("max_tokens") == limit
+ assert ("max_tokens" in kwargs) == (limit is not None)
+ assert kwargs["model"] == "configured-model"
+
+
+def test_native_deepseek_output_limit_reaches_http_request(monkeypatch):
+ """Exercise the optional real adapter and SDK without contacting a provider."""
+ import httpx
+ from src.providers import llm as provider
+
+ native = pytest.importorskip("langchain_deepseek")
+ requests = []
+
+ def respond(request):
+ requests.append(json.loads(request.content))
+ return httpx.Response(200, json={
+ "id": "offline-response", "object": "chat.completion", "created": 0,
+ "model": "deepseek-chat", "choices": [{
+ "index": 0, "finish_reason": "stop",
+ "message": {"role": "assistant", "content": "Hello"},
+ }], "usage": {"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12},
+ })
+
+ monkeypatch.setenv("LANGCHAIN_MAX_TOKENS", "2048")
+ monkeypatch.setattr(provider, "get_llm_credentials", lambda *args: {
+ "api_key": "offline-test-key", "base_url": "https://example.invalid/v1",
+ })
+ reset_env_config()
+ with httpx.Client(transport=httpx.MockTransport(respond)) as client:
+ def factory(**kwargs):
+ return native.ChatDeepSeek(**kwargs, http_client=client)
+
+ monkeypatch.setattr(provider, "import_module", lambda name: SimpleNamespace(ChatDeepSeek=factory))
+ adapter = provider._build_native_deepseek(model="deepseek-chat", temperature=0)
+ response = adapter.invoke("Hello")
+
+ assert response.content == "Hello"
+ assert len(requests) == 1
+ assert requests[0]["max_tokens"] == 2048
+
+
[email protected]("provider_name,model,field", [
+ ("deepseek", "deepseek-v4-pro", "max_tokens"),
+ ("openai", "gpt-4o", "max_completion_tokens"),
+])
+def test_compatible_output_limit_uses_the_provider_wire_field(provider_name, model, field):
+ import httpx
+ from src.providers.llm import ChatOpenAIWithReasoning
+
+ requests = []
+
+ def respond(request):
+ requests.append(json.loads(request.content))
+ return httpx.Response(200, json={
+ "id": "offline", "object": "chat.completion", "created": 0,
+ "model": model, "choices": [{"index": 0, "finish_reason": "stop",
+ "message": {"role": "assistant", "content": "Done"}}],
+ })
+
+ with httpx.Client(transport=httpx.MockTransport(respond)) as client:
+ adapter = ChatOpenAIWithReasoning(
+ model=model, api_key="offline", base_url="https://example.invalid/v1",
+ max_tokens=8192, http_client=client,
+ )
+ adapter._vibe_provider = provider_name
+ adapter.invoke("Hello")
+ assert requests[0][field] == 8192
+ other = "max_completion_tokens" if field == "max_tokens" else "max_tokens"
+ assert other not in requests[0]
+
+
[email protected]("recover", [True, False])
+def test_failed_stream_usage_is_counted_before_retry(tmp_path, monkeypatch, recover):
+ from src.providers.chat import ProviderStreamError
+ agent, llm = _agent(tmp_path)
+ monkeypatch.setattr("src.agent.loop._stream_retry_backoff_s", lambda _: 0)
+ def error():
+ return ProviderStreamError(
+ provider="deepseek", model="deepseek-v4-pro",
+ original=ConnectionError("EOF"), usage_metadata=USAGE,
+ )
+ llm.stream_chat.side_effect = [error(), LLMResponse(content="Hello", usage_metadata=USAGE)
+ if recover else error()]
+ result = agent.run("Say hello.")
+ assert result["status"] == ("success" if recover else "failed")
+ report = json.loads((tmp_path / "llm_usage.json").read_text())
+ assert report["totals"]["calls"] == 2
+ assert report["totals"]["total_tokens"] == USAGE["total_tokens"] * 2
+ assert report["per_iteration"][0]["purpose"] == "failed_stream"
diff --git a/agent/tests/test_model_tool_result_redaction.py b/agent/tests/test_model_tool_result_redaction.py
new file mode 100644
index 000000000..56f0b6f31
--- /dev/null
+++ b/agent/tests/test_model_tool_result_redaction.py
@@ -0,0 +1,83 @@
+"""Credentials in shell-style results never reach model requests or caches."""
+
+import copy
+import json
+from types import SimpleNamespace
+
+from src.agent.loop import AgentLoop
+from src.agent.tools import BaseTool, ToolRegistry
+from src.providers.chat import LLMResponse, ToolCallRequest
+from src.swarm.models import SwarmAgentSpec, SwarmTask
+import src.swarm.worker as worker
+from tests.test_agent_loop_deterministic_cache import _drive
+
+
+class _CredentialResult(BaseTool):
+ name = 'credential_result_probe'
+ description = 'Fixture result with ordinary financial data and a credential.'
+ parameters = {'type': 'object', 'properties': {}}
+ deterministic = True
+ repeatable = True
+ is_readonly = True
+
+ def __init__(self, secret):
+ self.secret = secret
+ self.calls = 0
+
+ def execute(self, **kwargs):
+ self.calls += 1
+ return json.dumps({'status': 'ok', 'stdout': f'TUSHARE_TOKEN={self.secret}\nclose=112.4 USD',
+ 'data': {'close': 112.4, 'currency': 'USD'}, 'content': 'Financial evidence'})
+
+
+def test_main_model_context_and_deterministic_replay_are_scrubbed(tmp_path):
+ secret = 'fixture-only-private-value'
+ tool = _CredentialResult(secret)
+ registry = ToolRegistry()
+ registry.register(tool)
+ agent = AgentLoop(registry=registry, llm=SimpleNamespace(), max_iterations=3)
+ agent.memory.run_dir = str(tmp_path)
+ messages, records = _drive(agent, tool.name, tmp_path, [{}, {}])
+ assert tool.calls == 1
+ assert secret not in json.dumps([messages, records, list(agent._called_identical.values())])
+ for message in messages:
+ payload = json.loads(message['content'])
+ assert payload['data'] == {'close': 112.4, 'currency': 'USD'}
+ assert payload['content'] == 'Financial evidence'
+ assert 'TUSHARE_TOKEN=[redacted]' in payload['stdout']
+
+
+def test_worker_model_request_and_persisted_conversation_are_scrubbed(tmp_path, monkeypatch):
+ secret = 'fixture-only-worker-private-value'
+ tool = _CredentialResult(secret)
+ registry = ToolRegistry()
+ registry.register(tool)
+ seen = []
+
+ class Model:
+ def stream_chat(self, messages, **kwargs):
+ seen.append(copy.deepcopy(messages))
+ if len(seen) == 1:
+ return LLMResponse(tool_calls=[ToolCallRequest(id='read', name=tool.name, arguments={})])
+ return LLMResponse(content='The requested financial evidence is collected and its source is retained. ' * 5)
+
+ def close(self):
+ pass
+
+ monkeypatch.setattr(worker, 'ChatLLM', lambda **kwargs: Model())
+ monkeypatch.setattr(worker, 'build_swarm_registry', lambda *args, **kwargs: registry)
+ result = worker.run_worker(
+ agent_spec=SwarmAgentSpec(id='analyst', role='Analyst', system_prompt='Read evidence.',
+ tools=[tool.name], max_iterations=3),
+ task=SwarmTask(id='t', agent_id='analyst', prompt_template='Read the evidence.'),
+ upstream_summaries={}, user_vars={}, run_dir=tmp_path,
+ )
+ assert result.status == 'completed'
+ assert len(seen) == 2
+ assert secret not in json.dumps(seen)
+ persisted = next(tmp_path.rglob('messages.json')).read_text()
+ assert secret not in persisted
+ output = next(m for m in seen[1] if m['role'] == 'tool')
+ payload = json.loads(output['content'])
+ assert payload['data'] == {'close': 112.4, 'currency': 'USD'}
+ assert 'TUSHARE_TOKEN=[redacted]' in payload['stdout']
diff --git a/agent/tests/test_readonly_replay_after_microcompact.py b/agent/tests/test_readonly_replay_after_microcompact.py
index bec79457d..97c10d223 100644
--- a/agent/tests/test_readonly_replay_after_microcompact.py
+++ b/agent/tests/test_readonly_replay_after_microcompact.py
@@ -1,5 +1,7 @@
from __future__ import annotations
+import json
+
import threading
from types import SimpleNamespace
@@ -531,7 +533,7 @@ def test_a_write_makes_earlier_readonly_results_unreplayable() -> None:
messages = []
loop._process_tool_calls([SimpleNamespace(name="factor_analysis", arguments=read, id="f2")], _Context(), messages, _Trace(), [], 3)
- assert '"version":"v2"' in messages[-1]["content"]
+ assert json.loads(messages[-1]["content"])["version"] == "v2"
assert "_vibe_replay" not in messages[-1]["content"]
diff --git a/agent/tests/test_redaction_shared.py b/agent/tests/test_redaction_shared.py
index e458e8aad..34cb06691 100644
--- a/agent/tests/test_redaction_shared.py
+++ b/agent/tests/test_redaction_shared.py
@@ -468,10 +468,11 @@ def test_plaintext_result_secret_reaches_neither_trace_nor_preview(
assert _PLAINTEXT_KEY_SECRET not in preview
assert _PLAINTEXT_BEARER_SECRET not in preview
- # Deliberate boundary: the LLM still receives the real result, because the
- # model needs the unredacted text to act on it. Only persisted/streamed
- # surfaces are scrubbed.
- assert _PLAINTEXT_KEY_SECRET in messages[0]["content"]
+ # Data-source credentials are consumed by tools, not forwarded to the
+ # model provider. The diagnostic remains useful without either secret.
+ assert _PLAINTEXT_KEY_SECRET not in messages[0]["content"]
+ assert _PLAINTEXT_BEARER_SECRET not in messages[0]["content"]
+ assert "connect failed" in messages[0]["content"]
def test_write_file_content_argument_never_lands_in_the_trace(tmp_path: Path) -> None:
diff --git a/agent/tests/test_result_artifacts.py b/agent/tests/test_result_artifacts.py
new file mode 100644
index 000000000..3ff2f41ff
--- /dev/null
+++ b/agent/tests/test_result_artifacts.py
@@ -0,0 +1,152 @@
+"""Large observations remain usable without resending or refetching them."""
+
+import copy
+import json
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+
+from src.agent.loop import AgentLoop
+from src.agent.result_artifacts import deliver_tool_result
+from src.agent.tools import BaseTool, ToolRegistry
+from src.config.limits import TOOL_RESULT_LIMIT
+from src.providers.chat import LLMResponse, ToolCallRequest
+from src.swarm.models import SwarmAgentSpec, SwarmTask
+from src.tools.read_file_tool import ReadFileTool
+import src.swarm.worker as worker
+from tests.test_agent_loop_deterministic_cache import _drive
+
+
[email protected](autouse=True)
+def allow_task_workspace(tmp_path, monkeypatch):
+ monkeypatch.setenv("VIBE_TRADING_ALLOWED_RUN_ROOTS", str(tmp_path))
+
+
+def large_result():
+ return json.dumps({
+ 'status': 'ok', '_provenance': {'source': 'fixture', 'quote_currency': 'USD'},
+ 'TUSHARE_TOKEN': 'fixture-private-key',
+ 'AAPL': {'rows': 900, 'returned': 800, 'truncated': True,
+ 'data': [{'date': f'row-{i}', 'close': i + .125,
+ 'note': 'quoted "field"\n中文'} for i in range(800)]},
+ }, ensure_ascii=False)
+
+
+def registry_with_reader():
+ registry = ToolRegistry()
+ registry.register(ReadFileTool())
+ return registry
+
+
+def test_complete_json_and_every_readable_part(tmp_path):
+ registry = registry_with_reader()
+ raw = large_result()
+ delivered = deliver_tool_result(raw, tmp_path, registry)
+ assert len(delivered) <= TOOL_RESULT_LIMIT
+ receipt = json.loads(delivered)
+ assert receipt['_provenance']['source'] == 'fixture'
+ artifact = receipt['_result_file']
+ original = Path(artifact['path']).read_text()
+ assert 'fixture-private-key' not in original + delivered
+ parsed = json.loads(original)
+ assert parsed['AAPL'] == json.loads(raw)['AAPL']
+ parts = []
+ for index in range(1, artifact['part_count'] + 1):
+ part = registry.execute('read_file', {
+ 'path': str(Path(artifact['parts_dir']) / f'{index}.txt'),
+ 'run_dir': str(tmp_path),
+ })
+ assert len(part) <= TOOL_RESULT_LIMIT
+ parts.append(json.loads(part)['content'])
+ assert ''.join(parts) == original
+ assert deliver_tool_result(raw, tmp_path, registry) == delivered
+
+
[email protected]('failure', ['no_reader', 'write_failure', 'symlink', 'modified'])
+def test_unavailable_artifact_is_not_claimed(tmp_path, monkeypatch, failure):
+ registry = registry_with_reader()
+ if failure == 'no_reader':
+ registry = ToolRegistry()
+ elif failure == 'write_failure':
+ def fail(*args, **kwargs):
+ raise OSError('disk full')
+ monkeypatch.setattr('src.agent.result_artifacts._write_original', fail)
+ elif failure == 'symlink':
+ outside = tmp_path.parent / f'{tmp_path.name}-outside'
+ outside.mkdir()
+ (tmp_path / 'tool-results').symlink_to(outside, target_is_directory=True)
+ else:
+ receipt = json.loads(deliver_tool_result(large_result(), tmp_path, registry))
+ Path(receipt['_result_file']['path']).write_text('changed by another writer')
+ result = deliver_tool_result(large_result(), tmp_path, registry)
+ assert 'TRUNCATED:' in result
+ assert '_result_file' not in result
+ assert 'fixture-private-key' not in result
+ if failure == 'symlink':
+ assert not list(outside.iterdir())
+ if failure == 'modified':
+ assert Path(receipt['_result_file']['path']).read_text() == 'changed by another writer'
+
+
+class _LargeResult(BaseTool):
+ name = 'large_result_probe'
+ description = 'Test large observation delivery.'
+ parameters = {'type': 'object', 'properties': {}}
+ deterministic = True
+ repeatable = True
+ is_readonly = True
+
+ def __init__(self, result):
+ self.result = result
+ self.calls = 0
+
+ def execute(self, **kwargs):
+ self.calls += 1
+ return self.result
+
+
+def test_main_fresh_and_cached_results_use_same_complete_file(tmp_path):
+ registry = registry_with_reader()
+ tool = _LargeResult(large_result())
+ registry.register(tool)
+ agent = AgentLoop(registry=registry, llm=SimpleNamespace(), max_iterations=3)
+ agent.memory.run_dir = str(tmp_path)
+ messages, _ = _drive(agent, tool.name, tmp_path, [{}, {}])
+ assert tool.calls == 1
+ paths = [json.loads(message['content'])['_result_file']['path'] for message in messages]
+ assert len(set(paths)) == 1
+ assert len(json.loads(Path(paths[0]).read_text())['AAPL']['data']) == 800
+
+
+def test_worker_model_can_recover_complete_result(tmp_path, monkeypatch):
+ registry = registry_with_reader()
+ registry.register(_LargeResult(large_result()))
+ seen = []
+
+ class Model:
+ def stream_chat(self, messages, **kwargs):
+ seen.append(copy.deepcopy(messages))
+ if len(seen) == 1:
+ return LLMResponse(tool_calls=[ToolCallRequest(
+ id='large', name='large_result_probe', arguments={})])
+ receipt = json.loads(next(m['content'] for m in messages if m['role'] == 'tool'))
+ data = json.loads(Path(receipt['_result_file']['path']).read_text())
+ assert data['AAPL']['data'][-1]['close'] == 799.125
+ assert data['AAPL']['truncated'] is True
+ return LLMResponse(content='The requested source observations are available in full, with source sampling retained. ' * 5)
+
+ def close(self):
+ pass
+
+ monkeypatch.setattr(worker, 'ChatLLM', lambda **kwargs: Model())
+ monkeypatch.setattr(worker, 'build_swarm_registry', lambda *args, **kwargs: registry)
+ result = worker.run_worker(
+ agent_spec=SwarmAgentSpec(id='analyst', role='Analyst', system_prompt='Read evidence.',
+ tools=['large_result_probe', 'read_file'], max_iterations=3),
+ task=SwarmTask(id='t', agent_id='analyst', prompt_template='Read evidence.'),
+ upstream_summaries={}, user_vars={}, run_dir=tmp_path,
+ )
+ assert result.status == 'completed'
+ assert len(seen) == 2
+ assert 'fixture-private-key' not in json.dumps(seen)
diff --git a/agent/tests/test_swarm_context.py b/agent/tests/test_swarm_context.py
new file mode 100644
index 000000000..07cb635a3
--- /dev/null
+++ b/agent/tests/test_swarm_context.py
@@ -0,0 +1,175 @@
+"""Worker evidence survives ordinary batches and remains readable under pressure."""
+
+import copy
+import json
+
+import pytest
+
+from src.agent.tools import BaseTool, ToolRegistry
+from src.providers.chat import LLMResponse, ToolCallRequest
+from src.config.limits import truncate_tool_result
+from src.swarm.context import WorkerContext
+from src.swarm.models import SwarmAgentSpec, SwarmTask
+import src.swarm.worker as worker
+from src.tools.read_file_tool import ReadFileTool
+
+
+class _EvidenceTool(BaseTool):
+ def __init__(self, name):
+ self.name = name
+
+ def execute(self, **kwargs):
+ return json.dumps({'status': 'ok', 'observed': self.name, 'detail': 'actual evidence ' * 50})
+
+
+def test_entire_batch_reaches_the_next_model_request(tmp_path, monkeypatch):
+ names = ['get_financial_statements', 'get_fund_flow', 'get_margin_trading',
+ 'get_research_reports', 'get_stock_news', 'a', 'b', 'c', 'd', 'e']
+ registry = ToolRegistry()
+ for name in names:
+ registry.register(_EvidenceTool(name))
+ seen = []
+
+ class Model:
+ def stream_chat(self, messages, **kwargs):
+ seen.append(copy.deepcopy(messages))
+ if len(seen) == 1:
+ return LLMResponse(tool_calls=[ToolCallRequest(id=n, name=n, arguments={}) for n in names])
+ return LLMResponse(content='The evidence was collected for all requested dimensions. ' * 5)
+
+ def close(self):
+ pass
+
+ monkeypatch.setattr(worker, 'ChatLLM', lambda **kwargs: Model())
+ monkeypatch.setattr(worker, 'build_swarm_registry', lambda *args, **kwargs: registry)
+ result = worker.run_worker(
+ agent_spec=SwarmAgentSpec(id='analyst', role='Analyst', system_prompt='Read all evidence.',
+ tools=names, max_iterations=3),
+ task=SwarmTask(id='t', agent_id='analyst', prompt_template='Review the company.'),
+ upstream_summaries={}, user_vars={}, run_dir=tmp_path,
+ )
+ assert result.status == 'completed'
+ outputs = [m for m in seen[1] if m['role'] == 'tool']
+ assert len(outputs) == 10
+ assert [json.loads(m['content'])['observed'] for m in outputs] == names
+
+
+def _pressured(tmp_path, *, can_read=True):
+ context = WorkerContext(model='deepseek-v4-pro', artifact_dir=tmp_path, can_read_files=can_read)
+ context.max_tokens = 12_000
+ messages = [{'role': 'system', 'content': 'Analyze the supplied observations.'}]
+ for i in range(4):
+ messages.append({'role': 'tool', 'tool_call_id': f'old-{i}', 'name': 'probe',
+ 'content': f'old observation {i} ' + 'x' * 16_000})
+ context.observe(messages, [], {'input_tokens': 19_000})
+ for i in range(2):
+ messages.append({'role': 'tool', 'tool_call_id': f'new-{i}', 'name': 'probe',
+ 'content': 'fresh observation ' + 'y' * 1_000})
+ context.pending.add(f'new-{i}')
+ return context, messages
+
+
+def test_pressure_saves_old_results_and_keeps_the_fresh_batch(tmp_path, monkeypatch):
+ monkeypatch.setenv('VIBE_TRADING_ALLOWED_RUN_ROOTS', str(tmp_path))
+ context, messages = _pressured(tmp_path)
+ originals = {m['tool_call_id']: m['content'] for m in messages[1:]}
+ receipts = context.compact(messages, [])
+ assert receipts
+ assert context.tokens(messages, []) <= context.budget(messages, []).micro_at
+ for receipt in receipts:
+ assert receipt['tool_call_id'].startswith('old-')
+ parts = []
+ for path in receipt['paths']:
+ read = json.loads(truncate_tool_result(ReadFileTool().execute(path=path, run_dir=str(tmp_path))))
+ assert read['status'] == 'ok'
+ parts.append(read['content'])
+ assert ''.join(parts) == originals[receipt['tool_call_id']]
+ assert messages[-1]['content'] == originals['new-1']
+ assert messages[-2]['content'] == originals['new-0']
+ assert context.compact(messages, []) == []
+
+
[email protected]('cause', ['no-reader', 'write-failure'])
+def test_unrecoverable_results_are_never_discarded(tmp_path, monkeypatch, cause):
+ context, messages = _pressured(tmp_path, can_read=cause != 'no-reader')
+ before = copy.deepcopy(messages)
+ if cause == 'write-failure':
+ def refuse(*args, **kwargs):
+ raise OSError('unwritable')
+ monkeypatch.setattr('src.agent.result_artifacts._write_original', refuse)
+ assert context.compact(messages, []) == []
+ assert messages == before
+ assert context.tokens(messages, []) > context.budget(messages, []).total
+
+
+def test_real_usage_and_schema_growth_are_counted(tmp_path):
+ context = WorkerContext(model='deepseek-v4-pro', artifact_dir=tmp_path, can_read_files=False)
+ messages = [{'role': 'system', 'content': 's'}, {'role': 'user', 'content': 'q'}]
+ schemas = [{'type': 'function', 'description': 'large schema ' * 500}]
+ assert context.tokens(messages, schemas) > context.tokens(messages, []) + 1000
+ context.observe(messages, schemas, {'input_tokens': 150_000})
+ assert context.tokens(messages, schemas) == 150_000
+ assert context.tokens(messages, []) < 150_000
+ context.max_tokens = 100_000
+ assert context.tokens(messages, schemas) > context.budget(messages, schemas).total
+
+
+def test_result_parts_survive_json_escaping_and_redact_secrets(tmp_path, monkeypatch):
+ monkeypatch.setenv('VIBE_TRADING_ALLOWED_RUN_ROOTS', str(tmp_path))
+ from src.tools.redaction import redact_tool_result
+ context, messages = _pressured(tmp_path)
+ messages[1]['content'] = json.dumps({'api_key': 'private-value', 'evidence': '\\"\n' * 5000})
+ original = redact_tool_result(messages[1]['content'])
+ receipts = context.compact(messages, [])
+ saved = next(r for r in receipts if r['tool_call_id'] == 'old-0')
+ parts = [json.loads(truncate_tool_result(ReadFileTool().execute(path=p, run_dir=str(tmp_path))))['content']
+ for p in saved['paths']]
+ assert ''.join(parts) == original
+ assert 'private-value' not in ''.join(parts)
+
+
+def test_context_file_symlink_cannot_redirect_writes(tmp_path):
+ context, messages = _pressured(tmp_path / 'worker')
+ context.artifact_dir.mkdir()
+ outside = tmp_path / 'outside'
+ outside.mkdir()
+ (context.artifact_dir / 'context-results').symlink_to(outside, target_is_directory=True)
+ before = copy.deepcopy(messages)
+ assert context.compact(messages, []) == []
+ assert messages == before
+ assert list(outside.iterdir()) == []
+
+
+def test_worker_refuses_over_budget_request_without_discarding_fresh_results(tmp_path, monkeypatch):
+ registry = ToolRegistry()
+ registry.register(_EvidenceTool('probe'))
+ calls = []
+
+ class Model:
+ def stream_chat(self, messages, **kwargs):
+ calls.append(copy.deepcopy(messages))
+ return LLMResponse(tool_calls=[ToolCallRequest(id='fresh', name='probe', arguments={})],
+ usage_metadata={'input_tokens': 999_999, 'output_tokens': 10})
+
+ def close(self):
+ pass
+
+ monkeypatch.setattr(worker, 'ChatLLM', lambda **kwargs: Model())
+ monkeypatch.setattr(worker, 'build_swarm_registry', lambda *args, **kwargs: registry)
+ result = worker.run_worker(
+ agent_spec=SwarmAgentSpec(id='a', role='Analyst', system_prompt='Analyze.', tools=['probe'], max_iterations=3),
+ task=SwarmTask(id='t', agent_id='a', prompt_template='Read.'),
+ upstream_summaries={}, user_vars={}, run_dir=tmp_path,
+ )
+ assert len(calls) == 1
+ assert result.status == 'token_limit'
+ messages = json.loads(next(tmp_path.rglob('messages.json')).read_text())
+ assert json.loads(messages[-1]['content'])['observed'] == 'probe'
+
+
[email protected]('tools', [['get_market_data'], ['write_file']])
+def test_prompt_names_actual_iteration_budget(tools):
+ spec = SwarmAgentSpec(id='a', role='Analyst', system_prompt='Analyze.', tools=tools, max_iterations=7)
+ prompt = worker.build_worker_prompt(spec, {}, '')
+ assert 'at most 7 model iterations' in prompt
+ assert 'HARD LIMIT of 20 tool calls' not in prompt
diff --git a/agent/tests/test_swarm_worker_stream_retry.py b/agent/tests/test_swarm_worker_stream_retry.py
index 442d3df83..f5725b00b 100644
--- a/agent/tests/test_swarm_worker_stream_retry.py
+++ b/agent/tests/test_swarm_worker_stream_retry.py
@@ -193,3 +193,15 @@ def test_non_retryable_4xx_fails_without_retry(monkeypatch, tmp_path):
assert result.status == "failed"
assert "LLM call failed at iteration 0" in (result.error or "")
assert llm.calls == 1
+
+
+def test_failed_stream_usage_survives_retry_and_worker_failure(monkeypatch, tmp_path):
+ """Paid partial calls stay in totals, even if no deliverable is released."""
+ usage = {"input_tokens": 100, "output_tokens": 20, "total_tokens": 120}
+ errors = [ProviderStreamError(provider="deepseek", model="test-model",
+ original=ConnectionError("incomplete stream"), usage_metadata=usage)
+ for _ in range(2)]
+ result = _run(monkeypatch, tmp_path, _FlakyChatLLM(errors, LLMResponse(content=FINAL_TEXT)))
+ assert result.status == "failed"
+ assert result.input_tokens == 200
+ assert result.output_tokens == 40
diff --git a/frontend/src/lib/api.ts b/frontend/src/lib/api.ts
index d21141fad..166cf148f 100644
--- a/frontend/src/lib/api.ts
+++ b/frontend/src/lib/api.ts
@@ -536,8 +536,8 @@ export const api = {
createSession: (title?: string) => request<SessionItem>("/sessions", { method: "POST", body: JSON.stringify({ title: title || "" }) }),
deleteSession: (sid: string) => request<{ status: string }>(`/sessions/${sid}`, { method: "DELETE" }),
renameSession: (sid: string, title: string) => request<{ status: string }>(`/sessions/${sid}`, { method: "PATCH", body: JSON.stringify({ title }) }),
- // Codex-style LLM summary title from the first exchange; backend refuses to
- // overwrite a manual rename, so this is safe to fire-and-forget.
+ // Optional paid title generation; ordinary chat uses its prompt-derived title.
+ // The backend refuses to overwrite a manual rename.
autoTitleSession: (sid: string) => request<{ status: string; title: string }>(`/sessions/${sid}/title/auto`, { method: "POST" }),
// Scheduled research: cadence + timezone are stored as authored (local
// wall-clock cron + IANA key), so list rows render without any UTC math.
diff --git a/frontend/src/pages/Agent.tsx b/frontend/src/pages/Agent.tsx
index 6ab9f10b1..f8152baa7 100644
--- a/frontend/src/pages/Agent.tsx
+++ b/frontend/src/pages/Agent.tsx
@@ -1037,13 +1037,9 @@ export function Agent() {
});
}
- // First completed exchange → ask the backend for a Codex-style
- // summary title, then nudge the sidebar to re-list sessions.
- if (act().messages.filter((message) => message.type === "user").length === 1) {
- api.autoTitleSession(sid)
- .then(() => window.dispatchEvent(new Event("vibe:sessions-refresh")))
- .catch(() => {});
- }
+ // Session creation already derives a title from the user's prompt.
+ // Refresh the sidebar without another paid model call after answering.
+ window.dispatchEvent(new Event("vibe:sessions-refresh"));
// Detect Shadow Account id if render_shadow_report fired successfully this turn
const shadowCall = completedTools.find(
diff --git a/frontend/src/pages/__tests__/AgentActivityTimerPersistence.test.tsx b/frontend/src/pages/__tests__/AgentActivityTimerPersistence.test.tsx
index e47b3137f..78999163d 100644
--- a/frontend/src/pages/__tests__/AgentActivityTimerPersistence.test.tsx
+++ b/frontend/src/pages/__tests__/AgentActivityTimerPersistence.test.tsx
@@ -17,6 +17,7 @@ const apiMock = vi.hoisted(() => ({
getLLMSettings: vi.fn(),
getRunPreview: vi.fn(),
getSessionMessages: vi.fn(),
+ autoTitleSession: vi.fn(),
sseUrl: vi.fn((sid: string) => `/sessions/${sid}/events`),
}));
@@ -47,6 +48,7 @@ describe("Agent activity elapsed clock survives navigation", () => {
beforeEach(() => {
useAgentStore.getState().reset();
sseMock.connect.mockClear();
+ apiMock.autoTitleSession.mockReset().mockResolvedValue({ status: "updated" });
apiMock.getGoal.mockResolvedValue(null);
apiMock.getSessionMessages.mockResolvedValue([]);
apiMock.getLLMSettings.mockResolvedValue({
@@ -69,6 +71,34 @@ describe("Agent activity elapsed clock survives navigation", () => {
});
});
+ it("finishes the first answer without starting a paid title request", async () => {
+ const router = createMemoryRouter(
+ [{ path: "/", element: <Agent /> }],
+ { initialEntries: ["/?session=session-one"] },
+ );
+ render(<RouterProvider router={router} />);
+ await waitFor(() => expect(sseMock.connect).toHaveBeenCalled());
+ act(() => {
+ useAgentStore.getState().addMessage({
+ id: "first-question", type: "user", content: "Hello", timestamp: Date.now(),
+ });
+ });
+ const refresh = vi.fn();
+ window.addEventListener("vibe:sessions-refresh", refresh);
+ try {
+ act(() => {
+ latestHandlers()["attempt.completed"]({
+ attempt_id: "first-attempt", summary: "Hello back", elapsed_ms: 100,
+ });
+ });
+ expect(useAgentStore.getState().messages.some((message) => message.content === "Hello back")).toBe(true);
+ expect(refresh).toHaveBeenCalled();
+ expect(apiMock.autoTitleSession).not.toHaveBeenCalled();
+ } finally {
+ window.removeEventListener("vibe:sessions-refresh", refresh);
+ }
+ });
+
it("keeps the running attempt's startedAt when leaving the page and coming back", async () => {
const router = createMemoryRouter(
[
diff --git a/pyproject.toml b/pyproject.toml
index 8a1acdd64..a949ba2a3 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -110,6 +110,7 @@ include = ["src*", "backtest*", "cli*", "evals*"]
"skills/**/*.json",
"skills/**/*.py",
"providers/*.json",
+ "agent/*.json",
"tools/paper_trade/*.json",
"tools/paper_trade/*.md",
"pixel_agents/*.json",