AI-powered virtual executive team — a single coherent executive persona backed by 8 specialist agents (FastAPI + Next.js).
Cache the agent tool loop's own transcript
Rufuse4af3bbe488f
diff --git a/packages/core/openexecutive/architecture/architecture-facts.yaml b/packages/core/openexecutive/architecture/architecture-facts.yamlindex 151c74a3..e091f441 100644--- a/packages/core/openexecutive/architecture/architecture-facts.yaml+++ b/packages/core/openexecutive/architecture/architecture-facts.yaml@@ -96,6 +96,17 @@ caching:Two cache_control blocks (down from four, consolidated in commit c27dcecto stay within the Anthropic 4-block API limit alongside any otherephemeral markers the request uses).+ breakpoint_budget: |+ The full 4-block budget, and why none of it is spare:+ 1. persona + knowledge (1h) — system block 0+ 2. company + org (5m, optional) — system block 1+ 3. last client tool (1h) — executive tool list+ 4. newest tool_result (5m) — the agent loop's intra-turn marker+ History turns deliberately carry NO marker. A rolling marker on the+ penultimate assistant turn used to claim a latent fifth breakpoint,+ which Anthropic rejects outright on the direct (non-OpenRouter) path.+ It never earned its slot: history turns are flat strings, and the+ system + tool blocks already cover the expensive stable prefix.block_order:- id: persona_and_knowledgettl: 1h@@ -119,11 +130,46 @@ caching:Shorter TTL so org edits propagate quickly. Only emitted whennon-empty.rag_context_placement: User turn (NOT the system prompt) — RAG hits vary per query, would poison the cache if cached.- rolling_history_cache: The penultimate assistant turn is marked ephemeral, building a rolling cache for long conversations.+ intra_turn_cache: |+ executive._apply_loop_cache_marker moves a single ephemeral marker onto+ the newest non-empty tool_result after each iteration of the tool-use+ loop. Without it the loop caches only the static tools+system prefix and+ re-sends its whole accumulated transcript at full input price on every+ iteration; the runtime signature of that bug is cache_read_input_tokens+ pinned flat while the prompt grows iteration over iteration.+ The marker MOVES rather than accumulates — each pass sweeps every+ tool_result clean before re-marking the newest. Left in place per+ iteration it would reach max_iterations breakpoints and blow the 4-block+ limit. Sweeping first also makes the helper idempotent and self-healing:+ there is no bookkeeping index to drift out of sync.+ The marker must stay at the tool_result block's own top level —+ providers.feature_gate._strip_cache_control walks one level deep only, so+ a marker nested inside tool_result["content"] would survive the gate and+ reach a backend that rejects the field.+ tool_result_bound: |+ TOOL_RESULT_MAX_CHARS (default 50,000) caps one tool result before it+ enters the prompt. Applied at the single point every result reaches the+ model (the tool_results comprehension in _stream_agent_loop), not at each+ producer: that also covers specialist output, tool errors and the+ unknown-tool fallback, and leaves non-model consumers — the+ propose_form_values json.loads, the audit trail — reading the full text.+ Capping upstream of that parse would feed it truncated JSON and silently+ drop the form_patch SSE event.+ A circuit breaker, not a routine clipper: the default is set so ordinary+ tool output never reaches it. A cut result carries an explicit marker+ naming the tool and stating the result is incomplete — silent truncation+ is worse than none, because the model reads the cut text as the whole+ answer and confabulates the rest.openrouter_passthrough: |When OPENROUTER_ENABLED=true and the selected model is in the Claudefamily, cache_control markers pass through the OpenRouter translator- unchanged. OpenRouter forwards them to Anthropic, so prompt-cache hit+ unchanged — on system content, user-turn content, tool entries, and+ tool-result content. The tool-result case is the agent loop's intra-turn+ marker and needs the translator to carry it onto the emitted `tool`+ message as a typed array: a marker text block appended to the user+ message would be re-ordered ahead of the tool messages (they are+ appended last) and land before the very bytes it must cover.+ OpenRouter forwards these markers to Anthropic, so prompt-cache hitrates stay roughly equivalent to direct routing. For non-Claudemodels the feature gate strips cache_control before translation —those backends have no equivalent feature and would reject thediff --git a/packages/core/openexecutive/architecture/prebuilt/caching.json b/packages/core/openexecutive/architecture/prebuilt/caching.jsonindex 9a924864..3f3f9bea 100644--- a/packages/core/openexecutive/architecture/prebuilt/caching.json+++ b/packages/core/openexecutive/architecture/prebuilt/caching.json@@ -1,7 +1,7 @@{"section_id": "caching","title": "Prompt Caching",- "markdown": "The Executive system prompt is partitioned for Anthropic prompt caching by `openexecutive.prompts.cache_manager.build_system_blocks`. It emits at most two `cache_control` blocks, consolidated down from four to stay within Anthropic's 4-block API limit alongside the tool definitions and the rolling message cache.\n\n- **Block 0 — persona + knowledge (1h TTL, always present):** the executive persona constant (or admin override) with the `{VOICE_PERSONA}` substitution, conditional `WEB_SEARCH_ADDENDUM` / `MCP_ADDENDUM`, the identity addendum (exec name/email, impersonation guard), the timezone addendum, and `KNOWLEDGE_INDEX_SUMMARY`. Every component is a constant or process-stable value, so the TTL is long.\n- **Block 1 — company + org (5m TTL, only if non-empty):** `company_profile.to_prompt_block()` plus `render_org_block()` (departments, heads, principals, charters). Shorter TTL so org edits propagate quickly.\n\nRAG context is **never** placed in a cached system block — retrieved hits vary per query and would poison the cache. It is injected into the **user turn** instead. The penultimate assistant turn is marked ephemeral to build a rolling cache for long conversations.\n\nTool definitions are sorted by name so their serialized order is stable across requests; the `web_search` server tool is appended after the sorted list so it never receives a `cache_control` marker.\n\n**The invariant:** never put dynamic content in a `cache_control` block. A single mutating value breaks the cache and drives roughly a 10x cost increase.",- "mermaid": "flowchart TD\n Tools[\"Tool defs<br/>(sorted by name)\"] --> B0\n B0[\"Block 0: persona + knowledge<br/>cache_control 1h\"] --> B1\n B1[\"Block 1: company + org<br/>cache_control 5m\"] --> Limit{{\"≤ 4 cache_control blocks\"}}\n Roll[\"Rolling history<br/>(penultimate turn ephemeral)\"] --> Limit\n Limit --> Turn[\"User turn\"]\n RAG[\"RAG hits<br/>(vary per query)\"] --> Turn\n Turn --> API[\"Anthropic API\"]\n class B0 cache\n class B1 cache\n class RAG hot\n class Turn hot\n class Tools compute\n class Roll cache\n class API external",- "generated_at": "2026-06-01T00:00:00+00:00"+ "markdown": "The Executive system prompt is partitioned for Anthropic prompt caching by `openexecutive.prompts.cache_manager.build_system_blocks`. It emits at most two `cache_control` blocks, consolidated down from four to stay within Anthropic's 4-block API limit alongside the tool definitions and the agent loop's intra-turn marker.\n\n- **Block 0 \u2014 persona + knowledge (1h TTL, always present):** the executive persona constant (or admin override) with the `{VOICE_PERSONA}` substitution, conditional `WEB_SEARCH_ADDENDUM` / `MCP_ADDENDUM`, the identity addendum (exec name/email, impersonation guard), the timezone addendum, and `KNOWLEDGE_INDEX_SUMMARY`. Every component is a constant or process-stable value, so the TTL is long.\n- **Block 1 \u2014 company + org (5m TTL, only if non-empty):** `company_profile.to_prompt_block()` plus `render_org_block()` (departments, heads, principals, charters). Shorter TTL so org edits propagate quickly.\n\nTool definitions are sorted by name so their serialized order is stable across requests; the `web_search` server tool is appended after the sorted list so it never receives a `cache_control` marker. The last client tool carries the third marker (1h TTL).\n\n## The intra-turn marker\n\nThe fourth and final breakpoint belongs to the tool-use loop. After each iteration appends its `tool_result` blocks, `executive._apply_loop_cache_marker` moves a single ephemeral marker onto the **newest non-empty tool result**. Without it the loop caches only the static tools+system prefix, so every iteration re-sends the whole accumulated transcript at full input price \u2014 the runtime signature is `cache_read_input_tokens` pinned flat while the prompt grows iteration over iteration.\n\nThe marker **moves rather than accumulates**: each pass first sweeps every `tool_result` clean, then re-marks the newest one. A marker left in place per iteration would reach `max_iterations` breakpoints and blow the 4-block limit. Sweeping first also makes the helper idempotent \u2014 there is no bookkeeping index to drift.\n\nHistory turns carry **no** marker. A rolling marker on the penultimate assistant turn used to claim a latent fifth breakpoint, which Anthropic rejects outright on the direct path; it never earned its slot, since history turns are flat strings and the system + tool blocks already cover the expensive stable prefix.\n\n**The breakpoint budget, in full:** persona+knowledge (1h) + company+org (5m, conditional) + last client tool (1h) + intra-turn tool result (5m) = at most 4.\n\nRAG context is **never** placed in a cached system block \u2014 retrieved hits vary per query and would poison the cache. It is injected into the **user turn** instead.\n\n## Tool-result bound\n\n`TOOL_RESULT_MAX_CHARS` (default 50,000) caps a single tool result before it enters the prompt, applied at the one point every result reaches the model rather than at each producer \u2014 so it also covers specialist output, tool errors and the unknown-tool fallback, while non-model consumers (the `propose_form_values` JSON parse, the audit trail) still read the full text. It is a circuit breaker, not a routine clipper: the default is set so ordinary tool output never reaches it. A cut result carries an explicit marker naming the tool and stating that the result is incomplete \u2014 truncating silently is worse than not truncating, because the model reads the cut text as the whole answer.\n\n**The invariant:** never put dynamic content in a `cache_control` block. A single mutating value breaks the cache and drives roughly a 10x cost increase.",+ "mermaid": "flowchart TD\n Tools[\"Tool defs (sorted by name)<br/>last one: cache_control 1h\"] --> B0\n B0[\"Block 0: persona + knowledge<br/>cache_control 1h\"] --> B1\n B1[\"Block 1: company + org<br/>cache_control 5m\"] --> Limit{{\"\u2264 4 cache_control blocks\"}}\n Loop[\"Newest tool_result<br/>cache_control 5m (moves each iteration)\"] --> Limit\n Hist[\"History turns<br/>(no marker)\"] --> Turn\n Limit --> Turn[\"User turn\"]\n RAG[\"RAG hits<br/>(vary per query)\"] --> Turn\n Turn --> API[\"Anthropic API\"]\n class B0 cache\n class B1 cache\n class RAG hot\n class Turn hot\n class Tools compute\n class Loop cache\n class API external",+ "generated_at": "2026-09-20T12:19:15Z"}diff --git a/packages/core/openexecutive/config.py b/packages/core/openexecutive/config.pyindex fc29c28f..2b5a5052 100644--- a/packages/core/openexecutive/config.py+++ b/packages/core/openexecutive/config.py@@ -462,6 +462,16 @@ class Settings(BaseSettings):)google_chat_project_number: str | None = Field(None, alias="GOOGLE_CHAT_PROJECT_NUMBER")+ # ---- Tool results ----+ # Upper bound on a single tool result's characters before it enters the+ # prompt. A circuit breaker against an unbounded result (a large document+ # fetch) dominating a turn and then being re-sent on every remaining+ # iteration of the tool loop — deliberately set high enough that ordinary+ # tool output never reaches it. Applies to every tool, not just MCP.+ tool_result_max_chars: int = Field(+ 50_000, alias="TOOL_RESULT_MAX_CHARS", ge=1_000+ )+mcp_servers_config_path: Path = Field(_ROOT / "company" / "mcp_servers.json", alias="MCP_SERVERS_CONFIG_PATH")diff --git a/packages/core/openexecutive/orchestrator/executive.py b/packages/core/openexecutive/orchestrator/executive.pyindex 8618d294..97e63935 100644--- a/packages/core/openexecutive/orchestrator/executive.py+++ b/packages/core/openexecutive/orchestrator/executive.py@@ -106,6 +106,142 @@ def _trunc(value: Any, limit: int = 200) -> str:return f"{s[:limit]}…[truncated {len(s) - limit} chars]"+# Smallest cap for which the never-exceeds-limit guarantee holds; mirrored+# by the ge= bound on TOOL_RESULT_MAX_CHARS in config.py.+_MIN_USEFUL_CAP = 1_000++# Longest tool name echoed into a truncation marker. Real names are well+# under this (the longest in the MCP surface is ~40 chars); the bound exists+# so a model-supplied name cannot inflate the marker past its own budget.+_TOOL_NAME_MARKER_MAX = 80+++def _cap_tool_result(text: Any, *, tool_name: str, limit: int) -> Any:+ """Bound one tool result before it enters the prompt.++ A circuit breaker, not a routine clipper: the default budget is set so+ it never fires on ordinary tool output. It exists because a single+ unbounded result (a large document fetch) otherwise lands in the+ context and is then re-sent on every remaining iteration of the tool+ loop, which is the dominant token cost of a long turn.++ The marker is load-bearing. Truncating silently is worse than not+ truncating: the model reads the cut text as the whole answer and+ confabulates the rest. Naming the tool and steering toward a+ *narrower* re-request (rather than a retry, which would re-trigger+ the cut and burn another iteration) is what makes the cut recoverable.++ Non-``str`` results pass through untouched — every producer returns a+ string today, and guessing at the size of some other type is not this+ function's job.++ ``limit`` is passed in rather than read here: ``get_settings()`` builds+ a fresh ``Settings`` on every call, and this runs once per tool result.++ Precondition: ``limit`` must be at least ``_MIN_USEFUL_CAP``. The+ "never exceeds ``limit``" guarantee holds by reserving the marker+ inside the budget, and the marker itself is ~284-383 chars — below+ that floor there is no room for it and the guarantee breaks. The+ config field enforces this with ``ge=1_000``.+ """+ if not isinstance(text, str):+ return text+ if len(text) <= limit:+ return text++ # ``tool_name`` comes from the model's tool_use block, so it is+ # attacker-influenceable via prompt injection. Two consequences:+ # * it is never passed through ``str.format`` — a name containing+ # braces would otherwise be interpreted as a field reference,+ # substituting our locals or raising and killing the turn;+ # * it is length-bounded, so the marker cannot outgrow the budget+ # it is supposed to fit inside.+ safe_name = tool_name[:_TOOL_NAME_MARKER_MAX] if isinstance(tool_name, str) else "?"++ def marker(shown: int, pct: int) -> str:+ return (+ f"\n\n[TRUNCATED by Open Executive: showed the first {shown:,} of "+ f"{len(text):,} characters from `{safe_name}` ({pct}% omitted). "+ "This is NOT the full result. To see more, call the tool again "+ "with a narrower request — a page range, a section name, a query "+ "or filter — rather than re-requesting the whole document.]"+ )++ # Reserve the marker inside the budget so the capped result never+ # exceeds ``limit``. Reserve against the widest form: ``shown`` can+ # never exceed ``limit``, and ``pct`` can round up to 100 (three+ # digits), so no real marker is longer than this one. With safe_name+ # bounded, the reserve stays well under the config's 1_000 minimum.+ shown = max(0, limit - len(marker(limit, 100)))+ pct = round((len(text) - shown) * 100 / len(text))+ logger.warning(+ "tool_result truncated tool=%s original_chars=%d shown_chars=%d limit=%d",+ safe_name, len(text), shown, limit,+ )+ return text[:shown] + marker(shown, pct)+++def _apply_loop_cache_marker(+ messages: list[dict[str, Any]], start: int = 0+) -> None:+ """Move the agent loop's intra-turn cache breakpoint to the newest+ tool result, in place.++ Without this the loop caches only the static tools+system prefix, so+ every iteration re-sends the whole accumulated transcript at full+ input price and ``cache_read_input_tokens`` stays pinned flat while+ the prompt grows. Marking the newest tool result makes each iteration+ read the previous iteration's write and extend it, so hits accrue.++ The marker MOVES rather than accumulates: a per-iteration marker left+ in place would reach ``max_iterations`` breakpoints and blow the+ 4-breakpoint API limit. Sweeping first also makes this idempotent and+ self-healing — there is no bookkeeping index to drift out of sync.++ Placement is the newest non-empty ``tool_result`` in a user message.+ That is normally the last block of the last user message; when the+ newest results are empty it falls back to the newest non-empty one+ before them, so an all-empty iteration still gets a breakpoint. Only+ the empty blocks then sit outside the cached prefix, which costs+ nothing.++ The marker must stay at the ``tool_result`` block's own top level:+ ``providers.feature_gate._strip_cache_control`` removes markers one+ level deep only, so a marker nested inside ``tool_result["content"]``+ would survive the gate and reach a provider that rejects the field.++ ``start`` bounds the sweep to messages this loop appended. The loop's+ ``current_messages`` is a SHALLOW copy, so the caller still owns the+ dicts before that index; without the bound a caller-supplied+ ``tool_result`` would be mutated in place. Today no caller builds one,+ but making that structural beats guarding it with a comment.+ """+ # Sweep every marker, and remember every markable block in order. The+ # newest one wins, but keeping the whole list means an iteration whose+ # last results are all empty falls back to the newest earlier result+ # rather than shipping with no breakpoint at all — losing the marker+ # costs the entire accumulated transcript at full price for that step.+ markable: list[dict[str, Any]] = []+ for msg in messages[start:]:+ content = msg.get("content")+ if not isinstance(content, list):+ continue+ is_user = msg.get("role") == "user"+ for block in content:+ if not isinstance(block, dict) or block.get("type") != "tool_result":+ continue+ block.pop("cache_control", None)+ # An empty result carries nothing worth caching and risks an+ # upstream 400 on the translated typed block. Non-str content+ # (a typed/image result) is markable — it is real bytes in the+ # prefix — but is left to the translator to render.+ body = block.get("content")+ if is_user and body:+ markable.append(block)+ if markable:+ markable[-1]["cache_control"] = {"type": "ephemeral"}++def _tool_error_result(tool_name: str, exc: BaseException) -> str:"""Render a crashed tool handler as a JSON tool_result the model can read.@@ -381,23 +517,17 @@ class Executive:messages: list[dict[str, Any]] = []history = session.get_recent_history()- for i, turn in enumerate(history):- msg: dict[str, Any] = {"role": turn["role"], "content": turn["content"]}- # Cache the penultimate assistant turn to build a rolling cache- if (- turn["role"] == "assistant"- and i == len(history) - 2- and self._settings.enable_caching- and isinstance(turn["content"], str)- ):- msg["content"] = [- {- "type": "text",- "text": turn["content"],- "cache_control": {"type": "ephemeral"},- }- ]- messages.append(msg)+ for turn in history:+ # No cache_control on history turns. The breakpoint budget is 4+ # per request and the other three are always spoken for: two+ # system blocks (cache_manager.build_system_blocks) plus the tool+ # block, leaving exactly one for the agent loop's intra-turn+ # marker (_apply_loop_cache_marker). A rolling marker here used+ # to claim a latent fifth, which Anthropic rejects outright on+ # the direct path; it never earned its slot anyway, since history+ # turns are flat strings and the system + tool blocks already+ # cover the expensive stable prefix.+ messages.append({"role": turn["role"], "content": turn["content"]})user_content_parts: list[dict[str, Any]] = []@@ -1122,6 +1252,8 @@ class Executive:Also yields debug event dicts when a debug_collector is provided."""current_messages = list(messages)+ # Shallow copy — the caller owns every dict up to this index.+ caller_message_count = len(current_messages)last_full_text = ""specialists_consulted: list[str] = []@@ -1552,17 +1684,29 @@ class Executive:yield debug_collector.to_sse_dict(evt)current_messages.append({"role": "assistant", "content": response_content})+ # Cap here, at the single point every result reaches the model,+ # rather than at each producer: this also covers specialist+ # output, tool errors and the unknown-tool fallback, and it+ # leaves non-model consumers (the propose_form_values JSON+ # parse above, the audit trail) reading the full text.tool_results = [{"type": "tool_result","tool_use_id": tu["id"],- "content": results_by_id.get(- tu["id"], f"Unknown tool: {tu['name']}"+ "content": _cap_tool_result(+ results_by_id.get(tu["id"], f"Unknown tool: {tu['name']}"),+ tool_name=tu["name"],+ limit=self._settings.tool_result_max_chars,),}for tu in tool_uses]current_messages.append({"role": "user", "content": tool_results})+ # Bounded to the messages this loop appended: current_messages+ # is a shallow copy, so anything at a lower index is still owned+ # by the caller and must not be mutated.+ if self._settings.enable_caching:+ _apply_loop_cache_marker(current_messages, caller_message_count)logger.warning("max_iterations=%d reached — returning partial result", max_iterations)yield last_full_text or "I was unable to complete the analysis. Please try again."diff --git a/packages/core/openexecutive/providers/feature_gate.py b/packages/core/openexecutive/providers/feature_gate.pyindex 6b33dc39..693e6da7 100644--- a/packages/core/openexecutive/providers/feature_gate.py+++ b/packages/core/openexecutive/providers/feature_gate.py@@ -82,8 +82,11 @@ def _strip_cache_control(kwargs: dict[str, Any]) -> None:if isinstance(t, dict):t.pop("cache_control", None)- # cache_control can also appear on user-turn content blocks for rolling- # cache. Strip it there too.+ # cache_control also appears on user-turn content blocks — notably the+ # agent loop's intra-turn marker, which rides on a tool_result block.+ # Strip it there too. One level deep is enough because every marker we+ # emit sits at a content block's own top level, never nested inside a+ # tool_result's own content list.messages = kwargs.get("messages")if isinstance(messages, list):for m in messages:diff --git a/packages/core/openexecutive/providers/translator.py b/packages/core/openexecutive/providers/translator.pyindex 0ee8af5a..3594caf9 100644--- a/packages/core/openexecutive/providers/translator.py+++ b/packages/core/openexecutive/providers/translator.py@@ -12,7 +12,8 @@ testable in isolation without an HTTP client or an event loop.What we DO NOT translate:* ``cache_control`` blocks pass through unchanged when present. We- preserve them on system content, user-turn content, and tool entries+ preserve them on system content, user-turn content, tool entries, and+ tool-result content (the agent loop's intra-turn breakpoint)so OpenRouter can forward the Anthropic cache hints to the upstreamAnthropic call (see OpenRouter prompt-caching docs:https://openrouter.ai/docs/guides/best-practices/prompt-caching).@@ -66,23 +67,33 @@ def _any_block_has_cache_control(blocks: Any) -> bool:)+def _typed_text_with_cc(text: str, cc: Any) -> dict[str, Any] | None:+ """Build one typed text block, carrying ``cache_control`` when it is real.++ The single place that decides how a cache marker is projected onto the+ wire, shared by the system/user text path (`_typed_text_block`) and the+ tool-result path. Pass the full ``cache_control`` dict through —+ ``ttl: "1h"`` and any future extension fields ride along untouched.+ OpenRouter forwards this to Anthropic verbatim.++ Returns ``None`` for empty text: an empty typed block carries no content+ to cache and risks an upstream 400.+ """+ if not isinstance(text, str) or not text:+ return None+ out: dict[str, Any] = {"type": "text", "text": text}+ if isinstance(cc, dict):+ out["cache_control"] = cc+ return out++def _typed_text_block(block: dict[str, Any]) -> dict[str, Any] | None:"""Project an Anthropic text block down to the typed-block shape OpenRouterforwards to Anthropic. Preserves ``cache_control`` (including the optional``ttl`` extension); drops anything else to keep the wire payload minimal."""if block.get("type") != "text":return None- txt = block.get("text", "")- if not isinstance(txt, str) or not txt:- return None- out: dict[str, Any] = {"type": "text", "text": txt}- cc = block.get("cache_control")- if isinstance(cc, dict):- # Pass the full cache_control dict through — ``ttl: "1h"`` and any- # future extension fields ride along untouched. OpenRouter forwards- # this to Anthropic verbatim.- out["cache_control"] = cc- return out+ return _typed_text_with_cc(block.get("text", ""), block.get("cache_control"))def _translate_system(system: Any) -> str | list[dict[str, Any]] | None:@@ -118,6 +129,37 @@ def _translate_system(system: Any) -> str | list[dict[str, Any]] | None:return str(system) or None+def _request_caches_tool_results(messages: list[Any]) -> bool:+ """True iff any ``tool_result`` block in the request carries a marker.++ Drives a UNIFORM tool-message shape for the whole request, which the+ prompt cache depends on. The agent loop's marker moves each iteration,+ so a given tool result is marked on the iteration it is newest and+ unmarked on every later one. If the shape followed the marker, that+ result would serialize as a typed array once and as a flat string+ afterwards — the cached prefix bytes would change under the very+ breakpoint that was supposed to read them, and every iteration would+ miss. (The ``cache_control`` key itself is not an invalidator; a+ content-shape change is.) So the decision is made per request, not per+ block: if anything in this request is cached, every tool message uses+ the typed form.+ """+ for m in messages:+ if not isinstance(m, dict):+ continue+ content = m.get("content")+ if not isinstance(content, list):+ continue+ for block in content:+ if (+ isinstance(block, dict)+ and block.get("type") == "tool_result"+ and isinstance(block.get("cache_control"), dict)+ ):+ return True+ return False++def _anthropic_messages_to_openai(messages: list[Any]) -> list[dict[str, Any]]:"""Convert Anthropic ``messages`` to OpenAI chat-completions ``messages``.@@ -127,11 +169,12 @@ def _anthropic_messages_to_openai(messages: list[Any]) -> list[dict[str, Any]]:tool results."""out: list[dict[str, Any]] = []+ typed_tool_results = _request_caches_tool_results(messages)for m in messages:role = m.get("role")content = m.get("content")if role == "user":- out.extend(_user_content_to_openai(content))+ out.extend(_user_content_to_openai(content, typed_tool_results))elif role == "assistant":out.append(_assistant_content_to_openai(content))else:@@ -154,7 +197,9 @@ def _content_to_text(content: Any) -> str:return ""-def _user_content_to_openai(content: Any) -> list[dict[str, Any]]:+def _user_content_to_openai(+ content: Any, typed_tool_results: bool = False+) -> list[dict[str, Any]]:"""User-turn content: emit one user message, plus one ``tool`` role messageper Anthropic ``tool_result`` block so the OpenAI chat history threadscorrectly through tool-use turns.@@ -191,11 +236,27 @@ def _user_content_to_openai(content: Any) -> list[dict[str, Any]]:tool_use_id = block.get("tool_use_id")inner = block.get("content")tool_text = _content_to_text(inner)+ # A cache_control marker on a tool_result is the agent loop's+ # intra-turn breakpoint (executive._apply_loop_cache_marker). It+ # must survive translation or the loop re-pays full input price+ # for its own transcript on every iteration. Carrying it on the+ # tool message keeps it AFTER the tool results in wire order —+ # a marker text block in the user message would be emitted+ # before them (tool messages are appended last, below).+ typed = _typed_text_with_cc(tool_text, block.get("cache_control"))+ # Shape is decided per REQUEST, not per block — see+ # _request_caches_tool_results. Following the marker would flip+ # this result between array and string as the marker moves,+ # changing the cached prefix out from under the breakpoint.+ # Uncached requests keep the flat string they have always sent.+ tool_content: Any = tool_text+ if typed_tool_results and typed is not None:+ tool_content = [typed]tool_messages.append({"role": "tool","tool_call_id": tool_use_id,- "content": tool_text,+ "content": tool_content,})@@ -205,9 +266,11 @@ def _user_content_to_openai(content: Any) -> list[dict[str, Any]]:elif text_chunks:# Fall back to the flat-string form even when ``preserve_typed`` is# True but ``text_blocks`` is empty — that happens when the only- # cache_control marker rides on a non-text block (e.g. tool_result).- # We don't want to drop the user's actual text just because we- # couldn't represent the marker.+ # cache_control marker rides on a non-text block. A tool_result+ # marker is carried on its own tool message above, so this is not+ # a dropped marker; it just means this user message has no text+ # block of its own to mark. We don't want to drop the user's+ # actual text just because the marker lives elsewhere.msgs.append({"role": "user", "content": "\n\n".join(text_chunks)})msgs.extend(tool_messages)return msgsdiff --git a/packages/core/tests/unit/test_executive_tool_loop_cache.py b/packages/core/tests/unit/test_executive_tool_loop_cache.pynew file mode 100644index 00000000..97aef0c0--- /dev/null+++ b/packages/core/tests/unit/test_executive_tool_loop_cache.py@@ -0,0 +1,407 @@+"""The agent loop must cache its own transcript as it grows.++Without an intra-turn breakpoint the loop caches only the static+tools+system prefix, so every iteration re-sends the whole accumulated+transcript at full input price. The runtime signature of that bug is+`cache_read_input_tokens` pinned flat while the prompt grows iteration+over iteration.++A fake provider cannot produce real cache reads, so this module pins the+*structure* that produces them — one marker, on the newest tool result,+moving each iteration — and the live check (a real two-iteration tool+loop, asserting cache_read grows) covers the wire behaviour. Simulating+cache accounting here would pass while production stayed broken, which is+precisely the silent regression being guarded against.++The contract pinned here:+ 1. the newest tool result carries the marker, and it is the last block+ of the last user message;+ 2. exactly one tool_result is marked per request — the marker MOVES,+ it does not accumulate (15 accumulated markers would blow the+ 4-breakpoint API limit);+ 3. total cache_control markers across system + tools + messages stay+ within the API limit of 4;+ 4. the loop marker is 5m (no `ttl` key) — the turn is over in seconds,+ so a 1h TTL would only double the write premium;+ 5. `enable_caching=False` emits no marker at all;+ 6. the caller's `messages` list is never mutated.+"""+from __future__ import annotations++import asyncio+import copy+from typing import Any+from unittest.mock import patch++import pytest++from openexecutive.orchestrator.executive import Executive++# --------------------------------------------------------------------- #+# Fake provider plumbing (same shape as test_executive_tool_error_isolation)+# --------------------------------------------------------------------- #+++class _TextBlock:+ type = "text"++ def __init__(self, text: str) -> None:+ self.text = text+++class _ToolUseBlock:+ type = "tool_use"++ def __init__(self, id_: str, name: str, input_: dict[str, Any]) -> None:+ self.id = id_+ self.name = name+ self.input = input_+++class _FinalMsg:+ usage = None # _emit_cache_event no-ops on usage=None++ def __init__(self, content: list[Any], stop_reason: str) -> None:+ self.content = content+ self.stop_reason = stop_reason+++class _FakeStream:+ def __init__(self, final_msg: _FinalMsg) -> None:+ self._final_msg = final_msg++ async def __aenter__(self) -> _FakeStream:+ return self++ async def __aexit__(self, *_a: Any) -> None:+ return None++ def __aiter__(self) -> _FakeStream:+ return self++ async def __anext__(self) -> Any:+ raise StopAsyncIteration++ async def get_final_message(self) -> _FinalMsg:+ return self._final_msg+++class _ScriptedProvider:+ """Records a deep snapshot of every request.++ The loop mutates one `current_messages` list in place and passes it by+ reference, so storing `kwargs` as-is would make every recorded call+ alias the same final state — and every per-call assertion below would+ silently pass regardless of what happened mid-loop. Snapshot at call+ time so "what did iteration N actually send" is a real question.+ """++ def __init__(self, final_msgs: list[_FinalMsg]) -> None:+ self._final_msgs = list(final_msgs)+ self.calls: list[dict[str, Any]] = []++ def messages_stream(self, **kwargs: Any) -> _FakeStream:+ self.calls.append(copy.deepcopy(kwargs))+ return _FakeStream(self._final_msgs.pop(0))++[email protected](autouse=True)+def _no_audit_writes(monkeypatch: pytest.MonkeyPatch) -> None:+ """`audit.log_event` otherwise writes to ./episodic_memory.db and leaks+ rows that break other modules' assertions in a full-suite run."""+ monkeypatch.setattr(+ "openexecutive.orchestrator.executive.audit_log",+ lambda *_a, **_k: None,+ )+++def _scripted_three_iterations() -> _ScriptedProvider:+ """tool_use -> tool_use -> text, i.e. two tool rounds then an answer."""+ return _ScriptedProvider(+ [+ _FinalMsg([_ToolUseBlock("tu_1", "search_knowledge", {"q": "a"})], "tool_use"),+ _FinalMsg([_ToolUseBlock("tu_2", "search_knowledge", {"q": "b"})], "tool_use"),+ _FinalMsg([_TextBlock("done")], "end_turn"),+ ]+ )+++def _run_loop(+ provider: _ScriptedProvider,+ messages: list[dict[str, Any]] | None = None,+ executive: Executive | None = None,+ system_blocks: list[dict[str, Any]] | None = None,+) -> list[Any]:+ exec_ = executive or Executive()+ msgs = messages if messages is not None else [+ {"role": "user", "content": "do the thing"}+ ]++ async def _go() -> list[Any]:+ items: list[Any] = []+ with patch(+ "openexecutive.orchestrator.executive.get_provider",+ return_value=provider,+ ):+ async for item in exec_._stream_agent_loop(+ system_blocks=system_blocks if system_blocks is not None else [],+ messages=msgs,+ model="claude-test",+ ):+ items.append(item)+ return items++ return asyncio.run(_go())+++def _marked_tool_results(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:+ """Every tool_result block in ``messages`` carrying a cache marker."""+ out: list[dict[str, Any]] = []+ for msg in messages:+ content = msg.get("content")+ if not isinstance(content, list):+ continue+ for block in content:+ if (+ isinstance(block, dict)+ and block.get("type") == "tool_result"+ and isinstance(block.get("cache_control"), dict)+ ):+ out.append(block)+ return out+++def _count_all_markers(call: dict[str, Any]) -> int:+ """cache_control markers across system + tools + messages for one request."""+ n = 0+ for block in call.get("system") or []:+ if isinstance(block, dict) and isinstance(block.get("cache_control"), dict):+ n += 1+ for tool in call.get("tools") or []:+ if isinstance(tool, dict) and isinstance(tool.get("cache_control"), dict):+ n += 1+ for msg in call.get("messages") or []:+ content = msg.get("content")+ if not isinstance(content, list):+ continue+ for block in content:+ if isinstance(block, dict) and isinstance(block.get("cache_control"), dict):+ n += 1+ return n+++# --------------------------------------------------------------------- #+# Tests+# --------------------------------------------------------------------- #+++def test_newest_tool_result_is_marked_and_is_the_last_block() -> None:+ provider = _scripted_three_iterations()+ _run_loop(provider)++ # Call 2 is the first request that carries a tool_result at all.+ messages = provider.calls[1]["messages"]+ marked = _marked_tool_results(messages)+ assert len(marked) == 1++ last_msg = messages[-1]+ assert last_msg["role"] == "user"+ assert last_msg["content"][-1]["tool_use_id"] == marked[0]["tool_use_id"], (+ "the breakpoint must sit on the newest tool result, otherwise the "+ "bytes after it are re-sent uncached on the next iteration"+ )+++def test_marker_moves_instead_of_accumulating() -> None:+ """The regression that blows the 4-breakpoint limit at max_iterations=15."""+ provider = _scripted_three_iterations()+ _run_loop(provider)++ call2_marked = _marked_tool_results(provider.calls[1]["messages"])+ call3_marked = _marked_tool_results(provider.calls[2]["messages"])++ assert len(call2_marked) == 1+ assert len(call3_marked) == 1, (+ "a second marker means the marker accumulated rather than moved"+ )+ # The block marked on call 2 must have been swept clean by call 3.+ assert call3_marked[0]["tool_use_id"] == "tu_2"+ assert call2_marked[0]["tool_use_id"] == "tu_1"+ prior = [+ b+ for msg in provider.calls[2]["messages"]+ if isinstance(msg.get("content"), list)+ for b in msg["content"]+ if isinstance(b, dict) and b.get("tool_use_id") == "tu_1"+ ]+ assert prior and "cache_control" not in prior[0]+++def test_total_markers_are_exactly_the_four_the_api_allows() -> None:+ """The live budget has ZERO headroom, so assert the real number.++ Passing `system_blocks=[]` would make this vacuous — 0 system + 1 tool+ + 1 loop = 2, and `<= 4` could never fail however the layout changed.+ Using the real `build_system_blocks` output pins the actual shipped+ budget: 2 system + 1 tool + 1 intra-turn = 4. A future third system+ block, or a second message-level marker, ships a 5-breakpoint request+ that Anthropic rejects outright — and this test goes red first.+ """+ from openexecutive.memory.company_profile import CompanyProfile+ from openexecutive.prompts.cache_manager import build_system_blocks++ system_blocks = build_system_blocks(+ company_profile=CompanyProfile(name="Test Co", industry="Testing"),+ )+ system_markers = sum(+ 1+ for b in system_blocks+ if isinstance(b, dict) and isinstance(b.get("cache_control"), dict)+ )+ assert system_markers == 2, (+ "cache_manager's documented budget is 2 system blocks; if this "+ "changed, the loop marker no longer fits under the API limit"+ )++ provider = _scripted_three_iterations()+ _run_loop(provider, system_blocks=system_blocks)++ # The iteration that carries a tool_result is the one at full budget.+ assert _count_all_markers(provider.calls[1]) == 4+ for i, call in enumerate(provider.calls):+ assert _count_all_markers(call) <= 4, f"call {i} exceeded the 4-block limit"+++def test_loop_marker_uses_the_default_five_minute_ttl() -> None:+ """A 1h TTL doubles the write premium; the turn is over in seconds."""+ provider = _scripted_three_iterations()+ _run_loop(provider)+ marked = _marked_tool_results(provider.calls[1]["messages"])+ assert marked[0]["cache_control"] == {"type": "ephemeral"}+ assert "ttl" not in marked[0]["cache_control"]+++def test_no_marker_when_caching_disabled(monkeypatch: pytest.MonkeyPatch) -> None:+ exec_ = Executive()+ monkeypatch.setattr(exec_._settings, "enable_caching", False)+ provider = _scripted_three_iterations()+ _run_loop(provider, executive=exec_)+ for call in provider.calls:+ assert _marked_tool_results(call["messages"]) == []+++def test_caller_messages_are_not_mutated() -> None:+ """The loop shallow-copies the caller's list; marking must not alias it.++ The caller message must contain a tool_result carrying a pre-existing+ marker — a flat-string message is skipped by the helper's+ `isinstance(content, list)` guard, so it would pass even if the sweep+ aggressively rewrote caller-owned blocks, which is the actual hazard+ the call-site comment reasons about.+ """+ caller_messages: list[dict[str, Any]] = [+ {"role": "user", "content": "do the thing"},+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": "caller_tu",+ "content": "caller-owned result",+ "cache_control": {"type": "ephemeral"},+ }+ ],+ },+ ]+ before = copy.deepcopy(caller_messages)+ provider = _scripted_three_iterations()+ _run_loop(provider, messages=caller_messages)+ assert caller_messages == before+++def test_build_messages_marks_no_history_turn() -> None:+ """The removed rolling-history marker was a latent 5th breakpoint.++ It lived in `_build_messages`, so this must assert on that method's+ output. Asserting on `_stream_agent_loop` (which takes `messages`+ pre-built) would pass with the deleted code fully restored — and+ restoring it ships a 5-breakpoint request Anthropic rejects outright.+ """+ from openexecutive.orchestrator.session import Session++ session = Session(session_id="s-hist")+ session.add_user_message("earlier question")+ session.add_assistant_message("earlier answer")+ session.add_user_message("second question")+ session.add_assistant_message("second answer")++ built = Executive()._build_messages(session, "do the thing")++ marked = [+ b+ for msg in built+ if isinstance(msg.get("content"), list)+ for b in msg["content"]+ if isinstance(b, dict) and "cache_control" in b+ ]+ assert marked == [], (+ "history turns must carry no cache_control — the budget's four "+ "slots are system x2 + tools + the intra-turn loop marker"+ )+++def test_marker_falls_back_when_the_newest_results_are_empty() -> None:+ """An all-empty iteration must still get a breakpoint.++ Losing it re-sends the whole accumulated transcript at full price for+ that step, which is the exact cost this feature exists to remove.+ """+ from openexecutive.orchestrator.executive import _apply_loop_cache_marker++ messages: list[dict[str, Any]] = [+ {+ "role": "user",+ "content": [+ {"type": "tool_result", "tool_use_id": "t1", "content": "real"}+ ],+ },+ {"role": "assistant", "content": [{"type": "text", "text": "x"}]},+ {+ "role": "user",+ "content": [{"type": "tool_result", "tool_use_id": "t2", "content": ""}],+ },+ ]+ _apply_loop_cache_marker(messages)++ marked = [+ b+ for msg in messages+ if isinstance(msg.get("content"), list)+ for b in msg["content"]+ if isinstance(b, dict) and "cache_control" in b+ ]+ assert len(marked) == 1+ assert marked[0]["tool_use_id"] == "t1"+++def test_non_string_tool_result_content_is_still_markable() -> None:+ """A typed/image result is real bytes in the prefix, so it must be+ able to carry the breakpoint — the cap deliberately passes non-str+ results through untouched, and dropping the marker for them would+ silently lose the cache hit for that whole iteration."""+ from openexecutive.orchestrator.executive import _apply_loop_cache_marker++ messages: list[dict[str, Any]] = [+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": "t1",+ "content": [{"type": "text", "text": "typed result"}],+ }+ ],+ }+ ]+ _apply_loop_cache_marker(messages)+ assert messages[0]["content"][0]["cache_control"] == {"type": "ephemeral"}diff --git a/packages/core/tests/unit/test_feature_gate.py b/packages/core/tests/unit/test_feature_gate.pyindex 245b0615..e17a3d30 100644--- a/packages/core/tests/unit/test_feature_gate.py+++ b/packages/core/tests/unit/test_feature_gate.py@@ -138,3 +138,37 @@ def test_strip_web_search_drops_tools_key_when_only_web_search_present() -> None}out = apply_feature_gates(_non_claude_spec(), kwargs)assert "tools" not in out+++def test_strips_cache_control_from_tool_result_blocks() -> None:+ """The agent loop's intra-turn breakpoint rides on a tool_result block.++ Before that loop existed, `messages` almost never carried a marker, so+ this branch of the strip was close to dead code. Now every tool-loop+ request carries one, and a non-Claude backend 400s on the unknown+ field — so the strip is load-bearing on every such request.++ The marker must stay at the tool_result block's own top level: the+ strip walks one level deep and does NOT recurse into a nested+ tool_result["content"] list, so moving it inside would silently leak+ it upstream.+ """+ kwargs = {+ "messages": [+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": "tu_1",+ "content": "output",+ "cache_control": {"type": "ephemeral"},+ }+ ],+ }+ ]+ }+ out = apply_feature_gates(_non_claude_spec(), kwargs)+ assert "cache_control" not in out["messages"][0]["content"][0]+ # Never mutates the caller's dict — the loop reuses it next iteration.+ assert "cache_control" in kwargs["messages"][0]["content"][0]diff --git a/packages/core/tests/unit/test_openrouter_translator.py b/packages/core/tests/unit/test_openrouter_translator.pyindex 9393731a..781e9f0e 100644--- a/packages/core/tests/unit/test_openrouter_translator.py+++ b/packages/core/tests/unit/test_openrouter_translator.py@@ -1114,3 +1114,238 @@ def test_stream_accumulator_drops_server_tool_call_continuation_chunks() -> Noneacc2.feed({"id": "s", "model": "m", "choices": [{"delta": {"content": "done"}, "finish_reason": "tool_calls"}]})final = acc2.finalize()assert [b.type for b in final.content] == ["text"] and final.stop_reason == "end_turn"+++# --------------------------------------------------------------------------+# tool_result cache_control — the agent loop's intra-turn breakpoint+# --------------------------------------------------------------------------+++def test_request_preserves_tool_result_cache_control_on_tool_message() -> None:+ """The agent loop marks the newest tool_result to cache its own+ transcript. Dropping that marker here is invisible in unit tests and+ costs full input price on every loop iteration in production, so the+ wire shape is pinned."""+ body = to_openai_request(+ "anthropic/claude-opus-4.7",+ {+ "messages": [+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": "tu_1",+ "content": "the tool output",+ "cache_control": {"type": "ephemeral"},+ }+ ],+ }+ ],+ },+ )+ tool_msgs = [m for m in body["messages"] if m["role"] == "tool"]+ assert len(tool_msgs) == 1+ assert tool_msgs[0]["tool_call_id"] == "tu_1"+ assert tool_msgs[0]["content"] == [+ {+ "type": "text",+ "text": "the tool output",+ "cache_control": {"type": "ephemeral"},+ }+ ]+++def test_request_tool_result_marker_lands_after_every_tool_message() -> None:+ """Ordering guard. A marker text block appended to the user message+ would be emitted BEFORE the tool messages (they are appended last),+ putting the breakpoint ahead of the very bytes it must cover. The+ marked message must be the last one for the turn."""+ body = to_openai_request(+ "anthropic/claude-opus-4.7",+ {+ "messages": [+ {+ "role": "user",+ "content": [+ {"type": "text", "text": "some user text"},+ {+ "type": "tool_result",+ "tool_use_id": "tu_a",+ "content": "first result",+ },+ {+ "type": "tool_result",+ "tool_use_id": "tu_b",+ "content": "second result",+ "cache_control": {"type": "ephemeral"},+ },+ ],+ }+ ],+ },+ )+ msgs = body["messages"]+ marked = [+ i+ for i, m in enumerate(msgs)+ if isinstance(m.get("content"), list)+ and any(+ isinstance(b, dict) and "cache_control" in b for b in m["content"]+ )+ ]+ assert len(marked) == 1+ assert marked[0] == len(msgs) - 1, "the breakpoint must be the last message"+ # And specifically after the unmarked sibling tool result.+ tu_a_index = next(+ i for i, m in enumerate(msgs) if m.get("tool_call_id") == "tu_a"+ )+ assert marked[0] > tu_a_index+++def test_request_unmarked_tool_result_stays_a_flat_string() -> None:+ """Every non-cached call keeps the legacy shape — no gratuitous+ array-form churn on the wire."""+ body = to_openai_request(+ "anthropic/claude-opus-4.7",+ {+ "messages": [+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": "tu_1",+ "content": "plain output",+ }+ ],+ }+ ],+ },+ )+ tool_msgs = [m for m in body["messages"] if m["role"] == "tool"]+ assert tool_msgs[0]["content"] == "plain output"+++def test_request_empty_tool_result_is_not_marked() -> None:+ """An empty typed text block carries nothing to cache and risks an+ upstream 400, so the marker is dropped rather than emitted empty."""+ body = to_openai_request(+ "anthropic/claude-opus-4.7",+ {+ "messages": [+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": "tu_1",+ "content": "",+ "cache_control": {"type": "ephemeral"},+ }+ ],+ }+ ],+ },+ )+ tool_msgs = [m for m in body["messages"] if m["role"] == "tool"]+ assert tool_msgs[0]["content"] == ""+ # The point of the test: the marker was dropped, not merely that the+ # content is empty (the pre-change code produced "" here too).+ assert not isinstance(tool_msgs[0]["content"], list)+ assert "cache_control" not in json.dumps(tool_msgs[0])+++def test_tool_result_wire_shape_is_stable_as_the_marker_moves() -> None:+ """The shape must not follow the marker.++ The agent loop's marker moves each iteration, so a given tool result+ is marked when it is newest and unmarked on every later iteration. If+ the wire shape followed the marker, that result would serialize as a+ typed array once and as a flat string afterwards — the cached prefix+ bytes would change underneath the very breakpoint meant to read them+ and every iteration would miss, making the whole feature a no-op on+ this path. (The `cache_control` key itself is not an invalidator; a+ content-shape change is.)+ """+ def body_for(marked_id: str) -> dict:+ return to_openai_request(+ "anthropic/claude-opus-4.7",+ {+ "messages": [+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": "tu_1",+ "content": "first result",+ **(+ {"cache_control": {"type": "ephemeral"}}+ if marked_id == "tu_1"+ else {}+ ),+ }+ ],+ },+ {"role": "assistant", "content": [{"type": "text", "text": "x"}]},+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": "tu_2",+ "content": "second result",+ **(+ {"cache_control": {"type": "ephemeral"}}+ if marked_id == "tu_2"+ else {}+ ),+ }+ ],+ },+ ]+ },+ )++ iter_n = body_for("tu_1")+ iter_n1 = body_for("tu_2")++ def tu1(body: dict) -> dict:+ return next(m for m in body["messages"] if m.get("tool_call_id") == "tu_1")++ # Strip the marker itself; everything else about tu_1 must be identical+ # across the two requests, or the prefix broke.+ a, b = dict(tu1(iter_n)), dict(tu1(iter_n1))+ for m in (a, b):+ if isinstance(m["content"], list):+ m["content"] = [+ {k: v for k, v in blk.items() if k != "cache_control"}+ for blk in m["content"]+ ]+ assert a == b, (+ "tu_1 serialized differently once the marker moved to tu_2 — the "+ "cached prefix changed and iteration N+1 cannot read N's write"+ )+++def test_uncached_request_keeps_the_legacy_flat_string_shape() -> None:+ """No marker anywhere -> every tool message keeps the shape it has+ always sent. The typed form is only paid for when caching is live."""+ body = to_openai_request(+ "anthropic/claude-opus-4.7",+ {+ "messages": [+ {+ "role": "user",+ "content": [+ {"type": "tool_result", "tool_use_id": "a", "content": "one"},+ {"type": "tool_result", "tool_use_id": "b", "content": "two"},+ ],+ }+ ]+ },+ )+ for m in body["messages"]:+ if m.get("role") == "tool":+ assert isinstance(m["content"], str)diff --git a/packages/core/tests/unit/test_tool_result_cap.py b/packages/core/tests/unit/test_tool_result_cap.pynew file mode 100644index 00000000..6143cf70--- /dev/null+++ b/packages/core/tests/unit/test_tool_result_cap.py@@ -0,0 +1,276 @@+"""A single tool result must not be able to dominate a turn.++An unbounded result (a large document fetch) lands in the context and is+then re-sent on every remaining iteration of the tool loop, so one call+can cost more than the rest of the turn combined. `_cap_tool_result` is a+circuit breaker against that — deliberately set high enough that ordinary+tool output never reaches it.++The contract pinned here:+ 1. under-limit results pass through byte-identically (the common case);+ 2. an over-limit result is cut to within the budget, marker included;+ 3. the marker names the tool, says the result is incomplete, and gives+ both character counts — silent truncation is worse than none, since+ the model reads the cut text as the whole answer;+ 4. the limit comes from settings, not a literal;+ 5. the cap sits DOWNSTREAM of the propose_form_values JSON parse, so an+ over-limit JSON result still produces its form_patch event. Capping+ at the producer instead would feed truncated JSON to json.loads and+ silently kill the SSE event.+"""+from __future__ import annotations++import asyncio+import json+from typing import Any+from unittest.mock import patch++import pytest++from openexecutive.orchestrator.executive import (+ _TOOL_NAME_MARKER_MAX,+ Executive,+ _cap_tool_result,+)++# --------------------------------------------------------------------- #+# _cap_tool_result unit behaviour+# --------------------------------------------------------------------- #+++def test_under_limit_result_is_unchanged() -> None:+ text = "a" * 100+ assert _cap_tool_result(text, tool_name="some_tool", limit=50_000) == text+++def test_non_string_result_passes_through() -> None:+ payload = {"not": "a string"}+ assert _cap_tool_result(payload, tool_name="some_tool", limit=10) is payload++[email protected]("limit", [1_000, 2_000, 9_999, 50_000])+def test_over_limit_result_never_exceeds_the_budget(limit: int) -> None:+ """The marker is reserved inside the budget, so the capped result is+ always <= limit — including when pct rounds up to three digits."""+ out = _cap_tool_result("x" * 500_000, tool_name="fetch_document", limit=limit)+ assert len(out) <= limit+ assert out.startswith("x" * 100)+++def test_marker_names_the_tool_and_both_counts() -> None:+ out = _cap_tool_result("x" * 50_000, tool_name="fetch_document", limit=2_000)++ assert "TRUNCATED" in out+ assert "fetch_document" in out, "the model must know which call to re-issue"+ assert "50,000" in out, "the original size must be stated"+ assert "NOT the full result" in out, "a bare ellipsis is reliably ignored"+ assert "narrower" in out, "a naive retry would re-trigger the cut"+++def test_limit_is_honoured_not_hardcoded() -> None:+ text = "y" * 5_000+ assert _cap_tool_result(text, tool_name="t", limit=50_000) == text+ assert _cap_tool_result(text, tool_name="t", limit=1_000) != text+++ "size",+ [+ 500, # a short JSON status result+ 4_000, # a specialist's analysis+ 20_000, # a substantial document excerpt+ 40_000, # the attachments extractor's own ceiling+ ],+)+def test_default_limit_does_not_fire_on_ordinary_output(size: int) -> None:+ """The breaker is insurance, not a routine clipper.++ Exercised behaviourally at the shipped default, across the range of+ result sizes real tools actually produce — a config-floor assertion+ alone would not notice the default being tuned down into the range+ where everyday document reads start getting cut.+ """+ from openexecutive.config import get_settings++ limit = get_settings().tool_result_max_chars+ text = "o" * size+ assert _cap_tool_result(text, tool_name="search_drive_files", limit=limit) == text+++# --------------------------------------------------------------------- #+# Placement: downstream of the form_patch JSON parse+# --------------------------------------------------------------------- #+++class _TextBlock:+ type = "text"++ def __init__(self, text: str) -> None:+ self.text = text+++class _ToolUseBlock:+ type = "tool_use"++ def __init__(self, id_: str, name: str, input_: dict[str, Any]) -> None:+ self.id = id_+ self.name = name+ self.input = input_+++class _FinalMsg:+ usage = None++ def __init__(self, content: list[Any], stop_reason: str) -> None:+ self.content = content+ self.stop_reason = stop_reason+++class _FakeStream:+ def __init__(self, final_msg: _FinalMsg) -> None:+ self._final_msg = final_msg++ async def __aenter__(self) -> _FakeStream:+ return self++ async def __aexit__(self, *_a: Any) -> None:+ return None++ def __aiter__(self) -> _FakeStream:+ return self++ async def __anext__(self) -> Any:+ raise StopAsyncIteration++ async def get_final_message(self) -> _FinalMsg:+ return self._final_msg+++class _ScriptedProvider:+ def __init__(self, final_msgs: list[_FinalMsg]) -> None:+ self._final_msgs = list(final_msgs)+ self.calls: list[dict[str, Any]] = []++ def messages_stream(self, **kwargs: Any) -> _FakeStream:+ import copy++ self.calls.append(copy.deepcopy(kwargs))+ return _FakeStream(self._final_msgs.pop(0))++[email protected](autouse=True)+def _no_audit_writes(monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setattr(+ "openexecutive.orchestrator.executive.audit_log",+ lambda *_a, **_k: None,+ )+++def test_oversized_json_result_still_emits_form_patch(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """The regression this placement exists to prevent.++ propose_form_values does `json.loads(result)` on the *uncapped* value.+ Capping upstream of that parse would raise JSONDecodeError, which the+ handler swallows — dropping the form_patch SSE event with no error+ anywhere. Capping at the prompt boundary keeps both correct.+ """+ from openexecutive.orchestrator.executive import PROPOSE_FORM_VALUES++ exec_ = Executive()+ monkeypatch.setattr(exec_._settings, "tool_result_max_chars", 1_000)++ # A valid-JSON handler result that comfortably exceeds the cap.+ big_values = {f"field_{i}": "v" * 100 for i in range(50)}+ handler_result = json.dumps({"values": big_values})+ assert len(handler_result) > 1_000++ provider = _ScriptedProvider(+ [+ _FinalMsg(+ [_ToolUseBlock("tu_1", PROPOSE_FORM_VALUES, {"values": big_values})],+ "tool_use",+ ),+ _FinalMsg([_TextBlock("done")], "end_turn"),+ ]+ )++ async def _fake_handler(*_a: Any, **_k: Any) -> str:+ return handler_result++ async def _go() -> list[Any]:+ items: list[Any] = []+ with patch(+ "openexecutive.orchestrator.executive.get_provider",+ return_value=provider,+ ), patch.dict(+ "openexecutive.orchestrator.executive._ALL_SKILL_HANDLERS",+ {PROPOSE_FORM_VALUES: _fake_handler},+ ):+ async for item in exec_._stream_agent_loop(+ system_blocks=[],+ messages=[{"role": "user", "content": "fill the form"}],+ model="claude-test",+ ):+ items.append(item)+ return items++ items = asyncio.run(_go())++ # The form_patch event survived the cap.+ patches = [+ it+ for it in items+ if isinstance(it, dict) and it.get("type") == "form_patch"+ ]+ assert patches, "capping upstream of json.loads would have dropped this"++ # ...and the prompt still got the capped text.+ sent = provider.calls[1]["messages"][-1]["content"][0]["content"]+ assert len(sent) <= 1_000+ assert "TRUNCATED" in sent+++# --------------------------------------------------------------------- #+# tool_name is model-supplied, therefore untrusted+# --------------------------------------------------------------------- #+++ "hostile_name",+ [+ "evil_{shown}_tool", # would substitute a local via str.format+ "evil_{0}_tool", # positional field reference+ "{", # unbalanced brace -> ValueError+ "}",+ "{limit}{pct}",+ "T" * 3_000, # would inflate the marker past the budget+ ],+)+def test_hostile_tool_name_neither_injects_nor_breaks_the_budget(+ hostile_name: str,+) -> None:+ """`tool_name` comes from the model's tool_use block, so prompt+ injection reaches it. It must never be interpreted as a format string+ (substitution or a raised ValueError that kills the turn), and must+ not be able to push the marker past the budget it fits inside."""+ out = _cap_tool_result("x" * 50_000, tool_name=hostile_name, limit=1_000)++ # Two things carry the weight here: no exception escaped (a raised+ # ValueError/KeyError from str.format would unwind the agent loop and+ # fail the whole turn), and the budget still held.+ assert len(out) <= 1_000, "a long tool name must not inflate the marker"+ assert "TRUNCATED" in out+ # The name is echoed as literal text, never interpreted as a field.+ # With str.format the brace forms below either raised or substituted;+ # as literals they survive verbatim (bounded to _TOOL_NAME_MARKER_MAX).+ literal = hostile_name[:_TOOL_NAME_MARKER_MAX]+ if "{" in hostile_name or "}" in hostile_name:+ assert literal in out, "braces must be literal text, not a format field"+++def test_hostile_tool_name_is_truncated_not_echoed_whole() -> None:+ out = _cap_tool_result("x" * 50_000, tool_name="N" * 500, limit=2_000)+ assert "N" * 500 not in out+ assert "N" * 80 in out