AI-powered virtual executive team — a single coherent executive persona backed by 8 specialist agents (FastAPI + Next.js).
perf(orchestrator): cache the conversation history between messages (#389)
Rufus2f9560b21c28
diff --git a/packages/core/openexecutive/architecture/architecture-facts.yaml b/packages/core/openexecutive/architecture/architecture-facts.yamlindex a9207804..fb35582f 100644--- a/packages/core/openexecutive/architecture/architecture-facts.yaml+++ b/packages/core/openexecutive/architecture/architecture-facts.yaml@@ -93,7 +93,7 @@ system:- A global operator pause (scheduler.pause, `executive_control` table) gates every autonomous loop — the scheduler, the Gmail poller and the workflow resumer. Inbound conversations (web chat, Slack, Discord, Telegram, Google Chat) are never gated. Pausing holds work, never cancels it.- Each specialist gets its own RAG fetch filtered by its declared domain alias.- System-prompt blocks marked with cache_control must remain byte-identical across turns — any dynamic content in them breaks Anthropic prompt caching and roughly 10× the cost.- - Total cache_control blocks must stay ≤ 4 (the Anthropic API hard limit). The current layout uses 2 of those slots for the system prompt (see `caching.block_order`), leaving headroom for the rolling-history ephemeral marker on the penultimate assistant turn.+ - Total cache_control blocks must stay ≤ 4 (the Anthropic API hard limit), with longer TTLs before shorter ones. The layout is tools 1h, persona 1h, then company 5m on a first message or the history marker 1h once there is history, then the agent loop's 5m marker (see `caching.breakpoint_budget`).- Tool definitions are sorted by name before being sent to the API so cache hits survive across processes.- Company-specific data lives under packages/core/company/ and is gitignored. Never commit it.- Audit writes never break a chat turn or tool call — exceptions are swallowed with a warning.@@ -130,15 +130,32 @@ caching:ephemeral markers the request uses).breakpoint_budget: |The full 4-block budget, and why none of it is spare:- 1. persona + knowledge (1h) — system block 0- 2. company + org (5m, optional) — system block 1- 3. last client tool (1h) — executive tool list+ 1. last client tool (1h) — executive tool list+ 2. persona + knowledge (1h) — system block 0+ 3. company + org (5m, optional) — system block 1, on a turn with no+ history; with history the slot goes to the history marker (1h) on+ the newest non-empty history turn (normally the previous reply)4. newest tool_result (5m) — the agent loop's intra-turn marker- History turns deliberately carry NO marker. A rolling marker on the- penultimate assistant turn used to claim a latent fifth breakpoint,- which Anthropic rejects outright on the direct (non-OpenRouter) path.- It never earned its slot: history turns are flat strings, and the- system + tool blocks already cover the expensive stable prefix.+ The history marker (executive._apply_history_cache_marker, applied by+ the agent loop, never by _build_messages) is what lets the next message+ read the conversation back. History turns are rebuilt from plain text,+ so the previous turn's loop entries (which ended inside that turn's+ retrieved context and tool calls) never match; without the marker every+ message re-wrote the whole history into the cache. The+ company block keeps its place in the prefix, covered by the history+ marker, and losing its own 5m marker keeps the TTLs in the order the API+ requires (1h before 5m). 1h because people reply in minutes; a 5m entry+ would mostly be gone. Session.get_recent_history moves its window's+ start in steps of 10 turns (20 to 30 turns kept) so the first history+ message, and with it the cached prefix, stays put between steps.+ On OpenRouter the translator keeps every assistant turn as typed text+ parts when a request carries the history marker+ (translator._request_caches_assistant_text), so the marker is forwarded+ and a reply has one shape whether marked or not.+ The committee revision call reuses the turn's messages without the+ history marker, so its request keeps the plain two-system-marker layout.+ A rolling marker on every history turn used to claim a latent fifth+ breakpoint, which Anthropic rejects outright on the direct path.block_order:- id: persona_and_knowledgettl: 1hdiff --git a/packages/core/openexecutive/architecture/prebuilt/caching.json b/packages/core/openexecutive/architecture/prebuilt/caching.jsonindex 5e4dd400..ae8f5538 100644--- a/packages/core/openexecutive/architecture/prebuilt/caching.json+++ b/packages/core/openexecutive/architecture/prebuilt/caching.json@@ -1,7 +1,7 @@{"section_id": "caching","title": "Prompt Caching",- "markdown": "The Executive system prompt is partitioned for Anthropic prompt caching by `openexecutive.prompts.cache_manager.build_system_blocks`. It emits at most two `cache_control` blocks, consolidated down from four to stay within Anthropic's 4-block API limit alongside the tool definitions and the agent loop's intra-turn marker.\n\n- **Block 0 — persona + knowledge (1h TTL, always present):** the executive persona constant — `EXECUTIVE_PERSONA_PROMPT` for a team, `EXECUTIVE_PERSONA_SOLO_PROMPT` in solo mode — or an admin override, which wins as-is in either mode, with the `{VOICE_PERSONA}` substitution, then the admin's Council **additional instructions** for the Executive (`persona_instructions`, wrapped in `<additional_instructions>`; blank leaves the block byte-identical, and like the override it changes only on save), conditional `WEB_SEARCH_ADDENDUM` / `MCP_ADDENDUM`, the constant `TOOL_BUILDING_ADDENDUM` (when to build a tool with `run_script`, check `list_saved_tools` first, keep what will be needed again, and talk about it without saying \"script\"; present with a gateway unless `CHAT_SCRIPTS=false`, a setting, so constant per deployment), the identity addendum (exec name/email, impersonation guard), the *Connected Systems* section, the constant `DELEGATION_ADDENDUM` while anyone on the install has Act as me on, the timezone addendum, and `KNOWLEDGE_INDEX_SUMMARY`. The timezone addendum names the user's zone, read fresh on every build from the workspace settings (`memory.workspace_settings.get_user_timezone`: the workspace's zone, else `USER_TIMEZONE`, else UTC). Every component is a constant or changes only when an admin or the user changes a setting, so the TTL is long. Setting a new time zone changes the addendum, so the 1h block misses once on that change and is warm again from the next turn — accepted, since a stale zone would mis-schedule follow-ups. Act as me works the same way: its addendum is keyed on the install-level setting (`delegation.settings.block0_delegation_on`), never on the turn or the speaker, so switching it on or off misses the 1h block once; off, block 0 is byte-identical to an install without the feature.\n The *Connected Systems* section (`prompts.connected_systems.render_connected_systems`) tells the model what is connected so it never has to discover it: Google Workspace on or off (on only when `google_workspace` is among the servers the running gateway started with, `MCPGateway.server_names`, so a gateway that failed to start never reads as connected), and when on, how to act from its own account plus the pinned Google tool names by service (`GOOGLE_TOOL_MANIFEST`: Gmail, Calendar, Drive, Docs, Sheets, as workspace-mcp 1.29.0 names them) to call directly; any other configured MCP servers by name (`_comment` keys skipped); Slack / Discord / Telegram / Google Chat on or off from their settings (names only, never a token); Notion knowledge sync; a Confluence knowledge sync line only when that sync is on (a setting, so constant per deployment); and one constant Act as me line that points at `DELEGATION_ADDENDUM`, so switching Act as me still changes block 0 by that addendum alone. The \"never ask which address to send from\" guidance lives here now, shown only while Google is connected; the identity addendum keeps the name, address and impersonation guard. Every input is fixed per process and the output is sorted, so block 0 stays warm; it is appended even under a persona override. It replaces what the model used to learn from `search_tools`, an uncached proxy whose results were gone by the next turn.\n- **Block 1 — company + org (5m TTL, only if non-empty):** `company_profile.to_prompt_block()` plus `render_org_block(mode=…)` — for a team: departments, heads, principals, charters and the People roster; in solo: the principal's own line, their role lines when set (a static lead-in marking the role private to the principal, then kind in plain words, title, reports to, remit, measured on — from the role pinned for the turn: the workspace settings, or a session's `principal_role` override), and their goals grouped by area. The role is set once per install and changes only when the principal edits it, so it is as stable as the rest of the block. Shorter TTL so org edits propagate quickly. The principal's contacts are private to them, so `build_system_blocks(include_contacts=...)` adds the `## Contacts` section, after the team or solo block, only on the principal's own verified turn (`executive._contacts_in_prompt`). That gives block 1 exactly two stable variants per mode — with and without contacts — and never anything per-request; every other turn's block is byte-identical to an install with no contacts.\n\n## Solo mode: one prefix per mode\n\nSolo mode — one person using Open Executive for themselves — changes three cached parts, all resolved **once per turn** by `pin_turn_workspace_mode(session)` (the session's override, else the workspace setting) so they always agree: the block-0 persona, the block-1 org block, and the tool list, which drops `send_company_broadcast`, `send_department_message` and `set_department_head` *before* the sort, so it stays sorted and keeps its 1h marker on the last tool. The resolved mode is pinned on the session for the rest of the turn, so the tool handlers use it too and a switch made mid-turn applies from the next turn. Every part is a constant or stable per install, so each mode keeps its own warm cache and switching modes misses once.\n\nThe two personas share most of their text. `prompts/executive_persona.py` holds plain string literals — a shared head, body and tail plus team or solo sections — concatenated at import and never formatted, so both prompts are constants. The team prompt is byte-for-byte what it was before the split; a test pins its sha256.\n\nTool schemas are static: nothing in a definition depends on company data — `create_goal` takes its area as free text rather than an enum of the current departments, which would change the cached tool prefix every time an area was added, and `record_decision_outcome` takes a plain decision id. The solo persona is still one constant: its goal and decision-outcome lines (`_SOLO_CREATE_GOAL`, `_SOLO_DECISION_OUTCOME`) are literals concatenated at import, and the team persona is byte-identical (a test pins its sha256). The scheduler's proactive-trigger run (a Session marked `unattended`) is sent the list without `schedule_tools.UNATTENDED_WITHHELD_TOOLS`, so it has a stable tool prefix of its own, as each mode does. So does a turn private to the principal (`Session.private_to_principal`: mail from one of their contacts, or mail they forwarded): it is sent the list without `schedule_tools.PRIVATE_TURN_WITHHELD_TOOLS`, dropped before the sort, so private turns in a mode share one sorted prefix with its marker on the last tool. Act as me adds tools rather than dropping any: on a turn `delegation.settings.pin_turn_delegation` offered it to (the principal's own verified, attended turn in a private conversation, with it on), `ghostwrite_email`, the four mailbox reads (`search_my_email`, `read_my_email`, `read_my_email_attachment`, `my_email_awaiting_reply`) and `remind_me` join the pool together before the sort, so those turns share one more sorted prefix and every other turn's list is byte-identical to before. The offer never depends on the Gmail connection, so a lapsed token never flips the prefix. `recall_history` (Always in the loop) joins the same way, before the sort, on a turn whose speaker may read their own notes; its offer reads only their switch and the surface, never the notes themselves. The chat note-taker runs after the turn, outside the cached prompt. Tool definitions are sorted by name so their serialized order is stable across requests; the `web_search` server tool is appended after the sorted list so it never receives a `cache_control` marker. The last client tool carries the third marker (1h TTL).\n\n## The intra-turn marker\n\nThe fourth and final breakpoint belongs to the tool-use loop. After each iteration appends its `tool_result` blocks, `executive._apply_loop_cache_marker` moves a single ephemeral marker onto the **newest non-empty tool result**. Without it the loop caches only the static tools+system prefix, so every iteration re-sends the whole accumulated transcript at full input price — the runtime signature is `cache_read_input_tokens` pinned flat while the prompt grows iteration over iteration.\n\nThe marker **moves rather than accumulates**: each pass first sweeps every `tool_result` clean, then re-marks the newest one. A marker left in place per iteration would reach `max_iterations` breakpoints and blow the 4-block limit. Sweeping first also makes the helper idempotent — there is no bookkeeping index to drift.\n\nHistory turns carry **no** marker. A rolling marker on the penultimate assistant turn used to claim a latent fifth breakpoint, which Anthropic rejects outright on the direct path; it never earned its slot, since history turns are flat strings and the system + tool blocks already cover the expensive stable prefix.\n\n**The breakpoint budget, in full:** persona+knowledge (1h) + company+org (5m, conditional) + last client tool (1h) + intra-turn tool result (5m) = at most 4.\n\nRAG context is **never** placed in a cached system block — retrieved hits vary per query and would poison the cache. It is injected into the **user turn** instead.\n\nSpecialist calls follow the same rule. Each specialist caches only its static domain prompt (`BaseAgent.analyze`, one ephemeral block); everything per call — retrieved knowledge, failure cases, past decisions, department memory, the company's stage (`<company_stage>`) and, in solo mode, the principal's role (`<principal_role>`) — goes in the specialist's user turn, so the cached domain prompt is byte-identical across companies and profile edits.\n\nThe ghostwriter's call (Act as me) carries no cache markers: its system prompt is the constant `GHOSTWRITER_PROMPT` plus that person's voice block — short, and sent only when a draft is written — and the thread and the intent go in its user turn.\n\n## Tool-result bound\n\n`TOOL_RESULT_MAX_CHARS` (default 50,000) caps a single tool result before it enters the prompt, applied at the one point every result reaches the model rather than at each producer — so it also covers specialist output, tool errors and the unknown-tool fallback, while non-model consumers (the `propose_form_values` JSON parse, the audit trail) still read the full text. It is a circuit breaker, not a routine clipper: the default is set so ordinary tool output never reaches it. A cut result carries an explicit marker naming the tool and stating that the result is incomplete — truncating silently is worse than not truncating, because the model reads the cut text as the whole answer.\n\n**The invariant:** never put dynamic content in a `cache_control` block. A single mutating value breaks the cache and drives roughly a 10x cost increase.",- "mermaid": "flowchart TD\n Tools[\"Tool defs (sorted by name)<br/>last one: cache_control 1h\"] --> B0\n Mode{{\"effective_workspace_mode<br/>(team or solo, once per turn)\"}} --> Tools\n Mode --> B0\n Mode --> B1\n Deleg{{\"Act as me on?<br/>(install-level, not per turn)\"}} --> B0\n Offer{{\"ghostwrite_email offered?<br/>(pinned once per turn)\"}} --> Tools\n B0[\"Block 0: persona (team or solo) + knowledge<br/>cache_control 1h\"] --> B1\n B1[\"Block 1: company + org (team or solo render)<br/>cache_control 5m\"] --> Limit{{\"≤ 4 cache_control blocks\"}}\n Loop[\"Newest tool_result<br/>cache_control 5m (moves each iteration)\"] --> Limit\n Hist[\"History turns<br/>(no marker)\"] --> Turn\n Limit --> Turn[\"User turn\"]\n RAG[\"RAG hits<br/>(vary per query)\"] --> Turn\n Turn --> API[\"Anthropic API\"]\n class B0 cache\n class B1 cache\n class RAG hot\n class Turn hot\n class Tools compute\n class Mode compute\n class Loop cache\n class API external\n class Deleg compute\n class Offer compute",- "generated_at": "2026-10-08T10:00:00Z"+ "markdown": "The Executive system prompt is partitioned for Anthropic prompt caching by `openexecutive.prompts.cache_manager.build_system_blocks`. It emits at most two `cache_control` blocks, consolidated down from four to stay within Anthropic's 4-block API limit alongside the tool definitions and the agent loop's intra-turn marker.\n\n- **Block 0 — persona + knowledge (1h TTL, always present):** the executive persona constant — `EXECUTIVE_PERSONA_PROMPT` for a team, `EXECUTIVE_PERSONA_SOLO_PROMPT` in solo mode — or an admin override, which wins as-is in either mode, with the `{VOICE_PERSONA}` substitution, then the admin's Council **additional instructions** for the Executive (`persona_instructions`, wrapped in `<additional_instructions>`; blank leaves the block byte-identical, and like the override it changes only on save), conditional `WEB_SEARCH_ADDENDUM` / `MCP_ADDENDUM`, the constant `TOOL_BUILDING_ADDENDUM` (when to build a tool with `run_script`, check `list_saved_tools` first, keep what will be needed again, and talk about it without saying \"script\"; present with a gateway unless `CHAT_SCRIPTS=false`, a setting, so constant per deployment), the identity addendum (exec name/email, impersonation guard), the *Connected Systems* section, the constant `DELEGATION_ADDENDUM` while anyone on the install has Act as me on, the timezone addendum, and `KNOWLEDGE_INDEX_SUMMARY`. The timezone addendum names the user's zone, read fresh on every build from the workspace settings (`memory.workspace_settings.get_user_timezone`: the workspace's zone, else `USER_TIMEZONE`, else UTC). Every component is a constant or changes only when an admin or the user changes a setting, so the TTL is long. Setting a new time zone changes the addendum, so the 1h block misses once on that change and is warm again from the next turn — accepted, since a stale zone would mis-schedule follow-ups. Act as me works the same way: its addendum is keyed on the install-level setting (`delegation.settings.block0_delegation_on`), never on the turn or the speaker, so switching it on or off misses the 1h block once; off, block 0 is byte-identical to an install without the feature.\n The *Connected Systems* section (`prompts.connected_systems.render_connected_systems`) tells the model what is connected so it never has to discover it: Google Workspace on or off (on only when `google_workspace` is among the servers the running gateway started with, `MCPGateway.server_names`, so a gateway that failed to start never reads as connected), and when on, how to act from its own account plus the pinned Google tool names by service (`GOOGLE_TOOL_MANIFEST`: Gmail, Calendar, Drive, Docs, Sheets, as workspace-mcp 1.29.0 names them) to call directly; any other configured MCP servers by name (`_comment` keys skipped); Slack / Discord / Telegram / Google Chat on or off from their settings (names only, never a token); Notion knowledge sync; a Confluence knowledge sync line only when that sync is on (a setting, so constant per deployment); and one constant Act as me line that points at `DELEGATION_ADDENDUM`, so switching Act as me still changes block 0 by that addendum alone. The \"never ask which address to send from\" guidance lives here now, shown only while Google is connected; the identity addendum keeps the name, address and impersonation guard. Every input is fixed per process and the output is sorted, so block 0 stays warm; it is appended even under a persona override. It replaces what the model used to learn from `search_tools`, an uncached proxy whose results were gone by the next turn.\n- **Block 1 — company + org (5m TTL, only if non-empty):** `company_profile.to_prompt_block()` plus `render_org_block(mode=…)` — for a team: departments, heads, principals, charters and the People roster; in solo: the principal's own line, their role lines when set (a static lead-in marking the role private to the principal, then kind in plain words, title, reports to, remit, measured on — from the role pinned for the turn: the workspace settings, or a session's `principal_role` override), and their goals grouped by area. The role is set once per install and changes only when the principal edits it, so it is as stable as the rest of the block. Shorter TTL so org edits propagate quickly. The principal's contacts are private to them, so `build_system_blocks(include_contacts=...)` adds the `## Contacts` section, after the team or solo block, only on the principal's own verified turn (`executive._contacts_in_prompt`). That gives block 1 exactly two stable variants per mode — with and without contacts — and never anything per-request; every other turn's block is byte-identical to an install with no contacts.\n\n## Solo mode: one prefix per mode\n\nSolo mode — one person using Open Executive for themselves — changes three cached parts, all resolved **once per turn** by `pin_turn_workspace_mode(session)` (the session's override, else the workspace setting) so they always agree: the block-0 persona, the block-1 org block, and the tool list, which drops `send_company_broadcast`, `send_department_message` and `set_department_head` *before* the sort, so it stays sorted and keeps its 1h marker on the last tool. The resolved mode is pinned on the session for the rest of the turn, so the tool handlers use it too and a switch made mid-turn applies from the next turn. Every part is a constant or stable per install, so each mode keeps its own warm cache and switching modes misses once.\n\nThe two personas share most of their text. `prompts/executive_persona.py` holds plain string literals — a shared head, body and tail plus team or solo sections — concatenated at import and never formatted, so both prompts are constants. The team prompt is byte-for-byte what it was before the split; a test pins its sha256.\n\nTool schemas are static: nothing in a definition depends on company data — `create_goal` takes its area as free text rather than an enum of the current departments, which would change the cached tool prefix every time an area was added, and `record_decision_outcome` takes a plain decision id. The solo persona is still one constant: its goal and decision-outcome lines (`_SOLO_CREATE_GOAL`, `_SOLO_DECISION_OUTCOME`) are literals concatenated at import, and the team persona is byte-identical (a test pins its sha256). The scheduler's proactive-trigger run (a Session marked `unattended`) is sent the list without `schedule_tools.UNATTENDED_WITHHELD_TOOLS`, so it has a stable tool prefix of its own, as each mode does. So does a turn private to the principal (`Session.private_to_principal`: mail from one of their contacts, or mail they forwarded): it is sent the list without `schedule_tools.PRIVATE_TURN_WITHHELD_TOOLS`, dropped before the sort, so private turns in a mode share one sorted prefix with its marker on the last tool. Act as me adds tools rather than dropping any: on a turn `delegation.settings.pin_turn_delegation` offered it to (the principal's own verified, attended turn in a private conversation, with it on), `ghostwrite_email`, the four mailbox reads (`search_my_email`, `read_my_email`, `read_my_email_attachment`, `my_email_awaiting_reply`) and `remind_me` join the pool together before the sort, so those turns share one more sorted prefix and every other turn's list is byte-identical to before. The offer never depends on the Gmail connection, so a lapsed token never flips the prefix. `recall_history` (Always in the loop) joins the same way, before the sort, on a turn whose speaker may read their own notes; its offer reads only their switch and the surface, never the notes themselves. The chat note-taker runs after the turn, outside the cached prompt. Tool definitions are sorted by name so their serialized order is stable across requests; the `web_search` server tool is appended after the sorted list so it never receives a `cache_control` marker. The last client tool carries the third marker (1h TTL).\n\n## The intra-turn marker\n\nThe fourth and final breakpoint belongs to the tool-use loop. After each iteration appends its `tool_result` blocks, `executive._apply_loop_cache_marker` moves a single ephemeral marker onto the **newest non-empty tool result**. Without it the loop caches only the static tools+system prefix, so every iteration re-sends the whole accumulated transcript at full input price — the runtime signature is `cache_read_input_tokens` pinned flat while the prompt grows iteration over iteration.\n\nThe marker **moves rather than accumulates**: each pass first sweeps every `tool_result` clean, then re-marks the newest one. A marker left in place per iteration would reach `max_iterations` breakpoints and blow the 4-block limit. Sweeping first also makes the helper idempotent — there is no bookkeeping index to drift.\n\n## The history marker\n\nOnce a conversation has history, the agent loop also marks the **newest non-empty history turn** (normally the previous reply) with a **1h** breakpoint (`executive._apply_history_cache_marker`), so the next message reads the whole conversation back from cache and writes only what is new: the last exchange, this turn's context and its tool calls. History turns are rebuilt from plain text, so the previous turn's loop entries, which ended inside that turn's retrieved context and tool results, never match; without this marker every message re-wrote the whole conversation into the cache. 1h because people reply in minutes, not seconds.\n\nThe history marker takes the company block's slot: block 1 stays in the prefix, covered by the history marker, and dropping its 5m marker keeps the TTLs in the order the API requires, longer before shorter. On a first message (no history) block 1 keeps its 5m marker as before. `_build_messages` itself never marks history, so the committee revision call, which reuses those messages, keeps the plain layout. `Session.get_recent_history` moves its window's start in steps of 10 turns (keeping 20 to 30), so a long conversation's first history message, and the cached prefix with it, stays put between steps instead of shifting every turn. On the OpenRouter path the translator keeps assistant turns as typed text parts whenever a request carries the history marker (`translator._request_caches_assistant_text`), so the marker reaches Anthropic and every reply has the same shape whether or not it is the one marked. A rolling marker on every history turn used to claim a latent fifth breakpoint, which Anthropic rejects outright.\n\n**The breakpoint budget, in full:** last client tool (1h) + persona+knowledge (1h) + company+org (5m, first message only) or history (1h) + intra-turn tool result (5m) = at most 4, longer TTLs first.\n\nRAG context is **never** placed in a cached system block — retrieved hits vary per query and would poison the cache. It is injected into the **user turn** instead.\n\nSpecialist calls follow the same rule. Each specialist caches only its static domain prompt (`BaseAgent.analyze`, one ephemeral block); everything per call — retrieved knowledge, failure cases, past decisions, department memory, the company's stage (`<company_stage>`) and, in solo mode, the principal's role (`<principal_role>`) — goes in the specialist's user turn, so the cached domain prompt is byte-identical across companies and profile edits.\n\nThe ghostwriter's call (Act as me) carries no cache markers: its system prompt is the constant `GHOSTWRITER_PROMPT` plus that person's voice block — short, and sent only when a draft is written — and the thread and the intent go in its user turn.\n\n## Tool-result bound\n\n`TOOL_RESULT_MAX_CHARS` (default 50,000) caps a single tool result before it enters the prompt, applied at the one point every result reaches the model rather than at each producer — so it also covers specialist output, tool errors and the unknown-tool fallback, while non-model consumers (the `propose_form_values` JSON parse, the audit trail) still read the full text. It is a circuit breaker, not a routine clipper: the default is set so ordinary tool output never reaches it. A cut result carries an explicit marker naming the tool and stating that the result is incomplete — truncating silently is worse than not truncating, because the model reads the cut text as the whole answer.\n\n**The invariant:** never put dynamic content in a `cache_control` block. A single mutating value breaks the cache and drives roughly a 10x cost increase.",+ "mermaid": "flowchart TD\n Tools[\"Tool defs (sorted by name)<br/>last one: cache_control 1h\"] --> B0\n Mode{{\"effective_workspace_mode<br/>(team or solo, once per turn)\"}} --> Tools\n Mode --> B0\n Mode --> B1\n Deleg{{\"Act as me on?<br/>(install-level, not per turn)\"}} --> B0\n Offer{{\"ghostwrite_email offered?<br/>(pinned once per turn)\"}} --> Tools\n B0[\"Block 0: persona (team or solo) + knowledge<br/>cache_control 1h\"] --> B1\n B1[\"Block 1: company + org (team or solo render)<br/>cache_control 5m on a first message\"] --> Limit{{\"≤ 4 cache_control blocks\"}}\n Loop[\"Newest tool_result<br/>cache_control 5m (moves each iteration)\"] --> Limit\n Hist[\"Newest history turn<br/>cache_control 1h (replaces block 1 marker)\"] --> Limit\n Limit --> Turn[\"User turn\"]\n RAG[\"RAG hits<br/>(vary per query)\"] --> Turn\n Turn --> API[\"Anthropic API\"]\n class B0 cache\n class B1 cache\n class RAG hot\n class Turn hot\n class Tools compute\n class Mode compute\n class Loop cache\n class API external\n class Deleg compute\n class Offer compute",+ "generated_at": "2026-10-08T11:30:00Z"}diff --git a/packages/core/openexecutive/architecture/prebuilt/lifecycle.json b/packages/core/openexecutive/architecture/prebuilt/lifecycle.jsonindex fe4a9310..a94e1798 100644--- a/packages/core/openexecutive/architecture/prebuilt/lifecycle.json+++ b/packages/core/openexecutive/architecture/prebuilt/lifecycle.json@@ -1,7 +1,7 @@{"section_id": "lifecycle","title": "Request Lifecycle",- "markdown": "A single chat message makes a full round-trip through the API, the orchestrator's tool-use loop, a parallel specialist fan-out, and synthesis — all streamed back as server-sent events.\n\n### Steps\n\n1. **Arrival** — the client mints a `client_turn_id` and posts to `POST /chat` (`openexecutive.api.routes.chat`). The turn carries a `<current_speaker>` hint resolved from the caller's `person_id` (followed by that person's `<working_style>` rules when they have any — see the attunement section), plus a `<briefing>` digest of current open alerts (built in parallel with RAG/episodic/peer-memory fetches) so the Executive can discuss any item shown on the `/today` briefing page. A turn arriving from a chat adapter instead of the web app also carries a `<channel>` block naming the surface and stating which approvals can be completed there; on Slack the `<briefing>` digest is gated to the principal's DMs. All three ride in the **user turn**, never a cached system block, as does a `<drive_memory>` block naming the Drive files this conversation already found or opened and the Drive searches that matched nothing (see the memory section). The server registers a stop switch under that id *before* the context fetch, so a Stop pressed during those first seconds halts the turn before any model call is made.\n2. **Tool-use loop** — the `Executive` calls Claude. For a cross-domain question the assistant turn emits multiple `consult_specialist` tool-use blocks at once.\n The person can keep typing while it works. A message sent then (`POST /chat/add`) joins the running turn: the loop adds it after the next round's tool results, so the answer follows it, and it is saved as its own message before the reply. One that arrives while the final answer is being written becomes the next turn instead. A conversation runs one turn at a time.\n3. **Parallel fan-out** — `openexecutive.orchestrator.router.route_parallel` dispatches those calls concurrently, bounded by `MAX_PARALLEL_SPECIALISTS`; calls past the cap get a skip tool_result. A specialist that raises (a timeout, an overloaded model) or returns no text no longer fails the batch: its tool_result says it is unavailable and tells the Executive not to invent its view, the others' results stand, and the turn goes on. It does not count as consulted, so it takes no committee seat and no department memory records the turn, and its documents are left out of the answer's sources. On the web chat the reply need not mention the gap, because the app shows it under the answer; on every other channel the tool_result asks the Executive to say in one sentence that part of the analysis is missing. A knowledge retrieval that fails likewise leaves that specialist without context rather than failing the turn.\n4. **Per-specialist RAG** — each specialist runs its own domain-filtered ChromaDB query, then reasons over the retrieved chunks and returns analysis as a tool_result. Its user turn also carries a `<company_stage>` tag from the session's company profile, so stage-sensitive advice (venture benchmarks vs. cash-first for a bootstrapped company) does not depend on the Executive remembering to mention it. Once the batch is done, the documents each answering specialist's retrieval returned are named, in call order, into the turn's `TurnSources` (`orchestrator/answer_sources.py`), which the web chat route owns and which also holds its own search's documents. Synced Google Drive files are recorded as `drive` sources with their Drive link, and the web chat lists them under **Google Drive**.\n5. **Synthesis** — the Executive feeds the tool_results back to Claude and streams one coherent reply. Every tool round — not just a specialist one — emits an `activity` event naming what is running, immediately followed by a `thinking` keepalive. The label names the *activity*, never the internal agent: \"Putting time on the calendar…\", \"Looking up people…\", or, for an MCP call, the underlying tool by name. A round that really does fan out collapses to the generic \"Consulting specialists…\" — the one voice holds even in the progress line. The keepalive carries no tool information of its own; it used to drive a hardcoded \"Consulting specialists…\" string, so an MCP or calendar round misreported itself as a fan-out.\n6. **Optional committee review** — when enabled, `Executive.stream_chat_with_committee()` runs a silent draft, parallel reviewer critiques, then a streamed revision pass.\n7. **Side effects** — after the reply, every rostered speaker's turn (not only the principal's) schedules a fire-and-forget **open-loop pass** (Attunement) beside the episodic extraction: a keyword prefilter, then one forced-tool call that may open or close loops, every item quote-gated against the speaker's own words. The user message is persisted with `sender_person_id` — the resolved speaker, never the principal fallback a Discord / Telegram session uses for ownership — and the session records the speaker for the turn (`Session.caller_person_id`), which `close_open_loop` reads. A DM sent while proactive work is tagged (`attunement.outcomes.tag_proactive` — scheduler dispatch, reflection, research, alert review) also opens an outcome-ledger row at `_record_outbound_context`; a user's own turn is never tagged. Any side-effecting tool call surfaces an `action_taken` chip, which unlike the ephemeral `activity` label is persisted with the assistant message; once the reply is over — also when it was stopped or timed out — the web chat route sends one `sources` event: the documents and web pages the turn looked at (its own search, each specialist that answered, web search results, and the pages the reply cites, which are always kept) and the areas (\"finance\", never a specialist) it had to go without. The web chat shows it under the reply, and it is persisted with the assistant message as `chat_messages.sources`; other channels don't collect it. The stream closes with a `done` event — preceded by a `stopped` event when the user halted the turn, in which case the partial reply is still persisted and flagged so it reads as cut short rather than complete, and is mirrored into the live in-memory session so the next turn's prompt matches what was saved. The stream still ends with `done` event. One side-effecting tool is gated server-side rather than by its description: `ack_alert` will only accept an `alert_id` from the live alert board the server itself derived this turn — the whole live set, not only the rows the `<briefing>` digest printed, since `/today` renders more cards than the digest lists and every one of them is Discuss-able — and it fails closed, so an id it cannot even parse is refused rather than waved through to the handler's own parse. That check runs on **every** session. It used to run only on turns that arrived from a chat channel, keyed on `origin_channel`, which the browser never sets — so the web, the surface the principal actually uses, was unchecked and the model could ack any id, including one quoted out of an alert body by the briefing page's Discuss handoff. What that buys is bounded and worth stating plainly: an id that is invented, closed, snoozed or expired is refused, including one quoted out of an attacker-written alert body; the model being argued into acking the wrong *live* card is not prevented. The one way to widen that set is `find_alerts`, for an item the principal names that has left the live board (already opened, snoozed, older than the board): a keyword search over alerts of every status whose own results — never an id the model supplies — join the trusted set. It runs only in a turn where `render_and_trust` showed the principal their own board (`Session.principal_board_shown`: the verified principal, and on a channel only in their DM), never merely because the session has a set — every `Session` starts with an empty one, and that empty set is how a Google Chat turn, someone else's DM or an unattended pass refuses every ack — and not on the principal's turn in a shared Slack or Discord thread either, where others read the reply and write into what the model reads. It trusts only rows still open (`unread`, `read`), never a roster-request card, and at most 25 ids per turn across calls, and refuses a query with no word of three or more letters. It also moves the admitted limit outward: a model argued into the wrong search can now reach up to 25 open alerts that are off the board, not only live ones. The board is recorded on the turn's `Session`, and the route binds that session for the **whole** SSE body (`schedule_tools.set_session`, alongside `set_turn`) rather than letting `stream_chat` bind it from inside its own generator. `_sse_body` drives the executive with `asyncio.wait_for(stream.__anext__())`, which runs each step in a fresh Task holding a *copy* of the context, so a binding made inside the generator is discarded with the step that made it: step one sees the session and every later step sees `None`. The tool-call loop runs on those later steps, so the whole guard read an empty set and refused every ack on the web — on live, open cards — until the binding moved out to the route. Anything reading `current_session` mid-turn has the same requirement, and two of them needed more than the binding to behave. `schedule_followup`'s seen-`channel_ref` anti-spam gate refuses a send to an address the session has not seen; with no session it skipped itself, so a web turn could schedule to **any** address, and with an empty one it refused even the principal's own. The route now seeds the caller's own addresses onto the session, so on web the gate means what it means on every other surface. Reply linkage (`_record_outbound_context`, read back by `inbound_hydration` to quote the originating conversation into the turn handling a recipient's reply) is the other: it was never reachable on web, and rather than switch that on for the principal's broadest surface as a side effect it is held back, keyed on an explicit `Session.from_web_chat` flag. Not on an empty `origin_channel` — alert review, the CLI and the unattended workflows all leave it empty while legitimately recording linkage, as does the email poller under its own `email` tag. `scheduled_actions` rows also take their `session_id` from the session. A session that was shown no board has an empty trusted set and can ack nothing, so the unattended passes (the nightly reflection, the research synthesis) are refused by default — they also still withhold the tool outright, which is belt-and-braces rather than the only thing stopping them. After the stream closes, the turn is also handed to the episodic memory extractor (`memory.episodic.schedule_extraction`, fire-and-forget) — on every turn where the principal said anything at all, and only the principal: a turn naming an `origin_channel` must resolve to a person marked `is_principal`, because the quote validator downstream cannot tell whose words it is checking. Both this path and the committee path call the same `should_extract` predicate, and it applies no length floor: the gate used to sum in the assistant's reply, which let a verbose answer drag a trivial turn in and discarded the short, decisive instructions that are the whole point of extracting. A user-side floor would fare no better — \"Do B.\" and \"Done\" are one character apart. See **Memory** for what that cost on a live tenant.\n8. **Tool failures** — a handler that raises instead of returning its JSON error is isolated: both tool gathers run with `return_exceptions=True`, the model gets an error `tool_result` naming only the exception's TYPE — a message can carry a filesystem path, a validation error echoing the input, or a third-party HTTP body, and anything in model context can be quoted back to the user, so the full repr stays in the log and the audit row — and the failure is audited with `ok=false`. Only `CancelledError` is re-raised, so turn-timeout and client-disconnect handling still work. Before this one crashing tool aborted the whole turn, which reached the user as an unexplained apology with no audit row (#136).\n\nWeb search, when invoked on the Anthropic path, executes inside the Anthropic API itself (`web_search_20250305`) and returns within the same generation; for any model routed via OpenRouter the same tool becomes OpenRouter's `openrouter:web_search` server tool, which also runs inside the generation with the same search cap and domain lists.\n\nThe install's workspace mode (solo or team, see Org) is read fresh at the start of each turn by `memory.workspace_settings.pin_turn_workspace_mode(session)` and pinned on the session for that turn only (`Session.turn_workspace_mode`): the persona, the org block, the tool list and every tool handler in the turn then agree, and a switch made mid-turn applies from the next turn. A session can instead carry an explicit override, `Session.workspace_mode` (`None` by default), which wins over both — so a caller that runs many conversations concurrently on one Executive, such as evals, can pick the mode per session rather than flipping the install-wide setting. Everything reads it through `effective_workspace_mode(session)` (override, then the turn's pin, then the setting). The principal's role is pinned the same way (`pin_turn_principal_role(session, mode)` → `Session.turn_principal_role`: in solo the `Session.principal_role` override, else the workspace settings read fresh; in team an empty role). The org block renders it, each specialist batch gets it as a `<principal_role>` user-turn tag, and a workflow the turn starts reads the same pin through `effective_principal_role` (override, then the pin, then the setting), so a role edited mid-turn applies from the next turn; a team turn sends none.\n\nAct as me is pinned beside them: `delegation.settings.pin_turn_delegation(session, speaker_text)` records on `Session.turn_delegation` whether the speaker has it on, whether `ghostwrite_email` is offered this turn (the principal, or a team member once the owner lets them, on a verified surface, in a conversation private to them, attended), and the speaker's own words. The pin is also held in a context variable for the turn's task, so two turns running at once on one session each read their own. The tool list, the tool handler and the turn's audit privacy all read that one pin, so a switch flipped mid-turn applies from the next turn. Once the tool touches the speaker's mailbox, the conversation is the speaker's alone (`session_store.mark_mail_private`, the principal included when it isn't theirs), the rest of the turn's audit rows are private (a team member's theirs alone) and the turn teaches no memory: no extraction, open-loop, working-style or Honcho pass runs on it.",+ "markdown": "A single chat message makes a full round-trip through the API, the orchestrator's tool-use loop, a parallel specialist fan-out, and synthesis — all streamed back as server-sent events.\n\n### Steps\n\n1. **Arrival** — the client mints a `client_turn_id` and posts to `POST /chat` (`openexecutive.api.routes.chat`). The turn carries a `<current_speaker>` hint resolved from the caller's `person_id` (followed by that person's `<working_style>` rules when they have any — see the attunement section), plus a `<briefing>` digest of current open alerts (built in parallel with RAG/episodic/peer-memory fetches) so the Executive can discuss any item shown on the `/today` briefing page. A turn arriving from a chat adapter instead of the web app also carries a `<channel>` block naming the surface and stating which approvals can be completed there; on Slack the `<briefing>` digest is gated to the principal's DMs. All three ride in the **user turn**, never a cached system block, as does a `<drive_memory>` block naming the Drive files this conversation already found or opened and the Drive searches that matched nothing (see the memory section). The server registers a stop switch under that id *before* the context fetch, so a Stop pressed during those first seconds halts the turn before any model call is made.\n2. **Tool-use loop** — the `Executive` calls Claude. For a cross-domain question the assistant turn emits multiple `consult_specialist` tool-use blocks at once. The call carries the conversation so far (`Session.get_recent_history`: 20 to 30 turns, its start moving in steps of 10) with a 1h cache marker on the previous reply, so each message reads the earlier conversation from the prompt cache and writes only what is new (see Caching); on the OpenRouter path the translator keeps that marker on the reply's text.\n The person can keep typing while it works. A message sent then (`POST /chat/add`) joins the running turn: the loop adds it after the next round's tool results, so the answer follows it, and it is saved as its own message before the reply. One that arrives while the final answer is being written becomes the next turn instead. A conversation runs one turn at a time.\n3. **Parallel fan-out** — `openexecutive.orchestrator.router.route_parallel` dispatches those calls concurrently, bounded by `MAX_PARALLEL_SPECIALISTS`; calls past the cap get a skip tool_result. A specialist that raises (a timeout, an overloaded model) or returns no text no longer fails the batch: its tool_result says it is unavailable and tells the Executive not to invent its view, the others' results stand, and the turn goes on. It does not count as consulted, so it takes no committee seat and no department memory records the turn, and its documents are left out of the answer's sources. On the web chat the reply need not mention the gap, because the app shows it under the answer; on every other channel the tool_result asks the Executive to say in one sentence that part of the analysis is missing. A knowledge retrieval that fails likewise leaves that specialist without context rather than failing the turn.\n4. **Per-specialist RAG** — each specialist runs its own domain-filtered ChromaDB query, then reasons over the retrieved chunks and returns analysis as a tool_result. Its user turn also carries a `<company_stage>` tag from the session's company profile, so stage-sensitive advice (venture benchmarks vs. cash-first for a bootstrapped company) does not depend on the Executive remembering to mention it. Once the batch is done, the documents each answering specialist's retrieval returned are named, in call order, into the turn's `TurnSources` (`orchestrator/answer_sources.py`), which the web chat route owns and which also holds its own search's documents. Synced Google Drive files are recorded as `drive` sources with their Drive link, and the web chat lists them under **Google Drive**.\n5. **Synthesis** — the Executive feeds the tool_results back to Claude and streams one coherent reply. Every tool round — not just a specialist one — emits an `activity` event naming what is running, immediately followed by a `thinking` keepalive. The label names the *activity*, never the internal agent: \"Putting time on the calendar…\", \"Looking up people…\", or, for an MCP call, the underlying tool by name. A round that really does fan out collapses to the generic \"Consulting specialists…\" — the one voice holds even in the progress line. The keepalive carries no tool information of its own; it used to drive a hardcoded \"Consulting specialists…\" string, so an MCP or calendar round misreported itself as a fan-out.\n6. **Optional committee review** — when enabled, `Executive.stream_chat_with_committee()` runs a silent draft, parallel reviewer critiques, then a streamed revision pass.\n7. **Side effects** — after the reply, every rostered speaker's turn (not only the principal's) schedules a fire-and-forget **open-loop pass** (Attunement) beside the episodic extraction: a keyword prefilter, then one forced-tool call that may open or close loops, every item quote-gated against the speaker's own words. The user message is persisted with `sender_person_id` — the resolved speaker, never the principal fallback a Discord / Telegram session uses for ownership — and the session records the speaker for the turn (`Session.caller_person_id`), which `close_open_loop` reads. A DM sent while proactive work is tagged (`attunement.outcomes.tag_proactive` — scheduler dispatch, reflection, research, alert review) also opens an outcome-ledger row at `_record_outbound_context`; a user's own turn is never tagged. Any side-effecting tool call surfaces an `action_taken` chip, which unlike the ephemeral `activity` label is persisted with the assistant message; once the reply is over — also when it was stopped or timed out — the web chat route sends one `sources` event: the documents and web pages the turn looked at (its own search, each specialist that answered, web search results, and the pages the reply cites, which are always kept) and the areas (\"finance\", never a specialist) it had to go without. The web chat shows it under the reply, and it is persisted with the assistant message as `chat_messages.sources`; other channels don't collect it. The stream closes with a `done` event — preceded by a `stopped` event when the user halted the turn, in which case the partial reply is still persisted and flagged so it reads as cut short rather than complete, and is mirrored into the live in-memory session so the next turn's prompt matches what was saved. The stream still ends with `done` event. One side-effecting tool is gated server-side rather than by its description: `ack_alert` will only accept an `alert_id` from the live alert board the server itself derived this turn — the whole live set, not only the rows the `<briefing>` digest printed, since `/today` renders more cards than the digest lists and every one of them is Discuss-able — and it fails closed, so an id it cannot even parse is refused rather than waved through to the handler's own parse. That check runs on **every** session. It used to run only on turns that arrived from a chat channel, keyed on `origin_channel`, which the browser never sets — so the web, the surface the principal actually uses, was unchecked and the model could ack any id, including one quoted out of an alert body by the briefing page's Discuss handoff. What that buys is bounded and worth stating plainly: an id that is invented, closed, snoozed or expired is refused, including one quoted out of an attacker-written alert body; the model being argued into acking the wrong *live* card is not prevented. The one way to widen that set is `find_alerts`, for an item the principal names that has left the live board (already opened, snoozed, older than the board): a keyword search over alerts of every status whose own results — never an id the model supplies — join the trusted set. It runs only in a turn where `render_and_trust` showed the principal their own board (`Session.principal_board_shown`: the verified principal, and on a channel only in their DM), never merely because the session has a set — every `Session` starts with an empty one, and that empty set is how a Google Chat turn, someone else's DM or an unattended pass refuses every ack — and not on the principal's turn in a shared Slack or Discord thread either, where others read the reply and write into what the model reads. It trusts only rows still open (`unread`, `read`), never a roster-request card, and at most 25 ids per turn across calls, and refuses a query with no word of three or more letters. It also moves the admitted limit outward: a model argued into the wrong search can now reach up to 25 open alerts that are off the board, not only live ones. The board is recorded on the turn's `Session`, and the route binds that session for the **whole** SSE body (`schedule_tools.set_session`, alongside `set_turn`) rather than letting `stream_chat` bind it from inside its own generator. `_sse_body` drives the executive with `asyncio.wait_for(stream.__anext__())`, which runs each step in a fresh Task holding a *copy* of the context, so a binding made inside the generator is discarded with the step that made it: step one sees the session and every later step sees `None`. The tool-call loop runs on those later steps, so the whole guard read an empty set and refused every ack on the web — on live, open cards — until the binding moved out to the route. Anything reading `current_session` mid-turn has the same requirement, and two of them needed more than the binding to behave. `schedule_followup`'s seen-`channel_ref` anti-spam gate refuses a send to an address the session has not seen; with no session it skipped itself, so a web turn could schedule to **any** address, and with an empty one it refused even the principal's own. The route now seeds the caller's own addresses onto the session, so on web the gate means what it means on every other surface. Reply linkage (`_record_outbound_context`, read back by `inbound_hydration` to quote the originating conversation into the turn handling a recipient's reply) is the other: it was never reachable on web, and rather than switch that on for the principal's broadest surface as a side effect it is held back, keyed on an explicit `Session.from_web_chat` flag. Not on an empty `origin_channel` — alert review, the CLI and the unattended workflows all leave it empty while legitimately recording linkage, as does the email poller under its own `email` tag. `scheduled_actions` rows also take their `session_id` from the session. A session that was shown no board has an empty trusted set and can ack nothing, so the unattended passes (the nightly reflection, the research synthesis) are refused by default — they also still withhold the tool outright, which is belt-and-braces rather than the only thing stopping them. After the stream closes, the turn is also handed to the episodic memory extractor (`memory.episodic.schedule_extraction`, fire-and-forget) — on every turn where the principal said anything at all, and only the principal: a turn naming an `origin_channel` must resolve to a person marked `is_principal`, because the quote validator downstream cannot tell whose words it is checking. Both this path and the committee path call the same `should_extract` predicate, and it applies no length floor: the gate used to sum in the assistant's reply, which let a verbose answer drag a trivial turn in and discarded the short, decisive instructions that are the whole point of extracting. A user-side floor would fare no better — \"Do B.\" and \"Done\" are one character apart. See **Memory** for what that cost on a live tenant.\n8. **Tool failures** — a handler that raises instead of returning its JSON error is isolated: both tool gathers run with `return_exceptions=True`, the model gets an error `tool_result` naming only the exception's TYPE — a message can carry a filesystem path, a validation error echoing the input, or a third-party HTTP body, and anything in model context can be quoted back to the user, so the full repr stays in the log and the audit row — and the failure is audited with `ok=false`. Only `CancelledError` is re-raised, so turn-timeout and client-disconnect handling still work. Before this one crashing tool aborted the whole turn, which reached the user as an unexplained apology with no audit row (#136).\n\nWeb search, when invoked on the Anthropic path, executes inside the Anthropic API itself (`web_search_20250305`) and returns within the same generation; for any model routed via OpenRouter the same tool becomes OpenRouter's `openrouter:web_search` server tool, which also runs inside the generation with the same search cap and domain lists.\n\nThe install's workspace mode (solo or team, see Org) is read fresh at the start of each turn by `memory.workspace_settings.pin_turn_workspace_mode(session)` and pinned on the session for that turn only (`Session.turn_workspace_mode`): the persona, the org block, the tool list and every tool handler in the turn then agree, and a switch made mid-turn applies from the next turn. A session can instead carry an explicit override, `Session.workspace_mode` (`None` by default), which wins over both — so a caller that runs many conversations concurrently on one Executive, such as evals, can pick the mode per session rather than flipping the install-wide setting. Everything reads it through `effective_workspace_mode(session)` (override, then the turn's pin, then the setting). The principal's role is pinned the same way (`pin_turn_principal_role(session, mode)` → `Session.turn_principal_role`: in solo the `Session.principal_role` override, else the workspace settings read fresh; in team an empty role). The org block renders it, each specialist batch gets it as a `<principal_role>` user-turn tag, and a workflow the turn starts reads the same pin through `effective_principal_role` (override, then the pin, then the setting), so a role edited mid-turn applies from the next turn; a team turn sends none.\n\nAct as me is pinned beside them: `delegation.settings.pin_turn_delegation(session, speaker_text)` records on `Session.turn_delegation` whether the speaker has it on, whether `ghostwrite_email` is offered this turn (the principal, or a team member once the owner lets them, on a verified surface, in a conversation private to them, attended), and the speaker's own words. The pin is also held in a context variable for the turn's task, so two turns running at once on one session each read their own. The tool list, the tool handler and the turn's audit privacy all read that one pin, so a switch flipped mid-turn applies from the next turn. Once the tool touches the speaker's mailbox, the conversation is the speaker's alone (`session_store.mark_mail_private`, the principal included when it isn't theirs), the rest of the turn's audit rows are private (a team member's theirs alone) and the turn teaches no memory: no extraction, open-loop, working-style or Honcho pass runs on it.","mermaid": "sequenceDiagram\n participant C as Client\n participant API as FastAPI /chat\n participant E as Executive\n participant CL as Claude\n participant S as Specialists\n participant KB as ChromaDB\n C->>API: POST /chat message\n API->>E: start turn\n E->>CL: messages (tool-use loop)\n CL-->>E: consult_specialist calls\n E->>S: route_parallel fan-out\n S->>KB: domain-filtered RAG\n KB-->>S: chunks\n S-->>E: analysis tool_results\n E->>CL: synthesize\n CL-->>E: streamed reply\n E-->>API: SSE chunks then done\n API-->>C: streamed response","generated_at": "2026-10-03T16:56:18Z"}diff --git a/packages/core/openexecutive/orchestrator/executive.py b/packages/core/openexecutive/orchestrator/executive.pyindex e4ef949e..59d4c538 100644--- a/packages/core/openexecutive/orchestrator/executive.py+++ b/packages/core/openexecutive/orchestrator/executive.py@@ -313,6 +313,80 @@ def _cap_tool_result(text: Any, *, tool_name: str, limit: int) -> Any:return text[:shown] + marker(shown, pct)+# The history breakpoint lives an hour: people answer in minutes, not+# seconds, and a 5m entry would be gone before most next messages.+_HISTORY_CACHE_CONTROL: dict[str, str] = {"type": "ephemeral", "ttl": "1h"}+++def _apply_history_cache_marker(+ system_blocks: list[dict[str, Any]], messages: list[dict[str, Any]]+) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:+ """Cache the conversation so far, so the next message reads it back.++ Marks the newest non-empty message before the caller's last one (the+ turn being answered): normally the previous reply. History turns are+ stored as plain text and rebuilt identically every turn, so on the next+ message everything up to that point is a cache read, and only what is+ new (the last exchange, this turn's context) is written. Without it the+ loop marker (``_apply_loop_cache_marker``) writes the whole history again+ on every turn, because the previous turn's cached prefix ended inside+ that turn's own context and tool calls, which history does not keep.++ The API allows four breakpoints and the other three are spoken for, so+ the history marker takes the slot of the 5m company-profile block's.+ That block still sits in the cached prefix (covered by this marker),+ and dropping its 5m marker keeps the TTLs in the order the API requires:+ tools 1h, persona 1h, history 1h, then the 5m loop marker.++ Returns new lists; the caller's blocks and messages are left as they+ were. With no history (a first message, an unattended run) both come+ back unchanged.+ """+ target = -1+ for i in range(len(messages) - 2, -1, -1):+ content = messages[i].get("content")+ if isinstance(content, str) and content:+ target = i+ break+ if isinstance(content, list) and any(+ isinstance(b, dict) and b.get("type") == "text" and b.get("text") for b in content+ ):+ target = i+ break+ if target < 0:+ return system_blocks, messages++ content = messages[target]["content"]+ if isinstance(content, str):+ marked_content: list[dict[str, Any]] = [+ {"type": "text", "text": content, "cache_control": dict(_HISTORY_CACHE_CONTROL)}+ ]+ else:+ marked_content = [+ {k: v for k, v in b.items() if k != "cache_control"} if isinstance(b, dict) else b+ for b in content+ ]+ last_text = max(+ j for j, b in enumerate(marked_content)+ if isinstance(b, dict) and b.get("type") == "text" and b.get("text")+ )+ marked_content[last_text] = {+ **marked_content[last_text], "cache_control": dict(_HISTORY_CACHE_CONTROL)+ }+ new_messages = list(messages)+ new_messages[target] = {**messages[target], "content": marked_content}++ new_system = [+ {k: v for k, v in b.items() if k != "cache_control"}+ if isinstance(b, dict)+ and isinstance(b.get("cache_control"), dict)+ and b["cache_control"].get("ttl") != "1h"+ else b+ for b in system_blocks+ ]+ return new_system, new_messages++def _apply_loop_cache_marker(messages: list[dict[str, Any]], start: int = 0) -> None:@@ -577,11 +651,15 @@ def _system_block_names(blocks: list[dict[str, Any]]) -> list[str]:the timeline can show which cached blocks were live during each step.Block layout in cache_manager.build_system_blocks() is fixed:0 = persona + knowledge_index (1h TTL)- 1 = company_profile + org_context (5m TTL)+ 1 = company_profile + org_context (5m TTL, or unmarked when the+ agent loop's history marker took its slot)"""names: list[str] = []for i, b in enumerate(blocks):- ttl = (b.get("cache_control") or {}).get("ttl", "5m")+ cc = b.get("cache_control")+ # No marker: the block rides inside a later breakpoint's prefix (the+ # history marker takes the company block's slot).+ ttl = cc.get("ttl", "5m") if isinstance(cc, dict) else "none"if i == 0:names.append(f"persona+knowledge_index (ttl={ttl})")elif i == 1:@@ -737,15 +815,12 @@ class Executive:history = session.get_recent_history()for turn in history:- # No cache_control on history turns. The breakpoint budget is 4- # per request and the other three are always spoken for: two- # system blocks (cache_manager.build_system_blocks) plus the tool- # block, leaving exactly one for the agent loop's intra-turn- # marker (_apply_loop_cache_marker). A rolling marker here used- # to claim a latent fifth, which Anthropic rejects outright on- # the direct path; it never earned its slot anyway, since history- # turns are flat strings and the system + tool blocks already- # cover the expensive stable prefix.+ # No cache_control here. The agent loop marks the newest history+ # turn itself (_apply_history_cache_marker), taking the company+ # block's slot so the request stays at the API's 4 breakpoints;+ # other users of these messages (the committee revision) keep+ # the plain layout. A rolling marker on every history turn used+ # to claim a latent fifth, which Anthropic rejects outright.prev = messages[-1] if messages else Noneif (prev is not None@@ -1809,6 +1884,10 @@ class Executive:if origin and not unattended_withheld:take_the_lead.wake(f"a message on {origin}")current_messages = list(messages)+ if self._settings.enable_caching:+ system_blocks, current_messages = _apply_history_cache_marker(+ system_blocks, current_messages+ )# Tool calls this turn's scripts have made, in all and per own tool# (settings.chat_script_max_calls, step_script.CHAT_OWN_TOOL_CAPS).script_counts: dict[str, int] = {}diff --git a/packages/core/openexecutive/orchestrator/session.py b/packages/core/openexecutive/orchestrator/session.pyindex 685673f8..c1d098ef 100644--- a/packages/core/openexecutive/orchestrator/session.py+++ b/packages/core/openexecutive/orchestrator/session.py@@ -158,8 +158,23 @@ class Session:def add_assistant_message(self, content: str | list[dict[str, Any]]) -> None:self.conversation_history.append({"role": "assistant", "content": content})- def get_recent_history(self, max_turns: int = 20) -> list[dict[str, Any]]:- history = self.conversation_history[-(max_turns * 2):]+ def get_recent_history(self, max_turns: int = 20, step_turns: int = 10) -> list[dict[str, Any]]:+ """The last ``max_turns`` turns, give or take ``step_turns``.++ The window's start moves in steps of ``step_turns`` rather than one+ turn at a time, so a long conversation keeps between ``max_turns``+ and ``max_turns + step_turns`` turns. A window that slid every turn+ would change the first history message on every request, and the+ prompt cache (``orchestrator.executive._apply_history_cache_marker``)+ would miss and re-write the whole conversation each time.+ """+ total = len(self.conversation_history)+ keep = max_turns * 2+ start = 0+ if total > keep:+ step = max(1, step_turns) * 2+ start = ((total - keep) // step) * step+ history = self.conversation_history[start:]# Anthropic requires messages to start with a user turn.# Drop a leading assistant message if history length is odd (can happen on error recovery).if history and history[0]["role"] != "user":diff --git a/packages/core/openexecutive/providers/translator.py b/packages/core/openexecutive/providers/translator.pyindex 7b22a0c7..3d02710c 100644--- a/packages/core/openexecutive/providers/translator.py+++ b/packages/core/openexecutive/providers/translator.py@@ -193,6 +193,28 @@ def _request_caches_tool_results(messages: list[Any]) -> bool:return False+def _request_caches_assistant_text(messages: list[Any]) -> bool:+ """True iff an assistant text block in the request carries a marker.++ That is the history breakpoint (``executive._apply_history_cache_marker``),+ normally on the previous reply. Like ``_request_caches_tool_results`` it+ sets one shape for every assistant turn in the request: the marker moves+ to the newest reply each message, so a reply typed while marked and+ flattened afterwards would change the cached prefix under the breakpoint+ meant to read it.+ """+ for m in messages:+ if not isinstance(m, dict) or m.get("role") != "assistant":+ continue+ content = m.get("content")+ if isinstance(content, list) and any(+ isinstance(b, dict) and b.get("type") == "text" and isinstance(b.get("cache_control"), dict)+ for b in content+ ):+ return True+ return False++def _anthropic_messages_to_openai(messages: list[Any]) -> list[dict[str, Any]]:"""Convert Anthropic ``messages`` to OpenAI chat-completions ``messages``.@@ -203,13 +225,14 @@ def _anthropic_messages_to_openai(messages: list[Any]) -> list[dict[str, Any]]:"""out: list[dict[str, Any]] = []typed_tool_results = _request_caches_tool_results(messages)+ typed_assistant_text = _request_caches_assistant_text(messages)for m in messages:role = m.get("role")content = m.get("content")if role == "user":out.extend(_user_content_to_openai(content, typed_tool_results))elif role == "assistant":- out.append(_assistant_content_to_openai(content))+ out.append(_assistant_content_to_openai(content, typed_assistant_text))else:# Unknown role — preserve as best-effort.out.append({"role": role, "content": _content_to_text(content)})@@ -328,14 +351,22 @@ def _user_content_to_openai(return msgs-def _assistant_content_to_openai(content: Any) -> dict[str, Any]:+def _assistant_content_to_openai(content: Any, typed_text: bool = False) -> dict[str, Any]:"""Assistant-turn content: collapse text blocks; lift tool_use blocks to- OpenAI ``tool_calls``."""+ OpenAI ``tool_calls``.++ ``typed_text`` (the request carries the history breakpoint, see+ ``_request_caches_assistant_text``) keeps the text as typed parts, each+ with its own ``cache_control``, so the marker reaches Anthropic through+ OpenRouter instead of being collapsed away."""if isinstance(content, str):+ if typed_text and content:+ return {"role": "assistant", "content": [{"type": "text", "text": content}]}return {"role": "assistant", "content": content}if not isinstance(content, list):return {"role": "assistant", "content": str(content)}+ typed_parts: list[dict[str, Any]] = []text_chunks: list[str] = []tool_calls: list[dict[str, Any]] = []reasoning_details: list[Any] = []@@ -347,6 +378,9 @@ def _assistant_content_to_openai(content: Any) -> dict[str, Any]:txt = block.get("text", "")if isinstance(txt, str) and txt:text_chunks.append(txt)+ part = _typed_text_with_cc(txt, block.get("cache_control"))+ if part is not None:+ typed_parts.append(part)elif btype == OPENROUTER_REASONING_BLOCK:details = block.get("reasoning_details")if isinstance(details, list):@@ -364,7 +398,10 @@ def _assistant_content_to_openai(content: Any) -> dict[str, Any]:)out: dict[str, Any] = {"role": "assistant"}- out["content"] = "\n\n".join(text_chunks) if text_chunks else None+ if typed_text and typed_parts:+ out["content"] = typed_parts+ else:+ out["content"] = "\n\n".join(text_chunks) if text_chunks else Noneif reasoning_details:out["reasoning_details"] = reasoning_detailsif tool_calls:diff --git a/packages/core/tests/unit/test_executive_tool_loop_cache.py b/packages/core/tests/unit/test_executive_tool_loop_cache.pyindex 97aef0c0..9f896bac 100644--- a/packages/core/tests/unit/test_executive_tool_loop_cache.py+++ b/packages/core/tests/unit/test_executive_tool_loop_cache.py@@ -24,7 +24,10 @@ The contract pinned here:4. the loop marker is 5m (no `ttl` key) — the turn is over in seconds,so a 1h TTL would only double the write premium;5. `enable_caching=False` emits no marker at all;- 6. the caller's `messages` list is never mutated.+ 6. the caller's `messages` list is never mutated;+ 7. with history, the previous reply carries a 1h marker that takes the+ 5m company block's slot, so the next message reads the conversation+ back instead of writing it again (still 4 markers, TTLs 1h before 5m)."""from __future__ import annotations@@ -405,3 +408,148 @@ def test_non_string_tool_result_content_is_still_markable() -> None:]_apply_loop_cache_marker(messages)assert messages[0]["content"][0]["cache_control"] == {"type": "ephemeral"}+++# --------------------------------------------------------------------- #+# History breakpoint (_apply_history_cache_marker)+# --------------------------------------------------------------------- #+++def _history_messages() -> list[dict[str, Any]]:+ return [+ {"role": "user", "content": "earlier question"},+ {"role": "assistant", "content": "earlier answer"},+ {"role": "user", "content": [{"type": "text", "text": "do the thing"}]},+ ]+++def _real_system_blocks() -> list[dict[str, Any]]:+ from openexecutive.memory.company_profile import CompanyProfile+ from openexecutive.prompts.cache_manager import build_system_blocks++ return build_system_blocks(+ company_profile=CompanyProfile(name="Test Co", industry="Testing"),+ )+++def _ttls_in_wire_order(call: dict[str, Any]) -> list[str]:+ """Marker TTLs in the order the API reads them: tools, system, messages."""+ out: list[str] = []+ blocks: list[Any] = [*(call.get("tools") or []), *(call.get("system") or [])]+ for msg in call.get("messages") or []:+ if isinstance(msg.get("content"), list):+ blocks.extend(msg["content"])+ for b in blocks:+ if isinstance(b, dict) and isinstance(b.get("cache_control"), dict):+ out.append(b["cache_control"].get("ttl", "5m"))+ return out+++def test_previous_reply_carries_a_one_hour_history_marker() -> None:+ """Without it every message re-writes the whole conversation into the+ cache: the previous turn's cached prefix ended inside that turn's own+ context and tool calls, which the rebuilt history does not keep."""+ provider = _scripted_three_iterations()+ _run_loop(provider, messages=_history_messages(), system_blocks=_real_system_blocks())++ for call in provider.calls:+ reply = call["messages"][1]+ assert reply["role"] == "assistant"+ assert reply["content"] == [+ {+ "type": "text",+ "text": "earlier answer",+ "cache_control": {"type": "ephemeral", "ttl": "1h"},+ }+ ]+++def test_history_marker_takes_the_company_blocks_slot() -> None:+ """Four breakpoints at most, and longer TTLs before shorter ones: the+ 5m company block gives up its marker (it stays inside the history+ marker's prefix), so the order is tools 1h, persona 1h, history 1h,+ loop 5m."""+ provider = _scripted_three_iterations()+ _run_loop(provider, messages=_history_messages(), system_blocks=_real_system_blocks())++ for i, call in enumerate(provider.calls):+ assert _count_all_markers(call) <= 4, f"call {i} exceeded the 4-block limit"+ system = call["system"]+ assert "cache_control" in system[0]+ assert "cache_control" not in system[1]+ assert _ttls_in_wire_order(provider.calls[1]) == ["1h", "1h", "1h", "5m"]+++def test_no_history_keeps_the_company_block_marker() -> None:+ provider = _scripted_three_iterations()+ _run_loop(provider, system_blocks=_real_system_blocks())+ for call in provider.calls:+ assert "cache_control" in call["system"][1]+++def test_history_marker_leaves_caller_lists_alone() -> None:+ messages = _history_messages()+ system_blocks = _real_system_blocks()+ before = (copy.deepcopy(messages), copy.deepcopy(system_blocks))+ provider = _scripted_three_iterations()+ _run_loop(provider, messages=messages, system_blocks=system_blocks)+ assert (messages, system_blocks) == before+++def test_no_history_marker_when_caching_disabled(monkeypatch: pytest.MonkeyPatch) -> None:+ exec_ = Executive()+ monkeypatch.setattr(exec_._settings, "enable_caching", False)+ provider = _scripted_three_iterations()+ _run_loop(provider, messages=_history_messages(), executive=exec_)+ assert provider.calls[0]["messages"][1]["content"] == "earlier answer"+++def test_history_marker_skips_an_empty_reply() -> None:+ """An empty text block with cache_control is a 400; mark the newest+ turn that has words instead."""+ from openexecutive.orchestrator.executive import _apply_history_cache_marker++ messages: list[dict[str, Any]] = [+ {"role": "user", "content": "earlier question"},+ {"role": "assistant", "content": ""},+ {"role": "user", "content": "do the thing"},+ ]+ _, out = _apply_history_cache_marker([], messages)+ assert out[0]["content"][0]["cache_control"] == {"type": "ephemeral", "ttl": "1h"}+ assert out[1]["content"] == ""+++def test_history_marker_on_a_typed_reply_marks_its_last_text_block() -> None:+ from openexecutive.orchestrator.executive import _apply_history_cache_marker++ messages: list[dict[str, Any]] = [+ {"role": "user", "content": "q"},+ {"role": "assistant", "content": [{"type": "text", "text": "a"}, {"type": "text", "text": "b"}]},+ {"role": "user", "content": "do the thing"},+ ]+ _, out = _apply_history_cache_marker([], messages)+ assert "cache_control" not in out[1]["content"][0]+ assert out[1]["content"][1]["cache_control"] == {"type": "ephemeral", "ttl": "1h"}+ assert "cache_control" not in messages[1]["content"][1]+++def test_history_window_moves_in_steps() -> None:+ """A window that slid one turn at a time would change the first history+ message on every request past turn 20, and the cache would miss on all+ of them."""+ from openexecutive.orchestrator.session import Session++ session = Session(session_id="s-window")+ firsts: list[str] = []+ for n in range(1, 61):+ session.add_user_message(f"q{n}")+ session.add_assistant_message(f"a{n}")+ history = session.get_recent_history()+ assert history[0]["role"] == "user"+ assert 20 <= len(history) // 2 <= 30 or n < 20+ firsts.append(history[0]["content"])+ # Past turn 20 the start moves once every 10 turns (at turns 30, 40,+ # 50 and 60), never on consecutive turns.+ changes = [i for i in range(1, len(firsts)) if firsts[i] != firsts[i - 1]]+ assert all(b - a >= 10 for a, b in zip(changes, changes[1:], strict=False))+ assert len(changes) == 4diff --git a/packages/core/tests/unit/test_openrouter_translator.py b/packages/core/tests/unit/test_openrouter_translator.pyindex 781e9f0e..a8a8c327 100644--- a/packages/core/tests/unit/test_openrouter_translator.py+++ b/packages/core/tests/unit/test_openrouter_translator.py@@ -1349,3 +1349,43 @@ def test_uncached_request_keeps_the_legacy_flat_string_shape() -> None:for m in body["messages"]:if m.get("role") == "tool":assert isinstance(m["content"], str)+++# --- history breakpoint on an assistant turn --------------------------------+++def _history_request(marked: bool) -> list[dict]:+ reply: object = (+ [{"type": "text", "text": "second answer", "cache_control": {"type": "ephemeral", "ttl": "1h"}}]+ if marked+ else "second answer"+ )+ return [+ {"role": "user", "content": "first question"},+ {"role": "assistant", "content": "first answer"},+ {"role": "user", "content": "second question"},+ {"role": "assistant", "content": reply},+ {"role": "user", "content": "third question"},+ ]+++def test_history_marker_on_a_reply_survives_translation() -> None:+ """The Executive's history breakpoint sits on the previous reply; OpenRouter+ only forwards it to Anthropic on a typed text part."""+ from openexecutive.providers.translator import _anthropic_messages_to_openai++ out = _anthropic_messages_to_openai(_history_request(marked=True))+ assert out[3]["content"] == [+ {"type": "text", "text": "second answer", "cache_control": {"type": "ephemeral", "ttl": "1h"}}+ ]+ # Every assistant turn takes the same typed shape in that request, so the+ # reply marked now reads back byte-identical when it is unmarked history.+ assert out[1]["content"] == [{"type": "text", "text": "first answer"}]+++def test_assistant_turns_stay_flat_without_a_history_marker() -> None:+ from openexecutive.providers.translator import _anthropic_messages_to_openai++ out = _anthropic_messages_to_openai(_history_request(marked=False))+ assert out[1]["content"] == "first answer"+ assert out[3]["content"] == "second answer"