AI-powered virtual executive team — a single coherent executive persona backed by 8 specialist agents (FastAPI + Next.js).
Remove the extraction length floor; audit every pass
Claude85ad10e22434
diff --git a/packages/core/openexecutive/architecture/architecture-facts.yaml b/packages/core/openexecutive/architecture/architecture-facts.yamlindex 6a938823..daaca830 100644--- a/packages/core/openexecutive/architecture/architecture-facts.yaml+++ b/packages/core/openexecutive/architecture/architecture-facts.yaml@@ -915,17 +915,25 @@ memory:populates `domain` but not `department`, so today the mirror onlyfires for explicit dept-scoped writers; falling back to `domain`is a deliberate follow-up decision, not silent drift.- Extraction is gated on MIN_USER_CHARS_FOR_EXTRACTION, measured on the- USER's message only. It was a combined user+assistant floor of 1500,- which on a live tenant blocked 9 of 16 turns — every instruction the- principal gave — while admitting the 7 long analytical exchanges that- contained no commitment; the extractor ran 13 times and stored nothing.- Decisive instructions are short and the analysis around them is long, so- a combined floor discards what it exists to capture, and the user's own- words are the only text `_is_valid_user_commitment` accepts a quote from.- Every pass now writes a `memory_extraction` audit row (proposed / stored /- dropped-with-reason) — before it, an extractor storing nothing looked- identical to one with nothing to store.+ Extraction is gated by `should_extract(user_message)`, which has NO+ length floor — it admits any non-blank user message. There used to be a+ combined user+assistant floor of 1500 chars, which on a live tenant+ blocked 9 of 16 turns — every instruction the principal gave — while+ admitting the 7 long analytical exchanges that contained no commitment;+ the extractor ran 13 times and stored nothing. Moving that floor to the+ user's side relocates the bug rather than fixing it: the canonical+ executive decision is a long analysis answered with "Approve option B."+ (17 chars) or "Do B." (5), and any floor high enough to skip "Done" (4)+ also skips those. "Do B." and "Done" differ by one character and mean+ opposite things, so length cannot separate a decision from an+ acknowledgement. `_is_valid_user_commitment` — the verbatim-quote check+ against the user's own words — is the only gate that can, and it is the+ one that runs. The cost of admitting acknowledgements is one utility-fast+ call per turn that proposes nothing.+ Every pass writes a `memory_extraction` audit row (proposed / stored /+ dropped-with-reason, plus a `failure` reason when the pass raises) —+ before it, an extractor storing nothing looked identical to one with+ nothing to store. A `proposed>0, stored=0` streak is the signal to watch.peer_memory:what: |diff --git a/packages/core/openexecutive/architecture/prebuilt/audit.json b/packages/core/openexecutive/architecture/prebuilt/audit.jsonindex 3978ed70..256f2e56 100644--- a/packages/core/openexecutive/architecture/prebuilt/audit.json+++ b/packages/core/openexecutive/architecture/prebuilt/audit.json@@ -1,7 +1,7 @@{"section_id": "audit","title": "Audit Log",- "markdown": "The audit log is a persistent, searchable, append-only record of what the system did — distinct from episodic memory (which biases future LLM behavior). Every event flows through `audit/logger.log_event`, a synchronous SQLite writer (WAL + 5s busy_timeout) that appends one row to the `audit_log` table in `episodic_memory.db`.\n\n### Event types\nRows are tagged by `event_type`, including `chat_turn`, `specialist_consult`, `tool_invocation`, `scheduled_action`, `alert`, `integration_inbound` (Slack / Discord / email / Telegram / Google Chat), plus `knowledge_retrieval`, `memory_extraction` (one row per episodic-extraction pass: how many items the model proposed, how many were stored, and why each drop was rejected — a `proposed>0, stored=0` streak means the quote validator is rejecting everything, which is otherwise invisible), `cache_event` (one row per model call from every call site — Executive turns, specialist consults (chat turns and workflow steps), research specialists, the research routing and watchlist passes, triage, memory extraction — carrying tokens, cache hits, cost, server-side search count and, inside a research run, its `run_id`), `memory_snapshot`, `committee_review`, `peer_memory`, and the external-signal/monitor events.\n\n### Session & turn context\nEach row carries `session_id`, `turn_id`, `actor`, `summary`, `details_json`, `full_json`, and `department`. A `turn_id` (`f\"t-{uuid12}\"`) is set per Executive turn and propagated automatically via ContextVars to retriever / specialist / tool / cache emits, grouping one user-assistant exchange. `summary` is the searchable column (LIKE, ~300 char cap); `details_json` is structured and not searched.\n\n### Views\n- `GET /audit` — filterable list with a \"Group by session\" toggle and session pills.\n- `GET /audit/session/{id}` — full ordered timeline plus a server-derived node/edge graph rendered as a three-pane React Flow chart.\n\n### Invariant\nAudit writes never break a chat turn: all writes swallow exceptions and log a warning, and instrumentation observes only — it never touches request building or the cached prompt blocks. Audit is append-only: no UPDATE or DELETE paths exist in product code.",+ "markdown": "The audit log is a persistent, searchable, append-only record of what the system did — distinct from episodic memory (which biases future LLM behavior). Every event flows through `audit/logger.log_event`, a synchronous SQLite writer (WAL + 5s busy_timeout) that appends one row to the `audit_log` table in `episodic_memory.db`.\n\n### Event types\nRows are tagged by `event_type`, including `chat_turn`, `specialist_consult`, `tool_invocation`, `scheduled_action`, `alert`, `integration_inbound` (Slack / Discord / email / Telegram / Google Chat), plus `knowledge_retrieval`, `memory_extraction` (one row per episodic-extraction pass, written in a `finally` so a crashed pass is recorded too: how many items the model proposed, how many were stored, a `{kind, reason}` record per drop, and a `failure` naming the exception type when the pass raised — a `proposed>0, stored=0` streak means the quote validator is rejecting everything, which is otherwise invisible), `cache_event` (one row per model call from every call site — Executive turns, specialist consults (chat turns and workflow steps), research specialists, the research routing and watchlist passes, triage, memory extraction — carrying tokens, cache hits, cost, server-side search count and, inside a research run, its `run_id`), `memory_snapshot`, `committee_review`, `peer_memory`, and the external-signal/monitor events.\n\n### Session & turn context\nEach row carries `session_id`, `turn_id`, `actor`, `summary`, `details_json`, `full_json`, and `department`. A `turn_id` (`f\"t-{uuid12}\"`) is set per Executive turn and propagated automatically via ContextVars to retriever / specialist / tool / cache emits, grouping one user-assistant exchange. `summary` is the searchable column (LIKE, ~300 char cap); `details_json` is structured and not searched.\n\n### Views\n- `GET /audit` — filterable list with a \"Group by session\" toggle and session pills.\n- `GET /audit/session/{id}` — full ordered timeline plus a server-derived node/edge graph rendered as a three-pane React Flow chart.\n\n### Invariant\nAudit writes never break a chat turn: all writes swallow exceptions and log a warning, and instrumentation observes only — it never touches request building or the cached prompt blocks. Audit is append-only: no UPDATE or DELETE paths exist in product code.","mermaid": "flowchart TD\n Chat[\"Chat turn / consult / tool call\"]\n Sched[\"Scheduled action\"]\n Inbound[\"Inbound integration\"]\n Log[\"audit/logger.log_event\"]\n DB[\"audit_log table (append-only)\"]\n ListView[\"/audit list view\"]\n SessView[\"/audit/session/{id} flow chart\"]\n Chat -->|event + turn_id| Log\n Sched --> Log\n Inbound --> Log\n Log -->|swallow errors, never block| DB\n DB --> ListView\n DB --> SessView\n class Chat entry\n class Sched entry\n class Inbound entry\n class Log compute\n class DB storage\n class ListView hot\n class SessView hot","generated_at": "2026-06-01T00:00:00+00:00"}diff --git a/packages/core/openexecutive/architecture/prebuilt/lifecycle.json b/packages/core/openexecutive/architecture/prebuilt/lifecycle.jsonindex 4724aa9c..2bd3bb47 100644--- a/packages/core/openexecutive/architecture/prebuilt/lifecycle.json+++ b/packages/core/openexecutive/architecture/prebuilt/lifecycle.json@@ -1,7 +1,7 @@{"section_id": "lifecycle","title": "Request Lifecycle",- "markdown": "A single chat message makes a full round-trip through the API, the orchestrator's tool-use loop, a parallel specialist fan-out, and synthesis — all streamed back as server-sent events.\n\n### Steps\n\n1. **Arrival** — the client posts to `POST /chat` (`openexecutive.api.routes.chat`). The turn carries a `<current_speaker>` hint resolved from the caller's `person_id`, plus a `<briefing>` digest of current open alerts (built in parallel with RAG/episodic/peer-memory fetches) so the Executive can discuss any item shown on the `/today` briefing page A turn arriving from a chat adapter instead of the web app also carries a `<channel>` block naming the surface and stating which approvals can be completed there; on Slack the `<briefing>` digest is gated to the principal's DMs. All three ride in the **user turn**, never a cached system block.\n2. **Tool-use loop** — the `Executive` calls Claude. For a cross-domain question the assistant turn emits multiple `consult_specialist` tool-use blocks at once.\n3. **Parallel fan-out** — `openexecutive.orchestrator.router.route_parallel` dispatches those calls concurrently, bounded by `MAX_PARALLEL_SPECIALISTS`; calls past the cap get a skip tool_result.\n4. **Per-specialist RAG** — each specialist runs its own domain-filtered ChromaDB query, then reasons over the retrieved chunks and returns analysis as a tool_result.\n5. **Synthesis** — the Executive feeds the tool_results back to Claude and streams one coherent reply; a `thinking` keepalive covers the specialist round.\n6. **Optional committee review** — when enabled, `Executive.stream_chat_with_committee()` runs a silent draft, parallel reviewer critiques, then a streamed revision pass.\n7. **Side effects** — any side-effecting tool call surfaces an `action_taken` chip; the stream closes with a `done` event. One side-effecting tool is gated server-side rather than by its description: on a turn that arrived from a chat channel, `ack_alert` will only accept an `alert_id` the session recorded as having been shown to the model this turn — and it fails closed, so an id it cannot even parse is refused rather than waved through to the handler's own parse. The unattended passes (the nightly reflection, the research synthesis) do not carry the tool at all, because they set no origin channel and so would not be checked. After the stream closes, the turn is also handed to the episodic memory extractor (`memory.episodic.schedule_extraction`, fire-and-forget) — but only if the USER's message clears `MIN_USER_CHARS_FOR_EXTRACTION`. Both this path and the committee path share that constant and both measure the user's side only: the floor used to sum in the assistant's reply, which let a verbose answer drag a trivial turn in and discarded the short, decisive instructions that are the whole point of extracting. See **Memory** for what that cost on a live tenant.\n8. **Tool failures** — a handler that raises instead of returning its JSON error is isolated: both tool gathers run with `return_exceptions=True`, the model gets an error `tool_result` naming only the exception's TYPE — a message can carry a filesystem path, a validation error echoing the input, or a third-party HTTP body, and anything in model context can be quoted back to the user, so the full repr stays in the log and the audit row — and the failure is audited with `ok=false`. Only `CancelledError` is re-raised, so turn-timeout and client-disconnect handling still work. Before this one crashing tool aborted the whole turn, which reached the user as an unexplained apology with no audit row (#136).\n\nWeb search, when invoked on the Anthropic path, executes inside the Anthropic API itself (`web_search_20250305`) and returns within the same generation; for any model routed via OpenRouter the same tool becomes OpenRouter's `openrouter:web_search` server tool, which also runs inside the generation with the same search cap and domain lists.",+ "markdown": "A single chat message makes a full round-trip through the API, the orchestrator's tool-use loop, a parallel specialist fan-out, and synthesis — all streamed back as server-sent events.\n\n### Steps\n\n1. **Arrival** — the client posts to `POST /chat` (`openexecutive.api.routes.chat`). The turn carries a `<current_speaker>` hint resolved from the caller's `person_id`, plus a `<briefing>` digest of current open alerts (built in parallel with RAG/episodic/peer-memory fetches) so the Executive can discuss any item shown on the `/today` briefing page A turn arriving from a chat adapter instead of the web app also carries a `<channel>` block naming the surface and stating which approvals can be completed there; on Slack the `<briefing>` digest is gated to the principal's DMs. All three ride in the **user turn**, never a cached system block.\n2. **Tool-use loop** — the `Executive` calls Claude. For a cross-domain question the assistant turn emits multiple `consult_specialist` tool-use blocks at once.\n3. **Parallel fan-out** — `openexecutive.orchestrator.router.route_parallel` dispatches those calls concurrently, bounded by `MAX_PARALLEL_SPECIALISTS`; calls past the cap get a skip tool_result.\n4. **Per-specialist RAG** — each specialist runs its own domain-filtered ChromaDB query, then reasons over the retrieved chunks and returns analysis as a tool_result.\n5. **Synthesis** — the Executive feeds the tool_results back to Claude and streams one coherent reply; a `thinking` keepalive covers the specialist round.\n6. **Optional committee review** — when enabled, `Executive.stream_chat_with_committee()` runs a silent draft, parallel reviewer critiques, then a streamed revision pass.\n7. **Side effects** — any side-effecting tool call surfaces an `action_taken` chip; the stream closes with a `done` event. One side-effecting tool is gated server-side rather than by its description: on a turn that arrived from a chat channel, `ack_alert` will only accept an `alert_id` the session recorded as having been shown to the model this turn — and it fails closed, so an id it cannot even parse is refused rather than waved through to the handler's own parse. The unattended passes (the nightly reflection, the research synthesis) do not carry the tool at all, because they set no origin channel and so would not be checked. After the stream closes, the turn is also handed to the episodic memory extractor (`memory.episodic.schedule_extraction`, fire-and-forget) — on every turn where the principal said anything at all. Both this path and the committee path call the same `should_extract` predicate, and it applies no length floor: the gate used to sum in the assistant's reply, which let a verbose answer drag a trivial turn in and discarded the short, decisive instructions that are the whole point of extracting. A user-side floor would fare no better — \"Do B.\" and \"Done\" are one character apart. See **Memory** for what that cost on a live tenant.\n8. **Tool failures** — a handler that raises instead of returning its JSON error is isolated: both tool gathers run with `return_exceptions=True`, the model gets an error `tool_result` naming only the exception's TYPE — a message can carry a filesystem path, a validation error echoing the input, or a third-party HTTP body, and anything in model context can be quoted back to the user, so the full repr stays in the log and the audit row — and the failure is audited with `ok=false`. Only `CancelledError` is re-raised, so turn-timeout and client-disconnect handling still work. Before this one crashing tool aborted the whole turn, which reached the user as an unexplained apology with no audit row (#136).\n\nWeb search, when invoked on the Anthropic path, executes inside the Anthropic API itself (`web_search_20250305`) and returns within the same generation; for any model routed via OpenRouter the same tool becomes OpenRouter's `openrouter:web_search` server tool, which also runs inside the generation with the same search cap and domain lists.","mermaid": "sequenceDiagram\n participant C as Client\n participant API as FastAPI /chat\n participant E as Executive\n participant CL as Claude\n participant S as Specialists\n participant KB as ChromaDB\n C->>API: POST /chat message\n API->>E: start turn\n E->>CL: messages (tool-use loop)\n CL-->>E: consult_specialist calls\n E->>S: route_parallel fan-out\n S->>KB: domain-filtered RAG\n KB-->>S: chunks\n S-->>E: analysis tool_results\n E->>CL: synthesize\n CL-->>E: streamed reply\n E-->>API: SSE chunks then done\n API-->>C: streamed response","generated_at": "2026-09-18T20:54:42.334412+00:00"}diff --git a/packages/core/openexecutive/architecture/prebuilt/memory.json b/packages/core/openexecutive/architecture/prebuilt/memory.jsonindex df755372..3f0fb90c 100644--- a/packages/core/openexecutive/architecture/prebuilt/memory.json+++ b/packages/core/openexecutive/architecture/prebuilt/memory.json@@ -1,7 +1,7 @@{"section_id": "memory","title": "Memory System",- "markdown": "The Executive's durable memory is an episodic SQLite store at `./episodic_memory.db`, managed by `openexecutive.memory.episodic`. It holds four tables:\n\n- `decisions` — choices made, with `domain`, `summary`, `rationale`, `outcome`, and `tags`.\n- `initiatives` — tracked efforts with a `status` and `summary` (idempotent upserts).\n- `advice_given` — query/advice pairs per domain.\n- `scheduled_actions` — a dispatch queue, treated separately from recall memory.\n\n### What gets extracted, and when\n\n`episodic.extract_and_store` runs after a turn and asks a utility-fast model for decisions, initiatives and advice. Every candidate must carry a `user_commitment_quote` that appears verbatim in the principal's own message (`_is_valid_user_commitment`) — the model has repeatedly tried to log the Executive's *recommendations* as the user's commitments, and that quote is the hard gate against it.\n\nA turn reaches the extractor only if the **user's** message clears `MIN_USER_CHARS_FOR_EXTRACTION`. This was a combined user+assistant floor, and on a live tenant it selected almost exactly the wrong turns: 9 of 16 exchanges were blocked, and the 9 held every instruction the principal gave, while the 7 admitted were long analytical exchanges with no commitment in them. The extractor ran 13 times and stored nothing. Decisive instructions are short (\"Correct it there\", \"Don't track this\"); the analysis around them is long, so a floor that scales with the Executive's verbosity discards exactly what it is meant to capture. The floor now measures only the user's side — the one place a commitment, and an acceptable quote, can come from.\n\nEach pass writes a `memory_extraction` audit row: how many items the model proposed, how many were stored, and the reason each drop was rejected. Without it, \"the model found nothing\" and \"the model found things and every one was rejected\" are indistinguishable from outside the database — which is why the failure above survived undetected and took a SQLite session on the tenant to find. A `proposed>0, stored=0` streak is the signal to watch.\n\n### Session scoping\n\n`decisions` and `advice_given` rows carry a `session_id`, so per-thread channels (Discord, Telegram) can pull just-this-thread history without leaking cross-conversation context. A null `session_id` falls back to a global view.\n\n### Surfacing\n\n`openexecutive.memory.episodic.format_for_prompt` renders recent decisions and initiatives into a compact `<past_decisions>` block injected into the user turn — alongside `<retrieved_context>` and peer memory — never into a cached system block.\n\nThe chat route also injects a `<briefing>` block into the same user turn: a compact digest of the current open (unread) alerts, built by `openexecutive.briefing.context.format_open_alerts_for_prompt` (company-wide, `status=\"unread\"`, mirroring `/today`). This is what lets the Executive discuss a briefing item the principal clicked or named on `/today` (e.g. \"Gulf Coast Port Cyberattack\"). It is resolved in parallel with the other context fetches, omitted when there are no open alerts, and never placed in a cached system block.\n\n### System of record\n\nSQL is the system of record; the peer-memory service is a recall layer mirrored from it, not a replacement. When `store_decision` / `store_initiative` / `store_advice` insert a row whose `department` is non-empty and not `\"general\"`, the row is mirrored as a kind-tagged note into that department's peer via `append_department_note`. The mirror fires only after a successful INSERT, is fire-and-forget, and never rolls back the SQL row on peer-memory failure. `scheduled_actions` is never mirrored — it is a queue, not memory.",+ "markdown": "The Executive's durable memory is an episodic SQLite store at `./episodic_memory.db`, managed by `openexecutive.memory.episodic`. It holds four tables:\n\n- `decisions` — choices made, with `domain`, `summary`, `rationale`, `outcome`, and `tags`.\n- `initiatives` — tracked efforts with a `status` and `summary` (idempotent upserts).\n- `advice_given` — query/advice pairs per domain.\n- `scheduled_actions` — a dispatch queue, treated separately from recall memory.\n\n### What gets extracted, and when\n\n`episodic.extract_and_store` runs after a turn and asks a utility-fast model for decisions, initiatives and advice. Every candidate must carry a `user_commitment_quote` that appears verbatim in the principal's own message (`_is_valid_user_commitment`) — the model has repeatedly tried to log the Executive's *recommendations* as the user's commitments, and that quote is the hard gate against it.\n\nA turn reaches the extractor if `should_extract(user_message)` is true, which means only that the principal said something — there is deliberately **no length floor**. There used to be one, on the combined user+assistant length, and on a live tenant it selected almost exactly the wrong turns: 9 of 16 exchanges were blocked, and the 9 held every instruction the principal gave, while the 7 admitted were long analytical exchanges with no commitment in them. The extractor ran 13 times and stored nothing.\n\nMoving that floor to the user's side would relocate the bug, not fix it. The canonical executive decision is a long analysis answered with \"Approve option B.\" (17 chars) or \"Do B.\" (5), and any floor high enough to skip \"Done\" (4) also skips those — \"Do B.\" and \"Done\" differ by one character and mean opposite things. Length cannot separate a decision from an acknowledgement; the verbatim-quote check can, so `_is_valid_user_commitment` is the only gate. The price is one utility-fast call on turns that propose nothing.\n\nEach pass writes a `memory_extraction` audit row — in a `finally`, so a pass that crashed is recorded too, with the exception type as its `failure` reason: how many items the model proposed, how many were stored, and a `{kind, reason}` record per drop (`bad_quote`, `missing_summary`, `not_a_list`, `item_not_a_dict`, `payload_not_a_dict`). The tool payload is not schema-checked anywhere, so a malformed shape is counted as a drop rather than raised — it used to abort the whole pass, discarding the other two kinds with it and suppressing the row, which read exactly like \"extraction never ran\". Without it, \"the model found nothing\" and \"the model found things and every one was rejected\" are indistinguishable from outside the database — which is why the failure above survived undetected and took a SQLite session on the tenant to find. A `proposed>0, stored=0` streak is the signal to watch.\n\n### Session scoping\n\n`decisions` and `advice_given` rows carry a `session_id`, so per-thread channels (Discord, Telegram) can pull just-this-thread history without leaking cross-conversation context. A null `session_id` falls back to a global view.\n\n### Surfacing\n\n`openexecutive.memory.episodic.format_for_prompt` renders recent decisions and initiatives into a compact `<past_decisions>` block injected into the user turn — alongside `<retrieved_context>` and peer memory — never into a cached system block.\n\nThe chat route also injects a `<briefing>` block into the same user turn: a compact digest of the current open (unread) alerts, built by `openexecutive.briefing.context.format_open_alerts_for_prompt` (company-wide, `status=\"unread\"`, mirroring `/today`). This is what lets the Executive discuss a briefing item the principal clicked or named on `/today` (e.g. \"Gulf Coast Port Cyberattack\"). It is resolved in parallel with the other context fetches, omitted when there are no open alerts, and never placed in a cached system block.\n\n### System of record\n\nSQL is the system of record; the peer-memory service is a recall layer mirrored from it, not a replacement. When `store_decision` / `store_initiative` / `store_advice` insert a row whose `department` is non-empty and not `\"general\"`, the row is mirrored as a kind-tagged note into that department's peer via `append_department_note`. The mirror fires only after a successful INSERT, is fire-and-forget, and never rolls back the SQL row on peer-memory failure. `scheduled_actions` is never mirrored — it is a queue, not memory.","mermaid": "flowchart TD\n Store[\"store_decision / store_initiative / store_advice\"]\n DB[\"episodic_memory.db<br/>decisions / initiatives<br/>advice_given / scheduled_actions\"]\n Fmt[\"format_for_prompt\"]\n Turn[\"User turn<br/>past_decisions block\"]\n Mirror[\"append_department_note\"]\n Peer[\"Department peer<br/>(Honcho)\"]\n Store --> DB\n DB --> Fmt\n Fmt --> Turn\n Store -->|\"dept set & not general\"| Mirror\n Mirror -->|\"fire-and-forget\"| Peer\n class Store entry\n class DB storage\n class Fmt compute\n class Turn hot\n class Peer external\n","generated_at": "2026-06-01T00:00:00+00:00"}diff --git a/packages/core/openexecutive/memory/episodic.py b/packages/core/openexecutive/memory/episodic.pyindex 16440611..2ad9b800 100644--- a/packages/core/openexecutive/memory/episodic.py+++ b/packages/core/openexecutive/memory/episodic.py@@ -1833,44 +1833,79 @@ def _is_valid_user_commitment(quote: str, user_message: str) -> bool:_MAX_INPUT_CHARS = 20_000 # cap each side to avoid runaway cost-# Minimum chars in the USER's message required to schedule extraction.-#-# This was a combined user+assistant floor of 1500, and measured against real-# traffic it selected almost exactly the wrong turns. On one tenant it blocked-# 9 of 16 exchanges, and the 9 included every substantive instruction the-# principal gave — "It's a mistake on the lp purchased properties worksheet.-# Correct it there. And record this as fixed." (1312 combined), "This is not-# relevant for us. Don't track this" (910), "The briefing is incorrect. We are-# not under water 2.9m" (1032) — while the 7 it admitted were long analytical-# exchanges containing no user commitment at all. The extractor ran 13 times-# and stored nothing, because it only ever saw the turns with nothing in them.-#-# The combined length was the wrong axis. A decision lives in what the-# PRINCIPAL said, which is also the only text `_is_valid_user_commitment` will-# accept a quote from; the Executive's analysis around it can run to thousands-# of characters without containing a single commitment. Decisive instructions-# are short by nature ("Correct it there", "Don't track this"), so a floor that-# scales with the assistant's verbosity systematically discards them.-#-# The floor now only has to separate a real instruction from a bare-# acknowledgement — "No", "Done", "Who am I?" — which is what the old comment-# meant by small talk. Both stream_chat and the committee path use this-# constant; keeping them symmetric prevents the committee path from firing more-# often than streaming, which it used to.-MIN_USER_CHARS_FOR_EXTRACTION = 40--# Back-compat alias. External callers (and older tests) may still import the-# old name; it no longer reflects what is measured, so nothing new should use-# it.-MIN_TURN_CHARS_FOR_EXTRACTION = MIN_USER_CHARS_FOR_EXTRACTION+def should_extract(user_message: str) -> bool:+ """True when this turn is worth an extraction pass.++ There is deliberately no length floor. There used to be one on the+ combined user+assistant length, which discarded short instructions+ answered at length — on a live tenant it blocked every commitment the+ principal made while admitting only the long analytical exchanges that+ had none, and the extractor ran 13 times storing nothing.++ Moving that floor to the user's side does not fix it, it relocates it: the+ canonical executive decision is a long analysis answered with "Approve+ option B." (17 chars) or "Do B." (5), and any floor high enough to skip+ "Done" (4) also skips those. Length cannot separate a decision from an+ acknowledgement — "Do B." and "Done" differ by one character and mean+ opposite things.++ `_is_valid_user_commitment` is the gate that actually tests for a+ commitment: it requires a verbatim quote from the user's own words. A pass+ over an acknowledgement costs one utility-fast call (~$0.005 observed) and+ stores nothing, which is the right trade against losing approvals. If+ per-turn cost ever becomes the binding constraint, the lever is a semantic+ prefilter or a per-session cap — not a length proxy for a property it+ cannot measure.++ The single decision point for both call sites in+ `orchestrator.executive`, so the rule is testable directly and the two+ paths cannot drift apart.+ """+ return bool(user_message.strip())+++# Payload key -> the `kind` a drop is recorded under. The per-item drops below+# are singular, so grouping the audit rows by `kind` has to see one spelling.+_ITEM_KINDS = {"decisions": "decision", "initiatives": "initiative", "advice": "advice"}+++def _iter_items(+ payload: dict[str, Any], key: str, dropped: list[dict[str, str]]+) -> list[dict[str, Any]]:+ """The dict-shaped items under ``key``, skipping anything malformed.++ The model's tool payload is not schema-checked, so `{"decisions": "..."}`+ or `{"decisions": ["text"]}` used to raise out of the whole pass — taking+ the other two kinds with it AND suppressing the audit row, which left the+ log looking exactly like "extraction never ran". Malformed shapes are now+ counted as drops and the pass continues.++ A missing key or an explicit `null` is not a drop: the model omitting a+ kind it found nothing for is the normal case.+ """+ kind = _ITEM_KINDS[key]+ raw = payload.get(key)+ if raw is None:+ return []+ if not isinstance(raw, list):+ dropped.append({"kind": kind, "reason": "not_a_list"})+ return []+ out: list[dict[str, Any]] = []+ for item in raw:+ if isinstance(item, dict):+ out.append(item)+ else:+ dropped.append({"kind": kind, "reason": "item_not_a_dict"})+ return outdef _audit_extraction(proposed: dict[str, int],stored: dict[str, int],- dropped: list[str],+ dropped: list[dict[str, str]],*,session_id: str,+ failure: str = "",) -> None:"""One `memory_extraction` audit row per extraction pass.@@ -1887,20 +1922,24 @@ def _audit_extraction("""total_proposed = sum(proposed.values())total_stored = sum(stored.values())+ prefix = f"FAILED({failure}) " if failure else ""try:from openexecutive.audit import log_eventlog_event("memory_extraction",- f"proposed={total_proposed} stored={total_stored} dropped={len(dropped)}",+ f"{prefix}proposed={total_proposed} stored={total_stored} "+ f"dropped={len(dropped)}",session_id=session_id or None,actor="memory_extractor",details={"proposed": proposed,"stored": stored,"dropped_count": len(dropped),- # Reasons, not content: enough to tell a validator rejection- # from an empty pass without copying company data into the log.+ "failure": failure,+ # Structured like `proposed`/`stored` so "how many bad-quote+ # drops on decisions this week" is a query, not a string split+ # over a field that can itself contain colons."dropped": dropped[:10],},)@@ -1958,6 +1997,10 @@ async def _extract_and_store(db_path: Path,session_id: str,) -> None:+ proposed = {"decisions": 0, "initiatives": 0, "advice": 0}+ stored = {"decisions": 0, "initiatives": 0, "advice": 0}+ dropped: list[dict[str, str]] = []+ failure = ""try:from openexecutive.audit.usage import log_model_usagefrom openexecutive.config import get_settings@@ -2009,15 +2052,15 @@ async def _extract_and_store(# total failure was an empty `decisions` table nobody was watching. It# took reading a tenant's SQLite to find that the turn gate had been# discarding every commitment for the whole life of the install.- proposed = {"decisions": 0, "initiatives": 0, "advice": 0}- stored = {"decisions": 0, "initiatives": 0, "advice": 0}- dropped: list[str] = []for block in response.content:if block.type != "tool_use" or block.name != "store_memories":continueinp = block.input+ if not isinstance(inp, dict):+ dropped.append({"kind": "pass", "reason": "payload_not_a_dict"})+ continue# Validate every item against the user's actual text. The LLM has# repeatedly proven willing to log the executive's recommendations# as if the user had committed to them — see the May 27 incident@@ -2026,15 +2069,16 @@ async def _extract_and_store(# The user_commitment_quote field + this validator are the# hard gate that catches that pattern; the prompt is now just# the soft instruction layer.- for d in inp.get("decisions", []):+ for d in _iter_items(inp, "decisions", dropped):proposed["decisions"] += 1domain = d.get("domain", "general")summary = d.get("summary", "")quote = d.get("user_commitment_quote", "")if not summary:+ dropped.append({"kind": "decision", "reason": "missing_summary"})continueif not _is_valid_user_commitment(quote, user_message):- dropped.append(f"decision:bad_quote:{summary[:60]}")+ dropped.append({"kind": "decision", "reason": "bad_quote", "summary": summary[:60]})logger.debug("Dropping decision — invalid user_commitment_quote %r (summary=%r)",quote[:120],@@ -2049,16 +2093,17 @@ async def _extract_and_store(session_id=session_id,db_path=db_path,)- for i in inp.get("initiatives", []):+ for i in _iter_items(inp, "initiatives", dropped):proposed["initiatives"] += 1title = i.get("title", "")status = i.get("status", "active")summary = i.get("summary", "")quote = i.get("user_commitment_quote", "")if not (title and summary):+ dropped.append({"kind": "initiative", "reason": "missing_field"})continueif not _is_valid_user_commitment(quote, user_message):- dropped.append(f"initiative:bad_quote:{title[:60]}")+ dropped.append({"kind": "initiative", "reason": "bad_quote", "title": title[:60]})logger.debug("Dropping initiative — invalid user_commitment_quote %r (title=%r)",quote[:120],@@ -2067,16 +2112,17 @@ async def _extract_and_store(continuestored["initiatives"] += 1store_initiative(title=title, status=status, summary=summary, db_path=db_path)- for a in inp.get("advice", []):+ for a in _iter_items(inp, "advice", dropped):proposed["advice"] += 1domain = a.get("domain", "general")query_summary = a.get("query_summary", "")advice_summary = a.get("advice_summary", "")quote = a.get("user_commitment_quote", "")if not (query_summary and advice_summary):+ dropped.append({"kind": "advice", "reason": "missing_field"})continueif not _is_valid_user_commitment(quote, user_message):- dropped.append(f"advice:bad_quote:{query_summary[:60]}")+ dropped.append({"kind": "advice", "reason": "bad_quote", "query_summary": query_summary[:60]})logger.debug("Dropping advice — invalid user_commitment_quote %r (query=%r)",quote[:120],@@ -2092,10 +2138,19 @@ async def _extract_and_store(db_path=db_path,)- _audit_extraction(proposed, stored, dropped, session_id=session_id)-- except Exception:+ except Exception as exc:+ # Recorded so the audit row can say the pass FAILED. Without it a+ # provider outage writes no rows at all, which reads identically to+ # "extraction was never scheduled" — the indistinguishable-failure+ # state this row exists to eliminate.+ failure = type(exc).__name__logger.exception("Episodic memory extraction failed — skipping silently")+ finally:+ # In `finally`, not the happy path: a pass that crashed is exactly the+ # one an operator needs to see.+ _audit_extraction(+ proposed, stored, dropped, session_id=session_id, failure=failure+ )def schedule_extraction(diff --git a/packages/core/openexecutive/orchestrator/executive.py b/packages/core/openexecutive/orchestrator/executive.pyindex 4c13e540..2a15a7d8 100644--- a/packages/core/openexecutive/orchestrator/executive.py+++ b/packages/core/openexecutive/orchestrator/executive.py@@ -792,8 +792,8 @@ class Executive:)from openexecutive.memory.episodic import (- MIN_USER_CHARS_FOR_EXTRACTION,schedule_extraction,+ should_extract,)# Re-bind the audit ContextVars for the duration of these calls so@@ -808,7 +808,7 @@ class Executive:# call time, so scheduling it out here would snapshot (None,# None) and the memory_extractor's model call would record# unattributed however correct the snapshot itself was.- if len(user_message.strip()) >= MIN_USER_CHARS_FOR_EXTRACTION:+ if should_extract(user_message):schedule_extraction(user_message, full_response, session_id=session.session_id)@@ -1220,10 +1220,10 @@ class Executive:)from openexecutive.memory.episodic import (- MIN_USER_CHARS_FOR_EXTRACTION,schedule_extraction,+ should_extract,)- if len(user_message.strip()) >= MIN_USER_CHARS_FOR_EXTRACTION:+ if should_extract(user_message):schedule_extraction(user_message, final_response, session_id=session.session_id)# Mirror the completed exchange into Honcho (see stream_chat fordiff --git a/packages/core/tests/unit/test_episodic.py b/packages/core/tests/unit/test_episodic.pyindex 6718b4ea..3ed21722 100644--- a/packages/core/tests/unit/test_episodic.py+++ b/packages/core/tests/unit/test_episodic.py@@ -443,52 +443,43 @@ def test_extraction_prompt_includes_source_attribution_rule() -> None:)-def test_extraction_floor_admits_instructions_and_blocks_acknowledgements() -> None:- """The floor gates extraction in both stream_chat and the committee path.-- This asserted `>= 1000` on a COMBINED user+assistant length, to stop- extraction firing on trivial Q&A. The cost concern was real; the axis was- not. Measured on a live tenant, that floor blocked 9 of 16 exchanges and- the 9 contained every instruction the principal gave, because decisive- instructions are short and the Executive's analysis around them is long.- The extractor ran 13 times and stored nothing.-- The floor now measures the user's own message — the only text a commitment- can come from, and the only text the quote validator will accept. Pinned to- real messages rather than a magic number, so a future change is judged on- what it admits and blocks, not on whether a constant looks big enough.+def test_extraction_gate_has_no_length_floor() -> None:+ """The floor was removed, not moved.++ It began as a combined user+assistant length and discarded short+ instructions answered at length. Relocating it to the user's side only+ relocates the bug: "Do B." (5 chars) and "Done" (4) differ by one+ character and mean opposite things, so no threshold separates them.+ `_is_valid_user_commitment` is the gate that actually tests for a+ commitment."""- from openexecutive.memory.episodic import MIN_USER_CHARS_FOR_EXTRACTION-- assert isinstance(MIN_USER_CHARS_FOR_EXTRACTION, int)- # Real instructions from the tenant that the old floor discarded.- for instruction in (- "This is not relevant for us. Don't track this",- "Well remember you fixed it or something. So we don't keep getting. Notified.",- ):- assert len(instruction.strip()) >= MIN_USER_CHARS_FOR_EXTRACTION-- # Bare acknowledgements still buy nothing — this is what the floor is for.- for noise in ("No", "Done", "ok", "yes", "Nope."):- assert len(noise.strip()) < MIN_USER_CHARS_FOR_EXTRACTION---def test_executive_call_sites_use_min_turn_chars_constant() -> None:- """Both extraction trigger sites in executive.py must reference the- constant, not a hard-coded number. Previously they were 300 (committee)- and 800 (stream) — asymmetric, low, and easy to drift further. Pin via- source inspection because the call sites live inside long async streaming- methods that aren't unit-testable end-to-end.+ from openexecutive.memory import episodic++ for decision in ("Do B.", "Approve option B.", "Kill it.", "no, drop that"):+ assert episodic.should_extract(decision)+ for blank in ("", " "):+ assert not episodic.should_extract(blank)+ assert not hasattr(episodic, "MIN_USER_CHARS_FOR_EXTRACTION")+ assert not hasattr(episodic, "MIN_TURN_CHARS_FOR_EXTRACTION")+++def test_executive_call_sites_use_the_shared_gate() -> None:+ """Both extraction trigger sites in executive.py must call the shared+ predicate, not inline a length comparison. Previously they were 300+ (committee) and 800 (stream) — asymmetric, low, and easy to drift further.+ Pin via source inspection because the call sites live inside long async+ streaming methods that aren't unit-testable end-to-end."""import inspectfrom openexecutive.orchestrator import executivesrc = inspect.getsource(executive)- # The constant must be referenced at least twice (once per call site).- assert src.count("MIN_USER_CHARS_FOR_EXTRACTION") >= 2, (- "executive.py must reference MIN_USER_CHARS_FOR_EXTRACTION in both "- "the stream_chat and stream_chat_with_committee extraction guards"+ # Each call site contributes an import AND a call, so >= 2 would be+ # satisfied by a single surviving site — a partial revert would pass.+ assert src.count("should_extract(user_message)") == 2, (+ "both the stream_chat and stream_chat_with_committee extraction "+ "guards must call should_extract(user_message)")# And they must measure the USER's message. Summing in the assistant's# length is the original defect: it let a verbose reply drag a trivial turndiff --git a/packages/core/tests/unit/test_extraction_gate.py b/packages/core/tests/unit/test_extraction_gate.pyindex 671b912f..df03c821 100644--- a/packages/core/tests/unit/test_extraction_gate.py+++ b/packages/core/tests/unit/test_extraction_gate.py@@ -11,14 +11,31 @@ The turns below are the real ones, with their real lengths."""from __future__ import annotations+from pathlib import Pathfrom typing import Anyfrom unittest import mockimport pytestfrom openexecutive.memory import episodic+from openexecutive.memory.episodic import initialize_db-# (user message, assistant length, what it is) — measured from the tenant.++def db(tmp_path: Path) -> Path:+ """An isolated episodic DB.++ Without it these passes write real rows into `./episodic_memory.db`, which+ other modules then read back as production data — the audit-log pollution+ trap in CLAUDE.md.+ """+ db_path = tmp_path / "episodic.db"+ initialize_db(db_path)+ return db_path++# Real turns from the tenant, with the assistant length that accompanied them.+# Each is the principal giving an instruction, and each was discarded by the+# combined-length floor._COMMITMENTS = [("It's a mistake on the lp purchased properties worksheet. That's it. ""Correct it there. And record this as fixed.", 1200),@@ -29,61 +46,50 @@ _COMMITMENTS = ["Notified.", 612),]-# Bare acknowledgements and one-line questions: nothing to extract from the-# user's own words, which is what the floor is actually for.-_NOISE = ["No", "Done", "Nope.", "Who am I?", "ok", "yes"]+# The shape a length floor cannot handle: a long analysis answered in a few+# words. A floor high enough to skip "Done" also skips "Do B.".+_SHORT_APPROVALS = ["Approve option B.", "Do B.", "Kill it. Approved.", "Yes, ship it."]@pytest.mark.parametrize("message,assistant_len", _COMMITMENTS)def test_real_commitments_reach_the_extractor(message: str, assistant_len: int) -> None:- """Every one of these was blocked by the old combined-length floor.-- Each is well under 1500 chars combined — `len(user) + assistant_len` — yet- each is the principal giving a decision. Asserted against the real lengths- so the test states the bug rather than restating the new constant.- """- assert len(message) + assistant_len < 1500, "should have been blocked before"- assert len(message.strip()) >= episodic.MIN_USER_CHARS_FOR_EXTRACTION+ """Drives the real gate, not arithmetic on literals."""+ assert len(message) + assistant_len < 1500, "blocked by the old combined floor"+ assert episodic.should_extract(message)[email protected]("message", _NOISE)-def test_bare_acknowledgements_are_still_skipped(message: str) -> None:- """The floor still has a job: an LLM call on "No" buys nothing."""- assert len(message.strip()) < episodic.MIN_USER_CHARS_FOR_EXTRACTION[email protected]("message", _SHORT_APPROVALS)+def test_short_approvals_reach_the_extractor(message: str) -> None:+ """The canonical executive decision: long analysis, three-word answer.--def test_a_long_assistant_reply_no_longer_drags_a_short_turn_in() -> None:- """The inverse of the bug: verbosity on the Executive's side is not- evidence the principal committed to anything, and the quote validator will- only ever accept text from the user's message anyway."""- assert len("Done".strip()) < episodic.MIN_USER_CHARS_FOR_EXTRACTION+ A user-side length floor regressed these — "Approve option B." is 17 chars+ and the old combined floor DID admit it after a long reply. Any floor that+ skips "Done" (4) also skips "Do B." (5), so there is no floor.+ """+ assert episodic.should_extract(message)-def test_old_constant_still_resolves_for_external_callers() -> None:- assert episodic.MIN_TURN_CHARS_FOR_EXTRACTION == episodic.MIN_USER_CHARS_FOR_EXTRACTION+def test_empty_turns_are_skipped() -> None:+ for blank in ("", " ", "\n\t "):+ assert not episodic.should_extract(blank)-async def test_extraction_audits_proposed_stored_and_dropped() -> None:- """A pass that proposes items and stores none must be visible.+async def _run_pass(+ payload: Any,+ db_path: Path,+ *,+ user_message: str = "Cut burn to 400k. Tell the team.",+) -> tuple[mock.MagicMock, list[dict[str, Any]]]:+ """Drive one real extraction pass over a model payload.- That is exactly what happened in production, and it was indistinguishable- from "the model found nothing" because drops were logger.debug and stores- wrote no row.+ The payload is whatever the model put in its `store_memories` tool block —+ it is NOT schema-checked anywhere, so tests hand it the shapes a model+ actually emits, including the malformed ones."""class _Block:type = "tool_use"name = "store_memories"- input: dict[str, Any] = {- "decisions": [- # Valid: the quote is the user's own words, terminated.- {"domain": "finance", "summary": "Cut burn to 400k",- "user_commitment_quote": "Cut burn to 400k"},- # Invalid: quote is not in the user message at all.- {"domain": "finance", "summary": "Sell the building",- "user_commitment_quote": "sell the building"},- ],- }+ input = payloadclass _Response:content = [_Block()]@@ -105,13 +111,143 @@ async def test_extraction_audits_proposed_stored_and_dropped() -> None:return_value=_Response())await episodic.extract_and_store(- "Cut burn to 400k. Tell the team.", "Understood.", session_id="s-1"+ user_message, "Understood.", db_path=db_path, session_id="s-1")+ return store, [r for r in rows if r["event_type"] == "memory_extraction"]+++async def test_extraction_audits_proposed_stored_and_dropped(db: Path) -> None:+ """A pass that proposes items and stores none must be visible.++ That is exactly what happened in production, and it was indistinguishable+ from "the model found nothing" because drops were logger.debug and stores+ wrote no row.+ """+ store, audit = await _run_pass(db_path=db, payload={+ "decisions": [+ # Valid: the quote is the user's own words, terminated.+ {"domain": "finance", "summary": "Cut burn to 400k",+ "user_commitment_quote": "Cut burn to 400k"},+ # Invalid: quote is not in the user message at all.+ {"domain": "finance", "summary": "Sell the building",+ "user_commitment_quote": "sell the building"},+ ],+ })+assert store.call_count == 1- audit = [r for r in rows if r["event_type"] == "memory_extraction"]assert len(audit) == 1assert audit[0]["details"]["proposed"]["decisions"] == 2assert audit[0]["details"]["stored"]["decisions"] == 1assert audit[0]["details"]["dropped_count"] == 1- assert "decision:bad_quote:Sell the building" in audit[0]["details"]["dropped"]+ assert audit[0]["details"]["dropped"] == [+ {"kind": "decision", "reason": "bad_quote", "summary": "Sell the building"}+ ]+ assert audit[0]["details"]["failure"] == ""+++# --------------------------------------------------------------------- #+# Malformed model payloads+#+# `block.input` is whatever the model emitted; nothing validates its shape+# before the loops index into it. A `null` or a list of bare strings used to+# raise out of the whole pass — which took the other two kinds with it AND+# skipped the audit row, leaving a log indistinguishable from "extraction+# never ran". That is the exact blind spot this work exists to remove, so the+# malformed shapes must still produce a row.+# --------------------------------------------------------------------- #+++ "payload,reason",+ [+ ({"decisions": "Cut burn"}, "not_a_list"),+ ({"decisions": 7}, "not_a_list"),+ ({"decisions": ["Cut burn to 400k"]}, "item_not_a_dict"),+ ({"decisions": [None]}, "item_not_a_dict"),+ ],+ ids=["str", "int", "list-of-str", "list-of-null"],+)+async def test_malformed_items_are_dropped_not_raised(+ payload: dict[str, Any], reason: str, db: Path+) -> None:+ store, audit = await _run_pass(payload, db)++ assert store.call_count == 0+ assert len(audit) == 1, "the pass must still audit itself"+ assert audit[0]["details"]["failure"] == "", "must not have raised"+ assert audit[0]["details"]["dropped"] == [+ {"kind": "decision", "reason": reason}+ ], "the kind must match the singular spelling used by per-item drops"+++async def test_a_null_kind_is_not_a_drop(db: Path) -> None:+ """The model omitting a kind is normal, not malformed."""+ _, audit = await _run_pass(+ db_path=db, payload={"decisions": None, "initiatives": []}+ )++ assert audit[0]["details"]["dropped"] == []+ assert audit[0]["details"]["proposed"] == {+ "decisions": 0, "initiatives": 0, "advice": 0+ }+++async def test_one_malformed_kind_does_not_lose_the_others(db: Path) -> None:+ """A bad `decisions` value used to abort the pass before `initiatives`+ was ever read, silently discarding good items alongside the bad one."""+ _, audit = await _run_pass(db_path=db, payload={+ "decisions": "oops",+ "initiatives": [+ {"title": "Cut burn", "summary": "Down to 400k",+ "user_commitment_quote": "Cut burn to 400k"},+ ],+ })++ assert audit[0]["details"]["stored"]["initiatives"] == 1+ assert audit[0]["details"]["dropped"] == [+ {"kind": "decision", "reason": "not_a_list"}+ ]+++async def test_a_non_dict_payload_is_dropped(db: Path) -> None:+ store, audit = await _run_pass(["decisions"], db)++ assert store.call_count == 0+ assert audit[0]["details"]["dropped"] == [+ {"kind": "pass", "reason": "payload_not_a_dict"}+ ]+++async def test_a_crashed_pass_still_audits_with_a_failure_reason(db: Path) -> None:+ """A provider outage that writes no row at all reads exactly like+ "extraction was never scheduled" — the state this row exists to rule out.+ """+ rows: list[dict[str, Any]] = []++ def _capture(event_type: str, summary: str, **kw: Any) -> None:+ rows.append({"event_type": event_type, "summary": summary, **kw})++ with (+ mock.patch("openexecutive.audit.log_event", _capture),+ mock.patch("openexecutive.config.get_settings"),+ mock.patch.object(episodic, "get_active_initiatives", return_value=[]),+ mock.patch("openexecutive.providers.get_provider") as provider,+ ):+ provider.return_value.messages_create = mock.AsyncMock(+ side_effect=RuntimeError("upstream 529")+ )+ await episodic.extract_and_store(+ "Cut burn to 400k.", "ok", db_path=db, session_id="s-1"+ )++ audit = [r for r in rows if r["event_type"] == "memory_extraction"]+ assert len(audit) == 1+ assert audit[0]["details"]["failure"] == "RuntimeError"+ assert audit[0]["summary"].startswith("FAILED(RuntimeError)")