Skip to content

Self-improving AI Employees for everyone

feat(chat): stream the answer, show working steps and stop a run

trohitg76e56cfddfc4
diff --git a/CLAUDE.md b/CLAUDE.md
index a11a6770..b5917f6f 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -14,7 +14,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire
| **[Temporal Cleanup & Resilience Plan (Waves 15-18)](./docs-internal/ARCHIVE/TEMPORAL_CLEANUP_AND_RESILIENCE_PLAN.md)** | *Archived — all four waves shipped.* 15 dead-code retirement (Redis-Streams branch in event_waiter, APScheduler cron stack, orphaned `connection_status` factory), 16 per-queue task routing (`TemporalWorkerPool`), 17 resilience hardening (cron `catchup_window`, SDK interceptors, `DEPLOYMENT_MODE`, periodic heartbeats; the one-shot LLM-step retry it records was later reverted to unlimited-with-backoff), 18 worker performance tuning. Current settings live in [Temporal Architecture](./docs-internal/TEMPORAL_ARCHITECTURE.md). |
| **[Node Allowlist](./docs-internal/node_allowlist.md)** | Single-config UI visibility — `server/config/node_allowlist.json` controls which nodes / credential categories / skill folders show in the UI. Five lists with two enforcement tiers (mode-gated allowlist + absolute blocklist). `useNodeAllowlist` hook exposes `isVisible` / `isBlocked` / `isAllowed` / `isCredentialCategoryDisabled` / `isSkillFolderDisabled`. Adding a new disabled domain = single JSON edit, no code change. |
| **[Normal Mode (Home)](./docs-internal/normal_mode.md)** | The owner-facing landing screen: describe a job, an LLM on the server drafts a setup screen (a JSON UI spec the client parses, normalises and renders from a fixed catalogue), and Hire builds, validates, saves and starts one workflow per AI employee (`services/employees/`, graph built from `config/employee_apps.json` plus the owner's skill library and a Canvas node, and checked against `node_allowlist.json` `enabled_nodes`). Covers the screen switch (`app/`, `shellMode` / `ui_shell_mode`, `VITE_NORMAL_MODE=false` kill switch, Home shows light/dark only), the employee page (the shared chat, `features/chat`, with the employee named in the header beside New conversation; Pause is in the Workspace header), deleting an employee from the team sidebar (stopped through Reset first, then gone from every tab), the `approvalGate` "ask me before sending" step, run records for "done today" (`machina-run-record-v1`), automatic-pause reasons, Settings (Profile fields; usage-only Billing via `get_employee_usage`; Skills, the user-skills library each new hire copies onto a Skills node; consumer Connectors with `publisher` / `verified`; Plugins, the starter bundles in `hire/starters.json`; the last three share one catalog page), the Workspace dock (an employee's Canvas board, named by the summary's `canvas_node_id`; a live Browser tab with user takeover, shared with Dev mode; Mobile shows the managed local Android phone), the "Web browser" app (read-only under "ask me first" through the tool's `ask_first_params`; while the agent waits in `request_user` the summary's `browser_request` shows Needs you and Help in browser), Talk on the employee page (a thread over the chat session whose id is the workflow id, rows written through `services/chat_thread.py` and, for the owner's messages and their chat runs, `services/chat/ledger.py` (see [Chat Protocol](./docs-internal/chat_protocol.md)); the talk line is chatTrigger → an agent → `chatReply` "Reply in Chat": a chat hire's own agent is its talk agent, other hires get a "Talk with {name}" agent sharing the worker's tools and skills with its own Context, and schedule hires also "Post to Talk"), Turn on Talk (for talk `off`) and Apply (while the summary's `pending_changes` is true) via `enable_employee_talk` / `apply_employee_changes`, each a restart on the saved graph through `services/deployment/restart.py`, self-extension from Talk through the Agent Builder within the Hire rule (`services/employees/policy.py`), the 3D orb, the wire contract and the `employee_lifecycle` / `approval_lifecycle` / `chat.updated` broadcasts. |
-| **[Chat Protocol](./docs-internal/chat_protocol.md)** | The wire contract between the chat runtime (`server/services/chat/`) and the shared chat UI: AG-UI's event model on the existing WebSocket, snake_case, in the CloudEvents envelope. A message a deployed chat trigger will answer starts a **chat run** admitted with it in one reserved write transaction (`Database.reserved_session`, SQLite `BEGIN IMMEDIATE`): one live run per session (the lane, also a partial unique index; a second send answers `run_in_progress`), idempotent `client_message_id`, claimed and finished by the MachinaWorkflow that answers it behind `machina-chat-run-v1` (the run id is trusted only from an event whose source is `opencompany://services/chat`; `run_scope` rides every node context; only the claimant finishes), the reply saved as `a_<run id>` by `chatReply`, every message appended after the session's active leaf (`chat_threads`) so concurrent writes never fork the thread. `chat_run_event` frames reach only sockets that passed `chat_subscribe`'s owner check (`services/chat/hub.py`: per-run `seq`, `hub_epoch`, bounded per-socket queues with a resync), never the status broadcast. A watchdog (`services/chat/watchdog.py`, started by `main.py`) ends runs nothing will finish (`not_delivered`, `timed_out`, `interrupted`). Lists every event, handler, part and error code, including the ones later phases build. |
+| **[Chat Protocol](./docs-internal/chat_protocol.md)** | The wire contract between the chat runtime (`server/services/chat/`) and the shared chat UI: AG-UI's event model on the existing WebSocket, snake_case, in the CloudEvents envelope. A message a deployed chat trigger will answer starts a **chat run** admitted with it in one reserved write transaction (`Database.reserved_session`, SQLite `BEGIN IMMEDIATE`): one live run per session (the lane, also a partial unique index; a second send answers `run_in_progress`), idempotent `client_message_id`, claimed and finished by the MachinaWorkflow that answers it behind `machina-chat-run-v1` (the run id is trusted only from an event whose source is `opencompany://services/chat`; `run_scope` rides every node context; only the claimant finishes), the reply saved as `a_<run id>` by `chatReply`, every message appended after the session's active leaf (`chat_threads`) so concurrent writes never fork the thread. `chat_run_event` frames reach only sockets that passed `chat_subscribe`'s owner check (`services/chat/hub.py`: per-run `seq`, `hub_epoch`, bounded per-socket queues with a resync), never the status broadcast. A watchdog (`services/chat/watchdog.py`, started by `main.py`) ends runs nothing will finish (`not_delivered`, `timed_out`, `interrupted`). The agent wired into Reply in Chat (`answers_chat_run`) streams its answer as `text.*` events (`services/chat/stream.py`, activity input only, no workflow patch), its tool calls show as steps (`services/chat/steps.py`; label from the plugin's `chat_step`), and `stop_chat_run` stops a run cooperatively: its LLM step polls the run and returns what was shown, unstarted tools are skipped, the reply is saved `stopped`, and the watchdog cancels a run still stopping after `runs.stop_grace_s`. Lists every event, handler, part and error code, including the ones later phases build. |
| **[Theme System](./docs-internal/theme_system.md)** | 12-way visual theme system — 2 base (light, dark) + 5 utopian (renaissance, greek, edo, steampunk, atomic) + 5 dystopian (cyber, wasteland, rot, plague, surveillance) — driven by `<html data-theme>` + per-theme CSS files in `client/src/themes/`. Token taxonomy (surface / fg / border / accent / typography / motion) in **hex + `color-mix()`**; `@theme inline` bridge maps `--color-X: var(--X)` (no `hsl()` wrapper); per-theme files own the colour hex, base.css owns the shared `--tint-*` alpha scale; action role tokens (incl. `-ink` readable text), **per-theme `--code-*` syntax tokens** (tier 6 — the code editor, console/output JSON viewers, and chat code blocks paint in each theme's own palette: keyword→trigger, string→success, number→agent, function→model, on an adaptive dark/light code surface; replaced the global dracula `--prism-*` block + dead `getPrismTokenCSS`; the `OutputPanel` JSON viewer reads the same vars), per-theme `--pulse-keyframe` animation system + `.opencompany-*` helpers in `animations.css`, decorative-layer wrappers (`.app-frame` / `.canvas-host` / `.modal-frame`), per-component decorative ornaments (panel textures + canvas decorations + node pseudo-element overlays + theme-specific keyframes), **canvas-node visual contract via `--node-color` CSS custom property** (no inline `background` / `border` on node components — base.css + per-theme CSS owns visuals; `NodeStyle` helper type at [types/NodeTypes.ts](./client/src/types/NodeTypes.ts) makes the inline custom-prop typecheck-clean), **`--node-pulse-color` separate from `--node-color`** so executing-node glow uses each theme's highest-contrast accent regardless of plugin accent (Cyber neon cyan, Surveillance REC red, Renaissance ultramarine, etc.), **`data-page-hidden` animation pause** (set by `app/usePageActivitySync.ts`, mounted in `AppShell` on both screens, while the tab is hidden or the window is blurred; base.css declares `html[data-page-hidden] *, *::before, *::after { animation-play-state: paused !important }` to prevent compositor stall on tab return), **per-theme icon glyph system** (290 inline SVG glyph strings across 29 keys × 10 themes via [themedGlyphs.ts](./client/src/assets/icons/themedGlyphs.ts) + theme-aware [NodeIcon.tsx](./client/src/assets/icons/NodeIcon.tsx)), **per-theme canvas-grid + custom cursors** via `--canvas-grid` / `--cursor-default` slots, **decorative HTML primitives** (`<SvgFilterDefs>` mounting `#ink-blot` / `#noise` / `#crt` filter IDs at app root, `<DropCap>` wrapper for Renaissance ornament rule), **parameter panel migrated to Tailwind tokens** (no `useAppTheme()` reads; section headers carry the display-typography triplet; raw `<Button>` swapped to `<ActionButton intent>`), **per-theme scrollbar webkit rules** in all 12 themes, 9-event WebAudio sound system (10 packs via `--sound-pack` token + `useSound()` hook + global hover delegate + sonner toast monkey-patch + `withSound()` HOC + `Sounds.unlock()` gesture-unlock for AudioContext autoplay-policy compliance), `@media (prefers-reduced-motion: reduce)` accessibility, 30 ms throttle on `type` / `hover`, migration recipe, anti-patterns. Read this before adding a new theme, migrating a component to the new contract, or adding a canvas-node component. |
| **[Design System Bundle](./docs-internal/design-system/IMPLEMENTATION.md)** | Canonical, vendored design-system reference (extracted from this repo; see its `IMPLEMENTATION.md` for the contract). Tokens are **hex + `color-mix()`** (`tokens/{colors,typography,spacing,motion,fonts,base}.css`) — the format the live codebase standardizes on. Includes 34 reference components (`components/`, inline-style reference only — recreate in Tailwind/shadcn idioms), full-app UI kit (`ui_kits/opencompany/index.html`), 12-theme docs (`guidelines/THEMES.md`), **the merged handoff brief (`HANDOFF.md` — glow-ladder traps, step-edge contract, toolbar layout traps, canvas-layer asymmetries, plus the recorded product amendments)**, and **the panel×theme fidelity target (`reference-mockup/Panel Theme Matrix.dc.html` — 17 panels × 12 themes, open in a browser)**. `reference/themes/` no longer vendors CSS snapshots — it points at the authoritative [client/src/themes/](./client/src/themes/). Copy token values verbatim; do not re-derive by eye. Pairs with [Theme System](./docs-internal/theme_system.md). |
| **[Schema Source of Truth RFC](./docs-internal/ARCHIVE/schema_source_of_truth_rfc.md)** | *Archived — shipped.* Backend is SSOT for node schemas, visual metadata, handlers, palette metadata, icons; `VITE_NODESPEC_BACKEND` defaults on and the frontend node definitions are gone. Plugin pattern: one `BaseNode` subclass in `server/nodes/<group>/<plugin>/__init__.py`. Wire format: `asset:<key>` / `<lib>:<brand>` / URL / emoji. Endpoint: `/api/schemas/nodes/{type}/spec.json`. Current reference: [Plugin System](./docs-internal/plugin_system.md). |
@@ -62,7 +62,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire
| **[Onboarding Service](./docs-internal/onboarding.md)** | First-launch welcome wizard with 4 steps, database persistence, and replay from Settings |
| **[CLI Services Integration](./docs-internal/cli_services_integration.md)** | Guide for integrating CLI-based services (Temporal, etc.) with proper lifecycle management |
| **[Temporal Architecture](./docs-internal/TEMPORAL_ARCHITECTURE.md)** | Distributed workflow execution: activities, FIRST_COMPLETED scheduling, horizontal scaling |
-| **[Native LLM SDK](./docs-internal/native_llm_sdk.md)** | Native SDK layer in `services/llm/`: protocol-based providers, config-driven base URLs, and 14 registered providers (11 cloud, Ollama / LM Studio, and `openai_compatible`, which serves any number of user-named endpoints — llama.cpp, vLLM, a LiteLLM proxy — added in the Credentials Modal with no code; xAI is agent-selectable without a standalone chat-model node). A JSON-driven `supports_model_listing: false` flag covers endpoints that speak the OpenAI chat wire format but ship no `/v1/models` route (Sarvam) — `OpenAIProvider.fetch_models` then serves the curated list and probes the key with a one-token completion. Every provider is native; `tests/llm/test_langchain_removed.py` locks the retired-dependency end state. |
+| **[Native LLM SDK](./docs-internal/native_llm_sdk.md)** | Native SDK layer in `services/llm/`: protocol-based providers, config-driven base URLs, and 14 registered providers (11 cloud, Ollama / LM Studio, and `openai_compatible`, which serves any number of user-named endpoints — llama.cpp, vLLM, a LiteLLM proxy — added in the Credentials Modal with no code; xAI is agent-selectable without a standalone chat-model node). A JSON-driven `supports_model_listing: false` flag covers endpoints that speak the OpenAI chat wire format but ship no `/v1/models` route (Sarvam) — `OpenAIProvider.fetch_models` then serves the curated list and probes the key with a one-token completion. `chat(on_event=...)` streams `StreamEvent`s only for providers whose block declares `"streaming": true` (Anthropic, OpenAI); the unifier replays the finished response into the sink for the rest, and the returned response never changes (`tests/llm/test_streaming.py`). Every provider is native; `tests/llm/test_langchain_removed.py` locks the retired-dependency end state. |
| **[RFC-0003 — OpenAI-Compatible Provider Contract](./RFC-0003-OPENAI-COMPATIBLE-PROVIDER-CONTRACT.md)** | Accepted, implemented. "OpenAI-compatible" is a family of divergent shapes with no spec behind it, so per-provider facts are **resolved at save time or declared**, never guessed. A configured `base_url` is copied **verbatim** from vendor docs and never rewritten (`/v1` is neither universal nor positionally predictable — DeepSeek is root-mounted, Groq is `/openai/v1`); a **user-supplied** URL is rooted by `services/llm/endpoints.py::resolve_base_url`, which sends the SDK's own `models.list()` to the URL then to it plus `/v1` and adopts only a body that **is an OpenAI list** — a status alone proves nothing, because LM Studio answers HTTP 200 to routes it does not serve. A failed save writes and broadcasts nothing. One `resolve_credential` sends the stored key or the vendor-documented placeholder (`auth.placeholder_key`) and never `None` (which would make the SDK read `OPENAI_API_KEY` and ship it to a third-party URL); no provider rewrites a key. A 2xx with no `choices` and an error body raises `PROTOCOL` naming the redacted URL. **Named endpoints (D13–D17)**: any server is a provider reference `openai_compatible:<slug>` stored exactly like Ollama (`{ref}` + `{ref}_proxy`), selectable from every agent (the `provider` field is a loader-driven `ProviderRef`, not a `Literal`), from the `openaiCompatibleChatModel` node (its field is `endpoint`, not `provider`, to dodge the panel's stored-key effect) and the global default picker; kind detected once from native routes by body shape; context and price from the server first (Ollama/LM Studio SDK probes, llama.cpp `/props`, vLLM `max_model_len`), then LiteLLM's table for generic servers only, then conservative defaults; local-model entries and the LiteLLM cache live under DATA_DIR, never in the tracked `config/model_registry.json`. RLM cannot route an endpoint and refuses one loudly. Gates AG1–AG20 (AG5 retired), `ProviderSpec` gains no field. |
| **[Event Waiter System](./docs-internal/event_waiter_system.md)** | In-memory asyncio.Future waiter for push-based trigger nodes on the canvas-Run path (WhatsApp, Telegram, Webhook, Chat, Task completion). Deployed triggers ride the Temporal canary path; the Redis-Streams backend was retired in Wave 15.3 |
| **[Credentials Encryption](./docs-internal/credentials_encryption.md)** | Fernet + PBKDF2 encryption pipeline, separate credentials.db, two credential systems (OAuth vs API keys), multi-backend abstraction |
diff --git a/docs-internal/agent_context_flow.md b/docs-internal/agent_context_flow.md
index 0c0a1e31..b5752473 100644
--- a/docs-internal/agent_context_flow.md
+++ b/docs-internal/agent_context_flow.md
@@ -114,6 +114,25 @@ checks below, even when the carried transcript is what seeds it. If that
stored row is over the 1 MB seed cap, the resumed run fails with
`ConversationTooLarge`.
+### Stopped turns and unanswered calls
+
+The owner can stop a chat run mid-answer ([Chat Protocol → Streaming, steps
+and Stop](./chat_protocol.md#streaming-steps-and-stop)). The LLM step then
+saves the turn as far as it went, like any other step: the exact sent list,
+plus the partial answer when the model had written something (its text block
+carries `metadata: {"stopped": true}`); with nothing written, the sent list
+alone, so the owner's message stays in the conversation unanswered. A tool call
+of a stopped run gets a result saying it was not run, so the transcript stays
+whole.
+
+A run cancelled while a tool ran (the watchdog cancels a run still stopping
+after the grace; a crash) can leave the last assistant turn with calls that
+have no result, which no provider will continue from. The load therefore
+passes the stored row through `close_unanswered_tool_calls`
+(`services/agent_context/conversation.py`), which answers each open call of the
+latest assistant turn with `UNANSWERED_TOOL_RESULT`, after the results the turn
+already has. A closed conversation is returned unchanged.
+
## Transcript size: capped results, cleared old results
The transcript is the `agent.execute_llm_step` input on every turn and the
@@ -297,6 +316,7 @@ workflow, not only the removed Context's agent.
| 8 | Specialized bridges record the ORIGINAL prompt, never the augmented one. | Recording the rendered transcript nests the conversation inside itself and grows without bound. |
| 9 | The store never imports `nodes/`; the plugin registers its broadcaster via `register_conversation_listener`, and a listener failure can never fail a save. | Same layering rule as every plugin registry; a UI notification must not break execution. |
| 10 | External tool results are capped before they enter the transcript; the latest turn is never cleared or summarized; the pressure rules are chosen by the recorded `context_pressure_version`. | Uncapped results bricked every later firing (`errors.md` #28); an unread turn summarized away loses what the model asked for; a replay must schedule the commands it recorded. |
+| 11 | A stopped turn is saved as far as it went, and a load answers any tool call the stored row left open. | A provider refuses a conversation with an unanswered call, so one cancelled run would otherwise break every later firing. |
## How this broke (August 2026 regression), and why the journal went away
diff --git a/docs-internal/chat_protocol.md b/docs-internal/chat_protocol.md
index 0946cc75..98cdf224 100644
--- a/docs-internal/chat_protocol.md
+++ b/docs-internal/chat_protocol.md
@@ -15,8 +15,9 @@ meaning.
Built so far: runs and their storage, the lifecycle events (`started`, `finished`, `failed`), subscriptions,
`get_chat_messages` v2 (with each message's `run`), `get_chat_run`, the watchdog and the MachinaWorkflow patch (see
-[Runs](#runs)), and the shared chat UI on both hosts (see [Client](#client)). Every other event, handler and part
-below is the contract the later phases build to.
+[Runs](#runs)); the answer streaming as text events, working steps and Stop (see [Streaming, steps and
+Stop](#streaming-steps-and-stop)); and the shared chat UI on both hosts (see [Client](#client)). Every other event,
+handler and part below is the contract the later phases build to.
## Concepts
@@ -53,8 +54,9 @@ internal socket, load the workflow, and compare its owner with the socket's exec
with `seq` at or below the snapshot's, and buffer events that arrive before the response.
- Each subscriber has a bounded queue (`hub.subscriber_queue_size` in `server/config/chat_defaults.json`). A socket
that falls that far behind has its queue replaced by one `custom` `opencompany.resync` per session it follows
- (`data: {session_id, hub_epoch, name, value}`, no `run_id`); the client then takes fresh snapshots. Merging
- consecutive text deltas of one segment comes with streaming.
+ (`data: {session_id, hub_epoch, name, value}`, no `run_id`); the client then takes fresh snapshots. Text deltas
+ are batched before they are published (see [Streaming, steps and Stop](#streaming-steps-and-stop)), not merged in
+ the queue.
- Delivery is in-process: events published by an activity on a worker in another process do not reach sockets here
yet (a relay is planned).
- `chat.updated` remains an identity-only broadcast (`{workflow_id, session_id, role}`) that tells every open thread
@@ -70,11 +72,11 @@ Every event's `data` carries `{workflow_id, session_id, run_id, seq, hub_epoch}`
| RUN_STARTED | `started` | `kind`, `parent_run_id?`, `user_message_id?`, `reply_message_id`, `started_at` |
| RUN_FINISHED | `finished` | `outcome` (below), `result {reply_message_id?, no_reply?}`, `duration_ms`, `step_count` |
| RUN_ERROR | `failed` | `message` (safe to show, at most 500 chars), `code`, `hint?`, `requires_user_action?` |
-| STEP_STARTED | `step.started` | `step_id`, `step_name`, `icon?`, `agent_node_id`, `tool_node_id?`, `attempt` |
-| STEP_FINISHED | `step.finished` | `step_id`, `step_name` (the done label), `state` (`done`, `failed`, `skipped`), `detail?`, `duration_ms`, `narration?` |
-| TEXT_MESSAGE_START | `text.started` | `message_id` (segment id `{run_id}.{iteration}.{attempt}`), `role: "assistant"`, `agent_node_id` |
+| STEP_STARTED | `step.started` | `step_id` (the tool call's id), `step_name`, `icon?` |
+| STEP_FINISHED | `step.finished` | `step_id`, `step_name`, `state` (`done`, `failed`, `skipped`), `detail?` (at most 200 chars), `duration_ms`, `narration?` |
+| TEXT_MESSAGE_START | `text.started` | `message_id` (segment id `{run_id}.{iteration}.{attempt}`), `role: "assistant"` |
| TEXT_MESSAGE_CONTENT | `text.content` | `message_id`, `delta` (never empty) |
-| TEXT_MESSAGE_END | `text.ended` | `message_id`, `final` (bool), `reply_message_id?`, `output_tokens?` |
+| TEXT_MESSAGE_END | `text.ended` | `message_id`, `final` (bool), `reply_message_id?` (with `final: true`) |
| TOOL_CALL_START | `tool_call.started` | `tool_call_id`, `tool_call_name`, `parent_message_id?`, `label` (sending tools only) |
| TOOL_CALL_ARGS | `tool_call.args` | `tool_call_id`, `delta` |
| TOOL_CALL_END | `tool_call.ended` | `tool_call_id` |
@@ -89,9 +91,9 @@ Every event's `data` carries `{workflow_id, session_id, run_id, seq, hub_epoch}`
`custom` names: `opencompany.segment_discarded` (`{message_id}`: a retried LLM attempt replaces this segment),
`opencompany.retrying` (`{retry_after?, attempt}`), `opencompany.stopping`, `opencompany.resync`.
-**Text segments.** Only the agent that answers the owner streams text. A segment that ends with `final: false`
-accompanied a tool call: the client folds it into the steps as narration. The segment with `final: true` is the
-reply. Text that could still turn out to be `NO_REPLY` or a `<followups>` block is held back on the server and never
+**Text segments.** Only the agent that answers the owner streams text. A segment that ends with `final: false` was
+written beside tool calls (narration, "Let me check the calendar."); the segment with `final: true` is the reply.
+Text that could still turn out to be `NO_REPLY` or a `<followups>` block is held back on the server and never
streamed.
**Outcomes** (`run.finished`):
@@ -121,7 +123,7 @@ All are WebSocket request/response handlers with snake_case payloads. Failures a
| `chat_subscribe` | `{session_id}` | `{success, session_id, hub_epoch, active_runs: [RunSnapshot]}` |
| `chat_unsubscribe` | `{session_id}` | `{success, session_id, hub_epoch}` |
| `get_chat_run` | `{run_id}` | `{success, run: RunSnapshot}`, or `not_found` |
-| `stop_chat_run` | `{run_id}` or `{session_id}` | `{success, run_id, state}` |
+| `stop_chat_run` | `{run_id}` | `{success, run_id, state: "stopping" \| "stopped"}`; `not_stoppable` (with `state`) for a run that ended otherwise |
| `edit_chat_message` | `{message_id, text, attachments?, expected_revision}` | `{success, message_id, run_id}` |
| `regenerate_chat_reply` | `{message_id, expected_revision}` | `{success, run_id}` |
| `switch_chat_branch` | `{message_id, target_id, expected_revision}` | `{success, thread}` |
@@ -170,7 +172,8 @@ summary adds `kind` (`gate` or `tool_call`), `node_type`, `tool_name`, `tool_cal
```
{id, legacy_id, role, kind, text, message, timestamp, run_key, run_id, parent_id, status, attachments,
parts, feedback, siblings: {index, count, ids}, editable, client_message_id?,
- run?: {run_id, state, outcome, error?: {message, code, hint?}}}
+ run?: {run_id, state, outcome, error?: {message, code, hint?}, steps?: [{step_id, name, state, detail?,
+ duration_ms?}], duration_ms?}}
```
- `id` is the message's stable id (`m_…` for the owner's, `a_<run id>` for a run's reply; rows saved before chat
@@ -178,9 +181,10 @@ summary adds `kind` (`gate` or `tool_call`), `node_type`, `tool_name`, `tool_cal
- `message` equals `text`; it is kept for older readers.
- `client_message_id` is echoed on the owner's messages that carried one, so an optimistic row and its saved row
keep one key.
-- `run` says how the run a message started (or answers) stood when the thread was read: its `state`, `outcome`, and
- for a failed run the error with the hint the run recorded. It is how a reload still shows that a message went
- unanswered; while the session is subscribed, the run's events are fresher (see [Client](#client)).
+- `run` says how the run a message started (or answers) stood when the thread was read: its `state`, `outcome`, for
+ a failed run the error with the hint the run recorded, the steps it saved, and once it has ended how long it
+ worked (`duration_ms`). It is how a reload still shows that a message went unanswered and what the employee did;
+ while the session is subscribed, the run's events are fresher (see [Client](#client)).
- Until branches land, `siblings` always holds the message alone and `editable` is false.
- `kind`: `text`, `report` (Post to Talk: no parent, not editable), `action` (a button press), `notice`.
- `status`: `complete`, `stopped`, `error`.
@@ -259,6 +263,43 @@ Approvals are joined live from the approvals store; the part only names them.
ends the runs nothing will finish (codes above). A pending run's wait (`runs.pickup_timeout_s`) counts from the
later of its creation and the server's start.
+## Streaming, steps and Stop
+
+Settings are in `server/config/chat_defaults.json` (`stream`, `steps`, `runs`).
+
+- **Who streams.** `agent.prepare_payload` gives an agent a `chat_stream` (`services/chat/stream.py`
+ `chat_stream_for`) when the run's `run_scope` reached it, it is not working for another agent (no
+ `parent_node_id`), and its output goes straight to a node whose plugin declares `answers_chat_run` (Reply in Chat).
+ The payload also carries `chat_run_id` for every agent of the run. Both ride the AgentWorkflow's activity inputs
+ only (LLM steps and tool calls), so no workflow patch was needed and recorded histories replay unchanged.
+- **Text.** `agent.execute_llm_step` passes a `ChatStreamEmitter` as the provider's `on_event` sink (providers that
+ declare `streaming` in `llm_defaults.json` stream; for the others the unifier replays the finished response as
+ events: see [Native LLM SDK](./native_llm_sdk.md)). Deltas go out every `stream.flush_ms` or `stream.flush_chars`.
+ A retried attempt first sends `opencompany.segment_discarded` for each earlier attempt's segment. Streaming never
+ changes the step's result.
+- **Steps.** `BaseNode.as_activity` wraps every tool call that carries a `chat_stream` and a `tool_call_id`
+ (`services/chat/steps.py`): `step.started` when it begins, `step.finished` when it ends, saved on the run as it
+ finishes (`ledger.record_step`, at most `steps.max_per_run`). The label is the plugin's `chat_step`
+ ("Searched the web"), else "Used <display name>"; plugins with `chat_step_hidden` (the clock, the checklist) show
+ none, and skill loads (`agent.skill.invoke`) never pass through it. A tool may put a short line in its result as
+ `_step_detail` ("3 events on Saturday"): it is shown under the step and taken out before the model reads the
+ result.
+- **Stop** (`stop_chat_run`, `ledger.request_stop`). A run nothing has picked up (pending, or queued until Resume)
+ ends `stopped` at once and frees the lane; a workflow that picks it up later claims it still `stopped`
+ (`chat_run.start`) and its agent answers nothing. A running run moves to `stopping` (`custom`
+ `opencompany.stopping`) and stops itself:
+ - its agent's model step polls the run every `runs.stop_poll_s` while the provider writes (an activity heartbeat
+ arrives too late to stop a stream mid-sentence), drops the call, and returns what the owner saw so far as the
+ final answer; a step about to start returns at once, before paying for a request;
+ - a tool call not started yet is answered "Not run: the owner stopped this answer." without running;
+ - Reply in Chat saves the partial answer with `status: "stopped"`, and the run finishes with outcome `stopped`.
+
+ A run still stopping `runs.stop_grace_s` after Stop (a long tool, a lost workflow) is ended by the watchdog, which
+ cancels its Temporal workflow. The conversation keeps the stopped turn: the request and the partial answer (its
+ text block marked `stopped`), or the request alone when nothing was written; a tool call a cancelled run left
+ without a result is answered when the conversation is next loaded (see
+ [Agent Context Flow](./agent_context_flow.md)).
+
## Client
`client/src/features/chat/` is the shared chat; only its `index.ts` is public (an ESLint rule keeps the rest
@@ -280,16 +321,24 @@ editor's console pane (`ConsoleChat`, compact, scope `live`) are the two hosts.
`active_runs` stand in; after it the store alone says which runs are live, and a message whose `run` still reads
live is read with `get_chat_run` (`useThreadRunReconcile`).
- **Turns** (`thread/model.ts`): one per message with a divider where `run_key` changes. A run that is going,
- failed or was stopped shows on its last answer, or on a turn of its own right after its last message before it
- has answered (a live run whose messages are out of view goes last). Failures whose code only says no answer came
+ failed or was stopped (or finished, while its answer is on the way) shows on its last answer, or on a turn of its
+ own right after its last message before it has answered (a live run whose messages are out of view goes last). Failures whose code only says no answer came
(`not_delivered`, `timed_out`, `interrupted`) disappear once an answer lands; `reset` and `cleared` never show. A
message sent from this tab and its saved row share a key (its `client_message_id`), and a run's first answer takes
the key its run's turn had, so neither remounts.
- **Sending** (`data/send.ts`): the message shows at once; the server's answer admits its run into the store
(`queued` or `pending`), so the employee shows working before the run's first event. While the lane is held, Send
- waits. A refused or failed send takes the message out of the thread and puts its text back in the box
+ is Stop. A refused or failed send takes the message out of the thread and puts its text back in the box
(`state/composerStore.ts`, a draft per session that survives switching conversations); after a failure in transit
the draft keeps its `client_message_id`, so sending it again is the same message.
+- **A working run** (`turns/AssistantTurn.tsx`): skeleton lines until text comes; then the latest segment, muted
+ while it is narration, with a caret while it streams (`ReplyMarkdown` renders each finished block once,
+ `markdown/blocks.ts`); a status line saying "Thinking", "Writing · N tok/s" or "Stopping…", with an Esc hint. The
+ steps disclosure (`turns/StepsDisclosure.tsx`) sits on the run's first turn: "Working…" and open while the run
+ works, "Worked for 12s · 3 steps" after; a run read back later starts it closed. A finished run keeps its turn
+ until its saved reply lands in the thread, so the streamed answer and the reply stay one element.
+- **Stop** (`data/stop.ts`): the Stop button, or Esc anywhere in the pane, sends `stop_chat_run` for the lane's run
+ and applies the answer to the store at once; the run's events take it from there.
## Notes to the employee
@@ -319,7 +368,8 @@ and server tests alike:
| `not_running` | send | The employee is not running and cannot queue messages. |
| `invalid_request` | send, save | A malformed field (`detail` says which): an empty message, a role other than the owner's, or a bad `client_message_id`. |
| `read_failed` | get_chat_messages, chat_subscribe | The thread could not be read. Never answered as an empty thread. |
-| `not_found` | get_chat_run | No such run. |
+| `not_found` | get_chat_run, stop_chat_run | No such run. |
+| `not_stoppable` | stop_chat_run | The run ended before Stop reached it; `state` says how. |
| `conflict` | edit, regenerate, switch, set_ask_first | `expected_revision` is stale. |
| `not_editable` | edit, regenerate, decide | The message belongs to an older generation, its prefix was compacted, or an edited argument is not editable. |
| `ui_event_rejected` | send | The element or action does not match the saved spec. |
diff --git a/docs-internal/plugin_system.md b/docs-internal/plugin_system.md
index 0c7194b4..a5e79826 100644
--- a/docs-internal/plugin_system.md
+++ b/docs-internal/plugin_system.md
@@ -79,6 +79,8 @@ class SpecializedAgentBase(ActionNode, abstract=True):
| `Output` | Pydantic `BaseModel` — runtime output shape. |
| `usable_as_tool` | `ActionNode` flag — mints a ToolNode adapter for AI invocation. Combined with `component_kind != "model"`, makes the plugin visible to `agentBuilder.add_tool` (catalogue + rebind paths). **Side effect:** setting it auto-sets `hide_input_handle` / `hide_output_handle` to `True` unless the class declares them (`BaseNode.__init_subclass__` in [base.py](../server/services/plugin/base.py), which does the same for `component_kind == "tool"`), and `_metadata_dict` adds an `output-tool` handle. The frontend (`SquareNode`) reads the hide flags only when a spec declares no handles; a spec with handles renders exactly what it declares. So a dual-purpose node that must stay wirable on the canvas declares `input-main` / `output-main` in `handles`; setting the flags `False` alone does not add them. |
| `needs_canvas` | `ClassVar[bool]` — when `True`, the F4.B `AgentWorkflow` tool-dispatch forwards the parent workflow's `nodes`/`edges` into the per-tool activity payload. Today only `AgentBuilderNode` opts in: `add_tool` compares the run's canvas with the saved graph, so a tool saved earlier but missing from the running snapshot is handed back to be bound. Its calling agent comes from `invoking_agent_node_id` / `parent_node_id`, not from edges. |
+| `chat_step` / `chat_step_hidden` | How a call of this tool reads as a working step in the chat while it answers the owner ("Searched the web"; default "Used <display name>"); `chat_step_hidden = True` shows none (the clock, the checklist). A tool may put a short line in its result as `_step_detail`, shown under the step and removed before the model reads the result. See [Chat Protocol](./chat_protocol.md#streaming-steps-and-stop). |
+| `answers_chat_run` | `ClassVar[bool]` — the node posts a chat run's answer (Reply in Chat). The agent wired straight into such a node streams its text into the chat. |
| `task_queue` | Temporal worker pool. See `TaskQueue` constants. |
| `retry_policy` | `RetryPolicy` dataclass (mirrors `temporalio.common.RetryPolicy`). |
| `start_to_close_timeout` / `heartbeat_timeout` | Per-node Temporal knobs. |
diff --git a/server/config/chat_defaults.json b/server/config/chat_defaults.json
index dc364f71..7ca66657 100644
--- a/server/config/chat_defaults.json
+++ b/server/config/chat_defaults.json
@@ -2,9 +2,18 @@
"runs": {
"pickup_timeout_s": 120,
"max_running_s": 86400,
- "watchdog_interval_s": 30
+ "watchdog_interval_s": 30,
+ "stop_poll_s": 1,
+ "stop_grace_s": 45
},
"hub": {
"subscriber_queue_size": 512
+ },
+ "stream": {
+ "flush_ms": 50,
+ "flush_chars": 160
+ },
+ "steps": {
+ "max_per_run": 100
}
}
diff --git a/server/core/database.py b/server/core/database.py
index 95968491..402161cf 100644
--- a/server/core/database.py
+++ b/server/core/database.py
@@ -138,6 +138,7 @@ class Database:
await self._migrate_workflow_controls()
await self._migrate_generation_scoped_runtime_data()
await self._migrate_chat_messages()
+ await self._migrate_chat_runs()
logger.info("Database initialized successfully")
@@ -524,6 +525,21 @@ class Database:
except Exception as exc:
logger.warning(f"Generation runtime-data migration check failed: {exc}")
+ async def _migrate_chat_runs(self):
+ """Give an older ``chat_runs`` table the columns added since."""
+ try:
+ async with self.engine.begin() as conn:
+ result = await conn.execute(text("PRAGMA table_info(chat_runs)"))
+ columns = {row[1] for row in result.fetchall()}
+ if not columns:
+ return
+ additions = {"stop_requested_at": "DATETIME"}
+ for column, definition in additions.items():
+ if column not in columns:
+ await conn.execute(text(f"ALTER TABLE chat_runs ADD COLUMN {column} {definition}"))
+ except Exception as exc:
+ logger.warning(f"Chat run migration check failed: {exc}")
+
async def _migrate_chat_messages(self):
"""Give older chat rows the columns chat runs need.
diff --git a/server/models/chat.py b/server/models/chat.py
index efb6b9ef..55538e5d 100644
--- a/server/models/chat.py
+++ b/server/models/chat.py
@@ -3,7 +3,8 @@
A run is admitted in the same transaction as the owner's message
(``services/chat/ledger.py``). It moves ``queued`` (the employee is paused)
or ``pending`` -> ``running`` -> ``finished`` | ``error`` | ``stopped``, each
-step a compare-and-swap on ``state``. ``kind`` says what started it:
+step a compare-and-swap on ``state``; Stop moves a pending or running run to
+``stopping`` on the way to ``stopped``. ``kind`` says what started it:
``message`` (the owner wrote), ``edit``, ``regenerate``, ``action`` (a button
in generated UI) or ``resume`` (an approved send executing later).
@@ -79,6 +80,9 @@ class ChatRun(SQLModel, table=True):
created_at: datetime = Field(default_factory=_utcnow, sa_column=Column(DateTime(timezone=True), nullable=False))
started_at: Optional[datetime] = Field(default=None, sa_column=Column(DateTime(timezone=True), nullable=True))
finished_at: Optional[datetime] = Field(default=None, sa_column=Column(DateTime(timezone=True), nullable=True))
+ #: When the owner pressed Stop; the watchdog ends a run still stopping
+ #: ``runs.stop_grace_s`` later.
+ stop_requested_at: Optional[datetime] = Field(default=None, sa_column=Column(DateTime(timezone=True), nullable=True))
class ChatRunPart(SQLModel, table=True):
diff --git a/server/nodes/chat/chat_reply/__init__.py b/server/nodes/chat/chat_reply/__init__.py
index 4e2063ab..d5f0b9d4 100644
--- a/server/nodes/chat/chat_reply/__init__.py
+++ b/server/nodes/chat/chat_reply/__init__.py
@@ -77,6 +77,9 @@ class ChatReplyNode(ActionNode):
hide_output_handle = True
annotations = {"destructive": False, "readonly": False, "open_world": False}
task_queue = TaskQueue.DEFAULT
+ # The agent wired into this node answers the chat run: its text streams
+ # into the chat as it writes (services/chat/stream.py).
+ answers_chat_run = True
Params = ChatReplyParams
Output = ChatReplyOutput
diff --git a/server/nodes/search/brave_search/__init__.py b/server/nodes/search/brave_search/__init__.py
index 06ceee8a..2732ca1e 100644
--- a/server/nodes/search/brave_search/__init__.py
+++ b/server/nodes/search/brave_search/__init__.py
@@ -89,6 +89,7 @@ class BraveSearchNode(ActionNode):
credentials = (BraveSearchCredential,)
annotations = {"destructive": False, "readonly": True, "open_world": True}
task_queue = TaskQueue.REST_API
+ chat_step = "Searched the web"
usable_as_tool = True
Params = BraveSearchParams
diff --git a/server/nodes/search/duckduckgo_search/__init__.py b/server/nodes/search/duckduckgo_search/__init__.py
index dcc7e70e..229d4a28 100644
--- a/server/nodes/search/duckduckgo_search/__init__.py
+++ b/server/nodes/search/duckduckgo_search/__init__.py
@@ -52,6 +52,7 @@ class DuckDuckGoSearchNode(ToolNode):
ui_hints = {"hideRunButton": True}
annotations = {"destructive": False, "readonly": True, "open_world": True}
task_queue = TaskQueue.REST_API
+ chat_step = "Searched the web"
Params = DuckDuckGoSearchParams
Output = DuckDuckGoSearchOutput
diff --git a/server/nodes/search/perplexity_search/__init__.py b/server/nodes/search/perplexity_search/__init__.py
index 22538cd2..299d05eb 100644
--- a/server/nodes/search/perplexity_search/__init__.py
+++ b/server/nodes/search/perplexity_search/__init__.py
@@ -93,6 +93,7 @@ class PerplexitySearchNode(ActionNode):
credentials = (PerplexityCredential,)
annotations = {"destructive": False, "readonly": True, "open_world": True}
task_queue = TaskQueue.REST_API
+ chat_step = "Searched the web"
usable_as_tool = True
Params = PerplexitySearchParams
diff --git a/server/nodes/search/serper_search/__init__.py b/server/nodes/search/serper_search/__init__.py
index 3fea2f1f..6feae6a3 100644
--- a/server/nodes/search/serper_search/__init__.py
+++ b/server/nodes/search/serper_search/__init__.py
@@ -88,6 +88,7 @@ class SerperSearchNode(ActionNode):
credentials = (SerperCredential,)
annotations = {"destructive": False, "readonly": True, "open_world": True}
task_queue = TaskQueue.REST_API
+ chat_step = "Searched the web"
usable_as_tool = True
Params = SerperSearchParams
diff --git a/server/nodes/tool/current_time_tool/__init__.py b/server/nodes/tool/current_time_tool/__init__.py
index 9c70d2f1..f42549da 100644
--- a/server/nodes/tool/current_time_tool/__init__.py
+++ b/server/nodes/tool/current_time_tool/__init__.py
@@ -39,6 +39,7 @@ class CurrentTimeToolNode(ToolNode):
ui_hints = {"hideRunButton": True}
annotations = {"destructive": False, "readonly": True, "open_world": False}
task_queue = TaskQueue.DEFAULT
+ chat_step_hidden = True
Params = CurrentTimeParams
Output = CurrentTimeOutput
diff --git a/server/nodes/tool/write_todos/__init__.py b/server/nodes/tool/write_todos/__init__.py
index 4badeb56..cb8ce10c 100644
--- a/server/nodes/tool/write_todos/__init__.py
+++ b/server/nodes/tool/write_todos/__init__.py
@@ -76,6 +76,7 @@ class WriteTodosNode(ToolNode):
ui_hints = {"hideRunButton": True, "isTodoEditor": True}
annotations = {"destructive": False, "readonly": False, "open_world": False}
task_queue = TaskQueue.DEFAULT
+ chat_step_hidden = True
Params = WriteTodosParams
Output = WriteTodosOutput
diff --git a/server/services/agent_context/conversation.py b/server/services/agent_context/conversation.py
index 64420eaa..66f61e55 100644
--- a/server/services/agent_context/conversation.py
+++ b/server/services/agent_context/conversation.py
@@ -243,8 +243,47 @@ async def list_conversations(
]
+#: The result a tool call gets when its run ended before the tool answered.
+UNANSWERED_TOOL_RESULT = "This step did not finish: the run ended before its result came back."
+
+
+def close_unanswered_tool_calls(messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+ """Answer the tool calls a stored conversation left open.
+
+ A run that ends between asking for a tool and getting its result (the
+ watchdog cancelling a stopped run mid-tool, a crash) saves a last
+ assistant turn whose calls have no result, and every provider refuses to
+ continue such a conversation. Each open call gets
+ :data:`UNANSWERED_TOOL_RESULT`, after the results the turn already has.
+ Returns ``messages`` itself when nothing is open.
+ """
+ from services.llm.protocol import Message, message_to_wire
+
+ latest = next((i for i in range(len(messages) - 1, -1, -1) if messages[i].get("role") == "assistant"), None)
+ if latest is None:
+ return messages
+ answered = {m.get("tool_call_id") for m in messages[latest + 1:] if m.get("role") == "tool"}
+ open_calls = [
+ call
+ for call in messages[latest].get("tool_calls") or []
+ if isinstance(call, dict) and call.get("id") and call.get("id") not in answered
+ ]
+ if not open_calls:
+ return messages
+ end = latest + 1
+ while end < len(messages) and messages[end].get("role") == "tool":
+ end += 1
+ closing = [
+ dict(message_to_wire(Message(role="tool", content=UNANSWERED_TOOL_RESULT, tool_call_id=call["id"], name=call.get("name"))))
+ for call in open_calls
+ ]
+ return [*messages[:end], *closing, *messages[end:]]
+
+
__all__ = [
+ "UNANSWERED_TOOL_RESULT",
"clear_conversation",
+ "close_unanswered_tool_calls",
"list_conversations",
"load_conversation",
"save_conversation",
diff --git a/server/services/chat/config.py b/server/services/chat/config.py
index 2a4c45b4..ebfcdc11 100644
--- a/server/services/chat/config.py
+++ b/server/services/chat/config.py
@@ -24,4 +24,13 @@ def hub_setting(name: str) -> int:
return int(load_chat_config()["hub"][name])
-__all__ = ["CONFIG_PATH", "hub_setting", "load_chat_config", "runs_setting"]
+def stream_setting(name: str) -> float:
+ """``stream.<name>``: how the answer's text is batched into events."""
+ return float(load_chat_config()["stream"][name])
+
+
+def steps_setting(name: str) -> int:
+ return int(load_chat_config()["steps"][name])
+
+
+__all__ = ["CONFIG_PATH", "hub_setting", "load_chat_config", "runs_setting", "steps_setting", "stream_setting"]
diff --git a/server/services/chat/handlers.py b/server/services/chat/handlers.py
index e7f279a8..adf17d31 100644
--- a/server/services/chat/handlers.py
+++ b/server/services/chat/handlers.py
@@ -45,13 +45,20 @@ _DENIED = {"success": False, "error": "access_denied"}
def _wire_run(run: Any) -> Dict[str, Any]:
"""The run a message started or answers, as a message carries it: enough
- to show how it ended after a reload (a failed run leaves no reply)."""
+ to show how it ended after a reload (a failed run leaves no reply), what
+ the employee did on the way and how long it took."""
wire: Dict[str, Any] = {"run_id": run.run_id, "state": run.state, "outcome": run.outcome}
if run.state == "error":
wire["error"] = {"message": run.error, "code": run.error_code}
hint = (run.result or {}).get("hint")
if hint:
wire["error"]["hint"] = hint
+ if run.steps:
+ wire["steps"] = [dict(step) for step in run.steps]
+ if run.started_at and run.finished_at:
+ started = run.started_at if run.started_at.tzinfo else run.started_at.replace(tzinfo=timezone.utc)
+ finished = run.finished_at if run.finished_at.tzinfo else run.finished_at.replace(tzinfo=timezone.utc)
+ wire["duration_ms"] = max(0, int((finished - started).total_seconds() * 1000))
return wire
@@ -302,6 +309,26 @@ async def handle_get_chat_run(data: Dict[str, Any], websocket: WebSocket) -> Dic
return {"success": True, "run": {**snapshot, "hub_epoch": hub.epoch}}
+@ws_handler("run_id")
+async def handle_stop_chat_run(data: Dict[str, Any], websocket: WebSocket) -> Dict[str, Any]:
+ """Stop a live run (``ledger.request_stop``). One nothing picked up yet
+ ends ``stopped`` at once; a running one stops itself at its next step,
+ keeping its reply so far. Answers the run's state: ``stopping`` or
+ ``stopped``, ``not_stoppable`` for a run that ended otherwise."""
+ database = container.database()
+ run = await ledger.get_run(database, str(data["run_id"]))
+ if run is None:
+ return {"success": False, "error": "not_found"}
+ try:
+ await authorize_session(database, websocket, run.session_id)
+ except ChatAccessDenied:
+ return dict(_DENIED)
+ run = await ledger.request_stop(database, run.run_id) or run
+ if run.state not in ("stopping", "stopped"):
+ return {"success": False, "error": "not_stoppable", "run_id": run.run_id, "state": run.state}
+ return {"success": True, "run_id": run.run_id, "state": run.state}
+
+
@ws_handler()
async def handle_clear_chat_messages(data: Dict[str, Any], websocket: WebSocket) -> Dict[str, Any]:
"""Clear a session's chat, every generation of it, with its runs. For a
@@ -339,6 +366,7 @@ WS_HANDLERS = {
"chat_subscribe": handle_chat_subscribe,
"chat_unsubscribe": handle_chat_unsubscribe,
"get_chat_run": handle_get_chat_run,
+ "stop_chat_run": handle_stop_chat_run,
"clear_chat_messages": handle_clear_chat_messages,
"save_chat_message": handle_save_chat_message,
}
diff --git a/server/services/chat/ledger.py b/server/services/chat/ledger.py
index 86599aa6..5cecdf89 100644
--- a/server/services/chat/ledger.py
+++ b/server/services/chat/ledger.py
@@ -10,9 +10,11 @@ writing a second one; the id maps to a message id that only this session can
produce (``client_message_uid``).
**Start** (``start_run``) moves ``pending``/``queued`` to ``running`` and
-records the Temporal workflow that claimed it. Several chatTrigger nodes in
-one graph each spawn a run for the same message: the first claims the chat
-run, the others run untracked. **Finish** (``finish_run``) is accepted only
+records the Temporal workflow that claimed it; a run stopped before anything
+picked it up is claimed still ``stopped``, so the workflow that answers it
+stops at its first step instead of answering untracked. Several chatTrigger
+nodes in one graph each spawn a run for the same message: the first claims
+the chat run, the others run untracked. **Finish** (``finish_run``) is accepted only
from the claimant. A retried start or finish publishes nothing new: events
carry an ``event_key`` the hub publishes once.
@@ -24,6 +26,18 @@ running run past ``runs.max_running_s`` (``timed_out``), and a running run
whose Temporal workflow closed without finishing it: finished when its reply
was saved, ``interrupted`` otherwise.
+**Stop** (``request_stop``) ends a run nothing has picked up at once
+(``stopped``, no reply) and moves a running run to ``stopping``. A running
+run stops itself: its agent's next model step returns what it has written so
+far, and a tool not started yet is skipped (``services/chat/stream.py``,
+``services/chat/steps.py``); its reply, if any, is saved ``stopped`` and the
+run finishes with outcome ``stopped``. A run still stopping
+``runs.stop_grace_s`` later (a tool that takes long, a lost workflow) is
+ended by the watchdog, which also cancels its workflow.
+
+**Steps** (``record_step``) are saved on the run as they finish, so a reload
+can say how long it worked and what it did.
+
Every transition commits before its event is published, so a client that
reads the run after an event never sees an older state than the event.
"""
@@ -43,7 +57,7 @@ from sqlmodel import select
from core.logging import get_logger
from models.chat import LIVE_STATES, TERMINAL_STATES, ChatRun
from models.database import ChatMessage
-from services.chat.config import runs_setting
+from services.chat.config import runs_setting, steps_setting
from services.chat.hub import get_chat_hub
from services.chat_thread import delivery_for
@@ -59,6 +73,9 @@ _CLOSED_GRACE = timedelta(seconds=60)
#: ``await status(temporal_workflow_id, temporal_run_id)`` -> True when the
#: workflow has closed, False while it runs, None when unknown.
TemporalStatus = Callable[[str, Optional[str]], Awaitable[Optional[bool]]]
+#: ``await cancel(temporal_workflow_id, temporal_run_id)``: ask Temporal to
+#: cancel a workflow (best effort).
+TemporalCancel = Callable[[str, Optional[str]], Awaitable[None]]
class RunInProgress(Exception):
@@ -269,6 +286,8 @@ async def post_reply(
uid = run.reply_message_uid or reply_uid(run.run_id)
if replies:
uid = f"{uid}.{len(replies) + 1}"
+ current = await session.get(ChatRun, run.run_id)
+ stopped = current is not None and current.state in ("stopping", "stopped")
row = await database.append_chat_row(
session,
session_id=run.session_id,
@@ -277,6 +296,7 @@ async def post_reply(
execution_id=execution_id,
uid=uid,
run_id=run.run_id,
+ status="stopped" if stopped else "complete",
meta={"node_id": node_id},
)
await session.commit()
@@ -362,16 +382,25 @@ async def start_run(
temporal_run_id=temporal_run_id,
)
)
+ # Stopped before anything picked it up: claimed as it is, so the
+ # workflow's agent stops at its first step rather than answering
+ # untracked.
+ await session.execute(
+ update(ChatRun)
+ .where(ChatRun.run_id == run_id, ChatRun.state == "stopped", ChatRun.temporal_workflow_id.is_(None))
+ .values(temporal_workflow_id=temporal_workflow_id, temporal_run_id=temporal_run_id)
+ )
await session.commit()
run = await get_run(database, run_id)
if (
run is None
- or run.state not in ("running", "stopping")
+ or run.state not in ("running", "stopping", "stopped")
or run.temporal_workflow_id != temporal_workflow_id
or run.temporal_run_id != temporal_run_id
):
return None
- publish_started(run)
+ if run.state != "stopped":
+ publish_started(run)
return run
@@ -437,6 +466,74 @@ async def finish_run(
return await _settle(database, run, values)
+async def request_stop(database: Any, run_id: str) -> Optional[ChatRun]:
+ """Stop a live run. Returns the run as it now is (unchanged when it was
+ not stoppable), None when it does not exist.
+
+ A run nothing has picked up yet (pending, or queued until the employee
+ resumes) ends ``stopped`` at once, which frees the lane; a workflow that
+ picks it up later claims it stopped (``start_run``) and answers nothing.
+ A running run moves to ``stopping`` and stops itself."""
+ now = _utcnow()
+ async with database.get_session() as session:
+ unclaimed = await session.execute(
+ update(ChatRun)
+ .where(
+ ChatRun.run_id == run_id,
+ ChatRun.state.in_(("pending", "queued")),
+ ChatRun.temporal_workflow_id.is_(None),
+ )
+ .values(
+ state="stopped",
+ outcome="stopped",
+ result={"no_reply": True},
+ stop_requested_at=now,
+ finished_at=now,
+ )
+ )
+ ended = bool(unclaimed.rowcount)
+ stopping = False
+ if not ended:
+ result = await session.execute(
+ update(ChatRun)
+ .where(ChatRun.run_id == run_id, ChatRun.state.in_(("pending", "queued", "running")))
+ .values(state="stopping", stop_requested_at=now)
+ )
+ stopping = bool(result.rowcount)
+ await session.commit()
+ run = await get_run(database, run_id)
+ if run is not None and ended:
+ publish_terminal(run)
+ elif run is not None and stopping:
+ _publish(run, "custom", {"name": "opencompany.stopping", "value": {}}, "stopping")
+ return run
+
+
+async def is_stopping(database: Any, run_id: str) -> bool:
+ """Whether the owner asked the run to stop (or it already has)."""
+ run = await get_run(database, run_id)
+ return run is not None and run.state in ("stopping", "stopped")
+
+
+async def record_step(database: Any, run_id: str, step: Dict[str, Any]) -> None:
+ """Save a finished step on the run (replacing an earlier save of the
+ same step), up to ``steps.max_per_run``."""
+ step_id = step.get("step_id")
+ if not step_id:
+ return
+ async with database.reserved_session() as session:
+ run = await session.get(ChatRun, run_id)
+ if run is None:
+ return
+ steps = [dict(item) for item in (run.steps or []) if item.get("step_id") != step_id]
+ if len(steps) >= steps_setting("max_per_run"):
+ return
+ steps.append(dict(step))
+ run.steps = steps
+ session.add(run)
+ await session.commit()
+
+
async def fail_run(database: Any, run: ChatRun, *, code: str, message: str) -> Optional[ChatRun]:
"""End a live run with an error. None when it moved meanwhile."""
if run.state not in LIVE_STATES:
@@ -461,6 +558,7 @@ async def sweep(
now: Optional[datetime] = None,
process_started: datetime = PROCESS_STARTED,
temporal_status: Optional[TemporalStatus] = None,
+ temporal_cancel: Optional[TemporalCancel] = None,
) -> List[str]:
"""End the live runs nothing will end. Returns their ids."""
now = now or _utcnow()
@@ -469,7 +567,10 @@ async def sweep(
ended: List[str] = []
for run in await live_runs(database):
try:
- closed = await _sweep_one(database, run, now, process_started, pickup, longest, temporal_status)
+ if run.state == "stopping":
+ closed = await _sweep_stopping(database, run, now, temporal_cancel)
+ else:
+ closed = await _sweep_one(database, run, now, process_started, pickup, longest, temporal_status)
except Exception: # noqa: BLE001 - one bad row must not stop the sweep
logger.warning("Chat run sweep failed for a run", run_id=run.run_id, exc_info=True)
continue
@@ -480,6 +581,22 @@ async def sweep(
return ended
+async def _sweep_stopping(
+ database: Any, run: ChatRun, now: datetime, temporal_cancel: Optional[TemporalCancel]
+) -> Optional[ChatRun]:
+ """End a run still stopping ``runs.stop_grace_s`` after Stop, cancelling
+ its workflow: its reply so far, if any, stays."""
+ requested = _aware(run.stop_requested_at) or _aware(run.started_at) or _aware(run.created_at) or now
+ if now - requested < timedelta(seconds=runs_setting("stop_grace_s")):
+ return None
+ if temporal_cancel is not None and run.temporal_workflow_id:
+ try:
+ await temporal_cancel(run.temporal_workflow_id, run.temporal_run_id)
+ except Exception: # noqa: BLE001 - the run ends either way
+ logger.warning("Could not cancel a stopped run's workflow", run_id=run.run_id, exc_info=True)
+ return await _settle(database, run, await _success_values(database, run, "stopped"))
+
+
async def _sweep_one(
database: Any,
run: ChatRun,
@@ -529,6 +646,7 @@ __all__ = [
"fail_run",
"finish_run",
"get_run",
+ "is_stopping",
"lane_run",
"live_runs",
"new_run_id",
@@ -536,7 +654,9 @@ __all__ = [
"publish_cleared",
"publish_started",
"publish_terminal",
+ "record_step",
"reply_uid",
+ "request_stop",
"runs_by_id",
"start_run",
"sweep",
diff --git a/server/services/chat/steps.py b/server/services/chat/steps.py
new file mode 100644
index 00000000..4b514847
--- /dev/null
+++ b/server/services/chat/steps.py
@@ -0,0 +1,159 @@
+"""Working steps: the tool calls of the agent answering a chat run, shown in
+the chat while it works ("Checked Google Calendar", docs-internal/
+chat_protocol.md).
+
+``BaseNode.as_activity`` calls these around a tool call. A call is a step
+only when it comes from the agent that answers the run: its payload carries
+``chat_stream`` (``agent.prepare_payload`` -> AgentWorkflow), and a tool
+call carries a ``tool_call_id``, which is the step id.
+
+- **The label** is the plugin's ``chat_step``, else "Used <display name>";
+ ``chat_step_hidden`` plugins (the clock, a checklist) show nothing.
+- **Detail.** A tool may put a short line in its result as ``_step_detail``
+ ("3 events on Saturday"); it is shown under the step and taken out of the
+ result before the model reads it.
+- **Saved** on the run as it finishes (``ledger.record_step``), so a reload
+ can say what the employee did.
+- **Stopped runs.** A tool call of a run the owner stopped is not run: it
+ gets a result saying so, which keeps the conversation whole (every call
+ answered) while the agent's next model step ends the run.
+
+Nothing here fails a tool call: publishing and saving are best effort.
+"""
+
+from __future__ import annotations
+
+import time
+from dataclasses import dataclass
+from typing import Any, Dict, Mapping, Optional
+
+from core.logging import get_logger
+
+logger = get_logger(__name__)
+
+#: The key a tool result may carry its step detail under.
+STEP_DETAIL_KEY = "_step_detail"
+#: What a tool call of a stopped run returns instead of running.
+STOPPED_RESULT = {"stopped": True, "message": "Not run: the owner stopped this answer."}
+
+
+def step_label(node_class: Any) -> Optional[str]:
+ """How a call of this tool reads as a step; None when it shows none."""
+ if getattr(node_class, "chat_step_hidden", False):
+ return None
+ label = getattr(node_class, "chat_step", "") or ""
+ if label:
+ return label
+ name = getattr(node_class, "display_name", "") or getattr(node_class, "type", "") or "a tool"
+ return f"Used {name}"
+
+
+@dataclass
+class Step:
+ run_id: str
+ session_id: str
+ workflow_id: Optional[str]
+ step_id: str
+ name: str
+ started: float
+
+
+def _stream_of(context: Mapping[str, Any]) -> Optional[Mapping[str, Any]]:
+ stream = context.get("chat_stream")
+ if not isinstance(stream, Mapping):
+ return None
+ if not isinstance(stream.get("run_id"), str) or not isinstance(stream.get("session_id"), str):
+ return None
+ return stream
+
+
+def _publish(step: Step, suffix: str, fields: Dict[str, Any], event_key: str) -> None:
+ try:
+ from services.chat.hub import get_chat_hub
+
+ get_chat_hub().publish(
+ run_id=step.run_id,
+ session_id=step.session_id,
+ workflow_id=step.workflow_id,
+ suffix=suffix,
+ fields=fields,
+ event_key=event_key,
+ )
+ except Exception: # noqa: BLE001 - a step is a courtesy
+ logger.warning("Chat step event could not be published", run_id=step.run_id, exc_info=True)
+
+
+def begin(context: Mapping[str, Any], node_class: Any) -> Optional[Step]:
+ """Announce a tool call as a step; None when it is not one."""
+ stream = _stream_of(context)
+ step_id = context.get("tool_call_id")
+ if stream is None or not isinstance(step_id, str) or not step_id:
+ return None
+ name = step_label(node_class)
+ if name is None:
+ return None
+ workflow_id = stream.get("workflow_id")
+ step = Step(
+ run_id=stream["run_id"],
+ session_id=stream["session_id"],
+ workflow_id=workflow_id if isinstance(workflow_id, str) else None,
+ step_id=step_id,
+ name=name,
+ started=time.monotonic(),
+ )
+ _publish(step, "step.started", {"step_id": step_id, "step_name": name}, f"step:{step_id}:start")
+ return step
+
+
+async def end(step: Optional[Step], *, state: str, detail: Optional[str] = None) -> None:
+ """Close a step (``done``, ``failed`` or ``skipped``) and save it."""
+ if step is None:
+ return
+ fields: Dict[str, Any] = {
+ "step_id": step.step_id,
+ "step_name": step.name,
+ "state": state,
+ "duration_ms": int((time.monotonic() - step.started) * 1000),
+ }
+ if detail:
+ fields["detail"] = detail[:200]
+ _publish(step, "step.finished", fields, f"step:{step.step_id}:end")
+ try:
+ from core.container import container
+ from services.chat import ledger
+
+ saved = {key: fields[key] for key in ("step_id", "state", "duration_ms", "detail") if key in fields}
+ saved["name"] = step.name
+ await ledger.record_step(container.database(), step.run_id, saved)
+ except Exception: # noqa: BLE001
+ logger.warning("Chat step could not be saved", run_id=step.run_id, exc_info=True)
+
+
+async def run_stopped(context: Mapping[str, Any]) -> bool:
+ """Whether the call belongs to a chat run the owner stopped."""
+ run_id = context.get("chat_run_id")
+ if not isinstance(run_id, str) or not run_id or not context.get("tool_call_id"):
+ return False
+ try:
+ from core.container import container
+ from services.chat import ledger
+
+ return await ledger.is_stopping(container.database(), run_id)
+ except Exception: # noqa: BLE001 - never skip a call over a failed read
+ logger.warning("Could not read whether a chat run is stopping", exc_info=True)
+ return False
+
+
+def take_detail(result: Any) -> Optional[str]:
+ """Remove a tool's ``_step_detail`` from its result, wherever the
+ plugin's result shape puts it, and return it."""
+ if not isinstance(result, dict):
+ return None
+ detail = result.pop(STEP_DETAIL_KEY, None)
+ inner = result.get("result")
+ if isinstance(inner, dict):
+ detail = inner.pop(STEP_DETAIL_KEY, None) or detail
+ return detail if isinstance(detail, str) and detail.strip() else None
+
+
+__all__ = ["STEP_DETAIL_KEY", "STOPPED_RESULT", "Step", "begin", "end", "run_stopped", "step_label", "take_detail"]
diff --git a/server/services/chat/stream.py b/server/services/chat/stream.py
new file mode 100644
index 00000000..652ca543
--- /dev/null
+++ b/server/services/chat/stream.py
@@ -0,0 +1,238 @@
+"""The employee's answer, streamed into the chat as it is written.
+
+One :class:`ChatStreamEmitter` per LLM step of the agent that answers a chat
+run (``chat_stream`` in the step's payload, set by ``agent.prepare_payload``
+for the agent whose output reaches a reply node). It turns the provider's
+:class:`~services.llm.protocol.StreamEvent` deltas into the run's text
+events (docs-internal/chat_protocol.md):
+
+- **One segment per attempt.** The segment id is ``<run>.<iteration>.<attempt>``.
+ A retried step first discards the segments its earlier attempts streamed
+ (``custom`` ``opencompany.segment_discarded``), so a client never shows
+ text twice.
+- **Batched.** Deltas go out every ``stream.flush_ms`` or ``stream.flush_chars``
+ (``config/chat_defaults.json``), and whatever is left when the step ends.
+- **Held back.** Text that could still turn out to be ``NO_REPLY`` (the agent
+ had nothing to say, so the reply node posts nothing) is not shown until it
+ stops matching; a ``<followups>`` block, and anything after it, never
+ streams (the server reads it when the run ends).
+- **Reply or narration.** ``text.ended`` says ``final: true`` when the step
+ answered, ``final: false`` when tool calls followed (narration beside them).
+
+Publishing never fails the step: the hub drops what it cannot deliver, and a
+problem here is logged, not raised.
+"""
+
+from __future__ import annotations
+
+import time
+from typing import Any, Callable, Dict, List, Mapping, Optional
+
+from core.logging import get_logger
+from services.approvals.contract import NO_REPLY
+from services.chat.config import stream_setting
+from services.llm.protocol import StreamEvent
+
+logger = get_logger(__name__)
+
+#: Opens the block the agent ends a reply with to offer follow-up questions.
+FOLLOWUPS_OPEN = "<followups>"
+
+Publish = Callable[..., Any]
+
+
+def _default_publish(**kwargs: Any) -> Any:
+ from services.chat.hub import get_chat_hub
+
+ return get_chat_hub().publish(**kwargs)
+
+
+def segment_id(run_id: str, iteration: int, attempt: int) -> str:
+ return f"{run_id}.{iteration}.{attempt}"
+
+
+def chat_run_id_of(context: Mapping[str, Any]) -> Optional[str]:
+ """The chat run a node works for (its ``run_scope``), or None."""
+ scope = context.get("run_scope")
+ run_id = scope.get("run_id") if isinstance(scope, Mapping) else None
+ return run_id if isinstance(run_id, str) and run_id else None
+
+
+def chat_stream_for(context: Mapping[str, Any]) -> Optional[Dict[str, Any]]:
+ """Where an agent's text streams, or None when it does not answer a chat
+ run.
+
+ An agent answers the run when the run's scope reached it (MachinaWorkflow
+ puts ``run_scope`` on every node of the run), it is not working for
+ another agent (a delegated child carries ``parent_node_id``), and its
+ output goes straight to a node that posts the run's answer
+ (``answers_chat_run``, Reply in Chat). Decided by the plugin attribute,
+ never by a node type name.
+ """
+ scope = context.get("run_scope")
+ if not isinstance(scope, Mapping) or context.get("parent_node_id"):
+ return None
+ run_id = chat_run_id_of(context)
+ session_id = scope.get("session_id")
+ if run_id is None or not isinstance(session_id, str) or not session_id:
+ return None
+ node_id = context.get("node_id")
+ nodes = {str(node.get("id")): node for node in context.get("nodes") or [] if isinstance(node, Mapping)}
+ from services.chat.ledger import reply_uid
+ from services.node_registry import get_node_class
+
+ for edge in context.get("edges") or []:
+ if not isinstance(edge, Mapping) or edge.get("source") != node_id:
+ continue
+ target = nodes.get(str(edge.get("target")))
+ node_class = get_node_class(str(target.get("type") or "")) if target else None
+ if node_class is not None and getattr(node_class, "answers_chat_run", False):
+ workflow_id = context.get("workflow_id")
+ return {
+ "run_id": run_id,
+ "session_id": session_id,
+ "workflow_id": workflow_id if isinstance(workflow_id, str) and workflow_id else session_id,
+ "reply_message_id": reply_uid(run_id),
+ }
+ return None
+
+
+def shown_text(text: str) -> str:
+ """The part of ``text`` a client may show: nothing while it could still
+ be ``NO_REPLY``, and nothing from a ``<followups>`` block on (nor a tag
+ that is still arriving)."""
+ stripped = text.lstrip()
+ if stripped and NO_REPLY.startswith(stripped):
+ return ""
+ cut = text.find(FOLLOWUPS_OPEN)
+ if cut != -1:
+ return text[:cut]
+ # A tail that could still become the tag waits for the next delta.
+ for length in range(min(len(FOLLOWUPS_OPEN) - 1, len(text)), 0, -1):
+ if text.endswith(FOLLOWUPS_OPEN[:length]):
+ return text[:-length]
+ return text
+
+
+class ChatStreamEmitter:
+ """Streams one step's text to a chat run's subscribers."""
+
+ def __init__(
+ self,
+ *,
+ run_id: str,
+ session_id: str,
+ workflow_id: Optional[str],
+ segment: str,
+ discarded: List[str],
+ reply_message_id: Optional[str],
+ flush_ms: Optional[float] = None,
+ flush_chars: Optional[int] = None,
+ publish: Publish = _default_publish,
+ clock: Callable[[], float] = time.monotonic,
+ ) -> None:
+ self.run_id = run_id
+ self.session_id = session_id
+ self.workflow_id = workflow_id
+ self.segment = segment
+ self.reply_message_id = reply_message_id
+ self._discarded = list(discarded)
+ self._flush_s = (flush_ms if flush_ms is not None else stream_setting("flush_ms")) / 1000.0
+ self._flush_chars = int(flush_chars if flush_chars is not None else stream_setting("flush_chars"))
+ self._publish = publish
+ self._clock = clock
+ self._text = ""
+ self._published = ""
+ self._opened = False
+ self._last_flush = clock()
+
+ @classmethod
+ def from_payload(cls, payload: Mapping[str, Any], *, attempt: int, **kwargs: Any) -> Optional["ChatStreamEmitter"]:
+ """The emitter for an LLM step's payload, or None when the step does
+ not answer a chat run."""
+ stream = payload.get("chat_stream")
+ if not isinstance(stream, Mapping):
+ return None
+ run_id = stream.get("run_id")
+ session_id = stream.get("session_id")
+ if not isinstance(run_id, str) or not run_id or not isinstance(session_id, str) or not session_id:
+ return None
+ iteration = int(payload.get("iteration") or 0)
+ attempt = max(1, int(attempt))
+ return cls(
+ run_id=run_id,
+ session_id=session_id,
+ workflow_id=stream.get("workflow_id") if isinstance(stream.get("workflow_id"), str) else None,
+ segment=segment_id(run_id, iteration, attempt),
+ discarded=[segment_id(run_id, iteration, earlier) for earlier in range(1, attempt)],
+ reply_message_id=stream.get("reply_message_id") if isinstance(stream.get("reply_message_id"), str) else None,
+ **kwargs,
+ )
+
+ # ---- publishing ------------------------------------------------------
+
+ def _send(self, suffix: str, fields: Dict[str, Any], event_key: Optional[str] = None) -> None:
+ try:
+ self._publish(
+ run_id=self.run_id,
+ session_id=self.session_id,
+ workflow_id=self.workflow_id,
+ suffix=suffix,
+ fields=fields,
+ event_key=event_key,
+ )
+ except Exception: # noqa: BLE001 - streaming is a courtesy; the answer is saved anyway
+ logger.warning("Chat stream event could not be published", run_id=self.run_id, suffix=suffix, exc_info=True)
+
+ def begin(self) -> None:
+ """Withdraw what earlier attempts of this step streamed."""
+ for segment in self._discarded:
+ self._send(
+ "custom",
+ {"name": "opencompany.segment_discarded", "value": {"message_id": segment}},
+ event_key=f"discard:{segment}",
+ )
+
+ def _flush(self) -> None:
+ visible = shown_text(self._text)
+ if len(visible) <= len(self._published) or not visible.startswith(self._published):
+ return
+ delta = visible[len(self._published):]
+ if not self._opened:
+ if not visible.strip():
+ return
+ self._opened = True
+ self._send("text.started", {"message_id": self.segment, "role": "assistant"}, event_key=f"start:{self.segment}")
+ self._published = visible
+ self._last_flush = self._clock()
+ self._send("text.content", {"message_id": self.segment, "delta": delta})
+
+ async def __call__(self, event: StreamEvent) -> None:
+ if event.kind != "text" or not event.delta:
+ return
+ self._text += event.delta
+ pending = len(self._text) - len(self._published)
+ if pending >= self._flush_chars or self._clock() - self._last_flush >= self._flush_s:
+ self._flush()
+
+ def end(self, *, final: bool) -> None:
+ """The step finished: send what is left and close the segment."""
+ self._flush()
+ if not self._opened:
+ return
+ fields: Dict[str, Any] = {"message_id": self.segment, "final": bool(final)}
+ if final and self.reply_message_id:
+ fields["reply_message_id"] = self.reply_message_id
+ self._send("text.ended", fields, event_key=f"end:{self.segment}")
+
+ @property
+ def text(self) -> str:
+ """Everything the provider wrote so far, held-back parts included."""
+ return self._text
+
+ @property
+ def published(self) -> str:
+ return self._published
+
+
+__all__ = ["ChatStreamEmitter", "FOLLOWUPS_OPEN", "chat_run_id_of", "chat_stream_for", "segment_id", "shown_text"]
diff --git a/server/services/chat/watchdog.py b/server/services/chat/watchdog.py
index ebc98d10..23b5dcf2 100644
--- a/server/services/chat/watchdog.py
+++ b/server/services/chat/watchdog.py
@@ -44,6 +44,27 @@ async def temporal_workflow_closed(workflow_id: str, run_id: Optional[str]) -> O
return description.status != WorkflowExecutionStatus.RUNNING
+async def cancel_temporal_workflow(workflow_id: str, run_id: Optional[str]) -> None:
+ """Ask Temporal to cancel a workflow (a stopped run's). Does nothing when
+ Temporal cannot be reached; a workflow that already closed is fine."""
+ try:
+ from core.container import container
+
+ wrapper = container.temporal_client()
+ except Exception: # noqa: BLE001 - Temporal is optional
+ return
+ client = getattr(wrapper, "client", None) if wrapper is not None else None
+ if client is None:
+ return
+ from temporalio.service import RPCError
+
+ try:
+ await client.get_workflow_handle(workflow_id, run_id=run_id).cancel()
+ except RPCError:
+ # Already closed, or unknown: nothing left to cancel.
+ return
+
+
class ChatRunWatchdog:
def __init__(self, database: Any, *, interval: Optional[float] = None) -> None:
self.database = database
@@ -67,7 +88,11 @@ class ChatRunWatchdog:
async def _loop(self) -> None:
while True:
try:
- await ledger.sweep(self.database, temporal_status=temporal_workflow_closed)
+ await ledger.sweep(
+ self.database,
+ temporal_status=temporal_workflow_closed,
+ temporal_cancel=cancel_temporal_workflow,
+ )
except asyncio.CancelledError:
raise
except Exception: # noqa: BLE001 - the next sweep tries again
@@ -75,4 +100,4 @@ class ChatRunWatchdog:
await asyncio.sleep(self.interval)
-__all__ = ["ChatRunWatchdog", "temporal_workflow_closed"]
+__all__ = ["ChatRunWatchdog", "cancel_temporal_workflow", "temporal_workflow_closed"]
diff --git a/server/services/plugin/base.py b/server/services/plugin/base.py
index b083af18..4c43c2a1 100644
--- a/server/services/plugin/base.py
+++ b/server/services/plugin/base.py
@@ -274,6 +274,17 @@ class BaseNode:
tool_name: ClassVar[str] = ""
tool_description: ClassVar[str] = ""
+ # Chat (docs-internal/chat_protocol.md). ``answers_chat_run``: this node
+ # posts the answer to a chat run (Reply in Chat), so the agent whose
+ # output it takes streams its text into the chat as it writes.
+ # ``chat_step``: how a call of this tool reads as a working step while
+ # the employee answers ("Checked Google Calendar"); empty falls back to
+ # "Used <display name>". ``chat_step_hidden``: housekeeping (the clock,
+ # a checklist) that shows no step.
+ answers_chat_run: ClassVar[bool] = False
+ chat_step: ClassVar[str] = ""
+ chat_step_hidden: ClassVar[bool] = False
+
# Set by __init_subclass__: {op_name: OperationSpec}
_operations: ClassVar[Dict[str, OperationSpec]] = {}
# Flag so concrete subclasses auto-register; abstract kinds don't.
@@ -1086,6 +1097,29 @@ class BaseNode:
)
return result
+ # Chat (services/chat/steps.py): a tool call of a run the owner
+ # stopped is answered without running; a tool call of the agent
+ # answering a chat run shows as a working step.
+ from services.chat import steps as chat_steps
+
+ if await chat_steps.run_stopped(context):
+ result = {
+ "success": True,
+ "node_id": node_id,
+ "node_type": cls.type,
+ "result": dict(chat_steps.STOPPED_RESULT),
+ "execution_id": execution_id,
+ "timestamp": datetime.now().isoformat(),
+ }
+ await broadcaster.update_node_status(
+ node_id,
+ "skipped",
+ {"stopped": True, "execution_id": execution_id},
+ workflow_id=workflow_id,
+ )
+ return result
+ chat_step = chat_steps.begin(context, cls)
+
# Broadcast executing — UI cyan-glow.
await broadcaster.update_node_status(
node_id,
@@ -1180,7 +1214,12 @@ class BaseNode:
# / TriggerNode return the {success, result} envelope.
# cls.interpret_result() normalizes both into (success,
# payload, error_message).
+ step_detail = chat_steps.take_detail(result)
success, payload, error = cls.interpret_result(result)
+ if chat_step is not None and not (
+ not success and result.get("error_type") == NODE_WAIT_INTERRUPTED
+ ):
+ await chat_steps.end(chat_step, state="done" if success else "failed", detail=step_detail)
if not success and result.get("error_type") == NODE_WAIT_INTERRUPTED:
# The body stopped waiting because this worker is going
# away, not because the work failed. Fail the attempt
@@ -1229,6 +1268,7 @@ class BaseNode:
raise
error_msg = f"{type(e).__name__}: {e}"
activity.logger.error(f"Node {node_id} crashed: {error_msg}")
+ await chat_steps.end(chat_step, state="failed")
await broadcaster.update_node_status(
node_id,
"error",
diff --git a/server/services/temporal/agent_activities.py b/server/services/temporal/agent_activities.py
index 57a338fb..9d7ff417 100644
--- a/server/services/temporal/agent_activities.py
+++ b/server/services/temporal/agent_activities.py
@@ -317,11 +317,86 @@ async def _await_with_llm_heartbeats(
await asyncio.gather(task, return_exceptions=True)
+#: ``_until_chat_run_stops`` result when the owner stopped the run.
+_STOPPED = object()
+
+
+async def _chat_run_stopping(run_id: str) -> bool:
+ """Whether the owner asked this chat run to stop. A failed read is
+ "no": stopping is never worth failing a step over."""
+ try:
+ from core.container import container
+ from services.chat import ledger
+
+ return await ledger.is_stopping(container.database(), run_id)
+ except Exception: # noqa: BLE001
+ logger.warning("Could not read whether a chat run is stopping", exc_info=True)
+ return False
+
+
+async def _until_chat_run_stops(call: Any, run_id: Optional[str]) -> Any:
+ """Await a model call, or return ``_STOPPED`` as soon as the chat run it
+ works for is stopped (polled every ``runs.stop_poll_s``: a heartbeat-
+ borne cancel arrives too late to stop a stream mid-sentence)."""
+ if not run_id:
+ return await call
+ from services.chat.config import runs_setting
+
+ poll = max(0.05, runs_setting("stop_poll_s"))
+
+ async def watch() -> None:
+ while not await _chat_run_stopping(run_id):
+ await asyncio.sleep(poll)
+
+ task = asyncio.ensure_future(call)
+ watcher = asyncio.ensure_future(watch())
+ try:
+ done, _ = await asyncio.wait({task, watcher}, return_when=asyncio.FIRST_COMPLETED)
+ if task in done:
+ return task.result()
+ return _STOPPED
+ finally:
+ for pending in (task, watcher):
+ if not pending.done():
+ pending.cancel()
+ await asyncio.gather(task, watcher, return_exceptions=True)
+
+
+async def _stopped_llm_step(payload: Dict[str, Any], messages: List[Any], *, partial: str) -> Dict[str, Any]:
+ """The step's result when the owner stopped the run: what was written so
+ far (possibly nothing) as the final answer. The conversation keeps the
+ turn as far as it went: the request (its tool results included, so no
+ call is left unanswered) and the partial answer, marked stopped; nothing
+ is added when nothing was written."""
+ from services.llm.protocol import ContentBlock, Message, Usage, message_to_wire
+
+ sent = [dict(message_to_wire(message)) for message in messages]
+ assistant = Message(
+ role="assistant",
+ content=partial,
+ blocks=[ContentBlock(type="text", text=partial, metadata={"stopped": True})] if partial else [],
+ )
+ assistant_wire = dict(message_to_wire(assistant))
+ await _save_conversation(payload, sent=sent, assistant_wire=assistant_wire if partial else None)
+ result: Dict[str, Any] = {
+ "kind": "final",
+ "assistant_message": assistant_wire,
+ "content": partial,
+ "thinking": None,
+ "usage": asdict(Usage()),
+ "stopped": True,
+ }
+ if payload.get("include_finish_reason"):
+ result["finish_reason"] = "stopped"
+ return result
+
+
async def _execute_native_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
"""Run one native-SDK model turn and retain the v1 activity envelope."""
from core.container import container
from services.agent_runtime import run_native_llm_step
+ from services.chat.stream import ChatStreamEmitter, shown_text
from services.llm.messages import filter_empty_messages
from services.llm.protocol import (
LLMError,
@@ -357,27 +432,39 @@ async def _execute_native_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
if isinstance(thinking_data, dict)
else None
)
+ # A chat run the owner stopped ends at this step, before paying for it.
+ chat_run_id = payload.get("chat_run_id")
+ if isinstance(chat_run_id, str) and chat_run_id and await _chat_run_stopping(chat_run_id):
+ return await _stopped_llm_step(payload, messages, partial="")
+ emitter = ChatStreamEmitter.from_payload(payload, attempt=_llm_activity_attempt())
+ if emitter is not None:
+ emitter.begin()
+
api_key = await _resolve_activity_api_key(payload)
try:
response = await _await_with_llm_heartbeats(
- run_native_llm_step(
- container.chat_unifier(),
- provider=payload["provider"],
- api_key=api_key,
- messages=messages,
- model=payload["model"],
- temperature=payload.get("temperature", 0.7),
- max_tokens=payload.get("max_tokens", 4096),
- thinking=thinking,
- tools=tool_defs,
- context_management=payload.get("context_management"),
- # Temporal owns retries; nested SDK retries would bypass
- # quota classification and hide the wait from the UI.
- sdk_max_retries=0,
- explicit_max_retries=0,
- # Keep structured metadata until this activity boundary,
- # where it is converted to a safe Temporal failure.
- translate_errors=False,
+ _until_chat_run_stops(
+ run_native_llm_step(
+ container.chat_unifier(),
+ provider=payload["provider"],
+ api_key=api_key,
+ messages=messages,
+ model=payload["model"],
+ temperature=payload.get("temperature", 0.7),
+ max_tokens=payload.get("max_tokens", 4096),
+ thinking=thinking,
+ tools=tool_defs,
+ context_management=payload.get("context_management"),
+ # Temporal owns retries; nested SDK retries would bypass
+ # quota classification and hide the wait from the UI.
+ sdk_max_retries=0,
+ explicit_max_retries=0,
+ # Keep structured metadata until this activity boundary,
+ # where it is converted to a safe Temporal failure.
+ translate_errors=False,
+ on_event=emitter,
+ ),
+ chat_run_id if isinstance(chat_run_id, str) else None,
),
detail=f"LLM step waiting: {payload.get('model')}",
)
@@ -396,6 +483,17 @@ async def _execute_native_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
# SDK bodies in chained tracebacks can include prompts or credentials.
raise failure from None
+ if response is _STOPPED:
+ # What the owner saw so far is the answer: the reply node posts it,
+ # saved as stopped, and the run finishes ``stopped``.
+ partial = shown_text(emitter.text).rstrip() if emitter is not None else ""
+ if emitter is not None:
+ emitter.end(final=True)
+ return await _stopped_llm_step(payload, messages, partial=partial)
+ if emitter is not None:
+ # Text beside tool calls is narration; without them it is the reply.
+ emitter.end(final=not response.tool_calls)
+
assistant = response.assistant_message or Message(
role="assistant",
content=response.content or "",
@@ -456,7 +554,7 @@ async def _save_conversation(
payload: Dict[str, Any],
*,
sent: List[Dict[str, Any]],
- assistant_wire: Dict[str, Any],
+ assistant_wire: Optional[Dict[str, Any]],
) -> None:
"""Persist the full transcript for this turn, when a key is attached.
@@ -464,7 +562,8 @@ async def _save_conversation(
exact list handed to the provider; ``sent + [assistant]`` IS the
conversation after this turn, saved whole (last-write-wins upsert per
key). Living inside the LLM activity means per-turn durability with
- zero extra activities and zero Temporal-history payload.
+ zero extra activities and zero Temporal-history payload. A step stopped
+ before the model wrote anything saves ``sent`` alone.
"""
key = payload.get("conversation_key")
if not isinstance(key, dict):
@@ -473,7 +572,7 @@ async def _save_conversation(
from core.container import container
from services.agent_context import save_conversation
- messages = [*sent, assistant_wire]
+ messages = [*sent, assistant_wire] if assistant_wire is not None else list(sent)
saved_bytes = len(json.dumps(messages, default=str).encode("utf-8"))
if saved_bytes > _SEED_TRANSCRIPT_MAX_BYTES // 2:
# The next firing refuses to load a row over the cap
@@ -1169,10 +1268,15 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
}
try:
from services.agent_context import load_conversation
-
- conversation = await load_conversation(
- database,
- **conversation_key,
+ from services.agent_context.conversation import close_unanswered_tool_calls
+
+ # A run stopped or crashed mid-tool left calls without results,
+ # which no provider accepts: answer them before continuing.
+ conversation = close_unanswered_tool_calls(
+ await load_conversation(
+ database,
+ **conversation_key,
+ )
)
except Exception as exc:
raise ApplicationError(
@@ -1381,8 +1485,13 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
CONTEXT_PRESSURE_VERSION,
transcript_budget_bytes,
)
+ from services.chat.stream import chat_run_id_of, chat_stream_for
return {
+ # The chat run this agent works for (stops when the owner presses
+ # Stop), and, for the agent that answers it, where its text streams.
+ "chat_run_id": chat_run_id_of(context),
+ "chat_stream": chat_stream_for(context),
"node_id": node_id,
"node_type": node_type,
"workflow_id": workflow_id,
diff --git a/server/services/temporal/agent_workflow.py b/server/services/temporal/agent_workflow.py
index 6156ab83..e0e36d3a 100644
--- a/server/services/temporal/agent_workflow.py
+++ b/server/services/temporal/agent_workflow.py
@@ -901,6 +901,20 @@ class AgentWorkflow:
# indistinguishable from a normal completion and gets
# returned to the user as a truncated final answer.
"include_finish_reason": True,
+ # The chat run this agent works for: the step stops when the
+ # owner presses Stop, and the agent that answers the run
+ # streams its text into the chat (services/chat/stream.py).
+ # Activity input only, so recorded histories replay as before.
+ **(
+ {"chat_run_id": payload["chat_run_id"]}
+ if payload.get("chat_run_id")
+ else {}
+ ),
+ **(
+ {"chat_stream": payload["chat_stream"]}
+ if payload.get("chat_stream")
+ else {}
+ ),
}
# Transient provider failures (429 rate limit, 5xx, network)
@@ -1785,6 +1799,19 @@ class AgentWorkflow:
call.get("id") or f"{iteration + 1}:{call_index + 1}"
),
**call_metadata,
+ # Chat (services/chat/steps.py): a call of a stopped run
+ # is not run, and a call of the agent answering the run
+ # shows as a working step. Activity input only.
+ **(
+ {"chat_run_id": payload["chat_run_id"]}
+ if payload.get("chat_run_id")
+ else {}
+ ),
+ **(
+ {"chat_stream": payload["chat_stream"]}
+ if payload.get("chat_stream")
+ else {}
+ ),
}
tool_activity_name = (
diff --git a/server/tests/services/agent_context/test_conversation.py b/server/tests/services/agent_context/test_conversation.py
index e14f0b45..360159f5 100644
--- a/server/tests/services/agent_context/test_conversation.py
+++ b/server/tests/services/agent_context/test_conversation.py
@@ -114,6 +114,31 @@ async def test_blocked_tool_turn_is_saved_once_and_stale_results_are_ignored(con
assert len(await load_conversation(conversation_database, **_KEY)) == 1
+def test_calls_a_stopped_run_left_open_are_answered():
+ """A run ended between asking for tools and getting their results (Stop
+ while a tool ran, a crash) leaves calls no provider will continue from;
+ each open call gets a result after the ones the turn already has."""
+ from services.agent_context.conversation import UNANSWERED_TOOL_RESULT, close_unanswered_tool_calls
+ from services.llm.protocol import ToolCall
+
+ assistant = dict(message_to_wire(Message(
+ role="assistant",
+ tool_calls=[ToolCall(id="c1", name="search", args={}), ToolCall(id="c2", name="calendar", args={})],
+ )))
+ answered = dict(message_to_wire(Message(role="tool", tool_call_id="c1", name="search", content="3 results")))
+ stored = [_wire("user", "Book Saturday"), assistant, answered]
+ repaired = close_unanswered_tool_calls(stored)
+ assert [(m["role"], m.get("tool_call_id")) for m in repaired] == [
+ ("user", None), ("assistant", None), ("tool", "c1"), ("tool", "c2"),
+ ]
+ assert (repaired[-1]["content"], repaired[-1]["name"]) == (UNANSWERED_TOOL_RESULT, "calendar")
+ # A closed turn, or no assistant turn at all, is returned as it is.
+ assert close_unanswered_tool_calls(repaired) is repaired
+ plain = [_wire("user", "hi"), _wire("assistant", "hello")]
+ assert close_unanswered_tool_calls(plain) is plain
+ assert close_unanswered_tool_calls([]) == []
+
+
def _sans_ts(messages: list[dict]) -> list[dict]:
return [{k: v for k, v in m.items() if k != "ts"} for m in messages]
diff --git a/server/tests/services/chat/test_handlers.py b/server/tests/services/chat/test_handlers.py
index 1fcf821c..aac12324 100644
--- a/server/tests/services/chat/test_handlers.py
+++ b/server/tests/services/chat/test_handlers.py
@@ -128,15 +128,24 @@ async def test_each_message_says_how_its_run_ended(chat):
await ledger.post_reply(chat.database, run=run, node_id="n", text="Booked.", execution_id="gen-1")
await ledger.finish_run(chat.database, run_id=answered["run_id"], temporal_workflow_id="tw2", temporal_run_id="tr", success=True)
+ await ledger.record_step(chat.database, answered["run_id"], {"step_id": "c1", "state": "done", "name": "Checked Google Calendar"})
+
result = await chat.handlers.handle_get_chat_messages({"session_id": "wf", "all_generations": True}, None)
first, second, reply = result["messages"]
+ # How long each run worked comes with it.
+ assert all(message["run"].pop("duration_ms") >= 0 for message in (first, second, reply))
assert first["run"] == {
"run_id": failed["run_id"],
"state": "error",
"outcome": None,
"error": {"message": "Calendar said no", "code": "run_failed", "hint": "Reconnect Google"},
}
- assert second["run"] == reply["run"] == {"run_id": answered["run_id"], "state": "finished", "outcome": "success"}
+ assert second["run"] == reply["run"] == {
+ "run_id": answered["run_id"],
+ "state": "finished",
+ "outcome": "success",
+ "steps": [{"step_id": "c1", "state": "done", "name": "Checked Google Calendar"}],
+ }
assert result["active_runs"] == []
diff --git a/server/tests/services/chat/test_step_activity.py b/server/tests/services/chat/test_step_activity.py
new file mode 100644
index 00000000..43f7204c
--- /dev/null
+++ b/server/tests/services/chat/test_step_activity.py
@@ -0,0 +1,107 @@
+"""A tool call run as a node activity (``BaseNode.as_activity``) for the
+agent answering a chat run: it shows as a working step with the detail the
+tool gave, the detail never reaches the model, and a call of a run the owner
+stopped is answered without running."""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+from temporalio.testing import ActivityEnvironment
+
+import nodes # noqa: F401 -- populate plugin registry
+from services.chat import ledger, steps
+from services.node_registry import get_node_class
+
+STREAM = {"run_id": "r_1", "session_id": "wf", "workflow_id": "wf", "reply_message_id": "a_r_1"}
+CONTEXT = {
+ "node_id": "wf:braveSearch:1",
+ "workflow_id": "wf",
+ "node_data": {},
+ "tool_call_id": "call_1",
+ "chat_run_id": "r_1",
+ "chat_stream": STREAM,
+}
+
+
+def surroundings(monkeypatch):
+ from core.container import container
+ import services.chat.hub as hub_module
+
+ workflow_service = MagicMock()
+ workflow_service.execute_node = AsyncMock(
+ return_value={"success": True, "result": {"results": ["a", "b", "c"], "_step_detail": "3 results"}}
+ )
+ monkeypatch.setattr(container, "workflow_service", lambda: workflow_service)
+ monkeypatch.setattr(container, "database", lambda: "db")
+
+ broadcaster = MagicMock()
+ broadcaster.update_node_status = AsyncMock()
+ broadcaster.update_node_output = AsyncMock()
+ monkeypatch.setattr("services.status_broadcaster.get_status_broadcaster", lambda: broadcaster)
+
+ published: List[Dict[str, Any]] = []
+
+ class Hub:
+ def publish(self, **kwargs):
+ published.append(kwargs)
+
+ monkeypatch.setattr(hub_module, "get_chat_hub", lambda: Hub())
+ saved: List[Any] = []
+
+ async def record_step(database, run_id, step):
+ saved.append((run_id, step))
+
+ stopping = {"value": False}
+
+ async def is_stopping(database, run_id):
+ return stopping["value"]
+
+ monkeypatch.setattr(ledger, "record_step", record_step)
+ monkeypatch.setattr(ledger, "is_stopping", is_stopping)
+ return type(
+ "Surroundings",
+ (),
+ {"service": workflow_service, "broadcaster": broadcaster, "published": published, "saved": saved, "stopping": stopping},
+ )
+
+
+async def test_a_tool_call_shows_as_a_step(surroundings):
+ activity_fn = get_node_class("braveSearch").as_activity()
+ result = await ActivityEnvironment().run(activity_fn, dict(CONTEXT))
+ # The model reads the result without the detail.
+ assert result["result"] == {"results": ["a", "b", "c"]}
+ started, finished = surroundings.published
+ assert (started["suffix"], started["fields"]["step_name"]) == ("step.started", "Searched the web")
+ assert (finished["suffix"], finished["fields"]["state"], finished["fields"]["detail"]) == ("step.finished", "done", "3 results")
+ [(run_id, step)] = surroundings.saved
+ assert run_id == "r_1" and step["detail"] == "3 results"
+
+
+async def test_a_failed_tool_call_is_a_failed_step(surroundings):
+ surroundings.service.execute_node.return_value = {"success": False, "error": "quota", "error_type": "NodeUserError"}
+ activity_fn = get_node_class("braveSearch").as_activity()
+ await ActivityEnvironment().run(activity_fn, dict(CONTEXT))
+ assert surroundings.published[-1]["fields"]["state"] == "failed"
+
+
+async def test_a_call_of_a_stopped_run_is_not_run(surroundings):
+ surroundings.stopping["value"] = True
+ activity_fn = get_node_class("braveSearch").as_activity()
+ result = await ActivityEnvironment().run(activity_fn, dict(CONTEXT))
+ assert result["success"] is True and result["result"] == steps.STOPPED_RESULT
+ surroundings.service.execute_node.assert_not_awaited()
+ assert surroundings.published == []
+ statuses = [call.args[1] for call in surroundings.broadcaster.update_node_status.await_args_list]
+ assert statuses == ["skipped"]
+
+
+async def test_a_node_outside_a_chat_run_is_untouched(surroundings):
+ surroundings.stopping["value"] = True
+ activity_fn = get_node_class("braveSearch").as_activity()
+ result = await ActivityEnvironment().run(activity_fn, {"node_id": "wf:braveSearch:1", "workflow_id": "wf", "node_data": {}})
+ surroundings.service.execute_node.assert_awaited_once()
+ assert result["success"] is True and surroundings.published == []
diff --git a/server/tests/services/chat/test_steps.py b/server/tests/services/chat/test_steps.py
new file mode 100644
index 00000000..a1510228
--- /dev/null
+++ b/server/tests/services/chat/test_steps.py
@@ -0,0 +1,133 @@
+"""Working steps (services/chat/steps.py): a tool call of the agent answering
+a chat run shows as a step labelled by its plugin, hidden tools show none,
+the step detail is taken out of the result, finished steps are saved on the
+run, and a tool call of a stopped run is not run."""
+
+from __future__ import annotations
+
+from types import SimpleNamespace
+from typing import Any, Dict, List
+
+import pytest
+
+from services.chat import ledger, steps
+
+STREAM = {"run_id": "r_1", "session_id": "wf", "workflow_id": "wf", "reply_message_id": "a_r_1"}
+
+
+def published(monkeypatch):
+ sent: List[Dict[str, Any]] = []
+
+ class Hub:
+ def publish(self, **kwargs):
+ sent.append(kwargs)
+
+ import services.chat.hub as hub_module
+
+ monkeypatch.setattr(hub_module, "get_chat_hub", lambda: Hub())
+ return sent
+
+
+def saved(monkeypatch):
+ calls: List[Any] = []
+
+ async def record_step(database, run_id, step):
+ calls.append((run_id, step))
+
+ monkeypatch.setattr(ledger, "record_step", record_step)
+ import core.container as container_module
+
+ monkeypatch.setattr(container_module, "container", SimpleNamespace(database=lambda: "db"))
+ return calls
+
+
+def test_labels_come_from_the_plugin():
+ assert steps.step_label(SimpleNamespace(chat_step="Searched the web", display_name="Brave Search")) == "Searched the web"
+ assert steps.step_label(SimpleNamespace(chat_step="", display_name="Gmail")) == "Used Gmail"
+ assert steps.step_label(SimpleNamespace(chat_step_hidden=True, display_name="Current Time")) is None
+
+
+def test_the_shipped_labels():
+ from nodes.search.brave_search import BraveSearchNode
+ from nodes.tool.current_time_tool import CurrentTimeToolNode
+ from nodes.tool.write_todos import WriteTodosNode
+
+ assert steps.step_label(BraveSearchNode) == "Searched the web"
+ assert steps.step_label(CurrentTimeToolNode) is None
+ assert steps.step_label(WriteTodosNode) is None
+
+
+async def test_a_tool_call_of_the_answering_agent_is_a_step(published, saved):
+ tool = SimpleNamespace(chat_step="Searched the web")
+ step = steps.begin({"chat_stream": STREAM, "tool_call_id": "call_1"}, tool)
+ assert step is not None
+ await steps.end(step, state="done", detail="3 results for salons near me" + "." * 300)
+ started, finished = published
+ assert (started["suffix"], started["fields"], started["event_key"]) == (
+ "step.started",
+ {"step_id": "call_1", "step_name": "Searched the web"},
+ "step:call_1:start",
+ )
+ assert finished["suffix"] == "step.finished" and finished["event_key"] == "step:call_1:end"
+ assert finished["fields"]["state"] == "done" and len(finished["fields"]["detail"]) == 200
+ assert finished["fields"]["duration_ms"] >= 0
+ [(run_id, record)] = saved
+ assert run_id == "r_1"
+ assert record["name"] == "Searched the web" and record["step_id"] == "call_1" and record["state"] == "done"
+
+
+def test_other_calls_are_not_steps(published):
+ tool = SimpleNamespace(chat_step="Searched the web")
+ # No chat stream (another agent, a canvas run), no call id, a hidden tool.
+ assert steps.begin({"tool_call_id": "call_1"}, tool) is None
+ assert steps.begin({"chat_stream": STREAM}, tool) is None
+ assert steps.begin({"chat_stream": {"run_id": "r_1"}, "tool_call_id": "call_1"}, tool) is None
+ assert steps.begin({"chat_stream": STREAM, "tool_call_id": "call_1"}, SimpleNamespace(chat_step_hidden=True)) is None
+ assert published == []
+
+
+async def test_ending_no_step_does_nothing(saved):
+ await steps.end(None, state="done")
+ assert saved == []
+
+
+ ("result", "remaining"),
+ [
+ ({"success": True, "_step_detail": "Found 3", "x": 1}, {"success": True, "x": 1}),
+ ({"success": True, "result": {"_step_detail": "Found 3", "x": 1}}, {"success": True, "result": {"x": 1}}),
+ ],
+)
+def test_the_detail_is_taken_out_of_the_result(result, remaining):
+ assert steps.take_detail(result) == "Found 3"
+ assert result == remaining
+
+
+def test_no_detail():
+ assert steps.take_detail({"success": True}) is None
+ assert steps.take_detail({"_step_detail": " "}) is None
+ assert steps.take_detail("plain") is None
+
+
+async def test_a_call_of_a_stopped_run_is_not_run(monkeypatch):
+ import core.container as container_module
+
+ monkeypatch.setattr(container_module, "container", SimpleNamespace(database=lambda: "db"))
+ stopping = {"r_1": True, "r_2": False}
+
+ async def is_stopping(database, run_id):
+ return stopping[run_id]
+
+ monkeypatch.setattr(ledger, "is_stopping", is_stopping)
+ assert await steps.run_stopped({"chat_run_id": "r_1", "tool_call_id": "call_1"}) is True
+ assert await steps.run_stopped({"chat_run_id": "r_2", "tool_call_id": "call_1"}) is False
+ # Only an agent's tool call is skipped: a node of the run itself runs.
+ assert await steps.run_stopped({"chat_run_id": "r_1"}) is False
+
+ async def broken(*_args):
+ raise RuntimeError("database is locked")
+
+ monkeypatch.setattr(ledger, "is_stopping", broken)
+ assert await steps.run_stopped({"chat_run_id": "r_1", "tool_call_id": "call_1"}) is False
diff --git a/server/tests/services/chat/test_stop.py b/server/tests/services/chat/test_stop.py
new file mode 100644
index 00000000..6d25dc80
--- /dev/null
+++ b/server/tests/services/chat/test_stop.py
@@ -0,0 +1,221 @@
+"""Stopping a chat run (services/chat/ledger.py, services/chat/handlers.py):
+a run nothing picked up ends at once and frees the lane, and a workflow that
+picks it up later claims it stopped; a running run moves to ``stopping``, its
+reply so far is saved stopped and it finishes ``stopped``; the watchdog ends a
+run still stopping after the grace and cancels its workflow. Steps are saved
+on the run as they finish."""
+
+from __future__ import annotations
+
+import asyncio
+from datetime import datetime, timedelta, timezone
+
+from sqlalchemy import update
+
+from models.chat import ChatRun
+from services.chat import ledger
+from tests.services.chat._helpers import FakeSocket, talking
+
+NOW = datetime(2026, 10, 4, 12, 0, tzinfo=timezone.utc)
+CLAIM = dict(temporal_workflow_id="tw-1", temporal_run_id="tr-1")
+
+
+async def admit(database, *, state="pending", session_id="wf"):
+ admission = await ledger.admit_message(
+ database, session_id=session_id, workflow_id=session_id, execution_id="gen-1",
+ text="Book Saturday", track=True, state=state,
+ )
+ return admission.run
+
+
+async def running(database):
+ run = await admit(database)
+ return await ledger.start_run(database, run_id=run.run_id, **CLAIM)
+
+
+def kinds(socket):
+ return [event["type"].removeprefix("com.opencompany.chat.run.") for event in socket.events()]
+
+
+# ----- a run nothing picked up -----
+
+
+async def test_stopping_a_pending_run_ends_it_at_once(database, hub):
+ socket = FakeSocket()
+ hub.subscribe(socket, "wf")
+ run = await admit(database)
+ stopped = await ledger.request_stop(database, run.run_id)
+ assert (stopped.state, stopped.outcome, stopped.result) == ("stopped", "stopped", {"no_reply": True})
+ assert stopped.stop_requested_at is not None and stopped.finished_at is not None
+ await asyncio.sleep(0)
+ [event] = socket.events()
+ assert event["type"] == "com.opencompany.chat.run.finished"
+ assert event["data"]["outcome"] == {"type": "stopped"}
+ # The lane is free at once.
+ assert await admit(database) is not None
+
+
+async def test_stopping_a_queued_run_ends_it_before_resume(database, hub):
+ run = await admit(database, state="queued")
+ assert (await ledger.request_stop(database, run.run_id)).state == "stopped"
+
+
+async def test_a_workflow_that_picks_up_a_stopped_run_claims_it_stopped(database, hub):
+ socket = FakeSocket()
+ hub.subscribe(socket, "wf")
+ run = await admit(database)
+ await ledger.request_stop(database, run.run_id)
+ claimed = await ledger.start_run(database, run_id=run.run_id, **CLAIM)
+ # Claimed as it is, so the agent stops at its first step instead of
+ # answering untracked; nothing says it started.
+ assert claimed is not None and claimed.state == "stopped"
+ assert (claimed.temporal_workflow_id, claimed.temporal_run_id) == ("tw-1", "tr-1")
+ assert await ledger.is_stopping(database, run.run_id) is True
+ # Another workflow does not get it, and the claimant's finish changes nothing.
+ assert await ledger.start_run(database, run_id=run.run_id, temporal_workflow_id="tw-2", temporal_run_id="tr-1") is None
+ finished = await ledger.finish_run(database, run_id=run.run_id, success=True, **CLAIM)
+ assert (finished.state, finished.outcome) == ("stopped", "stopped")
+ await asyncio.sleep(0)
+ assert kinds(socket) == ["finished"]
+
+
+# ----- a running run -----
+
+
+async def test_stopping_a_running_run_lets_it_stop_itself(database, hub):
+ socket = FakeSocket()
+ hub.subscribe(socket, "wf")
+ run = await running(database)
+ stopping = await ledger.request_stop(database, run.run_id)
+ assert stopping.state == "stopping" and stopping.stop_requested_at is not None
+ assert await ledger.is_stopping(database, run.run_id) is True
+ # A second press changes nothing and says nothing new.
+ assert (await ledger.request_stop(database, run.run_id)).state == "stopping"
+ await asyncio.sleep(0)
+ assert kinds(socket) == ["started", "custom"]
+ assert socket.events()[-1]["data"]["name"] == "opencompany.stopping"
+
+
+async def test_a_stopped_runs_reply_so_far_is_saved_stopped(database, hub):
+ run = await running(database)
+ await ledger.request_stop(database, run.run_id)
+ reply = await ledger.post_reply(database, run=run, node_id="wf:chatReply:1", text="I was about to", execution_id="gen-1")
+ assert (reply["uid"], reply["status"]) == (run.reply_message_uid, "stopped")
+ finished = await ledger.finish_run(database, run_id=run.run_id, success=True, **CLAIM)
+ assert (finished.state, finished.outcome) == ("stopped", "stopped")
+ assert finished.result == {"reply_message_id": run.reply_message_uid}
+ # The lane is free again.
+ assert await admit(database) is not None
+
+
+async def test_a_run_that_stopped_before_writing_ends_without_a_reply(database, hub):
+ run = await running(database)
+ await ledger.request_stop(database, run.run_id)
+ # A failure while stopping is still a stop: the owner asked for it.
+ finished = await ledger.finish_run(database, run_id=run.run_id, success=False, error="cancelled", **CLAIM)
+ assert (finished.state, finished.outcome, finished.result) == ("stopped", "stopped", {"no_reply": True})
+
+
+async def test_a_run_that_ended_is_not_stopped(database, hub):
+ run = await running(database)
+ await ledger.finish_run(database, run_id=run.run_id, success=True, **CLAIM)
+ assert (await ledger.request_stop(database, run.run_id)).state == "finished"
+ assert await ledger.request_stop(database, "r_unknown") is None
+ assert await ledger.is_stopping(database, "r_unknown") is False
+
+
+# ----- the watchdog -----
+
+
+async def stop_requested(database, run_id, moment):
+ async with database.get_session() as session:
+ await session.execute(update(ChatRun).where(ChatRun.run_id == run_id).values(stop_requested_at=moment))
+ await session.commit()
+
+
+async def test_the_watchdog_ends_a_run_still_stopping_after_the_grace(database, hub):
+ from services.chat.config import runs_setting
+
+ run = await running(database)
+ await ledger.request_stop(database, run.run_id)
+ await ledger.post_reply(database, run=run, node_id="n", text="Half an answer", execution_id="gen-1")
+ cancelled = []
+
+ async def cancel(workflow_id, temporal_run_id):
+ cancelled.append((workflow_id, temporal_run_id))
+
+ grace = timedelta(seconds=runs_setting("stop_grace_s"))
+ await stop_requested(database, run.run_id, NOW - grace + timedelta(seconds=5))
+ assert await ledger.sweep(database, now=NOW, temporal_cancel=cancel) == []
+ assert cancelled == []
+
+ await stop_requested(database, run.run_id, NOW - grace - timedelta(seconds=1))
+ assert await ledger.sweep(database, now=NOW, temporal_cancel=cancel) == [run.run_id]
+ assert cancelled == [("tw-1", "tr-1")]
+ ended = await ledger.get_run(database, run.run_id)
+ assert (ended.state, ended.outcome, ended.result) == ("stopped", "stopped", {"reply_message_id": run.reply_message_uid})
+
+
+async def test_a_failed_cancel_still_ends_the_run(database, hub):
+ run = await running(database)
+ await ledger.request_stop(database, run.run_id)
+ await stop_requested(database, run.run_id, NOW - timedelta(days=1))
+
+ async def broken(*_):
+ raise RuntimeError("temporal is down")
+
+ assert await ledger.sweep(database, now=NOW, temporal_cancel=broken) == [run.run_id]
+ assert (await ledger.get_run(database, run.run_id)).state == "stopped"
+
+
+# ----- steps -----
+
+
+async def test_steps_are_saved_once_each_up_to_the_cap(database, hub, monkeypatch):
+ run = await running(database)
+ await ledger.record_step(database, run.run_id, {"step_id": "c1", "state": "done", "name": "Searched the web"})
+ await ledger.record_step(database, run.run_id, {"step_id": "c2", "state": "failed", "name": "Used Gmail"})
+ # A retried step replaces its earlier save.
+ await ledger.record_step(database, run.run_id, {"step_id": "c1", "state": "done", "name": "Searched the web", "duration_ms": 900})
+ saved = (await ledger.get_run(database, run.run_id)).steps
+ assert [(step["step_id"], step.get("duration_ms")) for step in saved] == [("c2", None), ("c1", 900)]
+
+ monkeypatch.setattr(ledger, "steps_setting", lambda name: 2)
+ await ledger.record_step(database, run.run_id, {"step_id": "c3", "state": "done", "name": "Used Drive"})
+ assert len((await ledger.get_run(database, run.run_id)).steps) == 2
+ # Nothing to save without an id, or for a run that is gone.
+ await ledger.record_step(database, run.run_id, {"state": "done"})
+ await ledger.record_step(database, "r_unknown", {"step_id": "c4"})
+
+
+# ----- the command -----
+
+
+async def test_the_stop_command(chat):
+ await talking(chat.database)
+ sent = await chat.handlers.handle_send_chat_message({"message": "Book Saturday", "session_id": "wf"}, None)
+ stopped = await chat.handlers.handle_stop_chat_run({"run_id": sent["run_id"]}, None)
+ assert stopped == {"success": True, "run_id": sent["run_id"], "state": "stopped"}
+ # Pressed again, it answers the same.
+ assert (await chat.handlers.handle_stop_chat_run({"run_id": sent["run_id"]}, None))["state"] == "stopped"
+
+ second = await chat.handlers.handle_send_chat_message({"message": "And Sunday?", "session_id": "wf"}, None)
+ await ledger.start_run(chat.database, run_id=second["run_id"], **CLAIM)
+ assert (await chat.handlers.handle_stop_chat_run({"run_id": second["run_id"]}, None))["state"] == "stopping"
+ await ledger.finish_run(chat.database, run_id=second["run_id"], success=True, **CLAIM)
+
+ third = await chat.handlers.handle_send_chat_message({"message": "Thanks", "session_id": "wf"}, None)
+ await ledger.start_run(chat.database, run_id=third["run_id"], temporal_workflow_id="tw-3", temporal_run_id="tr-3")
+ await ledger.finish_run(chat.database, run_id=third["run_id"], temporal_workflow_id="tw-3", temporal_run_id="tr-3", success=True)
+ refused = await chat.handlers.handle_stop_chat_run({"run_id": third["run_id"]}, None)
+ assert refused == {"success": False, "error": "not_stoppable", "run_id": third["run_id"], "state": "finished"}
+ assert (await chat.handlers.handle_stop_chat_run({"run_id": "r_unknown"}, None))["error"] == "not_found"
+
+
+async def test_only_the_workflows_owner_may_stop_its_runs(chat):
+ await talking(chat.database)
+ await chat.database.save_workflow("wf", "Salon", "Salon_1", {"nodes": [], "edges": [], "owner_id": "alice"})
+ sent = await chat.handlers.handle_send_chat_message({"message": "Book Saturday", "session_id": "wf"}, FakeSocket(user_id="alice"))
+ for socket in (FakeSocket(user_id="mallory"), FakeSocket(path="/ws/internal")):
+ assert (await chat.handlers.handle_stop_chat_run({"run_id": sent["run_id"]}, socket))["error"] == "access_denied"
+ assert (await ledger.get_run(chat.database, sent["run_id"])).state == "pending"
diff --git a/server/tests/services/chat/test_stream.py b/server/tests/services/chat/test_stream.py
new file mode 100644
index 00000000..83b1962a
--- /dev/null
+++ b/server/tests/services/chat/test_stream.py
@@ -0,0 +1,214 @@
+"""The employee's answer streamed into the chat (services/chat/stream.py):
+which agent streams, what text may be shown while it is written, and the text
+events one step publishes (one segment per attempt, batched, held back while
+it could still be NO_REPLY or a follow-ups block, reply or narration)."""
+
+from __future__ import annotations
+
+from types import SimpleNamespace
+from typing import Any, Dict, List
+
+import pytest
+
+from services.chat import stream
+from services.llm.protocol import StreamEvent
+
+
+# ----- which agent streams -----
+
+
+def _graph(agent_id="wf:aiAgent:1", target_type="chatReply"):
+ nodes = [
+ {"id": agent_id, "type": "aiAgent"},
+ {"id": "wf:out:1", "type": target_type},
+ ]
+ edges = [{"source": agent_id, "target": "wf:out:1"}]
+ return nodes, edges
+
+
+def registry(monkeypatch):
+ import services.node_registry as node_registry
+
+ classes = {"chatReply": SimpleNamespace(answers_chat_run=True), "console": SimpleNamespace()}
+ monkeypatch.setattr(node_registry, "get_node_class", lambda node_type: classes.get(node_type))
+
+
+def _context(**overrides):
+ nodes, edges = _graph()
+ context = {
+ "node_id": "wf:aiAgent:1",
+ "workflow_id": "wf",
+ "nodes": nodes,
+ "edges": edges,
+ "run_scope": {"run_id": "r_1", "session_id": "wf"},
+ }
+ context.update(overrides)
+ return context
+
+
+def test_the_agent_whose_answer_is_the_reply_streams(registry):
+ assert stream.chat_stream_for(_context()) == {
+ "run_id": "r_1",
+ "session_id": "wf",
+ "workflow_id": "wf",
+ "reply_message_id": "a_r_1",
+ }
+ assert stream.chat_run_id_of(_context()) == "r_1"
+
+
+def test_no_other_agent_streams(registry):
+ nodes, edges = _graph(target_type="console")
+ # Its output goes elsewhere; it works for another agent; no chat run.
+ assert stream.chat_stream_for(_context(nodes=nodes, edges=edges)) is None
+ assert stream.chat_stream_for(_context(parent_node_id="wf:lead:1")) is None
+ assert stream.chat_stream_for(_context(run_scope=None)) is None
+ assert stream.chat_stream_for(_context(run_scope={"run_id": "r_1"})) is None
+ assert stream.chat_run_id_of({"run_scope": {"run_id": ""}}) is None
+
+
+def test_reply_in_chat_answers_the_run():
+ from nodes.chat.chat_reply import ChatReplyNode
+
+ assert ChatReplyNode.answers_chat_run is True
+
+
+# ----- what may be shown -----
+
+
+ ("text", "shown"),
+ [
+ ("", ""),
+ ("NO", ""),
+ (" NO_REP", ""),
+ ("NO_REPLY", ""),
+ ("No problem", "No problem"),
+ ("NOTE: done", "NOTE: done"),
+ ("Booked.\n<followups>[\"Change it\"]</followups>", "Booked.\n"),
+ ("Booked.\n<follow", "Booked.\n"),
+ ("Booked.\n<", "Booked.\n"),
+ ("a < b", "a < b"),
+ ],
+)
+def test_text_that_could_still_be_hidden_waits(text, shown):
+ assert stream.shown_text(text) == shown
+
+
+# ----- the events -----
+
+
+class Clock:
+ def __init__(self):
+ self.now = 0.0
+
+ def __call__(self):
+ return self.now
+
+
+def _emitter(published: List[Dict[str, Any]], *, attempt=1, flush_chars=10, clock=None, iteration=2):
+ def publish(**kwargs):
+ published.append(kwargs)
+
+ return stream.ChatStreamEmitter.from_payload(
+ {
+ "iteration": iteration,
+ "chat_stream": {"run_id": "r_1", "session_id": "wf", "workflow_id": "wf", "reply_message_id": "a_r_1"},
+ },
+ attempt=attempt,
+ publish=publish,
+ flush_ms=50,
+ flush_chars=flush_chars,
+ clock=clock or Clock(),
+ )
+
+
+def _text(published):
+ return "".join(event["fields"]["delta"] for event in published if event["suffix"] == "text.content")
+
+
+async def test_a_reply_streams_in_batches_and_ends_final():
+ published: List[Dict[str, Any]] = []
+ clock = Clock()
+ emitter = _emitter(published, clock=clock)
+ emitter.begin()
+ for piece in ["Your ", "booking ", "is set ", "for 10:00."]:
+ await emitter(StreamEvent("text", piece))
+ await emitter(StreamEvent("reasoning", "ignored"))
+ emitter.end(final=True)
+ suffixes = [event["suffix"] for event in published]
+ assert suffixes[0] == "text.started" and suffixes[-1] == "text.ended"
+ assert published[0]["fields"] == {"message_id": "r_1.2.1", "role": "assistant"}
+ assert _text(published) == "Your booking is set for 10:00."
+ # Batched: four deltas, fewer events than deltas.
+ assert suffixes.count("text.content") < 4
+ assert published[-1]["fields"] == {"message_id": "r_1.2.1", "final": True, "reply_message_id": "a_r_1"}
+ assert {event["run_id"] for event in published} == {"r_1"}
+
+
+async def test_time_flushes_a_short_delta():
+ published: List[Dict[str, Any]] = []
+ clock = Clock()
+ emitter = _emitter(published, clock=clock, flush_chars=1000)
+ await emitter(StreamEvent("text", "Hi"))
+ assert published == []
+ clock.now = 0.06
+ await emitter(StreamEvent("text", " there"))
+ assert _text(published) == "Hi there"
+
+
+async def test_narration_beside_tool_calls_is_not_the_reply():
+ published: List[Dict[str, Any]] = []
+ emitter = _emitter(published)
+ await emitter(StreamEvent("text", "Let me check the calendar."))
+ emitter.end(final=False)
+ assert published[-1]["fields"] == {"message_id": "r_1.2.1", "final": False}
+
+
+async def test_no_reply_never_shows():
+ published: List[Dict[str, Any]] = []
+ emitter = _emitter(published, flush_chars=1)
+ for piece in ["NO", "_RE", "PLY"]:
+ await emitter(StreamEvent("text", piece))
+ emitter.end(final=True)
+ assert published == []
+
+
+async def test_a_followups_block_never_streams():
+ published: List[Dict[str, Any]] = []
+ emitter = _emitter(published, flush_chars=1)
+ for piece in ["Booked.", " <fol", "lowups>[\"Move it\"]", "</followups>"]:
+ await emitter(StreamEvent("text", piece))
+ emitter.end(final=True)
+ assert _text(published) == "Booked. "
+ assert emitter.text.endswith("</followups>")
+
+
+async def test_a_retry_withdraws_what_earlier_attempts_streamed():
+ published: List[Dict[str, Any]] = []
+ emitter = _emitter(published, attempt=3)
+ emitter.begin()
+ discarded = [event for event in published if event["suffix"] == "custom"]
+ assert [event["fields"]["value"]["message_id"] for event in discarded] == ["r_1.2.1", "r_1.2.2"]
+ assert {event["fields"]["name"] for event in discarded} == {"opencompany.segment_discarded"}
+ assert [event["event_key"] for event in discarded] == ["discard:r_1.2.1", "discard:r_1.2.2"]
+ await emitter(StreamEvent("text", "Third time lucky."))
+ emitter.end(final=True)
+ assert published[-1]["fields"]["message_id"] == "r_1.2.3"
+
+
+async def test_a_publishing_failure_never_fails_the_step():
+ def broken(**_kwargs):
+ raise RuntimeError("hub is gone")
+
+ emitter = stream.ChatStreamEmitter.from_payload(
+ {"chat_stream": {"run_id": "r_1", "session_id": "wf"}}, attempt=1, publish=broken, flush_chars=1
+ )
+ await emitter(StreamEvent("text", "Still answering"))
+ emitter.end(final=True)
+ assert emitter.text == "Still answering"
+
+
+def test_a_step_without_a_chat_stream_has_no_emitter():
+ assert stream.ChatStreamEmitter.from_payload({}, attempt=1) is None
+ assert stream.ChatStreamEmitter.from_payload({"chat_stream": {"run_id": "r_1"}}, attempt=1) is None
diff --git a/server/tests/services/chat/test_watchdog.py b/server/tests/services/chat/test_watchdog.py
index 2ffe5f96..567bb5b6 100644
--- a/server/tests/services/chat/test_watchdog.py
+++ b/server/tests/services/chat/test_watchdog.py
@@ -1,6 +1,6 @@
"""The chat run watchdog (services/chat/watchdog.py): sweeps at once when it
-starts and then every interval, survives a failing sweep, stops cleanly, and
-asks Temporal whether a run's workflow has closed."""
+starts and then every interval, survives a failing sweep, stops cleanly, asks
+Temporal whether a run's workflow has closed, and cancels a stopped run's."""
from __future__ import annotations
@@ -15,8 +15,8 @@ from services.chat import watchdog
async def test_it_sweeps_at_once_and_keeps_sweeping_after_a_failure(monkeypatch):
sweeps = []
- async def sweep(database, *, temporal_status):
- sweeps.append((database, temporal_status))
+ async def sweep(database, *, temporal_status, temporal_cancel):
+ sweeps.append((database, temporal_status, temporal_cancel))
if len(sweeps) == 1:
raise RuntimeError("database is locked")
return []
@@ -32,7 +32,10 @@ async def test_it_sweeps_at_once_and_keeps_sweeping_after_a_failure(monkeypatch)
await dog.stop()
await dog.stop() # stopping twice is harmless
assert len(sweeps) >= 3
- assert all(database == "db" and status is watchdog.temporal_workflow_closed for database, status in sweeps)
+ assert all(
+ database == "db" and status is watchdog.temporal_workflow_closed and cancel is watchdog.cancel_temporal_workflow
+ for database, status, cancel in sweeps
+ )
count = len(sweeps)
await asyncio.sleep(0.05)
assert len(sweeps) == count, "the loop kept running after stop"
@@ -91,3 +94,31 @@ async def test_a_workflow_temporal_no_longer_knows_is_closed(monkeypatch):
assert await watchdog.temporal_workflow_closed("w", "r") is True
client.status = RPCStatusCode.UNAVAILABLE
assert await watchdog.temporal_workflow_closed("w", "r") is None
+
+
+async def test_cancelling_a_stopped_runs_workflow(monkeypatch):
+ from temporalio.service import RPCError, RPCStatusCode
+
+ cancelled = []
+
+ class Handle:
+ def __init__(self, workflow_id, run_id):
+ self.ids = (workflow_id, run_id)
+
+ async def cancel(self):
+ if self.ids[0] == "gone":
+ raise RPCError("already closed", RPCStatusCode.NOT_FOUND, b"")
+ cancelled.append(self.ids)
+
+ class Client:
+ def get_workflow_handle(self, workflow_id, run_id=None):
+ return Handle(workflow_id, run_id)
+
+ _container(monkeypatch, Client())
+ await watchdog.cancel_temporal_workflow("w", "r")
+ # A workflow that already closed has nothing left to cancel.
+ await watchdog.cancel_temporal_workflow("gone", "r")
+ assert cancelled == [("w", "r")]
+ # Without Temporal there is nothing to do.
+ _container(monkeypatch, None)
+ await watchdog.cancel_temporal_workflow("w", "r")
diff --git a/server/tests/temporal/test_agent_chat_stream.py b/server/tests/temporal/test_agent_chat_stream.py
new file mode 100644
index 00000000..9620332f
--- /dev/null
+++ b/server/tests/temporal/test_agent_chat_stream.py
@@ -0,0 +1,196 @@
+"""The LLM step of an agent answering a chat run (agent.execute_llm_step,
+services/temporal/agent_activities.py): its text streams to the run, a run the
+owner stopped ends at the next step without paying for it, a stream stopped
+midway keeps what was shown as the answer, and the conversation keeps the
+turn as far as it went."""
+
+from __future__ import annotations
+
+import asyncio
+from typing import Any, Dict, List
+
+import pytest
+
+from services.llm.protocol import LLMResponse, Message, StreamEvent, ToolCall, Usage, message_to_wire
+
+STREAM = {"run_id": "r_1", "session_id": "wf", "workflow_id": "wf", "reply_message_id": "a_r_1"}
+
+
+def _payload(**extra: Any) -> Dict[str, Any]:
+ return {
+ "provider": "anthropic",
+ "model": "test-model",
+ "api_key": "secret",
+ "iteration": 0,
+ "messages": [
+ message_to_wire(Message(role="system", content="Be brief.")),
+ message_to_wire(Message(role="user", content="Book Saturday")),
+ ],
+ "tools": [],
+ "conversation_key": {"workflow_id": "wf", "generation": 1, "agent_node_id": "wf:aiAgent:1"},
+ "chat_run_id": "r_1",
+ "chat_stream": dict(STREAM),
+ "include_finish_reason": True,
+ **extra,
+ }
+
+
+def env(monkeypatch):
+ """The step's surroundings: the hub, the conversation store and the stop
+ flag, all recorded."""
+ import core.container as container_module
+ import services.chat.config as chat_config
+ import services.chat.hub as hub_module
+ from services.temporal import agent_activities
+
+ published: List[Dict[str, Any]] = []
+ saved: List[Dict[str, Any]] = []
+ state = {"stopping": False, "polls": 0}
+
+ class Hub:
+ def publish(self, **kwargs):
+ published.append(kwargs)
+
+ async def save(payload, *, sent, assistant_wire):
+ saved.append({"sent": sent, "assistant": assistant_wire})
+
+ async def stopping(run_id):
+ state["polls"] += 1
+ return state["stopping"]
+
+ real_runs_setting = chat_config.runs_setting
+ monkeypatch.setattr(hub_module, "get_chat_hub", lambda: Hub())
+ monkeypatch.setattr(agent_activities, "_save_conversation", save)
+ monkeypatch.setattr(agent_activities, "_chat_run_stopping", stopping)
+ monkeypatch.setattr(chat_config, "runs_setting", lambda name: 0.0 if name == "stop_poll_s" else real_runs_setting(name))
+ monkeypatch.setattr(container_module.container, "chat_unifier", lambda: object())
+ return type("Env", (), {"published": published, "saved": saved, "state": state})
+
+
+def _run_step(monkeypatch, step):
+ import services.agent_runtime as runtime_module
+
+ monkeypatch.setattr(runtime_module, "run_native_llm_step", step)
+
+
+def _texts(published):
+ return "".join(event["fields"]["delta"] for event in published if event["suffix"] == "text.content")
+
+
+async def test_the_answer_streams_and_the_result_is_unchanged(env, monkeypatch):
+ from services.temporal.agent_activities import _execute_native_llm_step
+
+ async def step(unifier, **kwargs):
+ sink = kwargs["on_event"]
+ for piece in ["Booked ", "for 10:00."]:
+ await sink(StreamEvent("text", piece))
+ return LLMResponse(content="Booked for 10:00.", finish_reason="stop", usage=Usage(input_tokens=3, output_tokens=4))
+
+ _run_step(monkeypatch, step)
+ result = await _execute_native_llm_step(_payload())
+ assert (result["kind"], result["content"], result["finish_reason"]) == ("final", "Booked for 10:00.", "stop")
+ assert "stopped" not in result
+ assert _texts(env.published) == "Booked for 10:00."
+ assert env.published[-1]["suffix"] == "text.ended"
+ assert env.published[-1]["fields"] == {"message_id": "r_1.0.1", "final": True, "reply_message_id": "a_r_1"}
+ [turn] = env.saved
+ assert turn["assistant"]["content"] == "Booked for 10:00."
+
+
+async def test_text_beside_tool_calls_streams_as_narration(env, monkeypatch):
+ from services.temporal.agent_activities import _execute_native_llm_step
+
+ async def step(unifier, **kwargs):
+ await kwargs["on_event"](StreamEvent("text", "Checking the calendar."))
+ return LLMResponse(
+ content="Checking the calendar.",
+ tool_calls=[ToolCall(id="call_1", name="calendar", args={})],
+ finish_reason="tool_calls",
+ usage=Usage(),
+ )
+
+ _run_step(monkeypatch, step)
+ result = await _execute_native_llm_step(_payload())
+ assert result["kind"] == "tool_calls"
+ assert env.published[-1]["fields"]["final"] is False
+
+
+async def test_a_step_that_does_not_answer_the_run_streams_nothing(env, monkeypatch):
+ from services.temporal.agent_activities import _execute_native_llm_step
+
+ sinks = []
+
+ async def step(unifier, **kwargs):
+ sinks.append(kwargs["on_event"])
+ return LLMResponse(content="Done.", usage=Usage())
+
+ _run_step(monkeypatch, step)
+ payload = _payload()
+ payload.pop("chat_stream")
+ payload.pop("chat_run_id")
+ await _execute_native_llm_step(payload)
+ assert sinks == [None] and env.published == []
+ assert env.state["polls"] == 0
+
+
+async def test_a_stopped_run_ends_before_paying_for_a_step(env, monkeypatch):
+ from services.temporal.agent_activities import _execute_native_llm_step
+
+ async def step(unifier, **kwargs):
+ raise AssertionError("the model was called for a stopped run")
+
+ _run_step(monkeypatch, step)
+ env.state["stopping"] = True
+ result = await _execute_native_llm_step(_payload())
+ assert (result["kind"], result["content"], result["stopped"], result["finish_reason"]) == ("final", "", True, "stopped")
+ assert result["usage"]["total_tokens"] == 0
+ # The request is kept (its tool results answer every call); no answer is added.
+ [turn] = env.saved
+ assert [message["role"] for message in turn["sent"]] == ["system", "user"]
+ assert turn["assistant"] is None
+ assert env.published == []
+
+
+async def test_stopping_midway_keeps_what_was_shown(env, monkeypatch):
+ from services.temporal.agent_activities import _execute_native_llm_step
+
+ cancelled = asyncio.Event()
+
+ async def step(unifier, **kwargs):
+ sink = kwargs["on_event"]
+ await sink(StreamEvent("text", "Your Saturday slot is "))
+ await sink(StreamEvent("text", "10:00 <follow"))
+ env.state["stopping"] = True
+ try:
+ await asyncio.sleep(30)
+ except asyncio.CancelledError:
+ cancelled.set()
+ raise
+ return LLMResponse(content="never", usage=Usage())
+
+ _run_step(monkeypatch, step)
+ result = await asyncio.wait_for(_execute_native_llm_step(_payload()), timeout=5)
+ assert cancelled.is_set(), "the model call kept running after Stop"
+ assert (result["kind"], result["content"], result["stopped"]) == ("final", "Your Saturday slot is 10:00", True)
+ # The partial answer is the reply: its segment ends final.
+ assert env.published[-1]["suffix"] == "text.ended" and env.published[-1]["fields"]["final"] is True
+ [turn] = env.saved
+ assert turn["assistant"]["content"] == "Your Saturday slot is 10:00"
+ assert turn["assistant"]["blocks"][0]["metadata"] == {"stopped": True}
+
+
+async def test_a_retried_step_withdraws_the_earlier_attempts_text(env, monkeypatch):
+ from services.temporal import agent_activities
+ from services.temporal.agent_activities import _execute_native_llm_step
+
+ async def step(unifier, **kwargs):
+ await kwargs["on_event"](StreamEvent("text", "Second try."))
+ return LLMResponse(content="Second try.", usage=Usage())
+
+ _run_step(monkeypatch, step)
+ monkeypatch.setattr(agent_activities, "_llm_activity_attempt", lambda: 2)
+ await _execute_native_llm_step(_payload(iteration=3))
+ first = env.published[0]
+ assert (first["suffix"], first["fields"]["value"]) == ("custom", {"message_id": "r_1.3.1"})
+ assert env.published[1]["fields"]["message_id"] == "r_1.3.2"