Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
TypeScript agentaiai-agentsall-in-onecrmcrm-systememaillinearmcpmessagingnotesnotionofficerustslackslack-alternativestartupstartup-toolstypescriptworkspace
feat(agent_inmem): stream tool calls with throttled argument peeks (#7251)
Wolf Mermelsteinc5b49d5a7b4b
diff --git a/Cargo.lock b/Cargo.lockindex 69fbafefb9..2d20502121 100644--- a/Cargo.lock+++ b/Cargo.lock@@ -721,6 +721,7 @@ dependencies = ["memory","model-entity","model_owner",+ "nom 8.0.0","non_empty","opentelemetry","opentelemetry_sdk",diff --git a/crates/agent/AGENTS.md b/crates/agent/AGENTS.mdindex 17c452c28e..15df12297d 100644--- a/crates/agent/AGENTS.md+++ b/crates/agent/AGENTS.md@@ -15,6 +15,8 @@ and usage metering.behind `model/router.rs`, plus `predefined_model.rs`/`speed.rs` (theproduct-facing model/speed vocabulary) and `metering.rs` (usage recordinghook into `ai_usage`).+- `AgentLoop::with_streamed_tool_calls` opts hosts into early tool-call+ announcements and argument deltas; the accumulator persists only final calls.- `src/stream.rs`: `StreamPart`/`ChatCompletionStream`, the wire shapecallers consume.- `src/completion.rs`: one-shot, non-looping completions (no tools).diff --git a/crates/agent/src/accumulator.rs b/crates/agent/src/accumulator.rsindex 8c9ee7780b..57f56c1188 100644--- a/crates/agent/src/accumulator.rs+++ b/crates/agent/src/accumulator.rs@@ -26,8 +26,9 @@ impl StreamAccumulator {/// as, accumulate it, and return a reference to the stored part.////// Returns `None` (accumulating nothing) for parts that carry no- /// persistable content: token [`Usage`](StreamPart::Usage) events and empty- /// text or thinking deltas. The returned reference is the unmerged part, so+ /// persistable content: token [`Usage`](StreamPart::Usage) events, empty+ /// text or thinking deltas, and the progress of a tool call still being+ /// written. The returned reference is the unmerged part, so/// a consumer can forward it as an individual streamed chunk.pub fn push(&mut self, part: StreamPart) -> Option<&AssistantMessagePart> {let part = stream_part_to_message_part(part)?;@@ -93,7 +94,12 @@ fn stream_part_to_message_part(part: StreamPart) -> Option<AssistantMessagePart>description,id,}),- // Empty text/thinking deltas and usage events carry nothing to persist.- StreamPart::Content(_) | StreamPart::Thinking(_) | StreamPart::Usage(_) => None,+ // Empty text/thinking deltas and usage events carry nothing to persist,+ // and a call still being written persists once it is finished.+ StreamPart::Content(_)+ | StreamPart::Thinking(_)+ | StreamPart::Usage(_)+ | StreamPart::ToolCallStarted(_)+ | StreamPart::ToolCallArgs { .. } => None,}}diff --git a/crates/agent/src/agent_loop.rs b/crates/agent/src/agent_loop.rsindex 38c0cf4f10..597628ca1d 100644--- a/crates/agent/src/agent_loop.rs+++ b/crates/agent/src/agent_loop.rs@@ -136,6 +136,8 @@ pub struct AgentLoop {/// Whether this loop enriches the runtime's GenAI spans (see/// [`Self::with_genai_telemetry`]).genai_telemetry: bool,+ /// See [`Self::with_streamed_tool_calls`].+ stream_tool_calls: bool,/// The router this loop's model calls use instead of the shared one, when/// a caller-supplied key pays for them (see [`Self::with_provider_api_key`]).router: Option<ModelRouter<MeteredHttpClient>>,@@ -163,6 +165,7 @@ impl AgentLoop {conversation_id: None,agent_name: None,genai_telemetry: true,+ stream_tool_calls: false,router: None,}}@@ -279,6 +282,24 @@ impl AgentLoop {self}+ /// Report each tool call as soon as the model names it, then its+ /// arguments as they stream ([`StreamPart::ToolCallStarted`],+ /// [`StreamPart::ToolCallArgs`]), before the finished+ /// [`StreamPart::ToolCall`]. Off by default: a host that only renders+ /// finished calls has nothing to do with the extra parts.+ ///+ /// A streamed call uses the provider id available on its first delta,+ /// or rig's correlation id when none is available. Its final invocation+ /// and result keep that id so the host updates the existing row.+ ///+ /// [`StreamPart::ToolCallStarted`]: crate::StreamPart::ToolCallStarted+ /// [`StreamPart::ToolCallArgs`]: crate::StreamPart::ToolCallArgs+ /// [`StreamPart::ToolCall`]: crate::StreamPart::ToolCall+ pub fn with_streamed_tool_calls(mut self) -> Self {+ self.stream_tool_calls = true;+ self+ }+/// Start a new streaming session.////// `toolset` is the combined tool set (static + MCP) for this request.@@ -495,6 +516,7 @@ impl AgentLoop {register_loaded,tool_loads: tool_search.loads.clone(),user_tool_finisher: self.user_tool_finisher.clone(),+ stream_tool_calls: self.stream_tool_calls,result_limit: self.result_limit.clone(),},recorder: self.recorder.clone(),diff --git a/crates/agent/src/convert.rs b/crates/agent/src/convert.rsindex 77130348de..6961c26542 100644--- a/crates/agent/src/convert.rs+++ b/crates/agent/src/convert.rs@@ -221,8 +221,8 @@ fn model_view(limit: Option<&ToolResultLimit>, id: &str, text: String) -> String/// `call_id` replayed to the provider for a persisted tool call or result.///-/// The persisted id is the provider call id when one exists (OpenAI's-/// `call_…`) or an internal nanoid otherwise. OpenAI's Responses API pairs a+/// The persisted id is the provider call id (or a streamed item's `fc_…`+/// id), falling back to rig's correlation id when no provider id is available. OpenAI's Responses API pairs a/// call to its result through `call_id` and rejects ids ending in a separator,/// and a nanoid can end in `_` or `-` — so trim the trailing separator(s). A/// call and its result both run through this with the same `id`, so trimmingdiff --git a/crates/agent/src/hook.rs b/crates/agent/src/hook.rsindex d972f4cb27..059479513f 100644--- a/crates/agent/src/hook.rs+++ b/crates/agent/src/hook.rs@@ -1,7 +1,7 @@/// A [`rig_agent::agent::AgentHook`] that bridges rig lifecycle events into/// [`StreamPart`] items sent through a channel.use crate::AgentError;-use crate::stream::{McpInfo, StreamPart, ToolCall, ToolResponse, Usage};+use crate::stream::{McpInfo, StreamPart, ToolCall, ToolCallStart, ToolResponse, Usage};use crate::tool_result_limit::ToolResultLimit;use ai_toolset::{SearchableTool, ToolInfo};use rig_agent::agent::hook::{@@ -148,6 +148,28 @@ pub struct BridgeInputs {/// Cuts results too long for the model to read whole, when the host/// offers [`crate::ReadToolResult`] to page through them.pub result_limit: Option<ToolResultLimit>,+ /// Report tool calls while the model is still writing them (see+ /// [`StreamPart::ToolCallStarted`]).+ pub stream_tool_calls: bool,+}++/// A call announced from its first argument delta and not yet finished.+struct StartedCall {+ /// rig's correlation id, shared by the call's deltas and (for every+ /// provider rig ships) the finished call.+ internal_call_id: String,+ /// The id the announcement went out under.+ id: String,+}++/// The ids streamed calls were announced under, so the finished call and its+/// result are reported under the same one.+#[derive(Default)]+struct StreamedCalls {+ started: Vec<StartedCall>,+ /// Finished calls awaiting their result: rig's internal id to the id the+ /// call was reported under.+ finished: HashMap<String, String>,}/// The answer a user tool gives when it has not been finished.@@ -184,6 +206,9 @@ pub struct StreamBridge {/// Finishes a user tool's pending answer before the model reads it, on/// hosts that can reach the user mid-turn (see [`UserToolFinisher`]).user_tool_finisher: Option<UserToolFinisher>,+ /// Calls announced before they finished; `None` when the loop does not+ /// stream tool calls.+ streamed: Option<Arc<Mutex<StreamedCalls>>>,/// Cuts results too long for the model to read whole.result_limit: Option<ToolResultLimit>,/// the user has requested the stream stop@@ -217,6 +242,7 @@ impl StreamBridge {register_loaded,tool_loads,user_tool_finisher,+ stream_tool_calls,result_limit,} = inputs;let (tx, rx) = mpsc::unbounded_channel();@@ -229,6 +255,7 @@ impl StreamBridge {tool_loads,searchable_catalog,user_tool_finisher,+ streamed: stream_tool_calls.then(Arc::default),result_limit,cancel,provider_ids: Arc::default(),@@ -320,7 +347,7 @@ impl StreamBridge {}}- pub(crate) fn handle_tool_call_delta(&self, provider_id: &str, internal_call_id: &str) {+ fn capture_provider_id(&self, provider_id: &str, internal_call_id: &str) {if !provider_id.is_empty() {self.provider_ids.lock()@@ -345,6 +372,59 @@ impl StreamBridge {}}+ /// Announce a tool call on its first delta - the one that names it - and+ /// forward each argument fragment after it.+ ///+ /// The announcement uses the provider's id when available, preserving+ /// Anthropic's model-visible tool ids across replay and prompt caching.+ /// Rig's internal id correlates later deltas, execution and results.+ pub(crate) fn handle_tool_call_delta(+ &self,+ provider_id: &str,+ internal_call_id: &str,+ tool_name: Option<&str>,+ delta: &str,+ ) -> ObservationAction {+ if self.cancel.is_cancelled() {+ return ObservationAction::stop(CANCELLED_REASON);+ }+ self.capture_provider_id(provider_id, internal_call_id);+ let Some(streamed) = &self.streamed else {+ return ObservationAction::Continue;+ };+ let mut streamed = streamed.lock().expect("streamed calls poisoned");+ let known = streamed+ .started+ .iter()+ .find(|call| call.internal_call_id == internal_call_id)+ .map(|call| call.id.clone());+ let id = match (known, tool_name) {+ (Some(id), _) => id,+ (None, Some(tool_name)) => {+ let id = self.call_id(None, internal_call_id);+ streamed.started.push(StartedCall {+ internal_call_id: internal_call_id.to_owned(),+ id: id.clone(),+ });+ let _ = self.tx.send(Ok(StreamPart::ToolCallStarted(ToolCallStart {+ id: id.clone(),+ name: tool_name.to_owned(),+ mcp: self.mcp_info(tool_name),+ })));+ id+ }+ // rig names a call before forwarding any of its arguments.+ (None, None) => return ObservationAction::Continue,+ };+ if !delta.is_empty() {+ let _ = self.tx.send(Ok(StreamPart::ToolCallArgs {+ id,+ delta: delta.to_owned(),+ }));+ }+ ObservationAction::Continue+ }+pub(crate) fn handle_tool_call(&self,tool_name: &str,@@ -359,8 +439,18 @@ impl StreamBridge {.ok().filter(serde_json::Value::is_object).unwrap_or_else(|| serde_json::json!({}));- let id = self.call_id(tool_call_id, internal_call_id);- let mcp = (self.routing)(tool_name).map(|i| match i {+ let id = self.finished_call_id(tool_call_id, internal_call_id);+ let _ = self.tx.send(Ok(StreamPart::ToolCall(ToolCall {+ id,+ name: tool_name.to_owned(),+ json,+ mcp: self.mcp_info(tool_name),+ })));+ ToolCallAction::Run+ }++ fn mcp_info(&self, tool_name: &str) -> Option<McpInfo> {+ (self.routing)(tool_name).map(|i| match i {ToolInfo::ExternalTool {service_name,tool_name,@@ -370,14 +460,46 @@ impl StreamBridge {tool_name,display_name,},- });- let _ = self.tx.send(Ok(StreamPart::ToolCall(ToolCall {- id,- name: tool_name.to_owned(),- json,- mcp,- })));- ToolCallAction::Run+ })+ }++ /// The id a finished call is reported under: the one it was announced+ /// under when it streamed, else the provider's, else rig's.+ ///+ /// Match by rig's correlation id or the provider id, never by tool name:+ /// parallel calls of the same tool can finish in a different order.+ fn finished_call_id(&self, tool_call_id: Option<&str>, internal_call_id: &str) -> String {+ let unstreamed = self.call_id(tool_call_id, internal_call_id);+ let Some(streamed) = &self.streamed else {+ return unstreamed;+ };+ let mut streamed = streamed.lock().expect("streamed calls poisoned");+ let announced = streamed+ .started+ .iter()+ .position(|call| call.internal_call_id == internal_call_id || call.id == unstreamed);+ let id = match announced {+ Some(at) => streamed.started.remove(at).id,+ None => unstreamed,+ };+ streamed+ .finished+ .insert(internal_call_id.to_owned(), id.clone());+ id+ }++ /// The id a tool result is reported under: its call's.+ fn result_id(&self, tool_call_id: Option<&str>, internal_call_id: &str) -> String {+ self.streamed+ .as_ref()+ .and_then(|streamed| {+ streamed+ .lock()+ .expect("streamed calls poisoned")+ .finished+ .remove(internal_call_id)+ })+ .unwrap_or_else(|| self.call_id(tool_call_id, internal_call_id))}pub(crate) async fn handle_tool_result(@@ -408,7 +530,7 @@ impl StreamBridge {(self.register_loaded)(pending).await;}- let id = self.call_id(tool_call_id, internal_call_id);+ let id = self.result_id(tool_call_id, internal_call_id);let json = if is_success {presentation_json(presentation)} else {@@ -495,8 +617,12 @@ impl AgentHook for StreamBridge {_ctx: &HookContext,event: ToolCallDelta<'_>,) -> ObservationAction {- self.handle_tool_call_delta(event.tool_call_id, event.internal_call_id);- ObservationAction::Continue+ self.handle_tool_call_delta(+ event.tool_call_id,+ event.internal_call_id,+ event.tool_name,+ event.delta,+ )}async fn on_invalid_tool_call(diff --git a/crates/agent/src/lib.rs b/crates/agent/src/lib.rsindex 97af7cd805..e8837451fa 100644--- a/crates/agent/src/lib.rs+++ b/crates/agent/src/lib.rs@@ -31,7 +31,9 @@ pub use error::AgentError;pub use hook::{FinishedUserTool, PendingUserTool, UserToolFinisher};pub use model::metering::{MeteringContext, MeteringError, ProviderSupport, WireProtocol};pub use model::{ModelSpeed, PredefinedModel, ReasoningEffort};-pub use stream::{ChatCompletionStream, McpInfo, StreamPart, ToolCall, ToolResponse, Usage};+pub use stream::{+ ChatCompletionStream, McpInfo, StreamPart, ToolCall, ToolCallStart, ToolResponse, Usage,+};pub use tool_adapter::{DynToolSetAdapter, ToolsetToolAdapter, normalize_request_schema};pub use tool_result_limit::{ReadToolResult, ToolResultLimit};diff --git a/crates/agent/src/stream.rs b/crates/agent/src/stream.rsindex c9ba17dd81..0413f4cbcf 100644--- a/crates/agent/src/stream.rs+++ b/crates/agent/src/stream.rs@@ -15,6 +15,21 @@ pub enum StreamPart {Content(String),/// A thinking/reasoning text delta from the assistant.Thinking(String),+ /// The assistant began a tool call whose arguments are still streaming.+ /// Only sent by a loop built with+ /// [`AgentLoop::with_streamed_tool_calls`](crate::AgentLoop::with_streamed_tool_calls);+ /// the finished call follows as a [`StreamPart::ToolCall`] under the+ /// same id, unless the turn ends first.+ ToolCallStarted(ToolCallStart),+ /// A fragment of a started call's JSON arguments, in arrival order.+ /// Concatenated they are the arguments as the model wrote them, which+ /// need not parse until the call finishes.+ ToolCallArgs {+ /// The [`ToolCallStart::id`] the fragment belongs to.+ id: String,+ /// The newly received text.+ delta: String,+ },/// A complete tool invocation by the assistant.ToolCall(ToolCall),/// The result of executing a tool.@@ -34,6 +49,17 @@ pub struct McpInfo {pub display_name: Option<String>,}+/// A tool call the assistant has named but not finished writing.+#[derive(Debug, Clone, Serialize)]+pub struct ToolCallStart {+ /// The id the finished [`ToolCall`] and its [`ToolResponse`] carry.+ pub id: String,+ /// Tool name (mangled for MCP tools).+ pub name: String,+ /// Present when this is an MCP tool call.+ pub mcp: Option<McpInfo>,+}+/// A tool call made by the assistant.#[derive(Debug, Clone, Serialize)]pub struct ToolCall {diff --git a/crates/agent/src/test/agent_loop/mod.rs b/crates/agent/src/test/agent_loop/mod.rsindex 00a977ad56..9e5e3784b4 100644--- a/crates/agent/src/test/agent_loop/mod.rs+++ b/crates/agent/src/test/agent_loop/mod.rs@@ -8,6 +8,7 @@ mod util;mod test_cooperative_cancellation;mod test_eager_tools;mod test_prompt_cache;+mod test_streamed_tool_calls;mod test_telemetry;mod test_thinking;mod test_tool;diff --git a/crates/agent/src/test/agent_loop/test_prompt_cache.rs b/crates/agent/src/test/agent_loop/test_prompt_cache.rsindex 5450a0f5bc..f24bfbc52e 100644--- a/crates/agent/src/test/agent_loop/test_prompt_cache.rs+++ b/crates/agent/src/test/agent_loop/test_prompt_cache.rs@@ -220,6 +220,7 @@ async fn run_session(});let before = transport.bodies().len();let mut session = util::test_loop()+ .with_streamed_tool_calls().with_model(model).test_routed_session(&router(transport),@@ -401,6 +402,7 @@ async fn a_replayed_turn_repeats_its_live_requests() {eager: AsyncToolCollection::<()>::new().add_tool::<LoadToolsStandIn, ()>(),});let mut session = util::test_loop()+ .with_streamed_tool_calls().with_model("anthropic/claude-opus-4-8").test_routed_session(&router(&transport),diff --git a/crates/agent/src/test/agent_loop/test_streamed_tool_calls.rs b/crates/agent/src/test/agent_loop/test_streamed_tool_calls.rsnew file mode 100644index 0000000000..1cba1242cc--- /dev/null+++ b/crates/agent/src/test/agent_loop/test_streamed_tool_calls.rs@@ -0,0 +1,146 @@+//! A loop that streams tool calls reports each call as the model writes it.+//!+//! The model can spend seconds writing a call's arguments - a document body,+//! an email draft - before the call exists as far as rig's `on_tool_call` is+//! concerned. A host that renders tool activity live wants the call on screen+//! from the moment the model names it.++use super::util;+use crate::stream::{StreamPart, ToolResponse};+use ai_toolset::{+ AsyncTool, AsyncToolCollection, RequestContext, ServiceContext, ToolAnnotated, ToolAnnotations,+ ToolResult,+};+use async_trait::async_trait;+use rig_core::test_utils::{MockCompletionModel, MockStreamEvent};+use schemars::JsonSchema;+use serde::Deserialize;+use std::sync::Arc;++#[derive(Deserialize, JsonSchema)]+#[schemars(title = "echo_tool", description = "Echoes its input back.")]+struct EchoTool {+ value: String,+}++impl ToolAnnotated for EchoTool {+ const ANNOTATIONS: ToolAnnotations = ToolAnnotations::read_only("Echo");+}++#[async_trait]+impl AsyncTool<()> for EchoTool {+ type Output = serde_json::Value;++ async fn call(+ &self,+ _service_context: ServiceContext<()>,+ _request_context: RequestContext,+ ) -> ToolResult<Self::Output> {+ Ok(serde_json::json!({ "echo": self.value }))+ }+}++/// One call written in fragments, then a closing text turn.+fn streamed_echo() -> MockCompletionModel {+ MockCompletionModel::from_stream_turns([+ vec![+ MockStreamEvent::tool_call_name_delta("call-1", "internal-1", "echo_tool"),+ MockStreamEvent::tool_call_arguments_delta("call-1", "internal-1", "{\"value\":"),+ MockStreamEvent::tool_call_arguments_delta("call-1", "internal-1", "\"hi\"}"),+ MockStreamEvent::tool_call("call-1", "echo_tool", serde_json::json!({ "value": "hi" }))+ .with_call_id("call-1"),+ MockStreamEvent::final_response_with_default_usage(),+ ],+ vec![+ MockStreamEvent::text("done"),+ MockStreamEvent::final_response_with_default_usage(),+ ],+ ])+}++/// The call opens when it is named, its fragments follow in order, and the+/// finished call and its result preserve the announced provider id.+#[tokio::test]+async fn a_streamed_call_opens_early_and_finishes_under_the_same_id() {+ let mut session = util::test_loop()+ .with_streamed_tool_calls()+ .test_session(+ util::single_tool_set::<EchoTool, ()>(),+ Arc::new(()),+ "test preamble",+ util::usage_ctx(),+ streamed_echo(),+ )+ .await;++ let collected = util::drive(&mut session, "echo hi").await;+ assert!(collected.error.is_none(), "{:?}", collected.error);++ let started: Vec<(&str, &str)> = collected+ .parts+ .iter()+ .filter_map(|part| match part {+ StreamPart::ToolCallStarted(start) => Some((start.id.as_str(), start.name.as_str())),+ _ => None,+ })+ .collect();+ assert_eq!(started, vec![("call-1", "echo_tool")]);++ let fragments: Vec<(&str, &str)> = collected+ .parts+ .iter()+ .filter_map(|part| match part {+ StreamPart::ToolCallArgs { id, delta } => Some((id.as_str(), delta.as_str())),+ _ => None,+ })+ .collect();+ assert_eq!(+ fragments,+ vec![("call-1", "{\"value\":"), ("call-1", "\"hi\"}")]+ );++ let calls = collected.tool_calls();+ assert_eq!(calls.len(), 1);+ assert_eq!(calls[0].id, "call-1");+ assert_eq!(calls[0].json, serde_json::json!({ "value": "hi" }));+ assert!(matches!(+ collected.tool_response("call-1"),+ Some(ToolResponse::Json { json, .. }) if json == &serde_json::json!({ "echo": "hi" })+ ));++ let position = |wanted: fn(&StreamPart) -> bool| {+ collected+ .parts+ .iter()+ .position(wanted)+ .expect("the part was emitted")+ };+ let opened = position(|part| matches!(part, StreamPart::ToolCallStarted(_)));+ let first_fragment = position(|part| matches!(part, StreamPart::ToolCallArgs { .. }));+ let finished = position(|part| matches!(part, StreamPart::ToolCall(_)));+ assert!(opened < first_fragment && first_fragment < finished);+ assert_eq!(collected.content(), "done");+}++/// A loop that did not ask for streamed calls sees only finished ones, under+/// the provider's id, exactly as before.+#[tokio::test]+async fn a_loop_without_streaming_reports_only_finished_calls() {+ let mut session = util::session(+ util::tool_set(AsyncToolCollection::<()>::new().add_tool::<EchoTool, ()>()),+ Arc::new(()),+ streamed_echo(),+ )+ .await;++ let collected = util::drive(&mut session, "echo hi").await;++ assert!(!collected.parts.iter().any(|part| matches!(+ part,+ StreamPart::ToolCallStarted(_) | StreamPart::ToolCallArgs { .. }+ )));+ let calls = collected.tool_calls();+ assert_eq!(calls.len(), 1);+ assert_eq!(calls[0].id, "call-1");+ assert!(collected.tool_response("call-1").is_some());+}diff --git a/crates/agent/src/test/test_hook.rs b/crates/agent/src/test/test_hook.rsindex 433c648edb..9c9357e9a2 100644--- a/crates/agent/src/test/test_hook.rs+++ b/crates/agent/src/test/test_hook.rs@@ -45,6 +45,7 @@ fn inputs(register_loaded: register,tool_loads: ToolLoads::default(),user_tool_finisher,+ stream_tool_calls: false,result_limit: None,}}@@ -395,3 +396,182 @@ async fn results_other_than_pending_never_reach_the_finisher() {Ok(StreamPart::ToolResponse(ToolResponse::Err { .. }))));}++/// A bare bridge that streams tool calls.+fn streaming_bridge() -> (+ StreamBridge,+ tokio::sync::mpsc::UnboundedReceiver<Result<StreamPart, crate::AgentError>>,+) {+ let (register, _registered) = recording_register();+ StreamBridge::channel(+ BridgeInputs {+ stream_tool_calls: true,+ ..inputs(Arc::new(Mutex::new(Vec::new())), register, None)+ },+ Arc::new(vec![]),+ CancellationToken::new(),+ )+}++fn drain(+ rx: &mut tokio::sync::mpsc::UnboundedReceiver<Result<StreamPart, crate::AgentError>>,+) -> Vec<StreamPart> {+ std::iter::from_fn(|| rx.try_recv().ok())+ .map(|part| part.expect("no error parts"))+ .collect()+}++/// Two calls written at once - parallel tool calls stream interleaved - each+/// keep the id they opened under through their finished call and result,+/// even when the provider names the finished call by a `call_id` of its own.+#[tokio::test]+async fn interleaved_streamed_calls_keep_their_announced_ids() {+ let (bridge, mut rx) = streaming_bridge();++ bridge.handle_tool_call_delta("", "internal-a", Some("Search"), "");+ bridge.handle_tool_call_delta("", "internal-b", Some("Read"), "");+ bridge.handle_tool_call_delta("", "internal-b", None, "{\"id\":");+ bridge.handle_tool_call_delta("", "internal-a", None, "{\"query\":\"cats\"}");+ bridge.handle_tool_call_delta("", "internal-b", None, "\"doc\"}");+ bridge.handle_tool_call("Read", Some("call_b"), "internal-b", "{\"id\":\"doc\"}");+ bridge.handle_tool_call(+ "Search",+ Some("call_a"),+ "internal-a",+ "{\"query\":\"cats\"}",+ );+ bridge+ .handle_tool_result(+ "Read",+ Some("call_b"),+ "internal-b",+ "{}",+ &ToolOutput::json(serde_json::json!({"text": "hi"})),+ true,+ )+ .await;++ let parts = drain(&mut rx);+ let summary: Vec<String> = parts+ .iter()+ .map(|part| match part {+ StreamPart::ToolCallStarted(start) => format!("start {} {}", start.id, start.name),+ StreamPart::ToolCallArgs { id, delta } => format!("args {id} {delta}"),+ StreamPart::ToolCall(call) => format!("call {} {}", call.id, call.json),+ StreamPart::ToolResponse(ToolResponse::Json { id, .. }) => format!("result {id}"),+ other => format!("{other:?}"),+ })+ .collect();+ assert_eq!(+ summary,+ vec![+ "start internal-a Search",+ "start internal-b Read",+ "args internal-b {\"id\":",+ "args internal-a {\"query\":\"cats\"}",+ "args internal-b \"doc\"}",+ "call internal-b {\"id\":\"doc\"}",+ "call internal-a {\"query\":\"cats\"}",+ "result internal-b",+ ]+ );+}++/// A finished call with a different correlation id can still match its+/// announcement by the provider id, without guessing from its tool name.+#[tokio::test]+async fn a_finished_call_without_its_delta_id_claims_the_open_announcement() {+ let (bridge, mut rx) = streaming_bridge();++ bridge.handle_tool_call_delta("provider-id", "delta-id", Some("Search"), "{}");+ bridge.handle_tool_call("Search", Some("provider-id"), "final-id", "{}");+ bridge+ .handle_tool_result(+ "Search",+ Some("provider-id"),+ "final-id",+ "{}",+ &ToolOutput::json(serde_json::json!({})),+ true,+ )+ .await;++ let ids: Vec<String> = drain(&mut rx)+ .into_iter()+ .filter_map(|part| match part {+ StreamPart::ToolCall(call) => Some(call.id),+ StreamPart::ToolResponse(ToolResponse::Json { id, .. }) => Some(id),+ _ => None,+ })+ .collect();+ assert_eq!(ids, vec!["provider-id", "provider-id"]);+}++/// Without streaming the deltas emit no progress parts and ids are the provider's.+#[tokio::test]+async fn a_bridge_without_streaming_ignores_deltas() {+ let (bridge, mut rx) = bare_bridge();++ bridge.handle_tool_call_delta("", "internal-a", Some("Search"), "{}");+ bridge.handle_tool_call("Search", Some("call_a"), "internal-a", "{}");++ let parts = drain(&mut rx);+ assert_eq!(parts.len(), 1);+ assert!(matches!(&parts[0], StreamPart::ToolCall(call) if call.id == "call_a"));+}++#[tokio::test]+async fn parallel_calls_of_the_same_tool_keep_provider_ids_when_finishing_out_of_order() {+ let (bridge, mut rx) = streaming_bridge();+ bridge.handle_tool_call_delta("provider-a", "internal-a", Some("Search"), "{}");+ bridge.handle_tool_call_delta("provider-b", "internal-b", Some("Search"), "{}");+ bridge.handle_tool_call("Search", Some("provider-b"), "final-b", "{}");+ bridge.handle_tool_call("Search", Some("provider-a"), "final-a", "{}");+ for (provider, internal) in [("provider-b", "final-b"), ("provider-a", "final-a")] {+ bridge+ .handle_tool_result(+ "Search",+ Some(provider),+ internal,+ "{}",+ &ToolOutput::json(serde_json::json!({})),+ true,+ )+ .await;+ }+ let ids: Vec<String> = drain(&mut rx)+ .into_iter()+ .filter_map(|part| match part {+ StreamPart::ToolCall(call) => Some(call.id),+ StreamPart::ToolResponse(ToolResponse::Json { id, .. }) => Some(id),+ _ => None,+ })+ .collect();+ assert_eq!(+ ids,+ ["provider-b", "provider-a", "provider-b", "provider-a"]+ );+}++#[tokio::test]+async fn a_bridge_without_streaming_still_preserves_anthropic_provider_ids() {+ let (bridge, mut rx) = bare_bridge();+ bridge.handle_tool_call_delta("toolu_call", "internal", Some("Search"), "{}");+ bridge.handle_tool_call("Search", None, "internal", "{}");+ bridge+ .handle_tool_result(+ "Search",+ None,+ "internal",+ "{}",+ &ToolOutput::json(serde_json::json!({})),+ true,+ )+ .await;+ let parts = drain(&mut rx);+ assert!(matches!(&parts[0], StreamPart::ToolCall(call) if call.id == "toolu_call"));+ assert!(+ matches!(&parts[1], StreamPart::ToolResponse(ToolResponse::Json { id, .. }) if id == "toolu_call")+ );+ assert_eq!(parts.len(), 2);+}diff --git a/crates/agent/src/test/test_tool_result_limit.rs b/crates/agent/src/test/test_tool_result_limit.rsindex 0e46508824..5dc8dc349d 100644--- a/crates/agent/src/test/test_tool_result_limit.rs+++ b/crates/agent/src/test/test_tool_result_limit.rs@@ -25,6 +25,7 @@ fn limited_bridge(}),tool_loads: ToolLoads::default(),user_tool_finisher: None,+ stream_tool_calls: false,result_limit: Some(limit.clone()),},Arc::new(vec![]),diff --git a/crates/agent_inmem/AGENTS.md b/crates/agent_inmem/AGENTS.mdindex a3644623f4..c7dbba5f5b 100644--- a/crates/agent_inmem/AGENTS.md+++ b/crates/agent_inmem/AGENTS.md@@ -14,6 +14,8 @@ milliseconds instead of waiting on container provisioning.- `src/domain/engine.rs`: `TurnEngine` port — the seam `rig_engine` fills.- `src/outbound/manager.rs`: `InMemAgentManager` — provisions per-sessionagent tasks, implements the container-manager port the harness expects.+- `src/domain/tool_stream.rs`: throttled, bounded argument previews for pending+ calls; replay ignores previews and retains only calls that began execution.- `src/domain/session.rs` / `src/domain/replay.rs`: in-process session stateand log replay on reconnect.- `src/domain/admission.rs` + `src/outbound/approval_gate.rs`: usage/plandiff --git a/crates/agent_inmem/Cargo.toml b/crates/agent_inmem/Cargo.tomlindex 2902e9c448..28095f064c 100644--- a/crates/agent_inmem/Cargo.toml+++ b/crates/agent_inmem/Cargo.toml@@ -41,6 +41,7 @@ mcp_toolset = { path = "../mcp_toolset" }memory = { path = "../memory" }model-entity = { path = "../model-entity" }non_empty = { path = "../non_empty" }+nom = { workspace = true }prompt = { path = "../prompt" }rmcp = { workspace = true, features = ["client", "transport-streamable-http-client"] }schemars = { workspace = true }diff --git a/crates/agent_inmem/src/domain/agent.rs b/crates/agent_inmem/src/domain/agent.rsindex 5d46135fd4..bc81939908 100644--- a/crates/agent_inmem/src/domain/agent.rs+++ b/crates/agent_inmem/src/domain/agent.rs@@ -15,8 +15,8 @@ use std::time::{Duration, Instant};use crate::domain::model_access::{InMemModelAccess, ModelAccessError, SessionModels};use agent::types::{AssistantMessagePart, ChatMessage};+use agent::{McpInfo, StreamAccumulator, StreamPart, ToolResponse};use agent::{ModelSpeed, ReasoningEffort};-use agent::{StreamAccumulator, StreamPart, ToolResponse};use agent_api_keys::domain::{AgentKeys, ModelProvider, UsableAgentKey};use agent_client_protocol::schema::v1::{AgentCapabilities, AvailableCommand, AvailableCommandInput, AvailableCommandsUpdate,@@ -51,6 +51,7 @@ use crate::domain::engine::{AgentIdentity, AwaitingUser, TurnEngine, TurnPurposeuse crate::domain::mcp::{DynMcpToolConnector, dialable_servers};use crate::domain::model_options::REASONING_EFFORT_CONFIG_ID;use crate::domain::session::{HistoryEntry, SessionStore, UserPrompt, messages_for_turn};+use crate::domain::tool_stream::StreamingCalls;use crate::domain::user_input::{SharedUserInputRequester, UserInputError, UserInputOutcome, UserInputRequest,UserInputRequester,@@ -1032,6 +1033,7 @@ async fn run_turn(});let mut accumulator = StreamAccumulator::new();+ let mut streaming = StreamingCalls::default();let mut failure = None;let mut was_cancelled = false;let mut first_part_seen = false;@@ -1044,8 +1046,10 @@ async fn run_turn(StreamPart::Content(text) | StreamPart::Thinking(text) => {!text.trim().is_empty()}- StreamPart::ToolCall(_) => true,- StreamPart::ToolResponse(_) | StreamPart::Usage(_) => false,+ StreamPart::ToolCall(_) | StreamPart::ToolCallStarted(_) => true,+ StreamPart::ToolCallArgs { .. }+ | StreamPart::ToolResponse(_)+ | StreamPart::Usage(_) => false,};if !first_part_seen && is_output {first_part_seen = true;@@ -1060,7 +1064,7 @@ async fn run_turn(first_text_seen = true;span.record("agent.turn.first_text_ms", elapsed_ms(started));}- if let Some(update) = update_for_part(&part) {+ if let Some(update) = update_for_part(&part, &mut streaming, Instant::now()) {let notification = SessionNotification::new(acp_session_id.clone(), update);if connection.send_notification(notification).is_err() {// Nobody is listening; stop spending tokens.@@ -1096,7 +1100,13 @@ async fn run_turn(}let mut turn_parts = accumulator.into_parts();- for (id, _name) in close_dangling_tool_calls(&mut turn_parts) {+ // A call the turn ended while the model was still writing never ran and+ // is not in the history; its open row closes all the same.+ let unfinished = streaming.into_unfinished();+ let dangling = close_dangling_tool_calls(&mut turn_parts)+ .into_iter()+ .map(|(id, _name)| id);+ for id in unfinished.into_iter().chain(dangling) {let _ = connection.send_notification(SessionNotification::new(acp_session_id.clone(),SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(@@ -1149,7 +1159,11 @@ impl TurnOutput {self.tool_calls += 1;self.replied = false;}- StreamPart::Thinking(_) | StreamPart::ToolResponse(_) | StreamPart::Usage(_) => {}+ StreamPart::ToolCallStarted(_) => self.replied = false,+ StreamPart::ToolCallArgs { .. }+ | StreamPart::Thinking(_)+ | StreamPart::ToolResponse(_)+ | StreamPart::Usage(_) => {}}}@@ -1293,7 +1307,16 @@ fn parse_ask(question: &str) -> (String, Vec<String>) {}/// The `session/update` a stream part renders as, if any.-fn update_for_part(part: &StreamPart) -> Option<SessionUpdate> {+///+/// A call the model is still writing opens `pending` and is peeked at as its+/// arguments stream (see [`crate::domain::tool_stream`]); once finished it+/// moves to `in_progress` with the real arguments. A call that arrives whole+/// opens `in_progress` directly.+fn update_for_part(+ part: &StreamPart,+ streaming: &mut StreamingCalls,+ now: Instant,+) -> Option<SessionUpdate> {match part {StreamPart::Content(text) => Some(SessionUpdate::AgentMessageChunk(ContentChunk::new(ContentBlock::from(text.clone()),@@ -1301,20 +1324,41 @@ fn update_for_part(part: &StreamPart) -> Option<SessionUpdate> {StreamPart::Thinking(text) => Some(SessionUpdate::AgentThoughtChunk(ContentChunk::new(ContentBlock::from(text.clone()),))),- StreamPart::ToolCall(call) => {- let title = call- .mcp- .as_ref()- .and_then(|mcp| mcp.display_name.clone())- .unwrap_or_else(|| call.name.clone());+ StreamPart::ToolCallStarted(start) => {+ streaming.open(&start.id);Some(SessionUpdate::ToolCall(- AcpToolCall::new(call.id.clone(), title)- .kind(tool_kind(&call.name))- .status(ToolCallStatus::InProgress)- .raw_input(call.json.clone())- .meta(tool_call_meta(call)),+ AcpToolCall::new(+ start.id.clone(),+ tool_title(&start.name, start.mcp.as_ref()),+ )+ .kind(tool_kind(&start.name))+ .status(ToolCallStatus::Pending)+ .meta(tool_call_meta(&start.name, start.mcp.as_ref())),))}+ StreamPart::ToolCallArgs { id, delta } => {+ streaming.push_arguments(id, delta, now).map(|peek| {+ SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(+ id.clone(),+ ToolCallUpdateFields::new().raw_input(peek),+ ))+ })+ }+ StreamPart::ToolCall(call) if streaming.finish(&call.id) => {+ Some(SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(+ call.id.clone(),+ ToolCallUpdateFields::new()+ .status(ToolCallStatus::InProgress)+ .raw_input(call.json.clone()),+ )))+ }+ StreamPart::ToolCall(call) => Some(SessionUpdate::ToolCall(+ AcpToolCall::new(call.id.clone(), tool_title(&call.name, call.mcp.as_ref()))+ .kind(tool_kind(&call.name))+ .status(ToolCallStatus::InProgress)+ .raw_input(call.json.clone())+ .meta(tool_call_meta(&call.name, call.mcp.as_ref())),+ )),StreamPart::ToolResponse(ToolResponse::Json { id, json, .. }) => {Some(SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(id.clone(),@@ -1340,14 +1384,14 @@ fn update_for_part(part: &StreamPart) -> Option<SessionUpdate> {/// name rather than guess from the title: `macro.toolName` (an MCP tool as/// `mcp__<server>__<tool>`, the convention Claude Code set) and/// `macro.subagent` on a delegation.-fn tool_call_meta(call: &agent::ToolCall) -> Meta {- let tool_name = match &call.mcp {+fn tool_call_meta(name: &str, mcp: Option<&McpInfo>) -> Meta {+ let tool_name = match mcp {Some(mcp) => format!("mcp__{}__{}", mcp.service, mcp.tool_name),- None => call.name.clone(),+ None => name.to_owned(),};let mut ours = serde_json::Map::new();ours.insert("toolName".to_owned(), serde_json::Value::String(tool_name));- if call.mcp.is_none() && call.name == SUBAGENT_TOOL {+ if mcp.is_none() && name == SUBAGENT_TOOL {ours.insert("subagent".to_owned(), serde_json::Value::Bool(true));}let mut meta = Meta::new();@@ -1355,6 +1399,12 @@ fn tool_call_meta(call: &agent::ToolCall) -> Meta {meta}+/// A tool call's title: an MCP tool's display name, else the tool's name.+fn tool_title(name: &str, mcp: Option<&McpInfo>) -> String {+ mcp.and_then(|mcp| mcp.display_name.clone())+ .unwrap_or_else(|| name.to_owned())+}+/// A coarse [`ToolKind`] for a Macro tool name, for client iconography only.fn tool_kind(name: &str) -> ToolKind {let name = name.to_ascii_lowercase();diff --git a/crates/agent_inmem/src/domain/agent/test.rs b/crates/agent_inmem/src/domain/agent/test.rsindex 1e1d66c2b4..d049ecfd8e 100644--- a/crates/agent_inmem/src/domain/agent/test.rs+++ b/crates/agent_inmem/src/domain/agent/test.rs@@ -837,6 +837,155 @@ async fn tool_calls_are_stamped_with_their_names_and_subagent_flag() {);}+/// The tool frames of a turn, one line each: what a reader of the log sees+/// happen to its calls.+fn tool_frames(notifications: &[SessionNotification]) -> Vec<String> {+ let status = |status: Option<ToolCallStatus>| {+ status.map_or_else(+ || "-".to_owned(),+ |status| {+ serde_json::to_value(status)+ .expect("a status serializes")+ .to_string()+ },+ )+ };+ let input = |input: Option<&serde_json::Value>| {+ input.map_or_else(|| "-".to_owned(), serde_json::Value::to_string)+ };+ notifications+ .iter()+ .filter_map(|notification| match ¬ification.update {+ SessionUpdate::ToolCall(call) => Some(format!(+ "open {} {} {} {}",+ call.tool_call_id.0,+ call.title,+ status(Some(call.status)),+ input(call.raw_input.as_ref())+ )),+ SessionUpdate::ToolCallUpdate(update) => Some(format!(+ "update {} {} {}",+ update.tool_call_id.0,+ status(update.fields.status),+ input(update.fields.raw_input.as_ref())+ )),+ _ => None,+ })+ .collect()+}++/// A call the model is still writing opens `pending` at once, is peeked at+/// as its arguments stream - at most once per window, so a burst of+/// fragments is one peek - and starts under the same id once finished.+#[tokio::test]+async fn a_streamed_tool_call_opens_pending_and_is_peeked_at_while_written() {+ let engine = Arc::new(ScriptedEngine::new(vec![+ StreamPart::ToolCallStarted(agent::ToolCallStart {+ id: "call-1".into(),+ name: "NameSearch".into(),+ mcp: None,+ }),+ StreamPart::ToolCallArgs {+ id: "call-1".into(),+ delta: "{\"query\":\"road".into(),+ },+ StreamPart::ToolCallArgs {+ id: "call-1".into(),+ delta: "map\"}".into(),+ },+ StreamPart::ToolCall(agent::ToolCall {+ id: "call-1".into(),+ name: "NameSearch".into(),+ json: serde_json::json!({"query": "roadmap"}),+ mcp: None,+ }),+ StreamPart::ToolResponse(agent::ToolResponse::Json {+ id: "call-1".into(),+ json: serde_json::json!({"hits": 1}),+ name: "NameSearch".into(),+ }),+ ]));++ let (notifications, _, _) = with_agent(Arc::clone(&engine), async |connection, session| {+ connection+ .send_request(text_prompt(&session, "find the roadmap"))+ .block_task()+ .await+ .expect("the prompt should complete")+ })+ .await;++ assert_eq!(+ tool_frames(¬ifications),+ vec![+ r#"open call-1 NameSearch "pending" -"#,+ r#"update call-1 - {"query":"road"}"#,+ r#"update call-1 "in_progress" {"query":"roadmap"}"#,+ r#"update call-1 "completed" -"#,+ ]+ );+ let open = notifications+ .iter()+ .find_map(|notification| match ¬ification.update {+ SessionUpdate::ToolCall(call) => Some(call),+ _ => None,+ })+ .expect("the call opened");+ assert_eq!(+ open.meta.clone().map(serde_json::Value::Object),+ Some(serde_json::json!({"macro": {"toolName": "NameSearch"}})),+ "the fold reads the tool's name from the opening frame"+ );+}++/// A turn that ends while the model is still writing a call closes the+/// call's row as failed, and the call - which never ran - stays out of the+/// history the next turn is built from.+#[tokio::test]+async fn a_call_the_turn_ends_while_writing_closes_and_stays_out_of_history() {+ let engine = Arc::new(ScriptedEngine::new(vec![+ StreamPart::Content("On it.".into()),+ StreamPart::ToolCallStarted(agent::ToolCallStart {+ id: "call-1".into(),+ name: "CreateDocument".into(),+ mcp: None,+ }),+ StreamPart::ToolCallArgs {+ id: "call-1".into(),+ delta: "{\"title\":\"Pl".into(),+ },+ ]));++ let (notifications, _, _) = with_agent(Arc::clone(&engine), async |connection, session| {+ for prompt in ["write it up", "again"] {+ connection+ .send_request(text_prompt(&session, prompt))+ .block_task()+ .await+ .expect("the prompt should complete");+ }+ })+ .await;++ let frames = tool_frames(¬ifications);+ assert_eq!(+ frames[..3],+ [+ r#"open call-1 CreateDocument "pending" -"#.to_owned(),+ r#"update call-1 - {"title":"Pl"}"#.to_owned(),+ r#"update call-1 "failed" -"#.to_owned(),+ ]+ );+ assert_eq!(+ engine.requests()[1].messages,+ vec![+ "write it up".to_owned(),+ "On it.".to_owned(),+ "again".to_owned()+ ]+ );+}+#[tokio::test]async fn turns_accumulate_history_and_send_the_model() {let engine = Arc::new(ScriptedEngine::new(vec![StreamPart::Content("ok".into())]));diff --git a/crates/agent_inmem/src/domain/mod.rs b/crates/agent_inmem/src/domain/mod.rsindex 48588f9a1e..1d6b4f71de 100644--- a/crates/agent_inmem/src/domain/mod.rs+++ b/crates/agent_inmem/src/domain/mod.rs@@ -17,4 +17,5 @@ pub mod session;/// Whether one of Macro's own tools may run in this turn.pub mod tool_gate;pub mod tool_mode;+pub mod tool_stream;pub mod user_input;diff --git a/crates/agent_inmem/src/domain/replay.rs b/crates/agent_inmem/src/domain/replay.rsindex 98be3f68f4..3ab07ca4cc 100644--- a/crates/agent_inmem/src/domain/replay.rs+++ b/crates/agent_inmem/src/domain/replay.rs@@ -27,7 +27,7 @@ use agent_client_protocol::{JsonRpcMessage, RawJsonRpcMessage, RawJsonRpcParams}use agent_runtime_protocol::domain::schema::v0::{ToRuntimeMessage, ToServerMessage};use agent_session::domain::model::Message;use futures::future::BoxFuture;-use std::collections::HashSet;+use std::collections::{HashMap, HashSet};use crate::domain::agent::{META_NAMESPACE, close_dangling_tool_calls};use crate::domain::model_options::REASONING_EFFORT_CONFIG_ID;@@ -76,6 +76,7 @@ struct OpenTurn {pub fn replay_history(frames: impl IntoIterator<Item = Message>) -> Vec<HistoryEntry> {let mut history = Vec::new();let mut open: Option<OpenTurn> = None;+ let mut writing = WritingCalls::default();for frame in frames {match frame {@@ -92,6 +93,7 @@ pub fn replay_history(frames: impl IntoIterator<Item = Message>) -> Vec<HistoryEcontinue;};close_turn(&mut history, &mut open);+ writing = WritingCalls::default();open = Some(OpenTurn {id: request.id.clone(),prompt: UserPrompt::from_blocks(&prompt.prompt),@@ -147,7 +149,7 @@ pub fn replay_history(frames: impl IntoIterator<Item = Message>) -> Vec<HistoryE// An update outside any turn (status chatter) is// presentation, not conversation.if let Some(turn) = open.as_mut() {- apply_update(&mut turn.parts, notification.update);+ apply_update(&mut turn.parts, &mut writing, notification.update);}}_ => {}@@ -229,9 +231,54 @@ fn replay_config<T: Default>(frames: &[Message], read: impl Fn(&[SessionConfigOpcurrent}+/// Calls the turn opened `pending` - the model was still writing them - by+/// id, with the title they opened under. The live agent only adds a call to+/// its history once it is finished, so replay holds these back until the+/// update that starts them, and a call the turn ended in the middle of never+/// enters the history at all.+#[derive(Default)]+struct WritingCalls(HashMap<String, String>);+/// Fold one `session/update` back into the open turn's parts.-fn apply_update(parts: &mut Vec<AssistantMessagePart>, update: SessionUpdate) {+fn apply_update(+ parts: &mut Vec<AssistantMessagePart>,+ writing: &mut WritingCalls,+ update: SessionUpdate,+) {match update {+ SessionUpdate::ToolCall(call) if call.status == ToolCallStatus::Pending => {+ writing.0.insert(+ call.tool_call_id.0.to_string(),+ call.meta+ .as_ref()+ .and_then(|meta| meta.get(META_NAMESPACE))+ .and_then(|meta| meta.get("toolName"))+ .and_then(serde_json::Value::as_str)+ .unwrap_or(&call.title)+ .to_owned(),+ );+ }+ SessionUpdate::ToolCallUpdate(update)+ if writing.0.contains_key(update.tool_call_id.0.as_ref()) =>+ {+ let id = update.tool_call_id.0.to_string();+ match update.fields.status {+ // A peek at the arguments so far.+ Some(ToolCallStatus::Pending) | None => {}+ Some(ToolCallStatus::InProgress) => {+ let name = writing.0.remove(&id).unwrap_or_default();+ parts.push(AssistantMessagePart::ToolCall {+ name,+ json: update.fields.raw_input.unwrap_or(serde_json::Value::Null),+ id,+ });+ }+ // Closed before the model finished writing it: it never ran.+ Some(_) => {+ writing.0.remove(&id);+ }+ }+ }SessionUpdate::AgentMessageChunk(chunk) => {let ContentBlock::Text(text) = chunk.content else {return;diff --git a/crates/agent_inmem/src/domain/replay/test.rs b/crates/agent_inmem/src/domain/replay/test.rsindex 0407e847f3..1f148e3f28 100644--- a/crates/agent_inmem/src/domain/replay/test.rs+++ b/crates/agent_inmem/src/domain/replay/test.rs@@ -594,6 +594,85 @@ fn only_confirmed_effort_replays_and_removal_resets_it() {assert_eq!(replay_reasoning_effort(&frames), ReasoningEffort::Default);}+/// A call streamed as the model wrote it replays as the finished call: the+/// peeks at its arguments are presentation, and the update that started it+/// carries the arguments the model actually sent.+#[test]+fn a_streamed_call_replays_with_its_finished_arguments() {+ let history = replay_history(vec![+ prompt_frame("find the roadmap"),+ update_frame(SessionUpdate::ToolCall(+ AcpToolCall::new("call-1", "NameSearch").status(ToolCallStatus::Pending),+ )),+ update_frame(SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(+ "call-1",+ ToolCallUpdateFields::new().raw_input(serde_json::json!({"query": "road"})),+ ))),+ update_frame(SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(+ "call-1",+ ToolCallUpdateFields::new()+ .status(ToolCallStatus::InProgress)+ .raw_input(serde_json::json!({"query": "roadmap"})),+ ))),+ update_frame(SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(+ "call-1",+ ToolCallUpdateFields::new()+ .status(ToolCallStatus::Completed)+ .raw_output(serde_json::json!({"hits": 1})),+ ))),+ ]);++ let [HistoryEntry::User { .. }, HistoryEntry::Assistant(parts)] = history.as_slice() else {+ panic!("one turn should replay, got {history:#?}");+ };+ assert_eq!(+ parts.as_slice(),+ [+ AssistantMessagePart::ToolCall {+ name: "NameSearch".to_owned(),+ json: serde_json::json!({"query": "roadmap"}),+ id: "call-1".to_owned(),+ },+ AssistantMessagePart::ToolCallResponseJson {+ name: "NameSearch".to_owned(),+ json: serde_json::json!({"hits": 1}),+ id: "call-1".to_owned(),+ },+ ]+ );+}++/// A call the turn stopped in the middle of writing never ran, and the live+/// agent never put it in the history; replay leaves it out too.+#[test]+fn a_call_stopped_while_being_written_does_not_replay() {+ let history = replay_history(vec![+ prompt_frame("write it up"),+ update_frame(message_chunk("On it.")),+ update_frame(SessionUpdate::ToolCall(+ AcpToolCall::new("call-1", "CreateDocument").status(ToolCallStatus::Pending),+ )),+ update_frame(SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(+ "call-1",+ ToolCallUpdateFields::new().raw_input(serde_json::json!({"title": "Plan"})),+ ))),+ update_frame(SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(+ "call-1",+ ToolCallUpdateFields::new().status(ToolCallStatus::Failed),+ ))),+ ]);++ let [HistoryEntry::User { .. }, HistoryEntry::Assistant(parts)] = history.as_slice() else {+ panic!("one turn should replay, got {history:#?}");+ };+ assert_eq!(+ parts.as_slice(),+ [AssistantMessagePart::Text {+ text: "On it.".to_owned()+ }]+ );+}+#[test]fn speed_replays_only_from_confirmed_configuration() {use agent::ModelSpeed;@@ -613,3 +692,39 @@ fn speed_replays_only_from_confirmed_configuration() {frames.push(config_response(serde_json::json!({"configOptions":[]})));assert_eq!(replay_speed(&frames), ModelSpeed::Standard);}++#[test]+fn streamed_mcp_call_replays_its_tool_name_metadata_instead_of_its_label() {+ let history = replay_history(vec![+ prompt_frame("find a file"),+ update_frame(SessionUpdate::ToolCall(+ AcpToolCall::new("call-1", "Search files")+ .status(ToolCallStatus::Pending)+ .meta(serde_json::Map::from_iter([(+ "macro".to_owned(),+ serde_json::json!({"toolName": "mcp__files__search"}),+ )])),+ )),+ update_frame(SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(+ "call-1",+ ToolCallUpdateFields::new()+ .status(ToolCallStatus::InProgress)+ .raw_input(serde_json::json!({"query":"plan"})),+ ))),+ update_frame(SessionUpdate::ToolCallUpdate(ToolCallUpdate::new(+ "call-1",+ ToolCallUpdateFields::new()+ .status(ToolCallStatus::Completed)+ .raw_output(serde_json::json!({})),+ ))),+ ]);+ let [HistoryEntry::User { .. }, HistoryEntry::Assistant(parts)] = history.as_slice() else {+ panic!("one turn should replay");+ };+ assert!(+ matches!(&parts[0], AssistantMessagePart::ToolCall { name, json, .. } if name == "mcp__files__search" && json == &serde_json::json!({"query":"plan"}))+ );+ assert!(+ matches!(&parts[1], AssistantMessagePart::ToolCallResponseJson { name, .. } if name == "mcp__files__search")+ );+}diff --git a/crates/agent_inmem/src/domain/tool_stream.rs b/crates/agent_inmem/src/domain/tool_stream.rsnew file mode 100644index 0000000000..f2e80a4f58--- /dev/null+++ b/crates/agent_inmem/src/domain/tool_stream.rs@@ -0,0 +1,194 @@+//! Tool calls while the model is still writing them.+//!+//! A call opens `pending` the moment the model names it, so it takes its row+//! in the transcript's live tool window instead of appearing whole seconds+//! later. While its arguments stream, the reader gets a peek at them: the+//! prefix received so far, closed into a JSON object, sent as the call's+//! `rawInput`. Peeks go out at most once per [`PEEK_WINDOW`] - every frame is+//! logged, and a long argument can arrive in thousands of fragments - and+//! stop past [`PEEK_LIMIT`], where each one would carry a large prefix again.+//! The finished call replaces the last peek with the real arguments.++use std::collections::HashMap;+use std::time::{Duration, Instant};++use nom::branch::alt;+use nom::bytes::complete::take_while1;+use nom::character::complete::{anychar, char, one_of};+use nom::combinator::{consumed, iterator, map, opt, recognize};+use nom::multi::many0;+use nom::{IResult, Parser};+use serde_json::{Map, Value};++#[cfg(test)]+mod test;++/// The shortest gap between two peeks at one call's arguments.+pub const PEEK_WINDOW: Duration = Duration::from_millis(250);++/// Arguments longer than this are no longer peeked at; the call shows its+/// last peek until it finishes.+pub const PEEK_LIMIT: usize = 16 * 1024;++/// How many of the latest `,` boundaries a peek falls back to when the+/// prefix does not close into JSON as it stands.+const PEEK_CUTS: usize = 4;++/// The turn's calls that have opened and not finished.+#[derive(Default)]+pub struct StreamingCalls {+ calls: HashMap<String, StreamingCall>,+ /// Open order, so calls left open at the end of a turn close in order.+ order: Vec<String>,+}++#[derive(Default)]+struct StreamingCall {+ arguments: String,+ over_limit: bool,+ last_peek: Option<Instant>,+ peeked: Option<Value>,+}++impl StreamingCalls {+ /// The model named call `id`; its arguments follow.+ pub fn open(&mut self, id: &str) {+ if let std::collections::hash_map::Entry::Vacant(entry) = self.calls.entry(id.to_owned()) {+ entry.insert(StreamingCall::default());+ self.order.push(id.to_owned());+ }+ }++ /// Take in a fragment of call `id`'s arguments; returns the peek to send,+ /// if one is due and says something the last one did not.+ pub fn push_arguments(&mut self, id: &str, delta: &str, now: Instant) -> Option<Value> {+ let call = self.calls.get_mut(id)?;+ if call.over_limit {+ return None;+ }+ if delta.len() > PEEK_LIMIT.saturating_sub(call.arguments.len()) {+ call.over_limit = true;+ call.arguments.clear();+ return None;+ }+ call.arguments.push_str(delta);+ if call+ .last_peek+ .is_some_and(|at| now.saturating_duration_since(at) < PEEK_WINDOW)+ {+ return None;+ }+ call.last_peek = Some(now);+ let peek = peek_arguments(&call.arguments).filter(|peek| !peek.is_empty())?;+ let peek = Value::Object(peek);+ if call.peeked.as_ref() == Some(&peek) {+ return None;+ }+ call.peeked = Some(peek.clone());+ Some(peek)+ }++ /// Call `id` finished; whether it had opened early.+ pub fn finish(&mut self, id: &str) -> bool {+ self.order.retain(|open| open != id);+ self.calls.remove(id).is_some()+ }++ /// The calls that opened and never finished, oldest first.+ pub fn into_unfinished(self) -> Vec<String> {+ self.order+ }+}++/// The object a prefix of streamed JSON arguments says so far.+///+/// Open strings, arrays and objects are closed where the prefix stops; a+/// prefix that does not close into JSON that way - it ends inside a key, or+/// after one - falls back to its last complete member. `None` when nothing+/// complete has arrived yet.+#[must_use]+pub fn peek_arguments(prefix: &str) -> Option<Map<String, Value>> {+ let mut closers: Vec<char> = Vec::new();+ let mut in_string = false;+ let mut escaped = false;+ let mut cuts: Vec<(usize, Vec<char>)> = Vec::new();+ let mut tokens = iterator(prefix, consumed(json_token));+ let mut at = 0;+ for (raw, token) in &mut tokens {+ match token {+ JsonToken::String {+ open,+ trailing_escape,+ } => {+ in_string = open;+ escaped = trailing_escape;+ }+ JsonToken::Symbol('{') => closers.push('}'),+ JsonToken::Symbol('[') => closers.push(']'),+ JsonToken::Symbol('}' | ']') => {+ closers.pop();+ }+ JsonToken::Symbol(',') => {+ if cuts.len() == PEEK_CUTS {+ cuts.remove(0);+ }+ cuts.push((at, closers.clone()));+ }+ _ => {}+ }+ at += raw.len();+ }++ let mut whole = prefix.to_owned();+ if in_string {+ if escaped {+ whole.pop();+ }+ whole.push('"');+ }+ std::iter::once((whole, closers))+ .chain(+ cuts.into_iter()+ .rev()+ .map(|(at, closers)| (prefix[..at].to_owned(), closers)),+ )+ .find_map(|(mut attempt, closers)| {+ attempt.extend(closers.iter().rev());+ match serde_json::from_str(&attempt) {+ Ok(Value::Object(object)) => Some(object),+ _ => None,+ }+ })+}++/// A JSON token, allowing the final string to be incomplete.+enum JsonToken {+ String { open: bool, trailing_escape: bool },+ Symbol(char),+ Other,+}++fn json_token(input: &str) -> IResult<&str, JsonToken> {+ alt((+ map(+ (+ char('"'),+ many0(alt((+ recognize((char('\\'), anychar)),+ take_while1(|ch| ch != '"' && ch != '\\'),+ ))),+ opt(char('\\')),+ opt(char('"')),+ ),+ |(_, _, escape, close)| JsonToken::String {+ open: close.is_none(),+ trailing_escape: escape.is_some(),+ },+ ),+ map(one_of("{}[],"), JsonToken::Symbol),+ map(take_while1(|ch| !"\"{}[],".contains(ch)), |_| {+ JsonToken::Other+ }),+ ))+ .parse(input)+}diff --git a/crates/agent_inmem/src/domain/tool_stream/test.rs b/crates/agent_inmem/src/domain/tool_stream/test.rsnew file mode 100644index 0000000000..e9fa5df8e2--- /dev/null+++ b/crates/agent_inmem/src/domain/tool_stream/test.rs@@ -0,0 +1,168 @@+use std::time::{Duration, Instant};++use serde_json::json;++use super::{PEEK_LIMIT, PEEK_WINDOW, StreamingCalls, peek_arguments};++fn peek(prefix: &str) -> Option<serde_json::Value> {+ peek_arguments(prefix).map(serde_json::Value::Object)+}++#[test]+fn an_open_string_reads_as_far_as_it_has_arrived() {+ assert_eq!(+ peek(r#"{"to":"[email protected]","body":"Hi Al"#),+ Some(json!({ "to": "[email protected]", "body": "Hi Al" }))+ );+}++#[test]+fn open_containers_close_where_the_prefix_stops() {+ assert_eq!(+ peek(r#"{"ids":["a","b"#),+ Some(json!({ "ids": ["a", "b"] }))+ );+ assert_eq!(+ peek(r#"{"event":{"title":"Standup","attendees":[{"email":"[email protected]""#),+ Some(json!({ "event": { "title": "Standup", "attendees": [{ "email": "[email protected]" }] } }))+ );+}++#[test]+fn a_prefix_ending_in_a_key_falls_back_to_its_last_member() {+ assert_eq!(+ peek(r#"{"query":"cats","lim"#),+ Some(json!({ "query": "cats" }))+ );+ assert_eq!(+ peek(r#"{"query":"cats","limit":"#),+ Some(json!({ "query": "cats" }))+ );+ assert_eq!(+ peek(r#"{"query":"cats","flag":tr"#),+ Some(json!({ "query": "cats" }))+ );+}++#[test]+fn an_escape_cut_in_half_is_dropped() {+ assert_eq!(+ peek(r#"{"text":"say \"hi\"#),+ Some(json!({ "text": "say \"hi" }))+ );+ assert_eq!(peek(r#"{"text":"line\"#), Some(json!({ "text": "line" })));+}++#[test]+fn nothing_complete_is_no_peek() {+ assert_eq!(peek(""), None);+ assert_eq!(peek("{"), Some(json!({})));+ assert_eq!(peek(r#"{"que"#), None);+ assert_eq!(peek(r#"{"query":"#), None);+}++#[test]+fn whole_arguments_read_as_themselves() {+ assert_eq!(+ peek(r#"{"query":"cats","limit":3}"#),+ Some(json!({ "query": "cats", "limit": 3 }))+ );+}++#[test]+fn peeks_go_out_at_most_once_per_window() {+ let start = Instant::now();+ let mut calls = StreamingCalls::default();+ calls.open("call");++ assert_eq!(+ calls.push_arguments("call", r#"{"body":"a"#, start),+ Some(json!({ "body": "a" }))+ );+ assert_eq!(+ calls.push_arguments("call", "b", start + PEEK_WINDOW / 2),+ None,+ "inside the window"+ );+ assert_eq!(+ calls.push_arguments("call", "c", start + PEEK_WINDOW),+ Some(json!({ "body": "abc" })),+ "the next peek carries everything since"+ );+}++#[test]+fn a_peek_that_says_nothing_new_is_not_sent() {+ let start = Instant::now();+ let mut calls = StreamingCalls::default();+ calls.open("call");++ assert!(calls.push_arguments("call", r#"{"a":1,"#, start).is_some());+ assert_eq!(+ calls.push_arguments("call", r#""b"#, start + PEEK_WINDOW),+ None,+ "still only `a`"+ );+}++#[test]+fn long_arguments_stop_being_peeked() {+ let start = Instant::now();+ let mut calls = StreamingCalls::default();+ calls.open("call");++ assert!(+ calls+ .push_arguments("call", r#"{"body":""#, start)+ .is_some()+ );+ let long = "x".repeat(PEEK_LIMIT);+ assert_eq!(+ calls.push_arguments("call", &long, start + Duration::from_secs(1)),+ None+ );+ assert_eq!(+ calls.push_arguments("call", "y", start + Duration::from_secs(2)),+ None+ );+ assert!(calls.calls["call"].arguments.len() <= PEEK_LIMIT);+}++#[test]+fn fragments_of_an_unopened_call_are_ignored() {+ let mut calls = StreamingCalls::default();+ assert_eq!(+ calls.push_arguments("stray", r#"{"a":1}"#, Instant::now()),+ None+ );+}++#[test]+fn calls_left_open_are_reported_in_open_order() {+ let mut calls = StreamingCalls::default();+ calls.open("first");+ calls.open("second");+ calls.open("third");++ assert!(calls.finish("second"));+ assert!(!calls.finish("never-opened"));+ assert_eq!(calls.into_unfinished(), vec!["first", "third"]);+}++#[test]+fn invalid_prefixes_are_throttled_and_reopening_preserves_arguments() {+ let start = Instant::now();+ let mut calls = StreamingCalls::default();+ calls.open("call");+ assert_eq!(calls.push_arguments("call", "{", start), None);+ calls.open("call");+ assert_eq!(+ calls.push_arguments("call", r#""body":"a"#, start + PEEK_WINDOW / 2),+ None+ );+ assert_eq!(+ calls.push_arguments("call", "b", start + PEEK_WINDOW),+ Some(json!({ "body": "ab" }))+ );+ assert_eq!(calls.into_unfinished(), ["call"]);+}diff --git a/crates/agent_inmem/src/rig_engine.rs b/crates/agent_inmem/src/rig_engine.rsindex 16ddffe054..4797a98c1e 100644--- a/crates/agent_inmem/src/rig_engine.rs+++ b/crates/agent_inmem/src/rig_engine.rs@@ -606,11 +606,14 @@ async fn drive_turn(// reach the session actor as ACP frames, and the actor projects those onto// `invoke_agent` / `execute_tool` spans for every harness alike. Enriching// rig's spans too would report each turn twice.+ // Calls stream as the model writes them, so the transcript shows a call+ // from the moment it is named rather than once its arguments are whole.let mut agent_loop = AgentLoop::new(base_context.recorder.clone()).with_model(&model).with_reasoning_effort(reasoning_effort).with_speed(speed)?.with_genai_telemetry(false)+ .with_streamed_tool_calls().with_tool_result_limit(tool_results.clone());// Only the loop's own model calls run on the agent's key, so only they are// recorded against it; tools keep `usage_ctx`, since they call models ondiff --git a/docs/agent-guide/ai-chat.md b/docs/agent-guide/ai-chat.mdindex 08c58a0924..8e5bbf8191 100644--- a/docs/agent-guide/ai-chat.md+++ b/docs/agent-guide/ai-chat.md@@ -126,6 +126,13 @@ read project activity. Backend tool names use `Initiative`. These operate on thenative Projects views in Tasks. Agents set or clear a task's project with`SetEntityProperty` on the task's `Project` property.+In-memory agent tool rows appear as pending as soon as the model names the+call. Partial argument previews update at most every 250 ms and stop after+16 KiB; execution replaces the preview with the full input. Cancelling while+the model writes arguments closes the row as failed without executing it.+Verify with a long document creation request, cancel once mid-write, then+reconnect after a completed call and confirm its final input and result replay.+Each completed tool row has an expandable result toggle, including empty resultsand per-task failures. Project chips open the native project. Shift-click opensanother split. **Result data** reveals the complete returned response. Successful