Python
feat(sdk): expose safe attempt streams and provider diagnostics
Rohit P1dcaeaa4b93d
diff --git a/docs/harness/guides/providers.md b/docs/harness/guides/providers.mdindex 22839167..83e3539e 100644--- a/docs/harness/guides/providers.md+++ b/docs/harness/guides/providers.md@@ -27,6 +27,9 @@ and [guardrails guide](guardrails.md).A compatible endpoint does not guarantee equivalent tool or image behavior;qualify the specific model/configuration used by your application.+See [Streaming and retries](streaming.md) for provider reasoning display, explicit+Anthropic thinking settings and structured failure diagnostics.+## Streaming token usageChat Completions providers request `stream_options: { include_usage: true }` bydiff --git a/docs/harness/guides/streaming.md b/docs/harness/guides/streaming.mdnew file mode 100644index 00000000..c247407e--- /dev/null+++ b/docs/harness/guides/streaming.md@@ -0,0 +1,115 @@+# Streaming, retries and provider diagnostics++An agent turn can make several model calls, and a failed call can be retried. Build+the UI around attempt IDs rather than concatenating every text delta into one+permanent message. A successful model attempt is still provisional until the+turn completes; it can request tools or be followed by another model call.++```typescript+import { AttemptOutput } from "@openworkerai/harness-sdk";++const display = new AttemptOutput(); // Create one projection for this run.+const run = await session.send("Explain the trade-offs", {+ output: { reasoning: true, bufferAttempts: false },+});+for await (const item of run.events()) {+ display.apply(item);+ // Render as escaped text; never inject model text as HTML.+ answerElement.textContent = display.text;+ thinkingElement.textContent = display.reasoning;+}+const result = await run.result();+if (result.status !== "completed") display.interrupt();+// Save result.finalOutput as the final answer, not the accumulated stream.+```++The helper removes failed candidate text and reasoning as soon as the attempt+fails. Replayed event IDs are ignored. It limits retained attempts and characters+for display; these limits do not truncate the durable transcript. After cancelling+a run, call `interrupt()` to discard any remaining provisional output. To avoid+showing failed candidates at all, set `bufferAttempts: true`: the runtime releases+display deltas only after that provider attempt completes successfully. This+trades incremental output for a quieter UI during retries.++Rust exposes the same options through `TurnOptions.output` / `OutputOptions` and+typed `RunEvent::AttemptStarted`, `AttemptFinished`, `TextDelta`, `ReasoningDelta`,+`ModelActivity` and `ProviderError`. Keep separate buffers per+`(session_id, turn_id, attempt_id)` and discard a failed or interrupted buffer.+TypeScript summaries use `attemptId`; the raw journal retains `attempt_id`.++| Event | Meaning |+| --- | --- |+| `model_attempt_started` | A provider attempt began. |+| `assistant_delta` | Candidate answer text for that attempt. |+| `reasoning_delta` | Opt-in provider-designated display reasoning, separate from the answer. |+| `model_activity` | The request is still pending; includes elapsed milliseconds, at most once every five seconds. |+| `model_attempt_finished` | That attempt completed or failed. |+| `provider_error` | Safe structured diagnostics for a failed attempt. |++An activity event is a liveness signal, not a claim that the model has completed+a particular task. It does not extend the request timeout or add provider calls.++## Enable reasoning display++Reasoning display is off by default. `output.reasoning: true` exposes only the+provider's supported display text. It never decodes encrypted reasoning, exposes+signatures or reveals redacted thinking. Display is bounded to 64 KiB per attempt;+`reasoning_display_limited` records when that display bound is reached.++- **OpenAI Responses:** requests `reasoning.summary: "auto"` and forwards+ `response.reasoning_summary_text.delta`. Use a model that supports summaries.+- **Anthropic:** forwards `thinking_delta`. Explicitly configure thinking+ generation for the selected model; display alone does not enable generation.+- **Chat Completions:** forwards `reasoning_content` or `reasoning` text when the+ selected endpoint/model supplies it. The harness does not guess a vendor's+ generation settings or extract hidden reasoning from answer text.++For Anthropic, choose a mode supported by your exact model:++```typescript+await harness.setModelProvider({+ name: "Claude", api: "anthropic",+ credentials: { apiKeyEnv: "ANTHROPIC_API_KEY" },+ thinking: { type: "adaptive" },+ // For a model using an explicit thinking budget instead:+ // thinking: { type: "enabled", budgetTokens: 2048 },+});+```++Rust uses `ModelProvider::anthropic("Claude").thinking(AnthropicThinking::Adaptive)`+or `AnthropicThinking::Enabled { budget_tokens: 2048 }`. An enabled budget must+be at least 1,024 tokens and smaller than the current call's output limit. These+settings apply to main agent calls, not short connection probes, compactors or+reviewers. Omitted settings preserve an existing choice; explicitly use+`{ type: "disabled" }` to disable thinking generation. Provider configuration+persists across restart and credential rotation.++Output guardrails and blocking output/tool hooks suppress candidate reasoning as+well as candidate answer text, including for agents without tools. Display+opt-in does not bypass inspection or allow withheld content into telemetry.++See the provider contracts for [OpenAI summaries](https://developers.openai.com/api/docs/guides/reasoning#reasoning-summaries),+[Anthropic thinking streams](https://platform.claude.com/docs/en/build-with-claude/streaming#thinking-delta)+and [Together reasoning fields](https://docs.together.ai/docs/inference/chat/reasoning).++## Diagnose provider failures++Failed run results expose `providerError` in TypeScript or `provider_error` in+Rust. The diagnostic contains a category, retryability, and available HTTP status,+API, request ID and fixed parser/allowlisted upstream code. Probe failures expose+the same data through `HarnessError.context.providerError` or Rust+`Error::Provider`. An `invalid_tool_call_index` code, for example, identifies a+malformed streamed tool-call index without logging the tool arguments.++```typescript+const result = await run.result();+if (result.providerError) {+ const { category, httpStatus, requestId, code } = result.providerError;+ console.error({ category, httpStatus, requestId, code });+}+```++Upstream response bodies, prompts, keys and arbitrary error messages are not+included in these diagnostics. Use the request ID with the provider when more+detail is needed. A retryable provider error does not authorize an application to+replay tool actions: retain the SDK's run reference and reconcile unknown outcomes.diff --git a/harness/harness-sdk/src/agents/session.rs b/harness/harness-sdk/src/agents/session.rsindex 2c9c4def..5422ed3d 100644--- a/harness/harness-sdk/src/agents/session.rs+++ b/harness/harness-sdk/src/agents/session.rs@@ -123,6 +123,7 @@ pub struct RunResult {pub cursor: i64,/// Team board at the end of the run, for sessions with subagents.pub board: Option<Value>,+ pub provider_error: Option<openworker_runtime::provider::Diagnostic>,}impl RunResult {fn from_wire(v: &Value) -> Self {@@ -144,6 +145,7 @@ impl RunResult {submission_id: v["submission_id"].as_str().map(str::to_owned),cursor: v["cursor"].as_i64().unwrap_or(0),board: v.get("board").filter(|b| !b.is_null()).cloned(),+ provider_error: serde_json::from_value(v["provider_error"].clone()).ok(),}}}@@ -151,6 +153,12 @@ impl RunResult {/// Progress while a run is streaming.#[derive(Clone, Debug)]pub enum RunEvent {+ AttemptStarted { session_id: String, turn_id: Option<String>, attempt_id: String, attempt: u32 },+ AttemptFinished { session_id: String, turn_id: Option<String>, attempt_id: String, status: String, failure: Option<openworker_runtime::provider::Failure> },+ TextDelta { session_id: String, turn_id: Option<String>, attempt_id: String, text: String },+ ReasoningDelta { session_id: String, turn_id: Option<String>, attempt_id: String, kind: openworker_runtime::provider::ReasoningKind, text: String, block_index: u64 },+ ModelActivity { session_id: String, turn_id: Option<String>, attempt_id: String, elapsed_ms: u64 },+ ProviderError { session_id: String, turn_id: Option<String>, attempt_id: String, diagnostic: openworker_runtime::provider::Diagnostic },/// Named-team events retain both instance and definition identities.TeamActivity {agent_id: String,@@ -205,6 +213,9 @@ impl RunEvent {}let agent = item["agent"].as_str().unwrap_or_default().to_owned();let summary = &item["summary"];+ let session_id = item["event"]["session_id"].as_str().unwrap_or_default().to_owned();+ let turn_id = item["event"]["turn_id"].as_str().map(str::to_owned);+ let attempt_id = || summary["attempt_id"].as_str().unwrap_or_default().to_owned();let tool = || summary["tool"].as_str().unwrap_or_default().to_owned();// Older hosts omit this from summaries, but retain it in the journal.let call_id = || {@@ -215,6 +226,12 @@ impl RunEvent {.to_owned()};Some(match summary["type"].as_str() {+ Some("model_attempt_started") => RunEvent::AttemptStarted { session_id, turn_id, attempt_id:attempt_id(), attempt:summary["attempt"].as_u64()? as u32 },+ Some("model_attempt_finished") => RunEvent::AttemptFinished { session_id, turn_id, attempt_id:attempt_id(), status:summary["status"].as_str()?.into(), failure:serde_json::from_value(summary["kind"].clone()).ok() },+ Some("assistant_delta") => RunEvent::TextDelta { session_id, turn_id, attempt_id:attempt_id(), text:summary["text"].as_str()?.into() },+ Some("reasoning_delta") => RunEvent::ReasoningDelta { session_id, turn_id, attempt_id:attempt_id(), kind:serde_json::from_value(summary["kind"].clone()).ok()?, text:summary["text"].as_str()?.into(), block_index:summary["block_index"].as_u64()? },+ Some("model_activity") => RunEvent::ModelActivity { session_id, turn_id, attempt_id:attempt_id(), elapsed_ms:summary["elapsed_ms"].as_u64()? },+ Some("provider_error") => RunEvent::ProviderError { session_id, turn_id, attempt_id:attempt_id(), diagnostic:serde_json::from_value(summary["diagnostic"].clone()).ok()? },Some("message") => RunEvent::Message {agent,text: summary["text"].as_str().unwrap_or_default().to_owned(),@@ -843,4 +860,17 @@ mod tests {assert!(keys[2].0.is_none());}}++ #[test]+ fn attempt_reasoning_and_provider_diagnostics_are_typed() {+ let event=json!({"session_id":"s","turn_id":"t","data":{}});+ let item=|summary:Value|json!({"agent":"a","event":event,"summary":summary});+ assert!(matches!(RunEvent::from_wire(&item(json!({"type":"model_attempt_started","attempt_id":"a1","attempt":1}))),Some(RunEvent::AttemptStarted{attempt:1,attempt_id,..}) if attempt_id=="a1"));+ assert!(matches!(RunEvent::from_wire(&item(json!({"type":"model_attempt_finished","attempt_id":"a1","status":"failed","kind":"incomplete_stream"}))),Some(RunEvent::AttemptFinished{failure:Some(openworker_runtime::provider::Failure::IncompleteStream),..})));+ assert!(matches!(RunEvent::from_wire(&item(json!({"type":"reasoning_delta","attempt_id":"a1","kind":"summary","text":"display","block_index":0}))),Some(RunEvent::ReasoningDelta{kind:openworker_runtime::provider::ReasoningKind::Summary,text,..}) if text=="display"));+ let diagnostic=json!({"category":"authentication","retryable":false,"http_status":401,"request_id":"req_fixture","api":"responses","code":"invalid_api_key"});+ assert!(matches!(RunEvent::from_wire(&item(json!({"type":"provider_error","attempt_id":"a1","diagnostic":diagnostic}))),Some(RunEvent::ProviderError{diagnostic,..}) if diagnostic.http_status==Some(401)));+ let result=RunResult::from_wire(&json!({"status":"failed","provider_error":diagnostic}));+ assert_eq!(result.provider_error.unwrap().request_id.as_deref(),Some("req_fixture"));+ }}diff --git a/harness/harness-sdk/src/host_runs.rs b/harness/harness-sdk/src/host_runs.rsindex 20ec59ed..b0154657 100644--- a/harness/harness-sdk/src/host_runs.rs+++ b/harness/harness-sdk/src/host_runs.rs@@ -985,6 +985,7 @@ impl Host {}));if status == "failed" && cached.is_none() {out["failure"] = lead_turn["status"].clone();+ out["provider_error"] = lead_turn["provider_error"].clone();}if cached.is_none() && status != "running" && status != "interrupted" {if let Some(r) = record.as_mut() {@@ -1084,6 +1085,12 @@ fn final_output(snapshot: &Value) -> Value {/// A journal event plus a small typed summary SDKs can map directly.fn stream_item(agent: &str, e: Event) -> Value {let summary = match e.kind.as_str() {+ "model_attempt_started" | "model_attempt_finished" | "assistant_delta"+ | "reasoning_delta" | "model_activity" | "provider_error" => {+ let mut summary = e.data.clone();+ summary["type"] = json!(e.kind);+ Some(summary)+ }"interaction_resolved" => (e.data["origin"] == "app_policy").then(|| json!({"type": "auto_approved", "tool": e.data["tool"]})),"assistant_message" => {diff --git a/harness/harness-sdk/src/lib.rs b/harness/harness-sdk/src/lib.rsindex cba78732..383237bd 100644--- a/harness/harness-sdk/src/lib.rs+++ b/harness/harness-sdk/src/lib.rs@@ -36,6 +36,8 @@ pub use openworker_telemetry::otlp::{/// Opt-in prompt/tool content in telemetry (off by default).pub use openworker_telemetry::{Capture, ContentPolicy};pub use providers::{Credentials, Model, ModelApi, ModelProvider};+pub use openworker_runtime::anthropic::Thinking as AnthropicThinking;+pub use openworker_runtime::provider::{Diagnostic as ProviderDiagnostic, Failure as ProviderFailure, ReasoningKind};pub enum EmbeddedConfig {Local(RuntimeConfig),@@ -81,6 +83,7 @@ pub type Result<T> = std::result::Result<T, Error>;pub enum Error {InvalidInput(String),Remote(String),+ Provider(openworker_runtime::provider::Diagnostic),Setup(String),Unavailable,/// May have committed or started. Reconcile durable IDs; do not blindly retry.@@ -91,6 +94,7 @@ impl std::fmt::Display for Error {fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {match self {Self::InvalidInput(s) | Self::Remote(s) | Self::Setup(s) => f.write_str(s),+ Self::Provider(diagnostic) => std::fmt::Display::fmt(diagnostic, f),Self::Unavailable => f.write_str("Harness unavailable or recovering; request not sent"),Self::OutcomeUnknown => f.write_str("Harness response lost; execution outcome unknown. Reconcile saved session before retrying."),Self::Closed => f.write_str("Harness client closed"),diff --git a/harness/harness-sdk/src/providers.rs b/harness/harness-sdk/src/providers.rsindex 03e78f7d..cfef6a8b 100644--- a/harness/harness-sdk/src/providers.rs+++ b/harness/harness-sdk/src/providers.rs@@ -50,6 +50,7 @@ pub struct ModelProvider {base_url: Option<String>,stream_usage: Option<bool>,prompt_caching: Option<bool>,+ thinking: Option<openworker_runtime::anthropic::Thinking>,credentials: Option<Credentials>,pub(crate) access_auth: bool,pub(crate) refresh: Option<openworker_runtime::provider_auth::Refresh>,@@ -71,6 +72,7 @@ impl ModelProvider {base_url: None,stream_usage: None,prompt_caching: None,+ thinking: None,credentials: None,access_auth: false,refresh: None,@@ -93,6 +95,11 @@ impl ModelProvider {self.prompt_caching = Some(enabled);self}+ /// Explicit Anthropic generation mode; choose one supported by your model.+ pub fn thinking(mut self, thinking: openworker_runtime::anthropic::Thinking) -> Self {+ self.thinking = Some(thinking);+ self+ }pub fn credentials(mut self, value: Credentials) -> Self {self.credentials = Some(value);self@@ -128,6 +135,10 @@ impl ModelProvider {"prompt_caching is only supported for Anthropic Messages".into(),));}+ if let Some(thinking) = &self.thinking {+ if self.api != ModelApi::Anthropic { return Err(Error::InvalidInput("thinking is only supported for Anthropic".into())); }+ thinking.validate().map_err(|e|Error::InvalidInput(e.to_string()))?;+ }if self.stream_usage.is_some() && self.api != ModelApi::ChatCompletions {return Err(Error::InvalidInput("stream_usage is only supported for Chat Completions".into(),@@ -145,6 +156,7 @@ impl ModelProvider {if let Some(enabled) = self.prompt_caching {wire["prompt_caching"] = json!(enabled);}+ if let Some(thinking) = self.thinking { wire["thinking"] = json!(thinking); }if self.access_auth {wire["access_token_auth"] = json!(true);}diff --git a/harness/harness-sdk/src/transport.rs b/harness/harness-sdk/src/transport.rsindex c93b1757..4fcd2da3 100644--- a/harness/harness-sdk/src/transport.rs+++ b/harness/harness-sdk/src/transport.rs@@ -6,6 +6,13 @@ use tokio::{process::{Child, ChildStdin},};+fn provider_error(error: anyhow::Error) -> Error {+ match error.downcast_ref::<openworker_runtime::provider::Error>() {+ Some(error) => Error::Provider(error.diagnostic()),+ None => Error::Remote(error.to_string()),+ }+}+pub(crate) enum Backend {Host {host: Arc<crate::host::Host>,@@ -342,7 +349,7 @@ impl Backend {tokio::pin!(request);loop {tokio::select! {- result=&mut request=>break result.map_err(|e|Error::Remote(e.to_string())),+ result=&mut request=>break result.map_err(provider_error),value=application.recv()=>{let value=value.map_err(|_|Error::OutcomeUnknown)?;let _=sink.send(Notification::Application(value));},}}@@ -350,7 +357,7 @@ impl Backend {Self::Embedded { runtime, .. } => runtime.dispatch(method, params).await- .map_err(|e| Error::Remote(e.to_string())),+ .map_err(provider_error),Self::Process(p) => {p.next_id += 1;let id = p.next_id;@@ -380,6 +387,9 @@ impl Backend {return Err(Error::OutcomeUnknown);}if let Some(message) = v["error"]["message"].as_str() {+ if let Ok(diagnostic) = serde_json::from_value::<openworker_runtime::provider::Diagnostic>(v["error"]["provider_error"].clone()) {+ return Err(Error::Provider(diagnostic));+ }return Err(Error::Remote(message.into()));}return v.get("result").cloned().ok_or(Error::OutcomeUnknown);diff --git a/harness/runtime/examples/provider_probe.rs b/harness/runtime/examples/provider_probe.rsindex 7313222f..f16cd33d 100644--- a/harness/runtime/examples/provider_probe.rs+++ b/harness/runtime/examples/provider_probe.rs@@ -44,7 +44,7 @@ async fn main() -> Result<()> {p=rx.recv(),if !closed=>match p {Some(Progress::Delta(_))=>delta_count+=1,Some(Progress::Complete(c))=>complete=Some(c),- Some(Progress::OutputStarted|Progress::Usage(_))=>{},+ Some(Progress::Reasoning(_)|Progress::OutputStarted|Progress::Usage(_))=>{},None=>closed=true,}}diff --git a/harness/runtime/src/anthropic.rs b/harness/runtime/src/anthropic.rsindex 2205745c..a407ed22 100644--- a/harness/runtime/src/anthropic.rs+++ b/harness/runtime/src/anthropic.rs@@ -8,11 +8,39 @@ use serde_json::{json, Value};use std::collections::{BTreeMap, BTreeSet};use tokio::sync::mpsc;+/// Explicit generation configuration; display is controlled separately per turn.+#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]+#[serde(tag="type",rename_all="snake_case",deny_unknown_fields)]+pub enum Thinking {+ Disabled,+ Adaptive,+ Enabled { #[serde(rename="budgetTokens")] budget_tokens: u32 },+}+impl Thinking {+ pub fn validate(&self) -> anyhow::Result<()> {+ if let Self::Enabled { budget_tokens } = self {+ anyhow::ensure!((1024..=1_000_000).contains(budget_tokens), "Thinking budget must be between 1024 and 1000000 tokens");+ }+ Ok(())+ }+ pub fn request(&self, max_tokens: usize) -> Result<Value, Error> {+ self.validate().map_err(|_| config())?;+ Ok(match self {+ Self::Disabled => json!({"type":"disabled"}),+ Self::Adaptive => json!({"type":"adaptive"}),+ Self::Enabled { budget_tokens } => {+ if *budget_tokens as usize >= max_tokens { return Err(config()); }+ json!({"type":"enabled","budget_tokens":budget_tokens})+ }+ })+ }+}+fn invalid() -> Error {- Error::new(Failure::InvalidResponse)+ Error::new(Failure::InvalidResponse).with_code("invalid_anthropic_message")}fn config() -> Error {- Error::new(Failure::Configuration)+ Error::new(Failure::Configuration).with_code("invalid_anthropic_request")}fn append(messages: &mut Vec<Value>, role: &str, blocks: Vec<Value>) {@@ -171,8 +199,10 @@ pub struct Decoder {usage: Usage,output_known: bool,input_known: bool,+ reasoning: Option<crate::provider::ReasoningDelta>,}impl Decoder {+ pub fn take_reasoning(&mut self) -> Option<crate::provider::ReasoningDelta> { self.reasoning.take() }fn usage_report(&self) -> Option<crate::telemetry::TokenUsage> {self.input_known.then(|| {let mut u = self.usage.reported();@@ -296,6 +326,13 @@ impl Decoder {if field == "text" {return Ok(Some(fragment.into()));}+ if field == "thinking" {+ self.reasoning = Some(crate::provider::ReasoningDelta {+ kind: crate::provider::ReasoningKind::Thinking,+ text: fragment.into(),+ block_index: e["index"].as_u64().ok_or_else(invalid)?,+ });+ }}Some("content_block_stop") => {let block = self@@ -422,6 +459,7 @@ pub async fn request(.await.map_err(|_| Error::new(Failure::Transport))?;let status = response.status().as_u16();+ let request_id = crate::provider::request_id(response.headers(), key);let retry_after = response.headers().get("retry-after")@@ -431,7 +469,7 @@ pub async fn request(if !(200..300).contains(&status) {let mut bytes = vec![];while let Some(chunk) = stream.next().await {- let chunk = chunk.map_err(|_| Error::new(classify(status, &Value::Null)))?;+ let chunk = chunk.map_err(|_| Error {kind:classify(status,&Value::Null),retry_after,http_status:Some(status),request_id:request_id.clone(),api:Some(crate::model_api::Api::Anthropic),code:Some("http_error_body_unavailable")})?;if bytes.len() + chunk.len() > 16384 {break;}@@ -440,6 +478,9 @@ pub async fn request(let error: Value = serde_json::from_slice(&bytes).unwrap_or(Value::Null);return Err(Error {http_status: Some(status),+ request_id,+ api: Some(crate::model_api::Api::Anthropic),+ code: crate::provider::upstream_code(&error["error"]),kind: classify(status, &error["error"]),retry_after,});@@ -472,6 +513,9 @@ pub async fn request(previous_usage = Some(usage);}let decoded = decoded?;+ if let Some(delta) = decoder.take_reasoning() {+ tx.send(Progress::Reasoning(delta)).await.map_err(|_| Error::new(Failure::Transport))?;+ }if !first_sent && decoder.output_started() {tx.send(Progress::OutputStarted).awaitdiff --git a/harness/runtime/src/chat_completions.rs b/harness/runtime/src/chat_completions.rsindex 61aab61c..3e98fcee 100644--- a/harness/runtime/src/chat_completions.rs+++ b/harness/runtime/src/chat_completions.rs@@ -4,10 +4,10 @@ use crate::provider::{Completion, Error, Failure, StreamDecoder, Usage};use serde_json::{json, Value};use std::collections::{BTreeMap, BTreeSet};fn invalid() -> Error {- Error::new(Failure::InvalidResponse)+ Error::new(Failure::InvalidResponse).with_code("invalid_chat_completion")}fn config() -> Error {- Error::new(Failure::Configuration)+ Error::new(Failure::Configuration).with_code("invalid_chat_request_history")}const GOOGLE_PRIVATE: &str = "google-chat-completions";@@ -227,6 +227,7 @@ pub struct Decoder {accounting_tail: bool,usage: Option<Usage>,bytes: usize,+ reasoning: Option<crate::provider::ReasoningDelta>,binding: Option<String>,google_compat: bool,message_signature: Option<String>,@@ -347,6 +348,16 @@ impl Decoder {}None => None,};+ let reasoning_content = delta.get("reasoning_content").filter(|v| !v.is_null());+ let reasoning = delta.get("reasoning").filter(|v| !v.is_null());+ if reasoning_content.zip(reasoning).is_some_and(|(a,b)|a != b) { return Err(invalid()); }+ if let Some(text) = reasoning_content.or(reasoning) {+ self.reasoning = Some(crate::provider::ReasoningDelta {+ kind: crate::provider::ReasoningKind::Thinking,+ text: text.as_str().ok_or_else(invalid)?.to_owned(),+ block_index: 0,+ });+ }if let Some(calls) = delta.get("tool_calls") {for c in calls.as_array().ok_or_else(invalid)? {let signature = google_signature(c)?;@@ -378,7 +389,7 @@ impl Decoder {}self.calls.keys().next_back().map_or(0, |index| index + 1)} else {- c["index"].as_u64().ok_or_else(invalid)?+ c["index"].as_u64().ok_or_else(|| invalid().with_code("invalid_tool_call_index"))?};if index >= 64 || c.get("type").is_some_and(|v| v != "function") {return Err(invalid());@@ -429,6 +440,7 @@ impl Decoder {}}impl StreamDecoder for Decoder {+ fn take_reasoning(&mut self) -> Option<crate::provider::ReasoningDelta> { self.reasoning.take() }fn usage_report(&self) -> Option<crate::telemetry::TokenUsage> {self.usage.as_ref().map(Usage::reported)}diff --git a/harness/runtime/src/connections.rs b/harness/runtime/src/connections.rsindex b8e39034..49ce08c0 100644--- a/harness/runtime/src/connections.rs+++ b/harness/runtime/src/connections.rs@@ -74,6 +74,8 @@ struct Set {stream_usage: Option<bool>,#[serde(default)]prompt_caching: Option<bool>,+ #[serde(default)]+ thinking: Option<crate::anthropic::Thinking>,}impl Manager {/// Trusted server bootstrap only; not exposed by the model or stdio protocol.@@ -173,6 +175,7 @@ impl Manager {)?;}db.execute_batch("CREATE TABLE IF NOT EXISTS endpoints(id TEXT PRIMARY KEY, api TEXT NOT NULL, base_url TEXT NOT NULL);")?;+ db.execute_batch("CREATE TABLE IF NOT EXISTS thinking_options(id TEXT PRIMARY KEY, value TEXT NOT NULL);")?;let has_stream_usage = db.prepare("PRAGMA table_info(endpoints)")?.query_map([], |r| r.get::<_, String>(1))?@@ -254,6 +257,8 @@ impl Manager {}if api == "anthropic" {connections.last_mut().unwrap()["prompt_caching"] = json!(prompt_caching);+ let thinking: Option<String> = state.db.query_row("SELECT value FROM thinking_options WHERE id=?1", [&id], |r|r.get(0)).optional()?;+ if let Some(value) = thinking { connections.last_mut().unwrap()["thinking"] = serde_json::from_str(&value)?; }}}Ok(@@ -281,6 +286,10 @@ impl Manager {if input.prompt_caching.is_some() && api != crate::model_api::Api::Anthropic {bail!("prompt_caching is only supported for Anthropic Messages");}+ if let Some(thinking) = &input.thinking {+ anyhow::ensure!(api == crate::model_api::Api::Anthropic, "thinking is only supported for Anthropic");+ thinking.validate()?;+ }let base_url =crate::model_api::base_url(input.base_url.as_deref().unwrap_or(api.default_url()))?;let api_json = serde_json::to_string(&api)?;@@ -375,6 +384,9 @@ impl Manager {"INSERT INTO endpoints(id,api,base_url,stream_usage,prompt_caching) VALUES(?1,?2,?3,COALESCE(?4,1),COALESCE(?5,1)) ON CONFLICT(id) DO UPDATE SET stream_usage=COALESCE(?4,endpoints.stream_usage),prompt_caching=COALESCE(?5,endpoints.prompt_caching)",params![input.id, api_json, base_url, input.stream_usage, input.prompt_caching],)?;+ if let Some(thinking) = input.thinking {+ transaction.execute("INSERT INTO thinking_options(id,value) VALUES(?1,?2) ON CONFLICT(id) DO UPDATE SET value=excluded.value", params![input.id,serde_json::to_string(&thinking)?])?;+ }transaction.commit()?;#[cfg(target_os = "macos")]if was_secure && input.persistence != "secure" {@@ -410,6 +422,7 @@ impl Manager {state.db.execute("DELETE FROM endpoints WHERE id=?1", [id])?;+ state.db.execute("DELETE FROM thinking_options WHERE id=?1", [id])?;Ok(json!({"removed":true}))}pub fn defaults(&self) -> Result<Defaults> {@@ -635,6 +648,12 @@ impl Manager {|r| r.get(0),).context("Selected model provider is missing or its identity changed")}+ pub fn thinking(&self, selected: &Selection) -> Result<Option<crate::anthropic::Thinking>> {+ let value: Option<String> = self.state.lock().unwrap().db.query_row(+ "SELECT t.value FROM connections c LEFT JOIN thinking_options t ON c.id=t.id WHERE c.id=?1 AND c.identity=?2",+ params![selected.connection, selected.identity], |r|r.get(0))?;+ value.map(|v|serde_json::from_str(&v).map_err(Into::into)).transpose()+ }/// Harness-owned provider registration; never journal this secret-bearing request.pub fn set_provider(&self, value: Value) -> Result<Value> {#[derive(Deserialize)]@@ -649,6 +668,8 @@ impl Manager {#[serde(default)]prompt_caching: Option<bool>,#[serde(default)]+ thinking: Option<crate::anthropic::Thinking>,+ #[serde(default)]access_token_auth: bool,}#[derive(Deserialize)]@@ -666,6 +687,10 @@ impl Manager {if input.prompt_caching.is_some() && input.api != crate::model_api::Api::Anthropic {bail!("prompt_caching is only supported for Anthropic Messages");}+ if let Some(thinking) = &input.thinking {+ anyhow::ensure!(input.api == crate::model_api::Api::Anthropic, "thinking is only supported for Anthropic");+ thinking.validate()?;+ }anyhow::ensure!(!input.access_token_auth|| (input.credentials.is_none()@@ -730,6 +755,9 @@ impl Manager {params![input.name, serde_json::to_string(&input.api)?, base_url, enabled],)?;}+ if let Some(thinking) = input.thinking {+ self.state.lock().unwrap().db.execute("INSERT INTO thinking_options(id,value) VALUES(?1,?2) ON CONFLICT(id) DO UPDATE SET value=excluded.value", params![input.name,serde_json::to_string(&thinking)?])?;+ }return Ok(json!({"saved":true,"credentials_unchanged":true}));}}@@ -751,7 +779,7 @@ impl Manager {.into(),),};- self.set(json!({"id":input.name,"label":input.name,"provider":input.api.kind(),"api":input.api,"base_url":base_url,"stream_usage":input.stream_usage,"prompt_caching":input.prompt_caching,"api_key":key,"persistence":storage}))+ self.set(json!({"id":input.name,"label":input.name,"provider":input.api.kind(),"api":input.api,"base_url":base_url,"stream_usage":input.stream_usage,"prompt_caching":input.prompt_caching,"thinking":input.thinking,"api_key":key,"persistence":storage}))}}#[cfg(not(any(windows, target_os = "macos")))]diff --git a/harness/runtime/src/model.rs b/harness/runtime/src/model.rsindex 6a191b05..566919ea 100644--- a/harness/runtime/src/model.rs+++ b/harness/runtime/src/model.rs@@ -10,6 +10,8 @@ pub struct Model {pub observation: Option<CallContext>,/// Withhold candidate bodies from exporters until runtime safety checks finish.pub content_withheld: bool,+ /// Opt-in forwarding of provider-supported display reasoning, never signatures.+ pub stream_reasoning: bool,pub connection: Option<(std::sync::Arc<crate::connections::Manager>,crate::connections::Selection,@@ -31,6 +33,7 @@ pub type SampleHook = std::sync::Arc<pub type SummaryHook = std::sync::Arc<dyn Fn(String) -> TestFuture<String> + Send + Sync>;pub enum Progress {Delta(String),+ Reasoning(crate::provider::ReasoningDelta),/// Internal measurement only; never authorizes partial provider output.OutputStarted,/// Latest cumulative provider snapshot, not a token increment.@@ -94,6 +97,7 @@ impl Model {scripted,observation: None,content_withheld: false,+ stream_reasoning: false,connection: None,#[cfg(test)]sample_hook: None,@@ -277,6 +281,7 @@ impl Model {};match &progress {Progress::Delta(text) if !text.is_empty() => span.first_output(),+ Progress::Reasoning(delta) if !delta.text.is_empty() => span.first_output(),Progress::OutputStarted => span.first_output(),Progress::Usage(usage) => span.reported_usage(*usage),Progress::Complete(completion) => {@@ -396,7 +401,7 @@ impl Model {tx,crate::chat_completions::Decoder::with_binding(binding, google_compat),)- .await?;+ .await.map_err(|e| e.with_api(api))?;return Ok(());}if api == crate::model_api::Api::Anthropic {@@ -413,7 +418,16 @@ impl Model {body.as_object_mut().unwrap().remove("cache_control");}}- crate::anthropic::request(&client, &endpoint, key, body, tx).await?;+ // Short probes, reviewers and compactors have separate small+ // budgets; do not apply the agent's thinking generation budget.+ if self.observation.as_ref().is_some_and(|c| c.purpose == crate::telemetry::ModelPurpose::Main) {+ if let Some((manager, selection)) = &self.connection {+ if let Some(thinking) = manager.thinking(selection)? {+ body["thinking"] = thinking.request(max_tokens).map_err(|e|e.with_api(api))?;+ }+ }+ }+ crate::anthropic::request(&client, &endpoint, key, body, tx).await.map_err(|e| e.with_api(api))?;return Ok(());}if history.iter().any(|v| v.get("provider").is_some()) {@@ -431,10 +445,11 @@ impl Model {}).collect::<Vec<_>>();let mut body = json!({"model":self.name,"instructions":instructions,"input":input,"tools":tools,"stream":true,"store":false,"include":["reasoning.encrypted_content"],"parallel_tool_calls":false,"max_output_tokens":max_tokens});+ if self.stream_reasoning { body["reasoning"] = json!({"summary":"auto"}); }if let Some(format) = format {body["text"] = json!({"format":format});}- crate::provider::request(&client, &endpoint, key, body, tx).await?;+ crate::provider::request(&client, &endpoint, key, body, tx).await.map_err(|e| e.with_api(api))?;Ok(())}pub async fn summarize(@@ -546,7 +561,7 @@ impl Model {r=&mut request,if !finished=>{r?;finished=true;},p=rx.recv(),if !closed=>match p {Some(Progress::Complete(r))=>response=Some(r),- Some(Progress::Delta(_)|Progress::OutputStarted|Progress::Usage(_))=>{},None=>closed=true+ Some(Progress::Delta(_)|Progress::Reasoning(_)|Progress::OutputStarted|Progress::Usage(_))=>{},None=>closed=true}}}diff --git a/harness/runtime/src/protocol.rs b/harness/runtime/src/protocol.rsindex 5bb22c99..1d6b3e2a 100644--- a/harness/runtime/src/protocol.rs+++ b/harness/runtime/src/protocol.rs@@ -132,7 +132,7 @@ pub async fn serve_endpoint(rt: impl Endpoint + 'static) -> Result<()> {if method=="session/attach"{subscribed.insert(p["session_id"].as_str().unwrap().to_owned());}Ok::<_,anyhow::Error>(result)}.await;- match result{Ok(result)=>json!({"jsonrpc":"2.0","id":id,"result":result}),Err(e)=>json!({"jsonrpc":"2.0","id":id,"error":{"code":-32000,"message":e.to_string()}})}+ match result{Ok(result)=>json!({"jsonrpc":"2.0","id":id,"result":result}),Err(e)=>json!({"jsonrpc":"2.0","id":id,"error":{"code":-32000,"message":e.to_string(),"provider_error":e.downcast_ref::<crate::provider::Error>().map(|e|e.diagnostic())}})}}}},diff --git a/harness/runtime/src/provider.rs b/harness/runtime/src/provider.rsindex f839a62d..a798ddd6 100644--- a/harness/runtime/src/provider.rs+++ b/harness/runtime/src/provider.rs@@ -41,6 +41,18 @@ pub enum Failure {Configuration,SafetyBlocked,}+#[derive(Clone, Debug)]+pub struct Error {+ pub kind: Failure,+ pub retry_after: Option<u64>,+ /// Set only on an HTTP rejection, never on an error inside a stream.+ pub http_status: Option<u16>,+ pub request_id: Option<String>,+ pub api: Option<crate::model_api::Api>,+ /// Fixed parser code or an allowlisted upstream code; never arbitrary text.+ pub code: Option<&'static str>,+}+/// Safe structured context. Never contains request/response bodies or credentials.#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]#[serde(deny_unknown_fields)]@@ -56,20 +68,30 @@ pub struct Diagnostic {#[serde(default, skip_serializing_if = "Option::is_none")]pub code: Option<String>,}-#[derive(Clone, Debug)]-pub struct Error {- pub kind: Failure,- pub retry_after: Option<u64>,- /// Set only on an HTTP rejection, never on an error inside a stream.- pub http_status: Option<u16>,++impl std::fmt::Display for Diagnostic {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ write!(f, "Provider {:?}", self.category)?;+ if let Some(status) = self.http_status { write!(f, " (HTTP {status})")?; }+ if let Some(id) = &self.request_id { write!(f, "; request {id}")?; }+ if let Some(code) = &self.code { write!(f, "; {code}")?; }+ Ok(())+ }+}++/// Request IDs are identifiers, not arbitrary upstream diagnostic text.+pub(crate) fn request_id(headers: &reqwest::header::HeaderMap, key: &str) -> Option<String> {+ ["x-request-id", "request-id"].into_iter().find_map(|name| {+ let id = headers.get(name)?.to_str().ok()?;+ (!id.is_empty() && id.len() <= 128+ && id.bytes().all(|b| b.is_ascii_alphanumeric() || b"-_:".contains(&b))+ && !id.starts_with("sk-") && (key.is_empty() || !id.contains(key)))+ .then(|| id.to_owned())+ })}impl std::fmt::Display for Error {fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {- write!(- f,- "Provider {:?}; no tool actions accepted from this attempt",- self.kind- )+ write!(f, "{}; no tool actions accepted from this attempt", self.diagnostic())}}impl std::error::Error for Error {}@@ -79,8 +101,20 @@ impl Error {kind,retry_after: None,http_status: None,+ request_id: None,+ api: None,+ code: match kind { Failure::InvalidResponse=>Some("invalid_response"), Failure::IncompleteStream=>Some("incomplete_stream"), _=>None },}}+ pub fn diagnostic(&self) -> Diagnostic {+ Diagnostic { category:self.kind, retryable:self.retryable(), http_status:self.http_status,+ request_id:self.request_id.clone(), api:self.api, code:self.code.map(str::to_owned) }+ }+ pub fn with_api(mut self, api: crate::model_api::Api) -> Self {+ self.api = Some(api);+ self+ }+ pub fn with_code(mut self, code: &'static str) -> Self { self.code=Some(code);self }pub fn retryable(&self) -> bool {matches!(self.kind,@@ -91,6 +125,55 @@ impl Error {) && self.retry_after.is_none_or(|s| s <= 30)}}++pub(crate) fn upstream_code(value: &Value) -> Option<&'static str> {+ const CODES:&[&str]=&["invalid_api_key","authentication_error","permission_error","invalid_request_error","rate_limit_exceeded","rate_limit_error","insufficient_quota","context_length_exceeded","server_error","api_error","overloaded_error","misalignment_policy_violation","RESOURCE_EXHAUSTED","INVALID_ARGUMENT","UNAUTHENTICATED","PERMISSION_DENIED"];+ CODES.iter().copied().find(|code| value["code"]==*code || value["type"]==*code || value["status"]==*code)+}++#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]+#[serde(rename_all="snake_case")]+pub enum ReasoningKind { Summary, Thinking }++/// Provider-designated display text only. Opaque signatures never enter this type.+#[derive(Clone, Debug)]+pub struct ReasoningDelta {+ pub kind: ReasoningKind,+ pub text: String,+ pub block_index: u64,+}++#[cfg(test)]+mod diagnostics_tests {+ use super::*;+ use serde_json::json;+ #[test]+ fn safe_diagnostics_exclude_credentials_and_unbounded_header_text() {+ let mut headers = reqwest::header::HeaderMap::new();+ for value in ["sk-secret-token", "Bearer token", "arbitrary\ttext", "credential-sentinel"] {+ headers.insert("x-request-id",value.parse().unwrap());+ assert!(request_id(&headers,"credential-sentinel").is_none());+ }+ headers.insert("x-request-id","req_fixture-123".parse().unwrap());+ assert_eq!(request_id(&headers,"credential-sentinel").as_deref(),Some("req_fixture-123"));+ let mut e=Error::new(Failure::RateLimit).with_api(crate::model_api::Api::Responses);+ e.http_status=Some(429);e.request_id=request_id(&headers,"");+ assert_eq!(serde_json::to_value(e.diagnostic()).unwrap(),json!({"category":"rate_limit","retryable":true,"http_status":429,"request_id":"req_fixture-123","api":"responses"}));+ }+ #[test]+ fn responses_only_exposes_designated_summary_deltas() {+ let mut d=Decoder::default();+ for event in [json!({"type":"response.reasoning_text.delta","delta":"hidden"}),json!({"type":"response.output_item.added","item":{"type":"reasoning","encrypted_content":"opaque"}})] {+ assert!(d.event(&event).unwrap().is_none());+ assert!(d.take_reasoning().is_none());+ }+ d.event(&json!({"type":"response.reasoning_summary_text.delta","delta":"Display summary","output_index":0})).unwrap();+ let delta=d.take_reasoning().unwrap();+ assert_eq!(delta.kind,ReasoningKind::Summary);+ assert_eq!(delta.text,"Display summary");+ assert!(d.take_reasoning().is_none());+ }+}#[derive(Clone, Debug, Default, Serialize, Deserialize)]pub struct Usage {pub input_tokens: u64,@@ -175,6 +258,7 @@ pub struct Decoder {completion: Option<Completion>,reported: Option<Usage>,output_started: bool,+ reasoning: Option<ReasoningDelta>,}impl Decoder {pub fn event(&mut self, event: &Value) -> Result<Option<String>, Error> {@@ -201,6 +285,14 @@ impl Decoder {return Err(Error::new(Failure::InvalidResponse));}match event["type"].as_str().unwrap_or("") {+ "response.reasoning_summary_text.delta" => {+ self.reasoning = Some(ReasoningDelta {+ kind: ReasoningKind::Summary,+ text: event["delta"].as_str().ok_or_else(|| Error::new(Failure::InvalidResponse))?.to_owned(),+ block_index: event["output_index"].as_u64().ok_or_else(|| Error::new(Failure::InvalidResponse))?,+ });+ Ok(None)+ }"response.output_text.delta" => Ok(Some(event["delta"].as_str()@@ -287,8 +379,10 @@ pub(crate) trait StreamDecoder {fn output_started(&self) -> bool {false}+ fn take_reasoning(&mut self) -> Option<ReasoningDelta> { None }}impl StreamDecoder for Decoder {+ fn take_reasoning(&mut self) -> Option<ReasoningDelta> { self.reasoning.take() }fn usage_report(&self) -> Option<crate::telemetry::TokenUsage> {self.reported.as_ref().map(Usage::reported)}@@ -323,6 +417,7 @@ pub(crate) async fn request_decoded<D: StreamDecoder>(.map_err(|_| Error::new(Failure::Transport))?;if !response.status().is_success() {let status = response.status().as_u16();+ let request_id = request_id(response.headers(), key);let retry_after = response.headers().get("retry-after")@@ -331,7 +426,7 @@ pub(crate) async fn request_decoded<D: StreamDecoder>(let mut stream = response.bytes_stream();let mut bytes = vec![];while let Some(chunk) = stream.next().await {- let chunk = chunk.map_err(|_| Error::new(classify(status, "")))?;+ let chunk = chunk.map_err(|_| Error { kind:classify(status,""),retry_after,http_status:Some(status),request_id:request_id.clone(),api:None,code:Some("http_error_body_unavailable") })?;if bytes.len() + chunk.len() > 16384 {break;}@@ -342,6 +437,9 @@ pub(crate) async fn request_decoded<D: StreamDecoder>(kind: classify(status, error["error"]["code"].as_str().unwrap_or("")),retry_after,http_status: Some(status),+ request_id,+ api: None,+ code: upstream_code(&error["error"]),});}let mut stream = response.bytes_stream();@@ -369,7 +467,7 @@ pub(crate) async fn request_decoded<D: StreamDecoder>(continue;}let event: Value =- serde_json::from_str(data).map_err(|_| Error::new(Failure::InvalidResponse))?;+ serde_json::from_str(data).map_err(|_| Error::new(Failure::InvalidResponse).with_code("invalid_stream_json"))?;let decoded = decoder.event(&event);if let Some(usage) = decoder.usage_report()@@ -381,6 +479,9 @@ pub(crate) async fn request_decoded<D: StreamDecoder>(previous_usage = Some(usage);}let decoded = decoded?;+ if let Some(delta) = decoder.take_reasoning() {+ tx.send(Progress::Reasoning(delta)).await.map_err(|_| Error::new(Failure::Transport))?;+ }if !first_sent && decoder.output_started() {tx.send(Progress::OutputStarted).awaitdiff --git a/harness/runtime/src/runtime.rs b/harness/runtime/src/runtime.rsindex 54c0c18e..0bf80c13 100644--- a/harness/runtime/src/runtime.rs+++ b/harness/runtime/src/runtime.rs@@ -595,6 +595,10 @@ impl Runtime {.iter().any(|h| h.when == crate::safety::HookPhase::BeforeTool);model.content_withheld = guarded_output;+ let output_options = &s.turn.as_ref().context("No current turn")?.output;+ model.stream_reasoning = output_options.reasoning && !guarded_output;+ let buffer_attempts = output_options.buffer_attempts;+ let show_reasoning = model.stream_reasoning;let mut instructions = self.bundle.instructions_for(&s.skills, &s.loaded_skills);if let Some(addendum) = s.turn@@ -777,12 +781,36 @@ impl Runtime {let mut result = None;let mut closed = false;let mut reported_output = None;+ let attempt_started = tokio::time::Instant::now();+ let mut heartbeat = tokio::time::interval_at(+ attempt_started + std::time::Duration::from_secs(5),+ std::time::Duration::from_secs(5),+ );+ heartbeat.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);+ let mut reasoning = Vec::<crate::provider::ReasoningDelta>::new();+ let mut reasoning_bytes = 0usize;+ let mut reasoning_limited = false;while result.is_none() || !closed {tokio::select! {r=&mut request,if result.is_none()=>{result=Some(r);},+ _=heartbeat.tick(),if result.is_none()=>{+ self.progress(sid,"model_activity",json!({"attempt_id":id,"phase":"waiting_for_model","elapsed_ms":attempt_started.elapsed().as_millis() as u64}))?;+ },p=rx.recv(),if !closed=>match p {Some(Progress::Delta(text))=>{- if !guarded_output {let e=self.store.lock().unwrap().attempt_delta(sid,&id,&text)?;self.publish(e);}+ if !guarded_output && !buffer_attempts {let e=self.store.lock().unwrap().attempt_delta(sid,&id,&text)?;self.publish(e);}+ },+ Some(Progress::Reasoning(delta))=>{+ if show_reasoning && !delta.text.is_empty() {+ if reasoning_bytes.saturating_add(delta.text.len()) <= 65536 {+ reasoning_bytes += delta.text.len();+ if buffer_attempts { reasoning.push(delta); }+ else { self.progress(sid,"reasoning_delta",json!({"attempt_id":id,"kind":delta.kind,"text":delta.text,"block_index":delta.block_index}))?; }+ } else if !reasoning_limited {+ reasoning_limited = true;+ self.progress(sid,"reasoning_display_limited",json!({"attempt_id":id,"max_bytes":65536}))?;+ }+ }},Some(Progress::Usage(u))=>{reported_output=u.output;},Some(Progress::OutputStarted)=>{},@@ -829,6 +857,21 @@ impl Runtime {match result.unwrap() {Ok(()) => {let response = completed.context("Missing completed provider output")?;+ if buffer_attempts && !guarded_output {+ for delta in reasoning {+ self.progress(sid,"reasoning_delta",json!({"attempt_id":id,"kind":delta.kind,"text":delta.text,"block_index":delta.block_index}))?;+ }+ for item in &response.output {+ if item["type"] == "message" {+ for part in item["content"].as_array().into_iter().flatten() {+ if let Some(text) = part["text"].as_str() {+ let e = self.store.lock().unwrap().attempt_delta(sid,&id,text)?;+ self.publish(e);+ }+ }+ }+ }+ }let event =self.store.lock()@@ -839,6 +882,9 @@ impl Runtime {}Err(error) => {let failure = error.downcast_ref::<crate::provider::Error>();+ if let Some(failure) = failure {+ self.progress(sid,"provider_error",json!({"attempt_id":id,"diagnostic":failure.diagnostic()}))?;+ }let event = self.store.lock().unwrap().finish_attempt(sid,&id,@@ -1756,6 +1802,7 @@ impl Runtime {}}if let Err(e) = result {+ let provider_error = e.downcast_ref::<crate::provider::Error>().map(|e| e.diagnostic());let status = if e.is::<safety::Unavailable>() {"waiting_safety"} else if e.is::<safety::Blocked>() {@@ -1775,10 +1822,11 @@ impl Runtime {} else {"turn_completed"},- json!({"status":status,"error":e.to_string()}),+ json!({"status":status,"error":e.to_string(),"provider_error":provider_error}),|s| {if let Some(t) = &mut s.turn {t.status = status.into();+ t.provider_error = provider_error.clone();}Ok(())},diff --git a/harness/runtime/src/runtime_m2_tests.rs b/harness/runtime/src/runtime_m2_tests.rsindex ca4128b9..a32d5c6f 100644--- a/harness/runtime/src/runtime_m2_tests.rs+++ b/harness/runtime/src/runtime_m2_tests.rs@@ -2,6 +2,79 @@ use super::*;use crate::provider::{Completion, Error, Failure};use std::sync::atomic::AtomicUsize;+#[tokio::test]+async fn reasoning_is_opt_in_and_protected_output_never_streams_candidates() {+ for (enabled, protected) in [(false,false),(true,false),(true,true)] {+ let mut model = Model::new("fixture".into(), true);+ model.sample_hook = Some(Arc::new(|_,_,tx|Box::pin(async move {+ tx.send(Progress::Reasoning(crate::provider::ReasoningDelta { kind:crate::provider::ReasoningKind::Summary,text:"DISPLAY_SENTINEL".into(),block_index:0 })).await.unwrap();+ tx.send(Progress::Delta("answer".into())).await.unwrap();+ tx.send(Progress::Complete(Completion { output:vec![json!({"type":"message","role":"assistant","content":[{"type":"output_text","text":"answer"}]})],usage:None })).await.unwrap();+ Ok(())+ })));+ let (_tmp,rt,sid) = fixture(model,true).await;+ rt.update(&sid,"configure",json!({}),|s| {+ s.turn.as_mut().unwrap().output.reasoning = enabled;+ if protected { s.turn.as_mut().unwrap().safety.guardrails.model_output.push(crate::safety::Rule::regex("check","forbidden",crate::safety::Action::Block)); }+ Ok(())+ }).unwrap();+ rt.sample(&sid,&[],&[]).await.unwrap();+ let events = rt.store.lock().unwrap().events(&sid,0).unwrap();+ assert_eq!(events.iter().any(|e|e.kind=="reasoning_delta"),enabled && !protected);+ if !enabled || protected { assert!(!serde_json::to_string(&events).unwrap().contains("DISPLAY_SENTINEL")); }+ if protected { assert!(!events.iter().any(|e|e.kind=="assistant_delta")); }+ }+}++#[tokio::test]+async fn buffered_attempts_discard_failed_text_and_reasoning() {+ let count = Arc::new(AtomicUsize::new(0));+ let seen = count.clone();+ let mut model = Model::new("fixture".into(),true);+ model.sample_hook = Some(Arc::new(move |_,_,tx| {+ let failed = seen.fetch_add(1,Ordering::SeqCst)==0;+ Box::pin(async move {+ let text = if failed {"FAILED_CANDIDATE"} else {"accepted"};+ tx.send(Progress::Delta(text.into())).await.unwrap();+ tx.send(Progress::Reasoning(crate::provider::ReasoningDelta {kind:crate::provider::ReasoningKind::Summary,text:text.into(),block_index:0})).await.unwrap();+ if failed { return Err(Error::new(Failure::IncompleteStream).into()); }+ tx.send(Progress::Complete(Completion {output:vec![json!({"type":"message","role":"assistant","content":[{"type":"output_text","text":text}]})],usage:None})).await.unwrap();+ Ok(())+ })+ }));+ let (_tmp,rt,sid)=fixture(model,true).await;+ rt.update(&sid,"configure",json!({}),|s| { let o=&mut s.turn.as_mut().unwrap().output;o.reasoning=true;o.buffer_attempts=true;Ok(()) }).unwrap();+ rt.sample(&sid,&[],&[]).await.unwrap();+ let events=rt.store.lock().unwrap().events(&sid,0).unwrap();+ assert!(!serde_json::to_string(&events).unwrap().contains("FAILED_CANDIDATE"));+ assert_eq!(events.iter().filter(|e|e.kind=="assistant_delta").count(),1);+ assert_eq!(events.iter().filter(|e|e.kind=="reasoning_delta").count(),1);+ let finished:Vec<_>=events.iter().filter(|e|e.kind=="model_attempt_finished").collect();+ assert_eq!(finished.len(),2);+ assert_eq!(finished[0].data["status"],"failed");+ assert_eq!(finished[1].data["status"],"completed");+ assert_ne!(finished[0].data["attempt_id"],finished[1].data["attempt_id"]);+}++#[tokio::test]+async fn long_model_call_emits_bounded_truthful_activity() {+ let mut model=Model::new("fixture".into(),true);+ model.sample_hook=Some(Arc::new(|_,_,tx|Box::pin(async move {+ tokio::time::sleep(std::time::Duration::from_millis(5100)).await;+ tx.send(Progress::Complete(Completion {output:vec![json!({"type":"message","role":"assistant","content":[{"type":"output_text","text":"ready"}]})],usage:None})).await.unwrap();+ Ok(())+ })));+ let (_tmp,rt,sid)=fixture(model,true).await;+ tokio::time::timeout(std::time::Duration::from_secs(10),rt.sample(&sid,&[],&[])).await.unwrap().unwrap();+ let events=rt.store.lock().unwrap().events(&sid,0).unwrap();+ let ticks:Vec<_>=events.iter().filter(|e|e.kind=="model_activity").collect();+ assert_eq!(ticks.len(),1);+ assert_eq!(ticks[0].data["phase"],"waiting_for_model");+ assert!(ticks[0].data["elapsed_ms"].as_u64().unwrap()>=5000);+ let end=events.iter().find(|e|e.kind=="model_attempt_finished").unwrap();+ assert!(ticks[0].event_id<end.event_id);+}+#[tokio::test]async fn live_tool_access_reaches_model_and_refreshes_on_retry_without_entering_history() {use openworker_tools::{Tool, ToolContext, ToolDescriptor, ToolFuture};@@ -427,6 +500,9 @@ async fn retries_keep_partial_attempts_separate_and_do_not_touch_tools_or_historkind: Failure::RateLimit,retry_after: Some(0),http_status: None,+ request_id: None,+ api: None,+ code: None,}.into());}@@ -545,6 +621,9 @@ async fn repeated_transient_failure_stops_after_three_attempts() {kind: Failure::Transient,retry_after: Some(0),http_status: None,+ request_id: None,+ api: None,+ code: None,}.into())})@@ -577,6 +656,9 @@ async fn retry_backoff_is_cancellable_without_another_request() {kind: Failure::RateLimit,retry_after: Some(30),http_status: None,+ request_id: None,+ api: None,+ code: None,}.into())})@@ -637,6 +719,9 @@ async fn compaction_retry_and_pending_approval_survive_the_same_restart_scenariokind: Failure::Transient,retry_after: Some(0),http_status: None,+ request_id: None,+ api: None,+ code: None,}.into());}diff --git a/harness/runtime/tests/anthropic.rs b/harness/runtime/tests/anthropic.rsindex fdadf6a0..466f7b94 100644--- a/harness/runtime/tests/anthropic.rs+++ b/harness/runtime/tests/anthropic.rs@@ -142,6 +142,28 @@ fn private_thinking_is_retained_but_not_streamed_and_foreign_history_is_rejectedhistory[1] = json!({"type":"reasoning","encrypted_content":"OPENAI_ONLY"});assert!(anthropic::body("test", "system", &history, &[], 1024, None).is_err());}++#[test]+fn display_thinking_is_separate_from_answer_and_never_includes_signatures() {+ let mut d=Decoder::default();d.event(&start()).unwrap();+ d.event(&json!({"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":""}})).unwrap();+ assert!(d.event(&json!({"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"Display text"}})).unwrap().is_none());+ let text=d.take_reasoning().unwrap();assert_eq!(text.text,"Display text");+ d.event(&json!({"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"SECRET_SIGNATURE"}})).unwrap();+ assert!(d.take_reasoning().is_none());+ d.event(&json!({"type":"content_block_stop","index":0})).unwrap();+ d.event(&json!({"type":"content_block_start","index":1,"content_block":{"type":"redacted_thinking","data":"SECRET_REDACTED"}})).unwrap();+ assert!(d.take_reasoning().is_none());+}++#[test]+fn thinking_generation_configuration_is_explicit_and_budgeted() {+ use openworker_runtime::anthropic::Thinking;+ assert_eq!(Thinking::Adaptive.request(4096).unwrap(),json!({"type":"adaptive"}));+ assert_eq!(Thinking::Enabled{budget_tokens:1024}.request(4096).unwrap(),json!({"type":"enabled","budget_tokens":1024}));+ assert!(Thinking::Enabled{budget_tokens:1024}.request(1024).is_err());+ assert!(Thinking::Enabled{budget_tokens:5}.validate().is_err());+}#[test]fn input_pairing_images_denials_and_structured_reviewer_format() {let history = vec![diff --git a/harness/runtime/tests/connections.rs b/harness/runtime/tests/connections.rsindex 482a2e40..a6da131a 100644--- a/harness/runtime/tests/connections.rs+++ b/harness/runtime/tests/connections.rs@@ -290,6 +290,28 @@ fn chat_stream_usage_defaults_on_and_opt_out_survives_rotation_and_restart() {assert!(m.stream_usage(&selected).is_err());}+#[test]+fn anthropic_thinking_is_explicit_validated_and_persisted() {+ use openworker_runtime::anthropic::Thinking;+ let root=tempfile::tempdir().unwrap();+ let m=Manager::open(root.path()).unwrap();+ let config=json!({"name":"Claude","api":"anthropic","base_url":"https://api.anthropic.com/v1"});+ m.set_provider(config.clone()).unwrap();+ let selected=m.resolve(&Choice::new("Claude","fixture")).unwrap();+ assert!(m.thinking(&selected).unwrap().is_none());+ let mut enabled=config.clone();enabled["thinking"]=json!({"type":"enabled","budgetTokens":2048});+ m.set_provider(enabled).unwrap();m.set_provider(config.clone()).unwrap();+ assert_eq!(m.thinking(&selected).unwrap(),Some(Thinking::Enabled{budget_tokens:2048}));+ let mut invalid=config.clone();invalid["thinking"]=json!({"type":"enabled","budgetTokens":2});+ assert!(m.set_provider(invalid).is_err());+ drop(m);let m=Manager::open(root.path()).unwrap();+ assert_eq!(m.thinking(&selected).unwrap(),Some(Thinking::Enabled{budget_tokens:2048}));+ assert!(m.set_provider(json!({"name":"Wrong","api":"responses","thinking":{"type":"adaptive"}})).is_err());+ let mut disabled=config;disabled["thinking"]=json!({"type":"disabled"});m.set_provider(disabled).unwrap();+ assert_eq!(m.thinking(&selected).unwrap(),Some(Thinking::Disabled));+ m.remove("Claude").unwrap();assert!(m.thinking(&selected).is_err());+}+#[test]fn chat_stream_usage_migrates_existing_endpoint_rows() {let root = tempfile::tempdir().unwrap();diff --git a/harness/runtime/tests/provider.rs b/harness/runtime/tests/provider.rsindex 4584a72f..0ff44726 100644--- a/harness/runtime/tests/provider.rs+++ b/harness/runtime/tests/provider.rs@@ -128,3 +128,30 @@ async fn real_http_parser_handles_rate_limit_and_truncated_stream() {assert!(matches!(rx.recv().await,Some(Progress::Delta(t)) if t=="partial"));assert!(rx.recv().await.is_none());}++#[tokio::test]+async fn http_diagnostics_retain_status_and_request_id_without_upstream_body() {+ let body=r#"{"error":{"code":"invalid_api_key","message":"secret=BODY_SENTINEL"}}"#;+ let (url,server)=http_once(&format!("HTTP/1.1 401 Unauthorized\r\nX-Request-ID: req_fixture\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}",body.len(),body)).await;+ let (tx,_)=mpsc::channel(10);+ let e=provider::request(&reqwest::Client::new(),&url,"KEY_SENTINEL",json!({}),tx).await.unwrap_err();+ server.await.unwrap();+ assert_eq!(e.http_status,Some(401));+ assert_eq!(e.request_id.as_deref(),Some("req_fixture"));+ assert!(!e.retryable());+ for text in [e.to_string(),serde_json::to_string(&e.diagnostic()).unwrap()] {+ assert!(!text.contains("BODY_SENTINEL"));assert!(!text.contains("KEY_SENTINEL"));+ }+}++#[tokio::test]+async fn broken_http_error_body_keeps_known_status_and_request_id() {+ let (url,server)=http_once("HTTP/1.1 401 Unauthorized\r\nX-Request-ID: req_broken\r\nContent-Length: 999\r\nConnection: close\r\n\r\npartial").await;+ let (tx,_)=mpsc::channel(10);+ let error=provider::request(&reqwest::Client::new(),&url,"test-only",json!({}),tx).await.unwrap_err();+ server.await.unwrap();+ assert_eq!(error.http_status,Some(401));+ assert_eq!(error.request_id.as_deref(),Some("req_broken"));+ assert_eq!(error.code,Some("http_error_body_unavailable"));+ assert_eq!(error.kind,Failure::Authentication);+}diff --git a/harness/typescript-sdk/README.md b/harness/typescript-sdk/README.mdindex 479d13bc..da7450d7 100644--- a/harness/typescript-sdk/README.md+++ b/harness/typescript-sdk/README.md@@ -159,6 +159,10 @@ attempts separately; do not blindly concatenate a failed attempt into its retry.The game-builder starter's existing UI displays progress and completed messages;incremental-text rendering is a separate UI improvement.+For typed attempt events, opt-in reasoning display, optional attempt buffering and+safe provider diagnostics, see [Streaming and retries](../../docs/harness/guides/streaming.md).+`AttemptOutput` provides a bounded UI projection that removes failed candidates.+Application tools receive `context.callId`. Tool-start and tool-finish summariesexpose that same ID as `item.summary.callId`; the raw journal uses`item.event.data.call_id`. Use it to correlate calls, including repeated calls todiff --git a/harness/typescript-sdk/package.json b/harness/typescript-sdk/package.jsonindex d5e870da..b492dbaf 100644--- a/harness/typescript-sdk/package.json+++ b/harness/typescript-sdk/package.json@@ -24,7 +24,8 @@"test:team-placement": "npm run build && node --test test/team-placement.test.mjs","test:coding-presets:integration": "npm run build && node --test --test-concurrency=1 test/coding-presets-live.test.mjs","build": "tsc -p tsconfig.json",- "test": "npm run build && node --test test/config.test.mjs test/transport.test.mjs",+ "test": "npm run build && node --test test/config.test.mjs test/transport.test.mjs test/attempt-output.test.mjs",+ "test:stream-events": "npm run build && node --test --test-concurrency=1 test/stream-events.test.mjs","test:integration": "npm run build && node --test --test-concurrency=1 test/integration.test.mjs","test:dynamic-agents": "npm run build && node --test --test-concurrency=1 test/dynamic-agents.test.mjs","test:tool-free": "npm run build && node --test --test-concurrency=1 test/tool-free.test.mjs",diff --git a/harness/typescript-sdk/src/attempt-output.ts b/harness/typescript-sdk/src/attempt-output.tsnew file mode 100644index 00000000..0b7cacd6--- /dev/null+++ b/harness/typescript-sdk/src/attempt-output.ts@@ -0,0 +1,55 @@+import type { RunEvent } from "./types.js";++export interface AttemptView {+ sessionId: string;+ turnId: string | null;+ attemptId: string;+ status: "running" | "completed" | "failed" | "interrupted";+ text: string;+ reasoning: string;+}++/** UI projection for one run. Failed candidates disappear when their attempt+ * fails; replayed events are ignored. This never changes the durable transcript.+ * Render text with textContent/escaped text, never as HTML. */+export class AttemptOutput {+ private readonly rows = new Map<string, AttemptView>();+ private readonly cursors = new Map<string, number>();+ constructor(private readonly maxAttempts = 128, private readonly maxChars = 262144) {+ if (!Number.isSafeInteger(maxAttempts) || maxAttempts < 1 || maxAttempts > 1024 ||+ !Number.isSafeInteger(maxChars) || maxChars < 1 || maxChars > 1048576)+ throw new Error("Invalid attempt display limits");+ }+ apply(item: RunEvent): void {+ const { event, sessionId } = item;+ if (event.event_id <= (this.cursors.get(sessionId) ?? -1)) return;+ this.cursors.set(sessionId, event.event_id);+ const summary = item.summary;+ if (!summary || !("attemptId" in summary)) return;+ const key = JSON.stringify([sessionId, event.turn_id, summary.attemptId]);+ let row = this.rows.get(key);+ if (!row) {+ row = { sessionId, turnId:event.turn_id, attemptId:summary.attemptId,+ status:"running", text:"", reasoning:"" };+ this.rows.set(key, row);+ while (this.rows.size > this.maxAttempts) this.rows.delete(this.rows.keys().next().value!);+ }+ if (summary.type === "model_attempt_finished") {+ row.status = summary.status;+ if (row.status !== "completed") { row.text = ""; row.reasoning = ""; }+ } else if (row.status === "running") {+ if (summary.type === "assistant_delta") row.text = (row.text + summary.text).slice(0, this.maxChars);+ if (summary.type === "reasoning_delta") row.reasoning = (row.reasoning + summary.text).slice(0, this.maxChars);+ }+ // A cancelled/interrupted turn may end before an attempt-finished event.+ // Call interrupt() when the run terminates without completing.+ }+ interrupt(): void {+ for (const row of this.rows.values()) if (row.status === "running") {+ row.status = "interrupted"; row.text = ""; row.reasoning = "";+ }+ }+ get attempts(): readonly AttemptView[] { return [...this.rows.values()].map((row) => ({...row})); }+ get text(): string { return [...this.rows.values()].filter((row) => row.status === "running" || row.status === "completed").map((row) => row.text).join(""); }+ get reasoning(): string { return [...this.rows.values()].filter((row) => row.status === "running" || row.status === "completed").map((row) => row.reasoning).join(""); }+}diff --git a/harness/typescript-sdk/src/index.ts b/harness/typescript-sdk/src/index.tsindex ac261868..f8fe7e6c 100644--- a/harness/typescript-sdk/src/index.ts+++ b/harness/typescript-sdk/src/index.ts@@ -3,6 +3,7 @@ export * from "./safety.js";import { Service, type ServiceOptions } from "./service.js";export * from "./service.js";export * from "./coding.js";+export * from "./attempt-output.js";import { randomUUID } from "node:crypto";import { isAbsolute } from "node:path";import { Transport } from "./transport.js";@@ -12,6 +13,7 @@ import {ToolImageResult,checkMemberErrors,record,+ providerDiagnostic,HarnessError,type HarnessOptions,type SessionRef,@@ -70,6 +72,7 @@ function resultOf(value: unknown): RunResult {interruptions: v.interruptions.map(interruptionOf),submissionId: v.submission_id as string | null,failure: typeof v.failure === "string" ? v.failure : undefined,+ providerError: providerDiagnostic(v.provider_error),cursor: Number(v.cursor),board: (v.board ?? null) as Json,};@@ -393,6 +396,7 @@ export class Harness {base_url: provider.baseUrl,stream_usage: provider.streamUsage,prompt_caching: provider.promptCaching,+ thinking: provider.thinking,access_token_auth: provider.auth ? true : undefined,credentials: credentials? { api_key: apiKey, storage: credentials.storage ?? "secure" }@@ -1061,6 +1065,22 @@ export class RunStream implements AsyncIterable<RunEvent> {const callId = wireSummary.call_id ?? record(event.data).call_id;summary = { ...summary, callId: typeof callId === "string" ? callId : "" };}+ if (summary && ["model_attempt_started", "model_attempt_finished", "assistant_delta", "reasoning_delta", "model_activity", "provider_error"].includes(summary.type)) {+ const wire = record(item.summary);+ const attemptId = String(wire.attempt_id ?? "");+ switch (summary.type) {+ case "model_attempt_started": summary = { type:summary.type, attemptId, attempt:Number(wire.attempt) }; break;+ case "model_attempt_finished": summary = { type:summary.type, attemptId, status:wire.status as "completed" | "failed" | "interrupted", failure:typeof wire.kind === "string" ? wire.kind as import("./types.js").ProviderFailure : undefined }; break;+ case "assistant_delta": summary = { type:summary.type, attemptId, text:String(wire.text ?? "") }; break;+ case "reasoning_delta": summary = { type:summary.type, attemptId, kind:wire.kind as "summary" | "thinking", text:String(wire.text ?? ""), blockIndex:Number(wire.block_index) }; break;+ case "model_activity": summary = { type:summary.type, attemptId, elapsedMs:Number(wire.elapsed_ms) }; break;+ case "provider_error": {+ const diagnostic = providerDiagnostic(wire.diagnostic);+ summary = diagnostic ? { type:summary.type, attemptId, diagnostic } : null;+ break;+ }+ }+ }yield {agent: String(item.agent),...(typeof item.agent_id === "string"diff --git a/harness/typescript-sdk/src/transport.ts b/harness/typescript-sdk/src/transport.tsindex 0e6ab92e..294ef5c8 100644--- a/harness/typescript-sdk/src/transport.ts+++ b/harness/typescript-sdk/src/transport.ts@@ -10,6 +10,7 @@ import {HarnessOptions,ResolvedHarnessOptions,record,+ providerDiagnostic,type ConnectionState,} from "./types.js";@@ -322,10 +323,12 @@ export class Transport {clearTimeout(p.timer);if (frame.error) {// Do not reflect untrusted diagnostics, tool arguments or credentials in errors.+ const diagnostic = providerDiagnostic(frame.error.provider_error);p.reject(new HarnessError("Remote",- "Harness rejected the request; inspect its configuration or durable session state",+ diagnostic ? `Provider ${diagnostic.category}${diagnostic.httpStatus ? ` (HTTP ${diagnostic.httpStatus})` : ""}` : "Harness rejected the request; inspect its configuration or durable session state",+ diagnostic ? { providerError: diagnostic } : undefined,),);} else p.resolve(frame.result);diff --git a/harness/typescript-sdk/src/types.ts b/harness/typescript-sdk/src/types.tsindex 9170db7a..95833aa1 100644--- a/harness/typescript-sdk/src/types.ts+++ b/harness/typescript-sdk/src/types.ts@@ -49,6 +49,9 @@ export interface ModelProvider {streamUsage?: boolean;/** Anthropic automatic prompt caching, enabled by default. False omits cache_control. */promptCaching?: boolean;+ /** Anthropic generation mode. Choose a mode supported by the configured model.+ * Display requires per-turn output.reasoning; omission preserves existing settings. */+ thinking?: { type: "disabled" } | { type: "adaptive" } | { type: "enabled"; budgetTokens: number };/** Memory-only bearer access tokens; mutually exclusive with credentials. */auth?: ProviderAuth;credentials?: (@@ -242,6 +245,7 @@ export interface RunResult {interruptions: Interruption[];submissionId: string | null;failure?: string;+ providerError?: ProviderDiagnostic;cursor: number;board: Json | null;}@@ -309,6 +313,12 @@ export interface RunEvent {summary:| (| { type: "message"; text: string }+ | { type: "model_attempt_started"; attemptId: string; attempt: number }+ | { type: "model_attempt_finished"; attemptId: string; status: "completed" | "failed" | "interrupted"; failure?: ProviderFailure }+ | { type: "assistant_delta"; attemptId: string; text: string }+ | { type: "reasoning_delta"; attemptId: string; kind: "summary" | "thinking"; text: string; blockIndex: number }+ | { type: "model_activity"; attemptId: string; elapsedMs: number }+ | { type: "provider_error"; attemptId: string; diagnostic: ProviderDiagnostic }| {type: "tool_started" | "tool_finished";tool: string;@@ -377,6 +387,7 @@ export class HarnessError extends Error {sessionId?: string;submissionId?: string;requestId?: string;+ providerError?: ProviderDiagnostic;},) {super(message);@@ -384,6 +395,31 @@ export class HarnessError extends Error {}}+export type ProviderFailure = "rate_limit" | "transient" | "transport" | "incomplete_stream" | "invalid_response" | "context_overflow" | "truncated" | "authentication" | "configuration" | "safety_blocked";+export interface ProviderDiagnostic {+ category: ProviderFailure;+ retryable: boolean;+ httpStatus?: number;+ requestId?: string;+ api?: "responses" | "chat-completions" | "anthropic";+ /** Fixed parser code or allowlisted upstream code, not upstream message text. */+ code?: string;+}+/** Parse only the structured safe fields; never reflect arbitrary remote text. */+export function providerDiagnostic(value: unknown): ProviderDiagnostic | undefined {+ if (!value || typeof value !== "object" || Array.isArray(value)) return undefined;+ const v = value as Record<string, unknown>;+ const categories = ["rate_limit", "transient", "transport", "incomplete_stream", "invalid_response", "context_overflow", "truncated", "authentication", "configuration", "safety_blocked"];+ if (!categories.includes(String(v.category)) || typeof v.retryable !== "boolean") return undefined;+ return {+ category: v.category as ProviderFailure, retryable: v.retryable,+ ...(Number.isInteger(v.http_status) && Number(v.http_status) >= 100 && Number(v.http_status) <= 599 ? { httpStatus: Number(v.http_status) } : {}),+ ...(typeof v.request_id === "string" && /^[a-zA-Z0-9_:-]{1,128}$/.test(v.request_id) && !v.request_id.startsWith("sk-") ? { requestId: v.request_id } : {}),+ ...(["responses", "chat-completions", "anthropic"].includes(String(v.api)) ? { api: v.api as ProviderDiagnostic["api"] } : {}),+ ...(["invalid_api_key","authentication_error","permission_error","invalid_request_error","rate_limit_exceeded","rate_limit_error","insufficient_quota","context_length_exceeded","server_error","api_error","overloaded_error","misalignment_policy_violation","RESOURCE_EXHAUSTED","INVALID_ARGUMENT","UNAUTHENTICATED","PERMISSION_DENIED","http_error_body_unavailable","invalid_response","incomplete_stream","invalid_stream_json","invalid_anthropic_message","invalid_anthropic_request","invalid_chat_completion","invalid_chat_request_history","invalid_tool_call_index"].includes(String(v.code)) ? {code:String(v.code)} : {}),+ };+}+export function record(v: unknown): Record<string, unknown> {if (!v || typeof v !== "object" || Array.isArray(v))throw new HarnessError("Protocol", "Expected a protocol object");diff --git a/harness/typescript-sdk/test/attempt-output.test.mjs b/harness/typescript-sdk/test/attempt-output.test.mjsnew file mode 100644index 00000000..bddd5a48--- /dev/null+++ b/harness/typescript-sdk/test/attempt-output.test.mjs@@ -0,0 +1,35 @@+import test from "node:test";+import assert from "node:assert/strict";+import { AttemptOutput, providerDiagnostic } from "../dist/index.js";+function item(id, type, data = {}, session = "s") {+ return { sessionId:session, agent:"guide", event:{event_id:id,session_id:session,turn_id:"t",type,data:{}}, summary:{type,attemptId:"a",...data} };+}+test("attempt projection replaces failed candidates and deduplicates replay",()=>{+ const view=new AttemptOutput();+ view.apply(item(1,"model_attempt_started",{attempt:1}));+ view.apply(item(2,"assistant_delta",{text:"partial"}));+ view.apply(item(3,"reasoning_delta",{text:"candidate",kind:"summary",blockIndex:0}));+ assert.equal(view.text,"partial");+ view.apply(item(4,"model_attempt_finished",{status:"failed"}));+ assert.equal(view.text,"");assert.equal(view.reasoning,"");+ view.apply(item(5,"model_attempt_started",{attemptId:"b",attempt:2}));+ const delta=item(6,"assistant_delta",{attemptId:"b",text:"accepted"});+ view.apply(delta);view.apply(delta);+ view.apply(item(7,"model_attempt_finished",{attemptId:"b",status:"completed"}));+ assert.equal(view.text,"accepted");+ view.interrupt();assert.equal(view.text,"accepted");+});+test("attempt projection scopes sessions and drops cancelled provisional output",()=>{+ const view=new AttemptOutput(2,8);+ view.apply(item(1,"assistant_delta",{text:"abcdefghijk"},"one"));+ view.apply(item(1,"assistant_delta",{text:"second"},"two"));+ assert.equal(view.text,"abcdefghsecond");+ view.interrupt();assert.equal(view.text,"");+ assert.throws(()=>new AttemptOutput(0));+});+test("diagnostics expose only safe structured values",()=>{+ assert.deepEqual(providerDiagnostic({category:"authentication",retryable:false,http_status:401,request_id:"req_fixture",api:"anthropic",message:"SECRET",body:"SECRET"}),+ {category:"authentication",retryable:false,httpStatus:401,requestId:"req_fixture",api:"anthropic"});+ assert.equal(providerDiagnostic({category:"arbitrary secret",retryable:true}),undefined);+ assert.equal(providerDiagnostic({category:"transport",retryable:true,request_id:"sk-secret"}).requestId,undefined);+});diff --git a/harness/typescript-sdk/test/stream-events.test.mjs b/harness/typescript-sdk/test/stream-events.test.mjsnew file mode 100644index 00000000..e2a85a81--- /dev/null+++ b/harness/typescript-sdk/test/stream-events.test.mjs@@ -0,0 +1,94 @@+// Actual managed Rust host, synthetic loopback streams, no provider credentials.+import test from "node:test";+import assert from "node:assert/strict";+import { createServer } from "node:http";+import { mkdtemp, rm } from "node:fs/promises";+import { tmpdir } from "node:os";+import { join } from "node:path";+import { Harness, AttemptOutput, guard } from "../dist/index.js";++const options={skip:!process.env.OW_TS_TEST_HOST,timeout:90000};+const frame=value=>`data: ${JSON.stringify(value)}\n\n`;+function stream(api, text, reasoning, complete=true) {+ if(api==="responses") return frame({type:"response.reasoning_summary_text.delta",delta:reasoning,output_index:0,summary_index:0})++ frame({type:"response.output_text.delta",delta:text})++ (complete?frame({type:"response.completed",response:{status:"completed",output:[{type:"reasoning",encrypted_content:"PRIVATE_SIGNATURE"},{type:"message",role:"assistant",content:[{type:"output_text",text}]}],usage:{input_tokens:10,output_tokens:8}}}):"");+ if(api==="chat-completions") {+ const chunk=(delta,finish_reason=null)=>frame({id:"fixture",choices:[{index:0,delta,finish_reason}]});+ return chunk({reasoning_content:reasoning})+chunk({content:text})+(complete?chunk({},"stop")+"data: [DONE]\n\n":"");+ }+ const events=[+ {type:"message_start",message:{id:"fixture",role:"assistant",content:[],usage:{input_tokens:10,output_tokens:0}}},+ {type:"content_block_start",index:0,content_block:{type:"thinking",thinking:""}},+ {type:"content_block_delta",index:0,delta:{type:"thinking_delta",thinking:reasoning}},+ {type:"content_block_delta",index:0,delta:{type:"signature_delta",signature:"PRIVATE_SIGNATURE"}},+ {type:"content_block_stop",index:0},+ {type:"content_block_start",index:1,content_block:{type:"text",text:""}},+ {type:"content_block_delta",index:1,delta:{type:"text_delta",text}},+ ];+ if(complete)events.push({type:"content_block_stop",index:1},{type:"message_delta",delta:{stop_reason:"end_turn"},usage:{output_tokens:8}},{type:"message_stop"});+ return events.map(frame).join("");+}+async function setup(t,api,mode="normal") {+ const root=await mkdtemp(join(tmpdir(),"ow-stream-events-"));+ const requests=[];let serverError;+ const server=createServer(async(req,res)=>{+ try{+ let body="";for await(const part of req){body+=part;assert.ok(body.length<1_000_000);}+ const input=JSON.parse(body);requests.push(input);+ assert.deepEqual(input.tools??[],[]);+ if(mode==="error") {res.writeHead(401,{"x-request-id":"req_fixture"});res.end(JSON.stringify({error:{code:"invalid_api_key",type:"authentication_error",message:"SECRET_UPSTREAM_BODY"}}));return;}+ const failed=mode==="retry"&&requests.length===1;+ res.writeHead(200,{"content-type":"text/event-stream"});+ res.end(stream(api,failed?"FAILED_TEXT":"Accepted answer",failed?"FAILED_REASON":"Display reasoning",!failed));+ }catch(error){serverError=error;res.destroy();}+ });+ await new Promise(resolve=>server.listen(0,"127.0.0.1",resolve));+ let h;t.after(async()=>{await h?.shutdown();server.closeAllConnections();await new Promise(resolve=>server.close(resolve));await rm(root,{recursive:true,force:true});assert.equal(serverError,undefined);});+ h=await Harness.start({executable:process.env.OW_TS_TEST_HOST,stateDir:root,sandbox:"off"});+ await h.setModelProvider({name:"Fixture",api,baseUrl:`http://127.0.0.1:${server.address().port}/v1`,credentials:{apiKey:"SYNTHETIC_KEY_SENTINEL",storage:"memory"},...(api==="anthropic"?{thinking:{type:"adaptive"}}:{})});+ const agent=h.createAgent({name:"guide",instructions:"Give a hint.",tools:[],memory:false,planning:false,scheduling:false,defaultModel:{provider:"Fixture",name:"fixture"}});+ return {h,agent,requests};+}+for(const api of ["responses","chat-completions","anthropic"]){+ test(`reasoning is separate, opt-in and suppressed by output guardrails (${api})`,options,async t=>{+ const {agent,requests}=await setup(t,api);+ for(const [enabled,protectedOutput] of [[false,false],[true,false],[true,true]]){+ const session=await agent.newSession();+ const run=await session.send("Explain",{output:{reasoning:enabled},...(protectedOutput?{guardrails:{modelOutput:[guard.regex({name:"protect",pattern:"forbidden"})]}}:{})});+ const events=[];for await(const item of run.events())events.push(item);+ assert.equal((await run.result()).status,"completed");+ const reasoning=events.filter(e=>e.summary?.type==="reasoning_delta");+ assert.equal(reasoning.length,enabled&&!protectedOutput?1:0);+ if(reasoning.length){assert.equal(reasoning[0].summary.text,"Display reasoning");assert.ok(reasoning[0].summary.attemptId);}+ assert.ok(!JSON.stringify(events).includes("PRIVATE_SIGNATURE"));+ if(!enabled||protectedOutput)assert.ok(!JSON.stringify(events).includes("Display reasoning"));+ const body=requests.at(-1);+ if(api==="responses")assert.deepEqual(body.reasoning,enabled&&!protectedOutput?{summary:"auto"}:undefined);+ if(api==="anthropic")assert.deepEqual(body.thinking,{type:"adaptive"});+ }+ });+ for(const buffered of [false,true])test(`failed attempts are replaceable and optionally buffered (${api}, ${buffered})`,options,async t=>{+ const {agent,requests}=await setup(t,api,"retry");+ const run=await (await agent.newSession()).send("Explain",{output:{reasoning:true,bufferAttempts:buffered}});+ const projection=new AttemptOutput();const events=[];+ for await(const item of run.events()){projection.apply(item);events.push(item);}+ assert.equal((await run.result()).status,"completed");assert.equal(requests.length,2);+ assert.equal(projection.text,"Accepted answer");assert.equal(projection.reasoning,"Display reasoning");+ assert.equal(events.filter(e=>e.summary?.type==="model_attempt_started").length,2);+ assert.deepEqual(events.filter(e=>e.summary?.type==="model_attempt_finished").map(e=>e.summary.status),["failed","completed"]);+ assert.equal(JSON.stringify(events).includes("FAILED_TEXT"),!buffered);+ assert.equal(JSON.stringify(events).includes("FAILED_REASON"),!buffered);+ });+ test(`safe provider diagnostics survive result and probe errors (${api})`,options,async t=>{+ const {h,agent,requests}=await setup(t,api,"error");+ const run=await (await agent.newSession()).send("Explain");+ const events=[];for await(const item of run.events())events.push(item);+ const result=await run.result();assert.equal(result.status,"failed");+ assert.deepEqual(result.providerError,{category:"authentication",retryable:false,httpStatus:401,requestId:"req_fixture",api,code:"invalid_api_key"});+ const error=events.find(e=>e.summary?.type==="provider_error");assert.deepEqual(error.summary.diagnostic,result.providerError);+ await assert.rejects(h.testModelProvider({provider:"Fixture",name:"fixture"}),e=>e.code==="Remote"&&e.context?.providerError?.httpStatus===401);+ assert.equal(requests.length,2);+ assert.ok(!JSON.stringify([events,result]).includes("SECRET_UPSTREAM_BODY"));assert.ok(!JSON.stringify([events,result]).includes("SYNTHETIC_KEY_SENTINEL"));+ });+}