Python
fix(sdk): qualify provider flows and graceful budget summaries
Rohit P0f488afa6265
diff --git a/.github/workflows/rust-sdk.yml b/.github/workflows/rust-sdk.ymlindex 5e58b7c9..9df4a294 100644--- a/.github/workflows/rust-sdk.yml+++ b/.github/workflows/rust-sdk.yml@@ -227,8 +227,7 @@ jobs:npm --prefix typescript-sdk testnpm --prefix typescript-sdk run test:integrationnpm --prefix typescript-sdk run test:dynamic-agents- node --test --test-reporter=tap typescript-sdk/test/chat-usage.test.mjs typescript-sdk/test/openrouter-streaming.test.mjs- npm --prefix typescript-sdk run test:session-options+ node --test --test-concurrency=2 --test-reporter=tap typescript-sdk/test/chat-usage.test.mjs typescript-sdk/test/openrouter-streaming.test.mjs typescript-sdk/test/anthropic-caching.test.mjs typescript-sdk/test/gemini-compat.test.mjs typescript-sdk/test/session-options.test.mjs typescript-sdk/test/stream-events.test.mjsnode --test --test-reporter=tap typescript-sdk/test/desktop-api.test.mjs typescript-sdk/test/agent-team.test.mjsnode --test --test-reporter=tap typescript-sdk/test/coding.test.mjs typescript-sdk/test/assistant.test.mjs- name: Starter and partner packaging@@ -285,5 +284,6 @@ jobs:run: |cargo test --locked -p openworker-runtime -- --test-threads=1cargo test --locked -p openworker-telemetry --features otlp+ cargo test --locked -p openworker-store-postgres --test store imported_history_and_turn_context_ -- --ignored --test-threads=1- name: Shared PostgreSQL team contractrun: cargo test --locked -p openworker-store-postgres --features test-support --test team_contract --test agent_team_contract --test agent_team_worker -- --ignored --test-threads=1diff --git a/docs/harness/guides/releases.md b/docs/harness/guides/releases.mdindex f43338dd..195d3d7f 100644--- a/docs/harness/guides/releases.md+++ b/docs/harness/guides/releases.md@@ -32,5 +32,18 @@ Protocol negotiation does not make arbitrary SDK and host releases interchangeabThe application controls when it upgrades; the harness does not replace its runningexecutable automatically.+## Session and streaming API updates++The new turn options, history import, built-in tool controls and provider settings+require rebuilding the native harness alongside the SDK. They are not enabled by+updating a TypeScript declaration file alone. Existing sessions and omitted+configuration keep their previous tool selection and per-call limits.++Rust applications using explicit `TurnLimits` or `TurnOptions` literals should+include the new fields or use `..Default::default()`. Tool event variants now+carry call and turn IDs, and there are new attempt/reasoning/diagnostic variants;+update exhaustive matches, or use `..` when ignoring additional fields. See+[session options](sessions.md) and [streaming and retries](streaming.md) for examples.+See [troubleshooting](troubleshooting.md) for runtime discovery, version mismatchand storage errors.diff --git a/docs/harness/guides/sessions.md b/docs/harness/guides/sessions.mdindex 0cb8e45c..6a430f99 100644--- a/docs/harness/guides/sessions.md+++ b/docs/harness/guides/sessions.md@@ -14,7 +14,8 @@ unknown, inspect its status before retrying it.Use `instructionAddendum` in TypeScript, or `TurnOptions.instruction_addendum` inRust, for application-owned preferences that change between turns. The addendumaccompanies every model call and retry in that turn, survives recovery, and is-checked by model-input guardrails/hooks. It is stored with the turn rather than+included in the assembled request inspected by before-model hooks. User-input+guardrails apply to the user message, not this application-owned field. It is stored with the turn rather thanas user transcript text. It expires for the next turn. Keep user-supplied text inthe user message; an addendum does not grant tools, approvals or sandbox access.@@ -69,6 +70,11 @@ provider requires signed reasoning metadata to replay an old tool call, resumethe original harness session or import a text-only conversation summary instead.An import never restores permission grants or an in-progress operation.+Import validates structure and bounds; it does not retroactively run user-input+guardrails or tool-result hooks over the old transcript. The application must+sanitize migrated content before import. Before-model hooks can inspect the+assembled request, including imported history, before the next model call.+## Approvals and questionsRender durable approval/question interruptions in your UI, then send the user'sdiff --git a/docs/harness/guides/streaming.md b/docs/harness/guides/streaming.mdindex c247407e..075fdac2 100644--- a/docs/harness/guides/streaming.md+++ b/docs/harness/guides/streaming.md@@ -78,11 +78,21 @@ await harness.setModelProvider({Rust uses `ModelProvider::anthropic("Claude").thinking(AnthropicThinking::Adaptive)`or `AnthropicThinking::Enabled { budget_tokens: 2048 }`. An enabled budget must-be at least 1,024 tokens and smaller than the current call's output limit. These-settings apply to main agent calls, not short connection probes, compactors or-reviewers. Omitted settings preserve an existing choice; explicitly use-`{ type: "disabled" }` to disable thinking generation. Provider configuration-persists across restart and credential rotation.+be at least 1,024 tokens. The configured value is a ceiling: the harness clamps+it below the current call's output limit. If the remaining allowance cannot fit+the minimum thinking budget plus an answer token, the request omits that manual+thinking configuration. Final turn-limit summaries also omit the agent's thinking+configuration so the small summary reserve is not rejected by a larger manual+budget. Prior thinking/signature blocks are still replayed unchanged.++These settings apply to working-agent calls, not connection probes, compactors,+reviewers or final turn-limit summaries. Omitting a request setting uses the+provider/model default; it does not guarantee thinking is disabled on models where+thinking is always on. Output limits remain hard ceilings. Omitted settings during+provider registration preserve an existing choice. Use `{ type: "disabled" }`+only for models that support it; some adaptive-only models reject that mode.+Provider configuration persists across restart and credential rotation. See+[Anthropic's model-specific thinking rules](https://platform.claude.com/docs/en/build-with-claude/thinking-troubleshooting).Output guardrails and blocking output/tool hooks suppress candidate reasoning aswell as candidate answer text, including for agents without tools. Displaydiff --git a/harness/runtime/src/anthropic.rs b/harness/runtime/src/anthropic.rsindex 4d30a010..37e86be4 100644--- a/harness/runtime/src/anthropic.rs+++ b/harness/runtime/src/anthropic.rs@@ -29,18 +29,22 @@ impl Thinking {}Ok(())}- pub fn request(&self, max_tokens: usize) -> Result<Value, Error> {+ /// Output limits take precedence over manual thinking allowances. Return+ /// None below the manual minimum rather than sending an invalid request or+ /// forcing `disabled`, which some adaptive-only models reject.+ pub fn request(&self, max_tokens: usize) -> Result<Option<Value>, Error> {self.validate().map_err(|_| config())?;- Ok(match self {+ Ok(Some(match self {Self::Disabled => json!({"type":"disabled"}),Self::Adaptive => json!({"type":"adaptive"}),Self::Enabled { budget_tokens } => {- if *budget_tokens as usize >= max_tokens {- return Err(config());+ let budget_tokens = (*budget_tokens as usize).min(max_tokens.saturating_sub(1));+ if budget_tokens < 1024 {+ return Ok(None);}json!({"type":"enabled","budget_tokens":budget_tokens})}- })+ }))}}diff --git a/harness/runtime/src/model.rs b/harness/runtime/src/model.rsindex 6f0151bf..966e8bd9 100644--- a/harness/runtime/src/model.rs+++ b/harness/runtime/src/model.rs@@ -12,6 +12,9 @@ pub struct Model {pub content_withheld: bool,/// Opt-in forwarding of provider-supported display reasoning, never signatures.pub stream_reasoning: bool,+ /// A final turn-limit summary uses its small reserved output allowance,+ /// without inheriting the working agent's thinking-generation configuration.+ pub budget_summary: bool,pub connection: Option<(std::sync::Arc<crate::connections::Manager>,crate::connections::Selection,@@ -98,6 +101,7 @@ impl Model {observation: None,content_withheld: false,stream_reasoning: false,+ budget_summary: false,connection: None,#[cfg(test)]sample_hook: None,@@ -421,15 +425,19 @@ impl Model {}// Short probes, reviewers and compactors have separate small// budgets; do not apply the agent's thinking generation budget.- if self- .observation- .as_ref()- .is_some_and(|c| c.purpose == crate::telemetry::ModelPurpose::Main)+ if !self.budget_summary+ && self+ .observation+ .as_ref()+ .is_some_and(|c| c.purpose == crate::telemetry::ModelPurpose::Main){if let Some((manager, selection)) = &self.connection {if let Some(thinking) = manager.thinking(selection)? {- body["thinking"] =- thinking.request(max_tokens).map_err(|e| e.with_api(api))?;+ if let Some(request) =+ thinking.request(max_tokens).map_err(|e| e.with_api(api))?+ {+ body["thinking"] = request;+ }}}}diff --git a/harness/runtime/src/runtime.rs b/harness/runtime/src/runtime.rsindex 17deb9b0..b5ee452c 100644--- a/harness/runtime/src/runtime.rs+++ b/harness/runtime/src/runtime.rs@@ -570,7 +570,8 @@ impl Runtime {feedback: Option<&str>,) -> Result<crate::provider::Completion> {let started = tokio::time::Instant::now();- for number in 1..=if summary { 1 } else { 3 } {+ let max_attempts = if summary { 1 } else { 3 };+ for number in 1..=max_attempts {if started.elapsed().as_secs() >= 240 {return Err(crate::provider::Error::new(crate::provider::Failure::Transport).into(),@@ -596,7 +597,8 @@ impl Runtime {.any(|h| h.when == crate::safety::HookPhase::BeforeTool);model.content_withheld = guarded_output;let output_options = &s.turn.as_ref().context("No current turn")?.output;- model.stream_reasoning = output_options.reasoning && !guarded_output;+ model.budget_summary = summary;+ model.stream_reasoning = output_options.reasoning && !guarded_output && !summary;let buffer_attempts = output_options.buffer_attempts;let show_reasoning = model.stream_reasoning;let mut instructions = self.bundle.instructions_for(&s.skills, &s.loaded_skills);@@ -911,7 +913,7 @@ impl Runtime {},)?;}- if number == 3+ if number == max_attempts|| !failure.is_some_and(|e| e.retryable())|| started.elapsed().as_secs() >= 240{diff --git a/harness/runtime/src/runtime_budget_tests.rs b/harness/runtime/src/runtime_budget_tests.rsindex c2aceac5..5a44c440 100644--- a/harness/runtime/src/runtime_budget_tests.rs+++ b/harness/runtime/src/runtime_budget_tests.rs@@ -67,6 +67,170 @@ fn model(f: impl Fn(usize, String) -> Value + Send + Sync + 'static, output: u64m}+#[tokio::test]+async fn budget_summary_transport_failure_returns_error_without_retry_or_panic() {+ let calls = Arc::new(AtomicUsize::new(0));+ let seen = calls.clone();+ let mut m = Model::new("summary-error".into(), true);+ m.sample_hook = Some(Arc::new(move |_, _, _| {+ seen.fetch_add(1, Ordering::SeqCst);+ Box::pin(async {+ Err(crate::provider::Error::new(crate::provider::Failure::Transport).into())+ })+ }));+ let (_tmp, rt, sid) = fixture(m, None).await;+ let history = rt.store.lock().unwrap().get(&sid).unwrap().history;+ let error = rt.sample_mode(&sid, &history, &[], true).await.unwrap_err();+ assert_eq!(+ error.downcast_ref::<crate::provider::Error>().unwrap().kind,+ crate::provider::Failure::Transport+ );+ assert_eq!(calls.load(Ordering::SeqCst), 1);+ let events = rt.store.lock().unwrap().events(&sid, 0).unwrap();+ assert!(!events.iter().any(|e| e.kind == "model_retrying"));+ assert_eq!(+ events+ .iter()+ .filter(|e| e.kind == "model_attempt_finished" && e.data["status"] == "failed")+ .count(),+ 1+ );+}++#[tokio::test]+async fn anthropic_work_and_summary_requests_respect_remaining_budget_and_keep_replay_blocks() {+ use tokio::io::{AsyncReadExt, AsyncWriteExt};+ for total in [2048, 1024] {+ let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();+ let endpoint = format!("http://{}/v1", listener.local_addr().unwrap());+ let server = tokio::spawn(async move {+ for index in 0..2 {+ let (mut socket, _) = listener.accept().await.unwrap();+ let mut raw = vec![];+ let mut chunk = [0; 4096];+ let body: Value = loop {+ let n = socket.read(&mut chunk).await.unwrap();+ assert!(n > 0);+ raw.extend_from_slice(&chunk[..n]);+ assert!(raw.len() < 1_000_000);+ if let Some(end) = raw.windows(4).position(|w| w == b"\r\n\r\n") {+ let headers = String::from_utf8_lossy(&raw[..end]).to_ascii_lowercase();+ let length: usize = headers+ .lines()+ .find_map(|l| l.strip_prefix("content-length: "))+ .unwrap()+ .parse()+ .unwrap();+ if raw.len() >= end + 4 + length {+ break serde_json::from_slice(&raw[end + 4..end + 4 + length]).unwrap();+ }+ }+ };+ if index == 0 {+ assert_eq!(body["max_tokens"], total - 512);+ if total == 2048 {+ assert_eq!(+ body["thinking"],+ json!({"type":"enabled","budget_tokens":1535})+ );+ } else {+ assert!(body.get("thinking").is_none());+ }+ } else {+ assert_eq!(body["max_tokens"], 512);+ assert!(+ body.get("thinking").is_none(),+ "Reserved summary must not inherit manual thinking budget"+ );+ assert!(body+ .get("tools")+ .is_none_or(|v| v.as_array().is_some_and(Vec::is_empty)));+ let prior = body["messages"]+ .as_array()+ .unwrap()+ .iter()+ .flat_map(|m| m["content"].as_array().into_iter().flatten())+ .find(|b| b["type"] == "thinking")+ .unwrap();+ assert_eq!(+ prior,+ &json!({"type":"thinking","thinking":"PRIVATE_REPLAY","signature":"signature-retained"})+ );+ }+ let mut events = vec![+ json!({"type":"message_start","message":{"id":format!("r-{index}"),"role":"assistant","content":[],"usage":{"input_tokens":20,"output_tokens":0}}}),+ ];+ if index == 0 {+ events.push(json!({"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":"PRIVATE_REPLAY","signature":"signature-retained"}}));+ events.push(json!({"type":"content_block_stop","index":0}));+ events.push(json!({"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"bounded-plan","name":"update_plan","input":{"expected_revision":0,"explanation":"Plan","steps":[{"id":"work","description":"Find a name","status":"in_progress"}]}}}));+ events.push(json!({"type":"content_block_stop","index":1}));+ } else {+ events.push(json!({"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}));+ events.push(json!({"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Stopped at the configured limit; name lookup is incomplete."}}));+ events.push(json!({"type":"content_block_stop","index":0}));+ }+ events.push(json!({"type":"message_delta","delta":{"stop_reason":if index==0 {"tool_use"} else {"end_turn"}},"usage":{"output_tokens":10}}));+ events.push(json!({"type":"message_stop"}));+ let stream = events+ .iter()+ .map(|e| format!("data: {e}\n\n"))+ .collect::<String>();+ socket.write_all(format!("HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{stream}",stream.len()).as_bytes()).await.unwrap();+ }+ });+ let (_tmp, rt, sid) = fixture(+ Model::new("claude-fixture".into(), false)+ .with_provider(crate::provider::Kind::Anthropic),+ Some(Limits {+ max_model_calls: 1,+ max_output_tokens: total as u64,+ summary_tokens: 512,+ ..Default::default()+ }),+ )+ .await;+ rt.connections.set_provider(json!({"name":"Fixture","api":"anthropic","base_url":endpoint,"credentials":{"api_key":"fixture-key","storage":"memory"},"thinking":{"type":"enabled","budgetTokens":4096}})).unwrap();+ let selection = rt+ .connections+ .resolve(&crate::connections::Choice::new(+ "Fixture",+ "claude-fixture",+ ))+ .unwrap();+ rt.update(&sid, "model_bound", json!({}), |s| {+ s.model_selection = Some(selection);+ Ok(())+ })+ .unwrap();+ tokio::time::timeout(+ std::time::Duration::from_secs(10),+ rt.drive(&sid, Default::default(), vec![crate::planning::schema()]),+ )+ .await+ .unwrap()+ .unwrap();+ tokio::time::timeout(std::time::Duration::from_secs(10), server)+ .await+ .unwrap()+ .unwrap();+ let s = rt.store.lock().unwrap().get(&sid).unwrap();+ assert_eq!(s.turn.as_ref().unwrap().status, "completed");+ assert_eq!(+ s.turn.as_ref().unwrap().budget.stop_reason.as_deref(),+ Some("model_call_limit")+ );+ assert_eq!(+ s.history.last().unwrap()["content"][0]["text"],+ "Stopped at the configured limit; name lookup is incomplete."+ );+ assert!(+ s.plan.is_none(),+ "The work call stopped before its queued tool executed"+ );+ }+}+#[tokio::test]async fn instruction_addendum_applies_to_retries_and_tool_followups_only_in_its_turn() {let count = Arc::new(AtomicUsize::new(0));diff --git a/harness/runtime/src/runtime_safety.rs b/harness/runtime/src/runtime_safety.rsindex 9ca2e994..307e039d 100644--- a/harness/runtime/src/runtime_safety.rs+++ b/harness/runtime/src/runtime_safety.rs@@ -152,12 +152,13 @@ impl Runtime {.check_content(sid, &config, Phase::ModelOutput, text.clone(), None, None).await?;let mut rejected_calls = vec![];- // Google may sign only the first call in a parallel batch. Treat- // every call in that candidate as part of the signed response.- let signed = completion- .output- .iter()- .any(|item| crate::chat_completions::signed_tool_id(item).is_some());+ // Google may sign the first text part or only the first call in a+ // parallel batch. Every call in either candidate belongs to that+ // signed response; no sibling call may be silently rewritten.+ let signed = completion.output.iter().any(|item| {+ crate::chat_completions::signed_tool_id(item).is_some()+ || crate::chat_completions::message_signature(item)+ });if blocked.is_none() {for call in completion.outputdiff --git a/harness/runtime/src/runtime_safety_tests.rs b/harness/runtime/src/runtime_safety_tests.rsindex dde00cf3..a51710e5 100644--- a/harness/runtime/src/runtime_safety_tests.rs+++ b/harness/runtime/src/runtime_safety_tests.rs@@ -93,7 +93,9 @@ fn signed_tool_config(action: Action) -> Config {#[tokio::test]async fn signed_tool_redaction_and_denial_discard_candidate_with_bounded_retry() {for action in [Action::redact(), Action::Block] {- for recovers in [true, false] {+ for (recovers, message_level) in+ [(true, false), (false, false), (true, true), (false, true)]+ {let seen = Arc::new(AtomicUsize::new(0));let count = seen.clone();let mut model = Model::new("test".into(), true);@@ -111,6 +113,16 @@ async fn signed_tool_redaction_and_denial_discard_candidate_with_bounded_retry()// Only the first call is signed; an unsafe parallel sibling// must also cause the entire candidate to be discarded.let mut batch = signed_write("{\"path\":\"safe.txt\",\"content\":\"safe\"}");+ if message_level {+ // Some Google models sign the first text part instead+ // of a function call. The following tools remain part+ // of the same signed candidate.+ batch[0]["block"]["target"] = json!("message");+ batch[0]["block"]["id"] = json!("signed-message");+ let mut text = message("I will write both files.");+ text["id"] = json!("signed-message");+ batch.insert(1, text);+ }batch.push(json!({"type":"function_call","name":"write_file","call_id":"parallel-call","arguments":"{\"path\":\"no-write.txt\",\"content\":\"SECRET_SENTINEL\"}"}));batch};diff --git a/harness/runtime/tests/anthropic.rs b/harness/runtime/tests/anthropic.rsindex c3b0ee49..b990ccd8 100644--- a/harness/runtime/tests/anthropic.rs+++ b/harness/runtime/tests/anthropic.rs@@ -164,7 +164,7 @@ fn thinking_generation_configuration_is_explicit_and_budgeted() {use openworker_runtime::anthropic::Thinking;assert_eq!(Thinking::Adaptive.request(4096).unwrap(),- json!({"type":"adaptive"})+ Some(json!({"type":"adaptive"})));assert_eq!(Thinking::Enabled {@@ -172,13 +172,32 @@ fn thinking_generation_configuration_is_explicit_and_budgeted() {}.request(4096).unwrap(),- json!({"type":"enabled","budget_tokens":1024})+ Some(json!({"type":"enabled","budget_tokens":1024}))+ );+ assert_eq!(+ Thinking::Enabled {+ budget_tokens: 1024+ }+ .request(1024)+ .unwrap(),+ None+ );+ assert_eq!(+ Thinking::Enabled {+ budget_tokens: 4096+ }+ .request(2048)+ .unwrap(),+ Some(json!({"type":"enabled","budget_tokens":2047}))+ );+ assert_eq!(+ Thinking::Enabled {+ budget_tokens: 4096+ }+ .request(1025)+ .unwrap(),+ Some(json!({"type":"enabled","budget_tokens":1024})));- assert!(Thinking::Enabled {- budget_tokens: 1024- }- .request(1024)- .is_err());assert!(Thinking::Enabled { budget_tokens: 5 }.validate().is_err());}#[test]diff --git a/harness/store-postgres/tests/store.rs b/harness/store-postgres/tests/store.rsindex 59171a3e..4774c85c 100644--- a/harness/store-postgres/tests/store.rs+++ b/harness/store-postgres/tests/store.rs@@ -126,6 +126,12 @@ fn imported_history_and_turn_context_survive_postgres_atomicity_and_takeover() {"DROP TRIGGER reject_import_event ON events; DROP FUNCTION reject_import_event();",).unwrap();+ // A database fault deliberately fences this adapter. Reconcile rollback+ // through the independent admin connection above, then reconnect rather+ // than reusing a writer whose outcome was uncertain.+ assert!(owner.get(&s.id).is_err());+ drop(owner);+ let mut owner = PostgresStore::open(config.clone()).unwrap();owner.save(&s, "session_created", json!({})).unwrap();assert!(owner.get(&s.id).unwrap().turn.is_none());let context = Context {diff --git a/harness/typescript-sdk/test/live-tool-conversation.mjs b/harness/typescript-sdk/test/live-tool-conversation.mjsnew file mode 100644index 00000000..444d11b7--- /dev/null+++ b/harness/typescript-sdk/test/live-tool-conversation.mjs@@ -0,0 +1,85 @@+// Opt-in paid smoke test: streamed tool use, accounting, and durable replay.+// Only synthetic data and a local no-side-effect lookup are used.+import assert from "node:assert/strict";+import { randomUUID } from "node:crypto";+import { mkdtemp, rm } from "node:fs/promises";+import { tmpdir } from "node:os";+import { join } from "node:path";+import { Harness } from "../dist/index.js";++if (process.env.OW_RUN_LIVE !== "1") throw new Error("Set OW_RUN_LIVE=1 for paid qualification");+for (const key of ["OW_TS_TEST_HOST", "OW_LIVE_MODEL", "OW_LIVE_API", "OW_LIVE_KEY_ENV"])+ if (!process.env[key]) throw new Error(`${key} is required`);+assert.ok(["responses", "chat-completions", "anthropic"].includes(process.env.OW_LIVE_API));+assert.ok(process.env[process.env.OW_LIVE_KEY_ENV], "Provider key is missing");+if (process.env.OW_LIVE_API === "chat-completions")+ assert.ok(process.env.OW_LIVE_BASE_URL, "Compatible providers require an explicit endpoint");++const stateDir = await mkdtemp(join(tmpdir(), "ow-live-tools-"));+const receipt = `check-${randomUUID()}`;+let harness;+let toolCalls = 0;+async function start() {+ harness = await Harness.start({ executable: process.env.OW_TS_TEST_HOST, stateDir, sandbox: "off" });+ await harness.setModelProvider({+ name: "LiveQualification", api: process.env.OW_LIVE_API,+ ...(process.env.OW_LIVE_BASE_URL ? { baseUrl: process.env.OW_LIVE_BASE_URL } : {}),+ credentials: { apiKeyEnv: process.env.OW_LIVE_KEY_ENV, storage: "memory" },+ ...(process.env.OW_LIVE_API === "anthropic" ? { thinking: { type: "adaptive" } } : {}),+ });+ await harness.registerTools({ namespace: "app", tools: { lookup: {+ description: "Read a synthetic price and receipt for atlas.example. This performs no external action.",+ inputSchema: { type: "object", properties: { name: { type: "string", enum: ["atlas.example"] } }, required: ["name"], additionalProperties: false },+ execute(args) {+ assert.equal(args.name, "atlas.example");+ toolCalls++;+ assert.ok(toolCalls <= 1, "The previously completed lookup must not replay");+ return { name: args.name, priceUsd: 12.34, receipt };+ },+ } } });+ return harness.createAgent({+ name: "tool-qualification", instructions: "Follow the test request precisely. Use app.lookup only when requested. Copy the tool receipt exactly into your final answer. Keep final answers under 60 words.",+ tools: ["app.lookup"], builtinTools: { askUser: false, readHistory: false },+ planning: false, memory: false, scheduling: false,+ approvals: { mode: "only", tools: [] },+ limits: { maxModelCalls: 4, maxOutputTokens: 12288, maxOutputTokensPerCall: 4096, summaryTokens: 256 },+ defaultModel: { provider: "LiveQualification", name: process.env.OW_LIVE_MODEL },+ });+}+async function turn(session, input) {+ const run = await session.send(input, { output: { reasoning: true } });+ const totals = { input: 0, output: 0, usageEvents: 0, reasoningCharacters: 0, textCharacters: 0, attempts: 0 };+ for await (const item of run.events()) {+ if (item.summary?.type === "reasoning_delta") totals.reasoningCharacters += item.summary.text.length;+ if (item.summary?.type === "assistant_delta") totals.textCharacters += item.summary.text.length;+ if (item.summary?.type === "model_attempt_started") totals.attempts++;+ if (item.event.type === "model_usage") {+ totals.usageEvents++;+ totals.input += item.event.data.input_tokens;+ totals.output += item.event.data.output_tokens;+ }+ }+ const result = await run.result();+ if (result.status !== "completed") {+ console.log(JSON.stringify({ status: result.status, providerError: result.providerError, totals, toolCalls }));+ throw new Error("Live conversation did not complete");+ }+ assert.ok(result.finalOutput.includes(receipt), "Final answer must use the actual tool receipt");+ assert.ok(totals.usageEvents > 0 && totals.input > 0 && totals.output > 0, "Provider usage must be reported");+ return totals;+}+try {+ const agent = await start();+ const session = await agent.newSession();+ const first = await turn(session, "Call app.lookup once for atlas.example. Then give its price and exact receipt.");+ assert.equal(toolCalls, 1);+ const reference = session.reference();+ await harness.shutdown();+ const restarted = await start();+ const second = await turn(await restarted.resumeSession(reference), "Repeat the exact receipt from the previous lookup. Do not call any tool.");+ assert.equal(toolCalls, 1, "Restart/history replay must not execute the old call");+ console.log(JSON.stringify({ model: process.env.OW_LIVE_MODEL, api: process.env.OW_LIVE_API, first, afterRestart: second, toolCalls }));+} finally {+ try { await harness?.shutdown(); }+ finally { await rm(stateDir, { recursive: true, force: true }); }+}