Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

fix(llm-client): preserve plaintext Responses reasoning for non-OpenAI targets (#1223)

Graham King17c093196409
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 25da0ddfd..a7a528f9f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -46,6 +46,17 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
that selects escalation mode through its `escalation` table carries capability
keys that mode ignores.
+### Fixed
+
+- **Responses reasoning history reaches non-OpenAI targets** - the default
+ `responses_reasoning` is now `"auto"`. Targets other than OpenAI keep plaintext
+ reasoning `summary` and `content` in place, for any `store` value. Before, it
+ was removed, so models such as GLM lost their earlier reasoning. These targets
+ lose opaque `encrypted_content`, because its issuer is unknown. OpenAI targets
+ keep the previous behavior. **Rust source-breaking change:** callers that match
+ exhaustively on `ResponsesReasoningPolicy` must handle `Auto` and
+ `PreservePlaintext`.
+
## [0.3.0]
Switchyard 0.3.0 builds on the native server and Rust library introduced in
diff --git a/crates/libsy-llm-client/README.md b/crates/libsy-llm-client/README.md
index 837fb3720..23f43b98b 100644
--- a/crates/libsy-llm-client/README.md
+++ b/crates/libsy-llm-client/README.md
@@ -279,11 +279,13 @@ fn build_multi_format_client(
fields, array input, non-Codex endpoints, and the translation codec's exact
preservation remain unchanged.
- `ModelConfig::with_responses_reasoning` controls Responses reasoning replay.
- Every Responses model defaults to `PreserveEncrypted`: plaintext is removed,
- encrypted provider state is retained. Stored reasoning IDs are retained unless
- the request sets `store: false`. Switchyard's translated Anthropic reasoning is
- removed. Set `Drop` for
- a backend that cannot consume provider reasoning. Messages and tool history are retained.
+ The default, `Auto`, chooses per request from the target. OpenAI targets use
+ `PreserveEncrypted`: plaintext is removed and encrypted provider state is
+ retained. Stored reasoning IDs are retained unless the request sets
+ `store: false`. Other targets use `PreservePlaintext`: `summary` and `content`
+ are retained and opaque `encrypted_content` is removed. Switchyard's translated
+ Anthropic reasoning payload is always removed. Set `Drop` for a backend that
+ cannot consume provider reasoning. Messages and tool history are retained.
- `HttpBackendConfig::failure_cooldown` skips completion calls to a backend briefly
after an exhausted transient failure. Zero disables it. State is shared per model
within the client. Calls resume when the cooldown expires. Deployment TOML defaults
diff --git a/crates/libsy-llm-client/src/client.rs b/crates/libsy-llm-client/src/client.rs
index b17652a15..6158f42bf 100644
--- a/crates/libsy-llm-client/src/client.rs
+++ b/crates/libsy-llm-client/src/client.rs
@@ -378,7 +378,7 @@ impl TranslatingLlmClient {
.get(model)
.map(|config| config.responses_reasoning)
.unwrap_or_default()
- .normalize(&mut body);
+ .normalize(&mut body, backend.base_url(), model);
}
// After the merge on purpose: the effort override must win over both the caller's
// value and any `reasoning` default a target set through `extra_body`. Token counting
@@ -1727,13 +1727,6 @@ mod tests {
)]
}
- fn responses_map_with_drop_policy(base_url: &str) -> Vec<ModelConfig> {
- vec![
- ModelConfig::new("gpt", Backend::OpenAiResponses(config(base_url)), None)
- .with_responses_reasoning(crate::ResponsesReasoningPolicy::Drop),
- ]
- }
-
#[tokio::test]
async fn model_properties_uses_configured_auth_and_backend_path()
-> std::result::Result<(), Box<dyn Error + Sync + Send + 'static>> {
@@ -2847,18 +2840,32 @@ mod tests {
#[tokio::test]
async fn responses_reasoning_policy_preserves_replay_history()
-> std::result::Result<(), Box<dyn Error + Sync + Send + 'static>> {
+ use crate::ResponsesReasoningPolicy::{Auto, Drop, PreserveEncrypted, PreservePlaintext};
+ let summary = json!([{"type": "summary_text", "text": "Check the parser."}]);
let history = json!([
{"type": "message", "role": "user", "content": "continue"},
{"type": "reasoning", "encrypted_content": "opaque", "content": [{"type": "reasoning_text", "text": "private"}]},
{"type": "reasoning", "id": "rs_stored", "status": "completed", "summary": []},
- {"type": "reasoning", "encrypted_content": null},
- {"type": "reasoning", "encrypted_content": ""},
+ // GLM's plaintext forms, as Codex replays them.
+ {"type": "reasoning", "id": "rs_glm", "summary": summary, "encrypted_content": null},
+ {"type": "reasoning", "content": [{"type": "reasoning_text", "text": "Run the tests."}], "encrypted_content": ""},
{"type": "reasoning"},
{"type": "function_call", "call_id": "call_1", "name": "shell", "arguments": "{}"},
{"type": "function_call_output", "call_id": "call_1", "output": "ok"}
]);
+ // An explicit policy wins over the target. `Auto` follows the target.
+ let cases = [
+ (Auto, "openai/gpt-5", PreserveEncrypted),
+ (Auto, "nvidia/zai-org/glm-5.3-flash", PreservePlaintext),
+ (
+ PreserveEncrypted,
+ "nvidia/zai-org/glm-5.3-flash",
+ PreserveEncrypted,
+ ),
+ (Drop, "openai/gpt-5", Drop),
+ ];
for store in [Some(true), Some(false), None] {
- for is_drop in [false, true] {
+ for (policy, model, behavior) in cases {
for has_input_override in [false, true] {
let server = MockServer::start().await;
Mock::given(method("POST"))
@@ -2871,24 +2878,20 @@ mod tests {
.expect(1)
.mount(&server)
.await;
- let base_url = format!("{}/v1", server.uri());
- let mut models = if is_drop {
- responses_map_with_drop_policy(&base_url)
- } else {
- responses_map(&base_url)
- };
+ let mut backend = config(&format!("{}/v1", server.uri()));
if has_input_override {
- let Backend::OpenAiResponses(backend) = &mut models[0].default_backend
- else {
- return Err("expected Responses backend".into());
- };
backend.omit_body_fields.insert("input".to_string());
backend
.extra_body
.insert("input".to_string(), history.clone());
}
- let client = TranslatingLlmClient::new(&models)?;
- let mut request = json!({"model": "gpt", "input": history});
+ let client = TranslatingLlmClient::new(&[ModelConfig::new(
+ model,
+ Backend::OpenAiResponses(backend),
+ None,
+ )
+ .with_responses_reasoning(policy)])?;
+ let mut request = json!({"model": "switchyard", "input": history});
if let Some(store) = store {
request["store"] = json!(store);
}
@@ -2896,28 +2899,43 @@ mod tests {
.call_rewrite_model_raw(
request,
None,
- Some(&ModelId::from("gpt")),
+ Some(&ModelId::from(model)),
WireFormat::OpenAiResponses,
)
.await?;
let requests = server.received_requests().await.ok_or("missing requests")?;
let body: Value = serde_json::from_slice(&requests[0].body)?;
+ let stored = store != Some(false);
let mut expected = vec![history[0].clone()];
- if !is_drop {
- let mut encrypted = history[1].clone();
- encrypted["content"] = json!([]);
- expected.push(encrypted);
- if store != Some(false) {
- let mut stored = history[2].clone();
- stored["content"] = json!([]);
- expected.push(stored);
+ match behavior {
+ // OpenAI reads its own payload and stored IDs, but rejects plaintext.
+ PreserveEncrypted => {
+ let indexes: &[usize] = if stored { &[1, 2, 3] } else { &[1] };
+ for &index in indexes {
+ let mut item = history[index].clone();
+ item["content"] = json!([]);
+ expected.push(item);
+ }
+ }
+ // Other targets keep plaintext and lose payloads of unknown origin.
+ PreservePlaintext => {
+ let mut item = history[1].clone();
+ item.as_object_mut()
+ .ok_or("object")?
+ .remove("encrypted_content");
+ expected.push(item);
+ if stored {
+ expected.push(history[2].clone());
+ }
+ expected.extend([history[3].clone(), history[4].clone()]);
}
+ _ => {}
}
expected.extend([history[6].clone(), history[7].clone()]);
assert_eq!(
body["input"],
json!(expected),
- "store={store:?}, drop={is_drop}, replace={has_input_override}"
+ "store={store:?}, policy={policy:?}, model={model}, replace={has_input_override}"
);
}
}
@@ -4011,6 +4029,7 @@ mod tests {
// kinds: items with an ID but no `encrypted_content` when the request sets `store: false`,
// and items whose `encrypted_content` holds Anthropic thinking that Switchyard packed for
// the client. The Responses leg must drop both and keep GPT's own encrypted reasoning.
+ // A non-OpenAI target instead keeps the plaintext and drops both payloads.
#[tokio::test]
async fn openai_responses_backend_drops_reasoning_the_provider_cannot_read()
-> std::result::Result<(), Box<dyn Error + Sync + Send + 'static>> {
@@ -4091,25 +4110,22 @@ mod tests {
empty_reasoning
]);
- for has_input_override in [false, true] {
- let mut models = responses_map(&format!("{}/v1", server.uri()));
- if has_input_override {
- let Backend::OpenAiResponses(backend) = &mut models[0].default_backend else {
- return Err("expected Responses backend".into());
- };
- backend.omit_body_fields.insert("input".to_string());
- backend
- .extra_body
- .insert("input".to_string(), input.clone());
- }
- let client = TranslatingLlmClient::new(&models)?;
+ let models = ["openai/gpt-5", "nvidia/zai-org/glm-5.3-flash"].map(|model| {
+ ModelConfig::new(
+ model,
+ Backend::OpenAiResponses(config(&format!("{}/v1", server.uri()))),
+ None,
+ )
+ });
+ let client = TranslatingLlmClient::new(&models)?;
+ for model in ["openai/gpt-5", "nvidia/zai-org/glm-5.3-flash"] {
for store in [json!(false), json!(true)] {
let raw = json!({"model": "client-facing", "store": store, "input": input.clone()});
client
.call_rewrite_model_raw(
raw,
None,
- Some(&ModelId::from("gpt")),
+ Some(&ModelId::from(model)),
WireFormat::OpenAiResponses,
)
.await?;
@@ -4137,6 +4153,29 @@ mod tests {
"provider cannot read translated Anthropic reasoning"
);
}
+
+ // GLM keeps the plaintext beside the Anthropic envelope and drops GPT's payload.
+ let mut switchyard_plaintext = switchyard_reasoning.clone();
+ switchyard_plaintext
+ .as_object_mut()
+ .ok_or("object")?
+ .remove("encrypted_content");
+ assert_eq!(
+ sent[2]["input"],
+ json!([user, claude_reasoning, switchyard_plaintext, call, output])
+ );
+ assert_eq!(
+ sent[3]["input"],
+ json!([
+ user,
+ claude_reasoning,
+ switchyard_plaintext,
+ call,
+ output,
+ empty_reasoning
+ ])
+ );
+
for item in [
&mut claude_reasoning,
&mut gpt_reasoning,
@@ -4144,26 +4183,21 @@ mod tests {
] {
item["content"] = json!([]);
}
- for requests in sent.as_chunks::<2>().0 {
- // With `store: false`, only GPT's own encrypted reasoning survives.
- assert_eq!(
- requests[0]["input"],
- json!([user, call, output, gpt_reasoning])
- );
- // With `store: true`, IDs can refer to stored reasoning, but OpenAI still cannot
- // decrypt the Switchyard item.
- assert_eq!(
- requests[1]["input"],
- json!([
- user,
- claude_reasoning,
- call,
- output,
- gpt_reasoning,
- empty_reasoning
- ])
- );
- }
+ // With `store: false`, only GPT's own encrypted reasoning survives.
+ assert_eq!(sent[0]["input"], json!([user, call, output, gpt_reasoning]));
+ // With `store: true`, IDs can refer to stored reasoning, but OpenAI still cannot
+ // decrypt the Switchyard item.
+ assert_eq!(
+ sent[1]["input"],
+ json!([
+ user,
+ claude_reasoning,
+ call,
+ output,
+ gpt_reasoning,
+ empty_reasoning
+ ])
+ );
Ok(())
}
diff --git a/crates/libsy-llm-client/src/responses_reasoning.rs b/crates/libsy-llm-client/src/responses_reasoning.rs
index f5ba00d15..0dcc541ab 100644
--- a/crates/libsy-llm-client/src/responses_reasoning.rs
+++ b/crates/libsy-llm-client/src/responses_reasoning.rs
@@ -4,59 +4,176 @@
//! Target-specific replay policy for OpenAI Responses reasoning items.
use serde::Deserialize;
-use serde_json::Value;
+use serde_json::{Map, Value};
/// Controls which Responses reasoning items are replayed to an upstream.
#[derive(Clone, Copy, Debug, Default, Deserialize, PartialEq)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesReasoningPolicy {
- /// Preserve encrypted reasoning and stored IDs the provider can read.
+ /// Choose from the target on each request.
///
- /// This is the default for every Responses model, regardless of its URL.
+ /// OpenAI-hosted targets use [`Self::PreserveEncrypted`]. Every other target uses
+ /// [`Self::PreservePlaintext`].
#[default]
+ Auto,
+ /// Keep encrypted reasoning and stored IDs. Remove plaintext reasoning.
+ ///
+ /// OpenAI rejects plaintext reasoning `content` and unresolvable stored IDs.
PreserveEncrypted,
+ /// Keep plaintext reasoning. Remove opaque encrypted payloads.
+ ///
+ /// Switchyard cannot tell which provider issued an encrypted payload, so it
+ /// removes the payload and keeps any `summary` or `content` beside it.
+ PreservePlaintext,
/// Drop all reasoning items while preserving messages and tool-call history.
///
/// Use this for local Responses-compatible servers that cannot consume
- /// another provider's encrypted reasoning representation.
+ /// another provider's reasoning representation.
Drop,
}
impl ResponsesReasoningPolicy {
- /// Normalizes a Responses request body for this replay policy.
- pub(crate) fn normalize(self, body: &mut Value) {
+ /// Normalizes a Responses request body sent to `model` at `base_url`.
+ pub(crate) fn normalize(self, body: &mut Value, base_url: &str, model: &str) {
+ let policy = match self {
+ Self::Auto if is_openai_target(base_url, model) => Self::PreserveEncrypted,
+ Self::Auto => Self::PreservePlaintext,
+ policy => policy,
+ };
let has_stored_reasoning = body.get("store").and_then(Value::as_bool) != Some(false);
let Some(Value::Array(input)) = body.get_mut("input") else {
return;
};
- input.retain_mut(|item| self.normalize_item(item, has_stored_reasoning));
+ let mut stats = Stats::default();
+ input.retain_mut(|item| match item.as_object_mut() {
+ Some(item) if item.get("type").and_then(Value::as_str) == Some("reasoning") => {
+ stats.items += 1;
+ let keep = policy.keep(item, has_stored_reasoning, &mut stats.removed_payloads);
+ stats.removed_items += usize::from(!keep);
+ keep
+ }
+ _ => true,
+ });
+ if stats.items > 0 {
+ tracing::trace!(
+ policy = ?policy,
+ reasoning_items = stats.items,
+ removed_items = stats.removed_items,
+ removed_payloads = stats.removed_payloads,
+ "normalized Responses reasoning"
+ );
+ }
}
+ // Returns whether to keep a reasoning item, after removing parts the target cannot use.
// Stored reasoning can be replayed by ID without encrypted content.
- fn normalize_item(self, item: &mut Value, has_stored_reasoning: bool) -> bool {
- let Some(object) = item.as_object_mut() else {
- return true;
- };
- if object.get("type").and_then(Value::as_str) != Some("reasoning") {
- return true;
+ fn keep(
+ self,
+ item: &mut Map<String, Value>,
+ has_stored_reasoning: bool,
+ removed_payloads: &mut usize,
+ ) -> bool {
+ let encrypted_content = item
+ .get("encrypted_content")
+ .and_then(Value::as_str)
+ .filter(|content| !content.is_empty());
+ let has_stored_id = has_stored_reasoning
+ && item
+ .get("id")
+ .and_then(Value::as_str)
+ .is_some_and(|id| !id.is_empty());
+ match self {
+ Self::PreserveEncrypted => {
+ if encrypted_content
+ .is_some_and(switchyard_translation::is_anthropic_thinking_envelope)
+ || (encrypted_content.is_none() && !has_stored_id)
+ {
+ return false;
+ }
+ item.insert("content".to_string(), Value::Array(Vec::new()));
+ true
+ }
+ Self::PreservePlaintext => {
+ let has_plaintext = ["summary", "content"].into_iter().any(|field| {
+ item.get(field)
+ .and_then(Value::as_array)
+ .is_some_and(|parts| !parts.is_empty())
+ });
+ if encrypted_content.is_none() {
+ return has_plaintext || has_stored_id;
+ }
+ // The item ID belongs to the payload's issuer, so it cannot stand alone.
+ item.remove("encrypted_content");
+ *removed_payloads += 1;
+ has_plaintext
+ }
+ // `normalize` resolves `Auto` before calling this.
+ Self::Auto | Self::Drop => false,
}
+ }
+}
- let encrypted_content = object.get("encrypted_content").and_then(Value::as_str);
- if encrypted_content.is_some_and(switchyard_translation::is_anthropic_thinking_envelope) {
- return false;
- }
- let has_encrypted_content = encrypted_content.is_some_and(|content| !content.is_empty());
- let has_stored_id = object
- .get("id")
- .and_then(Value::as_str)
- .is_some_and(|id| !id.is_empty());
- if self == Self::PreserveEncrypted
- && (has_encrypted_content || (has_stored_reasoning && has_stored_id))
- {
- object.insert("content".to_string(), Value::Array(Vec::new()));
- true
- } else {
- false
+// Counts for one request. They hold no prompt or reasoning text.
+#[derive(Default)]
+struct Stats {
+ items: usize,
+ removed_items: usize,
+ removed_payloads: usize,
+}
+
+// OpenAI and Azure OpenAI reject plaintext reasoning in Responses input. A shared
+// gateway hostname says nothing about the model behind it, so other hosts are judged
+// by the upstream model name alone.
+fn is_openai_target(base_url: &str, model: &str) -> bool {
+ let host = reqwest::Url::parse(base_url)
+ .ok()
+ .and_then(|url| url.host_str().map(str::to_ascii_lowercase));
+ if host.is_some_and(|host| host == "api.openai.com" || host.ends_with(".openai.azure.com")) {
+ return true;
+ }
+ let name = model
+ .rsplit('/')
+ .next()
+ .unwrap_or(model)
+ .to_ascii_lowercase();
+ let is_o_series = name
+ .strip_prefix('o')
+ .is_some_and(|rest| rest.starts_with(|c: char| c.is_ascii_digit()));
+ (name.starts_with("gpt-") && !name.starts_with("gpt-oss"))
+ || name.starts_with("codex-")
+ || is_o_series
+}
+
+#[cfg(test)]
+mod tests {
+ use super::is_openai_target;
+
+ #[test]
+ fn openai_targets_come_from_host_or_model_name() {
+ let hub = "https://inference-api.nvidia.com/v1";
+ let proxy = "http://inference-proxy.internal:8080/v1";
+ for (base_url, model, expected) in [
+ ("https://api.openai.com/v1", "my-alias", true),
+ (
+ "https://acme.openai.azure.com/openai/v1",
+ "deployment",
+ true,
+ ),
+ (hub, "openai/openai/gpt-5.5", true),
+ (proxy, "azure/openai/gpt-5.6-sol", true),
+ (hub, "openai/o4-mini", true),
+ (proxy, "codex-mini-latest", true),
+ (hub, "nvidia/zai-org/glm-5.3-flash", false),
+ (proxy, "nvidia/zai-org/glm-5.3-flash", false),
+ (hub, "openai/gpt-oss-120b", false),
+ (proxy, "switchyard", false),
+ ("not a url", "deepseek-v4-pro", false),
+ ] {
+ assert_eq!(
+ is_openai_target(base_url, model),
+ expected,
+ "{base_url} {model}"
+ );
}
}
}
diff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rs
index 5adb4c20a..fdf100e36 100644
--- a/crates/switchyard-server/tests/server.rs
+++ b/crates/switchyard-server/tests/server.rs
@@ -6216,11 +6216,25 @@ async fn upstream_headers_forward_on_streaming_responses() -> TestResult {
Ok(())
}
+// Mirrors the GLM baseline: passthrough to a Responses target with a reasoning effort
+// default, called by Codex with `store: false`.
#[tokio::test]
async fn configured_responses_reasoning_policy_reaches_upstream() -> TestResult {
let upstream = MockUpstream::start().await?;
let base_url = upstream.base_url.replace("/v1", "/buffered");
- for (setting, expected_reasoning) in [("", 2), ("responses_reasoning = \"drop\"", 0)] {
+ let plaintext = json!({"type": "reasoning", "id": "rs_1", "summary": [{"type": "summary_text", "text": "plan"}], "encrypted_content": null});
+ let foreign =
+ json!({"type": "reasoning", "id": "rs_2", "summary": [], "encrypted_content": "opaque"});
+ let mut kept_foreign = foreign.clone();
+ kept_foreign["content"] = json!([]);
+ for (setting, expected_reasoning) in [
+ ("", vec![plaintext.clone()]),
+ (
+ "responses_reasoning = \"preserve_encrypted\"",
+ vec![kept_foreign],
+ ),
+ ("responses_reasoning = \"drop\"", vec![]),
+ ] {
let config = format!(
r#"
schema_version = 1
@@ -6228,40 +6242,42 @@ schema_version = 1
format = "openai_responses"
base_url = "{base_url}"
{setting}
-[targets]
-answer = {{ id = "model/efficient", llm_client = "upstream" }}
-[routes.route]
-id = "route"
+[targets.glm]
+id = "model/efficient"
+llm_client = "upstream"
+extra_body = {{ reasoning = {{ effort = "high" }} }}
+[routes.switchyard]
+id = "switchyard"
type = "passthrough"
-target = "answer"
+target = "glm"
"#
);
let app = build_switchyard_router(load_test_config(&config)?);
- let response = send(&app, "POST", "/v1/responses", Some(json!({
- "model": "route", "input": [
- {"type": "message", "role": "user", "content": "continue"},
- {"type": "reasoning", "encrypted_content": "opaque", "summary": []},
- {"type": "reasoning", "id": "rs_stored", "summary": []},
- {"type": "reasoning", "content": [{"type": "reasoning_text", "text": "local"}], "summary": []}
- ]
- }))).await?;
+ let response = send(
+ &app,
+ "POST",
+ "/v1/responses",
+ Some(json!({
+ "model": "switchyard", "store": false, "input": [
+ {"type": "message", "role": "user", "content": "continue"},
+ plaintext,
+ foreign
+ ]
+ })),
+ )
+ .await?;
assert_eq!(response.status, StatusCode::OK, "{}", response.json()?);
let calls = upstream.calls.lock().await;
- let input = calls.last().expect("upstream request")["input"]
+ let body = calls.last().expect("upstream request");
+ let reasoning: Vec<_> = body["input"]
.as_array()
- .expect("input array");
- assert_eq!(
- input
- .iter()
- .filter(|item| item["type"] == "reasoning")
- .count(),
- expected_reasoning
- );
- assert!(
- input
- .iter()
- .any(|item| item["type"] == "message" && item["role"] == "user")
- );
+ .expect("input array")
+ .iter()
+ .filter(|item| item["type"] == "reasoning")
+ .cloned()
+ .collect();
+ assert_eq!(reasoning, expected_reasoning, "{setting}");
+ assert_eq!(body["reasoning"], json!({"effort": "high"}));
}
Ok(())
}
diff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.md
index 370ebea9c..58373a6be 100644
--- a/docs/reference/toml_schema.md
+++ b/docs/reference/toml_schema.md
@@ -54,21 +54,37 @@ route reaches no upstream. A file without a `[targets]` table is rejected with
| `api_key_env` | No | unset | Name of the environment variable holding the key. Omit to send no authentication. |
| `forward_auth` | No | `false` | Forward the caller's provider credential and application headers. A route's forwarding clients must use one credential family unless they all use the same scheme, host, and port, such as one LLM gateway. |
| `extra_headers` | No | `{}` | Custom HTTP headers sent to the model server. Set credentials with `api_key_env` or `forward_auth`; the server rejects headers owned by the selected auth mode. Header names are case-insensitive. |
-| `responses_reasoning` | No | `"preserve_encrypted"` | Responses reasoning replay: `"preserve_encrypted"` or `"drop"`. Only valid for `openai_responses`. |
+| `responses_reasoning` | No | `"auto"` | Responses reasoning replay: `"auto"`, `"preserve_encrypted"`, `"preserve_plaintext"`, or `"drop"`. Only valid for `openai_responses`. |
| `fallback_on_rate_limit` | No | `true` | Try another completion candidate after an upstream HTTP 429 exhausts retries. Set to `false` to return the 429 instead. |
| `max_retries` | No | `2` | Retry budget, `0`–`10`. |
| `failure_cooldown_ms` | No | `5000` (5 seconds) | Skip a backend for this many milliseconds after an exhausted transient completion failure. Zero disables it. |
| `timeout_ms` | No | unset | Deadline in milliseconds for all attempts, retry delays, and the complete response, including stream reads. Must be at least `1`. Unset leaves the wait unbounded. |
For `openai_responses`, `responses_reasoning` controls replay of reasoning
-history. Every client defaults to `"preserve_encrypted"`: reasoning items with
-non-empty `encrypted_content` or a stored reasoning `id` are retained with empty
-`content`. ID-only reasoning is removed when the request sets `store: false`.
-Switchyard's translated Anthropic reasoning is always removed.
-Reasoning without either is removed. Set `responses_reasoning = "drop"` explicitly for a backend
-that cannot consume provider reasoning. Both modes keep messages, tool
-calls and tool results. The setting is rejected on other client formats, even
-when no target uses the client. It is not inferred from the model name or URL.
+history. Most configurations should leave it unset. The values are:
+
+- `"auto"` (default): choose per request from the target that serves it,
+ including fallback targets. Targets on `api.openai.com` or `*.openai.azure.com`
+ use `"preserve_encrypted"`. On other hosts, such as a shared gateway or proxy,
+ the choice comes from the last `/`-separated part of the target `id`. Names that
+ start with `gpt-` (except `gpt-oss`) or `codex-`, and o-series names such as
+ `o3` or `o4-mini`, use `"preserve_encrypted"`. All other targets use
+ `"preserve_plaintext"`.
+- `"preserve_encrypted"`: keep reasoning items with non-empty `encrypted_content`
+ or a stored reasoning `id`, and empty their `content`. ID-only reasoning is
+ removed when the request sets `store: false`. Switchyard's translated Anthropic
+ reasoning and plaintext-only reasoning are removed. OpenAI needs this.
+- `"preserve_plaintext"`: keep `summary` and `content` text unchanged and in place,
+ for any `store` value. Remove non-empty `encrypted_content`, because Switchyard
+ cannot tell which provider issued it. An item left with no text is removed. An
+ ID-only item is kept unless the request sets `store: false`.
+- `"drop"`: remove all reasoning items. Use it for a backend, such as llama.cpp,
+ that rejects reasoning from other providers.
+
+All modes keep messages, tool calls and tool results. Set an explicit value when
+`"auto"` picks the wrong mode, for example an Azure OpenAI deployment with a custom
+name behind a gateway. The setting is rejected on other client formats, even
+when no target uses the client.
The TOML never contains the secret itself. `api_key_env` names a variable that
must exist and be non-empty when the server loads.