Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
fix(libsy): drop empty classifier history turns after reasoning removal (#1273)
eric-liu-nvidia88fe33e323d4
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6b5848c98..dd4a7774f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -66,6 +66,12 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).### Fixed+- **Windowed classifiers drop empty history turns** - with `recent_turn_window`+ set, an assistant turn that holds only reasoning, as an OpenAI Chat message+ with `reasoning_content` and no `content` decodes, is left out of the judge+ request. Before, it reached the judge as an assistant message with empty text,+ and an Anthropic Messages judge rejected the request with HTTP 400, which+ failed the client request.- **Responses reasoning history reaches non-OpenAI targets** - the default`responses_reasoning` is now `"auto"`. Targets other than OpenAI keep plaintextreasoning `summary` and `content` in place, for any `store` value. Before, itdiff --git a/crates/libsy/src/algorithms/llm_class.rs b/crates/libsy/src/algorithms/llm_class.rsindex 10d0be8d4..ff86e89c8 100644--- a/crates/libsy/src/algorithms/llm_class.rs+++ b/crates/libsy/src/algorithms/llm_class.rs@@ -209,11 +209,16 @@ impl TaskInput {// Reasoning is provider-private and not required to classify the task. Some// upstreams also reject an unsigned reasoning item replayed without the// opaque state it was issued with, so it cannot travel through a windowed- // classifier request as ordinary history.+ // classifier request as ordinary history. Empty text goes with it: OpenAI+ // Chat decoding leaves an empty text placeholder beside the reasoning of a+ // reasoning-only turn, and an Anthropic judge rejects a message whose text+ // block is empty.for message in &mut messages {- message- .content- .retain(|block| !matches!(block, ContentBlock::Reasoning { .. }));+ message.content.retain(|block| match block {+ ContentBlock::Reasoning { .. } => false,+ ContentBlock::Text { text } => !text.is_empty(),+ _ => true,+ });}messages.retain(|message| !message.content.is_empty());messages@@ -2395,6 +2400,63 @@ mod tests {assert!(built.iter().all(|message| !message.content.is_empty()));}+ #[test]+ fn a_window_drops_a_turn_left_empty_once_its_reasoning_is_removed() {+ // OpenAI Chat decoding places an empty text placeholder beside the reasoning of+ // an assistant message that carries `reasoning_content` alone. Removing the+ // reasoning has to take the placeholder with it: an Anthropic judge rejects a+ // message whose text is empty, and the whole client request then fails.+ let body = serde_json::json!({+ "model": "qa/windowed",+ "messages": [+ {"role": "user", "content": "Reply with the number four."},+ {"role": "assistant", "reasoning_content": "private chain of thought"},+ {"role": "user", "content": "This is still a simple number lookup."}+ ]+ });+ let request = Request {+ llm_request: switchyard_translation::decode_request(+ switchyard_translation::WireFormat::OpenAiChat,+ &body,+ )+ .expect("client request decodes"),+ ..Request::default()+ };++ let built = TaskInput {+ recent_turn_window: Some(8),+ }+ .build_messages(&State::default(), &request);++ assert_eq!(+ built,+ vec![+ Message::text(Role::User, "Reply with the number four."),+ Message::text(Role::User, "This is still a simple number lookup."),+ Message::text(Role::User, TRAILING_ROUTING_INSTRUCTION),+ ]+ );+ let judge_request = LlmRequest {+ messages: built,+ ..LlmRequest::default()+ };+ let encoded = switchyard_translation::encode_request(+ &judge_request,+ switchyard_translation::WireFormat::AnthropicMessages,+ )+ .expect("judge request encodes");+ let has_empty_text = encoded["messages"]+ .as_array()+ .expect("messages")+ .iter()+ .flat_map(|message| match &message["content"] {+ Value::Array(blocks) => blocks.iter().map(|block| &block["text"]).collect(),+ content => vec![content],+ })+ .any(|text| text.as_str().is_some_and(str::is_empty));+ assert!(!has_empty_text, "{encoded}");+ }+#[test]fn the_default_path_is_left_unchanged() -> Result<()> {// No window means no conversation to be distracted by, so the defaultdiff --git a/docs/routing_algorithms/llm_classifier_routing.md b/docs/routing_algorithms/llm_classifier_routing.mdindex 893019a28..a27582c84 100644--- a/docs/routing_algorithms/llm_classifier_routing.md+++ b/docs/routing_algorithms/llm_classifier_routing.md@@ -241,7 +241,8 @@ Without affinity, the runtime judges every request. By default, it sends theopening task and the latest user follow-up when they differ, excluding tool calls,tool results, and reasoning. It keeps ordinary user content from messages thatalso contain tool results. Set `recent_turn_window` when intervening conversation-context affects the forecast.+context affects the forecast. The window still excludes reasoning, and a turn that+held only reasoning is left out.If a client sends only a follow-up fragment without the opening task, enableaffinity or include the task history. Threshold tuning changes routing policy;it cannot recover missing task context.