Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

fix(libsy): drop empty classifier history turns after reasoning removal (#1273)

eric-liu-nvidia88fe33e323d4
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6b5848c98..dd4a7774f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -66,6 +66,12 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### Fixed
+- **Windowed classifiers drop empty history turns** - with `recent_turn_window`
+ set, an assistant turn that holds only reasoning, as an OpenAI Chat message
+ with `reasoning_content` and no `content` decodes, is left out of the judge
+ request. Before, it reached the judge as an assistant message with empty text,
+ and an Anthropic Messages judge rejected the request with HTTP 400, which
+ failed the client request.
- **Responses reasoning history reaches non-OpenAI targets** - the default
`responses_reasoning` is now `"auto"`. Targets other than OpenAI keep plaintext
reasoning `summary` and `content` in place, for any `store` value. Before, it
diff --git a/crates/libsy/src/algorithms/llm_class.rs b/crates/libsy/src/algorithms/llm_class.rs
index 10d0be8d4..ff86e89c8 100644
--- a/crates/libsy/src/algorithms/llm_class.rs
+++ b/crates/libsy/src/algorithms/llm_class.rs
@@ -209,11 +209,16 @@ impl TaskInput {
// Reasoning is provider-private and not required to classify the task. Some
// upstreams also reject an unsigned reasoning item replayed without the
// opaque state it was issued with, so it cannot travel through a windowed
- // classifier request as ordinary history.
+ // classifier request as ordinary history. Empty text goes with it: OpenAI
+ // Chat decoding leaves an empty text placeholder beside the reasoning of a
+ // reasoning-only turn, and an Anthropic judge rejects a message whose text
+ // block is empty.
for message in &mut messages {
- message
- .content
- .retain(|block| !matches!(block, ContentBlock::Reasoning { .. }));
+ message.content.retain(|block| match block {
+ ContentBlock::Reasoning { .. } => false,
+ ContentBlock::Text { text } => !text.is_empty(),
+ _ => true,
+ });
}
messages.retain(|message| !message.content.is_empty());
messages
@@ -2395,6 +2400,63 @@ mod tests {
assert!(built.iter().all(|message| !message.content.is_empty()));
}
+ #[test]
+ fn a_window_drops_a_turn_left_empty_once_its_reasoning_is_removed() {
+ // OpenAI Chat decoding places an empty text placeholder beside the reasoning of
+ // an assistant message that carries `reasoning_content` alone. Removing the
+ // reasoning has to take the placeholder with it: an Anthropic judge rejects a
+ // message whose text is empty, and the whole client request then fails.
+ let body = serde_json::json!({
+ "model": "qa/windowed",
+ "messages": [
+ {"role": "user", "content": "Reply with the number four."},
+ {"role": "assistant", "reasoning_content": "private chain of thought"},
+ {"role": "user", "content": "This is still a simple number lookup."}
+ ]
+ });
+ let request = Request {
+ llm_request: switchyard_translation::decode_request(
+ switchyard_translation::WireFormat::OpenAiChat,
+ &body,
+ )
+ .expect("client request decodes"),
+ ..Request::default()
+ };
+
+ let built = TaskInput {
+ recent_turn_window: Some(8),
+ }
+ .build_messages(&State::default(), &request);
+
+ assert_eq!(
+ built,
+ vec![
+ Message::text(Role::User, "Reply with the number four."),
+ Message::text(Role::User, "This is still a simple number lookup."),
+ Message::text(Role::User, TRAILING_ROUTING_INSTRUCTION),
+ ]
+ );
+ let judge_request = LlmRequest {
+ messages: built,
+ ..LlmRequest::default()
+ };
+ let encoded = switchyard_translation::encode_request(
+ &judge_request,
+ switchyard_translation::WireFormat::AnthropicMessages,
+ )
+ .expect("judge request encodes");
+ let has_empty_text = encoded["messages"]
+ .as_array()
+ .expect("messages")
+ .iter()
+ .flat_map(|message| match &message["content"] {
+ Value::Array(blocks) => blocks.iter().map(|block| &block["text"]).collect(),
+ content => vec![content],
+ })
+ .any(|text| text.as_str().is_some_and(str::is_empty));
+ assert!(!has_empty_text, "{encoded}");
+ }
+
#[test]
fn the_default_path_is_left_unchanged() -> Result<()> {
// No window means no conversation to be distracted by, so the default
diff --git a/docs/routing_algorithms/llm_classifier_routing.md b/docs/routing_algorithms/llm_classifier_routing.md
index 893019a28..a27582c84 100644
--- a/docs/routing_algorithms/llm_classifier_routing.md
+++ b/docs/routing_algorithms/llm_classifier_routing.md
@@ -241,7 +241,8 @@ Without affinity, the runtime judges every request. By default, it sends the
opening task and the latest user follow-up when they differ, excluding tool calls,
tool results, and reasoning. It keeps ordinary user content from messages that
also contain tool results. Set `recent_turn_window` when intervening conversation
-context affects the forecast.
+context affects the forecast. The window still excludes reasoning, and a turn that
+held only reasoning is left out.
If a client sends only a follow-up fragment without the opening task, enable
affinity or include the task history. Threshold tuning changes routing policy;
it cannot recover missing task context.