Use Claude Code with your ChatGPT, Kimi, Cursor or Grok subscription via a local Anthropic-compatible proxy
Rust
preserve OpenAI parallel tool-call policy
Raine Virta8d63629d50e8
diff --git a/docs/src/content/docs/reference/http-api.md b/docs/src/content/docs/reference/http-api.mdindex 3b25b60..51a855d 100644--- a/docs/src/content/docs/reference/http-api.md+++ b/docs/src/content/docs/reference/http-api.md@@ -76,7 +76,7 @@ For Kimi, Grok, and Cursor, the route accepts:- `system`, `developer`, `user`, `assistant`, and `tool` messages- text, supported user images, function calls, and tool results-- function tools and `tool_choice`+- function tools, `tool_choice`, and `parallel_tool_calls`- `max_tokens` or `max_completion_tokens`- `reasoning_effort`- streaming, non-streaming responses, and `stream_options.include_usage`@@ -128,7 +128,7 @@ Codex models use native Responses passthrough, including native JSON and SSE out- string input or message items- `instructions`-- function calls, function-call outputs, tools, and `tool_choice`+- function calls, function-call outputs, tools, `tool_choice`, and `parallel_tool_calls`- `max_output_tokens`- `reasoning.effort`- streaming or non-streaming output@@ -137,6 +137,14 @@ Responses include the accepted tool settings. Grok search appears as a `web_sear`store: true` and other unsupported non-null fields return an error. Stored response retrieval, deletion, and WebSocket client connections are not supported.+### Parallel tool calls++The shared Kimi, Grok, and Cursor ingress accepts boolean `parallel_tool_calls` on both OpenAI routes. `false` preserves serial tool execution through translation, while `true` selects the existing parallel default. Omitting the field leaves the provider default unchanged. Non-boolean values return an `invalid_request_error` with `parallel_tool_calls` in `error.param`.++The setting applies without changing the requested `tool_choice` mode. This includes omitted or `auto` choices, `none`, `required`, and named functions. Internally, an explicit OpenAI setting determines the equivalent Anthropic `tool_choice.disable_parallel_tool_use` value. Anthropic Messages requests can set `disable_parallel_tool_use` directly. Kimi and Grok receive the corresponding upstream `parallel_tool_calls` value, and Cursor's bridged tool loop remains serial between client tool results.++Codex Responses uses native passthrough and forwards `parallel_tool_calls` unchanged. Codex Chat Completions has its own field allowlist and rejects `parallel_tool_calls` because that path does not support function tools.+## OpenAI routing, sessions, and errorsBoth OpenAI routes strip a trailing `[1m]`, resolve configured aliases, and choose the provider from `model`. Aliases follow `aliasProvider`, while explicit provider model IDs keep their provider. Unknown models return HTTP 400 with the supported model list.diff --git a/src/openai_compat/request.rs b/src/openai_compat/request.rsindex 659651b..c467d66 100644--- a/src/openai_compat/request.rs+++ b/src/openai_compat/request.rs@@ -114,11 +114,12 @@ pub fn parse_request());}validate_single_choice(object)?;- validate_parallel_tools(object)?;+ let parallel_tool_calls = optional_bool(object, "parallel_tool_calls")?;validate_store(surface, object)?;let max_tokens = parse_max_tokens(surface, object)?;let tools = parse_tools(object.get("tools"), surface)?;- let tool_choice = parse_tool_choice(object.get("tool_choice"), &tools, surface)?;+ let mut tool_choice = parse_tool_choice(object.get("tool_choice"), &tools, surface)?;+ apply_parallel_tool_calls(&mut tool_choice, parallel_tool_calls);let response_metadata = if surface == OpenAiSurface::Responses {OpenAiResponseMetadata {tools: object@@ -240,15 +241,18 @@ fn validate_single_choice(object: &Map<String, Value>) -> Result<(), OpenAiError}}-fn validate_parallel_tools(object: &Map<String, Value>) -> Result<(), OpenAiError> {- match object.get("parallel_tool_calls") {- None | Some(Value::Null) | Some(Value::Bool(false)) => Ok(()),- Some(Value::Bool(true)) => Err(OpenAiError::unsupported("parallel_tool_calls")),- Some(_) => Err(OpenAiError::invalid(- "'parallel_tool_calls' must be a boolean",- Some("parallel_tool_calls"),- )),- }+fn apply_parallel_tool_calls(tool_choice: &mut Option<Value>, parallel_tool_calls: Option<bool>) {+ let Some(parallel_tool_calls) = parallel_tool_calls else {+ return;+ };+ let choice = tool_choice.get_or_insert_with(|| json!({"type":"auto"}));+ choice+ .as_object_mut()+ .expect("translated tool choice is an object")+ .insert(+ "disable_parallel_tool_use".to_string(),+ Value::Bool(!parallel_tool_calls),+ );}fn parse_max_tokens(@@ -1135,6 +1139,78 @@ mod tests {assert_eq!(parsed.messages.extra["tool_choice"]["name"], "lookup");}+ #[test]+ fn parallel_tool_calls_sets_anthropic_tool_choice_policy() {+ let cases = [+ (None, "auto"),+ (Some(json!("auto")), "auto"),+ (Some(json!("none")), "none"),+ (Some(json!("required")), "any"),+ (+ Some(json!({"type":"function","function":{"name":"lookup"}})),+ "tool",+ ),+ ];+ for parallel in [false, true] {+ for (choice, expected_type) in &cases {+ let mut body = json!({+ "model":"kimi-k2.6",+ "messages":[{"role":"user","content":"look up x"}],+ "tools":[{"type":"function","function":{"name":"lookup","parameters":{"type":"object"}}}],+ "parallel_tool_calls":parallel,+ });+ if let Some(choice) = choice {+ body["tool_choice"] = choice.clone();+ }+ let parsed = parse_request(+ OpenAiSurface::ChatCompletions,+ body,+ "kimi",+ Some("session"),+ )+ .unwrap();+ let translated = &parsed.messages.extra["tool_choice"];+ assert_eq!(translated["type"], *expected_type);+ assert_eq!(translated["disable_parallel_tool_use"], !parallel);+ }+ }+ }++ #[test]+ fn responses_parallel_tool_calls_supports_named_choice() {+ let parsed = parse_request(+ OpenAiSurface::Responses,+ json!({+ "model":"grok-4.5",+ "input":"look up x",+ "tools":[{"type":"function","name":"lookup","parameters":{"type":"object"}}],+ "tool_choice":{"type":"function","name":"lookup"},+ "parallel_tool_calls":false,+ }),+ "grok",+ None,+ )+ .unwrap();+ assert_eq!(parsed.messages.extra["tool_choice"]["type"], "tool");+ assert_eq!(+ parsed.messages.extra["tool_choice"]["disable_parallel_tool_use"],+ true+ );+ }++ #[test]+ fn parallel_tool_calls_must_be_boolean() {+ let error = parse_request(+ OpenAiSurface::Responses,+ json!({"model":"grok-4.5","input":"hello","parallel_tool_calls":"false"}),+ "grok",+ None,+ )+ .unwrap_err();+ assert_eq!(error.param.as_deref(), Some("parallel_tool_calls"));+ assert!(error.code.is_none());+ }+#[test]fn responses_maps_function_items() {let parsed = parse_request(diff --git a/src/providers/codex/native.rs b/src/providers/codex/native.rsindex d392422..63995b9 100644--- a/src/providers/codex/native.rs+++ b/src/providers/codex/native.rs@@ -703,6 +703,19 @@ mod tests {assert_eq!(fast["service_tier"], "priority");}+ #[test]+ fn native_request_preserves_parallel_tool_calls() {+ for parallel in [false, true] {+ let mut body = request(json!({+ "model":"gpt-5.4",+ "input":[],+ "parallel_tool_calls":parallel+ }));+ shape_native_request(&mut body).unwrap();+ assert_eq!(body["parallel_tool_calls"], parallel);+ }+ }+#[test]fn explicit_service_tier_is_preserved() {let mut body = request(json!({diff --git a/src/providers/codex/translate/request.rs b/src/providers/codex/translate/request.rsindex b998039..ab1ab9b 100644--- a/src/providers/codex/translate/request.rs+++ b/src/providers/codex/translate/request.rs@@ -7,7 +7,8 @@ use serde_json::Value;use crate::anthropic::schema::MessagesRequest;use crate::config;use crate::providers::translate_shared::{- ContentBlock, flatten_system_text, image_source_to_url, normalize_content, read_effort,+ ContentBlock, flatten_system_text, image_source_to_url, normalize_content, parallel_tool_calls,+ read_effort,};use super::read_rewrite::{ReadOffsetRewrite, read_offset_rewrite};@@ -433,7 +434,7 @@ pub fn translate_request(let input = build_input(req);let tools = read_tools(req)?;let tool_choice = map_tool_choice(req)?;- let parallel_tool_calls = !disable_parallel_tool_use(req);+ let parallel_tool_calls = parallel_tool_calls(req).unwrap_or(true);let mut text = ResponsesText {verbosity: Some("low".to_string()),@@ -754,15 +755,6 @@ fn map_tool_choice(req: &MessagesRequest) -> Result<Option<ResponsesToolChoice>,}}-fn disable_parallel_tool_use(req: &MessagesRequest) -> bool {- req.extra- .get("tool_choice")- .and_then(Value::as_object)- .and_then(|choice| choice.get("disable_parallel_tool_use"))- .and_then(Value::as_bool)- .unwrap_or(false)-}-fn build_input(req: &MessagesRequest) -> Vec<ResponsesInputItem> {let mut out: Vec<ResponsesInputItem> = Vec::new();let mut read_tool_uses_with_offset = HashSet::new();diff --git a/src/providers/grok/translate/request.rs b/src/providers/grok/translate/request.rsindex 15c4ac2..328ebd3 100644--- a/src/providers/grok/translate/request.rs+++ b/src/providers/grok/translate/request.rs@@ -5,7 +5,7 @@ use serde_json::Value;use crate::anthropic::schema::{Message, MessagesRequest};use crate::config::GrokToolImageMode;-use crate::providers::translate_shared::{ImageSource, image_source_to_url};+use crate::providers::translate_shared::{ImageSource, image_source_to_url, parallel_tool_calls};#[derive(Debug, Clone, Serialize)]pub struct GrokResponsesRequest {@@ -17,6 +17,8 @@ pub struct GrokResponsesRequest {pub tools: Option<Vec<GrokTool>>,#[serde(skip_serializing_if = "Option::is_none")]pub tool_choice: Option<GrokToolChoice>,+ #[serde(skip_serializing_if = "Option::is_none")]+ pub parallel_tool_calls: Option<bool>,pub store: bool,pub stream: bool,#[serde(skip_serializing_if = "Option::is_none")]@@ -183,6 +185,7 @@ pub fn translate_request_with_mode(input,tools,tool_choice,+ parallel_tool_calls: parallel_tool_calls(req),store: false,stream: true,max_output_tokens: req.max_tokens,@@ -479,11 +482,29 @@ fn parse_tool_choice(.get("type").and_then(Value::as_str).ok_or_else(|| anyhow::anyhow!("tool_choice type is invalid"))?;+ let valid_policy = obj+ .get("disable_parallel_tool_use")+ .is_none_or(Value::is_boolean);match kind {- "auto" if obj.len() == 1 => Ok(Some(GrokToolChoice::Auto("auto".into()))),- "any" if obj.len() == 1 => Ok(Some(GrokToolChoice::Required("required".into()))),- "none" if obj.len() == 1 => Ok(Some(GrokToolChoice::None("none".into()))),- "tool" if obj.len() == 2 => {+ "auto" | "any" | "none"+ if valid_policy+ && obj+ .keys()+ .all(|key| ["type", "disable_parallel_tool_use"].contains(&key.as_str())) =>+ {+ Ok(Some(match kind {+ "auto" => GrokToolChoice::Auto("auto".into()),+ "any" => GrokToolChoice::Required("required".into()),+ "none" => GrokToolChoice::None("none".into()),+ _ => unreachable!(),+ }))+ }+ "tool"+ if valid_policy+ && obj.keys().all(|key| {+ ["type", "name", "disable_parallel_tool_use"].contains(&key.as_str())+ }) =>+ {let name = obj.get("name").and_then(Value::as_str)diff --git a/src/providers/kimi/translate/request.rs b/src/providers/kimi/translate/request.rsindex 0d676c6..229baff 100644--- a/src/providers/kimi/translate/request.rs+++ b/src/providers/kimi/translate/request.rs@@ -5,7 +5,7 @@ use super::model_allowlist::{KIMI_DEFAULT_MODEL, assert_allowed_model, is_k3, reuse crate::anthropic::schema::MessagesRequest;use crate::providers::translate_shared::{ContentBlock, flatten_system_text, image_block_to_url, image_source_to_url, normalize_content,- read_effort,+ parallel_tool_calls, read_effort,};// ---------------------------------------------------------------------------@@ -20,6 +20,8 @@ pub struct KimiChatRequest {pub tools: Option<Vec<KimiTool>>,#[serde(default, skip_serializing_if = "Option::is_none")]pub tool_choice: Option<KimiToolChoice>,+ #[serde(default, skip_serializing_if = "Option::is_none")]+ pub parallel_tool_calls: Option<bool>,pub stream: bool,pub stream_options: KimiStreamOptions,pub max_tokens: u32,@@ -153,6 +155,7 @@ pub fn translate_request(}),tools: if tools.is_empty() { None } else { Some(tools) },tool_choice,+ parallel_tool_calls: parallel_tool_calls(req),prompt_cache_key: opts.session_id,};diff --git a/src/providers/translate_shared.rs b/src/providers/translate_shared.rsindex 97201f5..77acd03 100644--- a/src/providers/translate_shared.rs+++ b/src/providers/translate_shared.rs@@ -57,6 +57,15 @@ pub fn flatten_system_text(system_val: Option<&Value>) -> Option<String> {}}+pub fn parallel_tool_calls(req: &MessagesRequest) -> Option<bool> {+ req.extra+ .get("tool_choice")+ .and_then(Value::as_object)+ .and_then(|choice| choice.get("disable_parallel_tool_use"))+ .and_then(Value::as_bool)+ .map(|disabled| !disabled)+}+pub fn read_effort(req: &MessagesRequest) -> Result<Option<&str>, anyhow::Error> {let output_config = match req.extra.get("output_config") {Some(Value::Object(m)) => m,diff --git a/tests/server.rs b/tests/server.rsindex 9087aff..f71f9c7 100644--- a/tests/server.rs+++ b/tests/server.rs@@ -14,7 +14,7 @@ use claude_code_proxy::{},};use serde_json::{Value, json};-use std::sync::Arc;+use std::sync::{Arc, Mutex};use tower::util::ServiceExt;fn body_string(json: &str) -> Body {@@ -95,6 +95,99 @@ impl Provider for FakeProvider {}}+struct TranslatingProvider {+ name: &'static str,+ model: &'static str,+ captured: Arc<Mutex<Option<Value>>>,+}++#[async_trait]+impl Provider for TranslatingProvider {+ fn name(&self) -> &'static str {+ self.name+ }++ fn supported_models(&self) -> Vec<String> {+ vec![self.model.to_string()]+ }++ fn cli(&self) -> &'static dyn CliHandlers {+ &FAKE_CLI+ }++ async fn handle_messages(+ &self,+ _body: MessagesRequest,+ _ctx: RequestContext,+ ) -> axum::response::Response {+ (StatusCode::NOT_IMPLEMENTED, "unused").into_response()+ }++ async fn handle_count_tokens(+ &self,+ _body: MessagesRequest,+ _ctx: RequestContext,+ ) -> axum::response::Response {+ (StatusCode::NOT_IMPLEMENTED, "unused").into_response()+ }++ async fn generate_anthropic_stream(+ &self,+ body: MessagesRequest,+ _ctx: RequestContext,+ ) -> Result<Generation, ProviderError> {+ let translated = match self.name {+ "kimi" => serde_json::to_value(+ claude_code_proxy::providers::kimi::translate::request::translate_request(+ &body,+ claude_code_proxy::providers::kimi::translate::request::TranslateOptions {+ session_id: None,+ },+ )+ .unwrap(),+ )+ .unwrap(),+ "grok" => serde_json::to_value(+ claude_code_proxy::providers::grok::translate::request::translate_request(+ &body,+ self.model.to_string(),+ )+ .unwrap(),+ )+ .unwrap(),+ _ => unreachable!(),+ };+ *self.captured.lock().unwrap() = Some(translated);+ let sse = concat!(+ "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"msg_fake\",\"model\":\"test\",\"usage\":{\"input_tokens\":1}}}\n\n",+ "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n",+ "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"ok\"}}\n\n",+ "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n",+ "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"},\"usage\":{\"output_tokens\":1}}\n\n",+ "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n"+ );+ Ok(Generation {+ body: GenerationBody::BufferedSse(sse.into()),+ resolved_model: self.model.to_string(),+ })+ }+}++fn translating_registry(+ name: &'static str,+ model: &'static str,+ captured: Arc<Mutex<Option<Value>>>,+) -> Arc<Registry> {+ Arc::new(Registry::from_providers(+ AliasProvider::Kimi,+ vec![Arc::new(TranslatingProvider {+ name,+ model,+ captured,+ }) as Arc<dyn Provider>],+ ))+}+fn routed_registry() -> Arc<Registry> {Arc::new(Registry::from_providers(AliasProvider::Kimi,@@ -756,6 +849,59 @@ async fn openai_routes_select_non_codex_providers_and_aliases() {}}+#[tokio::test]+async fn openai_routes_preserve_serial_tool_calls_upstream() {+ for (provider, model, uri, body, expected_choice) in [+ (+ "kimi",+ "kimi-k2.6",+ "/v1/chat/completions",+ json!({+ "model":"kimi-k2.6",+ "messages":[{"role":"user","content":"look up x"}],+ "tools":[{"type":"function","function":{"name":"lookup","parameters":{"type":"object"}}}],+ "tool_choice":{"type":"function","function":{"name":"lookup"}},+ "parallel_tool_calls":false+ }),+ json!({"type":"function","function":{"name":"lookup"}}),+ ),+ (+ "grok",+ "grok-4.5",+ "/v1/responses",+ json!({+ "model":"grok-4.5",+ "input":"look up x",+ "tools":[{"type":"function","name":"lookup","parameters":{"type":"object"}}],+ "tool_choice":"none",+ "parallel_tool_calls":false+ }),+ json!("none"),+ ),+ ] {+ let captured = Arc::new(Mutex::new(None));+ let response = app_with_options(+ translating_registry(provider, model, captured.clone()),+ None,+ true,+ )+ .oneshot(+ Request::builder()+ .method(Method::POST)+ .uri(uri)+ .header("content-type", "application/json")+ .body(body_string(&body.to_string()))+ .unwrap(),+ )+ .await+ .unwrap();+ assert_eq!(response.status(), StatusCode::OK);+ let translated = captured.lock().unwrap().clone().unwrap();+ assert_eq!(translated["parallel_tool_calls"], false);+ assert_eq!(translated["tool_choice"], expected_choice);+ }+}+#[tokio::test]async fn routed_openai_streams_use_surface_specific_events() {let chat = app_with_options(routed_registry(), None, true)diff --git a/tests/smoke_cutover.rs b/tests/smoke_cutover.rsindex f340cb7..7bbd602 100644--- a/tests/smoke_cutover.rs+++ b/tests/smoke_cutover.rs@@ -7,7 +7,10 @@ use axum::response::Response;use claude_code_proxy::providers::codex::compaction::clear_all_compactions_for_tests;use claude_code_proxy::providers::codex::continuation::clear_all_continuations_for_tests;use claude_code_proxy::providers::codex::websocket::clear_codex_websocket_pool_for_tests;-use claude_code_proxy::{registry::Registry, server::app};+use claude_code_proxy::{+ registry::Registry,+ server::{app, app_with_options},+};use futures_util::{SinkExt, StreamExt};use http_body_util::BodyExt;use serde_json::{Value, json};@@ -100,6 +103,22 @@ async fn call_messages_body(body: Value) -> Response {.unwrap()}+async fn call_responses_body(body: Value) -> Response {+ let _no_proxy_env = EnvGuard::set("NO_PROXY", "127.0.0.1,localhost");+ app_with_options(Arc::new(Registry::with_default_alias()), None, true)+ .oneshot(+ Request::builder()+ .method(Method::POST)+ .uri("/v1/responses")+ .header("content-type", "application/json")+ .header("x-claude-code-session-id", "smoke-session")+ .body(Body::from(body.to_string()))+ .unwrap(),+ )+ .await+ .unwrap()+}+fn collect_files(root: &Path) -> Vec<PathBuf> {let mut out = Vec::new();let Ok(entries) = std::fs::read_dir(root) else {@@ -832,6 +851,37 @@ async fn smoke_codex_http_messages_uses_mock_upstream() {assert_eq!(sent["stream"], true);}+#[allow(clippy::await_holding_lock)]+#[tokio::test]+async fn smoke_codex_native_responses_preserves_parallel_tool_calls() {+ let _guard = env_lock();+ let config = TempDir::new().unwrap();+ write_auth(config.path(), "codex");++ let captured = Arc::new(Mutex::new(None));+ let upstream = spawn_http_upstream({+ let captured = captured.clone();+ move |body: Value| {+ let _ = captured.lock().map(|mut guard| *guard = Some(body));+ br#"{"id":"resp_1","object":"response","status":"completed","output":[],"usage":{"input_tokens":1,"output_tokens":1}}"#.to_vec()+ }+ })+ .await;++ let _config_env = EnvGuard::set("CCP_CONFIG_DIR", config.path());+ let _base_url_env = EnvGuard::set("CCP_CODEX_BASE_URL", &upstream);+ let response = call_responses_body(json!({+ "model":"gpt-5.4",+ "input":"hello",+ "parallel_tool_calls":false+ }))+ .await;++ assert_eq!(response.status(), StatusCode::OK);+ let sent = captured.lock().unwrap().clone().unwrap();+ assert_eq!(sent["parallel_tool_calls"], false);+}+/// Resets the retry-delay override even when the test panics, so later tests/// in this process keep real backoff behavior.struct ZeroRetryDelayGuard;