LLM inference in C/C++
C++ ggml
server: support "reasoning_effort": "none" in OAI API (#26045)
Nigel Bosch27209a598da5
diff --git a/tools/server/README.md b/tools/server/README.mdindex e6b27a5576..d345654554 100644--- a/tools/server/README.md+++ b/tools/server/README.md@@ -1247,6 +1247,8 @@ The `response_format` parameter supports both plain JSON output (e.g. `{"type":`chat_template_kwargs`: Allows sending additional parameters to the json templating system. For example: `{"enable_thinking": false}`+`reasoning_effort`: If set to `none`, reasoning will be disabled for this request. Other values (e.g., `low`, `max`) have no effect on reasoning.+`reasoning_format`: The reasoning format to be parsed. If set to `none`, it will output the raw generated text.`reasoning_control`: Arms realtime reasoning control for this completion so it can be ended early via `/v1/chat/completions/control`. Defaults to `false`.diff --git a/tools/server/server-chat.cpp b/tools/server/server-chat.cppindex 31f94e0233..0322e54cce 100644--- a/tools/server/server-chat.cpp+++ b/tools/server/server-chat.cpp@@ -283,6 +283,15 @@ json server_chat_convert_responses_to_chatcmpl(const json & response_body) {chatcmpl_body["max_tokens"] = response_body["max_output_tokens"];}+ if (response_body.contains("reasoning")) {+ // Only "effort" is handled so far+ const json & reasoning = response_body.at("reasoning");+ if (reasoning.contains("effort")) {+ chatcmpl_body["reasoning_effort"] = reasoning.at("effort");+ }+ chatcmpl_body.erase("reasoning");+ }+return chatcmpl_body;}diff --git a/tools/server/server-common.cpp b/tools/server/server-common.cppindex 78b5c6819b..da58049308 100644--- a/tools/server/server-common.cpp+++ b/tools/server/server-common.cpp@@ -1086,6 +1086,14 @@ json oaicompat_chat_params_parse(throw std::invalid_argument("invalid type for \"enable_thinking\" (expected boolean, got string)");}+ // Parse also the OAI "reasoning_effort": "none" specific value+ if (body.contains("reasoning_effort")) {+ auto reasoning_effort = json_value(body, "reasoning_effort", std::string(""));+ if (reasoning_effort == "none") {+ inputs.enable_thinking = false;+ } // other reasoning_effort values are model-specific and not yet handled+ }+inputs.force_pure_content = opt.force_pure_content;// Apply chat template to the list of messages