Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
serve: /v1/responses accepts max_output_tokens of 0 and -1 (#4036)
Niko12219c2d35d978e2
diff --git a/docs/DETAILS.md b/docs/DETAILS.mdindex 61dfb584a..0ed983c37 100644--- a/docs/DETAILS.md+++ b/docs/DETAILS.md@@ -703,7 +703,7 @@ endpoints are refused with an error that says so.| `tools` | `function`, `namespace` and `custom` tools. Hosted tools (`web_search`, `file_search`, ...) are left out because the model cannot run them || `tool_choice` | `"none"` hides the tools; anything else lets the model choose (it cannot be forced) || `reasoning.effort` | `none`/`minimal`, `low`, `medium`, `high`/`xhigh`; without it the config's `reasoning_effort`, else the model's default |-| `max_output_tokens` | The output cap, thinking included. Running out ends the response `incomplete` |+| `max_output_tokens` | The output cap, thinking included. Running out ends the response `incomplete`. `0` and `-1` (or leaving it out) mean no cap, as for `max_tokens`; anything below `-1` is a 400 || `text.format` | `json_schema` and `json_object` use the JSON formats above (checked, not constrained) || `client_metadata["x-codex-turn-metadata"]` thread-title request | Opt-in `"codex_thread_titles": true`: answered without the engine as `{"title": "<the user's line, cut to 36 characters>"}` so the conversation keeps the prompt cache |diff --git a/serve/responses.py b/serve/responses.pyindex 53c4fb54f..a5b97bce1 100644--- a/serve/responses.py+++ b/serve/responses.py@@ -443,8 +443,10 @@ def check_request(req: dict) -> None:raise ResponsesError("stored prompt templates are not supported: send instructions and input","prompt", "unsupported_parameter")n = req.get("max_output_tokens")- if n is not None and (isinstance(n, bool) or not isinstance(n, int) or n < 1):- raise ResponsesError("max_output_tokens must be a positive whole number", "max_output_tokens")+ # -1 and 0 mean no cap, as max_tokens does on the chat path (docs/DETAILS.md); anything lower is a mistake+ if n is not None and (isinstance(n, bool) or not isinstance(n, int) or n < -1):+ raise ResponsesError("max_output_tokens must be a whole number, 1 or more, or -1 or 0 for no cap",+ "max_output_tokens")reasoning = req.get("reasoning")if reasoning is not None and not isinstance(reasoning, dict):raise ResponsesError("reasoning must be an object", "reasoning")diff --git a/serve/test_responses.py b/serve/test_responses.pyindex d6c4e857f..866582c34 100644--- a/serve/test_responses.py+++ b/serve/test_responses.py@@ -389,6 +389,21 @@ class OverHttp(Server):code, events = self.post({"model": "m", "input": "hi", "max_output_tokens": 5, "stream": True})self.assertEqual(events[-1]["type"], "response.incomplete")+ def test_max_output_tokens_zero_and_minus_one_mean_no_cap(self):+ # #4036: the documented "no cap" values; the answer is complete, not cut at the cap+ for n in (0, -1):+ code, r = self.post({"model": "m", "input": "hi", "max_output_tokens": n})+ self.assertEqual((code, r["status"]), (200, "completed"), (n, r))+ self.assertIsNone(r["incomplete_details"])+ code, events = self.post({"model": "m", "input": "hi", "max_output_tokens": n, "stream": True})+ self.assertEqual((code, events[-1]["type"]), (200, "response.completed"), n)++ def test_max_output_tokens_bad_values_are_400(self):+ for n in (-2, -100, 1.5, True, "5"):+ code, r = self.post({"model": "m", "input": "hi", "max_output_tokens": n})+ self.assertEqual(code, 400, (n, r))+ self.assertEqual(r["error"]["param"], "max_output_tokens", n)+def test_effort_none_does_not_think(self):self.engine.scripts = [self.tok.encode("Hello.<|im_end|>", parse_special=True)]self.engine.script = self.engine.scripts[0]