Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
serve: Anthropic thinking.budget_tokens is a thinking budget too; docs for the answer room and the budget; test
Niko1221194c26881a74
diff --git a/docs/DETAILS.md b/docs/DETAILS.mdindex 54eeba27e..bdd6acf06 100644--- a/docs/DETAILS.md+++ b/docs/DETAILS.md@@ -778,7 +778,16 @@ print(r.choices[0].message.content)answers from there (the engine continues from what it already holds, so nothing is read again). The wrap-up ispart of the thinking the client sees and counts as output tokens. `"reasoning_budget_tokens": N` in`strata-<model>.json` sets it for every request; a request's own value wins, and `0` means no budget. Off by default;- Anthropic's `"thinking": {"budget_tokens": N}` still only chooses the level, as above.+ Anthropic's `"thinking": {"budget_tokens": N}` chooses the level, as above, and since 0.1.45 is also the budget (a+ `reasoning_budget_tokens` in the same request wins).+- **max_tokens counts the thinking, so the answer keeps its room (0.1.45).** A client that caps the whole reply (the+ desktop agent's harness sends `max_completion_tokens: 8192` on every request) and a task the model thinks long about+ used to be cut off in the middle of the thinking, with no answer. Now a request with thinking on, no budget of its+ own and `max_tokens` of 1,024 or more gets one: the thinking is wrapped up when only a quarter of the cap (at least+ 512 tokens) is left, and the model answers in that room (8,192: thinking up to 6,144, answer 2,048). Requests without+ `max_tokens` (the rest of the context) get a budget of three quarters of it. `"answer_room": false` in+ `strata-<model>.json` turns it off. Qwen recommends up to 262,144 tokens of reasoning and 131,072 of answer: a client+ should not cap the reply far below what the context allows.- **A reply that ends inside its thinking (#1053, opt-in).** Some turns write a sentence of reasoning and then theend-of-turn token with no `</think>`: the content is empty and an agent stops. `"reasoning_close_retry": true` in`strata-<model>.json` closes the thinking once (as the thinking budget does) and continues, once per request, only fordiff --git a/serve/frontend.py b/serve/frontend.pyindex c36a44da1..893530862 100644--- a/serve/frontend.py+++ b/serve/frontend.py@@ -637,6 +637,11 @@ def anthropic_to_messages(req: dict, think_unasked: bool = True) -> tuple[list[d# "output_config": {"effort": "low" | "medium" | "high"}thinking = req.get("thinking")effort = (req.get("output_config") or {}).get("effort") if isinstance(req.get("output_config"), dict) else None+ # 0.1.45: Anthropic's budget_tokens is a real budget, not only a level: the thinking is closed (wrap-up, </think>,+ # the answer) after that many tokens. A reasoning_budget_tokens of the request's own wins.+ bt = thinking.get("budget_tokens") if isinstance(thinking, dict) and thinking.get("type") != "disabled" else None+ if isinstance(bt, int) and not isinstance(bt, bool) and bt > 0 and req.get("reasoning_budget_tokens") is None:+ req["reasoning_budget_tokens"] = btif isinstance(thinking, dict) and thinking.get("type") == "disabled":kwargs["enable_thinking"] = Falseelif effort:diff --git a/serve/test_server.py b/serve/test_server.pyindex ba1361d8d..febd25e97 100644--- a/serve/test_server.py+++ b/serve/test_server.py@@ -3837,6 +3837,17 @@ class ThinkingBudget(unittest.TestCase):"messages": [{"role": "user", "content": "2+2?"}]})self.assertEqual(len(self.engine.prompts), 3) # thinking off: no budget, one prompt each+ def test_anthropic_thinking_budget_tokens_is_a_budget(self):+ """0.1.45: thinking {type: enabled, budget_tokens: N} closes the thinking after N tokens (it was only a level)."""+ from serve.server import REASONING_WRAP_UP+ code, b = self.post("/v1/messages", {"model": "m", "max_tokens": 400, "thinking": {"type": "enabled", "budget_tokens": 30},+ "messages": [{"role": "user", "content": "2+2?"}]})+ self.assertEqual(code, 200, b)+ thinking = "".join(c.get("thinking", "") for c in b["content"] if c["type"] == "thinking")+ self.assertEqual(thinking, ThinkingEngine.THOUGHT[:30] + REASONING_WRAP_UP.split("</think>")[0])+ self.assertEqual("".join(c.get("text", "") for c in b["content"] if c["type"] == "text"), ThinkingEngine.ANSWER)+ self.assertEqual(len(self.engine.prompts), 2)+def test_a_budget_the_thinking_stays_under(self):code, b = self.openai(reasoning_budget_tokens=10_000)self.assertEqual(b["choices"][0]["message"]["reasoning_content"], ThinkingEngine.THOUGHT)