Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

serve: tool_call_recovery is on by default; "tool_call_recovery": false is the off switch

Niko12213fcb2106d00d
diff --git a/docs/DETAILS.md b/docs/DETAILS.md
index 73f7d8154..0ff215fc5 100644
--- a/docs/DETAILS.md
+++ b/docs/DETAILS.md
@@ -472,7 +472,7 @@ applies at the next start. Network, key, MCP and program keys are not editable t
| `repeat_stop_tokens` | `256` | A reply that repeats one token this many times in a row ends as `"length"`. `0` turns it off. |
| `effort_position` | `"start"` | `"end"` puts a `low` or `medium` effort in a short system turn right before the answer, so a request that only changes the effort reuses the conversation cache. How well the model follows a level in that place is not measured. |
| `anthropic_thinking` | as the template | `"on_request"` renders a `/v1/messages` request that asks for no thinking without thinking, as Anthropic does. Claude Code's short helper calls need it. |
-| `tool_call_recovery` | off | Also reads a declared tool's call written as `<parameter=NAME>`, as JSON inside `<tool_call>`, as a `<function=NAME>` block without `<tool_call>`, or as a second call inside one `<tool_call>`. On 1,462 agent turns 98.3% read as intended with it and 92.9% without. |
+| `tool_call_recovery` | on (since 0.2.8; off in 0.2.7 and before) | Also reads a declared tool's call written as `<parameter=NAME>`, as JSON inside `<tool_call>`, as a `<function=NAME>` block without `<tool_call>`, or as a second call inside one `<tool_call>`. On 1,462 agent turns 98.3% read as intended with it and 92.9% without; the 49 specimens of the test corpus read the same with it on or off. Only a declared tool's call is ever recovered; anything else stays text. `"tool_call_recovery": false` turns it off. |
| `env_to_user` | off | Moves opencode's `<env>` block (working directory, date) to the first user message so the system turn is the same for every session and the disk prefix cache restores it. |
| `return_progress` | off | Per request `"return_progress": true` adds llama.cpp's `prompt_progress` (`total`, `cache`, `processed`, `time_ms`) to the stream, one update per `--prefill` chunk. A prompt shorter than one chunk sends nothing. |
| `prevent_sleep` | off | Windows only: keeps the PC awake while a request runs or waits (`SetThreadExecutionState`). On Linux and macOS one startup line says it does nothing. |
@@ -1446,7 +1446,7 @@ with `--batch 2` at 262,144 context saw 538 MiB at the free line become 395 MiB
| not stated | RTX 5070 Ti Laptop, Swift 1.5 IQ3_XXS, a new opencode session in a second folder | `env_to_user`: first request 14,073 tokens with 13,693 from the cache, 1.1 s, against 14,753 read, 11.9 s |
| 0.1.40.2 | Tesla P100 | A question after a long document re-reads up to 16K tokens (55 s at 262K) without `strata_prefix` |
| not stated | Codex CLI 0.160.0, Q2_0, RTX 5070 | First prompt 9,443 tokens in 10 s; later tool-loop turns reused about 96% from the cache and read the new part in 1-2 s. Compaction after an 85,000-token conversation reused 84,895 of 84,997 tokens with `codex_compaction_cache` and 41 of 80,683 without |
-| not stated | Signalnine/q27's drift corpus, 1,462 agent turns (Qwen3.8 under Claude Code) | `tool_call_recovery`: 98.3% read as intended, 92.9% without |
+| not stated | Signalnine/q27's drift corpus, 1,462 agent turns (Qwen3.8 under Claude Code) | `tool_call_recovery`: 98.3% read as intended, 92.9% without (on by default from 0.2.8) |
| not stated | RTX 3080, RVN IQ3_S config, 8-turn chat replay on a 16K-token system prompt | NaN guard on and off: byte-identical answers in 3 pairs; cost 4.8 us of GPU time a window (7.8 us at most), 0.009% of decode time |
### Context beyond 262K (contributors' runs, RTX 5080, IQ3_S, native path, PR #84)
diff --git a/serve/frontend.py b/serve/frontend.py
index 63e319b82..9bae6a635 100644
--- a/serve/frontend.py
+++ b/serve/frontend.py
@@ -989,7 +989,7 @@ class OutputParser:
self.rescued = 0 # calls delivered from the reasoning
self.refused = 0 # declared calls kept as reasoning text (quoted, or the turn was cut)
self.fence, self.line, self.ticks = "", "", 0
- # recover (opt-in, the config's "tool_call_recovery"): a declared tool's call written next to the template's
+ # recover (on in the server since 0.2.8, off for a bare OutputParser; the config's "tool_call_recovery": false turns it off): a declared tool's call written next to the template's
# form is the call - `<parameter=NAME>` as the opener, JSON inside <tool_call>, a bare <function=NAME> at the
# start of a line outside code, or a second call in the same <tool_call> (which otherwise merges into one).
self.recover = recover
diff --git a/serve/server.py b/serve/server.py
index 8c5213b1f..62abfcfd5 100644
--- a/serve/server.py
+++ b/serve/server.py
@@ -4041,7 +4041,7 @@ class Service:
self.reasoning_budget_tokens = 0 # #123: the config's default thinking budget (0: none)
self.repeat_stop_tokens = REPEAT_STOP_TOKENS # #606: one token this many times in a row ends a reply (0: off)
self.anthropic_think_unasked = True # #278: "anthropic_thinking": "on_request" -> False
- self.tool_call_recovery = False # opt-in: tool calls in forms next to the template's
+ self.tool_call_recovery = True # default on (0.2.8): tool calls in forms next to the template's; config "tool_call_recovery": false turns it off
self.images = None # serve/images.py's ImageGen when the config has "images"
self.restore_after_load = None # a session file an image job saved before it unloaded
self.stop_ids = set(tokenizer.encode(IM_END, parse_special=True) +
@@ -8633,6 +8633,15 @@ def _component_exe(reference: str, cfg: dict) -> str:
raise SystemExit(e.exit_code) from None
+def tool_call_recovery_setting(cfg: dict) -> bool:
+ """The config's "tool_call_recovery" (see OutputParser's recover): ON when the config does not name it (since 0.2.8; 0.2.7 and before: off),
+ "tool_call_recovery": false turns it off. Anything but true or false stops the start."""
+ recovery = cfg.get("tool_call_recovery", True)
+ if not isinstance(recovery, bool):
+ raise SystemExit(f"[strata] config \"tool_call_recovery\" must be true or false, not {recovery!r}")
+ return recovery
+
+
def main(argv=None, *, lifecycle_event=None, interrupt_exit_code: int = 0) -> int:
try: # files made by a run as administrator still belong to the user
from strata_control import private_files as _pf
@@ -9097,10 +9106,7 @@ def main(argv=None, *, lifecycle_event=None, interrupt_exit_code: int = 0) -> in
if mode not in ("model", "on_request"):
raise SystemExit(f"[strata] config anthropic_thinking must be \"model\" or \"on_request\", not {mode!r}")
svc.anthropic_think_unasked = mode == "model"
- recovery = cfg.get("tool_call_recovery", False) # opt-in: see OutputParser's recover
- if not isinstance(recovery, bool):
- raise SystemExit(f"[strata] config \"tool_call_recovery\" must be true or false, not {recovery!r}")
- svc.tool_call_recovery = recovery
+ svc.tool_call_recovery = tool_call_recovery_setting(cfg)
etu = cfg.get("env_to_user", False) # opt-in: see frontend._env_to_user
if not isinstance(etu, bool):
raise SystemExit(f"[strata] config \"env_to_user\" must be true or false, not {etu!r}")
diff --git a/serve/test_server.py b/serve/test_server.py
index 09c880248..251f18376 100644
--- a/serve/test_server.py
+++ b/serve/test_server.py
@@ -1126,7 +1126,7 @@ class ToolCallTagInProse(unittest.TestCase):
class ToolCallRecovery(unittest.TestCase):
- """Opt-in ("tool_call_recovery": true): the model sometimes writes a call in a form next to the template's. These
+ """On by default in the server since 0.2.8 (config "tool_call_recovery": false turns it off): the model sometimes writes a call in a form next to the template's. These
shapes are from a corpus of 1,462 agent turns (Qwen3.8 under Claude Code, signalnine/q27): with the switch on, a
declared tool's call in one of them is the call; anything else stays content, verbatim."""
SCHEMA = ToolCallTerminators.SCHEMA
@@ -1255,6 +1255,29 @@ class ToolCallRecoverySwitch(unittest.TestCase):
self.assertEqual(self.run_reply(False), [])
self.assertEqual(self.run_reply(True), [ToolCallRecovery.CALL])
+ def test_on_by_default_and_the_config_turns_it_off(self):
+ """0.2.8: a server that is told nothing reads these calls; "tool_call_recovery": false is the off switch."""
+ from serve.server import tool_call_recovery_setting
+ tok = ByteTokenizer()
+ svc = Service(MockEngine(tok, "x", max_context=CTX), tok, ChatTemplate(ROOT / "serve/chat_template.jinja"))
+ self.assertIs(svc.tool_call_recovery, True)
+ self.assertIs(tool_call_recovery_setting({}), True)
+ self.assertIs(tool_call_recovery_setting({"model": "x"}), True)
+ self.assertIs(tool_call_recovery_setting({"tool_call_recovery": True}), True)
+ self.assertIs(tool_call_recovery_setting({"tool_call_recovery": False}), False)
+ for bad in ("false", 0, 1, None, "off"):
+ with self.assertRaises(SystemExit):
+ tool_call_recovery_setting({"tool_call_recovery": bad})
+
+ def test_a_fresh_service_reads_the_call_without_being_told(self):
+ tok = ByteTokenizer()
+ script = "</think>\n\n<function=write>\n" + ToolCallRecovery.PARAMS
+ svc = Service(MockEngine(tok, script, max_context=CTX), tok, ChatTemplate(ROOT / "serve/chat_template.jinja"))
+ with contextlib.redirect_stdout(io.StringIO()):
+ evs = [x for kind, x in svc.run(tok.encode("hi"), True, ToolCallRecovery.SCHEMA, 3000, {},
+ threading.Event()) if kind == "event"]
+ self.assertEqual([e.call.arguments for e in evs if e.kind == "tool_call"], [ToolCallRecovery.CALL])
+
class UnfinishedToolCall(unittest.TestCase):
"""#211: a call the output ends inside is not reported as a whole one - its streamed JSON is not closed and the