Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: the 1.5 route prior check - all five tasks the grader failed were read: three were grader flags, two were the long-winded answers

Niko12211b89043157b2
diff --git a/docs/release-notes-0.2.7.md b/docs/release-notes-0.2.7.md
index f12a67035..2f553b5ce 100644
--- a/docs/release-notes-0.2.7.md
+++ b/docs/release-notes-0.2.7.md
@@ -6,7 +6,7 @@ DRAFT of 27 October. Text in square brackets that starts with "pending" or "TO C
## What you will notice
-- **NVIDIA cards with 12 GB or less answer faster.** The router now prefers experts that are already on the card, which saves page-ins from RAM. On an RTX 3060 12 GB (Qwen3.8-Flash-Next IQ3_XXS, five paired runs against the same build with the setting off), a story answer went from 46.2 to 53.1 tok/s (+15%) and a code answer from 45.7 to 62.5 tok/s (+37%). It picks a slightly different set of experts, so the wording of answers changes. We checked the quality of the answers with 61 test tasks on that card. Retrieval (12 of 12), JSON (3 of 3) and unanswerable questions (9 of 9) were all right, and extraction found the same facts as with the setting off; we found no copy, code-constant or spelling defect. It did give two long-winded answers: one rambling paragraph in an agent task and one code answer that was cut off in the middle of a comment. Runs with the setting off show two or three such cases each too. The automatic grader passed 49 of the 61 tasks against 54 with the setting off; of the failing answers we read, the real defects are the two above and the rest were grader flags on names and hyphens. A stronger setting (2.0) did show copy and code defects, so the default is 1.5. It applies to a discrete NVIDIA card of 13 GiB or less; larger cards, Radeon and Strix Halo are not changed. Off: "Faster answers on 12 GB NVIDIA cards" in the model's settings, or `STRATA_ROUTE_PRIOR=0`.
+- **NVIDIA cards with 12 GB or less answer faster.** The router now prefers experts that are already on the card, which saves page-ins from RAM. On an RTX 3060 12 GB (Qwen3.8-Flash-Next IQ3_XXS, five paired runs against the same build with the setting off), a story answer went from 46.2 to 53.1 tok/s (+15%) and a code answer from 45.7 to 62.5 tok/s (+37%). It picks a slightly different set of experts, so the wording of answers changes. We checked the quality of the answers with 61 test tasks on that card. Retrieval (12 of 12), JSON (3 of 3) and unanswerable questions (9 of 9) were all right, and extraction found the same facts as with the setting off; we found no copy, code-constant or spelling defect. It did give two long-winded answers: one rambling paragraph in an agent task and one code answer that was cut off in the middle of a comment. Runs with the setting off show two or three such cases each too. The automatic grader passed 49 of the 61 tasks against 54 with the setting off; we read all five tasks it lost: three were grader flags (a name in a summary, a product name in a summary and a non-breaking hyphen in an agent task) and two were the long-winded answers above. A stronger setting (2.0) did show copy and code defects, so the default is 1.5. It applies to a discrete NVIDIA card of 13 GiB or less; larger cards, Radeon and Strix Halo are not changed. Off: "Faster answers on 12 GB NVIDIA cards" in the model's settings, or `STRATA_ROUTE_PRIOR=0`.
- **Radeon RX 9000 and AI PRO R9700: faster Qwen.** Two changes: the large per-token lookup table stays in RAM when the PC has room for it, and the decode split between card and processor starts at a better value. One R9700 (32 GB) with 125 GB of RAM on Linux, Qwen IQ3_S, medians of three runs:
| Context | Prompt before | Prompt now | Decode before | Decode now |