Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

release notes: the prompt-skip agreement at the final tip, 99.5% of 799 positions

Niko1221eec705cc9894
diff --git a/docs/release-notes-0.2.4.md b/docs/release-notes-0.2.4.md
index 67dec502a..9c4832987 100644
--- a/docs/release-notes-0.2.4.md
+++ b/docs/release-notes-0.2.4.md
@@ -36,7 +36,7 @@ These are the 2.00-bit set. The 2.25-bit set reads 13% more data per token, so e
- **Hot-expert profiles ship with the engine,** so the first prompts do not start cold.
- **Faster prompts.** The engine now reads long prompts layer by layer in big chunks instead of 16 tokens at a time. A 200-token prompt went from 3.6 to 6.4 tokens/s on the RTX 5070 PC, and 8K prompts reach 231 tokens/s.
- **Faster prompts on Radeon AI PRO R9700.** New matrix-core kernels (RDNA4) read prompts about 64% faster at 8K and 69% faster at 32K on 4 cards (196 to 320 and 445 to 755 tokens/s), and about 22% faster at 8K on one card. Answer speed does not change. They are on by default on these cards.
-- **Faster long prompts.** A prompt over about 3,700 tokens skips work that the last 20 layers cannot use (a 16K prompt is about 34% faster). In the 799 positions we compared, the model's top choice for the next word was the same with and without the skip in 99.75% of them.
+- **Faster long prompts.** A prompt over about 3,700 tokens skips work that the last 20 layers cannot use (a 16K prompt is about 34% faster). In 99.5% of the 799 positions we compared, the model's top choice for the next word was the same with and without the skip.
- **Thinking budget.** When neither your request nor your config sets one, DeepSeek thinks for at most 32,768 tokens. A request can set its own budget; 0 turns the limit off.
- **DSpark** (a small draft model from Unsloth) is on where it measured faster and off on CPU-bound PCs. On the RTX 3060 PC (20 threads) it raised decode from 6.1-6.7 to 8.2-8.7 tokens/s. On the 4x R9700 box it raised 31 tokens/s to 55 to 69. On the RTX 5070 PC with a 6-core Ryzen 5 7600 it gave no gain at any setting (code 4.64 tokens/s with it off against 4.33 at best with it on; agent 3.41 against 2.86 to 3.21), because checking the drafts costs the CPU about what the saved tokens are worth. There `--dspark auto` does not load the drafter.
- **Faster CPU rows.** The engine has new exact rows for the experts the CPU computes: AVX-512 on Zen 4 and Zen 5, AVX2 on other CPUs, chosen automatically. On the Ryzen 5 7600 PC with an RTX 5070 (2.00 set, DSpark on, three alternating pairs, identical text in all six runs), decode went from 4.08 to 5.11 tokens/s (+25%) and prompt reading rose 26%. This was measured on one CPU, a Zen 4. Zen 5 gained more in the kernel tests.