Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
release notes: the 99.75% is agreement between skip on and off over 799 positions
Niko12212e477ecd40f0
diff --git a/docs/release-notes-0.2.4.md b/docs/release-notes-0.2.4.mdindex d61ed03b1..8b09b94cc 100644--- a/docs/release-notes-0.2.4.md+++ b/docs/release-notes-0.2.4.md@@ -35,7 +35,7 @@ These are the 2.00-bit set. The 2.25-bit set reads 13% more data per token, so e- **A RAM expert store that sizes itself.** The model is a mixture of experts: each token uses a few of thousands of small blocks. Strata keeps the busiest ones in RAM, takes whatever RAM is free, and reads the rest from the SSD with direct reads. On the RTX 5070 PC with 64 GB this gave 4.3 to 4.9 tokens/s where memory-mapping the files gave 1.8 to 2.2.- **Hot-expert profiles ship with the engine,** so the first prompts do not start cold.- **Faster prompts.** The engine now reads long prompts layer by layer in big chunks instead of 16 tokens at a time. A 200-token prompt went from 3.6 to 6.4 tokens/s on the RTX 5070 PC, and 8K prompts reach 231 tokens/s.-- **Faster long prompts.** A prompt over about 3,700 tokens skips work that the last 20 layers cannot use (a 16K prompt is about 34% faster). The answers match the full computation in 99.75% of positions.+- **Faster long prompts.** A prompt over about 3,700 tokens skips work that the last 20 layers cannot use (a 16K prompt is about 34% faster). In the 799 positions we compared, the model's top choice for the next word was the same with and without the skip in 99.75% of them.- **Thinking budget.** When neither your request nor your config sets one, DeepSeek thinks for at most 32,768 tokens. A request can set its own budget; 0 turns the limit off.- **DSpark** (a small draft model from Unsloth) is on where it measured faster and off on CPU-bound PCs. On the RTX 3060 PC (20 threads) it raised decode from 6.1-6.7 to 8.2-8.7 tokens/s. On the 4x R9700 box it raised 31 tokens/s to 55 to 69. On the RTX 5070 PC with a 6-core Ryzen 5 7600 it gave no gain at any setting (code 4.64 tokens/s with it off against 4.33 at best with it on; agent 3.41 against 2.86 to 3.21), because checking the drafts costs the CPU about what the saved tokens are worth. There `--dspark auto` does not load the drafter.- **Faster CPU rows.** The engine has new exact rows for the experts the CPU computes: AVX-512 on Zen 4 and Zen 5, AVX2 on other CPUs, chosen automatically. On the Ryzen 5 7600 PC with an RTX 5070 (2.00 set, DSpark on, three alternating pairs, identical text in all six runs), decode went from 4.08 to 5.11 tokens/s (+25%) and prompt reading rose 26%. This was measured on one CPU, a Zen 4. Zen 5 gained more in the kernel tests.