Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Release notes 0.2.1: fill the iD, iE, zc and bimodal parts (measured defaults, fixes, known limits)

Niko122193aa62ed6466
diff --git a/docs/release-notes-0.2.1.md b/docs/release-notes-0.2.1.md
index 385ba1f91..ba3b05afc 100644
--- a/docs/release-notes-0.2.1.md
+++ b/docs/release-notes-0.2.1.md
@@ -1,7 +1,5 @@
**0.2.1: a task board in the app, pictures on the Mac and on Intel Arc cards, `strata bench run` with a fixed standard benchmark, faster downloads, Docker images, and your own engine settings are no longer overwritten.** It also fixes the crashes, stuck downloads and GPU-choice bugs reported on 0.2.0-beta. Each new default below says what it was measured on and whether the answers are identical to 0.2.0. TODO(release): engine package version and feed sequence number (0.2.0 was 0.1.45.3 and sequence 101). Update with `strata update` (Terminal), Settings > Update (app), or `UPDATE.bat` / `./update.sh` (source installs).
-TODO(streams still running, not described here): speed regressions (f-021-iD: #3252 #3262 #3237 #3204), Linux and Windows fixes (f-021-iE: #3238 #3269 #3191 #3202 #3186 #3190 #3268 #3208 #3219 #3172 #3271 #3217 and the community items #3201 #3173), packed-model loading (f-021-zc) and the R9700 prompt-speed modes (f-021-bimodal). Add each one when it is merged.
-
## Which file to download
TODO(release): file names. Below, `0.2.1` stands for the final version string (0.2.0 used `0.2.0-beta`).
@@ -46,8 +44,11 @@ Speed numbers are medians of interleaved pairs on the same box, a fresh engine p
- **Strix Halo (gfx1151): the hipBLASLt tuning table is on.** The packaged Strix Halo engines (Linux `strixhalo` and the Windows `rdna3` package) use the table for their card when their hipBLASLt version matches it. The measurement is a community A/B from `bench/results/2026-10-05-community-gfx1151` (Linux, ROCm 7.17, hipBLASLt 1.5.0): prompts 250.9 to 528.9 tok/s (2.11x). We have no Strix Halo and did not repeat it. The table changes which matrix-product algorithm reads the prompt, so the answers are not bit-identical to plain hipBLAS (on an R9700 the first-token KL was 0.016, against a run-to-run difference of 0.00004; top-1 the same on 24 of 24 prompts). `STRATA_HIPBLASLT_TABLE=off` turns it off. On every other card the table is opt-in (`STRATA_HIPBLASLT_TABLE=on`): on the R9700 a table calibrated for the packaged hipBLASLt gave about +1% prompt and -2% decode, so none is shipped as a default. TODO(owner): the table is packaged per card; confirm the final package contents when the engines are built (Windows `rdna3`: gfx1100 and gfx1151; Linux `rx7900`: gfx1100; Linux `strixhalo`: gfx1151; the R9700 packages carry none that matches).
- **Two NVIDIA cards, several requests at once: one batch group.** With a layer split over exactly two NVIDIA cards and `--batch N`, the engine now runs one serial group instead of two pipelined groups. Measured on two RTX A4000 16 GB with Qwen3.8-Flash-Next IQ3_XXS, 5 interleaved pairs, aggregate tokens per second: 2 requests +15.9% (5 of 5 pairs), 4 requests +12.0% (5 of 5), 8 requests +14.7% (3 of 3). Every slot's tokens equal the solo run's, and the old default's, under the fixed identity flags (2 and 4 slots, 64 greedy tokens). A single request is not changed. It is not faster everywhere: when most hot experts sit in the CPU, as under the identity gate's small cache, the pipelined groups were about 10% faster (24.4 against 22.0 tok/s), so a card with a very small cache can lose here. `STRATA_BATCH_GROUPS_NV_SERIAL=0` restores the old default; `--batch-groups auto` or a number asks for the pipeline. AMD cards, three or more cards and the single-request paths are unchanged. TODO(gate): the 5-pair A/B of the final engine at 4 slots (2 of 5 pairs done: new default faster 2 of 2) and the batch identity check on the final build.
- **A card that draws no display keeps less in reserve.** The reserve of graphics memory the engine leaves free is now 512 MiB (700 MiB with pictures) on a card the probe says drives no screen (a compute-only card), instead of 1536 MiB. A card that drives a display keeps the 1536 / 2048 MiB safety default, and a number you set yourself (0 included) is always used as it is. Only the amount of free memory changes, so the answers are identical and the expert cache gets room for more experts. We did not time this ourselves: the report behind it (#3215) measured 80 against 50 tok/s with the forced reserve. Where the probe cannot tell (AMD and Intel on Linux) the safety default stays.
-- **TODO(zc): packed models (`experts.bin`) read with the I/O prefetch by default when the page cache has room for at least 55% of the experts.** Measured on an RTX 3060 12 GB (Core Ultra 7 265, IQ3_XXS 43 GB pack, cgroup RAM 32 GB): decode 1.29 / 1.20 / 1.34x, identity gate identical, but 16 GB RAM prompts 4K -4% to -6% so the rule gates it. Not merged yet; write the final numbers and the rule once f-021-zc is merged.
-- **TODO(iD, iE, bimodal): any default or fix from those streams that changes speed, with the box it was measured on.**
+- **Packed models (`experts.bin`) on Linux: I/O prefetch and the decode gate are on when the page cache has room for at least 55% of the experts.** Measured on an RTX 3060 12 GB (Core Ultra 7 265), Qwen3.8-Flash-Next IQ3_XXS 43 GB pack with `--mmap-experts`, RAM limited to 32 GB by cgroup, cold page cache, 5 interleaved pairs: decode story / code / document 26.0 / 28.3 / 18.5 to 35.2 / 34.4 / 24.7 tok/s (x1.38 / x1.21 / x1.33, 5 of 5 pairs each), a 4K prompt x1.04 (4 of 5), a 20K prompt x1.06 (5 of 5). Same answers: the fixed-flag identity gate (10 prompts, 4K and 20K exact prompt checks) is identical with no RAM limit and with the 32 GB limit. With 16 GB of RAM the page cache holds 38% of the experts, so the default stays off: with it on, a cold 4K prompt read 4 to 6% slower. `STRATA_IO_PREFETCH=0` turns it off and `=1` forces it on. Linux only; only the IQ3_XXS pack was tested (not Q2_0, IQ3_S or Coder packs).
+- **AMD RX 9000 / R9700 on Linux: the engine opens its GPU queues in a fixed order at start.** Before, about half of the fresh engines with MTP read prompts 8 to 14% slower, depending on which compute pipe the card's firmware gave the engine's two busiest queues (both on one pipe was slow). Measured on a Radeon AI PRO R9700 (Linux, ROCm 7.10, Q2_0 with MTP), 12 interleaved pairs: the old engine had 4 slow or cold starts of 12 (4K prompt 1,973 to 2,094 tok/s against 2,287 for a fast start); the new one had 12 of 12 fast (4K 2,281 to 2,291, 20K 2,759 to 2,773). Over 108 starts with the queue order set 2 were slow (about 2%), against about 45% before. The same fix helps without MTP (1 to 4 of 12 slow before, 12 of 12 fast after). Decode is unchanged. Same answers: the identity gate (10 prompts, 4K and 20K exact checks) is identical for Q2_0 and IQ3_XXS, also with the 0.2.0 prompt defaults on. `STRATA_HIP_QUEUE_PRIME=0` turns it off. One card was measured; Windows and RX 7000 cards are not changed.
+- **Linux hybrid CPUs with low-power E cores (Core Ultra "Meteor Lake" / "Arrow Lake-H", #3262).** The engine and setup now count the cores listed in `cpu_lowpower` as efficiency cores. A Core Ultra 7 255H used to look like 8 P and 8 E cores, so the hybrid plan did not apply and 15 workers ran; it now plans 10. The reporter measured the 15-worker default about 35% slower than 9. We have no such CPU and ran unit tests only (the plan and a fake `/sys`); only the number of workers changes, not the answers.
+- **Two GPUs of different architectures (HIP): no start-up warm-up (#3204).** The warm-up product that runs on every card at start made prompts about 25% slower on a split of gfx1151 and gfx1201 for the reporter (`STRATA_PREFILL_WARM=0` restored them). It now runs only when every visible card has the same architecture. We have no mixed pair; on two R9700 (same architecture) the warm-up costs nothing (3,855 against 3,853 tok/s at a 32K prompt, 5 pairs), so the change leaves them as they were. The first request on a mixed pair pays the 1 to 2.5 s library load again, as before 0.2.0. `STRATA_PREFILL_WARM=1` / `0` overrides.
+- **Linux, models read in place: a drive probe decides between the file cache and unbuffered reads (#3237). Pending owner OK; not final for the release.** The room rule picks the file cache when RAM seems to have room, which was 2x slower in prompts and 3 to 4x slower in decode for a reporter with a PCIe 5 drive (about 8 GB/s). When the rule says "cache" and the cache would hold under half of the reads, the engine now reads 96 random 1 MiB pieces both ways at start (under 2 s) and goes unbuffered only if the drive reads at least 4 GB/s and is faster that way. Every drive we have (1.6 to 3.1 GB/s) keeps the file cache; on a 2 GB/s NVMe the cache was better for decode (up to x1.6) and unbuffered reads better for prompts, the opposite of the report. Untested on a fast drive; the answers cannot change (only where reads go), and no identity gate was run. `STRATA_FILE_TIER_PROBE=0` turns it off.
## Fixes from user reports
@@ -60,7 +61,16 @@ Speed numbers are medians of interleaved pairs on the same box, a fresh engine p
- **Crash on launch (Windows 11, exit code 0xC0000409, no log).** A failed start on Windows and Mac now leaves a log and a crash file in the app folder and shows a message box instead of closing. The reporter's own panic message is still unknown, so please send the new crash file if it happens again.
- **MTP, context and pictures setting kept coming back.** Every Load re-ran setup and rewrote them. A Load of an installed model is now direct, the picture-off choice sticks, a custom context size is shown and accepted, "MTP per request" is no longer frozen, and the new Draft layer switch is in Expert settings.
- **Your model library no longer moves.** `strata launch --data-dir` with a new folder used to move all existing models, packs and MTP files into it and rewrite your settings. It never moves them without asking now.
-- TODO(iD, iE): add the fixes of those streams with their issue numbers.
+- **Math in the chat (#3268).** Formulas are drawn with KaTeX: `$...$` and `\(...\)` inline, `$$...$$` and `\[...\]` as blocks. KaTeX and its fonts ship with the app, load only when an answer has a formula, and work offline. Prices like "$5 and $10" and anything in code stay text. Checked with the formulas from the report in a browser.
+- **Windows with a non-UTF-8 code page, such as Russian (#3202).** PowerShell output (the Documents folder, the save dialog, fonts, GPU names) was read in the wrong code page and came back as question marks; it is read as UTF-8 now. Python runs in UTF-8 mode from the launchers and the app's child processes. The engine and the picture reader carry a UTF-8 manifest, so a Cyrillic model folder arrives intact (Windows 10 1903 or newer). We reproduced the first two and tested the manifest with a small program; we could not run the engine on a Russian Windows, so a retest is welcome.
+- **Install folder (#3186, #3190).** The Windows installer asks for the folder (another drive when C: is short). Updates and silent installs keep the earlier folder. The docs say what stays on C: (the engine bundle, settings and chats) and that `STRATA_COMPONENTS_DIR` moves the engine bundle. We compiled the page logic with NSIS on a reduced script; the full installer is built at release.
+- **Linux engine killed with signal 9 and no out-of-memory (#3238).** A desktop with PipeWire and no rtkit can leave a finite `RLIMIT_RTTIME`. The server now warns at start when the limit is finite, and the engine-died message names it before the RAM hint. There is a troubleshooting entry.
+- **Tools find `gguf-py` in the Control package (#3269).** `tools/iq_pack.py` also looks in the install folder (`STRATA_INSTALL_DIR`, the app folder); the error names `STRATA_GGUF_PY`.
+- **Intel SYCL build (#3219).** `sycl/tools/build.sh` stops before the long compile when the `ocloc` of the base image is the one that crashes on the int8 kernels (Compute Runtime 26.22.38646.x, IGC 2.36.3) and says to `docker build --pull`; `STRATA_SYCL_I8_ESIMD=OFF` builds without those kernels (slower prompt experts, same answers). We could not reproduce the crash itself: a newer `ocloc` (26.31) builds the Battlemage target without errors.
+- **gfx906 (MI50, Radeon Pro VII) builds again (#3172, #3271).** The Qwen3.6 engine is not built for it, the W4A4 target gets `GGML_USE_HIP`, and the trellis overlay and `moe_f16_kq` sources are in the HIP list. All 161 targets compile for gfx906 and `strata` links; we have no gfx906 card, so it was not run on one.
+- **Trellis overlay with newer llama.cpp (#3217).** The overlay follows the signatures of the ggml-cuda it patches. On the pinned commit the generated files are byte-identical to before; against upstream 0ce8cd11 the old overlay failed to compile and the new one does.
+- **Settings text (#3208).** "Allow browser origins" says it does nothing in the app (the app's API already allows pages of any site) and where `cors_origins` goes, in all eight languages.
+- **Docs.** Build from git on Linux (#3191), the dual-socket CPU pool measurements, IOMMU check and no-numactl note (#3201; the NUMA-split pool from that report is not merged), and a two-card gfx906 tip with `--pipeline-windows 2` (+36% decode) and `--trim-stage-weights` (+5% prompt), measured by the reporter and off by default (#3173).
## Privacy
@@ -81,10 +91,12 @@ Speed numbers are medians of interleaved pairs on the same box, a fresh engine p
- **AMD and Intel on Linux:** the engine cannot tell whether a card drives a display, so the safety reserve (1536 MiB) stays unless you set one.
- **Benchmark:** `strata bench run` implements the draft protocol; a first real run on a release engine is still to come.
- **Two NVIDIA cards, batch:** a card with a very small expert cache can be slower with one group than with two (see New defaults).
-- TODO(bimodal): the R9700 prompt-speed modes (about half of fresh engines with MTP read prompts 10 to 17% slower) are in 0.2.0's limits; say here what changed.
+- **A100 (sm_80), #3252:** a report of decode -10% and prompts -20% since 0.1.45 was not reproduced. On a Tesla P100 (sm_60) and two RTX A4000 (sm_86) the 0.1.45 build against 0.1.44.1 measured decode x1.015 and x0.992, prompts x1.004 and x1.022 (5 interleaved rounds each), and the 0.2.1 tip is 7 to 10% faster in decode than 0.1.45 on both. Reading the commits found no sm_80-specific change. We have no A100; the reporter's numbers are needed.
+- **Fast drives, #3237:** the drive probe above is proposed and untested on a drive over 4 GB/s.
+- **NUMA-split CPU pool (#3201):** not part of 0.2.1; we have no dual-socket box to measure it.
- TODO(release): anything the gates find.
-Thanks to H3buss, who found that the Gyro import was missing `--compat-bf16` and where to put it (#3193); to bext1998, whose trace of the MTP download and the orphaned process (#3257) led to the progress and cancel fixes; and to Gazoo, who reported the card-selection, dropdown and re-download bugs (#3195, #3221, #3199). Thanks also to everyone who tested 0.2.0-beta and sent reports. TODO(release): add the names behind the iD and iE fixes if their analysis led directly to a fix.
+Thanks to H3buss, who found that the Gyro import was missing `--compat-bf16` and where to put it (#3193); to bext1998, whose trace of the MTP download and the orphaned process (#3257) led to the progress and cancel fixes; and to Gazoo, who reported the card-selection, dropdown and re-download bugs (#3195, #3221, #3199). Thanks also to everyone who tested 0.2.0-beta and sent reports.
---