Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Release notes 0.2.1: engine 0.1.46 and feed sequence 102 (proposed), file-name and Docker TODOs resolved, tuning tables per package

Niko1221d33bb832ca78
diff --git a/docs/release-notes-0.2.1.md b/docs/release-notes-0.2.1.md
index ba3b05afc..eac494fbc 100644
--- a/docs/release-notes-0.2.1.md
+++ b/docs/release-notes-0.2.1.md
@@ -1,9 +1,7 @@
-**0.2.1: a task board in the app, pictures on the Mac and on Intel Arc cards, `strata bench run` with a fixed standard benchmark, faster downloads, Docker images, and your own engine settings are no longer overwritten.** It also fixes the crashes, stuck downloads and GPU-choice bugs reported on 0.2.0-beta. Each new default below says what it was measured on and whether the answers are identical to 0.2.0. TODO(release): engine package version and feed sequence number (0.2.0 was 0.1.45.3 and sequence 101). Update with `strata update` (Terminal), Settings > Update (app), or `UPDATE.bat` / `./update.sh` (source installs).
+**0.2.1: a task board in the app, pictures on the Mac and on Intel Arc cards, `strata bench run` with a fixed standard benchmark, faster downloads, Docker images, and your own engine settings are no longer overwritten.** It also fixes the crashes, stuck downloads and GPU-choice bugs reported on 0.2.0-beta. Each new default below says what it was measured on and whether the answers are identical to 0.2.0. The engine packages are version 0.1.46 (0.2.0 had 0.1.45.3) and the update feed has sequence number 102. Update with `strata update` (Terminal), Settings > Update (app), or `UPDATE.bat` / `./update.sh` (source installs).
## Which file to download
-TODO(release): file names. Below, `0.2.1` stands for the final version string (0.2.0 used `0.2.0-beta`).
-
| System | Your graphics card | File |
| --- | --- | --- |
| Windows 10 / 11 | NVIDIA RTX, driver 580 or newer | `Strata-0.2.1-windows-cuda13.exe` |
@@ -26,7 +24,7 @@ The Terminal installs with one line and picks its engine by itself (`curl -fsSL
- **Pictures on Intel Arc cards.** Alchemist cards (A-series) read pictures by default through the CPU reader; Battlemage (B-series) stays opt-in (see Known limits). A new GPU picture reader for SYCL (`--vision gpu`) takes 0.16 s per picture on an A750 against 1.73 s on the CPU (its picture embeddings equal the CPU reader's to a cosine of 0.9959 to 0.9999). Ten varied pictures and a multi-picture chat all passed on the A750.
- **`strata bench run` runs the standard benchmark.** The default suite is `standard-v1`: fixed request files (34 of them, with a checksum list) in the Terminal archive, a scorer, a validator and a report that leaves out personal data. The Terminal archive now carries its commit, because a run without one does not count. The protocol text in `docs/BENCHMARK_PROTOCOL.md` is still a draft. See `docs/COMMUNITY_BENCHMARKS.md` for how to send a result.
- **Faster downloads.** Big model files come over 16 connections with several pieces per request (0.2.0 used 8 connections and a new connection for every piece). On the test PC, from Hugging Face, three interleaved pairs: 68.6 / 65.9 / 70.6 MB/s before, 74.4 / 74.8 / 74.9 MB/s now (+9%; the line itself tops out near 75 to 85 MB/s). The fetch of the draft layer (MTP) went from 25.0 / 28.1 / 37.6 MB/s to 47.4 / 50.5 / 52.2 MB/s, and its file list from 28 s to 4.6 s. ModelScope downloads, whose server reports no file length, now use several connections too. The files are the same and are still checked against their pinned SHA-256.
-- **Docker images.** Ready-made images for `ghcr.io/niko1221/strata:0.2.1-<flavor>`: `cuda13`, `cuda12`, `rocm-gfx120x` (RX 9000, R9700), `rocm-gfx110x` (RX 7000) and `rocm-gfx1151` (Strix Halo). They are built from the signed release packages by the repository's workflow. The start-up test passed on an RTX A4000 (cuda13 and cuda12) and on a Radeon AI PRO R9700 (rocm-gfx120x); the gfx110x and gfx1151 images were built but not run on a card. There is no Intel image yet. See `docs/DOCKER.md`. TODO(release): the workflow is started by the owner after the release; say so here or drop this line if the images are not up.
+- **Docker images.** Ready-made images for `ghcr.io/niko1221/strata:0.2.1-<flavor>`: `cuda13`, `cuda12`, `rocm-gfx120x` (RX 9000, R9700), `rocm-gfx110x` (RX 7000) and `rocm-gfx1151` (Strix Halo). They are built from the signed release packages by the repository's workflow. The start-up test passed on an RTX A4000 (cuda13 and cuda12) and on a Radeon AI PRO R9700 (rocm-gfx120x); the gfx110x and gfx1151 images were built but not run on a card. There is no Intel image yet. See `docs/DOCKER.md`. The workflow starts after this release is published, because it reads the release's own files, so the images appear a few hours later.
- **Your own engine settings stay.** The app and setup used to rewrite the run config on every Load. Options you set by hand (`--vram-reserve-mib`, control vectors, any option setup does not choose itself) are now kept in `user_args`, and what you set wins over what the app generates. See Fixes.
- **Draft layer (MTP) switch.** Settings > Expert has a Draft layer (MTP) on/off per model, and `strata launch --mtp on|off` does the same. A picture-off choice and a custom context size stick across loads.
- **Fit checks count the draft layer.** With MTP on, the draft layer takes about 1 GB of graphics memory; the model fit check and the expert-cache sizing now count it.
@@ -41,7 +39,7 @@ The Terminal installs with one line and picks its engine by itself (`curl -fsSL
Speed numbers are medians of interleaved pairs on the same box, a fresh engine per run; "pairs faster" counts the pairs the new default won. Each default has an off switch.
- **Mini PCs with a Radeon 780M (gfx1103), Linux: the matrix-core prompt set is on.** Measured on a Ryzen 7 8745HS with a Radeon 780M, 60 GB DDR5, Linux kernel 7.0.0-38, Qwen3.8-Flash-Next Q2_0 with MTP, 6 interleaved pairs: a 1K prompt 98.5 to 157.3 tok/s (x1.60, 6 of 6 pairs), a 4K prompt 115.1 to 233.3 (x2.03, 6 of 6), decode unchanged (x0.999 and x1.000). The greedy token ids equal the old default's in every pair (6 of 6, and 5 of 5 in the final build). Same answers, as far as the ids show. `STRATA_IGPU_PROMPT_SET=0` turns it off. It applies to the 780M and 760M on Linux only; 880M, 890M, Strix Halo and Windows stay off. The 780M's GPU can still reset on its own firmware (see Known limits); with the set on we saw no more resets than with it off, in small samples.
-- **Strix Halo (gfx1151): the hipBLASLt tuning table is on.** The packaged Strix Halo engines (Linux `strixhalo` and the Windows `rdna3` package) use the table for their card when their hipBLASLt version matches it. The measurement is a community A/B from `bench/results/2026-10-05-community-gfx1151` (Linux, ROCm 7.17, hipBLASLt 1.5.0): prompts 250.9 to 528.9 tok/s (2.11x). We have no Strix Halo and did not repeat it. The table changes which matrix-product algorithm reads the prompt, so the answers are not bit-identical to plain hipBLAS (on an R9700 the first-token KL was 0.016, against a run-to-run difference of 0.00004; top-1 the same on 24 of 24 prompts). `STRATA_HIPBLASLT_TABLE=off` turns it off. On every other card the table is opt-in (`STRATA_HIPBLASLT_TABLE=on`): on the R9700 a table calibrated for the packaged hipBLASLt gave about +1% prompt and -2% decode, so none is shipped as a default. TODO(owner): the table is packaged per card; confirm the final package contents when the engines are built (Windows `rdna3`: gfx1100 and gfx1151; Linux `rx7900`: gfx1100; Linux `strixhalo`: gfx1151; the R9700 packages carry none that matches).
+- **Strix Halo (gfx1151): the hipBLASLt tuning table is on.** The packaged Strix Halo engines (Linux `strixhalo` and the Windows `rdna3` package) use the table for their card when their hipBLASLt version matches it. The measurement is a community A/B from `bench/results/2026-10-05-community-gfx1151` (Linux, ROCm 7.17, hipBLASLt 1.5.0): prompts 250.9 to 528.9 tok/s (2.11x). We have no Strix Halo and did not repeat it. The table changes which matrix-product algorithm reads the prompt, so the answers are not bit-identical to plain hipBLAS (on an R9700 the first-token KL was 0.016, against a run-to-run difference of 0.00004; top-1 the same on 24 of 24 prompts). `STRATA_HIPBLASLT_TABLE=off` turns it off. On every other card the table is opt-in (`STRATA_HIPBLASLT_TABLE=on`): on the R9700 a table calibrated for the packaged hipBLASLt gave about +1% prompt and -2% decode, so none is shipped as a default. The tables are packaged per card (`tuning/` inside the engine package): Windows `rdna3` gfx1100 and gfx1151, Windows `hip` (RX 9000, R9700) gfx1201, Linux `rx7900` gfx1100, Linux `strixhalo` gfx1151; the Linux RX 9000 / R9700 package carries none, because no table matches its hipBLASLt. Only the gfx1151 table is used by default.
- **Two NVIDIA cards, several requests at once: one batch group.** With a layer split over exactly two NVIDIA cards and `--batch N`, the engine now runs one serial group instead of two pipelined groups. Measured on two RTX A4000 16 GB with Qwen3.8-Flash-Next IQ3_XXS, 5 interleaved pairs, aggregate tokens per second: 2 requests +15.9% (5 of 5 pairs), 4 requests +12.0% (5 of 5), 8 requests +14.7% (3 of 3). Every slot's tokens equal the solo run's, and the old default's, under the fixed identity flags (2 and 4 slots, 64 greedy tokens). A single request is not changed. It is not faster everywhere: when most hot experts sit in the CPU, as under the identity gate's small cache, the pipelined groups were about 10% faster (24.4 against 22.0 tok/s), so a card with a very small cache can lose here. `STRATA_BATCH_GROUPS_NV_SERIAL=0` restores the old default; `--batch-groups auto` or a number asks for the pipeline. AMD cards, three or more cards and the single-request paths are unchanged. TODO(gate): the 5-pair A/B of the final engine at 4 slots (2 of 5 pairs done: new default faster 2 of 2) and the batch identity check on the final build.
- **A card that draws no display keeps less in reserve.** The reserve of graphics memory the engine leaves free is now 512 MiB (700 MiB with pictures) on a card the probe says drives no screen (a compute-only card), instead of 1536 MiB. A card that drives a display keeps the 1536 / 2048 MiB safety default, and a number you set yourself (0 included) is always used as it is. Only the amount of free memory changes, so the answers are identical and the expert cache gets room for more experts. We did not time this ourselves: the report behind it (#3215) measured 80 against 50 tok/s with the forced reserve. Where the probe cannot tell (AMD and Intel on Linux) the safety default stays.
- **Packed models (`experts.bin`) on Linux: I/O prefetch and the decode gate are on when the page cache has room for at least 55% of the experts.** Measured on an RTX 3060 12 GB (Core Ultra 7 265), Qwen3.8-Flash-Next IQ3_XXS 43 GB pack with `--mmap-experts`, RAM limited to 32 GB by cgroup, cold page cache, 5 interleaved pairs: decode story / code / document 26.0 / 28.3 / 18.5 to 35.2 / 34.4 / 24.7 tok/s (x1.38 / x1.21 / x1.33, 5 of 5 pairs each), a 4K prompt x1.04 (4 of 5), a 20K prompt x1.06 (5 of 5). Same answers: the fixed-flag identity gate (10 prompts, 4K and 20K exact prompt checks) is identical with no RAM limit and with the 32 GB limit. With 16 GB of RAM the page cache holds 38% of the experts, so the default stays off: with it on, a cold 4K prompt read 4 to 6% slower. `STRATA_IO_PREFETCH=0` turns it off and `=1` forces it on. Linux only; only the IQ3_XXS pack was tested (not Q2_0, IQ3_S or Coder packs).