Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Release notes 0.2.1 (draft): downloads table, what is new, measured defaults, fixes, privacy, known limits, TODO markers for the running streams

Niko122124e56539c573
diff --git a/docs/release-notes-0.2.1.md b/docs/release-notes-0.2.1.md
new file mode 100644
index 000000000..385ba1f91
--- /dev/null
+++ b/docs/release-notes-0.2.1.md
@@ -0,0 +1,93 @@
+**0.2.1: a task board in the app, pictures on the Mac and on Intel Arc cards, `strata bench run` with a fixed standard benchmark, faster downloads, Docker images, and your own engine settings are no longer overwritten.** It also fixes the crashes, stuck downloads and GPU-choice bugs reported on 0.2.0-beta. Each new default below says what it was measured on and whether the answers are identical to 0.2.0. TODO(release): engine package version and feed sequence number (0.2.0 was 0.1.45.3 and sequence 101). Update with `strata update` (Terminal), Settings > Update (app), or `UPDATE.bat` / `./update.sh` (source installs).
+
+TODO(streams still running, not described here): speed regressions (f-021-iD: #3252 #3262 #3237 #3204), Linux and Windows fixes (f-021-iE: #3238 #3269 #3191 #3202 #3186 #3190 #3268 #3208 #3219 #3172 #3271 #3217 and the community items #3201 #3173), packed-model loading (f-021-zc) and the R9700 prompt-speed modes (f-021-bimodal). Add each one when it is merged.
+
+## Which file to download
+
+TODO(release): file names. Below, `0.2.1` stands for the final version string (0.2.0 used `0.2.0-beta`).
+
+| System | Your graphics card | File |
+| --- | --- | --- |
+| Windows 10 / 11 | NVIDIA RTX, driver 580 or newer | `Strata-0.2.1-windows-cuda13.exe` |
+| | NVIDIA, older card or driver older than 580 | `Strata-0.2.1-windows-cuda12.exe` |
+| | AMD Radeon RX 9000 series or Radeon AI PRO R9700 | `Strata-0.2.1-windows-hip.exe` |
+| | AMD Radeon RX 7000 series or Ryzen AI Max (Strix Halo) | `Strata-0.2.1-windows-hip-rdna3.exe` |
+| Linux | NVIDIA RTX, driver 580 or newer | `Strata-0.2.1-linux-nvidia-cuda13.AppImage` |
+| | NVIDIA, older card or driver older than 580 | `Strata-0.2.1-linux-nvidia-cuda12.AppImage` |
+| | AMD Radeon RX 9000 series or Radeon AI PRO R9700 | `Strata-0.2.1-linux-amd-rx9070.AppImage` |
+| | AMD Radeon RX 7000 series | `Strata-0.2.1-linux-amd-rx7900.AppImage` |
+| | AMD Ryzen AI Max (Strix Halo) | `Strata-0.2.1-linux-amd-strixhalo.AppImage` |
+| Mac | Apple silicon, macOS 15 or newer | `Strata-0.2.1-mac.dmg` (signed and notarised by Apple) |
+
+The Terminal installs with one line and picks its engine by itself (`curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh`, or `irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex` on Windows). The other files in this release are the engine and Terminal packages, the signed update feed and the in-app update files; you do not need to download them. Docker images are separate (below).
+
+## What is new
+
+- **A task board in the app.** On the Tasks screen, a Schedule | Board switch shows your scheduled tasks and loops as a board: what is running, what is waiting for your approval (from every chat), and what failed. Each run has one plain line, only the newest problem of a task is shown, and a problem can be dismissed. The sidebar shows how many tasks need you. All eight app languages are translated.
+- **Pictures on the Mac.** The Mac app and Terminal now read pictures by default. The reader runs on the Mac's GPU (Metal), built into `strata-vision` and signed with the app. Measured on an M4 Mac: a 260-token picture 0.45 s on the GPU reader against 50.7 s on the CPU, a 972-token picture 2.4 s against 194 s; the GPU reader takes about 1.2 GB of memory. On the 972-token picture 8 of 972 picture tokens differ from the CPU reader (cosine down to 0.48, in the text area); the answers were correct. `--vision cpu` is still there and warns that it takes about 50 seconds per picture.
+- **Pictures on Intel Arc cards.** Alchemist cards (A-series) read pictures by default through the CPU reader; Battlemage (B-series) stays opt-in (see Known limits). A new GPU picture reader for SYCL (`--vision gpu`) takes 0.16 s per picture on an A750 against 1.73 s on the CPU (its picture embeddings equal the CPU reader's to a cosine of 0.9959 to 0.9999). Ten varied pictures and a multi-picture chat all passed on the A750.
+- **`strata bench run` runs the standard benchmark.** The default suite is `standard-v1`: fixed request files (34 of them, with a checksum list) in the Terminal archive, a scorer, a validator and a report that leaves out personal data. The Terminal archive now carries its commit, because a run without one does not count. The protocol text in `docs/BENCHMARK_PROTOCOL.md` is still a draft. See `docs/COMMUNITY_BENCHMARKS.md` for how to send a result.
+- **Faster downloads.** Big model files come over 16 connections with several pieces per request (0.2.0 used 8 connections and a new connection for every piece). On the test PC, from Hugging Face, three interleaved pairs: 68.6 / 65.9 / 70.6 MB/s before, 74.4 / 74.8 / 74.9 MB/s now (+9%; the line itself tops out near 75 to 85 MB/s). The fetch of the draft layer (MTP) went from 25.0 / 28.1 / 37.6 MB/s to 47.4 / 50.5 / 52.2 MB/s, and its file list from 28 s to 4.6 s. ModelScope downloads, whose server reports no file length, now use several connections too. The files are the same and are still checked against their pinned SHA-256.
+- **Docker images.** Ready-made images for `ghcr.io/niko1221/strata:0.2.1-<flavor>`: `cuda13`, `cuda12`, `rocm-gfx120x` (RX 9000, R9700), `rocm-gfx110x` (RX 7000) and `rocm-gfx1151` (Strix Halo). They are built from the signed release packages by the repository's workflow. The start-up test passed on an RTX A4000 (cuda13 and cuda12) and on a Radeon AI PRO R9700 (rocm-gfx120x); the gfx110x and gfx1151 images were built but not run on a card. There is no Intel image yet. See `docs/DOCKER.md`. TODO(release): the workflow is started by the owner after the release; say so here or drop this line if the images are not up.
+- **Your own engine settings stay.** The app and setup used to rewrite the run config on every Load. Options you set by hand (`--vram-reserve-mib`, control vectors, any option setup does not choose itself) are now kept in `user_args`, and what you set wins over what the app generates. See Fixes.
+- **Draft layer (MTP) switch.** Settings > Expert has a Draft layer (MTP) on/off per model, and `strata launch --mtp on|off` does the same. A picture-off choice and a custom context size stick across loads.
+- **Fit checks count the draft layer.** With MTP on, the draft layer takes about 1 GB of graphics memory; the model fit check and the expert-cache sizing now count it.
+- **The model page shows the engine's own pick for your PC** (the memory tiers of 0.2.0), with Gyro-M named as the other choice at 40 GB.
+- **The chat footer speed is the writing speed.** It divides output tokens by decode time, so a tool-call turn no longer shows "632 tok/s" when the engine ran at 50 to 75.
+- **A wrong download says so.** An installer for another make of graphics card (an AMD build on an NVIDIA PC) tells you in plain words which file to get and does not unpack its engine. An installed engine with no model loaded is no longer reported as "No Strata engine".
+- **Terminal.** A model that is already on disk says "already downloaded". `strata monitor` finds a running `strata launch` without `--url`.
+- **The standard AMD packages carry the hipBLASLt tuning tables** and the engine start picks the one for the card (see New defaults for when it is used).
+
+## New defaults, with what we measured
+
+Speed numbers are medians of interleaved pairs on the same box, a fresh engine per run; "pairs faster" counts the pairs the new default won. Each default has an off switch.
+
+- **Mini PCs with a Radeon 780M (gfx1103), Linux: the matrix-core prompt set is on.** Measured on a Ryzen 7 8745HS with a Radeon 780M, 60 GB DDR5, Linux kernel 7.0.0-38, Qwen3.8-Flash-Next Q2_0 with MTP, 6 interleaved pairs: a 1K prompt 98.5 to 157.3 tok/s (x1.60, 6 of 6 pairs), a 4K prompt 115.1 to 233.3 (x2.03, 6 of 6), decode unchanged (x0.999 and x1.000). The greedy token ids equal the old default's in every pair (6 of 6, and 5 of 5 in the final build). Same answers, as far as the ids show. `STRATA_IGPU_PROMPT_SET=0` turns it off. It applies to the 780M and 760M on Linux only; 880M, 890M, Strix Halo and Windows stay off. The 780M's GPU can still reset on its own firmware (see Known limits); with the set on we saw no more resets than with it off, in small samples.
+- **Strix Halo (gfx1151): the hipBLASLt tuning table is on.** The packaged Strix Halo engines (Linux `strixhalo` and the Windows `rdna3` package) use the table for their card when their hipBLASLt version matches it. The measurement is a community A/B from `bench/results/2026-10-05-community-gfx1151` (Linux, ROCm 7.17, hipBLASLt 1.5.0): prompts 250.9 to 528.9 tok/s (2.11x). We have no Strix Halo and did not repeat it. The table changes which matrix-product algorithm reads the prompt, so the answers are not bit-identical to plain hipBLAS (on an R9700 the first-token KL was 0.016, against a run-to-run difference of 0.00004; top-1 the same on 24 of 24 prompts). `STRATA_HIPBLASLT_TABLE=off` turns it off. On every other card the table is opt-in (`STRATA_HIPBLASLT_TABLE=on`): on the R9700 a table calibrated for the packaged hipBLASLt gave about +1% prompt and -2% decode, so none is shipped as a default. TODO(owner): the table is packaged per card; confirm the final package contents when the engines are built (Windows `rdna3`: gfx1100 and gfx1151; Linux `rx7900`: gfx1100; Linux `strixhalo`: gfx1151; the R9700 packages carry none that matches).
+- **Two NVIDIA cards, several requests at once: one batch group.** With a layer split over exactly two NVIDIA cards and `--batch N`, the engine now runs one serial group instead of two pipelined groups. Measured on two RTX A4000 16 GB with Qwen3.8-Flash-Next IQ3_XXS, 5 interleaved pairs, aggregate tokens per second: 2 requests +15.9% (5 of 5 pairs), 4 requests +12.0% (5 of 5), 8 requests +14.7% (3 of 3). Every slot's tokens equal the solo run's, and the old default's, under the fixed identity flags (2 and 4 slots, 64 greedy tokens). A single request is not changed. It is not faster everywhere: when most hot experts sit in the CPU, as under the identity gate's small cache, the pipelined groups were about 10% faster (24.4 against 22.0 tok/s), so a card with a very small cache can lose here. `STRATA_BATCH_GROUPS_NV_SERIAL=0` restores the old default; `--batch-groups auto` or a number asks for the pipeline. AMD cards, three or more cards and the single-request paths are unchanged. TODO(gate): the 5-pair A/B of the final engine at 4 slots (2 of 5 pairs done: new default faster 2 of 2) and the batch identity check on the final build.
+- **A card that draws no display keeps less in reserve.** The reserve of graphics memory the engine leaves free is now 512 MiB (700 MiB with pictures) on a card the probe says drives no screen (a compute-only card), instead of 1536 MiB. A card that drives a display keeps the 1536 / 2048 MiB safety default, and a number you set yourself (0 included) is always used as it is. Only the amount of free memory changes, so the answers are identical and the expert cache gets room for more experts. We did not time this ourselves: the report behind it (#3215) measured 80 against 50 tok/s with the forced reserve. Where the probe cannot tell (AMD and Intel on Linux) the safety default stays.
+- **TODO(zc): packed models (`experts.bin`) read with the I/O prefetch by default when the page cache has room for at least 55% of the experts.** Measured on an RTX 3060 12 GB (Core Ultra 7 265, IQ3_XXS 43 GB pack, cgroup RAM 32 GB): decode 1.29 / 1.20 / 1.34x, identity gate identical, but 16 GB RAM prompts 4K -4% to -6% so the rule gates it. Not merged yet; write the final numbers and the rule once f-021-zc is merged.
+- **TODO(iD, iE, bimodal): any default or fix from those streams that changes speed, with the box it was measured on.**
+
+## Fixes from user reports
+
+- **Settings overwritten (#3215, #3270, #3216).** The app forced `--vram-reserve-mib` to 1536 (2048 with pictures) on every Load whatever your file said, which cut decode from 80 to 50 tok/s for one reporter, and a setup run that a Load started dropped every option it did not choose itself, control vectors included. Options set by hand are kept now and win over generated ones; an old file's hand edits are moved into `user_args`.
+- **Choosing graphics cards (#3203, #3195, #3256, #3221, #3211).** Automatic no longer picks a card the engine has no code for (an AMD integrated GPU in front of a discrete card). One selected card is sent as a single card, not as a list the installer refuses. A card order from the settings file (`"gpu": [1,0]`, `"gpu_order": "as_given"`) is kept instead of being replaced by the app's own pick. The card list no longer ticks two cards with one click when two makes both count from 0. The dropdown rows fit long card names. The Monitor opens on the cards the model is assigned to.
+- **Downloads (#3257, #3214, #3193).** The draft-layer download shows its progress in the app (it showed "0.0 / 0.0 GB"), runs again on a flaky link with fewer connections, and has one writer per folder. Cancelling, or a killed setup, now ends the whole process tree on Windows, so no orphan keeps writing. Gyro's low-RAM pack is written with `--compat-bf16`, which stopped the "command failed (exit 1): python.exe" at the packing step.
+- **Re-download of a model you already have (#3199).** A model loaded from your own GGUF folder keeps that folder, so unloading and loading it again no longer downloads it a second time.
+- **FP4-Local setup died at step 6 (#3227).** Setup now installs the pinned torch, transformers and safetensors the converter imports, before a prepare that needs them, and names a missing package at once instead of after 20 minutes.
+- **The model stopped while starting (#3192).** A start that stopped because no graphics memory was left for the expert cache is tried once more with less kept free.
+- **Crash on launch (Windows 11, exit code 0xC0000409, no log).** A failed start on Windows and Mac now leaves a log and a crash file in the app folder and shows a message box instead of closing. The reporter's own panic message is still unknown, so please send the new crash file if it happens again.
+- **MTP, context and pictures setting kept coming back.** Every Load re-ran setup and rewrote them. A Load of an installed model is now direct, the picture-off choice sticks, a custom context size is shown and accepted, "MTP per request" is no longer frozen, and the new Draft layer switch is in Expert settings.
+- **Your model library no longer moves.** `strata launch --data-dir` with a new folder used to move all existing models, packs and MTP files into it and rewrite your settings. It never moves them without asking now.
+- TODO(iD, iE): add the fixes of those streams with their issue numbers.
+
+## Privacy
+
+- Update checks run only when you click. Opening Settings, Engine and Update makes no network call; the last result is shown from your computer, and a new check starts only from the button. Offline, the check says "No internet connection. Strata works without it; try again when you are online."
+- The assistant asks before it first searches the web or opens a web page. In Manual mode you are asked once per chat and the answer holds for that chat. Settings > Tools has a switch to skip the question (off by default), and a scheduled task cannot search unless you turned that on. Bypass mode does not ask.
+- Starting, loading and chatting with a model on disk make no network calls. A test starts the host with every non-loopback request recorded and finds none. The old starter-model list can no longer download anything; it is display only.
+- What Strata downloads and when is in `docs/SAFETY.md`.
+
+## Known limits
+
+- **Windows:** the installers are not code-signed yet, so Windows shows "Windows protected your PC": choose **More info**, then **Run anyway**.
+- **Windows with an AMD card:** nothing has run on a real AMD card on Windows. The gfx12 prompt defaults and the 780M prompt set are off there. The Windows `rdna3` package (RX 7000 and Strix Halo) was built and scanned but not run on those cards by the maintainers.
+- **Radeon 780M:** the GPU can reset with a firmware error (MES REMOVE_QUEUE) when the kernel moves memory the GPU uses. On our box the default engine lost 1 to 4 of 10 starts to it with or without the new prompt set, and the driver recovered each time. The firmware is the cause; we did not install new firmware.
+- **Intel Arc Battlemage (B-series):** pictures stay opt-in. A first picture on a B70 once reset the card (xe driver); we could not repeat it on the A-series card and no B-series card was available for 0.2.1.
+- **Mac:** the GPU picture reader was measured on an M4 only. The CPU reader is about 50 seconds per picture.
+- **Strix Halo:** the hipBLASLt table default rests on a community measurement; please send the start-up line `hipBLASLt tuning table: ...` and a prompt run with the table on and off.
+- **Docker:** the `rocm-gfx110x` and `rocm-gfx1151` images were not run on a card, and there is no Intel image.
+- **AMD and Intel on Linux:** the engine cannot tell whether a card drives a display, so the safety reserve (1536 MiB) stays unless you set one.
+- **Benchmark:** `strata bench run` implements the draft protocol; a first real run on a release engine is still to come.
+- **Two NVIDIA cards, batch:** a card with a very small expert cache can be slower with one group than with two (see New defaults).
+- TODO(bimodal): the R9700 prompt-speed modes (about half of fresh engines with MTP read prompts 10 to 17% slower) are in 0.2.0's limits; say here what changed.
+- TODO(release): anything the gates find.
+
+Thanks to H3buss, who found that the Gyro import was missing `--compat-bf16` and where to put it (#3193); to bext1998, whose trace of the MTP download and the orphaned process (#3257) led to the progress and cancel fixes; and to Gazoo, who reported the card-selection, dropdown and re-download bugs (#3195, #3221, #3199). Thanks also to everyone who tested 0.2.0-beta and sent reports. TODO(release): add the names behind the iD and iE fixes if their analysis led directly to a fix.
+
+---
+
+Strata is free and open source. If it runs well on your PC, a coffee keeps the work on it going:
+
+<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>