Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs/STRIX_HALO_WINDOWS.md: a Strix Halo Windows guide, measured again on 0.2.7 (#3029)

Niko1221ee1a6efd5cea
diff --git a/docs/README.md b/docs/README.md
index ed4ebe063..9baad4415 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -22,7 +22,7 @@ Use the guide for your GPU and operating system. Model and feature support can d
| --- | --- |
| NVIDIA RTX 20 / 30 / 40 / 50 | [Install](INSTALL.md#what-you-need) |
| AMD Radeon (RX and AI PRO R9700) | [AMD / HIP](AMD_HIP.md) |
-| AMD Ryzen AI Max / Strix Halo | [Strix Halo](STRIX_HALO.md) |
+| AMD Ryzen AI Max / Strix Halo | [Strix Halo](STRIX_HALO.md), [on Windows](STRIX_HALO_WINDOWS.md) |
| Intel Arc on Linux | [Intel Arc](INTEL_ARC.md) |
| Apple Silicon | [Mac / Metal](MAC.md) |
| More than one GPU | [Multiple GPUs](MULTI_GPU.md) |
diff --git a/docs/STRIX_HALO.md b/docs/STRIX_HALO.md
index e37db19fc..72e55c066 100644
--- a/docs/STRIX_HALO.md
+++ b/docs/STRIX_HALO.md
@@ -32,6 +32,8 @@ change BIOS or kernel settings. The manual Linux toolchain below is for reproduc
DeepSeek V4.1-Flash comes with its own engine package on Windows and needs nothing extra: pick the model and the app or `strata launch`
downloads it ([DeepSeek on Strix Halo](#deepseek-v41-flash)).
+On Windows, [STRIX_HALO_WINDOWS.md](STRIX_HALO_WINDOWS.md) has the speeds measured on 0.2.7 with Qwen IQ3_S and DeepSeek, how memory is used and the traps.
+
## DeepSeek V4.1-Flash
The model runs through its own engine, `strata-dsv41`, as on the other platforms ([DEEPSEEK41.md](DEEPSEEK41.md)); on Windows the
diff --git a/docs/STRIX_HALO_WINDOWS.md b/docs/STRIX_HALO_WINDOWS.md
new file mode 100644
index 000000000..3f4fca8b4
--- /dev/null
+++ b/docs/STRIX_HALO_WINDOWS.md
@@ -0,0 +1,125 @@
+# Strix Halo (Ryzen AI Max, gfx1151) on Windows: install, what you get, what to know
+
+This page is for a Windows PC with a Ryzen AI Max "Strix Halo" chip (Radeon 8060S / 8050S, `gfx1151`). It says what Strata does there out
+of the box, what speed to expect with Qwen3.8-Flash-Next IQ3_S and with DeepSeek V4.1-Flash, and the traps on this hardware. The Linux
+build, the toolchain and the opt-in switches are on [STRIX_HALO.md](STRIX_HALO.md); the DeepSeek details are in
+[DEEPSEEK41.md](DEEPSEEK41.md).
+
+The structure of this page comes from the first Windows guide for this PC, written by HaiYunChuang (pull request #3029). It measured
+engines 0.1.41 to 0.1.47 with a hand-made configuration (`--mmap-experts`, a list of switches, a batch file). Since then the app and
+`strata launch` have taken over most of that, so this version describes the 0.2.7 install and every number in it was measured again.
+
+## 1. What this PC reaches
+
+Measured on 2026-10-26 on a GMKtec NucBox EVO-X2 (Ryzen AI Max+ 395, 128 GB shared memory: a 64 GB BIOS carve-out for the graphics chip
+and 64 GB that Windows sees; Windows 11 Pro, AMD Adrenalin 26.10.1 / driver 32.0.32015.7029). Strata was the 0.2.7 release candidate
+(Terminal and engine 0.1.51 built from commit `3e468f5de`, ROCm 10.2.0a20260930). The server was started with `strata launch` and
+nothing else was set by hand; the speeds come from `tools/strata_bench.py` against the running server (three runs per test, medians,
+the engine's own timings; the model is loaded before the clock starts). The numbers can still move before the release is cut.
+
+**Qwen3.8-Flash-Next IQ3_S** (`strata launch --family qwen --model iq3_s`; 131,072 tokens of context, int8 KV, pictures on):
+
+| Test | Result |
+| --- | --- |
+| Short chat, answer speed | **44.0 tok/s** (40.6 to 59.5: code-like answers are faster than prose) |
+| Short chat, time to first token | 0.60 s |
+| 4K prompt, read speed | **755 tok/s** (first token after 5.4 s) |
+| 16K prompt | **821 tok/s** (19.6 s) |
+| 32K prompt | **792 tok/s** (40.6 s) |
+| Coding-agent session, 6 turns | 69% of the prompt came from the conversation cache; answers 47 tok/s; 4.5 s to the first token per turn |
+
+**DeepSeek V4.1-Flash 2.00 bpw** (`strata launch --family deepseek41 --model ist-das-2.00bpw`, Faster mode, the drafter on; the 4K row is
+the default 8K context, the others 64K context with `--context 64k`):
+
+| Test | Result |
+| --- | --- |
+| Short chat, answer speed | **12.1 to 13.6 tok/s** |
+| 4K prompt | **214 to 226 tok/s** (about 19 s to the first token) |
+| 16K prompt | **326 tok/s** (51 s) |
+| 32K prompt | **342 tok/s** (97 s) |
+| Coding-agent session, 6 turns | 70% from the conversation cache; answers 14.2 tok/s; 17 to 31 s to the first token per turn |
+
+These are speed measurements. They say nothing about answer quality, and the short-chat range depends on the text. The DeepSeek 2.25 bpw set
+and the other Qwen sizes were not measured for this page.
+
+## 2. Memory and the BIOS
+
+- Allocate a **fixed carve-out** to the graphics chip in the BIOS (64 GB on the measured PC). The engine counts the carve-out and the
+ RAM that Windows sees as one pool and does not change any BIOS or driver setting.
+- The graphics chip and the CPU share that pool. For Qwen IQ3_S the engine reads the experts into RAM, keeps them there (it does not
+ register the arena with the GPU driver on an integrated chip) and gives the expert cache every slot: the start log says
+ `expert cache auto ... -> 24576 slots` and `expert arena: NOT registered with the GPU driver (integrated GPU, UMA mode)`. The process held
+ about 49 GB and 10 GB of RAM stayed free. Close other large programs before you load a model this size.
+- The default of four parallel slots takes 7.5 GiB of the shared pool for slot sessions (the log says `--batch: 4 slot sessions ... take
+ 7.54 GiB`). With IQ3_S the cache still held every expert, so there was nothing to gain from fewer slots here; with a bigger model there may be.
+
+## 3. Install
+
+1. Use the Windows app (`Strata-<version>-windows-x64-setup.exe`) or the Terminal. The engine for this chip is the `windows-x86_64-hip-rdna3`
+ package; the app or `strata launch` downloads it on the first start, and `strata update` keeps it current.
+2. Choose the model in the app, or `strata launch --family qwen --model iq3_s`. With model files you already have, add
+ `--model-path <folder>` (for IQ3_S the two shards are in one folder: shard 1 is the main model, shard 2 holds the per-layer embedding
+ table and must stay a separate file).
+3. The first start took about 20 minutes on the measured PC with the model files already on disk: the engine package (158 MB), the
+ check of the 84 GB of model files, the picture reader (0.9 GB), the draft layer (about 5 GB) and the one-time pack. The next start
+ took under two minutes.
+
+What `strata launch` wrote for IQ3_S, with no switches of mine (these are the settings to copy if you drive the engine by hand):
+
+```
+--expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp <draft layer>
+--max-context 131072 --kv int8 --vision --vram-reserve-mib 700 (backend hip, GPU picture reader)
+```
+
+and for DeepSeek 2.00 bpw: `--max-context 8192 --dspark auto --dspark-conf 0.65 --prefill-chunk 16384`, with `DS4_ASYNC_T=4` in the
+environment and the package's `rocm/bin` folder on the library path.
+
+## 4. What is on by default
+
+- On `gfx1151` the engine turns on 19 exact speed switches by itself (the start line lists them; `STRATA_GFX1151_DEFAULTS=0` turns them all
+ off, and a switch you set is kept). They are the ones that give the same answers as without. `STRATA_HC_Q8=1` is also on here (decode
+ about 9% faster, near-tie tokens can differ); `STRATA_HC_Q8=0` keeps the old read.
+- The hipBLASLt table for `gfx1151` ships in the package and is chosen without a setting (`prefill gemm: hipBLASLt tuning enabled (32 rows,
+ gfx1151, version 100500)` in the log). A hand-built engine has to be pointed at it: [AMD_HIP.md](AMD_HIP.md).
+- `--mmap-experts` is not needed. The first Windows guide found that without it the pinned expert arena used up the graphics memory
+ budget and decode fell to 28 tok/s on engine 0.1.41; the engine on this page sizes the cache from the shared pool and reaches every slot.
+- DeepSeek runs in "Faster" mode here: 5 of 6 experts per word. The app and `strata launch` say so; the quality change is small.
+
+**`STRATA_SH_STREAM` (the shared expert on its own stream) is on by default on `gfx1151`, and on this Windows PC it is slower.** The first
+Windows guide found the same on engine 0.1.41 (+13% decode with it off). Measured again on 0.2.7 with Qwen IQ3_S, three starts each, run
+in turn (on, off, on, off, on, off), the same quick suite after every start:
+
+| Start | `STRATA_SH_STREAM` | Chat decode, median of 9 answers (tok/s) | 4K prompt (tok/s) |
+| --- | --- | ---: | ---: |
+| 1 | 1 (default) | 43.9 | 742.9 |
+| 2 | 0 | 48.3 | 742.9 |
+| 3 | 1 | 43.6 | 746.8 |
+| 4 | 0 | 46.0 | 740.8 |
+| 5 | 1 | 43.2 | 745.5 |
+| 6 | 0 | 46.6 | 739.4 |
+
+With the switch off, decode was **6.9% faster** (46.6 against 43.6, the median of the three starts) and the code-like question 14% faster
+(67 against 59 tok/s); all three pairs went the same way. The prompt speed did not change. Setting it takes `strata launch ... --set
+STRATA_SH_STREAM=0`; it changes only the order the work runs in, so the answers stay the same. Whether Windows `gfx1151` gets this by
+default is a separate decision. This is one model, one PC and short answers; DeepSeek was not tested with it.
+
+
+## 5. Traps
+
+1. **Keep RAM free.** Qwen IQ3_S held 49 GB of 63.6 GB. A browser with many tabs, or a second model, is enough to push the PC into paging.
+2. **Do not run two engines.** One model at a time on this PC; `strata launch --status` shows what is serving, `strata launch --stop` ends it.
+3. **The graphics driver.** The 2025-03 driver failed at 32K context for DeepSeek. The measured driver is Adrenalin 26.10.1 (32.0.32015.7029).
+4. **If Windows resets the driver** (screen flicker, the engine dies mid-answer), see the driver-reset entry in
+ [TROUBLESHOOTING.md](TROUBLESHOOTING.md#amd-cards).
+5. **Other tuning from the first guide** (a longer `--prefill`, `STRATA_PF_*` switches, `--lookup-chain`): those results are for engines
+ 0.1.41 to 0.1.47 and a hand-made configuration. They were not repeated here, so they are not part of this page.
+
+## 6. Reproduce it
+
+```
+strata launch --family qwen --model iq3_s # then, from a source checkout:
+python tools/strata_bench.py --url http://127.0.0.1:8080 --runs 3 --long
+```
+
+`--long` adds the 32K prompt. For DeepSeek add `--context 64k` to `strata launch` first; the default 8K context skips the 16K prompt and
+the agent session.