Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

hip: 8-bit hyper-connection weights (STRATA_HC_Q8) on by default on gfx1151, STRATA_HC_Q8=0 keeps the bf16 read; one start-up line; docs and 0.2.2 release note with the KL table and the CPU-vs-GPU rounding baseline

Niko12216d4874a491cc
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.md
index eab17bdf7..e2e759071 100644
--- a/docs/AMD_HIP.md
+++ b/docs/AMD_HIP.md
@@ -1097,3 +1097,27 @@ passes. Additional unpublished upstream fixture suites are not covered.
The published performance table explicitly identifies the measured development
snapshot; it must not be read as a benchmark of every subsequent rebase. Vision, long-context stress, broad answer-quality
equivalence, other AMD cards, and mixed-vendor inference are not validated here.
+
+## Strix Halo: 8-bit hyper-connection weights (default on gfx1151)
+
+On a Strix Halo (gfx1151) the hyper-connection down / up projections are read as Q8_0 by default (quantized at load from the pack, or the GGUF's own Q8_0),
+which halves the 13.3 MB each read moves. Decode is about 9% faster (5 interleaved pairs per prompt on IQ3_S, MTP 4 drafts / min-p 0.5; json / code / tool
+call / chat / agent between +7.9% and +10.7%). `STRATA_HC_Q8=0` restores the bf16 read. Other cards are unchanged (gfx12 has had it on since 0.1.4x).
+Not bit-identical to the bf16 read: a greedy answer can pick another token at a near tie and then go its own way.
+
+How far it moves the model, measured with the quant-quality yardstick (teacher forced, 5 texts of 1,000 tokens, reference = bf16 read; RTX A4000, IQ3_XXS,
+2026-10-22 — the Q8_0 weights are the same bytes on every card):
+
+| text | top-1 same | KL |
+| --- | ---: | ---: |
+| code | 97.9% | 0.0077 |
+| documents | 93.5% | 0.0229 |
+| English | 94.7% | 0.0107 |
+| other languages | 98.0% | 0.0060 |
+| agent transcript | 93.0% | 0.0779 |
+| all 4,995 positions | 95.4% | 0.0251 |
+
+For scale, the same yardstick with the bf16 read on both sides and only the experts computed on the CPU instead of the GPU (a rounding-only difference the
+engine takes whenever an expert is not resident) gives code 0.0062, documents 0.0195, English 0.0088, other languages 0.0048, agent 0.0745, all 0.0228
+(top-1 95.4%). The 8-bit read is at that level. The agent text's number rests on a few near-tie positions (the top 20 of 999 carry half of it); finer blocks
+(16 or 8 values) did not lower it, and 8-bit hyper-connection weights in the first six layers alone give the full figure, in the last 24 layers a third of it.
diff --git a/docs/STRIX_HALO.md b/docs/STRIX_HALO.md
index 5d8f69348..2b89cdf53 100644
--- a/docs/STRIX_HALO.md
+++ b/docs/STRIX_HALO.md
@@ -100,7 +100,7 @@ they change speed, not answers. Nothing here applies to CUDA or to another AMD a
## 5. What is not on by default (it changes bits)
-**Five of the switches below (`STRATA_PF_GEMM`, `STRATA_HC_UPMIX`, `STRATA_PA_FAST`, `STRATA_HIP_WMMA`, `STRATA_SELECT_WMMA`) are one opt-in on an integrated Radeon: `STRATA_IGPU_PROMPT_SET=1` ([AMD_HIP.md](AMD_HIP.md), "Integrated GPUs"). It was measured on a Radeon 780M only, where it is the default.** `STRATA_PF_FUSED`, `STRATA_HC_Q8`, `STRATA_PF_SWITCH_MIN_T` and `STRATA_PREFILL_STREAM_MIN` stay opt-in as well.
+**Five of the switches below (`STRATA_PF_GEMM`, `STRATA_HC_UPMIX`, `STRATA_PA_FAST`, `STRATA_HIP_WMMA`, `STRATA_SELECT_WMMA`) are one opt-in on an integrated Radeon: `STRATA_IGPU_PROMPT_SET=1` ([AMD_HIP.md](AMD_HIP.md), "Integrated GPUs"). It was measured on a Radeon 780M only, where it is the default.** `STRATA_PF_FUSED`, `STRATA_PF_SWITCH_MIN_T` and `STRATA_PREFILL_STREAM_MIN` stay opt-in as well. `STRATA_HC_Q8` is on by default on gfx1151 (below); `STRATA_HC_Q8=0` turns it off.
These are faster on gfx1151 and **round differently** from the default path (a different summation order or a half-precision
intermediate), so they are opt-in and the maintainers gate them with KL against the default (`STRATA_PF_SWITCH_MIN_T=4096` keeps
@@ -114,15 +114,14 @@ export STRATA_HC_UPMIX=1 # the hyper-connection up projection with its ep
export STRATA_PA_FAST=1 # the prompt attention with single FP16 q and p
export STRATA_HIP_WMMA=1 # the prompt attention on matrix cores
export STRATA_SELECT_WMMA=1 # the block scorer on matrix cores
-export STRATA_HC_Q8=1 # the hyper-connection read from the GGUF's own Q8_0 projections
export STRATA_PF_SWITCH_MIN_T=4096
export STRATA_PREFILL_STREAM_MIN=128 # prompt reads of 128-1,023 tokens on the fused experts too (below)
```
-`STRATA_HC_Q8=1` measured on the Strix Halo (IQ3_S, gfx1151, 2026-10-21, MTP 4 drafts / min-p 0.5): **+8.4%**
+`STRATA_HC_Q8` (on by default on gfx1151 since 0.2.2; `STRATA_HC_Q8=0` restores the bf16 read) measured on the Strix Halo (IQ3_S, gfx1151, 2026-10-21, MTP 4 drafts / min-p 0.5): **+8.4%**
on `json_extract` at fixed windows (53.55 -> 58.03 tok/s), at the cost of **94 of 250 output tokens differing** from
the BF16 read (first at 60). It halves the hyper-connection weight bytes (13.29 -> 7.15 MB per read; cold read
-93.8 -> 20.3 us, 142 -> 352 GB/s). Experimental, opt-in only, never a default.
+93.8 -> 20.3 us, 142 -> 352 GB/s). The answers are not bit-identical to the bf16 read; how far they move is in docs/AMD_HIP.md ("Strix Halo: 8-bit hyper-connection weights").
`STRATA_PF_FUSED=1` reaches only chunks of `STRATA_PREFILL_STREAM_MIN` tokens or more (1,024 by default, the floor measured on
a discrete card, where a smaller chunk does not pay for streaming every expert); a smaller read runs its experts through MMQ.
diff --git a/docs/release-notes-0.2.2.md b/docs/release-notes-0.2.2.md
index 9486736b4..90ca53d21 100644
--- a/docs/release-notes-0.2.2.md
+++ b/docs/release-notes-0.2.2.md
@@ -22,6 +22,7 @@
- **The Linux CUDA 12 engine package has Pascal and Volta code again (#3503).** The 0.2.1 package was built for sm_75, 86 and 89 only because the repo held no recipe for it. `packaging/build_cuda_linux.sh` builds it for sm_60, 61, 70, 75, 80, 86 and 89, the packager refuses a CUDA 12 package without sm_60, 61 and 70, and the source Dockerfile takes `CUDA_IMAGE` and `CUDA_ARCHITECTURES`. A package built with it ran an IQ2_XS model on a Tesla P100. The 0.2.2 packages are built with it.
- **HIP queue priming stands aside on a restricted queue count (#3480).** When `GPU_MAX_HW_QUEUES` is 1 or 2, the engine no longer opens its two priming queues and says so in one line (`STRATA_HIP_QUEUE_PRIME=nn` still forces them). It compiles for gfx1201; it was not measured on a gfx12 card.
- **Docker.** An old (0.1.x) volume with `/data/models`, `/data/packs` and `/data/mtp` is moved into `/data/Strata-data` by rename instead of being downloaded and packed again (it used to start a 75 GB download). A card without `--gpus` stops at once with the `--gpus all` hint instead of after minutes of setup. A missing or empty `MODEL_PATH` gets a plain message. Settings that name another image's component folder (a CUDA 12 image on a CUDA 13 volume) are made again, and an engine folder renamed by an update no longer ends in `server_exited` when pictures are on. The terminal stage of the image build no longer needs git (it failed with `FileNotFoundError`); the commit comes in as a build argument, and all five image flavors built. Tested on one RTX 3060 PC with CUDA 12 and CUDA 13 images; no AMD or Intel image and no Docker Desktop.
+- **Strix Halo decodes about 9% faster.** The hyper-connection weights are read as 8-bit (Q8_0) on gfx1151 by default: 5 interleaved pairs on IQ3_S gave +7.9% to +10.7% on JSON, code, tool calls, chat and an agent transcript. Answers are not bit-identical: they can differ in near-tie tokens, about as much as the engine's own CPU-versus-GPU expert rounding does (the numbers are in docs/AMD_HIP.md). `STRATA_HC_Q8=0` restores the old read.
- **Re-runs do not tune again.** A tuning that found the standard settings already fastest counted as "not tuned", so `strata launch`, the TUI and the Docker entrypoint measured again at every start (20 minutes); it counts as tuned now. In the Terminal a first launch of an installed model took 521 s and the second 13 s on the test PC.
- **Problems with the computer's own setup are repaired or explained, not refused.** If the Windows runtime folder lists another account, Strata locks it down itself instead of stopping. A Documents folder that cannot be used (Controlled folder access, an offline OneDrive) falls back to the Strata home folder and Settings > Storage says so. A port that is taken or reserved gets one sentence with the next step. Local requests ignore a system proxy. File-in-use errors and a full disk say what happened. `strata launch` asks plain questions in a terminal that reports a tty late. `strata update` copes with a slow `--version`, a busy graphics card is not held against an engine, pre-release Control builds hand over correctly, and a wrong PC clock is named as the reason a feed or certificate looks bad. install.sh and install.ps1 give clearer messages (ARM, TLS 1.2, proxy credentials, no `tar.exe`). The Terminal parts are already in 0.2.1.3; the Documents fallback and the port handling are app parts.
- **A stale update notice goes away.** After a version change the app and the Terminal no longer keep showing an offer for a version you already have.
diff --git a/src/core/native_dense.cpp b/src/core/native_dense.cpp
index aba2882b8..25efb6f62 100644
--- a/src/core/native_dense.cpp
+++ b/src/core/native_dense.cpp
@@ -46,12 +46,26 @@ bool in_range(const std::string& name) {
bool hc_q8_env() {
// STRATA_HC_Q8: 1 = on (the GGUF's own Q8_0 projections, or - a bf16 pack - a Q8_0 copy quantized at load); 0 = off;
// unset / auto = on for gfx12 (RDNA4: the R9700 decode +2.5 to +4%, KL 0.0003-0.0010 over its noise floor) unless
- // STRATA_GFX12_DEFAULTS=0, off elsewhere
+ // STRATA_GFX12_DEFAULTS=0; also on for gfx1151 (Strix Halo, STRATA_HC_Q8=0 turns it off), off elsewhere
static const bool on = [] {
const char* v = std::getenv("STRATA_HC_Q8");
if (v != nullptr && v[0] == '1') return true;
if (v != nullptr && v[0] != '\0' && std::strcmp(v, "auto") != 0) return false;
#if defined(STRATA_USE_HIP)
+ { // Strix Halo (gfx1151): on by default (decode +9.5%, json/code/tool/chat/agent +7.9..+10.7%); the answers differ in
+ // near-tie tokens by about the size of the engine's own CPU-vs-GPU expert rounding (docs/AMD_HIP.md);
+ // STRATA_HC_Q8=0 keeps the bf16 read
+ int dev1151 = 0;
+ cudaDeviceProp p1151;
+ const bool g1151 = cudaGetDevice(&dev1151) == cudaSuccess && cudaGetDeviceProperties(&p1151, dev1151) == cudaSuccess &&
+ strata::kernels::gfx_arch_is_gfx1151(p1151.gcnArchName);
+ (void) cudaGetLastError();
+ if (g1151) {
+ std::fprintf(stderr, "strata: Strix Halo (gfx1151): 8-bit hyper-connection weights are on (decode about 9%% faster, "
+ "answers can differ in near-tie tokens); STRATA_HC_Q8=0 keeps the bf16 read\n");
+ return true;
+ }
+ }
const char* d = std::getenv("STRATA_GFX12_DEFAULTS");
if (d != nullptr && d[0] == '0') return false;
int dev = 0;