Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: the miss split is the gfx12 start value now (measured A/B on the shipped path: 8K +9.0%, 32K +23.6% decode)

Niko122113c4aebecafc
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.md
index d9ec92452..2ff7549e4 100644
--- a/docs/AMD_HIP.md
+++ b/docs/AMD_HIP.md
@@ -382,14 +382,16 @@ signed dot4 instruction and a 100 MHz wall clock. Switch defaults for gfx12 are
`ple_reader_selftest` proves the mapped rows are the same bytes as the direct ones. The table is 26.8 GiB
(320,001,536 rows of 90 B), so a start reads it once: about 3 s when it is in the file cache, up to a minute
from a cold disk, and it needs the RAM for it while the engine runs.
-- **Decode: the miss split pays on this card (0.2.7, opt-in).** `--pcie-frac` decides how many of the missing
- expert rows are fetched over PCIe and how many are computed by the CPU pool; the probe's default is 0.55. With
- 30 idle cores during decode, more CPU is faster here: IQ3_S, 3 rotated pairs, PLE direct, **8K decode 86.1 ->
- 94.6 tok/s at 0.15 (+9.9%) and 96.3 at 0.0 (+11.8%), 32K 67.2 -> 84.4 (+25.6%)**, prompt speed unchanged at 8K.
- The CPU lane rounds a missing expert's FP32 rows differently from the GPU lane, so this changes bits: the
- first-token KL against the default is 0.0013 mean (top-1 identical in 12 of 12 prompts), below the engine's own
- implementation-noise scale. It stays a recommendation (the owner decides a default), and it is the same class of
- finding as the pcie-frac default being wrong on the 3060 and P100 in the CUDA plan.
+- **Decode: the miss split starts at 0.15 on gfx12 (0.2.7).** `--pcie-frac` decides how many of the missing
+ expert rows are fetched over PCIe and how many are computed by the CPU pool. The x16 share (0.55) was measured on
+ a machine where the link was the cheap side; on an R9700 with idle CPU cores it is the pool: IQ3_S, 3 rotated
+ pairs, PLE direct, **8K decode 86.1 -> 94.6 tok/s at 0.15 (+9.9%), 32K 67.2 -> 84.4 (+25.6%)**, 64K 69.1 -> 72.3
+ (+4.7%), prompt speed unchanged. Under `serve/server.py` a short chat decodes 84.4 -> 93.2 (+10.4%). The CPU lane
+ rounds a missing expert's FP32 rows differently from the GPU lane, so this changes bits: the first-token KL of 0.0
+ against 0.55 is **0.0013 mean** (median 0.0011, max 0.0047, same top-1 in 12 of 12 prompts), below the engine's own
+ implementation-noise scale. gfx12 native packs therefore **start** at 0.15 (`STRATA_PCIE_FRAC_DEFAULT=old` restores
+ 0.55; an explicit `--pcie-frac` always wins); the CUDA and gfx11 parts keep their own measured values. This is the same
+ class of finding as the pcie-frac default being wrong on the 3060 and P100 in the CUDA plan.
- **Not validated:** answer-quality benchmarks and contexts beyond 16K on the 9070 XT.
> **Measured on:** Radeon AI PRO R9700 32 GB and RX 9070 XT 16 GB (gfx1201), Ryzen 9 3900X, 47 GB RAM, Ubuntu 24.04