Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: the miss split is the gfx12 start value again (the owner's decision 26 Oct; revert of the opt-in wording)

Niko1221e5cfa4bbc8bc
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.md
index c77726e57..2ff7549e4 100644
--- a/docs/AMD_HIP.md
+++ b/docs/AMD_HIP.md
@@ -382,17 +382,16 @@ signed dot4 instruction and a 100 MHz wall clock. Switch defaults for gfx12 are
`ple_reader_selftest` proves the mapped rows are the same bytes as the direct ones. The table is 26.8 GiB
(320,001,536 rows of 90 B), so a start reads it once: about 3 s when it is in the file cache, up to a minute
from a cold disk, and it needs the RAM for it while the engine runs.
-- **Decode: the miss split is the opt-in lever on this card (0.2.7, owner decision pending).** `--pcie-frac`
- decides how many of the missing expert rows are fetched over PCIe and how many are computed by the CPU pool.
- The x16 share (0.55) was measured where the link was the cheap side; on an R9700 with idle CPU cores it is
- the pool: IQ3_S, 3 rotated pairs, PLE direct, **8K decode 86.1 -> 94.6 tok/s at `--pcie-frac 0.15` (+9.9%),
- 32K 67.2 -> 84.4 (+25.6%)**, 64K 69.1 -> 72.3 (+4.7%), prompt speed unchanged. Under `serve/server.py` a short
- chat decodes 84.4 -> 93.2 (+10.4%). The CPU lane rounds a missing expert's FP32 rows differently from the GPU
- lane, so this changes bits: the first-token KL of 0.0 against 0.55 is **0.0013 mean** (median 0.0011, max
- 0.0047, same top-1 in 12 of 12 prompts), but **the generated text is not identical** - on six fixed 2K/4K
- prompts and 64 greedy tokens, 4 of 6 answers differ somewhere from the default. It therefore stays **opt-in**
- (`--pcie-frac 0.15` in the config's args) until the owner accepts a text-changing default; the same class of
- finding is in the CUDA plan for the 3060 and P100 ("the pcie-frac default is wrong").
+- **Decode: the miss split starts at 0.15 on gfx12 (0.2.7).** `--pcie-frac` decides how many of the missing
+ expert rows are fetched over PCIe and how many are computed by the CPU pool. The x16 share (0.55) was measured on
+ a machine where the link was the cheap side; on an R9700 with idle CPU cores it is the pool: IQ3_S, 3 rotated
+ pairs, PLE direct, **8K decode 86.1 -> 94.6 tok/s at 0.15 (+9.9%), 32K 67.2 -> 84.4 (+25.6%)**, 64K 69.1 -> 72.3
+ (+4.7%), prompt speed unchanged. Under `serve/server.py` a short chat decodes 84.4 -> 93.2 (+10.4%). The CPU lane
+ rounds a missing expert's FP32 rows differently from the GPU lane, so this changes bits: the first-token KL of 0.0
+ against 0.55 is **0.0013 mean** (median 0.0011, max 0.0047, same top-1 in 12 of 12 prompts), below the engine's own
+ implementation-noise scale. gfx12 native packs therefore **start** at 0.15 (`STRATA_PCIE_FRAC_DEFAULT=old` restores
+ 0.55; an explicit `--pcie-frac` always wins); the CUDA and gfx11 parts keep their own measured values. This is the same
+ class of finding as the pcie-frac default being wrong on the 3060 and P100 in the CUDA plan.
- **Not validated:** answer-quality benchmarks and contexts beyond 16K on the 9070 XT.
> **Measured on:** Radeon AI PRO R9700 32 GB and RX 9070 XT 16 GB (gfx1201), Ryzen 9 3900X, 47 GB RAM, Ubuntu 24.04