Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: integrated GPUs (mini PCs) - the matrix-core prompt set is on by default, numbers, off switches, Windows untested

Niko1221e5dc00c81eac
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.md
index cf9e27ba4..95d7c811f 100644
--- a/docs/AMD_HIP.md
+++ b/docs/AMD_HIP.md
@@ -543,6 +543,45 @@ an RX 9070 XT 16 GB and a Radeon AI PRO R9700 32 GB (both gfx1201), a Ryzen 9 39
against 2.42 s on its CPU). It has not run on an RDNA3 card (gfx1100-gfx1102) or on Windows with an AMD card.
- **Not validated:** long contexts beyond 16K, answer-quality benchmarks.
+## Integrated GPUs (mini PCs)
+
+On a Radeon that shares the computer's memory (the Ryzen mini PCs: Radeon 780M / 760M `gfx1103`, 890M / 880M `gfx1150`,
+Strix Halo `gfx1151`) the engine turns on the matrix-core prompt set by default. It picks this from the device's own
+"integrated" flag and the architecture, not from the name: a discrete card (an RX 7900 XTX `gfx1100`, an R9700 `gfx1201`, any other)
+never gets it, and neither does a computer that also holds a discrete card next to the iGPU. The engine prints one line at
+start-up that names what it set.
+
+| Switch | What it does |
+|---|---|
+| `STRATA_HIP_WMMA=1` | the prompt attention on the matrix cores (needs `--kv int8`) |
+| `STRATA_PF_GEMM=1` | the prompt's FP16 projections on the 128 x 256 WMMA GEMM |
+| `STRATA_PA_FAST=1` | the prompt attention with single FP16 q and p |
+| `STRATA_SELECT_WMMA=1` | the block scorer on the matrix cores |
+| `STRATA_HC_UPMIX=1` | the hyper-connection up projection with its epilogue |
+
+**Why.** The dense projections are 63% of a prompt's GPU time on a 12-CU iGPU (the experts only 16%), and they are what these
+switches speed up. Radeon 780M (P2006, Ryzen 7 255, Q2_0, `--kv int8`, 4K prompt, one run each; 148-amd): 109.5 tok/s with
+none of them, 214.7 with the first four (+96%), 233 with `STRATA_HC_UPMIX` as well (+111%); decode is the same (16.6 against 17-19
+tok/s, inside the run-to-run noise). The prompt switches do not touch the decode kernels.
+
+**What it costs.** They round differently from the old path (another summation order, half-precision intermediates), so the answers
+differ in the last bits. Measured on an R9700 with the same kernels (Q2_0): teacher-forced KL 0.0003 (28 prompts) / 0.0014 (eight
+~6K-token documents), and the task sets inside their run-to-run noise.
+
+**Turning it off.** Each switch keeps `=0` as its off switch (`STRATA_HC_UPMIX=0`); `STRATA_IGPU_DEFAULTS=0` turns the whole
+set off. A switch you set yourself, to anything, is never overridden. Put them in the server config's `env`, or in the shell.
+
+**Windows.** The same code applies and the set is on there too, but it has not been tested on Windows (there was no Windows iGPU
+machine to run it on); the start-up line says so. Report a problem with `STRATA_IGPU_DEFAULTS=0` as the first thing to try.
+
+**Strix Halo (`gfx1151`).** Its exact speed switches are separate and unchanged ([STRIX_HALO.md](STRIX_HALO.md)); the five above
+were already its "maintainers' fast configuration". The default was measured on the 780M only.
+
+**The 780M resets its GPU.** On the 780M's current Ubuntu firmware (gc 11.0.1 MES) a GPU reset can follow when the kernel evicts the
+process's GPU queues while a prompt kernel is running ([RADEON_780M.md](RADEON_780M.md), `notes-minipc.md`). `STRATA_ARENA_PIN_GIB=none`
+(experimental) leaves the expert arena unregistered, so that the kernel's page migration has no GPU mapping to invalidate;
+the experts then reach the GPU by copy. It is not the default and has not been validated.
+
## Community-validated cards
Run by their owners, not on the maintainers' machines; setup accepts them like gfx1100 / gfx1201. setup ships a
diff --git a/docs/RADEON_780M.md b/docs/RADEON_780M.md
index bc10836ec..c2f836111 100644
--- a/docs/RADEON_780M.md
+++ b/docs/RADEON_780M.md
@@ -18,7 +18,7 @@ GSQ-RCO Q2_0 with the MTP draft layer, 32K context, Int8 KV.
`hip_prefill_fused_iq`, `qsa_*`, `hip_gemm_f16_io_parity`, `hip_prefill_mmq_parity`).
- **A hipBLASLt table** for gfx1103 + hipBLASLt `100401`: `tools/hip/gfx1103-hipblaslt-100401.txt` (section 3).
- **The gfx1103 exact speed switches** (section 4), written into the config's `env` by setup for the opt-in install
- (the engine keeps gfx1103 out of its arch defaults, as it does for gfx1150). Whether the chip leaves the
+ (the engine keeps gfx1103 out of its exact-switch arch defaults (the prompt set of section 5 is separate), as it does for gfx1150). Whether the chip leaves the
`STRATA_EXPERIMENTAL_GFX1103=1` opt-in is the maintainers' call; the numbers here are what they need to make it.
## 2. Toolchain and build
@@ -66,7 +66,7 @@ skips because the table picks no 1176 / 1177 for the HC down GEMM (the gfx1150 t
## 4. The exact speed switches
With `STRATA_EXPERIMENTAL_GFX1103=1` setup writes the 17 exact switches below into the config's `env`, the same way
-it writes the hipBLASLt table; turning the opt-in off takes them out again. The engine keeps gfx1103 out of its
+it writes the hipBLASLt table; turning the opt-in off takes them out again. The engine keeps gfx1103 out of its exact-switch
arch defaults (`arch_default_env("gfx1103")` stays empty, checked by `arch_defaults_test`). Measured on the 780M:
+13% to +15% prompt with the table (183-186 -> 211-212 tok/s at 4K) and +3.2% decode (16.63 -> 17.17 tok/s, 640
greedy tokens, 3 of 3 pairs), with the greedy token ids unchanged. They are the gfx1151 table less
@@ -79,9 +79,11 @@ greedy tokens, 3 of 3 pairs), with the greedy token ids unchanged. They are the
| `STRATA_Q8_PACKED`, `STRATA_Q6_PACKED`, `STRATA_MMVF_ROWS`, `STRATA_ATTN_LANECELL` | decode layouts and kernels (packed Q8_0 / Q6_K weights, 4-row BF16 GEMV, one-cell-per-thread attention scores) |
| `STRATA_EXPERT_V2`, `STRATA_TSUM`, `STRATA_LFUSE`, `STRATA_GDN_SPLIT`, `STRATA_QFUSE`, `STRATA_PLE_BATCH`, `STRATA_SH_STREAM` | grouped decode experts, butterfly warp sums, fewer launches, the GDN step over four blocks per head, producer-written activation images, batched PLE projections, the shared expert on its own stream |
-## 5. Opt-in switches (they change bits)
+## 5. The matrix-core prompt set (on by default on an integrated GPU; it changes bits)
-Opt-in, and the measured wins are below. The prompt levers need the table to pay: with the table, the WMMA set
+On by default since the integrated-GPU defaults (AMD_HIP.md, "Integrated GPUs"): the engine sets `STRATA_HIP_WMMA`, `STRATA_PF_GEMM`, `STRATA_PA_FAST`,
+`STRATA_SELECT_WMMA` and `STRATA_HC_UPMIX` itself on an integrated Radeon; `STRATA_IGPU_DEFAULTS=0` or `NAME=0` turns them off. `STRATA_PF_FUSED` stays opt-in.
+The prompt levers need the table to pay: with the table, the WMMA set
read 248 tok/s at 4K against 186 without it, and at 32K 272 against 178. No KL was measured on this chip.
```sh
diff --git a/docs/STRIX_HALO.md b/docs/STRIX_HALO.md
index 651d9f9f6..8caa771eb 100644
--- a/docs/STRIX_HALO.md
+++ b/docs/STRIX_HALO.md
@@ -96,7 +96,9 @@ they change speed, not answers. Nothing here applies to CUDA or to another AMD a
| `STRATA_GDN_SPLIT`, `STRATA_QFUSE`, `STRATA_PLE_BATCH` | the GDN step over four blocks per head, activation images written by their producers, the PLE key / value projections of a verify window at once |
| `STRATA_SH_STREAM=1` | the shared expert on a second stream beside the routed experts. It is off by default on other AMD cards (#816), but it pays on gfx1151: decode +1.8% (UD-IQ4_XS) and +6.7% (UD-Q4_K_XL) at 8K |
-## 5. What is not on by default (it changes bits)
+## 5. What is not on by default (it changes bits), and the five that are
+
+**Since the integrated-GPU defaults, five of the switches below are on by default on a Strix Halo (any integrated Radeon, [AMD_HIP.md](AMD_HIP.md), "Integrated GPUs"): `STRATA_PF_GEMM`, `STRATA_HC_UPMIX`, `STRATA_PA_FAST`, `STRATA_HIP_WMMA`, `STRATA_SELECT_WMMA`.** Each keeps `=0` as its off switch and `STRATA_IGPU_DEFAULTS=0` turns the five off. The default was measured on a Radeon 780M; on the Strix Halo these five are the maintainers' earlier fast configuration. `STRATA_PF_FUSED`, `STRATA_HC_Q8`, `STRATA_PF_SWITCH_MIN_T` and `STRATA_PREFILL_STREAM_MIN` stay opt-in.
These are faster on gfx1151 and **round differently** from the default path (a different summation order or a half-precision
intermediate), so they are opt-in and the maintainers gate them with KL against the default (`STRATA_PF_SWITCH_MIN_T=4096` keeps
diff --git a/include/strata/core/arch_defaults.hpp b/include/strata/core/arch_defaults.hpp
index 359557fb0..5749af9d0 100644
--- a/include/strata/core/arch_defaults.hpp
+++ b/include/strata/core/arch_defaults.hpp
@@ -5,7 +5,8 @@
// 4K / 32K / 64K and on four models, and a bitwise harness per kernel. They are therefore on by default on that
// architecture and nowhere else: a CUDA build and every other HIP architecture run the code they always ran.
// Switches that change bits (STRATA_PF_GEMM, STRATA_PF_FUSED, STRATA_HC_UPMIX, STRATA_PA_FAST, STRATA_HIP_WMMA,
-// STRATA_SELECT_WMMA, STRATA_HC_Q8: rounding-level, KL-gated) stay opt-in; docs/STRIX_HALO.md lists them.
+// STRATA_SELECT_WMMA, STRATA_HC_Q8: rounding-level, KL-gated) stay opt-in in THIS table; docs/STRIX_HALO.md lists them. Five of them
+// (HIP_WMMA, PF_GEMM, PA_FAST, SELECT_WMMA, HC_UPMIX) are on by default on an integrated Radeon through igpu_default_env() below.
//
// A switch the user set (to anything) is never overridden; STRATA_GFX1151_DEFAULTS=0 turns the whole table off.
#pragma once