Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
docs: integrated GPUs (mini PCs) - the matrix-core prompt set is on by default, numbers, off switches, Windows untested
Niko1221e5dc00c81eac
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.mdindex cf9e27ba4..95d7c811f 100644--- a/docs/AMD_HIP.md+++ b/docs/AMD_HIP.md@@ -543,6 +543,45 @@ an RX 9070 XT 16 GB and a Radeon AI PRO R9700 32 GB (both gfx1201), a Ryzen 9 39against 2.42 s on its CPU). It has not run on an RDNA3 card (gfx1100-gfx1102) or on Windows with an AMD card.- **Not validated:** long contexts beyond 16K, answer-quality benchmarks.+## Integrated GPUs (mini PCs)++On a Radeon that shares the computer's memory (the Ryzen mini PCs: Radeon 780M / 760M `gfx1103`, 890M / 880M `gfx1150`,+Strix Halo `gfx1151`) the engine turns on the matrix-core prompt set by default. It picks this from the device's own+"integrated" flag and the architecture, not from the name: a discrete card (an RX 7900 XTX `gfx1100`, an R9700 `gfx1201`, any other)+never gets it, and neither does a computer that also holds a discrete card next to the iGPU. The engine prints one line at+start-up that names what it set.++| Switch | What it does |+|---|---|+| `STRATA_HIP_WMMA=1` | the prompt attention on the matrix cores (needs `--kv int8`) |+| `STRATA_PF_GEMM=1` | the prompt's FP16 projections on the 128 x 256 WMMA GEMM |+| `STRATA_PA_FAST=1` | the prompt attention with single FP16 q and p |+| `STRATA_SELECT_WMMA=1` | the block scorer on the matrix cores |+| `STRATA_HC_UPMIX=1` | the hyper-connection up projection with its epilogue |++**Why.** The dense projections are 63% of a prompt's GPU time on a 12-CU iGPU (the experts only 16%), and they are what these+switches speed up. Radeon 780M (P2006, Ryzen 7 255, Q2_0, `--kv int8`, 4K prompt, one run each; 148-amd): 109.5 tok/s with+none of them, 214.7 with the first four (+96%), 233 with `STRATA_HC_UPMIX` as well (+111%); decode is the same (16.6 against 17-19+tok/s, inside the run-to-run noise). The prompt switches do not touch the decode kernels.++**What it costs.** They round differently from the old path (another summation order, half-precision intermediates), so the answers+differ in the last bits. Measured on an R9700 with the same kernels (Q2_0): teacher-forced KL 0.0003 (28 prompts) / 0.0014 (eight+~6K-token documents), and the task sets inside their run-to-run noise.++**Turning it off.** Each switch keeps `=0` as its off switch (`STRATA_HC_UPMIX=0`); `STRATA_IGPU_DEFAULTS=0` turns the whole+set off. A switch you set yourself, to anything, is never overridden. Put them in the server config's `env`, or in the shell.++**Windows.** The same code applies and the set is on there too, but it has not been tested on Windows (there was no Windows iGPU+machine to run it on); the start-up line says so. Report a problem with `STRATA_IGPU_DEFAULTS=0` as the first thing to try.++**Strix Halo (`gfx1151`).** Its exact speed switches are separate and unchanged ([STRIX_HALO.md](STRIX_HALO.md)); the five above+were already its "maintainers' fast configuration". The default was measured on the 780M only.++**The 780M resets its GPU.** On the 780M's current Ubuntu firmware (gc 11.0.1 MES) a GPU reset can follow when the kernel evicts the+process's GPU queues while a prompt kernel is running ([RADEON_780M.md](RADEON_780M.md), `notes-minipc.md`). `STRATA_ARENA_PIN_GIB=none`+(experimental) leaves the expert arena unregistered, so that the kernel's page migration has no GPU mapping to invalidate;+the experts then reach the GPU by copy. It is not the default and has not been validated.+## Community-validated cardsRun by their owners, not on the maintainers' machines; setup accepts them like gfx1100 / gfx1201. setup ships adiff --git a/docs/RADEON_780M.md b/docs/RADEON_780M.mdindex bc10836ec..c2f836111 100644--- a/docs/RADEON_780M.md+++ b/docs/RADEON_780M.md@@ -18,7 +18,7 @@ GSQ-RCO Q2_0 with the MTP draft layer, 32K context, Int8 KV.`hip_prefill_fused_iq`, `qsa_*`, `hip_gemm_f16_io_parity`, `hip_prefill_mmq_parity`).- **A hipBLASLt table** for gfx1103 + hipBLASLt `100401`: `tools/hip/gfx1103-hipblaslt-100401.txt` (section 3).- **The gfx1103 exact speed switches** (section 4), written into the config's `env` by setup for the opt-in install- (the engine keeps gfx1103 out of its arch defaults, as it does for gfx1150). Whether the chip leaves the+ (the engine keeps gfx1103 out of its exact-switch arch defaults (the prompt set of section 5 is separate), as it does for gfx1150). Whether the chip leaves the`STRATA_EXPERIMENTAL_GFX1103=1` opt-in is the maintainers' call; the numbers here are what they need to make it.## 2. Toolchain and build@@ -66,7 +66,7 @@ skips because the table picks no 1176 / 1177 for the HC down GEMM (the gfx1150 t## 4. The exact speed switchesWith `STRATA_EXPERIMENTAL_GFX1103=1` setup writes the 17 exact switches below into the config's `env`, the same way-it writes the hipBLASLt table; turning the opt-in off takes them out again. The engine keeps gfx1103 out of its+it writes the hipBLASLt table; turning the opt-in off takes them out again. The engine keeps gfx1103 out of its exact-switcharch defaults (`arch_default_env("gfx1103")` stays empty, checked by `arch_defaults_test`). Measured on the 780M:+13% to +15% prompt with the table (183-186 -> 211-212 tok/s at 4K) and +3.2% decode (16.63 -> 17.17 tok/s, 640greedy tokens, 3 of 3 pairs), with the greedy token ids unchanged. They are the gfx1151 table less@@ -79,9 +79,11 @@ greedy tokens, 3 of 3 pairs), with the greedy token ids unchanged. They are the| `STRATA_Q8_PACKED`, `STRATA_Q6_PACKED`, `STRATA_MMVF_ROWS`, `STRATA_ATTN_LANECELL` | decode layouts and kernels (packed Q8_0 / Q6_K weights, 4-row BF16 GEMV, one-cell-per-thread attention scores) || `STRATA_EXPERT_V2`, `STRATA_TSUM`, `STRATA_LFUSE`, `STRATA_GDN_SPLIT`, `STRATA_QFUSE`, `STRATA_PLE_BATCH`, `STRATA_SH_STREAM` | grouped decode experts, butterfly warp sums, fewer launches, the GDN step over four blocks per head, producer-written activation images, batched PLE projections, the shared expert on its own stream |-## 5. Opt-in switches (they change bits)+## 5. The matrix-core prompt set (on by default on an integrated GPU; it changes bits)-Opt-in, and the measured wins are below. The prompt levers need the table to pay: with the table, the WMMA set+On by default since the integrated-GPU defaults (AMD_HIP.md, "Integrated GPUs"): the engine sets `STRATA_HIP_WMMA`, `STRATA_PF_GEMM`, `STRATA_PA_FAST`,+`STRATA_SELECT_WMMA` and `STRATA_HC_UPMIX` itself on an integrated Radeon; `STRATA_IGPU_DEFAULTS=0` or `NAME=0` turns them off. `STRATA_PF_FUSED` stays opt-in.+The prompt levers need the table to pay: with the table, the WMMA setread 248 tok/s at 4K against 186 without it, and at 32K 272 against 178. No KL was measured on this chip.```shdiff --git a/docs/STRIX_HALO.md b/docs/STRIX_HALO.mdindex 651d9f9f6..8caa771eb 100644--- a/docs/STRIX_HALO.md+++ b/docs/STRIX_HALO.md@@ -96,7 +96,9 @@ they change speed, not answers. Nothing here applies to CUDA or to another AMD a| `STRATA_GDN_SPLIT`, `STRATA_QFUSE`, `STRATA_PLE_BATCH` | the GDN step over four blocks per head, activation images written by their producers, the PLE key / value projections of a verify window at once || `STRATA_SH_STREAM=1` | the shared expert on a second stream beside the routed experts. It is off by default on other AMD cards (#816), but it pays on gfx1151: decode +1.8% (UD-IQ4_XS) and +6.7% (UD-Q4_K_XL) at 8K |-## 5. What is not on by default (it changes bits)+## 5. What is not on by default (it changes bits), and the five that are++**Since the integrated-GPU defaults, five of the switches below are on by default on a Strix Halo (any integrated Radeon, [AMD_HIP.md](AMD_HIP.md), "Integrated GPUs"): `STRATA_PF_GEMM`, `STRATA_HC_UPMIX`, `STRATA_PA_FAST`, `STRATA_HIP_WMMA`, `STRATA_SELECT_WMMA`.** Each keeps `=0` as its off switch and `STRATA_IGPU_DEFAULTS=0` turns the five off. The default was measured on a Radeon 780M; on the Strix Halo these five are the maintainers' earlier fast configuration. `STRATA_PF_FUSED`, `STRATA_HC_Q8`, `STRATA_PF_SWITCH_MIN_T` and `STRATA_PREFILL_STREAM_MIN` stay opt-in.These are faster on gfx1151 and **round differently** from the default path (a different summation order or a half-precisionintermediate), so they are opt-in and the maintainers gate them with KL against the default (`STRATA_PF_SWITCH_MIN_T=4096` keepsdiff --git a/include/strata/core/arch_defaults.hpp b/include/strata/core/arch_defaults.hppindex 359557fb0..5749af9d0 100644--- a/include/strata/core/arch_defaults.hpp+++ b/include/strata/core/arch_defaults.hpp@@ -5,7 +5,8 @@// 4K / 32K / 64K and on four models, and a bitwise harness per kernel. They are therefore on by default on that// architecture and nowhere else: a CUDA build and every other HIP architecture run the code they always ran.// Switches that change bits (STRATA_PF_GEMM, STRATA_PF_FUSED, STRATA_HC_UPMIX, STRATA_PA_FAST, STRATA_HIP_WMMA,-// STRATA_SELECT_WMMA, STRATA_HC_Q8: rounding-level, KL-gated) stay opt-in; docs/STRIX_HALO.md lists them.+// STRATA_SELECT_WMMA, STRATA_HC_Q8: rounding-level, KL-gated) stay opt-in in THIS table; docs/STRIX_HALO.md lists them. Five of them+// (HIP_WMMA, PF_GEMM, PA_FAST, SELECT_WMMA, HC_UPMIX) are on by default on an integrated Radeon through igpu_default_env() below.//// A switch the user set (to anything) is never overridden; STRATA_GFX1151_DEFAULTS=0 turns the whole table off.#pragma once