Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
arch_defaults: UMA arena is not a default (measured slower, resets not fixed): constant off, STRATA_ARENA_PIN_GIB=none stays opt-in; docs say the iGPU prompt set is opt-in with the 780M numbers
Niko1221b203b5f71e6f
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.mdindex 95d7c811f..84a97bdd1 100644--- a/docs/AMD_HIP.md+++ b/docs/AMD_HIP.md@@ -546,10 +546,11 @@ an RX 9070 XT 16 GB and a Radeon AI PRO R9700 32 GB (both gfx1201), a Ryzen 9 39## Integrated GPUs (mini PCs)On a Radeon that shares the computer's memory (the Ryzen mini PCs: Radeon 780M / 760M `gfx1103`, 890M / 880M `gfx1150`,-Strix Halo `gfx1151`) the engine turns on the matrix-core prompt set by default. It picks this from the device's own-"integrated" flag and the architecture, not from the name: a discrete card (an RX 7900 XTX `gfx1100`, an R9700 `gfx1201`, any other)-never gets it, and neither does a computer that also holds a discrete card next to the iGPU. The engine prints one line at-start-up that names what it set.+Strix Halo `gfx1151`) the matrix-core prompt set below is opt-in (`STRATA_IGPU_PROMPT_SET=1`): it changes bits, so it waits for the maintainer's+decision before it becomes the default. With the opt-in the engine picks the set from the device's own "integrated" flag and+the architecture, not from the name: a discrete card (an RX 7900 XTX `gfx1100`, an R9700 `gfx1201`, any other) never gets it,+and neither does a computer that also holds a discrete card next to the iGPU. The engine prints one line at start-up that names+what it set.| Switch | What it does ||---|---|@@ -568,19 +569,21 @@ tok/s, inside the run-to-run noise). The prompt switches do not touch the decodediffer in the last bits. Measured on an R9700 with the same kernels (Q2_0): teacher-forced KL 0.0003 (28 prompts) / 0.0014 (eight~6K-token documents), and the task sets inside their run-to-run noise.-**Turning it off.** Each switch keeps `=0` as its off switch (`STRATA_HC_UPMIX=0`); `STRATA_IGPU_DEFAULTS=0` turns the whole-set off. A switch you set yourself, to anything, is never overridden. Put them in the server config's `env`, or in the shell.+**Turning it on and off.** `STRATA_IGPU_PROMPT_SET=1` turns the whole set on; each switch keeps `=0` as its off switch+(`STRATA_HC_UPMIX=0`); `STRATA_IGPU_DEFAULTS=0` turns the set off whatever else is set. A switch you set yourself, to anything, is never overridden. Put them in the server config's `env`, or in the shell.-**Windows.** The same code applies and the set is on there too, but it has not been tested on Windows (there was no Windows iGPU-machine to run it on); the start-up line says so. Report a problem with `STRATA_IGPU_DEFAULTS=0` as the first thing to try.+**Windows.** The same code applies, but it has not been tested on Windows (there was no Windows iGPU machine to run it on).**Strix Halo (`gfx1151`).** Its exact speed switches are separate and unchanged ([STRIX_HALO.md](STRIX_HALO.md)); the five above-were already its "maintainers' fast configuration". The default was measured on the 780M only.+were already its "maintainers' fast configuration". The set was measured on the 780M only.**The 780M resets its GPU.** On the 780M's current Ubuntu firmware (gc 11.0.1 MES) a GPU reset can follow when the kernel evicts the-process's GPU queues while a prompt kernel is running ([RADEON_780M.md](RADEON_780M.md), `notes-minipc.md`). `STRATA_ARENA_PIN_GIB=none`-(experimental) leaves the expert arena unregistered, so that the kernel's page migration has no GPU mapping to invalidate;-the experts then reach the GPU by copy. It is not the default and has not been validated.+process's GPU queues while a kernel with barriers is running ([RADEON_780M.md](RADEON_780M.md), `notes-minipc.md`). It is+not caused by the prompt set: the default build without it resets the same way (forced compaction every 10 s killed 2 of 2 runs).+`STRATA_ARENA_PIN_GIB=none` (experimental, exact) leaves the expert arena unregistered, so that the kernel's page migration has+no big GPU mapping to invalidate; the experts then reach the GPU by copy. Measured on the 780M (30 interleaved runs, 021-minipc): the+eviction time drops from about 6 s to about 0.13 s per run, but resets still happened (4 of 20 runs against 1 of 10), and prompts/decode+were 1-4% slower, so it is not the default. A newer `linux-firmware` (MES) is the likely cure and is a host change the engine does not make.## Community-validated cardsdiff --git a/docs/RADEON_780M.md b/docs/RADEON_780M.mdindex c2f836111..c74f8fb22 100644--- a/docs/RADEON_780M.md+++ b/docs/RADEON_780M.md@@ -79,10 +79,10 @@ greedy tokens, 3 of 3 pairs), with the greedy token ids unchanged. They are the| `STRATA_Q8_PACKED`, `STRATA_Q6_PACKED`, `STRATA_MMVF_ROWS`, `STRATA_ATTN_LANECELL` | decode layouts and kernels (packed Q8_0 / Q6_K weights, 4-row BF16 GEMV, one-cell-per-thread attention scores) || `STRATA_EXPERT_V2`, `STRATA_TSUM`, `STRATA_LFUSE`, `STRATA_GDN_SPLIT`, `STRATA_QFUSE`, `STRATA_PLE_BATCH`, `STRATA_SH_STREAM` | grouped decode experts, butterfly warp sums, fewer launches, the GDN step over four blocks per head, producer-written activation images, batched PLE projections, the shared expert on its own stream |-## 5. The matrix-core prompt set (on by default on an integrated GPU; it changes bits)+## 5. The matrix-core prompt set (opt-in on an integrated GPU: `STRATA_IGPU_PROMPT_SET=1`; it changes bits)-On by default since the integrated-GPU defaults (AMD_HIP.md, "Integrated GPUs"): the engine sets `STRATA_HIP_WMMA`, `STRATA_PF_GEMM`, `STRATA_PA_FAST`,-`STRATA_SELECT_WMMA` and `STRATA_HC_UPMIX` itself on an integrated Radeon; `STRATA_IGPU_DEFAULTS=0` or `NAME=0` turns them off. `STRATA_PF_FUSED` stays opt-in.+Opt-in (AMD_HIP.md, "Integrated GPUs"); with `STRATA_IGPU_PROMPT_SET=1` the engine sets `STRATA_HIP_WMMA`, `STRATA_PF_GEMM`, `STRATA_PA_FAST`,+`STRATA_SELECT_WMMA` and `STRATA_HC_UPMIX` itself on an integrated Radeon; `NAME=0` turns one off. `STRATA_PF_FUSED` stays opt-in.The prompt levers need the table to pay: with the table, the WMMA setread 248 tok/s at 4K against 186 without it, and at 32K 272 against 178. No KL was measured on this chip.diff --git a/docs/STRIX_HALO.md b/docs/STRIX_HALO.mdindex 8caa771eb..9228d15f1 100644--- a/docs/STRIX_HALO.md+++ b/docs/STRIX_HALO.md@@ -96,9 +96,9 @@ they change speed, not answers. Nothing here applies to CUDA or to another AMD a| `STRATA_GDN_SPLIT`, `STRATA_QFUSE`, `STRATA_PLE_BATCH` | the GDN step over four blocks per head, activation images written by their producers, the PLE key / value projections of a verify window at once || `STRATA_SH_STREAM=1` | the shared expert on a second stream beside the routed experts. It is off by default on other AMD cards (#816), but it pays on gfx1151: decode +1.8% (UD-IQ4_XS) and +6.7% (UD-Q4_K_XL) at 8K |-## 5. What is not on by default (it changes bits), and the five that are+## 5. What is not on by default (it changes bits)-**Since the integrated-GPU defaults, five of the switches below are on by default on a Strix Halo (any integrated Radeon, [AMD_HIP.md](AMD_HIP.md), "Integrated GPUs"): `STRATA_PF_GEMM`, `STRATA_HC_UPMIX`, `STRATA_PA_FAST`, `STRATA_HIP_WMMA`, `STRATA_SELECT_WMMA`.** Each keeps `=0` as its off switch and `STRATA_IGPU_DEFAULTS=0` turns the five off. The default was measured on a Radeon 780M; on the Strix Halo these five are the maintainers' earlier fast configuration. `STRATA_PF_FUSED`, `STRATA_HC_Q8`, `STRATA_PF_SWITCH_MIN_T` and `STRATA_PREFILL_STREAM_MIN` stay opt-in.+**Five of the switches below (`STRATA_PF_GEMM`, `STRATA_HC_UPMIX`, `STRATA_PA_FAST`, `STRATA_HIP_WMMA`, `STRATA_SELECT_WMMA`) are one opt-in on an integrated Radeon: `STRATA_IGPU_PROMPT_SET=1` ([AMD_HIP.md](AMD_HIP.md), "Integrated GPUs"). It was measured on a Radeon 780M only.** `STRATA_PF_FUSED`, `STRATA_HC_Q8`, `STRATA_PF_SWITCH_MIN_T` and `STRATA_PREFILL_STREAM_MIN` stay opt-in as well.These are faster on gfx1151 and **round differently** from the default path (a different summation order or a half-precisionintermediate), so they are opt-in and the maintainers gate them with KL against the default (`STRATA_PF_SWITCH_MIN_T=4096` keepsdiff --git a/include/strata/core/arch_defaults.hpp b/include/strata/core/arch_defaults.hppindex 404b29aa3..b836867a8 100644--- a/include/strata/core/arch_defaults.hpp+++ b/include/strata/core/arch_defaults.hpp@@ -6,7 +6,7 @@// architecture and nowhere else: a CUDA build and every other HIP architecture run the code they always ran.// Switches that change bits (STRATA_PF_GEMM, STRATA_PF_FUSED, STRATA_HC_UPMIX, STRATA_PA_FAST, STRATA_HIP_WMMA,// STRATA_SELECT_WMMA, STRATA_HC_Q8: rounding-level, KL-gated) stay opt-in in THIS table; docs/STRIX_HALO.md lists them. Five of them-// (HIP_WMMA, PF_GEMM, PA_FAST, SELECT_WMMA, HC_UPMIX) are on by default on an integrated Radeon through igpu_default_env() below.+// (HIP_WMMA, PF_GEMM, PA_FAST, SELECT_WMMA, HC_UPMIX) are an opt-in set on an integrated Radeon (igpu_default_env() below).//// A switch the user set (to anything) is never overridden; STRATA_GFX1151_DEFAULTS=0 turns the whole table off.#pragma once@@ -22,7 +22,7 @@ namespace strata::core {std::vector<std::pair<std::string, std::string>> arch_default_env(const char* gcn_arch);/// The integrated Radeons (Phoenix / Hawk Point 780M gfx1103, Strix Point 890M / 880M gfx1150, Strix Halo gfx1151) whose-/// matrix-core prompt set is on by default (153-igpu). Pure architecture-name test; the caller adds the UMA test.+/// matrix-core prompt set is offered (opt-in; 021-minipc). Pure architecture-name test; the caller adds the UMA test.bool igpu_prompt_defaults_arch(const char* gcn_arch);/// 021-minipc: the matrix-core prompt set changes bits, so it is OFF by default until the owner approves it. Flip this constant@@ -31,10 +31,12 @@ bool igpu_prompt_defaults_arch(const char* gcn_arch);inline constexpr bool kIgpuPromptSetDefaultOn = false;bool igpu_prompt_set_enabled();-/// 021-minipc: does this device keep the expert arena unregistered (UMA mode) by default? Exact (no bit changes): only the-/// Radeon 780M (gfx1103, integrated, Linux). gfx1150 / gfx1151 are not included: no hardware to measure on, and Strix Halo-/// has a different firmware. STRATA_UMA_REGISTER=1 registers the arena as before.-bool igpu_uma_arena_default(const char* gcn_arch, bool integrated, bool windows);+/// 021-minipc: UMA mode (the expert arena stays unregistered). Exact (no bit changes) but NOT a default: measured on the 780M it was+/// 1-4% slower and did not stop the resets (4 of 20 runs against 1 of 10), so kIgpuUmaArenaDefaultOn is false. When true it applies+/// to the Radeon 780M only (gfx1103, integrated, Linux); gfx1150 / gfx1151 are not covered (no hardware). STRATA_UMA_REGISTER=1+/// registers the arena as before; STRATA_ARENA_PIN_GIB=none selects UMA mode per run.+inline constexpr bool kIgpuUmaArenaDefaultOn = false;+bool igpu_uma_arena_default(const char* gcn_arch, bool integrated, bool windows, bool on = kIgpuUmaArenaDefaultOn);/// The matrix-core prompt set an integrated GPU takes when `igpu_prompt_set_enabled()`: STRATA_HIP_WMMA, STRATA_PF_GEMM, STRATA_PA_FAST,/// STRATA_SELECT_WMMA, STRATA_HC_UPMIX, all "1". Empty unless `integrated` (hipDeviceProp_t::integrated: the devicediff --git a/src/core/arch_defaults.cpp b/src/core/arch_defaults.cppindex 7f10fb283..fe534c65b 100644--- a/src/core/arch_defaults.cpp+++ b/src/core/arch_defaults.cpp@@ -73,8 +73,8 @@ bool igpu_prompt_set_enabled() {return kIgpuPromptSetDefaultOn;}-bool igpu_uma_arena_default(const char* gcn_arch, bool integrated, bool windows) {- if (windows || !integrated || !strata::kernels::gfx_arch_is(gcn_arch, "gfx1103")) return false;+bool igpu_uma_arena_default(const char* gcn_arch, bool integrated, bool windows, bool on) {+ if (!on || windows || !integrated || !strata::kernels::gfx_arch_is(gcn_arch, "gfx1103")) return false;const char* reg = std::getenv("STRATA_UMA_REGISTER");return !(reg != nullptr && reg[0] == '1');}diff --git a/src/core/arch_defaults_test.cpp b/src/core/arch_defaults_test.cppindex 5a921de0e..dfea2c4fa 100644--- a/src/core/arch_defaults_test.cpp+++ b/src/core/arch_defaults_test.cpp@@ -156,17 +156,20 @@ int main() {CHECK(std::getenv("STRATA_HC_UPMIX") != nullptr && std::string(std::getenv("STRATA_HC_UPMIX")) == "0");for (const char* k : five) unset(k);unset("STRATA_IGPU_PROMPT_SET");+ // 021-minipc: UMA mode is NOT a default (slower, resets not fixed): off with nothing asked; the logic below runs with on=true+ CHECK(!kIgpuUmaArenaDefaultOn);+ CHECK(!igpu_uma_arena_default("gfx1103", true, false));// 021-minipc: UMA mode (the expert arena stays unregistered) is chosen for the Radeon 780M only: gfx1103 + integrated + not Windowsunset("STRATA_UMA_REGISTER");- CHECK(igpu_uma_arena_default("gfx1103", true, false));- CHECK(igpu_uma_arena_default("gfx1103:sramecc-:xnack-", true, false));- CHECK(!igpu_uma_arena_default("gfx1103", true, true)); // Windows: not measured- CHECK(!igpu_uma_arena_default("gfx1103", false, false)); // a gfx1103 that is not integrated+ CHECK(igpu_uma_arena_default("gfx1103", true, false, true));+ CHECK(igpu_uma_arena_default("gfx1103:sramecc-:xnack-", true, false, true));+ CHECK(!igpu_uma_arena_default("gfx1103", true, true, true)); // Windows: not measured+ CHECK(!igpu_uma_arena_default("gfx1103", false, false, true)); // a gfx1103 that is not integratedfor (const char* g : {"gfx1150", "gfx1151", "gfx1152", "gfx1100", "gfx1101", "gfx1200", "gfx1201", "gfx1030", "gfx11030", "", "gfx90a"})- CHECK(!igpu_uma_arena_default(g, true, false));- CHECK(!igpu_uma_arena_default(nullptr, true, false));+ CHECK(!igpu_uma_arena_default(g, true, false, true));+ CHECK(!igpu_uma_arena_default(nullptr, true, false, true));put("STRATA_UMA_REGISTER", "1");- CHECK(!igpu_uma_arena_default("gfx1103", true, false)); // the user registers the arena as before+ CHECK(!igpu_uma_arena_default("gfx1103", true, false, true)); // the user registers the arena as beforeunset("STRATA_UMA_REGISTER");// both platforms: no Windows guard on the iGPU set (the owner wants it there too; untested on Windows is only said in the log)std::printf(fails ? "arch_defaults_test: %d FAILED\n" : "arch_defaults_test: OK\n", fails);