Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Gyro on Intel Arc: README and MODELS say so, ctest cases for the trellis kernels, setup's Gyro memory warning reads the real RAM on the Intel path
Niko122179946b9c4fc4
diff --git a/README.de.md b/README.de.mdindex 1bfd321d5..72eb048e4 100644--- a/README.de.md+++ b/README.de.md@@ -164,7 +164,7 @@ Größen, mehr oder weniger stark komprimiert. Kleinere Größen sind schneller.| **Gyro-S** | 27,6 GB | 58,5 GB | Für PCs, auf denen nichts anderes läuft. Die Qualität ist geringer als bei IQ2_XS. || **Gyro-M** | 35 GB | 92,0 GB | Ungefähr die Qualität von IQ3_XXS bei etwa 20 % weniger GPU-Speicher. Es schreibt Antworten auf einer 12-GB-Karte und auf einer R9700 17-21 % langsamer als Q2_0. |-Gyro läuft auf NVIDIA- und AMD-Karten (bei AMD experimentell), nicht auf Intel Arc und nicht auf dem Mac. Details und Messwerte:+Gyro läuft auf NVIDIA- und AMD-Karten (bei AMD experimentell) und auf Intel-Arc-Karten unter Linux (getestet auf einer Arc A750), nicht auf dem Mac. Details und Messwerte:[docs/GYRO.md](docs/GYRO.md). Starte es mit `strata launch --model gyro-s`.### Die anderen Größendiff --git a/README.es.md b/README.es.mdindex 679fafd5a..0be4b744f 100644--- a/README.es.md+++ b/README.es.md@@ -164,7 +164,7 @@ comprimido. Los tamaños pequeños son más rápidos. Los grandes son algo más| **Gyro-S** | 27,6 GB | 58,5 GB | Para PC donde no cabe nada más. La calidad es menor que la de IQ2_XS. || **Gyro-M** | 35 GB | 92,0 GB | Calidad parecida a IQ3_XXS con un 20 % menos de memoria de GPU. Escribe las respuestas un 17-21 % más despacio que Q2_0 en una tarjeta de 12 GB y en una R9700. |-Gyro funciona en tarjetas NVIDIA y AMD (en AMD es experimental), no en Intel Arc ni en el Mac. Detalles y mediciones:+Gyro funciona en tarjetas NVIDIA y AMD (en AMD es experimental) y en tarjetas Intel Arc con Linux (probado en una Arc A750), no en el Mac. Detalles y mediciones:[docs/GYRO.md](docs/GYRO.md). Arráncalo con `strata launch --model gyro-s`.### Los demás tamañosdiff --git a/README.fr.md b/README.fr.mdindex 9ce7d44e1..085648539 100644--- a/README.fr.md+++ b/README.fr.md@@ -165,7 +165,7 @@ plus ou moins compressé. Les petites tailles sont plus rapides. Les grandes son| **Gyro-S** | 27,6 Go | 58,5 Go | Pour les PC où rien d'autre ne tient. La qualité est inférieure à celle d'IQ2_XS. || **Gyro-M** | 35 Go | 92,0 Go | À peu près la qualité d'IQ3_XXS avec environ 20 % de mémoire GPU en moins. Il écrit les réponses 17 à 21 % plus lentement que Q2_0 sur une carte de 12 Go et sur une R9700. |-Gyro fonctionne sur les cartes NVIDIA et AMD (expérimental sur AMD), pas sur Intel Arc ni sur Mac. Détails et mesures :+Gyro fonctionne sur les cartes NVIDIA et AMD (expérimental sur AMD) et sur les cartes Intel Arc sous Linux (testé sur une Arc A750), pas sur Mac. Détails et mesures :[docs/GYRO.md](docs/GYRO.md). Lancez-le avec `strata launch --model gyro-s`.### Les autres taillesdiff --git a/README.ja.md b/README.ja.mdindex e3d7e8353..8402120f0 100644--- a/README.ja.md+++ b/README.ja.md@@ -158,7 +158,7 @@ AI ツールは [MCP サーバー](docs/MCP_SERVER.md)を通じて、Strata の| **Gyro-S** | 27.6 GB | 58.5 GB | ほかのサイズが入らないPC向け。品質は IQ2_XS より下がります。 || **Gyro-M** | 35 GB | 92.0 GB | 品質は IQ3_XXS 程度で、GPUメモリの使用量は約20%少なくなります。12 GBのカードと R9700 では、回答の書き出しが Q2_0 より17〜21%遅くなります。 |-Gyro は NVIDIA と AMD のカードで動きます(AMD は実験的)。Intel Arc と Mac では動きません。詳細と測定結果は+Gyro は NVIDIA と AMD のカード(AMD は実験的)、および Linux 上の Intel Arc カード(Arc A750 で確認)で動きます。Mac では動きません。詳細と測定結果は[docs/GYRO.md](docs/GYRO.md) にあります。`strata launch --model gyro-s` で起動します。### ほかのサイズdiff --git a/README.md b/README.mdindex 4b7b92e40..394b1a3c6 100644--- a/README.md+++ b/README.md@@ -159,7 +159,7 @@ Smaller sizes are faster. Larger sizes are a bit smarter.| **Gyro-S** | 27.6 GB | 58.5 GB | For PCs that fit nothing else. Quality is lower than IQ2_XS. || **Gyro-M** | 35 GB | 92.0 GB | About IQ3_XXS quality with about 20% less GPU memory. It writes answers 17-21% slower than Q2_0 on a 12 GB card and on an R9700. |-Gyro runs on NVIDIA and AMD cards (AMD is experimental), not on Intel Arc or the Mac. Details and measurements:+Gyro runs on NVIDIA and AMD cards (AMD is experimental) and on Intel Arc cards on Linux (tested on an Arc A750), not on the Mac. Details and measurements:[docs/GYRO.md](docs/GYRO.md). Start it with `strata launch --model gyro-s`.### The other sizesdiff --git a/README.pt-BR.md b/README.pt-BR.mdindex d97e534fb..9e34e5aa9 100644--- a/README.pt-BR.md+++ b/README.pt-BR.md@@ -158,7 +158,7 @@ O [Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (da age| **Gyro-S** | 27,6 GB | 58,5 GB | Para PCs onde nenhum outro tamanho cabe. A qualidade é menor que a do IQ2_XS. || **Gyro-M** | 35 GB | 92,0 GB | Qualidade próxima à do IQ3_XXS, com cerca de 20% menos memória da GPU. Ele escreve as respostas 17 a 21% mais devagar que o Q2_0 numa placa de 12 GB e numa R9700. |-O Gyro roda em placas NVIDIA e AMD (AMD é experimental), e não roda em Intel Arc nem no Mac. Detalhes e medições:+O Gyro roda em placas NVIDIA e AMD (AMD é experimental) e em placas Intel Arc no Linux (testado numa Arc A750), e não roda no Mac. Detalhes e medições:[docs/GYRO.md](docs/GYRO.md). Inicie com `strata launch --model gyro-s`.### Os outros tamanhosdiff --git a/README.zh-CN.md b/README.zh-CN.mdindex d16844858..72ec629d7 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -157,7 +157,7 @@ Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow doc| **Gyro-S** | 27.6 GB | 58.5 GB | 给其他尺寸都放不下的电脑。质量低于 IQ2_XS。 || **Gyro-M** | 35 GB | 92.0 GB | 质量约相当于 IQ3_XXS,显存占用少约 20%。在 12 GB 显卡和 R9700 上,输出回答比 Q2_0 慢 17-21%。 |-Gyro 可在 NVIDIA 和 AMD 显卡上运行(AMD 为实验性支持),不支持 Intel Arc 和 Mac。详情和测试数据:+Gyro 可在 NVIDIA 和 AMD 显卡(AMD 为实验性支持)以及 Linux 上的 Intel Arc 显卡(已在 Arc A750 上测试)上运行,不支持 Mac。详情和测试数据:[docs/GYRO.md](docs/GYRO.md)。用 `strata launch --model gyro-s` 启动。### 其他尺寸diff --git a/docs/MODELS.md b/docs/MODELS.mdindex c51fae258..a828be85d 100644--- a/docs/MODELS.md+++ b/docs/MODELS.md@@ -19,7 +19,7 @@ other size stays a choice. A PC with a card of 12 GB and 32 GB of RAM is in the| Your memory (RAM + VRAM) | Pre-selected | Notes || --- | --- | --- || **32 GB** | **Gyro-S**, or the **Coder** for code | The Coder needs 32 GB of RAM and keeps half of the experts, so it is weaker outside code ([why](#coder)). Gyro-S has lower quality than the 2-3-bit sizes, and the 2-bit Q2_0 decodes faster |-| **40 GB** | **Swift 1.5** IQ2_XS (39.2 GB) | Gyro-M (35 GB) is the other choice: close to IQ3_XXS in quality, but it writes answers 17-21% slower than Q2_0. Gyro runs on NVIDIA and AMD cards, not on Intel Arc or the Mac |+| **40 GB** | **Swift 1.5** IQ2_XS (39.2 GB) | Gyro-M (35 GB) is the other choice: close to IQ3_XXS in quality, but it writes answers 17-21% slower than Q2_0. Gyro runs on NVIDIA, AMD and Intel Arc cards, not on the Mac || **48 GB** | **IQ3_XXS** (47.0 GB) | Slower, better || **64 GB** | **IQ3_S** (54.8 GB), the original. Swift 1.5's IQ3_S is an equal choice | The slowest, the best: it matches the full model on the published tests || **96 GB or more** | [Unsloth's UD-IQ4_XS to Q8_0](#unsloths-files-ud-iq4_xs-to-q8_0), 4 to 8 bit | UD-IQ4_XS keeps every expert in RAM from about 80 GB of RAM, Q8_0 from about 136 GB. Up to 192 GB, the 5-8-bit files fit in RAM. Setup still pre-selects IQ3_S here, except on a Strix Halo |@@ -183,7 +183,7 @@ experts in a rotated basis. They are the smallest options and never setup's defa| **Gyro-S** | 58.5 GB | 27.6 GB | the smallest option, for PCs with little memory; lower quality || **Gyro-M** | 92.0 GB | 35 GB | about IQ3_XXS quality with about 20% less GPU memory; decodes about 20% slower than Q2_0 on 12 GB cards |-NVIDIA, and AMD (experimental); not Intel Arc or the Mac. Gyro reads pictures (the same rule as above). It needs+Gyro runs on NVIDIA and AMD (experimental) cards and on Intel Arc cards on Linux (tested on an Arc A750; the Arc engine is built from this release's source, see [INTEL.md](INTEL.md)); not on the Mac. Gyro reads pictures (the same rule as above). On NVIDIA and AMD it needsengine 0.1.45.1 or newer; if the engine is older, the app says "This model needs a newer engine" and offers Updateengine. Gyro gives correct output on RTX 50 cards with engine 0.1.45.3. Details, measurements and credit:[GYRO.md](GYRO.md).diff --git a/sycl/CMakeLists.txt b/sycl/CMakeLists.txtindex 4dc7ae000..6f963d098 100644--- a/sycl/CMakeLists.txt+++ b/sycl/CMakeLists.txt@@ -429,6 +429,10 @@ if(STRATA_SYCL_PARITY)target_include_directories(tq_sycl_parity PRIVATE ${STRATA_ROOT}/third_party/ggml)target_compile_options(tq_sycl_parity PRIVATE -ffp-contract=off)target_link_libraries(tq_sycl_parity PRIVATE strata_engine strata_kernels strata_kernels_cpu)+ # random trellis codes (every bit pattern is valid): the GPU kernels against a host replay of their own arithmetic, bit for bit,+ # the prompt-path dequantizers against the fp16 codebook, and the Hadamard rotations against the CPU's+ add_test(NAME tq_sycl_parity_synthetic COMMAND tq_sycl_parity --synthetic 145/146 144/144 144/146 145/144)+ add_test(NAME tq_sycl_parity_synthetic_hadamard COMMAND tq_sycl_parity --synthetic --hadamard 145/146 144/144)# MODEL-UPDATE: the CPU pool's expert rows per format against the RAM bandwidth (as the top-level build's)add_executable(kq_cpu_bench ${STRATA_ROOT}/src/kernels/cpu/kq_cpu_bench.cpp)target_link_libraries(kq_cpu_bench PRIVATE strata_kernels_cpu)diff --git a/sycl/setup_intel.py b/sycl/setup_intel.pyindex c4395e424..d7b66829c 100644--- a/sycl/setup_intel.py+++ b/sycl/setup_intel.py@@ -88,6 +88,12 @@ def intel_gpus():return found+def real_mem_have(mem_have, real_ram: float, fake_ram: float):+ """setup.mem_have for the Intel path: the RAM it is given is the 1024 GB stand-in of setup's RAM rule, so a model that counts VRAM + RAM+ (Gyro) would never warn; it gets the real RAM."""+ return lambda model, ram, vram=0.0: mem_have(model, real_ram if ram >= fake_ram else ram, vram)++def sycl_engine():"""The SYCL build: (binary, why-not)."""if S.WIN:@@ -406,8 +412,7 @@ def install(argv) -> None:# Gyro (mem_total) counts VRAM + RAM together and is the one family whose experts are NOT streamed from the file into VRAM: on# an Arc they sit in the RAM arena and the CPU computes what the card does not hold. Its memory warning ("needs 27.6 GB of VRAM +# RAM together, you have ...", never a refusal) therefore reads the real RAM, not the 1024 GB above.- mem_have = S.mem_have- S.mem_have = lambda model, ram, vram=0.0: mem_have(model, real_ram if ram >= fake_ram else ram, vram)+ S.mem_have = real_mem_have(S.mem_have, real_ram, fake_ram)bench_tips = S.bench_tipsdiff --git a/tools/test_setup_sycl.py b/tools/test_setup_sycl.pyindex 10ba23aa9..339243c85 100644--- a/tools/test_setup_sycl.py+++ b/tools/test_setup_sycl.py@@ -206,6 +206,19 @@ class ToSycl(unittest.TestCase):self.assertEqual(c["env"]["STRATA_VERIFY_NO_HOST"], "0")self.assertEqual(c["env"]["STRATA_SYCL_BIN"], "build-sycl/strata")+ def test_gyro_memory_warning_reads_the_real_ram(self):+ # setup's RAM rule is fed 1024 GB on the Intel path; Gyro counts VRAM + RAM, so it must see the real RAM+ m = self.mod()+ import importlib.util+ spec = importlib.util.spec_from_file_location("setup_for_test", ROOT / "setup.py")+ s = importlib.util.module_from_spec(spec)+ spec.loader.exec_module(s)+ have = m.real_mem_have(s.mem_have, 16.0, 1024.0)+ self.assertEqual(have("Gyro-S", 1024.0, 8.0), 24.0) # 16 GB RAM + 8 GB VRAM: below Gyro-S's 27.6, setup warns+ self.assertLess(have("Gyro-S", 1024.0, 8.0), s.MODELS["Gyro-S"]["ram_gb"])+ self.assertEqual(have("Gyro-S", 40.0, 8.0), 48.0) # a real RAM figure passes through+ self.assertEqual(have("Q2_0", 1024.0, 8.0), 16.0) # the other sizes count RAM only, as before+def test_a_series_keeps_gyros_ngram_table_on_the_ssd(self):# Gyro's table is half of the 58-92 GB file: --ple-io ram would not fit beside the experts' arena on a 64 GB PCm = self.mod()