Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
DeepSeek table: the IQ4_NL Engram table row (README and 6 translations, DEEPSEEK41.md, release notes); notes: long-prompt skip and the 32,768 thinking budget
Niko12210e121c7a1363
diff --git a/README.de.md b/README.de.mdindex 3ff89dc5d..65a8279d4 100644--- a/README.de.md+++ b/README.de.md@@ -198,6 +198,7 @@ Alle Größen, Downloads und was wo läuft: [docs/MODELS.md](docs/MODELS.md).| --- | ---: | --- | --- || **IQ2_XXS** (2.00 Bit) | 257 GB | KL 0,169, Perplexität 3,64 (+16 %) | 6 bis 9 Tokens/s auf einer RTX 3060, etwa 5 auf einer RTX 5070 || **IQ2_XS** (2.25 Bit) | 274 GB | KL 0,110, Perplexität 3,40 (+8 %); Code +3,5 %, Agent-Aufgaben +2,5 % | etwa 11 Tokens/s auf einer Radeon AI PRO R9700 |+| **IQ4_NL** (Engram-Tabelle, kein drittes Modell) | 111 GB, schon in beiden Downloads oben enthalten | Die 4-Bit-Engram-Nachschlagetabelle, die das Modell aus RAM oder SSD liest. Beide Sets nutzen diese eine Datei (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |4x Radeon AI PRO R9700: etwa 31 Tokens/s, 55 bis 69 mit DSpark.diff --git a/README.es.md b/README.es.mdindex a0f818389..e18da76ef 100644--- a/README.es.md+++ b/README.es.md@@ -198,6 +198,7 @@ Todos los tamaños, descargas y lo que cabe en cada equipo: [docs/MODELS.md](doc| --- | ---: | --- | --- || **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,169, perplejidad 3,64 (+16 %) | de 6 a 9 tokens/s en una RTX 3060, unos 5 en una RTX 5070 || **IQ2_XS** (2.25 bits) | 274 GB | KL 0,110, perplejidad 3,40 (+8 %); código +3,5 %, tareas de agente +2,5 % | unos 11 tokens/s en una Radeon AI PRO R9700 |+| **IQ4_NL** (tabla Engram, no es un tercer modelo) | 111 GB, ya incluidos en las dos descargas de arriba | La tabla de consulta Engram de 4 bits que el modelo lee de la RAM o del SSD. Los dos conjuntos usan este único archivo (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |4x Radeon AI PRO R9700: unos 31 tokens/s, de 55 a 69 con DSpark.diff --git a/README.fr.md b/README.fr.mdindex 8fbfb4f0f..948b2422f 100644--- a/README.fr.md+++ b/README.fr.md@@ -199,6 +199,7 @@ Toutes les tailles, les téléchargements et ce qui tient où : [docs/MODELS.md]| --- | ---: | --- | --- || **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,169, perplexité 3,64 (+16 %) | 6 à 9 tokens/s sur une RTX 3060, environ 5 sur une RTX 5070 || **IQ2_XS** (2.25 bits) | 274 GB | KL 0,110, perplexité 3,40 (+8 %) ; code +3,5 %, tâches d’agent +2,5 % | environ 11 tokens/s sur une Radeon AI PRO R9700 |+| **IQ4_NL** (table Engram, pas un troisième modèle) | 111 Go, déjà compris dans les deux téléchargements ci-dessus | La table de consultation Engram en 4 bits que le modèle lit dans la RAM ou sur le SSD. Les deux versions utilisent ce seul fichier (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |4x Radeon AI PRO R9700 : environ 31 tokens/s, 55 à 69 avec DSpark.diff --git a/README.ja.md b/README.ja.mdindex 9a6a2c20b..26351c870 100644--- a/README.ja.md+++ b/README.ja.md@@ -191,6 +191,7 @@ Gyro は NVIDIA と AMD のカード(AMD は実験的)、および Linux 上| --- | ---: | --- | --- || **IQ2_XXS** (2.00 ビット) | 257 GB | KL 0.169、パープレキシティ 3.64(+16%) | RTX 3060 で毎秒 6 から 9 トークン、RTX 5070 で約 5 || **IQ2_XS** (2.25 ビット) | 274 GB | KL 0.110、パープレキシティ 3.40(+8%)、コード +3.5%、エージェント作業 +2.5% | Radeon AI PRO R9700 1 枚で毎秒約 11 トークン |+| **IQ4_NL**(Engram テーブル、3 つ目のモデルではありません) | 111 GB、上の 2 つのダウンロードに含まれています | モデルが RAM または SSD から読む 4 ビットの Engram 参照テーブルです。2 つのセットはこの 1 つのファイル(`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`)を共有します。 | 該当なし |Radeon AI PRO R9700 4 枚:毎秒約 31 トークン、DSpark ありで 55 から 69。diff --git a/README.md b/README.mdindex 94236f357..49845e0bd 100644--- a/README.md+++ b/README.md@@ -196,6 +196,7 @@ Every size, download and what fits where: [docs/MODELS.md](docs/MODELS.md).| --- | ---: | --- | --- || **IQ2_XXS** (2.00 bits) | 257 GB | KL 0.169, perplexity 3.64 (+16%) | 6 to 9 tokens/s on an RTX 3060, about 5 on an RTX 5070 || **IQ2_XS** (2.25 bits) | 274 GB | KL 0.110, perplexity 3.40 (+8%); code +3.5%, agent work +2.5% | about 11 tokens/s on one Radeon AI PRO R9700 |+| **IQ4_NL** (Engram table, not a third model) | 111 GB, already in both downloads above | The 4-bit Engram lookup table the model reads from RAM or SSD. Both sets use this one file (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |4x Radeon AI PRO R9700: about 31 tokens/s, 55 to 69 with DSpark.diff --git a/README.pt-BR.md b/README.pt-BR.mdindex afcf5a005..414b08e5d 100644--- a/README.pt-BR.md+++ b/README.pt-BR.md@@ -191,6 +191,7 @@ Todos os tamanhos, downloads e o que cabe em cada configuração: [docs/MODELS.m| --- | ---: | --- | --- || **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,169, perplexidade 3,64 (+16%) | 6 a 9 tokens/s em uma RTX 3060, cerca de 5 em uma RTX 5070 || **IQ2_XS** (2.25 bits) | 274 GB | KL 0,110, perplexidade 3,40 (+8%); código +3,5%, tarefas de agente +2,5% | cerca de 11 tokens/s em uma Radeon AI PRO R9700 |+| **IQ4_NL** (tabela Engram, não é um terceiro modelo) | 111 GB, já incluídos nos dois downloads acima | A tabela de consulta Engram de 4 bits que o modelo lê da RAM ou do SSD. Os dois conjuntos usam este único arquivo (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |4x Radeon AI PRO R9700: cerca de 31 tokens/s, 55 a 69 com DSpark.diff --git a/README.zh-CN.md b/README.zh-CN.mdindex 98c5b869a..02bd4a16d 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -190,6 +190,7 @@ Gyro 可在 NVIDIA 和 AMD 显卡(AMD 为实验性支持)以及 Linux 上的| --- | ---: | --- | --- || **IQ2_XXS** (2.00 位) | 257 GB | KL 0.169,困惑度 3.64(+16%) | RTX 3060 上每秒 6 到 9 个 token,RTX 5070 上约 5 个 || **IQ2_XS** (2.25 位) | 274 GB | KL 0.110,困惑度 3.40(+8%);代码 +3.5%,智能体任务 +2.5% | 一块 Radeon AI PRO R9700 上每秒约 11 个 token |+| **IQ4_NL**(Engram 表,不是第三个模型) | 111 GB,已包含在上面两个下载中 | 模型从内存或固态硬盘读取的 4 位 Engram 查找表。两个版本共用这一个文件(`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`)。 | 不适用 |4 块 Radeon AI PRO R9700:每秒约 31 个 token,使用 DSpark 时为 55 到 69。diff --git a/docs/DEEPSEEK41.md b/docs/DEEPSEEK41.mdindex 058abcd2c..0bd609bec 100644--- a/docs/DEEPSEEK41.md+++ b/docs/DEEPSEEK41.md@@ -31,6 +31,15 @@ same 111 GB n-gram table (downloaded once into `models/deepseek-v4.1-flash-share2.00, 274 GB for 2.25, 412 GB for both. Quality against the original model: perplexity +16% on 2.00, +8% on 2.25 (code and agent workabout +3.5% / +2.5% on 2.25).+| File | Size | What it is |+| --- | ---: | --- |+| IQ2_XXS shards (2.00 bpw), 4 files | 138 GB | the model, first set |+| IQ2_XS shards (2.25 bpw), 4 files | 155 GB | the model, second set |+| IQ4_NL: `DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf` | 111 GB | the 4-bit Engram lookup table the model reads from RAM or SSD. Not a third model: both sets use this one file, and it is already in their download sizes |+| DSpark drafter (Q8_0) | 8 GB | the small draft model (Unsloth) |++138 + 111 + 8 = 257 GB for 2.00; 155 + 111 + 8 = 274 GB for 2.25.+Measured: an RTX 5070 (12 GB) with 64 GB of RAM and a 3.6 GB/s NVMe drive answers at about 4.7 tokens/s and reads prompts at 121tokens/s (4K) to 231 tokens/s (8K, layer-major path); an RTX 3060 (12 GB) with 46 GB of RAM about 7.2 tokens/s; four R9700 about 31tokens/s. Under about 40 GB of RAM it is untested and slow; setup says so and still installs.diff --git a/docs/release-notes-0.2.4.md b/docs/release-notes-0.2.4.mdindex 8e634ae11..d61ed03b1 100644--- a/docs/release-notes-0.2.4.md+++ b/docs/release-notes-0.2.4.md@@ -12,6 +12,7 @@ Strata now runs DeepSeek V4.1-Flash on a PC with one 12 GB graphics card and 46|---|---|---|| IQ2_XXS (2.00 bits) | 257 GB | perplexity +16%; code and agent text +8.5% and +5.9% || IQ2_XS (2.25 bits) | 274 GB | perplexity +7.7% to +8.3%; code and agent text +3.5% and +2.5% |+ | IQ4_NL (Engram table, not a third model) | 111 GB, already in both downloads above | The 4-bit Engram lookup table the model reads from RAM or SSD; both sets use this one file |"Perplexity" is how surprised the model is by a fixed set of 4,995 test texts; lower is closer to the original. Pick 2.25 for coding and agent work when you have the disk. Setup shows both sizes and warns if your PC is short of disk or RAM. It does not stop you.- Setup downloads the files, builds the model folder and starts the engine. You set no flags.@@ -34,6 +35,8 @@ These are the 2.00-bit set. The 2.25-bit set reads 13% more data per token, so e- **A RAM expert store that sizes itself.** The model is a mixture of experts: each token uses a few of thousands of small blocks. Strata keeps the busiest ones in RAM, takes whatever RAM is free, and reads the rest from the SSD with direct reads. On the RTX 5070 PC with 64 GB this gave 4.3 to 4.9 tokens/s where memory-mapping the files gave 1.8 to 2.2.- **Hot-expert profiles ship with the engine,** so the first prompts do not start cold.- **Faster prompts.** The engine now reads long prompts layer by layer in big chunks instead of 16 tokens at a time. A 200-token prompt went from 3.6 to 6.4 tokens/s on the RTX 5070 PC, and 8K prompts reach 231 tokens/s.+- **Faster long prompts.** A prompt over about 3,700 tokens skips work that the last 20 layers cannot use (a 16K prompt is about 34% faster). The answers match the full computation in 99.75% of positions.+- **Thinking budget.** When neither your request nor your config sets one, DeepSeek thinks for at most 32,768 tokens. A request can set its own budget; 0 turns the limit off.- **DSpark** (a small draft model from Unsloth) is on where it measured faster and off on CPU-bound PCs. On the RTX 3060 PC (20 threads) it raised decode from 6.1-6.7 to 8.2-8.7 tokens/s. On the 4x R9700 box it raised 31 tokens/s to 55 to 69. On the RTX 5070 PC with a 6-core Ryzen 5 7600 it gave no gain at any setting (code 4.64 tokens/s with it off against 4.33 at best with it on; agent 3.41 against 2.86 to 3.21), because checking the drafts costs the CPU about what the saved tokens are worth. There `--dspark auto` does not load the drafter.- **Faster CPU rows.** The engine has new exact rows for the experts the CPU computes: AVX-512 on Zen 4 and Zen 5, AVX2 on other CPUs, chosen automatically. On the Ryzen 5 7600 PC with an RTX 5070 (2.00 set, DSpark on, three alternating pairs, identical text in all six runs), decode went from 4.08 to 5.11 tokens/s (+25%) and prompt reading rose 26%. This was measured on one CPU, a Zen 4. Zen 5 gained more in the kernel tests.- **Cards and engines:** tested on an RTX 3060 (Linux, full model), an RTX 5070 (Windows) and the Radeon AI PRO R9700 (gfx1201, Linux). The RTX 30 kernels (sm_86) were also checked on an RTX A4000. Windows on RTX 30 is built but was not run here. RX 9070 (gfx1200) is built but untested. The engine packages carry `strata-dsv41` next to `strata`.