Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: RTX 3060 speed with the release defaults (8 to 10 tokens/s, 149 tokens/s at 8K); DSpark stays off on 12 GB cards and adds 2-13% when turned on; README x7, DEEPSEEK41.md, release notes

Niko12218183c6b67162
diff --git a/README.de.md b/README.de.md
index e422baeec..108c21bc7 100644
--- a/README.de.md
+++ b/README.de.md
@@ -197,7 +197,7 @@ Alle Größen, Downloads und was wo läuft: [docs/MODELS.md](docs/MODELS.md).
| Set | Download | Qualität im Vergleich zum Originalmodell | Antwortet mit |
| --- | ---: | --- | --- |
| **IQ1_M** (1,75 Bit) | 239 GB | KL 0,217, Perplexität 3,80 (+21 %); Code +8,9 %, Agent-Aufgaben +10,2 %; schwächer bei einfacher englischer Prosa. Laut Modellkarte von ISTA-DASLab behält es 98,7 % des Aufgabendurchschnitts des Basismodells (AIME25, GPQA-Diamond, LiveCodeBench v6). | noch nicht gemessen |
-| **IQ2_XXS** (2.00 Bit) | 257 GB | KL 0,170, Perplexität 3,63 (+15,5 %); Code +8,6 %, Agent-Aufgaben +5,3 % | 6 bis 9 Tokens/s auf einer RTX 3060, etwa 5 auf einer RTX 5070 |
+| **IQ2_XXS** (2.00 Bit) | 257 GB | KL 0,170, Perplexität 3,63 (+15,5 %); Code +8,6 %, Agent-Aufgaben +5,3 % | etwa 8 bis 10 Tokens/s auf einer RTX 3060 (Code 9,9, Agent-Aufgaben 7,8), etwa 5 auf einer RTX 5070 |
| **IQ2_XS** (2.25 Bit) | 274 GB | KL 0,107, Perplexität 3,37 (+7,3 %); Code +3,2 %, Agent-Aufgaben +2,2 % | etwa 11 Tokens/s auf einer Radeon AI PRO R9700 |
| **IQ4_NL** (Engram-Tabelle, kein drittes Modell) | 111 GB, schon in allen drei Downloads oben enthalten | Die 4-Bit-Engram-Nachschlagetabelle, die das Modell aus RAM oder SSD liest. Alle drei Sets nutzen diese eine Datei (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |
diff --git a/README.es.md b/README.es.md
index c90d47fd9..b7d9f6d61 100644
--- a/README.es.md
+++ b/README.es.md
@@ -197,7 +197,7 @@ Todos los tamaños, descargas y lo que cabe en cada equipo: [docs/MODELS.md](doc
| Conjunto | Descarga | Calidad frente al modelo original | Responde a |
| --- | ---: | --- | --- |
| **IQ1_M** (1,75 bits) | 239 GB | KL 0,217, perplejidad 3,80 (+21 %); código +8,9 %, tareas de agente +10,2 %; más débil con prosa inglesa corriente. Según la ficha del modelo de ISTA-DASLab, conserva el 98,7 % del promedio de tareas del modelo base (AIME25, GPQA-Diamond, LiveCodeBench v6). | aún sin medir |
-| **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,170, perplejidad 3,63 (+15,5 %); código +8,6 %, tareas de agente +5,3 % | de 6 a 9 tokens/s en una RTX 3060, unos 5 en una RTX 5070 |
+| **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,170, perplejidad 3,63 (+15,5 %); código +8,6 %, tareas de agente +5,3 % | de 8 a 10 tokens/s en una RTX 3060 (código 9,9, tareas de agente 7,8), unos 5 en una RTX 5070 |
| **IQ2_XS** (2.25 bits) | 274 GB | KL 0,107, perplejidad 3,37 (+7,3 %); código +3,2 %, tareas de agente +2,2 % | unos 11 tokens/s en una Radeon AI PRO R9700 |
| **IQ4_NL** (tabla Engram, no es un tercer modelo) | 111 GB, ya incluidos en las tres descargas de arriba | La tabla de consulta Engram de 4 bits que el modelo lee de la RAM o del SSD. Los tres conjuntos usan este único archivo (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |
diff --git a/README.fr.md b/README.fr.md
index 9661a6bd4..3385a97b8 100644
--- a/README.fr.md
+++ b/README.fr.md
@@ -198,7 +198,7 @@ Toutes les tailles, les téléchargements et ce qui tient où : [docs/MODELS.md]
| Version | Téléchargement | Qualité par rapport au modèle d’origine | Écrit à |
| --- | ---: | --- | --- |
| **IQ1_M** (1,75 bit) | 239 Go | KL 0,217, perplexité 3,80 (+21 %) ; code +8,9 %, tâches d’agent +10,2 % ; plus faible sur la prose anglaise courante. D’après la fiche du modèle d’ISTA-DASLab, elle garde 98,7 % de la moyenne des tâches du modèle de base (AIME25, GPQA-Diamond, LiveCodeBench v6). | pas encore mesurée |
-| **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,170, perplexité 3,63 (+15,5 %) ; code +8,6 %, tâches d’agent +5,3 % | 6 à 9 tokens/s sur une RTX 3060, environ 5 sur une RTX 5070 |
+| **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,170, perplexité 3,63 (+15,5 %) ; code +8,6 %, tâches d’agent +5,3 % | environ 8 à 10 tokens/s sur une RTX 3060 (code 9,9, tâches d’agent 7,8), environ 5 sur une RTX 5070 |
| **IQ2_XS** (2.25 bits) | 274 GB | KL 0,107, perplexité 3,37 (+7,3 %) ; code +3,2 %, tâches d’agent +2,2 % | environ 11 tokens/s sur une Radeon AI PRO R9700 |
| **IQ4_NL** (table Engram, pas un troisième modèle) | 111 Go, déjà compris dans les trois téléchargements ci-dessus | La table de consultation Engram en 4 bits que le modèle lit dans la RAM ou sur le SSD. Les trois versions utilisent ce seul fichier (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |
diff --git a/README.ja.md b/README.ja.md
index b3ba4196c..34a243f35 100644
--- a/README.ja.md
+++ b/README.ja.md
@@ -190,7 +190,7 @@ Gyro は NVIDIA と AMD のカード(AMD は実験的)、および Linux 上
| セット | ダウンロード | 元のモデルとの品質差 | 出力速度 |
| --- | ---: | --- | --- |
| **IQ1_M**(1.75 ビット) | 239 GB | KL 0.217、パープレキシティ 3.80(+21%)、コード +8.9%、エージェント作業 +10.2%、通常の英語の散文では弱めです。ISTA-DASLab のモデルカードによると、基本モデルのタスク平均の 98.7% を保ちます(AIME25、GPQA-Diamond、LiveCodeBench v6)。 | 未測定 |
-| **IQ2_XXS** (2.00 ビット) | 257 GB | KL 0.170、パープレキシティ 3.63(+15.5%)、コード +8.6%、エージェント作業 +5.3% | RTX 3060 で毎秒 6 から 9 トークン、RTX 5070 で約 5 |
+| **IQ2_XXS** (2.00 ビット) | 257 GB | KL 0.170、パープレキシティ 3.63(+15.5%)、コード +8.6%、エージェント作業 +5.3% | RTX 3060 で毎秒約 8 から 10 トークン(コード 9.9、エージェント作業 7.8)、RTX 5070 で約 5 |
| **IQ2_XS** (2.25 ビット) | 274 GB | KL 0.107、パープレキシティ 3.37(+7.3%)、コード +3.2%、エージェント作業 +2.2% | Radeon AI PRO R9700 1 枚で毎秒約 11 トークン |
| **IQ4_NL**(Engram テーブル、3 つ目のモデルではありません) | 111 GB、上の 3 つのダウンロードに含まれています | モデルが RAM または SSD から読む 4 ビットの Engram 参照テーブルです。3 つのセットはこの 1 つのファイル(`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`)を共有します。 | 該当なし |
diff --git a/README.md b/README.md
index 06b39950c..6d09eb425 100644
--- a/README.md
+++ b/README.md
@@ -195,7 +195,7 @@ Every size, download and what fits where: [docs/MODELS.md](docs/MODELS.md).
| Set | Download | Quality against the original model | Writes at |
| --- | ---: | --- | --- |
| **IQ1_M** (1.75 bits) | 239 GB | KL 0.217, perplexity 3.80 (+21%); code +8.9%, agent work +10.2%; weaker on plain English prose. ISTA-DASLab's model card: keeps 98.7% of the base model's task average (AIME25, GPQA-Diamond, LiveCodeBench v6). | not measured yet |
-| **IQ2_XXS** (2.00 bits) | 257 GB | KL 0.170, perplexity 3.63 (+15.5%); code +8.6%, agent work +5.3% | 6 to 9 tokens/s on an RTX 3060, about 5 on an RTX 5070 |
+| **IQ2_XXS** (2.00 bits) | 257 GB | KL 0.170, perplexity 3.63 (+15.5%); code +8.6%, agent work +5.3% | about 8 to 10 tokens/s on an RTX 3060 (code 9.9, agent work 7.8), about 5 on an RTX 5070 |
| **IQ2_XS** (2.25 bits) | 274 GB | KL 0.107, perplexity 3.37 (+7.3%); code +3.2%, agent work +2.2% | about 11 tokens/s on one Radeon AI PRO R9700 |
| **IQ4_NL** (Engram table, not a third model) | 111 GB, already in all three downloads above | The 4-bit Engram lookup table the model reads from RAM or SSD. All three sets use this one file (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |
diff --git a/README.pt-BR.md b/README.pt-BR.md
index 7fe4ac539..3b6204bca 100644
--- a/README.pt-BR.md
+++ b/README.pt-BR.md
@@ -190,7 +190,7 @@ Todos os tamanhos, downloads e o que cabe em cada configuração: [docs/MODELS.m
| Conjunto | Download | Qualidade em relação ao modelo original | Responde a |
| --- | ---: | --- | --- |
| **IQ1_M** (1,75 bits) | 239 GB | KL 0,217, perplexidade 3,80 (+21%); código +8,9%, tarefas de agente +10,2%; mais fraco em prosa comum em inglês. Segundo o cartão do modelo da ISTA-DASLab, mantém 98,7% da média de tarefas do modelo base (AIME25, GPQA-Diamond, LiveCodeBench v6). | ainda não medida |
-| **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,170, perplexidade 3,63 (+15,5%); código +8,6%, tarefas de agente +5,3% | 6 a 9 tokens/s em uma RTX 3060, cerca de 5 em uma RTX 5070 |
+| **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,170, perplexidade 3,63 (+15,5%); código +8,6%, tarefas de agente +5,3% | cerca de 8 a 10 tokens/s em uma RTX 3060 (código 9,9, tarefas de agente 7,8), cerca de 5 em uma RTX 5070 |
| **IQ2_XS** (2.25 bits) | 274 GB | KL 0,107, perplexidade 3,37 (+7,3%); código +3,2%, tarefas de agente +2,2% | cerca de 11 tokens/s em uma Radeon AI PRO R9700 |
| **IQ4_NL** (tabela Engram, não é um terceiro modelo) | 111 GB, já incluídos nos três downloads acima | A tabela de consulta Engram de 4 bits que o modelo lê da RAM ou do SSD. Os três conjuntos usam este único arquivo (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |
diff --git a/README.zh-CN.md b/README.zh-CN.md
index cab50bb0b..951265096 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -189,7 +189,7 @@ Gyro 可在 NVIDIA 和 AMD 显卡(AMD 为实验性支持)以及 Linux 上的
| 版本 | 下载大小 | 与原模型相比的质量 | 生成速度 |
| --- | ---: | --- | --- |
| **IQ1_M**(1.75 位) | 239 GB | KL 0.217,困惑度 3.80(+21%);代码 +8.9%,智能体任务 +10.2%;普通英文散文表现较弱。据 ISTA-DASLab 的模型卡,它保留了基础模型任务平均分的 98.7%(AIME25、GPQA-Diamond、LiveCodeBench v6)。 | 尚未测量 |
-| **IQ2_XXS** (2.00 位) | 257 GB | KL 0.170,困惑度 3.63(+15.5%);代码 +8.6%,智能体任务 +5.3% | RTX 3060 上每秒 6 到 9 个 token,RTX 5070 上约 5 个 |
+| **IQ2_XXS** (2.00 位) | 257 GB | KL 0.170,困惑度 3.63(+15.5%);代码 +8.6%,智能体任务 +5.3% | RTX 3060 上每秒约 8 到 10 个 token(代码 9.9,智能体任务 7.8),RTX 5070 上约 5 个 |
| **IQ2_XS** (2.25 位) | 274 GB | KL 0.107,困惑度 3.37(+7.3%);代码 +3.2%,智能体任务 +2.2% | 一块 Radeon AI PRO R9700 上每秒约 11 个 token |
| **IQ4_NL**(Engram 表,不是第三个模型) | 111 GB,已包含在上面三个下载中 | 模型从内存或固态硬盘读取的 4 位 Engram 查找表。三个版本共用这一个文件(`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`)。 | 不适用 |
diff --git a/docs/DEEPSEEK41.md b/docs/DEEPSEEK41.md
index e3ad46cbd..fd2e2f5d4 100644
--- a/docs/DEEPSEEK41.md
+++ b/docs/DEEPSEEK41.md
@@ -42,7 +42,7 @@ perplexity +7.3% (code +3.2%, agent work +2.2%). IQ1_M: perplexity +21% (KL 0.21
121 + 111 + 8 = 239 GB for 1.75; 138 + 111 + 8 = 257 GB for 2.00; 155 + 111 + 8 = 274 GB for 2.25.
Measured: an RTX 5070 (12 GB) with 64 GB of RAM and a 3.6 GB/s NVMe drive answers at about 4.7 tokens/s and reads prompts at 121
-tokens/s (4K) to 231 tokens/s (8K, layer-major path); an RTX 3060 (12 GB) with 46 GB of RAM about 7.2 tokens/s; four R9700 about 31
+tokens/s (4K) to 231 tokens/s (8K, layer-major path); an RTX 3060 (12 GB) with 46 GB of RAM about 8 to 10 tokens/s (code 9.9, agent work 7.8; 149 tokens/s at 8K; DSpark stays off there, `--dspark on` adds 2-13%); four R9700 about 31
tokens/s. Under about 40 GB of RAM it is untested and slow; setup says so and still installs.
Setup downloads with resume and SHA-256, packs the model with the plain recipe (`pack_gguf.py --model SET --shards ... --engram
diff --git a/docs/release-notes-0.2.4.1.md b/docs/release-notes-0.2.4.1.md
index e94761ac4..31d4ad82a 100644
--- a/docs/release-notes-0.2.4.1.md
+++ b/docs/release-notes-0.2.4.1.md
@@ -4,6 +4,10 @@
Only the Terminal's Control changed (version 0.1.49.1). The app stays 0.2.3 and every engine package stays as it was, so there is nothing new to download for them.
+## Corrected speed figure for the RTX 3060
+
+The 0.2.4 notes said DSpark raised an RTX 3060 PC from 6.1-6.7 to 8.2-8.7 tokens/s. That was measured with DSpark forced on; on 12 GB cards DSpark stays off by default. Measured with the release defaults on an RTX 3060 PC with 46 GB of RAM: 8 to 10 tokens/s for answers (code 9.9, agent work 7.8, chat 8.9) and 149 tokens/s for an 8K prompt. Turning DSpark on (`--dspark on`) adds 2 to 13% there.
+
## Updating
- **From 0.2.4:** run `strata update` in the Terminal. It fetches Control 0.1.49.1; the next `strata` command already runs it (`strata --version` says 0.2.4.1). Then run your DeepSeek setup again.
diff --git a/docs/release-notes-0.2.4.md b/docs/release-notes-0.2.4.md
index ca06dd10e..3ead59873 100644
--- a/docs/release-notes-0.2.4.md
+++ b/docs/release-notes-0.2.4.md
@@ -25,7 +25,7 @@ Output is the speed of the answer. Prompt is how fast Strata reads what you send
| PC | output | prompt |
|---|---|---|
| RTX 5070 12 GB, 64 GB RAM | about 5 tokens/s (code) | 231 tokens/s at 8K |
-| RTX 3060 12 GB, 46 GB RAM | about 6 to 9 tokens/s (6.1-6.7 without DSpark, 8.2-8.7 with) | about 103 tokens/s at 8K |
+| RTX 3060 12 GB, 46 GB RAM | about 8 to 10 tokens/s (code 9.9, agent work 7.8) | 149 tokens/s at 8K |
| 1x Radeon AI PRO R9700 (AMD), IQ2_XS | about 11 tokens/s | not measured |
| 4x Radeon AI PRO R9700 (AMD) | about 31 tokens/s (55 to 69 with DSpark) | about 505 tokens/s at 8K, about 818 at 32K |
@@ -39,7 +39,7 @@ These are the 2.00-bit set. The 2.25-bit set reads 13% more data per token, so e
- **Faster prompts on Radeon AI PRO R9700.** New matrix-core kernels (RDNA4) read prompts 36% faster at 8K and 49% faster at 32K on 4 cards (370 to 505 and 550 to 818 tokens/s). Answer speed does not change. They are on by default on these cards.
- **Faster long prompts.** A prompt over about 3,700 tokens skips work that the last 20 layers cannot use (a 16K prompt is about 34% faster). In 99.5% of the 799 positions we compared, the model's top choice for the next word was the same with and without the skip.
- **Thinking budget.** When neither your request nor your config sets one, DeepSeek thinks for at most 32,768 tokens. A request can set its own budget; 0 turns the limit off.
-- **DSpark** (a small draft model from Unsloth) is on where it measured faster and off on CPU-bound PCs. On the RTX 3060 PC (20 threads) it raised decode from 6.1-6.7 to 8.2-8.7 tokens/s. On the 4x R9700 box it raised 31 tokens/s to 55 to 69. On the RTX 5070 PC with a 6-core Ryzen 5 7600 it gave no gain at any setting (code 4.64 tokens/s with it off against 4.33 at best with it on; agent 3.41 against 2.86 to 3.21), because checking the drafts costs the CPU about what the saved tokens are worth. There `--dspark auto` does not load the drafter.
+- **DSpark** (a small draft model from Unsloth) is on where it measured faster and off on CPU-bound PCs. On 12 GB cards DSpark stays off by default; turning it on (`--dspark on`) adds 2-13% on the RTX 3060 PC. On the 4x R9700 box it raised 31 tokens/s to 55 to 69. On the RTX 5070 PC with a 6-core Ryzen 5 7600 it gave no gain at any setting (code 4.64 tokens/s with it off against 4.33 at best with it on; agent 3.41 against 2.86 to 3.21), because checking the drafts costs the CPU about what the saved tokens are worth. There `--dspark auto` does not load the drafter.
- **Faster CPU rows.** The engine has new exact rows for the experts the CPU computes: AVX-512 on Zen 4 and Zen 5, AVX2 on other CPUs, chosen automatically. On the Ryzen 5 7600 PC with an RTX 5070 (2.00 set, DSpark on, three alternating pairs, identical text in all six runs), decode went from 4.08 to 5.11 tokens/s (+25%) and prompt reading rose 26%. This was measured on one CPU, a Zen 4. Zen 5 gained more in the kernel tests.
- **Cards and engines:** tested on an RTX 3060 (Linux, full model), an RTX 5070 (Windows) and the Radeon AI PRO R9700 (gfx1201, Linux). The RTX 30 kernels (sm_86) were also checked on an RTX A4000. Windows on RTX 30 is built but was not run here. RX 9070 (gfx1200) is built but untested. DeepSeek runs on its own engine package (see below).