Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
IQ1_M quality from the H100 (KL 0.217, ppl 3.80, +21%; code +8.9%, agent +10.2%, weaker on plain prose) in setup, the fit text, README x7, DEEPSEEK41.md and the release notes
Niko122152b104b1ac47
diff --git a/README.de.md b/README.de.mdindex 43b6184f9..fd164c25a 100644--- a/README.de.md+++ b/README.de.md@@ -196,7 +196,7 @@ Alle Größen, Downloads und was wo läuft: [docs/MODELS.md](docs/MODELS.md).| Set | Download | Qualität im Vergleich zum Originalmodell | Antwortet mit || --- | ---: | --- | --- |-| **IQ1_M** (1,75 Bit) | 239 GB | KL und Perplexität: werden noch gemessen. Laut Modellkarte von ISTA-DASLab behält es 98,7 % des Aufgabendurchschnitts des Basismodells (AIME25, GPQA-Diamond, LiveCodeBench v6). | noch nicht gemessen |+| **IQ1_M** (1,75 Bit) | 239 GB | KL 0,217, Perplexität 3,80 (+21 %); Code +8,9 %, Agent-Aufgaben +10,2 %; schwächer bei einfacher englischer Prosa. Laut Modellkarte von ISTA-DASLab behält es 98,7 % des Aufgabendurchschnitts des Basismodells (AIME25, GPQA-Diamond, LiveCodeBench v6). | noch nicht gemessen || **IQ2_XXS** (2.00 Bit) | 257 GB | KL 0,169, Perplexität 3,64 (+16 %) | 6 bis 9 Tokens/s auf einer RTX 3060, etwa 5 auf einer RTX 5070 || **IQ2_XS** (2.25 Bit) | 274 GB | KL 0,110, Perplexität 3,40 (+8 %); Code +3,5 %, Agent-Aufgaben +2,5 % | etwa 11 Tokens/s auf einer Radeon AI PRO R9700 || **IQ4_NL** (Engram-Tabelle, kein drittes Modell) | 111 GB, schon in allen drei Downloads oben enthalten | Die 4-Bit-Engram-Nachschlagetabelle, die das Modell aus RAM oder SSD liest. Alle drei Sets nutzen diese eine Datei (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |diff --git a/README.es.md b/README.es.mdindex a77d76ee8..2b17edbf7 100644--- a/README.es.md+++ b/README.es.md@@ -196,7 +196,7 @@ Todos los tamaños, descargas y lo que cabe en cada equipo: [docs/MODELS.md](doc| Conjunto | Descarga | Calidad frente al modelo original | Responde a || --- | ---: | --- | --- |-| **IQ1_M** (1,75 bits) | 239 GB | KL y perplejidad: se están midiendo. Según la ficha del modelo de ISTA-DASLab, conserva el 98,7 % del promedio de tareas del modelo base (AIME25, GPQA-Diamond, LiveCodeBench v6). | aún sin medir |+| **IQ1_M** (1,75 bits) | 239 GB | KL 0,217, perplejidad 3,80 (+21 %); código +8,9 %, tareas de agente +10,2 %; más débil con prosa inglesa corriente. Según la ficha del modelo de ISTA-DASLab, conserva el 98,7 % del promedio de tareas del modelo base (AIME25, GPQA-Diamond, LiveCodeBench v6). | aún sin medir || **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,169, perplejidad 3,64 (+16 %) | de 6 a 9 tokens/s en una RTX 3060, unos 5 en una RTX 5070 || **IQ2_XS** (2.25 bits) | 274 GB | KL 0,110, perplejidad 3,40 (+8 %); código +3,5 %, tareas de agente +2,5 % | unos 11 tokens/s en una Radeon AI PRO R9700 || **IQ4_NL** (tabla Engram, no es un tercer modelo) | 111 GB, ya incluidos en las tres descargas de arriba | La tabla de consulta Engram de 4 bits que el modelo lee de la RAM o del SSD. Los tres conjuntos usan este único archivo (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |diff --git a/README.fr.md b/README.fr.mdindex a62fb8010..a5516652d 100644--- a/README.fr.md+++ b/README.fr.md@@ -197,7 +197,7 @@ Toutes les tailles, les téléchargements et ce qui tient où : [docs/MODELS.md]| Version | Téléchargement | Qualité par rapport au modèle d’origine | Écrit à || --- | ---: | --- | --- |-| **IQ1_M** (1,75 bit) | 239 Go | KL et perplexité : en cours de mesure. D’après la fiche du modèle d’ISTA-DASLab, elle garde 98,7 % de la moyenne des tâches du modèle de base (AIME25, GPQA-Diamond, LiveCodeBench v6). | pas encore mesurée |+| **IQ1_M** (1,75 bit) | 239 Go | KL 0,217, perplexité 3,80 (+21 %) ; code +8,9 %, tâches d’agent +10,2 % ; plus faible sur la prose anglaise courante. D’après la fiche du modèle d’ISTA-DASLab, elle garde 98,7 % de la moyenne des tâches du modèle de base (AIME25, GPQA-Diamond, LiveCodeBench v6). | pas encore mesurée || **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,169, perplexité 3,64 (+16 %) | 6 à 9 tokens/s sur une RTX 3060, environ 5 sur une RTX 5070 || **IQ2_XS** (2.25 bits) | 274 GB | KL 0,110, perplexité 3,40 (+8 %) ; code +3,5 %, tâches d’agent +2,5 % | environ 11 tokens/s sur une Radeon AI PRO R9700 || **IQ4_NL** (table Engram, pas un troisième modèle) | 111 Go, déjà compris dans les trois téléchargements ci-dessus | La table de consultation Engram en 4 bits que le modèle lit dans la RAM ou sur le SSD. Les trois versions utilisent ce seul fichier (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |diff --git a/README.ja.md b/README.ja.mdindex d53a43289..07a2f41df 100644--- a/README.ja.md+++ b/README.ja.md@@ -189,7 +189,7 @@ Gyro は NVIDIA と AMD のカード(AMD は実験的)、および Linux 上| セット | ダウンロード | 元のモデルとの品質差 | 出力速度 || --- | ---: | --- | --- |-| **IQ1_M**(1.75 ビット) | 239 GB | KL とパープレキシティ:測定中。ISTA-DASLab のモデルカードによると、基本モデルのタスク平均の 98.7% を保ちます(AIME25、GPQA-Diamond、LiveCodeBench v6)。 | 未測定 |+| **IQ1_M**(1.75 ビット) | 239 GB | KL 0.217、パープレキシティ 3.80(+21%)、コード +8.9%、エージェント作業 +10.2%、通常の英語の散文では弱めです。ISTA-DASLab のモデルカードによると、基本モデルのタスク平均の 98.7% を保ちます(AIME25、GPQA-Diamond、LiveCodeBench v6)。 | 未測定 || **IQ2_XXS** (2.00 ビット) | 257 GB | KL 0.169、パープレキシティ 3.64(+16%) | RTX 3060 で毎秒 6 から 9 トークン、RTX 5070 で約 5 || **IQ2_XS** (2.25 ビット) | 274 GB | KL 0.110、パープレキシティ 3.40(+8%)、コード +3.5%、エージェント作業 +2.5% | Radeon AI PRO R9700 1 枚で毎秒約 11 トークン || **IQ4_NL**(Engram テーブル、3 つ目のモデルではありません) | 111 GB、上の 3 つのダウンロードに含まれています | モデルが RAM または SSD から読む 4 ビットの Engram 参照テーブルです。3 つのセットはこの 1 つのファイル(`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`)を共有します。 | 該当なし |diff --git a/README.md b/README.mdindex 1299c6caf..c41f87ae6 100644--- a/README.md+++ b/README.md@@ -194,7 +194,7 @@ Every size, download and what fits where: [docs/MODELS.md](docs/MODELS.md).| Set | Download | Quality against the original model | Writes at || --- | ---: | --- | --- |-| **IQ1_M** (1.75 bits) | 239 GB | KL and perplexity: being measured. ISTA-DASLab's model card: keeps 98.7% of the base model's task average (AIME25, GPQA-Diamond, LiveCodeBench v6). | not measured yet |+| **IQ1_M** (1.75 bits) | 239 GB | KL 0.217, perplexity 3.80 (+21%); code +8.9%, agent work +10.2%; weaker on plain English prose. ISTA-DASLab's model card: keeps 98.7% of the base model's task average (AIME25, GPQA-Diamond, LiveCodeBench v6). | not measured yet || **IQ2_XXS** (2.00 bits) | 257 GB | KL 0.169, perplexity 3.64 (+16%) | 6 to 9 tokens/s on an RTX 3060, about 5 on an RTX 5070 || **IQ2_XS** (2.25 bits) | 274 GB | KL 0.110, perplexity 3.40 (+8%); code +3.5%, agent work +2.5% | about 11 tokens/s on one Radeon AI PRO R9700 || **IQ4_NL** (Engram table, not a third model) | 111 GB, already in all three downloads above | The 4-bit Engram lookup table the model reads from RAM or SSD. All three sets use this one file (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |diff --git a/README.pt-BR.md b/README.pt-BR.mdindex f4c312254..4b75f7683 100644--- a/README.pt-BR.md+++ b/README.pt-BR.md@@ -189,7 +189,7 @@ Todos os tamanhos, downloads e o que cabe em cada configuração: [docs/MODELS.m| Conjunto | Download | Qualidade em relação ao modelo original | Responde a || --- | ---: | --- | --- |-| **IQ1_M** (1,75 bits) | 239 GB | KL e perplexidade: ainda sendo medidos. Segundo o cartão do modelo da ISTA-DASLab, mantém 98,7% da média de tarefas do modelo base (AIME25, GPQA-Diamond, LiveCodeBench v6). | ainda não medida |+| **IQ1_M** (1,75 bits) | 239 GB | KL 0,217, perplexidade 3,80 (+21%); código +8,9%, tarefas de agente +10,2%; mais fraco em prosa comum em inglês. Segundo o cartão do modelo da ISTA-DASLab, mantém 98,7% da média de tarefas do modelo base (AIME25, GPQA-Diamond, LiveCodeBench v6). | ainda não medida || **IQ2_XXS** (2.00 bits) | 257 GB | KL 0,169, perplexidade 3,64 (+16%) | 6 a 9 tokens/s em uma RTX 3060, cerca de 5 em uma RTX 5070 || **IQ2_XS** (2.25 bits) | 274 GB | KL 0,110, perplexidade 3,40 (+8%); código +3,5%, tarefas de agente +2,5% | cerca de 11 tokens/s em uma Radeon AI PRO R9700 || **IQ4_NL** (tabela Engram, não é um terceiro modelo) | 111 GB, já incluídos nos três downloads acima | A tabela de consulta Engram de 4 bits que o modelo lê da RAM ou do SSD. Os três conjuntos usam este único arquivo (`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`). | n/a |diff --git a/README.zh-CN.md b/README.zh-CN.mdindex 2b4abbb9f..98e313c66 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -188,7 +188,7 @@ Gyro 可在 NVIDIA 和 AMD 显卡(AMD 为实验性支持)以及 Linux 上的| 版本 | 下载大小 | 与原模型相比的质量 | 生成速度 || --- | ---: | --- | --- |-| **IQ1_M**(1.75 位) | 239 GB | KL 和困惑度:正在测量。据 ISTA-DASLab 的模型卡,它保留了基础模型任务平均分的 98.7%(AIME25、GPQA-Diamond、LiveCodeBench v6)。 | 尚未测量 |+| **IQ1_M**(1.75 位) | 239 GB | KL 0.217,困惑度 3.80(+21%);代码 +8.9%,智能体任务 +10.2%;普通英文散文表现较弱。据 ISTA-DASLab 的模型卡,它保留了基础模型任务平均分的 98.7%(AIME25、GPQA-Diamond、LiveCodeBench v6)。 | 尚未测量 || **IQ2_XXS** (2.00 位) | 257 GB | KL 0.169,困惑度 3.64(+16%) | RTX 3060 上每秒 6 到 9 个 token,RTX 5070 上约 5 个 || **IQ2_XS** (2.25 位) | 274 GB | KL 0.110,困惑度 3.40(+8%);代码 +3.5%,智能体任务 +2.5% | 一块 Radeon AI PRO R9700 上每秒约 11 个 token || **IQ4_NL**(Engram 表,不是第三个模型) | 111 GB,已包含在上面三个下载中 | 模型从内存或固态硬盘读取的 4 位 Engram 查找表。三个版本共用这一个文件(`DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf`)。 | 不适用 |diff --git a/docs/DEEPSEEK41.md b/docs/DEEPSEEK41.mdindex a3a70cc13..da2073886 100644--- a/docs/DEEPSEEK41.md+++ b/docs/DEEPSEEK41.md@@ -29,7 +29,7 @@ with an out-of-memory message. Both are said before anything is downloaded.Setup offers three official sets (ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF, pinned): **IQ1_M, 1.75 bpw** (`ist-das-1.75bpw`, 121 GB, slightly lower quality; setup recommends it where 2.00 is tight in memory or disk), **IQ2_XXS, 2.00 bpw** (138 GB) and **IQ2_XS, 2.25 bpw** (155 GB). All use thesame 111 GB n-gram table (downloaded once into `models/deepseek-v4.1-flash-shared`) and the 8 GB DSpark drafter: 257 GB of disk for2.00, 274 GB for 2.25, 239 GB for 1.75, 533 GB for all three. Quality against the original model: perplexity +16% on 2.00, +8% on 2.25 (code and agent work-about +3.5% / +2.5% on 2.25). IQ1_M: our KL and perplexity are being measured; ISTA-DASLab's model card says it keeps 98.7% of the base model's task average (AIME25, GPQA-Diamond, LiveCodeBench v6).+about +3.5% / +2.5% on 2.25). IQ1_M: perplexity +21% (KL 0.217, top-1 82.5%); code +8.9%, agent work +10.2%, multilingual +14%, docs +24%, plain English prose +53%, so it is weaker on prose. ISTA-DASLab's model card says it keeps 98.7% of the base model's task average (AIME25, GPQA-Diamond, LiveCodeBench v6).| File | Size | What it is || --- | ---: | --- |diff --git a/docs/release-notes-0.2.4.md b/docs/release-notes-0.2.4.mdindex 5428c83a3..5c6887811 100644--- a/docs/release-notes-0.2.4.md+++ b/docs/release-notes-0.2.4.md@@ -10,7 +10,7 @@ Strata now runs DeepSeek V4.1-Flash on a PC with one 12 GB graphics card and 46| set | download | quality against the original model ||---|---|---|- | IQ1_M (1.75 bits) | 239 GB | being measured; ISTA-DASLab's model card: keeps 98.7% of the base model's task average (AIME25, GPQA-Diamond, LiveCodeBench v6) |+ | IQ1_M (1.75 bits) | 239 GB | perplexity +21%; code and agent text +8.9% and +10.2%; weaker on plain English prose. ISTA-DASLab's model card: keeps 98.7% of the base model's task average (AIME25, GPQA-Diamond, LiveCodeBench v6) || IQ2_XXS (2.00 bits) | 257 GB | perplexity +16%; code and agent text +8.5% and +5.9% || IQ2_XS (2.25 bits) | 274 GB | perplexity +7.7% to +8.3%; code and agent text +3.5% and +2.5% || IQ4_NL (Engram table, not a model to pick) | 111 GB, already in all three downloads above | The 4-bit Engram lookup table the model reads from RAM or SSD; all three sets use this one file |diff --git a/setup.py b/setup.pyindex 6849cf86a..1da4267e3 100644--- a/setup.py+++ b/setup.py@@ -764,7 +764,7 @@ MULTI_FILE_FAMILIES["deepseek41"]["models"] = {"IST-DAS-1.75bpw": {"label": "IST-DAS 1.75 bpw","about": "the smallest official version: 121 GB of model + 111 GB n-gram table + 8 GB drafter (239 GB of disk, 18 GB less than 2.00). Runs "- "where 2.00 is tight (a 12 GB card and 32-40 GB of RAM); slightly lower quality than 2.00",+ "where 2.00 is tight (a 12 GB card and 32-40 GB of RAM). Smallest: fine for code and agent work, noticeably weaker on plain prose; pick 2.00 if it fits","ram_gb": 34, "recipe": "ista", "subdir": None, "legacy_subdir": None, "profile": "profile-2.00.bin", "hf": hf(DSV41_ISTA_REPO),"shards": [DSV41_ISTA_SHARD.format(quant="IQ1_M", i=i) for i in range(1, 5)],"shared": [DSV41_ISTA_ENGRAM, DSV41_SHARD.format(i=1), DSV41_DSPARK]},diff --git a/strata_control/dsv41_fit.py b/strata_control/dsv41_fit.pyindex c8879cf0e..1d3d641c6 100644--- a/strata_control/dsv41_fit.py+++ b/strata_control/dsv41_fit.py@@ -62,10 +62,11 @@ ISTA_SETS = {"ppl": "16%", "ram_ok_gb": 40.0, "ram_great_gb": 160.0, "disk_total": 257.0,"quality": "about 16% above the original model's perplexity (the 2-bit price)"},# IQ1_M (1.75 bpw): 120,721,120,256 bytes of shards; experts = the shards minus the 6.16 GB of dense weights. RAM class from the experts' size against- # the 2.00 set's measured 40 GB (not measured on a computer yet); quality: placeholder until the H100 score is in.+ # the 2.00 set's measured 40 GB (not measured on a computer yet); quality: the H100's 4,995-position score (KL 0.217, ppl 3.80, +21%; code +8.9%, agent +10.2%, English prose +53%)."IST-DAS-1.75bpw": {"title": "IST-DAS 1.75 bpw", "set_bytes": 120_721_120_256, "experts_gb": 114.56, "profile": "profile-2.00.bin",- "ram_ok_gb": 34.0, "disk_total": 239.0,- "quality": "slightly lower than the 2.00 version (the smallest; a quality figure follows when it is measured)"},+ "ppl": "21%", "ram_ok_gb": 34.0, "disk_total": 239.0,+ "quality": "about 21% above the original model's perplexity: the smallest, fine for code and agent work, noticeably weaker "+ "on plain prose; pick 2.00 if it fits"},"IST-DAS-2.25bpw": {"title": "IST-DAS 2.25 bpw", "set_bytes": 155_225_824_256, "experts_gb": 149.06, "profile": "profile-2.25.bin","ppl": "8%", "ram_ok_gb": 44.0, "ram_great_gb": 176.0, "disk_total": 274.0,"quality": "about 8% above the original model's perplexity (code and agent work about 2.5-3.5% above), "