Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

catalog: one plain line each for NVFP4 (best quality of the NVFP4 versions) and NVFP4-QAD (NVIDIA's FP4, trained for FP4, slightly faster on short prompts)

Niko1221c3e9480592d5
diff --git a/docs/MODELS.md b/docs/MODELS.md
index 9443dacbe..d07691e22 100644
--- a/docs/MODELS.md
+++ b/docs/MODELS.md
@@ -176,7 +176,7 @@ afterwards). Nothing of NVIDIA's weights is redistributed by Strata.
| Size | Download | Memory it needs | In plain words |
| --- | ---: | ---: | --- |
-| **NVFP4-QAD** | 102 GB (+110 GB of disk to prepare) | like NVFP4: ~80 GB of RAM keeps every expert in RAM | NVIDIA's own 4-bit model; RTX 50 cards have FP4 tensor cores, other NVIDIA cards run it at 8-bit precision |
+| **NVFP4-QAD** | 102 GB (+110 GB of disk to prepare) | like NVFP4: ~80 GB of RAM keeps every expert in RAM | NVIDIA's FP4, trained for FP4 - slightly faster on short prompts than NVFP4 (GPTQ is the best quality of the NVFP4 versions); RTX 50 cards have FP4 tensor cores, other NVIDIA cards run it at 8-bit precision |
NVIDIA cards only. Images are off for now. Its licenses are the NVIDIA Open Model Agreement and Qwen's Community License.
Technical details and measurements: [NVFP4.md](NVFP4.md#nvidias-fp4-local-checkpoint).
diff --git a/setup.py b/setup.py
index 11e299c8f..42d94b860 100644
--- a/setup.py
+++ b/setup.py
@@ -375,8 +375,8 @@ MODELS = {
"about 20% slower than Q2_0 on 12 GB cards",
"download_gb": 92.0, "ram_gb": 35.0, "arena_gb": 32.1, "families": ("gyro",), "mem_total": True,
"file": "Qwen3.8-Flash-Next-Gyro-M-TQ2_0.gguf", "shards": 1, "engine": GYRO_ENGINE, "vision": False},
- "NVFP4": {"about": "4-bit NVFP4 experts, GPTQ (sergqwer's strata-nvfp4 pack); RTX 50 has FP4 tensor cores, other "
- "NVIDIA cards run it at 8-bit precision", "download_gb": 128.0, "ram_gb": 80, "arena_gb": 68.0,
+ "NVFP4": {"about": "best quality of the NVFP4 versions (GPTQ, sergqwer's strata-nvfp4 pack); RTX 50 has FP4 tensor cores, "
+ "other NVIDIA cards run it at 8-bit precision", "download_gb": 128.0, "ram_gb": 80, "arena_gb": 68.0,
"families": ("nvfp4",), "nvidia_only": True, "vision": True, "repo": NVFP4_GPTQ_REPO,
"engine": NVFP4_ENGINE,
"license": "Qwen Community License 1.0 (Qwen3.8-Flash-Next's): https://huggingface.co/Qwen/Qwen3.8-Flash-Next"},
@@ -387,9 +387,9 @@ MODELS = {
"license": "NVIDIA Open Model License (Licensed by NVIDIA Corporation under the NVIDIA Open Model "
"License) with Qwen3.8-Flash-Next's Qwen Community License 1.0: "
"https://huggingface.co/nvidia/Qwen3.8-Flash-Next-NVFP4"},
- "NVFP4-QAD": {"about": "NVIDIA's own retrained 4-bit model (quantization-aware distillation, NVIDIA's Qwen3.8-Flash-Next-FP4-Local): "
- "a 102 GB download that this PC turns into a Strata pack (about 20-40 minutes and 110 GB more disk, "
- "once); RTX 50 cards have FP4 tensor cores, other NVIDIA cards run it at 8-bit precision. No images yet",
+ "NVFP4-QAD": {"about": "NVIDIA's FP4, trained for FP4 (NVIDIA's Qwen3.8-Flash-Next-FP4-Local) - slightly faster on short prompts; "
+ "a 102 GB download that this PC turns into a Strata pack once (about 20-40 minutes, 110 GB more disk); "
+ "RTX 50 cards have FP4 tensor cores, other NVIDIA cards run it at 8-bit precision. No images yet",
"download_gb": 102.2, "prepare_gb": 110.0, "ram_gb": 80, "arena_gb": 68.0,
"families": ("nvfp4",), "nvidia_only": True, "vision": False, "repo": NVFP4_FP4LOCAL_REPO,
"prepare": "fp4-local", "engine": FP4LOCAL_ENGINE,
diff --git a/tools/strata_mcp.py b/tools/strata_mcp.py
index ae54424b7..23848624d 100644
--- a/tools/strata_mcp.py
+++ b/tools/strata_mcp.py
@@ -91,7 +91,7 @@ FALLBACK_MODELS = {
"NVFP4-NVIDIA": {"about": "NVIDIA's own NVFP4 checkpoint (plain rounding), converted by sergqwer; the same size "
"and speed as NVFP4", "download_gb": 128.0, "ram_gb": 80, "arena_gb": 68.0,
"families": ("nvfp4",), "vision": True},
- "NVFP4-QAD": {"about": "NVIDIA's own retrained 4-bit model (quantization-aware distillation): a 102 GB download "
+ "NVFP4-QAD": {"about": "NVIDIA's FP4, trained for FP4 - slightly faster on short prompts; a 102 GB download "
"that this PC turns into a Strata pack once; no images yet",
"download_gb": 102.2, "ram_gb": 80, "arena_gb": 68.0, "families": ("nvfp4",), "vision": False},
}