Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
catalog: one plain line each for NVFP4 (best quality of the NVFP4 versions) and NVFP4-QAD (NVIDIA's FP4, trained for FP4, slightly faster on short prompts)
Niko1221c3e9480592d5
diff --git a/docs/MODELS.md b/docs/MODELS.mdindex 9443dacbe..d07691e22 100644--- a/docs/MODELS.md+++ b/docs/MODELS.md@@ -176,7 +176,7 @@ afterwards). Nothing of NVIDIA's weights is redistributed by Strata.| Size | Download | Memory it needs | In plain words || --- | ---: | ---: | --- |-| **NVFP4-QAD** | 102 GB (+110 GB of disk to prepare) | like NVFP4: ~80 GB of RAM keeps every expert in RAM | NVIDIA's own 4-bit model; RTX 50 cards have FP4 tensor cores, other NVIDIA cards run it at 8-bit precision |+| **NVFP4-QAD** | 102 GB (+110 GB of disk to prepare) | like NVFP4: ~80 GB of RAM keeps every expert in RAM | NVIDIA's FP4, trained for FP4 - slightly faster on short prompts than NVFP4 (GPTQ is the best quality of the NVFP4 versions); RTX 50 cards have FP4 tensor cores, other NVIDIA cards run it at 8-bit precision |NVIDIA cards only. Images are off for now. Its licenses are the NVIDIA Open Model Agreement and Qwen's Community License.Technical details and measurements: [NVFP4.md](NVFP4.md#nvidias-fp4-local-checkpoint).diff --git a/setup.py b/setup.pyindex 11e299c8f..42d94b860 100644--- a/setup.py+++ b/setup.py@@ -375,8 +375,8 @@ MODELS = {"about 20% slower than Q2_0 on 12 GB cards","download_gb": 92.0, "ram_gb": 35.0, "arena_gb": 32.1, "families": ("gyro",), "mem_total": True,"file": "Qwen3.8-Flash-Next-Gyro-M-TQ2_0.gguf", "shards": 1, "engine": GYRO_ENGINE, "vision": False},- "NVFP4": {"about": "4-bit NVFP4 experts, GPTQ (sergqwer's strata-nvfp4 pack); RTX 50 has FP4 tensor cores, other "- "NVIDIA cards run it at 8-bit precision", "download_gb": 128.0, "ram_gb": 80, "arena_gb": 68.0,+ "NVFP4": {"about": "best quality of the NVFP4 versions (GPTQ, sergqwer's strata-nvfp4 pack); RTX 50 has FP4 tensor cores, "+ "other NVIDIA cards run it at 8-bit precision", "download_gb": 128.0, "ram_gb": 80, "arena_gb": 68.0,"families": ("nvfp4",), "nvidia_only": True, "vision": True, "repo": NVFP4_GPTQ_REPO,"engine": NVFP4_ENGINE,"license": "Qwen Community License 1.0 (Qwen3.8-Flash-Next's): https://huggingface.co/Qwen/Qwen3.8-Flash-Next"},@@ -387,9 +387,9 @@ MODELS = {"license": "NVIDIA Open Model License (Licensed by NVIDIA Corporation under the NVIDIA Open Model ""License) with Qwen3.8-Flash-Next's Qwen Community License 1.0: ""https://huggingface.co/nvidia/Qwen3.8-Flash-Next-NVFP4"},- "NVFP4-QAD": {"about": "NVIDIA's own retrained 4-bit model (quantization-aware distillation, NVIDIA's Qwen3.8-Flash-Next-FP4-Local): "- "a 102 GB download that this PC turns into a Strata pack (about 20-40 minutes and 110 GB more disk, "- "once); RTX 50 cards have FP4 tensor cores, other NVIDIA cards run it at 8-bit precision. No images yet",+ "NVFP4-QAD": {"about": "NVIDIA's FP4, trained for FP4 (NVIDIA's Qwen3.8-Flash-Next-FP4-Local) - slightly faster on short prompts; "+ "a 102 GB download that this PC turns into a Strata pack once (about 20-40 minutes, 110 GB more disk); "+ "RTX 50 cards have FP4 tensor cores, other NVIDIA cards run it at 8-bit precision. No images yet","download_gb": 102.2, "prepare_gb": 110.0, "ram_gb": 80, "arena_gb": 68.0,"families": ("nvfp4",), "nvidia_only": True, "vision": False, "repo": NVFP4_FP4LOCAL_REPO,"prepare": "fp4-local", "engine": FP4LOCAL_ENGINE,diff --git a/tools/strata_mcp.py b/tools/strata_mcp.pyindex ae54424b7..23848624d 100644--- a/tools/strata_mcp.py+++ b/tools/strata_mcp.py@@ -91,7 +91,7 @@ FALLBACK_MODELS = {"NVFP4-NVIDIA": {"about": "NVIDIA's own NVFP4 checkpoint (plain rounding), converted by sergqwer; the same size ""and speed as NVFP4", "download_gb": 128.0, "ram_gb": 80, "arena_gb": 68.0,"families": ("nvfp4",), "vision": True},- "NVFP4-QAD": {"about": "NVIDIA's own retrained 4-bit model (quantization-aware distillation): a 102 GB download "+ "NVFP4-QAD": {"about": "NVIDIA's FP4, trained for FP4 - slightly faster on short prompts; a 102 GB download ""that this PC turns into a Strata pack once; no images yet","download_gb": 102.2, "ram_gb": 80, "arena_gb": 68.0, "families": ("nvfp4",), "vision": False},}