Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

docs+banner: DeepSeek V4 Flash REAP-150B is supported, and is not 284B

JustVugg1ac934bc3da3
diff --git a/README.md b/README.md
index 126c5c6e..0f31e2e9 100644
--- a/README.md
+++ b/README.md
@@ -409,7 +409,7 @@ the model's `config.json`):
> | **GLM-5.3-Flash** | ~195 GB converted | 25 GB (12 GB weights at int4 + expert cache) | not needed |
> | **Inkling** | ~469 GB | 25 GB with the int4 dense container, ~120 GB without | not needed |
> | **Kimi K3** | ~1.6 TB | 32 GB+ | not needed |
-> | **DeepSeek V4 Flash** | ~167 GB | 16 GB min, 32 GB comfortable | optional; any NVIDIA card from the GTX 10 series up (Pascal/Turing via `CUDA_ARCH=portable-pre-ampere NO_TC=1`, best on RTX 50) makes prefill 5-10x and decode ~2.5x faster |
+> | **DeepSeek V4 Flash** | ~167 GB (REAP 150B: ~85 GB) | 16 GB min, 32 GB comfortable | optional; any NVIDIA card from the GTX 10 series up (Pascal/Turing via `CUDA_ARCH=portable-pre-ampere NO_TC=1`, best on RTX 50) makes prefill 5-10x and decode ~2.5x faster |
> | **Qwen3.8-Flash-Next** | ~185.5 GB (official FP8 checkpoint) | 16 GB min, 24 GB comfortable at the default context | not supported; CPU only |
> | **Qwen3.6-35B-A3B** | ~20 GB (int4-gs64 container) | 24 GB (needs full RAM residency) | optional; the CUDA VRAM expert tier measured **1.44 -> 10.05 tok/s (7.0x)** on two 8 GB cards, output bit-identical to CPU |
>
@@ -423,7 +423,7 @@ the model's `config.json`):
| **Inkling** (Thinking Machines) | 975B / 41B | [`nbeerbower/Inkling-colibri-int4`](https://huggingface.co/nbeerbower/Inkling-colibri-int4) (469 GB) | `make -C c inkling` | [inkling.md](docs/inkling.md) |
| **GLM-5.3-Flash** (Z.ai) | 321B / 40B | [`zai-org/GLM-5.3-Flash`](https://huggingface.co/zai-org/GLM-5.3-Flash) β€” converted to **int4-gs64** routed experts, dense stays BF16 and the precision is a load-time choice; vision included | `make -C c glm53` | [glm53-flash.md](docs/glm53-flash.md) |
| **Kimi K3** (Moonshot) | 2.8T / 104B | [`moonshotai/Kimi-K3`](https://huggingface.co/moonshotai/Kimi-K3) β€” original checkpoint, routed experts stay **native MXFP4** | `make -C c kimi_k3` | [kimi_k3.md](docs/kimi_k3.md) |
-| **DeepSeek V4 Flash** | 284B / 13B | official sharded checkpoint β€” routed experts stay **native fp4**, dense stays fp8-e4m3 | `make -C c deepseek-v4` | [deepseek-v4.md](docs/deepseek-v4.md) |
+| **DeepSeek V4 Flash** | 284B / 13B | official sharded checkpoint β€” routed experts stay **native fp4**, dense stays fp8-e4m3; the **REAP-pruned 150B** ([`puwaer/DeepSeek-V4-Flash-0731-reap-150b`](https://huggingface.co/puwaer/DeepSeek-V4-Flash-0731-reap-150b), 85 GB, 132 of 256 experts) loads with the same engine and no conversion | `make -C c deepseek-v4` | [deepseek-v4.md](docs/deepseek-v4.md) |
| **Qwen3.8-Flash-Next** (Alibaba) | 125B + 51B n-gram / 6B | [`Qwen/Qwen3.8-Flash-Next-FP8`](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8) β€” original checkpoint; PLE stays pageable and experts stay **native block-FP8** | `make -C c qwen38` (CPU only) | [qwen38.md](docs/qwen38.md) |
| **Qwen3.6** (Alibaba) | 35B / 3B | [`Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64`](https://huggingface.co/Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64) (~20 GB, **recommended**) β€” hybrid Gated Attention + Gated DeltaNet | `make -C c qwen36` (`CUDA=1` for the VRAM expert tier) | [qwen36.md](docs/qwen36.md) |
| **OLMoE** (AI2) | 7B / 1B | converted with `c/tools/convert_olmoe_merged.py` β€” **int8** container, ~7 GB | `make -C c olmoe` | β€” |
diff --git a/c/coli b/c/coli
index ab9ee705..5fe9bc6d 100755
--- a/c/coli
+++ b/c/coli
@@ -193,6 +193,13 @@ def model_banner_line(model):
try:
family = resolve_model(model).descriptor
name, scale = family.display_name, family.display_scale
+ # display_scale e' la taglia del checkpoint di riferimento. Un REAP ha
+ # lo stesso model_type e meno esperti: "284B" su un 150B e' #1367
+ # di nuovo. Quando la geometria non e' quella del riferimento si
+ # lascia parlare la geometria, che e' misurata e non puo' mentire.
+ experts = cfg.get("n_routed_experts")
+ if family.reference_experts and experts and experts != family.reference_experts:
+ scale = ""
except (FamilyConfigError, UnknownFamilyError):
# Unknown checkpoint: say so with its own words rather than guess.
name = kind or "unknown model"
diff --git a/c/family_registry.py b/c/family_registry.py
index b86d36d8..eeddf2a8 100644
--- a/c/family_registry.py
+++ b/c/family_registry.py
@@ -109,6 +109,14 @@ class FamilyDescriptor:
converter: str = ""
converter_accepts: tuple = ()
converter_mtp_pass: bool = False
+ # Gli esperti per layer del checkpoint con cui display_scale e' stato
+ # scritto. display_scale e' un numero del checkpoint di riferimento, non
+ # della famiglia: un checkpoint potato (REAP) ha la stessa architettura e
+ # lo stesso model_type ma meno esperti, e annunciarlo con la taglia del
+ # riferimento e' lo stesso errore di #1367 -- con la differenza che qui
+ # la geometria lo rende visibile. 0 = nessun riferimento dichiarato, il
+ # banner stampa display_scale come sempre.
+ reference_experts: int = 0
@dataclass(frozen=True, slots=True)
@@ -1159,6 +1167,10 @@ FAMILIES = (
model_types=("deepseek_v4",),
display_name="DeepSeek V4 Flash",
display_scale="284B",
+ # deepseek-ai/DeepSeek-V4-Flash-0731 config.json: 43 layer, 256
+ # esperti, top-k 6. Il REAP a 150B (#1310) ne ha 132 sugli stessi 43
+ # layer: e' quello che fa scattare la geometria misurata nel banner.
+ reference_experts=256,
engine_artifact="deepseek_v4",
engine_aliases=(),
engine_group="deepseek_v4",
diff --git a/c/tests/test_cli_output.py b/c/tests/test_cli_output.py
index 84a1cda3..603e979c 100644
--- a/c/tests/test_cli_output.py
+++ b/c/tests/test_cli_output.py
@@ -261,6 +261,24 @@ class BannerModelLineTest(unittest.TestCase):
self.assertIn("unknown model", line)
self.assertNotIn("GLM-5.2", line)
+ def test_a_pruned_checkpoint_is_not_announced_with_the_reference_size(self):
+ """#1310 made the REAP-pruned DeepSeek V4 Flash load: same model_type,
+ same 43 layers, 132 routed experts instead of 256, 150B instead of
+ 284B. The family's display_scale is the reference checkpoint's number,
+ so printing it here is #1367 again -- except that this time the config
+ can tell the two apart, so the banner must."""
+ reap = self.line({"model_type": "deepseek_v4", "num_hidden_layers": 43,
+ "n_routed_experts": 132})
+ self.assertIn("DeepSeek V4 Flash", reap)
+ self.assertNotIn("284B", reap, "a 150B checkpoint announced as 284B")
+ self.assertIn("43L x 132E", reap)
+
+ def test_the_reference_checkpoint_keeps_its_declared_size(self):
+ official = self.line({"model_type": "deepseek_v4", "num_hidden_layers": 43,
+ "n_routed_experts": 256})
+ self.assertIn("284B", official)
+ self.assertNotIn("43L x 256E", official)
+
def test_no_model_keeps_the_generic_tagline(self):
"""What matters is that a tagline comes back and that it names no
model. It used to assert the substring "GLM-5.2", which pinned the
diff --git a/docs/deepseek-v4.md b/docs/deepseek-v4.md
index f870988c..684216b9 100644
--- a/docs/deepseek-v4.md
+++ b/docs/deepseek-v4.md
@@ -54,6 +54,21 @@ hf download deepseek-ai/DeepSeek-V4-Flash-0731 \
--local-dir /path/to/DeepSeek-V4-Flash
```
+The REAP-pruned 150B variant loads with the same engine and needs no
+conversion either. It keeps the official FP4 expert layout and the official
+dense FP8, and drops 124 of the 256 routed experts per layer:
+
+```bash
+hf download puwaer/DeepSeek-V4-Flash-0731-reap-150b \
+ --local-dir /path/to/DeepSeek-V4-Flash-reap-150b
+```
+
+85 GB on disk instead of 167. Its shards pack an expert's three weights and
+three scales non-contiguously, sometimes across shards; the engine reads such
+experts per matrix and takes the original contiguous fast path everywhere else
+(#1310). The banner reports it by its measured geometry, `43L x 132E`, rather
+than the 284B of the official checkpoint, because that number is not its.
+
A download can finish with a truncated shard even when the client reports
success. If `st.h` rejects a shard as out of bounds, compare every local shard
size with the Hugging Face repository before treating it as an engine failure.