The 80B Qwen3-Coder-Next on a 16 GB Mac, about 13 tok/s, fully offline. A mixture-of-experts inference engine for Apple silicon.
Rust
docs: squeeze takes 5 to 9 minutes, needs about 105 GB free while both files exist, and is deterministic (tester, 9 Oct)
Merlinbd964c59f58c
diff --git a/README.md b/README.mdindex f46c78d..701c381 100644--- a/README.md+++ b/README.md@@ -116,7 +116,7 @@ The Q3_K_M file writes faster than Q4_K_M on a 16 GB Mac, and the two scored theQwen3.5-122B-A10B is the biggest model Coachwhip runs on 16 GB, and it needs one extra step: its experts are too big to stream fast as they come, so Coachwhip squeezes the file once, on your Mac.1. Download the publisher's [Q3_K_M GGUF](https://huggingface.co/mradermacher/Qwen3.5-122B-A10B-GGUF) (59 GB) and the model's [tokenizer.json](https://huggingface.co/Qwen/Qwen3.5-122B-A10B).-2. Squeeze it. About five minutes on an M4; it writes a second file beside the first and leaves the download untouched:+2. Squeeze it. About 5 to 9 minutes on an M4. It writes a second file beside the first and leaves the download untouched, so you need about **105 GB free** before you start: the 59 GB download and the 46 GB squeezed file both exist until you delete the original:```sh~/coachwhip/target/release/coachwhip-prepare ~/Downloads/Qwen3.5-122B-A10B.Q3_K_M.gguf ~/Downloads/Qwen3.5-122B-A10B.Q2X.ggufdiff --git a/docs/MODELS.md b/docs/MODELS.mdindex b2a5733..a6529e2 100644--- a/docs/MODELS.md+++ b/docs/MODELS.md@@ -40,7 +40,7 @@ Measured 6 Oct 2026: one person, one run per bank, on the same Mac with a backgr## Qwen3.5-122B-A10B-The biggest model Coachwhip runs on 16 GB (architecture `qwen35moe`): 48 layers, 256 experts of which 8 are used per word. It runs from a **squeezed file** that `coachwhip-prepare` makes on your Mac from the publisher's Q3_K_M (59 GB): each expert's gate and up matrices to Q2_K on the GPU, its down matrix to Q3_K, the rest untouched; 45 GB written in 5 min 7 s on an M4. See the README, The 122B.+The biggest model Coachwhip runs on 16 GB (architecture `qwen35moe`): 48 layers, 256 experts of which 8 are used per word. It runs from a **squeezed file** that `coachwhip-prepare` makes on your Mac from the publisher's Q3_K_M (59 GB): each expert's gate and up matrices to Q2_K on the GPU, its down matrix to Q3_K, the rest untouched; 45 GB written in 5 to 9 minutes on an M4, and about 105 GB free needed while both files exist. See the README, The 122B.Fast mode (`--experts 4 --parallel`), Mac mini M4, 16 GB, bank 40:diff --git a/docs/releases/0.4.0.md b/docs/releases/0.4.0.mdindex a8ed78f..b99a7fe 100644--- a/docs/releases/0.4.0.md+++ b/docs/releases/0.4.0.md@@ -4,7 +4,7 @@ Mac mini M4, 16 GB, measured; your Mac and your prompt will move the numbers. Me## New model-- **Qwen3.5-122B-A10B** (`qwen35moe`), the biggest model Coachwhip runs on 16 GB, after a five-minute squeeze on your own Mac: `coachwhip-prepare` takes the publisher's Q3_K_M file and writes a second file beside it with each expert's gate and up matrices at Q2_K (done on the GPU) and its down matrix at Q3_K; everything else is copied as it is. 45 GB written in 5 minutes on an M4. Nothing is downloaded from us.+- **Qwen3.5-122B-A10B** (`qwen35moe`), the biggest model Coachwhip runs on 16 GB, after a squeeze of 5 to 9 minutes on your own Mac: `coachwhip-prepare` takes the publisher's Q3_K_M file and writes a second file beside it with each expert's gate and up matrices at Q2_K (done on the GPU) and its down matrix at Q3_K; everything else is copied as it is. 45 GB written in 5 to 9 minutes on an M4; about 105 GB free is needed while the download and the squeezed file both exist. The squeeze is deterministic: the same download gives the same bytes on every Mac. Nothing is downloaded from us.- **Two modes.** Exact mode uses the model's own 8 experts per word. Fast mode, `--experts 4 --parallel`, asks the router for its top 4 and lets the GPU run ahead of the CPU: about twice the speed, and the answers can differ from the full model's. Fast mode is a choice, never the default; the README says so where the command is.- Numbers on the squeezed file at bank 32, in [docs/MODELS.md](../MODELS.md): the tester's run.