The 80B Qwen3-Coder-Next on a 16 GB Mac, about 13 tok/s, fully offline. A mixture-of-experts inference engine for Apple silicon.
Rust
0.5.1 docs: coachwhip-models link filled in
1picassoai6fbc54eae1d3
diff --git a/README.md b/README.mdindex d1a3dc1..3375dd5 100644--- a/README.md+++ b/README.md@@ -17,7 +17,7 @@ curl -fsSL https://raw.githubusercontent.com/1picassoai/coachwhip/main/install.s**Then, two steps and you are talking to the coder:**-1. Download the ready-squeezed file, [`Qwen3-Coder-Next.Q2X.gguf`](HF_LINK) (29 GB, sha256 `cdc0584c1e4d30134cbb83f18e4c7607af90bd1145a64940c7a357b02a0216af`), and the model's [tokenizer.json](https://huggingface.co/Qwen/Qwen3-Coder-Next). Nothing to squeeze; it was squeezed once on our Mac with the same `coachwhip-prepare` you get in the install.+1. Download the ready-squeezed file, [`Qwen3-Coder-Next.Q2X.gguf`](https://huggingface.co/vinnj/coachwhip-models/resolve/main/Qwen3-Coder-Next.Q2X.gguf) (29 GB, sha256 `cdc0584c1e4d30134cbb83f18e4c7607af90bd1145a64940c7a357b02a0216af`), and the model's [tokenizer.json](https://huggingface.co/Qwen/Qwen3-Coder-Next). Nothing to squeeze; it was squeezed once on our Mac with the same `coachwhip-prepare` you get in the install.2. Start it. The chat opens in your browser:```sh~/coachwhip/coachwhip.sh ~/Downloads/Qwen3-Coder-Next.Q2X.gguf ~/Downloads/tokenizer.json 109 8090 --parallel --flash-reads --stand-insdiff --git a/docs/releases/0.5.1.md b/docs/releases/0.5.1.mdindex 61ecdce..0f38777 100644--- a/docs/releases/0.5.1.md+++ b/docs/releases/0.5.1.md@@ -5,7 +5,7 @@ Mac mini M4, 16 GB, measured; your Mac and your prompt will move the numbers. Th## The 80B coder, ready-squeezed- **Qwen3-Coder-Next, 80 billion parameters, 3 billion active, on a 16 GB Mac at about 13 tok/s.** The machine cannot hold the 38 GB file; Coachwhip streams the experts it needs and keeps the rest asleep.-- **Download the squeezed file instead of squeezing it yourself.** `Qwen3-Coder-Next.Q2X.gguf`, 29,335,336,000 bytes, sha256 `cdc0584c1e4d30134cbb83f18e4c7607af90bd1145a64940c7a357b02a0216af`, made by the shipped `coachwhip-prepare` from MaziyarPanahi's Q3_K_M (gate and up experts to Q2_K, down to Q3_K; everything else untouched). Link: (Hugging Face, under the author's account; filled in at publish). The installer checks the hash before it runs.+- **Download the squeezed file instead of squeezing it yourself.** `Qwen3-Coder-Next.Q2X.gguf`, 29,335,336,000 bytes, sha256 `cdc0584c1e4d30134cbb83f18e4c7607af90bd1145a64940c7a357b02a0216af`, made by the shipped `coachwhip-prepare` from MaziyarPanahi's Q3_K_M (gate and up experts to Q2_K, down to Q3_K; everything else untouched). Download: [huggingface.co/vinnj/coachwhip-models](https://huggingface.co/vinnj/coachwhip-models), the one place for every squeezed file we host; each file's sha256 and source are listed there. Check the hash after the download: `shasum -a 256 ~/Downloads/Qwen3-Coder-Next.Q2X.gguf`.- **One line to start it**, bank 109, fast mode on: `~/coachwhip/coachwhip.sh ~/Downloads/Qwen3-Coder-Next.Q2X.gguf ~/Downloads/tokenizer.json 109 8090 --parallel --flash-reads --stand-ins`## Measured, same prompt, same Mac