The 80B Qwen3-Coder-Next on a 16 GB Mac, about 13 tok/s, fully offline. A mixture-of-experts inference engine for Apple silicon.
Rust
docs: README demo GIF is gpt-oss-120b on the 0.5.0 release build
1picassoaifa42a2f75fc7
diff --git a/README.md b/README.mdindex 4ecc31b..c621f3d 100644--- a/README.md+++ b/README.md@@ -11,7 +11,7 @@Coachwhip is one Rust binary. It takes a model file in GGUF format, keeps the part of the model every word needs on the GPU, and streams the rest from the SSD as each word asks for it. A 120-billion-parameter model runs on a 16 GB Mac. Which models, how fast, and how to set each one up: [docs/MODELS.md](docs/MODELS.md).-<p align="center"><img src="docs/coachwhip-demo.gif" alt="A 122-billion-parameter model writing TypeScript in the Coachwhip chat on a 16 GB Mac mini, in real time" width="720"></p>+<p align="center"><img src="docs/coachwhip-demo.gif" alt="gpt-oss-120b thinking, then writing TypeScript in the Coachwhip chat on a 16 GB Mac mini, in real time" width="720"></p>## Why we built itdiff --git a/docs/coachwhip-demo.gif b/docs/coachwhip-demo.gifindex 3adc5fc..e0c997c 100644Binary files a/docs/coachwhip-demo.gif and b/docs/coachwhip-demo.gif differ