The 80B Qwen3-Coder-Next on a 16 GB Mac, about 13 tok/s, fully offline. A mixture-of-experts inference engine for Apple silicon.
Rust
Commits on main
Workspace: split into engine, server and cli crates; tests move beside their crate
72b97c8 1picassoai committed 1 month ago
docs: per-model measurements (speed, 8-task quality check) and the script to rerun them
413f3b6 1picassoai committed 1 month ago
Coachwhip v0.1.0: an 80B model, running comfortably on a 16 GB Mac
8bfefc2 1picassoai committed 1 month ago