The 80B Qwen3-Coder-Next on a 16 GB Mac, about 13 tok/s, fully offline. A mixture-of-experts inference engine for Apple silicon.
Rust
Commits on main
Build on GitHub's own Mac on every push to main (attested, macOS 15 minimum set in the binary); README header shows its build badge; the VirusTotal scan moves out of the header (kept in the Integrity section)
66e8e9b 1picassoai committed 1 month ago
README: our promise (built for ourselves first, a small team in England, supported for as long as people use it, a full roadmap soon)
513ea93 1picassoai committed 1 month ago
README: the new banner (Coachwhip, Metal and Rust powered local AI engine) under a new file name, metadata stripped
69c6025 1picassoai committed 1 month ago
README front in a cleaner layout: headings for requirements, integrity, quick start, tools and measured numbers; the 44 and 8090 explained; the SSD line stated as design
1363492 1picassoai committed 1 month ago
README: badges for macOS 15+, Rust and the VirusTotal scan of the v0.5.3 download (0 of 62 engines flag it), with the public reports linked
aa4b77f 1picassoai committed 1 month ago
Say the minimum, not the test box: macOS 15 or later everywhere (models page, gpt-oss page, every release note), and the installer refuses macOS below 15 with a plain message
eb4536b 1picassoai committed 1 month ago
0.5.3: the resident 35B line merged with main (the 0.5.2 docs); version 0.5.3, release notes, models page row, README front, install.sh pinned to v0.5.3
2797337 1picassoai committed 1 month ago
0.5.1 release notes, the models page row for the resident 35B, install.sh pinned to v0.5.1
3a59f4c Picassoai committed 1 month ago
0.5.1: version; the lab "gu check" print removed; the out-of-memory advice no longer mentions --bank (wrong in resident mode)
abdf5ad Picassoai committed 1 month ago
Wrangler default pieces 512: 1024 ran the GPU out of memory on a crew tester prompt of 3.4K words; 512 runs the whole crew clean (reading 282-311, 5/8)
180881a Picassoai committed 1 month ago
Wrangler defaults: 1024-word prompt pieces with a sync per layer (reading 326 tok/s, text identical)
3595df2 Picassoai committed 1 month ago
Reading 221 -> 326 tok/s, text identical: own GPU arena beside Candle (grow only, never given back), pool held after load, linear-attention state cut out of the piece buffer (the per-layer leak), conv/silu/norms done once per piece before the walk, 1024-word pieces
5d73db6 Picassoai committed 1 month ago
0.5.4 candidate: Qwen sampling defaults (top-k 20, min-p 0), repeat penalty off, per-pile split off by default; long-chat fixes, GPU top-k word pick, faster prompt-reading linear attention
00130bf Picassoai committed 1 month ago
lab: attention reads long pieces in query slices (COACHWHIP_ATTN_SLICE 128), scratch handed back every 4 layers while reading (COACHWHIP_PIECE_SYNC), error detail logged; identical text
b3271e0 Picassoai committed 1 month ago
lab: rolling drops a quarter of the space at once (writing at 16.8K context 11.5 -> 33.1 tok/s); prompt pieces sized by the reserved space, not the chat length
5717e7c Picassoai committed 1 month ago
lab: rolling conversation space: past the reserved space the full-attention layers keep the opening 256 words and the newest, the oldest middle rolls out; the door never refuses a long chat; mask rebuilt for the kept words
4ec4607 Picassoai committed 1 month ago
lab: fixed conversation space (no growing), chat limit follows it, prompt pieces shrink with the chat, snapshot store for compaction reuse (COACHWHIP_SNAPSHOTS=1), read-once short passes; held, not gated
972cd7f Picassoai committed 1 month ago
0.5.3 candidate: the wrangler keeps every expert resident when a Qwen3.5/3.6 MoE file fits; fast path on by default; the door honours request temperature and top_p
84eef0c Picassoai committed 1 month ago
lab: resident mode (COACHWHIP_RESIDENT=1): every expert loaded once at exact size, GPU routes and addresses directly
4ae67e3 Picassoai committed 1 month ago
IQ port: read unsloth IQ2_XXS / IQ3_XXS / IQ4_XS experts (GGUF types 16, 18, 23) through ggml Metal kernels already in Candle
5afa486 Picassoai committed 1 month ago
merge 0.5.2 engine (e21d97f, Galahad-signed) into the docs branch for main
39da9fe 1picassoai committed 1 month ago
0.5.2 docs: the tester's numbers, installer pinned to v0.5.2
a27cfa1 1picassoai committed 1 month ago
0.5.2 docs: README front cut to install, size, time, tok/s; the crew app under docs/crew; 0.5.2 notes (tester's numbers to fill)
80e1a83 1picassoai committed 1 month ago
0.5.2: fused Gated DeltaNet word kernel, on by default (COACHWHIP_ATTN_KERNEL=0 restores the op-by-op path)
e21d97f Picassoai committed 1 month ago
README: front page in the agreed shape (requirements, three paste blocks, API table, how it works); duplicate sections below removed
6a11d1d 1picassoai committed 1 month ago
README: the demo GIF is the 80B coder on the squeezed file, 13 tok/s on the page
37039d2 1picassoai committed 1 month ago