Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Commits on main
integer-dot kernels (Q8_0-style activations, AVX2 maddubs): int8 always, int4 when S>=2 — measured 1.4-2.5x
741f49a JustVugg committed 4 months ago
README: honest matmul-bound predictions (GFLOP math), updated sampling defaults
34fb900 JustVugg committed 4 months ago
sampling defaults for int4 reality: temp 0.7, nucleus 0.90 (official 1.0/0.95 tuned for full precision; the int4 tail is quantization noise)
2dd6800 JustVugg committed 4 months ago
prefill progress: engine reports layer N/78 on stderr, coli spinner shows it live
1a8d2bc JustVugg committed 4 months ago
reserve 2.5 GB for the page cache in the RAM budget: starving it drops buffered reads 800->180 MB/s (measured)
3278ea7 JustVugg committed 4 months ago
coli: user box adapts to multi-line messages; status lines one per row without paths
b56d58d JustVugg committed 4 months ago
coli convert: one command does model + MTP head; README documents the full zero-to-chat path
b2855c0 JustVugg committed 4 months ago
learning cache + true sampling + DSA indexer extraction mode
3e88e37 JustVugg committed 4 months ago
serve: :piu resumes a truncated answer from live KV; ngen default 1024 (stops end turns now)
257c4d0 JustVugg committed 4 months ago
README: try-it-on-better-hardware guide + predictions + support section; drop stage-0 research scripts from repo
2000874 JustVugg committed 4 months ago
colibrì: pure-C GLM-5.2 (744B MoE) engine with disk-streamed experts
1ae22a6 JustVugg committed 4 months ago