Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Commits on main

6dc16c5
d469c54
omp: reset CPU affinity before the tuning re-exec (fixes #471)
veerareddyvishal144 committed 4 months ago
4dfe581
quant: int4 IDOT at S=1 on AVX-512 VNNI (g_i4s=1), like ARM SDOT
veerareddyvishal144 committed 4 months ago
80cf3d4