Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
glm: COLI_VK_EXPERTS=0 disables the VK expert tier (experts stay on the parallel CPU loop)
Steve Markgraf936d13e081fa
diff --git a/c/colibri.c b/c/colibri.cindex 6f1d0c81..712296d5 100644--- a/c/colibri.c+++ b/c/colibri.c@@ -3297,7 +3297,8 @@ static void moe(Model *m, Layer *l, int layer, float *x, int S, float *out, int#endifint vk_active = 0; (void)vk_active;#ifdef COLI_VULKAN- vk_active = g_vulkan && !omp_in_parallel() && S<=4;+ vk_active = g_vulkan && g_vk_budget>0 && !omp_in_parallel() && S<=4; /* budget 0 = tier off,+ * experts stay on the normal (parallel) CPU loop */float *vk_xh = vk_active?falloc((int64_t)S*K*D):NULL;float *vk_yh = vk_active?falloc((int64_t)S*K*D):NULL;#endif