Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Do not fork an OpenMP team to run one iteration
ZacharyZcR230b75fd82e6
diff --git a/c/colibri.c b/c/colibri.cindex aa8bc12c..29553a0a 100644--- a/c/colibri.c+++ b/c/colibri.c@@ -3370,7 +3370,14 @@ static void attention_rows(Model *m, Layer *l, int layer, float *x, int S, int pm->dsa_sel=malloc((size_t)m->dsa_scap*sizeof(int));m->dsa_nsel=malloc((size_t)S*sizeof(int));}- #pragma omp parallel for schedule(dynamic,1)+ /* if(S>1): at decode S is 1, so this region has exactly ONE iteration --+ * and OpenMP still forks and joins the whole team to run it. That is+ * pure overhead on the hottest path there is: once per layer, per+ * token, whether or not the body does any work (both early `continue`+ * branches below are taken on short contexts, and the fork happens+ * anyway). The clause makes the single-iteration case run inline;+ * S>1 prefill is untouched, and the results are identical either way. */+ #pragma omp parallel for schedule(dynamic,1) if(S > 1)for(int s=0;s<S;s++){KVState *ks=kvs?kvs[s]:m->kv;int pos=positions?positions[s]:pos_base+s, nk=pos+1;