Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
perf(omp): keep OpenMP worker team hot across tiny per-expert matmul regions (seed + re-exec once, fully overridable) (#77)
Rodolfo Hansen704ed49c16a0
diff --git a/c/glm.c b/c/glm.cindex a6b73bbe..87706f47 100644--- a/c/glm.c+++ b/c/glm.c@@ -2494,8 +2494,37 @@ static void cap_for_ram(Model *m, double ram_gb, int ebits, int max_ctx){}int main(int argc, char **argv){- /* i thread OMP non devono girare a vuoto mentre il main aspetta il disco */- if(!getenv("OMP_WAIT_POLICY")) setenv("OMP_WAIT_POLICY","passive",1);+ /* ---- Permanent OpenMP hot-thread tuning. The per-expert matmul regions are+ * tiny and back-to-back; with the default passive wait policy libgomp parks+ * the worker team between regions and the re-wake latency dominates. Keeping+ * the threads hot (active spin) collapses that overhead β measured matmul+ * time 66.9s -> 20.9s on the Zen5 build, with no change to numerical output.+ *+ * libgomp reads the OMP_ / GOMP_ vars in a CONSTRUCTOR that runs before+ * main(), so setenv() here and continuing would be too late (verified:+ * setenv-in-main is ignored by the already-initialised runtime). Instead, on+ * first entry seed the winning defaults β respecting anything the user+ * already set (overwrite=0) β then re-exec self once so a fresh libgomp+ * constructor picks them up. The COLI_OMP_TUNED sentinel guards the exec so+ * we re-exec at most once. Fully overridable: any explicit OMP_/GOMP_ env the+ * user sets wins (overwrite=0), pre-setting COLI_OMP_TUNED=1 skips the+ * re-exec entirely (runs with whatever policy the environment already has),+ * and COLI_NO_OMP_TUNE=1 is a documented kill-switch that disables the whole+ * re-exec + tuning path (distinct from the internal COLI_OMP_TUNED sentinel).+ *+ * Must remain the FIRST statement in main(): argv is passed verbatim to execv(). */+ if(!getenv("COLI_OMP_TUNED") && !getenv("COLI_NO_OMP_TUNE")){+ setenv("OMP_WAIT_POLICY","active",0); /* keep the team hot across the tiny per-expert matmul regions */+ setenv("GOMP_SPINCOUNT","200000",0); /* spin briefly, then yield so long disk waits don't burn a core */+ setenv("OMP_PROC_BIND","close",0); /* pack the team onto adjacent cores for cache locality */+ setenv("OMP_DYNAMIC","FALSE",0); /* fixed team size: no per-region thread-count churn */+ setenv("COLI_OMP_TUNED","1",1);+#ifdef __linux__+ fprintf(stderr,"[OMP] hot-thread tuning: re-exec once (COLI_NO_OMP_TUNE=1 to skip)\n");+ execv("/proc/self/exe", argv); /* returns only on failure -> fall through and run untuned */+ perror("[OMP] execv self-reexec failed, running untuned");+#endif+ }const char *snap=getenv("SNAP"); if(!snap){fprintf(stderr,"SNAP=<dir>\n");return 1;}g_nopack = getenv("NOPACK")?1:0;g_drop = getenv("DROP")?1:0;