Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

perf(omp): keep OpenMP worker team hot across tiny per-expert matmul regions (seed + re-exec once, fully overridable) (#77)

Rodolfo Hansen704ed49c16a0
diff --git a/c/glm.c b/c/glm.c
index a6b73bbe..87706f47 100644
--- a/c/glm.c
+++ b/c/glm.c
@@ -2494,8 +2494,37 @@ static void cap_for_ram(Model *m, double ram_gb, int ebits, int max_ctx){
}
int main(int argc, char **argv){
- /* i thread OMP non devono girare a vuoto mentre il main aspetta il disco */
- if(!getenv("OMP_WAIT_POLICY")) setenv("OMP_WAIT_POLICY","passive",1);
+ /* ---- Permanent OpenMP hot-thread tuning. The per-expert matmul regions are
+ * tiny and back-to-back; with the default passive wait policy libgomp parks
+ * the worker team between regions and the re-wake latency dominates. Keeping
+ * the threads hot (active spin) collapses that overhead β€” measured matmul
+ * time 66.9s -> 20.9s on the Zen5 build, with no change to numerical output.
+ *
+ * libgomp reads the OMP_ / GOMP_ vars in a CONSTRUCTOR that runs before
+ * main(), so setenv() here and continuing would be too late (verified:
+ * setenv-in-main is ignored by the already-initialised runtime). Instead, on
+ * first entry seed the winning defaults β€” respecting anything the user
+ * already set (overwrite=0) β€” then re-exec self once so a fresh libgomp
+ * constructor picks them up. The COLI_OMP_TUNED sentinel guards the exec so
+ * we re-exec at most once. Fully overridable: any explicit OMP_/GOMP_ env the
+ * user sets wins (overwrite=0), pre-setting COLI_OMP_TUNED=1 skips the
+ * re-exec entirely (runs with whatever policy the environment already has),
+ * and COLI_NO_OMP_TUNE=1 is a documented kill-switch that disables the whole
+ * re-exec + tuning path (distinct from the internal COLI_OMP_TUNED sentinel).
+ *
+ * Must remain the FIRST statement in main(): argv is passed verbatim to execv(). */
+ if(!getenv("COLI_OMP_TUNED") && !getenv("COLI_NO_OMP_TUNE")){
+ setenv("OMP_WAIT_POLICY","active",0); /* keep the team hot across the tiny per-expert matmul regions */
+ setenv("GOMP_SPINCOUNT","200000",0); /* spin briefly, then yield so long disk waits don't burn a core */
+ setenv("OMP_PROC_BIND","close",0); /* pack the team onto adjacent cores for cache locality */
+ setenv("OMP_DYNAMIC","FALSE",0); /* fixed team size: no per-region thread-count churn */
+ setenv("COLI_OMP_TUNED","1",1);
+#ifdef __linux__
+ fprintf(stderr,"[OMP] hot-thread tuning: re-exec once (COLI_NO_OMP_TUNE=1 to skip)\n");
+ execv("/proc/self/exe", argv); /* returns only on failure -> fall through and run untuned */
+ perror("[OMP] execv self-reexec failed, running untuned");
+#endif
+ }
const char *snap=getenv("SNAP"); if(!snap){fprintf(stderr,"SNAP=<dir>\n");return 1;}
g_nopack = getenv("NOPACK")?1:0;
g_drop = getenv("DROP")?1:0;