Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
feat(prefetch): add opt-in lookahead-3 prefetching option for PILOT=3
Egon Ruiterb3fdb145f8c8
diff --git a/c/olmoe.c b/c/olmoe.cindex 72967dfb..f532a6dc 100644--- a/c/olmoe.c+++ b/c/olmoe.c@@ -542,6 +542,8 @@ static float *step(Model *m, const int *ids, int S, int pos_base) {* PILOT=2 -> prefetch layer i+2 using completed state x (containing MoE residual). */if (g_pilot >= 2 && S <= 8 && i + 2 < c->n_layers)pilot_prefetch(m, i + 2, x, S);+ if (g_pilot >= 3 && S <= 8 && i + 3 < c->n_layers)+ pilot_prefetch(m, i + 3, x, S);}/* IMPROVEMENT 2: count tokens; trigger hot pinning after warmup */m->token_count++; m->freq_token_count++;