Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Merge pull request #1630 from KhalidAlnujaidi/feat/olmoe-route-trace
Vincenzo Fornaro32d513dc8433
diff --cc c/olmoe.cindex 60fe72d0,1a05e5f6..92028443--- a/c/olmoe.c+++ b/c/olmoe.c@@@ -973,16 -950,17 +991,25 @@@ static void moe(Model *m, Layer *l, in}}free(logits); free(g); free(u); free(hh);+ /* Advance the trace call counter: once per moe() invocation, after all of+ * its rows are traced. rt_trace_end() is a no-op when no stream is open.+ *+ * Outside the row loop on purpose. A batch of S == 0 traces no rows and+ * must still consume a call id, or the ids stop being consecutive and+ * residency_sim.py rejects the trace outright ("trace lacks advancing GLM+ * call ids") rather than merging two forwards into one position space. GLM+ * and glm53 advance theirs the same way. */+ rt_trace_end();}+/* PROF phases (#1449): wall time in attention, in the MoE blocks (expert+ * loads included) and in the head, plus positions forwarded. The serve loop+ * reports per-turn deltas; expert_matmul_s is the MoE time minus the disk+ * seconds measured inside it, clamped at zero; forwards counts step() calls,+ * so tokens per forward reads 1.0 for this engine. Timing only. */+static double g_prof_attn_s = 0.0, g_prof_moe_s = 0.0, g_prof_head_s = 0.0;+static long long g_prof_forwards = 0;+static void layers_forward_range(Model *m, float *x, int S, int pos_base,int layer_begin, int layer_end,int allow_prefetch) {