Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Merge pull request #1630 from KhalidAlnujaidi/feat/olmoe-route-trace

Vincenzo Fornaro32d513dc8433
diff --cc c/olmoe.c
index 60fe72d0,1a05e5f6..92028443
--- a/c/olmoe.c
+++ b/c/olmoe.c
@@@ -973,16 -950,17 +991,25 @@@ static void moe(Model *m, Layer *l, in
}
}
free(logits); free(g); free(u); free(hh);
+ /* Advance the trace call counter: once per moe() invocation, after all of
+ * its rows are traced. rt_trace_end() is a no-op when no stream is open.
+ *
+ * Outside the row loop on purpose. A batch of S == 0 traces no rows and
+ * must still consume a call id, or the ids stop being consecutive and
+ * residency_sim.py rejects the trace outright ("trace lacks advancing GLM
+ * call ids") rather than merging two forwards into one position space. GLM
+ * and glm53 advance theirs the same way. */
+ rt_trace_end();
}
+/* PROF phases (#1449): wall time in attention, in the MoE blocks (expert
+ * loads included) and in the head, plus positions forwarded. The serve loop
+ * reports per-turn deltas; expert_matmul_s is the MoE time minus the disk
+ * seconds measured inside it, clamped at zero; forwards counts step() calls,
+ * so tokens per forward reads 1.0 for this engine. Timing only. */
+static double g_prof_attn_s = 0.0, g_prof_moe_s = 0.0, g_prof_head_s = 0.0;
+static long long g_prof_forwards = 0;
+
static void layers_forward_range(Model *m, float *x, int S, int pos_base,
int layer_begin, int layer_end,
int allow_prefetch) {