Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
fix(qwen36): skip RAM-derived cap for dense checkpoints (num_experts 0)
Jaapjan Tinbergena67674f7fe88
diff --git a/c/qwen36.c b/c/qwen36.cindex 80673482..1313c051 100644--- a/c/qwen36.c+++ b/c/qwen36.c@@ -2242,7 +2242,15 @@ static void model_init_range(Model *m, const char *snap, int cap, int bits,if (quantize_dense)fprintf(stderr, "[dense-i8] %d matrices %s during load, %.1f GB f32 freed\n", qcount,dense_bits() == 16 ? "kept in f16" : "quantized", qfreed/1073741824.0);- if (cap <= 0) {+ if (cap <= 0 && c->n_experts <= 0) {+ /* Dense checkpoint (#1757): nothing is ever routed, so the per-layer+ * expert cache is never touched by moe()/expert_get(). Sizing it+ * from RAM would be meaningless -- n_experts==0 has nothing to clamp+ * qwen36_cap_for_ram's derived value against, so an unclamped RAM+ * budget could otherwise calloc an absurd slot count for a cache+ * that will sit empty. cap=1 is a harmless placeholder. */+ cap = 1;+ } else if (cap <= 0) {/* cap<=0 sentinel: derive from host RAM, same "0 = auto" convention as* colibri.c/olmoe.c. rss_gb() here reflects all dense weights resident* (this loop just finished) but not yet DN_rec/DN_conv/m->cache