Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
serve: print the KV prefix-reuse diagnostic in mux mode too — [API] KV slot line existed only in the legacy \x02PROMPT path (#153)
ZacharyZcRd04d99e03990
diff --git a/c/glm.c b/c/glm.cindex cea69b77..1e10044c 100644--- a/c/glm.c+++ b/c/glm.c@@ -4070,6 +4070,7 @@ static int mux_submit(Model *m, Tok *T, ServeCtx *ctx, ServeReq *req, int nctx,kv_disk_truncate(m,sc->len); }int add=nt-sc->len;if(add>0) memcpy(sc->hist+sc->len,tmp+sc->len,(size_t)add*sizeof(int));+ fprintf(stderr,"[API] KV slot %d prefix %d/%d token, prefill %d\n",sub.slot,sc->len,nt,add);free(tmp);float *logit = add>0 ? step(m,sc->hist+sc->len,add,sc->len): step(m,sc->hist+sc->len-1,1,sc->len-1);