Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

serve: print the KV prefix-reuse diagnostic in mux mode too — [API] KV slot line existed only in the legacy \x02PROMPT path (#153)

ZacharyZcRd04d99e03990
diff --git a/c/glm.c b/c/glm.c
index cea69b77..1e10044c 100644
--- a/c/glm.c
+++ b/c/glm.c
@@ -4070,6 +4070,7 @@ static int mux_submit(Model *m, Tok *T, ServeCtx *ctx, ServeReq *req, int nctx,
kv_disk_truncate(m,sc->len); }
int add=nt-sc->len;
if(add>0) memcpy(sc->hist+sc->len,tmp+sc->len,(size_t)add*sizeof(int));
+ fprintf(stderr,"[API] KV slot %d prefix %d/%d token, prefill %d\n",sub.slot,sc->len,nt,add);
free(tmp);
float *logit = add>0 ? step(m,sc->hist+sc->len,add,sc->len)
: step(m,sc->hist+sc->len-1,1,sc->len-1);