Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Reclaim detached expert arena pages

ZacharyZcR66211fa9a7af
diff --git a/c/colibri.c b/c/colibri.c
index 6cebfe0e..909508c6 100644
--- a/c/colibri.c
+++ b/c/colibri.c
@@ -2282,7 +2282,19 @@ static void expert_host_release(Model *m, ESlot *s){
* fixed at the original expert_load site. fslab is plain malloc/falloc
* on the CPU path, so its free() stays plain (Metal path frees it before
* re-alloc and never reaches here with an aligned fslab on _WIN32). */
- if(s->aslab){ s->slab=NULL; s->fslab=NULL; } /* arena slice (#419): detach, keep caps, never free */
+ if(s->aslab){
+#ifdef __linux__
+ /* The arena owns the virtual address, not the resident pages. A GPU
+ * slot no longer needs its host copy after the synchronous H2D upload.
+ * Keeping those anonymous pages resident made every live REPIN pass
+ * grow RSS by one swapped batch (~300 MB at 16 experts). DONTNEED
+ * preserves the reusable slice while returning its pages to Linux;
+ * expert_host_ensure reloads them before the slot can run on CPU. */
+ madvise(s->slab,(size_t)s->slab_cap,MADV_DONTNEED);
+ madvise(s->fslab,(size_t)s->fslab_cap*sizeof(float),MADV_DONTNEED);
+#endif
+ s->slab=NULL; s->fslab=NULL; /* detach, keep caps/arena ownership */
+ }
else { compat_aligned_free(s->slab); free(s->fslab); s->slab=NULL; s->fslab=NULL; s->slab_cap=s->fslab_cap=0; }
QT *q[3]={&s->g,&s->u,&s->d};
for(int k=0;k<3;k++){ q[k]->qf=NULL; q[k]->q8=NULL; q[k]->q4=NULL; q[k]->s=NULL; }