Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
E5: hazard-audit round-2 fixes (re-register set hygiene, SUMMARY mutex ref)
monotophic8efa9ec6c3d1
diff --git a/SUMMARY.md b/SUMMARY.mdindex 599b8ca0..d196eb1a 100644--- a/SUMMARY.md+++ b/SUMMARY.md@@ -50,7 +50,10 @@ the `COLI_METAL_RESSET` `getenv` branch in `coli_metal_init`, so `resset_add`/`rdirty flag), calls `[rs addAllocation:b]` and sets `g_resset_dirty` β **it does notcommit**. No Metal call ever runs under `g_slab_mtx` (validator round-1 fix; E4's auditround 2 identified mutex-over-live-Metal-call as the leading suspect for its +12s- expert-disk regression).+ expert-disk regression). Re-registering a live base (no in-tree caller does today) drops+ the replaced wrapper from the set via `resset_remove(old)` before adding the new one+ (hazard-audit defensive fix β the set would otherwise retain the old buffer, and its+ pages' residency, forever), keeping set membership an exact mirror of `g_slabs`.- **`coli_metal_unregister`**: erases the `g_slabs` entry under `g_slab_mtx` (stashing thebuffer), then calls `resset_remove(b)` **outside `g_slab_mtx`**, before returning.`resset_remove` (under `g_resset_mtx`) calls `[rs removeAllocation:b]` **and commits@@ -58,7 +61,8 @@ the `COLI_METAL_RESSET` `getenv` branch in `coli_metal_init`, so `resset_add`/`rfunction returns. See UNCERTAINTIES for why this asymmetry is deliberate.- **`moe_submit`** (the one function whose `use` list β resolved expert weight/scale slabs βscales with LRU cache size): calls `resset_flush()` at the top (commits any pending adds- from `resset_add`, under `g_slab_mtx`), then, if `g_resset_enabled`, **skips** the+ from `resset_add`, under `g_resset_mtx` β it never touches the slab lock), then, if+ `g_resset_enabled`, **skips** the`for(auto&b:use) [e useResource:b usage:MTLResourceUsageRead];` loop entirely β residencyis already guaranteed by the queue-attached set. Every other `useResource:` call site in thefile (`bind_gemv`'s weight/scale buffers, `coli_metal_attn_decode`/`coli_metal_layer_decode`'sdiff --git a/c/backend_metal.mm b/c/backend_metal.mmindex be7dae8d..6a8e3a07 100644--- a/c/backend_metal.mm+++ b/c/backend_metal.mm@@ -399,13 +399,20 @@ extern "C" void coli_metal_register(void *base, size_t len) {id<MTLBuffer> b = [g_dev newBufferWithBytesNoCopy:base length:lenoptions:g_res_opts deallocator:nil];if (!b) return;+ id<MTLBuffer> old = nil; // E5: replaced wrapper on re-register of a live base (defensive){std::lock_guard<std::mutex> lk(g_slab_mtx); // called from parallel expert_load threadsbool found = false;- for (auto &s : g_slabs) if (s.base == base) { s.len = len; s.buf = b; found = true; break; }+ for (auto &s : g_slabs) if (s.base == base) { old = s.buf; s.len = len; s.buf = b; found = true; break; }if (!found) g_slabs.push_back({base, len, b});}- resset_add(b); // E5: outside g_slab_mtx (no Metal call under the slab lock); still before return+ // E5, outside g_slab_mtx (no Metal call under the slab lock), before returning. Invariant+ // defended: set membership mirrors g_slabs exactly -- a re-register of a live base must+ // drop the replaced wrapper from the set (ARC releases our reference, but the set retains+ // it and keeps its pages resident forever) before adding the new one. No in-tree caller+ // re-registers a live base today; defensive.+ if (old && old != b) resset_remove(old);+ if (old != b) resset_add(b);}extern "C" void coli_metal_unregister(void *base) {id<MTLBuffer> b = nil;