Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
docs/tools: document PILOT_EVICT_GUARD and add CLI overrides to test_olmoe_real
Egon Ruiterfbdb1099872f
diff --git a/c/olmoe.c b/c/olmoe.cindex 74df155b..a1cc0495 100644--- a/c/olmoe.c+++ b/c/olmoe.c@@ -13,6 +13,7 @@* WIDE=N : prefetch top-K*N candidates (default 1, try 2 or 3)* SMOOTH=F : EMA coefficient for routing momentum (default 0.3, range 0.0-0.95)* CONF_LIMIT=F : cumulative gate probability threshold for prefetch cutoff (default 0.92)+ * PILOT_EVICT_GUARD=0/1 : 1=enable LFRU prefetch eviction guard (default), 0=disable* (expert queue is sorted by eid for SSD read locality)*/#define _GNU_SOURCEdiff --git a/c/tools/test_olmoe_real.py b/c/tools/test_olmoe_real.pyindex 1abf64d0..384b2f9d 100644--- a/c/tools/test_olmoe_real.py+++ b/c/tools/test_olmoe_real.py@@ -30,8 +30,8 @@ BOOTSTRAP_REF = HERE / "ref_olmoe_bootstrap.json"PROMPT_IDS = [510, 5347, 273, 6181, 310] # "The capital of France is"MAX_NEW = 12-CACHE_SIZE = 32 # experts cached per layer-QUANT_BITS = 8 # engine supports 2-8; 8 = int8 (lossless vs our quant)+CACHE_SIZE = int(sys.argv[1]) if len(sys.argv) > 1 else 32 # experts cached per layer+QUANT_BITS = int(sys.argv[2]) if len(sys.argv) > 2 else 8 # engine supports 2-8; 8 = int8 (lossless vs our quant)# ββ Step 1: Write bootstrap ref with dummy full_ids = prompt_ids ββββββββββ# olmoe.exe needs full_ids to know how many tokens to generate (nfull - np).