Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Merge pull request #1386 from JustVugg/feat/inkling-olmoe-hits
Vincenzo Fornarob5649e643e66
diff --cc .github/workflows/ci.ymlindex bfac5cda,acbc192a..2c2e1b2e--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@@ -445,16 -445,40 +445,49 @@@ jobsgrep -q "config.json says 4 layers" bad.log || {echo "FAIL: refused, but not for the reason under test"; cat bad.log; exit 1; }echo "refused as expected:"; grep '^\[cfg\]' bad.log+ - name: Brain and Profile reach the dashboard+ run: |+ cd c+ # The oracle fixture has no tokenizer (weights only); `coli serve`+ # needs one to encode the prompt. The byte tokenizer is test data+ # sized to the fixture's vocab, the same as the GLM-5.3 gate uses.+ python3 tools/make_edge_tiny_tokenizer.py --vocab-size 320 qwen36_tiny_c+ make qwen36 >/dev/null+ COLI_QWEN36_FIXTURE=qwen36_tiny_c python3 -m unittest -v tests.test_qwen36_dashboard+ olmoe-tiny-check:+ name: OLMoE tiny oracle (token-exact + dashboard HITS)+ runs-on: ubuntu-latest+ steps:+ - uses: actions/checkout@v4+ - uses: actions/setup-python@v5+ with:+ python-version: '3.12'+ cache: pip+ cache-dependency-path: c/tools/oracle-requirements.txt+ - name: Install torch (CPU) + transformers+ run: pip install -r c/tools/oracle-requirements.txt+ - name: Tiny OLMoE checkpoint, converted+ run: |+ cd c+ # 4 layers x 8 experts at toy dimensions, greedy reference from the+ # HF model in ref_olmoe.json. The converter is the real one.+ python3 tools/make_olmoe_tiny.py --output olmoe_tiny+ python3 tools/convert_olmoe_merged.py --model olmoe_tiny --out olmoe_tiny_c+ # weights only: serve mode needs a tokenizer to encode the prompt+ python3 tools/make_edge_tiny_tokenizer.py --vocab-size 128 olmoe_tiny_c+ - name: Token-exact against the reference+ run: |+ cd c+ make olmoe+ SNAP=olmoe_tiny_c ./olmoe 8 8 olmoe_tiny/ref_olmoe.json | tee oracle.log+ # the engine prints the score and exits 0 either way: the gate is here+ grep -qE 'Matching tokens: ([0-9]+)/\1$' oracle.log+ - name: "Brain tab: HITS after every turn, on the grid EMAP declared"+ run: |+ cd c+ OLMOE_TINY=olmoe_tiny_c python3 -m unittest -v tests.test_olmoe_dashboard_hits+qwen38-tiny-check:name: Qwen3.8 tiny oracle (GDN + QSA + PLE + MoE)runs-on: ubuntu-latest