Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
qwen38 MTP: gated in CI; the harness leaves the test_*.py glob
JustVugg4ac9ed70fa30
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 6dad14ae..5546e466 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -1035,6 +1035,8 @@ jobs:run: make -C c qwen38-tiny-fp8-check qwen38-tier-engine-check- name: Routed experts as int4-g64 from the converter's sidecarrun: make -C c qwen38-tiny-int4-check+ - name: MTP speculative decoding against plain decoding (lossless gate)+ run: make -C c qwen38-tiny-mtp-check- name: Repeat the eviction path under ASan and UBSanrun: |cd c@@ -1052,6 +1054,10 @@ jobs:figrep -q "Matching tokens: 8/8" qwen38-san.loggrep -q "Matching tokens: 8/8" qwen38-san-int4.log+ # the MTP head, the S=2 verify and the rollback on every fixture kind+ ASAN_OPTIONS=detect_leaks=0 UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1 \+ python3 tests/qwen38_mtp_harness.py --engine ./qwen38 --bf16 ./qwen38_tiny_mtp \+ --fp8 ./qwen38_tiny_fp8_mtp --int4 ./qwen38_tiny_int4_mtp- name: Refuse config and tensor-shape disagreementrun: |cd cdiff --git a/c/Makefile b/c/Makefileindex dd2957c8..adf7ac3d 100644--- a/c/Makefile+++ b/c/Makefile@@ -1394,7 +1394,7 @@ qwen38-tiny-int4-check: qwen38-tiny-int4-generate qwen38-tiny-fp8-generate qwen3# "the MTP head"). The fixtures are the three above plus an MTP head (--mtp:# random head weights from their own seed under the release's mtp.* names; the# model and its references do not change) and a byte tokenizer for the text-# and serve gates. tests/test_qwen38_mtp.py checks:+# and serve gates. tests/qwen38_mtp_harness.py checks:# (a) the head's draft logits equal tools/qwen38_mtp_ref.py's float32# reference for the default wiring b and the diagnostic a, through the# one-row and the two-row (after an accepted draft) paths, on BF16 and@@ -1416,7 +1416,7 @@ qwen38-tiny-mtp-generate:$(PYTHON) tools/make_edge_tiny_tokenizer.py --vocab-size 64 ./$$d || exit $$?; doneqwen38-tiny-mtp-check: qwen38-tiny-mtp-generate qwen38$(EXE)- $(PYTHON) tests/test_qwen38_mtp.py --engine ./qwen38$(EXE) --bf16 ./qwen38_tiny_mtp \+ $(PYTHON) tests/qwen38_mtp_harness.py --engine ./qwen38$(EXE) --bf16 ./qwen38_tiny_mtp \--fp8 ./qwen38_tiny_fp8_mtp --int4 ./qwen38_tiny_int4_mtp# Context-size gates: KV layout, growth across requests, and the attentiondiff --git a/c/tests/test_qwen38_mtp.py b/c/tests/qwen38_mtp_harness.pysimilarity index 99%rename from c/tests/test_qwen38_mtp.pyrename to c/tests/qwen38_mtp_harness.pyindex 2e8f748e..6e338940 100644--- a/c/tests/test_qwen38_mtp.py+++ b/c/tests/qwen38_mtp_harness.py@@ -21,7 +21,7 @@ serve). Gates:serve a multi-turn serve session (prefix reuse, a pinned prefix,logprobs, a sampled turn) gives the same frames MTP on and off.- python tests/test_qwen38_mtp.py --engine ./qwen38 --bf16 ./qwen38_tiny_mtp \\+ python tests/qwen38_mtp_harness.py --engine ./qwen38 --bf16 ./qwen38_tiny_mtp \\--fp8 ./qwen38_tiny_fp8_mtp --int4 ./qwen38_tiny_int4_mtp"""import argparse