Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
Run the structural efficiency tests in CI
ZacharyZcR00129d2ecbf4
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 2e5f977d..4d992ea7 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -136,6 +136,53 @@ jobs:# why it runs here and not in `make check`.INKLING_TINY=tiny_inkling python3 -m unittest -v tests.test_inkling_prefix_serve+ # The efficiency suite against the tiny GLM oracle. These 8 tests existed but+ # had never run anywhere: they looked for c/glm.exe, a name that stopped+ # existing at the glm -> colibri rename, so _engine_present() was False on+ # every platform and all of them skipped. With that fixed they need only the+ # generated fixture, same shape as inkling-oracle above.+ #+ # Only the four that assert STRUCTURE are run, not the throughput floor:+ # a shared runner's tok/s is not reproducible, and a flaky perf gate gets+ # muted within a week. What these catch is real all the same -- the telemetry+ # lines the whole efficiency tooling parses, PROFILE phases going missing or+ # negative (double-counting), a fully-resident model becoming I/O bound, and+ # two identical greedy runs diverging.+ efficiency:+ name: Efficiency suite (tiny oracle)+ runs-on: ubuntu-latest+ steps:+ - uses: actions/checkout@v4+ - uses: actions/setup-python@v5+ with:+ python-version: '3.12'+ cache: pip+ cache-dependency-path: c/tools/oracle-requirements.txt+ - name: Install torch (CPU) + transformers+ run: pip install -r c/tools/oracle-requirements.txt+ - name: Build colibri+ run: make -C c colibri+ - name: Generate the glm_tiny fixture+ run: cd c && python3 tools/make_glm_oracle.py+ - name: Token-exact oracle (teacher forcing)+ # The fixture is only trustworthy if the engine reproduces it, so assert+ # that before reading anything else out of a run.+ run: cd c && SNAP=./glm_tiny TF=1 COLI_TEMP=0 ./colibri 64 16 16+ - name: Structural efficiency tests+ # test_cpu_vs_cpu_determinism is NOT in this list. Despite the name its+ # second assertion is a tok/s stability bound (two runs within 25%), and+ # the tiny replay is 15 ms end to end -- it measured 38.8% between two+ # identical runs on a quiet local box, so a shared runner would make it+ # flaky on day one. Its first assertion (hit-rate identical) IS a real+ # determinism check and does hold; splitting the two is worth doing, but+ # not silently from a CI change.+ run: |+ cd c+ python3 -m unittest -v \+ tests.test_inefficiency.TinyEfficiencyTest.test_telemetry_parses \+ tests.test_inefficiency.TinyEfficiencyTest.test_profile_phases_present_and_nonneg \+ tests.test_inefficiency.TinyEfficiencyTest.test_disk_wait_not_dominant+engine-hip-syntax:name: HIP syntax checkruns-on: ubuntu-latestdiff --git a/c/tests/bench_omp_grain b/c/tests/bench_omp_grainnew file mode 100755index 00000000..ed78f99aBinary files /dev/null and b/c/tests/bench_omp_grain differ