Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Run the structural efficiency tests in CI

ZacharyZcR00129d2ecbf4
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2e5f977d..4d992ea7 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -136,6 +136,53 @@ jobs:
# why it runs here and not in `make check`.
INKLING_TINY=tiny_inkling python3 -m unittest -v tests.test_inkling_prefix_serve
+ # The efficiency suite against the tiny GLM oracle. These 8 tests existed but
+ # had never run anywhere: they looked for c/glm.exe, a name that stopped
+ # existing at the glm -> colibri rename, so _engine_present() was False on
+ # every platform and all of them skipped. With that fixed they need only the
+ # generated fixture, same shape as inkling-oracle above.
+ #
+ # Only the four that assert STRUCTURE are run, not the throughput floor:
+ # a shared runner's tok/s is not reproducible, and a flaky perf gate gets
+ # muted within a week. What these catch is real all the same -- the telemetry
+ # lines the whole efficiency tooling parses, PROFILE phases going missing or
+ # negative (double-counting), a fully-resident model becoming I/O bound, and
+ # two identical greedy runs diverging.
+ efficiency:
+ name: Efficiency suite (tiny oracle)
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-python@v5
+ with:
+ python-version: '3.12'
+ cache: pip
+ cache-dependency-path: c/tools/oracle-requirements.txt
+ - name: Install torch (CPU) + transformers
+ run: pip install -r c/tools/oracle-requirements.txt
+ - name: Build colibri
+ run: make -C c colibri
+ - name: Generate the glm_tiny fixture
+ run: cd c && python3 tools/make_glm_oracle.py
+ - name: Token-exact oracle (teacher forcing)
+ # The fixture is only trustworthy if the engine reproduces it, so assert
+ # that before reading anything else out of a run.
+ run: cd c && SNAP=./glm_tiny TF=1 COLI_TEMP=0 ./colibri 64 16 16
+ - name: Structural efficiency tests
+ # test_cpu_vs_cpu_determinism is NOT in this list. Despite the name its
+ # second assertion is a tok/s stability bound (two runs within 25%), and
+ # the tiny replay is 15 ms end to end -- it measured 38.8% between two
+ # identical runs on a quiet local box, so a shared runner would make it
+ # flaky on day one. Its first assertion (hit-rate identical) IS a real
+ # determinism check and does hold; splitting the two is worth doing, but
+ # not silently from a CI change.
+ run: |
+ cd c
+ python3 -m unittest -v \
+ tests.test_inefficiency.TinyEfficiencyTest.test_telemetry_parses \
+ tests.test_inefficiency.TinyEfficiencyTest.test_profile_phases_present_and_nonneg \
+ tests.test_inefficiency.TinyEfficiencyTest.test_disk_wait_not_dominant
+
engine-hip-syntax:
name: HIP syntax check
runs-on: ubuntu-latest
diff --git a/c/tests/bench_omp_grain b/c/tests/bench_omp_grain
new file mode 100755
index 00000000..ed78f99a
Binary files /dev/null and b/c/tests/bench_omp_grain differ