Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
ci: build inkling + token-exact oracle in CI
Nicholas Beerbower2c396d821bb2
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 426ea413..a21bd4f1 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -12,8 +12,8 @@ jobs:runs-on: ubuntu-lateststeps:- uses: actions/checkout@v4- - name: Build colibri- run: cd c && make colibri+ - name: Build colibri + inkling+ run: cd c && make colibri inkling- name: C test suiterun: cd c && make test-c@@ -40,6 +40,38 @@ jobs:nvcc -O2 -std=c++17 -arch=sm_80 -c backend_cuda.cu -o /dev/null \-Xcompiler=-Wall,-Wextraecho "CUDA syntax check passed"+ - name: Compile backend_cuda_ink.cu (syntax only, no GPU)+ run: |+ cd c+ # inkling's own CUDA backend (bf16 residents + int4 expert GEMM); same+ # no-pipeline rule as above so a compile error can actually fail the job.+ nvcc -O2 -std=c++17 -arch=sm_80 -c backend_cuda_ink.cu -o /dev/null \+ -Xcompiler=-Wall,-Wextra+ echo "inkling CUDA syntax check passed"++ inkling-oracle:+ name: Inkling oracle (token-exact vs transformers)+ runs-on: ubuntu-latest+ steps:+ - uses: actions/checkout@v4+ - uses: actions/setup-python@v5+ with:+ python-version: '3.12'+ cache: pip+ cache-dependency-path: c/tools/oracle-requirements.txt+ - name: Install torch (CPU) + transformers+ run: pip install -r c/tools/oracle-requirements.txt+ - name: Build inkling+ run: cd c && make inkling+ - name: Tiny-model fixture + token-exact oracle (f32, bit-exact experts)+ run: |+ cd c+ # Random-init InklingForCausalLM via HF transformers, then the C engine+ # reproduces it token-for-token. bits=0 keeps experts f32 (bit-exact+ # forward); the greedy pass exits non-zero on any mismatch, so this+ # gate fails the moment a shared-code change regresses the engine.+ python3 tools/make_tiny_inkling.py tiny_inkling+ SNAP=tiny_inkling ./inkling 8 0 tiny_inkling/ref_inkling.jsonengine-hip-syntax:name: HIP syntax checkdiff --git a/.gitignore b/.gitignoreindex b6866801..7ac3789d 100644--- a/.gitignore+++ b/.gitignore@@ -69,3 +69,4 @@ olmoe_merged/olmoe_i4/c/olmoe_merged/c/olmoe_i4/+c/tiny_inkling/diff --git a/c/tools/oracle-requirements.txt b/c/tools/oracle-requirements.txtnew file mode 100644index 00000000..75a1e88d--- /dev/null+++ b/c/tools/oracle-requirements.txt@@ -0,0 +1,9 @@+# Deps for the Inkling tiny-model oracle (tools/make_tiny_inkling.py) — the+# ci.yml `inkling-oracle` job installs exactly this file, and setup-python+# hashes it for the pip cache key. CPU torch: the oracle is a tiny random-init+# model, no GPU wanted.+--index-url https://download.pytorch.org/whl/cpu+--extra-index-url https://pypi.org/simple+torch+transformers>=5.14 # Inkling architecture support (InklingForCausalLM)+safetensors