Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

ci: build inkling + token-exact oracle in CI

Nicholas Beerbower2c396d821bb2
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 426ea413..a21bd4f1 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -12,8 +12,8 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- - name: Build colibri
- run: cd c && make colibri
+ - name: Build colibri + inkling
+ run: cd c && make colibri inkling
- name: C test suite
run: cd c && make test-c
@@ -40,6 +40,38 @@ jobs:
nvcc -O2 -std=c++17 -arch=sm_80 -c backend_cuda.cu -o /dev/null \
-Xcompiler=-Wall,-Wextra
echo "CUDA syntax check passed"
+ - name: Compile backend_cuda_ink.cu (syntax only, no GPU)
+ run: |
+ cd c
+ # inkling's own CUDA backend (bf16 residents + int4 expert GEMM); same
+ # no-pipeline rule as above so a compile error can actually fail the job.
+ nvcc -O2 -std=c++17 -arch=sm_80 -c backend_cuda_ink.cu -o /dev/null \
+ -Xcompiler=-Wall,-Wextra
+ echo "inkling CUDA syntax check passed"
+
+ inkling-oracle:
+ name: Inkling oracle (token-exact vs transformers)
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-python@v5
+ with:
+ python-version: '3.12'
+ cache: pip
+ cache-dependency-path: c/tools/oracle-requirements.txt
+ - name: Install torch (CPU) + transformers
+ run: pip install -r c/tools/oracle-requirements.txt
+ - name: Build inkling
+ run: cd c && make inkling
+ - name: Tiny-model fixture + token-exact oracle (f32, bit-exact experts)
+ run: |
+ cd c
+ # Random-init InklingForCausalLM via HF transformers, then the C engine
+ # reproduces it token-for-token. bits=0 keeps experts f32 (bit-exact
+ # forward); the greedy pass exits non-zero on any mismatch, so this
+ # gate fails the moment a shared-code change regresses the engine.
+ python3 tools/make_tiny_inkling.py tiny_inkling
+ SNAP=tiny_inkling ./inkling 8 0 tiny_inkling/ref_inkling.json
engine-hip-syntax:
name: HIP syntax check
diff --git a/.gitignore b/.gitignore
index b6866801..7ac3789d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -69,3 +69,4 @@ olmoe_merged/
olmoe_i4/
c/olmoe_merged/
c/olmoe_i4/
+c/tiny_inkling/
diff --git a/c/tools/oracle-requirements.txt b/c/tools/oracle-requirements.txt
new file mode 100644
index 00000000..75a1e88d
--- /dev/null
+++ b/c/tools/oracle-requirements.txt
@@ -0,0 +1,9 @@
+# Deps for the Inkling tiny-model oracle (tools/make_tiny_inkling.py) — the
+# ci.yml `inkling-oracle` job installs exactly this file, and setup-python
+# hashes it for the pip cache key. CPU torch: the oracle is a tiny random-init
+# model, no GPU wanted.
+--index-url https://download.pytorch.org/whl/cpu
+--extra-index-url https://pypi.org/simple
+torch
+transformers>=5.14 # Inkling architecture support (InklingForCausalLM)
+safetensors