Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

merge: dev β€” keep engine-hip-syntax alongside the new windows-cuda-build job

noobdev-ph755a9a40c59d
diff --cc .github/workflows/ci.yml
index 500f6649,72b7149e..3857f6fc
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@@ -41,27 -41,78 +41,99 @@@ jobs
-Xcompiler=-Wall,-Wextra
echo "CUDA syntax check passed"
+ engine-hip-syntax:
+ name: HIP syntax check
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - name: Free runner disk
+ # rocm/dev images are large; hosted runners need the headroom freed
+ # BEFORE the pull (which is why this is not a container: job).
+ run: sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
+ - name: Compile backend_cuda.cu + test binary with hipcc (no GPU)
+ run: |
+ # Tag pinned: `latest` outgrew the runner disk once already. The
+ # gcc-side engine link is skipped: containerized gcc/ld cannot link
+ # ROCm runtime libs (both 22.04 and 24.04 bases); the hipcc-driver
+ # link of the test binary below covers the single-source claim, and
+ # the engine link is exercised on real ROCm hardware (GPU_BACKENDS.md).
+ docker run --rm -v "$PWD":/src -w /src rocm/dev-ubuntu-24.04:6.2 bash -c '
+ apt-get update && apt-get install -y --no-install-recommends make gcc g++ &&
+ make -C c gpu-compile HIP=1 HIP_ARCH=gfx1100'
+ echo "HIP syntax check passed"
+
+ windows-cuda-build:
+ # The Windows CUDA path has no coverage anywhere: `engine-cuda-syntax` above
+ # compiles with nvcc's *GCC* host on Linux, and check.yml's windows job is
+ # MinGW/UCRT64 CPU-only by design (#140). But nvcc on Windows requires MSVC
+ # as its host compiler β€” it does not accept MinGW β€” so `make cuda-dll` runs a
+ # toolchain nothing else in CI touches. That gap is not theoretical: every
+ # bug in #158 was a *build* failure on this path (MSVC rejects the GCC-style
+ # -Xcompiler=-Wall,-Wextra with "D8021 invalid numeric argument '/Wextra'",
+ # unresolvable CUDA_HOME/NVCC defaults, POSIX setenv in the kernel test), and
+ # #314 was CUDA_HOME with spaces β€” the layout the CUDA installer ships by
+ # default. Both classes are compile-time and need no GPU to catch.
+ #
+ # Build-only ON PURPOSE: GitHub's hosted runners have no NVIDIA device, so
+ # this job proves the Windows+MSVC CUDA build stays buildable, NOT that the
+ # kernels or the DLL loader behave on real silicon. That still needs hardware
+ # (see #157). Claiming otherwise would be the false confidence the
+ # engine-cuda-syntax comment above already warns about.
+ name: CUDA build (Windows, MSVC host)
+ # windows-2022, NOT windows-latest: the latest image now ships Visual Studio
+ # 18 (MSVC 14.5x), and CUDA's crt/host_config.h hard-errors on any host newer
+ # than VS 2022 ("Only the versions between 2017 and 2022 (inclusive) are
+ # supported"). That is a real constraint for every CUDA user on Windows, not
+ # a CI quirk β€” pinning tracks what the toolkit actually supports. Revisit when
+ # a CUDA release accepts VS 18; -allow-unsupported-compiler would only mask it.
+ runs-on: windows-2022
+ steps:
+ - uses: actions/checkout@v4
+ - name: MSVC environment (puts cl.exe on PATH for nvcc -ccbin)
+ uses: ilammy/msvc-dev-cmd@v1
+ - name: Install CUDA toolkit (compiler only)
+ uses: Jimver/[email protected]
+ with:
+ # Same pin as engine-cuda-syntax: v0.2.19's version table stops at
+ # 12.6.2. This installs to the default "C:\Program Files\NVIDIA GPU
+ # Computing Toolkit\..." path, so CUDA_HOME is space-bearing here β€”
+ # which is exactly the #314 regression this job would have caught.
+ cuda: '12.6.2'
+ method: network
+ # cudart as well as nvcc: unlike the Linux job (which only needs to
+ # *compile* backend_cuda.cu), cuda-dll links it, and on Windows the
+ # runtime headers/import lib ship as a separate installer component β€”
+ # with '["nvcc"]' alone this fails at `#include <cuda_runtime.h>`.
+ sub-packages: '["nvcc", "cudart"]'
+ - uses: msys2/setup-msys2@v2
+ with:
+ msystem: UCRT64
+ update: false
+ # inherit: cl.exe (msvc-dev-cmd) and nvcc (cuda-toolkit) are added to
+ # the *Windows* PATH by the steps above; without inheriting it the
+ # recipe's `command -v` guards fail inside the MSYS2 shell.
+ path-type: inherit
+ install: >-
+ make
+ mingw-w64-ucrt-x86_64-gcc
+ - name: make cuda-dll (nvcc + MSVC host)
+ shell: msys2 {0}
+ run: |
+ cd c
+ # CUDA_ARCH is pinned: the default is `native`, which asks the driver
+ # what card is present β€” there is none here, so it must be explicit.
+ # sm_80 matches engine-cuda-syntax and is supported by the 12.6 pin.
+ make cuda-dll CUDA_ARCH=sm_80
+ test -f coli_cuda.dll || { echo "cuda-dll reported success but produced no DLL" >&2; exit 1; }
+ echo "coli_cuda.dll built (MSVC host)"
+ - name: make glm CUDA_DLL=1 (host links backend_loader, not cudart)
+ shell: msys2 {0}
+ run: |
+ cd c
+ make glm CUDA_DLL=1
+ test -f glm.exe || { echo "glm CUDA_DLL=1 reported success but produced no exe" >&2; exit 1; }
+ echo "glm.exe built against the DLL loader"
+
web:
name: Web UI
runs-on: ubuntu-latest