Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
merge: dev β keep engine-hip-syntax alongside the new windows-cuda-build job
noobdev-ph755a9a40c59d
diff --cc .github/workflows/ci.ymlindex 500f6649,72b7149e..3857f6fc--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@@ -41,27 -41,78 +41,99 @@@ jobs-Xcompiler=-Wall,-Wextraecho "CUDA syntax check passed"+ engine-hip-syntax:+ name: HIP syntax check+ runs-on: ubuntu-latest+ steps:+ - uses: actions/checkout@v4+ - name: Free runner disk+ # rocm/dev images are large; hosted runners need the headroom freed+ # BEFORE the pull (which is why this is not a container: job).+ run: sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL+ - name: Compile backend_cuda.cu + test binary with hipcc (no GPU)+ run: |+ # Tag pinned: `latest` outgrew the runner disk once already. The+ # gcc-side engine link is skipped: containerized gcc/ld cannot link+ # ROCm runtime libs (both 22.04 and 24.04 bases); the hipcc-driver+ # link of the test binary below covers the single-source claim, and+ # the engine link is exercised on real ROCm hardware (GPU_BACKENDS.md).+ docker run --rm -v "$PWD":/src -w /src rocm/dev-ubuntu-24.04:6.2 bash -c '+ apt-get update && apt-get install -y --no-install-recommends make gcc g++ &&+ make -C c gpu-compile HIP=1 HIP_ARCH=gfx1100'+ echo "HIP syntax check passed"++ windows-cuda-build:+ # The Windows CUDA path has no coverage anywhere: `engine-cuda-syntax` above+ # compiles with nvcc's *GCC* host on Linux, and check.yml's windows job is+ # MinGW/UCRT64 CPU-only by design (#140). But nvcc on Windows requires MSVC+ # as its host compiler β it does not accept MinGW β so `make cuda-dll` runs a+ # toolchain nothing else in CI touches. That gap is not theoretical: every+ # bug in #158 was a *build* failure on this path (MSVC rejects the GCC-style+ # -Xcompiler=-Wall,-Wextra with "D8021 invalid numeric argument '/Wextra'",+ # unresolvable CUDA_HOME/NVCC defaults, POSIX setenv in the kernel test), and+ # #314 was CUDA_HOME with spaces β the layout the CUDA installer ships by+ # default. Both classes are compile-time and need no GPU to catch.+ #+ # Build-only ON PURPOSE: GitHub's hosted runners have no NVIDIA device, so+ # this job proves the Windows+MSVC CUDA build stays buildable, NOT that the+ # kernels or the DLL loader behave on real silicon. That still needs hardware+ # (see #157). Claiming otherwise would be the false confidence the+ # engine-cuda-syntax comment above already warns about.+ name: CUDA build (Windows, MSVC host)+ # windows-2022, NOT windows-latest: the latest image now ships Visual Studio+ # 18 (MSVC 14.5x), and CUDA's crt/host_config.h hard-errors on any host newer+ # than VS 2022 ("Only the versions between 2017 and 2022 (inclusive) are+ # supported"). That is a real constraint for every CUDA user on Windows, not+ # a CI quirk β pinning tracks what the toolkit actually supports. Revisit when+ # a CUDA release accepts VS 18; -allow-unsupported-compiler would only mask it.+ runs-on: windows-2022+ steps:+ - uses: actions/checkout@v4+ - name: MSVC environment (puts cl.exe on PATH for nvcc -ccbin)+ uses: ilammy/msvc-dev-cmd@v1+ - name: Install CUDA toolkit (compiler only)+ uses: Jimver/[email protected]+ with:+ # Same pin as engine-cuda-syntax: v0.2.19's version table stops at+ # 12.6.2. This installs to the default "C:\Program Files\NVIDIA GPU+ # Computing Toolkit\..." path, so CUDA_HOME is space-bearing here β+ # which is exactly the #314 regression this job would have caught.+ cuda: '12.6.2'+ method: network+ # cudart as well as nvcc: unlike the Linux job (which only needs to+ # *compile* backend_cuda.cu), cuda-dll links it, and on Windows the+ # runtime headers/import lib ship as a separate installer component β+ # with '["nvcc"]' alone this fails at `#include <cuda_runtime.h>`.+ sub-packages: '["nvcc", "cudart"]'+ - uses: msys2/setup-msys2@v2+ with:+ msystem: UCRT64+ update: false+ # inherit: cl.exe (msvc-dev-cmd) and nvcc (cuda-toolkit) are added to+ # the *Windows* PATH by the steps above; without inheriting it the+ # recipe's `command -v` guards fail inside the MSYS2 shell.+ path-type: inherit+ install: >-+ make+ mingw-w64-ucrt-x86_64-gcc+ - name: make cuda-dll (nvcc + MSVC host)+ shell: msys2 {0}+ run: |+ cd c+ # CUDA_ARCH is pinned: the default is `native`, which asks the driver+ # what card is present β there is none here, so it must be explicit.+ # sm_80 matches engine-cuda-syntax and is supported by the 12.6 pin.+ make cuda-dll CUDA_ARCH=sm_80+ test -f coli_cuda.dll || { echo "cuda-dll reported success but produced no DLL" >&2; exit 1; }+ echo "coli_cuda.dll built (MSVC host)"+ - name: make glm CUDA_DLL=1 (host links backend_loader, not cudart)+ shell: msys2 {0}+ run: |+ cd c+ make glm CUDA_DLL=1+ test -f glm.exe || { echo "glm CUDA_DLL=1 reported success but produced no exe" >&2; exit 1; }+ echo "glm.exe built against the DLL loader"+web:name: Web UIruns-on: ubuntu-latest