Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

docs: the engine is colibri(.exe) everywhere β€” kill the obsolete rename step

JustVugg10d9b580fa1e
diff --git a/README.md b/README.md
index 8b48ee59..ed1a512a 100644
--- a/README.md
+++ b/README.md
@@ -206,10 +206,10 @@ scale-granularity/rotation ablations live in
> through install β†’ build β†’ model β†’ first chat step by step for Linux, Windows,
> and macOS. On **Windows** you don't even need to build: download the
> `colibri-<version>-windows-x86_64.zip` from
-> [Releases](https://github.com/JustVugg/colibri/releases), unzip it, rename
-> `colibri-*-windows-x86_64.exe` β†’ `glm.exe`, install
-> [Python 3](https://www.python.org/downloads/), and run `coli chat` β€” full
-> details in the [Windows section](docs/quickstart.md#windows).
+> [Releases](https://github.com/JustVugg/colibri/releases), unzip it, install
+> [Python 3](https://www.python.org/downloads/), and run `coli chat` β€” the
+> engine ships ready to run as `colibri.exe`, the launcher finds it by itself.
+> Full details in the [Windows section](docs/quickstart.md#windows).
### 1. Get the model
@@ -245,10 +245,10 @@ and the optional API gateway.
**On Windows?** You don't need to build. Download the
`colibri-<version>-windows-x86_64.zip` from
-[Releases](https://github.com/JustVugg/colibri/releases), unzip it, rename
-`colibri-*-windows-x86_64.exe` β†’ `glm.exe` (so the `coli` launcher finds the
-engine), install [Python 3](https://www.python.org/downloads/), then run
-`coli chat`. Full walkthrough in the [Quick Start guide](docs/quickstart.md#windows).
+[Releases](https://github.com/JustVugg/colibri/releases), unzip it, install
+[Python 3](https://www.python.org/downloads/), then run `coli chat` β€” the
+engine is already named `colibri.exe` and the launcher finds it next to
+itself. Full walkthrough in the [Quick Start guide](docs/quickstart.md#windows).
Prefer a `coli` command on your PATH? From a checkout, `pip install -e .`
registers it (the engine itself still lives in `c/` β€” this is an editable
diff --git a/docs/cuda.md b/docs/cuda.md
index fab477d6..835ba971 100644
--- a/docs/cuda.md
+++ b/docs/cuda.md
@@ -10,7 +10,7 @@ cd c
make cuda-test CUDA=1 # q8/q4/q2/f32 kernel correctness
make CUDA=1
# optional dense-path experiment (hot experts are configured below)
-COLI_CUDA=1 COLI_GPU=0 CUDA_DENSE=1 SNAP=/nvme/glm52_i4 ./glm 64 4 4
+COLI_CUDA=1 COLI_GPU=0 CUDA_DENSE=1 SNAP=/nvme/glm52_i4 ./colibri 64 4 4
```
Requirements: Linux, an NVIDIA driver, and a CUDA Toolkit under
@@ -26,19 +26,19 @@ A measured `PIN` profile promotes its hottest experts into a persistent VRAM
tier while keeping the rest in RAM:
```bash
-STATS=stats.txt SNAP=/nvme/glm52_i4 ./glm 64 4 4 # collect routing frequencies first
+STATS=stats.txt SNAP=/nvme/glm52_i4 ./colibri 64 4 4 # collect routing frequencies first
COLI_CUDA=1 COLI_GPU=0 CUDA_EXPERT_GB=16 \
-PIN=stats.txt PIN_GB=160 SNAP=/nvme/glm52_i4 ./glm 64 4 4
+PIN=stats.txt PIN_GB=160 SNAP=/nvme/glm52_i4 ./colibri 64 4 4
# multi-GPU expert tier, 150 GB total budget across six 32 GB devices
COLI_CUDA=1 COLI_GPUS=0,1,2,3,4,5 CUDA_EXPERT_GB=150 \
CUDA_DENSE=1 PIN=stats.txt PIN_GB=300 RAM_GB=226 \
-SNAP=/nvme/glm52_i4 ./glm 64 4 4
+SNAP=/nvme/glm52_i4 ./colibri 64 4 4
# large-RAM host: fill safe VRAM, then keep every remaining expert in RAM
COLI_CUDA=1 COLI_GPUS=0,1,2,3,4,5 CUDA_EXPERT_GB=auto \
CUDA_DENSE=1 COLI_CUDA_ATTN=1 PIN=stats.txt PIN_GB=all RAM_GB=auto \
-SNAP=/nvme/glm52_i4 ./glm 64 4 4
+SNAP=/nvme/glm52_i4 ./colibri 64 4 4
```
Selected experts are uploaded during startup, so capacity failures occur before
diff --git a/docs/grammar-draft.md b/docs/grammar-draft.md
index cf5d1ac4..b3728316 100644
--- a/docs/grammar-draft.md
+++ b/docs/grammar-draft.md
@@ -31,7 +31,7 @@ Fewer forwards also means less LRU churn, which compounds with cache hit rate.
## Usage
```bash
-GRAMMAR=fit.gbnf COLI_TEMP=0 PROMPT="..." ./glm 64 4 8
+GRAMMAR=fit.gbnf COLI_TEMP=0 PROMPT="..." ./colibri 64 4 8
```
- `GRAMMAR=<file.gbnf>` β€” arm the draft source with a grammar. Byte-level GBNF
diff --git a/docs/metal.md b/docs/metal.md
index 3c94e5c0..5d983003 100644
--- a/docs/metal.md
+++ b/docs/metal.md
@@ -9,7 +9,7 @@ Token-exact vs the CPU path.
```bash
cd c
-make glm METAL=1 # macOS only; no Xcode needed (shader compiles at runtime)
+make colibri METAL=1 # macOS only; no Xcode needed (shader compiles at runtime)
make metal-test # standalone kernel/attention correctness vs CPU reference
COLI_METAL=1 COLI_MODEL=/path/glm52_i4 ./coli chat --ram 96
```
diff --git a/docs/quickstart.md b/docs/quickstart.md
index 60d48777..0cd95839 100644
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -48,23 +48,17 @@ Inside you'll find:
| File | What it is |
|---|---|
-| `colibri-<version>-windows-x86_64.exe` | **the engine** β€” the C program that actually runs the model |
+| `colibri.exe` | **the engine** β€” the C program that actually runs the model |
| `coli` | the command-line launcher (`chat`, `serve`, `convert`, `doctor`, …) |
| `openai_server.py`, `resource_plan.py`, `doctor.py` | Python support for the API server and placement planner |
-Two setup steps:
-
-1. **Rename the engine to `glm.exe`** so the launcher can find it (it looks for a
- binary named `glm`):
- ```powershell
- Rename-Item colibri-*-windows-x86_64.exe glm.exe
- ```
-2. **Install Python 3** from [python.org](https://www.python.org/downloads/) β€” the
- `coli` launcher and the API gateway are Python scripts (the engine itself is
- pure C and needs nothing).
+One setup step: **install Python 3** from
+[python.org](https://www.python.org/downloads/) β€” the `coli` launcher and the
+API gateway are Python scripts (the engine itself is pure C and needs nothing).
+No renaming, no configuration: the launcher finds `colibri.exe` next to itself.
Then continue to [step 3](#3-get-the-model). Prefer to skip the launcher? You can
-run the engine directly β€” `.\glm.exe` reads the model path from the `SNAP`
+run the engine directly β€” `.\colibri.exe` reads the model path from the `SNAP`
environment variable (see [docs/windows.md](windows.md)) β€” but `coli chat` is the
easy path.
diff --git a/docs/windows.md b/docs/windows.md
index ee4944d7..153ead0c 100644
--- a/docs/windows.md
+++ b/docs/windows.md
@@ -31,7 +31,7 @@ Use the container recommended in the README (with **int8 MTP heads** β€” int4 he
From a normal PowerShell, in the repo's `c\` directory:
```powershell
-make glm.exe ARCH=native # ARCH=native unlocks AVX-VNNI on Alder Lake+/Arrow Lake
+make colibri.exe ARCH=native # ARCH=native unlocks AVX-VNNI on Alder Lake+/Arrow Lake
make iobench.exe # disk benchmark, useful before committing to the download
```
@@ -39,10 +39,10 @@ Warnings about `#pragma comment` and unused variables are normal (MSVC-isms gcc
### ⚠️ Smart App Control will block your fresh binary
-On Windows 11 machines with **Smart App Control** enforced (`VerifiedAndReputablePolicyState = 1`), running your self-compiled `glm.exe` fails with:
+On Windows 11 machines with **Smart App Control** enforced (`VerifiedAndReputablePolicyState = 1`), running your self-compiled `colibri.exe` fails with:
```
-Program 'glm.exe' failed to run: An Application Control policy has blocked this file
+Program 'colibri.exe' failed to run: An Application Control policy has blocked this file
```
This is not Defender and not Mark-of-the-Web β€” SAC blocks *all* unsigned, unknown binaries, which includes anything you compile yourself. **Fix:** Windows Security β†’ App & browser control β†’ Smart App Control settings β†’ **Off**, then **reboot** (the policy only reloads on restart). Note SAC is one-way: re-enabling later requires resetting Windows. If the settings page is missing, the registry equivalent is setting `HKLM:\SYSTEM\CurrentControlSet\Control\CI\Policy\VerifiedAndReputablePolicyState` to `0` (admin PowerShell), then rebooting. Check your current state before touching anything:
@@ -64,13 +64,13 @@ nvcc needs MSVC as host compiler, so this one step must run from a shell with th
```cmd
make cuda-dll CUDA_ARCH=sm_120 # match your GPU: sm_120 Blackwell, sm_89 Ada, ...
-make glm.exe CUDA_DLL=1 ARCH=native # relink host with the runtime loader
+make colibri.exe CUDA_DLL=1 ARCH=native # relink host with the runtime loader
```
Two pitfalls, both fixed on current `dev` (#314) but worth knowing on older checkouts:
- **Spaces in `CUDA_HOME`** (`C:\Program Files\...`) used to break the recipe β†’ fixed; nvcc now comes from PATH and `"$(NVCC)"` is quoted.
-- **`make glm.exe CUDA_DLL=1` after a CPU-only build** used to report `up to date` and silently keep the CPU-only binary (GPU tier never engages, no error). Current `dev` has a build-config stamp that forces the relink. On older trees: delete `glm.exe` first.
+- **`make colibri.exe CUDA_DLL=1` after a CPU-only build** used to report `up to date` and silently keep the CPU-only binary (GPU tier never engages, no error). Current `dev` has a build-config stamp that forces the relink. On older trees: delete the binary (`colibri.exe`; `glm.exe` pre-rename) first.
Sanity check: first GPU run should print `[CUDA] device 0: <your GPU>, ... sm_XX` and `[CUDA] mode: routed experts + resident dense tensors`.
@@ -99,10 +99,10 @@ Size `CUDA_EXPERT_GB` so dense (~10 GB) + experts + working set stays under your
| Symptom | Cause | Fix |
|---|---|---|
-| `'printf' is not recognized` / `The system cannot find the path specified` during `make glm.exe` | scoop MinGW has no `sh.exe`; make fell back to cmd.exe (#478) | Β§0 β€” use MSYS2/w64devkit, or `set PATH=%PATH%;C:\msys64\usr\bin` |
+| `'printf' is not recognized` / `The system cannot find the path specified` during `make colibri.exe` | scoop MinGW has no `sh.exe`; make fell back to cmd.exe (#478) | Β§0 β€” use MSYS2/w64devkit, or `set PATH=%PATH%;C:\msys64\usr\bin` |
| `An Application Control policy has blocked this file` | Smart App Control | Β§2 β€” turn SAC off + **reboot** |
| `cuda-dll ... Error 1` immediately | old tree: spaced CUDA_HOME / MSVC rejects `-Wextra` | update to current `dev` (#314) |
-| `glm.exe is up to date` but GPU never engages | old tree: stale CPU-only binary | update to `dev`, or delete `glm.exe` and rebuild |
+| `colibri.exe is up to date` but GPU never engages | old tree: stale CPU-only binary | update to `dev`, or delete the binary and rebuild |
| `cl.exe (MSVC) not in PATH` | built from plain PowerShell | use the x64 Native Tools prompt |
| `nvcc fatal: unsupported gpu architecture 'sm_120'` | CUDA < 12.8 | install CUDA 12.8+ |
| MTP `0% (0/0)` on CPU path | int4 MTP heads in the container | use the int8-MTP container |
@@ -116,15 +116,15 @@ Size `CUDA_EXPERT_GB` so dense (~10 GB) + experts + working set stays under your
# dot-product instruction (VPDPBUSD) the engine can use for ~1.3x faster
# quantized matmul. The x86-64-v3 default (portable AVX2) compiles it out;
# build for THIS machine to enable it:
-make glm.exe ARCH=native # banner prints "idot: avx-vnni"
+make colibri.exe ARCH=native # banner prints "idot: avx-vnni"
# Verify (tiny model, 2.4 MB):
pip install torch transformers safetensors huggingface_hub
python tools/make_glm_oracle.py # generate tiny oracle
-SNAP=./glm_tiny TF=1 ./glm.exe 64 16 16 # expect "32/32 positions"
+SNAP=./glm_tiny TF=1 ./colibri.exe 64 16 16 # expect "32/32 positions"
# Run with real model:
-SNAP=D:\glm52_i4 ./glm.exe 64 4 16 # batch inference
+SNAP=D:\glm52_i4 ./colibri.exe 64 4 16 # batch inference
python coli chat --model D:\glm52_i4 # interactive chat
python coli serve --model D:\glm52_i4 # OpenAI-compatible API
```
@@ -149,7 +149,7 @@ completion.
On Windows the engine is built with MinGW gcc but CUDA kernels require MSVC +
nvcc. The split is clean: build the CUDA backend into a standalone
-`coli_cuda.dll` (nvcc + MSVC), then the host `glm.exe` loads it at runtime via
+`coli_cuda.dll` (nvcc + MSVC), then the host `colibri.exe` loads it at runtime via
`LoadLibrary` (`c/backend_loader.c`). The host never links cudart directly; if
the DLL is absent the engine falls back to CPU without error.
@@ -161,7 +161,7 @@ make cuda-dll CUDA_HOME="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.
# Build the host with the runtime loader (CUDA_DLL=1 adds -DCOLI_CUDA and
# links backend_loader.o instead of cudart):
-make glm.exe CUDA_DLL=1 ARCH=native
+make colibri.exe CUDA_DLL=1 ARCH=native
# Run with the GPU expert tier (8 GB VRAM budget here; scale to your free VRAM):
$env:COLI_CUDA="1"; $env:COLI_GPU="0"; $env:CUDA_EXPERT_GB="8"