Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
docs(windows): document isolated GLM oracle tooling
Kenneth-Javier Ortigozad27e182c68ec
diff --git a/c/tools/requirements.txt b/c/tools/requirements.txtindex 3d0087c6..88a8b644 100644--- a/c/tools/requirements.txt+++ b/c/tools/requirements.txt@@ -1,6 +1,21 @@# Python dependencies for the colibri offline converter / oracle / download tools# (c/tools/*.py and c/*.py). Minimum-version pins; for a fully reproducible env,# freeze exact versions into your own lockfile with `pip freeze > locked.txt`.+#+# DEVELOPMENT TOOLING ONLY β none of this is a Colibri runtime dependency. The+# engine is a C binary; once a model (or the generated glm_tiny oracle) exists on+# disk, running it needs no Python at all.+#+# Installing this whole file is only needed for the download/convert tools. To+# generate the tiny GLM oracle (tools/make_glm_oracle.py) the direct set is just:+#+# torch transformers>=5.11 safetensors+#+# transformers>=5.11 is a hard requirement there, not a preference: the generator+# exits rather than run below it, because older releases apply split-half Llama+# RoPE and silently emit an oracle the engine cannot match (issue #281). See+# docs/windows.md "Generating the synthetic GLM oracle model" for an isolated+# CPU-only setup.torch>=2.4safetensors>=0.4huggingface-hub>=0.24diff --git a/docs/windows.md b/docs/windows.mdindex c46c803b..b9409f66 100644--- a/docs/windows.md+++ b/docs/windows.md@@ -291,6 +291,96 @@ not been validated on hardware here.**CUDA is unaffected.** A `CUDA_DLL=1` host ignores `COLI_HIP_RUNTIME_DIR`entirely and keeps its existing `coli_cuda.dll` search behaviour unchanged.+### Generating the synthetic GLM oracle model++To exercise the engine end to end you need *a* model, and the smallest one is+generated locally rather than downloaded. `c/tools/make_glm_oracle.py` builds+**`glm_tiny`** β a ~0.6 MB GLM-5.2 (`glm_moe_dsa`) with the real architecture+(MLA + DSA indexer + sigmoid router + shared expert, 5 layers, 8 routed experts,+top-2) but random weights, plus `ref_glm.json`, a teacher-forcing oracle of+expected token IDs.++> **It is test tooling, not a language model.** Random weights produce+> meaningless text. Its only purpose is that the C engine must reproduce the+> reference token IDs exactly.++The packages below are **development tooling only** β the engine itself is a C+binary and needs no Python once the model files exist.++**1. Create an isolated environment.** Nothing is installed system-wide:++```powershell+$tooling = "C:\Development\colibri-validation\glm-oracle-tooling"+uv venv --python "$env:LOCALAPPDATA\Programs\Python\Python314\python.exe" "$tooling\venv"++# CPU-only Torch from the official PyTorch CPU index; the rest from PyPI+uv pip install --python "$tooling\venv\Scripts\python.exe" `+ --index-url https://download.pytorch.org/whl/cpu `+ --extra-index-url https://pypi.org/simple `+ --index-strategy unsafe-best-match `+ torch "transformers==5.11.0" "safetensors>=0.4"+```++Validated on **CPython 3.14.6 (Windows x64)** with **torch 2.13.0+cpu**,+**transformers 5.11.0** and **safetensors 0.8.0**. `transformers>=5.11` is a+hard floor β the generator exits below it, because older releases apply+split-half Llama RoPE and silently emit an oracle the engine cannot match+(issue #281). Pin 5.11.0 exactly to avoid oracle drift.++**2. Verify the environment** (imports only β builds nothing, downloads nothing):++```powershell+& "$tooling\venv\Scripts\python.exe" -c "import torch, transformers; from transformers import GlmMoeDsaConfig, GlmMoeDsaForCausalLM; print(torch.__version__, torch.version.cuda, transformers.__version__)"+```++Expect a `+cpu` Torch build and `None` for the CUDA version.++**3. Generate the model β from an external working directory.** The generator+takes no `--outdir`: it writes `glm_tiny/` **and** `ref_glm.json` relative to the+current directory. `c/ref_glm.json` is a tracked file, so generating inside `c/`+would overwrite it. Run it from outside the repository instead:++```powershell+$out = "$tooling\model"+New-Item -ItemType Directory -Force -Path $out | Out-Null+Push-Location $out+& "$tooling\venv\Scripts\python.exe" C:\Development\colibri\c\tools\make_glm_oracle.py+Pop-Location+# -> $out\glm_tiny\{config.json,model.safetensors} and $out\ref_glm.json+```++Generation is fully offline once the packages are installed: the model is built+from a literal config with a fixed seed, and nothing is fetched from the network.++**4. Run the engine against it.** `SNAP` and `REF` both accept external paths,+so the generated files never need to enter the repository:++```powershell+$env:SNAP = "$out\glm_tiny"+$env:REF = "$out\ref_glm.json"+$env:TF = "1"+.\colibri.exe 64 16 16+```++`TF=1` is the teacher-forcing self-test: it prints+`PREFILL (teacher-forcing) C vs oracle: N/32 positions` and one+`[ORACLE] mismatch pos=β¦ expected=β¦ got=β¦` line per disagreement. Add+`DEBUG_LOGITS=1` to dump the top-5 logits at a mismatch. Floating-point+near-ties are toolchain-dependent, so a small shortfall is expected; compare a+GPU run against the CPU run on the same machine rather than against a fixed+number.++To exercise the HIP backend instead, add the runtime settings from+[Runtime setup](#runtime-setup-experimental-developer-facing) to the same+command.++**Cleanup** removes the tooling and the generated model in one step; the+repository is untouched:++```powershell+Remove-Item -Recurse -Force $tooling+```+### AMD failure index| Symptom | Cause | Fix |