Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
README: dashboard screenshots + contents index + Web dashboard section (#202)
ZacharyZcRe21318e678a7
diff --git a/README.md b/README.mdindex 4109d5fe..86c30090 100644--- a/README.md+++ b/README.md@@ -17,6 +17,32 @@ $ ./coli chat◆ Ciao! 😊 Come posso aiutarti oggi?```++## See it running++<p align="center">+ <img src="docs/media/colibri-dashboard.png" width="900" alt="colibrì web dashboard — live metrics, hardware panel, expert tiers">+</p>+<p align="center"><em>The web dashboard (<code>./coli web</code>): a 744B model answering at 4+ tok/s end-to-end on 6× RTX 5090 —+with live token metrics, the hardware panel, and the VRAM/RAM/disk expert tiers.</em></p>++<p align="center">+ <img src="docs/media/colibri-brain.png" width="900" alt="the Brain page — 19,456 experts as a live cortex">+</p>+<p align="center"><em>The <strong>Brain</strong> page: all 19,456 experts as a living cortex — colour is the storage tier,+brightness is routing heat, and every expert routed in a turn flashes white. Hovering shows the expert's+<a href="https://github.com/JustVugg/colibri/issues/175">measured topic affinity</a>.</em></p>++## Contents++- [The idea](#the-idea)+- [See it running](#see-it-running)+- [What's implemented](#whats-implemented)+- [Honest numbers](#honest-numbers-wsl2-12-cores-25-gb-ram-nvme-via-vhdx)+- [Download the model](#download-the-model)+- [Web dashboard](#web-dashboard)+- [Got a better machine?](#got-a-better-machine-try-it--heres-what-to-expect)+## The ideaA 744B Mixture-of-Experts model activates only ~40B parameters per token — and only ~11 GB of those change from token to token (the routed experts). So:@@ -501,6 +527,23 @@ thrashing. Persistent `.coli_usage` remains the long-term signal and is not deca**Conversations reopen warm** (`.coli_kv`, since 2026-07-10): `coli chat` persists the compressed MLA KV-cache to disk after every turn (~182 KB/token, appended incrementally, crash-safe). Close the chat, reopen it tomorrow — the model still remembers the whole conversation and **zero re-prefill happens**: validated byte-identical to an uninterrupted session. `:reset` clears it, `KVSAVE=0` disables it.+## Web dashboard++One command serves the OpenAI-compatible API **and** the web console on the same port, then opens your browser when the engine is ready:++```bash+cd web && npm install && npm run build # once+./coli web --model <model-dir>+```++What you get:++- **Chat** with live metrics: a flashing token counter while generating, then tok/s, time-to-first-token, prompt→completion counts and queue wait;+- **Runtime panel**: your hardware (CPU, GPUs + VRAM, RAM, cores), the scheduler, and the live expert-tier bar — how many of the 19,456 experts sit in VRAM / RAM / disk right now;+- **Brain**: the whole model as a 76×256 cortex, one cell per expert. Colour = tier, brightness = routing heat, and the experts routed in each turn flash white and decay — you watch the model think. Hover any cell for its tier, heat and [measured topic affinity](https://github.com/JustVugg/colibri/issues/175) (specialists for code, Chinese, math, law… live in layers 11–22).++The dashboard talks to the engine over two tiny protocol lines (`TIERS`, `EMAP`/`HITS`) and plain JSON endpoints — nothing heavier than the engine itself.+## Got a better machine? Try it — here's what to expectcolibrì was built on deliberately humble hardware (12 cores, 25 GB RAM, an older DRAM-less NVMe behind a WSL2 VHDX that measured ~1 GB/s random on *this* drive — note WSL2 VHDX is not inherently slow: a community 5090 box measured 10.5 GB/s O_DIRECT through one, [#101](https://github.com/JustVugg/colibri/issues/101)). **Every one of those constraints is a knob your machine can turn up.** The engine needs: Linux (or WSL2), macOS, or **Windows 11 natively (MinGW-w64)**; gcc with OpenMP, AVX2, ≥16 GB RAM, and the ~370 GB int4 model on a local NVMe (ext4/NTFS — never a network/9p mount).diff --git a/docs/media/colibri-brain.png b/docs/media/colibri-brain.pngnew file mode 100644index 00000000..0cd6c905Binary files /dev/null and b/docs/media/colibri-brain.png differdiff --git a/docs/media/colibri-dashboard.png b/docs/media/colibri-dashboard.pngnew file mode 100644index 00000000..123fd3eeBinary files /dev/null and b/docs/media/colibri-dashboard.png differ