Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
docs: Get started now starts by getting the program
JustVuggac8dfbf26c22
diff --git a/README.it.md b/README.it.mdindex 76e3c21c..e59eec84 100644--- a/README.it.md+++ b/README.it.md@@ -15,7 +15,7 @@ precisione del modello né la semantica del router**.```$ ./coli chat- 🐦 colibrì v1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU+ 🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU✓ ready in 32s · resident 9.9 GB› ciao!◆ Ciao! 😊 Come posso aiutarti oggi?@@ -161,10 +161,42 @@ int4 e le ablazioni su granularità delle scale e rotazione sono in## Per iniziare-### 1. Scarica il modello+Ti servono due cose: **il programma** (poche centinaia di KB) e **il modello**+(372 GB). Guida passo passo per tutte le piattaforme nella+[Quick Start](docs/quickstart.md).++### 1. Procurati colibri++**Scarica una release già compilata** — Linux, macOS e Windows, nessun+compilatore necessario. Prendi l'archivio della tua piattaforma dalla pagina+[Releases](https://github.com/JustVugg/colibri/releases) e scompattalo:++```bash+mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri+python3 coli info # engine ready ✓+```++Dentro trovi il motore (`colibri`, `colibri.exe` su Windows), il launcher `coli`+e i suoi script Python di supporto. Niente da rinominare o configurare: `coli`+trova il motore accanto a sé. Serve solo avere+[Python 3](https://www.python.org/downloads/) installato — il launcher e il+gateway API sono script Python, mentre il motore è C puro senza dipendenze.++**Oppure compila dai sorgenti** — servono `gcc` (o clang) con OpenMP:++```bash+git clone https://github.com/JustVugg/colibri && cd colibri/c+./setup.sh # verifica gcc/OpenMP, compila, autotest+```++Vuoi `coli` nel PATH? Da un checkout, `pip install -e .` lo registra (il motore+resta in `c/` — è un'installazione editabile dal clone, non un wheel).++### 2. Scarica il modelloUn container **GLM-5.2 int4** pre-convertito è su Hugging Face — **usa la-versione con le teste MTP int8**:+versione con le teste MTP int8**. Pesa circa **372 GB**, quindi mettilo su un+disco che abbia lo spazio, meglio se veloce:**https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp**@@ -176,11 +208,10 @@ Oppure converti tu stesso dalla sorgente FP8 — un unico comando riprendibile cnon richiede mai i 756 GB completi su disco contemporaneamente:```bash-cd c && ./setup.sh # verifica gcc/OpenMP, compila, autotest./coli convert --model /nvme/glm52_i4 # scarica e converti shard per shard (python, una tantum)```-### 2. Esegui+### 3. Esegui```bashCOLI_MODEL=/nvme/glm52_i4 ./coli chat # budget RAM, cache e MTP rilevati automaticamente@@ -190,10 +221,11 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # controllo di idoneità (sola lettura./coli serve --model /nvme/glm52_i4 # solo API compatibile OpenAI```+Su Windows gli stessi comandi funzionano con `python coli chat --model D:\glm52_i4`.Il motore a runtime è puro C — python si usa solo per il convertitore (una tantum)e per il gateway API opzionale.-### 3. Approfondisci+### 4. Approfondisci| argomento | documento ||---|---|diff --git a/README.md b/README.mdindex ed1a512a..86aad79c 100644--- a/README.md+++ b/README.md@@ -21,7 +21,7 @@ semantics**.```$ ./coli chat- 🐦 colibrì v1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU+ 🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU✓ ready in 32s · resident 9.9 GB› ciao!◆ Ciao! 😊 Come posso aiutarti oggi?@@ -202,19 +202,43 @@ scale-granularity/rotation ablations live in## Get started-> **New here, or on Windows?** The [Quick Start guide](docs/quickstart.md) walks-> through install → build → model → first chat step by step for Linux, Windows,-> and macOS. On **Windows** you don't even need to build: download the-> `colibri-<version>-windows-x86_64.zip` from-> [Releases](https://github.com/JustVugg/colibri/releases), unzip it, install-> [Python 3](https://www.python.org/downloads/), and run `coli chat` — the-> engine ships ready to run as `colibri.exe`, the launcher finds it by itself.-> Full details in the [Windows section](docs/quickstart.md#windows).+You need two things: **the program** (a few hundred KB) and **the model**+(372 GB). Step-by-step for every platform in the+[Quick Start guide](docs/quickstart.md).-### 1. Get the model+### 1. Get colibri++**Download a prebuilt release** — Linux, macOS and Windows, no compiler needed.+Take the archive for your platform from+[Releases](https://github.com/JustVugg/colibri/releases) and unpack it:++```bash+mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri+python3 coli info # engine ready ✓+```++Inside you get the engine (`colibri`, `colibri.exe` on Windows), the `coli`+launcher and its Python helpers. Nothing to rename or configure — `coli` finds+the engine next to itself. You only need+[Python 3](https://www.python.org/downloads/) installed: the launcher and the+API gateway are Python scripts, while the engine itself is pure C with zero+dependencies.++**Or build from source** — needs `gcc` (or clang) with OpenMP:++```bash+git clone https://github.com/JustVugg/colibri && cd colibri/c+./setup.sh # checks gcc/OpenMP, builds, self-tests+```++Want `coli` on your PATH? From a checkout, `pip install -e .` registers it (the+engine still lives in `c/` — an editable install from the clone, not a wheel).++### 2. Get the modelA pre-converted **GLM-5.2 int4** container is on Hugging Face — **use the-version with the int8 MTP heads**:+version with the int8 MTP heads**. It is about **372 GB**, so put it on a disk+with the room, ideally a fast one:**https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp**@@ -226,11 +250,10 @@ Or convert from the FP8 source yourself — one resumable command that never neethe full 756 GB on disk at once:```bash-cd c && ./setup.sh # checks gcc/OpenMP, builds, self-tests./coli convert --model /nvme/glm52_i4 # download+convert shard by shard (python, one-time)```-### 2. Run it+### 3. Run it```bashCOLI_MODEL=/nvme/glm52_i4 ./coli chat # RAM budget, cache and MTP auto-detected@@ -240,21 +263,11 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # read-only readiness check./coli serve --model /nvme/glm52_i4 # OpenAI-compatible API only```+On Windows the same commands work with `python coli chat --model D:\glm52_i4`.The engine at runtime is pure C — python is only used by the one-time converterand the optional API gateway.-**On Windows?** You don't need to build. Download the-`colibri-<version>-windows-x86_64.zip` from-[Releases](https://github.com/JustVugg/colibri/releases), unzip it, install-[Python 3](https://www.python.org/downloads/), then run `coli chat` — the-engine is already named `colibri.exe` and the launcher finds it next to-itself. Full walkthrough in the [Quick Start guide](docs/quickstart.md#windows).--Prefer a `coli` command on your PATH? From a checkout, `pip install -e .`-registers it (the engine itself still lives in `c/` — this is an editable-install from the clone, not a standalone wheel).--### 3. Go deeper+### 4. Go deeper| topic | doc ||---|---|diff --git a/README.zh-CN.md b/README.zh-CN.mdindex 352de33c..90b1cfd4 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -14,7 +14,7 @@ Colibrì 是一套轻量、保持模型质量的 MoE 运行时,将 VRAM、RAM```$ ./coli chat- 🐦 colibrì v1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU+ 🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU✓ ready in 32s · resident 9.9 GB› ciao!◆ Ciao! 😊 Come posso aiutarti oggi?@@ -145,10 +145,38 @@ MTP head 必须是 **int8**(int4 head 的接受率会崩塌到 0–4%,见## 开始使用-### 1. 获取模型+你需要两样东西:**程序本体**(几百 KB)和**模型**(372 GB)。各平台的分步+指引见 [Quick Start 指南](docs/quickstart.md)。++### 1. 获取 colibri++**下载预编译版本**——Linux、macOS 与 Windows 均已提供,无需编译器。从+[Releases](https://github.com/JustVugg/colibri/releases) 下载对应平台的压缩包并解压:++```bash+mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri+python3 coli info # engine ready ✓+```++包内含引擎(`colibri`,Windows 上为 `colibri.exe`)、`coli` 启动器及其 Python+辅助脚本。无需重命名或配置:`coli` 会自动找到同目录下的引擎。你只需安装+[Python 3](https://www.python.org/downloads/)——启动器和 API gateway 是 Python+脚本,而引擎本身是零依赖的纯 C 程序。++**或者从源码构建**——需要带 OpenMP 的 `gcc`(或 clang):++```bash+git clone https://github.com/JustVugg/colibri && cd colibri/c+./setup.sh # 检查 gcc/OpenMP、构建并运行自测+```++想把 `coli` 加入 PATH?在 checkout 中执行 `pip install -e .` 即可注册(引擎仍位于+`c/` 目录——这是从克隆目录做的可编辑安装,而非独立 wheel)。++### 2. 获取模型Hugging Face 上已有预转换的 **GLM-5.2 int4** 容器——请务必使用-**含 int8 MTP head 的版本**:+**含 int8 MTP head 的版本**。它约为 **372 GB**,请放在空间足够的磁盘上,最好是快盘:**https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp**@@ -160,11 +188,10 @@ Hugging Face 上已有预转换的 **GLM-5.2 int4** 容器——请务必使用在磁盘上同时存放完整的 756 GB:```bash-cd c && ./setup.sh # 检查 gcc/OpenMP、构建并运行自测./coli convert --model /nvme/glm52_i4 # 逐 shard 下载并转换(仅此一次需要 python)```-### 2. 运行+### 3. 运行```bashCOLI_MODEL=/nvme/glm52_i4 ./coli chat # 自动检测 RAM 预算、缓存与 MTP@@ -174,9 +201,10 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # 只读就绪检查./coli serve --model /nvme/glm52_i4 # 仅提供 OpenAI 兼容 API```+在 Windows 上同样使用这些命令,写作 `python coli chat --model D:\glm52_i4`。引擎运行时是纯 C——python 只供一次性转换工具与可选的 API gateway 使用。-### 3. 深入了解+### 4. 深入了解| 主题 | 文档 ||---|---|diff --git a/README.zh-TW.md b/README.zh-TW.mdindex 2576eba1..55d5a3d0 100644--- a/README.zh-TW.md+++ b/README.zh-TW.md@@ -14,7 +14,7 @@ Colibrì 是一套輕量、維持品質的 MoE 執行環境,將 VRAM、RAM```$ ./coli chat- 🐦 colibrì v1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU+ 🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU✓ ready in 32s · resident 9.9 GB› ciao!◆ Ciao! 😊 Come posso aiutarti oggi?@@ -145,10 +145,38 @@ MTP head 必須是 **int8**(int4 head 的接受率會崩落到 0–4%,見## 開始使用-### 1. 取得模型+你需要兩樣東西:**程式本體**(幾百 KB)與**模型**(372 GB)。各平台的逐步+指引請見 [Quick Start 指南](docs/quickstart.md)。++### 1. 取得 colibri++**下載預先建置的版本**——Linux、macOS 與 Windows 均已提供,不需要編譯器。從+[Releases](https://github.com/JustVugg/colibri/releases) 下載對應平台的壓縮檔並解壓:++```bash+mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri+python3 coli info # engine ready ✓+```++裡面包含引擎(`colibri`,Windows 上為 `colibri.exe`)、`coli` 啟動器及其 Python+輔助腳本。不需重新命名或設定:`coli` 會自動找到同目錄下的引擎。你只需安裝+[Python 3](https://www.python.org/downloads/)——啟動器與 API gateway 是 Python+腳本,而引擎本身是零相依的純 C 程式。++**或者從原始碼建置**——需要具備 OpenMP 的 `gcc`(或 clang):++```bash+git clone https://github.com/JustVugg/colibri && cd colibri/c+./setup.sh # 檢查 gcc/OpenMP、建置並執行自我測試+```++想把 `coli` 加入 PATH?在 checkout 中執行 `pip install -e .` 即可註冊(引擎仍位於+`c/` 目錄——這是從複製目錄做的可編輯安裝,而非獨立 wheel)。++### 2. 取得模型Hugging Face 上已有預先轉換的 **GLM-5.2 int4** 容器——請務必使用-**含 int8 MTP head 的版本**:+**含 int8 MTP head 的版本**。它約為 **372 GB**,請放在空間足夠的硬碟上,最好是快碟:**https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp**@@ -160,11 +188,10 @@ Hugging Face 上已有預先轉換的 **GLM-5.2 int4** 容器——請務必使在硬碟上同時存放完整的 756 GB:```bash-cd c && ./setup.sh # 檢查 gcc/OpenMP、建置並執行自我測試./coli convert --model /nvme/glm52_i4 # 逐 shard 下載並轉換(僅此一次需要 python)```-### 2. 執行+### 3. 執行```bashCOLI_MODEL=/nvme/glm52_i4 ./coli chat # 自動偵測 RAM 預算、快取與 MTP@@ -174,9 +201,10 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # 唯讀就緒檢查./coli serve --model /nvme/glm52_i4 # 僅提供 OpenAI 相容 API```+在 Windows 上同樣使用這些指令,寫作 `python coli chat --model D:\glm52_i4`。引擎執行階段是純 C——python 只供單次轉換工具與選用的 API gateway 使用。-### 3. 深入了解+### 4. 深入了解| 主題 | 文件 ||---|---|diff --git a/docs/quickstart.md b/docs/quickstart.mdindex 0cd95839..77f13830 100644--- a/docs/quickstart.md+++ b/docs/quickstart.md@@ -27,6 +27,22 @@ CPU-only by default.## 1. Install the build tools+> **Shortcut — skip sections 1 and 2 entirely.** Prebuilt archives are published+> for **Linux, macOS and Windows** on the+> [Releases page](https://github.com/JustVugg/colibri/releases): unpack one,+> install [Python 3](https://www.python.org/downloads/), and jump straight to+> [step 3](#3-get-the-model). The engine ships ready to run and the `coli`+> launcher finds it next to itself — no compiler, no renaming, no configuration.+>+> ```bash+> mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri+> python3 coli info # engine ready ✓+> ```+>+> Build from source instead if you want the fastest binary for *your* CPU+> (`ARCH=native` unlocks the vector instructions your chip actually has), or if+> you plan to hack on the engine.+### Linux (Ubuntu / Debian)```bash