Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

docs: Get started now starts by getting the program

JustVuggac8dfbf26c22
diff --git a/README.it.md b/README.it.md
index 76e3c21c..e59eec84 100644
--- a/README.it.md
+++ b/README.it.md
@@ -15,7 +15,7 @@ precisione del modello né la semantica del router**.
```
$ ./coli chat
- 🐦 colibrì v1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
@@ -161,10 +161,42 @@ int4 e le ablazioni su granularità delle scale e rotazione sono in
## Per iniziare
-### 1. Scarica il modello
+Ti servono due cose: **il programma** (poche centinaia di KB) e **il modello**
+(372 GB). Guida passo passo per tutte le piattaforme nella
+[Quick Start](docs/quickstart.md).
+
+### 1. Procurati colibri
+
+**Scarica una release già compilata** — Linux, macOS e Windows, nessun
+compilatore necessario. Prendi l'archivio della tua piattaforma dalla pagina
+[Releases](https://github.com/JustVugg/colibri/releases) e scompattalo:
+
+```bash
+mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri
+python3 coli info # engine ready ✓
+```
+
+Dentro trovi il motore (`colibri`, `colibri.exe` su Windows), il launcher `coli`
+e i suoi script Python di supporto. Niente da rinominare o configurare: `coli`
+trova il motore accanto a sé. Serve solo avere
+[Python 3](https://www.python.org/downloads/) installato — il launcher e il
+gateway API sono script Python, mentre il motore è C puro senza dipendenze.
+
+**Oppure compila dai sorgenti** — servono `gcc` (o clang) con OpenMP:
+
+```bash
+git clone https://github.com/JustVugg/colibri && cd colibri/c
+./setup.sh # verifica gcc/OpenMP, compila, autotest
+```
+
+Vuoi `coli` nel PATH? Da un checkout, `pip install -e .` lo registra (il motore
+resta in `c/` — è un'installazione editabile dal clone, non un wheel).
+
+### 2. Scarica il modello
Un container **GLM-5.2 int4** pre-convertito è su Hugging Face — **usa la
-versione con le teste MTP int8**:
+versione con le teste MTP int8**. Pesa circa **372 GB**, quindi mettilo su un
+disco che abbia lo spazio, meglio se veloce:
**https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp**
@@ -176,11 +208,10 @@ Oppure converti tu stesso dalla sorgente FP8 — un unico comando riprendibile c
non richiede mai i 756 GB completi su disco contemporaneamente:
```bash
-cd c && ./setup.sh # verifica gcc/OpenMP, compila, autotest
./coli convert --model /nvme/glm52_i4 # scarica e converti shard per shard (python, una tantum)
```
-### 2. Esegui
+### 3. Esegui
```bash
COLI_MODEL=/nvme/glm52_i4 ./coli chat # budget RAM, cache e MTP rilevati automaticamente
@@ -190,10 +221,11 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # controllo di idoneità (sola lettura
./coli serve --model /nvme/glm52_i4 # solo API compatibile OpenAI
```
+Su Windows gli stessi comandi funzionano con `python coli chat --model D:\glm52_i4`.
Il motore a runtime è puro C — python si usa solo per il convertitore (una tantum)
e per il gateway API opzionale.
-### 3. Approfondisci
+### 4. Approfondisci
| argomento | documento |
|---|---|
diff --git a/README.md b/README.md
index ed1a512a..86aad79c 100644
--- a/README.md
+++ b/README.md
@@ -21,7 +21,7 @@ semantics**.
```
$ ./coli chat
- 🐦 colibrì v1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
@@ -202,19 +202,43 @@ scale-granularity/rotation ablations live in
## Get started
-> **New here, or on Windows?** The [Quick Start guide](docs/quickstart.md) walks
-> through install → build → model → first chat step by step for Linux, Windows,
-> and macOS. On **Windows** you don't even need to build: download the
-> `colibri-<version>-windows-x86_64.zip` from
-> [Releases](https://github.com/JustVugg/colibri/releases), unzip it, install
-> [Python 3](https://www.python.org/downloads/), and run `coli chat` — the
-> engine ships ready to run as `colibri.exe`, the launcher finds it by itself.
-> Full details in the [Windows section](docs/quickstart.md#windows).
+You need two things: **the program** (a few hundred KB) and **the model**
+(372 GB). Step-by-step for every platform in the
+[Quick Start guide](docs/quickstart.md).
-### 1. Get the model
+### 1. Get colibri
+
+**Download a prebuilt release** — Linux, macOS and Windows, no compiler needed.
+Take the archive for your platform from
+[Releases](https://github.com/JustVugg/colibri/releases) and unpack it:
+
+```bash
+mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri
+python3 coli info # engine ready ✓
+```
+
+Inside you get the engine (`colibri`, `colibri.exe` on Windows), the `coli`
+launcher and its Python helpers. Nothing to rename or configure — `coli` finds
+the engine next to itself. You only need
+[Python 3](https://www.python.org/downloads/) installed: the launcher and the
+API gateway are Python scripts, while the engine itself is pure C with zero
+dependencies.
+
+**Or build from source** — needs `gcc` (or clang) with OpenMP:
+
+```bash
+git clone https://github.com/JustVugg/colibri && cd colibri/c
+./setup.sh # checks gcc/OpenMP, builds, self-tests
+```
+
+Want `coli` on your PATH? From a checkout, `pip install -e .` registers it (the
+engine still lives in `c/` — an editable install from the clone, not a wheel).
+
+### 2. Get the model
A pre-converted **GLM-5.2 int4** container is on Hugging Face — **use the
-version with the int8 MTP heads**:
+version with the int8 MTP heads**. It is about **372 GB**, so put it on a disk
+with the room, ideally a fast one:
**https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp**
@@ -226,11 +250,10 @@ Or convert from the FP8 source yourself — one resumable command that never nee
the full 756 GB on disk at once:
```bash
-cd c && ./setup.sh # checks gcc/OpenMP, builds, self-tests
./coli convert --model /nvme/glm52_i4 # download+convert shard by shard (python, one-time)
```
-### 2. Run it
+### 3. Run it
```bash
COLI_MODEL=/nvme/glm52_i4 ./coli chat # RAM budget, cache and MTP auto-detected
@@ -240,21 +263,11 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # read-only readiness check
./coli serve --model /nvme/glm52_i4 # OpenAI-compatible API only
```
+On Windows the same commands work with `python coli chat --model D:\glm52_i4`.
The engine at runtime is pure C — python is only used by the one-time converter
and the optional API gateway.
-**On Windows?** You don't need to build. Download the
-`colibri-<version>-windows-x86_64.zip` from
-[Releases](https://github.com/JustVugg/colibri/releases), unzip it, install
-[Python 3](https://www.python.org/downloads/), then run `coli chat` — the
-engine is already named `colibri.exe` and the launcher finds it next to
-itself. Full walkthrough in the [Quick Start guide](docs/quickstart.md#windows).
-
-Prefer a `coli` command on your PATH? From a checkout, `pip install -e .`
-registers it (the engine itself still lives in `c/` — this is an editable
-install from the clone, not a standalone wheel).
-
-### 3. Go deeper
+### 4. Go deeper
| topic | doc |
|---|---|
diff --git a/README.zh-CN.md b/README.zh-CN.md
index 352de33c..90b1cfd4 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -14,7 +14,7 @@ Colibrì 是一套轻量、保持模型质量的 MoE 运行时,将 VRAM、RAM
```
$ ./coli chat
- 🐦 colibrì v1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
@@ -145,10 +145,38 @@ MTP head 必须是 **int8**(int4 head 的接受率会崩塌到 0–4%,见
## 开始使用
-### 1. 获取模型
+你需要两样东西:**程序本体**(几百 KB)和**模型**(372 GB)。各平台的分步
+指引见 [Quick Start 指南](docs/quickstart.md)。
+
+### 1. 获取 colibri
+
+**下载预编译版本**——Linux、macOS 与 Windows 均已提供,无需编译器。从
+[Releases](https://github.com/JustVugg/colibri/releases) 下载对应平台的压缩包并解压:
+
+```bash
+mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri
+python3 coli info # engine ready ✓
+```
+
+包内含引擎(`colibri`,Windows 上为 `colibri.exe`)、`coli` 启动器及其 Python
+辅助脚本。无需重命名或配置:`coli` 会自动找到同目录下的引擎。你只需安装
+[Python 3](https://www.python.org/downloads/)——启动器和 API gateway 是 Python
+脚本,而引擎本身是零依赖的纯 C 程序。
+
+**或者从源码构建**——需要带 OpenMP 的 `gcc`(或 clang):
+
+```bash
+git clone https://github.com/JustVugg/colibri && cd colibri/c
+./setup.sh # 检查 gcc/OpenMP、构建并运行自测
+```
+
+想把 `coli` 加入 PATH?在 checkout 中执行 `pip install -e .` 即可注册(引擎仍位于
+`c/` 目录——这是从克隆目录做的可编辑安装,而非独立 wheel)。
+
+### 2. 获取模型
Hugging Face 上已有预转换的 **GLM-5.2 int4** 容器——请务必使用
-**含 int8 MTP head 的版本**:
+**含 int8 MTP head 的版本**。它约为 **372 GB**,请放在空间足够的磁盘上,最好是快盘:
**https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp**
@@ -160,11 +188,10 @@ Hugging Face 上已有预转换的 **GLM-5.2 int4** 容器——请务必使用
在磁盘上同时存放完整的 756 GB:
```bash
-cd c && ./setup.sh # 检查 gcc/OpenMP、构建并运行自测
./coli convert --model /nvme/glm52_i4 # 逐 shard 下载并转换(仅此一次需要 python)
```
-### 2. 运行
+### 3. 运行
```bash
COLI_MODEL=/nvme/glm52_i4 ./coli chat # 自动检测 RAM 预算、缓存与 MTP
@@ -174,9 +201,10 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # 只读就绪检查
./coli serve --model /nvme/glm52_i4 # 仅提供 OpenAI 兼容 API
```
+在 Windows 上同样使用这些命令,写作 `python coli chat --model D:\glm52_i4`。
引擎运行时是纯 C——python 只供一次性转换工具与可选的 API gateway 使用。
-### 3. 深入了解
+### 4. 深入了解
| 主题 | 文档 |
|---|---|
diff --git a/README.zh-TW.md b/README.zh-TW.md
index 2576eba1..55d5a3d0 100644
--- a/README.zh-TW.md
+++ b/README.zh-TW.md
@@ -14,7 +14,7 @@ Colibrì 是一套輕量、維持品質的 MoE 執行環境,將 VRAM、RAM
```
$ ./coli chat
- 🐦 colibrì v1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
@@ -145,10 +145,38 @@ MTP head 必須是 **int8**(int4 head 的接受率會崩落到 0–4%,見
## 開始使用
-### 1. 取得模型
+你需要兩樣東西:**程式本體**(幾百 KB)與**模型**(372 GB)。各平台的逐步
+指引請見 [Quick Start 指南](docs/quickstart.md)。
+
+### 1. 取得 colibri
+
+**下載預先建置的版本**——Linux、macOS 與 Windows 均已提供,不需要編譯器。從
+[Releases](https://github.com/JustVugg/colibri/releases) 下載對應平台的壓縮檔並解壓:
+
+```bash
+mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri
+python3 coli info # engine ready ✓
+```
+
+裡面包含引擎(`colibri`,Windows 上為 `colibri.exe`)、`coli` 啟動器及其 Python
+輔助腳本。不需重新命名或設定:`coli` 會自動找到同目錄下的引擎。你只需安裝
+[Python 3](https://www.python.org/downloads/)——啟動器與 API gateway 是 Python
+腳本,而引擎本身是零相依的純 C 程式。
+
+**或者從原始碼建置**——需要具備 OpenMP 的 `gcc`(或 clang):
+
+```bash
+git clone https://github.com/JustVugg/colibri && cd colibri/c
+./setup.sh # 檢查 gcc/OpenMP、建置並執行自我測試
+```
+
+想把 `coli` 加入 PATH?在 checkout 中執行 `pip install -e .` 即可註冊(引擎仍位於
+`c/` 目錄——這是從複製目錄做的可編輯安裝,而非獨立 wheel)。
+
+### 2. 取得模型
Hugging Face 上已有預先轉換的 **GLM-5.2 int4** 容器——請務必使用
-**含 int8 MTP head 的版本**:
+**含 int8 MTP head 的版本**。它約為 **372 GB**,請放在空間足夠的硬碟上,最好是快碟:
**https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp**
@@ -160,11 +188,10 @@ Hugging Face 上已有預先轉換的 **GLM-5.2 int4** 容器——請務必使
在硬碟上同時存放完整的 756 GB:
```bash
-cd c && ./setup.sh # 檢查 gcc/OpenMP、建置並執行自我測試
./coli convert --model /nvme/glm52_i4 # 逐 shard 下載並轉換(僅此一次需要 python)
```
-### 2. 執行
+### 3. 執行
```bash
COLI_MODEL=/nvme/glm52_i4 ./coli chat # 自動偵測 RAM 預算、快取與 MTP
@@ -174,9 +201,10 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # 唯讀就緒檢查
./coli serve --model /nvme/glm52_i4 # 僅提供 OpenAI 相容 API
```
+在 Windows 上同樣使用這些指令,寫作 `python coli chat --model D:\glm52_i4`。
引擎執行階段是純 C——python 只供單次轉換工具與選用的 API gateway 使用。
-### 3. 深入了解
+### 4. 深入了解
| 主題 | 文件 |
|---|---|
diff --git a/docs/quickstart.md b/docs/quickstart.md
index 0cd95839..77f13830 100644
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -27,6 +27,22 @@ CPU-only by default.
## 1. Install the build tools
+> **Shortcut — skip sections 1 and 2 entirely.** Prebuilt archives are published
+> for **Linux, macOS and Windows** on the
+> [Releases page](https://github.com/JustVugg/colibri/releases): unpack one,
+> install [Python 3](https://www.python.org/downloads/), and jump straight to
+> [step 3](#3-get-the-model). The engine ships ready to run and the `coli`
+> launcher finds it next to itself — no compiler, no renaming, no configuration.
+>
+> ```bash
+> mkdir colibri && tar xzf colibri-v1.1.0-linux-x86_64.tar.gz -C colibri && cd colibri
+> python3 coli info # engine ready ✓
+> ```
+>
+> Build from source instead if you want the fastest binary for *your* CPU
+> (`ARCH=native` unlocks the vector instructions your chip actually has), or if
+> you plan to hack on the engine.
+
### Linux (Ubuntu / Debian)
```bash