Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
docs: link the GLM-5.3 int4 container from the model section (#1403)
HoneyTyagiie982635cb815
diff --git a/README.it.md b/README.it.mdindex a6329fea..a866a84a 100644--- a/README.it.md+++ b/README.it.md@@ -284,6 +284,12 @@ disco che abbia lo spazio, meglio se veloce:**https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp**+**GLM-5.3** è la stessa famiglia e si carica con lo stesso motore. Ha il suo+container group-scaled (gs64), circa **419 GB**, e arriva **senza** la testa MTP,+quindi la decodifica speculativa resta disattivata:++**https://huggingface.co/Justvugg/GLM-5.3-colibri-int4-g64**+> ⚠️ Usa il container **gs64** qui sopra, non i vecchi mirror int4 per-row> (`mateogrgic/…`, `jlnsrk/…`): misurano circa 9 punti percentuali in meno sulla> qualità e causavano i loop in think-mode e le generazioni senza termine originalidiff --git a/README.md b/README.mdindex f204abcc..6b8b366c 100644--- a/README.md+++ b/README.md@@ -374,6 +374,12 @@ so put it on a disk with the room, ideally a fast one:**https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp**+**GLM-5.3** is the same family and loads with the same engine. It has its own+container, also group-scaled (gs64), about **419 GB**. It ships **without** the+MTP head, so speculative decoding stays off:++**https://huggingface.co/Justvugg/GLM-5.3-colibri-int4-g64**+> ⚠️ Use the **gs64** container above, not the older per-row int4 mirrors> (`mateogrgic/…`, `jlnsrk/…`): those measure ~9pp worse on quality and are the> root cause of the original think-mode loops and never-terminating generations@@ -405,7 +411,7 @@ the model's `config.json`):> | Model | Disk for the weights | RAM | GPU |> |---|---|---|---|> | **OLMoE** | ~7 GB (int8 container) | 8 GB | not needed |-> | **GLM-5.2/5.3** | ~372 GB | 16 GB min, 24 GB comfortable | not needed |+> | **GLM-5.2/5.3** | ~372 GB (5.2) / ~419 GB (5.3) | 16 GB min, 24 GB comfortable | not needed |> | **GLM-5.3-Flash** | ~195 GB converted | 25 GB (12 GB weights at int4 + expert cache) | not needed |> | **Inkling** | ~469 GB | 25 GB with the int4 dense container, ~120 GB without | not needed |> | **Kimi K3** | ~1.6 TB | 32 GB+ | not needed |@@ -419,7 +425,7 @@ the model's `config.json`):| Family | Total / active | Weights | Build | Docs ||---|---|---|---|---|-| **GLM-5.2/5.3** | 744B / 40B | [`mastouri/…-int4-g64-with-int8-mtp`](https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp) (372 GB) | `make -C c glm` | this page |+| **GLM-5.2/5.3** | 744B / 40B | [`mastouri/…-int4-g64-with-int8-mtp`](https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp) (372 GB) or [`Justvugg/GLM-5.3-colibri-int4-g64`](https://huggingface.co/Justvugg/GLM-5.3-colibri-int4-g64) (419 GB) | `make -C c glm` | this page || **Inkling** (Thinking Machines) | 975B / 41B | [`nbeerbower/Inkling-colibri-int4`](https://huggingface.co/nbeerbower/Inkling-colibri-int4) (469 GB) | `make -C c inkling` | [inkling.md](docs/inkling.md) || **GLM-5.3-Flash** (Z.ai) | 321B / 40B | [`zai-org/GLM-5.3-Flash`](https://huggingface.co/zai-org/GLM-5.3-Flash) — converted to **int4-gs64** routed experts, dense stays BF16 and the precision is a load-time choice; vision included | `make -C c glm53` | [glm53-flash.md](docs/glm53-flash.md) || **Kimi K3** (Moonshot) | 2.8T / 104B | [`moonshotai/Kimi-K3`](https://huggingface.co/moonshotai/Kimi-K3) — original checkpoint, routed experts stay **native MXFP4** | `make -C c kimi_k3` | [kimi_k3.md](docs/kimi_k3.md) |diff --git a/README.zh-CN.md b/README.zh-CN.mdindex d13cbf2d..fb257d77 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -243,6 +243,11 @@ Hugging Face 上已有预转换的 **GLM-5.2 int4** 容器——请务必使用**https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp**+**GLM-5.3** 属于同一家族,使用同一引擎加载。它有自己的 group-scaled(gs64)容器,+约 **419 GB**,且**不含** MTP head,因此推测解码保持关闭:++**https://huggingface.co/Justvugg/GLM-5.3-colibri-int4-g64**+> ⚠️ 请使用上面的 **gs64** 容器,不要使用较旧的 per-row int4 镜像>(`mateogrgic/…`、`jlnsrk/…`):后者质量实测低约 9 个百分点,也是> [#455](https://github.com/JustVugg/colibri/issues/455) 最初 think-mode 循环与生成不终止的根因。diff --git a/README.zh-TW.md b/README.zh-TW.mdindex cc085678..e1f9e972 100644--- a/README.zh-TW.md+++ b/README.zh-TW.md@@ -244,6 +244,11 @@ Hugging Face 上已有預先轉換的 **GLM-5.2 int4** 容器——請務必使**https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp**+**GLM-5.3** 屬於同一家族,使用同一引擎載入。它有自己的 group-scaled(gs64)容器,+約 **419 GB**,且**不含** MTP head,因此推測解碼保持關閉:++**https://huggingface.co/Justvugg/GLM-5.3-colibri-int4-g64**+> ⚠️ 請使用上面的 **gs64** 容器,不要使用較舊的 per-row int4 鏡像>(`mateogrgic/…`、`jlnsrk/…`):後者品質實測低約 9 個百分點,也是> [#455](https://github.com/JustVugg/colibri/issues/455) 最初 think-mode 迴圈與生成不終止的根因。diff --git a/docs/quickstart.md b/docs/quickstart.mdindex d0ea21d7..4e183b6a 100644--- a/docs/quickstart.md+++ b/docs/quickstart.md@@ -146,6 +146,12 @@ A pre-converted **GLM-5.2 int4** model is on Hugging Face. Use the**https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp**+**GLM-5.3** is the same family and loads with the same engine. Its own+group-scaled (gs64) container is about **419 GB** and ships **without** the MTP+head, so speculative decoding stays off:++**https://huggingface.co/Justvugg/GLM-5.3-colibri-int4-g64**+Group scales matter: the older per-row int4 containers(`mateogrgic/…-int4-with-int8-mtp`, `jlnsrk/…`) measure ~9pp worse on qualitybenchmarks and are the root cause of the think-mode loops and never-terminating