Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

release: 1.11.0

JustVuggc6ae5d4788ce
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 675dd189..249343e9 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -3,6 +3,150 @@
All notable changes to colibrì are documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/).
+## [1.11.0] — 2026-09-13
+
+54 pull requests since v1.10.2. A ninth model family, five real bugs closed
+across four engines, and the two platforms the C tests never built on now
+building them in CI.
+
+### A ninth engine: DeepSeek V4.1 Flash
+
+- **#1453**: DeepSeek V4.1 Flash (552B, 510 GB on disk) runs on a CPU box
+ streaming experts from an SSD, with no conversion: the released checkpoint
+ is read natively, fp8 dense with 32x32 ue8m0 tiles and fp4 experts whose
+ layout is byte-identical to the mxfp4 the Kimi K3 engine already reads.
+ Everything in the architecture that is not V4 is in: Engram (two n-gram
+ memories of 384M rows, 203 GB, that never enter RAM), the DSA indexer with
+ its two-level candidate source, hyper-connections, the compressor, the
+ 32-layer vision tower with its aligner, DSpark speculative decoding (3
+ stages, blocks of 5, verified in one batched forward with a rollback for
+ the rejected rows) and tool calling in the checkpoint's own DSML format.
+- Held token-exact in CI against a torch-only CPU reference
+ (`c/tools/dsv41_ref.py`, written because the vendor's own forward needs
+ tilelang GPU kernels), at three cache capacities, on a short and a
+ 40-token prompt, under all three speculative modes; the vision tower is
+ matched to 5e-06. The CI also asserts that the vendor's index-key
+ republication policy and the tidy reading disagree, so the shipped default
+ cannot be "corrected" by accident (`V41_INDEX_OWNER`).
+- Measured on the released checkpoint, cold, caches dropped before every run,
+ same prompt and seed: a turn went from 78.7 s to 25.1 s during the work
+ (0.305 to 0.957 tok/s) through batched expert reads (`V41_READ_DEPTH`,
+ default 8 is the measured knee), attention matrices read once per block of
+ positions instead of once per token, and an expert-major MoE. Every step
+ is bit-exact against what it replaced. A five-turn chat session runs at
+ 1.14 to 1.58 tok/s; an `image_url` part goes through the gateway into the
+ vision tower on the same engine.
+- Two ways of hiding the expert reads behind the matmuls were built,
+ measured worse, and removed; both are written up in `docs/deepseek-v41.md`
+ with the numbers, so the next attempt starts from them.
+- `coli chat`, `coli serve` and `coli web` work; `coli run` is deliberately
+ unwired for this family, as for qwen36 and qwen38. Per-turn accounting is
+ behind `V41_STATS`, off by default, so the chat stays clean.
+
+### Fixed
+
+- **#1390** (@crichalchemist): the qwen36 GPU tier's `qt_fill_wait` returned
+ when the upload queue was empty, but the uploader frees the ring slot at
+ dequeue, before the backend copies anything, so the warmstart freed the RAM
+ int8 copy of experts still in flight. An in-flight count that the uploader
+ decrements only once the slot is resident (#1360).
+- **#1434** (@njloof): `fmt=0` is raw f32 and has no scale array, but the
+ Metal sizing helper returned a per-row scale size for it, so
+ `coli_metal_matmul` wrapped a NULL pointer and segfaulted inside
+ `newBufferWithBytes`, or silently produced all-zero output when the size
+ happened to be a page multiple. The kernel no longer applies a scale for
+ `fmt=0`, and a fail-closed guard sized off the same helper falls back to
+ the CPU path for any format that needs scales and got none.
+- **#1389** (@bherald): GLM-5.3 teardown leaked the lazily allocated dashboard
+ HITS table.
+- **#1461**: `coli cluster worker` could never start without an explicit
+ `--cap`: the default is `None`, and `str(None)` reached the engine's
+ argument check as the literal `"None"` (#1452). The worker now resolves the
+ cap the way every other launcher does.
+- **#1460**: olmoe's serve-mode `PROF` line published five literal zeros, so
+ `/profile` reported `expert_disk_s = 0.0` on turns where the expert reads
+ were the workload. The disk phase is measured now; the other four fields
+ stay zero because they are unmeasured, not because a guess would look
+ better (#1449, half of it).
+- **#1445**: olmoe's `--ram` was inert and "no `--cap`" meant a constant eight
+ slots per layer regardless of memory; the cap is derived from the RAM budget
+ (#1443). On the reporter's box a 2,291-token prompt went from 675 s to 353 s
+ with no flags (#1442).
+- **#1377**: available RAM is measured on every platform (glm53 read
+ `/proc/meminfo` unconditionally, so on Windows and macOS it read 0 and the
+ expert budget collapsed to 1 GB, which is how `coli tune` went OOM in
+ #1375), and the Windows commit limit is respected.
+- **#1381**: the context ceiling is announced, a request that cannot be
+ honoured is refused up front, and the number reported is the right one
+ (#1376).
+- **#1393**: GLM-5.3 replies always start inside the think block (#1278).
+- **#1414** (@dajiaohuang): the GLM-5.3 serve loop honours `CANCEL` while the
+ turn is still running (#1332).
+- **#1423** (@kreuzzelg): the lazy HITS table is built privately and published
+ under a lock in qwen36, kimi_k3, glm53 and qwen38; the parallel warmstart
+ segfaulted on first touch about one run in twelve (#1422).
+- **#1404** (@Petsku01): the qwen36 tier offers int8 experts on the decode
+ path (#1391). **#1388** (@crichalchemist): `QT_MAX_ROWS` replaces seven
+ literal 32s that had to agree, and the int8 copy is freed by ownership
+ rather than by format.
+- **#1421** (@bherald): `coli` no longer overwrites an explicit
+ `CUDA_EXPERT_GB` when it decides to place the dense trunk on the card.
+- **#1244** (@Unknown-Findout): the CUDA expert tier is charged real VRAM, not
+ logical bytes (#687). **#1394**: the VRAM prefix is priced per row, not at
+ the container's widest (#1351). **#1410**, **#1411**: on a single GPU the
+ VRAM prefix is sized from the VRAM budget and the measured headroom, not
+ from the RAM pin plan capped by the LRU reserve (#1409, #1405).
+- **#1447** (@trigger2k20): `coli mirror verify` accepts a complete mirror
+ copied by other means, with no receipt, and fails closed on a short or
+ corrupt shard.
+- **#1396** (@dmoraesrs): the web reasoning selector drops "Medium" for
+ GLM 5.3, which the engine renders identically to "High".
+- **#1428**, **#1435** (@iiEliJas), **#1433** and **#1432** (@texasich),
+ **#1440**, **#1459**, **#1458** (@trigger2k20), **#1463**: `setenv` visible
+ to `getenv` in-process on Windows, `malloc_trim` guarded to glibc so the
+ engines build on musl, a missing bench recipe restored, the per-test
+ `_putenv_s` helpers dropped, tests including `compat.h` directly, and
+ Clang warnings cleaned up.
+
+### Added
+
+- **#1462** (@trigger2k20): an opt-in Metal path for GLM-5.3-Flash's routed
+ experts on Apple Silicon (`COLI_METAL=1`): gate, up, clamped SwiGLU, down
+ and the route-weighted scatter on the GPU, CPU path unchanged when Metal
+ is off. Validated against the CPU implementation to 2.8e-09 and measured
+ on an M4 Max: 0.469 to 0.621 tok/s at 32 tokens, up to 2.15 tok/s with
+ the expert cache tuned. Also fixes `coli run --cap N` being silently
+ ignored on the glm53 one-shot path.
+- **#1454** (@yuripourre): the qwen36 GPU tier compiles with `HIP=1`.
+- **#1399** (@SebaWag): `TRUNK_RESIDENT_LAYERS=N` streams the dense trunk
+ through mmap (#826).
+- **#1361**, **#1374** (@kreuzzelg): the qwen36 dense trunk places itself
+ (`COLI_PLACE=auto`, priced in bytes saved per byte of VRAM), fp8 streaming
+ mode, VRAM accounting at allocator granularity, thread affinity, `COLI_GPU`.
+- **#1383**, **#1385**, **#1386**, **#1387**: Brain and Profile tabs (EMAP,
+ HITS, PROF) on every engine. **#1382** (@dmoraesrs): a reasoning depth
+ selector in the web UI (#1311).
+- **#1407**, **#1408**, **#1406**: CI builds the portable Windows CUDA DLL and
+ publishes it as an artifact; the Makefile refuses the 32-bit `cl.exe` before
+ nvcc runs (#1405).
+- **#1436**: CI builds against musl (Alpine) and compiles the environment
+ tests on Windows, the two holes that let #1430 and #1420 ship.
+- **#1360** (@kreuzzelg): tier invariants on the fake backend, and the
+ reservation leak they found. **#1419**, **#1415**, **#1418**
+ (@dajiaohuang): the V4 and fp8 test harnesses build on Windows.
+- **#1439** (@bherald): the planner tests no longer materialise four 3 GB
+ zero-filled payloads. **#1358** (@monotophic): a raw-evidence adapter for
+ the scoring corpus. **#1238** (@ZacharyZcR): reproducible performance
+ records are validated.
+
+### Docs
+
+- **#1446**: the qwen36 `--ram` section said the engine does not stream at
+ all. **#1392**: Azure GLM-5.2 benchmark rows (#1379, #1380, #1384), the rANS
+ cross-reference (#1273), and the int8 probe pinned in CI (#1331).
+- **#1427** (@ZH1995): README.zh-CN formatting. **#1438** (@iiEliJas): the
+ `malloc_trim` comment.
+
## [1.10.2] — 2026-09-06
Patch release. Three of these fixes answer reports made against 1.10.1 in the
diff --git a/README.it.md b/README.it.md
index c201e6cc..fbfb7f79 100644
--- a/README.it.md
+++ b/README.it.md
@@ -12,7 +12,7 @@ miliardi a 2,8 mila miliardi di parametri** — su hardware consumer ed eterogen
in C puro e senza dipendenze del motore, trattando storage, RAM e VRAM come
un'unica gerarchia di inferenza.
-Oggi girano otto famiglie: **GLM-5.2/5.3** (744B), **GLM-5.3-Flash** (321B, con
+Oggi girano nove famiglie: **GLM-5.2/5.3** (744B), **GLM-5.3-Flash** (321B, con
vision), **Inkling** (975B), **Kimi K3** (2,8T), **DeepSeek V4 Flash** (284B), **DeepSeek V4.1 Flash** (552B, con visione),
**Qwen3.8-Flash-Next** (125B + 51B n-gram), **Qwen3.6** (35B-A3B) e
**OLMoE** (7B) — un
@@ -34,7 +34,7 @@ ma non ridefinire il modello di nascosto.
```
$ ./coli chat
- 🐦 colibri v1.10.2 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.11.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
@@ -338,8 +338,8 @@ e per il gateway API opzionale.
end-to-end, revisionati e sviluppati apertamente.
- **Più modelli aperti.** L'algoritmo di tiering è indipendente dal modello:
qualsiasi MoE con expert instradati può essere organizzato allo stesso modo.
- Otto famiglie funzionano già (GLM-5.2, GLM-5.3-Flash con la vision,
- Inkling, Kimi K3, DeepSeek V4 Flash, Qwen3.8-Flash-Next, Qwen3.6, OLMoE);
+ Nove famiglie funzionano già (GLM-5.2, GLM-5.3-Flash con la vision,
+ Inkling, Kimi K3, DeepSeek V4 Flash, DeepSeek V4.1 Flash, Qwen3.8-Flash-Next, Qwen3.6, OLMoE);
altre famiglie open-weight, **MiniMax** tra le candidate, si guadagnano un
engine come le prime otto: quando qualcuno le misura end-to-end.
diff --git a/README.md b/README.md
index 4028cecd..7cb05c9c 100644
--- a/README.md
+++ b/README.md
@@ -18,7 +18,7 @@ parameters** — on consumer and heterogeneous hardware, in pure C with zero
engine dependencies, by treating storage, RAM, and VRAM as a single inference
hierarchy (AI memory multitiering).
-Eight families run today: **GLM-5.2/5.3** (744B), **GLM-5.3-Flash** (321B, with
+Nine families run today: **GLM-5.2/5.3** (744B), **GLM-5.3-Flash** (321B, with
vision), **Inkling** (975B), **Kimi K3** (2.8T), **DeepSeek V4 Flash** (284B), **DeepSeek V4.1 Flash** (552B, with vision),
**Qwen3.8-Flash-Next** (125B + 51B n-gram), **Qwen3.6** (35B-A3B) and
**OLMoE** (7B) —
@@ -40,7 +40,7 @@ may reduce speed; it must not quietly redefine the model.
```
$ ./coli chat
- 🐦 colibri v1.10.2 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.11.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
@@ -584,8 +584,8 @@ checkpoint validation, and the generated tiny independent oracle.
lower cost per useful token. Everything lands the way this project works:
measured end to end, reviewed, and developed in the open.
- **More open models.** The tiering algorithm is model-agnostic: any MoE with
- routed experts can be staged the same way. Eight families run today (GLM-5.2,
- GLM-5.3-Flash, Inkling, Kimi K3, DeepSeek V4 Flash, Qwen3.8-Flash-Next,
+ routed experts can be staged the same way. Nine families run today (GLM-5.2,
+ GLM-5.3-Flash, Inkling, Kimi K3, DeepSeek V4 Flash, DeepSeek V4.1 Flash, Qwen3.8-Flash-Next,
Qwen3.6, OLMoE); further open-weight families — **MiniMax** among the
candidates — earn an engine the way the first eight did: when someone
measures one end to end.
diff --git a/README.zh-CN.md b/README.zh-CN.md
index c15773bd..ef32c684 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -10,7 +10,7 @@
**小巧引擎,庞大模型。**在消费级与异构硬件上运行**前沿 MoE 模型——从 744B 到
2.8T 参数**——以引擎零依赖的纯 C 实现,将存储、RAM 与 VRAM 视为统一的推理层级。
-目前可运行八个模型家族:**GLM-5.2/5.3**(744B)、**GLM-5.3-Flash**(321B,含视觉)、
+目前可运行九个模型家族:**GLM-5.2/5.3**(744B)、**GLM-5.3-Flash**(321B,含视觉)、
**Inkling**(975B)、**Kimi K3**(2.8T)、**DeepSeek V4 Flash**(284B)、**DeepSeek V4.1 Flash**(552B,含视觉)、
**Qwen3.8-Flash-Next**(125B + 51B n-gram)、**Qwen3.6**(35B-A3B)与 **OLMoE**(7B)
——各自一个 C 文件,共用同一套 `coli chat` / `coli serve` / `coli web` 前端。[完整列表](README.md#other-supported-models)
@@ -25,7 +25,7 @@ Colibrì 刻意用于验证激进的系统思路——因此**对速度不作 SL
```
$ ./coli chat
- 🐦 colibri v1.10.2 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.11.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
@@ -311,7 +311,7 @@ oracle 说明,请参阅[中文版 DeepSeek V4 文档](docs/deepseek-v4.zh-CN.m
放置、调度、I/O、CPU/GPU 内核、异构重叠、KV 状态与路由感知推测。目标是降低硬件要求
和每个有效 token 的成本,所有成果都以端到端测量为准、经审查并公开开发。
- **支持更多开放模型**。层级算法与模型无关,任何带路由专家的 MoE 都能用相同方式分层。
- 目前已有八个模型家族可用(GLM-5.2、GLM-5.3-Flash、Inkling、Kimi K3、DeepSeek V4 Flash、
+ 目前已有九个模型家族可用(GLM-5.2、GLM-5.3-Flash、Inkling、Kimi K3、DeepSeek V4 Flash、DeepSeek V4.1 Flash、
Qwen3.8-Flash-Next、Qwen3.6、OLMoE);更多开放权重家族(候选包括 **MiniMax**)将沿用同样的
规则获得引擎支持:有人完成端到端实测之后。
diff --git a/README.zh-TW.md b/README.zh-TW.md
index f397e739..8211a0b1 100644
--- a/README.zh-TW.md
+++ b/README.zh-TW.md
@@ -10,7 +10,7 @@
**小巧引擎,龐大模型。**在消費級與異質硬體上執行**前沿 MoE 模型——從 744B 到
2.8T 參數**——以引擎零相依套件的純 C 實作,將儲存、RAM 與 VRAM 視為統一的推論階層。
-目前可執行八個模型家族:**GLM-5.2/5.3**(744B)、**GLM-5.3-Flash**(321B,含視覺)、
+目前可執行九個模型家族:**GLM-5.2/5.3**(744B)、**GLM-5.3-Flash**(321B,含視覺)、
**Inkling**(975B)、**Kimi K3**(2.8T)、**DeepSeek V4 Flash**(284B)、**DeepSeek V4.1 Flash**(552B,含視覺)、
**Qwen3.8-Flash-Next**(125B + 51B n-gram)、**Qwen3.6**(35B-A3B)與 **OLMoE**(7B)
——各自一個 C 檔案,共用同一套 `coli chat` / `coli serve` / `coli web` 前端。[完整清單](README.md#other-supported-models)
@@ -25,7 +25,7 @@ Colibrì 刻意用於驗證激進的系統構想——因此**對速度不作 SL
```
$ ./coli chat
- 🐦 colibri v1.10.2 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.11.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
@@ -291,7 +291,7 @@ COLI_MODEL=/nvme/glm52_i4 ./coli doctor # 唯讀就緒檢查
配置、排程、I/O、CPU/GPU 核心、異質重疊、KV 狀態與路由感知推測。目標是降低硬體要求
和每個有效 token 的成本,所有成果都以端到端測量為準、經審查並公開開發。
- **支援更多開放模型。**階層演算法與模型無關,任何帶路由專家的 MoE 都能用相同方式分層。
- 目前已有八個模型家族可用(GLM-5.2、GLM-5.3-Flash、Inkling、Kimi K3、DeepSeek V4 Flash、
+ 目前已有九個模型家族可用(GLM-5.2、GLM-5.3-Flash、Inkling、Kimi K3、DeepSeek V4 Flash、DeepSeek V4.1 Flash、
Qwen3.8-Flash-Next、Qwen3.6、OLMoE);更多開放權重家族(候選包括 **MiniMax**)將沿用同樣的
規則獲得引擎支援:有人完成端到端實測之後。
diff --git a/c/version.py b/c/version.py
index 06573da8..fe23c815 100644
--- a/c/version.py
+++ b/c/version.py
@@ -1,3 +1,3 @@
"""Single source of truth for the colibri version number."""
-__version__ = "1.10.2"
+__version__ = "1.11.0"
diff --git a/site/index.html b/site/index.html
index 10f85750..e8e5b55a 100644
--- a/site/index.html
+++ b/site/index.html
@@ -4,9 +4,9 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>colibrì — run models bigger than your machine</title>
-<meta name="description" content="An inference engine in pure C that streams Mixture-of-Experts weights from disk. A 2.8-trillion-parameter model runs on a 32 GB box. Six model families, zero dependencies, CPU or GPU.">
+<meta name="description" content="An inference engine in pure C that streams Mixture-of-Experts weights from disk. A 2.8-trillion-parameter model runs on a 32 GB box. Nine model families, zero dependencies, CPU or GPU.">
<meta property="og:title" content="colibrì — run models bigger than your machine">
-<meta property="og:description" content="Six frontier MoE families, up to 2.8T parameters, on consumer hardware: pure C, zero dependencies, experts tiered across VRAM, RAM and disk. Run it, study it, improve it.">
+<meta property="og:description" content="Nine frontier MoE families, up to 2.8T parameters, on consumer hardware: pure C, zero dependencies, experts tiered across VRAM, RAM and disk. Run it, study it, improve it.">
<meta property="og:type" content="website">
<meta property="og:image" content="https://raw.githubusercontent.com/JustVugg/colibri/main/docs/media/colibri-atlas.png">
<link rel="icon" type="image/svg+xml" href="colibri-icon.svg">
@@ -229,7 +229,7 @@ footer li a:hover{color:var(--on-dark)}
<div><div class="n">510</div><div class="l">pull requests merged</div></div>
<div><div class="n">6</div><div class="l">families · 4 backends · 3 platforms</div></div>
<div class="since">All of it in <b data-gh-age>six weeks</b>, in the open: issues filed from strangers' machines,
- reproduced and fixed the same day. Currently shipping <b>v1.10.2</b>.</div>
+ reproduced and fixed the same day. Currently shipping <b>v1.11.0</b>.</div>
</div>
</section>