Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

release: 1.12.1

JustVuggda7bc4f44730
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 39d9bb45..a9847b86 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -3,6 +3,123 @@
All notable changes to colibrì are documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/).
+## [1.12.1] — 2026-09-21
+
+30 pull requests since v1.12.0, 22 of them from contributors. Two tokenizers
+brought back to the reference, brio on the ninth engine, `coli chat` working
+again at the default context on two families, and a placement decision that
+is now measured on the card in front of it instead of predicted.
+
+### Tokenizers, measured against the reference
+
+- **#1654**: qwen36 tokenized differently from HF `tokenizers` in two ways.
+ An added token right after punctuation was encoded as text (`X.<|im_end|>`
+ was 7 tokens instead of 3, every chat turn ending in punctuation paid +4,
+ #1653), and a whitespace run followed by a non-space was one piece where
+ the regex's `\s+(?!\S)` leaves the last char to the next one, so every
+ indented line of code tokenized differently. Measured on the real
+ vocabulary: 2,803 lines and blocks of code, Markdown, Chinese and
+ Japanese went from 757 identical to 2,803, with 5.4% fewer tokens.
+- **#1656**: OLMoE's `tokenizer.json` has no Split, a bare ByteLevel with
+ `use_regex`, for which HF runs the original GPT-2 pattern; `tok.h` applied
+ cl100k. A GPT-2 family in `tok.h`: 1,560/1,708 identical before, 1,708/1,708
+ after. The same measurement on GLM-5.2/5.3/5.3-Flash, DeepSeek V4 and V4.1,
+ Inkling and Qwen3.8 came back identical on every case.
+
+### Brio and the serve contract
+
+- **#1655**: the DeepSeek V4 engine speaks the numeric channel (`logprobs=k`,
+ `pin=1`, `max_tokens=0`), so `/v1/brio` works on the ninth engine instead
+ of answering 500 (#1648). The head that used to keep only its argmax now
+ returns the whole row; `ECHO` per prompt position during prefill, the
+ prompt-end scores kept with a state snapshot on `pin`, and a logprob tail
+ on every `DATA` frame during generation. The tiny fixture pins that the
+ best `ECHO` token equals the greedy token from the same prefix, and that
+ the pinned predictor equals a cold prefill's.
+- **#1659**: qwen36 and qwen38 refused a request when `prompt + max_tokens`
+ exceeded the context, and the gateway's default budget for these two
+ families is 8192, the whole default context: every request without
+ `max_tokens` and every `coli chat` message answered 400 on a two-token
+ prompt (#1641). `max_tokens` is now a ceiling, clamped to the room the
+ prompt leaves, as GLM and DeepSeek V4 already did; only a prompt that does
+ not fit is refused. `docs/api.md` states the rule, `docs/qwen38.md` names
+ `Q38_MAXT` as the variable `--ctx` becomes.
+
+### The dense trunk in VRAM, measured before it is placed
+
+- **#1657**: qwen36 offers the rest of its dense trunk to the VRAM placer:
+ the DeltaNet out_proj (`dnout`), the attention q/k/v/o (`attnproj`) and
+ the shared expert (`shexp`), about 650 MB more of int8 on the 35B beside
+ `lmhead` and `dnproj`. Measured on four Tesla M10 by the reporter of
+ #1652, every placed component ran slower than the CPU (lm_head 68.8 ms
+ against 41.7), so the engine now times one GEMV both ways at startup and
+ withdraws the whole automatic placement when the GPU loses, giving the
+ VRAM back to the experts: `auto` equals `off` on that box, byte-identical
+ output. A hand-written `COLI_PLACE` stands; `COLI_TRUNK_PROBE=0` trusts
+ the placer.
+
+### Performance, from contributors
+
+- **#1606**: the K1b grouped int4 family gets a multi-row tile and AVX-512
+ and AMX arms, and is no longer switched off on AVX-512 builds; exact on
+ all eight engines on a 16-core AVX-512 host.
+- **#1239**: an SSE4.1 tier for the olmoe and qwen36 int8 GEMV, for hosts
+ with SSE4.1 but no AVX2; on the Sandy Bridge of #1652 decode went from
+ 2.45 to 3.54 tok/s.
+- **#1313**: `matmul_fp8` computes four output rows per pass under clang,
+ where the contraction makes it bit-exact; GCC keeps the one-row kernel.
+- **#1612**: qwen36 gains the GLM engine's `CACHE_ROUTE` lever, with the
+ VRAM tier as the first residency level, opt-in.
+- **#906**: `DEGRADE_ZERO`, an opt-in policy that zero-fills a missed
+ expert slot below a gate-weight threshold instead of blocking on the
+ load (#865).
+
+### Fixed
+
+- **#1626**: `SNAP` is the model directory for every non-GLM engine, so
+ `coli run` stops handing them a leftover environment (#1600).
+- **#1604**: glm53 honours `Mat.resident` in the Vulkan gate and frees the
+ Vulkan handle in `mat_release`.
+- **#1321**: glm53 sizes its expert cache around the model rather than
+ around `MemAvailable`, which the page cache had been inflating.
+- **#1588**: qwen36 refuses loudly on a failed encode-buffer realloc instead
+ of writing through NULL.
+- **#1546**: `coli convert` routes OLMoE to `convert_olmoe_merged.py`.
+- **#1630**: olmoe emits the `ROUTE_TRACE` records it announced; the stream
+ used to be a zero-byte file.
+- **#1610**: `v41_dsml.py` is staged during installation (and the nix flake
+ bumped).
+- **#1511**: the GPU test suite builds under HIP on gfx1151.
+- **#1658**: `test_mem_available` compared two reads of available memory
+ with `==` and failed on a busy Windows runner; a quarter of a GB of
+ tolerance.
+
+### Tools and the gateway
+
+- **#1425**: a general GGUF reader, pure stdlib, and a converter from GGUF
+ OLMoE checkpoints to a colibri container, with the numerical evidence in
+ its own CI job.
+- **#1497**: opt-in prompt-injected tool calling for the families without
+ native tool tokens (OLMoE, Qwen3.6), behind `COLI_TOOL_FALLBACK=1`, with a
+ two-turn end-to-end test.
+- **#1355**, **#1357**: durable per-request results and strict stdout
+ classification in the eval harness, and the logprob-gap check gated on
+ the engine preamble.
+
+### Docs
+
+- **#1639**, **#1644**: the README shows brio mode and the dashboard as it
+ is: the workspace, the Brain page (the measured expert atlas as a cortex,
+ and a region inside it) and the Profiling page, in four languages;
+ `docs/api.md` describes the four pages instead of the old console.
+- **#1492**, **#1643**: a Japanese README, and its banner at the shipping
+ version, which the banner test now checks in every language.
+- **#1634**: the multi-disk guide states measured gains and limits instead
+ of "twice the bandwidth", with Bash and PowerShell examples.
+- **#1617**: connecting the pi coding agent to `coli serve`.
+- **#1619**: `expected_bytes` identity versus physical extent for
+ int4-rans256-g0 (#1273).
+
## [1.12.0] — 2026-09-20
81 pull requests since v1.11.0. A new way to ask a model a closed question,
diff --git a/README.it.md b/README.it.md
index fa66b2a7..86dbed61 100644
--- a/README.it.md
+++ b/README.it.md
@@ -34,7 +34,7 @@ ma non ridefinire il modello di nascosto.
```
$ ./coli chat
- 🐦 colibri v1.12.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.12.1 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
diff --git a/README.ja.md b/README.ja.md
index 82ebd90b..9f4bbdca 100644
--- a/README.ja.md
+++ b/README.ja.md
@@ -39,7 +39,7 @@ Colibrì は VRAM・RAM・ストレージを単一のマルチティア階層と
```
$ ./coli chat
- 🐦 colibri v1.12.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.12.1 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
diff --git a/README.md b/README.md
index 3dae7bc7..32b1f37a 100644
--- a/README.md
+++ b/README.md
@@ -40,7 +40,7 @@ may reduce speed; it must not quietly redefine the model.
```
$ ./coli chat
- 🐦 colibri v1.12.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.12.1 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
diff --git a/README.zh-CN.md b/README.zh-CN.md
index c25436c6..7f9a9420 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -25,7 +25,7 @@ Colibrì 刻意用于验证激进的系统思路——因此**对速度不作 SL
```
$ ./coli chat
- 🐦 colibri v1.12.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.12.1 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
diff --git a/README.zh-TW.md b/README.zh-TW.md
index 05e8b7ba..c208ac41 100644
--- a/README.zh-TW.md
+++ b/README.zh-TW.md
@@ -25,7 +25,7 @@ Colibrì 刻意用於驗證激進的系統構想——因此**對速度不作 SL
```
$ ./coli chat
- 🐦 colibri v1.12.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
+ 🐦 colibri v1.12.1 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
diff --git a/c/version.py b/c/version.py
index 58ab5498..db24fa2d 100644
--- a/c/version.py
+++ b/c/version.py
@@ -1,3 +1,3 @@
"""Single source of truth for the colibri version number."""
-__version__ = "1.12.0"
+__version__ = "1.12.1"
diff --git a/site/index.html b/site/index.html
index 292ba59c..ed5bc742 100644
--- a/site/index.html
+++ b/site/index.html
@@ -488,7 +488,7 @@ footer{border-top:1px solid var(--line)}
<footer><div class="fin">
<span>&copy; 2026 colibri</span>
- <span>Currently shipping <b>v1.12.0</b></span>
+ <span>Currently shipping <b>v1.12.1</b></span>
<span class="sp">
<a href="https://github.com/JustVugg/colibri/">GitHub</a><a href="https://github.com/JustVugg/colibri/blob/main/docs/quickstart.md">Docs</a>
<a href="https://github.com/JustVugg/colibri/blob/main/docs/brio.md">Brio</a><a href="https://github.com/JustVugg/colibri/blob/main/docs/api.md">API</a>