Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
CHANGELOG 1.12.1: eight more pull requests
JustVugg5e11f4649cc0
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex db8a9198..4f1317fb 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,7 +5,7 @@ Format follows [Keep a Changelog](https://keepachangelog.com/).## [1.12.1] — 2026-09-22-87 pull requests since v1.12.0, 75 of them from contributors. Two tokenizers+95 pull requests since v1.12.0, 82 of them from contributors. Two tokenizersbrought back to the reference, brio on the ninth engine, `coli chat` workingagain at the default context on two families, and a placement decision thatis now measured on the card in front of it instead of predicted.@@ -130,6 +130,16 @@ is now measured on the card in front of it instead of predicted.int8 down alone recovers a quarter of the gap between gs64 and all-int8,the rest sits in gate/up. A measurement tool and a middle step, not theanswer to the gap.+- **#1730** (mfethe1): DeepSeek V4's FP4 expert kernels, the prefill batch+ and the decode matvec, get a NEON arm; arm64 used to take the scalar+ arm, which is why Apple Silicon prefilled at decode speed (#1696). Bit+ identical to the scalar arm, and the ARM CI job now checks that on every+ change; 17 to 22x on the kernel at the V4 expert shapes on an M-series+ Mac, as measured by the author.+- **#1716** (jtinbergen): qwen36 quantizes its dense weights to int8 while+ loading instead of keeping an f32 copy first, and converts f16/bf16 with+ SIMD. On the 35B the resident set after load goes from 9.2 to 4.8 GB;+ the generated text and the perplexity are identical to before.- **#1286** (cameron): the grouped int4 GEMV and the fused gate/up GEMV getan SSE4.1 arm for CPUs without AVX2 (Ivy Bridge and older). It vectorizesacross output rows, so each lane runs the scalar row's exact sequence and@@ -218,6 +228,20 @@ is now measured on the card in front of it instead of predicted.gateway and the whole answer came back as `reasoning_content`. Thenon-special added tokens are decoded now; special ones such as`<|im_start|>` still decode to nothing.+- **#1734** (tarazum): stopping `coli serve` closes the engine's stdin and+ waits for it to exit on its own before the hard-stop ladder, so the+ engine's teardown runs; qwen36 never saved its `HEAT_FILE` under `coli+ serve` (#1733). On Windows the gateway handles SIGBREAK and the engine+ runs in its own process group.+- **#1726** (kevin9327): Inkling measured no RAM on Windows and sized its+ expert cache to 16 per layer; it uses the shared probe now, which on+ Linux and macOS reads the same numbers as before.+- **#1735**: on GNU Make 3.81, the system make on macOS, `.build-config`+ was never written and every build relinked (#1732).+- **#1731** (bokiko): the DeepSeek V4 CUDA object rebuilds when the nvcc+ command changes, so a new `CUDA_ARCH` no longer links the old object.+- **#1728** (crichalchemist): `make test-c VK=1` built 43 test binaries+ without the Vulkan object; they link it now.- **#1712** (kevin9327): Kimi K3, Inkling and OLMoE now treat `max_tokens`as a ceiling like the other engines; `coli chat`'s default of 16384answered 400 on every Kimi and Inkling message against their 8192-token@@ -301,6 +325,10 @@ is now measured on the card in front of it instead of predicted.dependency. The admission scheduler distinguishes completion, failureand cancellation, lets a request use a free slot that no earlier waiterreserved, and joins the keepalive pump before the slot is released.+- **#1717** (enitimeago): the web chat offers Continue on the last+ assistant message when it stopped at the token limit, by hand or on an+ error, and only when `/health` says the server continues assistant+ turns (#1699).- **#1402** (enitimeago): a request whose last message is a non-empty`assistant` turn continues that turn instead of answering in a new one, on`/v1/chat/completions` and `/v1/messages`, for all nine families (Kimi K3