Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

docs: the shipped expert profiles, and what they do to a first run

JustVuggd1e29696b731
diff --git a/docs/qwen36.md b/docs/qwen36.md
index 3a31475b..7e1af17a 100644
--- a/docs/qwen36.md
+++ b/docs/qwen36.md
@@ -95,7 +95,8 @@ trunk on the CPU anywhere and `COLI_VK_DENSE=1` puts it on the device anywhere;
This engine kept no expert history before; with the tier on it keeps route_trace.h's
`.coli_usage` beside the container (`COLI_USAGE` moves it), saved at the end of
-every run and serve turn, and fills the tier from it at the next start. With
+every run and serve turn, and fills the tier from it at the next start. `coli
+setup` gives Qwen3.6-35B-A3B a starting one ([vulkan.md](vulkan.md#the-routed-expert-tier-vk_tierc)). With
`COLI_CUDA=1` as well, the CUDA tier wins.
On an integrated Radeon 780M, with the int4 gs64 container at cap 64 and the trunk
diff --git a/docs/qwen38.md b/docs/qwen38.md
index e377ef66..1ad0e73e 100644
--- a/docs/qwen38.md
+++ b/docs/qwen38.md
@@ -367,7 +367,8 @@ alternating.
In a `VK=1` build, `COLI_VULKAN=1` puts the trunk on the Vulkan device (as before)
and the routed experts of the model's layers on the shared Vulkan expert tier
(`c/vk_tier.c`, [vulkan.md](vulkan.md#the-routed-expert-tier-vk_tierc)): a cache of
-experts in device memory, filled at startup from `.coli_usage` and adapted while
+experts in device memory, filled at startup from `.coli_usage` (`coli setup` ships a
+starting one) and adapted while
you chat, whose experts the device computes while the CPU computes the rest of the
step. It takes every expert form this engine reads: the int4-g64 sidecar's planar
records (as int4 groups of 64 on the device), the release's FP8 with the 128x128
diff --git a/docs/vulkan.md b/docs/vulkan.md
index a825f173..6127d7ad 100644
--- a/docs/vulkan.md
+++ b/docs/vulkan.md
@@ -396,6 +396,17 @@ of them on the device the way a GPU-equipped PC should use its card:
than 1.25 x their heat + 4). A promotion copies the expert's bytes once on the engine thread
(at most `COLI_VK_TIER_RATE` per token, 16) and an uploader thread writes it to
the device; it serves from the next layer step on.
+- **A fresh install starts warm.** A model downloaded a minute ago has no history,
+ so its first runs would fill the tier one routing at a time. `coli setup` copies a
+ starting history into the model folder when it has none, for the catalog models
+ that ship one (`c/profiles/<catalog id>.coli_usage`: Qwen3.6-35B-A3B and Qwen3.8
+ Flash Next). It comes from a calibration session (16 prompts: chat in five
+ languages, code, reasoning, JSON), and the engine adds every run's routing to it,
+ so your own use takes over; an existing history is never replaced. Measured on a
+ Radeon 780M with Qwen3.6-35B-A3B, two prompts outside the calibration set, 128
+ tokens, two rounds each: the first run went from 5.9-6.2 tok/s without it to
+ 10.9-11.1 with it, the device serving 92-94% of the routed experts instead of
+ 15-16%, for a warm start of about 3 s.
- **The device and the CPU compute at the same time.** For each MoE layer step the
routed (row, expert) pairs whose expert is resident go to the device as ONE
submit that nobody waits for: per expert, its rows run gate+up and the activation