Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Merge MiMo's dense chain into the inkling and olmoe chain

JustVugg9e3f6dcd69e7
diff --cc c/tests/vulkan_engines.sh
index 5334b72f,45336929..a3d11f16
--- a/c/tests/vulkan_engines.sh
+++ b/c/tests/vulkan_engines.sh
@@@ -7,7 -7,7 +7,8 @@@
# bash tests/vulkan_engines.sh glm | glm-sanitize # GLM-5.2 (colibri) and GLM-5.3 Flash (glm53)
# bash tests/vulkan_engines.sh shader # the qmatmul formats, the expert batch and the tier, no engine
# bash tests/vulkan_engines.sh qwen-chain | qwen-chain-sanitize # the dense chain (COLI_VK_CHAIN=1)
+ # bash tests/vulkan_engines.sh mimo-chain | mimo-chain-sanitize # MiMo-V2.6's dense chain
+# bash tests/vulkan_engines.sh inkling-olmoe-chain | inkling-olmoe-chain-sanitize
#
# Needs libvulkan-dev, glslc and mesa-vulkan-drivers, plus the Python packages of
# the family's tiny fixtures (see the vulkan-engines job in .github/workflows/ci.yml).
@@@ -1552,7 -1514,5 +1711,7 @@@ case "${1:-}" i
glm-sanitize) family_glm_sanitize ;;
qwen-chain) family_qwen_chain ;;
qwen-chain-sanitize) family_qwen_chain_sanitize ;;
- *) echo "usage: $0 shader|qwen|qwen-sanitize|inkling-olmoe|inkling-olmoe-sanitize|mimo-qwenimage|deepseek|deepseek-sanitize|kimi|kimi-mimo-sanitize|glm|glm-sanitize|qwen-chain|qwen-chain-sanitize|mimo-chain|mimo-chain-sanitize" >&2; exit 2 ;;
+ inkling-olmoe-chain) family_inkling_olmoe_chain ;;
+ inkling-olmoe-chain-sanitize) family_inkling_olmoe_chain_sanitize ;;
- *) echo "usage: $0 shader|qwen|qwen-sanitize|inkling-olmoe|inkling-olmoe-sanitize|mimo-qwenimage|deepseek|deepseek-sanitize|kimi|kimi-mimo-sanitize|glm|glm-sanitize|qwen-chain|qwen-chain-sanitize|inkling-olmoe-chain|inkling-olmoe-chain-sanitize" >&2; exit 2 ;;
++ *) echo "usage: $0 shader|qwen|qwen-sanitize|inkling-olmoe|inkling-olmoe-sanitize|mimo-qwenimage|deepseek|deepseek-sanitize|kimi|kimi-mimo-sanitize|glm|glm-sanitize|qwen-chain|qwen-chain-sanitize|mimo-chain|mimo-chain-sanitize|inkling-olmoe-chain|inkling-olmoe-chain-sanitize" >&2; exit 2 ;;
esac
diff --cc docs/ENVIRONMENT.md
index 3f582522,5ba2fe32..b2b584ef
--- a/docs/ENVIRONMENT.md
+++ b/docs/ENVIRONMENT.md
@@@ -245,7 -245,7 +245,7 @@@ With `COLI_VULKAN=1` every MoE engine (
| `COLI_VK_TIER_GEMM_ROWS` | `16` | Rows from which an expert of a step takes the tiled GEMM (prefill) instead of the per-row GEMV; `0` never. Below it a row's bits do not depend on how many rows share the dispatch. |
| `COLI_VK_TIER_QUEUE` | a second queue | `0`: the tier's batches share the main queue with the dense matmuls (they then serialize). Unset: a second queue of the main family, else a compute-only family's (RADV), else shared (Lavapipe). |
| `COLI_VK_DENSE` | on; off on a device sharing the CPU's RAM while the tier is on | Where the dense (non-expert) matrices of qwen36, qwen38, inkling, olmoe, kimi_k3 (its shared experts), mimo, deepseek_v41, deepseek_v4, glm53 and qwenimage run. `0`: on the CPU (with the tier on, the device takes the routed experts only); `1`: on the device whatever the device. Unset: on the device, except on an integrated GPU or a CPU device (Lavapipe) while the engine runs the expert tier (every one of them but qwenimage, which has no routed experts), where they stay on the CPU: there the dense matmuls cost more than the tier gains (measured on a Radeon 780M). The `[VK] <engine>: device ready, dense matrices on the ...` line says which and why. The GLM engine reads it through the same rule with its own default, off (see above). |
- | `COLI_VK_CHAIN` | on for a discrete GPU; qwen36 on, qwen38 off, olmoe on, inkling off (not measured) on an integrated GPU with the tier; off on a CPU device | qwen36, qwen38, olmoe and inkling: every layer's dense chain recorded into one submission per layer, the residual stream and the recurrent and KV state on the device (see [vulkan.md](vulkan.md#the-dense-chain-vk_chainc)). `0` off, `1` on, `2` on for prompts only (decode on the CPU). The `[VK] <engine>: dense chain ...` line says which and why. |
-| `COLI_VK_CHAIN` | on for a discrete GPU; qwen36 on, qwen38 and mimo off on an integrated GPU with the tier; off on a CPU device | qwen36, qwen38 and mimo: every layer's dense chain recorded into one submission per layer, the residual stream and the recurrent and KV state on the device (see [vulkan.md](vulkan.md#the-dense-chain-vk_chainc)). `0` off, `1` on, `2` on for prompts only (decode on the CPU). The `[VK] <engine>: dense chain ...` line says which and why. |
++| `COLI_VK_CHAIN` | on for a discrete GPU; qwen36 and olmoe on, qwen38 off, mimo and inkling off (not measured) on an integrated GPU with the tier; off on a CPU device | qwen36, qwen38, olmoe, inkling and mimo: every layer's dense chain recorded into one submission per layer, the residual stream and the recurrent and KV state on the device (see [vulkan.md](vulkan.md#the-dense-chain-vk_chainc)). `0` off, `1` on, `2` on for prompts only (decode on the CPU). The `[VK] <engine>: dense chain ...` line says which and why. |
| `COLI_VK_CHAIN_ROWS` | `512` | Prompt rows per chunk of the chain's prefill. |
| `COLI_VK_CHAIN_GEMV` | on | `0`: the chain's decode matrices take `qmatmul.comp`'s GEMV instead of `chain_gemv.comp`'s. |
| `COLI_VK_CHAIN_SPIN_US` | `2000` | How long a wait on a chain frame polls before blocking. |
diff --cc docs/vulkan.md
index cd894397,9fe68932..41830dd3
--- a/docs/vulkan.md
+++ b/docs/vulkan.md
@@@ -826,128 -826,74 +826,196 @@@ prediction of it. Not timed either: con
selects blocks instead of attending to all of them), serve sessions, Qwen3-Coder and
Qwen3.8-27B (no checkpoints on the box); their correctness is the Lavapipe gates'.
+ ### MiMo-V2.6 on the chain
+
+ `mimo_chain.h` runs MiMo-V2.6's layers (Flash and Pro) as frames on the device, as
+ qwen36's chain does, with what this model has instead of a DeltaNet:
+
+ | | mimo |
+ |---|---|
+ | device, one frame per MoE layer | the previous layer's routed sum joining the residual (the CPU's `h += out`); the input RMSNorm; the fused qkv projection in its dense form (`MIMO_DENSE_BITS`: the release's FP8 with its 128-column block scales, int8 rows or f32); partial RoPE on q and k (rotate-half on the first `rope_dim` dims, from a host table of the CPU's own `cosf`/`sinf`, one per layer kind: the two kinds have their own theta); the value scale (`chain_ew` SCALE); the new K/V rows into the device's cache; attention with the layer's sliding window and its sink logits (`chain_attn.comp` with a window, a ring and a sink); o_proj; the residual add; the post-attention RMSNorm. The dense layer (layer 0 on the release) runs its MLP there too and has no host step |
+ | host | `moe()` as the CPU runs it, on the device's normalized rows: the router (sigmoid scores, the correction bias picks, top-k renormalized) and the routed experts (the tier's batch and the CPU's share, joined in each row's routing order). There is no shared expert, so nothing runs beside it |
+ | last frame | the final norm and lm_head on the last row, every row for a read-out (`MIMO_LOGITS`, logprobs) |
+
+ **The caches.** The host's stay canonical, in the layout the CPU keeps: a
+ full-attention layer `[ctx][kvh][hd]` (V `[ctx][kvh][vd]`), a windowed layer a ring of
+ `min(window, ctx)` rows with position `p` in row `p % rows`. The device mirrors each
+ one in exactly that layout (`chain_attn.comp` reads position-major rows and a ring) behind
+ a watermark per layer: positions below it (for a ring, the last `rows` of them) equal
+ the host's. A step lowers it to its first position and uploads what the window still
+ sees; a CPU step lowers it to its own; a restored photo (`pin_state_load` rewrites the
+ host's rings) drops the rings' watermarks to 0, so the next step uploads them again.
+ Prefix reuse needs nothing else.
+
+ **A windowed layer's attention.** One decode row reads the ring in place: its window
+ is exactly the ring's rows. A block of rows cannot (its later rows would overwrite slots
+ its earlier rows still see), so it gathers the window's earlier rows from the ring and
+ its own new rows into one position-major scratch, as the CPU's `attention()` copies
+ them, attends there, and writes its last `rows` rows into the ring. Both visit the
+ positions in the same order, so a row gets the same bits either way: with every matrix
+ on the per-row GEMV (`COLI_VK_GEMM_MIN_S=0`) and the tier off, the logits of every
+ position are the same bytes for a prompt in one block, one token at a time, and in
+ blocks of 3, 8 (the fixture's window) and 9 rows.
+
+ **A lost device.** MiMo has no recurrent state, and the new K/V rows reach the host's
+ caches only when the step's last frame has completed: a device lost in the middle of a
+ step loses nothing the host holds, and the CPU runs the step's remaining rows from where
+ the caches end. Nothing is rebuilt, and a turn with a picture recovers too. The line is
+ `[VK] mimo chain: the device was lost at position P; ...`.
+
+ **What stays off the device.** The router's top-k and the routed experts the tier does
+ not hold, the embedding gather, the vision tower (on the device only with
+ `COLI_VK_DENSE=1`, through the per-matrix path), and runs with `MIMO_TRACE` (the CPU's
+ per-layer dump), which the chain declines. MiMo has no MTP head. The chain also
+ declines a head dim above 256 or a dense matrix that did not reach the device.
+
+ **Correctness, on the tiny fixture** (6 layers: 4 windowed with a window of 8 and a sink
+ logit, 2 full, partial RoPE with two thetas, a value scale of 0.707, a dense layer 0, a
+ vision tower), on Lavapipe and on the Radeon 780M (`tests/vulkan_engines.sh
+ mimo-chain`): Xiaomi's vendor oracle passes with the chain on (greedy and teacher-forced
+ in every case, the native FP8/BF16 trunk, prefill in blocks of 3 and of 1, the picture);
+ every configuration gives the CPU's tokens and every position's logits within 1e-4 of
+ the largest one (measured over 42 configurations at most 2.0e-6 on Lavapipe and 2.7e-6
+ on the 780M, both with the picture's tower on the device; 1.7e-6 without it); serve
+ sessions give the CPU's frames (logprobs within 2.9e-4 on Lavapipe and 1.8e-4 on the
+ 780M, on logits that reach 300); the prefix-reuse and photo tests pass bit for bit
+ against a cold engine. With `COLI_VULKAN` unset, the tokens and the logits bytes of 36
+ configurations (`MIMO_DENSE_BITS` 32, 0 and 8, blocks of 64, 3 and 1, four cases with
+ the picture) are the previous build's; with `COLI_VULKAN=1` and `COLI_VK_CHAIN=0` or
+ unset, those of 54 (the tier balanced deterministically, `COLI_VK_TIER_BALANCE=0`, since
+ its balance moves bits from run to run on either build).
+
+ **The default.** The chain's speed on a real MiMo model is not measured: no MiMo
+ checkpoint is on the test box (the smallest, Flash, has 309B parameters). The chain is
+ on for a discrete GPU (the common rule), off on an integrated GPU (`COLI_VK_CHAIN=1`
+ turns it on) and on a CPU device:
+
+ ```
+ [VK] mimo: dense chain off (an integrated GPU with the expert tier: not measured; COLI_VK_CHAIN=0 off, 1 on, 2 prompts only)
+ ```
+
+### OLMoE and Inkling
+
+olmoe (`olmoe_chain.h`) and inkling (`inkling_chain.h`) run the chain too, with the same
+knobs, default rule and `[VK] <engine> chain: ...` line as qwen36.
+
+| | olmoe | inkling |
+|---|---|---|
+| device, frame A1 | the layer before's routed sum joining the residual; the input norm; q/k/v; OLMoE's q and k norms over the whole projection; RoPE from the CPU's table; the K/V rows into the device mirror; attention; o_proj; the residual add; the post-attention norm; the router logits | the layer before's MoE output joining the residual (the routed sum, then each shared expert times its combine weight, `moe()`'s order, then the MLP's short convolution and the add); the input norm; q/k/v and the bias projection r; the short convolutions on K and V with their rings; the per-head q/k norms; the attention (`chain_relattn.comp`); the K/V rows into the device ring; o_proj; the attention's short convolution; the residual add; the post-attention norm; the router logits (routed and shared). A dense-MLP layer runs its MLP, global scale, short convolution and add in the same frame |
+| host | softmax, top-k, the routed experts (the tier's batch and the CPU's share in rank order: the code `moe()` runs); the K/V rows into the host's cache; `PILOT`'s prefetch (below) | the sigmoid router with its bias, top-k, the joint combine weights, `TOPP`; the routed experts (`moe_ex`: `moe()`'s code, the device's logits handed in, the shared experts left out); the K/V rows into the host's cache |
+| device, frame A2 (not waited for) | none: OLMoE has no shared expert | the shared experts, unweighted |
+| last frame | the final norm and lm_head on the last row | the final norm, the division by the width multiplier, lm_head; the per-position heads (logprobs, teacher forcing) read the final rows on the host |
+
+Inkling's attention is not qwen36's: a learned relative-position bias (an r projection
+per row mixed through a per-layer bank, one bias per backward distance), the log-length
+scale tau, a sliding window on five layers of six over a ring of K/V rows, and short
+depthwise convolutions (residual inside) on K, V, the attention output and the MLP
+output. Two shaders do it, made on first use so that the qwen chains never depend on
+them: `chain_relattn.comp` (grouped attention with the bias bank mixed in the CPU's
+order, tau from a host table computed as the CPU computes it, the window, the ring, and
+the step's own rows read from its K/V scratch as `attention()` reads them, so a step
+that wraps the ring never loses a row an earlier query of it reads) and
+`chain_sconv.comp` (the convolution with its ring, in `sconv_apply`'s order, plus the
+scalar multiply and divide of the dense MLP's global scale and the logits' width
+multiplier). The shared experts join through `chain_ew.comp`'s `HC_APPLY` over one stream
+(y += w[row] * x), the CPU's `os[d] += w * hh[d]`. Nothing in the chain stages a row
+in shared memory past what the device allows: Inkling's hidden size (6144) and its
+24576-wide dense MLP run through `chain_gemv.comp` or, past its staging, `qmatmul.comp`'s
+GEMV reading from the buffer; `tools/make_tiny_inkling.py --wide` makes a two-layer
+model at D = 6144 for the tests.
+
+**State.** Both keep the host's K/V cache canonical and copy each step's rows back.
+olmoe mirrors it behind qwen36's watermark. Inkling's mirror has the host's layout, a
+ring on sliding layers, and a range of positions per layer that the host wrote alone
+(a CPU step): the next chain step uploads the rows of its last `cap` positions first,
+whatever positions they now hold, so a rewind to a pinned snapshot over a wrapped ring
+reads on the device what the CPU reads. Inkling's four convolution states per layer run
+on the device and come back at the end of every chain step (a few KB a layer), so the
+host's copy is always current; a reset, a restored snapshot or a CPU step sends them
+up again before the next chain step.
+
+**Where they decline** (the per-matrix path, the state marked as the host's): inkling
+under CUDA or Metal; inkling's bf16 matrices on a CPU whose bf16 dot rounds the
+activations (AVX512-BF16), which the per-matrix path keeps on the CPU for the same
+reason (`[VK] inkling chain: ... bf16 stays on the CPU`); a geometry outside the shaders
+(head dim above 256, a bias bank wider than 64, more than 9 taps). olmoe runs `PILOT`:
+the rows after attention come down with the frame and the prefetch reads them, the
+rows after the MoE are those plus the routed sum (the float add the device makes).
+Running PILOT under the chain showed a race of the CPU path's own: the prefetcher could
+take the slot the forward pass was multiplying with as its LRU victim (at cap 1 the only
+slot) and read another expert into it mid-matmul. A slot now counts its readers, and
+neither eviction takes one being read. A device lost mid-step: olmoe redoes the step on
+the CPU (attention only, nothing to rebuild); inkling rebuilds its K/V and convolution
+states on the CPU from the prefix record, as qwen36 does.
+
+**OLMoE-1B-7B on the Radeon 780M** (`allenai/OLMoE-1B-7B-0924`, converted with
+`tools/convert_olmoe_merged.py`: int8 experts, f32 trunk; cap 64, `OMP_NUM_THREADS=8`,
+1-min load under 2, no other engine running, the same binary for every arm, every tier
+arm from the same history of an unrelated prompt). Cold: the model files dropped from
+the page cache first (`posix_fadvise`, as the qwen bench does); warm: the run after.
+Decode is 100 steps after a 25-token prompt: the engine's time for 101 new tokens minus
+its time for 1 (the prefill alone), both from `TUNE decode`; prefill is a 512-token
+prompt with one new token. Where a warm cell lists two numbers, they are separate
+rounds.
+
+| Decode, 100 tokens | cold | warm |
+|---|---|---|
+| CPU | 22.2 tok/s | 23.1, 23.2 tok/s |
+| tier, trunk on the CPU | 12.6 tok/s | 12.8, 12.8 tok/s |
+| tier and chain (`COLI_VK_CHAIN=1`) | 16.8 tok/s | 17.3, 17.2 tok/s |
+| chain without the tier (`COLI_VK_TIER=0`) | 12.0 tok/s | 12.8 tok/s |
+
+| Prefill, 512 tokens | cold | warm |
+|---|---|---|
+| CPU | 11.7 s | 10.5, 10.6 s |
+| tier, trunk on the CPU | 6.3 s | 6.4, 6.4 s |
+| tier and chain | 5.5 s | 5.5, 5.5 s |
+| chain without the tier | 12.7 s | 11.4 s |
+
+What the numbers say:
+- **Against the tier alone the chain wins both**: decode 35% faster, the 512-token
+ prompt 14% sooner. That is the comparison `coli_vk_chain_decide` makes on an
+ integrated GPU with the tier on, so olmoe passes ON there.
+- **Against the CPU, decode loses**: 17.3 against 23.1 tok/s. OLMoE's trunk is f32
+ (1.49 GB read every token, more than its eight routed experts' 0.8 GB of int8); the CPU
+ reads it faster than the GPU does at the clock it mostly holds. Prefill wins: 5.5
+ against 10.5 s. `COLI_VULKAN=1` is opt-in, and on this box the CPU alone is the faster
+ way to decode OLMoE; with the device on, the chain is the better of its two modes.
+ `COLI_VK_CHAIN_PROF=1` put 95% of the chain's device time in the trunk's f32 GEMVs,
+ 34.8 ms a token (about 43 GB/s for its 1.49 GB); the GPU sat at its 800 MHz floor in
+ 78% of the clock samples with the chain, 95% with the tier alone.
+- **Without the tier** the chain only moves the f32 trunk to the device and leaves every
+ expert on the CPU: slower than the CPU in both. The default keeps it off there.
+- **The text.** Every arm, cold and warm, printed the same 100 decode tokens, and the
+ same first token after the 512-token prompt.
+
+**Not measured**: a discrete GPU; OLMoE in serve sessions, past a 537-token context, with
+`PILOT` (its experts fit in RAM here) or with the dense trunk on the device beside the
+chain (`COLI_VK_DENSE=1`).
+
+**Inkling: not measured.** No Inkling checkpoint runs on the box (the model is 975B),
+so its integrated-GPU default is off (`COLI_VK_CHAIN_UNMEASURED`: the `[VK]` line says
+"not measured"); `COLI_VK_CHAIN=1` turns it on. Its correctness is the tiny fixtures',
+on Lavapipe and on the 780M (below), including the two-layer model at D = 6144.
+
+**Tests.** `tests/vulkan_engines.sh inkling-olmoe-chain` gates every configuration on the
+CPU run's tokens and every forward's logits within 1e-4 of the largest (`DUMP=<path>` in
+both engines' ref mode writes them; measured 2e-7 relative on the fixtures, 6e-7 at
+D = 6144): inkling's f32, dense-int4g64 and bf16 snapshots (on an AVX512-BF16 host the
+bf16 one checks the clean decline instead), its int4 and int8 expert containers and
+runtime quantizations, `TOPP`, the tier off, the trunk's device copies shared with the
+per-matrix path, prefill in chunks of 3, the tiled GEMM, the per-row GEMV, prompts only,
+D = 6144 (and the tier's expert batch at that width); olmoe's caps, 4-bit experts,
+`PILOT` at caps 1 and 2 and `PILOT=3`, the tier off, the shared trunk, an eviction
+budget, chunks, the GEMM, the per-row GEMV, prompts only; the device lost mid-decode in
+both (inkling in a shared-expert frame and at a router); `tests/vulkan_chain_serve.py`
+sessions (pins, prompt-cache extensions, a divergent prompt, logprobs) frame for frame
+against the CPU, with the chain and with prompts only; and both engines' prefix-reuse,
+dashboard and Brio tests with the chain on. `inkling-olmoe-chain-sanitize` runs the
+chain's ops, a set of those configurations and a serve session of each engine under
+ASan and UBSan. On the 780M (RADV) every one of those configurations and serve sessions
+gave the CPU's tokens.
+
### Adding an engine to the chain
The recipe qwen36_chain.h and qwen38_chain.h follow, for the engines still on the
@@@ -979,9 -925,9 +1047,9 @@@ What each remaining architecture needs
| colibri.c (GLM-5.2), glm53, deepseek_v41, kimi_k3 (MLA layers) | MLA: q_a/kv_a, the latent norms, q_b, RoPE on the rope dims, a latent + rope cache | the absorb core (`attention_absorb.comp`) already reads a device KV mirror with GLM's watermark (`vk_kv_valid`): record it as a chain op with the o-projection fused (the universal layout's eight bindings hold its seven); DSA's index keys and top-k are `chain_qsa.comp`'s selection with blocks of one position, and the absorb shader takes the selection list as `chain_attn.comp` does; GLM's MTP layer stays on the CPU like qwen38's head, or runs in the chain (MLA has no recurrent state: a rejected draft is a lowered watermark) |
| deepseek_v4 | MLA with compressed (CSA) and hierarchical (HCA) KV, mHC | the manifold hyper-connections are qwen38's stream read/apply with a Sinkhorn normalization (an element-wise op that iterates); the compressors' rolling windows are rings like the conv's, snapshotted the same way; the CPU rounds activations to E4M3 before its fp8 matmuls, so the chain needs that rounding as an element-wise op to keep the same arithmetic |
| kimi_k3 (KDA layers) | Kimi Delta Attention: a gated delta rule whose decay is a vector over the key channels | `chain_dnrec.comp` with the decay per key row (one `exp(g_k)` per row of the column, loaded beside q and k in shared memory) instead of one per head; the short convolution is `chain_dnconv.comp`; its output gate and norm as the gated norm |
- | mimo | sliding-window attention (and full layers) | `chain_attn.comp` with a window (positions from `max(0, pos - W + 1)`: one more push constant), the cache optionally a ring of W rows (the host mirror then indexes `t % W`) |
+ | mimo | sliding-window attention (and full layers) | done: [MiMo-V2.6 on the chain](#mimo-v26-on-the-chain) (`chain_attn.comp` with a window, a ring of W rows, V's own head dim and a sink) |
-| inkling | grouped attention, MoE with a shared expert | qwen36's attention and combine as they are; its per-position heads are matmuls of the residual rows |
-| olmoe | attention with q/k norm, MoE without a shared expert | qwen36's Qwen3-Coder geometry (all attention, no gate, no shared expert) is the same chain |
+| inkling | grouped attention with a relative-position bias, a sliding window and short convolutions; MoE with shared experts | in the chain ([OLMoE and Inkling](#olmoe-and-inkling)): two shaders of its own, `chain_relattn.comp` and `chain_sconv.comp`; the shared experts join through `HC_APPLY` |
+| olmoe | attention with q/k norm, MoE without a shared expert | in the chain ([OLMoE and Inkling](#olmoe-and-inkling)) with qwen36's ops as they are |
## Adding an engine to the tier
@@@ -1064,10 -1010,11 +1132,15 @@@ in short
(measured 2e-7 and below on the fixtures), prefill in chunks, an image, MTP drafts
rejected, accepted and alternating, a device lost mid-run, the qwen38 oracle targets,
the prefix-reuse contract and serve sessions frame for frame; `qwen-chain-sanitize`
- runs the chain under ASan and UBSan. `inkling-olmoe-chain` and
+ runs the chain under ASan and UBSan. `mimo-chain` does it for MiMo: the vendor
+ oracle with the chain on, the CPU's tokens and every position's logits in every dense
+ form, block size, case and tier setting, the window boundary bit for bit across block
+ sizes, prompts only, a device lost in four places, the prefix-reuse and photo tests
+ and serve sessions; `mimo-chain-sanitize` under ASan and UBSan.
++ `inkling-olmoe-chain` and
+ `inkling-olmoe-chain-sanitize` do the same for inkling and olmoe
+ ([OLMoE and Inkling](#olmoe-and-inkling)); `vk-chain-check` covers their ops
+ (`chain_sconv.comp`, `chain_relattn.comp`, the GEMV and the norm at D = 6144).
- int4 weights decode as offset-binary (nibble−8), byte-identical layout to
the CPU path — no repacking.
- Khronos validation layers: the backend never enables them, so the loader