Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
docs(deepseek-v41): record the DSpark measurement, not the intention
JustVugg6a0f1b709eef
diff --git a/docs/ENVIRONMENT.md b/docs/ENVIRONMENT.mdindex f86544b4..744d0035 100644--- a/docs/ENVIRONMENT.md+++ b/docs/ENVIRONMENT.md@@ -497,7 +497,7 @@ COLI_TEMP=0 COLI_METAL=1 DIRECT=1 COLI_NO_OMP_TUNE=1 PIPE=1 PIPE_WORKERS=6 MTP=0| `V41_INDEX_OWNER` | unset | DeepSeek V4.1: score each layer against its OWN index keys instead of the last published cache. The default reproduces the released inference code; this changes the model's behaviour, see docs/deepseek-v41.md. || `V41_MAX_IMAGE_TOKENS` | the checkpoint's `max_image_tokens` | DeepSeek V4.1: ceiling on what one image costs in prompt tokens. || `V41_TRACE` | unset | DeepSeek V4.1: print per-sublayer checksums, matching tools/dsv41_ref.py's, to locate a divergence by diffing two columns. `2` follows the first row of a speculative step rather than the last. |-| `V41_DSPARK` | on when the checkpoint carries the head | DeepSeek V4.1: `0` disables the DSpark draft head, which is then not loaded at all. Drafts never change what a turn produces, only how many forwards it takes. |+| `V41_DSPARK` | off | DeepSeek V4.1: `1` loads the DSpark draft head. Drafts never change what a turn produces, only how many forwards it takes; measured a net loss on CPU at 68% expert residency (9 forwards instead of 24, but 114 s instead of 86) and a win where per-row compute is cheap. || `V41_DSPARK_MAX` | the checkpoint's `dspark_block_size` | DeepSeek V4.1: how many drafted tokens go in front of the main model per round. Fewer costs less when a round is rejected and caps the win when it is not. |-| `V41_DSPARK_MINACC` | 30 | DeepSeek V4.1: percent of drafts that must be accepted over a window of 24 before drafting pauses for 64 tokens. |+| `V41_DSPARK_MINACC` | 60 | DeepSeek V4.1: percent of drafts that must be accepted over a window of ten before drafting pauses for 64 tokens. 60 is the measured break-even. || `V41_SPEC_FORCE` | unset | DeepSeek V4.1, oracle mode only: draft the reference's own tokens (`1`), corrupt the last one (`2`), or keep the head's (`3`), so the verification path runs on a fixture whose draft head is random noise. |diff --git a/docs/deepseek-v41.md b/docs/deepseek-v41.mdindex 54bd90cd..d401cb68 100644--- a/docs/deepseek-v41.md+++ b/docs/deepseek-v41.md@@ -95,11 +95,28 @@ the oracle drafts the reference's own next tokens (`1`), or corrupts the last ofso a round is rejected part way (`2`), or keeps the head's own (`3`); all three have toreproduce the reference token for token, and CI runs them.-Drafting is on whenever the checkpoint carries the head. It is not always worth it: a-round that is rejected has still read its stages' experts, so acceptance is measured-over a window and the drafts pause for 64 tokens when it falls under-`V41_DSPARK_MINACC`. `V41_DSPARK=0` turns the head off entirely, and it is then not-even loaded.+Drafting is **off unless you ask for it**, and the reason is a measurement rather+than caution. On a 16-thread CPU server holding 68% of the experts, DeepSeek's own+draft head accepts half of what it proposes, which is a respectable rate, and still+loses:++| | forwards for 24 tokens | wall |+|---|---|---|+| `V41_DSPARK=1` | 9 | 114 s |+| off | 24 | 86 s |++A six-row verify amortizes the disk almost perfectly, expert reads up 17% for six+positions, and the per-row dense work not at all, attention up 40%. Six positions+cost six times the attention projections and buy 2.7 tokens. That balance flips+where per-row compute is cheap: a GPU, or a machine holding the whole expert set,+which is exactly where someone would reach for it.++So it is a switch. `V41_DSPARK=1` turns it on, and the guard still watches: a round+that is rejected has still read its stages' experts, so acceptance is measured over+ten proposals and the drafts pause for 64 tokens when it falls under+`V41_DSPARK_MINACC`, whose default of 60% is where the measurement above puts the+break-even. That is the same crossing colibri.c measured for MTP on GLM-5.2, where+90% acceptance gains 22% and 19% loses 25%.## The index-key slot, and why the default is the odd one@@ -131,9 +148,9 @@ default cannot be "tidied" by accident.| `V41_INDEX_OWNER` | unset | each layer scores against its own owner's index keys (see above). Changes the model's behaviour. || `V41_MAX_IMAGE_TOKENS` | the checkpoint's `max_image_tokens` | a ceiling on what one image costs in prompt tokens. || `V41_TRACE` | unset | print a checksum of the tensors the reference prints too, for locating a divergence by diffing two columns. `2` follows the first row of a speculative step instead of the last, which is the row a sequential decode is comparable to. |-| `V41_DSPARK` | on when the checkpoint carries the head | `0` disables the draft head and does not load it. |+| `V41_DSPARK` | off | `1` loads the DSpark draft head and verifies its blocks. A net loss on a CPU box at 68% expert residency, a win where per-row compute is cheap; see above. || `V41_DSPARK_MAX` | the checkpoint's `dspark_block_size` | how many of the drafted tokens are put in front of the main model. Fewer means a cheaper rejected round and a lower ceiling on the win. |-| `V41_DSPARK_MINACC` | 30 | percent of drafts that must be accepted over a window of 24 for drafting to continue; below it, drafts pause for 64 tokens. |+| `V41_DSPARK_MINACC` | 60 | percent of drafts that must be accepted over a window of ten for drafting to continue; below it, drafts pause for 64 tokens. 60 is the measured break-even, not a guess. || `V41_SPEC_FORCE` | unset | oracle mode only: draft the reference's tokens (`1`), corrupt the last one (`2`), or use the head's own (`3`), to exercise the verification path on a fixture whose draft head is random. |## How it is tested