Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

registry+tests: fix round 1 -- name parity, MXFP4 row precision, collision-family pinning

monotophic575612c9cb90
diff --git a/c/colibri.c b/c/colibri.c
index b144b1dd..6ff1d9f5 100644
--- a/c/colibri.c
+++ b/c/colibri.c
@@ -1490,10 +1490,19 @@ static int qt_resolve_fmt(const char *name, int O, int I, int64_t nb, int64_t ns
* corner to fmt=1 (the stamp confirms it really is plain int8, a format
* this build CAN decode); a stamp naming "fp8-e4m3-b128" does NOT resolve
* it, for the same "recognized but not implemented" reason the clean
- * (non-colliding) ue8m0 case below refuses regardless of any stamp. No
- * real GLM tensor has these shapes; see also the SECOND DESIGN LANDMINE
- * above for the analogous ue8m0-vs-fmt=6 corner this same signature can
- * hit. */
+ * (non-colliding) ue8m0 case below refuses regardless of any stamp.
+ * The colliding FAMILY is exactly: nb==O*I && ns==O*4 &&
+ * ceil(O/128)*ceil(I/128)==4*O (is_row && is_blk_ue8m0 below; is_blk can
+ * never co-hold since nblk==O and nblk==4*O are disjoint for O>=1).
+ * At I<=16384 (nblkI<=128) membership forces O<=32; every GLM-5.2
+ * resident/routed role has O>=576 (kv_a's kv_lora+qk_rope is the
+ * smallest -- tools/fp8_collision_census.py enumerates the roles, and
+ * the census over the real checkpoint was run in this PR's rev5 round),
+ * so no GLM-5.2 tensor is a member -- the discipline exists for
+ * untrusted containers. tests/test_fp8_load.c's Part A3b pins the
+ * family's polarity at member shapes [1,400], [2,1024], [129,33000].
+ * See also the SECOND DESIGN LANDMINE above for the analogous
+ * ue8m0-vs-fmt=6 corner this same signature can hit. */
if(fmt==1){
int64_t nblkO=fp8_nblk(O), nblkI=fp8_nblk(I);
int64_t ns_row=(int64_t)O*4, ns_blk=nblkO*nblkI*4, ns_blk_ue8m0=nblkO*nblkI;
@@ -1532,7 +1541,8 @@ static int qt_resolve_fmt(const char *name, int O, int I, int64_t nb, int64_t ns
fprintf(stderr,"%s: [%d,%d] fp8-e4m3-b128 with ue8m0 scales recognized but not "
"implemented; only f32 block scales are supported in this build (nb=%lld "
"bytes matches raw e4m3 weight bytes, ns=%lld bytes matches %lld blocks x "
- "1 byte/block)%s%s\n",
+ "1 byte/block)%s%s -- refusing rather than misreading the sidecar "
+ "(untrusted container)\n",
name,O,I,(long long)nb,(long long)ns,(long long)(nblkO*nblkI),
is_row ? " -- scale array ALSO matches per-row int8 (fmt=1)" : "",
stamped_name ? " -- a metadata stamp cannot grant this build a decoder it doesn't have"
diff --git a/c/st.h b/c/st.h
index 67aa126d..2a22df54 100644
--- a/c/st.h
+++ b/c/st.h
@@ -246,9 +246,13 @@ static void st_pread_full(int fd, void *buf, int64_t n, int64_t off, const char
* carries (tools/repack_fp8_passthrough.py never stamps routed experts). A
* container whose combined __metadata__["colibri.fmt"] entries exceed this
* cap is not using the convention as designed -- CAP, not a switch to a hash
- * table: refuse loudly rather than grow an unbounded array for an untrusted
- * container that might be trying to force a large ingest allocation during
- * header parsing, before any other validation has run. */
+ * table. Precisely what this bounds: the colibri.fmt blob is json_parse'd in
+ * FULL before the per-entry check below fires, so the parse allocation
+ * itself is bounded by ST_MAX_HEADER (the shard-header size cap), not by
+ * this constant -- what the cap bounds is the PERSISTENT fmt_name/fmt_val
+ * strdup arrays on `shards` (and every later st_fmt_stamp linear scan over
+ * them), which would otherwise grow with an adversarial map. Refuse loudly
+ * rather than carry an absurd stamp map forward. */
#define ST_FMT_STAMP_MAX 4096
/* Parses one shard's __metadata__["colibri.fmt"] value (a safetensors metadata
diff --git a/c/tests/test_formats_registry.py b/c/tests/test_formats_registry.py
new file mode 100644
index 00000000..1d028d4f
--- /dev/null
+++ b/c/tests/test_formats_registry.py
@@ -0,0 +1,96 @@
+"""docs/FORMATS.md <-> colibri.c FMT_NAMES parity (fix round 1, adopted
+review test-candidate).
+
+The registry's name column IS the canonical stamp string: a container's
+__metadata__["colibri.fmt"] entry is matched byte-for-byte against
+FMT_NAMES by qt_fmt_by_name(), and FORMATS.md is what a third-party tool
+author will read to learn the string to write. A divergence between the
+two (the class of defect this test pins: the registry shipped "e8-iq3"
+while FMT_NAMES said "e8-iq3-lattice") means a tool stamping the
+documented name produces containers the reader REFUSES as unrecognized.
+
+Mechanism: regex over the two authoritative TEXTS themselves (the C
+source and the markdown), not a generated header. Justification: a
+generated header is a third copy that can go stale exactly like the doc
+did, plus build machinery for one test; the C table's syntax is a
+trivially stable one-entry-per-line initializer and the doc's table is
+the reviewed artifact itself. Both parses assert plausibility floors so
+an anchor drifting (renamed array, retitled section) fails loudly as a
+parse error, never as a silent empty-vs-empty pass.
+
+Parity rules:
+ 1. Every FMT_NAMES (name, fmt) entry must appear in the doc table as a
+ row whose ordinal matches and whose FIRST backticked token in the
+ name cell equals the C name exactly.
+ 2. Every doc table row whose ordinal has NO FMT_NAMES entry must say so
+ explicitly ("no in-tree name string" -- the MXFP4 row's marker), so
+ a new registry row can't silently claim a name the reader won't
+ recognize.
+"""
+import os
+import re
+import unittest
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+C_DIR = os.path.normpath(os.path.join(HERE, ".."))
+COLIBRI_C = os.path.join(C_DIR, "colibri.c")
+FORMATS_MD = os.path.normpath(os.path.join(C_DIR, "..", "docs", "FORMATS.md"))
+
+
+def parse_fmt_names(path):
+ with open(path, encoding="utf-8") as f:
+ src = f.read()
+ m = re.search(r"FMT_NAMES\[\]\s*=\s*\{(.*?)\};", src, re.S)
+ if not m:
+ raise AssertionError("FMT_NAMES[] initializer not found in colibri.c "
+ "(anchor drifted -- update this test's regex)")
+ entries = re.findall(r'\{\s*"([^"]+)"\s*,\s*(\d+)\s*\}', m.group(1))
+ return {int(fmt): name for name, fmt in entries}
+
+
+def parse_doc_rows(path):
+ rows = {}
+ with open(path, encoding="utf-8") as f:
+ lines = f.readlines()
+ for line in lines:
+ m = re.match(r"\|\s*\*{0,2}(\d+)\*{0,2}\s*\|\s*(.+)$", line)
+ if not m:
+ continue
+ ordinal = int(m.group(1))
+ name_cell = m.group(2).split("|")[0]
+ first_tick = re.search(r"`([^`]+)`", name_cell)
+ rows[ordinal] = (first_tick.group(1) if first_tick else None, name_cell)
+ return rows
+
+
+class RegistryParityTest(unittest.TestCase):
+ def setUp(self):
+ self.c_names = parse_fmt_names(COLIBRI_C)
+ self.doc_rows = parse_doc_rows(FORMATS_MD)
+ # plausibility floors: an anchor drift must fail HERE, loudly.
+ self.assertGreaterEqual(len(self.c_names), 8,
+ "FMT_NAMES parse found suspiciously few entries")
+ self.assertGreaterEqual(len(self.doc_rows), 9,
+ "FORMATS.md table parse found suspiciously few rows")
+
+ def test_every_c_name_matches_its_doc_row(self):
+ for fmt, name in sorted(self.c_names.items()):
+ self.assertIn(fmt, self.doc_rows,
+ f"FMT_NAMES has fmt={fmt} ('{name}') but FORMATS.md has no row for it")
+ doc_name, cell = self.doc_rows[fmt]
+ self.assertEqual(doc_name, name,
+ f"fmt={fmt}: FORMATS.md names '{doc_name}' but FMT_NAMES "
+ f"(the string qt_fmt_by_name actually matches) says '{name}'")
+
+ def test_doc_rows_without_c_entry_are_marked(self):
+ for fmt, (doc_name, cell) in sorted(self.doc_rows.items()):
+ if fmt in self.c_names:
+ continue
+ self.assertIn("no in-tree name string", cell,
+ f"FORMATS.md row fmt={fmt} ('{doc_name}') has no FMT_NAMES entry "
+ f"and is not marked 'no in-tree name string' -- a tool stamping "
+ f"this documented name would be refused as unrecognized")
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/c/tests/test_fp8_load.c b/c/tests/test_fp8_load.c
index c6104f4c..4a509d26 100644
--- a/c/tests/test_fp8_load.c
+++ b/c/tests/test_fp8_load.c
@@ -203,7 +203,16 @@ static int expect_refuse_stamped(int O, int I, int64_t nb, int64_t ns, const cha
printf("FAIL %s: expected exit(1) refusal, got status=%d, stderr=%.200s\n", tag, status, err);
return 0;
}
- if(!strstr(err,"refus")){
+ /* Search the ENGINE's message, not this test's own words: qt_resolve_fmt
+ * prints "%s: ..." with the caller-supplied name -- which is `tag` here --
+ * so a tag containing "refuses" would satisfy a naive strstr all by
+ * itself and turn this assertion into a tautology (fix round 1, found
+ * while pinning the ue8m0 family: the ue8m0 refusal message itself
+ * lacked any "refus" wording and the check never noticed). Skip the
+ * echoed tag prefix before searching. */
+ const char *msg = err; size_t tl = strlen(tag);
+ if(!strncmp(err, tag, tl)) msg = err + tl;
+ if(!strstr(msg,"refus")){
printf("FAIL %s: exited(1) but message lacked a refusal explanation: %.200s\n", tag, err);
return 0;
}
@@ -404,6 +413,35 @@ static void test_ue8m0_scale_refusal(void){
"ue8m0/fmt=1 collision O=1 I=400, stamped fp8-e4m3-b128 -> STILL refuses (no ue8m0 decoder)"));
}
+/* ---- Part A3b (fix round 1): the ue8m0-vs-fmt=1 unstamped-refusal FAMILY,
+ * pinned at two more member shapes. Membership is exactly:
+ * nb == O*I && ns == O*4 && ceil(O/128)*ceil(I/128) == 4*O
+ * (qt_resolve_fmt's is_row && is_blk_ue8m0; is_blk can never co-hold, since
+ * nblk==O and nblk==4*O are disjoint for O>=1). The [1,400] case above is
+ * the O=1 member; these pin O=2 (nblkO=1) and O=129 (nblkO=2, past the
+ * block edge) so the family's O-dependence is exercised, not one corner.
+ * CURRENT POLARITY, pinned deliberately: an UNSTAMPED tensor whose bytes
+ * are genuine plain int8 at a member shape REFUSES here (the named ue8m0
+ * refusal, carrying its "ALSO matches per-row int8" clause), while a build
+ * without this PR's fp8 branches loads the same bytes as fmt=1 -- this is
+ * a knowing, disclosed strictness trade for untrusted containers, and the
+ * stamp ("int8-row") is the designed escape hatch, verified here too.
+ * No GLM-5.2 resident tensor is a member: at I<=16384 (nblkI<=128),
+ * membership forces O<=32, and every repack-eligible GLM-5.2 role has
+ * O>=576 (tools/fp8_collision_census.py enumerates the roles). */
+static void test_ue8m0_family_sweep(void){
+ /* [2,1024]: nblkO=1, nblkI=8 -> nblk=8 == 4*O. ns = O*4 = nblk = 8. */
+ CHECK(expect_refuse(2,1024, (int64_t)2*1024, 8,
+ "ue8m0 family [2,1024] (nblk=8==4*O), unstamped int8-shaped -> refuses (current polarity)"));
+ CHECK(expect_fmt_stamped(2,1024, (int64_t)2*1024, 8, "int8-row", 1,
+ "ue8m0 family [2,1024], stamped int8-row -> resolves to fmt=1 (the escape hatch)"));
+ /* [129,33000]: nblkO=2, nblkI=258 -> nblk=516 == 4*129. ns = 516. */
+ CHECK(expect_refuse(129,33000, (int64_t)129*33000, 516,
+ "ue8m0 family [129,33000] (nblk=516==4*O, nblkO=2), unstamped int8-shaped -> refuses (current polarity)"));
+ CHECK(expect_fmt_stamped(129,33000, (int64_t)129*33000, 516, "int8-row", 1,
+ "ue8m0 family [129,33000], stamped int8-row -> resolves to fmt=1"));
+}
+
/* ---- Part B: qt_from_disk loader-seam (real safetensors file) ---- */
static void deq_fmt8(const QT *t, float *dq){
@@ -1071,6 +1109,7 @@ int main(void){
test_disambiguation();
test_fmt6_fp8_collision();
test_ue8m0_scale_refusal();
+ test_ue8m0_family_sweep();
test_loader_seam();
check_fp8_bytes(2048,6144, "qt_bytes fmt=8 gate/up-shaped O=2048 I=6144 (spec example)");
check_fp8_bytes(6144,2048, "qt_bytes fmt=8 down-shaped O=6144 I=2048");
diff --git a/docs/FORMATS.md b/docs/FORMATS.md
index a2b95d4c..fbec9f1c 100644
--- a/docs/FORMATS.md
+++ b/docs/FORMATS.md
@@ -62,8 +62,8 @@ the bytes alone say.
| 3 | `int2-row` | `O*ceil(I/4)` bytes (`q4`, 4 packed 2-bit values/byte) | 1 `f32` per **row** (`O` entries) | stable, upstream | `1ae22a6` (initial commit) |
| 4 | `int4-grouped` | `O*ceil(I/2)` bytes (`q4`, same packed layout as fmt=2) | 1 `f32` per **group** of `gs` inputs, `O*ceil(I/gs)` entries | stable, upstream | `498ab0c` / merged PR [#242](https://github.com/JustVugg/colibri/pull/242) |
| 5 | `int3-g64` | `O*ceil(I/64)*24` bytes (`q4`; 24B/group = 16B low-plane + 8B high-plane, `I3_GROUP=64`/`I3_GBYTES=24`) | 1 `f32` per **64-input group**, `O*ceil(I/64)` entries | stable, upstream | `5e42e70` |
-| 6 | `e8-iq3` (E8/IQ3 lattice) | `O*ceil(I/256)*98` bytes (98-byte self-contained super-blocks: E8 lattice indices + parity signs + fp16 super-scales; `E8_QK=256`, `E8_BBYTES=98`) | none as a separate array β€” scales live **inside** the super-blocks; the `.qs` sidecar is a single-float tag (`ns==4` is the loader's discriminator). NOTE: stores `W@Q` (block-diagonal FWHT rotation) β€” activations must be rotated before the kernel | **stable, upstream** β€” [#465](https://github.com/JustVugg/colibri/pull/465) MERGED 2026-07-21 | dev `dce7012` |
-| 7 | `mxfp4` (no in-tree name string β€” dev has no stamp feature) | `O*ceil(I/2)` bytes (e2m1 nibbles, 2 packed values/byte β€” same nibble packing as fmt=2/4) | 1 `f32` per **group** of `gs` inputs (`gs=32` for Kimi K3; host gate requires `gs % 8 == 0`) | **stable, upstream** β€” Kimi K3 native routed-expert tier, **Vulkan backend only** (`c/backend_vulkan.c`, `c/shaders/qmatmul.comp`); NOT a CPU/`QT` format β€” `qt_resolve_fmt` never returns 7 and the Metal backend refuses it | merged [#676](https://github.com/JustVugg/colibri/issues/676)/[#705](https://github.com/JustVugg/colibri/pull/705) |
+| 6 | `e8-iq3-lattice` (E8/IQ3 lattice) | `O*ceil(I/256)*98` bytes (98-byte self-contained super-blocks: E8 lattice indices + parity signs + fp16 super-scales; `E8_QK=256`, `E8_BBYTES=98`) | none as a separate array β€” scales live **inside** the super-blocks; the `.qs` sidecar is a single-float tag (`ns==4` is the loader's discriminator). NOTE: stores `W@Q` (block-diagonal FWHT rotation) β€” activations must be rotated before the kernel | **stable, upstream** β€” [#465](https://github.com/JustVugg/colibri/pull/465) MERGED 2026-07-21 | dev `dce7012` |
+| 7 | `mxfp4` (no in-tree name string β€” dev has no stamp feature) | `O*ceil(I/2)` bytes (e2m1 nibbles, 2 packed values/byte β€” same nibble packing as fmt=2/4) | in the **container**: 1 **UE8M0** byte (u8 power-of-two exponent) per **group** of `gs=32` inputs; expanded to 1 `f32` per group at Vulkan upload (`c/kimi_k3.c`'s `mx4_scale` loop β€” the shader is float-only). Host gate: `gs >= 8 && gs % 8 == 0` (`c/backend_vulkan.c`) | **stable, upstream** β€” Kimi K3 native routed-expert tier, **Vulkan backend only** (`c/backend_vulkan.c`, `c/shaders/qmatmul.comp`); NOT a CPU/`QT` format β€” `qt_resolve_fmt` never returns 7 and the Metal backend refuses it | merged [#676](https://github.com/JustVugg/colibri/issues/676)/[#705](https://github.com/JustVugg/colibri/pull/705) |
| **8** | `fp8-e4m3-b128` | `O*I` Γ— 1 byte (`q8`, raw e4m3, byte-identical layout to fmt=1) | a **declared property**, not one fixed layout β€” see "Scale encoding is a declared property" below. **f32** (1 per 128Γ—128 block, `ceil(O/128)*ceil(I/128)` entries) is IMPLEMENTED; **UE8M0** (1 byte/block, same block grid) is RECOGNIZED and refused by name, not yet implemented | **this PR pair** β€” CPU/Metal/repack/collision-refusal on `fmt7/fp8-passthrough`, this stamp+registry PR stacked on top; developed under the PRIVATE ORDINAL BLOCK as `fmt=100`; assigned fmt=7 by the maintainer on #524, renumbered to **fmt=8** after #705 merged claiming 7 for MXFP4 while this pair was open (see "ID assignment" below) | `fmt7/fp8-passthrough` (PR 1) |
With fmt 0–8 all assigned, the next free public ordinal is **9** β€” per the
@@ -95,7 +95,7 @@ pair's current restack, base dev `292ed4c`):
(`quant.h:295`), `i3_rowbytes` (`quant.h:296`); kernel `matmul_i3`
(`quant.h:354`); pack helper `pack_int3_g64` (`quant.h:956`). Allocation:
`qt_alloc`'s `bits==3` branch (inside `c/colibri.c:1105`).
-- **fmt=6** (`e8-iq3`) β€” upstream's merged code: format section header
+- **fmt=6** (`e8-iq3-lattice`) β€” upstream's merged code: format section header
precedes `quant.h:1008`; constants `E8_QK=256` (`quant.h:1008`),
`E8_SUB=32` (`quant.h:1009`), `E8_BBYTES=98` (`quant.h:1010`); row-byte
helpers `e8_blocks`/`e8_rowbytes` (`quant.h:1011-1012`); rotation contract
@@ -109,9 +109,11 @@ pair's current restack, base dev `292ed4c`):
branch `p.fmt == 4 || p.fmt == 7` in `c/shaders/qmatmul.comp` (e2m1 nibble
decode, per-`gs`-group scale folded into the accumulation), host gate and
sizing in `c/backend_vulkan.c` (`upload_tensor`'s `fmt == 4 || fmt == 7`
- allow-list, `gs % 8 == 0`). Wired for Kimi K3's routed-expert tier
- (`c/kimi_k3.c`); no CPU (`quant.h`) or Metal kernel exists for it, and
- `qt_resolve_fmt` has no byte-arithmetic branch that returns 7.
+ allow-list, `gs >= 8 && gs % 8 == 0`). Wired for Kimi K3's routed-expert
+ tier (`c/kimi_k3.c`, which expands the container's per-32-group UE8M0 u8
+ exponents to the f32 group scales the shader consumes β€” `mx4_scale`); no
+ CPU (`quant.h`) or Metal kernel exists for it, and `qt_resolve_fmt` has
+ no byte-arithmetic branch that returns 7.
- **fmt=8** (`fp8-e4m3-b128`, this branch) β€” decode table `E4M3_LUT`
(`quant.h:446`) / `e4m3_decode` (`quant.h:480`), block size
`FP8_BLOCK=128` (`quant.h:482`), kernel `matmul_fp8` (`quant.h:491`).
@@ -172,8 +174,14 @@ LANDMINE) also produces a UE8M0-vs-fmt=1 collision at different shapes (e.g.
`O=1, I` in `(384,512]`), and the same small-shape corner of the fmt=6
collision (SECOND DESIGN LANDMINE, `I=98`) has a UE8M0-scaled analogue at
`O` in `(384,512]`. Both are handled explicitly in `qt_resolve_fmt` and
-covered by `tests/test_fp8_load.c`'s Part A3 suite. No real GLM tensor has
-these shapes; the discipline exists for untrusted containers.
+covered by `tests/test_fp8_load.c`'s Part A3/A3b suites. The
+UE8M0-vs-fmt=1 colliding family is exactly `nb == O*I && ns == O*4 &&
+ceil(O/128)*ceil(I/128) == 4*O`; at `I <= 16384` (`nblkI <= 128`)
+membership forces `O <= 32`, and every GLM-5.2 resident/routed role has
+`O >= 576` (`kv_a`'s `kv_lora+qk_rope` is the smallest β€” the role census
+in `c/tools/fp8_collision_census.py`, run against the real checkpoint in
+this PR's rev5 round), so no GLM-5.2 tensor is a member; the discipline
+exists for untrusted containers.
## PRIVATE ORDINAL BLOCK convention (this repo, pending upstream review)
@@ -269,11 +277,15 @@ handful to a few hundred tensors per model (`q_a`/`q_b`/`kv_a`/`kv_b_proj`,
census in `c/tools/fp8_collision_census.py`), **never** the tens of
thousands of routed-expert tensors a large MoE checkpoint carries. A
container whose combined stamp map exceeds 4096 entries is not using this
-convention as it's designed to be used β€” most plausibly a malformed or
-adversarial container trying to force an unbounded ingest allocation during
-header parsing, before any size/shape validation has even run. Refusing
-loudly at that bound is the same "untrusted container, refuse rather than
-guess" discipline `qt_resolve_fmt` applies everywhere else in this feature.
+convention as it's designed to be used. To be precise about what the cap
+bounds: the `colibri.fmt` blob is JSON-parsed in full **before** the
+per-entry cap is checked, so the parse allocation itself is bounded by the
+shard-header size cap (`ST_MAX_HEADER`), not by this constant β€” what the
+cap bounds is the **persistent** per-tensor arrays (`fmt_name`/`fmt_val`
+strdups on `shards`) that would otherwise grow with an adversarial map,
+plus every later `st_fmt_stamp` linear scan over them. Refusing loudly at
+that bound is the same "untrusted container, refuse rather than guess"
+discipline `qt_resolve_fmt` applies everywhere else in this feature.
### Discovery-time abort surface