Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
docs+stamp: non-retroactivity, stamp-map cap, discovery-time abort, .qs scope
monotophic1a18eb6feede
diff --git a/c/st.h b/c/st.hindex b01ffb03..67aa126d 100644--- a/c/st.h+++ b/c/st.h@@ -239,6 +239,18 @@ static void st_pread_full(int fd, void *buf, int64_t n, int64_t off, const char}}+/* Stamps are a resident-tensor convention (see docs/FORMATS.md's "Stamp-map+ * scan bound"): a handful to a few hundred entries per model+ * (q_a/q_b/kv_a/kv_b_proj, o_proj, shared-expert and dense-MLP gate/up/down),+ * NEVER the tens of thousands of routed-expert tensors a large MoE model+ * carries (tools/repack_fp8_passthrough.py never stamps routed experts). A+ * container whose combined __metadata__["colibri.fmt"] entries exceed this+ * cap is not using the convention as designed -- CAP, not a switch to a hash+ * table: refuse loudly rather than grow an unbounded array for an untrusted+ * container that might be trying to force a large ingest allocation during+ * header parsing, before any other validation has run. */+#define ST_FMT_STAMP_MAX 4096+/* Parses one shard's __metadata__["colibri.fmt"] value (a safetensors metadata* value is always a plain string, so colibri.fmt's VALUE is itself JSON text --* a flat {tensor_name: format_name} object -- parsed a second time here) and@@ -250,7 +262,22 @@ static void st_pread_full(int fd, void *buf, int64_t n, int64_t off, const char* loudly -- same "untrusted container" discipline qt_resolve_fmt applies* elsewhere: a stamp the engine cannot make sense of must not be silently* ignored (that would be indistinguishable from a real mismatch going- * unnoticed). */+ * unnoticed).+ *+ * DISCOVERY-TIME ABORT SURFACE: every exit(1) below (malformed stamp value,+ * malformed entry, or the ST_FMT_STAMP_MAX cap) fires from inside+ * st_init_multi's shard-header-parse loop -- i.e. at CONTAINER DISCOVERY+ * time, while the engine is still building its tensor index, before it has+ * resolved a single tensor against the model's architecture or read one byte+ * of weight data. This is coarser-grained and EARLIER than+ * qt_resolve_fmt/qt_verify_fmt_stamp's own per-tensor refusals in colibri.c+ * (which fire much later, during weight load, once a specific tensor's+ * [O,I] shape and stamp are both known): a malformed stamp anywhere in any+ * shard aborts the ENTIRE model load immediately, before the user ever sees+ * which layer or tensor was implicated -- these messages name a shard FILE,+ * never a tensor, which is how to tell this abort surface apart from the+ * later per-tensor one at a glance. See docs/FORMATS.md's own section on+ * this. */static void st_fmt_stamp_ingest(shards *S, jval *root, const char *shard_path) {jval *meta = json_get(root, "__metadata__");if (!meta || meta->t != J_OBJ) return; /* no metadata object: unstamped, fine */@@ -269,6 +296,12 @@ static void st_fmt_stamp_ingest(shards *S, jval *root, const char *shard_path) {if (v->t != J_STR) {fprintf(stderr, "%s: colibri.fmt entry '%s' is not a string -- malformed stamp, refusing (untrusted container)\n",shard_path, inner->keys[i]); exit(1); }+ if (S->fmt_n >= ST_FMT_STAMP_MAX) {+ fprintf(stderr, "%s: __metadata__[\"colibri.fmt\"] stamps more than %d tensor names across "+ "this container's shards -- stamps are a resident-tensor convention (docs/FORMATS.md), "+ "not a bulk migration path; a container stamping this many names is malformed, "+ "refusing (untrusted container)\n",+ shard_path, ST_FMT_STAMP_MAX); exit(1); }if (S->fmt_n == S->fmt_cap) {S->fmt_cap = S->fmt_cap ? S->fmt_cap * 2 : 16;S->fmt_name = realloc(S->fmt_name, S->fmt_cap * sizeof(char*));@@ -286,7 +319,19 @@ static void st_fmt_stamp_ingest(shards *S, jval *root, const char *shard_path) {/* Stamped format NAME for `name`, or NULL if this tensor carries no stamp* (either because no shard stamped it, or the container predates this* feature). Linear scan: S->fmt_n is a subset of S->n (only stamped tensors),- * small in practice -- see the shards struct comment. */+ * small in practice -- see the shards struct comment.+ *+ * SCOPE: .qs-BACKED TENSORS ONLY. This function itself will happily look up+ * ANY name that got stamped -- st_fmt_stamp_ingest doesn't know or check+ * whether a stamped name belongs to a quantized (.qs-backed) tensor -- but+ * qt_from_disk (colibri.c) only ever CALLS this inside its `st_has(name++ * ".qs")` branch, i.e. only for a tensor that already carries a quantized+ * scale sidecar. A colibri.fmt entry naming a raw f32/bf16 weight, a norm, a+ * router, or embed/lm_head is stored here like any other entry but never+ * looked up: it is silently ignored BY DESIGN, not an oversight -- the+ * convention exists to disambiguate a byte-count collision among quantized+ * formats, and only a .qs-backed tensor can have one. See docs/FORMATS.md's+ * "Scope: .qs-backed tensors only". */static const char *st_fmt_stamp(shards *S, const char *name) {for (int i = 0; i < S->fmt_n; i++)if (!strcmp(S->fmt_name[i], name)) return S->fmt_val[i];diff --git a/c/tests/test_fp8_load.c b/c/tests/test_fp8_load.cindex a6bdd216..40c6b191 100644--- a/c/tests/test_fp8_load.c+++ b/c/tests/test_fp8_load.c@@ -959,6 +959,114 @@ static void test_stamp_absent(void){char p[300]; snprintf(p,sizeof p,"%s/model.safetensors",dir); unlink(p); rmdir(dir);}+/* ---- Stamp-map scan bound (maintainer review, #529): st_fmt_stamp_ingest+ * (st.h) caps the number of stamped-tensor entries it will ingest across a+ * container's shards at ST_FMT_STAMP_MAX and refuses (exit(1)) past it --+ * see st.h's own comment and docs/FORMATS.md's "Stamp-map scan bound" for+ * why (stamps are a resident-tensor convention, never a bulk migration path+ * for the tens of thousands of routed-expert tensors a large MoE model+ * carries). Writes a minimal ONE-tensor shard (the shard's actual tensor+ * content is irrelevant to this check -- the cap fires during+ * st_init_multi's header-parse loop, at CONTAINER DISCOVERY time, before any+ * tensor is resolved against the model, see st_fmt_stamp_ingest's own+ * "DISCOVERY-TIME ABORT SURFACE" comment) carrying an oversized+ * __metadata__["colibri.fmt"] blob of N distinct, entirely fictitious tensor+ * names -- st_fmt_stamp_ingest parses that blob independently of the+ * shard's real tensor list, so the names never need to correspond to+ * anything real for the cap to trigger. Tests BOTH sides of the boundary:+ * exactly ST_FMT_STAMP_MAX entries must still load fine (no off-by-one+ * false refusal), one more must refuse. */+static void write_stamp_cap_fixture(const char *dir, int n_entries){+#ifdef _WIN32+ mkdir(dir);+#else+ mkdir(dir,0755);+#endif+ size_t inner_cap = (size_t)n_entries*40 + 64;+ char *inner = malloc(inner_cap);+ size_t p = 0; inner[p++]='{';+ for(int i=0;i<n_entries;i++)+ p += (size_t)snprintf(inner+p, inner_cap-p, "%s\"stamp_%d\":\"int8-row\"", i?",":"", i);+ inner[p++]='}'; inner[p]=0;++ /* JSON-escape `inner` into a quoted string literal for the OUTER header's+ * colibri.fmt VALUE (double-JSON-encoding, same shape write_stamp_fixture's+ * callers hand-escape for small fixtures -- built programmatically here+ * since n_entries is too many to hand-escape). */+ char *esc = malloc(p*2 + 4);+ size_t q = 0; esc[q++]='"';+ for(size_t i=0;i<p;i++){+ char c = inner[i];+ if(c=='"' || c=='\\') esc[q++]='\\';+ esc[q++]=c;+ }+ esc[q++]='"'; esc[q]=0;+ free(inner);++ enum { O=1, I=1 };+ static uint8_t q1[O*I]; static float s1[O];+ q1[0]=1; s1[0]=0.01f;+ char path[300]; snprintf(path,sizeof path,"%s/model.safetensors",dir);+ int64_t nb=(int64_t)O*I, ns=(int64_t)O*4;+ char *hdr = malloc(q + 512);+ int hl = snprintf(hdr, q+512,+ "{\"__metadata__\":{\"colibri.fmt\":%s},"+ "\"w\":{\"dtype\":\"U8\",\"shape\":[%lld],\"data_offsets\":[0,%lld]},"+ "\"w.qs\":{\"dtype\":\"F32\",\"shape\":[%lld],\"data_offsets\":[%lld,%lld]}}",+ esc,+ (long long)nb,(long long)nb,+ (long long)(ns/4),(long long)nb,(long long)(nb+ns));+ free(esc);+ FILE *f=fopen(path,"wb");+ if(!f){ printf("FAIL: cannot create %s (run from c/, like tools/run_tests.py does)\n", path); fails++; free(hdr); return; }+ uint64_t hlen=(uint64_t)hl;+ fwrite(&hlen,8,1,f); fwrite(hdr,1,(size_t)hl,f);+ fwrite(q1,1,(size_t)nb,f); fwrite(s1,1,(size_t)ns,f);+ fclose(f);+ free(hdr);+}++static void test_stamp_map_cap_boundary_ok(void){+ const char *dir="tests/tmp_fp8_stamp_cap_ok";+ write_stamp_cap_fixture(dir, ST_FMT_STAMP_MAX); /* exactly at the cap: must NOT refuse */+ static Model gm; memset(&gm,0,sizeof gm);+ st_init(&gm.S, dir);+ CHECK(gm.S.fmt_n == ST_FMT_STAMP_MAX);+ char p[300]; snprintf(p,sizeof p,"%s/model.safetensors",dir); unlink(p); rmdir(dir);+}++static void test_stamp_map_cap_exceeded(void){+ const char *dir="tests/tmp_fp8_stamp_cap_over";+ write_stamp_cap_fixture(dir, ST_FMT_STAMP_MAX+1); /* one past the cap: must refuse */+#ifndef _WIN32+ int pipefd[2];+ if(pipe(pipefd)==0){+ pid_t pid = fork();+ if(pid == 0){+ dup2(pipefd[1],2); close(pipefd[0]); close(pipefd[1]);+ static Model gm; memset(&gm,0,sizeof gm);+ st_init(&gm.S, dir); /* must exit(1) inside st_fmt_stamp_ingest's cap check */+ _exit(42); /* reaching here is the bug */+ } else if(pid > 0){+ close(pipefd[1]);+ char err[1024]={0}; ssize_t n=read(pipefd[0],err,sizeof(err)-1); (void)n;+ close(pipefd[0]);+ int status=0; waitpid(pid,&status,0);+ int ok = WIFEXITED(status) && WEXITSTATUS(status)==1;+ if(!ok) printf("FAIL stamp-map cap exceeded: expected exit(1), got status=%d, stderr=%.200s\n", status, err);+ CHECK(ok);+ if(ok && !strstr(err,"refus")){+ printf("FAIL stamp-map cap exceeded: exited(1) but message lacked a refusal explanation: %.200s\n", err);+ fails++;+ }+ } else fails++;+ } else fails++;+#else+ printf("skipped on Windows (no fork): stamp-map cap exceeded\n");+#endif+ char p[300]; snprintf(p,sizeof p,"%s/model.safetensors",dir); unlink(p); rmdir(dir);+}+int main(void){test_disambiguation();test_fmt6_fp8_collision();@@ -982,6 +1090,8 @@ int main(void){test_stamp_mismatching();test_stamp_unrecognized_name();test_stamp_absent();+ test_stamp_map_cap_boundary_ok();+ test_stamp_map_cap_exceeded();if(fails){ printf("fp8 loader-seam tests: %d FAILED\n", fails); return 1; }printf("fp8 loader-seam tests: ok\n");return 0;diff --git a/docs/FORMATS.md b/docs/FORMATS.mdindex 6c52379f..dd2f27b1 100644--- a/docs/FORMATS.md+++ b/docs/FORMATS.md@@ -203,6 +203,80 @@ reference implementation of this proposal:documentation for the exact rule, including the cases where even acorrect stamp still can't resolve one (the UE8M0 corners above).+### Non-retroactivity++The stamp is a **forward convention**, not a migration path. It describes+containers written *going forward* by a tool that has been updated to write+`__metadata__["colibri.fmt"]` β it says nothing about, and does nothing to,+containers that already exist. A pre-convention container (anything written+before this feature existed β including every container this repo's own+tooling has produced to date, and every upstream Z.ai/DeepSeek checkpoint)+is simply **unstamped**: `st_fmt_stamp` returns `NULL` for every tensor in+it, `qt_verify_fmt_stamp` is a no-op, and `qt_resolve_fmt` resolves it by+byte arithmetic alone, exactly as it did before this PR. There is no+in-place upgrade path, no "adopt the stamp on an existing container"+tooling, and none is implied by anything in this document β the only way a+container gains a stamp is to be produced (or re-produced) by a tool that+writes one. An absent stamp is therefore never itself a signal that+anything is wrong with a container; it is the default, expected state for+everything that predates this convention (which, as of this PR, is+everything).++### Stamp-map scan bound++`st_fmt_stamp_ingest` (`c/st.h`) caps the total number of stamped-tensor+entries it will ingest across a container's shards at `ST_FMT_STAMP_MAX`+(4096) and refuses loudly (`exit(1)`) if a container's combined+`__metadata__["colibri.fmt"]` entries exceed it. This is a **cap, not a+switch to a hash table**: stamps are a resident-tensor convention β a+handful to a few hundred tensors per model (`q_a`/`q_b`/`kv_a`/`kv_b_proj`,+`o_proj`, shared-expert and dense-MLP gate/up/down; see the resident-role+census in `c/tools/fp8_collision_census.py`), **never** the tens of+thousands of routed-expert tensors a large MoE checkpoint carries. A+container whose combined stamp map exceeds 4096 entries is not using this+convention as it's designed to be used β most plausibly a malformed or+adversarial container trying to force an unbounded ingest allocation during+header parsing, before any size/shape validation has even run. Refusing+loudly at that bound is the same "untrusted container, refuse rather than+guess" discipline `qt_resolve_fmt` applies everywhere else in this feature.++### Discovery-time abort surface++`st_fmt_stamp_ingest`'s `exit(1)` calls (a `colibri.fmt` value that isn't a+JSON string, one that doesn't parse to a JSON object, an entry whose value+isn't a string, or the 4096-entry cap above) all fire from inside+`st_init_multi`'s shard-header-parse loop β i.e. at **container discovery+time**, while the engine is still building its tensor index, before it has+resolved a single tensor against the model's architecture or read one byte+of weight data. This is a coarser-grained, EARLIER abort surface than+`qt_resolve_fmt`/`qt_verify_fmt_stamp`'s own per-tensor refusals (which fire+much later, once a *specific* tensor's `[O,I]` shape and stamp are both+known during weight load): a malformed stamp anywhere in any shard aborts+the *entire* model load immediately, before the user ever sees which layer+or tensor was implicated. Operationally, a stamp-related `exit(1)` whose+message names a *file* (shard path) rather than a *tensor* is this+discovery-time surface; one that names a tensor is the later, per-tensor+one.++### Scope: `.qs`-backed tensors only++The stamp lookup (`st_fmt_stamp`, called from `qt_from_disk`) is consulted+**only** inside `qt_from_disk`'s `st_has(&m->S, name+".qs")` branch β i.e.+only for a tensor that already carries a quantized `.qs` scale sidecar. A+`colibri.fmt` entry naming some *other* tensor (a raw f32/bf16 weight, a+norm, a router, embed/lm_head) is parsed and stored exactly like any other+entry β `st_fmt_stamp_ingest` has no way to know, and does not check,+whether the name it's ingesting belongs to a `.qs`-backed tensor β but it is+then **silently ignored by design**: `qt_from_disk`'s plain f32/bf16 read+path never calls `st_fmt_stamp` for it, so nothing ever consults that entry.+This is intentional, not an oversight this document is patching over: the+whole point of the stamp is to disambiguate a *byte-count* collision among+quantized formats, and only a `.qs`-backed tensor can have one β stamping+anything else has no failure mode to guard against, so the convention simply+doesn't extend its verification there. A future stamping tool that writes+`colibri.fmt` entries for non-`.qs` tensors would not break anything, but+those entries would have no effect.+This is offered as a **reference implementation**, not a mandate: a futurepublic format registry could standardize the metadata key and stamp shape(`colibri.fmt` and its JSON-map-of-name convention are this PR's proposal,