Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Merge pull request #1861 into integ/release-batch-5

JustVuggc5a91fb03704
diff --cc c/st.h
index 1460a876,bc6aaeaf..ccd70e84
--- a/c/st.h
+++ b/c/st.h
@@@ -721,11 -724,9 +724,11 @@@ static void st_init_multi(shards *S, co
hdr[hlen] = 0;
int64_t data_start = 8 + (int64_t)hlen;
char *arena = NULL;
- jval *root = json_parse(hdr, &arena);
+ /* A partial object must not index weights or authorize an overlay.
+ * Header padding is JSON whitespace; embedded NUL is not padding. */
+ jval *root = memchr(hdr, 0, (size_t)hlen) ? NULL : json_parse_checked(hdr);
if (!root || root->t != J_OBJ) {
- fprintf(stderr, "%s: safetensors header is not a JSON object\n", files[fi]); exit(1); }
+ coli_load_fail(COLI_LOAD_FAIL_FORMAT, "%s: safetensors header is not a JSON object", files[fi]); }
st_fmt_stamp_ingest(S, root, files[fi]);
for (int i = 0; i < root->len; i++) {
const char *name = root->keys[i];
diff --cc c/tests/test_serve_codec.c
index fd8e9c5f,1e502f84..31298ff3
--- a/c/tests/test_serve_codec.c
+++ b/c/tests/test_serve_codec.c
@@@ -290,8 -282,8 +332,9 @@@ int main(void
test_invalid_headers_and_bodies();
test_writer_golden_bytes();
test_ready_caps_golden_bytes();
+ test_decide_and_decision();
test_allocation_failures_are_fatal_to_the_caller();
+ test_load_fail_kinds_and_line();
puts("serve codec tests: ok");
return 0;
}
diff --cc docs/serve_protocol.md
index 1b7889fe,11704a8c..e89ead31
--- a/docs/serve_protocol.md
+++ b/docs/serve_protocol.md
@@@ -38,15 -38,25 +38,34 @@@ declares a tower whose tensors are not
leave the server's flag unknown. glm53, qwen38, qwen36 and deepseek_v41 emit it;
`openai_server.py` derives `/v1/models` `input_modalities` from it.
+Two more keys say what kind of engine answers. `decide=1`: the engine takes
+`DECIDE` (below), and the gateway sends `POST /v1/systemone` to it as one record
+instead of scoring options through the logprob channel. `chat=0`: it has nothing
+else, so the generating endpoints answer 400 with a pointer to `/v1/systemone`.
+A decision engine (laya, gliner_decide) says `decide=1 chat=0`; an engine that chats and also
+decides natively says `decide=1` alone (qwen36 with Clef's head). `decide_record=raw`
+asks for the record in its raw form, the caller's own values (docs/systemone.md,
+"Decision engines").
+
+ An engine that cannot load its model says why on this channel, instead of
+ `READY`, and exits:
+
+ ```
+ LOAD_FAIL kind=<nomem|io|format|unsupported> <detail>
+ ```
+
+ It is the last line such an engine writes. `nomem`: the host refused memory
+ (ENOMEM/EAGAIN from malloc, mmap or mlock); `io`: the file could not be reached or
+ read (ENOENT, EACCES, EIO, ...); `format`: the file was read but is not what it
+ claims (short read, bad header); `unsupported`: well-formed, but a dtype or geometry
+ this engine does not serve. The detail is the same text the engine wrote to
+ stderr. `openai_server.py` raises `EngineLoadError(kind, detail)` from it and logs
+ `[gateway] engine load failed: kind=<kind> <detail>`; an engine that exits with no
+ such line is still reported as "colibri engine exited unexpectedly". The shared
+ safetensors path (`st.h`, every family) emits it through `coli_load_fail()`
+ (`load_fail.h`); a family's own load-time refusals adopt the same call as they are
+ touched.
+
## Requests (server → engine)
```