Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

docs: decision engines, Laya, and switching from Jev

JustVugg4f88fbd37564
diff --git a/docs/ENVIRONMENT.md b/docs/ENVIRONMENT.md
index 442f67f5..b5828fc2 100644
--- a/docs/ENVIRONMENT.md
+++ b/docs/ENVIRONMENT.md
@@ -561,6 +561,15 @@ Read **only** by `c/olmoe.c`. This is the sister engine used for streaming-cache
| `CONF_LIMIT` | `0.92` | Confidence ceiling for the router prediction. Clamped to [0.1, 1.0]. |
| `EXPERT_DROP` | `0` (off) | Drop experts below the confidence threshold instead of loading them (quality/speed experiment). |
+## Laya engine (`laya`)
+
+Read **only** by `c/laya.c`, the decision engine ([laya.md](laya.md)).
+
+| Variable | Default | Effect |
+|---|---|---|
+| `COLI_LAYA_MAX_LEN` | the checkpoint's `max_len` | Tokens per question sequence (512 on the English checkpoint, capped at the encoder's 8192 positions). |
+| `COLI_LAYA_HEAD_MAX_LEN` | the checkpoint's `head_max_len` | Tokens shared by a question's instructions and options; raise it for questions with many options. |
+
---
## Server / CLI (`openai_server.py`, `coli`)
diff --git a/docs/brio.md b/docs/brio.md
index 01b7dba4..5c7d67a6 100644
--- a/docs/brio.md
+++ b/docs/brio.md
@@ -181,12 +181,38 @@ const { json, fields } = await r.json();
// json.queue, json.urgent are guaranteed to be values from your lists
```
-### For code written against Jev: `POST /v1/systemone`
+### Switching from Jev: change the base URL, keep your code
-If your application already speaks TypeSafe's Jev API, point it at colibri
-and change the base URL; the request and the reply are the same shape. The
-route accepts any `model` name (a Jev client sends `jev-latest`) and answers
-with the model the server actually runs.
+colibri answers `POST /v1/systemone` with the request and the reply of
+TypeSafe's Jev API. Code written against Jev keeps working: point it at
+colibri and change nothing else. The two official SDKs, unmodified, are
+tested against `coli serve` on a language model and on a decision engine
+(`tests/test_jev_sdk.py`, `make -C c jev-sdk-check`).
+
+Python, `typesafe_sdk`:
+
+```python
+from typesafe_sdk import TypeSafeClient
+
+client = TypeSafeClient(api_key="your-colibri-key", base_url="http://127.0.0.1:8000")
+result = client.system_one(model="jev-latest", state=ticket, questions=questions)
+print(result.choices["department"].choice, result.nouls["urgency"].noul)
+```
+
+TypeScript, `@typesafe-ai/sdk`:
+
+```ts
+import { TypeSafeClient } from "@typesafe-ai/sdk";
+
+const client = new TypeSafeClient({ apiKey: "your-colibri-key", baseURL: "http://127.0.0.1:8000" });
+const { answers } = await client.systemOne({ state: ticket, questions });
+```
+
+Or leave the code as it is and set `TYPESAFE_BASE_URL=http://127.0.0.1:8000`
+and `TYPESAFE_API_KEY` (the server's `COLI_API_KEY`; the SDKs want a
+non-empty key, and a server started without one accepts any). The SDKs
+POST to `<base_url>/v1/systemone` and list models at `<base_url>/v1/models`;
+both routes accept any `model` name (a Jev client sends `jev-latest`).
```bash
curl -s http://127.0.0.1:8000/v1/systemone \
@@ -210,45 +236,65 @@ The reply, from Qwen3.6-35B-A3B on a CPU box (1m46 with the experts streamed
from disk, the state read once for the three questions):
```json
-{"model": "qwen36",
+{"id": "req_5f0c...", "model": "qwen36", "provider": "colibri",
"answers": {
"urgency": {"type": "noul", "noul": 0.847606},
"department": {"type": "choice", "choice": "technical",
"probabilities": {"billing": 0.059806, "technical": 0.939983, "sales": 0.000211},
"confidence": 0.909974},
- "severity": {"type": "score", "score": 3.831772,
- "legend": {"1": "no impact", "2": "minor inconvenience", "3": "blocked on one task",
- "4": "losing money", "5": "business down"},
- "probabilities": {"1": 0.044403, "2": 0.02624, "3": 0.034899, "4": 0.842097, "5": 0.052361},
+ "severity": {"type": "score", "score": 2.831772,
+ "legend": {"0": "no impact", "1": "minor inconvenience", "2": "blocked on one task",
+ "3": "losing money", "4": "business down"},
+ "probabilities": {"0": 0.044403, "1": 0.02624, "2": 0.034899, "3": 0.842097, "4": 0.052361},
"confidence": 0.802621}},
- "usage": {"input_tokens": 86, "output_tokens": 15}}
+ "usage": {"input_tokens": 86, "output_tokens": 15, "cost": 0}}
```
-How the three primitives map onto the `questions` form above, so you know
-what the model is actually asked:
+How the three primitives map onto a language model, so you know what it is
+actually asked:
| Jev primitive | what is scored | what goes into the question text | reply |
|---|---|---|---|
| `noul` | `yes` / `no` | `instructions`, then `yes: <criteria.true>` and `no: <criteria.false>` when given, then "Answer yes or no." | `noul` = probability of yes |
| `choice` | the labels of `criteria` (up to 255) | `instructions`, then one line per label with its description | `choice`, `probabilities` by label, `confidence` |
-| `score` | the level numbers `"1".."n"` (2 to 10 levels) | `instructions`, then `k: <description>` per level, then "Answer with the number." | `score` = expected value under the distribution, `legend` by number, `probabilities`, `confidence` |
+| `score` | the level numbers `"1".."n"` (1 to 255 levels) | `instructions`, then `k: <description>` per level, then "Answer with the number." | `score` = expected level under the distribution, `legend` and `probabilities` keyed `"0".."n-1"` as Jev numbers them, `confidence` |
-`state` and `instructions` may be a string, an object or an array; JSON is
-serialized as text. `confidence` is the formula their documentation gives,
-`(n * peak - 1) / (n - 1)`: 1 when all the mass sits on one label, 0 when
-flat; `noul` carries none, as in theirs.
+`state`, `instructions` and every criterion may be a string, an object or an
+array; JSON is serialized as text. `confidence` is the formula their
+documentation gives, `(n * peak - 1) / (n - 1)`: 1 when all the mass sits on
+one label, 0 when flat; `noul` carries none, as in theirs. A question with a
+single label or level is answered without asking the model.
What differs, stated rather than hidden:
-- `model` in the reply is the served model, not a Jev version.
-- `usage.output_tokens` counts the option tokens the engine **read**; nothing
- is generated. `input_tokens` is the longest prompt of the request.
+- `model` is the served model, `provider` is `"colibri"`, `usage.cost` is 0,
+ and `id` is the request id (also the `x-typesafe-request-id` header).
+- `usage.output_tokens` counts the option tokens a language model **read**;
+ nothing is generated. `input_tokens` is the longest prompt of the request.
+ A decision engine reports the tokens it read as `input_tokens` and 0 output.
- Validation errors are `422`, as theirs, with this server's error envelope
- (`error.message`, `error.param`).
+ (`error.message`, `error.param`) and Jev's `detail` list beside it.
+- `GET /v1/models` lists the Jev card (`models`: name, description,
+ release_date) next to OpenAI's `data`.
- The probabilities are this model's, normalised over the options with the
- `mean` rule above. The numbers will not match Jev's on the same input; the
+ `sum` rule below. The numbers will not match Jev's on the same input; the
contract is the same, the model is yours.
+#### colibri's options
+
+None of these is sent by a Jev client; each is optional.
+
+| field | meaning |
+|---|---|
+| `normalize` | `"sum"` (default) or `"mean"`: how a language model's option log-probabilities become one score per option, see below |
+| `pin_state` | `true` / `false`: photograph the state for the next request. Left out, the server photographs it when two or more questions read it, or when the same state came back from an earlier request. A state that changes on every call (a game, a sensor feed) is read straight through, one engine call fewer per request |
+| `prefix` | a fixed text read before the state, photographed once per KV slot and retaken if the engine evicted it. For rules or instructions that stay while the state changes: on the tiny MiMo fixture, rules of 690 positions plus a frame cost 143 positions per request instead of 690 |
+| `cache_slot` | the KV slot, as on the chat endpoints; by default it is derived from the state (from the prefix, when there is one) |
+
+A decision engine has no log-probabilities and no photos: it ignores
+`normalize` and `pin_state` and refuses `prefix` (put that text in `state` or
+`instructions`); `cache_slot` reaches it as the DECIDE slot.
+
### Do not put the options in the prompt
Write the state and the question; leave the option list to the `options` field. On a
@@ -256,13 +302,18 @@ real case that list was 48 tokens of 123, and it is about half of what the mode
saves. Naming the options in the text also biases the scoring towards whichever one
the sentence happens to mention last.
-### `mean` or `sum`
+### `sum` or `mean`
-`logprob` is the log probability of the whole option string, so a two-token option
-is penalised against a one-token option simply for being longer. Measured on qwen36:
-summing picked `merge` where the same model's own greedy decoding said
-`request changes`; the mean per token agreed with generation. `mean` is therefore
-the default. `sum` is available when you want the literal probability of the string.
+`logprob` is the log probability of the whole option string; `mean_logprob`
+divides it by the option's tokens. `sum` is the default. Measured on a
+30-case code-review benchmark (ALLOW / REVIEW / DENY), `mean` answered DENY on
+all 30, a one-word README fix included, because DENY is two tokens and ALLOW
+one: a per-token average favours whichever option has more tokens. With `sum`
+the same suite scored ALLOW 10/10 and DENY 9/10. `mean` stays available, and
+the server warns when it is asked for options whose token counts differ. On
+qwen36 one case went the other way (`sum` picked `merge` where the model's
+own greedy answer was `request changes`), so when your options have very
+different lengths, look at both.
## Reading the entropy
@@ -440,3 +491,110 @@ Note that these are on a disk-streaming engine, where the prefill costs about
interleaved clients evict each other.
- **`normalize` is a policy, not a fact.** Neither mean nor sum is right for every
option set; if your options have very different lengths, look at both.
+
+## Decision engines
+
+A language model answers `/v1/systemone` by scoring each option through its
+logprob channel: a prompt per question, a read-out per option. A **decision
+engine** is a model built for the question itself. It reads the state and the
+typed questions and returns a probability per option in one forward pass,
+with the calibration its authors fitted. colibri serves it on the same
+endpoint and with the same reply, so a Jev client, the SDKs and the dashboard
+cannot tell which kind answered.
+
+| engine | model | doc |
+|---|---|---|
+| `c/laya` | Laya (Convai Innovations): ModernBERT encoder + decision head | [laya.md](laya.md) |
+
+A decision engine serves `POST /v1/systemone` only. Chat, completions,
+messages and `/v1/brio` answer 400 with a pointer to `/v1/systemone`, and
+`/v1/models` lists the model with `capabilities: ["systemone", "decision"]`
+(a language model says `["chat", "systemone"]`). The path is one round trip:
+the gateway checks the request, sends it to the engine as one record, and
+shapes the answer. Nothing is rendered into a prompt and no option is scored
+on its own. On Laya the gateway's share is about 2 ms per request: 8.1 ms at
+the client for three questions on the tiny fixture, 6.1 of them in the
+engine; 882 ms on the real checkpoint, 879.5 in the engine. The engine's own
+time is in the `x-colibri-engine-ms` header.
+
+### The contract, for an engine author
+
+Everything below is generic; `c/decide_serve.h` implements the engine side,
+so a new engine writes only its model and one function.
+
+**Handshake.** The engine announces itself on the `CAPS` line of the serve
+protocol: `decide=1` (it takes `DECIDE`) and `chat=0` (it generates nothing).
+An engine that also chats leaves `chat` out; the gateway then sends
+`/v1/systemone` to `DECIDE` and the chat endpoints to `SUBMIT` as before.
+
+**Request.** `DECIDE <id> <slot> <bytes>` followed by one JSON record:
+
+```json
+{"state": "{\"subject\": \"Duplicate charge\", \"body\": \"...\"}",
+ "state_type": "object",
+ "questions": [
+ {"id": "department", "type": "choice", "instructions": "Which department?",
+ "options": [{"label": "billing", "text": "invoices, payments, refunds"},
+ {"label": "other", "text": null}]},
+ {"id": "urgency", "type": "score", "instructions": "How urgent is this?",
+ "options": [{"label": "0", "text": "not urgent"}, {"label": "1", "text": "blocking"}]},
+ {"id": "churn", "type": "noul", "instructions": "Does the user threaten to leave?",
+ "options": [{"label": "false", "text": null}, {"label": "true", "text": null}]}]}
+```
+
+- Every text is already text. A string is kept exactly as sent; an object or
+ an array (a state, instructions, a criterion) is written as
+ `json.dumps(value, ensure_ascii=False)`, the serialization the reference
+ packages apply, so the engine reads the bytes its model was trained on.
+ `state_type` says what the caller sent (`string`, `object`, `array`, ...),
+ because a model may treat a list (a conversation) differently.
+- Options come in the caller's order: a choice's labels with their
+ descriptions (`null` for none), a score's levels from 0, a noul's `false`
+ then `true` with the optional criteria. Missing instructions get the same
+ default text the language-model path asks with.
+- `slot` is the KV slot, for an engine that keeps state between requests.
+
+**Answer.** One `DECISION <id> <bytes>` frame, then `DONE`:
+
+```json
+{"answers": [{"id": "department", "logits": [4.68, -2.78], "probs": [0.9804, 0.0196],
+ "temperature": 1.906, "actions": {"act": 1.0, "escalate": 0.0},
+ "tokens": 96, "state_tokens": 50, "state_dropped": 0}],
+ "input_tokens": 348, "engine_ms": 879.5}
+```
+
+`probs` are the engine's final probabilities, one per option in the record's
+order; the gateway builds the Jev answer from them alone (argmax for
+`choice`, the expected level for `score`, `probs[1]` for `noul`). `logits`,
+`temperature` and `actions` (a policy head such as Laya's act / escalate) are
+the engine's own record of how it got there. A record the engine refuses is
+`ERROR <id> DECIDE_INVALID <field>: <reason>`, which the client receives as a
+422 naming that field; `DECIDE_FAILED` is the engine's own failure (500).
+
+**Registry.** A family with `modality="decision"` and
+`FamilyCapabilities(decision=True)` in `c/family_registry.py`, and a way for
+`resolve_model` to recognise its checkpoint (Laya: `rl_agent_config.json`
+plus `encoder/config.json`, keyed `laya_<encoder model_type>`). `coli serve`
+and `coli web` then serve it, `coli info` and `coli plan` describe it, and
+`coli chat` / `coli run` refuse it with a pointer to the endpoint.
+
+**Tests.** A tiny fixture whose reference answers come from the model's own
+package (`tools/make_laya_tiny.py`, `tools/make_laya_ref.py`,
+`tests/test_laya_tiny.py`), and the fixture behind the real gateway
+(`tests/test_decision_serve.py`, `tests/test_jev_sdk.py`).
+
+### The next two
+
+- **GLiNER2.5-Decide** (DeBERTa-v3 encoder, GLiNER2 heads): a second encoder
+ engine on the same contract. It needs the DeBERTa-v3 encoder (relative
+ position buckets and disentangled attention, a SentencePiece tokenizer) and
+ its heads, its own rendering of a record into its input, and a
+ `resolve_model` rule for its checkpoint. `decide_serve.h`, the gateway and
+ the SDK tests stay as they are.
+- **Clef** (a joint schema head over Qwen3.8-27B, dense, with vision): the
+ backbone is the qwen36 engine's. It would answer `DECIDE` in that engine's
+ serve loop next to `SUBMIT`, announce `decide=1` without `chat=0` (the same
+ process still chats), run the backbone over the rendered record and apply
+ the head to its final hidden states. The record already carries what a
+ schema head needs (every question with its typed options); a picture in the
+ state would need the record to carry image parts, which today it does not.
diff --git a/docs/docs.json b/docs/docs.json
index fabfa78c..d33e8d54 100644
--- a/docs/docs.json
+++ b/docs/docs.json
@@ -23,7 +23,7 @@
},
{
"group": "Model engines",
- "pages": ["deepseek-v4", "kimi_k3", "qwen38", "qwen36", "inkling", "mimo", "qwen-image"]
+ "pages": ["deepseek-v4", "kimi_k3", "qwen38", "qwen36", "inkling", "mimo", "qwen-image", "laya"]
},
{
"group": "Configure and tune",
diff --git a/docs/laya.md b/docs/laya.md
new file mode 100644
index 00000000..5e80104d
--- /dev/null
+++ b/docs/laya.md
@@ -0,0 +1,142 @@
+# Laya on colibri
+
+[Laya](https://huggingface.co/convaiinnovations/laya), by Convai Innovations
+(Apache-2.0), is a decision model: give it a state (text or JSON) and typed
+questions (`choice`, `score`, `noul`) and it answers every question with
+calibrated probabilities in one forward pass. It never generates text.
+
+`c/laya` runs it from the official checkpoint with no conversion, and
+`coli serve` exposes it on `POST /v1/systemone`, the same request and reply
+as TypeSafe's Jev API and as every other model colibri serves
+([brio.md](brio.md#decision-engines)).
+
+## Download and run
+
+The English checkpoint is the root of the repository (842 MB):
+
+```sh
+hf download convaiinnovations/laya model.safetensors rl_agent_config.json \
+ "encoder/*" "tokenizer/*" --local-dir ~/Models/laya
+make -C c laya
+./c/coli serve --model ~/Models/laya
+```
+
+`coli` recognises the checkpoint by its `rl_agent_config.json` and the
+encoder's `encoder/config.json`; there is no root `config.json` to add.
+`coli info --model ~/Models/laya` shows the encoder and the token budget,
+`coli plan` the RAM.
+
+```sh
+curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
+ "model": "jev-latest",
+ "state": "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan.",
+ "questions": {
+ "department": {"type": "choice", "instructions": "Which department should handle this?",
+ "criteria": {"billing": "invoices, payments, refunds",
+ "technical": "bugs, outages, system errors", "other": "everything else"}},
+ "urgency": {"type": "score", "instructions": "How urgent is this?",
+ "criteria": ["not urgent", "soon", "blocking"]},
+ "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"}}}'
+```
+
+```json
+{"id": "req_e622...", "model": "laya", "provider": "colibri",
+ "answers": {
+ "department": {"type": "choice", "choice": "billing",
+ "probabilities": {"billing": 0.986516, "technical": 0.007975, "other": 0.005509},
+ "confidence": 0.979774},
+ "urgency": {"type": "score", "score": 1.772169,
+ "legend": {"0": "not urgent", "1": "soon", "2": "blocking"},
+ "probabilities": {"0": 0.038904, "1": 0.150022, "2": 0.811074}, "confidence": 0.71661},
+ "churn_risk": {"type": "noul", "noul": 0.879019}},
+ "usage": {"input_tokens": 164, "output_tokens": 0, "cost": 0}}
+```
+
+`input_tokens` is what the encoder read, every question's sequence counted.
+Chat, completions and messages answer 400 with a pointer to `/v1/systemone`,
+and `coli chat` says the same before starting anything.
+
+## What the engine computes
+
+Each step follows the `laya` package (0.3.24), which is the reference:
+
+| step | reference | in `c/laya.c` |
+|---|---|---|
+| one sequence per question | `common.build_head`, `build_sequence`, `render_options` | `[CLS] <type> question: <instructions> [SEP] [MASK] option ... [SEP] state [SEP]`; options capped at 48 tokens, shrunk evenly when they overflow `head_max_len`; the state cut at `max_len`, from the left for a list (a conversation keeps its newest turns) |
+| tokenizer | `tokenizer/tokenizer.json` | NFC, added tokens split out first, GPT-2 byte-level pre-tokenizer, BPE (`tok.h`) |
+| encoder | ModernBERT | 28 pre-norm layers, no norm before layer 0's attention; RoPE theta 160000 on the global layers (every third, from 0) and 10000 on the others, which see +-64 positions; GeGLU MLP; no biases; final LayerNorm |
+| head | `DecisionModel.forward` | `+ type_emb[type]`, two norm-first transformer layers (ReLU, 16 heads), the scorer (LayerNorm, Linear, GELU, Linear) at each option's `[MASK]` |
+| calibration | `Agent._decode_answers` | one temperature per type and option count (`choice:3-5`, `noul:2`, ...), clamped to [0.5, 5] like the package (`choice:11+` ships 0.1006), softmax over the question's options |
+| act head | `DecisionModel.forward` | `[h[0], top1, top1 - top2, entropy, k/255]` -> act / escalate; carried in the DECIDE answer, not in the Jev reply (the model card says it carries no usable signal yet) |
+
+Weights are read as stored (F16) and kept in f32: 1.7 GB of RAM. Every
+question of a request is one batch of rows; attention stays inside each
+question's sequence.
+
+## Checked against the reference
+
+- **Tiny oracle** (`make -C c laya-tiny-check`, in CI): a random checkpoint
+ in the release's layout (`tools/make_laya_tiny.py`, numpy) and what the
+ `laya` package answers on it (`laya_tiny/ref.json`, written by
+ `tools/make_laya_ref.py`): 12 requests with 25 questions plus one the
+ package refuses, covering every type, 2 to 20 options, option shrinking,
+ long states and conversations cut at `max_len`, JSON states and
+ instructions, `[MASK]` in the text, NFC and the added tokens. Token ids and
+ markers identical, probabilities within 4.8e-7 (tolerance 2e-5), every
+ decision the same; then the same fixture through `coli serve`.
+- **The real checkpoints** (`tools/compare_laya.py`): 36 requests, 117
+ questions (the model card's examples, the package's presets, documents cut
+ at 512 tokens, a 40-option question, unicode, emoji). Root checkpoint:
+ 117/117 decisions identical, max |dp| 2.7e-6. `typed-decisions`: 117/117,
+ max |dp| 1.2e-6.
+
+## Speed
+
+Measured on an i7-1355U laptop (10 cores), 6 threads for both, with other
+processes on the machine, so the numbers move by 20-30% from run to run:
+
+| | `c/laya` | `laya` package (torch, CPU, f32) |
+|---|---|---|
+| 36 requests, median per request | 1.9 s | 3.1 s |
+| one short question (41 tokens) | 300 ms | 250 ms |
+| README request, 3 questions, through `coli serve` | 882 ms, of which 2.5 ms gateway | |
+| Jev request, 1 question, through `coli serve` | 219 ms, of which 2.5 ms gateway | |
+
+The engine is ahead when a request has several questions and slightly behind
+on a single short one. There is no int8 option: measured on the same 117
+questions, int8 weights changed 1 decision and int8 weights with int8
+activations changed 4, which is not a trade a decision model should make.
+
+## Knobs
+
+| variable | default | meaning |
+|---|---|---|
+| `COLI_LAYA_MAX_LEN` | the checkpoint's `max_len` (512) | tokens per question sequence; the encoder reads up to 8192 |
+| `COLI_LAYA_HEAD_MAX_LEN` | the checkpoint's `head_max_len` (192) | tokens shared by the instructions and the options; raise it for many options |
+| `OMP_NUM_THREADS` | physical cores | as for the other engines |
+
+A question whose options do not fit is refused with a 422 that says how many
+markers fit, as the package refuses it.
+
+## The other checkpoints
+
+- **`typed-decisions/`** (same architecture, fine-tuned, `max_len` 1024):
+ point `--model` at the subfolder. Checked as above.
+- **`multilingual/`** (`laya-multilingual`, mmBERT-base, 22 layers x 768,
+ 256k vocabulary) is not supported yet. What it needs: a SentencePiece-style
+ BPE (`Metaspace` pre-tokenizer, a `Replace` normalizer, byte fallback,
+ 580k merges) in the tokenizer, `<bos>`/`<eos>`/`<mask>` as the special
+ tokens, RoPE 160000 on every layer (both thetas in its config), and the
+ long-context default (`max_len` 1024, up to 8192). The encoder and the head
+ are otherwise the code above; the registry already names it
+ (`Laya multilingual`, model id `laya-multilingual`).
+
+## Limits
+
+- English only on this checkpoint: the model card measures it collapsing on
+ non-Latin scripts while staying confident.
+- The shipped temperatures are fitted on the authors' data; the model card
+ advises refitting on yours. colibri applies the checkpoint's own values.
+- `noul` can follow its option labels rather than the state on this
+ checkpoint (model card); a two-option `choice` with neutral keys is the
+ workaround they give.
diff --git a/docs/serve_protocol.md b/docs/serve_protocol.md
index bbd0b0a4..0b046e5b 100644
--- a/docs/serve_protocol.md
+++ b/docs/serve_protocol.md
@@ -38,6 +38,13 @@ declares a tower whose tensors are not in the container). Engines that say nothi
leave the server's flag unknown. glm53, qwen38, qwen36 and deepseek_v41 emit it;
`openai_server.py` derives `/v1/models` `input_modalities` from it.
+Two more keys say what kind of engine answers. `decide=1`: the engine takes
+`DECIDE` (below), and the gateway sends `POST /v1/systemone` to it as one record
+instead of scoring options through the logprob channel. `chat=0`: it has nothing
+else, so the generating endpoints answer 400 with a pointer to `/v1/systemone`.
+A decision engine (laya) says `decide=1 chat=0`; an engine that chats and also
+decides natively would say `decide=1` alone.
+
## Requests (server β†’ engine)
```
@@ -45,6 +52,7 @@ SUBMIT <id> <slot> <bytes> <max_tokens> <temperature> <top_p>\n<payload>\n
IMAGE <id> <bytes> <grid_h> <grid_w>\n<payload>\n
STOP <id>\n
CANCEL <id>\n
+DECIDE <id> <slot> <bytes>\n<payload>\n
```
- `id` β€” non-zero u64, unique among in-flight requests.
@@ -67,6 +75,14 @@ CANCEL <id>\n
usage history, and KV state are persisted; the HTTP gateway uses it after a
client-provided stop sequence matches.
- `CANCEL` aborts a request after its client disconnects and returns `CANCELLED`.
+- `DECIDE` (only to an engine that announced `decide=1`) carries one decision
+ record, UTF-8 JSON: the state and the typed questions with their options in
+ order. The engine answers with one `DECISION` frame and `DONE`, or with
+ `ERROR <id> DECIDE_INVALID <reason>` for a record it refuses (the client's
+ 422; the reason starts with the field, `questions.<id>: ...`) and
+ `ERROR <id> DECIDE_FAILED <reason>` for its own failure. The record and the
+ answer are specified in [docs/brio.md, Decision engines](brio.md#decision-engines);
+ `c/decide_serve.h` parses one and writes the other.
- EOF on stdin = graceful shutdown: in-flight requests finish first.
Prefill is serial; decode is continuously batched β€” every active slot contributes
@@ -86,6 +102,13 @@ REPIN <layer> <eid> <old_tier> <gpu> # live re-pin swap events, as they ha
DONE <id> STAT <emitted> <tok_s> <hit_pct> <rss_gb> <prompt_tokens> <length_limited>
```
+and for `DECIDE`:
+
+```
+DECISION <id> <n>\n<n bytes of JSON>\n # once
+DONE <id> STAT 0 <tok_s> 0.0 <rss_gb> <tokens_read> 0
+```
+
Kimi K3 emits a zero-byte `TOOL` frame immediately after `ACCEPT` for every
chat request. That frame declares the sideband authoritative even when no tool
call follows. Real K3 special-token structure and its enclosed tool payload use
@@ -94,7 +117,8 @@ only resembles an XTML marker cannot be promoted into a client tool call.
Errors replace the stream: `ERROR <id> <CODE>` with codes `BAD_FRAME`, `BAD_REQUEST`,
`SLOT_BUSY`, `DUPLICATE_ID`, `EMPTY_PROMPT`, `NOT_FOUND` (CANCEL of unknown id),
-`CANCELLED`. A `CANCEL` is acknowledged by `ERROR <id> CANCELLED` after the slot's KV
+`CANCELLED`, and for decision engines `DECIDE_INVALID`, `DECIDE_FAILED` and
+`NOT_SUPPORTED` (a `SUBMIT` to an engine that does not generate). A `CANCEL` is acknowledged by `ERROR <id> CANCELLED` after the slot's KV
is persisted.
Immediately before each `DONE` the engine emits a telemetry block for the finished