Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Merge origin/dev (GLiNER2.5-Decide) into feat/decide-clef

JustVuggcca4c1dcdd16
diff --cc .github/workflows/ci.yml
index 43710713,e6c9c94d..d2fee69e
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@@ -1165,21 -1165,19 +1165,34 @@@ jobs
- name: Generate the fixture, build laya, compare with the committed reference
run: make -C c laya-tiny-check
+ gliner-decide-tiny-check:
+ name: GLiNER2.5-Decide tiny oracle (vs the gliner2 package) and POST /v1/systemone end to end
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-python@v5
+ with:
+ python-version: '3.12'
+ - name: Install the fixture generator's one dependency
+ run: pip install numpy
+ - name: Generate the fixture, build gliner_decide, compare with the committed reference
+ run: make -C c gliner-decide-tiny-check
+
+ clef-tiny-check:
+ name: Clef tiny oracle (vs joint_schema_model.py) and POST /v1/systemone on qwen36
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-python@v5
+ with:
+ python-version: '3.12'
+ cache: pip
+ cache-dependency-path: c/tools/oracle-requirements.txt
+ - name: Install the generator's numpy and the converter's torch + safetensors
+ run: pip install numpy -r c/tools/oracle-requirements.txt
+ - name: Generate and convert the fixture, build qwen36, compare with the committed reference
+ run: make -C c clef-tiny-check
+
jev-sdk-check:
name: TypeSafe's SDKs (Python, TypeScript) against coli serve, base URL changed only
runs-on: ubuntu-latest
@@@ -1191,12 -1189,11 +1204,12 @@@
- uses: actions/setup-node@v4
with:
node-version: '20'
- - name: Install the pinned SDKs
+ - name: Install the pinned SDKs, and torch for the Clef fixture's converter
run: |
pip install -r c/tools/requirements-jev-sdk.txt
+ pip install numpy -r c/tools/oracle-requirements.txt
cd c/tools/jev-sdk && npm ci
- - name: Both SDKs against a language model (tiny MiMo), a decision engine (tiny Laya) and Clef
- - name: Both SDKs against a language model (tiny MiMo) and the decision engines (tiny Laya, tiny GLiNER2.5-Decide)
++ - name: Both SDKs against a language model (tiny MiMo), the decision engines (tiny Laya, tiny GLiNER2.5-Decide) and Clef
run: make -C c jev-sdk-check
qwenimage-tiny-check:
diff --cc README.it.md
index 9f3e22ea,19c1cd47..01ea067d
--- a/README.it.md
+++ b/README.it.md
@@@ -551,7 -551,7 +551,8 @@@ altre nove famiglie di modelli linguist
| **OLMoE** (AI2) | 7B / 1B | convertito con `c/tools/convert_olmoe_merged.py`: container **int8**, ~7 GB | `make -C c olmoe` | - |
| **Qwen-Image-2.1** (Alibaba) | modello di immagini | [`Qwen/Qwen-Image-2.1`](https://huggingface.co/Qwen/Qwen-Image-2.1) (~33 GB), checkpoint diffusers ufficiale, **nessuna conversione**: il text encoder e il diffusion transformer vengono quantizzati a int8 durante il caricamento. Immagini dentro `coli chat`, `POST /v1/images/generations` su `coli serve`. Qwen Research License: solo uso non commerciale | `make -C c qwenimage` | [qwen-image.md](docs/qwen-image.md) |
| **Laya** (Convai Innovations) | modello di decisione, 421M | [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) (842 MB), checkpoint ufficiale, **nessuna conversione**: un encoder ModernBERT con una testa di decisione che risponde a domande tipizzate (choice, score, noul) con probabilità calibrate invece di generare. Servito su `POST /v1/systemone` da `coli serve`. Apache-2.0 | `make -C c laya` | [laya.md](docs/laya.md) |
+ | **GLiNER2.5-Decide** (fastino) | modello di decisione, 340M | [`fastino/GLiNER2.5-Decide`](https://huggingface.co/fastino/GLiNER2.5-Decide) (1,9 GB), checkpoint ufficiale, **nessuna conversione**: un encoder DeBERTa-v3 con la testa di classificazione di GLiNER2; le domande di una richiesta e lo stato si leggono in un solo passaggio e ogni opzione riceve una probabilità. Servito su `POST /v1/systemone` da `coli serve`. Apache-2.0 | `make -C c gliner_decide` | [gliner_decide.md](docs/gliner_decide.md) |
+| **Clef** (Cloudflare) | modello di decisione + chat, 27B | [`Cloudflare/clef`](https://huggingface.co/Cloudflare/clef) (55 GB), convertito con `c/tools/convert_qwen36.py` (52 GB, testa inclusa): Qwen3.8-27B riaddestrato con una testa congiunta che dà un punteggio a ogni opzione di ogni domanda in un solo passaggio. `POST /v1/systemone` risponde in modo nativo, la chat funziona come su Qwen3.8-27B. Apache-2.0 | `make -C c qwen36` | [clef.md](docs/clef.md) |
Qwen3.6 offre tre container pre-convertiti: **int4-gs64** (consigliato: coseno
misurato rispetto all'ancora int8 da 0,98777 a 0,99313 e KL da 0,109 a 0,080
diff --cc README.ja.md
index 621c4d37,fabfd4d2..4fa4218f
--- a/README.ja.md
+++ b/README.ja.md
@@@ -511,7 -511,7 +511,8 @@@ GLM-5.2 がリファレンスモデル�
| **OLMoE**(AI2) | 7B / 1B | `c/tools/convert_olmoe_merged.py` で変換 — **int8** コンテナ、約 7 GB | `make -C c olmoe` | — |
| **Qwen-Image-2.1**(Alibaba) | 画像モデル | [`Qwen/Qwen-Image-2.1`](https://huggingface.co/Qwen/Qwen-Image-2.1)(約 33 GB)、公式 diffusers チェックポイント、**変換不要**: テキストエンコーダと拡散トランスフォーマーはロード時に int8 に量子化。`coli chat` では画像をインライン表示し、`coli serve` では `POST /v1/images/generations` で提供。Qwen Research License: 非商用利用のみ | `make -C c qwenimage` | [qwen-image.md](docs/qwen-image.md) |
| **Laya**(Convai Innovations) | 判定モデル、421M | [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya)(842 MB)、公式チェックポイント、**変換不要**: ModernBERT エンコーダと判定ヘッドで、型付きの質問(choice、score、noul)に生成ではなく較正済みの確率で答える。`coli serve` の `POST /v1/systemone` で提供。Apache-2.0 | `make -C c laya` | [laya.md](docs/laya.md) |
+ | **GLiNER2.5-Decide**(fastino) | 判定モデル、340M | [`fastino/GLiNER2.5-Decide`](https://huggingface.co/fastino/GLiNER2.5-Decide)(1.9 GB)、公式チェックポイント、**変換不要**: DeBERTa-v3 エンコーダと GLiNER2 の分類ヘッド。リクエストの全質問と状態を一度に読み、各選択肢に確率を返す。`coli serve` の `POST /v1/systemone` で提供。Apache-2.0 | `make -C c gliner_decide` | [gliner_decide.md](docs/gliner_decide.md) |
+| **Clef**(Cloudflare) | 判定モデル + チャット、27B | [`Cloudflare/clef`](https://huggingface.co/Cloudflare/clef)(55 GB)、`c/tools/convert_qwen36.py` で変換(52 GB、ヘッド込み): Qwen3.8-27B を追加学習し、すべての質問のすべての選択肢を 1 回のパスで採点する結合スキーマヘッドを付けたモデル。`POST /v1/systemone` にネイティブに答え、チャットは Qwen3.8-27B と同様に動作。Apache-2.0 | `make -C c qwen36` | [clef.md](docs/clef.md) |
Qwen3.6 には変換済みコンテナが 3 つあります: **int4-gs64**(推奨 — int8 のアンカーに対するコサイン類似度は
行単位と比べて 0.98777 → 0.99313、KL は 0.109 → 0.080 と計測されており、量子化誤差が約 44% 少ない)、
diff --cc README.md
index 2b0161c7,1f2dec8b..9c832821
--- a/README.md
+++ b/README.md
@@@ -532,7 -532,7 +532,8 @@@ or from `model_index.json` for the imag
| **OLMoE** (AI2) | 7B / 1B | converted with `c/tools/convert_olmoe_merged.py` — **int8** container, ~7 GB | `make -C c olmoe` | — |
| **Qwen-Image-2.1** (Alibaba) | image model | [`Qwen/Qwen-Image-2.1`](https://huggingface.co/Qwen/Qwen-Image-2.1) (~33 GB), official diffusers checkpoint, **no conversion**: the text encoder and the diffusion transformer are quantized to int8 while loading. Pictures inline in `coli chat`, `POST /v1/images/generations` on `coli serve`. Qwen Research License: non-commercial use only | `make -C c qwenimage` | [qwen-image.md](docs/qwen-image.md) |
| **Laya** (Convai Innovations) | decision model, 421M | [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) (842 MB), official checkpoint, **no conversion**: a ModernBERT encoder with a decision head that answers typed questions (choice, score, noul) with calibrated probabilities instead of generating. Served on `POST /v1/systemone` by `coli serve`. Apache-2.0 | `make -C c laya` | [laya.md](docs/laya.md) |
+ | **GLiNER2.5-Decide** (fastino) | decision model, 340M | [`fastino/GLiNER2.5-Decide`](https://huggingface.co/fastino/GLiNER2.5-Decide) (1.9 GB), official checkpoint, **no conversion**: a DeBERTa-v3 encoder with GLiNER2's classification head; the questions of a request and the state are read in one pass and every option gets a probability. Served on `POST /v1/systemone` by `coli serve`. Apache-2.0 | `make -C c gliner_decide` | [gliner_decide.md](docs/gliner_decide.md) |
+| **Clef** (Cloudflare) | decision model + chat, 27B | [`Cloudflare/clef`](https://huggingface.co/Cloudflare/clef) (55 GB), converted with `c/tools/convert_qwen36.py` (52 GB, head included): Qwen3.8-27B post-trained with a joint schema head that scores every option of every question in one pass. `POST /v1/systemone` answers natively, chat works as on Qwen3.8-27B. Apache-2.0 | `make -C c qwen36` | [clef.md](docs/clef.md) |
Qwen3.6 ships three pre-converted containers: **int4-gs64** (recommended — measured
cosine to the int8 anchor 0.98777 → 0.99313 and KL 0.109 → 0.080 against per-row, i.e.
diff --cc README.zh-CN.md
index 318f7d70,2d636a5d..f265e3e0
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@@ -464,7 -464,7 +464,8 @@@ GLM-5.2 是参考模型,但同样的�
| **OLMoE**(AI2) | 7B / 1B | 用 `c/tools/convert_olmoe_merged.py` 转换,**int8** 容器,约 7 GB | `make -C c olmoe` | 无 |
| **Qwen-Image-2.1**(Alibaba) | 图像模型 | [`Qwen/Qwen-Image-2.1`](https://huggingface.co/Qwen/Qwen-Image-2.1)(约 33 GB),官方 diffusers checkpoint,**无需转换**:文本编码器与扩散 transformer 在加载时量化为 int8。`coli chat` 中直接显示图片,`coli serve` 提供 `POST /v1/images/generations`。Qwen Research License:仅限非商业用途 | `make -C c qwenimage` | [qwen-image.md](docs/qwen-image.md) |
| **Laya**(Convai Innovations) | 决策模型,421M | [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya)(842 MB),官方 checkpoint,**无需转换**:ModernBERT 编码器加决策头,对类型化问题(choice、score、noul)给出校准后的概率,而不是生成文本。由 `coli serve` 在 `POST /v1/systemone` 上提供。Apache-2.0 | `make -C c laya` | [laya.md](docs/laya.md) |
+ | **GLiNER2.5-Decide**(fastino) | 决策模型,340M | [`fastino/GLiNER2.5-Decide`](https://huggingface.co/fastino/GLiNER2.5-Decide)(1.9 GB),官方 checkpoint,**无需转换**:DeBERTa-v3 编码器加 GLiNER2 的分类头,一次读完请求中的所有问题和状态,为每个选项给出概率。由 `coli serve` 在 `POST /v1/systemone` 上提供。Apache-2.0 | `make -C c gliner_decide` | [gliner_decide.md](docs/gliner_decide.md) |
+| **Clef**(Cloudflare) | 决策模型 + 聊天,27B | [`Cloudflare/clef`](https://huggingface.co/Cloudflare/clef)(55 GB),用 `c/tools/convert_qwen36.py` 转换(52 GB,含决策头):在 Qwen3.8-27B 上继续训练并加上联合模式头,一次前向为每个问题的每个选项打分。`POST /v1/systemone` 原生作答,聊天与 Qwen3.8-27B 相同。Apache-2.0 | `make -C c qwen36` | [clef.md](docs/clef.md) |
Qwen3.6 提供三个预转换容器:**int4-gs64**(推荐:与 per-row 相比,对 int8 基准的余弦相似度
实测从 0.98777 提升到 0.99313,KL 从 0.109 降到 0.080,即量化误差减少约 44%)、作为 A/B 基线的
diff --cc README.zh-TW.md
index 0491a121,1f06df47..7c41a3bc
--- a/README.zh-TW.md
+++ b/README.zh-TW.md
@@@ -465,7 -465,7 +465,8 @@@ GLM-5.2 是參考模型,但同樣的�
| **OLMoE**(AI2) | 7B / 1B | 以 `c/tools/convert_olmoe_merged.py` 轉換,**int8** 容器,約 7 GB | `make -C c olmoe` | 無 |
| **Qwen-Image-2.1**(Alibaba) | 圖像模型 | [`Qwen/Qwen-Image-2.1`](https://huggingface.co/Qwen/Qwen-Image-2.1)(約 33 GB),官方 diffusers checkpoint,**無需轉換**:文字編碼器與擴散 transformer 在載入時量化為 int8。`coli chat` 中直接顯示圖片,`coli serve` 提供 `POST /v1/images/generations`。Qwen Research License:僅限非商業用途 | `make -C c qwenimage` | [qwen-image.md](docs/qwen-image.md) |
| **Laya**(Convai Innovations) | 決策模型,421M | [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya)(842 MB),官方 checkpoint,**無需轉換**:ModernBERT 編碼器加決策頭,對型別化問題(choice、score、noul)給出校準後的機率,而不是生成文字。由 `coli serve` 在 `POST /v1/systemone` 上提供。Apache-2.0 | `make -C c laya` | [laya.md](docs/laya.md) |
+ | **GLiNER2.5-Decide**(fastino) | 決策模型,340M | [`fastino/GLiNER2.5-Decide`](https://huggingface.co/fastino/GLiNER2.5-Decide)(1.9 GB),官方 checkpoint,**無需轉換**:DeBERTa-v3 編碼器加 GLiNER2 的分類頭,一次讀完請求中的所有問題和狀態,為每個選項給出機率。由 `coli serve` 在 `POST /v1/systemone` 上提供。Apache-2.0 | `make -C c gliner_decide` | [gliner_decide.md](docs/gliner_decide.md) |
+| **Clef**(Cloudflare) | 決策模型 + 聊天,27B | [`Cloudflare/clef`](https://huggingface.co/Cloudflare/clef)(55 GB),用 `c/tools/convert_qwen36.py` 轉換(52 GB,含決策頭):在 Qwen3.8-27B 上繼續訓練並加上聯合模式頭,一次前向為每個問題的每個選項打分。`POST /v1/systemone` 原生作答,聊天與 Qwen3.8-27B 相同。Apache-2.0 | `make -C c qwen36` | [clef.md](docs/clef.md) |
Qwen3.6 提供三個預先轉換的容器:**int4-gs64**(推薦:與 per-row 相比,對 int8 基準的餘弦相似度
實測從 0.98777 提升到 0.99313,KL 從 0.109 降到 0.080,即量化誤差減少約 44%)、作為 A/B 基準的
diff --cc c/Makefile
index 6be1d41d,7500f825..cc65ea14
--- a/c/Makefile
+++ b/c/Makefile
@@@ -1303,27 -1314,27 +1314,42 @@@ laya-tiny-generate
laya-tiny-check: laya-tiny-generate laya$(EXE)
LAYA_TINY_REQUIRED=1 $(PYTHON) -m unittest -v tests.test_laya_tiny tests.test_decision_serve
+ # Tiny GLiNER2.5-Decide checkpoint + the gliner2 package's own answers on it,
+ # the same arrangement: tools/make_gliner_decide_tiny.py (numpy, deterministic)
+ # writes the checkpoint, gliner_decide_tiny/ref.json is committed and comes from
+ # the `gliner2` package (tools/make_gliner_decide_ref.py, maintainer-run,
+ # fixture SHA-256 pinned). The second module serves the fixture through the
+ # gateway: POST /v1/systemone end to end.
+ .PHONY: gliner-decide-tiny-generate gliner-decide-tiny-check
+ gliner-decide-tiny-generate:
+ $(PYTHON) tools/make_gliner_decide_tiny.py --output ./gliner_decide_tiny
+
+ gliner-decide-tiny-check: gliner-decide-tiny-generate gliner_decide$(EXE)
+ GLINER_DECIDE_TINY_REQUIRED=1 $(PYTHON) -m unittest -v tests.test_gliner_decide_tiny tests.test_decision_serve
+
+# Tiny Clef checkpoint (Cloudflare/clef's layout at toy widths) + the answers of
+# Clef's own joint_schema_model.py on it. The generator is numpy and
+# deterministic, the converter is the one users run (torch + safetensors);
+# clef_tiny/ref.json is committed and comes from tools/make_clef_ref.py
+# (maintainer-run, transformers 5.10.2, fixture SHA-256 pinned). The second
+# module is the same fixture through `coli serve`: POST /v1/systemone and a chat.
+.PHONY: clef-tiny-generate clef-tiny-check
+clef-tiny-generate:
+ $(PYTHON) tools/make_clef_tiny.py --output ./clef_tiny
+ $(PYTHON) tools/convert_qwen36.py --model ./clef_tiny --out ./clef_tiny_c --ebits 8 --no-readme >/dev/null
+
+clef-tiny-check: clef-tiny-generate qwen36$(EXE)
+ CLEF_TINY_REQUIRED=1 $(PYTHON) -m unittest -v tests.test_clef_tiny tests.test_decision_serve
+
# TypeSafe's own SDKs (Python and TypeScript), unmodified, against `coli serve`
- # on a language model (tiny MiMo), a decision engine (tiny Laya) and a chat
- # model with a decision head (tiny Clef on qwen36): switching from Jev is a
- # base-URL change. Needs the pinned SDKs first, and torch for the Clef converter:
-# on a language model (tiny MiMo) and the decision engines (tiny Laya, tiny
-# GLiNER2.5-Decide): switching from Jev is a base-URL change. Needs the pinned
-# SDKs first:
++# on a language model (tiny MiMo), the decision engines (tiny Laya, tiny
++# GLiNER2.5-Decide) and a chat model with a decision head (tiny Clef on qwen36):
++# switching from Jev is a base-URL change. Needs the pinned SDKs first, and
++# torch for the Clef converter:
# pip install -r tools/requirements-jev-sdk.txt && (cd tools/jev-sdk && npm ci)
.PHONY: jev-sdk-check
- jev-sdk-check: mimo$(EXE) laya$(EXE) qwen36$(EXE) mimo-tiny-generate laya-tiny-generate clef-tiny-generate
-jev-sdk-check: mimo$(EXE) laya$(EXE) gliner_decide$(EXE) mimo-tiny-generate laya-tiny-generate \
- gliner-decide-tiny-generate
++jev-sdk-check: mimo$(EXE) laya$(EXE) gliner_decide$(EXE) qwen36$(EXE) mimo-tiny-generate \
++ laya-tiny-generate gliner-decide-tiny-generate clef-tiny-generate
JEV_SDK_REQUIRED=1 $(PYTHON) -m unittest -v tests.test_jev_sdk
# In make check: every format and ragged edges against the naive product, and
diff --cc c/coli
index 8c4060ed,f0514c0a..4b537a47
--- a/c/coli
+++ b/c/coli
@@@ -273,25 -274,25 +274,40 @@@ def is_decision_model(model)
except (FamilyConfigError, UnknownFamilyError, OSError, TypeError, ValueError):
return False
+ # What bounds the tokens a decision engine reads, and the variable that moves it.
+ DECISION_LENGTH = {
+ "laya": ("at most its checkpoint's max_len per question", "COLI_LAYA_MAX_LEN"),
+ "gliner_decide": ("every question and the state in one sequence of at most 4096 tokens",
+ "COLI_GLINER_MAX_LEN"),
+ }
+
+ def is_gliner_decide(model):
+ try:
+ return resolve_model(model).descriptor.id == "gliner_decide"
+ except (FamilyConfigError, UnknownFamilyError, OSError, TypeError, ValueError):
+ return False
+
+def decision_head_line(model, resolved):
+ """`coli info`'s line for a decision head over a chat model (Clef): what it
+ adds, and that the chat stays."""
+ from family_registry import decision_head_of
+ head = decision_head_of(resolved)
+ try:
+ with open(os.path.join(model, head.files[0]), encoding="utf-8") as f:
+ cfg = json.load(f)
+ shape = (f"{cfg.get('routing_layers')} routing + {cfg.get('layers')} decoder layers, "
+ f"width {cfg.get('width')}")
+ except (OSError, ValueError, AttributeError):
+ shape = "head config unreadable"
+ return (f"{head.display_name} joint schema head ({shape}): POST /v1/systemone answers "
+ f"natively; chat, completions and messages work as on any {resolved.descriptor.id} model")
+
def decision_refusal(family):
- """What every chatting command says about a decision model, and where to go."""
+ """What every chatting command says about a decision model, and where to go.
+ Laya ships a calibration; GLiNER2.5-Decide gives its classifier's softmax."""
+ answers = "calibrated probabilities" if family.id == "laya" else "a probability per option"
return (f"{C.yel}{family.display_name} is a decision model:{C.r} it answers typed questions "
- f"(choice, score, noul) with calibrated probabilities and does not chat.\n"
+ f"(choice, score, noul) with {answers} and does not chat.\n"
f" Start it with: coli serve --model <dir> (or coli web for the dashboard)\n"
f" then POST /v1/systemone (docs/systemone.md, \"Decision engines\")")
diff --cc c/tests/test_decision_serve.py
index 32a2d53b,2bcd1e5c..06efe04b
--- a/c/tests/test_decision_serve.py
+++ b/c/tests/test_decision_serve.py
@@@ -16,15 -17,14 +17,19 @@@ pinned here
gateway: the probabilities that come back over HTTP are
the `laya` package's (laya_tiny/ref.json), and the
latency is split into gateway and engine time.
+ GlinerDecideTinyEndToEnd
+ the same for the gliner_decide engine on its tiny
+ fixture, against the `gliner2` package's answers
+ (gliner_decide_tiny/ref.json).
+ ClefTinyEndToEnd qwen36 with Clef's decision head (clef_tiny_c) behind the
+ real gateway: CAPS decide=1 decide_record=raw and no
+ chat=0, /v1/systemone gives Clef's own probabilities
+ (clef_tiny/ref.json), and the same engine still chats,
+ with a decision before and after the chat unchanged.
- LayaTinyEndToEnd and ClefTinyEndToEnd are skipped without their fixture and
- binary (`make laya-tiny-check` and `make clef-tiny-check` build them and turn
- the skip into a failure).
+ Each end-to-end class is skipped without its fixture and binary (`make
-laya-tiny-check` / `make gliner-decide-tiny-check` build them and turn the skip
-into a failure).
++laya-tiny-check` / `make gliner-decide-tiny-check` / `make clef-tiny-check`
++build them and turn the skip into a failure).
"""
import json
import math
@@@ -49,10 -49,9 +54,13 @@@ from test_openai_server import Blocking
FIXTURE = HERE / "laya_tiny"
BINARY = HERE / ("laya.exe" if os.name == "nt" else "laya")
REQUIRED = os.environ.get("LAYA_TINY_REQUIRED") == "1"
+ GLINER_FIXTURE = HERE / "gliner_decide_tiny"
+ GLINER_BINARY = HERE / ("gliner_decide.exe" if os.name == "nt" else "gliner_decide")
+ GLINER_REQUIRED = os.environ.get("GLINER_DECIDE_TINY_REQUIRED") == "1"
+CLEF_REF = HERE / "clef_tiny" / "ref.json"
+CLEF_CONTAINER = HERE / "clef_tiny_c"
+QWEN36 = HERE / ("qwen36.exe" if os.name == "nt" else "qwen36")
+CLEF_REQUIRED = os.environ.get("CLEF_TINY_REQUIRED") == "1"
TICKET = {"from": "[email protected]", "subject": "Duplicate charge on invoice #4411",
"body": "Hi, we were billed twice for March. Please refund the duplicate today or we "
@@@ -399,46 -402,23 +445,49 @@@ class DecisionRegistryAndLauncher(unitt
def coli(*args):
return subprocess.run([sys.executable, str(HERE / "coli"), *args], capture_output=True,
text=True, timeout=120, cwd=str(HERE))
- info = coli("info", "--model", str(FIXTURE))
- self.assertIn("decision model", info.stdout)
- plan = coli("plan", "--model", str(FIXTURE), "--json")
- self.assertEqual(json.loads(plan.stdout)["modality"], "decision")
- doctor = json.loads(coli("doctor", "--model", str(FIXTURE), "--json").stdout)
- self.assertIn({"id": "model.family", "status": "pass"},
- [{"id": c["id"], "status": c["status"]} for c in doctor["checks"]])
- for command in (("chat", "--model", str(FIXTURE), "--no-attach"),
- ("run", "--model", str(FIXTURE), "hello")):
- refused = coli(*command)
- self.assertNotEqual(refused.returncode, 0)
- self.assertIn("POST /v1/systemone", refused.stdout + refused.stderr)
+ for fixture in fixtures:
+ with self.subTest(fixture=fixture.name):
+ info = coli("info", "--model", str(fixture))
+ self.assertIn("decision model", info.stdout)
+ plan = coli("plan", "--model", str(fixture), "--json")
+ self.assertEqual(json.loads(plan.stdout)["modality"], "decision")
+ doctor = json.loads(coli("doctor", "--model", str(fixture), "--json").stdout)
+ checks = [{"id": c["id"], "status": c["status"]} for c in doctor["checks"]]
+ self.assertIn({"id": "model.family", "status": "pass"}, checks)
+ self.assertNotIn({"id": "model.files", "status": "fail"}, checks)
+ for command in (("chat", "--model", str(fixture), "--no-attach"),
+ ("run", "--model", str(fixture), "hello")):
+ refused = coli(*command)
+ self.assertNotEqual(refused.returncode, 0)
+ self.assertIn("POST /v1/systemone", refused.stdout + refused.stderr)
+ def test_the_registry_knows_a_clef_checkpoint(self):
+ import tempfile
+ from family_registry import (FamilyConfigError, checkpoint_decides, default_model_id,
+ display_for, resolve_model)
+ directory = tempfile.TemporaryDirectory()
+ self.addCleanup(directory.cleanup)
+ root = Path(directory.name)
+ text = {"model_type": "qwen3_5_text", "num_hidden_layers": 64, "hidden_size": 5120,
+ "intermediate_size": 17408}
+ (root / "config.json").write_text(json.dumps({"model_type": "qwen3_5", "text_config": text}),
+ encoding="utf-8")
+ stock = resolve_model(root)
+ self.assertEqual((stock.decision_head, display_for(stock)), ("", ("Qwen3.8-27B", "27B")))
+ self.assertFalse(checkpoint_decides(stock))
+ (root / "joint_head_config.json").write_text(json.dumps({"hidden_size": 5120}), encoding="utf-8")
+ with self.assertRaises(FamilyConfigError): # half a head
+ resolve_model(root)
+ (root / "joint_head.safetensors").write_bytes(b"")
+ clef = resolve_model(root)
+ self.assertEqual((clef.descriptor.id, clef.descriptor.modality), ("qwen36", "text"))
+ self.assertEqual(clef.decision_head, "clef")
+ self.assertTrue(checkpoint_decides(clef))
+ self.assertEqual(display_for(clef), ("Clef", "27B"))
+ self.assertEqual(default_model_id(clef), "clef")
+
+
def _laya_ready():
return (FIXTURE / "model.safetensors").exists() and BINARY.exists()
@@@ -542,148 -522,114 +591,257 @@@ class LayaTinyEndToEnd(unittest.TestCas
self.assertGreater(wall, engine)
+ def _gliner_ready():
+ return (GLINER_FIXTURE / "model.safetensors").exists() and GLINER_BINARY.exists()
+
+
+ class GlinerDecideTinyEndToEnd(unittest.TestCase):
+ """The gliner_decide engine behind the real gateway, on its tiny fixture."""
+
+ @classmethod
+ def setUpClass(cls):
+ if not _gliner_ready():
+ message = "gliner_decide or gliner_decide_tiny missing: run make gliner-decide-tiny-check"
+ if GLINER_REQUIRED:
+ raise AssertionError(message)
+ raise unittest.SkipTest(message)
+ ref = json.loads((GLINER_FIXTURE / "ref.json").read_text(encoding="utf-8"))
+ cls.max_len = ref["max_len"]
+ cls.ref = {case["name"]: case for case in ref["cases"]}
+
+ def setUp(self):
+ env = dict(os.environ, OMP_NUM_THREADS="2", COLI_GLINER_MAX_LEN=str(self.max_len))
+ engine = Engine(GLINER_BINARY, GLINER_FIXTURE, env=env, family=family_by_id("gliner_decide"))
+ self.addCleanup(engine.process.stdout.close)
+ self.addCleanup(engine.close)
+ self.server = APIServer(("127.0.0.1", 0), engine, "gliner-decide-tiny")
+ self.addCleanup(self.server.server_close)
+ self.addCleanup(self.server.shutdown)
+ threading.Thread(target=self.server.serve_forever, daemon=True).start()
+ self.base = f"http://127.0.0.1:{self.server.server_port}"
+
+ def post(self, path, body):
+ request = Request(self.base + path, data=json.dumps(body).encode(),
+ headers={"Content-Type": "application/json"})
+ with urlopen(request, timeout=120) as response:
+ return json.loads(response.read()), dict(response.headers)
+
+ def test_systemone_answers_with_the_reference_probabilities(self):
+ # the cases the route accepts as Jev requests (it refuses a bare number as a
+ # criterion, which the engine-level oracle still covers)
+ for name in ("ticket_three_types", "two_options_and_noul_criteria",
+ "twelve_labels_no_instructions", "long_state_cut", "conversation",
+ "json_state_and_instructions", "special_tokens_and_whitespace",
+ "urls_mail_handles", "unicode", "many_options", "empty_state"):
+ case = self.ref[name]
+ with self.subTest(case=name):
+ out, headers = self.post("/v1/systemone", {"model": "jev-latest",
+ "state": case["state"],
+ "questions": case["questions"]})
+ self.assertEqual(out["model"], "gliner-decide-tiny")
+ for want in case["answers"]:
+ got = out["answers"][want["id"]]
+ probs = want["probs"]
+ if got["type"] == "noul":
+ self.assertAlmostEqual(got["noul"], probs[1], delta=2e-5)
+ continue
+ values = list(got["probabilities"].values())
+ self.assertEqual(len(values), len(probs))
+ for a, b in zip(values, probs):
+ self.assertAlmostEqual(a, b, delta=2e-5)
+ if got["type"] == "choice":
+ labels = list(case["questions"][want["id"]]["criteria"])
+ self.assertEqual(got["choice"], labels[probs.index(max(probs))])
+ else:
+ expected = sum(i * p for i, p in enumerate(probs))
+ self.assertAlmostEqual(got["score"], expected, delta=1e-4)
+ # one sequence for the whole request
+ self.assertEqual(out["usage"]["input_tokens"], len(case["ids"]))
+ self.assertEqual(out["usage"]["output_tokens"], 0)
+ self.assertIn("x-colibri-engine-ms", headers)
+
+ def test_questions_that_do_not_fit_are_a_422(self):
+ case = self.ref["questions_over_max_len"]
+ request = Request(self.base + "/v1/systemone",
+ data=json.dumps({"state": case["state"], "questions": case["questions"]}).encode(),
+ headers={"Content-Type": "application/json"})
+ with self.assertRaises(HTTPError) as caught:
+ urlopen(request, timeout=60)
+ self.assertEqual(caught.exception.code, 422)
+ error = json.loads(caught.exception.read())["error"]
+ self.assertEqual(error["param"], "questions")
+ self.assertIn(case["refused"].split(": ", 1)[1], error["message"])
+
+ def test_chat_is_refused(self):
+ request = Request(self.base + "/v1/chat/completions",
+ data=json.dumps({"model": "gliner-decide-tiny",
+ "messages": [{"role": "user", "content": "hi"}]}).encode(),
+ headers={"Content-Type": "application/json"})
+ with self.assertRaises(HTTPError) as caught:
+ urlopen(request, timeout=10)
+ self.assertEqual(caught.exception.code, 400)
+
+ def test_latency_gateway_and_engine(self):
+ """Not a gate: wall time at the client, the engine's own compute time, and
+ the rest (HTTP, JSON, the pipe) as the gateway's share."""
+ case = self.ref["ticket_three_types"]
+ body = {"model": "jev-latest", "state": case["state"], "questions": case["questions"]}
+ self.post("/v1/systemone", body) # warm
+ walls, engines = [], []
+ for _ in range(10):
+ started = time.perf_counter()
+ _, headers = self.post("/v1/systemone", body)
+ walls.append((time.perf_counter() - started) * 1e3)
+ engines.append(float(headers["x-colibri-engine-ms"]))
+ walls.sort(), engines.sort()
+ wall, engine = walls[len(walls) // 2], engines[len(engines) // 2]
+ sys.stderr.write(f"\n[gliner_decide tiny e2e] /v1/systemone, 4 questions: median {wall:.1f} ms "
+ f"at the client, {engine:.1f} ms in the engine, {wall - engine:.1f} ms gateway\n")
+ self.assertGreater(wall, engine)
+
+
+def _clef_ready():
+ return CLEF_REF.exists() and (CLEF_CONTAINER / "joint_head.safetensors").exists() and QWEN36.exists()
+
+
+class ClefTinyEndToEnd(unittest.TestCase):
+ """qwen36 with Clef's head behind the real gateway, on the tiny fixture."""
+
+ # the cases the route accepts as Jev requests (it refuses a bare number as a
+ # criterion or an instruction, which the engine-level oracle still covers)
+ CASES = ("model_card_invoice", "model_card_systemone", "ticket_four_questions", "noul_variants",
+ "json_state", "conversation", "special_text", "many_questions", "single_option",
+ "empty_state")
+
+ @classmethod
+ def setUpClass(cls):
+ if not _clef_ready():
+ message = "qwen36, clef_tiny or clef_tiny_c missing: run make clef-tiny-check"
+ if CLEF_REQUIRED:
+ raise AssertionError(message)
+ raise unittest.SkipTest(message)
+ cls.ref = {case["name"]: case for case in
+ json.loads(CLEF_REF.read_text(encoding="utf-8"))["cases"]}
+
+ def setUp(self):
+ arch = patch.object(openai_server, "ARCH", "qwen36")
+ arch.start()
+ self.addCleanup(arch.stop)
+ engine = Engine(QWEN36, CLEF_CONTAINER, family=family_by_id("qwen36"),
+ env=dict(os.environ, COLI_DENSE_I8="0", OMP_NUM_THREADS="2",
+ COLI_NO_OMP_TUNE="1"))
+ self.engine = engine
+ self.addCleanup(engine.process.stdout.close)
+ self.addCleanup(engine.close)
+ self.server = APIServer(("127.0.0.1", 0), engine, "clef-tiny")
+ self.addCleanup(self.server.server_close)
+ self.addCleanup(self.server.shutdown)
+ threading.Thread(target=self.server.serve_forever, daemon=True).start()
+ self.base = f"http://127.0.0.1:{self.server.server_port}"
+
+ def post(self, path, body):
+ request = Request(self.base + path, data=json.dumps(body).encode(),
+ headers={"Content-Type": "application/json"})
+ with urlopen(request, timeout=300) as response:
+ return json.loads(response.read()), dict(response.headers)
+
+ def check(self, name, out):
+ case = self.ref[name]
+ for want in case["answers"]:
+ got = out["answers"][want["id"]]
+ p = dict(zip(want["option_ids"], want["probs"]))
+ if got["type"] == "noul":
+ self.assertAlmostEqual(got["noul"], p["true"], delta=2e-5, msg=name)
+ continue
+ self.assertEqual(set(got["probabilities"]), set(p), name)
+ for label, value in got["probabilities"].items():
+ self.assertAlmostEqual(value, p[label], delta=2e-5, msg=f"{name}.{want['id']}.{label}")
+ if got["type"] == "choice":
+ self.assertEqual(got["choice"], max(p, key=p.get), name)
+ self.assertEqual(out["usage"], {"input_tokens": len(case["input_ids"]), "output_tokens": 0,
+ "cost": 0})
+
+ def test_caps_say_decide_raw_and_chat(self):
+ self.assertTrue(self.engine.decides)
+ self.assertTrue(self.engine.chats)
+ self.assertEqual(self.engine.decide_record, "raw")
+ with urlopen(self.base + "/v1/models", timeout=10) as response:
+ card = json.loads(response.read())
+ self.assertEqual(card["data"][0]["capabilities"], ["chat", "systemone"])
+
+ def test_systemone_answers_with_clefs_probabilities(self):
+ for name in self.CASES:
+ case = self.ref[name]
+ with self.subTest(case=name):
+ out, headers = self.post("/v1/systemone", {"model": "jev-latest", "state": case["state"],
+ "questions": case["questions"]})
+ self.check(name, out)
+ self.assertIn("x-colibri-engine-ms", headers)
+
+ def test_the_same_engine_chats_between_two_decisions(self):
+ case = self.ref["model_card_systemone"]
+ body = {"state": case["state"], "questions": case["questions"]}
+ first, _ = self.post("/v1/systemone", body)
+ chat, _ = self.post("/v1/chat/completions", {"model": "clef-tiny", "max_tokens": 6,
+ "temperature": 0,
+ "messages": [{"role": "user", "content": "hi"}]})
+ self.assertEqual(chat["object"], "chat.completion")
+ self.assertGreater(chat["usage"]["completion_tokens"], 0)
+ second, _ = self.post("/v1/systemone", body)
+ self.assertEqual(second["answers"], first["answers"])
+ self.check("model_card_systemone", second)
+
+ def test_the_gateway_keeps_clefs_trunk_in_f16_when_the_budget_holds_it(self):
+ """int8 moves Clef's probabilities by up to 0.22 on the real checkpoint, f16 by
+ 0.012 (docs/clef.md): the gateway picks f16 when the planner's RAM budget
+ holds it, int8 otherwise, and an operator's own COLI_DENSE_BITS wins."""
+ env = {}
+ openai_server.decision_head_env(env, str(CLEF_CONTAINER))
+ self.assertEqual(env.get("COLI_DENSE_BITS"), "16")
+ env = {"RAM_GB": "0.001"}
+ self.assertIn("int8", openai_server.decision_head_env(env, str(CLEF_CONTAINER)))
+ self.assertNotIn("COLI_DENSE_BITS", env)
+ env = {"COLI_DENSE_BITS": "4"}
+ self.assertIsNone(openai_server.decision_head_env(env, str(CLEF_CONTAINER)))
+ self.assertEqual(env, {"COLI_DENSE_BITS": "4"})
+ self.assertIsNone(openai_server.decision_head_env({}, str(HERE / "laya_tiny")))
+
+ def test_a_schema_past_the_budget_is_a_422(self):
+ """COLI_CLEF_MAX_LEN is the reference's max_length: past it the schema alone
+ does not fit and the request is the caller's 422, not a 500."""
+ self.engine.close()
+ engine = Engine(QWEN36, CLEF_CONTAINER, family=family_by_id("qwen36"),
+ env=dict(os.environ, COLI_DENSE_I8="0", OMP_NUM_THREADS="2",
+ COLI_NO_OMP_TUNE="1", COLI_CLEF_MAX_LEN="600"))
+ self.addCleanup(engine.process.stdout.close)
+ self.addCleanup(engine.close)
+ self.server.engine = engine
+ case = self.ref["schema_too_long"]
+ request = Request(self.base + "/v1/systemone",
+ data=json.dumps({"state": case["state"], "questions": case["questions"]}).encode(),
+ headers={"Content-Type": "application/json"})
+ with self.assertRaises(HTTPError) as caught:
+ urlopen(request, timeout=120)
+ self.assertEqual(caught.exception.code, 422)
+ self.assertIn("1389", json.loads(caught.exception.read())["error"]["message"])
+
+ def test_latency_gateway_and_engine(self):
+ """Not a gate: wall time at the client against the engine's own time."""
+ case = self.ref["ticket_four_questions"]
+ body = {"model": "jev-latest", "state": case["state"], "questions": case["questions"]}
+ self.post("/v1/systemone", body)
+ walls, engines = [], []
+ for _ in range(5):
+ started = time.perf_counter()
+ _, headers = self.post("/v1/systemone", body)
+ walls.append((time.perf_counter() - started) * 1e3)
+ engines.append(float(headers["x-colibri-engine-ms"]))
+ walls.sort(), engines.sort()
+ wall, engine = walls[len(walls) // 2], engines[len(engines) // 2]
+ sys.stderr.write(f"\n[clef tiny e2e] /v1/systemone, 4 questions: median {wall:.1f} ms at the "
+ f"client, {engine:.1f} ms in the engine, {wall - engine:.1f} ms gateway\n")
+ self.assertGreater(wall, engine)
+
+
if __name__ == "__main__":
unittest.main()
diff --cc c/tests/test_jev_sdk.py
index 2a6b9de9,8611f50b..e44d7986
--- a/c/tests/test_jev_sdk.py
+++ b/c/tests/test_jev_sdk.py
@@@ -11,14 -11,13 +11,14 @@@ The official clients, unmodified, point
@typesafe-ai/sdk TypeScript, new TypeSafeClient({apiKey, baseURL})
.systemOne(...) and .models.list(), run under node
--Each one is driven against three servers, each a real `coli serve` process:
++Each one is driven against four servers, each a real `coli serve` process:
a language model (the tiny MiMo fixture, which scores options through the
- logprob channel), a decision engine (the tiny Laya fixture, DECIDE) and a
- chat model with a decision head (the tiny Clef on qwen36, DECIDE too). The
- same requests go to both: every question type, several questions at once,
-logprob channel) and two decision engines (the tiny Laya and GLiNER2.5-Decide
-fixtures, DECIDE). The same requests go to all three: every question type, several questions at once,
--text, object and array states, objects and arrays as instructions and
--criteria, and a request the server refuses (422), which each SDK must raise
--as its own UnprocessableEntity error.
++logprob channel), two decision engines (the tiny Laya and GLiNER2.5-Decide
++fixtures, DECIDE) and a chat model with a decision head (the tiny Clef on
++qwen36, DECIDE too). The same requests go to all four: every question type,
++several questions at once, text, object and array states, objects and arrays
++as instructions and criteria, and a request the server refuses (422), which
++each SDK must raise as its own UnprocessableEntity error.
Skipped when an SDK, the engines or the fixtures are missing. CI installs the
pinned SDKs (tools/requirements-jev-sdk.txt, tools/jev-sdk/package.json) and
@@@ -45,8 -44,8 +45,10 @@@ import mimo_serve_fixture # noqa: E40
REQUIRED = os.environ.get("JEV_SDK_REQUIRED") == "1"
LAYA_FIXTURE = HERE / "laya_tiny"
LAYA_BINARY = HERE / ("laya.exe" if os.name == "nt" else "laya")
+ GLINER_FIXTURE = HERE / "gliner_decide_tiny"
+ GLINER_BINARY = HERE / ("gliner_decide.exe" if os.name == "nt" else "gliner_decide")
+CLEF_CONTAINER = HERE / "clef_tiny_c"
+QWEN36_BINARY = HERE / ("qwen36.exe" if os.name == "nt" else "qwen36")
TS_SDK_DIR = Path(os.environ.get("JEV_TS_SDK_DIR", HERE / "tools" / "jev-sdk"))
TS_SCRIPT = HERE / "tests" / "jev_sdk_client.mjs"
API_KEY = "sk-colibri-test"
@@@ -151,12 -150,14 +153,18 @@@ def servers()
found.append(("laya-tiny (decision engine)", LAYA_FIXTURE, {}))
elif REQUIRED:
raise AssertionError("laya or laya_tiny missing (make laya laya-tiny-generate)")
+ if GLINER_BINARY.exists() and (GLINER_FIXTURE / "model.safetensors").exists():
+ found.append(("gliner-decide-tiny (decision engine)", GLINER_FIXTURE, {}))
+ elif REQUIRED:
+ raise AssertionError("gliner_decide or gliner_decide_tiny missing "
+ "(make gliner_decide gliner-decide-tiny-generate)")
+ if QWEN36_BINARY.exists() and (CLEF_CONTAINER / "joint_head.safetensors").exists():
+ found.append(("clef-tiny (chat model with a decision head)", CLEF_CONTAINER, {}))
+ elif REQUIRED:
+ raise AssertionError("qwen36 or clef_tiny_c missing (make qwen36 clef-tiny-generate)")
if not found:
- raise unittest.SkipTest("no tiny fixture to serve (make mimo laya laya-tiny-generate)")
+ raise unittest.SkipTest("no tiny fixture to serve (make mimo laya laya-tiny-generate "
- "gliner_decide gliner-decide-tiny-generate)")
++ "gliner_decide gliner-decide-tiny-generate qwen36 clef-tiny-generate)")
return found
diff --cc docs/api.md
index 2dc450ef,5d00d856..7e5d42f3
--- a/docs/api.md
+++ b/docs/api.md
@@@ -26,7 -26,8 +26,8 @@@ Implemented endpoints are `GET /v1/mode
`POST /v1/systemone`, colibri's one decision API ([systemone.md](systemone.md)):
the request and reply of TypeSafe's Jev API, scored by a language model through
its logprob channel or answered natively by a decision model such as
- [Laya](laya.md) or [Clef](clef.md). Chat and
-[Laya](laya.md) or
-[GLiNER2.5-Decide](gliner_decide.md). Chat and
++[Laya](laya.md),
++[GLiNER2.5-Decide](gliner_decide.md) or [Clef](clef.md). Chat and
completion requests support JSON responses, SSE streaming, usage counts,
`max_tokens`/`max_completion_tokens`, `temperature`, `top_p`, and up to four
custom `stop` sequences. `max_tokens` is a ceiling, not a target: when the
diff --cc docs/docs.json
index fa9e4b09,399551c3..ccd5bacb
--- a/docs/docs.json
+++ b/docs/docs.json
@@@ -23,7 -23,7 +23,7 @@@
},
{
"group": "Model engines",
- "pages": ["deepseek-v4", "kimi_k3", "qwen38", "qwen36", "inkling", "mimo", "qwen-image", "laya", "clef"]
- "pages": ["deepseek-v4", "kimi_k3", "qwen38", "qwen36", "inkling", "mimo", "qwen-image", "laya", "gliner_decide"]
++ "pages": ["deepseek-v4", "kimi_k3", "qwen38", "qwen36", "inkling", "mimo", "qwen-image", "laya", "gliner_decide", "clef"]
},
{
"group": "Configure and tune",
diff --cc docs/serve_protocol.md
index d0099eed,a3a852bd..1b7889fe
--- a/docs/serve_protocol.md
+++ b/docs/serve_protocol.md
@@@ -42,10 -42,8 +42,10 @@@ Two more keys say what kind of engine a
`DECIDE` (below), and the gateway sends `POST /v1/systemone` to it as one record
instead of scoring options through the logprob channel. `chat=0`: it has nothing
else, so the generating endpoints answer 400 with a pointer to `/v1/systemone`.
- A decision engine (laya) says `decide=1 chat=0`; an engine that chats and also
+ A decision engine (laya, gliner_decide) says `decide=1 chat=0`; an engine that chats and also
-decides natively would say `decide=1` alone.
+decides natively says `decide=1` alone (qwen36 with Clef's head). `decide_record=raw`
+asks for the record in its raw form, the caller's own values (docs/systemone.md,
+"Decision engines").
## Requests (server → engine)
diff --cc docs/systemone.md
index 92ce61e8,224a5d40..90431285
--- a/docs/systemone.md
+++ b/docs/systemone.md
@@@ -5,7 -5,7 +5,8 @@@ option. There is one API for it, `POST
the reply of TypeSafe's Jev: a state, typed questions (`noul`, `choice`,
`score`), and for every question a probability per option. A language model
answers by **scoring** instead of writing; a decision model such as
--[Laya](laya.md) answers natively. Same server and same model as the chat: the
++[Laya](laya.md), [GLiNER2.5-Decide](gliner_decide.md) or [Clef](clef.md)
++answers natively. Same server and same model as the chat: the
mode is the endpoint, not a build flag or a separate process.
## Why you would want it
@@@ -168,7 -168,7 +169,8 @@@ const { answers } = await r.json()
colibri answers `POST /v1/systemone` with the request and the reply of
TypeSafe's Jev API. Code written against Jev keeps working: point it at
colibri and change nothing else. The two official SDKs, unmodified, are
--tested against `coli serve` on a language model and on a decision engine
++tested against `coli serve` on a language model, on the decision engines
++(Laya, GLiNER2.5-Decide) and on a chat model with a decision head (Clef)
(`tests/test_jev_sdk.py`, `make -C c jev-sdk-check`).
Python, `typesafe_sdk`:
@@@ -461,14 -461,12 +463,15 @@@ cannot tell which kind answered
| engine | model | doc |
|---|---|---|
| `c/laya` | Laya (Convai Innovations): ModernBERT encoder + decision head | [laya.md](laya.md) |
+ | `c/gliner_decide` | GLiNER2.5-Decide (fastino): DeBERTa-v3 encoder + GLiNER2's classification head | [gliner_decide.md](gliner_decide.md) |
+| `c/qwen36` | Clef (Cloudflare): Qwen3.8-27B, post-trained, + joint schema head; it also chats | [clef.md](clef.md) |
- A decision-only engine (Laya) serves `POST /v1/systemone` only. Chat,
-A decision engine serves `POST /v1/systemone` only. Chat, completions and
-messages answer 400 with a pointer to `/v1/systemone`, and
++A decision-only engine (Laya, GLiNER2.5-Decide) serves `POST /v1/systemone` only. Chat,
+completions and messages answer 400 with a pointer to `/v1/systemone`, and
`/v1/models` lists the model with `capabilities: ["systemone", "decision"]`
-(a language model says `["chat", "systemone"]`). The path is one round trip:
+(a language model says `["chat", "systemone"]`). A chat model with a
+decision head (Clef) keeps every endpoint and says `["chat", "systemone"]`;
+only its `/v1/systemone` takes the native path. The path is one round trip:
the gateway checks the request, sends it to the engine as one record, and
shapes the answer. Nothing is rendered into a prompt and no option is scored
on its own. On Laya the gateway's share is about 2 ms per request: 8.1 ms at
@@@ -514,18 -511,6 +517,18 @@@ An engine that also chats leaves `chat
default text the language-model path asks with.
- `slot` is the KV slot, for an engine that keeps state between requests.
+**The raw form.** A model whose reference renders the request into a prompt
+itself (Clef) needs what the caller sent, not the gateway's reading of it. An
+engine that says `decide_record=raw` gets the record with `"record": "raw"`
+and the same shape, with three differences: instructions are `null` when the
+caller sent none (no default text); a description is kept as sent, `""`
+included, and a noul side the caller did not describe has no `"text"` key at
+all (`null` when it was described as `null`); and a JSON value (the state,
+instructions, a description) is written with sorted keys and compact
+separators, `json.dumps(value, ensure_ascii=False, separators=(",", ":"),
+sort_keys=True)`, with `"json": true` on an option whose text is one.
- Laya gets the default form, byte for byte as before.
++Laya and GLiNER2.5-Decide get the default form, byte for byte as before.
+
**Answer.** One `DECISION <id> <bytes>` frame, then `DONE`:
```json
@@@ -546,20 -531,27 +549,22 @@@ the engine's own record of how it got t
**Registry.** A family with `modality="decision"` and
`FamilyCapabilities(decision=True)` in `c/family_registry.py`, and a way for
`resolve_model` to recognise its checkpoint (Laya: `rl_agent_config.json`
- plus `encoder/config.json`, keyed `laya_<encoder model_type>`). `coli serve`
+ plus `encoder/config.json`, keyed `laya_<encoder model_type>`;
+ GLiNER2.5-Decide: a `config.json` whose `model_type` is `extractor` plus
+ `encoder_config/config.json`, keyed
+ `gliner2_<architecture>_<encoder model_type>`). `checkpoint_files` names the
+ files `coli doctor` checks. `coli serve`
and `coli web` then serve it, `coli info` and `coli plan` describe it, and
--`coli chat` / `coli run` refuse it with a pointer to the endpoint.
++`coli chat` / `coli run` refuse it with a pointer to the endpoint. A decision
++head over a chat family is a `DecisionHead` instead (Clef: a qwen36 checkpoint
++with `joint_head_config.json` and `joint_head.safetensors`): the family stays
++the chat one, `resolve_model` sets `decision_head`, and nothing is refused.
**Tests.** A tiny fixture whose reference answers come from the model's own
package (`tools/make_laya_tiny.py`, `tools/make_laya_ref.py`,
- `tests/test_laya_tiny.py`), and the fixture behind the real gateway
- (`tests/test_decision_serve.py`, `tests/test_jev_sdk.py`).
+ `tests/test_laya_tiny.py`; `tools/make_gliner_decide_tiny.py`,
-`tools/make_gliner_decide_ref.py`, `tests/test_gliner_decide_tiny.py`), and
-the fixture behind the real gateway (`tests/test_decision_serve.py`,
++`tools/make_gliner_decide_ref.py`, `tests/test_gliner_decide_tiny.py`;
++`tools/make_clef_tiny.py`, `tools/make_clef_ref.py`, `tests/test_clef_tiny.py`),
++and the fixture behind the real gateway (`tests/test_decision_serve.py`,
+ `tests/test_jev_sdk.py`).
--### The next one
--
- - **GLiNER2.5-Decide** (DeBERTa-v3 encoder, GLiNER2 heads): a second encoder
- engine on the same contract. It needs the DeBERTa-v3 encoder (relative
- position buckets and disentangled attention, a SentencePiece tokenizer) and
- its heads, its own rendering of a record into its input, and a
- `resolve_model` rule for its checkpoint. `decide_serve.h`, the gateway and
- the SDK tests stay as they are.
-- **Clef** (a joint schema head over Qwen3.8-27B, dense, with vision): the
- backbone is the qwen36 engine's. It would answer `DECIDE` in that engine's
- serve loop next to `SUBMIT`, announce `decide=1` without `chat=0` (the same
- process still chats), run the backbone over the rendered record and apply
- the head to its final hidden states. The record already carries what a
- schema head needs (every question with its typed options); a picture in the
- state would need the record to carry image parts, which today it does not.