Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
jev-sdk-check: TypeSafe's SDKs against GLiNER2.5-Decide too
JustVuggcd0b86b6d509
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex e54c0db3..975193b6 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -1129,7 +1129,7 @@ jobs:run: |pip install -r c/tools/requirements-jev-sdk.txtcd c/tools/jev-sdk && npm ci- - name: Both SDKs against a language model (tiny MiMo) and a decision engine (tiny Laya)+ - name: Both SDKs against a language model (tiny MiMo) and the decision engines (tiny Laya, tiny GLiNER2.5-Decide)run: make -C c jev-sdk-checkqwenimage-tiny-check:diff --git a/c/Makefile b/c/Makefileindex 30131de6..0b4a23e7 100644--- a/c/Makefile+++ b/c/Makefile@@ -1310,11 +1310,13 @@ gliner-decide-tiny-check: gliner-decide-tiny-generate gliner_decide$(EXE)GLINER_DECIDE_TINY_REQUIRED=1 $(PYTHON) -m unittest -v tests.test_gliner_decide_tiny tests.test_decision_serve# TypeSafe's own SDKs (Python and TypeScript), unmodified, against `coli serve`-# on a language model (tiny MiMo) and a decision engine (tiny Laya): switching-# from Jev is a base-URL change. Needs the pinned SDKs first:+# on a language model (tiny MiMo) and the decision engines (tiny Laya, tiny+# GLiNER2.5-Decide): switching from Jev is a base-URL change. Needs the pinned+# SDKs first:# pip install -r tools/requirements-jev-sdk.txt && (cd tools/jev-sdk && npm ci).PHONY: jev-sdk-check-jev-sdk-check: mimo$(EXE) laya$(EXE) mimo-tiny-generate laya-tiny-generate+jev-sdk-check: mimo$(EXE) laya$(EXE) gliner_decide$(EXE) mimo-tiny-generate laya-tiny-generate \+ gliner-decide-tiny-generateJEV_SDK_REQUIRED=1 $(PYTHON) -m unittest -v tests.test_jev_sdk# In make check: every format and ragged edges against the naive product, anddiff --git a/c/tests/test_jev_sdk.py b/c/tests/test_jev_sdk.pyindex 2023449c..8611f50b 100644--- a/c/tests/test_jev_sdk.py+++ b/c/tests/test_jev_sdk.py@@ -11,10 +11,10 @@ The official clients, unmodified, pointed at colibri with nothing but@typesafe-ai/sdk TypeScript, new TypeSafeClient({apiKey, baseURL}).systemOne(...) and .models.list(), run under node-Each one is driven against two servers, each a real `coli serve` process:+Each one is driven against three servers, each a real `coli serve` process:a language model (the tiny MiMo fixture, which scores options through the-logprob channel) and a decision engine (the tiny Laya fixture, DECIDE). The-same requests go to both: every question type, several questions at once,+logprob channel) and two decision engines (the tiny Laya and GLiNER2.5-Decide+fixtures, DECIDE). The same requests go to all three: every question type, several questions at once,text, object and array states, objects and arrays as instructions andcriteria, and a request the server refuses (422), which each SDK must raiseas its own UnprocessableEntity error.@@ -44,6 +44,8 @@ import mimo_serve_fixture # noqa: E402REQUIRED = os.environ.get("JEV_SDK_REQUIRED") == "1"LAYA_FIXTURE = HERE / "laya_tiny"LAYA_BINARY = HERE / ("laya.exe" if os.name == "nt" else "laya")+GLINER_FIXTURE = HERE / "gliner_decide_tiny"+GLINER_BINARY = HERE / ("gliner_decide.exe" if os.name == "nt" else "gliner_decide")TS_SDK_DIR = Path(os.environ.get("JEV_TS_SDK_DIR", HERE / "tools" / "jev-sdk"))TS_SCRIPT = HERE / "tests" / "jev_sdk_client.mjs"API_KEY = "sk-colibri-test"@@ -138,7 +140,7 @@ def long_mimo_fixture():def servers():- """(name, model dir, env) of the two servers, or a skip naming what is missing."""+ """(name, model dir, env) of the three servers, or a skip naming what is missing."""found = []if mimo_serve_fixture.available():found.append(("mimo-tiny (language model)", long_mimo_fixture(), {"CTX": "4096"}))@@ -148,8 +150,14 @@ def servers():found.append(("laya-tiny (decision engine)", LAYA_FIXTURE, {}))elif REQUIRED:raise AssertionError("laya or laya_tiny missing (make laya laya-tiny-generate)")+ if GLINER_BINARY.exists() and (GLINER_FIXTURE / "model.safetensors").exists():+ found.append(("gliner-decide-tiny (decision engine)", GLINER_FIXTURE, {}))+ elif REQUIRED:+ raise AssertionError("gliner_decide or gliner_decide_tiny missing "+ "(make gliner_decide gliner-decide-tiny-generate)")if not found:- raise unittest.SkipTest("no tiny fixture to serve (make mimo laya laya-tiny-generate)")+ raise unittest.SkipTest("no tiny fixture to serve (make mimo laya laya-tiny-generate "+ "gliner_decide gliner-decide-tiny-generate)")return found