Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
qwen-image: the qwen_image family, detected from model_index.json
JustVugg88facffd440d
diff --git a/README.it.md b/README.it.mdindex 86dbed61..8c2f3de6 100644--- a/README.it.md+++ b/README.it.md@@ -17,6 +17,9 @@ vision), **Inkling** (975B), **Kimi K3** (2,8T), **DeepSeek V4 Flash** (284B), ***Qwen3.8-Flash-Next** (125B + 51B n-gram), **Qwen3.6** (35B-A3B) e**OLMoE** (7B) — unfile C ciascuna, la stessa interfaccia `coli chat` / `coli serve` / `coli web`. [Elenco completo](README.md#other-supported-models)+Anche immagini: **Qwen-Image-2.1** genera figure da un testo, mostrate dentro al+terminale da `coli chat` e servite su `POST /v1/images/generations`+([qwen-image.md](docs/qwen-image.md)).> **Colibrì è un motore di inferenza che puoi usare oggi, e una piattaforma di> ricerca aperta.** Il suo obiettivo principale è migliorare le prestazioni didiff --git a/README.ja.md b/README.ja.mdindex 9f4bbdca..f271ce28 100644--- a/README.ja.md+++ b/README.ja.md@@ -22,6 +22,8 @@**Qwen3.8-Flash-Next**(125B + 51B n-gram)、**Qwen3.6**(35B-A3B)、そして**OLMoE**(7B)——それぞれが C ファイル 1 つで、同じ `coli chat` / `coli serve` / `coli web` フロントエンドを共有します。+画像生成も可能です: **Qwen-Image-2.1** がテキストから画像を生成し、`coli chat` はそれをターミナル内に+直接表示、`coli serve` は `POST /v1/images/generations` で提供します([qwen-image.md](docs/qwen-image.md))。[全モデル一覧 ↓](#other-supported-models)> **Colibrì は今日すぐに動かせる推論エンジンであり、同時にオープンな研究diff --git a/README.md b/README.mdindex 3a0ad7d9..962d8ef7 100644--- a/README.md+++ b/README.md@@ -23,6 +23,9 @@ vision), **Inkling** (975B), **Kimi K3** (2.8T), **DeepSeek V4 Flash** (284B), ***Qwen3.8-Flash-Next** (125B + 51B n-gram), **Qwen3.6** (35B-A3B) and**OLMoE** (7B) —one C file each, the same `coli chat` / `coli serve` / `coli web` front end.+Images too: **Qwen-Image-2.1** generates pictures from text, shown inline in+the terminal by `coli chat` and served at `POST /v1/images/generations`+([qwen-image.md](docs/qwen-image.md)).[Full roster ↓](#other-supported-models)> **Colibrì is an inference engine you can run today, and an open researchdiff --git a/README.zh-CN.md b/README.zh-CN.mdindex 7f9a9420..188d97cf 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -14,6 +14,8 @@**Inkling**(975B)、**Kimi K3**(2.8T)、**DeepSeek V4 Flash**(284B)、**DeepSeek V4.1 Flash**(552B,含视觉)、**Qwen3.8-Flash-Next**(125B + 51B n-gram)、**Qwen3.6**(35B-A3B)与 **OLMoE**(7B)——各自一个 C 文件,共用同一套 `coli chat` / `coli serve` / `coli web` 前端。[完整列表](README.md#other-supported-models)+也能生成图像:**Qwen-Image-2.1** 根据文字生成图片,`coli chat` 直接在终端中显示,+`coli serve` 通过 `POST /v1/images/generations` 提供([qwen-image.md](docs/qwen-image.md))。> **Colibrì 既是今天就能运行的推理引擎,也是一个开放的研究平台**。它的首要目标是在> 完整的软硬件边界上追求推理侧性能——模型格式、内存层级、存储 I/O、放置、调度、内核、diff --git a/README.zh-TW.md b/README.zh-TW.mdindex c208ac41..db1e978b 100644--- a/README.zh-TW.md+++ b/README.zh-TW.md@@ -14,6 +14,8 @@**Inkling**(975B)、**Kimi K3**(2.8T)、**DeepSeek V4 Flash**(284B)、**DeepSeek V4.1 Flash**(552B,含視覺)、**Qwen3.8-Flash-Next**(125B + 51B n-gram)、**Qwen3.6**(35B-A3B)與 **OLMoE**(7B)——各自一個 C 檔案,共用同一套 `coli chat` / `coli serve` / `coli web` 前端。[完整清單](README.md#other-supported-models)+也能生成圖像:**Qwen-Image-2.1** 依據文字生成圖片,`coli chat` 直接在終端機中顯示,+`coli serve` 透過 `POST /v1/images/generations` 提供([qwen-image.md](docs/qwen-image.md))。> **Colibrì 既是今天就能執行的推論引擎,也是一個開放的研究平台。**它的首要目標是在> 完整的軟硬體邊界上追求推論側效能——模型格式、記憶體階層、儲存 I/O、配置、排程、核心、diff --git a/c/family_registry.py b/c/family_registry.pyindex d1fb3877..eb13de8a 100644--- a/c/family_registry.py+++ b/c/family_registry.py@@ -143,6 +143,15 @@ class FamilyDescriptor:# la geometria lo rende visibile. 0 = nessun riferimento dichiarato, il# banner stampa display_scale come sempre.reference_experts: int = 0+ # "text" for the chat engines, "image" for a text-to-image pipeline. An+ # image family has no KV cache, no experts and no chat template: coli+ # routes it to the image REPL, the image planner and POST+ # /v1/images/generations, and every text-only invariant (context variable,+ # segment conformance, tuning) is scoped to modality "text".+ modality: str = "text"+ # Where the tokenizer lives, relative to the model directory. A diffusers+ # pipeline keeps it in processor/, not at the root.+ tokenizer_file: str = "tokenizer.json"@dataclass(frozen=True, slots=True)@@ -1439,6 +1448,42 @@ FAMILIES = (# coli chat, coli serve and coli web already use.has_cli_adapter=False,),+ FamilyDescriptor(+ id="qwen_image",+ # A diffusers pipeline has no config.json at its root: the family is+ # read from model_index.json's _class_name (see resolve_model), which+ # is what this entry holds, normalized like every model_type.+ model_types=("qwenimage21pipeline",),+ display_name="Qwen-Image-2.1",+ display_scale="",+ engine_artifact="qwenimage",+ engine_aliases=(),+ engine_group="qwenimage",+ internal_arch="qwenimage",+ build_target="qwenimage",+ process_names=("qwenimage",),+ default_model_id="qwen-image-2.1-colibri",+ cli_adapter="qwen_image",+ gateway_adapter="qwen_image",+ planner_id="qwen_image",+ planner_geometry=None,+ planner_unsupported_reason=(+ "an image model has no KV cache and no experts; coli plan sizes its "+ "text encoder, DiT and VAE instead (image_engine.plan_image_model)"),+ expert_inventory=lambda _name, _size, _config, _dtype=None: (),+ config_section="root",+ # One image at a time, and no context or output-token budget: the+ # limits exist because every descriptor has them. No context variable+ # either (empty): the launcher refuses --ctx for this modality rather+ # than inventing a knob the engine does not read.+ limits=FamilyLimits(1, 1, 1, 1, 1, 0, ""),+ capabilities=FamilyCapabilities(False, False, False, False),+ has_gateway_adapter=True,+ has_cli_adapter=True,+ supports_accelerator=False,+ modality="image",+ tokenizer_file="processor/tokenizer.json",+ ),)@@ -1461,7 +1506,9 @@ def _build_registry(families):not isinstance(family.has_gateway_adapter, bool) ornot isinstance(family.has_cli_adapter, bool) ornot isinstance(family.tune_prompt_template, str) or- "{prompt}" not in family.tune_prompt_template):+ "{prompt}" not in family.tune_prompt_template or+ family.modality not in ("text", "image") or+ not isinstance(family.tokenizer_file, str) or not family.tokenizer_file):raise RegistryError(f"incomplete family descriptor: {family.id}")try:family.tune_prompt_template.format(prompt="test", prompt_len=4)@@ -1526,6 +1573,9 @@ def family_by_id(family_id):def family_for_config(config):if not isinstance(config, dict):raise FamilyConfigError("config.json is not a JSON object")+ if "model_type" not in config and isinstance(config.get("_class_name"), str):+ # A diffusers model_index.json: the pipeline class is its model type.+ return family_for_index(config)model_type = _normalize_model_type(config.get("model_type"))try:return _BY_TYPE[model_type]@@ -1533,15 +1583,48 @@ def family_for_config(config):raise UnknownFamilyError(f"unsupported model_type: {model_type}") from error+def family_for_index(index):+ """The family of a diffusers pipeline, from its model_index.json."""+ if not isinstance(index, dict):+ raise FamilyConfigError(f"{MODEL_INDEX} is not a JSON object")+ name = index.get("_class_name")+ if not isinstance(name, str) or not name.strip():+ raise FamilyConfigError(f"{MODEL_INDEX} has no non-empty string _class_name")+ try:+ family = _BY_TYPE[_normalize_model_type(name)]+ except KeyError as error:+ raise UnknownFamilyError(f"unsupported diffusers pipeline: {name}") from error+ if family.modality != "image":+ raise UnknownFamilyError(f"unsupported diffusers pipeline: {name}")+ return family++def tuning_replay_prompt(family, prompt):if not isinstance(prompt, str):raise ValueError("tuning prompt must be a string")return family.tune_prompt_template.format(prompt=prompt, prompt_len=len(prompt))+MODEL_INDEX = "model_index.json"++def resolve_model(model_dir):model = Path(model_dir).expanduser().resolve()path = model / "config.json"+ if not path.is_file() and (model / MODEL_INDEX).is_file():+ # A diffusers pipeline (Qwen-Image): the root carries model_index.json+ # and each component keeps its own config.json in its own directory.+ # config.json wins when both exist, so a text checkpoint that happens+ # to ship an index is never read as an image model.+ try:+ index = json.loads((model / MODEL_INDEX).read_text(encoding="utf-8"))+ except OSError as error:+ raise FamilyConfigError(f"cannot read {MODEL_INDEX}: {model}") from error+ except json.JSONDecodeError as error:+ raise FamilyConfigError(f"invalid {MODEL_INDEX}: {error}") from error+ family = family_for_index(index)+ return ResolvedFamily(family, _normalize_model_type(index["_class_name"]),+ index, index, str(model))try:config = json.loads(path.read_text(encoding="utf-8"))except OSError as error:@@ -1549,7 +1632,8 @@ def resolve_model(model_dir):f"cannot read config.json: {model}\n"" coli picks the engine from config.json, so nothing runs without it. Copy the\n"" checkpoint's config.json (with tokenizer.json and model.safetensors.index.json)\n"- " from the model repo next to the shards.") from error+ " from the model repo next to the shards. An image model (a diffusers pipeline)\n"+ " carries model_index.json instead.") from errorexcept json.JSONDecodeError as error:raise FamilyConfigError(f"invalid config.json: {error}") from errorfamily = family_for_config(config)@@ -1684,6 +1768,8 @@ def public_metadata(family):"gateway_adapter": family.gateway_adapter,"planner_id": family.planner_id,"supports_accelerator": family.supports_accelerator,+ "modality": family.modality,+ "tokenizer_file": family.tokenizer_file,"limits": {"default_context": family.limits.default_context,"max_context": family.limits.max_context,diff --git a/c/tests/test_family_registry.py b/c/tests/test_family_registry.pyindex 4a3e95e2..81cb9f84 100644--- a/c/tests/test_family_registry.py+++ b/c/tests/test_family_registry.py@@ -1032,6 +1032,9 @@ class FamilyRegistryTest(unittest.TestCase):# V4.1 ships its chat encoding as a Python module (encoding/encoding.py),# not a jinja template, so the replay prompt stays the bare text like V4."deepseek_v41": "hello {world}",+ # An image model has no chat template and nothing to tune; the+ # template is the identity and `coli tune` refuses the modality.+ "qwen_image": "hello {world}",}self.assertEqual({family.id: tuning_replay_prompt(family, prompt) for family in FAMILIES},diff --git a/c/tests/test_registry_engine_agreement.py b/c/tests/test_registry_engine_agreement.pyindex d90e931c..1bd93a6a 100644--- a/c/tests/test_registry_engine_agreement.py+++ b/c/tests/test_registry_engine_agreement.py@@ -48,6 +48,11 @@ def engine_rule(artifact: str) -> str:class RegistryEngineAgreementTest(unittest.TestCase):def test_every_engine_reads_its_declared_context_variable(self):for family in fr.FAMILIES:+ if family.modality != "text":+ # No context window to cap: the launcher refuses --ctx for an+ # image model instead of setting a variable (coli, env_for_engine).+ self.assertEqual(family.limits.context_env, "", family.id)+ continuewith self.subTest(family=family.id):env = family.limits.context_envsource = (ROOT / f"{family.engine_artifact}.c").read_text(diff --git a/c/tests/test_segment_conformance_manifest.py b/c/tests/test_segment_conformance_manifest.pyindex 9770ff1e..dc33facb 100644--- a/c/tests/test_segment_conformance_manifest.py+++ b/c/tests/test_segment_conformance_manifest.py@@ -22,7 +22,10 @@ class SegmentConformanceManifestTest(unittest.TestCase):self.assertEqual(self.manifest["release_policy"], "all_registered_families")manifest_ids = [entry["family_id"] for entry in self.entries]- registry_ids = [family.id for family in FAMILIES]+ # The segment runtime's contract is about sequence state (KV caches,+ # recurrent and convolution state). A text-to-image family keeps none+ # between requests, so the gate covers every registered TEXT family.+ registry_ids = [family.id for family in FAMILIES if family.modality == "text"]self.assertEqual(manifest_ids, registry_ids)self.assertEqual(len(manifest_ids), len(set(manifest_ids)))diff --git a/c/tools/clean.py b/c/tools/clean.pyindex 06a588b7..517b93a9 100644--- a/c/tools/clean.py+++ b/c/tools/clean.py@@ -35,6 +35,7 @@ FILES = ["coli_hip.dll", "coli_hip.lib", "coli_hip.exp", "coli_hip.pdb","deepseek_v4", "deepseek_v4.exe", "deepseek_v4.cflags", "deepseek_v4.cudaflags","deepseek_v41", "deepseek_v41.exe",+ "qwenimage", "qwenimage.exe","native_quant.o", "native_quant_parallel.o", "native_quant_dual.o","native_quant_batch_avx512.o", "native_quant_fp4_rows16.o",]diff --git a/c/tools/convert_fp8_to_int4.py b/c/tools/convert_fp8_to_int4.pyindex 6f87f9ad..8c569abe 100644--- a/c/tools/convert_fp8_to_int4.py+++ b/c/tools/convert_fp8_to_int4.py@@ -64,6 +64,8 @@ OTHER_FAMILY_PATHS = {"deepseek_v4": "DeepSeek V4: see docs/deepseek-v4.md, section Download","deepseek_v41": "DeepSeek V4.1: no conversion needed -- its experts already ship fp4 and its dense fp8; run tools/prepare_dsv41.py once, see docs/deepseek-v41.md","deepseek_v41_text": "DeepSeek V4.1: no conversion needed -- see docs/deepseek-v41.md",+ "qwenimage21pipeline": "Qwen-Image-2.1 is NOT converted: the image engine reads the "+ "diffusers checkpoint as downloaded; see docs/qwen-image.md",}diff --git a/site/index.html b/site/index.htmlindex ed5bc742..ca0b9d10 100644--- a/site/index.html+++ b/site/index.html@@ -384,7 +384,8 @@ footer{border-top:1px solid var(--line)}<div class="chart"><div class="crow"><span class="cl">kimi-k3<s>104B active</s></span><span class="ct"><i style="width:100.0%"></i></span><span class="cv">2.8T</span></div><div class="crow"><span class="cl">inkling<s>41B active</s></span><span class="ct"><i style="width:34.8%"></i></span><span class="cv">975B</span></div><div class="crow"><span class="cl">glm-5.2<s>40B active</s></span><span class="ct"><i style="width:26.6%"></i></span><span class="cv">744B</span></div><div class="crow"><span class="cl">deepseek-v4.1-flash<s>16B active</s></span><span class="ct"><i style="width:19.7%"></i></span><span class="cv">552B</span></div><div class="crow"><span class="cl">glm-5.3-flash<s>18B active</s></span><span class="ct"><i style="width:11.4%"></i></span><span class="cv">320B</span></div><div class="crow"><span class="cl">deepseek-v4-flash<s>13B active</s></span><span class="ct"><i style="width:10.1%"></i></span><span class="cv">284B</span></div><div class="crow"><span class="cl">qwen3.8-flash-next<s>6B active</s></span><span class="ct"><i style="width:6.4%"></i></span><span class="cv">180B</span></div><div class="crow"><span class="cl">qwen3.6-35b-a3b<s>3B active</s></span><span class="ct"><i style="width:1.2%"></i></span><span class="cv">35B</span></div><div class="crow"><span class="cl">olmoe<s>1B active</s></span><span class="ct"><i style="width:0.8%"></i></span><span class="cv">7B</span></div></div><p class="src">Total parameters. Only the active ones are computed per token, and only therouted experts move. The engines, by the names the docs use:- GLM-5.3-Flash, GLM-5.2/5.3, Inkling, Kimi K3, OLMoE, Qwen3.6-35B-A3B, Qwen3.8-Flash-Next, DeepSeek V4 Flash, DeepSeek V4.1 Flash.</p>+ GLM-5.3-Flash, GLM-5.2/5.3, Inkling, Kimi K3, OLMoE, Qwen3.6-35B-A3B, Qwen3.8-Flash-Next, DeepSeek V4 Flash, DeepSeek V4.1 Flash.+ And for images, Qwen-Image-2.1: text to picture, drawn inside the terminal.</p><a class="more" href="#models">Browse all models →</a></div>