Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

qwen-image: the qwen_image family, detected from model_index.json

JustVugg88facffd440d
diff --git a/README.it.md b/README.it.md
index 86dbed61..8c2f3de6 100644
--- a/README.it.md
+++ b/README.it.md
@@ -17,6 +17,9 @@ vision), **Inkling** (975B), **Kimi K3** (2,8T), **DeepSeek V4 Flash** (284B), *
**Qwen3.8-Flash-Next** (125B + 51B n-gram), **Qwen3.6** (35B-A3B) e
**OLMoE** (7B) — un
file C ciascuna, la stessa interfaccia `coli chat` / `coli serve` / `coli web`. [Elenco completo](README.md#other-supported-models)
+Anche immagini: **Qwen-Image-2.1** genera figure da un testo, mostrate dentro al
+terminale da `coli chat` e servite su `POST /v1/images/generations`
+([qwen-image.md](docs/qwen-image.md)).
> **Colibrì è un motore di inferenza che puoi usare oggi, e una piattaforma di
> ricerca aperta.** Il suo obiettivo principale è migliorare le prestazioni di
diff --git a/README.ja.md b/README.ja.md
index 9f4bbdca..f271ce28 100644
--- a/README.ja.md
+++ b/README.ja.md
@@ -22,6 +22,8 @@
**Qwen3.8-Flash-Next**(125B + 51B n-gram)、**Qwen3.6**(35B-A3B)、そして
**OLMoE**(7B)——
それぞれが C ファイル 1 つで、同じ `coli chat` / `coli serve` / `coli web` フロントエンドを共有します。
+画像生成も可能です: **Qwen-Image-2.1** がテキストから画像を生成し、`coli chat` はそれをターミナル内に
+直接表示、`coli serve` は `POST /v1/images/generations` で提供します([qwen-image.md](docs/qwen-image.md))。
[全モデル一覧 ↓](#other-supported-models)
> **Colibrì は今日すぐに動かせる推論エンジンであり、同時にオープンな研究
diff --git a/README.md b/README.md
index 3a0ad7d9..962d8ef7 100644
--- a/README.md
+++ b/README.md
@@ -23,6 +23,9 @@ vision), **Inkling** (975B), **Kimi K3** (2.8T), **DeepSeek V4 Flash** (284B), *
**Qwen3.8-Flash-Next** (125B + 51B n-gram), **Qwen3.6** (35B-A3B) and
**OLMoE** (7B) —
one C file each, the same `coli chat` / `coli serve` / `coli web` front end.
+Images too: **Qwen-Image-2.1** generates pictures from text, shown inline in
+the terminal by `coli chat` and served at `POST /v1/images/generations`
+([qwen-image.md](docs/qwen-image.md)).
[Full roster ↓](#other-supported-models)
> **Colibrì is an inference engine you can run today, and an open research
diff --git a/README.zh-CN.md b/README.zh-CN.md
index 7f9a9420..188d97cf 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -14,6 +14,8 @@
**Inkling**(975B)、**Kimi K3**(2.8T)、**DeepSeek V4 Flash**(284B)、**DeepSeek V4.1 Flash**(552B,含视觉)、
**Qwen3.8-Flash-Next**(125B + 51B n-gram)、**Qwen3.6**(35B-A3B)与 **OLMoE**(7B)
——各自一个 C 文件,共用同一套 `coli chat` / `coli serve` / `coli web` 前端。[完整列表](README.md#other-supported-models)
+也能生成图像:**Qwen-Image-2.1** 根据文字生成图片,`coli chat` 直接在终端中显示,
+`coli serve` 通过 `POST /v1/images/generations` 提供([qwen-image.md](docs/qwen-image.md))。
> **Colibrì 既是今天就能运行的推理引擎,也是一个开放的研究平台**。它的首要目标是在
> 完整的软硬件边界上追求推理侧性能——模型格式、内存层级、存储 I/O、放置、调度、内核、
diff --git a/README.zh-TW.md b/README.zh-TW.md
index c208ac41..db1e978b 100644
--- a/README.zh-TW.md
+++ b/README.zh-TW.md
@@ -14,6 +14,8 @@
**Inkling**(975B)、**Kimi K3**(2.8T)、**DeepSeek V4 Flash**(284B)、**DeepSeek V4.1 Flash**(552B,含視覺)、
**Qwen3.8-Flash-Next**(125B + 51B n-gram)、**Qwen3.6**(35B-A3B)與 **OLMoE**(7B)
——各自一個 C 檔案,共用同一套 `coli chat` / `coli serve` / `coli web` 前端。[完整清單](README.md#other-supported-models)
+也能生成圖像:**Qwen-Image-2.1** 依據文字生成圖片,`coli chat` 直接在終端機中顯示,
+`coli serve` 透過 `POST /v1/images/generations` 提供([qwen-image.md](docs/qwen-image.md))。
> **Colibrì 既是今天就能執行的推論引擎,也是一個開放的研究平台。**它的首要目標是在
> 完整的軟硬體邊界上追求推論側效能——模型格式、記憶體階層、儲存 I/O、配置、排程、核心、
diff --git a/c/family_registry.py b/c/family_registry.py
index d1fb3877..eb13de8a 100644
--- a/c/family_registry.py
+++ b/c/family_registry.py
@@ -143,6 +143,15 @@ class FamilyDescriptor:
# la geometria lo rende visibile. 0 = nessun riferimento dichiarato, il
# banner stampa display_scale come sempre.
reference_experts: int = 0
+ # "text" for the chat engines, "image" for a text-to-image pipeline. An
+ # image family has no KV cache, no experts and no chat template: coli
+ # routes it to the image REPL, the image planner and POST
+ # /v1/images/generations, and every text-only invariant (context variable,
+ # segment conformance, tuning) is scoped to modality "text".
+ modality: str = "text"
+ # Where the tokenizer lives, relative to the model directory. A diffusers
+ # pipeline keeps it in processor/, not at the root.
+ tokenizer_file: str = "tokenizer.json"
@dataclass(frozen=True, slots=True)
@@ -1439,6 +1448,42 @@ FAMILIES = (
# coli chat, coli serve and coli web already use.
has_cli_adapter=False,
),
+ FamilyDescriptor(
+ id="qwen_image",
+ # A diffusers pipeline has no config.json at its root: the family is
+ # read from model_index.json's _class_name (see resolve_model), which
+ # is what this entry holds, normalized like every model_type.
+ model_types=("qwenimage21pipeline",),
+ display_name="Qwen-Image-2.1",
+ display_scale="",
+ engine_artifact="qwenimage",
+ engine_aliases=(),
+ engine_group="qwenimage",
+ internal_arch="qwenimage",
+ build_target="qwenimage",
+ process_names=("qwenimage",),
+ default_model_id="qwen-image-2.1-colibri",
+ cli_adapter="qwen_image",
+ gateway_adapter="qwen_image",
+ planner_id="qwen_image",
+ planner_geometry=None,
+ planner_unsupported_reason=(
+ "an image model has no KV cache and no experts; coli plan sizes its "
+ "text encoder, DiT and VAE instead (image_engine.plan_image_model)"),
+ expert_inventory=lambda _name, _size, _config, _dtype=None: (),
+ config_section="root",
+ # One image at a time, and no context or output-token budget: the
+ # limits exist because every descriptor has them. No context variable
+ # either (empty): the launcher refuses --ctx for this modality rather
+ # than inventing a knob the engine does not read.
+ limits=FamilyLimits(1, 1, 1, 1, 1, 0, ""),
+ capabilities=FamilyCapabilities(False, False, False, False),
+ has_gateway_adapter=True,
+ has_cli_adapter=True,
+ supports_accelerator=False,
+ modality="image",
+ tokenizer_file="processor/tokenizer.json",
+ ),
)
@@ -1461,7 +1506,9 @@ def _build_registry(families):
not isinstance(family.has_gateway_adapter, bool) or
not isinstance(family.has_cli_adapter, bool) or
not isinstance(family.tune_prompt_template, str) or
- "{prompt}" not in family.tune_prompt_template):
+ "{prompt}" not in family.tune_prompt_template or
+ family.modality not in ("text", "image") or
+ not isinstance(family.tokenizer_file, str) or not family.tokenizer_file):
raise RegistryError(f"incomplete family descriptor: {family.id}")
try:
family.tune_prompt_template.format(prompt="test", prompt_len=4)
@@ -1526,6 +1573,9 @@ def family_by_id(family_id):
def family_for_config(config):
if not isinstance(config, dict):
raise FamilyConfigError("config.json is not a JSON object")
+ if "model_type" not in config and isinstance(config.get("_class_name"), str):
+ # A diffusers model_index.json: the pipeline class is its model type.
+ return family_for_index(config)
model_type = _normalize_model_type(config.get("model_type"))
try:
return _BY_TYPE[model_type]
@@ -1533,15 +1583,48 @@ def family_for_config(config):
raise UnknownFamilyError(f"unsupported model_type: {model_type}") from error
+def family_for_index(index):
+ """The family of a diffusers pipeline, from its model_index.json."""
+ if not isinstance(index, dict):
+ raise FamilyConfigError(f"{MODEL_INDEX} is not a JSON object")
+ name = index.get("_class_name")
+ if not isinstance(name, str) or not name.strip():
+ raise FamilyConfigError(f"{MODEL_INDEX} has no non-empty string _class_name")
+ try:
+ family = _BY_TYPE[_normalize_model_type(name)]
+ except KeyError as error:
+ raise UnknownFamilyError(f"unsupported diffusers pipeline: {name}") from error
+ if family.modality != "image":
+ raise UnknownFamilyError(f"unsupported diffusers pipeline: {name}")
+ return family
+
+
def tuning_replay_prompt(family, prompt):
if not isinstance(prompt, str):
raise ValueError("tuning prompt must be a string")
return family.tune_prompt_template.format(prompt=prompt, prompt_len=len(prompt))
+MODEL_INDEX = "model_index.json"
+
+
def resolve_model(model_dir):
model = Path(model_dir).expanduser().resolve()
path = model / "config.json"
+ if not path.is_file() and (model / MODEL_INDEX).is_file():
+ # A diffusers pipeline (Qwen-Image): the root carries model_index.json
+ # and each component keeps its own config.json in its own directory.
+ # config.json wins when both exist, so a text checkpoint that happens
+ # to ship an index is never read as an image model.
+ try:
+ index = json.loads((model / MODEL_INDEX).read_text(encoding="utf-8"))
+ except OSError as error:
+ raise FamilyConfigError(f"cannot read {MODEL_INDEX}: {model}") from error
+ except json.JSONDecodeError as error:
+ raise FamilyConfigError(f"invalid {MODEL_INDEX}: {error}") from error
+ family = family_for_index(index)
+ return ResolvedFamily(family, _normalize_model_type(index["_class_name"]),
+ index, index, str(model))
try:
config = json.loads(path.read_text(encoding="utf-8"))
except OSError as error:
@@ -1549,7 +1632,8 @@ def resolve_model(model_dir):
f"cannot read config.json: {model}\n"
" coli picks the engine from config.json, so nothing runs without it. Copy the\n"
" checkpoint's config.json (with tokenizer.json and model.safetensors.index.json)\n"
- " from the model repo next to the shards.") from error
+ " from the model repo next to the shards. An image model (a diffusers pipeline)\n"
+ " carries model_index.json instead.") from error
except json.JSONDecodeError as error:
raise FamilyConfigError(f"invalid config.json: {error}") from error
family = family_for_config(config)
@@ -1684,6 +1768,8 @@ def public_metadata(family):
"gateway_adapter": family.gateway_adapter,
"planner_id": family.planner_id,
"supports_accelerator": family.supports_accelerator,
+ "modality": family.modality,
+ "tokenizer_file": family.tokenizer_file,
"limits": {
"default_context": family.limits.default_context,
"max_context": family.limits.max_context,
diff --git a/c/tests/test_family_registry.py b/c/tests/test_family_registry.py
index 4a3e95e2..81cb9f84 100644
--- a/c/tests/test_family_registry.py
+++ b/c/tests/test_family_registry.py
@@ -1032,6 +1032,9 @@ class FamilyRegistryTest(unittest.TestCase):
# V4.1 ships its chat encoding as a Python module (encoding/encoding.py),
# not a jinja template, so the replay prompt stays the bare text like V4.
"deepseek_v41": "hello {world}",
+ # An image model has no chat template and nothing to tune; the
+ # template is the identity and `coli tune` refuses the modality.
+ "qwen_image": "hello {world}",
}
self.assertEqual(
{family.id: tuning_replay_prompt(family, prompt) for family in FAMILIES},
diff --git a/c/tests/test_registry_engine_agreement.py b/c/tests/test_registry_engine_agreement.py
index d90e931c..1bd93a6a 100644
--- a/c/tests/test_registry_engine_agreement.py
+++ b/c/tests/test_registry_engine_agreement.py
@@ -48,6 +48,11 @@ def engine_rule(artifact: str) -> str:
class RegistryEngineAgreementTest(unittest.TestCase):
def test_every_engine_reads_its_declared_context_variable(self):
for family in fr.FAMILIES:
+ if family.modality != "text":
+ # No context window to cap: the launcher refuses --ctx for an
+ # image model instead of setting a variable (coli, env_for_engine).
+ self.assertEqual(family.limits.context_env, "", family.id)
+ continue
with self.subTest(family=family.id):
env = family.limits.context_env
source = (ROOT / f"{family.engine_artifact}.c").read_text(
diff --git a/c/tests/test_segment_conformance_manifest.py b/c/tests/test_segment_conformance_manifest.py
index 9770ff1e..dc33facb 100644
--- a/c/tests/test_segment_conformance_manifest.py
+++ b/c/tests/test_segment_conformance_manifest.py
@@ -22,7 +22,10 @@ class SegmentConformanceManifestTest(unittest.TestCase):
self.assertEqual(
self.manifest["release_policy"], "all_registered_families")
manifest_ids = [entry["family_id"] for entry in self.entries]
- registry_ids = [family.id for family in FAMILIES]
+ # The segment runtime's contract is about sequence state (KV caches,
+ # recurrent and convolution state). A text-to-image family keeps none
+ # between requests, so the gate covers every registered TEXT family.
+ registry_ids = [family.id for family in FAMILIES if family.modality == "text"]
self.assertEqual(manifest_ids, registry_ids)
self.assertEqual(len(manifest_ids), len(set(manifest_ids)))
diff --git a/c/tools/clean.py b/c/tools/clean.py
index 06a588b7..517b93a9 100644
--- a/c/tools/clean.py
+++ b/c/tools/clean.py
@@ -35,6 +35,7 @@ FILES = [
"coli_hip.dll", "coli_hip.lib", "coli_hip.exp", "coli_hip.pdb",
"deepseek_v4", "deepseek_v4.exe", "deepseek_v4.cflags", "deepseek_v4.cudaflags",
"deepseek_v41", "deepseek_v41.exe",
+ "qwenimage", "qwenimage.exe",
"native_quant.o", "native_quant_parallel.o", "native_quant_dual.o",
"native_quant_batch_avx512.o", "native_quant_fp4_rows16.o",
]
diff --git a/c/tools/convert_fp8_to_int4.py b/c/tools/convert_fp8_to_int4.py
index 6f87f9ad..8c569abe 100644
--- a/c/tools/convert_fp8_to_int4.py
+++ b/c/tools/convert_fp8_to_int4.py
@@ -64,6 +64,8 @@ OTHER_FAMILY_PATHS = {
"deepseek_v4": "DeepSeek V4: see docs/deepseek-v4.md, section Download",
"deepseek_v41": "DeepSeek V4.1: no conversion needed -- its experts already ship fp4 and its dense fp8; run tools/prepare_dsv41.py once, see docs/deepseek-v41.md",
"deepseek_v41_text": "DeepSeek V4.1: no conversion needed -- see docs/deepseek-v41.md",
+ "qwenimage21pipeline": "Qwen-Image-2.1 is NOT converted: the image engine reads the "
+ "diffusers checkpoint as downloaded; see docs/qwen-image.md",
}
diff --git a/site/index.html b/site/index.html
index ed5bc742..ca0b9d10 100644
--- a/site/index.html
+++ b/site/index.html
@@ -384,7 +384,8 @@ footer{border-top:1px solid var(--line)}
<div class="chart"><div class="crow"><span class="cl">kimi-k3<s>104B active</s></span><span class="ct"><i style="width:100.0%"></i></span><span class="cv">2.8T</span></div><div class="crow"><span class="cl">inkling<s>41B active</s></span><span class="ct"><i style="width:34.8%"></i></span><span class="cv">975B</span></div><div class="crow"><span class="cl">glm-5.2<s>40B active</s></span><span class="ct"><i style="width:26.6%"></i></span><span class="cv">744B</span></div><div class="crow"><span class="cl">deepseek-v4.1-flash<s>16B active</s></span><span class="ct"><i style="width:19.7%"></i></span><span class="cv">552B</span></div><div class="crow"><span class="cl">glm-5.3-flash<s>18B active</s></span><span class="ct"><i style="width:11.4%"></i></span><span class="cv">320B</span></div><div class="crow"><span class="cl">deepseek-v4-flash<s>13B active</s></span><span class="ct"><i style="width:10.1%"></i></span><span class="cv">284B</span></div><div class="crow"><span class="cl">qwen3.8-flash-next<s>6B active</s></span><span class="ct"><i style="width:6.4%"></i></span><span class="cv">180B</span></div><div class="crow"><span class="cl">qwen3.6-35b-a3b<s>3B active</s></span><span class="ct"><i style="width:1.2%"></i></span><span class="cv">35B</span></div><div class="crow"><span class="cl">olmoe<s>1B active</s></span><span class="ct"><i style="width:0.8%"></i></span><span class="cv">7B</span></div></div>
<p class="src">Total parameters. Only the active ones are computed per token, and only the
routed experts move. The engines, by the names the docs use:
- GLM-5.3-Flash, GLM-5.2/5.3, Inkling, Kimi K3, OLMoE, Qwen3.6-35B-A3B, Qwen3.8-Flash-Next, DeepSeek V4 Flash, DeepSeek V4.1 Flash.</p>
+ GLM-5.3-Flash, GLM-5.2/5.3, Inkling, Kimi K3, OLMoE, Qwen3.6-35B-A3B, Qwen3.8-Flash-Next, DeepSeek V4 Flash, DeepSeek V4.1 Flash.
+ And for images, Qwen-Image-2.1: text to picture, drawn inside the terminal.</p>
<a class="more" href="#models">Browse all models &rarr;</a>
</div>