Run frontier MoE models on hardware you already own â pure C, zero deps, experts streamed from disk. Tiny engine, immense model. ðŠ
C
docs: add Japanese README
Ikko Eltociear Ashimine90d115f007bb
diff --git a/README.it.md b/README.it.mdindex a6329fea..ef689868 100644--- a/README.it.md+++ b/README.it.md@@ -4,7 +4,7 @@<p align="center"><a href="https://discord.gg/RXV83nSZdk"><b>Discord</b></a> ·- <a href="README.md">English</a> · <a href="README.zh-CN.md">ç®äœäžæ</a> · <a href="README.zh-TW.md">ç¹é«äžæ</a> · Italiano+ <a href="README.md">English</a> · <a href="README.zh-CN.md">ç®äœäžæ</a> · <a href="README.zh-TW.md">ç¹é«äžæ</a> · Italiano · <a href="README.ja.md">æ¥æ¬èª</a></p>**Motore piccolo, modello immenso.** Esegui **modelli MoE di frontiera â da 744diff --git a/README.ja.md b/README.ja.mdnew file mode 100644index 00000000..51c77847--- /dev/null+++ b/README.ja.md@@ -0,0 +1,699 @@+<p align="center">+ <img src="assets/colibri-logo.svg" width="560" alt="colibrì â å°ããªãšã³ãžã³ã巚倧ãªã¢ãã«">+</p>++<p align="center">+ <a href="https://justvugg.github.io/colibri"><img src="https://img.shields.io/badge/website-justvugg.github.io%2Fcolibri-1f6feb" alt="Website"></a>+ <a href="https://github.com/JustVugg/colibri/releases"><img src="https://img.shields.io/github/v/release/JustVugg/colibri?color=2ea043" alt="Latest release"></a>+</p>++<p align="center">+ <a href="https://justvugg.github.io/colibri"><b>Website</b></a> ·+ <a href="https://discord.gg/RXV83nSZdk"><b>Discord</b></a> ·+ <a href="README.md">English</a> · <a href="README.zh-CN.md">ç®äœäžæ</a> · <a href="README.zh-TW.md">ç¹é«äžæ</a> · <a href="README.it.md">Italiano</a> · æ¥æ¬èª+</p>++**å°ããªãšã³ãžã³ã巚倧ãªã¢ãã«ã** ã¹ãã¬ãŒãžã»RAMã»VRAM ãåäžã®æšè«éå±€ãšããŠæ±ã+ïŒAI ã¡ã¢ãªã®ãã«ããã£ã¢åïŒããšã§ã**744B ãã 2.8T ãã©ã¡ãŒã¿ã®ããã³ãã£ã¢ MoE ã¢ãã«**ãã+ã³ã³ã·ã¥ãŒããŒåããç°ç𮿷·åšã®ããŒããŠã§ã¢äžã§ããšã³ãžã³äŸåãŒãã®çŽç²ãª C ã§å®è¡ããŸãã++çŸåšåäœããã®ã¯ 9 ã€ã®ãã¡ããªãŒã§ã: **GLM-5.2/5.3**ïŒ744BïŒã**GLM-5.3-Flash**ïŒ321Bã+ããžã§ã³å¯Ÿå¿ïŒã**Inkling**ïŒ975BïŒã**Kimi K3**ïŒ2.8TïŒã**DeepSeek V4 Flash**ïŒ284BïŒã**DeepSeek V4.1 Flash**ïŒ552Bãããžã§ã³å¯Ÿå¿ïŒã+**Qwen3.8-Flash-Next**ïŒ125B + 51B n-gramïŒã**Qwen3.6**ïŒ35B-A3BïŒããããŠ+**OLMoE**ïŒ7BïŒââ+ããããã C ãã¡ã€ã« 1 ã€ã§ãåã `coli chat` / `coli serve` / `coli web` ããã³ããšã³ããå ±æããŸãã+[å šã¢ãã«äžèЧ â](#other-supported-models)++> **Colibrì ã¯ä»æ¥ããã«åãããæšè«ãšã³ãžã³ã§ãããåæã«ãªãŒãã³ãªç ç©¶+> ãã©ãããã©ãŒã ã§ããããŸãã** äž»ãªç®æšã¯ããœãããŠã§ã¢ãšããŒããŠã§ã¢ã®å¢çå šäœââ+> ã¢ãã«ãã©ãŒããããã¡ã¢ãªéå±€ãã¹ãã¬ãŒãž I/Oãé 眮ãã¹ã±ãžã¥ãŒãªã³ã°ãã«ãŒãã«ã+> ææ©çãã³ãŒããCPU/GPU ã®ãªãŒããŒã©ããââã«ããã£ãŠæšè«åŽã®æ§èœã远æ±ãã+> å€§èŠæš¡ã¢ãã«ãåžå°ãªããŒããŠã§ã¢ã«äŸåãããããäœã³ã¹ãã§åãããã«ããããšã§ãã++Colibrì 㯠VRAMã»RAMã»ã¹ãã¬ãŒãžãåäžã®ãã«ããã£ã¢éå±€ãšããŠæ±ããæå³çã«+æ»ããã·ã¹ãã äžã®ã¢ã€ãã¢ã詊ãå Žãšãªã£ãŠããŸãããã®ãã **é床㫠SLA ã¯ãããŸãããã+ã»ãã³ãã£ã¯ã¹ã¯å³æ Œã«ä¿èšŒããŸã**ãå®éšã¯åçŸå¯èœãªãšã³ãããŒãšã³ãã®èšæž¬ã«ãã£ãŠ+æ¡çšã«å€ããããšã瀺ããªããã°ãªãããããã©ã«ãã®ããªã·ãŒã¯ **ã¢ãã«ã®ç²ŸåºŠãã«ãŒã¿ãŒã®+ã»ãã³ãã£ã¯ã¹ãé»ã£ãŠå€æŽããããšã¯æ±ºããŠãããŸãã**ãé«éã¡ã¢ãªãäžè¶³ãããšé床ã¯+èœã¡ããããããŸããããããã«ãã£ãŠã¢ãã«ãå¯ãã«å¥ç©ã«ãªãããšã¯ãã£ãŠã¯ãªããŸããã++```+$ ./coli chat+ ðŠ colibri v1.11.0 â GLM-5.2 · 744B MoE · int4 · streaming CPU+ â ready in 32s · resident 9.9 GB+ ⺠ciao!+ â Ciao! ð Come posso aiutarti oggi?+```++## åäœã®æ§å++<p align="center">+ <img src="docs/media/colibri-dashboard.png" width="900" alt="colibrì Web ããã·ã¥ããŒã â ã©ã€ãã¡ããªã¯ã¹ãããŒããŠã§ã¢ããã«ããšãã¹ããŒãã®ãã£ã¢">+</p>+<p align="center"><em>Web ããã·ã¥ããŒãïŒ<code>./coli web</code>ïŒ: 744B ã¢ãã«ã <strong>4 tok/sãTTFT 1.6 ç§ããã£ã¹ã¯ 0</strong> ã§åäœ â+6à RTX 5090 äžã§ãšãã¹ããŒããå®å šåžžé§ãããã©ã€ãã®ããŒã¯ã³ã¡ããªã¯ã¹ãã¿ãŒã³ããšã®æéå èš³ã+VRAM/RAM/ãã£ã¹ã¯ã®ãã£ã¢ããŒãé ã«ã¯ã©ã€ãã®ããè³ã衚瀺ããŠããŸãã</em></p>++<p align="center">+ <img src="docs/media/colibri-brain.png" width="900" alt="Brain ããŒãž â 19,456 åã®ãšãã¹ããŒããçããç®è³ªãšããŠè¡šç€º">+</p>+<p align="center"><em><strong>Brain</strong> ããŒãž: 19,456 åã®ãšãã¹ããŒããã¹ãŠãçããç®è³ªãšããŠè¡šç€º â è²ã¯ã¹ãã¬ãŒãžã®ãã£ã¢ã+æããã¯ã«ãŒãã£ã³ã°ã®ç±éã衚ãã1 ã¿ãŒã³ã§ã«ãŒãã£ã³ã°ããããšãã¹ããŒãã¯çœãå ããŸãããããŒãããšãã®ãšãã¹ããŒãã®+<a href="https://github.com/JustVugg/colibri/issues/175">èšæž¬ããããããã¯èŠªåæ§</a>ã衚瀺ãããŸãã</em></p>++<p align="center">+ <img src="docs/media/colibri-atlas.png" width="900" alt="Atlas ããŒãž â èšæž¬ããããšãã¹ããŒãã¢ãã©ã¹ã 3D éæ²³ãšããŠè¡šç€º">+</p>+<p align="center"><em><strong>Atlas</strong> ããŒãž: <a href="https://github.com/JustVugg/colibri/issues/175">èšæž¬ããããšãã¹ããŒãã¢ãã©ã¹</a>+ã 3D éæ²³ãšããŠè¡šç€º â ç¹æ§ãæããã«ãªã£ã 13,260 åã®ãšãã¹ããŒããšããããã¯ïŒè©©ãæ³åŸãäžåœèªãSQLâŠïŒããšã«+ã¯ã©ã¹ã¿ã圢æããåçŸæ§ã®ãã 1,041 åã®ã¹ãã·ã£ãªã¹ããäœçœ®ã¯åŠç¿ãããåã蟌ã¿ã§ã¯ãªããèšæž¬ãããã«ãŒãã£ã³ã°èŠªåæ§ã§ãããã©ãã°ã§å転ã§ããŸãã</em></p>++## ç ç©¶ã®ããã·ã§ã³++Colibrì ãããã°ããã©ã€ããŒããªããã³ãã£ã¢ã¢ãã«ãžã®ã¢ã¯ã»ã¹ãããã€ããŒã¹ã±ãŒã©ãŒçŽããŒããŠã§ã¢ã®å ¥æå¯èœæ§ã«å¶éãããããšã¯ãããŸããã++ãã«ããã£ã¢æ©èœã«ãã£ãŠãColibrì 㯠**æšè«ãšã³ãžã³ã®ãã€ãã©ã€ã³ãç©æ¥µçã«æé©åãã+ãããã©ã€ãšã¿ãªãªããŒããŠã§ã¢ãžã®äŸåãåãé€ããŸã**ã++ç§ãã¡ã®å®åäžã®ããã·ã§ã³ã«ã¯ãéã¿ã®è¡šçŸæ¹æ³ãšç§»åæ¹æ³ãå€ããããšãäœã VRAMã»RAMã»+ã¹ãã¬ãŒãžã«çœ®ãããæ±ºããããšãç°çš®èšç®è³æºããªãŒããŒã©ãããããããšãèµ·åãšåæã®+ãªãŒããŒããããæžããããšãã¹ããŒã¹æ§ãšåå©çšã掻çšããããšããããŠæ°ãããã³ãŒã+ã¢ã«ãŽãªãºã ã詊ãããšãå«ãŸããŸããæ £ç¿çã ãããšããçç±ã ãã§å®ããããã®ã¯ãªãã+ãã€ã¯ããã³ãããŒã¯ã§éãèŠãããšããçç±ã ãã§æ¡çšããããã®ããããŸãããæ±ºãæãšãªãã®ã¯+宿©ã§ã®ãšã³ãããŒãšã³ãã®æšè«çµæã§ãããã¹ã«ãŒããããã¬ã€ãã³ã·ãã¡ã¢ãªãã³ã¹ããš+䞊ãã§ãæ£ãããšå質ãèšæž¬ãããŸãã++ãã®å®éçãªåž°çµã **ã¢ã¯ã»ã·ããªãã£** ã§ãããã§ã«æã£ãŠããããŒããŠã§ã¢ã§+744B ãã©ã¡ãŒã¿ã®ã¢ãã«ãåããããã¹ãŠã®ãšãã¹ããŒããçºç«ããæ§åããªã¢ã«ã¿ã€ã ã§çºãã+ãããå®çŸããŠããã³ãŒãã倿Žã§ããŸããAPI ã®åããã«ããç¥èœãåããã®ã§ã¯ãªãã+ããã *æã«ãã* ââ調ã¹ãèšæž¬ããæ¹åããââã®ã§ãããšã³ãžã³ã¯æå³çã«å°ããä¿ãããŠããã+æ¬¡ã®æçšãªæé©åã¯ããããèšæž¬ããããšãã誰ããã§ãçãŸãåŸãŸãã++## ã³ã¢æè¡ãšèšæž¬çµæ++- **ãã£ã¢å®¹éã«çžãããªãåäžã®éå±€ã** VRAMã»RAMã»NVMe ã¯åãéã¿ã眮ãé 眮ãã£ã¢ã§ãã+ é«éã¡ã¢ãªã®å¶çŽã¯é床ãå€ããã ãã§ãã¢ãã«ã®ã»ãã³ãã£ã¯ã¹ã¯å€ããŸããã+- **éã¿ã®ããã® JITã** ãã¹ãŠã®ãšãã¹ããŒããããŒããã代ããã«ãèšæž¬ãããã«ãŒãã£ã³ã°ã®ç±éã+ ã¬ã€ã€ãŒããšã® LRUãåŠç¿ããããã³çããããã¹ãã¢ã1 ã¬ã€ã€ãŒå ã®ããªãã§ãããé§åããŸãã+ ç¹°ãè¿ãã®ããã¯ãŒã¯ããŒãã§ã¯å¹æããããŸãããå±¥æŽã¯éåŠç¿ãåŸãããå èªã¿ã¯äžéšã®+ ãã¹ãã§ã¯é广ã«ãªãåŸããããã©ã¡ããçŽæã§ã¯ãªãèšæž¬å¯èœãªããªã·ãŒãšããŠæ±ãããŸãã+- **I/O ã¯ãšã³ãžã³ã®äžéšã** ãããåããããšãã¹ããŒãã®åéåãèªã¿èŸŒã¿ãšèšç®ã®ãªãŒããŒã©ããã+ `O_DIRECT`ãéã¿ä»ããã¥ã¢ã« SSD ã¹ãã©ã€ãã³ã°ã«ãã£ãŠãã¹ãã¬ãŒãžã®ã¬ã€ãã³ã·ã+ ã¿ãã§ãããã®ããã«è£ ãã®ã§ã¯ãªããã¹ããªãŒãã³ã°çµè·¯ãã®ãã®ã«åãçµã¿ãŸãã`O_DIRECT` ã¯+ ãã©ã€ãäŸåã§ããããã¥ã¢ã« SSD ã¯ãŸã ããå¹ åºãã³ãã¥ããã£ã«ãããšã³ãããŒãšã³ãã® A/B ãå¿ èŠãšããŠããŸãã+- **ç°ç𮿷·åšå®è¡ã** CPUãCUDAãMetalãNUMA ã¡ã¢ãªãéšåçãŸãã¯å®å šãªãšãã¹ããŒãåžžé§ã¯+ 1 ã€ã®ã©ã³ã¿ã€ã ãå ±æãããã·ã³ã«å¿ããŠçµã¿åãããããŸããã©ã®çµã¿åãããæå©ãã¯ã+ èšç®èœåã垯åå¹ ãåžžé§ç¶æ³ãã¯ãŒã¯ããŒãã«ãã£ãŠæ±ºãŸããŸãã+- **å¥ã®ã¢ãã«ã«ããããšãªãç¶æ ãå§çž®ã** ããŒã¯ã³åäœã§å®å šäžèŽãããã©ã¯ãŒãæ€èšŒã+ 57 åã® 1 ã«çž®å°ããã MLA KV ç¶æ ãæ°žç¶åããããŠã©ãŒã ãªäŒè©±ãå¿ å®ãª DSA ã«ããã+ æé©åãæ£ããã«çµã³ä»ããŠããŸãããããã¯ã¡ã¢ãªã»ã¬ã€ãã³ã·ã»æ£ããã®ç¹æ§ã§ããã+ äžåŸã®ã¹ã«ãŒãããåäžã䞻匵ãããã®ã§ã¯ãããŸããã+- **å ãåããå Žåã«ã ãäœ¿ãææ©çãã³ãŒãã** ãã€ãã£ã MTP ãšææ³åŒ·å¶ãã©ããã¯+ ãšã³ãããŒãšã³ãã§èšæž¬ãããåççãæ€èšŒã³ã¹ãã«èŠåããªãå Žåã¯ç¡å¹åã§ããŸãã++## æªæ€èšŒã®ä»®èª¬ãå®éšããããŠååã®æ¹æ³++Colibrì ã¯ãå¶åŸ¡ããããšã³ãããŒãšã³ãã® A/B ã瀺ããŸã§ãæé©åã仮説ãšããŠæ±ããŸãã+çŸåšã®äž»ãªåãã¯æ¬¡ã®ãšããã§ã:++| 仮説 | ãããŸã§ã®ãšããã³ã¹ | ãŸã å¿ èŠãªå®éš |+|---|---|---|+| ã«ãŒãã£ã³ã°å±¥æŽã¯åçŽãª LRU ãããããŸããšãã¹ããŒããé 眮ã§ãã | åŠç¿ããããã³ã¯ç¹°ãè¿ãã®ã¯ãŒã¯ããŒããæ¹åããããããã³ããã«éåŠç¿ãåŸã | ã³ãŒãã£ã³ã°ããã£ãããå€èšèªãé·ã³ã³ããã¹ãã®ã¯ãŒã¯ããŒãã«ããããããŒã«ãã¢ãŠããã€ã»ãã·ã§ã³æšªæã® A/B |+| è€æ°ã® SSD ã¯ç¬ç«ãã垯åå¹ ããã³ãŒãé床ã«å€ãããã | éã¿ä»ããã©ãŒ/åå²ã«ãŒãã£ã³ã°ã¯å®è£ ã»æ€èšŒæžã¿ã§ã垯åå¹ ã¢ãã«ã¯åŠ¥åœ | å®éã«ç¬ç«ããã³ã³ãããŒã©äžã§ã®ãã³ãŒã«ããã£ãã·ã¥ç¶æ ã® 1 ãã©ã€ã察 2 ãã©ã€ãã«ãã GLM-5.2 å®è¡ |+| ããŒããŠã§ã¢ãèæ ®ãããã©ã³ããŒã¯ãåãã·ã³ã®æé©æ§æã«èªåã§è¿ã¥ãã | RAM/VRAM ã®äºç®ãšããã€ãã®ããã¯ãšã³ãã¯çŸåšãã§ã«æ€åºããã | çæããããã©ã³ããã©ããããããã¯ãŒã¯ã¹ããŒã·ã§ã³ãNUMA ãã¹ãããã«ã GPU ã·ã¹ãã ã«ãããå¶åŸ¡ããããã©ã¡ãŒã¿ã¹ã€ãŒããšæ¯èŒãã |+| ãã¹ã¬ã¹ãŸãã¯å質äžéä»ãã®è¡šçŸã§ãéã¿ã®ç§»åãæå³ã®ããã»ã©æžããã | æ£ãã/å質ã²ãŒãä»ãã®ãã©ãŒããããšéååã®ã¢ãã¬ãŒã·ã§ã³ãååšãã | å§çž®çã ãã§ãªããå質ãç§»åãã€ãæ°ãã¬ã€ãã³ã·ãæçšããŒã¯ã³ãããã®ã³ã¹ããåæã«åçŸãã |+| ã«ãŒãã£ã³ã°ãèæ ®ããææ©çãã³ãŒãã¯ãã»ãŒå®å šåžžé§ã«éããåã§ãå ãåãã | MTP ãšææ³ãã©ããã¯åäœããããMTP ã¯ãšãã¹ããŒããããççŽ 85% ä»è¿ã§ 32% ã®æå€±ãèšæž¬ãããŠãã | åççããšãã¹ããŒããããçããããã®åéåããã©ããæ·±ãã«ãããæçåå²é¢ããããã³ã°ãã |+| CPU/GPU ã®ãªãŒããŒã©ããã¯ãããã«ããã¯ãç§»ãã ãã§ãªã転éãšåæãé èœã§ãã | CUDA ãš Metal ã§ã®æ¹åã¯ããããé«é㪠CPU ãäœãåžžé§çã§ã¯ãããæã¡æ¶ããåŸã | PCIeããŠããã¡ã€ãã¡ã¢ãªãå®å šåžžé§ãã·ã³ã«ããããã¹ããŒãžããšã®ãããã¡ã€ã«ãš 1 倿°ãã€ã® A/B |++ååãããã§ãã? ããããã®è¡ãéžã³ããã¬ãã£ããªçµæãå ¬éããŠãã ãããããŒããŠã§ã¢ã+ã³ããããã¢ãã«/ã³ã³ãããæ£ç¢ºãªã³ãã³ããããã³ããããã£ãã·ã¥ç¶æ ãã¹ã«ãŒãããã+TTFTããšãã¹ããŒããããçãèªã¿èŸŒã¿ãã€ãæ°ãå質ãã§ãã¯ãèšé²ãã1 ã€ã®å€æ°ã ããå€ããŠ+åå®è¡ããçã®ãã°ãæ·»ä»ããŠãã ããããŸãã¯+[CONTRIBUTING.md](CONTRIBUTING.md) ããå§ãã+[ãã³ãããŒã¯ãããã³ã«](docs/benchmarking.md) ãšæ¯èŒããããã§ã+[å®éš issue ãäœæ](https://github.com/JustVugg/colibri/issues/new) ããŠãã ããã+ããã§ã¯ã説æã®ã€ããªãéãæ°å€ããããããå¶åŸ¡ããã倱æã®ã»ãã䟡å€ããããŸãã++## ã¢ã€ãã¢++744B ã® Mixture-of-Experts ã¢ãã«ã¯ã1 ããŒã¯ã³ãããçŽ 40B ã®ãã©ã¡ãŒã¿ããæŽ»æ§åããã+ãã®ãã¡ããŒã¯ã³ããšã«å ¥ãæ¿ããã®ã¯çŽ 11 GBïŒã«ãŒãã£ã³ã°ããããšãã¹ããŒãïŒã ãã§ã:++<p align="center">+ <img src="docs/media/sparse.png" width="880" alt="1 ããŒã¯ã³ãããæŽ»æ§åããã®ã¯ãã©ã¡ãŒã¿ã®çŽ 5.4% ã®ã¿">+</p>++ã€ãŸããã¢ãã«ã¯é«éã¡ã¢ãªã« *åãŸã* å¿ èŠã¯ãªãã**é 眮** ãããã°ããã®ã§ã:++- **å¯ãªéšå**ïŒã¢ãã³ã·ã§ã³ãå ±æãšãã¹ããŒããåã蟌㿠â çŽ 17B ãã©ã¡ãŒã¿ïŒã¯+ **int4 ã§ RAM ã«åžžé§** ããŸãïŒçŽ 9.9 GBïŒã+- **19,456 åã®ã«ãŒãã£ã³ã°ãšãã¹ããŒã**ïŒ75 MoE ã¬ã€ã€ãŒ à 256 + MTP ããããint4 ã§åçŽ 19 MBïŒã¯+ **ãã£ã¹ã¯äž** ã«çœ®ããïŒçŽ 370 GBïŒãã¬ã€ã€ãŒããšã® LRU ãã£ãã·ã¥ãåŠç¿ããããã³çããããã¹ãã¢ã+ ãªãã·ã§ã³ã® VRAM ãã£ã¢ãšãšãã« **ãªã³ããã³ãã§ã¹ããªãŒãã³ã°** ãããŸãã++ã³ã¢ã¢ã«ãŽãªãºã 㯠**éã¿ã®ããã® JIT** ã ãšèããŠãã ãããã³ã³ãã€ã©ã® JIT ã¯+ããã°ã©ã å šäœãã³ã³ãã€ã«ããããšã¯ãªããå®éã«å®è¡ãããéšåã芳å¯ããŠãããããã¹ã+ãžã£ã¹ãã€ã³ã¿ã€ã ã§ã³ã³ãã€ã«ããŸããcolibrì 㯠744B ã®ãã©ã¡ãŒã¿ç©ºéã«å¯ŸããŠåãè³ããããŸãã+ãã©ã¡ãŒã¿ã¯ä¿æãã¹ãåžžé§ç¶æ ã§ã¯ãªããã«ãŒã¿ãŒãå¿ èŠã ãšèšŒæãããŸãã«ãã®ãšãã«ã+ç°ç𮿷·åšã®ã¹ãã¬ãŒãžéå±€ïŒVRAM / RAM / NVMeïŒã«ããã£ãŠ **ã¹ããŒãžã³ã°ãããããŒã¿** ãªã®ã§ãã+èšæž¬ãããã«ãŒãã£ã³ã°ã®ç±éãã©ã®ãšãã¹ããŒããã©ã®ãã£ã¢ã«çœ®ãããæ±ºããã«ãŒã¿ãŒã¯+1 ã¬ã€ã€ãŒå ãèµ°ã£ãŠããªãã§ãããã¹ããŒãžã³ã°ã®ã¬ã€ãã³ã·ãé ãããã㊠JIT ãšåæ§ã«ã+ãšã³ãžã³ã¯ããªãã®ã¯ãŒã¯ããŒããåŠç¿ããŸãã䜿ãã°äœ¿ãã»ã©ãé©åãªãšãã¹ããŒãããããã«ãªã£ãŠãããŸãã+ãããæ©èœããã®ã¯ãã«ãŒãã£ã³ã°ã«èšæž¬å¯èœãªæ§é ãããããã§ãïŒ+[ãšãã¹ããŒãã¢ãã©ã¹](https://github.com/JustVugg/colibri/issues/175) ãåç §ïŒã+ãããŠæ§é ã¯ãã£ãã·ã¥å¯èœã§ãã++ãšã³ãžã³ã¯åäžã® C ãã¡ã€ã«ïŒ`c/colibri.c`ïŒãšå°ããªããã矀ã ãã§æ§æãããŠããŸããBLAS ãã+å®è¡æã® Python ããGPU ãå¿ èŠãããŸããã++### ããŒã«ã«ã¯ã©ã¹ã¿ã¢ãŒã++ã³ãŒãã£ããŒã¿ãŒã¯ããŒã¯ã³çæãã«ãŒãã£ã³ã°ãKV ç¶æ ãããŒã«ã«ã«ä¿æãããã£ã¹ã¯ã+ããã¯ãšã³ããšãããšãã¹ããŒãã¯ãŒã«ãŒããã«ãŒãã£ã³ã°ããã FFN ãä»ã® Mac äžã§å®è¡ããŸãã+ããã¬ã€ã€ãŒã§ã«ãŒãã£ã³ã°ããããããã®åéå㯠1 ã€ã®æ°žç¶ TCP ãªã¯ãšã¹ããšããŠéãããããã+1 ããŒã¯ã³ã§ãšãã¹ããŒãããšã«ã©ãŠã³ãããªãããçºçããããšã¯ãããŸããã++ãªãã·ã§ã³ã®ç»é²ãµãŒãã¹ãèµ·åããŸã:++```bash+./coli cluster coordinator --host 0.0.0.0 --port 8765+```++åã¯ãŒã«ãŒã§ãåã倿æžã¿ã¢ãã«ãããŒã«ã«ã«çšæããããã§:++```bash+./coli cluster worker --model /nvme/glm52_i4 --port 9100 \+ --coordinator http://COORDINATOR:8765 --advertise-host WORKER_IP+```++ãã£ã¹ã«ããªãŒã䜿ã£ãŠã³ãŒãã£ããŒã¿ãŒãå®è¡ããããéçãªæ§æã§ã¯ `--cluster-workers+HOST:PORT,...` ãæå®ããŸã:++```bash+./coli serve --model /nvme/glm52_i4 \+ --cluster-coordinator http://127.0.0.1:8765+```++ã¯ãŒã«ãŒãèšå®ãããŠããªãéããã©ã³ã¹ããŒãã¯ç¡å¹ãªã®ã§ãæ¢åã®åäžãã·ã³ã§ã®çµè·¯ã¯+å€ãããŸãããå¯ã¬ã€ã€ãŒã®ã·ã£ãŒãã£ã³ã°ããã©ãŠã¶/WebGPU ã¯ãŒã«ãŒã¯ãå¥ã®ä»åŸã®æ¡åŒµãã€ã³ãã§ãã++## ä»çµã¿++### ããŒã¯ã³ããšã®çµè·¯++<p align="center">+ <img src="docs/media/token-path.png" width="880" alt="ã«ãŒãã£ã³ã° â åéå â é 眮 â ãªãŒããŒã©ãã â åŠç¿">+</p>++ãã¹ãŠã®ããŒã¯ã³ã®ãã¹ãŠã®ã¬ã€ã€ãŒããåã 5 ã€ã®ã¹ãããããã©ããŸããèšèšäžã®ç®æšã¯ã+**é çœ®ãæ±ºããã®ã¯åžžã«é床ã ã** ãšããããšã§ãããšãã¹ããŒãã VRAM ããå¿çããããš+ãã£ã¹ã¯ããå¿çããããšãã«ãŒã¿ãŒã®å€æãéã¿ã®ç²ŸåºŠãåãã§ãã++### 1 ã€ã®ã¡ã¢ãªèŠä»¶ã§ã¯ãªãã1 ã€ã®ã¡ã¢ãªéå±€++<p align="center">+ <img src="docs/media/tiers.png" width="880" alt="VRAM / RAM / NVMe ã® 3 ãã£ã¢ã«ãããšãã¹ããŒãåžžé§">+</p>++### ãã¥ã¢ã« SSD: ã¢ãã«ã®ã³ã㌠2 ã€ã§ãèªã¿èŸŒã¿åž¯åå¹ ã 2 åã«++ã»ãšãã©ã®ãã·ã³ã§ãã³ãŒãã¯ãã£ã¹ã¯åŸéã§ããããšãã¹ããŒãã®èªã¿èŸŒã¿ã¯èªã¿åãå°çšã§ããããã§ **2 å°ç®ã® SSD** ããããªããããã«ã¢ãã«ã®å®å šãªã³ããŒã眮ãããšã³ãžã³ã«äž¡æ¹ã®ãã©ã€ãããåæã«ã¹ããªãŒãã³ã°ãããŸããã:++```bash+COLI_MODEL=/fast/glm52_i4 COLI_MODEL_MIRROR=/second/glm52_i4 ./coli chat+COLI_DISK_WEIGHTS=9,3 ... # ãªãã·ã§ã³: ãã©ã€ããª,ãã©ãŒã®åž¯åå¹ æ¯ïŒæªæå®ãªãèµ·åæã«èšæž¬ïŒ+```++åãšãã¹ããŒãã¯ã2 å°ã®ãã©ã€ãã®èšæž¬ãããïŒãŸãã¯å®£èšãããïŒåž¯åå¹ ã§éã¿ä»ããããæ±ºå®è«çããã·ã¥ã«ãã£ãŠäžæ¹ã®ãã©ã€ãã«å²ãåœãŠãããŸãããã®ãã readahead/PILOT ã®ããªãã§ãããšèŠæ±æã®èªã¿èŸŒã¿ã¯åžžã«åããã©ã€ãã«åœãããäºéã«ãã£ãã·ã¥ãããããšã¯ãããŸãããåèšåž¯åå¹ ã¯äž¡ãã©ã€ãã®åã«ãªããŸã â 9 GB/s + 3 GB/s ã®çµã¿åããã§ã¯ãé«éãªãã©ã€ãåäœããããšãã¹ããŒãã®èªã¿èŸŒã¿ãçŽ 33% éããªããOMP 䞊åã®ãã³çã/ãŠã©ãŒã ã¢ããã®ããŒããäž¡æ¹ããã¹ããªãŒãã³ã°ãããŸããç¥ã£ãŠããã¹ã詳现:++- ãã©ãŒã¯ **èµ·åæã«æ€èšŒ** ãããŸãïŒãã¡ã€ã«ããšã®ãµã€ãºãš safetensors ãããããã©ã€ããªãšãã€ãåäœã§äžèŽããå¿ èŠããããŸãïŒãäžèŽããªãããŸãã¯æ¬ ããŠãããã¡ã€ã«ã¯é»ã£ãŠãã©ã€ããªã®ãŸãŸã«ãªãã®ã§ã**éšåçãªãã©ãŒã§ãåé¡ãããŸãã** â äžéšã®ã·ã£ãŒããã眮ããªãå°ããã® 2 å°ç® SSD ã§ã广ããããŸãã+- ãã©ãŒã«ã¯ **äžåæžã蟌ãŸããŸãã**: `.coli_usage`ã`.coli_kv` ããã³ãã¹ãŠã®ãµã€ãã«ãŒãã¡ã€ã«ã¯ãã©ã€ããªã«æ®ããŸãã+- ãã©ãŒã§ã®èªã¿èŸŒã¿ãšã©ãŒã¯ãã©ã€ããªã«ãã©ãŒã«ããã¯ããŸãïŒèŠå 1 åãã¯ã©ãã·ã¥ãªãïŒããã®ããå®è¡äžã« 2 å°ç®ã®ãã©ã€ããæããŠãããµãŒããŒãèœã¡ãã®ã§ã¯ãªãæ§èœãäœäžããã ãã§ãã+- ã«ãŒãã£ã³ã°ãããŒã¯ã³ãå€ããããšã¯ãããŸãã â äž¡ã³ããŒã¯ãã€ãåäœã§åäžã§ãããå®è¡ããšã® `MIRROR:` çµ±èšè¡ã«ã¯ãã©ã€ãããšã«æäŸãã GB æ°ã衚瀺ãããŸãã++åããšã³ãžã³ãããããèŠæš¡ãã«ããŒããŸãã25 GB ã®ã©ãããããã§ã¯ãã¹ãŠããã£ã¹ã¯ãã+ã¹ããªãŒãã³ã°ããïŒé ããæ£ããïŒã倧ããªãã¹ãã§ã¯ãšãã¹ããŒãå šäœãåžžé§ã+ïŒ`CUDA_EXPERT_GB=auto PIN_GB=all`ïŒããã£ã¹ã¯ã¯ãã³ãŒãçµè·¯ããå®å šã«å€ããŸãã+ãã£ã¢ã®éã«ã¯ **åŠç¿ãã£ãã·ã¥** ããããŸãããšã³ãžã³ã¯ *ããªãã®* ã¯ãŒã¯ããŒããã©ã®+ãšãã¹ããŒãã«ã«ãŒãã£ã³ã°ãããããèšé²ãïŒ`.coli_usage`ãã¿ãŒã³ããšã«æŽæ°ïŒãæãããããª+ãã®ãèªåã§ãã³çãããŸã â colibrì ã¯æåã©ããã䜿ãã°äœ¿ãã»ã©éããªããŸãããã«ããœã±ããã®+ãã¹ãã§ã¯ã`COLI_NUMA=1` ã«ãã£ãŠåžžé§ããéã¿ãã¡ã¢ãªã³ã³ãããŒã©éã§ã€ã³ã¿ãŒãªãŒãããŸã+ïŒ[#82](https://github.com/JustVugg/colibri/issues/82)ïŒã++ã¢ãã«å šäœã眮ããªã 2 å°ç®ã®ãã©ã€ãåãã«ãColibri ã¯ãã§ã«åŠç¿ããŠãããšãã¹ããŒãå±¥æŽãã+éšåãã©ãŒã®åªå é äœãä»ããããŸãããŸãããã€ãã®ä»£è¡šçãªããã³ãããå®è¡ã㊠`.coli_usage` ã«+ã¯ãŒã¯ããŒããåæ ãããŠããããã©ãŒãèšç»ã»ã¹ããŒãžã³ã°ã»æ€èšŒããŸã:++```bash+./c/coli mirror plan --model /fast/glm52_i4 --mirror /second/glm52_i4 \+ --budget-gib 200 --reserve-gib 20+./c/coli mirror stage --model /fast/glm52_i4 --mirror /second/glm52_i4 \+ --budget-gib 200 --reserve-gib 20+./c/coli mirror verify --model /fast/glm52_i4 --mirror /second/glm52_i4+```++ãã©ã³ããŒã¯ safetensors ããããçŽæ¥èªã¿ã`COLI_MODEL_DIRS` ããåå²ã¢ãã«ã®ãã£ã¬ã¯ããªããã©ãã+æãããããªã«ãŒãã£ã³ã°ãšãã¹ããŒããæäŸã§ããã·ã£ãŒããåªå ããŸããã¹ããŒãžã³ã°ããã©ã€ããªã®+ã¢ãã«ã倿Žããããšã¯ãããŸãããäžæãã¡ã€ã«çµç±ã§ã³ããŒããæå®ããã空ã容éã®äºåã確ä¿ãã+ãã¹ãŠã®ã·ã£ãŒãã SHA-256 ã§æ€èšŒããæ¢åã®ãã©ãŒã·ã£ãŒããåé€ããããšã¯ãªããéžæããã+ãã©ãŒã®æºåãæŽã£ãŠåããŠã¬ã·ãŒããã¢ãããã¯ã«å ¬éããŸãã++### ãã£ã¹ã¯ãäºåºŠåŸ ããªã++ãã¹ã¯ã³ã¹ããé«ãããããšã³ãžã³ã¯å·¥å€«ã®å€§éšåããã¹ã®åé¿ãšãªãŒããŒã©ããã«è²»ãããŸãã+åãšãã¹ããŒãã® 3 ã€ã®è¡åã¯é£æ¥ããŠæ ŒçŽããã1 åã® `pread` ã§èªã¿èŸŒãŸããŸããäžéä»ãã®+éåæ I/O ããŒã«ïŒ`PIPE=1`ãããã©ã«ãïŒã¯ãåžžé§ããŠãããšãã¹ããŒããèšç®ããŠããéã«+æ¬ ããŠãããšãã¹ããŒããããŒãããŸãããããåãããäœçœ®ã§ã¯äžæãªãšãã¹ããŒãã 1 åã ãèªã¿èŸŒã¿+ïŒ**batch-union**ïŒãã«ãŒã¿ãŒå èªã¿ã¹ã¬ããïŒ`PILOT=1`ïŒã次ã®ã¬ã€ã€ãŒã®ãšãã¹ããŒãã+ããªãã§ããããŸã â ã«ãŒãã£ã³ã°ã¯ **1 ã¬ã€ã€ãŒå ã 71.6% ã®ç²ŸåºŠã§äºæž¬å¯èœ** ã§ããããšãèšæž¬ãããŠããŸãã+GPU ã§ã¯ãåžžé§ãã€ãã©ã€ã³ïŒ`COLI_CUDA_PIPE=2`ïŒãæ®å·®ã¹ããªãŒã ãã¬ã€ã€ãŒããŸããã§ããã€ã¹äžã«+ä¿æãããããCPU ã®ãšãã¹ããŒãã«ãŒãã¯äžæãããã«å®è¡ãããŸããApple Silicon ã§ã¯å®éšçãª+[Metal ããã¯ãšã³ã](docs/metal.md) ããŠããã¡ã€ãã¡ã¢ãª GPU äžã§ãããåããããšãã¹ããŒãæŒç®ãè¡ãã+[Vulkan ããã¯ãšã³ã](docs/vulkan.md) ã¯ãšãã¹ããŒããã£ã¢ãå¯ãªå°åœ±ãMLA ã¢ãã³ã·ã§ã³ã®ã³ã¢ãã+Vulkan 1.2 ãã©ã€ããæã€ãããã GPU ã«ãããããŸã â Mesa/RADV çµç±ã® AMD ã«ãŒããå«ãŸããŸã+ïŒRX 580 ã®ããã«ãã³ããŒã®ã¹ã¿ãã¯ããµããŒããçµäºããã«ãŒãã§ã¯å¯äžã®ããã¯ãšã³ãã§ããã+RDNA4 ã§ã¯ ROCm ãšäºè§ã§ã â [ãã³ãããŒã¯ã«é¢ããã¡ã¢](docs/vulkan.md) ãåç §ïŒã++> **å®éã® NVMe ã§ã¯ `DIRECT=1` ãèšæž¬ããŠãã ããã** O_DIRECT ã¯ããŒãžãã£ãã·ã¥ããã€ãã¹ãã+> DRAM ãã£ãã·ã¥ãšåž¯åå¹ ã«äœè£ã®ãããã©ã€ãã§ã¯å€§ããªæ¹åã«ãªãããšãå€ãã§ãïŒBlackwell/Windows+> ãã·ã³ã§ `PIPE=1` ãšäœµçšããŠãã³ãŒã +34% ãèšæž¬ãGB10 ã® iobench ã§ 4.25â9.69 GB/sïŒã+> ãã ããã©ã€ãäŸåã§ãããQLC/DRAM ã¬ã¹ãä»®æ³åããããã£ã¹ã¯ã§ã¯å¹æãªããé广ã«ãªãåŸãŸãã+> ãŸã詊ããŠãããŒããŠã§ã¢ãå ±ããŠãããèšå®ãæ®ããŠãã ããã++### å¿ å®ãªã¢ãã«ãå§çž®ãããç¶æ ++ãã©ã¯ãŒããã¹ã¯ `transformers` ã®ãªã©ã¯ã«ã«å¯ŸããŠæ€èšŒãããŠããŸãïŒteacher-forcing ã§+éåžž 30ã32/32ãå°ããªãªã©ã¯ã«ã® 2 ã€ã®äœçœ®ã¯æµ®åå°æ°ç¹äžã®ã»ãŒåå€ã§ãããŒã«ãã§ãŒã³ã«äŸåããŸãïŒã+MLA ã¢ãã³ã·ã§ã³ã¯å§çž®ããã KV ç¶æ ãä¿åããŸã â 1 ããŒã¯ã³ããã 32,768 åã§ã¯ãªã 576 åã®+æµ®åå°æ°ç¹æ°ïŒ**57 åã® 1**ïŒâ ãããŠãããåèµ·åããŸããã§æ°žç¶åããŸãïŒ`.coli_kv`ïŒã+äŒè©±ã¯åããªãã£ã«ãªãã§ãŠã©ãŒã ãªç¶æ ããåéãããäžæãããªãã£ãã»ãã·ã§ã³ãšãã€ãåäœã§åäžã§ãã+DSA ã¹ããŒã¹ã¢ãã³ã·ã§ã³ïŒGLM-5.2 ã® lightning indexerïŒã¯å¿ å®ã«å®è£ ãããŠãããå šããŒã匷å¶çã«+éžæããããšãã«å¯ãªã¢ãã³ã·ã§ã³ãæ£ç¢ºã«åçŸããããšã§æ€èšŒãããŠããŸãã++### ææ©çãã³ãŒãããèª å®ã«++GLM-5.2 ã®ãã€ãã£ã MTP ããããããŒã¯ã³ããã©ããããã¡ã€ã³ã¢ãã«ã 1 åã®ãããåããã+ãã©ã¯ãŒãã§ãããæ€èšŒããŸã â 广ãããå Žåã¯ãã©ã¯ãŒãããã 2.2ã2.8 ããŒã¯ã³ãèŠåŽããŠ+åŸã 2 ã€ã®ã«ãŒã«ãããã©ã«ããšããŠçµã¿èŸŒãŸããŠããŸããMTP ããã㯠**int8** ã§ãªããã°ãªããªãããš+ïŒint4 ã®ãããã¯åççã 0ã4% ã«åŽ©å£ããŸãã[#8](https://github.com/JustVugg/colibri/issues/8)ïŒã+ãããŠãã©ãããšæ€èšŒã¯ **åã颿°** ãèšç®ããªããã°ãªããªãããš â `SPEC_PIN=1` ã¯äž¡è ã+1 ã€ã®ã«ãŒãã«ãã¡ããªãŒã«åºå®ããŸãïŒè©³ãã調æ»ã®çµç·¯ã¯ [#163](https://github.com/JustVugg/colibri/issues/163) ã«ãããŸãïŒã+ææ³åŒ·å¶ãã©ããïŒ[`GRAMMAR=file.gbnf`](docs/grammar-draft.md)ïŒã¯ãå¶çŽä»ã JSON åºåã§+ã»ãŒã¿ãã§åççãäžããŸããææ©çãã³ãŒããæ£å³ã§ãã©ã¹ã«ãªããã¯ãã£ãã·ã¥ã®æž©ãŸãå ·åã«+äŸåããŸã â èšæž¬ãã广ããªããã° `DRAFT=0` ã䜿ã£ãŠãã ããã++## äœãå®çŸããŠããã++<p align="center">+ <img src="docs/media/ladder.png" width="880" alt="ããŒããŠã§ã¢ã¯ã©ã¹å¥ã®èšæž¬ãã³ãŒãé床">+</p>++åããšã³ãžã³ãåã int4 ã³ã³ãã â ããŒããŠã§ã¢ãå€ããã®ã¯ãšãã¹ããŒãã®çœ®ãå Žæã ãã§ãã+[å®å šãªãã³ãããŒã¯è¡š](docs/benchmarks.md) ããã®ãã€ã©ã€ã:++- **6à RTX 5090ãå®å šåžžé§:** ãã³ãŒã 5.8ã6.8 tok/sãTTFT çŽ 13 ç§+ ïŒ[å®éšãã°](docs/experiments/glm52-6x5090-2026-07-12.md)ïŒ+- **128 GB ã® CPU ã®ã¿ã®ãã¹ã¯ããã:** ãŠã©ãŒã æ çŽ 1.8 tok/sïŒ[#200](https://github.com/JustVugg/colibri/issues/200)ïŒ+- **RTX 5070 Ti 1 æã®ã©ãããããçŽãã·ã³:** GPU åžžé§ãã€ãã©ã€ã³ã§ 1.07 tok/s+ ïŒ[#273](https://github.com/JustVugg/colibri/issues/273)ïŒ+- **25 GB ã®éçºãã·ã³:** ã³ãŒã«ãæ 0.05ã0.1 tok/s â ãã®ãããžã§ã¯ããå§ãŸã£ãå®èšŒæžã¿ã®äžéã§ããã+ ä»ãèª å®ãªããŒã¹ã©ã€ã³ã§ãã++å質ã¯ä»®å®ã§ã¯ãªãèšæž¬ãããŠããŸããint4 ã³ã³ããã®éååã³ã¹ããšãã¹ã±ãŒã«ç²åºŠ/å転ã®+ã¢ãã¬ãŒã·ã§ã³ã¯ [docs/benchmarks.md](docs/benchmarks.md#quality-benchmark) ãš+[#108](https://github.com/JustVugg/colibri/issues/108)/[#81](https://github.com/JustVugg/colibri/issues/81) ã«ãããŸãã++## ã¯ããã«++å¿ èŠãªãã®ã¯ 2 ã€ã§ã: **ããã°ã©ã **ïŒæ°çŸ KBïŒãš **ã¢ãã«**ïŒ372 GBïŒã+å šãã©ãããã©ãŒã åãã®æé 㯠[ã¯ã€ãã¯ã¹ã¿ãŒãã¬ã€ã](docs/quickstart.md) ã«ãããŸãã++### 1. colibri ãå ¥æãã++**ãã«ãæžã¿ãªãªãŒã¹ãããŠã³ããŒã** â LinuxãmacOSãWindows ã«å¯Ÿå¿ããã³ã³ãã€ã©ã¯äžèŠã§ãã+[Releases](https://github.com/JustVugg/colibri/releases) ããèªåã®ãã©ãããã©ãŒã çšã®+ã¢ãŒã«ã€ããååŸããŠå±éããŸã:++```bash+mkdir colibri && tar xzf colibri-v1.8.0-linux-x86_64.tar.gz -C colibri && cd colibri+python3 coli info # engine ready â+```++äžã«ã¯ãšã³ãžã³ïŒ`colibri`ãWindows ã§ã¯ `colibri.exe`ïŒã`coli` ã©ã³ãã£ãŒããã® Python+ãã«ããŒãå ¥ã£ãŠããŸããååã®å€æŽãèšå®ã¯äžèŠã§ã â `coli` ã¯èªåã®é£ã«ãããšã³ãžã³ãèŠã€ããŸãã+å¿ èŠãªã®ã¯ [Python 3](https://www.python.org/downloads/) ã®ã€ã³ã¹ããŒã«ã ãã§ããã©ã³ãã£ãŒãš+API ã²ãŒããŠã§ã€ã¯ Python ã¹ã¯ãªããã§ããããšã³ãžã³èªäœã¯äŸåãŒãã®çŽç²ãª C ã§ãã++**ãŸãã¯ãœãŒã¹ãããã«ã** â OpenMP 察å¿ã® `gcc`ïŒãŸã㯠clangïŒãå¿ èŠã§ã:++```bash+git clone https://github.com/JustVugg/colibri && cd colibri/c+./setup.sh # gcc/OpenMP ã確èªãããã«ããšã»ã«ããã¹ããå®è¡+```++`coli` ã PATH ã«çœ®ãããå Žåã¯ããã§ãã¯ã¢ãŠããã `pip install -e .` ãå®è¡ãããšç»é²ãããŸã+ïŒãšã³ãžã³ã¯åŒãç¶ã `c/` ã«ãããŸã â wheel ã§ã¯ãªããã¯ããŒã³ããã® editable ã€ã³ã¹ããŒã«ã§ãïŒã++### 2. ã¢ãã«ãå ¥æãã++倿æžã¿ã® **GLM-5.2 int4** ã³ã³ããã Hugging Face ã«ãããŸã â **int8 MTP ããã** ä»ãã®+**ã°ã«ãŒãã¹ã±ãŒã«ïŒgs64ïŒ** ãã«ãã䜿ã£ãŠãã ããããµã€ãºã¯çŽ **372 GB** ãªã®ã§ã+ååãªå®¹éã®ãããã§ããã°é«éãªãã£ã¹ã¯ã«çœ®ããŠãã ãã:++**https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp**++> â ïž å€ãè¡åäœ int4 ã®ãã©ãŒïŒ`mateogrgic/âŠ`ã`jlnsrk/âŠ`ïŒã§ã¯ãªããäžèšã® **gs64** ã³ã³ããã+> 䜿ã£ãŠãã ããããããã¯å質ãçŽ 9 ãã€ã³ãå£ãããšãèšæž¬ãããŠããã+> [#455](https://github.com/JustVugg/colibri/issues/455) ã§å ±åãããåœåã®æèã¢ãŒãã®ã«ãŒãã+> çµãããªãçæã®æ ¹æ¬åå ã§ãããgs64 ã³ã³ããã¯å¶åŸ¡ãããè¡åäœã® A/B ã§èŠããããããã®åé¡ã+> è§£æ¶ããŸããããç¹°ãè¿ãã EOS æ¬ ä¹ã«å¯Ÿããæ±çšçãªã¬ãŒãã§ã¯ãããŸãããMTP ãããã+> **int4 ã§ã¯ãªã int8** ã§ããå¿ èŠããããŸã+> ïŒint4 â ãã©ããåçç 0%ã[#8](https://github.com/JustVugg/colibri/issues/8)ïŒ:+> `ls -l <model>/out-mtp-*` â int8ïŒæ£ããïŒãªã 3 ãã¡ã€ã«ã§ `3527131672 / 5366238584 / 1065950496`ã+> ãŸãã¯åäžã® `out-mtp-00000.safetensors` ã§ `9959321520` ãã€ãã§ã+> ïŒæšå¥šã³ã³ããã®çŸåšã®ã¢ããããŒã㯠1 ãã¡ã€ã«ã§é åžãããŠããŸã: äžèº«ã¯åã+> int8 ãã³ãœã«ã§ã1 èŠçŽ 1 ãã€ãã®ãã®ã 777 åã§ãïŒã++ããã㯠FP8 ã®ãœãŒã¹ããèªåã§å€æããããšãã§ããŸã â 756 GB å šäœãäžåºŠã«ãã£ã¹ã¯ã«çœ®ãå¿ èŠã®ãªãã+åéå¯èœãªã³ãã³ã 1 ã€ã§è¡ããŸã:++```bash+./coli convert --model /nvme/glm52_i4 # ã·ã£ãŒãããšã«ããŠã³ããŒã+倿ïŒpythonãååã®ã¿ïŒ+```++<a id="other-supported-models"></a>+#### ãã®ä»ã®å¯Ÿå¿ã¢ãã«++GLM-5.2 ããªãã¡ã¬ã³ã¹ã¢ãã«ã§ãããåãã¹ããªãŒãã³ã°ææ³ã§ããã« 6 ã€ã®ãã¡ããªãŒãåäœããŸãã+ããããã **å åŒãšã³ãžã³** ã§ã â C ãã¡ã€ã« 1 ã€ã§ç¬èªã®ã¢ãŒããã¯ãã£ãæã¡ãåã+`coli chat` / `coli serve` / `coli web` ããã³ããšã³ãã䜿ããŸãïŒã©ã³ãã£ãŒã¯ã¢ãã«ã®+`config.json` ãããã€ããªãéžã³ãŸãïŒ:++> **ããããã«å¿ èŠãªãã®ã** ãããã¯å€§ããç°ãªãã2 ã€ã䞊ã¹ãŠèªãã 人ãèŠä»¶ãççŸããŠãããš+> 誀解ããããšããããŸãïŒ[#191](https://github.com/JustVugg/colibri/issues/191)ïŒãççŸããŠã¯ããŸãã â+> å¥ã ã®ã¢ãã«ãªã®ã§ãã**ã©ãã GPU ã¯å¿ èŠãããŸããã**+>+> | ã¢ãã« | éã¿çšã®ãã£ã¹ã¯ | RAM | GPU |+> |---|---|---|---|+> | **OLMoE** | çŽ 7 GBïŒint8 ã³ã³ããïŒ | 8 GB | äžèŠ |+> | **GLM-5.2/5.3** | çŽ 372 GB | æäœ 16 GBãå¿«é©ã«ã¯ 24 GB | äžèŠ |+> | **GLM-5.3-Flash** | å€æåŸ çŽ 195 GB | 25 GBïŒint4 ã®éã¿ 12 GB + ãšãã¹ããŒããã£ãã·ã¥ïŒ | äžèŠ |+> | **Inkling** | çŽ 469 GB | int4 å¯ã³ã³ããäœ¿çšæ 25 GBãæªäœ¿çšæ çŽ 120 GB | äžèŠ |+> | **Kimi K3** | çŽ 1.6 TB | 32 GB ä»¥äž | äžèŠ |+> | **DeepSeek V4 Flash** | çŽ 167 GBïŒREAP 150B: çŽ 85 GBïŒ | æäœ 16 GBãå¿«é©ã«ã¯ 32 GB | ãªãã·ã§ã³ãGTX 10 ã·ãªãŒãºä»¥éã®ä»»æã® NVIDIA ã«ãŒãïŒPascal/Turing 㯠`CUDA_ARCH=portable-pre-ampere NO_TC=1` ã§ãRTX 50 ã§æè¯ïŒã«ãããããªãã£ã«ã 5ã10 åããã³ãŒããçŽ 2.5 åé«éå |+> | **Qwen3.8-Flash-Next** | çŽ 185.5 GBïŒå ¬åŒ FP8 ãã§ãã¯ãã€ã³ãïŒ | æäœ 16 GBãããã©ã«ãã®ã³ã³ããã¹ãã§å¿«é©ã«ã¯ 24 GB | é察å¿ãCPU ã®ã¿ |+> | **Qwen3.6-35B-A3B** | çŽ 20 GBïŒint4-gs64 ã³ã³ããïŒ | 24 GBïŒRAM ãžã®å®å šåžžé§ãå¿ èŠïŒ | ãªãã·ã§ã³ãCUDA VRAM ãšãã¹ããŒããã£ã¢ã¯ 8 GB ã«ãŒã 2 æã§ **1.44 -> 10.05 tok/sïŒ7.0 åïŒ** ãèšæž¬ãåºå㯠CPU ãšãããåäœã§åäž |+>+> GPU ã¯ãããŸã§éãããã ãã§ãããšãã¹ããŒãã¯ãã£ã¹ã¯ããã¹ããªãŒãã³ã°ããããããé床ã¯+> ãã£ã¹ã¯ã§æ±ºãŸããŸã â é ããã©ã€ãã§ã¯ 1 ç§ããã 1 ããŒã¯ã³æªæºãé«éãªãã©ã€ãã§ãã£ãã·ã¥ã+> æž©ãŸã£ãŠããã° 1 ç§ãããæ°ããŒã¯ã³ãæ³å®ããŠãã ããã++| ãã¡ããªãŒ | ç·æ° / ã¢ã¯ãã£ã | éã¿ | ãã«ã | ããã¥ã¡ã³ã |+|---|---|---|---|---|+| **GLM-5.2/5.3** | 744B / 40B | [`mastouri/âŠ-int4-g64-with-int8-mtp`](https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp)ïŒ372 GBïŒ | `make -C c glm` | ãã®ããŒãž |+| **Inkling**ïŒThinking MachinesïŒ | 975B / 41B | [`nbeerbower/Inkling-colibri-int4`](https://huggingface.co/nbeerbower/Inkling-colibri-int4)ïŒ469 GBïŒ | `make -C c inkling` | [inkling.md](docs/inkling.md) |+| **GLM-5.3-Flash**ïŒZ.aiïŒ | 321B / 40B | [`zai-org/GLM-5.3-Flash`](https://huggingface.co/zai-org/GLM-5.3-Flash) â ã«ãŒãã£ã³ã°ãšãã¹ããŒãã **int4-gs64** ã«å€æãå¯éšå㯠BF16 ã®ãŸãŸã§ç²ŸåºŠã¯ããŒãæã«éžæãããžã§ã³å¯Ÿå¿ | `make -C c glm53` | [glm53-flash.md](docs/glm53-flash.md) |+| **Kimi K3**ïŒMoonshotïŒ | 2.8T / 104B | [`moonshotai/Kimi-K3`](https://huggingface.co/moonshotai/Kimi-K3) â ãªãªãžãã«ã®ãã§ãã¯ãã€ã³ããã«ãŒãã£ã³ã°ãšãã¹ããŒã㯠**ãã€ãã£ã MXFP4** ã®ãŸãŸ | `make -C c kimi_k3` | [kimi_k3.md](docs/kimi_k3.md) |+| **DeepSeek V4 Flash** | 284B / 13B | å ¬åŒã®ã·ã£ãŒãåãã§ãã¯ãã€ã³ã â ã«ãŒãã£ã³ã°ãšãã¹ããŒã㯠**ãã€ãã£ã fp4**ãå¯éšå㯠fp8-e4m3 ã®ãŸãŸã**REAP ã§æåããã 150B**ïŒ[`puwaer/DeepSeek-V4-Flash-0731-reap-150b`](https://huggingface.co/puwaer/DeepSeek-V4-Flash-0731-reap-150b)ã85 GBã256 åäž 132 åã®ãšãã¹ããŒãïŒãåããšã³ãžã³ã§å€æãªãã«ããŒãå¯èœ | `make -C c deepseek-v4` | [deepseek-v4.md](docs/deepseek-v4.md) |+| **DeepSeek V4.1 Flash** | 552B / 16B | å ¬åŒãã§ãã¯ãã€ã³ãã**倿äžèŠ**: ãšãã¹ããŒãã¯ãã§ã« fp4ãå¯éšå㯠fp8-e4m3ããã®ãã¡ 203 GB ã¯äžåºŠã«æ°çŸãã€ããã€ãã£ã¹ã¯ããèªãŸãã n-gram ã¡ã¢ãªã§ãã«ãŒãã£ã³ã°ãšãã¹ããŒãã®ã³ã¹ã㯠GLM-5.2 ã® 12.7 GB ã«å¯Ÿã㊠**1 ããŒã¯ã³ããã 4.5 GB**ãããžã§ã³ãããŒã«åŒã³åºããDSpark ãã©ãããããã¯ãã¹ãŠæå¹ | `make -C c deepseek_v41` | [deepseek-v41.md](docs/deepseek-v41.md) |+| **Qwen3.8-Flash-Next**ïŒAlibabaïŒ | 125B + 51B n-gram / 6B | [`Qwen/Qwen3.8-Flash-Next-FP8`](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8) â ãªãªãžãã«ã®ãã§ãã¯ãã€ã³ããPLE ã¯ããŒãžã³ã°å¯èœãªãŸãŸããšãã¹ããŒã㯠**ãã€ãã£ãã®ããã㯠FP8** ã®ãŸãŸ | `make -C c qwen38`ïŒCPU ã®ã¿ïŒ | [qwen38.md](docs/qwen38.md) |+| **Qwen3.6**ïŒAlibabaïŒ | 35B / 3B | [`Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64`](https://huggingface.co/Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64)ïŒçŽ 20 GBã**æšå¥š**ïŒâ Gated Attention + Gated DeltaNet ã®ãã€ããªãã | `make -C c qwen36`ïŒVRAM ãšãã¹ããŒããã£ã¢ã«ã¯ `CUDA=1`ïŒ | [qwen36.md](docs/qwen36.md) |+| **OLMoE**ïŒAI2ïŒ | 7B / 1B | `c/tools/convert_olmoe_merged.py` ã§å€æ â **int8** ã³ã³ãããçŽ 7 GB | `make -C c olmoe` | â |++Qwen3.6 ã«ã¯å€ææžã¿ã³ã³ããã 3 ã€ãããŸã: **int4-gs64**ïŒæšå¥š â int8 ã®ã¢ã³ã«ãŒã«å¯Ÿããã³ãµã€ã³é¡äŒŒåºŠã¯+è¡åäœãšæ¯ã¹ãŠ 0.98777 â 0.99313ãKL 㯠0.109 â 0.080 ãšèšæž¬ãããŠãããéåå誀差ãçŽ 44% å°ãªãïŒã+A/B ã®ããŒã¹ã©ã€ã³ãšããŠã® [int4 è¡åäœ](https://huggingface.co/Kreuzzelg/qwen36-35b-a3b-colibri-i4)ã+ãã㊠[KAT-Coder v2.5](https://huggingface.co/Kreuzzelg/kat-coder-v2.5-dev-colibri-i4-gs64) ã§ãã+KAT-Coder ã¯åããšã³ãžã³ã§ãã®ãŸãŸåããŸã â ã¢ãŒããã¯ãã£ãåäžã®ãã§ãã¯ãã€ã³ãã§ããã°ã+å°çšã®ã³ãŒããã¹ãªãã§åäœããŸãã`CUDA=1` ã䜿ããšãVRAM ãšãã¹ããŒããã£ã¢ã¯+**8 GB ã«ãŒã 2 æã§ 1.44 â 10.05 tok/sïŒ7.0 åïŒ** ãèšæž¬ããåºå㯠CPU ã®çµè·¯ãšãããåäœã§åäžã§ããã++Kimi K3 ã¯å€æäžèŠã§ããQAT ã§åŠç¿ããã MXFP4 ãšãã¹ããŒãã¯ãªãªãžãã«ã® Hugging Face ã·ã£ãŒããã+çŽæ¥ã¹ããªãŒãã³ã°ãããbf16 ã®å¯ãªéã¿ã»ããã¯ããŒãæã«éååãããŸããé·ããšãŒãžã§ã³ãã»ãã·ã§ã³ã§ã¯ã+ãªã«ã¬ã³ãç¶æ ã®ãã§ãã¯ãã€ã³ãããªããã€ã³ã§ããŸãïŒRAM äžã« `COLI_K3_CKPT=N` ã¹ãããããŸãã¯+`COLI_K3_CKPT_DIR` ã§ãã£ã¹ã¯ã«éé¿ïŒãç·šéãããããã³ããããã©ããŒã¢ããããã³ããã¯ãæ®ã£ãŠãã+æãæ·±ããã§ãã¯ãã€ã³ãã埩å ããŠæ«å°Ÿã ããåããªãã£ã«ãããããäŒè©±å šäœã SSM ã¬ã€ã€ãŒã§+åçãçŽãå¿ èŠããããŸãããVulkan ãã¹ãã§ã¯ `K3_VK_UP=auto` ããèšæž¬ããã垯åå¹ ãã+ãšãã¹ããŒããã£ã¢ã®ã¢ããããŒãéãæ±ºããŸãããšã³ãžã³ã® KDA ãš MLA ã®çµè·¯ã¯ãCI ã§ãã³ããŒå®è£ ã«+察ããŠããŒã¯ã³åäœã§å®å šäžèŽããããšãæ€èšŒãããŠããŸãã++Inkling 㯠int4 ã®ãšãã¹ããŒããš **bf16 ã®å¯ãªéã¿**ïŒåžžé§ 49.4 GBïŒã§æäŸãããŠããŸããããã+ä¿æã§ããªããã¹ãåãã«ã[inkling.md](docs/inkling.md) ã«ã¯å¯ãªéã¿ã»ããã 15.3 GB ã«ãã+ã¯ã³ãã¹ã®ããŒã«ãããã975B ã 25 GB ã®ãã·ã³ã§åãããŸã â ãã¬ãŒããªããèª å®ã«æžãããŠããŸãã++### 3. å®è¡ãã++```bash+COLI_MODEL=/nvme/glm52_i4 ./coli chat # RAM äºç®ããã£ãã·ã¥ãMTP ãèªåæ€åº+COLI_MODEL=/nvme/glm52_i4 ./coli plan # èšç»ããã VRAM/RAM/ãã£ã¹ã¯ã®é 眮ã確èª+COLI_MODEL=/nvme/glm52_i4 ./coli doctor # èªã¿åãå°çšã®æºåç¶æ³ãã§ãã¯+COLI_MODEL=/nvme/glm52_i4 ./coli doctor --deep # ãã³ãœã«/ã·ã£ãŒã/ã€ã³ããã¯ã¹/ãã©ãŒã®å³å¯ãªäºåãã§ãã¯+COLI_MODEL=/nvme/glm52_i4 ./coli tune # ãã®ãã·ã³ã§æéãã€å®å šãªå®è¡ãããã¡ã€ã«ãèšæž¬ããŠä¿å+./coli web --model /nvme/glm52_i4 # API + ããã·ã¥ããŒãããã©ãŠã¶ãéã+./coli serve --model /nvme/glm52_i4 # API + ããã·ã¥ããŒãããã©ãŠã¶ãªãïŒãããã¬ã¹ïŒ+```++Windows ã§ã¯ãªãªãŒã¹ã¢ãŒã«ã€ãã« `coli.cmd` ã忢±ãããŠããŸããããã«ã¯ãªãã¯ã§ã¯ã€ãã¯ã¹ã¿ãŒãã+ãŸã㯠cmd ã PowerShell ãã `coli.cmd chat --model D:\glm52_i4` ãå®è¡ããŠãã ããã+ãœãŒã¹ã®ãã§ãã¯ã¢ãŠãããã¯ãåãã³ãã³ãã `python coli chat --model+D:\glm52_i4` ãšããŠå®è¡ã§ããŸãã`.exe` ãã¡ã€ã«ã¯ãšã³ãžã³ã§ããã©ã³ãã£ãŒã§ã¯ãããŸããã+åäœã§èµ·åãããšããŒãããã¢ãã«ããªããããããã«çµäºããŸãã+å®è¡æã®ãšã³ãžã³ã¯çŽç²ãª C ã§ã â Python ã¯ååã®ã¿ã®å€æããŒã«ãšãªãã·ã§ã³ã®+API ã²ãŒããŠã§ã€ã§ãã䜿ãããŸããã++#### åãã³ãã³ãã§ã©ã®ã¢ãã«ãåã++`coli` ã¯ã¢ãã«ã® `config.json` ãèªã¿ã察å¿ãããšã³ãžã³ãã€ããªãéžã³ããã®ãã¡ããªãŒã®+ãã£ãããã³ãã¬ãŒããé©çšããŸã â ãã®ãã **ã¢ãã«ãå€ãã£ãŠãã³ãã³ãã©ã€ã³ã¯äœãå€ãããŸãã**ã+䜿ããããšã³ãžã³ãäžåºŠãã«ãããããããšã¯ `COLI_MODEL` ãé©åãªãã£ã¬ã¯ããªã«åããã ãã§ã:++```bash+make -C c glm # GLM-5.2+make -C c inkling # Inkling+make -C c kimi_k3 # Kimi K3++COLI_MODEL=/nvme/glm52_i4 ./coli chat # TUI+COLI_MODEL=/nvme/inkling_i4 ./coli chat+COLI_MODEL=/nvme/kimi_k3 ./coli chat++./coli web --model /nvme/inkling_i4 # API + ããã·ã¥ããŒãããã©ãŠã¶ãéã+./coli web --model /nvme/kimi_k3+./coli serve --model /nvme/inkling_i4 # API + ããã·ã¥ããŒãããã©ãŠã¶ãªã+```++GLM 以å€ã®ãšã³ãžã³ã§ã¯ã`coli chat` ãããŒã«ã«ã§ã²ãŒããŠã§ã€ãèµ·åã㊠TUI ãããã«æ¥ç¶ããŸãã+ãã®ãã TUIãAPIãããã·ã¥ããŒãã¯ãã¹ãŠåãã¢ãŒããã¯ãã£å¯Ÿå¿ã®ãã£ãããã³ãã¬ãŒããéãã+ãã³ãã¬ãŒããèªåã§æå®ããå¿ èŠã¯ãããŸããã++ã¢ãã«ããšã«ç°ãªãç¹ã 2 ã€ãããã©ã¡ããåã¢ãã«ã®ããŒãžã«èšèŒãããŠããŸã:++- **RAM ã«äœè£ã®ãªããã¹ãã§ã® Inkling** ã«ã¯ãint4 ã®å¯ã³ã³ãããšå°ããªãšãã¹ããŒããã£ãã·ã¥ã+ å¿ èŠã§ã: `./coli chat --model /nvme/inkling_i4 --cap 2`+ ïŒ[inkling.md](docs/inkling.md) ãåç § â ããã©ã«ãã® `--cap 8` ã§ã¯ãåžžé§ã»ããã«å ããŠ+ çŽ 14 GB ã®ãã£ãã·ã¥ãå¿ èŠã§ãïŒã+- **Kimi K3** 㯠MXFP4 ãšãã¹ããŒãããªãªãžãã«ã®ãã§ãã¯ãã€ã³ãããã¹ããªãŒãã³ã°ããããã+ 倿ãããã®ã¯ãããŸãã â ãã ãã¹ãããã·ã§ããã¯çŽ 1.6 TB ã§ã+ ïŒ[kimi_k3.md](docs/kimi_k3.md) ãåç §ïŒã++### 4. ããã«è©³ãã++| ããã㯠| ããã¥ã¡ã³ã |+|---|---|+| ãã³ãããŒã¯ãã³ãã¥ããã£ã®ããŒã¿ãã€ã³ããåè³ªèšæž¬ | [docs/benchmarks.md](docs/benchmarks.md) |+| åçŸå¯èœãªãã³ãããŒã¯ãããã³ã«ãšæäœéã®ã¬ããŒã | [docs/benchmarking.md](docs/benchmarking.md) |+| ãã¥ãŒãã³ã°é ç®ãããªã·ãŒãåŠç¿ãã£ãã·ã¥ãããªãã§ãã | [docs/tuning.md](docs/tuning.md) |+| Windows 11 ãã€ãã£ããã«ãïŒ+ CUDA DLLïŒ | [docs/windows.md](docs/windows.md) |+| CUDA ããã¯ãšã³ããVRAM ãšãã¹ããŒããã£ã¢ãå®å šåžžé§ | [docs/cuda.md](docs/cuda.md) |+| Vulkan ããã¯ãšã³ãïŒä»»æã® GPU: RADV çµç±ã® AMDãROCm ããµããŒããçµäºããã«ãŒããå«ãïŒ | [docs/vulkan.md](docs/vulkan.md) |+| Apple Silicon Metal ããã¯ãšã³ã | [docs/metal.md](docs/metal.md) |+| OpenAI äºæ APIãKV ã¹ããããWeb ããã·ã¥ããŒã | [docs/api.md](docs/api.md) |+| å®éšçãªã¬ã€ã€ãŒã»ã°ã¡ã³ãåã蟌㿠ABI | [docs/segment-runtime.md](docs/segment-runtime.md) |+| å®éšçãªããŒã¯ãã€ã¶/åã蟌ã¿/ãããã® Edge ABI | [docs/edge-runtime.md](docs/edge-runtime.md) |+| ææ³åŒ·å¶ãã©ããïŒæ§é ååºåïŒ | [docs/grammar-draft.md](docs/grammar-draft.md) |+| ç°å¢å€æ°äžèЧ | [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md) |++## DeepSeek V4++**DeepSeek V4 Flash** ã¯å ¬åŒãã§ãã¯ãã€ã³ãã倿ãªãã§ã¹ããªãŒãã³ã°ããŸããã«ãŒãã£ã³ã°+ãšãã¹ããŒã㯠**ãã€ãã£ã fp4** ã®ãŸãŸãå¯ãªéã¿ã»ãã㯠UE8M0 ãããã¯ã¹ã±ãŒã«ä»ãã®+**fp8-e4m3** ã®ãŸãŸã§ããMLA + DSA ã¹ããŒã¹ã¢ãã³ã·ã§ã³ã43 ã¬ã€ã€ãŒã256 åã®ã«ãŒãã£ã³ã°+ãšãã¹ããŒããš 1 åã®å ±æãšãã¹ããŒããtop-6ãx86-64/aarch64 ã® Linux ãš Windows/MSYS2ïŒCPUïŒã«+察å¿ãããªãã·ã§ã³ã® CUDA ãã£ã¢ïŒWindows ã§ã¯ã©ã³ã¿ã€ã DLLãLinux ã§ã¯ `CUDA=1` ã«ããçŽæ¥ãªã³ã¯ã+WSL2 ã§æ€èšŒæžã¿ïŒã¯ããã¹ãŠã®ã¹ããŒãžã CPU åºæºã«ä¿ã¡ã€ã€ã¹ããŒãžããšã«ãã©ãŒã«ããã¯ããŸãã++```bash+cd c+make deepseek-v4+python ./coli chat --model /path/to/DeepSeek-V4-Flash --ram 32+# coli run / coli serve / coli web ãå¯+# Windows CUDA ãã£ã¢: make cuda-dsv4-dll CUDA_ARCH=portable ïŒRTX 50 ã§ã¯ + make cuda-dsv4-dg-dllïŒ+```++æ°ãã远å ããã 2 ã€ã®ãªããã€ã³ GPU ã¬ããŒããããã³ãã¥ããã£ã«ããèšæž¬å€ãæ±ããŠããŸãã+ã©ã¡ããããã©ã«ãã§ãªãã§ãæªèšå®æã¯ãã€ãåäœã§åäžã§ãã`DSV4_HYBRID=1` ã¯ãå®è¡æã«èšæž¬ãã+垯åå¹ ã䜿ã£ãŠãVRAM ãã£ã¢ã®ãã¹ã GPU ã®ãã£ã«åå²ãš CPU ã®åå²ã«æ¯ãåããŸãã+`COLI_CUDA_MOE_DOUBLE=1`ïŒ`COLI_CUDA_MOE_BATCH=1` ãšäœµçšïŒã¯ãçŸåšã®ã¬ã€ã€ãŒãèšç®ããŠããéã«+次ã®ã¬ã€ã€ãŒã®ãšãã¹ããŒãã»ããå šäœã 2 ã€ç®ã® VRAM ãã³ã¯ã«ããªãã§ããããVRAM ãè¶³ããªããã°+åäžãã³ã¯ã«ãã©ãŒã«ããã¯ããŸãã+CUDA ãã£ã¢ã¯ Pascal ãš Turing ã®ã«ãŒãïŒGTX 10 / RTX 20 ã·ãªãŒãºïŒã§ãåäœããããã«ãªããŸãã:+`CUDA_ARCH=portable-pre-ampere NO_TC=1` ã§ãã«ãããŠãã ããã++ã°ãªãŒãã£ãã³ãŒãã§ KV ã¹ããã㯠1 ã€ã§ããããŒã«åŒã³åºãã¯ãV4 ãã€ãã£ãã®ããã³ãããš DSML ã®+åŒã³åºããããã¯ã§ HTTP ã²ãŒããŠã§ã€ãéããŠæ¥ç¶ãããŠããŸããææ³ã¯ãµããŒããããŠããŸããã+[ãšã³ãžã³ããšã® API ãããªã¯ã¹](docs/api.md#tool-calling-support) ãåç §ããŠãã ããããã¬ãã£ãã¯ã¹+ãã§ãã¯ãã€ã³ãïŒã¡ã¢ãªäž + ãã£ã¹ã¯äžïŒã«ãããã·ã¹ãã ããã³ããã®ååããªãã£ã«åŸã¯ã+ãšãŒãžã§ã³ãã»ãã·ã§ã³ããã©ããŒã¢ããã®ã¿ãŒã³ãæ°ç§ã§å§ãŸããŸããRTX 5080 + NVMe 2 å°ã§ã®èšæž¬:+3324 ããŒã¯ã³ã®ããªãã£ã«ã 90 ç§ã8.3k ããŒã¯ã³ã®ååã¿ãŒã³ãååã®ã¿çŽ 4 åã以éã®+ã»ãã·ã§ã³/ã¿ãŒã³ã¯ 6ã9 ç§ã3k ã³ã³ããã¹ãã§ãã³ãŒãçŽ 1.6 tok/s â 詳现ã¯+[docs/deepseek-v4.md](docs/deepseek-v4.md) ãåç §ããŠãã ããã++**RAM ãäžããŠãã ããã** 43 à 256 åã®ã«ãŒãã£ã³ã°ãšãã¹ããŒãã¯ãã£ã¹ã¯äžã§çŽ 137 GiB ããã+1 ããŒã¯ã³ããã®ãã¡ 301 åã«è§Šããããããšãã¹ããŒããã£ãã·ã¥ã®ãããçã tok/s ãæ±ºããŸã â+`--ram` ã¯æã䟡å€ã®ããåäžã®èšå®é ç®ã§ãããå€ããã®ã¯é床ã ãã§ãåºåã¯æ±ºããŠå€ãããŸããã++**ææ©çãã©ããã¯ååšããŸããããªãã§ãã** DSpark ã®ãã«ã³ããã©ãã¿ãŒãšå®å šãª MTP ã¯ã©ã¡ãã+å®è£ ã»æ€èšŒæžã¿ã§ãããã©ããã¯ãã©ã¯ãŒããã¹ãç¯çŽã§ããŸãããããŒã¯ã³ãå€ããããšã¯æ±ºããŠãããŸããã+åçãããããŒã¯ã³ã¯ãã¹ãŠã¿ãŒã²ããèªèº«ã® argmax ã ããã§ããå®éã®ãã«ãã¿ãŒã³ãã£ããã§èšæž¬ãããšããã+åçç㯠15 åäž 1 åãš 24 åäž 10 åã§ããã®ãšã³ãžã³ã®ãªã«ã¬ã³ããªã¢ãã³ã·ã§ã³ç¶æ ã«ã€ããŠ+æ£åŽããããµãã£ãã¯ã¹ãåçããã³ã¹ãããã©ããã«ããç¯çŽãäžåããŸãã â 14 ããŒã¯ã³ã®åç 1 ã€ã«+495 ç§ããããŸããããã®ãã `V4_DRAFT` ãš `V4_MTP` ã¯ããã©ã«ãã§ `0` ãšããããé«éãªã¹ãã¬ãŒãžã§+åææŠãã人ã®ããã«ãã³ãŒãã¯èšæž¬å€ãšãšãã«æ®ããŠãããŸãã++CUDA ãã£ã¢ïŒãã«ããDLL ã®éžæãGPU ã®å¯Ÿå¿ç¯å²ïŒãç°å¢å€æ°ãªãã¡ã¬ã³ã¹ãæ§èœå€ã+ãã§ãã¯ãã€ã³ãã®æ€èšŒãçæãããå°ããªç¬ç«ãªã©ã¯ã«ã«ã€ããŠã¯+[docs/deepseek-v4.md](docs/deepseek-v4.md) ãåç §ããŠãã ããã++## ä»åŸã®å±æ++- **æšè«ã·ã¹ãã ã®ç ç©¶ããããããã¯ãã§ãã** çŸåšã®é局㯠LRU + åŠç¿ããããã³ã»ããã§ãã+ é²è¡äžã®äœæ¥ã¯ãã¢ãã«ãã©ãŒããããå§çž®ãé 眮ãã¹ã±ãžã¥ãŒãªã³ã°ãI/OãCPU/GPU ã«ãŒãã«ã+ ç°ç𮿷·åšã®ãªãŒããŒã©ãããKV ç¶æ ãã«ãŒãã£ã³ã°ãèæ ®ããææ©çãã³ãŒãã«ããããŸãã+ ç®çã¯ããŒããŠã§ã¢èŠä»¶ãšæçšããŒã¯ã³ãããã®ã³ã¹ããäžããããšã§ãããã¹ãŠã¯ãã®ãããžã§ã¯ãã®+ ããæ¹ã§åã蟌ãŸããŸã: ãšã³ãããŒãšã³ãã§èšæž¬ãããã¬ãã¥ãŒããããªãŒãã³ã«éçºãããŸãã+- **ããå€ãã®ãªãŒãã³ã¢ãã«ã** ãã£ã¢ãªã³ã°ã¢ã«ãŽãªãºã ã¯ã¢ãã«ã«äŸåããŸãããã«ãŒãã£ã³ã°+ ãšãã¹ããŒããæã€ MoE ã§ããã°ãã©ããåãæ¹æ³ã§ã¹ããŒãžã³ã°ã§ããŸããçŸåš 9 ã€ã®ãã¡ããªãŒã+ åäœããŠããŸãïŒGLM-5.2ãGLM-5.3-FlashãInklingãKimi K3ãDeepSeek V4 FlashãDeepSeek V4.1 Flashã+ Qwen3.8-Flash-NextãQwen3.6ãOLMoEïŒããããªããªãŒãã³ãŠã§ã€ãã®ãã¡ããªãŒ â åè£ã«ã¯+ **MiniMax** ãå«ãŸããŸã â ã¯ãæåã® 8 ã€ãšåãæ¹æ³ã§ãšã³ãžã³ãç²åŸããŸã:+ 誰ãããšã³ãããŒãšã³ãã§èšæž¬ãããšãã«ã§ãã++## ãããžã§ã¯ããžã®æ¯æŽ++colibrì ã¯ãRAM 25 GB ã® 12 ã³ã¢ã®ã©ãããããäžã§ã1 人ã®ãããžã§ã¯ããšããŠå§ãŸããŸããã+ä»ã§ã¯ãã®æ°å€ã¯ã宿©ãæã€ã³ãã¥ããã£ããéãŸã£ãŠããŸãã圹ã«ç«ã£ããšæããã:++- â ãªããžããªã«ã¹ã¿ãŒãä»ããŠå ±æããŠãã ããã+- ð ããªãã®ããŒããŠã§ã¢ã§ã®ãã³ãããŒã¯æ°å€ãæ·»ã㊠issue ãäœæããŠãã ãã â ããŒã¿ãã€ã³ãã¯+ äœããããã®ãããžã§ã¯ããåé²ãããŸãã+- ð¬ [Discord ã³ãã¥ããã£](https://discord.gg/RXV83nSZdk) ã«åå ããŠãå®éšãããŒããŠã§ã¢ã§ã®çµæã+ ç ç©¶ã®æ¹åæ§ã«ã€ããŠè°è«ããŠãã ããã+- ð¬ éçºã®ã¹ãã³ãµãŒãããŒããŠã§ã¢ã®å¯èŽã«ã€ããŠã¯ãGitHub ã® issue ãããé£çµ¡ãã ããã++## ãªããžããªæ§æ++```+Makefile ã«ãŒãã®ãã«ã/ãã§ãã¯ã®ãšã³ããªãã€ã³ã+c/+âââ colibri.c GLM-5.2 ãšã³ãžã³ (make glm)+âââ inkling.c Inkling ãšã³ãžã³ (make inkling)+âââ kimi_k3.c Kimi K3 ãšã³ãžã³ (make kimi_k3)+âââ deepseek_v4.c DeepSeek V4 Flash ãšã³ãžã³ (make deepseek-v4)+âââ qwen38.c Qwen3.8-Flash-Next ããã¹ããšã³ãžã³ (make qwen38)+âââ qwen36.c Qwen3.6 ãšã³ãžã³ (make qwen36)+âââ olmoe.c OLMoE ãšã³ãžã³ (make olmoe)+â+âââ st.h safetensors ã®ã€ã³ããã¯ã¹ãšç¯å²èªã¿èŸŒã¿+âââ quant.h æ£èŠã®ã³ã³ãããã³ãŒã+âââ expert_ffn.h MoE ãšã³ãžã³å ±éã®ã«ãŒããããšãã¹ããŒã FFN ã«ãŒãã«ïŒplanar int4ãã¬ã€ã€ãŒã©ã³ããŒïŒ+âââ tok.h, json.h ããŒã¯ãã€ã¶ãš JSON ããŒãµ+âââ compat.h Windows/macOS çšã·ã ïŒPOSIX åãäžãæã«ïŒ+âââ expert_store.h ã¹ããªãŒãã³ã°ãšãã¹ããŒããã£ãã·ã¥+âââ route_trace.h ã«ãŒãã£ã³ã°ã®ãã¬ã¡ããªãš .coli_usageïŒãšã³ãžã³éäŸåïŒ+âââ kv_prefix.h ã¿ãŒã³ããŸããã KV ãã¬ãã£ãã¯ã¹ã®åå©çš+â+âââ backend_cuda.* ãªãã·ã§ã³ã® CUDA ãã£ã¢ (CUDA=1)+âââ backend_metal.* ãªãã·ã§ã³ã® Metal ãã£ã¢ (METAL=1)+âââ backend_vulkan.* ãªãã·ã§ã³ã® Vulkan ãã£ã¢ (VULKAN=1)+â+âââ Makefile ãã«ããšããŒã«ã«ãã§ãã¯+âââ coli ãŠãŒã¶ãŒåã CLI+âââ openai_server.py OpenAI äºæ HTTP ã²ãŒããŠã§ã€+âââ resource_plan.py `coli plan` ãš `coli doctor` ã®èåŸã«ãã RAM/VRAM ãã©ã³ããŒ+âââ tools/ ãªãã©ã€ã³å€æããã£ã¯ã¹ãã£ããã³ãããŒã¯+âââ scripts/ é·æéå®è¡ã®å€æãã«ããŒ+âââ tests/ äŸåé¢ä¿ã®ãªã C ãš Python ã®ãã¹ã+web/ ãã©ãŠã¶ UIïŒçŽç²ãª OpenAI API ã¯ã©ã€ã¢ã³ãïŒ+desktop/ Web UI ãã©ãããã Tauri v2 ãã¹ã¯ãããã·ã§ã«+docker/ ã³ã³ããã€ã¡ãŒãž+docs/ ãªãã¡ã¬ã³ã¹ããã¥ã¡ã³ããå®éšãã¡ãã£ã¢+```++**ã¢ãã«ãã¡ããªãŒããšã« `.c` ã 1 ã€ãå ±æã®åäžãããã®äžã«ã** ãšã³ãžã³ã¯èªèº«ã®ã¢ãŒããã¯ãã£ã ãã+æã¡ããã以å€ã¯æã¡ãŸããã2 ã€ã®ãšã³ãžã³ãå ±ã«å¿ èŠãšãããã® â safetensors ãªãŒããŒãã³ã³ãã+ãã³ãŒããããŒã¯ãã€ã¶ããšãã¹ããŒããã£ãã·ã¥ â ã¯äž¡è ãã€ã³ã¯ã«ãŒããããããã«çœ®ãããããã+ä¿®æ£ã¯äžåºŠã«ãã¹ãŠã®ãšã³ãžã³ã«å±ããŸãããã®ã«ãŒã«ã¯é£Ÿãã§ã¯ãããŸãããããã§ç¹°ãè¿ãçºçãã+äžå ·åã¯ãããä»çµã¿ã 1 ã€ã®ãšã³ãžã³ã«ã ãå ¥ããå åŒãšã³ãžã³ã«å±ããªãã£ãã±ãŒã¹ãªã®ã§ãã++ãªããžããªã®ã«ãŒãããã¯ã`make`ã`make check`ã`make clean` ããšã³ãžã³ã® Makefile ã«å§è²ãããŸãã++## ãªããcolibrìããªã®ã++ããããªïŒã€ã¿ãªã¢èªã§ colibrìïŒã¯äœéãããæ°ã°ã©ã ã§ã空äžã«ãšã©ãŸãã1 æ¥ã«åãã®è±ã蚪ããŸãã+ãã®ãšã³ãžã³ã¯ã7,440 åãã©ã¡ãŒã¿ã®å·šäººãããããªã®é£äºéã§çããç¶ããŸã: RAM 25 GBã+CPU 12 ã³ã¢ããããŠãã£ã¹ã¯ãžã®ãã£ã·ãã®å¿èã§ãã++## è¬èŸ++colibrì ã¯ãšã³ãžã³ã«ãããããããåããç¥æ§ã¯èŽãç©ã§ããããã³ãã£ã¢çŽã®éã¿ããªãŒãã³ã«+å ¬éããŠããããŒã â **Z.ai**ïŒGLMïŒã**Moonshot AI**ïŒKimiïŒã**Alibaba Qwen**ã**MiniMax**ã+**Allen AI**ïŒOLMoEïŒâ ãããŠããã³ãããŒã¯ãåãããã€ã»ã¯ãããã¢ãã©ã¹ã®å®è¡ãåçŸãã+ããããéã£ãŠããããã¹ãŠã®ã³ã³ããªãã¥ãŒã¿ãŒã«æè¬ããŸãã+ãã®ãããžã§ã¯ãã¯ããªãŒãã³ãŠã§ã€ããäœãå¯èœã«ãããã®èšŒæã§ãã++ãã®ãããžã§ã¯ãã®ãšãã¹ããŒãé 眮ãå§çž®ãã«ãŒãã£ã³ã°ã®å®éšã¯ã以äžã®ãªãŒãã³ãªç ç©¶ãš+ã·ã¹ãã ã®ã¢ã€ãã¢ããšããã³ã¹ã«ãåºã¥ããŠããŸã:++- åºåãèæ ®ããããã¡ã€ã³åºæã®ãšãã¹ããŒãéèŠåºŠã«ã€ããŠã¯+ [REAP](https://github.com/CerebrasResearch/reap) ãš+ [EASY-EP](https://github.com/RUCAIBox/EASYEP)ã+- é¡äŒŒåºŠã«åºã¥ããšãã¹ããŒãã®åã«ãŒãã£ã³ã°ã«ã€ããŠã¯ [SERE](https://github.com/JL-Cheng/SERE)ã+ ãã£ãã·ã¥å±ææ§ãèæ ®ããã«ãŒã¿ãŒã®ãã¡ã€ã³ãã¥ãŒãã³ã°ã«ã€ããŠã¯+ [ReMoE](https://github.com/BUAA-OSCAR/ReMoE)ã+- ã«ãŒãã£ã³ã°ã«å°ããããšãã¹ããŒãã®ããŒãžãšå§çž®ã«ã€ããŠã¯+ [MC-SMoE](https://github.com/UNITES-Lab/MC-SMoE)ã+- å ±æãšãã¹ããŒãåºåºãšäœã©ã³ã¯ã®ãšãã¹ããŒãå·®åã«ã€ããŠã¯+ [MoBE](https://github.com/inclusionAI/MoBE) ãš+ [D²-MoE](https://github.com/lliai/D2MoE)ã+- CPU/GPU ãã€ããªããã®ãšãã¹ããŒãã¹ã±ãžã¥ãŒãªã³ã°ã«ã€ããŠã¯+ [HybriMoE](https://github.com/PKU-SEC-Lab/HybriMoE)ããšãã¹ããŒãéä¿¡ãšèšç®ã®ãªãŒããŒã©ããã«ã€ããŠã¯+ [ScMoE](https://arxiv.org/abs/2404.05019)ã忣ãªã³ããã³ãã®ãšãã¹ããŒãããŒãã«ã€ããŠã¯+ [OD-MoE](https://arxiv.org/abs/2512.03927)ã+- æ¯èŒãåçŸå¯èœã«ããŠãããªãŒãã³ãªæšè«ã·ã¹ãã ãšãšãã¹ããŒããªãããŒãã®åãçµã¿ã«ã€ããŠã¯+ [vLLM](https://github.com/vllm-project/vllm)ã+ [llama.cpp](https://github.com/ggml-org/llama.cpp)ã+ [kTransformers](https://github.com/kvcache-ai/ktransformers)ã++ãšã³ãžã³ã¯ã¢ã€ãã¢ã ãã§ãªããå ·äœçãªãšã³ãžãã¢ãªã³ã°ã®ææã®äžã«ãæãç«ã£ãŠããŸãã以äžã¯ãããã+çŸåšããªãŒå ã§äœ¿ãããŠããããåå®è£ ãããŠããŸã:++- [safetensors](https://github.com/huggingface/safetensors) â ãã¹ãŠã®ãšã³ãžã³ãèªãã³ã³ãã+ ïŒ`c/st.h`ïŒãfp8 ãš I64 ã® dtype ãå«ã¿ãŸãã+- [tiktoken](https://github.com/openai/tiktoken) â `c/tok.h` ã¯ãã® `byte_pair_encode` ã+ æ£ç¢ºã«åå®è£ ããŠãããé£çµããçµæã®èªåœ ID ãæãå°ãã飿¥ãã¢ãããŒãžããããã+ tiktoken ç±æ¥ã®èªåœã«ã¯ããŒãžãªã¹ããäžèŠã§ãã+- [llama.cpp](https://github.com/ggml-org/llama.cpp) â `c/grammar.h` ã® GBNF ææ³ãµãã»ããã¯+ ãã®æ§æãšã¹ã¿ãã¯éåã«ãã PDA ã«åŸã£ãŠãããMetal ã®çµè·¯ã¯ãã®+ `newBufferWithBytesNoCopy` ã«ããåžžé§ãã¯ããã¯ãåçšããŠããŸãã+- [vLLM](https://github.com/vllm-project/vllm) â ãšã³ãžã³ãäœçœ®ããšã«äžèŽãããŠããåºå+ ã»ãã³ãã£ã¯ã¹ã®ãªãã¡ã¬ã³ã¹ïŒäŸ: æçµãã«ã ã LM ãããã«å¯ŸããŠã©ãã«å ¥ããïŒã+- [transformers](https://github.com/huggingface/transformers) â ãªã©ã¯ã«:+ CI ã¯ã©ã³ãã åæåã¢ãã«ãããã«å¯ŸããŠããŒã¯ã³åäœã§åçŸããŸãã+- [DietGPU](https://github.com/facebookresearch/dietgpu) â å®éšçãªå§çž®ãšãã¹ããŒããã£ã¢+ ïŒ`COLI_ANS`ïŒã®èåŸã«ãã GPU ANS ã³ãŒããã¯ã+- [rocWMMA](https://github.com/ROCm/rocWMMA) â HIP ããã¯ãšã³ã㯠CUDA ã®+ `nvcuda::wmma` ã® fragment/mma_sync API ãããã«ãããã³ã°ããŠããïŒ`c/backend_gpu_compat.h`ïŒã+ ããã«ãã£ãŠ 1 ã€ã® .cu ãœãŒã¹ãäž¡ãã³ããŒåãã«ã³ã³ãã€ã«ã§ããŸãã++## ã©ã€ã»ã³ã¹++Apache 2.0ãGLM-5.2 ã®éã¿ã¯ Z.ai ã«ãã MIT ã©ã€ã»ã³ã¹ã§å ¬éãããŠããŸããdiff --git a/README.md b/README.mdindex 3bfa7bcf..0f8688a8 100644--- a/README.md+++ b/README.md@@ -10,7 +10,7 @@<p align="center"><a href="https://justvugg.github.io/colibri"><b>Website</b></a> ·<a href="https://discord.gg/RXV83nSZdk"><b>Discord</b></a> ·- English · <a href="README.zh-CN.md">ç®äœäžæ</a> · <a href="README.zh-TW.md">ç¹é«äžæ</a> · <a href="README.it.md">Italiano</a>+ English · <a href="README.zh-CN.md">ç®äœäžæ</a> · <a href="README.zh-TW.md">ç¹é«äžæ</a> · <a href="README.it.md">Italiano</a> · <a href="README.ja.md">æ¥æ¬èª</a></p>**Tiny engine, immense model.** Run **frontier MoE models â 744B to 2.8Tdiff --git a/README.zh-CN.md b/README.zh-CN.mdindex d13cbf2d..6dec0ca3 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -4,7 +4,7 @@<p align="center"><a href="https://discord.gg/RXV83nSZdk"><b>Discord</b></a> ·- <a href="README.md">English</a> · ç®äœäžæ · <a href="README.zh-TW.md">ç¹é«äžæ</a> · <a href="README.it.md">Italiano</a>+ <a href="README.md">English</a> · ç®äœäžæ · <a href="README.zh-TW.md">ç¹é«äžæ</a> · <a href="README.it.md">Italiano</a> · <a href="README.ja.md">æ¥æ¬èª</a></p>**å°å·§åŒæïŒåºå€§æš¡åã**åšæ¶èŽ¹çº§äžåŒæç¡¬ä»¶äžè¿è¡**åæ²¿ MoE æš¡åââä» 744B å°diff --git a/README.zh-TW.md b/README.zh-TW.mdindex cc085678..011df552 100644--- a/README.zh-TW.md+++ b/README.zh-TW.md@@ -4,7 +4,7 @@<p align="center"><a href="https://discord.gg/RXV83nSZdk"><b>Discord</b></a> ·- <a href="README.md">English</a> · <a href="README.zh-CN.md">ç®äœäžæ</a> · ç¹é«äžæ · <a href="README.it.md">Italiano</a>+ <a href="README.md">English</a> · <a href="README.zh-CN.md">ç®äœäžæ</a> · ç¹é«äžæ · <a href="README.it.md">Italiano</a> · <a href="README.ja.md">æ¥æ¬èª</a></p>**å°å·§åŒæïŒéŸå€§æš¡åã**åšæ¶è²»çŽèç°è³ªç¡¬é«äžå·è¡**åæ²¿ MoE æš¡åââåŸ 744B å°