Run frontier MoE models on hardware you already own β pure C, zero deps, experts streamed from disk. Tiny engine, immense model. π¦
C
docs(api): add "Connect a coding CLI or editor" recipe (#373)
KingIcyCreamProjects73e0e5ec82e9
diff --git a/docs/api.md b/docs/api.mdindex d3b09f16..cb29eb22 100644--- a/docs/api.md+++ b/docs/api.md@@ -49,6 +49,67 @@ errors before streaming headers are sent. `GET /health` exposesactive/queued/completed/rejected counters, and successful generation responsesinclude `x-colibri-queue-wait-ms`.+## Connect a coding CLI or editor++The API is OpenAI-compatible, so most coding CLIs and editor extensions work by+pointing them at Colibri as an *OpenAI-compatible* provider. Three settings:++- **Base URL** β `http://localhost:8000/v1`+- **Model** β `glm-5.2-colibri` (or whatever you pass to `--model-id`)+- **API key** β any non-empty string, e.g. `local`++Colibri needs **no** API key by default, but many clients refuse to start without+one β give them any dummy value. The key is only enforced if you set `COLI_API_KEY`.++Smoke-test the endpoint first (no key needed unless you set one):++```bash+curl http://127.0.0.1:8000/v1/chat/completions \+ -H 'Content-Type: application/json' \+ -d '{"model":"glm-5.2-colibri","messages":[{"role":"user","content":"hi"}]}'+```++**aider**++```bash+export OPENAI_API_BASE=http://localhost:8000/v1+export OPENAI_API_KEY=local+aider --model openai/glm-5.2-colibri # the openai/ prefix routes to OPENAI_API_BASE+```++**crush** β add a provider to `crush.json` (`~/.config/crush/crush.json`, or+`%USERPROFILE%\AppData\Local\crush\crush.json` on Windows):++```json+{+ "$schema": "https://charm.land/crush.json",+ "providers": {+ "colibri": {+ "name": "Colibri",+ "type": "openai-compat",+ "base_url": "http://localhost:8000/v1/",+ "api_key": "local",+ "models": [+ { "name": "GLM-5.2 (Colibri)", "id": "glm-5.2-colibri",+ "context_window": 131072, "default_max_tokens": 1024 }+ ]+ }+ }+}+```++The `"api_key": "local"` dummy is what satisfies clients that demand a key.+`context_window` is only the client's budget display β set it to whatever your+KV configuration actually allows.++**Continue, Cline / Roo, `llm`, the OpenAI SDKs, β¦** β set the provider's base+URL to `http://localhost:8000/v1`, the model to `glm-5.2-colibri`, and any dummy+key (`OPENAI_API_KEY` / `OPENAI_BASE_URL` for env-based tools).++> On the CPU-streaming path a large model decodes at roughly 1 tok/s, so+> interactive agent loops will feel slow β it connects and works, but the latency+> is very different from a hosted model.+## Isolated KV contexts`coli serve --kv-slots N` allocates up to 16 independent sequence contexts.