Skip to content

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

Merge dev: document tool-calling, and keep dev's stop-sequence correction

JustVugg00a0ffc8a43c
diff --cc docs/api.md
index 52c7b605,4daf784b..6344f641
--- a/docs/api.md
+++ b/docs/api.md
@@@ -30,12 -41,23 +41,25 @@@ sequences. The extensio
The server is deliberately text-only and serves one generation at a time: the
744B model stays in one persistent process, so concurrent HTTP requests queue
-instead of loading duplicate model copies. Image/audio input, log probabilities,
-and token penalties return an explicit error
-rather than being silently ignored. The default bind address is localhost; set
-`COLI_API_KEY` before exposing the server beyond the machine.
+instead of loading duplicate model copies. Tool-calling **is** supported on this
+path — pass OpenAI `tools` and (optionally) `tool_choice`, mirroring the
- Anthropic endpoint below. Image/audio input, custom stop sequences, log
- probabilities, and token penalties return an explicit error rather than being
- silently ignored. The default bind address is localhost; set `COLI_API_KEY`
- before exposing the server beyond the machine.
++Anthropic endpoint below. Image/audio input, log probabilities, and token
++penalties return an explicit error rather than being silently ignored. The
++default bind address is localhost; set `COLI_API_KEY` before exposing the
++server beyond the machine.
+
+ When a reverse proxy or MagicDNS hostname preserves a public `Host` header,
+ trust that exact hostname with repeatable `--allowed-host` options. The
+ comma-separated `COLI_ALLOWED_HOSTS` environment variable is equivalent:
+
+ ```bash
+ COLI_ALLOWED_HOSTS=llm.example.com ./coli serve --model /nvme/glm52_i4
+ # or: ./coli serve --model /nvme/glm52_i4 --allowed-host llm.example.com
+ ```
+
+ Only configure hostnames or IP addresses you control; there is no wildcard.
+ This setting extends the DNS-rebinding allowlist and is independent of CORS and
+ API-key authentication.
Browser access from the Vite development server and Tauri local origins is
enabled by default. Repeat `--cors-origin https://your-ui.example` to allow