Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
C
docs: lead with the one-step setup; AI_SETUP.md and MCP_SERVER.md
JustVuggd85e58822601
diff --git a/README.it.md b/README.it.mdindex b22891d1..87b83903 100644--- a/README.it.md+++ b/README.it.md@@ -400,6 +400,31 @@ Ti servono due cose: **il programma** (poche centinaia di KB) e **il modello**(372 GB). Guida passo passo per tutte le piattaforme nella[Quick Start](docs/quickstart.md).+### In un solo passo++**Windows:** scarica il repository (**Code**, poi **Download ZIP**, oppure+`git clone`), scompattalo e fai doppio clic su **`START-HERE.bat`**.+**Linux e macOS:**++```bash+git clone https://github.com/JustVugg/colibri && cd colibri+./start-here.sh+```++Rileva RAM, disco e GPU, consiglia un modello adatto a questa macchina (Invio+lo accetta), compila il motore con Vulkan o CUDA quando la tua GPU può usarli+(oppure scarica quello già compilato), scarica il modello con ripresa+(interrompilo quando vuoi, rilancialo per continuare) e apre la dashboard nel+browser. Stampa anche gli URL base OpenAI e Anthropic per le altre app.+Rilancialo più tardi e colibri parte subito; `c/coli stop` lo ferma. Cosa fa+ogni passo: [quickstart.md](docs/quickstart.md#the-one-step-way).++Usi un assistente di programmazione AI? Chiedigli di installare colibri seguendo [docs/AI_SETUP.md](docs/AI_SETUP.md).+Gli assistenti che parlano il Model Context Protocol possono usare `coli mcp`+([MCP_SERVER.md](docs/MCP_SERVER.md)).++Segue il percorso manuale.+### 1. Procurati colibri**Scarica una release già compilata** — Linux, macOS e Windows, nessundiff --git a/README.ja.md b/README.ja.mdindex 3dfbd590..f5c1af1f 100644--- a/README.ja.md+++ b/README.ja.md@@ -368,6 +368,31 @@ n=64 の GLM-5.2 ではまだ捕捉されていません。必要なものは 2 つです: **プログラム**(数百 KB)と **モデル**(372 GB)。全プラットフォーム向けの手順は [クイックスタートガイド](docs/quickstart.md) にあります。+### ワンステップで始める++**Windows:** リポジトリをダウンロードし(**Code** から **Download ZIP**、または+`git clone`)、展開して **`START-HERE.bat`** をダブルクリックします。+**Linux と macOS:**++```bash+git clone https://github.com/JustVugg/colibri && cd colibri+./start-here.sh+```++RAM・ディスク・GPU を検出し、このマシンに収まるモデルを推奨し(Enter でそのまま+選択)、GPU が使える場合は Vulkan または CUDA でエンジンをビルドし(またはビルド済み+のものを取得し)、モデルを再開可能な形でダウンロードし(いつ中断しても、もう一度+実行すれば続きから再開します)、ブラウザでダッシュボードを開きます。他のアプリ向けに+OpenAI と Anthropic のベース URL も表示します。次回からは実行するだけで colibri が+すぐに起動し、`c/coli stop` で停止します。各ステップの内容:+[quickstart.md](docs/quickstart.md#the-one-step-way)。++AI コーディングアシスタントを使っていますか? [docs/AI_SETUP.md](docs/AI_SETUP.md) に従って colibri をセットアップするよう頼んでください。+Model Context Protocol に対応したアシスタントは `coli mcp` を使えます+([MCP_SERVER.md](docs/MCP_SERVER.md))。++以下は手動での手順です。+### 1. colibri を入手する**ビルド済みリリースをダウンロード** — Linux、macOS、Windows に対応し、コンパイラは不要です。diff --git a/README.md b/README.mdindex 828771fd..2c8d9e9f 100644--- a/README.md+++ b/README.md@@ -382,6 +382,31 @@ You need two things: **the program** (a few hundred KB) and **the model**(372 GB). Step-by-step for every platform in the[Quick Start guide](docs/quickstart.md).+### The one-step way++**Windows:** download the repository (**Code**, then **Download ZIP**, or+`git clone`), unzip it and double-click **`START-HERE.bat`**.+**Linux and macOS:**++```bash+git clone https://github.com/JustVugg/colibri && cd colibri+./start-here.sh+```++It finds your RAM, disk and GPU, recommends a model that fits this machine+(Enter takes it), builds the engine with Vulkan or CUDA when your GPU can use+it (or fetches the prebuilt one), downloads the model with resume (interrupt+it any time, run it again to continue), and opens the dashboard in your+browser. It also prints the OpenAI and Anthropic base URLs for other apps.+Run it again later and colibri starts straight away; `c/coli stop` stops it.+What each step does: [quickstart.md](docs/quickstart.md#the-one-step-way).++Use an AI coding assistant? Ask it to set up colibri following [docs/AI_SETUP.md](docs/AI_SETUP.md).+Assistants that speak the Model Context Protocol can use `coli mcp`+([MCP_SERVER.md](docs/MCP_SERVER.md)).++The manual path follows.+### 1. Get colibri**Download a prebuilt release** — Linux, macOS and Windows, no compiler needed.diff --git a/README.zh-CN.md b/README.zh-CN.mdindex 5f119952..3dfc4c15 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -329,6 +329,29 @@ context 点积改为累加整数乘积并只舍入一次,因此验证行中的你需要两样东西:**程序本体**(几百 KB)和**模型**(372 GB)。各平台的分步指引见 [Quick Start 指南](docs/quickstart.md)。+### 一步完成++**Windows:** 下载仓库(**Code** 中的 **Download ZIP**,或 `git clone`),解压后+双击 **`START-HERE.bat`**。+**Linux 和 macOS:**++```bash+git clone https://github.com/JustVugg/colibri && cd colibri+./start-here.sh+```++它会检测内存、磁盘和 GPU,推荐一个适合这台机器的模型(按回车即采用推荐),在 GPU+可用时用 Vulkan 或 CUDA 编译引擎(或获取预编译版本),以可续传的方式下载模型(随时+可以中断,再次运行即从断点继续),并在浏览器中打开仪表盘。它还会打印供其他应用使用的+OpenAI 和 Anthropic 基础 URL。之后再次运行,colibri 会直接启动;`c/coli stop` 可以+停止它。每一步做什么:[quickstart.md](docs/quickstart.md#the-one-step-way)。++在用 AI 编程助手?让它按照 [docs/AI_SETUP.md](docs/AI_SETUP.md) 来安装 colibri。+支持 Model Context Protocol 的助手可以使用 `coli mcp`+([MCP_SERVER.md](docs/MCP_SERVER.md))。++下面是手动安装的步骤。+### 1. 获取 colibri**下载预编译版本**——Linux、macOS 与 Windows 均已提供,无需编译器。从diff --git a/README.zh-TW.md b/README.zh-TW.mdindex 8b70e84e..f9bbf64f 100644--- a/README.zh-TW.md+++ b/README.zh-TW.md@@ -330,6 +330,29 @@ context 內積改為累加整數乘積並只捨入一次,因此驗證列中的你需要兩樣東西:**程式本體**(幾百 KB)與**模型**(372 GB)。各平台的逐步指引請見 [Quick Start 指南](docs/quickstart.md)。+### 一步完成++**Windows:** 下載儲存庫(**Code** 中的 **Download ZIP**,或 `git clone`),解壓縮後+雙擊 **`START-HERE.bat`**。+**Linux 與 macOS:**++```bash+git clone https://github.com/JustVugg/colibri && cd colibri+./start-here.sh+```++它會偵測記憶體、磁碟與 GPU,推薦一個適合這台機器的模型(按 Enter 即採用推薦),在+GPU 可用時以 Vulkan 或 CUDA 編譯引擎(或取得預先編譯的版本),以可續傳的方式下載模型+(隨時可以中斷,再次執行即從中斷處繼續),並在瀏覽器中開啟儀表板。它也會印出供其他+應用程式使用的 OpenAI 與 Anthropic 基礎 URL。之後再次執行,colibri 會直接啟動;+`c/coli stop` 可以停止它。每一步做什麼:[quickstart.md](docs/quickstart.md#the-one-step-way)。++在用 AI 程式設計助手嗎?請它依照 [docs/AI_SETUP.md](docs/AI_SETUP.md) 安裝 colibri。+支援 Model Context Protocol 的助手可以使用 `coli mcp`+([MCP_SERVER.md](docs/MCP_SERVER.md))。++以下是手動安裝的步驟。+### 1. 取得 colibri**下載預先建置的版本**——Linux、macOS 與 Windows 均已提供,不需要編譯器。從diff --git a/docs/AI_SETUP.md b/docs/AI_SETUP.mdnew file mode 100644index 00000000..578de051--- /dev/null+++ b/docs/AI_SETUP.md@@ -0,0 +1,244 @@+# Setting up colibri with an AI coding assistant++This page is written for an AI coding assistant that a user asked to install,+start, check or stop colibri on their machine. Every step is a command with a+machine-readable result. A person can follow it too; the one-step path for+people is `START-HERE.bat` (Windows) or `./start-here.sh` (Linux, macOS), see+[quickstart.md](quickstart.md).++If your client speaks the Model Context Protocol, the same operations are+available as tools: see [MCP_SERVER.md](MCP_SERVER.md).++## Ground rules++- **Ask before downloading.** Models are 19 GB to 1.6 TB. Show the user the+ recommended model, its size and where it will go, and get a yes.+- **Ask before installing system packages.** The setup never runs `sudo`,+ `apt`, `pacman` or `winget` itself. When it needs a package it prints the+ exact command; show it to the user and run it only with their consent.+- **Reruns are safe.** Every step is idempotent. If a command stops (network,+ timeout, Ctrl+C), run the same command again: it continues where it stopped.+- **Prefer the Linux disk on WSL.** Never put the model under `/mnt/c` (or any+ `/mnt/<drive>`): the engine streams the model from disk, and that path is+ many times slower. The default, `~/colibri-models`, is right.++## 0. Prerequisites++Python 3.10 or newer and a copy of the repository.++```bash+python3 --version # Windows: py -3 --version+git clone https://github.com/JustVugg/colibri # or use the folder the user already has+cd colibri+```++On Windows run the commands below from PowerShell or cmd in the repository+folder, with `py -3 c\coli` where this page writes `python3 c/coli`. If Python+is missing on Windows, `winget install -e --id Python.Python.3.12 --scope user`+installs it for the current user (ask first).++A C compiler is optional. With one (gcc and make on Linux, Xcode tools on+macOS, MSYS2 UCRT64 on Windows) the engine is built for this machine, with the+GPU when one is usable. Without one, the prebuilt engine from the GitHub+release is used, which runs on the CPU.++## 1. Look at the machine++```bash+python3 c/setup_hw.py --json # RAM, disk, CPU features, GPUs+python3 c/coli setup --list --json # every model against this machine+```++`setup_hw.py --json` reports `memory.total` and `memory.available` (bytes),+`disk.free_bytes` for `~/colibri-models` (pass another folder as an argument),+`cpu.features`, `gpu.vulkan` (the Vulkan device the engine would use, with its+`type`, `integrated` or `discrete`, and `budget_bytes` or `device_local_bytes`),+`gpu.vulkan_icd` (a driver manifest found under the home folder, typical on+WSL) and `gpu.nvidia` (from nvidia-smi).++`setup --list --json` returns one object per model:++| Field | Meaning |+|---|---|+| `id` | what `--model` takes |+| `download_gb` | size of the download |+| `ram_min_gb`, `ram_good_gb` | below the first it does not run; from the second it runs as measured |+| `dense_gb` | the part that always stays in RAM |+| `runs` | `from RAM` (small models) or `streams experts from the SSD` (large ones) |+| `status` | `good`, `tight`, `needs-ram`, `needs-disk` or `unsupported` |+| `fits`, `reason` | whether it can be installed here, and why |+| `recommended` | exactly one model is the default |++"Fits" means: the dense part, which always stays in RAM, plus a minimum+expert cache fit in RAM, and the download fits on the disk.++Tell the user the recommendation and the alternatives that fit, with their+sizes. Large models stream from the SSD and are much slower than the small+ones that run from RAM.++## 2. Install++Non-interactive, with the model the user chose:++```bash+python3 c/coli setup --yes --model qwen36-35b --no-start+```++What it does, in order: detects the hardware, picks the engine build (CUDA+for an NVIDIA card when the CUDA toolkit is installed and the engine has a+CUDA path, else Vulkan when a Vulkan GPU answered and the Vulkan headers and+`glslc` are installed, else the CPU), builds or fetches the engine, downloads+the model with resume, runs the planner on the files, and writes the run+configuration. It prints each step.++The download can take hours. If your command runner has a time limit, start+it detached and poll the status instead of waiting:++```bash+# Linux, macOS, WSL+nohup python3 c/coli setup --yes --model qwen36-35b --no-start > colibri-setup.log 2>&1 &+```++```powershell+# Windows+Start-Process -WindowStyle Hidden py -ArgumentList '-3','c\coli','setup','--yes','--model','qwen36-35b','--no-start' `+ -RedirectStandardOutput colibri-setup.log -RedirectStandardError colibri-setup.err+```++Then poll every minute or so:++```bash+python3 c/coli status --json+```++`install.phase` moves through `build`, `download`, `ready` (configured) and+`started` (server launched); `error` and `interrupted` end it early, and+`install.running` says whether the install process is still alive. During the+download `install.done` and `install.total` are bytes and `install.rate` is+bytes per second. On `error`, `install.message` says why; fix the cause and+rerun the same setup command.++### The setup options++| Option | Effect |+|---|---|+| `--yes`, `-y` | no questions: the recommendation and every default |+| `--model ID` | the model to install (ids from `--list`) |+| `--model-dir DIR` | use a model already on disk instead of downloading |+| `--dir DIR` | where models are downloaded (default `~/colibri-models`) |+| `--backend auto\|cpu\|vulkan\|cuda` | force the engine build; `--no-gpu` is `--backend cpu` |+| `--host H`, `--port N` | where the server listens (default `127.0.0.1:8000`) |+| `--no-start` | stop after writing the configuration |+| `--background` | start the server detached at the end |+| `--no-browser` | do not open a browser tab |+| `--reconfigure` | choose again even if a setup is complete |+| `--list`, `--all` | show the catalog against this machine and exit |+| `--via-windows auto\|yes\|no` | WSL only: download through Windows' `curl.exe` |+| `--no-verify` | skip the checksums of downloaded files |+| `--json` | machine-readable result (implies `--yes` and a background start) |++### When a GPU package is missing++If the machine has a GPU that the build cannot use yet, the setup continues+on the CPU and prints a line like:++```+ to use the GPU through VULKAN, first run: sudo apt install libvulkan-dev glslc mesa-vulkan-drivers+```++Show that command to the user. If they run it (or let you run it), run+`python3 c/coli setup --yes --no-start` again: the setup remembers that a GPU+was waiting for packages, sees that they are there, rebuilds the engine for+the GPU and rewrites the configuration. The download is not repeated.++On Windows the GPU build needs MSYS2 with its UCRT64 packages; the line names+them. The prebuilt Windows engine runs on the CPU.++### WSL++The setup measures the download speed from WSL and from Windows with a short+ranged request when the download is larger than a few GB. If Windows is much+faster (it can be: 63 KB/s against 3.3 MB/s was measured on one machine), it+downloads through Windows' `curl.exe` into the same Linux folder. Force it with+`--via-windows yes`, or disable it with `--via-windows no`.++A Vulkan driver installed by hand under the home folder (Mesa's Dozen driver is+the usual one on WSL) is found and written into the run configuration as+`VK_ICD_FILENAMES`.++## 3. Start++```bash+python3 c/coli start --background --no-browser+python3 c/coli status --json+```++Wait until `server.state` is `ready` (`loading` first; loading takes from+seconds to minutes depending on the model). Then give the user:++| Field in `status --json` | Use |+|---|---|+| `server.urls.browser` | the dashboard, for the user |+| `server.urls.openai_base_url` | `.../v1`, for OpenAI-compatible clients |+| `server.urls.anthropic_base_url` | for Anthropic-compatible clients (they append `/v1/messages`) |+| `server.model_id` | the `model` value API requests must send |+| `server.tokens_per_second` | speed of the last answer, prompt time included |++A quick check that it answers:++```bash+curl -s http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \+ -d '{"model": "<server.model_id>", "messages": [{"role": "user", "content": "Hello"}]}'+```++`python3 c/coli start` without `--background` runs in the foreground and opens+the browser: that is what a person wants, not a tool call.++If the configured port is taken by another program, the server moves to the+next free port and the configuration is updated; read the URLs from+`status`.++## 4. Check, read the logs, stop++```bash+python3 c/coli status # human-readable; --json for the fields above+python3 c/coli logs -n 50 # server log of a background start+python3 c/coli logs --install # log of a detached install+python3 c/coli stop # stops the configured server and its engine+```++`coli stop` without `--port` stops the server the setup configured. Rerunning+`python3 c/coli setup` later starts the configured server directly (in the+foreground; add `--background --no-browser` for a detached start), with no+download and no build.++## Files++Everything the setup writes lives in one folder: `~/.local/share/colibri` on+Linux, `~/Library/Application Support/colibri` on macOS,+`%LOCALAPPDATA%\colibri` on Windows. `COLI_SETUP_HOME` overrides it.++| File | Content |+|---|---|+| `setup.json` | the run configuration: model, engine, backend, environment, launcher arguments, URLs |+| `install-state.json` | phase and progress of the current or last install |+| `logs/serve.log`, `logs/install.log`, `logs/build.log` | logs |+| `runtime/` | a prebuilt release, when there is no compiler |++The model folder holds `.colibri-download.json`, which lists the files and+says when the download is complete. Partial files end in `.part`.++`HF_TOKEN` (or the token `hf auth login` saved) is used for gated repositories.+`HF_ENDPOINT` points the downloads at a mirror.++## Troubleshooting++| Symptom | What to do |+|---|---|+| `download stopped: ...` | rerun the same command; it resumes from the bytes on disk |+| `checksum mismatch` | the partial file was removed; rerun |+| `needs N GB free for the download` | pass `--dir` with a folder on a bigger disk |+| `the published release ... predates <model>` | there is no compiler and the last release cannot run that model: install the compiler (the message names the command) or pick another model |+| `Already set up: ...` but the user wants another model | `--reconfigure`, or `--model ID` |+| `server.state` stays `loading` | `coli logs -n 50`; large models take minutes to load |+| `plan warning: RAM budget cannot hold one expert slot` | the model does not fit in the free RAM: close programs or choose a smaller model |diff --git a/docs/MCP_SERVER.md b/docs/MCP_SERVER.mdnew file mode 100644index 00000000..f43facd9--- /dev/null+++ b/docs/MCP_SERVER.md@@ -0,0 +1,136 @@+# MCP server: install and run colibri from an AI assistant++colibri ships a [Model Context Protocol](https://modelcontextprotocol.io) server,+so an AI coding assistant can detect the hardware, recommend a model, install+it, and start, check and stop the server through tool calls instead of shell+commands. It is the same code `coli setup` runs (`c/setup_flow.py`), written+with the Python standard library only: nothing to install beyond Python 3.10.++The shell path for assistants without MCP is [AI_SETUP.md](AI_SETUP.md).++## Run it++The transport is stdio: the client starts the process and exchanges JSON-RPC+2.0 messages with it, one per line. Either form works:++```bash+python3 /path/to/colibri/c/coli mcp+python3 /path/to/colibri/c/mcp_server.py+```++After `make install` the launcher is on the PATH and `coli mcp` is enough.++## Client configuration++Most MCP clients take a JSON entry of this shape (the file and the top-level+key differ by client; see its documentation for where it goes). Use absolute+paths.++Linux, macOS, WSL:++```json+{+ "mcpServers": {+ "colibri": {+ "command": "python3",+ "args": ["/home/me/colibri/c/coli", "mcp"]+ }+ }+}+```++Windows:++```json+{+ "mcpServers": {+ "colibri": {+ "command": "py",+ "args": ["-3", "C:\\Users\\me\\colibri\\c\\coli", "mcp"]+ }+ }+}+```++Optional environment, in an `"env"` object next to `"args"`:++| Variable | Use |+|---|---|+| `COLI_SETUP_HOME` | where the run configuration, state and logs live (default: `~/.local/share/colibri`, `~/Library/Application Support/colibri`, `%LOCALAPPDATA%\colibri`) |+| `HF_TOKEN` | token for gated model repositories |+| `HF_ENDPOINT` | a mirror of the model hub |++A client that only takes a command line: `python3 /home/me/colibri/c/coli mcp`.++## Tools++| Tool | Arguments | Returns |+|---|---|---|+| `detect_hardware` | `target_dir` (optional) | RAM total and free, free disk in `target_dir` (default `~/colibri-models`) with the WSL `/mnt/<drive>` warning, CPU name and features, the Vulkan device the engine would use (type, memory budget), NVIDIA cards, and a `summary` text |+| `recommend_models` | `target_dir`, `include_unfit` | the models that fit, each with `download_gb`, `ram_min_gb`, `ram_good_gb`, `dense_gb`, `status`, `reason`, and the `recommended` id; `fits_means` explains what fitting means |+| `install` | `model`, `target_dir`, `model_dir`, `backend` (`auto`, `cpu`, `vulkan`, `cuda`), `port`, `start` (default true), `wait_seconds` (default 0) | starts a background `coli setup --yes` job and returns its pid and log; with `wait_seconds` it waits that long and reports progress |+| `start` | `open_browser` (default false), `wait_seconds` | starts the configured server in the background; returns its state and URLs |+| `stop` | none | stops the configured server and its engine |+| `status` | none | the configuration (model, engine, backend, environment), install progress, server state (`stopped`, `loading`, `ready`), the browser, OpenAI and Anthropic URLs, the API `model_id`, tok/s of the last answer |+| `logs` | `which` (`server`, `install`, `build`), `lines` | the tail of that log |++Results carry the data twice, as JSON text in `content` and as+`structuredContent`. A tool that cannot do what was asked returns a result+with `isError: true` and the reason (for example `start` before anything is+installed); malformed calls get a JSON-RPC error (`-32602` for an unknown tool+or a bad argument, `-32601` for an unknown method).++### The install job++Downloads take minutes to hours, longer than a tool call should block. `install`+therefore starts the setup as a separate process that keeps running if the+assistant or this server exits, and returns at once. Follow it with `status`:+`install.phase` is `build`, `download`, `ready`, `started`, or `error` with+`install.message`; during the download `install.done` and `install.total` are+bytes. A second `install` while one runs is refused. Calling `install` again+after an error or an interruption resumes: finished files are kept, partial+ones continue.++With `wait_seconds`, and a `progressToken` in the request's `_meta`, the call+sends `notifications/progress` while it waits (bytes during the download, a+step count before).++## A typical session++1. `detect_hardware`, then `recommend_models`.+2. Tell the user the recommended model, its download size and where it goes;+ ask before downloading.+3. `install` with the chosen `model`. Poll `status` until `install.phase` is+ `started` (or `ready` with `start: false`).+4. If `logs` with `which: "install"` shows a line `to use the GPU through ...,+ first run: <command>`, the GPU needs a package: show the command to the user,+ and after they install it call `install` again with the same `model`.+5. `status` until `server.state` is `ready`, then give the user+ `server.urls.browser`, and for other apps `server.urls.openai_base_url`,+ `server.urls.anthropic_base_url` and `server.model_id`.+6. `stop` when the user is done.++## Protocol details++- Protocol versions `2025-06-18`, `2025-03-26` and `2024-11-05`; the server+ answers with the client's version when it supports it, else the newest.+- Capabilities: `tools`. The tool list does not change at runtime.+- stdout carries protocol messages only; diagnostics go to stderr, prefixed+ `[colibri-mcp]`.+- Requests are handled concurrently: a long `install` or `start` wait does not+ block `status` or `ping`.+- When stdin closes, calls in flight get up to a minute to answer, then the+ process exits. Background installs and servers it started keep running.++## Tests++`c/tests/test_mcp_server.py` drives the protocol (initialize, tools/list,+tools/call, errors, progress) in process and through a child process over real+stdin and stdout. By hand:++```bash+printf '%s\n' \+ '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"t","version":"0"}}}' \+ '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"status","arguments":{}}}' \+ | python3 c/coli mcp+```diff --git a/docs/SETTINGS.md b/docs/SETTINGS.mdindex 5d5f0074..91c8fded 100644--- a/docs/SETTINGS.md+++ b/docs/SETTINGS.md@@ -28,7 +28,12 @@ Flags may also be given **after** the subcommand. Most flags map onto an engine| `chat` | Interactive REPL chat. || `serve` | Start the OpenAI-compatible HTTP server. || `web` | `serve`, then open the dashboard in the browser. |-| `stop` | Shut down a running `coli serve` and its engine. |+| `stop` | Shut down a running `coli serve` and its engine (default port: the one `coli setup` configured, else `8000`). |+| `setup` | One step: detect the hardware, recommend a model, build or fetch the engine (Vulkan/CUDA when usable), download with resume, write the run configuration, start. A rerun starts directly. See [AI_SETUP.md](AI_SETUP.md). |+| `start` | Start what `coli setup` configured (`--background`, `--no-browser`, `--json`). |+| `status` | What is set up, install progress, server state, URLs, API model id, tok/s of the last answer (`--json`). |+| `logs` | Tail of the background server log (`-n N`, `--install` for the install log). |+| `mcp` | Model Context Protocol server on stdio for AI coding assistants. See [MCP_SERVER.md](MCP_SERVER.md). || `cluster coordinator` / `cluster worker` | Run the local-cluster control plane, or an expert worker that serves experts to it. || `bench [tasks]` | Run benchmark tasks (`--limit`, `--data`). || `convert` | Convert an FP8 repo to a colibrì int4 snapshot. |@@ -73,6 +78,32 @@ Flags may also be given **after** the subcommand. Most flags map onto an engine| `--queue-timeout` | `$COLI_QUEUE_TIMEOUT` or `300` | Seconds a request may wait. || `--kv-slots` | `$COLI_KV_SLOTS` or `1` | Independent KV conversation slots (→ `KV_SLOTS`). |+**`setup`** (no common flags: `--model` here is a catalog id, not a directory)++| Flag | Default | Meaning |+|---|---|---|+| `--yes`, `-y` | off | Non-interactive: the recommendation and every default. |+| `--model ID` | the recommendation | Catalog model to install (`--list` shows the ids). |+| `--model-dir DIR` | none | Use a model already on disk instead of downloading one. |+| `--dir DIR` | `~/colibri-models` | Folder for downloaded models. |+| `--backend` | `auto` | `auto`, `cpu`, `vulkan` or `cuda`: the engine build. `auto` takes CUDA, then Vulkan, then the CPU. |+| `--no-gpu` | off | Same as `--backend cpu`. |+| `--host`, `--port` | `127.0.0.1`, `8000` | Where the configured server listens. |+| `--no-start` | off | Stop after writing the run configuration. |+| `--background` | off | Start the server detached (log in the setup folder). |+| `--no-browser` | off | Do not open the browser. |+| `--reconfigure` | off | Choose again even if a setup is complete. |+| `--list`, `--all` | off | List the catalog against this machine and exit; `--all` also shows what does not fit. |+| `--via-windows` | `auto` | WSL only: download through Windows' `curl.exe` (`auto`: when it measures much faster). |+| `--no-verify` | off | Skip the checksums of downloaded files. |+| `--json` | off | Machine-readable result; implies `--yes` and a background start. |++The run configuration, state and logs live in `COLI_SETUP_HOME` (default+`~/.local/share/colibri`, `~/Library/Application Support/colibri`,+`%LOCALAPPDATA%\colibri`). `COLI_SETUP_CATALOG` names a JSON list of extra+catalog entries (a mirror or a private repository); `HF_ENDPOINT` and+`HF_TOKEN` are honoured by the downloads.+**`convert`**| Flag | Default | Meaning |diff --git a/docs/docs.json b/docs/docs.jsonindex fabfa78c..9ba215f3 100644--- a/docs/docs.json+++ b/docs/docs.json@@ -15,7 +15,7 @@"groups": [{"group": "Get started",- "pages": ["index", "quickstart"]+ "pages": ["index", "quickstart", "AI_SETUP", "MCP_SERVER"]},{"group": "Serve models",diff --git a/docs/quickstart.md b/docs/quickstart.mdindex 74f4597d..23fa8772 100644--- a/docs/quickstart.md+++ b/docs/quickstart.md@@ -9,6 +9,95 @@ stuck, `./coli doctor` (below) tells you exactly what's missing.> experts from disk instead of needing them all in RAM. The engine is a single> C program; Python is only used once, to prepare the model files.+Use an AI coding assistant? Ask it to set up colibri following [docs/AI_SETUP.md](AI_SETUP.md).++---++## The one-step way++**Windows:** download the repository (on GitHub: **Code**, then **Download ZIP**,+and unzip it; or `git clone`), then double-click **`START-HERE.bat`** in the+top folder.++**Linux and macOS:**++```bash+git clone https://github.com/JustVugg/colibri.git+cd colibri+./start-here.sh+```++Both run `coli setup`, which does the rest:++1. **Finds your hardware**: RAM, free disk where the model goes, CPU features,+ and GPUs (any Vulkan GPU with its memory, NVIDIA cards through nvidia-smi).+2. **Recommends a model that fits**, in a short numbered menu with download+ sizes; Enter takes the recommendation. "Fits" means the part of the model+ that always stays in RAM (the dense part) plus a minimum expert cache fit+ in your RAM, and the download fits on your disk.+3. **Gets the engine**: it builds it for your machine when a compiler is+ there, with Vulkan or CUDA when your GPU can use it, or downloads the+ prebuilt one (CPU) when there is no compiler. If a package is missing for+ your GPU it prints the exact command to install it and carries on with the+ CPU; nothing is installed system-wide without you. Run the setup again once+ you have installed it, and it rebuilds the engine for the GPU.+4. **Downloads the model** with progress and resume. Interrupt it whenever you+ like: running it again continues where it stopped.+5. **Starts colibri** and opens the dashboard in your browser, and prints the+ addresses other apps can use.++What you see on Linux (on Windows the same, in the window `START-HERE.bat` opens):++```+$ ./start-here.sh+colibri setup++Your machine+ CPU 13th Gen Intel(R) Core(TM) i7-1355U, 6 cores (12 threads), AVX2, AVX_VNNI+ RAM 27.3 GB (24.7 GB free now)+ Disk 787 GB free in /home/me/colibri-models+ GPU Intel(R) Iris(R) Xe Graphics via Vulkan 1.3 (integrated, shares RAM)+ System Ubuntu 24.04 LTS++Models that fit this machine+ (fits = the dense part, which always stays in RAM, plus a minimum expert cache fit in RAM, and the download fits on the disk)+ 1) Qwen3.6-35B-A3B 23 GB runs from RAM [recommended]+ 2) Qwen3-Coder-30B-A3B 19 GB runs from RAM+ 3) DeepSeek V4 Flash REAP 150B 85 GB streams from the SSD+ ...+Choose a model [Enter = 1]:++Engine: qwen36, VULKAN (Intel(R) Iris(R) Xe Graphics (integrated) through Vulkan)+ building: make qwen36 ARCH=native VK=1+Download: Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64 (23.1 GB); safe to interrupt, rerun to continue+ [##########..............] 41.0% 9.5 GB/23.1 GB 38.2 MB/s 6 min left+...+Starting colibri+ Browser: http://127.0.0.1:8000/+ OpenAI base URL: http://127.0.0.1:8000/v1+ Anthropic base URL: http://127.0.0.1:8000+ stop: press Ctrl+C here (or close this window)+```++**Later:** run `START-HERE.bat` or `./start-here.sh` again and colibri starts+straight away, with no second download or build. To stop it, press Ctrl+C in+its window, or run `c/coli stop` from another terminal (`c\coli.cmd stop` on+Windows). `c/coli status` shows what is installed, whether it runs, its+addresses and the speed of the last answer. To pick another model:+`./start-here.sh --reconfigure`.++**On WSL** keep the model on the Linux disk (the default, `~/colibri-models`),+never under `/mnt/c`. If WSL's own network is much slower than Windows' (it can+be: 63 KB/s against 3.3 MB/s was measured on one machine), the setup offers to+download through Windows' `curl.exe` into the same folder.++Useful options (`c/coli setup --help` lists them all): `--yes` takes every+default without asking, `--model ID` picks a model (`--list` shows the ids+against your machine), `--dir DIR` puts the models elsewhere, `--no-gpu` keeps+everything on the CPU, `--model-dir DIR` uses a model you already have.++The rest of this page is the same thing done by hand, step by step.+---## 0. What you need first (prerequisites)diff --git a/docs/windows.md b/docs/windows.mdindex 0c37afb8..ebf9e6c6 100644--- a/docs/windows.md+++ b/docs/windows.md@@ -4,10 +4,22 @@ A start-to-finish, reproducible path from a fresh Windows 11 machine to GLM-5.2---+## The one-step way++Download the repository (**Code**, then **Download ZIP**, or `git clone`), unzip+it and double-click **`START-HERE.bat`**. It checks for Python (and offers to+install it with winget), finds your hardware, recommends a model that fits,+gets the engine (built with MSYS2 when it is installed, with Vulkan when your+GPU can use it; otherwise the prebuilt CPU engine from the release), downloads+the model with resume and opens the dashboard. Running it again starts colibri+directly. The same from a terminal: `py -3 c\coli setup`. See+[quickstart.md](quickstart.md#the-one-step-way).+## If you downloaded a release archive, start hereThe archive contains **`coli.cmd`**: that is the program to run. Double-click it-for the quick start, or from cmd/PowerShell:+for the quick start (it offers `coli.cmd setup`, the one-step path above), or+from cmd/PowerShell:```coli.cmd chat --model D:\models\glm52_i4