LLM inference in C/C++
C++ ggml
doc: document MCP stdio servers and CORS defaults in the server README [no release] [no ci] (#26847)
Pascal087f94d82e59
diff --git a/tools/server/README.md b/tools/server/README.mdindex 017981fd16..f1e1faceeb 100644--- a/tools/server/README.md+++ b/tools/server/README.md@@ -343,6 +343,58 @@ The server includes a set of built-in tools that enable the LLM to access the loTo use this feature, start the server with `--tools all`. You can also enable only specific tools by passing a comma-separated list: `--tools name1,name2,...`. Run `--help` for the full list of available tool names.+### MCP servers++Besides the built-in tools, the server can expose tools coming from MCP servers, added in [#26062](https://github.com/ggml-org/llama.cpp/pull/26062). Only the stdio transport is supported: such a server is a child process reading JSON-RPC messages on its stdin and writing replies on its stdout, so nothing has to be started or maintained outside `llama-server`.++Servers are declared in a Cursor-compatible JSON file:++```json+{+ "mcpServers": {+ "example": { "command": "/path/to/server", "args": [] }+ }+}+```++```sh+llama-server -m model.gguf --mcp-servers-config mcp.json+```++The same JSON can be passed inline with `--mcp-servers-json`. Each entry under `mcpServers` accepts:++| Key | Explanation |+| --- | ----------- |+| `command` | executable to spawn, required, entries without it are skipped |+| `args` | array of arguments |+| `env` | object merged over the parent environment |+| `cwd` | working directory of the child process |+| `timeout_ms` | per-tool-call timeout (default: 30000) |++Every server is spawned once at startup to list its tools, then stopped, and respawned on demand when one of its tools is called. Tools are exposed as `<server>_<tool>` alongside the built-in ones: they show up in the Web UI and in `GET /tools`, and the model calls them like any other tool. A name colliding with an already registered tool is skipped. This is independent of `--tools`, MCP servers can be the only tools available.++The child process runs with the same privileges as the server, so only declare commands you trust. As with `--tools`, `--cors-origins` then defaults to `localhost`.++Note: `--ui-mcp-proxy` is unrelated, it only lets the Web UI reach remote MCP servers from the browser.++Any server written against the [MCP specification](https://modelcontextprotocol.io) works as is, whether it uses an official SDK or not: the transport is one JSON-RPC message per line on stdio, so a script wrapping an existing program is a valid server too.++### CORS++By default the server reflects any `Origin` header back with credentials allowed. This matches the old, always-on `*` behavior and is fine as long as the server only exposes stateless, read-only endpoints.++Enabling `--tools` or `--agent` exposes file read/write over the API, so in that case `--cors-origins` defaults to `localhost` instead: only pages served from localhost can reach the server. Pass `--cors-origins` explicitly to override either default.++Recommended `--cors-origins` setting, depending on where the server runs:++| Deployment | Recommendation |+| ---------- | --------------- |+| Public | set an API key, put the server behind a reverse proxy, `--cors-origins` optional |+| Local network | set `--cors-origins` to your frontend's origin |+| Same machine | `--cors-origins localhost` (default once `--agent` is set) |++Related flags: `--cors-origins`, `--cors-methods`, `--cors-headers`, `--cors-credentials` / `--no-cors-credentials`. Background and rationale: [#25655](https://github.com/ggml-org/llama.cpp/pull/25655).+## Build`llama-server` is built alongside everything else from the root of the project