Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

gateway: Codex talks Responses to magpie over a WebSocket, turns after the first as a delta (#4058)

yetoneffa07395eb27
diff --git a/docs/reference.md b/docs/reference.md
index cacb1bc5b..6bcab60bc 100644
--- a/docs/reference.md
+++ b/docs/reference.md
@@ -1973,7 +1973,7 @@ It exposes:
| Path | API |
| ------------------------ | -------------------------- |
| `/v1/chat/completions` | OpenAI chat completions |
-| `/v1/responses` | OpenAI Responses (HTTP, streamed as SSE; a WebSocket upgrade is answered 426) |
+| `/v1/responses` | OpenAI Responses (HTTP, streamed as SSE, or a WebSocket as Codex speaks it) |
| `/v1/messages` | Anthropic Messages |
| `/v1/messages/count_tokens` | Anthropic token counting |
| `/v1beta/models/{model}:generateContent` | Google Gemini (also `:streamGenerateContent`, `:countTokens`) |
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.md
index f221f90b1..990886d3b 100644
--- a/docs/subsystems/gateway-routing.md
+++ b/docs/subsystems/gateway-routing.md
@@ -29,7 +29,7 @@ backfilled; see [Gateway Session Aggregation](gateway-sessions.md).
| Address and caller | `DefaultAddr`. The local token `magpie` and the per-agent `magpie-<agent>` (`TokenFor`). `agentOf` tells which agent asked, first by token, then by User-Agent. A Codex turn that names itself Claude Code is Codex's: OpenAI's Codex plugin for Claude Code (/codex:rescue, /codex:review) runs `codex app-server` with clientInfo "Claude Code", which Codex sends as its originator and User-Agent, and only Codex's turn headers (`x-codex-window-id`, `x-codex-turn-metadata`) tell it apart from Claude Code's own requests (`codexForClaudeCode`); an app that runs Codex as its own engine under its own name (Lody) stays that app. Claude Science, whose User-Agent is the Anthropic SDK's, is told by its client's `X-Anthropic-Surface: operon` (`isClaudeScience`). It is listed every model as `claude-magpie-<number>` (`scienceModels`; its picker keeps only a `claude-` id whose name isn't a lower-case slug), and a Claude id magpie doesn't serve (its default before a pick, its haiku side tasks) goes to the model its last turn with tools was on, else the first it is listed (`scienceTurn`, kept in `claude-science.model`). | [`gateway.go`](../../internal/gateway/gateway.go) |
| WSL under NAT | On Windows, while the gateway listens on loopback alone, `serveWSL` also serves it, in plain HTTP, at each address Windows has on a WSL network (`WSLNets`: the `vEthernet (WSL…)` adapters, looked for every 10s, since WSL's VM brings its adapter up and down). That is where a distro in NAT networking reaches Windows, and what `place` writes into its agents (zaidu259434 on X: Codex in WSL failed with sharing off). `lanGuard` takes a request there as this computer's (`fromWSL`) only when it arrives at that address, from inside that network, with the WSL key: `WSLKey`, a `wsl-` secret in magpie's folder (`wsl-key`, 0600), optionally followed by `-<agent>`. It becomes a loopback request carrying `magpie` or `magpie-<agent>` (`asFromLoopback`). Anything else there, a Docker Desktop container on the same network included, is judged as another machine's. Windows' firewall still decides whether WSL gets in: WSL's network is a Public one. | [`wsl.go`](../../internal/gateway/wsl.go) |
| TLS | `--tls-cert`/`--tls-key` (`MAGPIE_TLS_CERT`, `MAGPIE_TLS_KEY`) make `ListenAndServe`, `Relisten` (both through `Server.listen`) and `magpie web`'s page (`ServeTLS`) wrap the listener in `tlsListener`. A connection from another computer is `tls.Server` at once, so its plain HTTP gets net/http's 400. One from this computer (`fromThisComputer`: loopback, or a peer address equal to the local one) is sniffed on its own goroutine, 10s at most (`sniffWait`), and 0x16 makes it TLS, so `URL()`, the agents, MCP servers, `ServedBy` and the healthcheck keep `http://127.0.0.1`. `certFiles.get` reloads the pair when either file's mtime or size changes; a pair that doesn't load keeps the one served (logged once per change). `tlsConfig` refuses half a pair or files that don't load before binding. HTTP/1.1 only over TLS (hijack, Codex's websocket). `Scheme()` gives the scheme of `LANURLs`, a scheme-less `MAGPIE_PUBLIC_URL` and the page's network links. Host and Origin checks (`foreignPage`, `rebound`) don't depend on the scheme. | [`tls.go`](../../internal/gateway/tls.go), [`lan.go`](../../internal/gateway/lan.go) |
-| HTTP surface | `Handler` routes `/v1/chat/completions`, `/v1/responses`, `/v1/messages` (each also without `/v1`), `/v1/models`, Gemini's `/v1beta/models/…`, the Codex backend path, `/v1/codex/models` (Codex's catalog shape for a Codex elsewhere with a gateway key, `codexCatalog`, kept to the key's models by `keyAllowed`; #1281), Ollama's `GET /api/version`, `GET /api/tags` and `POST /api/show` (`ollama.go`, for a client that takes a custom endpoint only as an Ollama server: Visual Studio's and VS Code's Copilot, wani on Discord; the list is `catalogFor`'s, so a key's models alone, `/api/show` tells `capabilities` (`completion`, `tools`, `vision` from `Images`, `thinking` from `Thinks`) and `model_info` with `general.architecture` `magpie`, `general.basename`, `magpie.context_length` (32768, VS Code's own default, when the window isn't known, since VS Code would take a 0) and the `general.parameter_count` and `magpie.embedding_length` the ollama CLI before 0.7 reads unchecked; a name ending `:latest`, as the ollama CLI asks, is the bare id unless a model is really called that (`ollamaName`); the client then chats on `/v1/chat/completions`, or on Ollama's own `POST /api/chat` and `POST /api/generate` (`ollama_chat.go`, Zed's Ollama provider, super_lalalalala on Discord), which `ollamaChatBody` turns into a `/v1/chat/completions` body handed to the same `handle(provider.Chat)` on a clone of the request, so they are routed, held to a key's models and counted as it is: `messages` with `images` (base64, its type sniffed, as data-URL parts; a tool message's in a user message after it), an assistant's `thinking` as `reasoning_content` and its `tool_calls` given ids where Ollama sent none, a tool result matched to its call by `tool_call_id`, else `tool_name`, else in order (one with no call is a 400), `tools`, `format` (`json` or a schema), `think` (a level is `reasoning_effort`), and `options`' `temperature`, `top_p`, `seed`, `stop`, the two penalties and a positive `num_predict` as `max_tokens` (`num_ctx`, `top_k` and the rest of Ollama's own are dropped); `ollamaWriter` turns the reply into Ollama's NDJSON lines (streamed unless `stream:false`, then one object), tool calls with their `arguments` as an object and an `id`, and a last `done` line with `done_reason` and the token counts, and an error into `{"error":"…"}` with the status `/v1` would give; a request with no messages or prompt is answered `done_reason:"load"`, as Ollama does to warm a model), `/mcp/{name}`, `/mcp/magpie/web-search`, `/mcp/magpie/vision` and `/mcp/magpie/image` with its `/mcp/magpie/image/files/{name}` (magpie's own web search, image recognition and image generation as MCP servers, below) and the `/v1/magpie/*` status endpoints. `/v1/magpie/heap` (`heapProfile`, [`heap.go`](../../internal/gateway/heap.go), for `magpie heap`) is held tighter than the rest by `heapRefused` in the handler itself, so it holds where `Handler` is served without `lanGuard` too: only a loopback caller that isn't proxied (`local`), carries no browser mark (`fromBrowser`) and names a loopback `Host` (`loopbackHost`) gets the profile; a gateway key never lets anything else in. `/v1/magpie/cpu` (`cpuProfile`, [`cpu.go`](../../internal/gateway/cpu.go), for `magpie cpu`) records a CPU profile for `?seconds=` (30, from 5 to 120) and is held to the same callers by `cpuRefused`, which shares `profileRefused` with `heapRefused`; one profile records at a time (`cpuRecording`), a second is answered 409, and a caller that hangs up stops it. `/v1/magpie/goroutines` (`goroutineDump`, [`goroutines.go`](../../internal/gateway/goroutines.go), for `magpie goroutines`, #3362) answers every goroutine's stack and wait as text (`runtime/pprof`'s `goroutine` profile at debug 2) to the same callers, through `goroutinesRefused`, and is answered ahead of everything below (`dumpFirst`, around `Handler`'s wrappers and around `lanGuard` in `Front`), since it is asked for while magpie hangs: the allow list reads settings.json under its lock, and `holdCatalog` takes `provider.Hold`'s and `filememo`'s, so a stuck holder of any of them would hold the dump too; a caller `goroutinesRefused` lets in is one those let in anyway, and any other goes through them as before (`TestGoroutineDumpAnswersWhileRequestsWait`). `/v1/alpha/search` (also without `/v1`, and on the Codex backend path for a magpie model) is Codex's web.run search (`alphaSearch`, `codex_search.go`): a ChatGPT account's model goes to the ChatGPT backend's `/alpha/search` on the account's sign-in, the next account asked on 401, 403, 429 or 5xx; any other model's `search_query` is searched by `webSearch`, and when nothing answers the reason is a 200 with `output`, since Codex ends the turn on any failed web.run (anton on Discord). A GET on `/v1/responses` (a client opening Responses over a WebSocket) is answered 426, a plain one 405: Responses is relayed over HTTP only. Every route is wrapped in `counted`, `corsGuard`, `callerGuard`, `withCaller` and `keyLimited`; `/v1/messages`, `/messages` and `/v1/messages/count_tokens` in `keepsSignIn` too. `corsGuard` (#1051) serves CORS only to the origins listed in Settings › Network › Web pages (`settings.CORSOrigins`, cleaned to `scheme://host[:port]` by `settings.CleanOrigins`, which refuses a wildcard). A browser extension's own pages, popup and service worker are listed the same way by their origin, which is the browser's scheme and the extension's id: `chrome-extension://<id>` (Chromium, the id 32 letters a–p), `moz-extension://<uuid>` (Firefox, the Internal UUID about:debugging shows, not the add-on id) and `safari-web-extension://<uuid>` (Safari, which picks a new UUID on each launch, so a listing lasts until Safari restarts); the id must have that browser's shape, cased any way, and nothing else is taken — no bare scheme, no wildcard, no path (SkyAerope on Discord: an extension without host permissions is held to CORS like a page). A content script's fetch carries the page's origin, not the extension's. A listed extension is treated exactly as a listed page, below; one not listed passes as before (other schemes, below) with no CORS headers. For a listed origin it answers the preflight (204) and adds `Access-Control-Allow-Origin`, and every other request from it needs an enabled gateway key (`access.Authenticate`), even on loopback, or gets a 401. It takes the key from the caller `lanGuard` recorded too, since on the running server `lanGuard` has already put magpie's own token in the key's place. Any other web page is refused 403 on every route (`foreignPage`): on loopback a call needs no key and the body's Content-Type isn't checked, so a page the user visits could otherwise send a "simple" `text/plain` POST and spend their subscriptions blind. Only an `http`/`https` Origin is judged, and one on this computer (`localhost`, `*.localhost`, `127.0.0.0/8`, `[::1]`, any port) or the request's own origin (its host and port equal to `Host`, such as a page on the gateway's LAN address) passes, as does usemagpie.ai on `/v1/magpie/omarchy`. Other schemes (`app://`, `file://`, `tauri://`, `vscode-webview://`, `chrome-extension://`) come from desktop apps, webviews and extensions that don't enforce CORS, and pass keyless as before; `null` is not one of them, since any web page sends it from a sandboxed frame and an opened file sends it too, so a call with it needs an enabled gateway key as a listed origin's does, and is answered 403 without one (no CORS headers either way, so a frame or a local page still reads no reply). A page at a subdomain of localhost (`localhostSubdomain`, #2256) needs a key the same way: Chrome and Firefox resolve `*.localhost` to loopback themselves, but Safari and other WebKit views ask the system resolver, which before macOS 26 asks the network's DNS (WebKit bug 160504), so a resolver that answers serves such a page from anywhere. `tauri.localhost` and `wails.localhost` stay keyless on Windows only, where Tauri's and Wails' webviews serve themselves at those names; on macOS and Linux those apps use `tauri://` and `wails://`. A request with no Origin (every agent's) passes through with no CORS headers, as before. A page whose own name resolves to 127.0.0.1 (DNS rebinding) would count as same-origin here, so `lanGuard` refuses it first (`rebound`, 403): a loopback request carrying a browser's marks (`Origin`, `Sec-Fetch-Site`/`Sec-Fetch-Mode`, a `Mozilla/` user agent) whose `Host` isn't this computer's (an IP, `localhost` but not a subdomain of it, which WebKit asks the DNS for (#2256), a container runtime's name for the host such as `host.docker.internal`, `MAGPIE_PUBLIC_URL`'s or `MAGPIE_ADDR`'s host, the machine's own name) needs an enabled gateway key. Agents send none of those marks and pass under any name. Before any of that, `lanGuard` lets in one request without a key from anywhere: `POST /_magpie/claude-mcp/<token>` with the token of a Claude subscription run now live (`ownCallback`, see [the Claude subscription bridge](claude-subscription-bridge.md)), the run's MCP helper calling back to a `MAGPIE_ADDR` that isn't loopback (#1546). Next, before the key or `rebound` is looked at, the allow list (lc on Discord, magpie in Docker on a public host): with `MAGPIE_ALLOW_IPS` or Settings › Network › Allowed addresses (`settings.AllowIPs`, this computer's own, never brought by a sync or restore: see [sync and backup](sync-backup.md#what-stays-on-each-computer); the variable wins and the page then only shows it) naming IPs and CIDR ranges, a request whose connection (`RemoteAddr`, IPv4-mapped IPv6 unmapped, zone dropped) is outside them is refused 403 naming its address, even with an enabled gateway key (`refuseNotAllowed`). Loopback and this computer's own interface addresses always pass. A forwarding header is never taken for the address, so a proxy or tunnel on this computer passes (it arrives from loopback) and its clients are limited at the proxy. An entry that doesn't parse lets nothing in and is logged once; the list stays on. Settings' list is read with `settings.LoadChecked`: a settings.json that can't be read, or doesn't parse with no last good generation, is unknown, never "no list", so the list read last in this process stays in force (`lastAllow`), and with none read yet only this computer passes, the 403 saying settings.json couldn't be read; logged once. No file, or an empty one, is no list. The Settings page's saves read with `LoadChecked` too and refuse such a file, as `settings.Save` does. `magpie web`'s server is wrapped in the same check (`AllowGuard`), and its port hands the gateway's client routes (`ClientPath`) to `Server.Front`, the same lanGuard-over-Handler the gateway's listener serves, each marked by `FromPagePort` so `local` is false for it (`viaPagePort`; `lanGuard` refuses it 403 while magpie isn't shared, saying to call the gateway's own address, and asks a gateway key otherwise): a raw TCP forward or a proxy with no forwarding header reaches that port from loopback, so no request there counts as an agent here (`TestClientPathCoversHandler` fails when a route Handler adds is in neither `ClientPath` nor its list of routes that stay on the gateway's port). Settings refuses a list that would leave out the address the page saving it is open from (`KeepsOut`); the page names that address, which behind Docker or a proxy is the proxy's, and offers to add it to the refused list (`allowSays`, Mathresearcher on Discord). | [`gateway.go`](../../internal/gateway/gateway.go), [`allow.go`](../../internal/gateway/allow.go), [`internal/settings/allowips.go`](../../internal/settings/allowips.go), [`cors.go`](../../internal/gateway/cors.go), [`codex_catalog.go`](../../internal/gateway/codex_catalog.go), [`ollama.go`](../../internal/gateway/ollama.go), [`ollama_chat.go`](../../internal/gateway/ollama_chat.go), [`internal/settings/origins.go`](../../internal/settings/origins.go) |
+| HTTP surface | `Handler` routes `/v1/chat/completions`, `/v1/responses`, `/v1/messages` (each also without `/v1`), `/v1/models`, Gemini's `/v1beta/models/…`, the Codex backend path, `/v1/codex/models` (Codex's catalog shape for a Codex elsewhere with a gateway key, `codexCatalog`, kept to the key's models by `keyAllowed`; #1281), Ollama's `GET /api/version`, `GET /api/tags` and `POST /api/show` (`ollama.go`, for a client that takes a custom endpoint only as an Ollama server: Visual Studio's and VS Code's Copilot, wani on Discord; the list is `catalogFor`'s, so a key's models alone, `/api/show` tells `capabilities` (`completion`, `tools`, `vision` from `Images`, `thinking` from `Thinks`) and `model_info` with `general.architecture` `magpie`, `general.basename`, `magpie.context_length` (32768, VS Code's own default, when the window isn't known, since VS Code would take a 0) and the `general.parameter_count` and `magpie.embedding_length` the ollama CLI before 0.7 reads unchecked; a name ending `:latest`, as the ollama CLI asks, is the bare id unless a model is really called that (`ollamaName`); the client then chats on `/v1/chat/completions`, or on Ollama's own `POST /api/chat` and `POST /api/generate` (`ollama_chat.go`, Zed's Ollama provider, super_lalalalala on Discord), which `ollamaChatBody` turns into a `/v1/chat/completions` body handed to the same `handle(provider.Chat)` on a clone of the request, so they are routed, held to a key's models and counted as it is: `messages` with `images` (base64, its type sniffed, as data-URL parts; a tool message's in a user message after it), an assistant's `thinking` as `reasoning_content` and its `tool_calls` given ids where Ollama sent none, a tool result matched to its call by `tool_call_id`, else `tool_name`, else in order (one with no call is a 400), `tools`, `format` (`json` or a schema), `think` (a level is `reasoning_effort`), and `options`' `temperature`, `top_p`, `seed`, `stop`, the two penalties and a positive `num_predict` as `max_tokens` (`num_ctx`, `top_k` and the rest of Ollama's own are dropped); `ollamaWriter` turns the reply into Ollama's NDJSON lines (streamed unless `stream:false`, then one object), tool calls with their `arguments` as an object and an `id`, and a last `done` line with `done_reason` and the token counts, and an error into `{"error":"…"}` with the status `/v1` would give; a request with no messages or prompt is answered `done_reason:"load"`, as Ollama does to warm a model), `/mcp/{name}`, `/mcp/magpie/web-search`, `/mcp/magpie/vision` and `/mcp/magpie/image` with its `/mcp/magpie/image/files/{name}` (magpie's own web search, image recognition and image generation as MCP servers, below) and the `/v1/magpie/*` status endpoints. `/v1/magpie/heap` (`heapProfile`, [`heap.go`](../../internal/gateway/heap.go), for `magpie heap`) is held tighter than the rest by `heapRefused` in the handler itself, so it holds where `Handler` is served without `lanGuard` too: only a loopback caller that isn't proxied (`local`), carries no browser mark (`fromBrowser`) and names a loopback `Host` (`loopbackHost`) gets the profile; a gateway key never lets anything else in. `/v1/magpie/cpu` (`cpuProfile`, [`cpu.go`](../../internal/gateway/cpu.go), for `magpie cpu`) records a CPU profile for `?seconds=` (30, from 5 to 120) and is held to the same callers by `cpuRefused`, which shares `profileRefused` with `heapRefused`; one profile records at a time (`cpuRecording`), a second is answered 409, and a caller that hangs up stops it. `/v1/magpie/goroutines` (`goroutineDump`, [`goroutines.go`](../../internal/gateway/goroutines.go), for `magpie goroutines`, #3362) answers every goroutine's stack and wait as text (`runtime/pprof`'s `goroutine` profile at debug 2) to the same callers, through `goroutinesRefused`, and is answered ahead of everything below (`dumpFirst`, around `Handler`'s wrappers and around `lanGuard` in `Front`), since it is asked for while magpie hangs: the allow list reads settings.json under its lock, and `holdCatalog` takes `provider.Hold`'s and `filememo`'s, so a stuck holder of any of them would hold the dump too; a caller `goroutinesRefused` lets in is one those let in anyway, and any other goes through them as before (`TestGoroutineDumpAnswersWhileRequestsWait`). `/v1/alpha/search` (also without `/v1`, and on the Codex backend path for a magpie model) is Codex's web.run search (`alphaSearch`, `codex_search.go`): a ChatGPT account's model goes to the ChatGPT backend's `/alpha/search` on the account's sign-in, the next account asked on 401, 403, 429 or 5xx; any other model's `search_query` is searched by `webSearch`, and when nothing answers the reason is a 200 with `output`, since Codex ends the turn on any failed web.run (anton on Discord). A WebSocket upgrade on `/v1/responses`, `/responses` or the Codex backend's `/responses` is Responses over a WebSocket, as Codex speaks it (below, [A Responses WebSocket](#a-responses-websocket)); one on another Codex backend path is answered 426, and a plain GET on `/v1/responses` 405. Every route is wrapped in `counted`, `corsGuard`, `callerGuard`, `withCaller` and `keyLimited`; `/v1/messages`, `/messages` and `/v1/messages/count_tokens` in `keepsSignIn` too. `corsGuard` (#1051) serves CORS only to the origins listed in Settings › Network › Web pages (`settings.CORSOrigins`, cleaned to `scheme://host[:port]` by `settings.CleanOrigins`, which refuses a wildcard). A browser extension's own pages, popup and service worker are listed the same way by their origin, which is the browser's scheme and the extension's id: `chrome-extension://<id>` (Chromium, the id 32 letters a–p), `moz-extension://<uuid>` (Firefox, the Internal UUID about:debugging shows, not the add-on id) and `safari-web-extension://<uuid>` (Safari, which picks a new UUID on each launch, so a listing lasts until Safari restarts); the id must have that browser's shape, cased any way, and nothing else is taken — no bare scheme, no wildcard, no path (SkyAerope on Discord: an extension without host permissions is held to CORS like a page). A content script's fetch carries the page's origin, not the extension's. A listed extension is treated exactly as a listed page, below; one not listed passes as before (other schemes, below) with no CORS headers. For a listed origin it answers the preflight (204) and adds `Access-Control-Allow-Origin`, and every other request from it needs an enabled gateway key (`access.Authenticate`), even on loopback, or gets a 401. It takes the key from the caller `lanGuard` recorded too, since on the running server `lanGuard` has already put magpie's own token in the key's place. Any other web page is refused 403 on every route (`foreignPage`): on loopback a call needs no key and the body's Content-Type isn't checked, so a page the user visits could otherwise send a "simple" `text/plain` POST and spend their subscriptions blind. Only an `http`/`https` Origin is judged, and one on this computer (`localhost`, `*.localhost`, `127.0.0.0/8`, `[::1]`, any port) or the request's own origin (its host and port equal to `Host`, such as a page on the gateway's LAN address) passes, as does usemagpie.ai on `/v1/magpie/omarchy`. Other schemes (`app://`, `file://`, `tauri://`, `vscode-webview://`, `chrome-extension://`) come from desktop apps, webviews and extensions that don't enforce CORS, and pass keyless as before; `null` is not one of them, since any web page sends it from a sandboxed frame and an opened file sends it too, so a call with it needs an enabled gateway key as a listed origin's does, and is answered 403 without one (no CORS headers either way, so a frame or a local page still reads no reply). A page at a subdomain of localhost (`localhostSubdomain`, #2256) needs a key the same way: Chrome and Firefox resolve `*.localhost` to loopback themselves, but Safari and other WebKit views ask the system resolver, which before macOS 26 asks the network's DNS (WebKit bug 160504), so a resolver that answers serves such a page from anywhere. `tauri.localhost` and `wails.localhost` stay keyless on Windows only, where Tauri's and Wails' webviews serve themselves at those names; on macOS and Linux those apps use `tauri://` and `wails://`. A request with no Origin (every agent's) passes through with no CORS headers, as before. A page whose own name resolves to 127.0.0.1 (DNS rebinding) would count as same-origin here, so `lanGuard` refuses it first (`rebound`, 403): a loopback request carrying a browser's marks (`Origin`, `Sec-Fetch-Site`/`Sec-Fetch-Mode`, a `Mozilla/` user agent) whose `Host` isn't this computer's (an IP, `localhost` but not a subdomain of it, which WebKit asks the DNS for (#2256), a container runtime's name for the host such as `host.docker.internal`, `MAGPIE_PUBLIC_URL`'s or `MAGPIE_ADDR`'s host, the machine's own name) needs an enabled gateway key. Agents send none of those marks and pass under any name. Before any of that, `lanGuard` lets in one request without a key from anywhere: `POST /_magpie/claude-mcp/<token>` with the token of a Claude subscription run now live (`ownCallback`, see [the Claude subscription bridge](claude-subscription-bridge.md)), the run's MCP helper calling back to a `MAGPIE_ADDR` that isn't loopback (#1546). Next, before the key or `rebound` is looked at, the allow list (lc on Discord, magpie in Docker on a public host): with `MAGPIE_ALLOW_IPS` or Settings › Network › Allowed addresses (`settings.AllowIPs`, this computer's own, never brought by a sync or restore: see [sync and backup](sync-backup.md#what-stays-on-each-computer); the variable wins and the page then only shows it) naming IPs and CIDR ranges, a request whose connection (`RemoteAddr`, IPv4-mapped IPv6 unmapped, zone dropped) is outside them is refused 403 naming its address, even with an enabled gateway key (`refuseNotAllowed`). Loopback and this computer's own interface addresses always pass. A forwarding header is never taken for the address, so a proxy or tunnel on this computer passes (it arrives from loopback) and its clients are limited at the proxy. An entry that doesn't parse lets nothing in and is logged once; the list stays on. Settings' list is read with `settings.LoadChecked`: a settings.json that can't be read, or doesn't parse with no last good generation, is unknown, never "no list", so the list read last in this process stays in force (`lastAllow`), and with none read yet only this computer passes, the 403 saying settings.json couldn't be read; logged once. No file, or an empty one, is no list. The Settings page's saves read with `LoadChecked` too and refuse such a file, as `settings.Save` does. `magpie web`'s server is wrapped in the same check (`AllowGuard`), and its port hands the gateway's client routes (`ClientPath`) to `Server.Front`, the same lanGuard-over-Handler the gateway's listener serves, each marked by `FromPagePort` so `local` is false for it (`viaPagePort`; `lanGuard` refuses it 403 while magpie isn't shared, saying to call the gateway's own address, and asks a gateway key otherwise): a raw TCP forward or a proxy with no forwarding header reaches that port from loopback, so no request there counts as an agent here (`TestClientPathCoversHandler` fails when a route Handler adds is in neither `ClientPath` nor its list of routes that stay on the gateway's port). Settings refuses a list that would leave out the address the page saving it is open from (`KeepsOut`); the page names that address, which behind Docker or a proxy is the proxy's, and offers to add it to the refused list (`allowSays`, Mathresearcher on Discord). | [`gateway.go`](../../internal/gateway/gateway.go), [`allow.go`](../../internal/gateway/allow.go), [`internal/settings/allowips.go`](../../internal/settings/allowips.go), [`cors.go`](../../internal/gateway/cors.go), [`codex_catalog.go`](../../internal/gateway/codex_catalog.go), [`ollama.go`](../../internal/gateway/ollama.go), [`ollama_chat.go`](../../internal/gateway/ollama_chat.go), [`internal/settings/origins.go`](../../internal/settings/origins.go) |
| Model resolution | `serve` resolves the model the agent asked for: Claude tier stand-ins, a routing group (`provider.GroupFor`: its id, the model's id however a vendor spells it, else the one group whose name it is, unless a provider serves a model by that id; the user's own groups only while the `groups` plugin is on, [the plugin runtime](plugin-runtime.md#gateway-extension-points-p3-router-strategy)), auto stand-ins, the agent's own stand-in (`StandIn`: Codex's config.toml `model` when magpie is its provider — in a WSL distro, the distro's own, of the ones running now — for a name it sends that magpie doesn't serve, such as its auto-review's `codex-auto-review`), itself taken through `GroupFor` and auto stand-ins so a config naming a group bare reaches it, then `provider.Resolve`. A Codex memory call or auto-review (`x-openai-subagent: guardian`, or `codex-auto-review`) that still names no served model goes to the model Codex's last turn was answered on (`codexMemoryStandIn`, per agent and gateway key; a memory call first waits beside the turn it was started with, through `codexLeads`, see the side turns below), so a Codex on magpie through another provider table, a profile or another computer isn't answered 404 (Adam on Discord, `TestCodexAutoReviewReachesItsTurnsModel`). A bare model id several providers serve (MOMO on Discord: `glm-5.3` under opencode-go and a6api) is never refused: with found groups on (the default) `GroupFor` takes it to the group magpie found for it (`auto-<slug>`), its members in the Providers order, routed and failed over as any group, a conversation staying with the member that answered it while its cache is worth it (affinity); with them off, `Resolve` takes the first provider in the Providers order that exposes it, with no fallback. magpie writes only `provider/model` ids into agents' configs (`magpieModels`), so a bare id is one the user wrote. `/v1/models` lists bare ids only when the user asks for them ([Model ids without the provider](#model-ids-without-the-provider)). Every reply says who answered in `X-Magpie-Provider` and `X-Magpie-Model` (`noteMember`), and the usage log and the Routing page keep it. A model id with `~` and no slash that resolves as nothing else is the one served id (a provider's model or a group) whose slashes it spells as `~` (`unflat`, `provider.Unflat`; MiniMax Code's spelling, #1387), taken back in `validateModel` before key limits, middleware and usage see it; two ids spelled alike are neither. A Claude model id whose version is dashed (`relay/claude-opus-4-6-us`, with or without `:level` and `[1m]`) that is not served as spelled is the one served id it dashes (`provider.ClaudeUndashed`; the spelling magpie gives Claude Code, which reads a dotted version as Claude Opus 4), taken back in `validateModel` and `unprefixed` the same way. An unknown model gets a 404 naming why: a switched-off provider, a disabled group or an empty group. Otherwise it names up to three ids nearest to what was asked (`provider.Closest`: words of a model's display name count before words of its id, and a provider's id or name only alongside them, so "Zen-Space-Bunny" is told `opencode-zen-free/space-bunny-free`, MOMO on Discord), then the ways a model can be named, then every id (`unknownModel`; chat and embeddings). | [`gateway.go`](../../internal/gateway/gateway.go), [`internal/provider/group.go`](../../internal/provider/group.go), [`internal/provider/flat.go`](../../internal/provider/flat.go), [`internal/provider/claudedash.go`](../../internal/provider/claudedash.go) |
| A found group's two ids | A group magpie found is listed (`/v1/models`, the catalog, the lists written into agents, the pickers, the CLI and TUI) as `auto/<model>`: the model's id as a vendor spells it, dots kept and with no vendor prefix (`auto/claude-opus-5.5`; `provider.Group.Ref`, `spelledIn`), so a client that looks the part after the slash up on models.dev finds it (ZeroClover on X). The spelling is the one models.dev gives the model most (`catalog.Spelling`, its `normal` index: `glm-5.3-flash` whether the members serve `glm-5-3-flash`, Ark's dated id or `glm-5.3-flash`), so it no longer follows the order the providers were added in; for a model models.dev doesn't know, the first member's spelling it lists, else the first member's. The group's name comes from the same choice (`autoGroups`): the name models.dev gives that spelling (`catalog.NameOf`), else a vendor's own name for it, else the spelling itself; they were picked apart, a name of `glm-5-3-flash` beside `auto/glm-5.3-flash` (XYenon on Discord, `TestFoundGroupIDAndNameAreOneSpelling`). A found group the user changed keeps the name they saved. Every spelling it was listed by before is still the group (`CanonicalRef`), and `PublicRef` takes one to the spelling now. Its old id `group/auto-<slug>` is served forever. Inside magpie the group keeps that one id: `validateModel` takes `auto/<model>` (with `:level` or `[1m]`) back to it with `provider.CanonicalRef` before key limits, routing, affinity and the usage log see the request, and the request body is rewritten to it, so both ids count as one group. A gateway key naming either id, `group/*` or `auto/*` passes for both (`allows` in `key_models.go`). `SaveGroup` stores a member, rule or classifier picked as `auto/<model>` by the old id. An agent field magpie set to `group/auto-<slug>`, or to an `auto/<model>` spelt otherwise than the group is listed now, moves to `auto/<model>` on the next `SyncCatalog` (`agent.renameFoundGroups`, `TestFoundGroupMovedOffAnOldSpelling`), but only while its value is still what applied.json says magpie wrote; a value the user set or changed stays, and is still served. A pick of the group by its old id written now (`Agent.Apply`) is written as `auto/<model>`. An agent's hidden, picked and ordered models saved by the old id hold for the new one (`PublicRef` in `visible.go`), and a bare model id still reaches its found group (`bareTarget`). Claude Code is given a dotted Claude version dashed, so it may send `auto/claude-sonnet-4-5`: that is the same group, and `Reseat` still moves an agent off it once the group is gone (`samePick`). `auto` is no provider id: `Save` and `Rename` refuse it for a new provider, and a plugin's `auto` is `auto-plugin` (`PluginID`). A provider of the user's saved as `auto` before keeps its models: while it exists, `auto/<x>` is its model and found groups list as `group/auto-<slug>` (`autoRefsOn`). Hand-made groups stay `group/<id>`. | [`internal/provider/autoref.go`](../../internal/provider/autoref.go), [`key_models.go`](../../internal/gateway/key_models.go), [`internal/agent/autoref.go`](../../internal/agent/autoref.go) |
| Candidates | `plan` lists one provider's candidates: one per key or account on, then each `Fallback` model, with resting ones last (`restLast`). A fallback the user unchecked in its own provider's model list is skipped, as a group's member is (`provider.ResolveRoute`, #3911; `fallback_unpicked_test.go`), and so is it where fallbacks are read for careful accounts, native compaction, redaction and Codex's own turns. A fallback can be a routing group (`group/<id>`, 01huadalang on Discord): `planPaused` hands its members to `planLevel` as a request to the group would (`planFallbackGroup`), in the group's own routing and sink, its pause rules applied in `serve`, and its trace rows marked `Fallback` with the group in `Via`. Its turn rules (`ruleFor`, which may ask a classifier) are not applied: the plan is made before the primary is tried, and a classifier call on every request the primary answers costs too much. Each provider/model is tried once a round: a member that is the primary, an earlier fallback or another fallback group's is left out where it came second, so a group holding this provider is no loop (groups have no fallbacks of their own, and a member's are not followed). A group that is gone is passed over. Every place that reads fallbacks (careful accounts, native compaction, redaction, Codex's own turns, Claude's review) reads a group's members through `provider.FallbackModels`. `fallback_group_test.go`. `planGroup` and `planLevel` weigh a group's members together. A nested group plans by its own routing (#576); one left on the default (Smart) in a group that pools (Smart, In turn, Least used, Weekly pace, Free first, Speed first, a plugin's) takes the outer group's, as a provider left on the default does (`provider.Group.Within`, set on `Member.Via`, so the trace's `Subs` and the Routing page read the routing that applies, #3847). In a group In order, By weight or Manual it stays Smart. `TestFreeFirstReachesAGroupInItLeftOnTheDefault`. So does a member whose provider routes other than Smart (`ownRouting`): weighed as one by its first not resting (`firstAwake`), tried whole in its own order; `leastHeld` leaves such a group's order alone too. A provider on Smart, or routed as the group is, is pooled with the group's other keys and accounts. `group_own_routing_test.go`. A member's own fallbacks are not the group's. | [`fallback.go`](../../internal/gateway/fallback.go) |
@@ -415,9 +415,10 @@ request alone. `Server.gatewayTrace` in
A remote provider serves the conversation, token-counting, System One,
embeddings, rerank, speech-to-text, image generation/edit and video
creation/status/content APIs. Gemini clients enter through the local Gemini adapter and follow the
-same model routing as other conversation clients. Responses uses HTTP/SSE;
-its WebSocket route still answers 426 (magpie's own leg to the ChatGPT
-backend is a WebSocket for a streamed turn: see
+same model routing as other conversation clients. Responses is served
+over HTTP/SSE and over a WebSocket ([A Responses
+WebSocket](#a-responses-websocket)); magpie's own leg to the ChatGPT
+backend is a WebSocket for a streamed turn (see
[Providers and accounts](providers-accounts.md)). Discovery and older-peer behavior
are described in [Providers and accounts](providers-accounts.md#remote-magpie).
`GET /v1/magpie/limit` (`keyLimit` in
@@ -1185,6 +1186,69 @@ the row of the entry that id reaches, with its label, under the bare id.
`internal/gui/tests/bare-ids.test.cjs`.
- A URL magpie downloads but didn't choose itself is downloaded from public addresses only, wherever the request came from ([`image_fetch_guard.go`](../../internal/gateway/image_fetch_guard.go)). That is an image a caller names (an `/v1/images/edits` or `/v1/images/generations` reference image, for an images API that takes files, Gemini's own API or Code Assist: `fetchPicture` in [`draw.go`](../../internal/gateway/draw.go), `publicClient`), a page `web_fetch` reads (`webFetchClient`), and an image or video a vendor answers by URL, which what the model was asked to draw or do can steer (the inline download for a `b64_json` caller in `draw.go` and `/v1/videos/{id}/content` in [`video.go`](../../internal/gateway/video.go), `vendorClient`; generate_image's own download in [`internal/imagemcp`](../../internal/imagemcp/imagemcp.go), `PublicClient`). Every address the host resolves to must be public (`publicAddr`, which is [`internal/netguard`](../../internal/netguard/netguard.go)'s `Public`, the rule a provider's icon uses too: not loopback, RFC 1918, fc00::/7, link-local with 169.254.169.254, CGNAT 100.64/10, unspecified, multicast, documentation, reserved, benchmarking, Teredo, nor an IPv4-mapped, NAT64 or 6to4 form of one; an AAAA record Loon synthesizes for an A answer in the same reply, `fd27:712::/96` with that IPv4 address inside, is left out and the A answer judged in its place, `dropLoon`), and the request goes to the address checked, so a name that resolves elsewhere the second time changes nothing; a direct connection is checked again on the address dialled (`guardDialer`). Through a proxy the proxy is sent that address, never the name (an `https://` CONNECT or SOCKS keeps the name for TLS and Host; a plain `http://` URL through an HTTP proxy is asked for by the address in Host too). A fake-IP resolver's answer is the one shape let past that (#2593; ggbdpq, with h4rk8s's and TomXPRIME's reports): Clash's TUN with fake-ip, Clash Verge, Surge's enhanced mode and Loon answer every name in 198.18.0.0/15 (`netguard.FakeIP`), so with one on every download was refused as a private address. A name (never an address literal) whose every answer is in that range is fetched: with no proxy (pure TUN) its fake address is pinned and dialled, the dial guard letting that one address in, and the TUN carries it; through a proxy on this computer (a loopback address or `localhost`, `onThisComputer`) the proxy is given the name, never the fake address, which a real proxy can't route (h4rk8s: a CONNECT by the name went through where one by its 198.18 address failed), and a plain `http://` request's Host is the checked name, never the caller's own; through a proxy elsewhere it is refused. An answer with any other address beside the fake ones (private or public) is judged as any, so a fake one beside 192.168.x is refused. The trust this asks for is the fake-IP resolver's: which host the name stands for, and so where the TUN or the proxy connects, is decided where magpie can't see, and a name that really points at the user's network is reached if that resolver's rules send it there. A provider's icon was already fetched this way (`publicIP` in [`internal/provider/icon.go`](../../internal/provider/icon.go), #252), and shares the rule. `TestFakeIPNameGoesToALocalProxyByName`, `TestFakeIPNameDialledThroughTheTUN`, `TestFakeIPRefusesTheRest` in [`fakeip_test.go`](../../internal/gateway/fakeip_test.go). Each redirect, up to 5, is checked the same, and a caller's image must sniff as an image. A request from this computer gets nothing more: a reverse proxy or relay in front of magpie on the same host makes every request it carries arrive from loopback, so where a request came from is never a reason to reach a private address. Two things are: settings.json's `fetchPrivate` (`settings.FetchPrivate`, none by default, no Settings page, this computer's own and never brought by a sync or restore), a list of names, `name:port`s, IPs and CIDR ranges the user lets magpie download from (`allowFor`; a settings.json that can't be read lets nothing extra in), and for a vendor's URL the `host:port` of that provider's own endpoints, which the user named themselves (a provider on the LAN that serves its images from itself). A vendor's image that can't be downloaded is answered by its URL, never dropped; its video is a 502 naming why. A URL left for the vendor to fetch (a chat model's message) is the vendor's to fetch.
+## A Responses WebSocket
+
+Codex speaks Responses over a WebSocket to its own provider, which magpie's
+wiring is (`openai_base_url` on the Codex backend path), and only there
+names the reply a turn follows (`previous_response_id`) and sends what came
+after it; over HTTP it sends the whole conversation, images and all, each
+turn (#4058, kevinzhow and neighbads on X). magpie answered the upgrade 426
+(a87dc1ff2) until #4058. [`responses_ws.go`](../../internal/gateway/responses_ws.go)
+serves it on `/v1/responses`, `/responses` and the Codex backend's
+`/responses`, with Codex's protocol (codex-rs
+`responses_websocket.rs`):
+
+- Each `response.create` text frame is one turn, served as the POST it
+ stands for: the upgrade's own request (URL, Host, address, headers but
+ the socket's own and `OpenAI-Beta: responses_websockets`) with the frame
+ as its body, `type`, `previous_response_id` and `generate` taken out, and
+ `client_metadata`'s turn state and responses-lite flag as the
+ `X-Codex-Turn-State` and `X-OpenAI-Internal-Codex-Responses-Lite`
+ headers Codex sends over HTTP. It goes through the whole handler the
+ upgrade came in on (`socketEntry`, outside `Front` and `Handler`), so
+ the allow list, `rebound`, the key, `corsGuard`, `keyLimited`, routing,
+ failover, usage and the log judge every turn as an HTTP one: a key
+ turned off or past its limit refuses the next turn
+ (`TestResponsesSocketGuards`). The upgrade itself passed those too, and
+ coder/websocket's `Accept` refuses a browser Origin whose host isn't the
+ `Host`. `holdCatalog` holds each turn, not the socket.
+- The reply's events go back a text frame each; the reply's headers as
+ Codex reads them on a socket (`response.metadata` with
+ `x-codex-turn-state`, `openai-model` and `x-reasoning-included`;
+ `codex.response.metadata` with `x-models-etag`, the backend's own event
+ given magpie's tag as `modelsEtag` gives it; `codex.rate_limits` from the
+ `x-codex-*` windows). A whole JSON reply is told as its stream would be. A
+ refused turn is an `error` frame with the status, the error object and
+ the headers Codex acts on (`Retry-After`, `x-codex-*`, `x-ratelimit-*`).
+- The conversation a turn continues is the socket's own: each completed
+ reply's input and output (in the order of `output_item.done`, as Codex
+ adds them), the last two kept (`socketKeep`; a warmup's and its first
+ turn's). A turn naming one is sent the whole conversation, so a provider
+ or account other than the last one's sees what it would over HTTP. An id
+ the socket doesn't hold, from another socket, or a failed reply's, is
+ answered `previous_response_not_found`, on which Codex opens a new socket
+ and sends everything. `prevResponses` isn't used: Codex says `store:
+ false`.
+- `generate: false` (Codex's warmup) is answered locally with a
+ `response.created` and `response.completed` of its own id; nothing is
+ sent upstream.
+- `response.interrupt` naming the reply in flight cancels its request and
+ is answered `response.interrupt.accepted`, then an interrupted
+ `response.incomplete`; one naming another reply, or none in flight,
+ `response.interrupt.failed`. Other frames during a turn wait for it.
+- An upstream cut the gateway tells as `response.failed` goes back as
+ that frame, and the reply isn't kept; a stream that still ends with no
+ terminal event closes the socket 1011, as an HTTP stream would be cut
+ (`TestResponsesSocketCutReplyFails`). Binary frames close it 1003.
+- `Run`'s shutdown ends each socket going away (1001) once its turn is
+ answered, through `clientSockets.drain` beside `srv.Shutdown` (a hijacked
+ connection isn't the server's), and refuses new ones 503.
+- Not covered: the leg to the vendor is the whole conversation each turn,
+ as before (for the ChatGPT backend, a socket per turn: see
+ [Providers and accounts](providers-accounts.md)); what is saved is
+ Codex's upload to magpie. A socket's conversation doesn't outlive the
+ socket or the gateway.
+
## A Responses request that names the response it follows
A Responses client may send only its new input with `previous_response_id`
@@ -1200,6 +1264,7 @@ included). `previousResponse` in
request with `store: false` keeps nothing. Codex over HTTP sends
`store: false` with the whole input and never names a previous response
(only its websocket request has the field), so nothing changes for it.
+ On a WebSocket it names one, which the socket keeps itself (below).
- A turn is kept for the caller that made it (`prevScope`): its gateway
key, the remote magpie it came through, and the address of a keyless
remote caller. Another caller naming the id is told it isn't there.
diff --git a/docs/subsystems/providers-accounts.md b/docs/subsystems/providers-accounts.md
index f4b7e1ec8..95f5f7853 100644
--- a/docs/subsystems/providers-accounts.md
+++ b/docs/subsystems/providers-accounts.md
@@ -256,7 +256,7 @@ and `TestHistoricalWindowHoldsUsesEachWindowsOwnReading`.
- A Copilot account is known by its GitHub login and host (`CopilotAccountName`, #1220): `mona` on github.com, `[email protected]` on an enterprise's GHE.com. That name is its `logins.json` user, its row in every list, its usage card and its per-account routing settings, and it is what `SwitchLogin`, `SetLoginOn` and `ForgetLogin` take, so one login on two hosts is two accounts, neither written over by the other nor dropped as the editors' own. One saved before on an enterprise's host under its login alone is read under its name with the host (`copilotSavedName`), keeping its id; the editors' own remembered so is renamed in place, still hidden if it was (`copilotRenamed`). Either rename takes the account's per-account settings in `providers.json` (proxy, models, cap, concurrency) from the bare login to the new name before the new name is written to `logins.json` (`renameSettled`): a setting the new name has already stays, and while `providers.json` can't be read or written the account stays saved under its old name, its settings with it. `magpie accounts switch|forget copilot` takes the full name, the login with `--host <name>.ghe.com`, or the login alone when it names one account.
- A Codex account is known by its email, with the plan for a workspace seat (`codexUser`), and a second seat of one email in another workspace by that name with its workspace after it (`codexName`): `[email protected] · Team · ws-two`. The account Codex is signed in to goes by the name it is saved under (`codexLiveName`) everywhere: its provider's `Account.User` (`codexAccount`), `liveLogin`, and its session identity (`SessionIdentities`). Its allowance, rests and sticks (`restKey`), cap, proxy, model list, a gateway key's account list (`LoginID`) and the Routing page all go by that name, so two seats of one email are never read as one (#1424).
- Every request to the ChatGPT backend (`CodexBase`'s host) goes over HTTP/1.1, as Codex itself speaks to it: each transport [`netproxy.Dispatch`](../../internal/netproxy/netproxy.go) makes (`http.DefaultClient`'s, the gateway's) sends a request `netproxy.HTTP1` names through an HTTP/1.1 copy of itself whose TLS hello offers no h2, and the provider package sets `HTTP1` to `codexBackend` ([codex_http1.go](../../internal/provider/codex_http1.go)). That covers the gateway's turns, the models list, the warm-up, the allowance, its resets and spending one, through the proxy in force, an account's own included. Every other vendor is still offered HTTP/2.
-- A streamed turn to the ChatGPT backend (POST `CodexBase`/responses with `"stream":true`) goes over the backend's WebSocket, as Codex's own client sends it, and comes back as the server-sent events the HTTP route answers with (`codexSocket` in [codex_socket.go](../../internal/gateway/codex_socket.go), first in the gateway's `upstreamTransport`, so Codex's own sign-in passed through and magpie's ChatGPT accounts both take it, through the proxy in force). On some networks a streamed HTTP reply from chatgpt.com is cut at about 120 s, before `response.completed`, while the WebSocket runs to the end (brickhao on Discord). The handshake carries the turn's headers with `OpenAI-Beta: responses_websockets=2026-02-06`, without `x-codex-turn-state` and the responses-lite header, which go in the message's `client_metadata` as Codex puts them; the message is the body with `"type":"response.create"` ahead of it (a zstd body is decoded first). `codex.rate_limits` and `codex.response.metadata`, read ahead of the reply, are also told as the `x-codex-*` and `x-codex-turn-state` headers the HTTP route carries; `responsesapi.websocket_timing` is dropped. An `error` frame ahead of the reply is that status with its error as the body; `websocket_connection_limit_reached`, a refused handshake or a body that isn't a streamed turn goes over HTTP instead, and a host that refused the handshake (other than 401, 429 or 5xx) is spoken HTTP for 10 minutes (`codexSocketShun`). A socket that ends, or won't take the message, before any frame of the reply was handed on (any close status, a dropped connection, a normal close with no reply) sends the same body over HTTP (`codexSocketEnded`): Lamrfu's long conversation was closed 1009 (message too big) by chatgpt.com and got a 502. magpie opens a socket per turn and sends the whole body, where Codex's own client sends a delta after `previous_response_id` on a kept socket. A message at least as big as one a host closed 1009 goes to it over HTTP for 10 minutes (`codexTooBig`). A socket closed after the reply began reads as a cut stream (`io.ErrUnexpectedEOF`), and is not asked again. Codex still speaks HTTP to magpie: magpie answers its WebSocket handshake 426, and Codex falls back for the session; only the leg to the vendor changed. The gateway's tests turn it off in TestMain (`codexSocketOff`); codex_socket_test's fakes speak it.
+- A streamed turn to the ChatGPT backend (POST `CodexBase`/responses with `"stream":true`) goes over the backend's WebSocket, as Codex's own client sends it, and comes back as the server-sent events the HTTP route answers with (`codexSocket` in [codex_socket.go](../../internal/gateway/codex_socket.go), first in the gateway's `upstreamTransport`, so Codex's own sign-in passed through and magpie's ChatGPT accounts both take it, through the proxy in force). On some networks a streamed HTTP reply from chatgpt.com is cut at about 120 s, before `response.completed`, while the WebSocket runs to the end (brickhao on Discord). The handshake carries the turn's headers with `OpenAI-Beta: responses_websockets=2026-02-06`, without `x-codex-turn-state` and the responses-lite header, which go in the message's `client_metadata` as Codex puts them; the message is the body with `"type":"response.create"` ahead of it (a zstd body is decoded first). `codex.rate_limits` and `codex.response.metadata`, read ahead of the reply, are also told as the `x-codex-*` and `x-codex-turn-state` headers the HTTP route carries; `responsesapi.websocket_timing` is dropped. An `error` frame ahead of the reply is that status with its error as the body; `websocket_connection_limit_reached`, a refused handshake or a body that isn't a streamed turn goes over HTTP instead, and a host that refused the handshake (other than 401, 429 or 5xx) is spoken HTTP for 10 minutes (`codexSocketShun`). A socket that ends, or won't take the message, before any frame of the reply was handed on (any close status, a dropped connection, a normal close with no reply) sends the same body over HTTP (`codexSocketEnded`): Lamrfu's long conversation was closed 1009 (message too big) by chatgpt.com and got a 502. magpie opens a socket per turn and sends the whole body, where Codex's own client sends a delta after `previous_response_id` on a kept socket. A message at least as big as one a host closed 1009 goes to it over HTTP for 10 minutes (`codexTooBig`). A socket closed after the reply began reads as a cut stream (`io.ErrUnexpectedEOF`), and is not asked again. Codex speaks to magpie over its own WebSocket since #4058 ([A Responses WebSocket](gateway-routing.md#a-responses-websocket)), sending a turn as a delta after `previous_response_id`; magpie gives each turn the whole conversation before it goes on, so this leg still sends the whole body. The gateway's tests turn it off in TestMain (`codexSocketOff`); codex_socket_test's fakes speak it.
- A provider the user added may have its certificate left unchecked (`InsecureTLS`, the editor's Certificate row, #2383: a vLLM on the LAN, Ollama behind a self-signed cert, a company CA). Only one with a base URL of the user's own takes it (`TakesInsecureTLS`: no account, no preset, not Vertex, not a subscription's or plugin's id); `Save` refuses it on any other, and one written into providers.json by hand is ignored. `Provider.Via` then adds `netproxy.WithUncheckedTLS` with the hosts of its own base URLs (`UncheckedHosts`: Chat, Responses, Anthropic, Gemini, Decide), so every request made on its behalf through a `Dispatch` transport (the gateway's turns, count_tokens, models, test and detect, balance) skips the check for those hosts alone, on a transport kept apart for each (`route.unchecked`). A redirect or a models, balance or image URL on another host is checked, and `netproxy.With` drops the leave for a request made for another provider. An `https://` proxy would be dialled with the same TLS config, so its certificate would go unchecked too: such a request is refused (`ErrUncheckedOverHTTPSProxy`); an `http://` or `socks5://` proxy is taken. Backup, export and sync carry the field with the provider; an import link doesn't.
- A provider the user added may be set never to rest for a limit (`LimitRestOff`, `limitRestOff`; the editor's Advanced fold, "Set it aside when out of quota or rate limited", ticked by default; zoecats on Discord: a local aggregator's "quota exhausted" while it switches upstreams). Off, a quota, credit or rate refusal and any 429 rest none of its keys and the next request is sent to it as usual (`restAfterMarked` in the gateway, see [gateway routing](gateway-routing.md)). The editor offers it only to a provider the user added (`custom`); a save that leaves it out keeps it (`TestProviderSaveLimitRestOff`, `provider-limit-rest.test.cjs`).
- The Copilot account the editors and CLI keep is read by `copilotLogin` ([account.go](../../internal/provider/account.go)): an editor's github.com sign-in in `~/.config/github-copilot/apps.json` or `hosts.json` first, then the Copilot CLI's, then an editor's on GHE.com. When GitHub refuses the editor token's trade for a session token (401 or 403, `copilotRefusals`, retried after an hour), the CLI's sign-in of the same login on the same host takes its place, sent as the CLI sends it, from that request on and on the Usage card (`copilotStandIn`); a CLI signed in to another account is never used for it, and the error names the editors' file (#1238). A sign-in from magpie of the account whose editor token is refused is kept and stands for it (`copilotProbeEditor`, `addCopilotLogin`); while the editor token works, it is let go as the agent's own, as before.
diff --git a/internal/gateway/codex_backend.go b/internal/gateway/codex_backend.go
index 9a69fa7f7..098405c9d 100644
--- a/internal/gateway/codex_backend.go
+++ b/internal/gateway/codex_backend.go
@@ -64,9 +64,14 @@ Be concise, structured, and focused on helping the next LLM seamlessly continue
const codexSummaryPrefix = codexsummary.Prefix
func (s *Server) codexBackend(w http.ResponseWriter, r *http.Request) {
- // Responses over a WebSocket: 426 sends Codex to plain HTTP at once
- if strings.EqualFold(r.Header.Get("Upgrade"), "websocket") {
- http.Error(w, "magpie speaks HTTP", http.StatusUpgradeRequired)
+ // Responses over a WebSocket, Codex's own way to its provider: each
+ // turn is served as its POST is (responses_ws.go), and Codex sends
+ // only a turn's new input after the reply it follows (#4058)
+ if wantsSocket(r) {
+ if strings.TrimPrefix(r.URL.Path, CodexPath) == "/responses" && s.serveResponsesSocket(w, r) {
+ return
+ }
+ http.Error(w, "magpie serves a WebSocket on "+CodexPath+"/responses alone", http.StatusUpgradeRequired)
return
}
// the pool's accounts and their model lists say they are this Codex, or
diff --git a/internal/gateway/codex_backend_test.go b/internal/gateway/codex_backend_test.go
index bb7af2c20..45a8d66af 100644
--- a/internal/gateway/codex_backend_test.go
+++ b/internal/gateway/codex_backend_test.go
@@ -869,19 +869,6 @@ func TestCodexModelsEtagTagged(t *testing.T) {
}
}
-// Responses over a WebSocket are turned away so Codex uses HTTP at once.
-func TestCodexWebSocketUpgradeRequired(t *testing.T) {
- chatgpt(t, func(w http.ResponseWriter, r *http.Request) { t.Error("sent to ChatGPT") })
- rec := httptest.NewRecorder()
- req := httptest.NewRequest("GET", CodexPath+"/responses", nil)
- req.Header.Set("Upgrade", "websocket")
- req.Header.Set("Connection", "Upgrade")
- New().Handler().ServeHTTP(rec, req)
- if rec.Code != http.StatusUpgradeRequired {
- t.Errorf("%d", rec.Code)
- }
-}
-
// A routing group in Codex's model list is named as one, not as its first
// member's provider, which would read as that provider's own model.
func TestCodexModelsNameGroups(t *testing.T) {
diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.go
index e7c6bd97a..710b520ed 100644
--- a/internal/gateway/gateway.go
+++ b/internal/gateway/gateway.go
@@ -460,6 +460,8 @@ type Server struct {
// the connections taken whose first request isn't read yet, which a
// handover's end waits for (handover.go)
unread unreadConns
+ // sockets are the Responses WebSockets being served (responses_ws.go)
+ sockets clientSockets
// what it serves other computers TLS with, nil without (tls.go)
tls *tls.Config
// the images described for models that can't see them (vision.go)
@@ -720,7 +722,12 @@ func (s *Server) ListenAndServe(ctx context.Context) error {
s.stopAccepting()
s.unread.wait(c)
}
+ // a socket's connection was taken from the server (hijacked), so
+ // Shutdown doesn't wait for it: each ends after its turn in flight
+ sockets := make(chan struct{})
+ go func() { defer close(sockets); s.sockets.drain(c) }()
srv.Shutdown(c)
+ <-sockets
s.subscription.abortAll()
provider.FlushServed() // what answered last, kept for the next start and the CLI
flushSticks() // who answered which conversation, for the next start or the magpie handed over to
@@ -800,8 +807,8 @@ func (s *Server) Handler() http.Handler {
mux.HandleFunc("POST /chat/completions", s.handle(provider.Chat))
mux.HandleFunc("POST /v1/responses", s.handle(provider.Responses))
mux.HandleFunc("POST /responses", s.handle(provider.Responses))
- mux.HandleFunc("GET /v1/responses", responsesOverHTTP)
- mux.HandleFunc("GET /responses", responsesOverHTTP)
+ mux.HandleFunc("GET /v1/responses", s.responsesOverHTTP)
+ mux.HandleFunc("GET /responses", s.responsesOverHTTP)
mux.HandleFunc("POST /v1/messages", keepsSignIn(s.handle(provider.Anthropic)))
mux.HandleFunc("POST /messages", keepsSignIn(s.handle(provider.Anthropic)))
mux.HandleFunc("POST /v1/systemone", s.serveSystemOne)
@@ -849,16 +856,17 @@ func (s *Server) Handler() http.Handler {
})
// the goroutine dump first, ahead of every lock the rest can be stuck
// on (dumpFirst)
- return s.dumpFirst(s.counted(holdCatalog(corsGuard(callerGuard(withCaller(keyLimited(mux)))))))
+ // a socket's turns go through all of it again (socketEntry)
+ return socketEntry(s.dumpFirst(s.counted(holdCatalog(corsGuard(callerGuard(withCaller(keyLimited(mux))))))))
}
-// responsesOverHTTP answers a GET on /v1/responses, which is how a client
-// opens Responses over a WebSocket (#1005): magpie relays Responses over
-// HTTP alone, its stream as SSE, so the upgrade is told 426 — the answer a
-// client falls back to HTTP on at once (Codex does, as on CodexPath) — in
-// place of the 404 every unknown path gets, which read as a broken
-// gateway. A plain GET is told to POST.
-func responsesOverHTTP(w http.ResponseWriter, r *http.Request) {
+// responsesOverHTTP answers a GET on /v1/responses. A WebSocket upgrade
+// is Responses over a WebSocket (#1005, #4058), each turn served as its
+// POST is (responses_ws.go). A plain GET is told to POST.
+func (s *Server) responsesOverHTTP(w http.ResponseWriter, r *http.Request) {
+ if wantsSocket(r) && s.serveResponsesSocket(w, r) {
+ return
+ }
if strings.EqualFold(r.Header.Get("Upgrade"), "websocket") {
writeError(w, provider.Responses, http.StatusUpgradeRequired, "magpie relays Responses over HTTP only: send POST /v1/responses (streamed as SSE); Responses over a WebSocket is not served")
return
diff --git a/internal/gateway/holdcatalog.go b/internal/gateway/holdcatalog.go
index 50baf4865..fface5690 100644
--- a/internal/gateway/holdcatalog.go
+++ b/internal/gateway/holdcatalog.go
@@ -17,6 +17,11 @@ import (
// and a file another program writes is seen within the second.
func holdCatalog(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ // a WebSocket lasts a session: each of its turns holds it instead
+ if wantsSocket(r) {
+ h.ServeHTTP(w, r)
+ return
+ }
defer provider.Hold()()
defer filememo.Hold()()
h.ServeHTTP(w, r)
diff --git a/internal/gateway/oneport.go b/internal/gateway/oneport.go
index 18921cdd8..6cf480d41 100644
--- a/internal/gateway/oneport.go
+++ b/internal/gateway/oneport.go
@@ -15,7 +15,8 @@ import (
func (s *Server) Front() http.Handler {
// the goroutine dump ahead of lanGuard too, whose allow list reads
// settings.json under its lock (dumpFirst)
- s.frontOnce.Do(func() { s.front = s.dumpFirst(lanGuard(s.Handler())) })
+ // (socketEntry: a socket's turns go through all of it again)
+ s.frontOnce.Do(func() { s.front = socketEntry(s.dumpFirst(lanGuard(s.Handler()))) })
return s.front
}
diff --git a/internal/gateway/responses_socket_test.go b/internal/gateway/responses_socket_test.go
deleted file mode 100644
index a99de87ab..000000000
--- a/internal/gateway/responses_socket_test.go
+++ /dev/null
@@ -1,34 +0,0 @@
-package gateway
-
-import (
- "net/http"
- "net/http/httptest"
- "strings"
- "testing"
-
- "github.com/yetone/magpie/internal/provider"
-)
-
-// A client opening Responses over a WebSocket (Pi's Responses WS
-// extension, #1005) is told 426 with the reason, not the 404 of an unknown
-// path, and nothing goes upstream; a plain GET is told to POST.
-func TestResponsesWebSocketUpgradeRequired(t *testing.T) {
- setup(t, provider.Responses, &fake{t: t})
- for _, path := range []string{"/v1/responses", "/responses"} {
- rec := httptest.NewRecorder()
- req := httptest.NewRequest("GET", path, nil)
- req.Header.Set("Upgrade", "websocket")
- req.Header.Set("Connection", "Upgrade")
- req.Header.Set("Sec-WebSocket-Version", "13")
- req.Header.Set("Sec-WebSocket-Key", "dGhlIHNhbXBsZSBub25jZQ==")
- New().Handler().ServeHTTP(rec, req)
- if rec.Code != http.StatusUpgradeRequired || !strings.Contains(rec.Body.String(), "POST /v1/responses") {
- t.Errorf("%s upgrade: %d %s", path, rec.Code, rec.Body)
- }
- rec = httptest.NewRecorder()
- New().Handler().ServeHTTP(rec, httptest.NewRequest("GET", path, nil))
- if rec.Code != http.StatusMethodNotAllowed || rec.Header().Get("Allow") != "POST" {
- t.Errorf("%s GET: %d %q", path, rec.Code, rec.Header().Get("Allow"))
- }
- }
-}
diff --git a/internal/gateway/responses_ws.go b/internal/gateway/responses_ws.go
new file mode 100644
index 000000000..6c91065ae
--- /dev/null
+++ b/internal/gateway/responses_ws.go
@@ -0,0 +1,822 @@
+package gateway
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "log"
+ "net/http"
+ "strconv"
+ "strings"
+ "sync"
+ "time"
+ "unicode/utf8"
+
+ "github.com/coder/websocket"
+ "github.com/tidwall/gjson"
+)
+
+// Responses over a WebSocket, as Codex speaks it to its own provider
+// (codex-rs codex-api/src/endpoint/responses_websocket.rs and
+// core/src/client.rs at openai/codex 2f1e4f9). Codex opens one socket
+// for a session and sends each turn on it as a text frame, the HTTP
+// body with "type":"response.create" added; the reply comes back as one
+// text frame for each event the HTTP route streams. On a socket it has
+// had a reply on, Codex names that reply (previous_response_id) and
+// sends only what came after it, where over HTTP it sends the whole
+// conversation every turn, images and all (#4058, kevinzhow and
+// neighbads on X). magpie answered the upgrade 426 (a87dc1ff2), which
+// has Codex use HTTP for the session.
+//
+// Each response.create is served as the POST it stands for: the request
+// the socket was opened with is made again, with the frame as its body,
+// and sent through the whole handler the upgrade came in on (socketEntry):
+// the allow list, the key and its limits, the Origin and Host checks,
+// routing, failover, usage and the log judge every turn as they judge an
+// HTTP one, and a key turned off ends the next turn, not the socket.
+//
+// The conversation a turn continues is the socket's own: each reply's
+// input and output, kept for the socket's life (Codex says store false on
+// every request, and OpenAI's socket keeps the last response for the
+// connection all the same). A turn naming one is given the conversation
+// whole before it is sent, so what serves it, a provider or another
+// account, sees what it would over HTTP. An id the socket doesn't hold
+// is answered previous_response_not_found, on which Codex opens a new
+// socket and sends the conversation whole.
+
+// socketOriginKey holds the request a socket was opened with.
+type socketOriginKey struct{}
+
+type socketOrigin struct {
+ ctx context.Context // the request's own, before any guard's
+ req *http.Request // as it arrived: URL, Host, RemoteAddr, headers
+ via http.Handler // where it arrived, which each turn goes through
+}
+
+// wantsSocket: r asks for a WebSocket.
+func wantsSocket(r *http.Request) bool {
+ return r.Method == http.MethodGet && strings.EqualFold(r.Header.Get("Upgrade"), "websocket")
+}
+
+// socketEntry is where a request reaches the gateway (Front, Handler): a
+// socket's turns are sent through it again, as they arrived.
+func socketEntry(next http.Handler) http.Handler {
+ var self http.Handler
+ self = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ if wantsSocket(r) && r.Context().Value(socketOriginKey{}) == nil {
+ o := &socketOrigin{ctx: r.Context(), req: r.Clone(r.Context()), via: self}
+ r = r.WithContext(context.WithValue(r.Context(), socketOriginKey{}, o))
+ }
+ next.ServeHTTP(w, r)
+ })
+ return self
+}
+
+const (
+ // socketKeep is how many replies a socket holds: Codex names only the
+ // last, and a warmup's before its first turn
+ socketKeep = 2
+ // socketErrMax is the most of an error reply sent on
+ socketErrMax = 1 << 20
+)
+
+// clientSockets are the sockets being served, which end with the gateway.
+type clientSockets struct {
+ mu sync.Mutex
+ m map[*responsesSocket]bool
+ closing bool
+ done chan struct{} // closed when the last one ends while closing
+}
+
+func (cs *clientSockets) add(s *responsesSocket) bool {
+ cs.mu.Lock()
+ defer cs.mu.Unlock()
+ if cs.closing {
+ return false
+ }
+ if cs.m == nil {
+ cs.m = map[*responsesSocket]bool{}
+ }
+ cs.m[s] = true
+ return true
+}
+
+func (cs *clientSockets) ending() bool {
+ cs.mu.Lock()
+ defer cs.mu.Unlock()
+ return cs.closing
+}
+
+func (cs *clientSockets) remove(s *responsesSocket) {
+ cs.mu.Lock()
+ defer cs.mu.Unlock()
+ delete(cs.m, s)
+ if cs.closing && len(cs.m) == 0 && cs.done != nil {
+ close(cs.done)
+ cs.done = nil
+ }
+}
+
+// drain ends every socket once its turn in flight is answered, and waits
+// for them until ctx ends, when the rest are cut. Codex opens a new one
+// for its next turn, to the gateway serving the port then, and sends the
+// conversation whole on it.
+func (cs *clientSockets) drain(ctx context.Context) {
+ cs.mu.Lock()
+ cs.closing = true
+ var done chan struct{}
+ if len(cs.m) > 0 {
+ done = make(chan struct{})
+ cs.done = done
+ }
+ all := make([]*responsesSocket, 0, len(cs.m))
+ for s := range cs.m {
+ all = append(all, s)
+ }
+ cs.mu.Unlock()
+ for _, s := range all {
+ s.goingAway()
+ }
+ if done == nil {
+ return
+ }
+ select {
+ case <-done:
+ case <-ctx.Done():
+ for _, s := range all {
+ s.c.CloseNow()
+ }
+ }
+}
+
+// heldTurn is one reply a socket holds: the conversation up to and
+// with it.
+type heldTurn struct {
+ id string
+ items []json.RawMessage
+}
+
+type responsesSocket struct {
+ s *Server
+ o *socketOrigin
+ path string
+ c *websocket.Conn
+ ctx context.Context
+ live context.Context // the socket's, for writes (send)
+
+ writeMu sync.Mutex
+ turns []heldTurn // the newest last
+
+ mu sync.Mutex
+ away bool // the gateway is ending: close after this turn
+ turning bool
+
+ // for the log
+ frames, continued int
+ got, sent int64
+}
+
+// serveResponsesSocket serves a WebSocket upgrade on a Responses route,
+// and false when r didn't come through socketEntry.
+func (s *Server) serveResponsesSocket(w http.ResponseWriter, r *http.Request) bool {
+ o, _ := r.Context().Value(socketOriginKey{}).(*socketOrigin)
+ if o == nil {
+ return false
+ }
+ // a browser's page from another site is turned away (the Origin
+ // check here is the library's: Origin's host must be the Host); the
+ // guards the upgrade passed judged it already
+ if s.sockets.ending() {
+ http.Error(w, "magpie is restarting", http.StatusServiceUnavailable)
+ return true
+ }
+ c, err := websocket.Accept(w, r, &websocket.AcceptOptions{CompressionMode: websocket.CompressionNoContextTakeover})
+ if err != nil {
+ log.Printf("responses socket from %s: %v", r.RemoteAddr, err)
+ return true
+ }
+ c.SetReadLimit(defaultBodyLimit)
+ sock := &responsesSocket{s: s, o: o, path: r.URL.Path, c: c, ctx: o.ctx}
+ if !s.sockets.add(sock) {
+ c.Close(websocket.StatusGoingAway, "magpie is restarting")
+ return true
+ }
+ defer s.sockets.remove(sock)
+ sock.serve()
+ if sock.frames > 0 {
+ log.Printf("responses socket from %s ended: %d turns, %d continued, %d bytes received, %d sent on as whole requests",
+ r.RemoteAddr, sock.frames, sock.continued, sock.got, sock.sent)
+ }
+ return true
+}
+
+func (rs *responsesSocket) goingAway() {
+ rs.mu.Lock()
+ rs.away = true
+ idle := !rs.turning
+ rs.mu.Unlock()
+ if idle {
+ // Close waits for the client's reply: not one socket after another
+ go rs.c.Close(websocket.StatusGoingAway, "magpie is restarting")
+ }
+}
+
+type socketMsg struct {
+ data []byte
+ err error
+}
+
+func (rs *responsesSocket) serve() {
+ ctx, cancel := context.WithCancel(rs.ctx)
+ defer cancel()
+ defer rs.c.CloseNow()
+ rs.live = ctx
+ in := make(chan socketMsg)
+ go func() {
+ for {
+ typ, data, err := rs.c.Read(ctx)
+ if err == nil && typ != websocket.MessageText {
+ err = errors.New("binary frame")
+ rs.c.Close(websocket.StatusUnsupportedData, "text frames only")
+ }
+ select {
+ case in <- socketMsg{data, err}:
+ case <-ctx.Done():
+ return
+ }
+ if err != nil {
+ return
+ }
+ }
+ }()
+ var queued []socketMsg
+ for {
+ var m socketMsg
+ if len(queued) > 0 {
+ m, queued = queued[0], queued[1:]
+ } else {
+ select {
+ case m = <-in:
+ case <-ctx.Done():
+ return
+ }
+ }
+ if m.err != nil {
+ return
+ }
+ kind := gjson.GetBytes(m.data, "type").String()
+ switch kind {
+ case "response.create":
+ case "response.interrupt":
+ // nothing in flight: it ended before the ask came
+ rs.write(ctx, interruptReply("response.interrupt.failed", gjson.GetBytes(m.data, "response_id").String()))
+ continue
+ default:
+ rs.writeError(ctx, http.StatusBadRequest, "invalid_request_error", "unsupported_message",
+ fmt.Sprintf("magpie serves response.create on this socket, not %q", kind), "type")
+ continue
+ }
+ rs.mu.Lock()
+ rs.turning = true
+ rs.mu.Unlock()
+ turnCtx, stop := context.WithCancel(ctx)
+ t := &socketTurnRun{}
+ done := make(chan struct{})
+ go func() {
+ defer close(done)
+ rs.turn(turnCtx, t, m.data)
+ }()
+ for waiting := true; waiting; {
+ select {
+ case <-done:
+ waiting = false
+ case n := <-in:
+ if n.err != nil {
+ // the client went: what is in flight ends with it
+ stop()
+ <-done
+ return
+ }
+ if gjson.GetBytes(n.data, "type").String() != "response.interrupt" {
+ queued = append(queued, n)
+ continue
+ }
+ id := gjson.GetBytes(n.data, "response_id").String()
+ if !t.interrupt(id) {
+ rs.write(ctx, interruptReply("response.interrupt.failed", id))
+ continue
+ }
+ stop()
+ <-done
+ waiting = false
+ rs.write(ctx, interruptReply("response.interrupt.accepted", id))
+ // the reply ends as interrupted: Codex keeps none of it
+ // (discard_partial_items) and waits for its end
+ end, _ := json.Marshal(map[string]any{"type": "response.incomplete", "response": map[string]any{
+ "id": id, "object": "response", "status": "incomplete",
+ "incomplete_details": map[string]any{"reason": "interrupted"}, "output": []any{},
+ }})
+ rs.write(ctx, end)
+ }
+ }
+ stop()
+ rs.mu.Lock()
+ rs.turning = false
+ away := rs.away
+ rs.mu.Unlock()
+ if t.cut {
+ // the reply ended before its end: as over HTTP, Codex asks
+ // again, on a new socket, with the conversation whole
+ rs.c.Close(websocket.StatusInternalError, "the reply ended before response.completed")
+ return
+ }
+ if away {
+ rs.c.Close(websocket.StatusGoingAway, "magpie is restarting")
+ return
+ }
+ }
+}
+
+func interruptReply(kind, id string) []byte {
+ b, _ := json.Marshal(map[string]string{"type": kind, "response_id": id})
+ return b
+}
+
+// write sends frame unless ctx, a turn's, has ended (interrupted, or the
+// client gone). The write itself is under the socket's context: one whose
+// context ends mid-write closes the socket, which an interrupt mustn't.
+func (rs *responsesSocket) write(ctx context.Context, frame []byte) error {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+ rs.writeMu.Lock()
+ defer rs.writeMu.Unlock()
+ return rs.c.Write(rs.live, websocket.MessageText, frame)
+}
+
+// writeError is an error frame as Codex reads one: the HTTP status and
+// the error object an HTTP reply would have carried.
+func (rs *responsesSocket) writeError(ctx context.Context, status int, typ, code, msg, param string) error {
+ e := map[string]any{"type": typ, "code": code, "message": msg}
+ if param != "" {
+ e["param"] = param
+ }
+ b, _ := json.Marshal(map[string]any{"type": "error", "status": status, "error": e})
+ return rs.write(ctx, b)
+}
+
+// socketTurnRun is what a turn in flight has said so far.
+type socketTurnRun struct {
+ mu sync.Mutex
+ ids []string // the response ids it gave
+ cut bool // its reply ended before its end
+ cancelled bool
+}
+
+func (t *socketTurnRun) interrupt(id string) bool {
+ t.mu.Lock()
+ defer t.mu.Unlock()
+ for _, have := range t.ids {
+ if have == id {
+ t.cancelled = true
+ return true
+ }
+ }
+ return false
+}
+
+// held is the conversation a reply the socket holds ends.
+func (rs *responsesSocket) held(id string) ([]json.RawMessage, bool) {
+ for i := len(rs.turns) - 1; i >= 0; i-- {
+ if rs.turns[i].id == id {
+ return rs.turns[i].items, true
+ }
+ }
+ return nil, false
+}
+
+func (rs *responsesSocket) keep(t heldTurn) {
+ rs.turns = append(rs.turns, t)
+ if len(rs.turns) > socketKeep {
+ rs.turns = append(rs.turns[:0:0], rs.turns[len(rs.turns)-socketKeep:]...)
+ }
+}
+
+// turn serves one response.create.
+func (rs *responsesSocket) turn(ctx context.Context, t *socketTurnRun, frame []byte) {
+ rs.frames++
+ rs.got += int64(len(frame))
+ var q map[string]json.RawMessage
+ if decodeJSONView(frame, &q) != nil {
+ rs.writeError(ctx, http.StatusBadRequest, "invalid_request_error", "invalid_json", "response.create is not a JSON object", "")
+ return
+ }
+ _, fresh := prevInput(q["input"])
+ if fresh == nil {
+ rs.writeError(ctx, http.StatusBadRequest, "invalid_request_error", "invalid_value", "input is neither text nor a list of items", "input")
+ return
+ }
+ input := fresh
+ if raw := q["previous_response_id"]; len(raw) > 0 && !bytes.Equal(raw, []byte("null")) {
+ var id string
+ json.Unmarshal(raw, &id)
+ prior, ok := rs.held(id)
+ if !ok {
+ // Codex opens a new socket and sends the conversation whole
+ rs.writeError(ctx, http.StatusBadRequest, "invalid_request_error", "previous_response_not_found",
+ "Previous response with id '"+id+"' not found.", "previous_response_id")
+ return
+ }
+ input = append(prior[:len(prior):len(prior)], fresh...)
+ rs.continued++
+ }
+ // a warmup (generate false) asks for no reply: Codex sends it with
+ // the conversation ahead of its first turn, and that turn follows it
+ var generate bool
+ if raw := q["generate"]; len(raw) > 0 && json.Unmarshal(raw, &generate) == nil && !generate {
+ id := "resp_" + newID()
+ t.mu.Lock()
+ t.ids = append(t.ids, id)
+ t.mu.Unlock()
+ resp := map[string]any{"id": id, "object": "response", "created_at": time.Now().Unix(), "status": "in_progress", "output": []any{}}
+ if m := q["model"]; len(m) > 0 {
+ resp["model"] = m
+ }
+ b, _ := json.Marshal(map[string]any{"type": "response.created", "response": resp})
+ if rs.write(ctx, b) != nil {
+ return
+ }
+ resp["status"] = "completed"
+ resp["usage"] = map[string]any{"input_tokens": 0, "output_tokens": 0, "total_tokens": 0}
+ b, _ = json.Marshal(map[string]any{"type": "response.completed", "response": resp})
+ if rs.write(ctx, b) == nil {
+ rs.keep(heldTurn{id: id, items: input})
+ }
+ return
+ }
+ // the body the HTTP route would have had
+ delete(q, "type")
+ delete(q, "previous_response_id")
+ delete(q, "generate")
+ in, err := json.Marshal(input)
+ if err != nil {
+ rs.writeError(ctx, http.StatusBadRequest, "invalid_request_error", "invalid_value", err.Error(), "input")
+ return
+ }
+ q["input"] = in
+ body, err := json.Marshal(q)
+ if err != nil {
+ rs.writeError(ctx, http.StatusBadRequest, "invalid_request_error", "invalid_json", err.Error(), "")
+ return
+ }
+ rs.sent += int64(len(body))
+ req := rs.request(ctx, body)
+ out := &socketTurnWriter{rs: rs, ctx: ctx, run: t, h: http.Header{}}
+ rs.o.via.ServeHTTP(out, req)
+ out.finish()
+ if ctx.Err() != nil {
+ return
+ }
+ if !out.ended {
+ t.cut = true
+ return
+ }
+ if out.id != "" && out.completed {
+ rs.keep(heldTurn{id: out.id, items: append(input[:len(input):len(input)], out.items...)})
+ }
+}
+
+// request is the POST a turn stands for: the upgrade's own request, its
+// URL, Host, address and headers, with the turn as its body.
+func (rs *responsesSocket) request(ctx context.Context, body []byte) *http.Request {
+ o := rs.o.req
+ req := o.Clone(ctx)
+ req.Method = http.MethodPost
+ req.Header = http.Header{}
+ for k, vs := range o.Header {
+ ck := http.CanonicalHeaderKey(k)
+ if hopHeader(ck) || strings.HasPrefix(ck, "Sec-Websocket-") {
+ continue
+ }
+ switch ck {
+ case "Content-Type", "Content-Encoding", "Accept", "Accept-Encoding":
+ continue
+ case "Openai-Beta":
+ // the socket's own beta (responses_websockets=…) isn't the turn's
+ var keep []string
+ for _, v := range vs {
+ if !strings.Contains(v, "responses_websockets") {
+ keep = append(keep, v)
+ }
+ }
+ if len(keep) > 0 {
+ req.Header[ck] = keep
+ }
+ continue
+ }
+ req.Header[ck] = append([]string(nil), vs...)
+ }
+ req.Header.Set("Content-Type", "application/json")
+ req.Header.Set("Accept", "text/event-stream")
+ // what Codex says in client_metadata here and in a header over HTTP
+ // (codexSocketMessage, the other way)
+ if v := gjson.GetBytes(body, `client_metadata.x-codex-turn-state`).String(); v != "" {
+ req.Header.Set("X-Codex-Turn-State", v)
+ }
+ if strings.EqualFold(gjson.GetBytes(body, `client_metadata.ws_request_header_x_openai_internal_codex_responses_lite`).String(), "true") {
+ req.Header.Set("X-OpenAI-Internal-Codex-Responses-Lite", "true")
+ }
+ req.Body = io.NopCloser(bytes.NewReader(body))
+ req.GetBody = func() (io.ReadCloser, error) { return io.NopCloser(bytes.NewReader(body)), nil }
+ req.ContentLength = int64(len(body))
+ req.Header.Set("Content-Length", strconv.Itoa(len(body)))
+ req.RequestURI = ""
+ return req
+}
+
+// socketTurnWriter takes the HTTP reply a turn is given and sends it on
+// as the socket's frames: each event of a stream as one, an error as an
+// error frame.
+type socketTurnWriter struct {
+ rs *responsesSocket
+ ctx context.Context
+ run *socketTurnRun
+ h http.Header
+
+ status int
+ stream bool
+ pw *io.PipeWriter
+ read chan struct{}
+ buf bytes.Buffer // a reply that isn't a stream
+
+ // what the reply said
+ id string
+ items []json.RawMessage
+ ended, completed bool
+ wroteErr bool
+}
+
+func (w *socketTurnWriter) Header() http.Header { return w.h }
+
+func (w *socketTurnWriter) WriteHeader(code int) {
+ if w.status != 0 || code < 200 {
+ return
+ }
+ w.status = code
+ ct := w.h.Get("Content-Type")
+ if code/100 == 2 && strings.Contains(ct, "event-stream") {
+ w.startStream()
+ }
+}
+
+func (w *socketTurnWriter) Write(b []byte) (int, error) {
+ if w.status == 0 {
+ if w.h.Get("Content-Type") == "" && (bytes.HasPrefix(b, []byte("event:")) || bytes.HasPrefix(b, []byte("data:"))) {
+ w.h.Set("Content-Type", "text/event-stream")
+ }
+ w.WriteHeader(http.StatusOK)
+ }
+ if w.stream {
+ return w.pw.Write(b)
+ }
+ if w.buf.Len() < socketErrMax || w.status/100 == 2 {
+ w.buf.Write(b)
+ }
+ return len(b), nil
+}
+
+func (w *socketTurnWriter) Flush() {}
+
+func (w *socketTurnWriter) startStream() {
+ w.stream = true
+ for _, f := range w.headerFrames() {
+ w.send(f)
+ }
+ pr, pw := io.Pipe()
+ w.pw, w.read = pw, make(chan struct{})
+ go func() {
+ defer close(w.read)
+ err := readSSE(pr, func(_, data string) error {
+ data = strings.TrimSpace(data)
+ if data == "" || data == "[DONE]" {
+ return nil
+ }
+ return w.event([]byte(data))
+ })
+ pr.CloseWithError(err)
+ if err == nil {
+ io.Copy(io.Discard, pr)
+ }
+ }()
+}
+
+// headerFrames say what the HTTP route says in the reply's headers, as
+// Codex reads it on a socket: the turn state and the model in
+// response.metadata, the model list's tag in codex.response.metadata,
+// and the account's windows in codex.rate_limits.
+func (w *socketTurnWriter) headerFrames() [][]byte {
+ var out [][]byte
+ meta := map[string]string{}
+ for _, k := range []string{"x-codex-turn-state", "openai-model", "x-reasoning-included"} {
+ if v := w.h.Get(k); v != "" {
+ meta[k] = v
+ }
+ }
+ if len(meta) > 0 {
+ b, _ := json.Marshal(map[string]any{"type": "response.metadata", "headers": meta})
+ out = append(out, b)
+ }
+ if v := w.h.Get("X-Models-Etag"); v != "" {
+ b, _ := json.Marshal(map[string]any{"type": "codex.response.metadata", "headers": map[string]string{"x-models-etag": v}})
+ out = append(out, b)
+ }
+ if f := codexLimitsFrame(w.h); f != nil {
+ out = append(out, f)
+ }
+ return out
+}
+
+// codexLimitsFrame is the codex.rate_limits event the x-codex-* headers
+// of a reply say (codexLimitHeaders, the other way), nil for none.
+func codexLimitsFrame(h http.Header) []byte {
+ windows := map[string]any{}
+ for _, which := range []string{"primary", "secondary"} {
+ used := h.Get("x-codex-" + which + "-used-percent")
+ p, err := strconv.ParseFloat(used, 64)
+ if err != nil {
+ continue
+ }
+ w := map[string]any{"used_percent": p}
+ if v, err := strconv.ParseInt(h.Get("x-codex-"+which+"-window-minutes"), 10, 64); err == nil {
+ w["window_minutes"] = v
+ }
+ if v, err := strconv.ParseInt(h.Get("x-codex-"+which+"-reset-at"), 10, 64); err == nil {
+ w["reset_at"] = v
+ }
+ windows[which] = w
+ }
+ if len(windows) == 0 {
+ return nil
+ }
+ ev := map[string]any{"type": "codex.rate_limits", "rate_limits": windows}
+ if p := h.Get("x-codex-plan-type"); p != "" {
+ ev["plan_type"] = p
+ }
+ has, unl := h.Get("x-codex-credits-has-credits"), h.Get("x-codex-credits-unlimited")
+ if has != "" && unl != "" {
+ cr := map[string]any{"has_credits": strings.EqualFold(has, "true"), "unlimited": strings.EqualFold(unl, "true")}
+ if b := h.Get("x-codex-credits-balance"); b != "" {
+ cr["balance"] = b
+ }
+ ev["credits"] = cr
+ }
+ b, _ := json.Marshal(ev)
+ return b
+}
+
+// event is one event of the reply's stream.
+func (w *socketTurnWriter) event(data []byte) error {
+ var e struct {
+ Type string `json:"type"`
+ Response json.RawMessage `json:"response"`
+ Item json.RawMessage `json:"item"`
+ }
+ if json.Unmarshal(data, &e) != nil {
+ return nil // not JSON: Codex would skip it too
+ }
+ switch e.Type {
+ case "response.created", "response.in_progress":
+ w.note(gjson.GetBytes(e.Response, "id").String())
+ case "response.output_item.done":
+ // in the order they came, as Codex adds them to its side
+ if len(e.Item) > 0 {
+ w.items = append(w.items, append(json.RawMessage(nil), e.Item...))
+ }
+ case "response.completed":
+ w.note(gjson.GetBytes(e.Response, "id").String())
+ w.ended, w.completed = true, true
+ case "response.incomplete", "response.failed", "response.cancelled", "error":
+ w.ended = true
+ case "codex.response.metadata":
+ // the model list's tag as magpie gives it (modelsEtag), which the
+ // reply's header has
+ if v := w.h.Get("X-Models-Etag"); v != "" && gjson.GetBytes(data, "headers.x-models-etag").Exists() {
+ if b, ok := withModelsEtag(data, v); ok {
+ data = b
+ }
+ }
+ }
+ return w.send(data)
+}
+
+func (w *socketTurnWriter) note(id string) {
+ if id == "" {
+ return
+ }
+ w.id = id
+ w.run.mu.Lock()
+ for _, have := range w.run.ids {
+ if have == id {
+ w.run.mu.Unlock()
+ return
+ }
+ }
+ w.run.ids = append(w.run.ids, id)
+ w.run.mu.Unlock()
+}
+
+func (w *socketTurnWriter) send(frame []byte) error { return w.rs.write(w.ctx, frame) }
+
+// finish is the end of the handler's reply.
+func (w *socketTurnWriter) finish() {
+ if w.status == 0 {
+ w.WriteHeader(http.StatusOK)
+ }
+ if w.stream {
+ w.pw.Close()
+ <-w.read
+ return
+ }
+ if w.ctx.Err() != nil {
+ return
+ }
+ if w.status/100 != 2 {
+ w.sendHTTPError()
+ return
+ }
+ // a whole reply: what its stream would have said
+ body := bytes.TrimSpace(w.buf.Bytes())
+ r := gjson.ParseBytes(body)
+ if r.Get("object").String() != "response" || r.Get("id").String() == "" {
+ w.status = http.StatusBadGateway
+ w.buf.Reset()
+ w.buf.WriteString(`{"error":{"type":"server_error","message":"the reply was neither a stream nor a response"}}`)
+ w.sendHTTPError()
+ return
+ }
+ for _, f := range w.headerFrames() {
+ w.send(f)
+ }
+ w.event([]byte(`{"type":"response.created","response":` + string(body) + `}`))
+ for i, it := range r.Get("output").Array() {
+ ev, _ := json.Marshal(map[string]any{"type": "response.output_item.done", "output_index": i, "item": json.RawMessage(it.Raw)})
+ w.event(ev)
+ }
+ kind := "response.completed"
+ switch r.Get("status").String() {
+ case "incomplete":
+ kind = "response.incomplete"
+ case "failed":
+ kind = "response.failed"
+ }
+ w.event([]byte(`{"type":"` + kind + `","response":` + string(body) + `}`))
+}
+
+// sendHTTPError is an error reply as Codex reads it on a socket: the
+// status, the error object and the headers it acts on (Retry-After, the
+// account's x-codex-* windows).
+func (w *socketTurnWriter) sendHTTPError() {
+ w.ended = true
+ body := bytes.TrimSpace(w.buf.Bytes())
+ var e json.RawMessage
+ if v := gjson.GetBytes(body, "error"); v.IsObject() {
+ e = json.RawMessage(v.Raw)
+ } else {
+ msg := string(body)
+ if !utf8.Valid(body) || msg == "" {
+ msg = http.StatusText(w.status)
+ }
+ e, _ = json.Marshal(map[string]string{"type": "server_error", "message": msg})
+ }
+ headers := map[string]string{}
+ for k := range w.h {
+ lk := strings.ToLower(k)
+ if lk == "retry-after" || lk == "retry-after-ms" || strings.HasPrefix(lk, "x-codex-") ||
+ strings.HasPrefix(lk, "x-ratelimit-") || lk == "x-request-id" || lk == "x-error-json" {
+ headers[lk] = w.h.Get(k)
+ }
+ }
+ frame := map[string]any{"type": "error", "status": w.status, "error": e}
+ if len(headers) > 0 {
+ frame["headers"] = headers
+ }
+ b, _ := json.Marshal(frame)
+ w.send(b)
+}
+
+// withModelsEtag is a codex.response.metadata event with its
+// x-models-etag set to v, every other member as it was.
+func withModelsEtag(data []byte, v string) ([]byte, bool) {
+ var top map[string]json.RawMessage
+ var h map[string]json.RawMessage
+ if json.Unmarshal(data, &top) != nil || json.Unmarshal(top["headers"], &h) != nil {
+ return nil, false
+ }
+ h["x-models-etag"], _ = json.Marshal(v)
+ top["headers"], _ = json.Marshal(h)
+ b, err := json.Marshal(top)
+ return b, err == nil
+}
diff --git a/internal/gateway/responses_ws_test.go b/internal/gateway/responses_ws_test.go
new file mode 100644
index 000000000..d1e7adb1e
--- /dev/null
+++ b/internal/gateway/responses_ws_test.go
@@ -0,0 +1,674 @@
+package gateway
+
+import (
+ "context"
+ "fmt"
+ "io"
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "sync"
+ "testing"
+ "time"
+
+ "github.com/coder/websocket"
+ "github.com/tidwall/gjson"
+
+ "github.com/yetone/magpie/internal/access"
+ "github.com/yetone/magpie/internal/budget"
+ "github.com/yetone/magpie/internal/codexcat"
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// respUp is a Responses provider that answers each turn with one message
+// and a usage of 300 in and 50 out, keeping the bodies it was sent.
+type respUp struct {
+ mu sync.Mutex
+ bodies [][]byte
+ head []http.Header
+ reply func(w http.ResponseWriter, r *http.Request, n int) // nil: the usual
+}
+
+func (u *respUp) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+ b, _ := io.ReadAll(r.Body)
+ u.mu.Lock()
+ u.bodies = append(u.bodies, b)
+ u.head = append(u.head, r.Header.Clone())
+ n := len(u.bodies)
+ reply := u.reply
+ u.mu.Unlock()
+ if reply != nil {
+ reply(w, r, n)
+ return
+ }
+ w.Header().Set("Content-Type", "text/event-stream")
+ id := fmt.Sprintf("resp_up%d", n)
+ item := fmt.Sprintf(`{"type":"message","id":"msg_%d","role":"assistant","status":"completed","content":[{"type":"output_text","text":"answer %d"}]}`, n, n)
+ io.WriteString(w, sse(
+ `data: {"type":"response.created","response":{"id":"`+id+`","object":"response","status":"in_progress","output":[]}}`,
+ `data: {"type":"response.output_item.done","output_index":0,"item":`+item+`}`,
+ `data: {"type":"response.completed","response":{"id":"`+id+`","object":"response","status":"completed","output":[`+item+`],"usage":{"input_tokens":300,"output_tokens":50,"total_tokens":350}}}`))
+}
+
+func (u *respUp) calls() int {
+ u.mu.Lock()
+ defer u.mu.Unlock()
+ return len(u.bodies)
+}
+
+func (u *respUp) body(i int) []byte {
+ u.mu.Lock()
+ defer u.mu.Unlock()
+ return u.bodies[i]
+}
+
+// sockUp saves a Responses provider served by u.
+func sockUp(t *testing.T, u *respUp) {
+ t.Helper()
+ up := httptest.NewServer(u)
+ t.Cleanup(up.Close)
+ if err := provider.Save(provider.Provider{ID: "fake", Name: "Fake", Key: "k", Responses: up.URL + "/v1", Models: []string{"m1"}}); err != nil {
+ t.Fatal(err)
+ }
+}
+
+// sockServer serves the gateway as the real server does (Front), where a
+// request's address and Host can be set by the test (X-Test-Remote,
+// X-Test-Host): a test's dial comes from 127.0.0.1 to the server's own.
+func sockServer(t *testing.T) (*Server, *httptest.Server) {
+ t.Helper()
+ s := New()
+ front := s.Front()
+ ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ if v := r.Header.Get("X-Test-Remote"); v != "" {
+ r.RemoteAddr = v
+ }
+ if v := r.Header.Get("X-Test-Host"); v != "" {
+ r.Host = v
+ }
+ r.Header.Del("X-Test-Remote")
+ r.Header.Del("X-Test-Host")
+ front.ServeHTTP(w, r)
+ }))
+ t.Cleanup(ts.Close)
+ // a socket's handler outlives ts.Close (hijacked): it ends before
+ // the test's folders go
+ t.Cleanup(func() {
+ ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+ defer cancel()
+ s.sockets.drain(ctx)
+ })
+ return s, ts
+}
+
+type sockClient struct {
+ t *testing.T
+ c *websocket.Conn
+}
+
+func dialSock(t *testing.T, ts *httptest.Server, path string, h http.Header) (*sockClient, *http.Response, error) {
+ t.Helper()
+ ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+ defer cancel()
+ c, resp, err := websocket.Dial(ctx, "ws"+strings.TrimPrefix(ts.URL, "http")+path, &websocket.DialOptions{HTTPHeader: h})
+ if err != nil {
+ return nil, resp, err
+ }
+ c.SetReadLimit(64 << 20)
+ t.Cleanup(func() { c.CloseNow() })
+ return &sockClient{t: t, c: c}, resp, nil
+}
+
+func (s *sockClient) send(frame string) {
+ s.t.Helper()
+ ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+ defer cancel()
+ if err := s.c.Write(ctx, websocket.MessageText, []byte(frame)); err != nil {
+ s.t.Fatal(err)
+ }
+}
+
+// until reads frames up to and with one of the given types, or until the
+// socket ends (err).
+func (s *sockClient) until(kinds ...string) ([]gjson.Result, error) {
+ ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+ defer cancel()
+ var out []gjson.Result
+ for {
+ typ, b, err := s.c.Read(ctx)
+ if err != nil {
+ return out, err
+ }
+ if typ != websocket.MessageText {
+ s.t.Fatalf("a %v frame", typ)
+ }
+ r := gjson.ParseBytes(b)
+ out = append(out, r)
+ for _, k := range kinds {
+ if r.Get("type").String() == k {
+ return out, nil
+ }
+ }
+ }
+}
+
+func (s *sockClient) turn(frame string) []gjson.Result {
+ s.t.Helper()
+ s.send(frame)
+ got, err := s.until("response.completed", "response.failed", "response.incomplete", "error")
+ if err != nil {
+ s.t.Fatalf("%v after %v", err, got)
+ }
+ return got
+}
+
+func last(rs []gjson.Result) gjson.Result { return rs[len(rs)-1] }
+
+const userHi = `{"type":"message","role":"user","content":[{"type":"input_text","text":"hi"}]}`
+const userMore = `{"type":"message","role":"user","content":[{"type":"input_text","text":"and more"}]}`
+
+func create(prev, input string, extra ...string) string {
+ f := `{"type":"response.create","model":"fake/m1","stream":true,"store":false,"instructions":"be brief","input":[` + input + `]`
+ if prev != "" {
+ f += `,"previous_response_id":"` + prev + `"`
+ }
+ for _, e := range extra {
+ f += "," + e
+ }
+ return f + "}"
+}
+
+// Codex's socket is served (#4058): the upgrade is taken on each
+// Responses route; a turn naming the reply before it, with only what
+// came after, reaches the provider as the whole conversation, the reply
+// in its place, and none of the socket's own members. Over HTTP Codex
+// sends the conversation whole each turn; magpie answered the upgrade
+// 426 (a87dc1ff2), so it never sent less.
+func TestResponsesSocketContinues(t *testing.T) {
+ fresh(t)
+ for _, path := range []string{"/v1/responses", "/responses"} {
+ u := &respUp{}
+ sockUp(t, u)
+ _, ts := sockServer(t)
+ c, resp, err := dialSock(t, ts, path, nil)
+ if err != nil {
+ code := 0
+ if resp != nil {
+ code = resp.StatusCode
+ }
+ t.Fatalf("%s: %v (%d)", path, err, code)
+ }
+ got := c.turn(create("", userHi))
+ end := last(got)
+ if end.Get("type").String() != "response.completed" {
+ t.Fatalf("%s turn 1: %v", path, got)
+ }
+ id := end.Get("response.id").String()
+ if id == "" || got[0].Get("type").String() != "response.created" {
+ t.Fatalf("%s turn 1: %v", path, got)
+ }
+ var done bool
+ for _, f := range got {
+ done = done || f.Get("type").String() == "response.output_item.done"
+ }
+ if !done {
+ t.Errorf("%s: no item: %v", path, got)
+ }
+ got = c.turn(create(id, userMore))
+ if last(got).Get("type").String() != "response.completed" {
+ t.Fatalf("%s turn 2: %v", path, got)
+ }
+ if u.calls() != 2 {
+ t.Fatalf("%s: %d calls", path, u.calls())
+ }
+ b := u.body(1)
+ in := gjson.GetBytes(b, "input").Array()
+ if len(in) != 3 || in[0].Get("content.0.text").String() != "hi" || in[1].Get("role").String() != "assistant" ||
+ in[1].Get("content.0.text").String() != "answer 1" || in[2].Get("content.0.text").String() != "and more" {
+ t.Errorf("%s turn 2 upstream input: %s", path, b)
+ }
+ for _, k := range []string{"type", "previous_response_id", "generate"} {
+ if gjson.GetBytes(b, k).Exists() {
+ t.Errorf("%s: %s went upstream: %s", path, k, b)
+ }
+ }
+ if gjson.GetBytes(b, "instructions").String() != "be brief" {
+ t.Errorf("%s: %s", path, b)
+ }
+ // a third turn continues the second
+ got = c.turn(create(last(got).Get("response.id").String(), userHi))
+ if last(got).Get("type").String() != "response.completed" || len(gjson.GetBytes(u.body(2), "input").Array()) != 5 {
+ t.Errorf("%s turn 3: %v %s", path, got, u.body(2))
+ }
+ }
+}
+
+// A plain GET on a Responses route is still told to POST.
+func TestResponsesSocketPlainGET(t *testing.T) {
+ fresh(t)
+ sockUp(t, &respUp{})
+ for _, path := range []string{"/v1/responses", "/responses"} {
+ rec := httptest.NewRecorder()
+ New().Handler().ServeHTTP(rec, httptest.NewRequest("GET", path, nil))
+ if rec.Code != http.StatusMethodNotAllowed || rec.Header().Get("Allow") != "POST" {
+ t.Errorf("%s GET: %d %q", path, rec.Code, rec.Header().Get("Allow"))
+ }
+ }
+}
+
+// A reply the socket doesn't hold is previous_response_not_found, on
+// which Codex opens a new socket with the conversation whole; nothing goes
+// upstream. So is one from another socket: each holds its own.
+func TestResponsesSocketUnknownPrevious(t *testing.T) {
+ fresh(t)
+ u := &respUp{}
+ sockUp(t, u)
+ _, ts := sockServer(t)
+ a, _, err := dialSock(t, ts, "/v1/responses", nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ got := a.turn(create("", userHi))
+ id := last(got).Get("response.id").String()
+ b, _, err := dialSock(t, ts, "/v1/responses", nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, prev := range []string{"resp_never", id} {
+ got = b.turn(create(prev, userMore))
+ e := last(got)
+ if e.Get("type").String() != "error" || e.Get("status").Int() != 400 || e.Get("error.code").String() != "previous_response_not_found" {
+ t.Errorf("%s: %v", prev, got)
+ }
+ }
+ if u.calls() != 1 {
+ t.Errorf("%d calls", u.calls())
+ }
+ // the socket goes on
+ if got = b.turn(create("", userHi)); last(got).Get("type").String() != "response.completed" {
+ t.Errorf("after: %v", got)
+ }
+}
+
+// A warmup (generate false) is answered without a provider; the turn
+// after it names it and sends only what came after, and the provider gets
+// the whole.
+func TestResponsesSocketWarmup(t *testing.T) {
+ fresh(t)
+ u := &respUp{}
+ sockUp(t, u)
+ _, ts := sockServer(t)
+ c, _, err := dialSock(t, ts, "/v1/responses", nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ got := c.turn(create("", userHi, `"generate":false`))
+ if len(got) != 2 || got[0].Get("type").String() != "response.created" || last(got).Get("type").String() != "response.completed" {
+ t.Fatalf("warmup: %v", got)
+ }
+ if u.calls() != 0 {
+ t.Fatalf("a warmup went upstream")
+ }
+ got = c.turn(create(last(got).Get("response.id").String(), userMore))
+ if last(got).Get("type").String() != "response.completed" || u.calls() != 1 {
+ t.Fatalf("turn: %v", got)
+ }
+ in := gjson.GetBytes(u.body(0), "input").Array()
+ if len(in) != 2 || in[0].Get("content.0.text").String() != "hi" || in[1].Get("content.0.text").String() != "and more" {
+ t.Errorf("upstream: %s", u.body(0))
+ }
+}
+
+// A refused turn is an error frame with the HTTP status and error object,
+// as Codex reads one; the socket goes on, and the refused turn isn't one a
+// later turn can name.
+func TestResponsesSocketUpstreamError(t *testing.T) {
+ fresh(t)
+ u := &respUp{}
+ u.reply = func(w http.ResponseWriter, r *http.Request, n int) {
+ w.Header().Set("Content-Type", "application/json")
+ w.Header().Set("X-Request-Id", "req-bad")
+ w.WriteHeader(http.StatusBadRequest)
+ io.WriteString(w, `{"error":{"type":"invalid_request_error","code":"context_length_exceeded","message":"too long for this model"}}`)
+ }
+ sockUp(t, u)
+ _, ts := sockServer(t)
+ c, _, err := dialSock(t, ts, "/v1/responses", nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ got := c.turn(create("", userHi))
+ e := last(got)
+ if e.Get("type").String() != "error" || e.Get("status").Int() != 400 || !strings.Contains(e.Get("error.message").String(), "too long") {
+ t.Fatalf("%v", got)
+ }
+ u.mu.Lock()
+ u.reply = nil
+ u.mu.Unlock()
+ if got = c.turn(create("", userHi)); last(got).Get("type").String() != "response.completed" {
+ t.Errorf("after: %v", got)
+ }
+}
+
+// An interrupt of the reply in flight stops it: response.interrupt
+// .accepted and an interrupted response.incomplete, as OpenAI's socket
+// answers Codex; the provider's request is cancelled, and the socket
+// serves the next turn. One naming another reply fails.
+func TestResponsesSocketInterrupt(t *testing.T) {
+ fresh(t)
+ u := &respUp{}
+ gone := make(chan struct{}, 1)
+ u.reply = func(w http.ResponseWriter, r *http.Request, n int) {
+ w.Header().Set("Content-Type", "text/event-stream")
+ io.WriteString(w, sse(`data: {"type":"response.created","response":{"id":"resp_long","object":"response","status":"in_progress","output":[]}}`,
+ `data: {"type":"response.output_item.added","output_index":0,"item":{"type":"message","id":"msg_long","role":"assistant","status":"in_progress","content":[]}}`,
+ `data: {"type":"response.output_text.delta","output_index":0,"content_index":0,"item_id":"msg_long","delta":"wor"}`))
+ w.(http.Flusher).Flush()
+ <-r.Context().Done()
+ gone <- struct{}{}
+ }
+ sockUp(t, u)
+ _, ts := sockServer(t)
+ c, _, err := dialSock(t, ts, "/v1/responses", nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ c.send(create("", userHi))
+ if got, err := c.until("response.created"); err != nil || last(got).Get("response.id").String() != "resp_long" {
+ t.Fatalf("%v %v", got, err)
+ }
+ c.send(`{"type":"response.interrupt","response_id":"resp_other","mode":"discard_partial_items"}`)
+ got, err := c.until("response.interrupt.failed")
+ if err != nil || last(got).Get("response_id").String() != "resp_other" {
+ t.Fatalf("%v %v", got, err)
+ }
+ c.send(`{"type":"response.interrupt","response_id":"resp_long","mode":"discard_partial_items"}`)
+ got, err = c.until("response.incomplete")
+ if err != nil {
+ t.Fatalf("%v %v", got, err)
+ }
+ var accepted bool
+ for _, f := range got {
+ accepted = accepted || (f.Get("type").String() == "response.interrupt.accepted" && f.Get("response_id").String() == "resp_long")
+ }
+ end := last(got)
+ if !accepted || end.Get("response.id").String() != "resp_long" || end.Get("response.incomplete_details.reason").String() != "interrupted" {
+ t.Fatalf("%v", got)
+ }
+ select {
+ case <-gone:
+ case <-time.After(30 * time.Second):
+ t.Fatal("the provider's request went on")
+ }
+ u.mu.Lock()
+ u.reply = nil
+ u.mu.Unlock()
+ if got = c.turn(create("", userHi)); last(got).Get("type").String() != "response.completed" {
+ t.Errorf("after: %v", got)
+ }
+}
+
+// A reply cut before its end is a failed turn, as the HTTP route tells
+// it; the socket goes on, and the failed reply isn't one to name.
+func TestResponsesSocketCutReplyFails(t *testing.T) {
+ fresh(t)
+ u := &respUp{}
+ u.reply = func(w http.ResponseWriter, r *http.Request, n int) {
+ w.Header().Set("Content-Type", "text/event-stream")
+ io.WriteString(w, sse(`data: {"type":"response.created","response":{"id":"resp_cut","object":"response","status":"in_progress","output":[]}}`,
+ `data: {"type":"response.output_text.delta","delta":"par"}`))
+ }
+ sockUp(t, u)
+ _, ts := sockServer(t)
+ c, _, err := dialSock(t, ts, "/v1/responses", nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ got := c.turn(create("", userHi))
+ if last(got).Get("type").String() != "response.failed" {
+ t.Fatalf("%v", got)
+ }
+ got = c.turn(create("resp_cut", userMore))
+ if last(got).Get("error.code").String() != "previous_response_not_found" {
+ t.Errorf("%v", got)
+ }
+}
+
+// Everything that judges an HTTP turn judges each turn on the socket: a
+// remote caller needs a key to open it, a key turned off refuses the next
+// turn, a key's limit counts each turn; a web page elsewhere, a sandboxed
+// frame (Origin null), a localhost subdomain and a rebound name can't open
+// one at all.
+func TestResponsesSocketGuards(t *testing.T) {
+ fresh(t)
+ t.Setenv("MAGPIE_ADDR", "")
+ t.Setenv("MAGPIE_PUBLIC_URL", "")
+ budget.Forget()
+ pinLimitClock(t)
+ u := &respUp{}
+ sockUp(t, u)
+ keys, secrets := newCaller(t, "Laptop", "Capped")
+ _, ts := sockServer(t)
+ remote := func(secret string) http.Header {
+ h := http.Header{"X-Test-Remote": {"192.168.1.9:5000"}}
+ if secret != "" {
+ h.Set("Authorization", "Bearer "+secret)
+ }
+ return h
+ }
+ refused := func(what string, h http.Header) {
+ t.Helper()
+ c, resp, err := dialSock(t, ts, "/v1/responses", h)
+ if err == nil {
+ // opened: its turn must be refused, never served
+ got := c.turn(create("", userHi))
+ t.Errorf("%s opened a socket: %v", what, got)
+ return
+ }
+ if resp == nil || resp.StatusCode < 400 {
+ t.Errorf("%s: %v %v", what, err, resp)
+ }
+ }
+ before := u.calls()
+ refused("a remote caller without a key", remote(""))
+ refused("a remote caller with a wrong key", remote("sk-magpie-wrong"))
+ for _, origin := range []string{"https://evil.example", "null", "http://evil.localhost:3425"} {
+ refused("Origin "+origin, http.Header{"Origin": {origin}})
+ }
+ refused("a rebound page", http.Header{
+ "X-Test-Host": {"rebind.attacker.example:3425"},
+ "User-Agent": {"Mozilla/5.0 (Macintosh) Chrome/140"}, "Sec-Fetch-Site": {"same-origin"}, "Sec-Fetch-Mode": {"websocket"},
+ })
+ if u.calls() != before {
+ t.Fatalf("a refused socket reached the provider")
+ }
+
+ // a key: served, until it is turned off
+ c, _, err := dialSock(t, ts, "/v1/responses", remote(secrets[0]))
+ if err != nil {
+ t.Fatal(err)
+ }
+ got := c.turn(create("", userHi))
+ if last(got).Get("type").String() != "response.completed" {
+ t.Fatalf("keyed: %v", got)
+ }
+ if h := u.head[len(u.head)-1].Get("Authorization"); h != "Bearer k" {
+ t.Errorf("the provider got %q", h)
+ }
+ if _, err := access.Update("off-key", access.Change{Key: keys[0].ID}); err != nil {
+ t.Fatal(err)
+ }
+ got = c.turn(create(last(got).Get("response.id").String(), userMore))
+ if e := last(got); e.Get("type").String() != "error" || e.Get("status").Int() != http.StatusUnauthorized {
+ t.Errorf("a key turned off: %v", got)
+ }
+ if u.calls() != 1 {
+ t.Errorf("%d calls", u.calls())
+ }
+
+ // a limit counts each turn: 350 tokens a turn, 700 allowed
+ setLimit(t, keys[1].ID, &access.Limit{Period: "day", Tokens: 700})
+ c, _, err = dialSock(t, ts, "/v1/responses", remote(secrets[1]))
+ if err != nil {
+ t.Fatal(err)
+ }
+ prev := ""
+ for i := 0; i < 2; i++ {
+ got = c.turn(create(prev, userHi))
+ if last(got).Get("type").String() != "response.completed" {
+ t.Fatalf("turn %d: %v", i, got)
+ }
+ prev = last(got).Get("response.id").String()
+ }
+ got = c.turn(create(prev, userMore))
+ if e := last(got); e.Get("type").String() != "error" || e.Get("status").Int() != http.StatusTooManyRequests {
+ t.Errorf("past the limit: %v", got)
+ }
+ if u.calls() != 3 {
+ t.Errorf("%d calls", u.calls())
+ }
+}
+
+// When the gateway ends, a socket is closed going away after its turn,
+// and Codex opens a new one to the gateway serving the port then.
+func TestResponsesSocketDrains(t *testing.T) {
+ fresh(t)
+ sockUp(t, &respUp{})
+ s, ts := sockServer(t)
+ c, _, err := dialSock(t, ts, "/v1/responses", nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if got := c.turn(create("", userHi)); last(got).Get("type").String() != "response.completed" {
+ t.Fatal(got)
+ }
+ // Codex reads all along, the close's answer too
+ read := make(chan error, 1)
+ go func() {
+ _, err := c.until("never")
+ read <- err
+ }()
+ ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+ defer cancel()
+ s.sockets.drain(ctx)
+ if ctx.Err() != nil {
+ t.Fatal("drain waited out")
+ }
+ if err := <-read; websocket.CloseStatus(err) != websocket.StatusGoingAway {
+ t.Errorf("%v", err)
+ }
+ if _, resp, err := dialSock(t, ts, "/v1/responses", nil); err == nil || resp == nil || resp.StatusCode != http.StatusServiceUnavailable {
+ t.Errorf("a socket while the gateway ends: %v %v", err, resp)
+ }
+}
+
+// Codex's own route serves its socket too: each turn goes to the ChatGPT
+// backend as Codex's HTTP turn would, the sign-in, the turn state and the
+// whole conversation with it, and the reply's headers come back as the
+// frames Codex reads them from on a socket (response.metadata,
+// codex.response.metadata with magpie's model-list tag, codex.rate_limits).
+func TestCodexSocketServed(t *testing.T) {
+ setup(t, provider.Chat, &fake{t: t})
+ var mu sync.Mutex
+ var bodies [][]byte
+ var heads []http.Header
+ n := 0
+ chatgpt(t, func(w http.ResponseWriter, r *http.Request) {
+ if r.Method != http.MethodPost {
+ t.Errorf("%s %s", r.Method, r.URL.Path)
+ return
+ }
+ b, _ := io.ReadAll(r.Body)
+ mu.Lock()
+ bodies = append(bodies, b)
+ heads = append(heads, r.Header.Clone())
+ n++
+ id := fmt.Sprintf("resp_c%d", n)
+ mu.Unlock()
+ w.Header().Set("Content-Type", "text/event-stream")
+ w.Header().Set("X-Models-Etag", `W/"v1"`)
+ w.Header().Set("X-Codex-Turn-State", "ts-"+id)
+ w.Header().Set("X-Codex-Primary-Used-Percent", "12.5")
+ w.Header().Set("X-Codex-Primary-Window-Minutes", "300")
+ item := `{"type":"message","role":"assistant","content":[{"type":"output_text","text":"ok"}]}`
+ io.WriteString(w, sse(
+ `data: {"type":"response.created","response":{"id":"`+id+`"}}`,
+ `data: {"type":"codex.response.metadata","headers":{"x-models-etag":"W/\"v1\"","x-other":"kept"}}`,
+ `data: {"type":"response.output_item.done","output_index":0,"item":`+item+`}`,
+ `data: {"type":"response.completed","response":{"id":"`+id+`","usage":{"input_tokens":9,"output_tokens":2}}}`))
+ })
+ _, ts := sockServer(t)
+ h := http.Header{"Authorization": {"Bearer chatgpt-token"}, "Chatgpt-Account-Id": {"acct-1"},
+ "Openai-Beta": {"responses_websockets=2026-02-06"}}
+ c, resp, err := dialSock(t, ts, CodexPath+"/responses", h)
+ if err != nil {
+ code := 0
+ if resp != nil {
+ code = resp.StatusCode
+ }
+ t.Fatalf("%v (%d)", err, code)
+ }
+ frame := func(prev, input string) string {
+ f := `{"type":"response.create","model":"gpt-5.5","stream":true,"store":false,"input":[` + input + `],` +
+ `"client_metadata":{"x-codex-turn-state":"ts-prev","ws_request_header_x_openai_internal_codex_responses_lite":"true"}`
+ if prev != "" {
+ f += `,"previous_response_id":"` + prev + `"`
+ }
+ return f + "}"
+ }
+ got := c.turn(frame("", userHi))
+ if last(got).Get("type").String() != "response.completed" {
+ t.Fatalf("%v", got)
+ }
+ tag := codexcat.Tag(provider.CodexListed())
+ var meta, etag, limits bool
+ for _, f := range got {
+ switch f.Get("type").String() {
+ case "response.metadata":
+ meta = f.Get(`headers.x-codex-turn-state`).String() == "ts-resp_c1"
+ case "codex.response.metadata":
+ v := f.Get(`headers.x-models-etag`).String()
+ if !codexcat.Tagged(v, tag) {
+ t.Errorf("x-models-etag %q", v)
+ }
+ etag = etag || f.Get(`headers.x-other`).String() == "kept"
+ case "codex.rate_limits":
+ limits = f.Get("rate_limits.primary.used_percent").Float() == 12.5 && f.Get("rate_limits.primary.window_minutes").Int() == 300
+ }
+ }
+ if !meta || !etag || !limits {
+ t.Errorf("meta %v etag %v limits %v: %v", meta, etag, limits, got)
+ }
+ got = c.turn(frame(last(got).Get("response.id").String(), userMore))
+ if last(got).Get("type").String() != "response.completed" {
+ t.Fatalf("%v", got)
+ }
+ mu.Lock()
+ defer mu.Unlock()
+ if len(bodies) != 2 {
+ t.Fatalf("%d calls", len(bodies))
+ }
+ in := gjson.GetBytes(bodies[1], "input").Array()
+ if len(in) != 3 || in[1].Get("role").String() != "assistant" || gjson.GetBytes(bodies[1], "previous_response_id").Exists() || gjson.GetBytes(bodies[1], "type").Exists() {
+ t.Errorf("turn 2: %s", bodies[1])
+ }
+ hd := heads[1]
+ if hd.Get("Authorization") != "Bearer chatgpt-token" || hd.Get("Chatgpt-Account-Id") != "acct-1" ||
+ hd.Get("X-Codex-Turn-State") != "ts-prev" || strings.Contains(hd.Get("Openai-Beta"), "responses_websockets") {
+ t.Errorf("headers %v", hd)
+ }
+}
+
+// Only Codex's Responses route takes a socket; another of its paths is
+// still told 426.
+func TestCodexSocketOtherPaths(t *testing.T) {
+ chatgpt(t, func(w http.ResponseWriter, r *http.Request) { t.Error("sent to ChatGPT") })
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("GET", CodexPath+"/models", nil)
+ req.Header.Set("Upgrade", "websocket")
+ req.Header.Set("Connection", "Upgrade")
+ New().Handler().ServeHTTP(rec, req)
+ if rec.Code != http.StatusUpgradeRequired {
+ t.Errorf("%d", rec.Code)
+ }
+}