Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
gateway: an error in a kept-alive stream names the model the agent asked for (#3728)
JoeyWu-tech47175a68fffe
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.mdindex d130c3588..8f41350af 100644--- a/docs/subsystems/gateway-routing.md+++ b/docs/subsystems/gateway-routing.md@@ -187,7 +187,7 @@ recorded as a prediction failure (`noPrediction`).- A 2xx from Zhipu or Z.ai holding the platform's refusal envelope (`{"code":1000,"msg":…,"success":false}`, which their Responses API at `/api/v1` answers a refused key) is the refusal it stands for (`provider.ZhipuEnveloped` in `forwardOnce`): 401 for a key refused, 429 for a limit or no balance, as Zhipu's error codes give, so it fails over and rests as those do and the agent is told why. `TestZhipuResponsesRefusalIsRefusal`.- A streamed request is never left silent while its vendor says nothing. `holdWriter.watch` runs beside each try: past `keepHeldAfter` (15s) it sends a held try's agent the stream's 200 and SSE comments (`keepAlive`), none of the try, so another candidate may still answer in the same stream; once the reply passes, it puts a comment between two of its lines after `keepaliveEvery` of quiet. It stops after `keepaliveLongest` (5 minutes) with nothing from the vendor. A translated stream (`streamTranslated`) whose upstream has begun its reply and then says nothing at all, not even a comment (a relay that sends the rest of a long reply in one burst at its end), is kept alive in the client's own protocol by a ticker beside its reader (`keepThroughSilence`): every `keepaliveEvery` it sends the client's keepalive (`enc.keepalive`: a ping for Anthropic, `response.in_progress` for Responses, a comment for Chat, nothing for Gemini), which also counts as the try heard from, so the held stream's comments go on too; it stops once the upstream has been silent for `translatedQuietLongest` (30 minutes), past the 10 minutes `upstreamHeaderWait` gives a vendor to begin and long enough for a long reasoning turn sent whole at its end, after which the client's own timeout ends the wait (youyo.p on Discord: before, Claude Code heard only comments, which its event watchdog doesn't count, and gave up 300s on; `TestTranslatedSilenceKeptAliveForABound`). An upstream that has answered its headers but sent no byte yet is still waited for under the held stream's comments alone (`notAnAPIReply` reads its first bytes before the reply is translated). An Anthropic client is sent a ping at most once each `pingEvery` (1 minute), comments between, and none before `message_start`: Claude Code's watchdog counts only 30 pings in a row as life, and a ping on every upstream keepalive spent them in the first minute; the SDKs' `MessageStream` fails on any event before `message_start` (`anthropicEncoder.keepalive`, `TestTranslatedStreamKeepsClientAlive`). A try waiting for its key's or account's slot or for room in its minute (`lanes.take`, `rpms.wait`) is kept alive the same way (`keepQueued`), for as long as it waits; once the 200 is out, the try is held, and its failure or its queue turning it away reaches the agent as the stream's error (`failTo`), not a status (coeo91 on Discord). A wait shorter than `keepHeldAfter` sends nothing, so its 429 keeps its `Retry-After`. A proxy in front such as Cloudflare (125s) otherwise ends the request (#947). Gemini streams get no comments (#934). A vendor that then ignores `stream:true` and answers 200 with its reply whole has it sent as the agent's stream (`wholeAsStream`: Chat, Responses, Anthropic), not as the stream's error, while a reply under a type that says neither JSON nor a stream is held as a stream when its first bytes are one (`sseStart`), read event by event like one that says so, so an error among its frames still goes to the next candidate (#1418: it went through at once); a 200 that is no reply of the protocol's fails as a translated reply that didn't stream does (502, "… did not stream"), for the next candidate to answer, and the last one's reaches the agent as the stream's error. A held stream that has sent nothing but the frames before a reply (`response.created`, `response.in_progress`, `codex.rate_limits`, pings, an empty part begun, and any event `streamEvent` doesn't know to carry text, reasoning, a call or a reply's end (`replyEvent`), or an item not known that isn't a call or a tool's (`streamPart.said`): `eventLead`; Aliceonly on #1418: an unknown event was taken for content and let a frames-only 200 through, and its `server_is_overloaded` reached Codex with five candidates left) stays held as long as its agent is kept alive, up to `holdLead` (4 minutes, under the 300s Codex waits for an event; `keptQuiet`), so a `server_is_overloaded` that comes after it is a failure the next candidate answers (#1418: let through at 15s, the ChatGPT backend's failure 36s in ended Codex's turn with sub2api, next in the group, never asked). A stream whose agent can't be kept alive (Gemini's) is let through after `holdLongest` (15s), as before. Codex counts only events toward that 300s, not comments, so the bound is the request's, not each try's: once Codex on Responses has heard no event for `agentQuietMost` (4 minutes) since the request began or its last event (`keptAlive.tooQuiet`), the try in hand is let through (`scan`, `keepQuiet`), and no group replan or same-one retry is taken whose pause would end past it (fadenoob on Discord: held tries and replan pauses added up past 300s and Codex reconnected on "idle timeout waiting for SSE"). A translated reply whose upstream stream stops before it said its reply had — an Anthropic one with no stop_reason and no message_stop, a Responses one with no `response.completed`, the same two a relayed one is told by (`sniff.whole`; a Chat stream may end without `[DONE]` and be whole), a Gemini or Code Assist one whose last chunk has no candidate's `finishReason` (`geminiEnds`, #1643: `usageMetadata` is no end, Vertex AI sends it on every chunk; a chunk with no candidate at all is the empty reply's to answer, #667), and a relayed Gemini one told the same way (#1644; it ends in Gemini's own `{"error":{"code":502,…,"status":"UNAVAILABLE"}}` data line from `streamFailure`, `TestGeminiRelayEndedShort`; whatever a relayed stream gets appended, that error or a filled-in `finish_reason`, begins after a blank line, two newlines when the upstream's last write stopped inside a line, so it is an event of its own rather than more of the half-written one, #2446, `TestPassthroughStreamCutMidLine`, `TestGeminiCompatFinishAfterPartialLine`), and a Chat one from Gemini's OpenAI-compatible API (`geminiCompat`) with no `finish_reason`, relayed or translated (`saysItsEnd`; Linka on Discord: the upstream hung up between a whole function call and its `finish_reason`, and Pi threw the call away as "Stream ended without finish_reason"; that API counts the usage only once the generation is done, so when the last chunk carried it only the `finish_reason` was lost and it is filled in, `tool_calls` after a call and `stop` otherwise, as a relayed chunk before `[DONE]` or a translated reply's end, and the reply counts as whole (`chatTail`); only one with no usage at its end is a cut, `TestGeminiCompatRelayEndedShort`, `TestGeminiCompatTranslatedEndedShort`) — ends the turn with "the reply ended before it was complete" (`errEndedShort`, `TestResponsesUpstreamEndedShort`, `TestGeminiUpstreamEndedShort`), asked again in place where the upstream goes on from a prefill (`continuation.go`). A client that asked for no stream is told the same 502 of such a cut (`translate`, `TestNonStreamEndedShort`, `TestGeminiNonStreamEndedShort`), rather than handed what came as its whole reply. A translated reply from an Anthropic or Responses upstream that ended but said nothing (no text, call, search or image) and counted no token at all is no reply either, whatever its end says (`repliesCounted`, #2086: an Anthropic-compatible proxy's 200 ended a Codex turn as `response.completed` with no output, no usage and no served model, and Codex didn't ask again). It is the empty reply's to answer (#667): asked again in place, then failed with "an empty reply", streamed or not, for every client (`TestUncountedEmptyReplyFails`, `TestUncountedEmptyReplyFailsEveryClient`). Claude's own empty `end_turn` counts its prompt and stays a whole reply (`TestWellFormedStreamsStillComplete`). A reply passed through in its own protocol isn't read this way.- A vendor's error keeps its `code` and `param` on its way to an OpenAI-shaped client (`writeVendorError`), where `writeError` used to set both to null: a status error relayed or translated, and a request refused inside a Responses stream. A Responses stream's `error` event or failed response whose error is of type `invalid_request_error` is the request at fault (`requestRefused`, read by its type, not its words): its `Event` carries status 400 and the vendor's error (`Event.Said`), so a client answered whole gets the vendor's 400 with its code and param (the whole-translated path wrote 502 `api_error`), a reply cut by it isn't resumed (`continuation.again`), and a translated stream's error says the vendor's type, code and param (`saidError`), so `holdWriter` reads it as the 400 it is. Before, the hold read magpie's `api_error` as a 502: the same request was sent three times, then to the other accounts, and every one rested (magpie-releases#18: the ChatGPT backend answers `access_programs` naming a program the workspace hasn't got with a 200 stream whose error is `access_program_not_enabled`). Asked once, nobody rests ([`codex_access_programs_test.go`](../../internal/gateway/codex_access_programs_test.go)).-- An error that reaches the agent in a stream already begun can't carry its status, so `streamError` says it in the words (`saysPassing`, #2372, zminds): for a status that may pass (408, 429, 5xx) whose text names none of the words a text-judging agent retries on (Pi decides by its `errorMessage` alone), it appends `(<status>: you can retry your request)`; a request's own fault (400, 401, 404 …) and a text that already says so are left as they were. `TestStreamErrorSaysItPasses` holds Pi's own lists; `TestGroupLastFailureInStreamSaysItPasses` runs a group whose last member fails after the keepalive. A 413 byte cap in that stream says `request_too_large` in the error text, and Chat and Anthropic name that type, so Pi compacts; Gemini keeps its own status shape (`TestStreamedByteCapSaysRequestTooLargeToPi`, `TestStreamErrorByteCapInEveryShape`).+- An error that reaches the agent in a stream already begun can't carry its status, so `streamError` says it in the words (`saysPassing`, #2372, zminds): for a status that may pass (408, 429, 5xx) whose text names none of the words a text-judging agent retries on (Pi decides by its `errorMessage` alone), it appends `(<status>: you can retry your request)`; a request's own fault (400, 401, 404 …) and a text that already says so are left as they were. `TestStreamErrorSaysItPasses` holds Pi's own lists; `TestGroupLastFailureInStreamSaysItPasses` runs a group whose last member fails after the keepalive. A 413 byte cap in that stream says `request_too_large` in the error text, and Chat and Anthropic name that type, so Pi compacts; Gemini keeps its own status shape (`TestStreamedByteCapSaysRequestTooLargeToPi`, `TestStreamErrorByteCapInEveryShape`). Wherever the protocol says a model with the error (Chat's first chunk, Anthropic's `message_start`, Responses' `response.created` and the events after it), it is the model the agent asked for, as `serve` has it, an effort on it kept (`keptAlive.model`; on Codex's own sign-in, the one Codex wrote) (#3728, jilen: with `"model":""` fx took Chat's first chunk for a broken one and stopped there; `TestStreamErrorNamesTheModel`, `TestKeptAliveChatErrorNamesTheModel`, `TestKeptAliveChatOnCodexErrorNamesTheModel`, `TestKeptAliveCodexErrorNamesTheModel`). An error after a reply that already went, a loop cut's, still starts a message of its own after the vendor's, under a new id and no model (#3792).- Claude's, GPT's and Gemini's reasoning is held (`refusesAfterThinking`, up to `holdThinking`, 4 minutes) so that a refusal after it goes to the next candidate unseen (#248). It is held only while one could: on the last candidate a refusal isn't asked again, so their reasoning streams as it comes there, and a refusal after it reaches the agent as the vendor's own. Held there, a lone relay's Claude or GPT showed Claude Code its thinking all at once with the text, and Codex none until then (iTianbao on X). Gemini's streams there too: a reply that only reasoned is asked again in place by `streamTranslated`, the reasoning the agent has kept and the next try's after it (#667), so holding it only kept Codex on a Gemini model with nothing at all, not even `response.created`, for the whole of its thinking (LuckymanJC on Discord; `TestThinkingStreamsWithNoOneElse`'s gemini cases). Any other model's reasoning is never held.- A streamed reply stuck in a loop is ended, not let run to its output limit (#1359, Moody-Sin: WorkBuddy's DeepSeek v4.1 flash reasoned round "Now." "Writing." "Go." "Let me write." "OK." until max tokens while dsh and Pi waited). `holdWriter.Write` feeds every streamed try, passed through or translated, a key's, a built-in subscription's or a plugin's (a moved WorkBuddy's fetch in the plugin host, `TestLoopEndsWorkBuddysReply`), in each of the agent's protocols, to a `loopGuard` ([`loop_guard.go`](../../internal/gateway/loop_guard.go)). It cuts the reasoning and the text, each apart, into units, its lines and sentences (a list's `1.` or a decimal is no sentence's end), compared without case or a closing stop. A unit with no word in it (a letter run with no digit: not a brace, a rule, `0x00`, a hash or a row of numbers) isn't counted. When the last `loopWindow` (256) counted units of either are among no more than `loopDistinct` (24) different ones, the try is cut (`cutLoop`): what came with that write goes out, then, after a blank line (two newlines when that write stopped inside a line, `holdWriter.atLine`, #2446), the protocol's stream error (`streamError`, 502) naming the loop and quoting its units, and the vendor's request is cancelled. It falls back to nobody and rests nobody (`failLoop`); its usage record's error type is `reply_loop` (`usage.LoopErrType`), and Usage counts them per model (`Totals.Loops`), as "{n} looped" beside the model's failures in the Overview's rows and the Requests ranking (#1858, `TestLoopsCountedPerModel`, `usage-loops.test.cjs`). A reply magpie translates into the agent's protocol (`streamTranslated`) whose *reasoning* loops before it has said a text or a call, and isn't a continuation, is asked again once in place instead (`loopReasks`, through `holdWriter.onLoop`; cdredfox on X, WorkBuddy AI's DeepSeek v4 flash): the agent keeps the reasoning it has, the next try's follows it in the same stream, as an empty reply's re-ask does (#667), and only a loop on that try ends the reply with the error (`TestReasoningLoopIsAskedAgain`). A reply passed through in the agent's own protocol is asked again once too, at the fallback loop (`holdWriter.mayReloop`, `letGoLoop`; #2436, lqisgod: pi's Chat stream to WorkBuddy was cut at once): when nothing of the try has reached the agent yet, in any protocol, its held bytes are dropped; when its reasoning is already passing, only a Chat stream goes on, written up to the end of the tripping event, with the next try's chunks after it. An Anthropic or Responses stream already passing still ends with the error, since a second `message_start` or `response.created` can't follow. Each re-ask, in place or at the fallback loop, counts in the usage record's tries. A loop in the text, or after a text or call, is ended at once (`TestLoopAfterTextIsNotAskedAgain`). A cycle need not be exact. Code, tables, lists, logs, CSV and base64 differ line to line or have no words, and don't trip it (`TestLoopGuardLeavesOrdinaryRepliesAlone` feeds magpie's own sources and docs through it). A reply sent whole (no stream, or Gemini without `alt=sse`) isn't read. Settings › Models › Replies › *Stop looping replies* (`noLoopGuard`) turns it off.- A conversation going round one tool call is answered by magpie and sent nowhere (audie on Discord: Codex Desktop on WorkBuddy's deepseek-v4-pro read the same file 308 times in half an hour, and 340 times that evening, 537 credits). `stoppedToolLoop` ([`tool_loop.go`](../../internal/gateway/tool_loop.go)) runs in `serveAgent` before hooks, middleware and routing, and in the Codex backend's passthrough on Codex's own sign-in. It reads the request's own history, so it is per conversation and keeps no state: when the last `toolLoopRepeats` (8) turns are each the same calls (name and arguments, keys in any order, call ids ignored) answered with the same results (with Codex's `Wall time`, `Chunk ID` and `duration_seconds` taken out, `runNoise`) and nothing else from the user, the turn is answered in the agent's own protocol, streamed or whole (`writeLocalText` through `relayStatus`), with a notice naming the call. Any different result, or a word of the user's, starts the count again; the notice is a turn without calls, so the user's next word goes on. A body with fewer than 8 tool results isn't parsed (`mayLoop`), and only the newest 64 items of a long one are (`tailOf`, gjson), ~1ms for a 1.5MB turn (`BenchmarkToolLoopCheck`). The turn is recorded as Codex's title turns are (`answeredHere`): the Routing trace as "Tool loop stopped", and a usage row with provider `magpie`, rejected, its session and an error starting `tool loop:`. A poll that gets the same answer 8 times in a row is stopped too. Settings › Models › Replies › *Stop looping replies* (`noLoopGuard`) turns it off with the reply guard. Verification: `go test -tags nogui ./internal/gateway -run 'TestToolLoop'`.diff --git a/internal/gateway/codex_backend.go b/internal/gateway/codex_backend.goindex 091fd0bbd..264da4ede 100644--- a/internal/gateway/codex_backend.go+++ b/internal/gateway/codex_backend.go@@ -964,6 +964,7 @@ func (s *Server) codexUpstreamOn(w http.ResponseWriter, r *http.Request, rest stwho = "Codex's API key"}model := modelOf(body)+ kept.model = model // only a turn is kept alive (overload)effort = requestEffort(provider.Responses, body)seat := Weighed{ID: "codex", Provider: "openai", Name: "OpenAI", Icon: "openai", Who: who, Kind: "account", Agent: "codex", Model: model}link := s.titlePrompts.observe(r, body, metadata, kind, start)@@ -1031,7 +1032,7 @@ func (s *Server) codexUpstreamOn(w http.ResponseWriter, r *http.Request, rest st// error once it was sent the stream's headers while it waitedfailWith := func(status int, msg string) {if kept.sent {- streamError(w, provider.Responses, status, msg)+ streamError(w, provider.Responses, kept.model, status, msg)return}writeError(w, provider.Responses, status, msg)diff --git a/internal/gateway/fallback.go b/internal/gateway/fallback.goindex f93efcb6d..77b014ccb 100644--- a/internal/gateway/fallback.go+++ b/internal/gateway/fallback.go@@ -1979,7 +1979,9 @@ func (h *holdWriter) cutLoop(b []byte, t loopTrip) (int, error) {} else {io.WriteString(h.w, "\n\n")}- streamError(h.w, proto, http.StatusBadGateway, msg)+ // no model: the reply went ahead under its own id and model, and an+ // error that starts a message of its own after it is #3792+ streamError(h.w, proto, "", http.StatusBadGateway, msg)h.looped, h.ended = msg, trueh.flush()if h.stop != nil {@@ -2231,6 +2233,9 @@ var keepHeldAfter = holdLongest// a reply: the stream's headers and SSE comments, nothing of the reply.type keptAlive struct {proto provider.Protocol+ // model is the one the agent asked for, which an error sent in the+ // stream names (streamError, #3728)+ model stringsent bool // the stream's 200 and headers went outat time.Time // the last comment// told is when the agent was last sent anything of a reply, else when@@ -2508,10 +2513,10 @@ func (h *holdWriter) release() {case h.failure != 0:// a 200 that is no reply of the protocol's (settle)who := cmp.Or(h.header.Get(providerHeader), "the provider")- streamError(h.w, h.alive.proto, h.failure, who+" "+h.failMsg)+ streamError(h.w, h.alive.proto, h.alive.model, h.failure, who+" "+h.failMsg)case h.status >= 400:// an error status, once the agent has a stream: told as its error- streamError(h.w, h.alive.proto, h.status, provider.APIError(h.held.Bytes(), http.StatusText(h.status)))+ streamError(h.w, h.alive.proto, h.alive.model, h.status, provider.APIError(h.held.Bytes(), http.StatusText(h.status)))default:// a reply given whole by a vendor that ignored stream:true,// after the agent was kept alive: sent as the stream it askeddiff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.goindex 6477fc5c9..72a647623 100644--- a/internal/gateway/gateway.go+++ b/internal/gateway/gateway.go@@ -1705,7 +1705,8 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro}// a model asked for at an effort of its own (a Claude Code tier, #536)// is the model, every try of it asked for that effort- asked, askedEffort := askedAt(modelOf(body))+ askedFor := modelOf(body) // as it reaches serve, which an error in its stream names (keptAlive)+ asked, askedEffort := askedAt(askedFor)if askedEffort != "" {body = rewriteModel(body, asked)plain = rewriteModel(plain, asked)@@ -2438,7 +2439,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Prounlisted, unlistedW = nil, nil}// what the tries' held streams sent the agent ahead of a reply (#751)- kept := &keptAlive{proto: from, told: start}+ kept := &keptAlive{proto: from, model: askedFor, told: start}if k := handedOnOf(r); k != nil {// Codex's own sign-in waited for first, with Codex kept alivekept = k@@ -7351,13 +7352,17 @@ func failTo(w http.ResponseWriter, a *keptAlive, proto provider.Protocol, statusif a == nil || !a.sent {return writeError(w, proto, status, msg)}- streamError(w, proto, status, msg)+ streamError(w, proto, a.model, status, msg)return status}// streamError writes an error as an event of the client's stream, its-// headers already sent.-func streamError(w http.ResponseWriter, proto provider.Protocol, status int, msg string) {+// headers already sent. Wherever the protocol says a model with it (Chat's+// first chunk, Anthropic's message_start, Responses' response.created and+// the events after it), it is model, the one the agent asked for, none+// from cutLoop (#3792): with none, fx took Chat's first chunk for a broken+// one and stopped there (#3728).+func streamError(w http.ResponseWriter, proto provider.Protocol, model string, status int, msg string) {sw := newSSEWriter(w)sw.begun = truetext := saysPassing(status, msg)@@ -7376,7 +7381,7 @@ func streamError(w http.ResponseWriter, proto provider.Protocol, status int, msgif byteCap && !ownShape && !alreadySaid {ev.Text = text + " (request_too_large)"}- makeEncoder(proto, sw, &Request{}).event(ev)+ makeEncoder(proto, sw, &Request{Model: model}).event(ev)}// passingSaid matches the words an agent that reads only an error's textdiff --git a/internal/gateway/stream_error_model_test.go b/internal/gateway/stream_error_model_test.gonew file mode 100644index 000000000..9c196eb5b--- /dev/null+++ b/internal/gateway/stream_error_model_test.go@@ -0,0 +1,246 @@+package gateway++import (+ "encoding/json"+ "io"+ "net/http"+ "net/http/httptest"+ "strings"+ "sync"+ "testing"+ "time"++ "github.com/tidwall/gjson"+ "github.com/yetone/magpie/internal/provider"+)++// streamModels is every "model" the data lines of an SSE body hold, at+// any depth.+func streamModels(t *testing.T, body string) []string {+ t.Helper()+ var models []string+ var walk func(v any)+ walk = func(v any) {+ switch v := v.(type) {+ case map[string]any:+ if m, ok := v["model"].(string); ok {+ models = append(models, m)+ }+ for _, x := range v {+ walk(x)+ }+ case []any:+ for _, x := range v {+ walk(x)+ }+ }+ }+ for _, line := range strings.Split(body, "\n") {+ d, ok := strings.CutPrefix(line, "data: ")+ if !ok || d == "[DONE]" {+ continue+ }+ var v any+ if err := json.Unmarshal([]byte(d), &v); err != nil {+ t.Fatalf("a data line that isn't JSON: %q", d)+ }+ walk(v)+ }+ return models+}++// lateUpstream is a vendor that says nothing until the agent has heard a+// keepalive (or 5s), then answers x; with no code, it hangs up with no+// reply, the Go client's "EOF", as chatgpt.com's in #3728.+func lateUpstream(t *testing.T, heard <-chan struct{}, x reply) *httptest.Server {+ t.Helper()+ up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ io.ReadAll(r.Body)+ select {+ case <-heard:+ case <-time.After(5 * time.Second):+ }+ if x.code != 0 {+ w.Header().Set("Content-Type", x.ctype)+ w.WriteHeader(x.code)+ io.WriteString(w, x.body)+ return+ }+ if c, _, err := w.(http.Hijacker).Hijack(); err == nil {+ c.Close()+ }+ }))+ t.Cleanup(up.Close)+ return up+}++// An error sent in a stream already begun names the model the agent+// asked for wherever its protocol says one ahead of it (#3728). Chat's+// first chunk went with "model":"", which fx takes for a broken chunk:+// it stopped there and never read the error after it. Anthropic's+// message_start and Responses' response.created, response.in_progress and+// response.failed had it empty too. Gemini's error says no model.+func TestStreamErrorNamesTheModel(t *testing.T) {+ const asked = "codex/gpt-6.1-sol"+ for _, c := range []struct {+ proto provider.Protocol+ named bool // the protocol says a model ahead of the error+ }{+ {provider.Chat, true},+ {provider.Anthropic, true},+ {provider.Responses, true},+ {provider.Gemini, false},+ } {+ rec := httptest.NewRecorder()+ streamError(rec, c.proto, asked, http.StatusBadGateway, `Post "https://chatgpt.com/backend-api/codex/responses": EOF`)+ body := rec.Body.String()+ models := streamModels(t, body)+ if c.named && len(models) == 0 {+ t.Errorf("%s: no model named: %s", c.proto, body)+ }+ if !c.named && len(models) > 0 {+ t.Errorf("%s: names %q, want none: %s", c.proto, models, body)+ }+ for _, m := range models {+ if m != asked {+ t.Errorf("%s: model %q, want %q: %s", c.proto, m, asked, body)+ }+ }+ if !strings.Contains(body, "EOF") {+ t.Errorf("%s: the error isn't in the stream: %s", c.proto, body)+ }+ }+}++// #3728's case: a Chat stream kept alive while its vendor said nothing,+// then the vendor gone with no reply. The error goes in the stream, and+// the chunk ahead of it names the model the agent asked for; the same+// for each way the vendor can fail after the stream's headers went out:+// an error status, told as it is (holdWriter.release), a 413, told with+// the byte cap's words (failTo), a 200 that isn't a stream, and a JSON+// 200 that is no reply, which release tells as the failure it is. A model+// asked for at an effort (#536) is named with it, as the agent asked.+func TestKeptAliveChatErrorNamesTheModel(t *testing.T) {+ for _, c := range []struct {+ name string+ model string+ x reply+ }{+ {"gone", "a/m", reply{}},+ {"gone, at an effort", "a/m:high", reply{}},+ {"502", "a/m", reply{http.StatusBadGateway, "application/json", `{"error":{"message":"upstream connect error","type":"upstream_error"}}`}},+ {"413", "a/m", reply{http.StatusRequestEntityTooLarge, "application/json", `{"error":{"message":"request entity too large","type":"invalid_request_error"}}`}},+ {"a 200 that isn't a stream", "a/m", reply{http.StatusOK, "text/html", "<html><body>down for maintenance</body></html>"}},+ {"a 200 JSON that isn't a reply", "a/m", reply{http.StatusOK, "application/json", `{"status":"ok"}`}},+ } {+ t.Run(c.name, func(t *testing.T) {+ quietFast(t)+ fresh(t)+ pause := retryPause+ retryPause = time.Millisecond+ t.Cleanup(func() { retryPause = pause })+ heard := make(chan struct{})+ up := lateUpstream(t, heard, c.x)+ if err := provider.Save(provider.Provider{ID: "a", Name: "A", Key: "k", Models: []string{"m"}, Chat: up.URL + "/v1"}); err != nil {+ t.Fatal(err)+ }+ code, body, _ := askChat(t, New(), c.model, heard)+ keptThenFailed(t, code, body, c.model)+ if c.x.code == http.StatusRequestEntityTooLarge && !strings.Contains(body, "request_too_large") {+ t.Fatalf("the byte cap's words aren't in the error: %q", body)+ }+ })+ }+}++// keptThenFailed checks a Chat stream kept alive and then failed: its+// headers went out ahead of the failure, its first chunk names model, and+// the error follows.+func keptThenFailed(t *testing.T, code int, body, model string) {+ t.Helper()+ if code != http.StatusOK || !strings.Contains(body, ": keepalive") {+ t.Fatalf("the stream's headers didn't go out ahead of the failure: %d %q", code, body)+ }+ var first string+ for _, line := range strings.Split(body, "\n") {+ if d, ok := strings.CutPrefix(line, "data: "); ok {+ first = d+ break+ }+ }+ if got := gjson.Get(first, "model"); got.String() != model {+ t.Fatalf("the stream's first chunk names %s, want %s: %q", got.Raw, model, body)+ }+ if !strings.Contains(body, `{"error":`) {+ t.Fatalf("no error in the stream: %q", body)+ }+}++// #3728's own route: fx on magpie's codex provider (Codex's sign-in), its+// Chat turn asked on the ChatGPT backend, which hangs up once fx was kept+// alive, each time it is asked. The error is the report's EOF.+func TestKeptAliveChatOnCodexErrorNamesTheModel(t *testing.T) {+ quietFast(t)+ codexSignedIn(t)+ pause := retryPause+ retryPause = time.Millisecond+ t.Cleanup(func() { retryPause = pause })+ heard := make(chan struct{})+ up := lateUpstream(t, heard, reply{})+ was := provider.CodexBase+ provider.CodexBase = up.URL + "/backend-api/codex"+ t.Cleanup(func() { provider.CodexBase = was })+ code, body, _ := askChat(t, New(), "codex/gpt-5.5", heard)+ keptThenFailed(t, code, body, "codex/gpt-5.5")+ if !strings.Contains(body, "EOF") {+ t.Fatalf("the error isn't the backend's EOF: %q", body)+ }+}++// The same on Codex's own sign-in (codexBackend): the backend overloaded,+// Codex kept alive while the turn waits to be asked again, then the+// backend gone. The response.created and response.failed Codex is sent+// name the model it asked for.+func TestKeptAliveCodexErrorNamesTheModel(t *testing.T) {+ quietFast(t)+ codexSignedIn(t)+ var mu sync.Mutex+ asked := 0+ up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ io.ReadAll(r.Body)+ mu.Lock()+ asked+++ n := asked+ mu.Unlock()+ if n == 1 {+ w.Header().Set("Content-Type", "application/json")+ w.WriteHeader(529)+ io.WriteString(w, overloadedStatus)+ return+ }+ if c, _, err := w.(http.Hijacker).Hijack(); err == nil {+ c.Close()+ }+ }))+ t.Cleanup(up.Close)+ was := provider.CodexBase+ provider.CodexBase = up.URL + "/backend-api/codex"+ t.Cleanup(func() { provider.CodexBase = was })+ // asked again after 500ms, Codex kept alive past keepHeldAfter's 40ms+ pause := retryPause+ retryPause = 500 * time.Millisecond+ t.Cleanup(func() { retryPause = pause })+ code, body := codexPost(t, oneAccountTurn)+ if code != http.StatusOK || !strings.Contains(body, ": keepalive") || !strings.Contains(body, "event: response.failed") {+ t.Fatalf("not kept alive, then failed in the stream: %d %q", code, body)+ }+ models := streamModels(t, body)+ if len(models) == 0 {+ t.Fatalf("no model named: %q", body)+ }+ for _, m := range models {+ if m != "gpt-5.5" {+ t.Fatalf("model %q, want gpt-5.5: %q", m, body)+ }+ }+}diff --git a/internal/gateway/stream_error_passing_test.go b/internal/gateway/stream_error_passing_test.goindex c9e6af441..7b759fe3d 100644--- a/internal/gateway/stream_error_passing_test.go+++ b/internal/gateway/stream_error_passing_test.go@@ -157,7 +157,7 @@ func TestStreamErrorByteCapInEveryShape(t *testing.T) {{provider.Chat, "A: 413 prompt is too long: 300000 tokens > 200000 maximum", "", `"code":"context_length_exceeded"`},} {rec := httptest.NewRecorder()- streamError(rec, c.proto, http.StatusRequestEntityTooLarge, c.msg)+ streamError(rec, c.proto, "a/m", http.StatusRequestEntityTooLarge, c.msg)body := rec.Body.String()if !strings.Contains(body, c.kind) {t.Errorf("%s %q: want %s in %s", c.proto, c.msg, c.kind, body)