Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

gateway: Claude Desktop's model check goes to the model it names; a relay gets a cache key

yetone6e331afb45e8
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.md
index 98811c6df..073737f60 100644
--- a/docs/subsystems/gateway-routing.md
+++ b/docs/subsystems/gateway-routing.md
@@ -270,6 +270,8 @@ recorded as a prediction failure (`noPrediction`).
- Claude Code's auto mode asks the API to review each action ([`automode.go`](../../internal/gateway/automode.go), Anthropic's [auto-mode-classifier-billing](https://code.claude.com/docs/en/auto-mode-classifier-billing)): the `dangerous-tool-use-…` beta, a `safeguards` field, and `safeguard_results` in the reply (a stream's in its `message_delta`). Only Anthropic runs that classifier, so they reach only a route it answers on (`AutoModeReviewed`): the Claude subscription (the bridge hands them to Claude Code's own binary), a Claude model under any vendor's spelling on a provider with a key (Anthropic's API, a relay, Bedrock, Vertex), and anything on `api.anthropic.com`. Another subscription's or a plugin's Claude model (Copilot, Kiro, Cursor…) is that vendor's service and isn't reviewed by its name: Copilot turns `safeguards` away (#1756), the others translate the request, so neither the beta nor `safeguards` is sent there (`TestAutoModeReviewedRoutes`, `TestAutoModeReviewLeftOffOtherSubscriptions`). There the beta, `safeguards` and the results pass as sent, a whole reply's results moved into a stream's `message_delta` (`anthropicWholeEvents`), tool-use ids unchanged. To any other model they are taken off (`serverReview`; a translation never carries them), and magpie never answers a result on Anthropic's behalf. Claude Code then classifies locally and names the gateway in a notice; the wiring turns the review off in its settings while no model it can reach through magpie is reviewed (`claudeReviewEnv`, [agent wiring](agent-wiring.md)), which is the page's remedy for a gateway that can't run the checks (1EFLOPS on X: Codex's model behind magpie). Reviewed there means `AutoModeReviewable`: a reviewed model on a route that reaches the Messages API, since a Claude model asked on a relay's Chat or Responses URL is translated and never carries the review (nexnull on Discord, `TestAutoModeReviewableAsksOnMessages`).
- Locally, Claude Code's classifier is a request of its own on the session's model: no tools, a system prompt asking for `<block>yes</block>` or `<block>no</block>`, the transcript in `<transcript>`, a small `max_tokens`, and no stream; Claude Code 2.1.310 waits 60 seconds an attempt for it. To a model that isn't Claude it goes with `classifierRoom` (2048) added to a limit below it and asked to reason least (`fitAutoModeClassifier`): none, or the lowest of the model's known levels. A model whose levels magpie doesn't know (a relay's own name, gpt6.1-sol) is asked for none too, as magpie's own ask (`Request.OwnEffort`): left to the vendor, a reasoning model reasoned at its default past the 60 seconds and every classifier request ended 499 (CyberCaius on Discord). Its 400 to that effort (`ownEffortRefused`) is asked again at low when it named the level, and without the effort when it named the field ("Unrecognized request argument supplied: reasoning_effort"), remembered in `unfit` until magpie restarts (`TestAutoModeClassifierReasonsLeastOnAnUnknownModel`, `TestAutoModeClassifierEffortRefusedGoesWithout`).
- Claude Code's billing line (`x-anthropic-billing-header: cc_version=…; cc_entrypoint=…; cch=…;`, its own first system block, `cch` changing every turn) goes on only to a Claude model asked on Anthropic's Messages API (`keepsBillingLine` in [`billing_line.go`](../../internal/gateway/billing_line.go): Anthropic's, a relay's, Bedrock's), as Claude Code sent it. Anywhere else it is left out, the rest of the system as sent (`withoutBillingLine`): a vendor's model relayed on Messages (`passthrough`), every translation to Chat, Responses or Gemini (`forwardTranslated`, Claude models too). Kept, a vendor read it as the prompt's first words and its prefix cache missed from byte 0 every turn (#1600, justrun1983: 4% read from DeepSeek's cache behind Command Code's plan). The Claude subscription's run sends its own (`withoutBillingHeader`), another magpie decides for the provider it picks, and Code Assist (Antigravity, Gemini CLI) is sent it as before, which `antigravityTurnsAway` reads (#666). Once Antigravity has turned such a request away, for `agForeseeing` (6 hours) after, a candidate loop with someone left after an Antigravity account skips it for a request that carries the line, without asking it (`antigravityForeseen`, #2156: each Claude Desktop turn waited about a second on the same refusal before the next member). The skip goes in `Fallback` with the hint, nothing rests, and the agent is told the refusal if nobody after it answers but with a quota. The last one left is still asked, and an old refusal foresees nothing, so a block Google lifts is noticed (`TestAntigravityTurnedAwayIsForeseen`). Codex's own identity line for its GPT-5 models ("You are Codex, an agent based on GPT-5", "…a coding agent based on GPT-5", any case) is turned away the same way (`antigravityTurnsCodexAway`): Codex sends its model's prompt as the thread's first developer message, which goes in as the system instruction, and a thread begun on a GPT model keeps it after `/model`, `resume -m` or a fork as an agent's thread, so such a thread on an Antigravity model 429'd until Codex's retry limit while dsh on the same account answered (Jodiezzzzzz on X, 10-27). It gets its own hint (`antigravityCodexHint`: start a new chat on the model, which is sent magpie's own Codex prompt "You are Codex, based on GPT-5.", which Antigravity answers), chosen by `antigravityHint`. Codex sends its `web_search` tool every turn, so an Antigravity model's Codex turn goes through magpie's own search (`searchReply`); its rounds are wrapped in `antigravityRound`, which tells the 429 (or the reply's error event) the same way and marks the attempt, where before the search path returned the bare 429 that Codex retried to its limit.
+- A request translated to a Chat or Responses upstream that carries no `prompt_cache_key` of the client's (every Anthropic client's) gets one for its conversation when the provider is OpenAI's API with a key or one the user added by its address (`ownCacheKey`, `cacheKeyOf` in [`gateway.go`](../../internal/gateway/gateway.go)): a hash, in a UUID's shape, of the session the client names, else of its first user message, so every turn of it has the same key and nothing in it names the client or magpie. A relay can turn a request away without one ("prompt_cache_key is required", CyberCaius's AnyRouter). A vendor's preset, a signed-in account, a plugin, Bedrock and Vertex get none, nor does a provider whose extraBody names `prompt_cache_key` (the user's value is sent). An upstream whose 400 names the field is asked again without it and not sent it again; another 400 is not asked again for it, so an unrelated refusal is tried as often as before.
+- Claude Desktop's check that a gateway model answers (`{"max_tokens":1,"messages":[{"role":"user","content":"."}]}`, sent as it starts and by its settings' Test on the first haiku it is listed, else sonnet, else opus) goes to the model it names (`desktopProbe` in [`desktop.go`](../../internal/gateway/desktop.go)), not to the session's model as a title does: sent there, a failure of the session's model was shown as the haiku rejected (CyberCaius on Discord).
- Antigravity answers a system instruction holding one of a few lines with a 429 "Resource has been exhausted" whatever quota is left, on every model (`agTurnedAway` in [`codeassist.go`](../../internal/gateway/codeassist.go)): Claude Code's billing line, the Claude Agent SDK's identity line, Codex's identity lines for its GPT-5 models (which a Codex thread begun on a GPT model keeps after a switch), and Hermes Agent's earlier default persona. An Antigravity account's system instruction goes with a zero-width space after each such line's first letter (`antigravityCloak`, in `codeAssistBody`), which Google lets through and the model reads the same; every other provider, Gemini CLI's sign-in on the same envelope included, is sent the prompt as written. `magpie provider set antigravity prompt=as-written` (`Provider.PromptAsWritten`) turns it off. The refusal is still told against the system prompt as the agent wrote it (`antigravityTurnedAway`): a 400 with the hint for that prompt, the account not rested, and a group skipping Antigravity for that prompt for 6 hours after a refusal (`antigravityForeseen`).
- Claude Code's advisor (`/advisor`, `--advisor`) is Anthropic's server tool `advisor_20260301`, its `model` named as Claude Code names its own: on magpie, by magpie's id (the tier an alias like `opus` is on), which no vendor knows, so the request was turned away whole (JackChou on Discord). Relayed on Messages (`passthrough`), the advisor's model goes as the upstream's own name for it when it is the same provider's Claude model, or a Claude id naming no provider (`advisorTools` in [`advisor.go`](../../internal/gateway/advisor.go)); one it can't run there (another provider's, a group, not Claude's) is left out, as is every advisor when the request's own model isn't a Claude model (a vendor's Messages API knows no such tool), and the turn is answered without one. A translation drops it with every other server tool. Claude Code (2.1.300) also puts the advisor in the conversation: a `{role: "system"}` message whose text ends in its instructions ("# Advisor Tool … call advisor BEFORE substantive work") and a `tool_addition` naming it. Where the advisor is left out, that `tool_addition` goes too (and a message it leaves empty), and the instructions become Claude Code's own "The advisor tool is no longer available; disregard the earlier advisor instructions." (`withoutAdvisor` on Messages, `advisorGoneIn` in `parseAnthropic` for a translation): left there, the model called `advisor()` as one of the agent's tools and Claude Code answered "No such tool available: advisor" (JackChou on Discord). The Claude subscription's run declares only magpie's tools, so it never has one. `TestAdvisorModelAsTheUpstreamNamesIt`, `TestAdvisorLeftOutOnTranslation`.
- Kiro is told where to cache the prompt (plugins#110: a Kiro subscription read 0 from its cache). Its API takes Bedrock's `cachePoint: {"type":"default"}` on a `userInputMessage`, an `assistantResponseMessage` or as a tool entry. The built-in (`buildKiro` in [`kiro.go`](../../internal/gateway/kiro.go)) marks the first message when there is a system prompt, which it folds into that message, and the current message, as `buildAnthropic` marks Anthropic's. A moved Kiro's plugin (opencode-kiro-auth 0.1.10, the mover's `min`) turns each `cache_control` in the Anthropic body it is handed into one, at most 4, the last kept.
diff --git a/internal/gateway/desktop.go b/internal/gateway/desktop.go
index 463863b83..4e5c0aee0 100644
--- a/internal/gateway/desktop.go
+++ b/internal/gateway/desktop.go
@@ -483,6 +483,10 @@ func desktopTurn(asked string, body []byte, byClaudeCode, bySubagent bool) strin
if m := desktopTierTurn(asked); m != "" {
return m
}
+ if desktopProbe(body) && !unserved(asked) {
+ // Desktop's check that the model works: asked of it alone
+ return asked
+ }
tools := hasTools(body)
desktopPicked.Lock()
defer desktopPicked.Unlock()
@@ -589,6 +593,32 @@ func desktopDefault(asked string) string {
return shown[0].ID
}
+// desktopProbe: the request is Claude Desktop's check that a gateway model
+// answers, which it runs as it starts and from its gateway settings' Test:
+// {"model":<it>,"max_tokens":1,"messages":[{"role":"user","content":"."}]}
+// (lWt with JUt in its app.asar, 2.7032), on the shortest id with haiku
+// in it, else sonnet, else opus (YT). It is small and tool-less like a
+// title, but it asks whether that model answers: sent to the session's
+// model instead, a failure there was shown as the haiku model rejected
+// ("Gateway rejected model …haiku…", CyberCaius on Discord, whose session
+// was on a relay's GPT model that turned the request away).
+func desktopProbe(body []byte) bool {
+ var v struct {
+ MaxTokens int `json:"max_tokens"`
+ System json.RawMessage `json:"system"`
+ Tools json.RawMessage `json:"tools"`
+ Messages []json.RawMessage `json:"messages"`
+ }
+ if json.Unmarshal(body, &v) != nil || v.MaxTokens != 1 || len(v.System) > 0 || len(v.Tools) > 0 || len(v.Messages) != 1 {
+ return false
+ }
+ var m struct {
+ Role string `json:"role"`
+ Content string `json:"content"`
+ }
+ return json.Unmarshal(v.Messages[0], &m) == nil && m.Role == "user" && m.Content == "."
+}
+
// desktopTitlePrompt is how the system prompt Claude Desktop asks a
// session's title and branch name with starts (dcr in its app.asar), sent
// by its own client or, when that fails, as Claude Code's --system-prompt.
diff --git a/internal/gateway/desktop_probe_test.go b/internal/gateway/desktop_probe_test.go
new file mode 100644
index 000000000..e20a7b12c
--- /dev/null
+++ b/internal/gateway/desktop_probe_test.go
@@ -0,0 +1,154 @@
+package gateway
+
+import (
+ "bytes"
+ "encoding/json"
+ "net/http/httptest"
+ "strings"
+ "testing"
+
+ "github.com/tidwall/gjson"
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// relayNeedsCacheKey answers as CyberCaius's AnyRouter did a request
+// without a prompt_cache_key.
+func relayNeedsCacheKey(body []byte) (int, string) {
+ if gjson.GetBytes(body, "prompt_cache_key").String() == "" {
+ return 400, `{"error":{"message":"Arouter: prompt_cache_key is required (request id: x)","type":"invalid_request_error"},"type":"error"}`
+ }
+ return 0, ""
+}
+
+// Claude Desktop checks a gateway model as it starts with
+// {"max_tokens":1,"messages":[{"role":"user","content":"."}]} on the first
+// haiku it is listed. With a session on a relay's GPT model, magpie took
+// that check for a title and sent it to the relay, which turned it away,
+// and Desktop said the haiku model was rejected (CyberCaius on Discord).
+// The check goes to the model it names; a title still goes to the
+// session's model, and reaches the relay with a cache key it needs.
+func TestClaudeDesktopProbeAsked(t *testing.T) {
+ claude := &fake{t: t, reply: sse(
+ `event: message_start`+"\n"+`data: {"type":"message_start","message":{"id":"m","type":"message","role":"assistant","model":"claude-haiku-5-5","content":[],"usage":{"input_tokens":1,"output_tokens":0}}}`,
+ `event: message_stop`+"\n"+`data: {"type":"message_stop"}`)}
+ relay := &fake{t: t, reply: deepseekReply, refuse: relayNeedsCacheKey}
+ up := setup(t, provider.Anthropic, claude)
+ _ = up
+ ru := httptest.NewServer(relay)
+ t.Cleanup(ru.Close)
+ if err := provider.Save(provider.Provider{ID: "leee", Name: "leee", Key: "k", Anthropic: up.URL,
+ Models: []string{"claude-haiku-5-5"}}); err != nil {
+ t.Fatal(err)
+ }
+ if err := provider.Save(provider.Provider{ID: "anyrouter", Name: "AnyRouter", Key: "k", Chat: ru.URL + "/v1",
+ Models: []string{"gpt-6.1-sol"}}); err != nil {
+ t.Fatal(err)
+ }
+ desktopPicked.Lock()
+ desktopPicked.model, desktopPicked.from = "", ""
+ desktopPicked.Unlock()
+ srv := New()
+ send := func(body string) (int, string) {
+ t.Helper()
+ req := httptest.NewRequest("POST", "/v1/messages", strings.NewReader(body))
+ req.Header.Set("x-api-key", Token+"-claude-desktop")
+ req.Header.Set("User-Agent", "Mozilla/5.0 (Macintosh) Claude/2.7032.0 Chrome/140.0 Electron/38.0")
+ rec := httptest.NewRecorder()
+ srv.Handler().ServeHTTP(rec, req)
+ return rec.Code, rec.Body.String()
+ }
+ // the session's turn, on the relay's model
+ if code, body := send(`{"model":"anyrouter/gpt-6.1-sol","max_tokens":32000,"stream":true,"tools":[{"name":"Bash","input_schema":{"type":"object"}}],"messages":[{"role":"user","content":"hi"}]}`); code != 200 {
+ t.Fatalf("turn: %d %s", code, body)
+ }
+ first := gjson.GetBytes(relay.got, "prompt_cache_key").String()
+ if first == "" || strings.Contains(strings.ToLower(first), "magpie") {
+ t.Fatalf("turn's cache key %q", first)
+ }
+
+ // Desktop's check of its haiku, by the id it was listed
+ haiku := "magpie-" + aliasNumber("leee/claude-haiku-5-5") + ".anthropic.claude-haiku-5-5[1m]"
+ relay.calls, claude.got = 0, nil
+ if code, body := send(`{"model":"` + haiku + `","max_tokens":1,"messages":[{"role":"user","content":"."}]}`); code != 200 {
+ t.Fatalf("probe: %d %s", code, body)
+ }
+ if relay.calls != 0 || !bytes.Contains(claude.got, []byte(`"claude-haiku-5-5"`)) {
+ t.Fatalf("probe went to the relay (%d calls); leee got %s", relay.calls, claude.got)
+ }
+
+ // a title still goes to the session's model, and the relay takes it
+ relay.got = nil
+ if code, body := send(`{"model":"` + haiku + `","max_tokens":200,"stream":true,"system":"You write short session titles.","messages":[{"role":"user","content":"hi"}]}`); code != 200 {
+ t.Fatalf("title: %d %s", code, body)
+ }
+ if modelOf(relay.got) != "gpt-6.1-sol" {
+ t.Fatalf("title went to %q", modelOf(relay.got))
+ }
+ // the same conversation keeps its key
+ if code, body := send(`{"model":"anyrouter/gpt-6.1-sol","max_tokens":32000,"stream":true,"tools":[{"name":"Bash","input_schema":{"type":"object"}}],"messages":[{"role":"user","content":"hi"},{"role":"assistant","content":"ok"},{"role":"user","content":"more"}]}`); code != 200 {
+ t.Fatalf("second turn: %d %s", code, body)
+ }
+ if k := gjson.GetBytes(relay.got, "prompt_cache_key").String(); k != first {
+ t.Fatalf("second turn's key %q, first %q", k, first)
+ }
+}
+
+// desktopProbe knows Desktop's check alone, not a title or a turn.
+func TestDesktopProbe(t *testing.T) {
+ for body, want := range map[string]bool{
+ `{"model":"x","max_tokens":1,"messages":[{"role":"user","content":"."}]}`: true,
+ `{"model":"x","max_tokens":200,"messages":[{"role":"user","content":"."}]}`: false,
+ `{"model":"x","max_tokens":1,"system":"You write short session titles.","messages":[{"role":"user","content":"."}]}`: false,
+ `{"model":"x","max_tokens":1,"messages":[{"role":"user","content":"hi"}]}`: false,
+ `{"model":"x","max_tokens":1,"tools":[{"name":"a"}],"messages":[{"role":"user","content":"."}]}`: false,
+ } {
+ if desktopProbe([]byte(body)) != want {
+ t.Errorf("desktopProbe(%s) = %v", body, !want)
+ }
+ }
+}
+
+// magpie's own cache key goes only to OpenAI's API and a provider the user
+// added by its address; a vendor's preset, a signed-in account and a
+// plugin get none. A relay that turns the field away is asked again
+// without it, once.
+func TestOwnCacheKey(t *testing.T) {
+ for _, c := range []struct {
+ p provider.Provider
+ want bool
+ }{
+ {provider.Provider{ID: "relay", Chat: "https://relay.example/v1"}, true},
+ {provider.Provider{ID: "openai", Preset: "openai", Responses: "https://api.openai.com/v1"}, true},
+ {provider.Provider{ID: "deepseek", Preset: "deepseek", Chat: "https://api.deepseek.com/v1"}, false},
+ {provider.Provider{ID: "codex", Account: &provider.Account{}, Responses: "https://chatgpt.com/backend-api/codex"}, false},
+ {provider.Provider{ID: "relay-own", Chat: "https://relay.example/v1", ExtraBody: provider.ExtraBody{"prompt_cache_key": json.RawMessage(`"${sessionId}"`)}}, false},
+ {provider.Provider{ID: "relay-forced", Chat: "https://relay.example/v1", ExtraBody: provider.ExtraBody{"!prompt_cache_key": json.RawMessage(`"k"`)}}, false},
+ } {
+ if got := ownCacheKey(c.p); got != c.want {
+ t.Errorf("ownCacheKey(%s) = %v", c.p.ID, got)
+ }
+ }
+
+ refusing := func(body []byte) (int, string) {
+ if gjson.GetBytes(body, "prompt_cache_key").Exists() {
+ return 400, `{"error":{"message":"Unrecognized request argument supplied: prompt_cache_key"}}`
+ }
+ return 0, ""
+ }
+ f := &fake{t: t, reply: deepseekReply, refuse: refusing}
+ setup(t, provider.Chat, f)
+ srv := New()
+ for turn := range 2 {
+ calls := f.calls
+ req := httptest.NewRequest("POST", "/v1/messages", strings.NewReader(`{"model":"fake/m1","max_tokens":100,"stream":true,"messages":[{"role":"user","content":"hi"}]}`))
+ req.Header.Set("x-api-key", Token)
+ rec := httptest.NewRecorder()
+ srv.Handler().ServeHTTP(rec, req)
+ if rec.Code != 200 {
+ t.Fatalf("turn %d: %d %s", turn+1, rec.Code, rec.Body)
+ }
+ if n, want := f.calls-calls, 2-turn; n != want {
+ t.Errorf("turn %d: %d upstream calls, want %d", turn+1, n, want)
+ }
+ }
+}
diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.go
index e9fee3752..9e6c9daf4 100644
--- a/internal/gateway/gateway.go
+++ b/internal/gateway/gateway.go
@@ -5371,6 +5371,13 @@ func (s *Server) forwardTranslated(ctx context.Context, p provider.Provider, to
web := req.WebSearch
// the cache key was left out to see if it was what the upstream refused
dropped := false
+ // the conversation's cache key given a provider that may need one when
+ // the client named none (ownCacheKey); "" once it was left out for a
+ // refusal
+ ownKey := ""
+ if req.CacheKey == "" && ownCacheKey(p) {
+ ownKey = cacheKeyOf(in, req)
+ }
// Codex's image tool was left out to see if it was (#949), and the
// request with it, to go back to when it wasn't; tried once
var withImage *Request
@@ -5399,6 +5406,17 @@ func (s *Server) forwardTranslated(ctx context.Context, p provider.Provider, to
r := *req
r.WebSearch, req = want, &r
}
+ if ownKey != "" {
+ // only Chat and Responses carry it
+ want := ""
+ if (to == provider.Chat || to == provider.Responses) && s.fits(p.ID, cacheKeyField, to) {
+ want = ownKey
+ }
+ if req.CacheKey != want {
+ r := *req
+ r.CacheKey, req = want, &r
+ }
+ }
if req.CacheKey != "" && !s.fits(p.ID, cacheKeyField, to) {
r := *req
r.CacheKey, req = "", &r
@@ -5811,7 +5829,10 @@ func (s *Server) forwardTranslated(ctx context.Context, p provider.Provider, to
r.Tier, req = "", &r
continue
}
- if req.CacheKey != "" && badRequest(res.StatusCode) && !wrongEndpoint(res.StatusCode, b) && !provider.CopilotRefusal(b) {
+ // magpie's own key is what a refusal is about only when it says so:
+ // another 400 is not asked again for it
+ ownKeyKept := ownKey != "" && req.CacheKey == ownKey && !refusesField(res.StatusCode, b, cacheKeyField)
+ if req.CacheKey != "" && !ownKeyKept && badRequest(res.StatusCode) && !wrongEndpoint(res.StatusCode, b) && !provider.CopilotRefusal(b) {
// a vendor that turns away fields it doesn't know is asked again
// without the cache key, and not sent it again once that works —
// at once when its error names the key; not every error does
@@ -5820,6 +5841,7 @@ func (s *Server) forwardTranslated(ctx context.Context, p provider.Provider, to
} else {
dropped = true
}
+ ownKey = ""
r := *req
r.CacheKey, req = "", &r
continue
@@ -5892,6 +5914,61 @@ func loadsInPlace(p provider.Provider) bool {
// upstream; a provider that refused it is remembered under it in unfit.
const cacheKeyField = "prompt_cache_key"
+// ownCacheKey says a request translated for p goes with a prompt cache
+// key for its conversation when the client sent none (an Anthropic
+// client's never has one): OpenAI's API routes a conversation by it to
+// where its prompt is cached, and a relay in front of it can refuse a
+// request without it ("prompt_cache_key is required", CyberCaius's
+// AnyRouter on Discord). Only OpenAI's own API with a key, and a provider
+// the user added by its address, which may be such a relay: a vendor's
+// preset, a signed-in account (the ChatGPT backend gets its own, as Codex
+// CLI sends it), a plugin, Bedrock and Vertex are sent nothing they didn't
+// get before, nor is a provider whose extraBody names the field: the
+// user's value is the one sent. One whose 400 names the field is asked again without it and
+// not sent it again (forwardTranslated); another 400 is not retried for
+// it.
+func ownCacheKey(p provider.Provider) bool {
+ if p.Account != nil || p.IsPlugin() || p.IsBedrock() || p.IsVertex() {
+ return false
+ }
+ if _, ok := p.ExtraBody[cacheKeyField]; ok {
+ return false // the user's own extraBody key (#1604) is theirs to fill
+ }
+ if _, ok := p.ExtraBody["!"+cacheKeyField]; ok {
+ return false
+ }
+ if p.AddedByAddress() {
+ return true
+ }
+ h := p.Host()
+ return h == "api.openai.com" || strings.HasSuffix(h, ".api.openai.com")
+}
+
+// cacheKeyOf is the prompt cache key of the conversation req is a turn
+// of: the session the client names (sessionOf), else its first user
+// message, which every later turn repeats, hashed into a UUID's shape, so
+// it is the same for every turn and names nothing of the client, the
+// user or magpie.
+func cacheKeyOf(in http.Header, req *Request) string {
+ h := sha256.New()
+ if v := sessionOf(in); v != "" {
+ io.WriteString(h, "session\x00"+v)
+ } else {
+ io.WriteString(h, "first\x00")
+ for _, m := range req.Messages {
+ if m.Role != "user" {
+ continue
+ }
+ for _, pt := range m.Parts {
+ fmt.Fprintf(h, "%v\x00%s\x00%s\x00%s\x00%s\x00", pt.Kind, pt.Text, pt.MediaType, pt.Data, pt.URL)
+ }
+ break
+ }
+ }
+ x := hex.EncodeToString(h.Sum(nil)[:16])
+ return x[:8] + "-" + x[8:12] + "-" + x[12:16] + "-" + x[16:20] + "-" + x[20:]
+}
+
// refusesField recognizes a vendor turning a request away for a field it
// doesn't take — Gemini's "Unknown name", Mistral's extra_forbidden,
// Groq's "unsupported" — by the field's name in the error.
diff --git a/internal/gateway/testdata/upstream_bodies/anthropic-to-chat.json b/internal/gateway/testdata/upstream_bodies/anthropic-to-chat.json
index f7cbdb383..20467995f 100644
--- a/internal/gateway/testdata/upstream_bodies/anthropic-to-chat.json
+++ b/internal/gateway/testdata/upstream_bodies/anthropic-to-chat.json
@@ -1 +1 @@
-{"max_tokens":1024,"messages":[{"content":"You are é / \"careful\"\n\ttabs","role":"system"},{"content":"read it","role":"user"},{"content":"ok 漢字 😀","role":"assistant","tool_calls":[{"function":{"arguments":"{\"path\":\"/a b/c.go\",\"n\":1.50e2}","name":"Read"},"id":"toolu_1","type":"function"}]},{"content":"line\u0001one\r\nline two","role":"tool","tool_call_id":"toolu_1"},{"content":"go on","role":"user"}],"model":"m1","stream":true,"stream_options":{"include_usage":true},"temperature":0.7,"tools":[{"function":{"description":"Reads a file","name":"Read","parameters":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}},"type":"function"}]}
\ No newline at end of file
+{"max_tokens":1024,"messages":[{"content":"You are é / \"careful\"\n\ttabs","role":"system"},{"content":"read it","role":"user"},{"content":"ok 漢字 😀","role":"assistant","tool_calls":[{"function":{"arguments":"{\"path\":\"/a b/c.go\",\"n\":1.50e2}","name":"Read"},"id":"toolu_1","type":"function"}]},{"content":"line\u0001one\r\nline two","role":"tool","tool_call_id":"toolu_1"},{"content":"go on","role":"user"}],"model":"m1","prompt_cache_key":"2062df63-eb9f-15ae-9e58-9bc2c61248d3","stream":true,"stream_options":{"include_usage":true},"temperature":0.7,"tools":[{"function":{"description":"Reads a file","name":"Read","parameters":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}},"type":"function"}]}
\ No newline at end of file
diff --git a/internal/gateway/testdata/upstream_bodies/anthropic-to-responses.json b/internal/gateway/testdata/upstream_bodies/anthropic-to-responses.json
index 34fd05ae9..e6c8d49e7 100644
--- a/internal/gateway/testdata/upstream_bodies/anthropic-to-responses.json
+++ b/internal/gateway/testdata/upstream_bodies/anthropic-to-responses.json
@@ -1 +1 @@
-{"input":[{"content":[{"text":"what is this","type":"input_text"},{"detail":"auto","image_url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==","type":"input_image"}],"role":"user","type":"message"}],"max_output_tokens":64,"model":"m1","store":false,"stream":true}
\ No newline at end of file
+{"input":[{"content":[{"text":"what is this","type":"input_text"},{"detail":"auto","image_url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==","type":"input_image"}],"role":"user","type":"message"}],"max_output_tokens":64,"model":"m1","prompt_cache_key":"c0369fb3-13da-2218-18ed-5ee8ed47454f","store":false,"stream":true}
\ No newline at end of file
diff --git a/internal/gateway/testdata/upstream_bodies/large-responses-to-chat.sha256 b/internal/gateway/testdata/upstream_bodies/large-responses-to-chat.sha256
index 85edc5ee1..6f1c603b7 100644
--- a/internal/gateway/testdata/upstream_bodies/large-responses-to-chat.sha256
+++ b/internal/gateway/testdata/upstream_bodies/large-responses-to-chat.sha256
@@ -1 +1 @@
-ea4a95413224377e614047c41a8abae72aff0f2692ff5751562da720d7701a29
+c7cf9a32e597de0ba64ec1e04cdb0079e91f286790d8c40a0fa903a220cd362a
diff --git a/internal/gateway/testdata/upstream_bodies/responses-to-chat.json b/internal/gateway/testdata/upstream_bodies/responses-to-chat.json
index 83c1f9898..9f0203da5 100644
--- a/internal/gateway/testdata/upstream_bodies/responses-to-chat.json
+++ b/internal/gateway/testdata/upstream_bodies/responses-to-chat.json
@@ -1 +1 @@
-{"messages":[{"content":"be brief — ok","role":"system"},{"content":[{"text":"hi","type":"text"},{"image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="},"type":"image_url"}],"role":"user"},{"role":"assistant","tool_calls":[{"function":{"arguments":"{\"path\":\"/x\"}","name":"Read"},"id":"call_1","type":"function"}]},{"content":"done\t\"x\"","role":"tool","tool_call_id":"call_1"},{"content":"go on","role":"user"}],"model":"m1","reasoning_effort":"medium","service_tier":"priority","stream":true,"stream_options":{"include_usage":true},"tools":[{"function":{"description":"Reads","name":"Read","parameters":{"type":"object","properties":{"path":{"type":"string"}}}},"type":"function"}]}
\ No newline at end of file
+{"messages":[{"content":"be brief — ok","role":"system"},{"content":[{"text":"hi","type":"text"},{"image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="},"type":"image_url"}],"role":"user"},{"role":"assistant","tool_calls":[{"function":{"arguments":"{\"path\":\"/x\"}","name":"Read"},"id":"call_1","type":"function"}]},{"content":"done\t\"x\"","role":"tool","tool_call_id":"call_1"},{"content":"go on","role":"user"}],"model":"m1","prompt_cache_key":"792f8d82-337c-0905-4730-10546e0bd386","reasoning_effort":"medium","service_tier":"priority","stream":true,"stream_options":{"include_usage":true},"tools":[{"function":{"description":"Reads","name":"Read","parameters":{"type":"object","properties":{"path":{"type":"string"}}}},"type":"function"}]}
\ No newline at end of file