Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
gateway: a remote magpie's providers are each a vendor of their own to a conversation (fremontdango)
yetoneff16f6c552e3
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.mdindex 1c2d17239..d130c3588 100644--- a/docs/subsystems/gateway-routing.md+++ b/docs/subsystems/gateway-routing.md@@ -37,7 +37,7 @@ backfilled; see [Gateway Session Aggregation](gateway-sessions.md).| Limits | Each key or account (a candidate's `who()`) has at most `MaxConcurrency` requests out at once (its own `AccountConcurrency` over it); one more waits in its lane's queue, bounded by `QueueLimit` and `QueueWait` and turned away with a 429 past either (`lanes.take`). `MaxRPM` (coeo91 on Discord: OpenRouter's free models take 20 a minute) bounds how many requests it sends the vendor in any rolling 60 seconds (`rpms`), counting each for a second more (`rpmSlack`), since the vendor counts a request when it arrives and the first on a new connection arrives later by its dial: one more waits for room in the minute, in the order they came, after it has its slot and right before it is sent. A wait that would be longer than 2 minutes (or the provider's `QueueWait`, when shorter: `rpmWait`) is turned away at once with a 429 whose `Retry-After` is when the minute has room; nothing rests for it. An agent gone while it waits gives its place back. Every request that reaches the vendor counts: each try of the candidate loop (a retry, the next key, a fallback) waits in `serve`, and the requests a try makes on the way and the gateway's own asks (count_tokens, System One's decisions, embeddings, drawing, Kimi's search) are counted by `rpmTransport` on `s.client` (`metered`, `paidFor`). count_tokens never waits: a candidate with no room is skipped, and the local estimate answers. A full lane or a minute with no room gives way to a ready key or account of the provider or the group (`laneMate`, `ready`); a fallback is never asked for want of either. A plugin's account sends through its own transport, so only its tries are counted, not a plugin's own extra requests. Set in the provider editor (Concurrency, Requests per minute, Queue size, Queue wait) or with `magpie provider account-concurrency`, `queue` and `rpm`; a subscription's are kept with its picks, and every account of it keeps them, those on beside the first as well (`withPicks`, called by `All` and `AlsoOn`; #1899: a second WorkBuddy AI account had none, and queued 62s with the wait set to 1s). | [`concurrency.go`](../../internal/gateway/concurrency.go), [`rpm.go`](../../internal/gateway/rpm.go), [`internal/provider/concurrency.go`](../../internal/provider/concurrency.go) || Sink | With `Sink` on, a key or account that answers 429 while it still has quota goes to the back, behind every one not rate limited since. Held in memory until restart, or until its rest is lifted by hand (Try now, `Unrest`/`UnrestAll` call `unsink`), which puts it back in its place (`TestTryNowPutsASunkAccountBack`). A group in a group sinks by its own Sink, within its own order (`sinkOwn`, #2255), and the group asked for then sinks the whole by its own; `TestNestedGroupSinksItsOwnOrder`. | [`sink.go`](../../internal/gateway/sink.go) || Image generation | `/v1/images/generations` and `/v1/images/edits` are translated to the selected model's image API. OpenRouter image-only models use its `/images` endpoint and send reference images as `input_references`; OpenRouter models that return images through chat stay on Chat Completions. Volcengine Ark (`isArk`: an `ark.*.volces.com` base, the preset's plans or one added by hand) is asked at `/images/generations` only, as its doc (https://www.volcengine.com/docs/82379/1541523) takes it (`drawArk`): `watermark: false`, since it marks every image otherwise; reference images as JSON `image` (a URL or data URL, an array for several), never a multipart edit; no `n`, so one request per image (`eachDrawing`); no ratio as `size`, so a ratio its doc lists is sent as that ratio's 2K size and any other is a 400 that says to put it in the prompt; no mask, no quality (JL on Discord, `TestArkDrawsWithoutWatermarkOncePerImage`). A routing group named for a drawing or a video (`drawingGroup`: `group/<id>`, or a bare id the group goes by that resolved to one of its members) is tried member by member in the order its routing plans for a chat (`drawingTries` over `planGroup`), each member over its own keys or accounts (`drawOnAccounts`, `startVideo`); the next member is asked while one fails as a chat's would move on (`retryable`: its vendor 5xx, out of quota, the model not served), never for a refusal of the request itself, and the call's `Fallback` names those tried first (#2005, tianxia3111: a group of two relays answered the first one's 502). Pause rules and a gateway key's group hold apply as for a chat. A failed drawing rests no member and no key. An account out of quota for the image model (`failure` reads `failQuota`) rests that model on it alone (`restDrawer`, keyed by `restID`), until the reset its refusal names (`resetsAt`: Antigravity's "Your quota will reset after 3h19m30s.") or `quotaRest`, so the next drawing asks the accounts that still have it first (`restLast`), and the account's text models stay in. Antigravity also tells a model's quota used up as a 200 whose stream is the error; `drawCodeAssist` gives such an error the status its words stand for (`saidStatus`: quota or rate 429, credit 402, else 502), as the images API's 200 with no image does, so a spent account hands the drawing to the next one as its 429 does (01huadalang on Discord: a 502 ended the drawing on the spent account; `TestAntigravityDrawingOutOfQuotaInsideTheReplyGoesOn`, `TestAntigravityDrawingQuotaRestsTheImageModel`). `TestImageGroupMovesOnToTheNextMember`, `TestVideoGroupMovesOnToTheNextMember`. An account's own list of models (#474) never bars a ChatGPT account's GPT Image, a Grok subscription's Grok Imagine or a WorkBuddy plan's image models (`listsCantName`): the list is picked from the editor's chat models, which they are never among, so a list kept for some chat models had every drawing refused as "set not to be served by any account" (heyiwei5 on X). A key's list, picked from models that include its image ones, still bars them. `TestCodexDrawsPastAccountModelLists`. A drawing is given Settings → Images → Image generation timeout (`imageTimeout`, 1 to 60 minutes, 5 when unset; neighbads on X: a slow image endpoint needed more) as its context's deadline (`drawTimeout`, `drawWithin`), and so is downloading an image answered as a link (`imageLongest`, never under `ImageLongest`). A drawing's request goes through a transport with no `ResponseHeaderTimeout` (`drawsApart`), since an images API sends its headers only once it has drawn, and every other request keeps `upstreamHeaderWait` (10 minutes). A video's submit, status and download calls keep 5 minutes each (`videoTimeout`). The `magpie mcp image` process, which may not read these settings (WSL), waits on a call up to `callLongest` (two hours and a minute), so it never cuts one the gateway still waits on. `TestLongImageTimeoutLetsASlowVendorFinish`, `TestImageTimeoutSettingReachesTheVendor`, `TestOnlyDrawingsWaitPastTheHeaderWait`. | [`draw.go`](../../internal/gateway/draw.go), [`draw_volcengine.go`](../../internal/gateway/draw_volcengine.go), [`video.go`](../../internal/gateway/video.go), [`catalog.go`](../../internal/catalog/catalog.go) |-| Affinity | A conversation stays with the key or account that answered it, so the vendor's prompt cache is read again. Modes: auto (""), `session`, `turn`, `off`, `strict`. Stored in `affinity.json` next to `providers.json`. A conversation is kept by its id (`conversationID`), and a Claude Code subagent's apart from its lead's by `x-claude-code-agent-id` (`stickKey`): Claude Code sends the session's id with its subagents' requests too, and a subagent's answer took the lead's stick over, so the lead's next turn was judged by the subagent's cache and moved to another account, where its whole conversation was written again (slkassinthehood on X, `TestClaudeConversationKeepsItsAccountsCache`). | [`affinity.go`](../../internal/gateway/affinity.go) |+| Affinity | A conversation stays with the key or account that answered it, so the vendor's prompt cache is read again. Modes: auto (""), `session`, `turn`, `off`, `strict`. Stored in `affinity.json` next to `providers.json`. A conversation is kept by its id (`conversationID`), and a Claude Code subagent's apart from its lead's by `x-claude-code-agent-id` (`stickKey`): Claude Code sends the session's id with its subagents' requests too, and a subagent's answer took the lead's stick over, so the lead's next turn was judged by the subagent's cache and moved to another account, where its whole conversation was written again (slkassinthehood on X, `TestClaudeConversationKeepsItsAccountsCache`). Behind a remote magpie, the key or account is the remote's key and the provider there that serves the model, its id's first part (`remoteUpstream`, `candidate.vendor`, `sameVendor`): the remote is one key here, but each of its providers keeps its own cache and seals, so a conversation is kept, held strictly, told its sealed reasoning was refused and given its native compaction's reader by that pair, as the same group on the remote does (fremontdango on Discord: a group of one model from two of the remote's providers kept a conversation on the second as the same account when the first was gone, and a strict one failed over to it; `TestRemoteMagpieStrictKeepsTheRemotesProvider`, `TestRemoteMagpieOtherProviderIsNotTheSameAccount`). A remote's group is one vendor by its whole id, its own routing deciding inside it; a model named without a provider is the key alone. Lanes and requests a minute still go by the key here, the limits the user set on it. | [`affinity.go`](../../internal/gateway/affinity.go) || One try | `attempt` sends one candidate the request. It goes through the proxy of the account or key it uses, else the provider's (`p.Via`, `ProxyChoice`: `AccountProxies` holds an account's by its name, a key's by its `KeyID`). A request magpie makes for one provider while serving another's (a web search, an image description, a classifier call) takes its own provider's proxy or the global one, never the outer one: `netproxy.With` replaces a choice the context already names, "" too, and `webSearch` starts from none, so the search APIs follow the global proxy (#1522). A provider of the user's set not to check its certificate (`InsecureTLS`, #2383) has it skipped for its own base URLs' hosts alone, through `p.Via` ([Providers and accounts](providers-accounts.md)). Every request to one vendor's host shares one HTTP/2 connection (`upstreamTransport`); one with no frame for 20s is pinged, and dropped when no answer comes within 10s (`netproxy.HTTP2Health`, on `http.DefaultTransport` too), so a connection gone silent without closing (a proxy's way out died, a NAT forgot the flow) fails the requests on it, which move on as a 502 does, and the next request dials a new one. A live vendor answers a ping whatever its streams are doing, so a long wait for a first byte is not cut by it (#2843: a group's requests hung with no first byte until a restart; `TestGroupMovesOnFromSilentHTTP2Connection`). A subscription with its own client is served specially, such as Claude through the genuine Claude Code. The request is relayed as is (`passthrough`) when the model is usable on the agent's API, except that a Claude model where the provider has an Anthropic URL is translated to Messages from Chat or Responses too (`OnMessages`: OpenCode speaks Chat for every model, and a relay drops cache_control there), unless the user set the model's API or the vendor's list names its APIs. An account that only streams (`Account.Stream`: a plugin's, and most subscriptions magpie signs in to) has a request that asked for one body translated, which magpie always asks streamed, except a plugin's on Responses (`gathers`): that is relayed all the same, asked with `stream:true`, and the client given the `response` of the stream's `response.completed` (or `response.incomplete`) as one JSON body (`gathered` in [`gathered.go`](../../internal/gateway/gathered.go); its `output` filled from the stream's `response.output_item.done` items, or else its text, when that event's is empty). So the tools the vendor runs itself, and its response's `tools` and usage details, reach the client (plugins#112: a translation dropped Grok's `x_search`, and Grok answered without searching; `TestGrokPluginNonStreamKeepsHostedTools`). A stream that fails or ends before that event is a 502 (`TestGrokPluginNonStreamEndedShort`), never an empty 200. A Messages request is still translated, its stream having no event that holds the whole reply, and so is a subscription magpie signs in to itself (the ChatGPT backend's, Qoder's): its own code fits its requests. A request offering web search to a provider that doesn't search by itself is translated too, so magpie can search for it (`search.go`; a provider that searches for some of its models only, Command Code's Provider API, lists them in `searchModels` by API, and none count under its `x-cmd-zdr` header, #1590), but on the agent's own API where the provider serves it and no `ResponsesFirst`/`MessagesFirst` rule prefers another: Codex offers web_search on every turn, and each went out as Chat (#997). Otherwise it is translated (`translate`) to the first API `usable` gives: Chat, except Responses for OpenAI's models where their maker serves them (`ResponsesFirst`), Responses for every model at ModelScope, whose Chat API reports neither the tokens its cache answered nor reasoning while its Responses API reports both (`ResponsesTellsCache`, #1578; a 404 there sends the model to Chat, and only there), and Anthropic's Messages for a Claude model where the provider has an Anthropic URL (`MessagesFirst`, which keeps its cache breakpoints, #997). An API that answers the model is not served there (`wrongEndpoint`) is remembered in `unfit` until magpie restarts. A Chat API that refuses function tools with reasoning, with no other API to take them to, is asked once more with `reasoning_effort` `none` (#176), unless `none` is known refused (`turnedAway`); `none` refused naming the levels, the tools refusal is the answer, not a retry at the lowest level (#2670). When every API has turned the model away, the API the user picked for it (`ModelAPI`) is asked again, and without a pick each API the provider has a URL for (`Speaks`), Chat first. `Provider.Native`, the API agents with a per-model API (Pi, omp, droid) are wired to ask on, follows the same order. A custom provider with a Gemini URL (#1346) has Gemini last among the APIs it speaks: a Gemini client's request is relayed there, and any other is translated to it when it has no other, or when the user set Gemini as the model's API (`SetModelAPI` takes chat, responses, anthropic or gemini, each only where the provider has its URL; a model set to Gemini has no `Native`, so those agents ask it on their own API and it is translated). Upstream it is always asked `models/{model}:streamGenerateContent?alt=sse` under the base (`upstreamPath`), the model only in the path (`forwardOnce` drops the body's), so a Gemini client that asked for one JSON body is translated, as for Factory's generate route. | [`gateway.go`](../../internal/gateway/gateway.go) || Trace | The Routing view's record of each decision, written where the decision is made: candidates, rests, which try answered. `Route.Folder` is the folder the agent says it works in (`workDir`), its last two levels only (`shortFolder`, e.g. `workspace/magpie`), never the full path; a request that says none (a title, a topic check) takes its session's as said last (`trace.sessionFolder`), one without a session id has none (#1068). It is kept in the routing history on disk for as long as the route is (`history.go`: 30 days, 200 MB). The GUI shows it in By session's headings, a request row's tooltip, the stage's agent working in more than one folder, and the tray panel's agent tooltip. | [`trace.go`](../../internal/gateway/trace.go) || Prompt | What each request's prompt held, part by part: the agent's system prompt, tool definitions (an MCP server's tools as one item), instruction files and skills, files its tools read, their other output, and the conversation. `inspectPrompt` reads it from the body as the agent sent it, before translation; `calibrated` scales the estimate to the prompt the vendor counted and sets the window of the model that answered. The route keeps names and sizes only, never the text, so the routing history on disk holds none. For the card's text (masaka on Discord), the trace keeps the last `promptBodies` (10) request bodies in memory only (at most 32 MiB each, 64 MiB together), masked as the vendor got them; for a model let have it unmasked, the masked copy is kept all the same. `PromptText` reads a part's items and their text from that body again (`prompt_text.go`), served to the GUI as `/api/context/text`; once a body is dropped, or magpie quits, its text is gone. It goes on the trace as `Route.Prompt`, with `Route.Conv` naming a conversation that has no session id. The GUI's `/api/context` scores each agent from it (`internal/gui/context.go`). A session's line and its compactions are its own turns': a call with a `Kind`, a Claude Code subagent's (`Route.Sub`, from `x-claude-code-agent-id`), and, in a session whose turns carry tool definitions, a request without any (auto mode's classifier, the title, the quota check) are left out, so a classifier's 40K after a 100K turn is no compaction (`toolSessions`, `TestContextSideCallsAreNoCompaction`). | [`prompt.go`](../../internal/gateway/prompt.go) |diff --git a/internal/gateway/affinity.go b/internal/gateway/affinity.goindex 73d09681f..835691a1f 100644--- a/internal/gateway/affinity.go+++ b/internal/gateway/affinity.go@@ -380,7 +380,7 @@ func strictKeep(scope, mode string, in http.Header, body []byte, cs []candidate,sticks.Lock()st, had := stickOf(stickKey(scope, in, body))sticks.Unlock()- if !had || time.Since(st.at) > stickKeep || slices.ContainsFunc(cs, func(c candidate) bool { return c.who() == st.who }) {+ if !had || time.Since(st.at) > stickKeep || slices.ContainsFunc(cs, func(c candidate) bool { return c.sameVendor(st.who, st.model) }) {return cs, pl, nil}// the left's line for c, as it was left out@@ -392,7 +392,7 @@ func strictKeep(scope, mode string, in http.Header, body []byte, cs []candidate,var back []candidatevar order []Weighedfor _, c := range pl.capped {- if c.who() != st.who {+ if !c.sameVendor(st.who, st.model) {continue}i := leftAt(c)@@ -405,12 +405,12 @@ func strictKeep(scope, mode string, in http.Header, body []byte, cs []candidate,back, order = append(back, c), append(order, w)}if len(back) > 0 {- pl.capped = slices.DeleteFunc(slices.Clone(pl.capped), func(c candidate) bool { return c.who() == st.who })+ pl.capped = slices.DeleteFunc(slices.Clone(pl.capped), func(c candidate) bool { return c.sameVendor(st.who, st.model) })return append(back, cs...), planned{order: append(order, pl.order...), left: pl.left, held: pl.held, capped: pl.capped}, nil}var held []Weighedfor _, c := range pl.held {- if c.who() != st.who {+ if !c.sameVendor(st.who, st.model) {continue}if i := slices.IndexFunc(pl.left, func(w Weighed) bool { return w.ID == c.rest && w.NoCredits }); i >= 0 {@@ -419,7 +419,7 @@ func strictKeep(scope, mode string, in http.Header, body []byte, cs []candidate,}if len(held) > 0 {// only its own reset, should it spend one by itself, brings it back- pl.held = slices.DeleteFunc(slices.Clone(pl.held), func(c candidate) bool { return c.who() != st.who })+ pl.held = slices.DeleteFunc(slices.Clone(pl.held), func(c candidate) bool { return !c.sameVendor(st.who, st.model) })return nil, planned{order: nil, left: pl.left, held: pl.held, capped: pl.capped}, held[:1]}return cs, pl, nil@@ -430,10 +430,10 @@ func strictKeep(scope, mode string, in http.Header, body []byte, cs []candidate,// group has on it, which a rule may still pick between. Nobody else is// asked, nor a reset spent elsewhere: its error is the agent's.func strictOnly(cs []candidate, pl planned, at int) ([]candidate, planned) {- who := cs[at].who()+ who, model := cs[at].who(), cs[at].modelkeep := []int{at}for i, c := range cs {- if i != at && c.who() == who {+ if i != at && c.sameVendor(who, model) {keep = append(keep, i)}}@@ -448,7 +448,7 @@ func strictOnly(cs []candidate, pl planned, at int) ([]candidate, planned) {}}pl.order = order- pl.held = slices.DeleteFunc(slices.Clone(pl.held), func(c candidate) bool { return c.who() != who })+ pl.held = slices.DeleteFunc(slices.Clone(pl.held), func(c candidate) bool { return !c.sameVendor(who, model) })return out, pl}@@ -494,12 +494,13 @@ func affine(scope, mode string, rotate bool, in http.Header, from provider.Proto// the account and the model that answered: a group with Opus and// Sonnet both on one Claude account, a rule sending the turn to Opus,// had the turn's next request kept to the account and so to Sonnet,- // its first member there; the account alone when the model is gone+ // its first member there; the account alone when the model is gone,+ // and behind a remote magpie the provider there alone (sameVendor)at := -1for _, same := range []func(candidate) bool{func(c candidate) bool { return c.who() == st.who && c.model == st.model && c.effort == st.effort },func(c candidate) bool { return c.who() == st.who && c.model == st.model },- func(c candidate) bool { return c.who() == st.who },+ func(c candidate) bool { return c.sameVendor(st.who, st.model) },} {for i, c := range cs {if had && at < 0 && same(c) {@@ -710,7 +711,7 @@ func opened(key string) {func unanswered(key string, c candidate) {sticks.Lock()st, ok := stickOf(key)- ok = ok && st.who == c.who()+ ok = ok && c.sameVendor(st.who, st.model)if ok {delete(sticks.m, key)}diff --git a/internal/gateway/codex_backend.go b/internal/gateway/codex_backend.goindex c312a05e9..091fd0bbd 100644--- a/internal/gateway/codex_backend.go+++ b/internal/gateway/codex_backend.go@@ -516,13 +516,13 @@ func leadFirst(scope, parent string, cands []candidate, pl planned) ([]candidate}}for i, c := range cands {- if i > 0 && c.who() == st.who {+ if i > 0 && c.sameVendor(st.who, st.model) {cands = append(append([]candidate{c}, cands[:i]...), cands[i+1:]...)pl.order = append(append([]Weighed{pl.order[i]}, pl.order[:i]...), pl.order[i+1:]...)return cands, pl, c.rest}}- if len(cands) > 0 && cands[0].who() == st.who {+ if len(cands) > 0 && cands[0].sameVendor(st.who, st.model) {return cands, pl, cands[0].rest}return cands, pl, ""diff --git a/internal/gateway/fallback.go b/internal/gateway/fallback.goindex 6ae086e33..f93efcb6d 100644--- a/internal/gateway/fallback.go+++ b/internal/gateway/fallback.go@@ -147,6 +147,49 @@ func (c candidate) who() string {return c.restKey()}+// remoteUpstream is, for a remote magpie's model, the provider there that+// serves it, its id's first part (anthropic of anthropic/claude-…), or a+// group of the remote's, its whole id; "" for any other provider's model,+// and for a remote's model named without one. The remote magpie is one+// key here, but each of its providers is a vendor of its own, with its+// own cache and its own seals (fremontdango on Discord: a group of one+// model from two of the remote's providers kept a conversation on the+// second when the first was resting, as one account, where the same+// group on the remote moved it as gone).+func remoteUpstream(p provider.Provider, model string) string {+ if !p.IsRemoteMagpie() {+ return ""+ }+ if strings.HasPrefix(model, provider.GroupPrefix) {+ return model+ }+ up, _, _ := strings.Cut(model, "/")+ if up == model {+ return ""+ }+ return up+}++// vendor is who as the vendor a conversation is with knows it: the key or+// account, and behind a remote magpie the provider there (remoteUpstream).+// What a vendor keeps of a conversation — its cache, the reasoning and+// compactions it sealed — goes by it; what the user limits on the key+// here — its lanes, requests a minute — still goes by who.+func (c candidate) vendor() string {+ if up := remoteUpstream(c.p, c.model); up != "" {+ return c.who() + ">" + up+ }+ return c.who()+}++// sameVendor says whether c is with the vendor that who, asked for model,+// was: the same key or account, and behind a remote magpie the same+// provider there. With no model known (an answer the usage log has) it+// is the key or account alone.+func (c candidate) sameVendor(who, model string) bool {+ return c.who() == who && (model == "" || remoteUpstream(c.p, c.model) == remoteUpstream(c.p, model))+}+// perKey is a provider once per key it has on, in order — or, for a// signed-in agent, once per account it has on, its own first. A key made// for one protocol only serves on that one's endpoint, and the keys thatdiff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.goindex e5347e75e..6477fc5c9 100644--- a/internal/gateway/gateway.go+++ b/internal/gateway/gateway.go@@ -2137,7 +2137,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Proif cp, ok := checkpointOf(body); ok {checkpoint = &cpvar left bool- if cands, pl, left = checkpointReaders(cands, pl, cp.Who); !left {+ if cands, pl, left = checkpointReaders(cands, pl, cp); !left {call.Status, call.Error = 400, "native compaction sealed elsewhere"writeError(w, from, 400, checkpointError(call.Model, cp))turnedAway()@@ -2580,7 +2580,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Proif from == provider.Responses {// reasoning another sealed, which this one refused earlier in// the conversation, isn't sent to be refused again; its own is- if b, ok := withoutRefused(stuck, c.who(), attemptBody); ok {+ if b, ok := withoutRefused(stuck, c.vendor(), attemptBody); ok {attemptBody = b}}@@ -3048,7 +3048,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Proif resealed == 0 {resealed = 1if b, ok = withoutReasoning(body); ok {- refused(stuck, c.who(), attemptBody, "reasoning")+ refused(stuck, c.vendor(), attemptBody, "reasoning")}}// a compaction sealed where native compaction is on is never@@ -3056,7 +3056,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Proif !ok && checkpoint == nil && !s.compactsNatively(c.p, c.model) {resealed = 2if b, ok = withoutCompaction(body); ok {- refused(stuck, c.who(), attemptBody, compactionKinds...)+ refused(stuck, c.vendor(), attemptBody, compactionKinds...)}}if ok {diff --git a/internal/gateway/native_compaction.go b/internal/gateway/native_compaction.goindex 841d347f3..86d5863f9 100644--- a/internal/gateway/native_compaction.go+++ b/internal/gateway/native_compaction.go@@ -279,22 +279,25 @@ func saveCheckpoints() {}}-// checkpointReaders keeps of cands those of the key or account who, pl's+// checkpointReaders keeps of cands those of the key or account that+// sealed cp (behind a remote magpie, of the provider there too), pl's// order with them; left says one is left, if only held or capped. One// whose own list lacks the model now (planned.unlisted) is kept too: it// is tried as the last resort an account whose list lacks the model is,// and none other can read what it sealed. Its row goes from left to the// order with it, so the Routing page doesn't tell it both tried and// unlisted, skipped.-func checkpointReaders(cands []candidate, pl planned, who string) (_ []candidate, _ planned, left bool) {- kept, pl, ok := keepCandidates(cands, pl, func(c candidate) bool { return c.who() == who })+func checkpointReaders(cands []candidate, pl planned, cp checkpointWriter) (_ []candidate, _ planned, left bool) {+ // behind a remote magpie, the provider there that sealed it too+ reads := func(c candidate) bool { return c.sameVendor(cp.Who, cp.Model) }+ kept, pl, ok := keepCandidates(cands, pl, reads)if ok {return kept, pl, true}var found []candidatevar ws []Weighedfor i, c := range pl.unlisted {- if c.who() != who {+ if !reads(c) {continue}found = append(found, c)diff --git a/internal/gateway/remote_magpie_vendor_test.go b/internal/gateway/remote_magpie_vendor_test.gonew file mode 100644index 000000000..8b3cec754--- /dev/null+++ b/internal/gateway/remote_magpie_vendor_test.go@@ -0,0 +1,158 @@+package gateway++import (+ "encoding/json"+ "io"+ "net/http"+ "net/http/httptest"+ "strings"+ "sync"+ "testing"++ "github.com/yetone/magpie/internal/provider"+)++// fremontdango on Discord: a group of one model from two of a remote+// magpie's providers, made on the magpie that uses the remote, kept its+// conversations as if the two were one vendor, since both are the one+// remote magpie key here. The same group made on the remote itself+// doesn't. remoteVendors is the remote: a magpie serving m from two+// providers, "first/m" and "second/m", each answering "from <provider>",+// first out of its usage (429) while breaking is set.+type remoteVendors struct {+ srv *httptest.Server+ mu sync.Mutex+ tried []string+ breaking bool+}++func newRemoteVendors(t *testing.T) *remoteVendors {+ rv := &remoteVendors{}+ rv.srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ if r.Method == http.MethodGet {+ w.Header().Set("Content-Type", "application/json")+ io.WriteString(w, `{"object":"list","data":[{"id":"first/m","object":"model"},{"id":"second/m","object":"model"},{"id":"third/m","object":"model"}]}`)+ return+ }+ var q struct {+ Model string `json:"model"`+ }+ b, _ := io.ReadAll(r.Body)+ json.Unmarshal(b, &q)+ up, _, _ := strings.Cut(q.Model, "/")+ rv.mu.Lock()+ rv.tried = append(rv.tried, up)+ br := rv.breaking && up == "first"+ rv.mu.Unlock()+ w.Header().Set("X-Magpie-Provider", up)+ if br {+ w.Header().Set("Content-Type", "application/json")+ w.WriteHeader(http.StatusTooManyRequests)+ io.WriteString(w, `{"error":{"message":"first: usage limit reached","type":"rate_limit_error"}}`)+ return+ }+ w.Header().Set("Content-Type", "text/event-stream")+ io.WriteString(w, sse(`data: {"id":"c2","choices":[{"index":0,"delta":{"role":"assistant","content":"from `+up+`"}}]}`,+ `data: {"id":"c2","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}`,+ `data: [DONE]`))+ }))+ t.Cleanup(rv.srv.Close)+ return rv+}++func (rv *remoteVendors) post(t *testing.T, s *Server, session string) (string, string) {+ t.Helper()+ rv.mu.Lock()+ rv.tried = nil+ rv.mu.Unlock()+ rec := httptest.NewRecorder()+ req := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(`{"model":"group/g","stream":true,"messages":[{"role":"user","content":"hi `+session+`"}]}`))+ req.Header.Set("x-session-id", session)+ s.Handler().ServeHTTP(rec, req)+ rv.mu.Lock()+ defer rv.mu.Unlock()+ return rec.Body.String(), strings.Join(rv.tried, ",")+}++func remoteVendorsGroup(t *testing.T, rv *remoteVendors, affinity string, members ...string) {+ t.Helper()+ if err := provider.Save(provider.Provider{ID: "office", Name: "Office", Preset: provider.RemoteMagpiePreset, Key: "sk-office", Chat: rv.srv.URL, Models: []string{"first/m", "second/m", "third/m"}}); err != nil {+ t.Fatal(err)+ }+ if err := provider.SaveGroup(provider.Group{Name: "G", Members: members, Routing: provider.Ordered, Affinity: affinity}); err != nil {+ t.Fatal(err)+ }+}++// Strictly kept, a conversation the remote's first provider answered stays+// on it when it runs out, its error the agent's, as the same group on the+// remote keeps it: the remote's second provider is another vendor, not another model+// of the same account, and isn't asked.+func TestRemoteMagpieStrictKeepsTheRemotesProvider(t *testing.T) {+ fresh(t)+ rv := newRemoteVendors(t)+ remoteVendorsGroup(t, rv, provider.AffinityStrict, "office/first/m", "office/second/m")+ s := New()+ if out, got := rv.post(t, s, "s1"); !strings.Contains(out, "from first") || got != "first" {+ t.Fatalf("first turn: %s tried %s", out, got)+ }+ rv.mu.Lock()+ rv.breaking = true+ rv.mu.Unlock()+ if out, got := rv.post(t, s, "s1"); !strings.Contains(out, "usage limit") || got != "first" {+ t.Fatalf("strict, its provider out: went to the remote's other provider: %s tried %s", out, got)+ }+ // resting now, it is still its error the agent gets, the other not asked+ if out, got := rv.post(t, s, "s1"); !strings.Contains(out, "usage limit") || got == "second" {+ t.Fatalf("strict, its provider resting: %s tried %s", out, got)+ }+ if why := lastWhy(t, s); why != "strict" {+ t.Fatalf("the retry traced as %q, want strict", why)+ }+}++// A conversation whose remote provider has left the group is gone, as on+// the remote, not kept on the remote's other provider as its account: the+// trace said "session", kept, where nothing of it was cached.+func TestRemoteMagpieOtherProviderIsNotTheSameAccount(t *testing.T) {+ fresh(t)+ rv := newRemoteVendors(t)+ remoteVendorsGroup(t, rv, provider.AffinitySession, "office/first/m", "office/second/m")+ s := New()+ if out, got := rv.post(t, s, "s1"); !strings.Contains(out, "from first") || got != "first" {+ t.Fatalf("first turn: %s tried %s", out, got)+ }+ remoteVendorsGroup(t, rv, provider.AffinitySession, "office/second/m", "office/third/m")+ if out, got := rv.post(t, s, "s1"); !strings.Contains(out, "from second") || got != "second" {+ t.Fatalf("second turn: %s tried %s", out, got)+ }+ if why := lastWhy(t, s); why != "gone" {+ t.Fatalf("traced as %q, want gone: the remote's other provider kept it as its own", why)+ }+}++func TestRemoteUpstream(t *testing.T) {+ remote := provider.Provider{ID: "office", Preset: provider.RemoteMagpiePreset, Key: "k"}+ plain := provider.Provider{ID: "openrouter", Key: "k"}+ for _, c := range []struct {+ p provider.Provider+ model, want string+ }{+ {remote, "anthropic/claude-sonnet-4-5", "anthropic"},+ {remote, "openrouter/anthropic/claude-sonnet-4-5", "openrouter"},+ {remote, "group/fast", "group/fast"},+ {remote, "m", ""},+ {plain, "anthropic/claude-sonnet-4-5", ""},+ } {+ if got := remoteUpstream(c.p, c.model); got != c.want {+ t.Errorf("remoteUpstream(%s, %q) = %q, want %q", c.p.ID, c.model, got, c.want)+ }+ }+ a, b := candidate{p: remote, model: "first/m"}, candidate{p: remote, model: "second/m"}+ if a.who() != b.who() || a.vendor() == b.vendor() {+ t.Errorf("who %q %q, vendor %q %q: one key here, two vendors there", a.who(), b.who(), a.vendor(), b.vendor())+ }+ if !a.sameVendor(b.who(), "first/x") || a.sameVendor(b.who(), "second/m") || !a.sameVendor(b.who(), "") {+ t.Error("sameVendor")+ }+}