Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

provider: every request to a Remote magpie carries the provider's Headers; its key always goes as x-api-key (Yirasumi)

yetone66b5e4a844cd
diff --git a/docs/subsystems/providers-accounts.md b/docs/subsystems/providers-accounts.md
index 8903211bc..abf2172b0 100644
--- a/docs/subsystems/providers-accounts.md
+++ b/docs/subsystems/providers-accounts.md
@@ -23,6 +23,7 @@ keys and accounts per request is in [Gateway routing and fallback](gateway-routi
| Zhipu's 200 refusals | Zhipu's and Z.ai's `/api/v1` (the GLM Coding Plan's Responses API, from their Codex pages) and `/api/monitor` answer a refusal 200 in the platform's envelope, `{"code":1000,"msg":"身份验证失败。","success":false}`, where their OpenAI and Anthropic endpoints answer the same key 401 (@xanahopper on X: a refused key tested green on the Responses row only). `ZhipuEnveloped` turns a 2xx JSON body from `open.bigmodel.cn` or `api.z.ai` with `success: false` into `{"error":{"code","message"}}` at the status Zhipu's error-code table gives (1000–1005 → 401, 1113 and 1302–1399 → 429, 1220 → 403, 1210–1229, 1261 and 1301 → 400, anything else → 502), in the provider test's `probe` (endpoint tests, model tests, Detect APIs) and the gateway's `forwardOnce`. A stream, a compressed body, a body over 64 KB and every other host pass as they came. The plan's quota card for a key refused there (1000–1005, or a 401) says `<Zhipu or Z.ai> didn't take this key (<their words>): …` with the vendor's API-keys page (`zhipuRefused`), not asked again as a team's key, and the GUI shows it as that (`quotaError`) rather than "Allowance unavailable". | [`zhipu_envelope.go`](../../internal/provider/zhipu_envelope.go), [`planquota.go`](../../internal/provider/planquota.go), [`test.go`](../../internal/provider/test.go) |
| Extra body | `ExtraBody` (`extraBody` in providers.json, the editor's *Extra body*, #1604) is a JSON object whose fields `WithExtraBody` adds at the top level of every JSON body sent to the provider: Messages, Chat, Responses and Gemini, relayed as they came or translated, in the gateway's `forwardOnce` and in Test and Detect (`tested`). A field the body has already is kept; a name written `"!name"` replaces it. Values are put in as written, objects whole, not merged. `${sessionId}` in any string, at any depth, becomes the request's session: the agent's session header (`sessionOf`), else the id `conversationID` derives from the first user message (`magpie-` and its hash), so it stays the same through one conversation; Test and Detect send `magpie-test-` and 12 random letters, a new one each time. `${threadId}` is the thread the agent names within that session: Codex's `thread_id` (its turn metadata or `thread-id` header, `threadOf`), which a subagent has of its own while it shares its lead's session; read only when the provider's headers or extraBody name it (`AsksThread`). No other agent names a thread, and there `${threadId}` is the session's id. `model`, `stream` and the conversation (`messages`, `input`, `contents`, `system`, `systemInstruction`, `system_instruction`, `instructions`, `tools`, `ExtraBodyReserved`) are magpie's: Save refuses them with or without `!` (`ExtraBodyOK`), the editor says so as they are typed, and one written into providers.json by hand is dropped on read (`cleanExtraBody`). Not sent with it: `count_tokens`, embeddings, images, video and System One's decision model (other APIs), and a signed-in account's requests, except a ChatGPT (Codex) account's (magpie-releases#18): its requests are built as Codex CLI's (`codexBody`) and take the fields Codex CLI adds by choice, such as `access_programs` (OpenAI's Daybreak), while `store`, `tool_choice` and `include`, which the backend needs as `codexBody` sets them, can't be set (`codexOwned`). Its extraBody is written into providers.json by hand under the account's entry and kept onto the account by `withPicks`; the editor has no box for it. A client's own top-level `access_programs` on Responses or Chat is carried to a Responses build as it came (`Request.AccessPrograms`), so a request translated (a Codex account is streamed, so a whole Responses request is made again) keeps it as a relayed one does. It reaches only the upstreams Codex itself sends it to (`keepsAccessPrograms`, #3172, see [gateway routing](gateway-routing.md)); it is taken out before the extraBody merge, so an `access_programs` the user puts in a provider's extraBody still goes. One object goes to every API; fields for one API alone are not a thing yet. A `thinking` budget it puts in on an Anthropic endpoint is fitted to a Claude that thinks only adaptively after the merge, as an agent's is (`adaptiveThinking`, `TestExtraBodyBudgetFittedToAdaptiveModel`). | [`extrabody.go`](../../internal/provider/extrabody.go), [`gateway.go`](../../internal/gateway/gateway.go) |
| Headers' conversation ids | `${sessionId}` and `${threadId}` in a value of the provider's `Headers` (Axel Burks on Discord: a gateway in front of several Azure accounts routes one conversation to one account by a session header) are the ids extraBody gets, on every request `Sign` signs: the gateway carries the conversation to it in the request's context (`WithConversation`), on every API and `count_tokens` too. A request that is no turn of a conversation (a model list, a balance, an embedding, image or video, System One's decision model, a search) gets an id of its own, `magpie-` and 12 random letters, new each time (`OwnConversation`); Test and Detect's is their `magpie-test-` id, the same in headers and body. A literal placeholder is never sent. In a header the ids go without control characters (`headerSafe`): an agent's session id with CR or LF in it would split the header, or Go would fail the request. A value with neither placeholder goes as written (`SentHeaders`). | [`conversation.go`](../../internal/provider/conversation.go), [`account.go`](../../internal/provider/account.go), [`gateway.go`](../../internal/gateway/gateway.go), [`codex_resume.go`](../../internal/gateway/codex_resume.go) |
+| A remote magpie's headers | Every request magpie makes to a Remote magpie carries the provider's `Headers` (Yirasumi on Discord: the Usage page's read went without the header a proxy in front of the remote asks for): its API requests and model list as any provider's (`Sign`, `listHeaders`), and its cards, a card's refresh, its quota history and each gateway key's limit (`signRemote`), the last four as no turn of a conversation (`${sessionId}` an id of its own). The key always reaches the remote as `x-api-key` (`remoteKey`), which a gateway reads beside `Authorization`: a user's `Authorization` (a proxy's own scheme) goes in place of magpie's `Bearer` copy without taking the key away, and a user's `x-api-key` is never sent to a remote magpie, so a key's limit is read with that key. A user's `Authorization` that is itself another gateway key of that magpie is read first there, as on any request. | [`remote_quotas.go`](../../internal/provider/remote_quotas.go), [`remote_key_limits.go`](../../internal/provider/remote_key_limits.go), [`remote_magpie.go`](../../internal/provider/remote_magpie.go), [`account.go`](../../internal/provider/account.go) |
| Reply headers | `ReplyHeaders` (`replyHeaders` in providers.json, the editor's *Reply headers* under Advanced, a comma-separated list, #4145) names headers of the vendor's reply the gateway passes on to the agent, matched whatever their case and saved canonical (`cleanReplyHeaders`). `DefaultReplyHeaders` (`Request-Id`, `X-Request-Id`, `X-Oneapi-Request-Id`) go from every provider without being listed: opaque request ids, under names magpie sets nothing of its own. Never passed on, listed or not (`ReplyHeaderPassable`): the reply's framing (`Content-*`, `Transfer-Encoding`, hop-by-hop), cookies and sign-ins (`Set-Cookie`, `Authorization`, `WWW-Authenticate`, `Proxy-*`), CORS (`Access-Control-*`), `Sec-*`, redirects and browser policy (`Location`, `Strict-Transport-Security`, `Content-Security-Policy`…), `Retry-After`/`Cache-Control`/`Date`/`Server`/`Vary` (magpie writes those) and `X-Magpie-*` (the gateway reads them as notes of the try). Save refuses such a name (`ReplyHeadersOK`), the editor says so before the save (`replyHeadersOf`, its list held to Go's by `TestReplyHeadersNeverAsTheProvidersAre`), and one written into providers.json by hand is dropped on read. A preset's provider and a signed-in account's requests take it too (`withPicks`). How the gateway carries them is in [gateway routing](gateway-routing.md). | [`replyheaders.go`](../../internal/provider/replyheaders.go), [`reply_headers.go`](../../internal/gateway/reply_headers.go) |
| Native compaction | `NativeCompaction` (`nativeCompaction` in providers.json, the editor's *Compaction* row, `magpie provider set <id> compaction=native\|magpie`; #1099) has Codex's compaction asked of the provider as Codex sent it and its sealed item returned unchanged. Off by default. Only a provider whose wire carries it can be set so (`CarriesCompaction`: a Responses API, a ChatGPT account, GitHub Copilot; not another agent's sign-in or a plugin's), and `CompactsOn(model)` also needs the model asked on Responses. `SetNativeCompaction`, the CLI and the GUI's save refuse it elsewhere. An account keeps it through `Save`'s picks (`withPicks`), and `AddCopy` carries it. The gateway's half is in [gateway-routing](gateway-routing.md). | [`native_compaction.go`](../../internal/provider/native_compaction.go) |
| providers.json | `Path` is `providers.json` in magpie's config folder. A read that fails returns `ErrUnreadable` and is never treated as empty. `store` writes atomically through a rename and drops the cached catalog (`catalog.Touched`). | [`provider.go`](../../internal/provider/provider.go) |
diff --git a/internal/provider/account.go b/internal/provider/account.go
index f228c8ef2..7d2e320e3 100644
--- a/internal/provider/account.go
+++ b/internal/provider/account.go
@@ -241,6 +241,12 @@ func (p Provider) Sign(ctx context.Context, req *http.Request, proto Protocol, b
}
catalog.PutUserHeader(req.Header, k, v)
}
+ // a remote magpie is always sent its key as x-api-key, which no header
+ // of the user's replaces: their Authorization (a proxy's) goes on
+ // without taking the key away (remoteKey)
+ if p.IsRemoteMagpie() && p.Key != "" {
+ remoteKey(req.Header, p.Key)
+ }
return nil
}
diff --git a/internal/provider/remote_headers_test.go b/internal/provider/remote_headers_test.go
new file mode 100644
index 000000000..0dfdf5b44
--- /dev/null
+++ b/internal/provider/remote_headers_test.go
@@ -0,0 +1,124 @@
+package provider
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "net/http"
+ "regexp"
+ "strings"
+ "sync"
+ "testing"
+)
+
+// headerPeer is a remote magpie behind a proxy that wants its own
+// headers: it answers every request of magpie's own (cards, a refresh,
+// the quota history, a key's limit, the model list) and keeps each one's
+// headers by its path.
+type headerPeer struct {
+ sync.Mutex
+ seen map[string][]http.Header
+}
+
+func (f *headerPeer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+ f.Lock()
+ f.seen[r.Method+" "+r.URL.Path] = append(f.seen[r.Method+" "+r.URL.Path], r.Header.Clone())
+ f.Unlock()
+ switch r.URL.Path {
+ case RemoteKeyLimitPath:
+ w.Write([]byte(`{"limited":false,"key":"k"}`))
+ case "/v1/magpie/quotas/history":
+ json.NewEncoder(w).Encode(map[string]any{"object": "list", "data": []QuotaHistory{{Provider: "codex"}}})
+ case "/v1/models":
+ w.Write([]byte(`{"object":"list","data":[{"id":"gpt-6","object":"model"}]}`))
+ default:
+ json.NewEncoder(w).Encode(map[string]any{"object": "list", "data": []SubscriptionQuota{{Provider: "codex", Name: "Codex", Kind: "subscription"}}})
+ }
+}
+
+// Yirasumi on Discord: a Remote magpie behind a proxy that asks for a
+// header of its own answered the chat, but the Usage page's read of its
+// cards went without the provider's Headers. Every request magpie makes
+// to a remote magpie now carries them, ${sessionId} as an id of its own,
+// and the key still reaches the remote as x-api-key: a user's
+// Authorization (the proxy's) goes on in place of magpie's Bearer, and a
+// user's x-api-key never replaces the key.
+func TestRemoteMagpieRequestsCarryHeaders(t *testing.T) {
+ for _, userAuth := range []bool{false, true} {
+ t.Run(map[bool]string{false: "own auth", true: "user's Authorization"}[userAuth], func(t *testing.T) {
+ peer := &headerPeer{seen: map[string][]http.Header{}}
+ remoteQuotaHome(t, peer)
+ p, err := Find("office")
+ if err != nil {
+ t.Fatal(err)
+ }
+ p.Headers = map[string]string{"X-Proxy-Token": "tok", "X-Session": "${sessionId}", "X-Api-Key": "not-the-key"}
+ if userAuth {
+ p.Headers["Authorization"] = "Basic cHJveHk6cHc="
+ }
+ p.Keys = []KeyAccount{{Name: "spare", Key: "sk-magpie-spare"}}
+ if err := Save(*p); err != nil {
+ t.Fatal(err)
+ }
+ ForgetRemoteCardsForTest()
+ if p, err = Find("office"); err != nil {
+ t.Fatal(err)
+ }
+ ctx := context.Background()
+ RemoteCards(ctx)
+ RefreshUsage(ctx, "office/codex", "")
+ RemoteQuotaHistories(ctx, "7")
+ if _, err := p.Fetch(ctx); err != nil {
+ t.Fatalf("model list: %v", err)
+ }
+ req, _ := http.NewRequest(http.MethodPost, p.Chat+"/chat/completions", bytes.NewReader([]byte(`{}`)))
+ if err := p.Sign(ctx, req, Chat, []byte(`{}`)); err != nil {
+ t.Fatal(err)
+ }
+ res, err := p.Do(http.DefaultClient, req)
+ if err != nil {
+ t.Fatal(err)
+ }
+ res.Body.Close()
+
+ own := regexp.MustCompile(`^magpie-[a-z0-9]{12}$`)
+ peer.Lock()
+ defer peer.Unlock()
+ for _, path := range []string{
+ "GET " + RemoteCardsPath, "POST " + RemoteRefreshPath, "GET /v1/magpie/quotas/history",
+ "GET " + RemoteKeyLimitPath, "GET /v1/models", "POST /v1/chat/completions",
+ } {
+ hs := peer.seen[path]
+ if len(hs) == 0 {
+ t.Errorf("%s: never asked", path)
+ continue
+ }
+ keys := map[string]bool{}
+ for _, h := range hs {
+ if h.Get("X-Proxy-Token") != "tok" {
+ t.Errorf("%s: X-Proxy-Token %q", path, h.Get("X-Proxy-Token"))
+ }
+ if s := h.Get("X-Session"); !own.MatchString(s) {
+ t.Errorf("%s: X-Session %q, want an id of its own", path, s)
+ }
+ if v := h.Values("X-Api-Key"); len(v) != 1 || !strings.HasPrefix(v[0], "sk-magpie-") {
+ t.Errorf("%s: x-api-key %q, want magpie's key", path, v)
+ } else {
+ keys[v[0]] = true
+ }
+ want := "Bearer " + h.Get("X-Api-Key")
+ if userAuth {
+ want = "Basic cHJveHk6cHc="
+ }
+ if a := h.Values("Authorization"); len(a) != 1 || a[0] != want {
+ t.Errorf("%s: Authorization %q, want %q", path, a, want)
+ }
+ }
+ // each key's limit is read with that key
+ if path == "GET "+RemoteKeyLimitPath && (!keys["sk-magpie-office"] || !keys["sk-magpie-spare"]) {
+ t.Errorf("key limits read with %v", keys)
+ }
+ }
+ })
+ }
+}
diff --git a/internal/provider/remote_key_limits.go b/internal/provider/remote_key_limits.go
index 9f2d8efb6..5770a8b53 100644
--- a/internal/provider/remote_key_limits.go
+++ b/internal/provider/remote_key_limits.go
@@ -238,8 +238,8 @@ func askRemoteKeyLimit(ctx context.Context, p Provider, key string) (*remoteKeyA
if err != nil {
return nil, err
}
- req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Accept", "application/json")
+ p.signRemote(req, key)
res, err := p.Do(http.DefaultClient, req)
if err != nil {
return nil, fmt.Errorf("%s %s: %w", errRemoteUnreachable, remoteName(p), err)
diff --git a/internal/provider/remote_magpie.go b/internal/provider/remote_magpie.go
index 6f08794cf..9ff7f03cc 100644
--- a/internal/provider/remote_magpie.go
+++ b/internal/provider/remote_magpie.go
@@ -109,6 +109,7 @@ func (p *Provider) remoteMagpieEndpoints() {
// listHeaders are the headers p's model list is asked with: a remote
// magpie is asked for its image, video and decision models too, and its prices.
+// Its key goes as x-api-key whatever the user's headers say (remoteKey).
// A list is no turn of a conversation: ${sessionId} and ${threadId} in a
// value are an id of the list's own (conversation.go).
func (p Provider) listHeaders() map[string]string {
@@ -120,6 +121,11 @@ func (p Provider) listHeaders() map[string]string {
if h == nil {
h = map[string]string{}
}
+ for k := range h {
+ if strings.EqualFold(k, "x-api-key") {
+ delete(h, k)
+ }
+ }
h[DrawersHeader] = "1"
h[VideomakersHeader] = "1"
h[PricesHeader] = "1"
diff --git a/internal/provider/remote_quotas.go b/internal/provider/remote_quotas.go
index e83ac1ad9..37de272f5 100644
--- a/internal/provider/remote_quotas.go
+++ b/internal/provider/remote_quotas.go
@@ -12,6 +12,8 @@ import (
"strings"
"sync"
"time"
+
+ "github.com/yetone/magpie/internal/catalog"
)
// A remote magpie's quotas (莫 on Discord): a computer whose providers are
@@ -360,6 +362,35 @@ func askRemote(ctx context.Context, p Provider, method, path string) ([]Subscrip
return cards, nil
}
+// signRemote puts on req, a request of magpie's own to the remote magpie
+// p (its cards, a refresh, its quota history, a key's limit), key and p's
+// Headers, as its model list and its API requests carry them (Yirasumi on
+// Discord: the Usage page's read of the remote went without the headers a
+// proxy in front of it asks for). It is no turn of a conversation:
+// ${sessionId} and ${threadId} are an id of its own (conversation.go).
+// The key goes as Authorization: Bearer, which a user's Authorization
+// replaces (a proxy's own scheme), and always as x-api-key (remoteKey).
+func (p Provider) signRemote(req *http.Request, key string) {
+ req.Header.Set("Authorization", "Bearer "+key)
+ for k, v := range p.SentHeaders(context.Background()) {
+ catalog.PutUserHeader(req.Header, k, v)
+ }
+ remoteKey(req.Header, key)
+}
+
+// remoteKey writes key as x-api-key, in place of the header in any case:
+// to a remote magpie that header is magpie's key, which a gateway reads
+// beside Authorization, so a user's header (a proxy's Authorization) never
+// leaves the request without it, and a key's limit is read with that key.
+func remoteKey(h http.Header, key string) {
+ for k := range h {
+ if strings.EqualFold(k, "x-api-key") {
+ delete(h, k)
+ }
+ }
+ h.Set("x-api-key", key)
+}
+
// remoteJSON asks p's gateway, with its key, for a {"data": …} list.
func remoteJSON(ctx context.Context, p Provider, method, path string, dst any) error {
ctx, cancel := context.WithTimeout(ctx, 8*time.Second)
@@ -368,8 +399,8 @@ func remoteJSON(ctx context.Context, p Provider, method, path string, dst any) e
if err != nil {
return err
}
- req.Header.Set("Authorization", "Bearer "+p.Key)
req.Header.Set("Accept", "application/json")
+ p.signRemote(req, p.Key)
res, err := p.Do(http.DefaultClient, req)
if err != nil {
return fmt.Errorf("%s %s: %w", errRemoteUnreachable, remoteName(p), err)