Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
provider: Usage only keeps the ChatGPT account's models out of Codex's own picker, spawn_agent and its list's ETag (#2530)
yetone32f7c4de1421
diff --git a/docs/subsystems/providers-accounts.md b/docs/subsystems/providers-accounts.mdindex ae50ed2c4..fca125915 100644--- a/docs/subsystems/providers-accounts.md+++ b/docs/subsystems/providers-accounts.md@@ -26,7 +26,7 @@ keys and accounts per request is in [Gateway routing and fallback](gateway-routi| providers.json | `Path` is `providers.json` in magpie's config folder. A read that fails returns `ErrUnreadable` and is never treated as empty. `store` writes atomically through a rename and drops the cached catalog (`catalog.Touched`). | [`provider.go`](../../internal/provider/provider.go) || A provider renamed | `Rename` (the editor's id field, `magpie provider set <id> id=<new>`) keeps the old id in `Was`, so a model still picked by it reaches the provider. Usage is recorded under the id a call went out by and never rewritten; whatever reads it names the provider by its id now through `UsageRenames().Of(id, time)`: the summary, the Requests page (its blocks, the probe rows, other computers' days), the Sessions tab's vias, window holds, `NewPricer`/`NewCoster` and the gateway's `loggedAnswers` (cybercaius on Discord: a renamed provider was listed under both ids). `Visit` hands records out as recorded, so `Of` is taken once. An id another provider takes later (a rename onto it, or `Save` of a new provider under it) moves from the first one's `Was` to its `Gave` with the time (`giveID`): calls recorded under it before then stay the first one's, later ones are the new one's. An id held by a provider or a subscription now is never renamed (`Renamed`), and `freeID` doesn't hand out an id in any `Was`, so adding a vendor again gets `-2`. Ids reused before `Gave` existed have no time and go to whoever holds the id now. A providers.json that can't be read keeps the last renames read, not none. The summary and Requests caches key on `Renames.Key`, the vias cache too. | [`rename.go`](../../internal/provider/rename.go), [`block_price.go`](../../internal/usage/block_price.go) || The list | `All` merges stored providers with `Accounts()`. An entry with no URL stores the user's picks for an account. The order is `file.Order`. Reads within one request share one build (`Hold`, #746). | [`provider.go`](../../internal/provider/provider.go) |-| Model picks | `Models` are the models the user picked for agents; none means the vendor's list, up to 24 (`Exposed`), unless the user chose Select none (`NoModels`, `magpie provider models <id> none`): then none of its models is served, not to agents, not to any routing group (auto groups included), not by a `provider/model` id, not as a classifier, while its balance and usage are still read (`ServesNone`, #1818). The editor's serves control names it Usage only, beside Vendor's list and Picked only, and the card says "usage only" (#2068); Picked only from there picks the models the vendor's list serves, to unpick from. A file without `noModels` reads as before, and a pick clears it. `Save` keeps beside new picks the models listed then (`PickedFrom`, `listedWith`), and the same picks saved again keep it. When the picks held every model listed then, one the vendor lists since is served first beside them (`withNewlyListed`, `Picks`); one listed then and left unpicked stays out. Picks saved before `PickedFrom` was kept are taken to hold every model when they hold every one released up to the newest of them (models.dev's `release_date`), so a Claude account whose user picked them all is served Haiku 5.5 (wakaka). The editor and the TUI start from `Picks`, so a later Save keeps the new model. While some picks are set aside (the list is known to be short), a save with new picks keeps the models the earlier `PickedFrom` had beside those listed now, so a model the user saw and left unpicked isn't new when that list has it again; a list short of no pick is taken as it is. A pick a list fetched again stops listing (`Refetch`, `KeepListsFresh`, and a plugin's hourly `KeepPluginsListed`) leaves `Models` for `Delisted` (`dropGonePicks`), not served, and goes back among the picks when a later list has it (`relistPicks`): Our Free Model's plugin lists its account channels' models only once its service has read them after a restart, and its free pools only while they answer, so each magpie update had its user pick every model again (多喝热水 on Discord). Every save keeps `Delisted` but for what it picks again (`delistedWith`); Select none clears it. A provider whose every pick is in `Delisted` serves none of its models (`waitsForPicks`), not the vendor's list. `KeepPluginsListed` asks again after `relistSoon` (5 minutes) when a list left a pick out. `TestPluginPicksComeBackWithTheirList`, `TestEveryPickDelistedServesNone` and `TestRelayPicksComeBackWithTheirList` cover it. | [`models.go`](../../internal/provider/models.go), [`provider.go`](../../internal/provider/provider.go) |+| Model picks | `Models` are the models the user picked for agents; none means the vendor's list, up to 24 (`Exposed`), unless the user chose Select none (`NoModels`, `magpie provider models <id> none`): then none of its models is served, not to agents, not to any routing group (auto groups included), not by a `provider/model` id, not as a classifier, while its balance and usage are still read (`ServesNone`, #1818). Codex's own list at `/backend-api/codex/models` drops the ChatGPT account's visible models too, as spawn_agent's list of them does, and the list's ETag changes so a running Codex asks again (`CodexNativePicked` in [`codex_models.go`](../../internal/provider/codex_models.go), #2530); the hidden ones Codex runs jobs of its own on (codex-auto-review, gpt-reserve) and other providers' models stay. The editor's serves control names it Usage only, beside Vendor's list and Picked only, and the card says "usage only" (#2068); Picked only from there picks the models the vendor's list serves, to unpick from. A file without `noModels` reads as before, and a pick clears it. `Save` keeps beside new picks the models listed then (`PickedFrom`, `listedWith`), and the same picks saved again keep it. When the picks held every model listed then, one the vendor lists since is served first beside them (`withNewlyListed`, `Picks`); one listed then and left unpicked stays out. Picks saved before `PickedFrom` was kept are taken to hold every model when they hold every one released up to the newest of them (models.dev's `release_date`), so a Claude account whose user picked them all is served Haiku 5.5 (wakaka). The editor and the TUI start from `Picks`, so a later Save keeps the new model. While some picks are set aside (the list is known to be short), a save with new picks keeps the models the earlier `PickedFrom` had beside those listed now, so a model the user saw and left unpicked isn't new when that list has it again; a list short of no pick is taken as it is. A pick a list fetched again stops listing (`Refetch`, `KeepListsFresh`, and a plugin's hourly `KeepPluginsListed`) leaves `Models` for `Delisted` (`dropGonePicks`), not served, and goes back among the picks when a later list has it (`relistPicks`): Our Free Model's plugin lists its account channels' models only once its service has read them after a restart, and its free pools only while they answer, so each magpie update had its user pick every model again (多喝热水 on Discord). Every save keeps `Delisted` but for what it picks again (`delistedWith`); Select none clears it. A provider whose every pick is in `Delisted` serves none of its models (`waitsForPicks`), not the vendor's list. `KeepPluginsListed` asks again after `relistSoon` (5 minutes) when a list left a pick out. `TestPluginPicksComeBackWithTheirList`, `TestEveryPickDelistedServesNone` and `TestRelayPicksComeBackWithTheirList` cover it. | [`models.go`](../../internal/provider/models.go), [`provider.go`](../../internal/provider/provider.go) || List schedule and rules | A provider of the user's own (not an account, a plugin's or a decision-only one) can set `RefreshEvery` (`magpie provider refresh-every <id> 6h`, or 1d, or off, at least 10m) and `AutoInclude` (`magpie provider auto-include <id> 'gpt-*' 're:claude-.*'`, or none; a glob with `*`, or `re:` matching the whole id, both in any case (`compilePattern`; `(?-i)` inside a `re:` makes case matter); a plain id is refused) (#2135). The gateway (`magpie serve` or the app) looks each minute and fetches a list once `Fetched` is that old, through the same `Refetch` as Fetch models (`KeepListsFresh`); a failed fetch waits `min(every, 15m)`. `RefetchTaking` adds to the picks a model the fetch newly lists that a rule matches (`takeMatching`), starting from what is served now (`Exposed`), so nothing served is taken away and a model the user unticks stays out (it is not new to the next fetch). Setting the rules takes the current list's matches. Every save that doesn't set them keeps them, the window's editor among them; the editor has no fields for them yet. A Remote magpie with no `RefreshEvery` is fetched every 10m (`RemoteMagpieEvery`, `RefreshSchedule`), so a model the other magpie lists later (ZCode's GLM-5.3-Trial once its plugin read the gift, 6igsteven on Discord) is served without Fetch models; `off` set on it is kept as `"off"` and asks it only when told (`TestRemoteMagpieListFollowsTheOtherMagpie`). | [`autolist.go`](../../internal/provider/autolist.go), [`models.go`](../../internal/provider/models.go), [`providers_cli.go`](../../internal/cli/providers_cli.go) || Keys | Several keys per provider. Requests go to the first key that is on; the gateway moves on when it is out or rate limited. `Key` stays the first, so other code reads it as before. A key can be made for one protocol (`KeyProtocol`). A key made for a protocol its provider has no URL for is skipped (`perKeyBarred`); when every key is, the provider is still tried whole on the URLs it has, and the candidate carries `noURL` so it isn't quiet: Routing's row (`Weighed.NoURL`) says "key set to {api} only, no {api} URL · sent on the others", and the editor's key row shows a "No {api} URL" chip (`keyLacksURL`; #1702, `TestKeyForAProtocolWithNoURLIsSaid`, `key-nourl.test.cjs`). A key's value can be replaced in place (`ReplaceKey`, #1869: a key rotated at the vendor): it keeps its name, place, protocol, weight and on or off, and what was set for it by its fingerprint (`AccountModels`, `AccountProxies`, `AccountConcurrency`) and the gateway keys held to it (`access.ReplaceAccount`) move to the new one; the GUI's `keys/replace` also lifts its rests (`UnrestAll`). Its past requests stay under the old fingerprint, by the same name. A provider may hold thousands of keys (@NetRozee on X: a thousand froze magpie). Nothing goes over its keys once for each key: `KeysOn` drops a repeat by a set, a row's rest keys come from `RestKeys` (whether several are on, found once), and whether each key serves a model from `Serving`, which reads the lists once and keeps each model's keys as a set until the lists are read anew (`keyLists`). `KeysListed` and `ListsRead` count them for the tests (`TestProviderInfoListsKeysOnce`, `TestPerKeyReadsListsOnce`). A provider's keys have their balances, plans and remote limits read `keyReadsAtOnce` (8) at a time (`keySlots`); other providers' beside them. The editor's per-key proxy lines fold as its Accounts list does (`proxyLines`, `provider-many-keys-proxy.test.cjs`). Asking a key's models (`fetchPerKey`) still goes one key after another. | [`keys.go`](../../internal/provider/keys.go) || A key a provider | One vendor's keys saved a provider each — an app that keeps an entry a key, as CC Switch does, imported (yuluo_yx on X: "NVIDIA #001" to "#600" at one address, "Mistral #0001"…) — are folded into one provider with those keys (`FoldCopies` at start-up, `foldAdded` after `ImportFromApps`): hundreds of copies listed every model hundreds of times in every agent's picker and found group, and 3,000 made `/api/state` 86 MB. At start-up only copies by name are folded: a base name and that name with a number after it (`copyBase`: " #12", or `freeName`'s " 2"), one at least numbered; an import folds what it added whatever their names. Either way they must be alike in all but their key (`foldShape`: the provider with its id, name, keys and on/off left out), what the settings say of their models (`modelPrefs`) and where they are named (`foldRefs`: Visible, the agents' picked, hidden and ordered models, the groups, the providers falling back to them), so one key shown to one agent and another to another stays apart. Not folded: a subscription's, a hidden one, one without a key, Vertex, one with a StepFun sign-in of its own. The first on, in the providers' order, takes the rest in; its name becomes the base and its key is named its old name. Each other is a key in order, named its name, off if it was off (all off: the provider is off, its keys on); its id goes in `Was`, so its calls count with the provider by their key (`ProviderKeyID` is the key's fingerprint, unchanged) and a model picked by the old id still reaches it. Group members, rules, weights and fallbacks naming them name the provider once; their per-model settings and fetched lists go; balance and window history kept by their id moves to their key's name (`foldHistories`). A providers.json that doesn't parse is left alone. It runs at every start and writes nothing when there is nothing to fold. `TestFoldCopies`, `TestFoldCopiesIntoOneOn`, `TestFoldCopiesUnreadable`, `TestImportFoldsAKeyAnEntry`. | [`fold.go`](../../internal/provider/fold.go), [`importapps.go`](../../internal/provider/importapps.go), [`main.go`](../../internal/cli/main.go) |diff --git a/internal/gateway/codex_spawn_models.go b/internal/gateway/codex_spawn_models.goindex dbbc22614..a290cd932 100644--- a/internal/gateway/codex_spawn_models.go+++ b/internal/gateway/codex_spawn_models.go@@ -261,6 +261,9 @@ func spawnChoices(r *http.Request, lead string, codexPath bool) []spawnChoice {// Codex's own, as its backend lists them to it, less the ones// taken out of its list on the Agents pagehidden := provider.CodexNativeHidden()+ // and those the codex provider's picks leave out, every one under+ // Usage only (#2530), as codexModels leaves them out of its list+ keep, narrowed := provider.CodexNativePicked()// and the ones kept for routing groups (#1602)all, unlisted := provider.CodexNativeUnlisted()if codexOwnOnly() {@@ -272,7 +275,7 @@ func spawnChoices(r *http.Request, lead string, codexPath bool) []spawnChoice {}var owns []ownfor slug, e := range codexcat.CacheEntries() {- if v, _ := e["visibility"].(string); v != "list" || hidden[slug] || all || unlisted[slug] {+ if v, _ := e["visibility"].(string); v != "list" || hidden[slug] || narrowed && !keep[slug] || all || unlisted[slug] {continue}name, _ := e["display_name"].(string)diff --git a/internal/gateway/codex_usage_only_test.go b/internal/gateway/codex_usage_only_test.gonew file mode 100644index 000000000..c6d4ab8b9--- /dev/null+++ b/internal/gateway/codex_usage_only_test.go@@ -0,0 +1,98 @@+package gateway++import (+ "encoding/json"+ "io"+ "net/http"+ "net/http/httptest"+ "os"+ "path/filepath"+ "slices"+ "testing"++ "github.com/yetone/magpie/internal/provider"+)++// Usage only keeps the ChatGPT account's visible models out of Codex's own+// list, as it does out of the general catalog (#2530): the hidden jobs and+// other providers' models stay. The list's ETag changes with it, so a+// running Codex asks again, and spawn_agent names none of them either.+func TestCodexUsageOnlyCatalog(t *testing.T) {+ codexSignedIn(t)+ // models_cache.json as Codex wrote it from the whole list before+ os.MkdirAll(filepath.Join(os.Getenv("HOME"), ".codex"), 0o700)+ os.WriteFile(filepath.Join(os.Getenv("HOME"), ".codex", "models_cache.json"), []byte(`{"etag":"\"v1\"","models":[{"slug":"gpt-6-sol","visibility":"list","priority":1},{"slug":"gpt-5.5","visibility":"list","priority":2},{"slug":"codex-auto-review","visibility":"hide","priority":3}]}`), 0o600)+ tags := map[string]string{}+ up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ io.WriteString(w, `{"models":[{"slug":"gpt-6-sol","priority":1,"visibility":"list"},{"slug":"gpt-5.5","priority":2,"visibility":"list"},{"slug":"codex-auto-review","priority":3,"visibility":"hide"},{"slug":"gpt-reserve","priority":4,"visibility":"hide"}]}`)+ }))+ t.Cleanup(up.Close)+ was := provider.CodexBase+ provider.CodexBase = up.URL + "/backend-api/codex"+ t.Cleanup(func() { provider.CodexBase = was })+ if err := provider.Save(provider.Provider{ID: "fake", Name: "Fake", Key: "k", Models: []string{"m1"}, Chat: up.URL + "/v1"}); err != nil {+ t.Fatal(err)+ }+ gw := httptest.NewServer(New().Handler())+ t.Cleanup(gw.Close)+ for _, tc := range []struct {+ name string+ p provider.Provider+ want []string+ }{+ {"vendor list", provider.Provider{ID: "codex"}, []string{"gpt-6-sol", "gpt-5.5"}},+ {"usage only", provider.Provider{ID: "codex", NoModels: true}, nil},+ {"picked again", provider.Provider{ID: "codex", Models: []string{"gpt-6-sol"}}, []string{"gpt-6-sol"}},+ {"off keeps native list", provider.Provider{ID: "codex", Off: true, NoModels: true}, []string{"gpt-6-sol", "gpt-5.5"}},+ } {+ t.Run(tc.name, func(t *testing.T) {+ if err := provider.Save(tc.p); err != nil {+ t.Fatal(err)+ }+ req, err := http.NewRequest(http.MethodGet, gw.URL+CodexPath+"/models?client_version=0.163.0", nil)+ if err != nil {+ t.Fatal(err)+ }+ req.Header.Set("Authorization", "Bearer chatgpt-token")+ res, err := gw.Client().Do(req)+ if err != nil {+ t.Fatal(err)+ }+ defer res.Body.Close()+ var list struct {+ Models []struct {+ Slug string `json:"slug"`+ } `json:"models"`+ }+ if err := json.NewDecoder(res.Body).Decode(&list); err != nil {+ t.Fatal(err)+ }+ if res.StatusCode != http.StatusOK {+ t.Fatalf("model list status %d", res.StatusCode)+ }+ var got []string+ for _, m := range list.Models {+ got = append(got, m.Slug)+ }+ want := append(slices.Clone(tc.want), "codex-auto-review", "gpt-reserve", "fake/m1")+ slices.Sort(got)+ slices.Sort(want)+ if !slices.Equal(got, want) {+ t.Fatalf("models = %v, want %v", got, want)+ }+ tags[tc.name] = res.Header.Get("ETag")+ var spawn []string+ for _, c := range spawnChoices(httptest.NewRequest("POST", CodexPath+"/responses", nil), "gpt-6-sol", true) {+ if c.slug == "gpt-6-sol" || c.slug == "gpt-5.5" {+ spawn = append(spawn, c.slug)+ }+ }+ if !slices.Equal(spawn, tc.want) {+ t.Errorf("spawn_agent offers the account's %v, want %v", spawn, tc.want)+ }+ })+ }+ if tags["usage only"] == tags["vendor list"] || tags["picked again"] == tags["usage only"] {+ t.Errorf("ETag kept across Usage only: %q", tags)+ }+}diff --git a/internal/provider/codex_models.go b/internal/provider/codex_models.goindex b965b651e..c5bbe4967 100644--- a/internal/provider/codex_models.go+++ b/internal/provider/codex_models.go@@ -508,6 +508,15 @@ func buildCodexListTag(kind string) string {for _, slug := range off {ms = append(ms, catalog.Model{ID: "-" + slug})}+ // and the account's own kept to the codex provider's picks, none of+ // them under Usage only (#2530), so a change of either has Codex ask+ // for the list again rather than keep the one it has+ if keep, narrowed := CodexNativePicked(); narrowed {+ ms = append(ms, catalog.Model{ID: "~picked"})+ for _, slug := range slices.Sorted(maps.Keys(keep)) {+ ms = append(ms, catalog.Model{ID: "~picked:" + slug})+ }+ }// and those kept out of the lists for routing groups (#1602)all, unlisted := CodexNativeUnlisted()if all {@@ -551,10 +560,17 @@ func buildCodexListTag(kind string) string {//// A provider switched off picks nothing, as it serves no agent anything// (Provider.Off): its picks are kept for when it is switched on again and-// are not a narrowing now.+// are not a narrowing now. Usage only narrows the visible native list to+// none, as it does the general catalog; codexModels keeps hidden jobs.func CodexNativePicked() (map[string]bool, bool) {p, ok := find(All(), "codex")- if !ok || p.Off || len(p.Models) == 0 {+ if !ok || p.Off {+ return nil, false+ }+ if p.ServesNone() {+ return nil, true+ }+ if len(p.Models) == 0 {return nil, false}keep := make(map[string]bool, len(p.Models))