Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

providers: a pool stops at its caps and reads usage again near them (plugins#349)

yetone08b02cf18cac
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.md
index e485b8e13..09bea4efa 100644
--- a/docs/subsystems/gateway-routing.md
+++ b/docs/subsystems/gateway-routing.md
@@ -169,7 +169,7 @@ recorded as a prediction failure (`noPrediction`).
- A rule by agent (`Rule.Agents`, `agents=codex,sillytavern`) matches the agent the gateway names the request's (`agentOf` in [`gateway.go`](../../internal/gateway/gateway.go): its `magpie-<name>` key, else its User-Agent), in lower case (`cleanRules`, `MatchesBesidesIntent`, `Pauses`). A client magpie doesn't know is named by its User-Agent's product name, before the first `/` or space (`usage.AgentOf`), so a rule can hold SillyTavern or one's own app (raikyou on Discord). The editor's agent picker lists the agents magpie knows, then those `/api/usage?period=30d` heard from that it doesn't (not `other`, nor `magpie`), and takes a name typed in its filter as the gateway reads a User-Agent (`uaName` in [`plugins/routing/client.js`](../../internal/gui/assets/plugins/routing/client.js), through the picker's `field.typed`): "MyApp/2.0 (Linux)" is `myapp`. `routing-rule-agent-typed.test.cjs`.
- A pause rule (`Rule.Pause`, `pause=<model>`; John on Discord: a model of the group paused in its vendor's peak hours) leaves the model it names out of the group while it holds (`provider.PausedOut`, on `ruleClock`): the group's own rules and each nested group's on the way down to a member, a group named taking all its models, and a model named taking it wherever a group in the group has it (#625). It runs before planning, on every request rather than once a turn, so the model is not first, not a failover, and not kept by affinity, and comes back as soon as the hours end. It holds by hours, days, agents and folders only (`cleanRules` refuses another condition), and is not a send-to rule: `Matches`, `MatchRule`, `ThenUses`, `Ruled` and the compaction rules leave it out. A send-to rule naming a paused model finds it unready and the group routes as without it. The trace keeps what was paused (`GroupRef.Paused`), and the Routing page says so. Every model paused answers 503 (`pausedError`) naming each rule. Embeddings go by the same pauses; manual groups have no rules (`Live`), so none pause there. A pause by folder (#1606, Ti800: a subscription's quota kept for the long task in one folder, `pause=<model> except-folders=~/big-task`) reads the folder only when such a pause is on the way (`provider.PausesByFolder`, then `workDir`): outside its except-folders, which includes a request that says no folder (a subagent's or a small model's that doesn't), the model is left out; with `folders=`, only requests known to be in them. Embeddings and System One's decision calls have no folder. `Paused.Folder`/`ByFolder` carry what was read, and the Routing page says the folder or that none was said. `rules_pause_test.go`, `TestPauseByFolder`, `TestPauseByFolderRule`, `TestParsePauseByFolder`, `routing-pause-folder.test.cjs`.
- A rule's hours and days (`Rule.Time`, `provider.TimeWindow`) are read on `ruleClock` in the zone they were set in (`TimeWindow.Zone`, an IANA name; gaubeebangeel on X: magpie on a cloud server in UTC, its user in Beijing, the hours 8 off). The GUI saves the browser's zone when the hours or days are changed in the editor and leaves a rule's zone alone otherwise; the CLI (`magpie group rule add … time=09:00-18:00@Asia/Shanghai` or `tz=`) and the TUI save the one typed, or this computer's (`provider.LocalZone`: `$TZ`, `time.Local`, the `/etc/localtime` link) when it has an IANA name. A window with no zone, every one saved before zones were kept, stays on magpie's own clock. A zone magpie doesn't know is refused on save (`TimeWindow.clean`), and one in a file edited by hand makes the window never hold (`zoneOf`): the rule is as if absent, never always on and never hours off. Zones come from Go's embedded tzdata, so Windows knows them too. `/api/groups` gives magpie's clock (`clock`: its zone and offset in seconds), and when it differs from the hours' zone the Routing page tells the zone beside them, on the group's rules and in the editor (`zoneNote` in [`plugins/routing/client.js`](../../internal/gui/assets/plugins/routing/client.js)): the zone's name, or the server's offset for hours with none. `TestRuleTimeWindowInItsZone`, `TestRuleTimeWindowInItsZoneAtTurnStart`, `TestRuleTimeWindowUnknownZone`, `TestParseRuleTimeZone`, `TestGroupRuleTimeZoneCmd`, `routing-time-zone.test.cjs`.
-3. Candidates come from `planGroup` for a group or `plan` for a provider. They are filtered by `sealedReaders` and `allowedCandidates`. An account past its share is held out of the candidates (`capHeld`): a usage cap's share, each window at its own where the account gives it one (`AccountWindowCaps`, willz on Discord: 50% of the five hours, 40% of the week) and at the account's cap otherwise, or 100% of a window for a Codex account set not to spend its credits (`provider.HoldCaps`, `WindowCaps.Of`, `CodexNoCredits`). A ChatGPT account's call on `codex-auto-review`, which OpenAI counts in no window, is held by no usage cap, only by credits it may not spend (`freeOnChatGPT`, #1593, `TestAccountCapLetsAutoReviewThrough`); a Free plan's call is capped as any other (`TestAccountCapHoldsAutoReviewOnAFreePlan`). The account is held while any window is past its own share, and `cappedError` and the trace name the fullest such window's share and cap. When every candidate is held and one held for its credits spends its resets by itself, `autoReset` spends a reset on it and it is tried (`planned.held`); otherwise a capped or barred set returns `cappedError` or `barredError`. Then `affine` moves the conversation's last answerer first, unless it is resting or nearly used up. An `X-Magpie-Account` header (`AccountHeader`) pins the request to one account: only that account is tried, and when it can't answer the caller is told why.
+3. Candidates come from `planGroup` for a group or `plan` for a provider. They are filtered by `sealedReaders` and `allowedCandidates`. An account past its share is held out of the candidates (`capHeld`): a usage cap's share, each window at its own where the account gives it one (`AccountWindowCaps`, willz on Discord: 50% of the five hours, 40% of the week) and at the account's cap otherwise, or 100% of a window for a Codex account set not to spend its credits (`provider.HoldCaps`, `WindowCaps.Of`, `CodexNoCredits`). A ChatGPT account's call on `codex-auto-review`, which OpenAI counts in no window, is held by no usage cap, only by credits it may not spend (`freeOnChatGPT`, #1593, `TestAccountCapLetsAutoReviewThrough`); a Free plan's call is capped as any other (`TestAccountCapHoldsAutoReviewOnAFreePlan`). The account is held while any window is past its own share, and `cappedError` and the trace name the fullest such window's share and cap. When every candidate is held and one held for its credits spends its resets by itself, `autoReset` spends a reset on it and it is tried (`planned.held`); otherwise a capped or barred set returns `cappedError` or `barredError`. A provider with Stop at the caps on (`CapStop`, plugins#349) answers that 429 instead of going on to its fallbacks when every primary account is held (`plan`, `TestCapStopSkipsTheFallbacks`); off, the default, the fallbacks take it as before. A careful provider (`CapCareful`, plugins#349, off by default, `cap_careful.go`) has `readCareful` read a capped account's usage again before planning when its reading is over 15s old within 5 points of a cap, or over 5m old at all, waiting at most 10s and asking a failed account again at most every 15s; `capHold` then holds an account whose usage was never read, whose last read failed or whose reading is still that old (`carefulDoubt`, `Weighed.Unread` in the trace), as one at its cap is: unknown is not spent (`TestCarefulHoldsWhatItCantTell`, `TestCarefulReadsAgainNearTheCap`, `TestCarefulEveryAccountUnread`, `TestCarefulOffByDefault`). Then `affine` moves the conversation's last answerer first, unless it is resting or nearly used up. An `X-Magpie-Account` header (`AccountHeader`) pins the request to one account: only that account is tried, and when it can't answer the caller is told why.
4. The candidate loop calls `attempt` on each candidate in turn, through a `holdWriter`. The writer holds the reply until it is clearly passing. A candidate that fails before anything is sent rests (`failure`): out of credit for 30 minutes, out of quota until reset, rate limited until retry-after, otherwise a minute and longer on repeats: doubled each time, up to `longestRetry` (10 minutes) for a failure and `longestRateRest` (30 minutes) for a rate limit back as soon as its rest is over, or up to the provider's own `restCap` in minutes (`Provider.RestCapOr`, at most `MaxQuotaRest`; `magpie provider rest-cap <id> 4h`, #2008: a free key down for hours was asked every 10 minutes). A time the vendor names stands over it as before (`TestOwnRestCap`). The failure count is kept per key or account (`restKey`), and any of its models' answers clears it (`served`). Each model's own count (`restID`) is kept beside it and cleared only by that model's answer (`servedCandidate`): once it is past the account's, a sibling model answered since it last did, so the failure is the model's and it climbs its own ladder and rests alone, its siblings still served (#2241, weizsw: an Antigravity seat's gemini-3.5-flash-lite rested a minute 41 times of 43 while gemini-3.8-flash kept answering; `TestModelLadderClimbsPastSiblingAnswers`). Every model failing with no answer between still climbs the account's ladder and rests the account, and a candidate naming no model has the account's count alone. `rest-cap <id> off` (`provider.RestCapOff`, `restCap: -1`) is for a relay or account pool whose unknown errors come now and then and pass by the next request (Kismet Alan on Discord): a failure that says nothing of why (`failOther`: a 5xx, a dropped connection) then rests nobody and isn't counted, its trace rest `by: "off"` with no time left. A 401 or 403, a model not served, an account's window full, a rate limit (magpie's own 30-minute cap), quota, credit and a restriction still rest it as before (`TestRestCapOff`). A provider the user added can be set never to rest for a limit instead (`provider.LimitRestOff`, `limitRestOff` in providers.json; the editor's Advanced fold, "Set it aside when out of quota or rate limited", ticked by default): a refusal `restAfterMarked` reads as out of quota, out of credit or rate limited, and any 429, then rests nothing, isn't counted or sunk, and is `by: "off"`; `restDrawer` rests no image model of it. The next request is sent to it as usual. It is for a relay whose limits pass in a moment (zoecats on Discord: a local aggregator said "quota exhausted" while it switched upstreams, and magpie answered itself for 30 minutes; `TestLimitRestOffKeepsAsking`, `TestLimitRestOffRestsNothingForALimit`). A 401, a model not served and a plain failure still rest it. The shortest cap otherwise is a minute, the first rest; `rest-cap <id> 30s` is turned away and names `off`. A 400, 403, 404 or 422 that says the model isn't served (`unservedWords`) and says nothing of quota, credit or a rate limit rests only that key's or account's model (`modelRefused`, keyed by `restID`, its backoff counted apart): its other models are still served first in order (#1235, SenseNova's 403 "model is not available in the current token plan"). A Google account's Code Assist (the Antigravity and Gemini CLI built-ins) answering 404 `NOT_FOUND` ("Requested entity was not found.") rests its model the same way, as its endpoint is the same for every model (`codeAssistModelMissing`; Packing1 on Discord: Argon, listed for a Google AI Pro account, answered 404 and rested the whole account; `TestCodeAssistMissingModelRestsTheModel`). A refusal for want of quota, credit or rate, for an account its vendor restricted, or for a verify it asks of its account there, from another computer's magpie (a Remote magpie provider, LAN share) rests only the model it was about (`remoteModelRest`, keyed by `restID`, in `restAfterMarked` and `rateRest`): that magpie serves each of its models from its own providers and rests them itself, so one out of quota there says nothing of the others (bigsteven on Discord: a 429 for one model rested every model on the remote for the 3 hours it named, so a group never moved on to the remote's other models; `TestRemoteMagpieQuotaRestsTheModel`, `TestRemoteMagpieRestrictedRestsTheModel`). Another magpie added by hand, as an OpenAI-compatible provider of the user's own name rather than the preset, is read as one by its reply: a magpie names the provider that answered or held it (`X-Magpie-Provider`, `providerHeader`; `noteMember`, also on a `quotaHeld` or `verifyHeld` answer), as no vendor does, and `keepRetry` notes it as `answeredThereHeader`, since this magpie writes its own provider into the same header of its reply and the note is kept from the client (lagrangepoint1813 on Discord: "Remote Magpie: Claude Code: You've hit your session limit" answered every model of it, from `quotaHeld`, for the five hours it named; `TestAnotherMagpieAddedByHandRestsTheModel`, asked there and held there). Its key refused, the computer unreachable, or the key's own limit there (`keyLimited`'s 429, which carries `X-Magpie-Limit-Reset`, `remoteKeyLimitHeader`), still rests it whole: #2077's reporter had a Claude Code session limit on the other magpie refuse its Qoder models here (`TestRemoteMagpieRestsTheModel`, two gateways over HTTP; `TestRemoteMagpieKeyLimitRestsTheKey`). The held try keeps that marker only as a note of magpie's own (`keepRetry` sets `keyLimitThereHeader`, which `holdWriter` never passes on), so the rest is whole however the 429 came, and the marker never reaches the agent or a magpie this one answers, which would read it as this magpie's own key limit (#2791, ggbdpq; `TestRemoteMagpieKeyLimitRestsTheRemote`, `TestRemoteMagpieKeyLimitMarkerStaysHere`). A 400, 404, 410 or 422 that says the model is retired (`retiredWords`: OpenCode Zen's 410 `ModelDeprecated` "Model exo-free has been deprecated.", whose own model list still names it; MOMO on Discord) rests only that model the same way, is retryable so a group's next member or another provider of it is asked, and is remembered (`provider.Retire`, `retired.json` beside `providers.json`): the model leaves that provider's entries in the catalog (`buildEntries`), so /v1/models, every agent's model list and the found groups go without it, until a request on it is answered again (`Unretire` in `servedCandidate`) or a week has passed. A refresh of the vendor's list doesn't bring it back early, since that list is where it still is. A `provider/model` request for it is still sent. A 400 or 422 that says this upstream's API can't take the request's shape (`shapeWords`: xAI's 422 "unknown item type" (#350), OpenAI's "Unknown parameter", and a limit of a model's own on tool schemas, OpenCode Go's 400 "JSON schema exceeds the maximum nesting depth of 10 levels" on muse-spark-1.3-contributor, #1870), or that it is served on another API than the one asked (`wrongEndpoint`, once the member has tried every API it has a URL for: #2670, a relay with a Chat URL alone refusing tools with reasoning and `none` too), rests nobody: the next candidate is asked once (`shapeRefused`, `failShape`), and the last one's error reaches the agent. A request missing what every API requires ("field required") isn't one, nor is a 200 that mentions it. Command Code's Provider API turning a request away with nothing but a trace id (`unsaidWords`: `{"message":"invalid request error trace_id: <hex>","type":"invalid_request_error"}`, a 400, or a 422 before) is handled the same way: it turns away a request whose history and asked-for output together are over a limit below the model's listed context (the reporter's A/B: one ~748K-token agent conversation answered 200 with `max_completion_tokens` 64000 and this 400 with 384000, the model's listed max output), which another member's model may take, and the key is fine (#2313, `TestGroupFailsOverCommandCodesBare400`). Its overflow, which names the model's context length before the trace id, and an error that names a `param`, aren't one. A relay's own cap on one request's input that says so (`tooLongRe`: B.ai's 400 "Input token exceed the limit", on a 380K request under the model's 1M window; #2445, worksyang) is taken as an overflow (`overflowed`), which no word list of `retryable` matched: a group's next member that may hold the request is asked (`withRoom`, `failOverflow`), nobody rests, another account of the same model isn't asked, and with no member left the agent is told the conversation is too long, so it compacts. A rate limit worded with input tokens (`paceWords`) isn't one, and a plain bad request is still asked of nobody else (`TestRelayInputCapGoesToTheNextMember`). A subscription account that fails any other way with a window full, by magpie's latest reading of its windows, rests until that window renews. A 401 or 403 is the exception: one that names a quota is out of quota already (`usedUpWords`), and one that doesn't is about the key or the request, so it rests on the backoff ladder whatever the reading says (plugins#243, zygs1083-dotcom: a Factory account read at 98% sat out its week over one plain 403; `TestPlain403DoesNotRestUntilTheWindow`). A plain 403 (`plainForbidden`: no quota, credit, rate, verification, restriction or model not served) from a second key or account of the same provider, model and reasoning in one request is the request refused, not each account: neither rests, the first one's rest and counted failure are taken back (`forbiddenAlike`, `unforbid`), and the provider's other mates are asked last (`matesLast`). Before, that one request walked every account of the provider into backoff (`TestPlain403OnTwoMatesRestsNeither`). One account's plain 403 alone still rests it. Both quota rests and window rests are held to `longestQuota` (8 days), so a reset time read wrong (a year off) doesn't bench the account for that long. The next candidate is then tried. A failed provider waits `fallbackCooldown` (1 minute) at the back. A WorkBuddy account the vendor restricted (#1849: 403 `{"code":11140,"msg":"request illegal","displayMsg":{"en":"Your account has been restricted due to unusual activity",…}}`) rests `restrictedRest` (2 hours) as `restricted` (`failRestricted`), not the minutes of an unknown failure: only WorkBuddy lifting the restriction, after the user's appeal, brings it back. Both shapes are known: the built-in's `wbExplain` and the plugin's `explained()` (0.1.17, the mover's `min`) end the error with `provider.WBRestrictedHint` (`restrictedSaid`), and the vendor's own body, its code 11140 or its words (`provider.WBRestricted`), counts only from a WorkBuddy candidate (`restrictedRefused`, `isWorkBuddy`), as another vendor's 11140 is its own. Bare "request illegal" isn't matched. WorkBuddy's 11128 channel refusal stays `promptRefused` and rests nobody. The Routing page names it "account restricted by the vendor", says the appeal is the way back, and gives the account's row a button that lifts the rest (`/api/gateway/unrest`) once the vendor has. Every other rest a key or account takes (quota, rate limit, retry-after, a failure's backoff) can be lifted by hand the same way (#1869): "Try now" on its row on the Routing page and beside its rest note in the provider's key list. It ends the rest, and with Sink on puts the key back from the back of the order; the backoff count stays, and a refusal after it rests the key again the usual way. Every lift before its time (Try now, `UnrestAll` for a replaced key, `renewed` for windows that renewed) is noted by its rest key (`restingUntil.lifted`) and handed to the Routing page in the trace (`TraceState.Lifted`, `lifted` in `/api/gateway/trace`), and a lift wakes a page waiting on the trace though no route changed (`trace.lifted`). The page stops showing a rest or a sink found before its lift, wherever the lift was asked for (zoecats on Discord: Try now in the provider's editor left Routing at 429 until a restart; `TestTryNowToldToTheRoutingPage`, `routing-lifted-elsewhere.test.cjs`). With another magpie serving the gateway, Try now lifts nothing (#3555). The vendor's safety filter refusing a reply rests nobody, whether it refuses the turn before any of the reply is said or once some of it has been said (`refused`, `refusedAfter`). So does one said in a stream a client is answered whole from (a Codex account is streamed upstream for a `stream:false` Responses, Chat or Messages request): each decoder gives the refusal's error event status 400 and its code (`filterRefused`), a whole reply it cut is answered the refusal rather than its words so far, and Anthropic's error shape, which has no code, carries it in `refusedHeader`; before, it was answered 502 `api_error`, asked again, and each account rested (#3274). `restAfterMarked` itself rests nobody, and counts no failure, for a 400 whose error object `policyRefusal` reads as a refusal, by its code, never by words a body may echo of the request. Its note on the try (and in the error the agent gets when nobody is left) is the vendor's reason, with Anthropic's `stop_details` category and explanation when it gave them (`refusedDetail`, #1768: "safety filter: refusal (category cyber)"); none is made up when it gave none. `stop_details` is carried through the IR (`Event.StopDetails`), so a Claude subscription run through the claude CLI and a translated Anthropic route keep it for an Anthropic client too. How many more candidates a refusal before any of the reply is asked of is Settings' `RefusalRetries` (`refusalAsksAgain`, counted per request in the candidate loop; a group's other members count too): unset, every one left (#248); 0, none (#1925, DD on Discord). When nobody is left, an agent on Anthropic's Messages API gets the held refusal itself, the vendor's 200 with `stop_reason` `refusal` and its `stop_details`, streamed or whole (`holdWriter.handBack`), since that is what Claude Code switches to its fallback model on; a 400 is only an API error to it. Codex and Chat clients still get `failTo`'s 400 (`refusedStatus`), which they don't ask again for. `TestRefusalRetries*` in [`refusal_retries_test.go`](../../internal/gateway/refusal_retries_test.go) cover keys, a group and the Claude subscription. The ChatGPT backend's 502 "response protection is unavailable" is about the request's content, not the account: the same history fails the same on each of its accounts and models. It rests nobody, isn't tried again after a moment, and isn't asked of the provider's other accounts or models; another provider left in a group is asked once (`protectionRefused`, `elsewhere`). One cause is known (#1270): a history that replays the backend's own `web_search_call` in a request that doesn't declare the `web_search` tool is refused so, whatever the account, model or turn (a plain turn, a thread's description, a compaction, Codex's own text compaction for a custom provider with `tools: []`). Every request to the ChatGPT backend with such a history therefore declares cached-only search (`{"type":"web_search","external_web_access":false}`, `provider.DeclareSearch`): an account's in `codexBody`, Codex's own sign-in relayed in `codexUpstreamOn`. It goes in the tools, or for Responses Lite (its header, `reasoning.context` `all_turns`, or an `additional_tools` item already there), which refuses a top-level `web_search`, in the `additional_tools` item. A request that offered no tools gets `tool_choice` `none` so the model can't search; one with a forced choice keeps it. A history without a search, or a request that declares search already, is relayed unchanged; Codex's own `web.run` (a function in the `web` namespace, which Codex offers in place of the hosted tool on a Responses Lite model or with its standalone web search on, and runs through `/alpha/search`) counts as declaring it, since the backend refuses a request carrying both `web.run` and the hosted `web_search` (Savin on Discord, Codex 0.163.0-alpha; `TestSearchedConversationUnderWebRun`). siwc, API-key requests to openai.com and the legacy `/responses/compact` are left as they were. Command Code turning a model away for its Zero Data Retention setting (its plugin's switch, 0.1.14, which sends `x-cmd-zdr: 1`; magpie-community/plugins#246) is a refusal of that request's setting, not of the model or the account: the Provider API's 422 with `"code":"cmd_zdr_no_providers"` and the 400 the plugin makes of Go's error line with `"type":"cmd_zdr_no_providers"` (`zdrCoded`, matched as the error's code or type, never by its words, whose "…a different model.","type":"invalid_request_error" read as a model unserved and rested the model on the account). `keepRetry` notes it beside the error (`zdrHeader`), since the error the agent gets may keep no code (Anthropic's shape, or the 400's numeric code). It rests nobody, anywhere (`restAfterMarked` returns `by: "off"`), so the model answers as soon as the switch is off; it isn't asked of the provider's other accounts at that model, which carry the same header (`notMates`), nor again after a moment; another member, the provider's other models among them, is asked once, and with none left the agent gets Command Code's words (`TestCommandCodeZDRRefusalRestsNothing`, `TestZDRRefusalNeverRests`). Another vendor's refusal in the same words, without the code, still rests the model (`TestZDRRefusalIsCommandCodesCodeOnly`).
5. The last candidate's error reaches the agent, with one exception for a group (#1418, `patience.go`), which a plain provider shares once its same-one retries are spent (below). When every member has failed in a way that passes (`busy`: 408, 500, 502, 503, 504 or 529 that isn't a proxy, a quota, response protection, or an upstream that answered a stream request with a whole reply, which fails the same however often it is asked (`didNotStream`; the same-one retries of `passing` leave it alone too, `TestStreamlessMemberIsAskedOnce`), or a 429 that is a short rate limit, not a plan or credit), and nothing of the reply has been sent, the members that failed so are asked again in turn after a pause (`replan`). The pause is `retryPause` doubled each round up to `replanLongest()` (30 times `retryPause`, 30s), and never shorter than the soonest Retry-After; a member whose Retry-After ends past the pause is asked after the others. This goes on while the group's `Patience` lasts. A member of a group in the group uses the innermost that sets `Patience` (`innerPatience`, #2410), as FirstToken (#2246) and Sink (#2255) do; unset falls back to the group asked for, and an inner `PatienceOff` skips replan for that member (`TestNestedGroupPatienceThroughOuter`, `TestNestedGroupPatienceInnerOff`, `TestNestedGroupPatienceFallsBackToOuter`). `replanRounds` (12) is only a backstop that never comes first: the pauses alone add up to `PatienceLongest` in 11 rounds (with 6, a group set to 240s gave up after 90s of pauses, real-jacket on #1418; `TestPatienceIsUsedUp`). The budget is the group's `Patience`, counted from the first time every member had failed, when the agent would have had the error (`waitFrom`; `Group.Waits`: 0 is `PatienceDefault`, 60s; -1, `PatienceOff`, turns it off; at most `PatienceLongest`, 240s). A pause that would end past the budget isn't taken, and the agent gets the error; a member the vendor said is back within a second past its budget (read in whole seconds) is asked as the budget ends (`TestReporterResetFitsTheDefaultPatience`). While another member is busy too, the last one isn't first tried again alone (`passing`): the members take turns, and once the budget has begun, the same-one retries aren't taken either. While the budget lasts, the last candidate's failure is held (`newHoldWriter`), and a streaming agent is kept alive through the pause with SSE comments (`pauseAlive`). The try is traced with `Again` (the pause), `Replan` (how many are asked again) and `Patience` (the budget in seconds). The group editor's "Every member busy" row sets it. A plain provider (a `provider/model` request, its accounts and fallbacks the candidates) first takes the same-one retries `passing` gives the last one left: `lastRetries` for a busy vendor, `rateRetries` for a rate limit, each pause `retryPause<<n` or a Retry-After of at most `longestPause` (8s). Past those, or for a Retry-After longer than `longestPause`, it is replanned the same way, its accounts taking turns as members do, for the provider's own `Patience` (`Provider.Patience`, the same values as a group's, set with `magpie provider set <id> patience=2m|off|default`; the provider editor shows no row for it and its save keeps it, `TestProviderSaveKeepsPatience`), unset `singlePatience` (`PatienceDefault`, 60s), counted from when the agent would have had the error, so it works with no setting and the user's value wins either way (`TestSingleProviderOwnPatienceWins`; ChatGPT's reset of a large request, `codexReset`, is left out: Codex is told it at its 503 with what may help, as `codexBusy` leaves it). A provider's own Patience is for a request to it alone: a group it is in, set or unset, keeps the group's (zoecats on Discord: a local relay's short 429 went straight to Claude Code once the few retries were spent). So a 429 whose Retry-After is 20s is waited out, the stream kept alive past `keepHeldAfter`; one of an hour, a quota, credit or plan limit, a 401 or 403, a refusal, response protection's 502 and `didNotStream` are not `busy` or don't fit the budget, and reach the agent at once as the vendor said them. Its tries are traced with `Again` alone, no `Replan` or `Patience`, so the Routing page tells it as the last one left asked again rather than as a group's members (`TestSingleProviderWaitsOutAShortRateLimit`, `TestSingleProviderPassesALongLimitThrough`, `TestSingleProviderAgentGoneInTheWait` in [`single_busy_test.go`](../../internal/gateway/single_busy_test.go)). Codex's own one-account relay (`codexUpstreamOn`, below) doesn't join it: its request is relayed unchanged outside the candidate loop, and the backend's 429 is always the plan's usage limit (`codexBusy` never retries one), so the short rate limit this waits out doesn't arise there; its overloads keep their `lastRetries` and then the subscription's fallbacks. An agent that goes away in either pause (the same-one retry's or a group's replan) ends the request there with 499: nobody is asked again, and the try whose pause it was is marked `Gone` (#1894, heng91: a relay down while its proxy was off, the agent's window closed). The Routing page says the agent canceled the request before the retry and lists it as canceled, rather than ending its story on "tried again in 2s", a retry that never came (`TestAgentGoneInTheRetryPause`, `routing-agent-gone.test.cjs`). A try that answered is recorded (`served`, `answered`) for routing, affinity and usage. A try is added to the route's `Tries` only as it is asked, so a later try is the record that the gateway went on from this one, and the Routing page tells what came of a try from that, never from where it stood in the order (#2281, xishuchkui-code: an Antigravity 502 empty reply that went on to the next account was told as "an error another account wouldn't fix, so Claude Code gets it"). The one case the record can't show is the last try left out of its allowance after an earlier failure that wasn't one, where the agent is told the earlier error: that try is marked `Earlier` (`routing-went-on.test.cjs`).
diff --git a/docs/subsystems/providers-accounts.md b/docs/subsystems/providers-accounts.md
index ff8d940e6..1184951c2 100644
--- a/docs/subsystems/providers-accounts.md
+++ b/docs/subsystems/providers-accounts.md
@@ -244,7 +244,7 @@ and `TestHistoricalWindowHoldsUsesEachWindowsOwnReading`.
- Not read for Vertex AI: the metadata server of Compute Engine, Cloud Run and GKE; workload identity federation (`external_account`, `external_account_authorized_user`, refused by name); credentials whose `universe_domain` isn't `googleapis.com`; Vertex AI's express-mode API keys (Save refuses a key). No quota project is sent: Vertex AI bills the project in the URL.
- `GOOGLE_APPLICATION_CREDENTIALS` and `CLOUDSDK_CONFIG` are in `agentenv.Vars`. The desktop app takes them from the login shell when it wasn't started with them (`proc.UserPath`, macOS and Linux), and `testenv` clears both. Vertex AI reads them as set (`vertexEnvPath`), not through `appdir.Getenv`, which takes a relative value as unset: a `~/` that starts one is the home, and one that still isn't a full path is refused, naming the variable. Passed over, it would have the request signed from gcloud's own file, which may be another account's (a person's, where the variable named a service account's key).
- The Vertex AI preset's `HeaderHints` are Google's Priority PayGo and Flex PayGo headers, `X-Vertex-AI-LLM-Request-Type` and `X-Vertex-AI-LLM-Shared-Request-Type`: the editor offers to add them, the values are the user's, and they go as `Headers` do. Priority PayGo couldn't be confirmed: on the project it was tried on, Vertex AI answered with `trafficType` `ON_DEMAND`.
-- A usage cap is kept per account (`AccountCaps`, 1–99) and, beside it, per window (`AccountWindowCaps`, by the account and the window's `capId`, its name in lower case: `WindowCapID`). A window's own share wins over the account's; 100 is no cap on that window. `CapsOf` gives both (`WindowCaps`), `CapHeld` and `capReached` judge each window by its own, and `SetWindowCap` (GUI `provider/windowcap`, `magpie provider account-cap <id> <account> --window <name>`) sets one. A config without window caps reads as before.
+- A usage cap is kept per account (`AccountCaps`, 1–99) and, beside it, per window (`AccountWindowCaps`, by the account and the window's `capId`, its name in lower case: `WindowCapID`). A window's own share wins over the account's; 100 is no cap on that window. `CapsOf` gives both (`WindowCaps`), `CapHeld` and `capReached` judge each window by its own, and `SetWindowCap` (GUI `provider/windowcap`, `magpie provider account-cap <id> <account> --window <name>`) sets one. A config without window caps reads as before. How a pool ends at its caps is kept with the provider (plugins#349): `CapStop` answers 429 rather than going on to the fallbacks once every account in use is at its cap, and `CapCareful` reads usage again near a cap and holds an account whose usage can't be told (both off by default; `SetCapStop`, `SetCapCareful`, GUI `provider/capstop` and `provider/capcareful`, ticks under the account list once an account has a cap). `UsageRead` says when an account's usage was last read well and whether the last try failed (a failed try keeps the good reading's time), and `ReadUsageNow` reads it again. `magpie provider pool <id> [--accounts a,b,…] [--cap percent|off] [--stop on|off] [--careful on|off]` shows and sets the whole pool in one command: which accounts are on and in what order, each one's cap, and both settings, and names the accounts in use with no cap.
- A window whose reset time has passed has started again, whatever share it was read at (`resetPassed`). That holds for a reading kept from before (`AsOf`) and for a fresh one: Claude Code's `/usage` gives a reset to the minute and can lag it, so a run just after a reset can still say 100% with that reset. `claudeWindows` hands such a window back empty on every path, and `usedPast`, `usedUp`, `BackAt` and `capReached` leave it out when they judge an account spent, for Codex, Claude Code and `magpie quota wait` alike. A card can still show a kept reading as it was read (`reported`, for a Claude card while `/usage` is unavailable); that changes what it shows, never which account is used. A reset `/usage` gives as a time alone ("resets 3pm") can be read as its next one, a day on (`claudeResetTime`), and is then not seen as passed here.
- A restart to update can run two magpies for up to the old one's drain
([GUI shell](gui-shell.md#a-restart-to-update-hands-the-gateway-over)), each
diff --git a/internal/cli/pool_cli.go b/internal/cli/pool_cli.go
new file mode 100644
index 000000000..878f697a2
--- /dev/null
+++ b/internal/cli/pool_cli.go
@@ -0,0 +1,237 @@
+package cli
+
+// `magpie provider pool`: a subscription's accounts as one pool, the
+// GUI's account list in one command (plugins#349) — which take part and in
+// what order, each one's cap, whether a request ends at the caps or goes
+// on to the fallbacks, and whether usage is read again near a cap.
+
+import (
+ "fmt"
+ "slices"
+ "strconv"
+ "strings"
+
+ "github.com/yetone/magpie/internal/agent"
+ "github.com/yetone/magpie/internal/provider"
+)
+
+const poolUsage = "magpie provider pool <id> [--accounts a,b,…] [--cap percent|off] [--stop on|off] [--careful on|off]"
+
+// poolCmd shows a subscription's pool, after setting what the flags say:
+// --accounts puts those accounts in use in that order, the rest out of
+// it; --cap caps each account in use (the listed ones, with --accounts);
+// --stop answers 429 while every account is at its cap rather than going
+// on to the fallbacks (provider.Provider.CapStop); --careful reads usage
+// again near a cap and holds an account whose usage can't be told
+// (provider.Provider.CapCareful).
+func poolCmd(rest []string) error {
+ usage := fmt.Errorf("%s", poolUsage)
+ var id string
+ var accounts []string
+ accountsSet, capSet := false, false
+ capN := 0
+ var stop, careful *bool
+ onOff := func(v string) (*bool, error) {
+ switch strings.ToLower(strings.TrimSpace(v)) {
+ case "on", "true", "yes":
+ b := true
+ return &b, nil
+ case "off", "false", "no":
+ b := false
+ return &b, nil
+ }
+ return nil, fmt.Errorf("%q: on or off\n%s", v, poolUsage)
+ }
+ for i := 0; i < len(rest); i++ {
+ a := rest[i]
+ name, val, hasVal := strings.Cut(a, "=")
+ if !strings.HasPrefix(name, "--") {
+ if id != "" {
+ return usage
+ }
+ id = a
+ continue
+ }
+ if !hasVal {
+ if i+1 >= len(rest) {
+ return usage
+ }
+ i++
+ val = rest[i]
+ }
+ var err error
+ switch name {
+ case "--accounts":
+ accountsSet = true
+ for _, u := range strings.Split(val, ",") {
+ if u = strings.TrimSpace(u); u != "" {
+ accounts = append(accounts, u)
+ }
+ }
+ if len(accounts) == 0 {
+ return fmt.Errorf("--accounts names at least one account\n%s", poolUsage)
+ }
+ case "--cap":
+ capSet = true
+ if capN, err = provider.ParseCap(val); err != nil {
+ return err
+ }
+ case "--stop":
+ stop, err = onOff(val)
+ case "--careful":
+ careful, err = onOff(val)
+ default:
+ return usage
+ }
+ if err != nil {
+ return err
+ }
+ }
+ if id == "" {
+ return usage
+ }
+ p, err := provider.Find(id)
+ if err != nil {
+ return err
+ }
+ if p.Account == nil {
+ return fmt.Errorf("%s has keys, not subscription accounts with usage windows to pool", p.Name)
+ }
+ agentID := poolAgent(*p)
+ ls := provider.Logins(agentID)
+ if accountsSet {
+ // every one named is known before anything changes
+ var order []string
+ for _, u := range accounts {
+ i := slices.IndexFunc(ls, func(l provider.Login) bool { return strings.EqualFold(l.User, u) })
+ if i < 0 {
+ return fmt.Errorf("%s has no account %q", p.Name, u)
+ }
+ order = append(order, ls[i].User)
+ }
+ for _, l := range ls {
+ if !slices.ContainsFunc(order, func(u string) bool { return strings.EqualFold(u, l.User) }) {
+ order = append(order, l.User)
+ }
+ }
+ if err := provider.SetAccountOrder(p.ID, order); err != nil {
+ return err
+ }
+ moved, err := agent.Reseat(func() error {
+ // on first, so the one the agent is signed in to can be
+ // paused once another is on
+ for _, u := range order[:len(accounts)] {
+ if err := provider.SetLoginOn(agentID, u, true); err != nil {
+ return err
+ }
+ }
+ for _, u := range order[len(accounts):] {
+ if err := provider.SetLoginOn(agentID, u, false); err != nil {
+ return err
+ }
+ }
+ return nil
+ })
+ if err != nil {
+ return err
+ }
+ printMoved(moved)
+ agent.SyncCatalogSoon()
+ ls = provider.Logins(agentID)
+ }
+ if capSet {
+ var users []string
+ if accountsSet {
+ users = accounts
+ } else {
+ for _, l := range ls {
+ if poolOn(l) {
+ users = append(users, l.User)
+ }
+ }
+ }
+ for _, u := range users {
+ if err := provider.SetAccountCap(p.ID, u, capN); err != nil {
+ return err
+ }
+ }
+ }
+ if stop != nil {
+ if err := provider.SetCapStop(p.ID, *stop); err != nil {
+ return err
+ }
+ }
+ if careful != nil {
+ if err := provider.SetCapCareful(p.ID, *careful); err != nil {
+ return err
+ }
+ }
+ if accountsSet || capSet || stop != nil || careful != nil {
+ fmt.Println(green.Render("✓"), "saved", p.Name, muted.Render("("+p.ID+")"))
+ if p, err = provider.Find(p.ID); err != nil {
+ return err
+ }
+ }
+ printPool(*p, ls)
+ return nil
+}
+
+// poolAgent is the agent p's accounts are listed and switched under: a
+// plugin's by its provider's id.
+func poolAgent(p provider.Provider) string {
+ if p.IsPlugin() {
+ return p.ID
+ }
+ return p.Account.Agent
+}
+
+// poolOn says whether l takes part in the pool: on, the one signed in
+// unless paused.
+func poolOn(l provider.Login) bool {
+ return l.On && !l.Paused || l.Active && !l.Paused
+}
+
+// printPool shows p's pool: each account in order, whether it is in use
+// and its cap, then how the pool ends at its caps.
+func printPool(p provider.Provider, ls []provider.Login) {
+ if len(ls) == 0 {
+ fmt.Println(muted.Render(p.Name + " has no account"))
+ return
+ }
+ uncapped := 0
+ for i, l := range ls {
+ mark := muted.Render("off")
+ if poolOn(l) {
+ mark = green.Render("on ")
+ }
+ caps := p.CapsOf(l.User)
+ capText := muted.Render("no cap: used to 100%")
+ switch {
+ case caps.All > 0:
+ capText = "capped at " + strconv.Itoa(caps.All) + "%"
+ case len(caps.Windows) > 0:
+ capText = "capped on some windows"
+ default:
+ if poolOn(l) {
+ uncapped++
+ }
+ }
+ fmt.Printf("%d. %s %s · %s\n", i+1, mark, l.User, capText)
+ }
+ stop := "on: a request is answered 429 while every account in use is at its cap"
+ if !p.CapStop {
+ stop = "off: at the caps, a request goes on to the fallbacks"
+ if len(p.Fallback) == 0 {
+ stop = "off: at the caps, a request goes on to the fallbacks (none set, so it is answered 429)"
+ }
+ }
+ careful := "on: near a cap (within 5 points) usage is read again if older than 15s, any reading older than 5m too, and an account whose usage can't be read is held"
+ if !p.CapCareful {
+ careful = "off: routing goes by the last reading, read again each minute"
+ }
+ fmt.Println("stop at caps ·", stop)
+ fmt.Println("check usage near the cap ·", careful)
+ if uncapped > 0 {
+ fmt.Println(muted.Render(fmt.Sprintf("%d account(s) in use have no cap, so nothing holds them before the vendor does: magpie provider pool %s --cap 90", uncapped, p.ID)))
+ }
+}
diff --git a/internal/cli/pool_cli_test.go b/internal/cli/pool_cli_test.go
new file mode 100644
index 000000000..9fa059d21
--- /dev/null
+++ b/internal/cli/pool_cli_test.go
@@ -0,0 +1,75 @@
+package cli
+
+import (
+ "encoding/base64"
+ "encoding/json"
+ "os"
+ "path/filepath"
+ "testing"
+ "time"
+
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// magpie provider pool (plugins#349) sets a subscription's caps, Stop at
+// the caps and Check usage near the cap from one command, and shows them;
+// a bad value or an account the subscription hasn't changes nothing.
+func TestPoolCmd(t *testing.T) {
+ home := t.TempDir()
+ t.Setenv("HOME", home)
+ t.Setenv("USERPROFILE", home)
+ t.Setenv("XDG_CONFIG_HOME", t.TempDir())
+ t.Setenv("XDG_CACHE_HOME", t.TempDir())
+ t.Setenv("DSH_HOME", "")
+ jwt := func(m map[string]any) string {
+ b, _ := json.Marshal(m)
+ return "h." + base64.RawURLEncoding.EncodeToString(b) + ".s"
+ }
+ auth, _ := json.Marshal(map[string]any{"auth_mode": "chatgpt", "tokens": map[string]any{
+ "id_token": jwt(map[string]any{"email": "[email protected]"}),
+ "access_token": jwt(map[string]any{"exp": time.Now().Add(time.Hour).Unix()}),
+ "refresh_token": "r", "account_id": "acct-1"}})
+ os.MkdirAll(filepath.Join(home, ".codex"), 0o755)
+ if err := os.WriteFile(filepath.Join(home, ".codex", "auth.json"), auth, 0o600); err != nil {
+ t.Fatal(err)
+ }
+ provider.ForgetAccounts()
+ t.Cleanup(provider.ForgetAccounts)
+
+ if err := providerCmd([]string{"provider", "pool", "codex"}); err != nil {
+ t.Fatal(err)
+ }
+ if p, _ := provider.Find("codex"); p.CapStop || p.CapCareful {
+ t.Fatalf("on by default: stop %v careful %v", p.CapStop, p.CapCareful)
+ }
+ if err := providerCmd([]string{"provider", "pool", "codex", "--cap", "85%", "--stop=on", "--careful", "on"}); err != nil {
+ t.Fatal(err)
+ }
+ p, _ := provider.Find("codex")
+ if c := provider.AccountCapOf("codex", "[email protected]"); c != 85 || !p.CapStop || !p.CapCareful {
+ t.Fatalf("cap %d stop %v careful %v", c, p.CapStop, p.CapCareful)
+ }
+ for _, bad := range [][]string{
+ {"provider", "pool", "codex", "--stop", "maybe"},
+ {"provider", "pool", "codex", "--cap", "150"},
+ {"provider", "pool", "codex", "--accounts", "[email protected]", "--careful", "off"},
+ {"provider", "pool", "codex", "--careful"},
+ {"provider", "pool", "codex", "--fast", "on"},
+ {"provider", "pool"},
+ } {
+ if err := providerCmd(bad); err == nil {
+ t.Fatalf("%v taken", bad)
+ }
+ }
+ p, _ = provider.Find("codex")
+ if c := provider.AccountCapOf("codex", "[email protected]"); c != 85 || !p.CapStop || !p.CapCareful {
+ t.Fatalf("a refused command changed it: cap %d stop %v careful %v", c, p.CapStop, p.CapCareful)
+ }
+ if err := providerCmd([]string{"provider", "pool", "codex", "--accounts", "[email protected]", "--cap", "off", "--stop", "off"}); err != nil {
+ t.Fatal(err)
+ }
+ p, _ = provider.Find("codex")
+ if c := provider.AccountCapOf("codex", "[email protected]"); c != 0 || p.CapStop || !p.CapCareful {
+ t.Fatalf("cap %d stop %v careful %v", c, p.CapStop, p.CapCareful)
+ }
+}
diff --git a/internal/cli/providers_cli.go b/internal/cli/providers_cli.go
index dadad973c..65461a3a1 100644
--- a/internal/cli/providers_cli.go
+++ b/internal/cli/providers_cli.go
@@ -76,6 +76,10 @@ const providerUsage = `usage:
magpie provider account-cap <id> <account> --window <name> [percent|none|default]
a share for one window alone (e.g. "5 hours" 50): none is no cap on it,
default has it follow the account's cap
+ magpie provider pool <id> [--accounts a,b,…] [--cap percent|off] [--stop on|off] [--careful on|off]
+ a subscription's accounts as one pool: which are in use and in what order,
+ each one's cap, --stop: answer 429 at the caps rather than go on to the
+ fallbacks, --careful: read usage again near a cap, holding what can't be read
magpie provider account-concurrency <id> [account|key [n|off|default]]
how many requests one account or key has out at once, over every model,
routing group and agent: its own, off for none, default for the provider's
@@ -602,6 +606,8 @@ func providerCmd(args []string) error {
return accountModelsCmd(rest)
case "account-cap", "account-caps":
return accountCapCmd(rest)
+ case "pool":
+ return poolCmd(rest)
case "account-concurrency", "account-limit":
return accountConcurrencyCmd(rest)
case "queue":
diff --git a/internal/gateway/cap_careful.go b/internal/gateway/cap_careful.go
new file mode 100644
index 000000000..e7d221aef
--- /dev/null
+++ b/internal/gateway/cap_careful.go
@@ -0,0 +1,163 @@
+package gateway
+
+// Careful caps (provider.Provider.CapCareful, plugins#349): a capped
+// account's usage is read again before a request goes to it near its cap,
+// and while its usage can't be told it is held, as one at its cap is:
+// unknown is not spent. Off, the default, routing goes by the last
+// reading, read again each minute in the background.
+
+import (
+ "context"
+ "fmt"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// What careful means, as the thread pins no figures: an account within
+// carefulNear points of a cap is read again when its reading is older than
+// carefulFresh, any capped account when older than carefulOld, and a
+// request waits at most carefulWait for the reading.
+const (
+ carefulNear = 5.0
+ carefulFresh = 15 * time.Second
+ carefulOld = 5 * time.Minute
+ carefulWait = 10 * time.Second
+ // carefulRetry: an account whose reading failed is asked again at
+ // most this often, held meanwhile, so a vendor's usage endpoint that
+ // is down isn't asked at every request
+ carefulRetry = 15 * time.Second
+)
+
+// usageRead and readUsageNow are provider's, stood in for by tests.
+var (
+ usageRead = provider.UsageRead
+ readUsageNow = provider.ReadUsageNow
+)
+
+// carefulDoubt says why the account of a careful provider, capped by caps,
+// can't be sent model at now on what is known of its usage: never read
+// well, its last read failed, its reading older than carefulOld, or older
+// than carefulFresh within carefulNear points of a cap; "" when it can.
+func carefulDoubt(a *provider.Account, al provider.Allowance, known bool, model string, caps provider.WindowCaps, now time.Time) string {
+ at, failed, ok := usageRead(a.UsageAgent(), a.User)
+ age := now.Sub(at)
+ switch {
+ case !ok:
+ return "its usage hasn't been read yet"
+ case failed:
+ return "its usage couldn't be read"
+ case at.IsZero() || !known:
+ return "its usage hasn't been read yet"
+ case age > carefulOld:
+ return fmt.Sprintf("its usage was last read %s ago", age.Round(time.Second))
+ case age > carefulFresh && al.NearCap(model, caps, carefulNear, now):
+ return fmt.Sprintf("it is near its cap and its usage was last read %s ago", age.Round(time.Second))
+ }
+ return ""
+}
+
+// carefulTried: when each account's usage was last read again for a
+// careful provider, by agent/user (carefulRetry).
+var carefulTried struct {
+ sync.Mutex
+ at map[string]time.Time
+}
+
+// readCareful reads again, before a request is planned, the usage of each
+// capped account of the careful providers among ps (with the model each
+// is asked for) that carefulDoubt would hold, and waits for it at most
+// carefulWait: a reading back in time lets the plan send to it.
+func readCareful(ctx context.Context, ps []provider.Provider, models []string) {
+ now := time.Now()
+ users := map[string][]string{} // usage agent → accounts
+ carefulTried.Lock()
+ if carefulTried.at == nil {
+ carefulTried.at = map[string]time.Time{}
+ }
+ for i, p := range ps {
+ if !p.CapCareful || p.Account == nil {
+ continue
+ }
+ for _, acct := range append([]provider.Provider{p}, p.AlsoOn()...) {
+ a := acct.Account
+ if a == nil {
+ continue
+ }
+ caps := provider.HoldCaps(p, a.Agent, a.User)
+ if !caps.Holds() {
+ continue
+ }
+ al, known := allowances(a.UsageAgent())[a.User]
+ if carefulDoubt(a, al, known, models[i], caps, now) == "" {
+ continue
+ }
+ key := a.UsageAgent() + "/" + strings.ToLower(a.User)
+ if now.Sub(carefulTried.at[key]) < carefulRetry {
+ continue
+ }
+ carefulTried.at[key] = now
+ users[a.UsageAgent()] = append(users[a.UsageAgent()], a.User)
+ }
+ }
+ carefulTried.Unlock()
+ if len(users) == 0 {
+ return
+ }
+ ctx, cancel := context.WithTimeout(ctx, carefulWait)
+ defer cancel()
+ var wg sync.WaitGroup
+ for agent, us := range users {
+ wg.Go(func() { readUsageNow(ctx, agent, us...) })
+ }
+ wg.Wait()
+}
+
+// carefulOf is the providers a request for model on p may go to, with the
+// model each is asked: p, then its fallbacks, as plan takes them; a
+// group's members.
+func carefulOf(p provider.Provider, model string, ms []provider.Member) ([]provider.Provider, []string) {
+ var ps []provider.Provider
+ var models []string
+ if ms != nil {
+ for _, m := range ms {
+ ps, models = append(ps, m.Provider), append(models, m.Model)
+ }
+ return ps, models
+ }
+ ps, models = append(ps, p), append(models, model)
+ for _, id := range p.Fallback {
+ if fp, fm, ok := provider.Resolve(id); ok {
+ ps, models = append(ps, fp), append(models, fm)
+ }
+ }
+ return ps, models
+}
+
+// anyCareful says whether any of ps is careful, so a request to none of
+// them reads nothing more.
+func anyCareful(ps []provider.Provider) bool {
+ for _, p := range ps {
+ if p.CapCareful {
+ return true
+ }
+ }
+ return false
+}
+
+// capShare is the cap caps hold an account at, for saying so: its own,
+// else the lowest a window of it has, else 100 (credits not spent).
+func capShare(caps provider.WindowCaps) int {
+ if caps.All >= provider.MinCap && caps.All <= provider.MaxCap {
+ return caps.All
+ }
+ share := 100
+ for _, v := range caps.Windows {
+ if v >= provider.MinCap && v < share {
+ share = v
+ }
+ }
+ return share
+}
diff --git a/internal/gateway/cap_pool_test.go b/internal/gateway/cap_pool_test.go
new file mode 100644
index 000000000..b73ebc240
--- /dev/null
+++ b/internal/gateway/cap_pool_test.go
@@ -0,0 +1,279 @@
+package gateway
+
+import (
+ "context"
+ "slices"
+ "strings"
+ "sync"
+ "testing"
+ "time"
+
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// plugins#349: a pool of capped accounts that ends at its caps (CapStop)
+// and reads usage again near them (CapCareful).
+
+// poolRead stands in for when each account's usage was last read, and
+// whether that read failed, by user; readUsageNow is stood in for by
+// again, which is told the users asked and may change the readings. A
+// user with no entry was never read.
+type poolRead struct {
+ mu sync.Mutex
+ at map[string]time.Time
+ failed map[string]bool
+ asked []string
+ again func(user string)
+}
+
+func standInReads(t *testing.T) *poolRead {
+ t.Helper()
+ r := &poolRead{at: map[string]time.Time{}, failed: map[string]bool{}}
+ oldRead, oldNow := usageRead, readUsageNow
+ t.Cleanup(func() { usageRead, readUsageNow = oldRead, oldNow })
+ usageRead = func(_, user string) (time.Time, bool, bool) {
+ r.mu.Lock()
+ defer r.mu.Unlock()
+ at, ok := r.at[user]
+ return at, r.failed[user], ok
+ }
+ readUsageNow = func(_ context.Context, _ string, users ...string) bool {
+ r.mu.Lock()
+ defer r.mu.Unlock()
+ r.asked = append(r.asked, users...)
+ for _, u := range users {
+ if r.again != nil {
+ r.again(u)
+ }
+ }
+ return true
+ }
+ carefulTried.Lock()
+ carefulTried.at = nil
+ carefulTried.Unlock()
+ return r
+}
+
+func (r *poolRead) set(user string, at time.Time, failed bool) {
+ r.mu.Lock()
+ defer r.mu.Unlock()
+ r.at[user], r.failed[user] = at, failed
+}
+
+func (r *poolRead) askedFor() []string {
+ r.mu.Lock()
+ defer r.mu.Unlock()
+ return slices.Clone(r.asked)
+}
+
+// carefulPool: Codex's two accounts, me first in the order with a 90% cap
+// and spare with none, their five hours used as given.
+func carefulPool(t *testing.T, me, spare float64, careful bool) *resetBackend {
+ t.Helper()
+ codexSignedIn(t, "[email protected]")
+ b := newResetBackend(t)
+ capUsage(t, map[string]float64{"[email protected]": me, "[email protected]": spare})
+ if err := provider.SetRouting("codex", provider.Ordered); err != nil {
+ t.Fatal(err)
+ }
+ if err := provider.SetAccountCap("codex", "[email protected]", 90); err != nil {
+ t.Fatal(err)
+ }
+ if careful {
+ if err := provider.SetCapCareful("codex", true); err != nil {
+ t.Fatal(err)
+ }
+ }
+ return b
+}
+
+// Off, the default: an account under its cap takes the request on its
+// last reading, however old or failed that read, and nothing is read
+// again before it.
+func TestCarefulOffByDefault(t *testing.T) {
+ b := carefulPool(t, 88, 10, false)
+ r := standInReads(t)
+ r.set("[email protected]", time.Now().Add(-time.Hour), true)
+ if p, _ := provider.Find("codex"); p.CapCareful || p.CapStop {
+ t.Fatalf("on by default: careful %v stop %v", p.CapCareful, p.CapStop)
+ }
+ code, body := resetPost(t, New())
+ if tried, _ := b.seen(); code != 200 || !strings.Contains(body, "pong") || tried != "acct-1" {
+ t.Fatalf("%d %s tried %s", code, body, tried)
+ }
+ if asked := r.askedFor(); len(asked) != 0 {
+ t.Fatalf("read again with careful off: %v", asked)
+ }
+}
+
+// On, an account whose usage read failed, was never read, or was read
+// over five minutes ago is held, as one at its cap is, and the request
+// goes to the next; a fresh good reading lets it through.
+func TestCarefulHoldsWhatItCantTell(t *testing.T) {
+ for _, c := range []struct {
+ name string
+ read func(*poolRead)
+ want string
+ }{
+ {"failed", func(r *poolRead) { r.set("[email protected]", time.Now(), true) }, "acct-2"},
+ {"never read", func(*poolRead) {}, "acct-2"},
+ {"stale", func(r *poolRead) { r.set("[email protected]", time.Now().Add(-10*time.Minute), false) }, "acct-2"},
+ {"fresh", func(r *poolRead) { r.set("[email protected]", time.Now(), false) }, "acct-1"},
+ } {
+ t.Run(c.name, func(t *testing.T) {
+ b := carefulPool(t, 50, 10, true)
+ r := standInReads(t)
+ c.read(r)
+ srv := New()
+ code, body := resetPost(t, srv)
+ if tried, _ := b.seen(); code != 200 || !strings.Contains(body, "pong") || tried != c.want {
+ t.Fatalf("%d %s tried %s, want %s", code, body, tried, c.want)
+ }
+ if c.want == "acct-1" {
+ return
+ }
+ // read again first, the reading still not told: held
+ if asked := r.askedFor(); !slices.Equal(asked, []string{"[email protected]"}) {
+ t.Fatalf("read again for %v", asked)
+ }
+ var unread []string
+ for _, rt := range srv.Trace(t.Context(), 0, 0).Routes {
+ for _, w := range rt.Left {
+ if w.Unread != "" {
+ unread = append(unread, w.Who+": "+w.Unread)
+ }
+ }
+ }
+ if len(unread) != 1 || !strings.HasPrefix(unread[0], "[email protected]: its usage") {
+ t.Fatalf("traced as unread: %v", unread)
+ }
+ })
+ }
+}
+
+// On, an account within five points of its cap on a reading over 15
+// seconds old is read again before the request: a reading back under the
+// cap sends it there, one at the cap moves on. Far from its cap, a
+// minute-old reading is used as it is.
+func TestCarefulReadsAgainNearTheCap(t *testing.T) {
+ t.Run("far from the cap", func(t *testing.T) {
+ b := carefulPool(t, 50, 10, true)
+ r := standInReads(t)
+ r.set("[email protected]", time.Now().Add(-time.Minute), false)
+ if code, _ := resetPost(t, New()); code != 200 {
+ t.Fatal(code)
+ }
+ if tried, _ := b.seen(); tried != "acct-1" || len(r.askedFor()) != 0 {
+ t.Fatalf("tried %s, read again for %v", tried, r.askedFor())
+ }
+ })
+ t.Run("read again, room left", func(t *testing.T) {
+ b := carefulPool(t, 87, 10, true)
+ r := standInReads(t)
+ r.set("[email protected]", time.Now().Add(-time.Minute), false)
+ r.again = func(u string) { r.at[u] = time.Now() }
+ if code, _ := resetPost(t, New()); code != 200 {
+ t.Fatal(code)
+ }
+ if tried, _ := b.seen(); tried != "acct-1" || !slices.Equal(r.askedFor(), []string{"[email protected]"}) {
+ t.Fatalf("tried %s, read again for %v", tried, r.askedFor())
+ }
+ })
+ t.Run("read again, at the cap", func(t *testing.T) {
+ b := carefulPool(t, 87, 10, true)
+ r := standInReads(t)
+ r.set("[email protected]", time.Now().Add(-time.Minute), false)
+ r.again = func(u string) {
+ r.at[u] = time.Now()
+ capUsage(t, map[string]float64{"[email protected]": 91, "[email protected]": 10})
+ }
+ if code, _ := resetPost(t, New()); code != 200 {
+ t.Fatal(code)
+ }
+ if tried, _ := b.seen(); tried != "acct-2" || !slices.Equal(r.askedFor(), []string{"[email protected]"}) {
+ t.Fatalf("tried %s, read again for %v", tried, r.askedFor())
+ }
+ })
+ t.Run("the read fails", func(t *testing.T) {
+ b := carefulPool(t, 87, 10, true)
+ r := standInReads(t)
+ r.set("[email protected]", time.Now().Add(-time.Minute), false)
+ r.again = func(u string) { r.failed[u] = true }
+ if code, _ := resetPost(t, New()); code != 200 {
+ t.Fatal(code)
+ }
+ if tried, _ := b.seen(); tried != "acct-2" {
+ t.Fatalf("tried %s", tried)
+ }
+ })
+}
+
+// On, with every account capped and none of their usage readable, the
+// client is answered 429 and nothing reaches the vendor; the message says
+// why and how to turn it off.
+func TestCarefulEveryAccountUnread(t *testing.T) {
+ b := carefulPool(t, 50, 10, true)
+ if err := provider.SetAccountCap("codex", "[email protected]", 90); err != nil {
+ t.Fatal(err)
+ }
+ r := standInReads(t)
+ r.set("[email protected]", time.Now(), true)
+ r.set("[email protected]", time.Now(), true)
+ srv := New()
+ code, body := resetPost(t, srv)
+ if tried, _ := b.seen(); code != 429 || tried != "" {
+ t.Fatalf("%d tried %q: %s", code, tried, body)
+ }
+ for _, want := range []string{"rate_limit_error", "couldn't be read", "--careful off"} {
+ if !strings.Contains(body, want) {
+ t.Errorf("no %q in %s", want, body)
+ }
+ }
+}
+
+// Every account at its cap, with a fallback set: it takes the request,
+// as before; with Stop at the caps on, the client is answered 429 and
+// neither the vendor nor the fallback is sent anything.
+func TestCapStopSkipsTheFallbacks(t *testing.T) {
+ b := carefulPool(t, 95, 92, false)
+ if err := provider.SetAccountCap("codex", "[email protected]", 90); err != nil {
+ t.Fatal(err)
+ }
+ fb := &keyed{}
+ serveOn(t, "spare", "kf", []string{"m2"}, fb)
+ p, _ := provider.Find("codex")
+ p.Fallback = []string{"spare/m2"}
+ if err := provider.Save(*p); err != nil {
+ t.Fatal(err)
+ }
+ post := func() (int, string) {
+ return postAs(t, New(), "", `{"model":"codex/gpt-5.5","messages":[{"role":"user","content":"hi"}]}`)
+ }
+ code, body := post()
+ if tried, _ := b.seen(); code != 200 || !strings.Contains(body, "from kf") || tried != "" {
+ t.Fatalf("stop off: %d %s, codex tried %q", code, body, tried)
+ }
+ if err := provider.SetCapStop("codex", true); err != nil {
+ t.Fatal(err)
+ }
+ fb.tried = nil
+ code, body = post()
+ if tried, _ := b.seen(); code != 429 || tried != "" || len(fb.tried) != 0 {
+ t.Fatalf("stop on: %d %s, codex tried %q, fallback %v", code, body, tried, fb.tried)
+ }
+ for _, want := range []string{"rate_limit_error", "usage cap reached", "past its 90% cap"} {
+ if !strings.Contains(body, want) {
+ t.Errorf("no %q in %s", want, body)
+ }
+ }
+ // an account with room still takes it (the first request's stick to
+ // the fallback let go, as it would be once the session moves on)
+ capUsage(t, map[string]float64{"[email protected]": 50, "[email protected]": 92})
+ sticks.Lock()
+ sticks.m = map[string]stick{}
+ sticks.Unlock()
+ code, body = post()
+ if tried, _ := b.seen(); code != 200 || tried != "acct-1" || len(fb.tried) != 0 {
+ t.Fatalf("room left: %d %s, codex tried %q, fallback %v", code, body, tried, fb.tried)
+ }
+}
diff --git a/internal/gateway/fallback.go b/internal/gateway/fallback.go
index f73c60cd1..f365dd70f 100644
--- a/internal/gateway/fallback.go
+++ b/internal/gateway/fallback.go
@@ -79,6 +79,9 @@ type capHold struct {
used float64
back time.Time
noCredits bool
+ // unread: held by a careful provider as its usage can't be told, why
+ // (carefulDoubt), cap then the account's own
+ unread string
}
// label names a candidate in a call's record: the provider, and the key
@@ -422,6 +425,11 @@ func (s *Server) plan(p provider.Provider, model string, from provider.Protocol)
return cs
}
out := add(p, model, false)
+ if p.CapStop && len(out) == 0 && len(pl.capped)+len(pl.held) > 0 {
+ // every account held at its cap: the user set the provider to end
+ // there (plugins#349), answered 429 rather than sent on
+ return restLast(out, pl)
+ }
seen := map[string]bool{p.ID + "/" + model: true}
for _, id := range p.Fallback {
fp, fm, ok := provider.Resolve(id)
@@ -713,8 +721,16 @@ func capHeld(p, acct provider.Provider, model string, now time.Time) *capHold {
if !caps.Holds() {
return nil
}
- h := allowances(a.UsageAgent())[a.User].CapHeld(model, caps, now)
+ al, known := allowances(a.UsageAgent())[a.User]
+ h := al.CapHeld(model, caps, now)
if h == nil {
+ if p.CapCareful {
+ // what can't be told is not spent (plugins#349)
+ if why := carefulDoubt(a, al, known, model, caps, now); why != "" {
+ used, _ := al.For(model, now)
+ return &capHold{cap: capShare(caps), used: used, unread: why}
+ }
+ }
return nil
}
return &capHold{cap: h.Cap, used: h.Used, back: h.Back, noCredits: h.Credits}
@@ -726,6 +742,9 @@ func capHeld(p, acct provider.Provider, model string, now time.Time) *capHold {
func cappedError(model string, ws []Weighed, now time.Time) (string, time.Time) {
held, soonest := heldReasons(ws, now)
capped, noCredits := heldBy(ws)
+ if unread(ws) {
+ return fmt.Sprintf("usage cap reached: every account that serves %q is held at the usage cap set on it in magpie, or held as its usage can't be told right now — %s. magpie reads it again at the next request and uses it once it is under its cap; %s, or turn off Check usage near the cap (Providers → the provider's accounts, or magpie provider pool <id> --careful off)", model, strings.Join(held, "; "), heldAdvice(capped, noCredits)), soonest
+ }
switch {
case !noCredits:
return fmt.Sprintf("usage cap reached: every account that serves %q is held at the usage cap set on it in magpie — %s. magpie uses it again once that window renews; %s", model, strings.Join(held, "; "), heldAdvice(capped, noCredits)), soonest
@@ -758,6 +777,9 @@ func heldReasons(ws []Weighed, now time.Time) ([]string, time.Time) {
if w.NoCredits {
s = fmt.Sprintf("%s (%s) has used up a usage window and is set not to spend its credits", w.Name, w.Who)
}
+ if w.Unread != "" {
+ s = fmt.Sprintf("%s (%s) is held, as %s and its provider is set to check usage near the cap", w.Name, w.Who, w.Unread)
+ }
if w.CapBack != nil {
s += ", until " + heldClock(*w.CapBack, now)
if soonest.IsZero() || w.CapBack.Before(soonest) {
@@ -806,6 +828,9 @@ func checkpointCappedRecord(ws []Weighed) string {
// cappedRecord is the call's record of the same refusal as cappedError,
// told apart the same way.
func cappedRecord(ws []Weighed) string {
+ if unread(ws) {
+ return "every account at its usage cap or its usage unread"
+ }
switch capped, noCredits := heldBy(ws); {
case !noCredits:
return "every account at its usage cap"
@@ -815,6 +840,17 @@ func cappedRecord(ws []Weighed) string {
return "every account held: at its usage cap or set not to spend its credits"
}
+// unread says whether one of ws is held as its usage can't be told
+// (carefulDoubt).
+func unread(ws []Weighed) bool {
+ for _, w := range ws {
+ if w.Capped > 0 && w.Unread != "" {
+ return true
+ }
+ }
+ return false
+}
+
// creditsHeld are those of cs held only as they won't spend their
// credits: out of their allowance at the vendor's own 100%, where one
// that spends its resets by itself spends one (codex_autoreset.go). One
@@ -850,7 +886,7 @@ func barredOf(cs []candidate, q provider.Provider, fallback bool, from provider.
w := weighed(c, q, weighing{}, fallback, from)
w.Unlisted, w.Via = true, via
if h := c.capped; h != nil {
- w.Capped, w.Used, w.NoCredits = h.cap, h.used, h.noCredits
+ w.Capped, w.Used, w.NoCredits, w.Unread = h.cap, h.used, h.noCredits, h.unread
if !h.back.IsZero() {
w.CapBack = &h.back
}
diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.go
index 0e279f17c..36f989561 100644
--- a/internal/gateway/gateway.go
+++ b/internal/gateway/gateway.go
@@ -2092,6 +2092,17 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
return
}
}
+ // a careful provider's capped accounts read again now where what is
+ // known of their usage is too old to send on (plugins#349)
+ if isGroup {
+ if ps, models := carefulOf(p, model, ms); anyCareful(ps) {
+ readCareful(r.Context(), ps, models)
+ }
+ } else if p.CapCareful || len(p.Fallback) > 0 {
+ if ps, models := carefulOf(p, model, nil); anyCareful(ps) {
+ readCareful(r.Context(), ps, models)
+ }
+ }
// the primary, then its fallbacks while it can't take the request and
// nothing has been sent yet
var cands []candidate
diff --git a/internal/gateway/plugin_cap_pool_test.go b/internal/gateway/plugin_cap_pool_test.go
new file mode 100644
index 000000000..a54bac31f
--- /dev/null
+++ b/internal/gateway/plugin_cap_pool_test.go
@@ -0,0 +1,229 @@
+package gateway
+
+import (
+ "context"
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "net/http/httptest"
+ "os"
+ "path/filepath"
+ "strings"
+ "sync"
+ "testing"
+ "time"
+
+ "github.com/yetone/magpie/internal/plugin"
+ "github.com/yetone/magpie/internal/provider"
+ "github.com/yetone/magpie/internal/testenv"
+)
+
+// plugins#349 through a real plugin host: the fake plugin as Factory, two
+// accounts in order, each capped at 90%, their usage read from the
+// plugin's usage call as it says it now (FAKE_WINDOWS). With Check usage
+// near a cap on, a usage call that fails holds both accounts: the agent is
+// answered 429 and nothing reaches the vendor; off, the last reading is
+// used as before; once usage reads again, A serves. With Stop at the caps
+// on, both at their caps, the fallback isn't sent the request either.
+func TestPluginCarefulPool(t *testing.T) {
+ bun := testenv.Bun(t)
+ fresh(t)
+ t.Setenv("MAGPIE_BUN", bun)
+ t.Setenv("FAKE_ID", "factory")
+ windows := filepath.Join(t.TempDir(), "windows.json")
+ t.Setenv("FAKE_WINDOWS", windows)
+ t.Cleanup(plugin.Settle)
+ old := allowances
+ allowances = provider.Allowances
+ t.Cleanup(func() { allowances = old })
+ carefulTried.Lock()
+ carefulTried.at = nil
+ carefulTried.Unlock()
+
+ var mu sync.Mutex
+ var tried []string
+ up := testenv.FakeVendor(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ io.ReadAll(r.Body)
+ who := strings.TrimPrefix(r.Header.Get("Authorization"), "Bearer fresh-r-")
+ mu.Lock()
+ tried = append(tried, who)
+ mu.Unlock()
+ w.Header().Set("Content-Type", "text/event-stream")
+ fmt.Fprint(w, `data: {"id":"c","object":"chat.completion.chunk","model":"fake-1","choices":[{"index":0,"delta":{"role":"assistant","content":"from `+who+`"},"finish_reason":"stop"}]}`+"\n\n")
+ fmt.Fprint(w, "data: [DONE]\n\n")
+ }))
+ defer up.Close()
+
+ dir := filepath.Dir(provider.Path())
+ if err := os.MkdirAll(dir, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ mig, _ := json.Marshal(map[string]any{"factory": map[string]any{"state": provider.MovePlugin}})
+ if err := os.WriteFile(filepath.Join(dir, "migrations.json"), mig, 0o600); err != nil {
+ t.Fatal(err)
+ }
+ resets := time.Now().Add(2 * time.Hour)
+ five := func(used float64) []map[string]any {
+ return []map[string]any{{"name": "5 hours", "used": used, "span": 18000, "resetsAt": resets.UTC().Format(time.RFC3339)}}
+ }
+ write := func(t *testing.T, raw []byte) {
+ t.Helper()
+ if err := os.WriteFile(windows+".tmp", raw, 0o600); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Rename(windows+".tmp", windows); err != nil {
+ t.Fatal(err)
+ }
+ }
+ // says writes what the plugin says of each account, and waits for
+ // routing to have read it: a's share, or with broken, a failed read
+ says := func(t *testing.T, a, b float64, broken bool) {
+ t.Helper()
+ if broken {
+ write(t, []byte("{not json"))
+ } else {
+ raw, _ := json.Marshal(map[string]any{"a@fake": five(a), "b@fake": five(b)})
+ write(t, raw)
+ }
+ for _, u := range []string{"a@fake", "b@fake"} {
+ provider.StaleAllowance("plugin:factory", u)
+ }
+ deadline := time.Now().Add(time.Minute)
+ for {
+ got := provider.Allowances("plugin:factory")
+ _, failed, ok := provider.UsageRead("plugin:factory", "a@fake")
+ if broken && ok && failed || !broken && ok && !failed && len(got["a@fake"]) > 0 && got["a@fake"][0].Used == a && len(got["b@fake"]) > 0 && got["b@fake"][0].Used == b {
+ return
+ }
+ if time.Now().After(deadline) {
+ t.Fatalf("routing never read the windows (broken %v): %+v", broken, got)
+ }
+ time.Sleep(10 * time.Millisecond)
+ }
+ }
+ raw, _ := json.Marshal(map[string]any{"a@fake": five(40), "b@fake": five(10)})
+ write(t, raw)
+
+ ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
+ defer cancel()
+ abs, _ := filepath.Abs("../plugin/testdata/fake/index.js")
+ if _, err := plugin.Add(ctx, abs); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := plugin.Providers(ctx); err != nil {
+ t.Fatal(err)
+ }
+ for _, team := range []string{"a", "b"} {
+ st, err := provider.StartPluginSignIn("factory", 1, map[string]string{"where": "work", "team": team})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := provider.SubmitSignInCallback(st.ID, "good"); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if _, err := plugin.Providers(ctx); err != nil {
+ t.Fatal(err)
+ }
+ if ls := provider.Logins("factory"); len(ls) != 2 {
+ t.Fatalf("accounts %+v", ls)
+ }
+ if err := provider.SwitchLogin("factory", "a@fake"); err != nil {
+ t.Fatal(err)
+ }
+ if err := provider.SetLoginOn("factory", "b@fake", true); err != nil {
+ t.Fatal(err)
+ }
+ if err := provider.SetRouting("factory", provider.Ordered); err != nil {
+ t.Fatal(err)
+ }
+ for _, u := range []string{"a@fake", "b@fake"} {
+ if err := provider.SetAccountCap("factory", u, 90); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if p, _ := provider.Find("factory"); p.CapCareful || p.CapStop {
+ t.Fatalf("on by default: careful %v stop %v", p.CapCareful, p.CapStop)
+ }
+
+ s := New()
+ ask := func(t *testing.T) (int, string, []string) {
+ t.Helper()
+ mu.Lock()
+ tried = nil
+ mu.Unlock()
+ sticks.Lock()
+ sticks.m = map[string]stick{}
+ sticks.Unlock()
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(`{"model":"factory/fake-1","stream":true,"messages":[{"role":"user","content":"hello"}]}`))
+ s.Handler().ServeHTTP(rec, req)
+ mu.Lock()
+ defer mu.Unlock()
+ return rec.Code, rec.Body.String(), append([]string(nil), tried...)
+ }
+ served := func(t *testing.T, want string) {
+ t.Helper()
+ code, body, seen := ask(t)
+ if code != 200 || !strings.Contains(body, "from "+want) || strings.Join(seen, " ") != want {
+ t.Fatalf("%d %s (tried %v), want %s alone", code, body, seen, want)
+ }
+ }
+ careful := func(t *testing.T, on bool) {
+ t.Helper()
+ if err := provider.SetCapCareful("factory", on); err != nil {
+ t.Fatal(err)
+ }
+ carefulTried.Lock()
+ carefulTried.at = nil
+ carefulTried.Unlock()
+ }
+
+ says(t, 40, 10, false)
+ served(t, "a")
+
+ // usage can't be read: careful holds both, nothing reaches Factory
+ says(t, 0, 0, true)
+ careful(t, true)
+ code, body, seen := ask(t)
+ if code != 429 || len(seen) != 0 {
+ t.Fatalf("careful, unread: %d %s (tried %v)", code, body, seen)
+ }
+ for _, want := range []string{"rate_limit", "couldn't be read", "a@fake", "b@fake", "--careful off"} {
+ if !strings.Contains(body, want) {
+ t.Errorf("no %q in %s", want, body)
+ }
+ }
+ // off: the last reading (A at 40%) is used, as before
+ careful(t, false)
+ served(t, "a")
+
+ // read again: careful sends to A
+ careful(t, true)
+ raw, _ = json.Marshal(map[string]any{"a@fake": five(40), "b@fake": five(10)})
+ write(t, raw)
+ served(t, "a")
+
+ // both at their caps with a fallback: it takes the request, unless
+ // Stop at the caps is on
+ careful(t, false)
+ fb := &keyed{}
+ serveOn(t, "spare", "kf", []string{"m2"}, fb)
+ p, _ := provider.Find("factory")
+ p.Fallback = []string{"spare/m2"}
+ if err := provider.Save(*p); err != nil {
+ t.Fatal(err)
+ }
+ says(t, 95, 92, false)
+ if code, body, seen := ask(t); code != 200 || !strings.Contains(body, "from kf") || len(seen) != 0 {
+ t.Fatalf("stop off: %d %s (tried %v)", code, body, seen)
+ }
+ if err := provider.SetCapStop("factory", true); err != nil {
+ t.Fatal(err)
+ }
+ fb.tried = nil
+ if code, body, seen := ask(t); code != 429 || len(seen) != 0 || len(fb.tried) != 0 || !strings.Contains(body, "usage cap reached") {
+ t.Fatalf("stop on: %d %s (tried %v, fallback %v)", code, body, seen, fb.tried)
+ }
+}
diff --git a/internal/gateway/trace.go b/internal/gateway/trace.go
index 9de5ef492..92e339ea0 100644
--- a/internal/gateway/trace.go
+++ b/internal/gateway/trace.go
@@ -300,6 +300,10 @@ type Weighed struct {
// NoCredits: held at 100% (Capped), a Codex account the user set not
// to spend its credits once its allowance is used up
NoCredits bool `json:"noCredits,omitempty"`
+ // Unread: held (Capped) by a careful provider as its usage can't be
+ // told, why: not read yet, the read failed, or too old near its cap
+ // (provider.Provider.CapCareful)
+ Unread string `json:"unread,omitempty"`
// Credits: a Codex account that goes on past its allowance on its
// credits, and is let spend them: all but used up, it still answers,
// and goes before those used up with nothing to go on with
diff --git a/internal/gui/assets/app.css b/internal/gui/assets/app.css
index 40c8f8a31..a1c9c8f3e 100644
--- a/internal/gui/assets/app.css
+++ b/internal/gui/assets/app.css
@@ -2504,8 +2504,9 @@ body.window .ag-sk-icon { width: 22px; height: 22px; border-radius: 6px; }
.accts { display: grid; grid-template-columns: minmax(0, 1fr); border: 1px solid var(--line); border-radius: 10px; background: var(--card); overflow: hidden; }
.acc { display: flex; align-items: center; gap: 9px; min-height: 38px; padding: 5px 8px 5px 11px; font-size: 12.5px; text-align: left; }
.acc + .acc, .accts .signing { border-top: 1px solid var(--line-2); }
-.editor .accts .keep-login, .editor .accts .switch-only { display: flex; align-items: center; flex-wrap: wrap; gap: 6px; min-height: 34px; padding: 6px 11px; font-size: 12px; border-top: 1px solid var(--line-2); }
+.editor .accts .keep-login, .editor .accts .switch-only, .editor .accts .pool-ends { display: flex; align-items: center; flex-wrap: wrap; gap: 6px; min-height: 34px; padding: 6px 11px; font-size: 12px; border-top: 1px solid var(--line-2); }
.editor .accts .keep-login .keep-as { max-width: 240px; }
+.editor .accts .pool-ends .note { flex-basis: 100%; color: var(--fg-2); }
.acc .n { font-weight: 500; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.acc .plan { color: var(--muted); font-size: 11px; padding: 1px 6px; border-radius: 5px; background: var(--pill); white-space: nowrap; }
.acc .grow { flex: 1; }
diff --git a/internal/gui/assets/app.js b/internal/gui/assets/app.js
index 8cfabeda5..fc4a55327 100644
--- a/internal/gui/assets/app.js
+++ b/internal/gui/assets/app.js
@@ -12890,6 +12890,13 @@ function renderAccounts(a, p) {
for (const l of ls) {
const on = !unusable(l) && !l.paused && (l.active || l.on);
const row = el("div", "acc" + (on ? " in-use" : " off") + (l.user === justAdded ? " new" : ""));
+ // its Remove, titled as given; true: the own account's, else the plain one
+ const forgetOf = (title) => {
+ const f = el("button", "text quiet", t("Remove"));
+ if (title) f.title = title !== true ? title : l.own ? forgetOwnTitle(a) : t("magpie forgets this account's sign-in; the account itself is untouched");
+ f.onclick = () => accountAction("login/forget", { agent: a.agent, user: l.user }, t("{user} removed", { user: l.user }));
+ return f;
+ };
row.dataset.accountId = l.user;
const dot = el("button", "dot tick");
if (on) dot.append(svg(CHECK, 10, 2.2));
@@ -12933,19 +12940,14 @@ function renderAccounts(a, p) {
row.append(again);
// the one Claude Code is signed in to can't be forgotten (ForgetLogin)
if (!l.active) {
- const forget = el("button", "text quiet", t("Remove"));
- forget.onclick = () => accountAction("login/forget", { agent: a.agent, user: l.user }, t("{user} removed", { user: l.user }));
- row.append(forget);
+ row.append(forgetOf());
}
} else if (signedOut(l)) {
row.classList.add("signed-out");
row.append(el("span", "using", t("Sign-in required")));
const again = el("button", "text", t("Sign in again"));
again.onclick = () => startSignIn(a.agent, false, againSite(l), { user: l.user });
- const forget = el("button", "text quiet", t("Remove"));
- forget.title = l.own ? forgetOwnTitle(a) : t("magpie forgets this account's sign-in; the account itself is untouched");
- forget.onclick = () => accountAction("login/forget", { agent: a.agent, user: l.user }, t("{user} removed", { user: l.user }));
- row.append(again, forget);
+ row.append(again, forgetOf(true));
} else if (l.active && kept && l.user !== firstUser) {
// signed in to, kept so, and tried at its place in the order
const signed = el("span", "using", l.paused ? t("Paused") : t("Signed in"));
@@ -12957,10 +12959,7 @@ function renderAccounts(a, p) {
if (back && !l.paused) using.title = t("{user} was nearly used up, so magpie signed {agent} in to this one; it goes back to {user} once that has room again", { user: back.user, agent: a.agentName });
row.append(using);
if (a.agent === "qoder" || a.agent === "qoder-cn" || l.own) {
- const forget = el("button", "text quiet", t("Remove"));
- if (l.own) forget.title = forgetOwnTitle(a);
- forget.onclick = () => accountAction("login/forget", { agent: a.agent, user: l.user }, t("{user} removed", { user: l.user }));
- row.append(forget);
+ row.append(forgetOf(l.own));
} else if (a.agent === "codex" && ls.length === 1) {
// Codex's only account: no other to sign it in to first, so it is
// signed out, as codex logout does (ForgetLogin)
@@ -12971,15 +12970,10 @@ function renderAccounts(a, p) {
} else if (a.agent === "codex") {
// Codex is signed in to another of its accounts first, then this
// one goes (ForgetLogin), rather than Codex signed out
- const forget = el("button", "text quiet", t("Remove"));
- forget.title = t("Codex is signed in to another of its accounts, and magpie forgets this one; the account itself is untouched");
- forget.onclick = () => accountAction("login/forget", { agent: a.agent, user: l.user }, t("{user} removed", { user: l.user }));
- row.append(forget);
+ row.append(forgetOf(t("Codex is signed in to another of its accounts, and magpie forgets this one; the account itself is untouched")));
}
} else {
- const forget = el("button", "text quiet", t("Remove"));
- forget.title = l.own ? forgetOwnTitle(a) : t("magpie forgets this account's sign-in; the account itself is untouched");
- forget.onclick = () => accountAction("login/forget", { agent: a.agent, user: l.user }, t("{user} removed", { user: l.user }));
+ const forget = forgetOf(true);
if (l === back) {
const again = el("span", "using", t("First again once it has room"));
again.title = t("magpie signs {agent} back in to this account once it has room again", { agent: a.agentName });
@@ -13060,24 +13054,30 @@ function renderAccounts(a, p) {
box.append(only);
list.append(box);
}
+ // how a pool ends at its caps (plugins#349)
+ const capped = (l) => accountCapOf(p, l.user) || windowCapsOf(p, l.user);
+ if (p && (ls.some(capped) || p.capStop || p.capCareful)) {
+ const box = el("div", "pool-ends");
+ for (const [k, say, tip] of [["capStop", "Stop when every account is at its cap", "On: once every account in use is at its cap, requests get 429 instead of going to the fallbacks."],
+ ["capCareful", "Check usage again near a cap", "On: an account within 5 points of its cap has its usage read again if older than 15s (any account: 5m) before a request, waiting up to 10s; one whose usage can't be read is held."]]) {
+ const [l, c] = tick(t(say), !!p[k]);
+ l.title = t(tip);
+ c.onchange = () => accountAction("provider/" + k.toLowerCase(), { id: p.id, [k]: c.checked }, t("Saved"));
+ box.append(l);
+ }
+ const open = ls.filter((l) => !unusable(l) && !l.paused && (l.active || l.on) && !capped(l)).map((l) => l.user);
+ if (open.length) box.append(el("span", "note", t("No cap on {users}: used to 100% of each window", { users: open.join(", ") })));
+ list.append(box);
+ }
const pv = a.agent === "codex" ? providers || {} : {};
if (pv.codexDaemon) list.append(renderCodexDaemon(pv.codexDaemon));
if (pv.codexApp) list.append(renderCodexApp(pv.codexApp));
if (signing?.agent === a.agent) list.append(renderSigning(sub));
else {
- const add = el("button", "acc add");
- const ic = el("span", "dot");
- ic.append(svg(PLUS, 10, 1.8));
- add.append(ic, el("span", "n", t(sub.single && sub.plugin ? "Sign in again" : sub.single ? "Sign in to another {name} account" : "Add another {name} account", { name: sub.name })));
- add.onclick = () => startSignIn(a.agent);
- list.append(add);
+ list.append(addRow(t(sub.single && sub.plugin ? "Sign in again" : sub.single ? "Sign in to another {name} account" : "Add another {name} account", { name: sub.name }), () => startSignIn(a.agent)));
if (sub.importable) {
- const imp = el("button", "acc add");
- const ic2 = el("span", "dot");
- ic2.append(svg(PLUS, 10, 1.8));
- imp.append(ic2, el("span", "n", importSay(a.agent).row || t("Import accounts from a file…")));
+ const imp = addRow(importSay(a.agent).row || t("Import accounts from a file…"), () => startImport(a.agent));
imp.title = importSay(a.agent).from;
- imp.onclick = () => startImport(a.agent);
list.append(imp);
}
}
@@ -13574,6 +13574,14 @@ function accountNameButton(p, user) {
wrap.append(hideAccounts.eye(user));
return wrap;
}
+// an account or key list's last row, adding one
+function addRow(say, go) {
+ const b = el("button", "acc add"), ic = el("span", "dot");
+ ic.append(svg(PLUS, 10, 1.8));
+ b.append(ic, el("span", "n", say));
+ b.onclick = go;
+ return b;
+}
function accountCapOf(p, user) { return p?.accountCaps?.[String(user).toLowerCase()] || 0; }
// A window's own cap (provider.AccountWindowCaps, willz on Discord): a
// friend's account stops at 50% of its five hours while its week may go to
@@ -14433,12 +14441,7 @@ function renderKeyAccounts(p) {
list.append(box);
queueMicrotask(() => key.focus());
} else {
- const add = el("button", "acc add");
- const ic = el("span", "dot");
- ic.append(svg(PLUS, 10, 1.8));
- add.append(ic, el("span", "n", t("Add another key")));
- add.onclick = () => { addingKey = { id: p.id, name: "", key: "" }; renderProviders(); };
- list.append(add);
+ list.append(addRow(t("Add another key"), () => { addingKey = { id: p.id, name: "", key: "" }; renderProviders(); }));
}
// rows folded away keep their places when the shown ones are dragged
if (fold.folded) list.accountOrder = p.keyList.map((k) => k.id);
diff --git a/internal/gui/assets/i18n.js b/internal/gui/assets/i18n.js
index a214eeb68..77f28d93d 100644
--- a/internal/gui/assets/i18n.js
+++ b/internal/gui/assets/i18n.js
@@ -1599,6 +1599,11 @@ const I18N = {
"Keep {agent} signed in to the first account": "{agent} 始终登录首选账号",
"Keep {agent} signed in to": "{agent} 始终登录",
"Only switch {agent}'s sign-in": "只切换 {agent} 的登录",
+ "Stop when every account is at its cap": "所有账号都到上限时停止",
+ "On: once every account in use is at its cap, requests get 429 instead of going to the fallbacks.": "开启:在用的账号全部到达上限后,请求返回 429,不再转到备用。",
+ "Check usage again near a cap": "接近上限时重新读取用量",
+ "On: an account within 5 points of its cap has its usage read again if older than 15s (any account: 5m) before a request, waiting up to 10s; one whose usage can't be read is held.": "开启:距上限 5 个百分点以内的账号,用量读数超过 15 秒(任何账号超过 5 分钟)时,请求前重新读取,最多等 10 秒;读不到用量的账号暂停使用。",
+ "No cap on {users}: used to 100% of each window": "{users} 没有上限:每个窗口都会用到 100%",
"magpie signs {agent} in to the next account with room when the one it is on is used up, and back once that has room again, by writing {agent}'s own sign-in; {agent}'s requests go straight to OpenAI, never through magpie. A {agent} app or session already open stays on its account until it is opened again. Connected to magpie, {agent}'s requests go through it as before.": "当前账号额度用完时,magpie 改写 {agent} 自己的登录,把它切到下一个还有额度的账号,等原账号额度恢复后再切回;{agent} 的请求直接发往 OpenAI,不经过 magpie。已经打开的 {agent} 应用或会话会停留在原账号,直到重新打开。{agent} 接入 magpie 时,请求仍照常经过 magpie。",
"magpie only switches {agent}'s sign-in; its requests go straight to OpenAI": "magpie 只切换 {agent} 的登录;它的请求直接发往 OpenAI",
"{agent}'s requests go through magpie to fail over to your other accounts again": "{agent} 的请求重新经过 magpie,以便转到你的其他账号",
@@ -5623,6 +5628,11 @@ const I18N = {
"Keep {agent} signed in to the first account": "{agent} 始終登入首選帳號",
"Keep {agent} signed in to": "{agent} 始終登入",
"Only switch {agent}'s sign-in": "只切換 {agent} 的登入",
+ "Stop when every account is at its cap": "所有帳號都到上限時停止",
+ "On: once every account in use is at its cap, requests get 429 instead of going to the fallbacks.": "開啟:使用中的帳號全部到達上限後,請求回傳 429,不再轉到備援。",
+ "Check usage again near a cap": "接近上限時重新讀取用量",
+ "On: an account within 5 points of its cap has its usage read again if older than 15s (any account: 5m) before a request, waiting up to 10s; one whose usage can't be read is held.": "開啟:距上限 5 個百分點以內的帳號,用量讀數超過 15 秒(任何帳號超過 5 分鐘)時,請求前重新讀取,最多等 10 秒;讀不到用量的帳號暫停使用。",
+ "No cap on {users}: used to 100% of each window": "{users} 沒有上限:每個視窗都會用到 100%",
"magpie signs {agent} in to the next account with room when the one it is on is used up, and back once that has room again, by writing {agent}'s own sign-in; {agent}'s requests go straight to OpenAI, never through magpie. A {agent} app or session already open stays on its account until it is opened again. Connected to magpie, {agent}'s requests go through it as before.": "目前帳號額度用完時,magpie 改寫 {agent} 自己的登入,把它切到下一個還有額度的帳號,等原帳號額度恢復後再切回;{agent} 的請求直接送往 OpenAI,不經過 magpie。已經開啟的 {agent} 應用程式或工作階段會停留在原帳號,直到重新開啟。{agent} 接入 magpie 時,請求仍照常經過 magpie。",
"magpie only switches {agent}'s sign-in; its requests go straight to OpenAI": "magpie 只切換 {agent} 的登入;它的請求直接送往 OpenAI",
"{agent}'s requests go through magpie to fail over to your other accounts again": "{agent} 的請求重新經過 magpie,以便轉到你的其他帳號",
@@ -9624,6 +9634,11 @@ const I18N = {
"Keep {agent} signed in to the first account": "{agent} を最初のアカウントにサインインしたままにする",
"Keep {agent} signed in to": "{agent} のサインイン先を固定:",
"Only switch {agent}'s sign-in": "{agent} のサインインだけを切り替える",
+ "Stop when every account is at its cap": "すべてのアカウントが上限に達したら止める",
+ "On: once every account in use is at its cap, requests get 429 instead of going to the fallbacks.": "オン:使用中のアカウントがすべて上限に達すると、リクエストはフォールバックに送らず 429 を返します。",
+ "Check usage again near a cap": "上限に近づいたら使用量を読み直す",
+ "On: an account within 5 points of its cap has its usage read again if older than 15s (any account: 5m) before a request, waiting up to 10s; one whose usage can't be read is held.": "オン:上限まで 5 ポイント以内のアカウントは、読み取りが 15 秒(どのアカウントも 5 分)より古ければリクエスト前に使用量を読み直し、最大 10 秒待ちます。使用量を読めないアカウントは使いません。",
+ "No cap on {users}: used to 100% of each window": "{users} には上限がありません:各ウィンドウを 100% まで使います",
"magpie signs {agent} in to the next account with room when the one it is on is used up, and back once that has room again, by writing {agent}'s own sign-in; {agent}'s requests go straight to OpenAI, never through magpie. A {agent} app or session already open stays on its account until it is opened again. Connected to magpie, {agent}'s requests go through it as before.": "{agent} がサインイン中のアカウントの利用枠を使い切ると、magpie は {agent} 自身のサインインを書き換えて、枠の残っている次のアカウントに切り替え、元のアカウントの枠が戻ったら切り戻します。{agent} のリクエストは magpie を経由せず、OpenAI に直接送られます。すでに開いている {agent} のアプリやセッションは、開き直すまで元のアカウントのままです。{agent} を magpie に接続している場合、リクエストはこれまでどおり magpie を経由します。",
"magpie only switches {agent}'s sign-in; its requests go straight to OpenAI": "magpie は {agent} のサインインだけを切り替えます。リクエストは OpenAI に直接送られます",
"{agent}'s requests go through magpie to fail over to your other accounts again": "{agent} のリクエストは再び magpie を経由し、他のアカウントへ切り替えられます",
@@ -13655,6 +13670,11 @@ const I18N = {
"Keep {agent} signed in to the first account": "{agent} beim ersten Konto angemeldet lassen",
"Keep {agent} signed in to": "{agent} angemeldet lassen bei",
"Only switch {agent}'s sign-in": "Nur die Anmeldung von {agent} wechseln",
+ "Stop when every account is at its cap": "Anhalten, wenn jedes Konto sein Limit erreicht hat",
+ "On: once every account in use is at its cap, requests get 429 instead of going to the fallbacks.": "An: Hat jedes genutzte Konto sein Limit erreicht, bekommen Anfragen 429, statt an die Fallbacks zu gehen.",
+ "Check usage again near a cap": "Nutzung nahe am Limit neu lesen",
+ "On: an account within 5 points of its cap has its usage read again if older than 15s (any account: 5m) before a request, waiting up to 10s; one whose usage can't be read is held.": "An: Bei einem Konto höchstens 5 Punkte unter seinem Limit wird die Nutzung vor einer Anfrage neu gelesen, wenn sie älter als 15 s ist (bei jedem Konto: 5 min), mit bis zu 10 s Wartezeit; ein Konto, dessen Nutzung sich nicht lesen lässt, wird zurückgehalten.",
+ "No cap on {users}: used to 100% of each window": "Kein Limit für {users}: jedes Fenster wird bis 100 % genutzt",
"magpie signs {agent} in to the next account with room when the one it is on is used up, and back once that has room again, by writing {agent}'s own sign-in; {agent}'s requests go straight to OpenAI, never through magpie. A {agent} app or session already open stays on its account until it is opened again. Connected to magpie, {agent}'s requests go through it as before.": "Ist das Konto, mit dem {agent} angemeldet ist, aufgebraucht, schreibt magpie die eigene Anmeldung von {agent} auf das nächste Konto mit Kontingent um, und zurück, sobald das erste wieder Kontingent hat; die Anfragen von {agent} gehen direkt an OpenAI, nie über magpie. Eine bereits geöffnete {agent}-App oder -Sitzung bleibt beim bisherigen Konto, bis sie neu geöffnet wird. Ist {agent} mit magpie verbunden, laufen seine Anfragen wie bisher über magpie.",
"magpie only switches {agent}'s sign-in; its requests go straight to OpenAI": "magpie wechselt nur die Anmeldung von {agent}; seine Anfragen gehen direkt an OpenAI",
"{agent}'s requests go through magpie to fail over to your other accounts again": "Die Anfragen von {agent} laufen wieder über magpie, um auf deine anderen Konten auszuweichen",
diff --git a/internal/gui/core_budget_test.go b/internal/gui/core_budget_test.go
index f437a6b1f..1e61a724c 100644
--- a/internal/gui/core_budget_test.go
+++ b/internal/gui/core_budget_test.go
@@ -15,8 +15,8 @@ import (
// so a page can't quietly grow back into the core; a change that needs more
// raises them and says why in its commit.
const (
- appScriptBudget = 1_310_460 // app.js: 1,198,239 after Usage left (P2); 1,218,098 with the 10-15 fixes to pages still in the core; 1,232,466 with the provider editor's Advanced fold and every dialog holding its place, the core's own (10-16); 1,250,533 with the quota refresh kept through redraws, the core's own since the tray panel draws it too, plus 1.5% (#2397); 1,269,083 with Free first's option in the provider editor's ROUTINGS, which the Routing page reads too, plus 1.5%; 1,272,726 with the Providers page drawing its last list at once (#1519), the core's page, and the Routing trace's account names read without Object.entries (#1481), plus 1.5%; a dialog growing out of the row that opened it and folding back into it, the core's own since every page's dialogs open through it (10-20); Codex's and Claude Code's own settings moved from Settings into the agent's row on the Agents page, the core's own since the tray panel and the window both draw them (10-20); 1,314,432 with Settings › Privacy's record of what was masked (its summary, the records view and the local model's controls), which belongs to Settings, the core's own, plus 1.5%; 1,336,481 with a provider's Names & levels saying Provider in model names is Settings' and opening it there (joe), and Settings' Image recognition saying where a custom provider's model is marked (nexnull), both in the core's own pages, with no room added (10-23); 1,337,633 with Settings › Models › Replies' Intent classifier timeout (#2934, reyoung), Settings' own row, drawn by waitRow, which the Image generation timeout's row now shares, with no room added; 1,337,735 with the provider editor's link naming a partner's page by its own host (siteHost) instead of usemagpie.ai's /go link, the core's own editor, with no room added (10-23); 1,338,084 with Check in now saying what the press came to (claimed +credits, already today, not available yet) where it said "refreshed" (#3326), the check-in row the tray panel draws too, the core's own, with no room added (10-23); 1,338,929 with a new partner's tile keeping a small dot until it is opened, in the add sheet, the core's own, with no room added (10-23); 1,343,324 with the sidebar's open-page row marked by one shape that moves on a spring (navTopGo), the window's own shell, which every page sits beside, with no room added (10-23); 1,343,465 with a dialog kept centred while its content comes in, until it is pressed (holdModal, every page's dialogs), the core's own, with no room added (10-24); 1,317,486 with what masking did moved out to Insights › Masking (plugins/redact), the budget lowered to it (10-24); 1,321,336 with Codex's one subagent model, its card's square saying plainly what each choice does and a row with a name in its sheet, where two squares read alike (the owner), the Agents page's, which the tray panel draws too, the core's own, with no room added (10-24); 1,321,424 with Settings › Models' Image generation MCP server row beside Image recognition's (JonahFu on X), the two drawn by one helper (mcpRow), Settings' own, with no room added (10-25); 1,321,994 with a provider of the user's set never to rest for a limit (zoecats), the provider editor's Advanced row, the core's own editor, with no room added (10-25); 1,322,289 with Settings › Privacy's Keep a key's form under Mask secrets (Charles Cao on X), Settings' own row, with no room added (10-25); 1,322,537 with no Codex squares beside its named settings rows (the owner), with no room added (10-25); 1,323,401 with the tray panel drawing its last allowances at once and asking for them beside the state (#3160, #3134), the tray panel's, the core's own, and the 4 bytes 84c50f032 added, with no room added (10-25); 1,326,105 with many releases' notes merged into one list by kind, each line with its version (the owner), in the update sheet and What's new, with no room added (10-25); 1,321,319 with Settings › Observability moved out to its plugin (plugins/otel/page.js) and its Langfuse setup there (an X user), the budget lowered to it (10-26); a quota card whose window is used but names no reset no longer saying "Not used yet" (windy on Discord) raised it to 1,321,347, and back to 1,321,319 with Settings' model pickers sharing one row helper (modelOpt) where six kept their own copy, no raise (10-26); 1,325,845 with every slow page's skeletons getting magpie's bird hopping bar to bar (skelBird, the owner after the Plugins page's README), one bird for the window over whatever page or dialog is loading, the core's own, with no room added (10-26); 1,310,129 with Settings › Network and sharing's gateway rows (its port, bare ids, local network, allowed addresses, web pages) moved out to the gateway plugin (plugins/gateway/settings.js), the budget lowered to it (10-27); 1,310,269 with Test models naming the first model that failed and its error in the status line, where it said only how many (ysm119 on Discord), the provider editor's, the core's own, with no room added (10-27); 1,310,460 with Settings › About's Website row linking usemagpie.ai (the owner), Settings' own row, with no room added (10-27)
- coreScriptBudget = 1_526_556 // every core script: 1,358,034 after Usage left; 1,380,003 with them; 1,405,937 with the fold; 1,426,102 with 10-16's core work (the GitHub list loading, the move to plugins, Settings › Long conversations), plus 1.5% (#2612); 1,447,368 with loadBack, the window's one state read per return, which every page needs and so is the core's own (1,446,845 before it, 155 bytes under), plus 1.5%; OpenCode Zen's card saying OpenCode tells no key its balance adds ~390 bytes inside it; 1,472,589 with the dialog that grows out of its row and the agents' own settings in their rows, on main's 10-20 core work (the Providers page's last list, the Routing trace's names), plus 1.5% (10-20); 1,506,192 with Settings › Privacy's record of what was masked, plus 1.5%; 1,529,393 with the Intent classifier timeout row (#2934), with no room added; 1,529,495 with the editor's partner link host, with no room added; 1,529,844 with Check in now's press summary (#3326), with no room added; 1,530,689 with the new partner tile's dot, with no room added; 1,535,084 with the sidebar's spring, with no room added; 1,549,478 with the context window card's map (tiles by tokens, the default look, the block grid a switch away) and its text view showing an inline image, in context.js, which the Routing page draws as well as Usage, so the core's own, with no room added (10-24); 1,550,768 with a usage card's refresh opening after its text (#840's card, which the tray panel draws too), the Add key spinner, and a dialog kept centred while its content comes in, with no room added (10-24); 1,524,789 with what masking did moved out to Insights › Masking, lowered to it; 1,528,525 with Codex's one subagent model, with no room added (10-24); 1,528,972 with the Image generation MCP server row (JonahFu on X), with no room added (10-25); 1,529,542 with the limit-rest row (zoecats), with no room added (10-25); 1,529,837 with Keep a key's form (Charles Cao on X), with no room added (10-25); 1,530,050 with no Codex squares beside its named settings rows (the owner), with no room added (10-25); 1,530,631 with a plugin's page growing to its README on the dialog's spring instead of in one jump (the owner), the Plugins page's, in plugins.js, with no room added (10-25); 1,533,445 with the context map counting its tokens on as a live request grows, a tile pointed at keeping its tip, a tool's gathered results saying how many and the small files their folder (the owner, after flohoeller's map on X), in context.js, which the Routing page draws as well as Usage, and the 4 bytes 84c50f032 added to app.js (Settings' Version row) without a raise, with no room added (10-25); 1,534,312 with the tray panel's last allowances drawn at once (#3160, #3134), with no room added (10-25); 1,537,016 with many releases' notes merged into one list by kind (the owner), in app.js, with no room added (10-25); 1,542,244 with a plugin's README loading under magpie's bird bouncing down its lines instead of pulsing bars (the owner), the Plugins page's, in plugins.js, with no room added (10-26); 1,542,418 with a page plugin's sidebar tab coming and going as it is installed, switched on or off, or removed, instead of on the Plugins page's next load (the owner), in plugins.js, with no room added (10-26); 1,537,645 with Settings › Observability moved out to plugins/otel, lowered to it (10-26); 1,537,673 with a used window with no reset not "Not used yet" (windy on Discord), and back to 1,537,645 with Settings' model pickers sharing modelOpt (10-26); 1,541,967 with every slow page's skeletons getting magpie's bird (skelBird, the owner), in app.js, the README's bird sharing its drawing and styles, with no room added (10-26); 1,526,251 with Settings' gateway rows moved out to plugins/gateway/settings.js, lowered to it (10-27); 1,526,391 with Test models naming the first failed model and its error (ysm119 on Discord), in app.js, with no room added (10-27); 1,526,556 with About's Website row (the owner), in app.js, with no room added (10-27)
+ appScriptBudget = 1_310_460 // app.js: 1,198,239 after Usage left (P2); 1,218,098 with the 10-15 fixes to pages still in the core; 1,232,466 with the provider editor's Advanced fold and every dialog holding its place, the core's own (10-16); 1,250,533 with the quota refresh kept through redraws, the core's own since the tray panel draws it too, plus 1.5% (#2397); 1,269,083 with Free first's option in the provider editor's ROUTINGS, which the Routing page reads too, plus 1.5%; 1,272,726 with the Providers page drawing its last list at once (#1519), the core's page, and the Routing trace's account names read without Object.entries (#1481), plus 1.5%; a dialog growing out of the row that opened it and folding back into it, the core's own since every page's dialogs open through it (10-20); Codex's and Claude Code's own settings moved from Settings into the agent's row on the Agents page, the core's own since the tray panel and the window both draw them (10-20); 1,314,432 with Settings › Privacy's record of what was masked (its summary, the records view and the local model's controls), which belongs to Settings, the core's own, plus 1.5%; 1,336,481 with a provider's Names & levels saying Provider in model names is Settings' and opening it there (joe), and Settings' Image recognition saying where a custom provider's model is marked (nexnull), both in the core's own pages, with no room added (10-23); 1,337,633 with Settings › Models › Replies' Intent classifier timeout (#2934, reyoung), Settings' own row, drawn by waitRow, which the Image generation timeout's row now shares, with no room added; 1,337,735 with the provider editor's link naming a partner's page by its own host (siteHost) instead of usemagpie.ai's /go link, the core's own editor, with no room added (10-23); 1,338,084 with Check in now saying what the press came to (claimed +credits, already today, not available yet) where it said "refreshed" (#3326), the check-in row the tray panel draws too, the core's own, with no room added (10-23); 1,338,929 with a new partner's tile keeping a small dot until it is opened, in the add sheet, the core's own, with no room added (10-23); 1,343,324 with the sidebar's open-page row marked by one shape that moves on a spring (navTopGo), the window's own shell, which every page sits beside, with no room added (10-23); 1,343,465 with a dialog kept centred while its content comes in, until it is pressed (holdModal, every page's dialogs), the core's own, with no room added (10-24); 1,317,486 with what masking did moved out to Insights › Masking (plugins/redact), the budget lowered to it (10-24); 1,321,336 with Codex's one subagent model, its card's square saying plainly what each choice does and a row with a name in its sheet, where two squares read alike (the owner), the Agents page's, which the tray panel draws too, the core's own, with no room added (10-24); 1,321,424 with Settings › Models' Image generation MCP server row beside Image recognition's (JonahFu on X), the two drawn by one helper (mcpRow), Settings' own, with no room added (10-25); 1,321,994 with a provider of the user's set never to rest for a limit (zoecats), the provider editor's Advanced row, the core's own editor, with no room added (10-25); 1,322,289 with Settings › Privacy's Keep a key's form under Mask secrets (Charles Cao on X), Settings' own row, with no room added (10-25); 1,322,537 with no Codex squares beside its named settings rows (the owner), with no room added (10-25); 1,323,401 with the tray panel drawing its last allowances at once and asking for them beside the state (#3160, #3134), the tray panel's, the core's own, and the 4 bytes 84c50f032 added, with no room added (10-25); 1,326,105 with many releases' notes merged into one list by kind, each line with its version (the owner), in the update sheet and What's new, with no room added (10-25); 1,321,319 with Settings › Observability moved out to its plugin (plugins/otel/page.js) and its Langfuse setup there (an X user), the budget lowered to it (10-26); a quota card whose window is used but names no reset no longer saying "Not used yet" (windy on Discord) raised it to 1,321,347, and back to 1,321,319 with Settings' model pickers sharing one row helper (modelOpt) where six kept their own copy, no raise (10-26); 1,325,845 with every slow page's skeletons getting magpie's bird hopping bar to bar (skelBird, the owner after the Plugins page's README), one bird for the window over whatever page or dialog is loading, the core's own, with no room added (10-26); 1,310,129 with Settings › Network and sharing's gateway rows (its port, bare ids, local network, allowed addresses, web pages) moved out to the gateway plugin (plugins/gateway/settings.js), the budget lowered to it (10-27); 1,310,269 with Test models naming the first model that failed and its error in the status line, where it said only how many (ysm119 on Discord), the provider editor's, the core's own, with no room added (10-27); 1,310,460 with Settings › About's Website row linking usemagpie.ai (the owner), Settings' own row, with no room added (10-27); 1,309,669 with a pool's Stop at the caps and Check usage near a cap ticks under a provider's accounts (plugins#349), the core's own editor, its account rows' Remove and add rows each drawn by one helper (forgetOf, addRow), with no room added (10-28)
+ coreScriptBudget = 1_526_556 // every core script: 1,358,034 after Usage left; 1,380,003 with them; 1,405,937 with the fold; 1,426,102 with 10-16's core work (the GitHub list loading, the move to plugins, Settings › Long conversations), plus 1.5% (#2612); 1,447,368 with loadBack, the window's one state read per return, which every page needs and so is the core's own (1,446,845 before it, 155 bytes under), plus 1.5%; OpenCode Zen's card saying OpenCode tells no key its balance adds ~390 bytes inside it; 1,472,589 with the dialog that grows out of its row and the agents' own settings in their rows, on main's 10-20 core work (the Providers page's last list, the Routing trace's names), plus 1.5% (10-20); 1,506,192 with Settings › Privacy's record of what was masked, plus 1.5%; 1,529,393 with the Intent classifier timeout row (#2934), with no room added; 1,529,495 with the editor's partner link host, with no room added; 1,529,844 with Check in now's press summary (#3326), with no room added; 1,530,689 with the new partner tile's dot, with no room added; 1,535,084 with the sidebar's spring, with no room added; 1,549,478 with the context window card's map (tiles by tokens, the default look, the block grid a switch away) and its text view showing an inline image, in context.js, which the Routing page draws as well as Usage, so the core's own, with no room added (10-24); 1,550,768 with a usage card's refresh opening after its text (#840's card, which the tray panel draws too), the Add key spinner, and a dialog kept centred while its content comes in, with no room added (10-24); 1,524,789 with what masking did moved out to Insights › Masking, lowered to it; 1,528,525 with Codex's one subagent model, with no room added (10-24); 1,528,972 with the Image generation MCP server row (JonahFu on X), with no room added (10-25); 1,529,542 with the limit-rest row (zoecats), with no room added (10-25); 1,529,837 with Keep a key's form (Charles Cao on X), with no room added (10-25); 1,530,050 with no Codex squares beside its named settings rows (the owner), with no room added (10-25); 1,530,631 with a plugin's page growing to its README on the dialog's spring instead of in one jump (the owner), the Plugins page's, in plugins.js, with no room added (10-25); 1,533,445 with the context map counting its tokens on as a live request grows, a tile pointed at keeping its tip, a tool's gathered results saying how many and the small files their folder (the owner, after flohoeller's map on X), in context.js, which the Routing page draws as well as Usage, and the 4 bytes 84c50f032 added to app.js (Settings' Version row) without a raise, with no room added (10-25); 1,534,312 with the tray panel's last allowances drawn at once (#3160, #3134), with no room added (10-25); 1,537,016 with many releases' notes merged into one list by kind (the owner), in app.js, with no room added (10-25); 1,542,244 with a plugin's README loading under magpie's bird bouncing down its lines instead of pulsing bars (the owner), the Plugins page's, in plugins.js, with no room added (10-26); 1,542,418 with a page plugin's sidebar tab coming and going as it is installed, switched on or off, or removed, instead of on the Plugins page's next load (the owner), in plugins.js, with no room added (10-26); 1,537,645 with Settings › Observability moved out to plugins/otel, lowered to it (10-26); 1,537,673 with a used window with no reset not "Not used yet" (windy on Discord), and back to 1,537,645 with Settings' model pickers sharing modelOpt (10-26); 1,541,967 with every slow page's skeletons getting magpie's bird (skelBird, the owner), in app.js, the README's bird sharing its drawing and styles, with no room added (10-26); 1,526,251 with Settings' gateway rows moved out to plugins/gateway/settings.js, lowered to it (10-27); 1,526,391 with Test models naming the first failed model and its error (ysm119 on Discord), in app.js, with no room added (10-27); 1,526,556 with About's Website row (the owner), in app.js, with no room added (10-27); 1,526,500 with a pool's cap ticks (plugins#349), in app.js, with no room added (10-28)
)
func TestCoreScriptBudget(t *testing.T) {
diff --git a/internal/gui/providers.go b/internal/gui/providers.go
index 25a1c7554..704dde7da 100644
--- a/internal/gui/providers.go
+++ b/internal/gui/providers.go
@@ -220,6 +220,10 @@ type providerJSON struct {
// SwitchOnly: magpie only moves Codex's own sign-in between its
// accounts, never routing its requests for failover (#1907)
SwitchOnly bool `json:"switchOnly,omitempty"`
+ // CapStop and CapCareful: how its accounts end at their caps
+ // (plugins#349), set on their own with capstop and capcareful
+ CapStop bool `json:"capStop,omitempty"`
+ CapCareful bool `json:"capCareful,omitempty"`
// how many requests each of its keys or accounts has out at once, the
// rest queued: the user's (null: not set), and what its plugin says
// when the user set none (provider.Concurrency)
@@ -596,7 +600,7 @@ func providerInfoWith(p provider.Provider, agents []agentUse, desktop map[string
Catalog: p.Catalog, Website: p.Website, KeysURL: p.KeysURL,
Proxy: p.Proxy, AccountProxies: p.AccountProxies, AccountModels: p.AccountModels, AccountCaps: p.AccountCaps, AccountWindowCaps: p.AccountWindowCaps, AccountNames: p.AccountNames, Headers: p.Headers, ExtraBody: p.ExtraBody, Searches: p.Searches, Cline: p.ClinePinnable(), PinUpstream: p.PinUpstream, Unredacted: p.Unredacted, InsecureTLS: p.InsecureTLS, TakesInsecureTLS: p.TakesInsecureTLS(), LimitRestOff: p.LimitRestOff, NativeCompaction: p.NativeCompaction, CarriesCompaction: p.CarriesCompaction(), PassUserAgent: p.PassUserAgent, OwnClient: p.OwnClient(), AuthHeader: p.AuthHeader, BothAuth: p.BothAuth(), BalanceURL: p.BalanceURL, BalancePath: p.BalancePath, BalanceMethod: p.BalanceMethod, ModelsURL: p.ModelsURL,
Ready: p.Ready(), Chosen: p.Picks(), Models: []modelJSON{}, Agents: []providerAgent{},
- Fallback: p.Fallback, Routing: p.Routing, Sink: p.Sink, Affinity: p.Affinity, KeepLogin: p.KeepLogin, KeepLoginAs: p.KeepLoginAs, SwitchOnly: p.SwitchOnly, Unlisted: p.Unlisted, NoModels: p.ServesNone(), Off: p.Off, Contexts: p.Contexts,
+ Fallback: p.Fallback, Routing: p.Routing, Sink: p.Sink, Affinity: p.Affinity, KeepLogin: p.KeepLogin, KeepLoginAs: p.KeepLoginAs, SwitchOnly: p.SwitchOnly, CapStop: p.CapStop, CapCareful: p.CapCareful, Unlisted: p.Unlisted, NoModels: p.ServesNone(), Off: p.Off, Contexts: p.Contexts,
MaxConcurrency: p.MaxConcurrency, PluginConcurrency: p.PluginConcurrency(), PriceRate: p.PriceRate,
AccountConcurrency: p.AccountConcurrency, QueueLimit: p.QueueLimit, QueueWait: p.QueueWait, MaxRPM: p.MaxRPM,
Outputs: provider.OutputsOf(p.ID), Compacts: provider.CompactsOf(p.ID),
@@ -1646,7 +1650,8 @@ func providerRoutes(mux *http.ServeMux, w Windows) {
in.KeepLogin = old.KeepLogin // set on its own, with keeplogin
in.KeepLoginAs = old.KeepLoginAs
in.SwitchOnly = old.SwitchOnly
- in.Off = old.Off // and this with off and on
+ in.CapStop, in.CapCareful = old.CapStop, old.CapCareful // capstop, capcareful
+ in.Off = old.Off // and this with off and on
if in.Contexts == nil {
in.Contexts = old.Contexts // a save that doesn't say
}
@@ -1845,6 +1850,20 @@ func providerRoutes(mux *http.ServeMux, w Windows) {
return
}
agent.SyncCatalogSoon()
+ case "capstop":
+ // a request ends at the accounts' caps, 429, or goes on to the
+ // fallbacks (plugins#349)
+ if err := provider.SetCapStop(in.ID, in.CapStop); err != nil {
+ fail(rw, err)
+ return
+ }
+ case "capcareful":
+ // usage read again near a cap, an account held while it can't
+ // be (plugins#349)
+ if err := provider.SetCapCareful(in.ID, in.CapCareful); err != nil {
+ fail(rw, err)
+ return
+ }
case "affinity":
if err := provider.SetAffinity(in.ID, in.Affinity); err != nil {
fail(rw, err)
diff --git a/internal/gui/tests/pool-ends.test.cjs b/internal/gui/tests/pool-ends.test.cjs
new file mode 100644
index 000000000..0a1074574
--- /dev/null
+++ b/internal/gui/tests/pool-ends.test.cjs
@@ -0,0 +1,132 @@
+// Run with Node's test runner and Playwright on the module path; see README.md.
+// plugins#349: under a subscription's accounts, once any has a cap, two
+// ticks say how the pool ends at its caps: stop there (429, no fallback)
+// and check usage again near a cap. Each posts its own action and the
+// redraw keeps it ticked; a note names the accounts in use with no cap.
+// With no cap and neither on, there is no box. The page doesn't move. In
+// English, Chinese, Japanese and German, Chromium and WebKit, at 1100 and
+// 420 wide.
+const assert = require("node:assert/strict");
+const fs = require("node:fs/promises");
+const path = require("node:path");
+const { test } = require("node:test");
+const { chromium, webkit } = require("playwright");
+
+const assets = path.resolve(__dirname, "../assets");
+const codex = ({ capped = true, capStop = false, capCareful = false } = {}) => ({
+ id: "codex", name: "Codex", icon: "codex-color", chat: "", responses: "", anthropic: "", catalog: "", routing: "ordered", capStop, capCareful,
+ accountCaps: capped ? { "[email protected]": 90 } : {},
+ models: [{ id: "gpt-6", name: "GPT-6", on: true }], agents: [], fallback: [], headers: {}, keyList: [],
+ account: {
+ agent: "codex", agentName: "Codex", user: "[email protected]", plan: "PLUS",
+ logins: [
+ { user: "[email protected]", plan: "PLUS", active: true, on: true },
+ { user: "[email protected]", plan: "PRO", on: true },
+ ],
+ },
+});
+
+function serve(lang, first, posts) {
+ let list = [first];
+ const state = { agents: [], profiles: [], settings: { lang, theme: "light" } };
+ const providers = () => ({ providers: list, presets: [], excluded: [], gateway: { running: true, window: true } });
+ return async (route) => {
+ const url = new URL(route.request().url());
+ const json = (data) => route.fulfill({ json: data });
+ if (url.pathname === "/boot.js") return route.fulfill({ contentType: "text/javascript", body: `window.bootPrefs = {lang:"${lang}",theme:"light",web:true};` });
+ if (url.pathname === "/wails/runtime.js") return route.fulfill({ contentType: "text/javascript", body: "export const Window = {};" });
+ if (url.pathname === "/api/state") return json(state);
+ if (url.pathname === "/api/providers") return json(providers());
+ if (url.pathname === "/api/provider/capstop" || url.pathname === "/api/provider/capcareful") {
+ const body = route.request().postDataJSON();
+ posts.push(body);
+ list = [{ ...list[0], ...body }];
+ return json(providers());
+ }
+ if (url.pathname === "/api/groups") return json({ groups: [] });
+ if (url.pathname === "/api/plugins") return json({ plugins: [] });
+ if (url.pathname.startsWith("/api/")) return json({});
+ const file = path.join(assets, url.pathname === "/" ? "index.html" : url.pathname);
+ const contentType = { ".html": "text/html", ".js": "text/javascript", ".css": "text/css", ".svg": "image/svg+xml", ".png": "image/png" }[path.extname(file)];
+ await route.fulfill({ body: await fs.readFile(file), contentType });
+ };
+}
+
+const label = {
+ en: ["Stop when every account is at its cap", "Check usage again near a cap", "No cap on [email protected]: used to 100% of each window"],
+ zh: ["所有账号都到上限时停止", "接近上限时重新读取用量", "[email protected] 没有上限:每个窗口都会用到 100%"],
+ ja: ["すべてのアカウントが上限に達したら止める", "上限に近づいたら使用量を読み直す", "[email protected] には上限がありません:各ウィンドウを 100% まで使います"],
+ de: ["Anhalten, wenn jedes Konto sein Limit erreicht hat", "Nutzung nahe am Limit neu lesen", "Kein Limit für [email protected]: jedes Fenster wird bis 100 % genutzt"],
+};
+
+for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
+ for (const width of [1100, 420]) {
+ for (const lang of Object.keys(label)) {
+ const open = async (t, opts) => {
+ const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));
+ t.after(() => browser.close());
+ const page = await (await browser.newContext({ viewport: { width, height: 760 }, reducedMotion: "reduce" })).newPage();
+ page.setDefaultTimeout(5000);
+ const errors = [], posts = [];
+ page.on("pageerror", (e) => errors.push(e.message));
+ await page.route("**/*", serve(lang, codex(opts), posts));
+ await page.goto("http://magpie.test/?view=providers");
+ await page.locator(".row.provider", { hasText: "Codex" }).first().click();
+ await page.locator(".editor .acc").first().waitFor();
+ return { page, errors, posts };
+ };
+
+ test(`${engine} ${width} ${lang}: a capped pool's ticks stop at the caps and read usage again`, async (t) => {
+ const { page, errors, posts } = await open(t, {});
+ const ticks = page.locator(".editor .accts .pool-ends label");
+ assert.equal(await ticks.count(), 2);
+ assert.deepEqual((await ticks.allTextContents()).map((s) => s.trim()), label[lang].slice(0, 2));
+ for (const i of [0, 1]) {
+ assert.ok(await ticks.nth(i).getAttribute("title"), "it says what it does");
+ assert.equal(await ticks.nth(i).locator("input").isChecked(), false, "off by default");
+ }
+ assert.equal((await page.locator(".editor .accts .pool-ends .note").textContent()).trim(), label[lang][2], "the uncapped account is named");
+ assert.equal(await page.locator(".editor .accts .keep-login.pool-ends, .editor .accts .keep-login .pool-ends").count(), 0, "apart from keep-login's box");
+ if (lang === "en") {
+ const strings = await page.evaluate(() => [...document.querySelectorAll(".editor .accts .pool-ends label")].map((l) => l.title));
+ const keys = ["Stop when every account is at its cap", "Check usage again near a cap", "No cap on {users}: used to 100% of each window",
+ "On: once every account in use is at its cap, requests get 429 instead of going to the fallbacks.",
+ "On: an account within 5 points of its cap has its usage read again if older than 15s (any account: 5m) before a request, waiting up to 10s; one whose usage can't be read is held."];
+ assert.deepEqual(strings, keys.slice(3));
+ const missing = await page.evaluate((ks) => ["zh", "zh-TW", "ja", "de"].flatMap((l) => ks.filter((k) => !I18N[l][k]).map((k) => `${l}: ${k}`)), keys);
+ assert.deepEqual(missing, [], "every string has its translations");
+ }
+ await ticks.nth(1).scrollIntoViewIfNeeded();
+ for (const i of [0, 1]) {
+ const box = await ticks.nth(i).boundingBox();
+ assert.ok(box.x >= 0 && box.x + box.width <= width, `tick ${i} fits across ${width}`);
+ }
+ const scrollers = () => page.evaluate(() => [...document.querySelectorAll("*")].filter((e) => e.scrollTop).map((e) => e.scrollTop).concat(document.scrollingElement.scrollTop));
+ const before = await scrollers();
+ const nth = (i) => `document.querySelectorAll(".editor .accts .pool-ends input")[${i}]?.checked`;
+ await ticks.nth(0).locator("input").click();
+ await page.waitForFunction(`${nth(0)} === true`);
+ assert.deepEqual(posts, [{ id: "codex", capStop: true }]);
+ await page.locator(".editor .accts .pool-ends input").nth(1).click();
+ await page.waitForFunction(`${nth(1)} === true && ${nth(0)} === true`);
+ assert.deepEqual(posts, [{ id: "codex", capStop: true }, { id: "codex", capCareful: true }]);
+ assert.deepEqual(await scrollers(), before, "the page doesn't move");
+ await page.locator(".editor .accts .pool-ends input").nth(0).click();
+ await page.waitForFunction(`${nth(0)} === false && ${nth(1)} === true`);
+ assert.deepEqual(posts.at(-1), { id: "codex", capStop: false });
+ assert.deepEqual(errors, []);
+ });
+
+ test(`${engine} ${width} ${lang}: the pool's ticks are drawn as set, and not at all with no cap`, async (t) => {
+ const on = await open(t, { capStop: true, capCareful: true });
+ assert.deepEqual(await on.page.locator(".editor .accts .pool-ends input").evaluateAll((is) => is.map((i) => i.checked)), [true, true]);
+ const none = await open(t, { capped: false });
+ assert.equal(await none.page.locator(".editor .accts .pool-ends").count(), 0);
+ // no cap left but a tick still on: shown, so it can be turned off
+ const left = await open(t, { capped: false, capStop: true });
+ assert.equal(await left.page.locator(".editor .accts .pool-ends input").count(), 2);
+ assert.deepEqual([...on.errors, ...none.errors, ...left.errors], []);
+ });
+ }
+ }
+}
diff --git a/internal/provider/account_caps.go b/internal/provider/account_caps.go
index 835bb4138..577205615 100644
--- a/internal/provider/account_caps.go
+++ b/internal/provider/account_caps.go
@@ -316,6 +316,29 @@ func (a Allowance) CapHeld(model string, caps WindowCaps, now time.Time) *CapHol
return h
}
+// NearCap says whether an allowance, as last known, has some window
+// counting model within points of its own share of caps (or past it) at
+// now, not renewed since: a careful provider reads such an account again
+// before it is sent a request (Provider.CapCareful). No caps are near
+// nothing.
+func (a Allowance) NearCap(model string, caps WindowCaps, points float64, now time.Time) bool {
+ if !caps.Holds() {
+ return false
+ }
+ model = strings.ToLower(model)
+ for _, l := range a {
+ share, _ := caps.Of(l.Name)
+ if share <= 0 || !l.applies(model) || l.Used < float64(share)-points {
+ continue
+ }
+ if !l.Resets.IsZero() && !l.Resets.After(now) {
+ continue // renewed since it was read
+ }
+ return true
+ }
+ return false
+}
+
// capReached says whether an account's windows that stop it for every
// model are, one of them, at or past its own share of caps at now —
// usedPast at the share, those renewed since they were read left out. No
diff --git a/internal/provider/cards_heard_claude_test.go b/internal/provider/cards_heard_claude_test.go
index 2d26d1e1a..c1dd4333e 100644
--- a/internal/provider/cards_heard_claude_test.go
+++ b/internal/provider/cards_heard_claude_test.go
@@ -43,7 +43,7 @@ func TestCachedCardsTellHeardClaudeLimits(t *testing.T) {
readAt := time.Now()
old := SubscriptionQuota{Provider: "claude", User: "[email protected]", ReadAt: &readAt, Windows: []QuotaWindow{{Name: "5 hours", Used: 10, ResetsAt: &resetAt}}, readSeq: before}
loginUsageCache.Lock()
- loginUsageCache.m = map[string]loginUsageEntry{"claude/[email protected]": {time.Now(), old, old}}
+ loginUsageCache.m = map[string]loginUsageEntry{"claude/[email protected]": {at: time.Now(), q: old, read: old}}
loginUsageCache.Unlock()
if got := used(); got != 42 {
t.Fatalf("a reading started before the header took its place: used %v, want 42", got)
@@ -54,7 +54,7 @@ func TestCachedCardsTellHeardClaudeLimits(t *testing.T) {
newer.Windows = []QuotaWindow{{Name: "5 hours", Used: 55, ResetsAt: &resetAt}}
newer.readSeq = quotaReadSequence.Add(1)
loginUsageCache.Lock()
- loginUsageCache.m = map[string]loginUsageEntry{"claude/[email protected]": {time.Now(), newer, newer}}
+ loginUsageCache.m = map[string]loginUsageEntry{"claude/[email protected]": {at: time.Now(), q: newer, read: newer}}
loginUsageCache.Unlock()
if got := used(); got != 55 {
t.Fatalf("a reading started after the header: used %v, want 55", got)
diff --git a/internal/provider/login_usage.go b/internal/provider/login_usage.go
index d7c50030a..39f907e77 100644
--- a/internal/provider/login_usage.go
+++ b/internal/provider/login_usage.go
@@ -27,6 +27,33 @@ type loginUsageEntry struct {
// read is the reading itself, as the Usage page shows it: q is the
// same, but where a hiccup kept what was known
read SubscriptionQuota
+ // readAt, when set, is when read was really made: one handed back
+ // held (allowances read only when asked) is dated now in at
+ readAt time.Time
+}
+
+// readState is when an account's allowance was last read well, and
+// whether the last try failed or was held back (a careful provider holds
+// such an account: unknown is not spent, Provider.CapCareful).
+type readState struct {
+ at time.Time
+ failed bool
+}
+
+// state is what e says of the account's reading: a reading kept from
+// before (AsOf) or one that failed counts as failed, as of the good one.
+func (e loginUsageEntry) state() readState {
+ at := e.at
+ if !e.readAt.IsZero() {
+ at = e.readAt
+ }
+ switch {
+ case e.read.AsOf != nil:
+ return readState{at: *e.read.AsOf, failed: true}
+ case e.read.Error != "" || len(e.read.Windows) == 0:
+ return readState{failed: true}
+ }
+ return readState{at: at}
}
// loginRead is an account's allowance being read: whoever asks for it
@@ -69,26 +96,32 @@ func loginUsageStandIn(ctx context.Context, agent string) (map[string]Subscripti
// comes from the cache; the rest is asked for at once, as long as ctx
// allows.
func LoginUsage(ctx context.Context, agent string) map[string]SubscriptionQuota {
- out, _ := loginUsageAt(ctx, agent)
+ out, _, _ := loginUsageAt(ctx, agent)
return out
}
// loginUsageAt is LoginUsage and when the oldest of its readings was made:
// one taken from the cache is up to a minute old already, and Allowances
-// counts its own minute from then, not from when it asked (#1295).
-func loginUsageAt(ctx context.Context, agent string) (map[string]SubscriptionQuota, time.Time) {
+// counts its own minute from then, not from when it asked (#1295). And
+// each account's reading's state: when it was last read well, and
+// whether this try failed.
+func loginUsageAt(ctx context.Context, agent string) (map[string]SubscriptionQuota, time.Time, map[string]readState) {
// nor of a provider switched off or removed (leftAlone)
if settings.AccountDiscoveryOff() || leftAlone(agent) {
- return nil, time.Time{}
+ return nil, time.Time{}, nil
}
asked := time.Now()
+ states := map[string]readState{}
if out, ok := loginUsageStandIn(ctx, agent); ok {
- return out, asked
+ for user, q := range out {
+ states[user] = loginUsageEntry{at: asked, read: q}.state()
+ }
+ return out, asked, states
}
out := map[string]SubscriptionQuota{}
logins, ok := usageLogins(agent)
if !ok {
- return out, asked
+ return out, asked, states
}
var mu sync.Mutex
var wg sync.WaitGroup
@@ -100,6 +133,7 @@ func loginUsageAt(ctx context.Context, agent string) (map[string]SubscriptionQuo
e := loginReading(ctx, l)
mu.Lock()
out[l.User] = e.q
+ states[l.User] = e.state()
if e.q.Error == "" && e.at.Before(oldest) {
oldest = e.at
}
@@ -130,7 +164,7 @@ func loginUsageAt(ctx context.Context, agent string) (map[string]SubscriptionQuo
}
noteHistory(qs, time.Now())
}
- return out, oldest
+ return out, oldest, states
}
// noteHistory notes a batch of accounts' readings in the quota history.
@@ -179,6 +213,9 @@ func cachedLoginReading(ctx context.Context, l Login) loginUsageEntry {
if ok && (held || time.Since(e.at) < time.Minute) {
c.Unlock()
if held {
+ if e.readAt.IsZero() {
+ e.readAt = e.at // as it really was: a careful provider counts its age
+ }
e.at = time.Now() // as good as it gets until asked: Allowances keeps it its minute
}
return e
@@ -188,7 +225,7 @@ func cachedLoginReading(ctx context.Context, l Login) loginUsageEntry {
if q.Error != "" && ok && q.Provider != "claude" {
q = e.q // a hiccup keeps what was known
}
- return loginUsageEntry{time.Now(), q, read}
+ return loginUsageEntry{at: time.Now(), q: q, read: read}
}
r := c.pending[key]
if r == nil && held {
diff --git a/internal/provider/pool_read_test.go b/internal/provider/pool_read_test.go
new file mode 100644
index 000000000..345dd082a
--- /dev/null
+++ b/internal/provider/pool_read_test.go
@@ -0,0 +1,142 @@
+package provider
+
+import (
+ "context"
+ "sync/atomic"
+ "testing"
+ "time"
+)
+
+// plugins#349: a careful provider asks when each account's usage was
+// last read well and whether the last try failed (UsageRead), and has it
+// read again before a request (ReadUsageNow). A failed try keeps the last
+// good reading's time, so its age still counts.
+func TestUsageReadAndReadAgain(t *testing.T) {
+ signIn(t)
+ resets := time.Now().Add(3 * time.Hour)
+ var meFails atomic.Bool
+ LoginUsageVia(func(context.Context, string) map[string]SubscriptionQuota {
+ me := SubscriptionQuota{Windows: []QuotaWindow{{Span: 5 * time.Hour, Used: 40, ResetsAt: &resets}}}
+ if meFails.Load() {
+ me = SubscriptionQuota{Error: "usage endpoint said 503"}
+ }
+ return map[string]SubscriptionQuota{
+ "[email protected]": {Error: "usage endpoint said 401"},
+ }
+ })
+ clear := func() {
+ usedCache.Lock()
+ usedCache.m, usedCache.at, usedCache.loading = map[string]map[string]Allowance{}, map[string]time.Time{}, map[string]chan struct{}{}
+ usedCache.stale, usedCache.renewed, usedCache.seen, usedCache.read = nil, nil, nil, nil
+ usedCache.Unlock()
+ }
+ clear()
+ t.Cleanup(func() {
+ usedCache.Lock()
+ done := usedCache.loading["codex"]
+ usedCache.Unlock()
+ if done != nil {
+ <-done
+ }
+ LoginUsageVia(nil)
+ clear()
+ })
+ if _, _, ok := UsageRead("codex", "[email protected]"); ok {
+ t.Fatal("known before any read")
+ }
+ ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second)
+ defer cancel()
+ before := time.Now()
+ if !ReadUsageNow(ctx, "codex", "[email protected]") {
+ t.Fatal("no reading came back")
+ }
+ at, failed, ok := UsageRead("codex", "[email protected]")
+ if !ok || failed || at.Before(before) {
+ t.Fatalf("me: at %v (asked %v) failed %v ok %v", at, before, failed, ok)
+ }
+ if _, failed, ok := UsageRead("codex", "[email protected]"); !ok || !failed {
+ t.Fatalf("other: failed %v ok %v", failed, ok)
+ }
+
+ // read again: a newer good reading
+ good := at
+ if !ReadUsageNow(ctx, "codex", "[email protected]") {
+ t.Fatal("no second reading")
+ }
+ if at, failed, _ := UsageRead("codex", "[email protected]"); failed || !at.After(good) {
+ t.Fatalf("read again: at %v, was %v, failed %v", at, good, failed)
+ }
+ good, _, _ = UsageRead("codex", "[email protected]")
+
+ // it fails: failed, the last good reading's time kept
+ meFails.Store(true)
+ if !ReadUsageNow(ctx, "codex", "[email protected]") {
+ t.Fatal("no third reading")
+ }
+ if at, failed, _ := UsageRead("codex", "[email protected]"); !failed || !at.Equal(good) {
+ t.Fatalf("failed read: at %v, good %v, failed %v", at, good, failed)
+ }
+}
+
+// An account within the given points of a window's cap, or past it, is
+// near it; one further off, a window renewed since, or no cap is not.
+func TestNearCap(t *testing.T) {
+ now := time.Now()
+ al := func(used float64, resets time.Time) Allowance {
+ return Allowance{{Used: used, Resets: resets, Span: 5 * time.Hour}}
+ }
+ later, earlier := now.Add(time.Hour), now.Add(-time.Minute)
+ caps := WindowCaps{All: 90}
+ for _, c := range []struct {
+ name string
+ a Allowance
+ caps WindowCaps
+ want bool
+ }{
+ {"within five points", al(86, later), caps, true},
+ {"past it", al(95, later), caps, true},
+ {"further off", al(80, later), caps, false},
+ {"renewed since", al(88, earlier), caps, false},
+ {"no cap", al(99, later), WindowCaps{}, false},
+ } {
+ if got := c.a.NearCap("gpt-5.5", c.caps, 5, now); got != c.want {
+ t.Errorf("%s: %v", c.name, got)
+ }
+ }
+}
+
+// Stop at the caps and Check usage near the cap are kept with the
+// provider, off until set, and a save of the rest keeps them.
+func TestCapStopAndCarefulKept(t *testing.T) {
+ isolate(t)
+ if err := Save(Provider{ID: "plan", Name: "Plan", Key: "k", Chat: "http://127.0.0.1:1/v1", Models: []string{"m"}}); err != nil {
+ t.Fatal(err)
+ }
+ if p, _ := Find("plan"); p.CapStop || p.CapCareful {
+ t.Fatalf("on by default: %+v", p)
+ }
+ if err := SetCapStop("plan", true); err != nil {
+ t.Fatal(err)
+ }
+ if err := SetCapCareful("plan", true); err != nil {
+ t.Fatal(err)
+ }
+ p, _ := Find("plan")
+ if !p.CapStop || !p.CapCareful {
+ t.Fatalf("not kept: stop %v careful %v", p.CapStop, p.CapCareful)
+ }
+ p.Name = "Plan 2"
+ if err := Save(*p); err != nil {
+ t.Fatal(err)
+ }
+ if p, _ := Find("plan"); !p.CapStop || !p.CapCareful || p.Name != "Plan 2" {
+ t.Fatalf("after a save: %+v", p)
+ }
+ if err := SetCapStop("plan", false); err != nil {
+ t.Fatal(err)
+ }
+ if p, _ := Find("plan"); p.CapStop || !p.CapCareful {
+ t.Fatalf("stop off: stop %v careful %v", p.CapStop, p.CapCareful)
+ }
+}
diff --git a/internal/provider/provider.go b/internal/provider/provider.go
index 87106afa3..725819bcc 100644
--- a/internal/provider/provider.go
+++ b/internal/provider/provider.go
@@ -352,6 +352,17 @@ type Provider struct {
// and then the window's WindowCapID: 1–99, or 100 for none on it. A
// window not in it takes the account's AccountCaps.
AccountWindowCaps map[string]map[string]int `json:"accountWindowCaps,omitempty"`
+ // CapStop, for a subscription whose accounts are capped, ends there
+ // (plugins#349): while every account is held at its cap, a request is
+ // answered 429, as the vendor would, rather than sent on to Fallback.
+ // Off, the default, a fallback takes it. See gateway.Server.plan.
+ CapStop bool `json:"capStop,omitempty"`
+ // CapCareful is a capped account's usage read again before a request
+ // is sent near its cap, and the account held while its usage can't be
+ // told (plugins#349): unknown is not spent. Off, the default, routing
+ // goes by the last reading, read again each minute. See
+ // gateway.carefulDoubt.
+ CapCareful bool `json:"capCareful,omitempty"`
// AccountNames is, for a subscription, the name the user gave each
// account they named (#1515: a team's seats told apart by who holds
// them, not by a masked email), by the account's name in lower case.
@@ -737,6 +748,7 @@ func withPicks(a, pk Provider) Provider {
a.PromptAsWritten = pk.PromptAsWritten
a.Proxy, a.AccountProxies, a.AccountModels = pk.Proxy, pk.AccountProxies, pk.AccountModels
a.AccountCaps, a.AccountWindowCaps = pk.AccountCaps, pk.AccountWindowCaps
+ a.CapStop, a.CapCareful = pk.CapStop, pk.CapCareful
a.AccountNames = pk.AccountNames
a.MaxConcurrency, a.PinUpstream = pk.MaxConcurrency, pk.PinUpstream
a.AccountConcurrency, a.QueueLimit, a.QueueWait = pk.AccountConcurrency, pk.QueueLimit, pk.QueueWait
@@ -871,7 +883,7 @@ func Save(p Provider) error {
if p.ID == "kiro" {
key = p.Key
}
- p = Provider{ID: p.ID, Key: key, Models: p.Models, NoModels: p.NoModels, PickedFrom: p.PickedFrom, Delisted: p.Delisted, Unlisted: p.Unlisted, Off: p.Off, Fallback: p.Fallback, Routing: p.Routing, Sink: p.Sink, Affinity: p.Affinity, KeepLogin: p.KeepLogin, KeepLoginAs: p.KeepLoginAs, SwitchOnly: p.SwitchOnly, Contexts: p.Contexts, Family: p.Family, Proxy: p.Proxy, AccountProxies: p.AccountProxies, AccountModels: p.AccountModels, AccountCaps: p.AccountCaps, AccountWindowCaps: p.AccountWindowCaps, AccountNames: p.AccountNames, MaxConcurrency: p.MaxConcurrency, AccountConcurrency: p.AccountConcurrency, QueueLimit: p.QueueLimit, QueueWait: p.QueueWait, MaxRPM: p.MaxRPM, QuotaRest: p.QuotaRest, QuotaResetAt: p.QuotaResetAt, RestCap: p.RestCap, PinUpstream: p.PinUpstream, NativeCompaction: p.NativeCompaction, PromptAsWritten: p.PromptAsWritten, Hidden: hiddenAccount(p.ID), Quiet: quietAccount(p.ID), Tucked: tuckedAccount(p.ID)}
+ p = Provider{ID: p.ID, Key: key, Models: p.Models, NoModels: p.NoModels, PickedFrom: p.PickedFrom, Delisted: p.Delisted, Unlisted: p.Unlisted, Off: p.Off, Fallback: p.Fallback, Routing: p.Routing, Sink: p.Sink, Affinity: p.Affinity, KeepLogin: p.KeepLogin, KeepLoginAs: p.KeepLoginAs, SwitchOnly: p.SwitchOnly, Contexts: p.Contexts, Family: p.Family, Proxy: p.Proxy, AccountProxies: p.AccountProxies, AccountModels: p.AccountModels, AccountCaps: p.AccountCaps, AccountWindowCaps: p.AccountWindowCaps, CapStop: p.CapStop, CapCareful: p.CapCareful, AccountNames: p.AccountNames, MaxConcurrency: p.MaxConcurrency, AccountConcurrency: p.AccountConcurrency, QueueLimit: p.QueueLimit, QueueWait: p.QueueWait, MaxRPM: p.MaxRPM, QuotaRest: p.QuotaRest, QuotaResetAt: p.QuotaResetAt, RestCap: p.RestCap, PinUpstream: p.PinUpstream, NativeCompaction: p.NativeCompaction, PromptAsWritten: p.PromptAsWritten, Hidden: hiddenAccount(p.ID), Quiet: quietAccount(p.ID), Tucked: tuckedAccount(p.ID)}
} else {
p.AccountProxies = keyProxies(p) // a provider of keys proxies each key apart
if subscriptionID(p.ID) && !stored(p.ID) {
diff --git a/internal/provider/routing.go b/internal/provider/routing.go
index 7578f9bf5..de8e8ae9f 100644
--- a/internal/provider/routing.go
+++ b/internal/provider/routing.go
@@ -70,6 +70,30 @@ func SetSink(id string, sink bool) error {
return Save(*p)
}
+// SetCapStop has a subscription answer 429 while every account is held at
+// its usage cap, rather than go on to its fallbacks (Provider.CapStop), or
+// go on to them again.
+func SetCapStop(id string, on bool) error {
+ p, err := Find(id)
+ if err != nil {
+ return err
+ }
+ p.CapStop = on
+ return Save(*p)
+}
+
+// SetCapCareful has a subscription's capped accounts read again near
+// their cap and held while their usage can't be told
+// (Provider.CapCareful), or go by the last reading again.
+func SetCapCareful(id string, on bool) error {
+ p, err := Find(id)
+ if err != nil {
+ return err
+ }
+ p.CapCareful = on
+ return Save(*p)
+}
+
// SetKeepLogin keeps Codex or Claude Code signed in to the account the
// user made first (Provider.KeepLogin), or lets magpie move it on again.
// Either way it is no longer kept on one of the user's choosing
@@ -155,6 +179,9 @@ var usedCache struct {
// agent; that reading was asked before an account said it was out,
// so what it tells is kept only until the next ask reads again
stale map[string]bool
+ // read: when each account's allowance was last read well, and whether
+ // the last try failed, by agent/user (UsageRead)
+ read map[string]readState
}
// firstWait is how long a request waits for an agent's allowances the
@@ -555,7 +582,7 @@ func Allowances(agent string) map[string]Allowance {
defer cancel()
share := renewalShare(agent) // read before the lock: it reads providers.json
all := map[string]Allowance{}
- readings, readAt := loginUsageAt(ctx, agent)
+ readings, readAt, states := loginUsageAt(ctx, agent)
for user, q := range readings {
if q.Error != "" || len(q.Windows) == 0 {
continue
@@ -574,6 +601,16 @@ func Allowances(agent string) map[string]Allowance {
if c.seen == nil {
c.seen = map[string]reading{}
}
+ if c.read == nil {
+ c.read = map[string]readState{}
+ }
+ for user, st := range states {
+ key := agent + "/" + strings.ToLower(user)
+ if was := c.read[key].at; st.failed && was.After(st.at) {
+ st.at = was // the last good one's time stands
+ }
+ c.read[key] = st
+ }
var renewed []string
for user, a := range all {
key := agent + "/" + strings.ToLower(user)
@@ -624,6 +661,50 @@ func Allowances(agent string) map[string]Allowance {
return m
}
+// UsageRead is when user's allowance under agent was last read well, and
+// whether the last try to read it failed (or was held back); ok is false
+// when it hasn't been tried since magpie started. A careful provider holds
+// an account whose reading is too old or failed (Provider.CapCareful).
+func UsageRead(agent, user string) (at time.Time, failed, ok bool) {
+ c := &usedCache
+ c.Lock()
+ defer c.Unlock()
+ st, ok := c.read[agent+"/"+strings.ToLower(user)]
+ return st.at, st.failed, ok
+}
+
+// ReadUsageNow has agent's accounts users read again now, past the
+// minute their readings are kept and though allowances are read only when
+// asked, and waits, till ctx ends, for a reading begun after it asked. It
+// says whether one came back (UsageRead then tells how it went).
+func ReadUsageNow(ctx context.Context, agent string, users ...string) bool {
+ for _, u := range users {
+ StaleAllowance(agent, u)
+ }
+ c := &usedCache
+ for range 3 {
+ Allowances(agent)
+ c.Lock()
+ done, back := c.loading[agent], !c.at[agent].IsZero()
+ c.Unlock()
+ if done == nil {
+ return back // read already, the first ask waiting for it; or nothing to read it with
+ }
+ select {
+ case <-done:
+ case <-ctx.Done():
+ return false
+ }
+ c.Lock()
+ back = !c.at[agent].IsZero() // zero: asked before it went stale
+ c.Unlock()
+ if back {
+ return true
+ }
+ }
+ return false
+}
+
// OnRenewed has f told when an account's usage windows were started again
// (a Codex reset spent), or a reading of them (Allowances) finds one it
// was full in full no more, so what sat out waiting for them can come