Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

gateway: a model slow to start again and again rests on its account (小八)

yetone398803e3f575
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.md
index faa19cb64..1145c01dc 100644
--- a/docs/subsystems/gateway-routing.md
+++ b/docs/subsystems/gateway-routing.md
@@ -136,6 +136,7 @@ recorded as a prediction failure (`noPrediction`).
- An account resting out of its window (a Claude account by the time Claude Code's refusal named, `resets`, or any by `window`) is back once magpie's reading of its windows (`provider.Allowances`) finds it full till sooner: the window used below `SpentShareOf`, or its reset passed, with no other window over the same models still full. The reading tells `provider.OnRenewed` as the account's `UsageAgent` and user, and `renewed` lifts that account's rests out of quota or by `window`; its rests for a rate limit, its credit, a verification or a failure stay, as a key's do, and so does every rest of other accounts still out of their windows. Planning reads every account candidate's allowance, ordered groups included, so this needs no step of its own in `restingOf`.
- Only an account is held at a usage cap (`capHeld`) or rested out of one model's pool (`pooled`); a key's windows order it and rest it, nothing more.
- A rest of one model alone (keyed by `restID`: a pool, a model not served, a paid model beside free ones) names that model in the trace (`Rest.Model`). The Routing page shows it on that model's rows only: the account's other models' rows on the stage, and the account's row under Accounts and keys, don't read as resting. The account's row names the models resting on a line of their own under what it has left, and its Try now lifts their rests. A kept request from before `Rest.Model` has the model on the end of `Rest.Key`, which the page reads the same (`restModel` in `plugins/routing/client.js`, #1960, `routing-model-rest.test.cjs`).
+- A group member let go for no first content within the group's `FirstToken` (`hw.slow` in [`gateway.go`](../../internal/gateway/gateway.go), 504 "no first token in Ns", `failSlow`) doesn't rest the first time: a long prompt is slow to start anywhere. Slow again on the same account's model with no answer from it in between, and within `slowForget` (30 minutes) of its last slow start, that model rests on that account (`restSlow` in [`routing.go`](../../internal/gateway/routing.go), keyed by `restID`, `Rest.Model` set): a minute from the second time (`slowRests`), doubling each time after, up to the provider's rest cap (`RestCapOr(longestRetry)`), and never for a provider whose cap is off. Its answer clears the count (`servedCandidate`). A request its client cancelled counts nothing, as the cancel is read before `hw.slow`. Before it, each new request asked the stalled member first again and waited out the whole `FirstToken` (小八 on Discord: a WorkBuddy account's kimi-k3-1 and glm-5.3 stalled 58 times on the vendor's per-model token limit, 15 times in 17 minutes in one session). The same model on another account and another model on the same account are asked as before (`TestSlowStartsRestTheModelOnItsAccount`, `TestCanceledSlowStartRestsNothing`; the Routing page's story, `routing-went-on.test.cjs`).
- A 402 is out of credit, unless it speaks of a usage limit (`usedUpWords`) and nothing of money (`creditWords`, `insufficient_quota`): then it is out of quota, and rests only the model when the account's windows say its pool is the one used up (`pooled`). Factory answers Standard's pool used up with 402 "You've reached your weekly standard usage limit (resets in …). Switch to Droid Core …"; read as credit, it rested the account 30 minutes and its Droid Core models with it (#1979). Both the built-in's windows and the plugin's (Standard's `notModels`, Core's `models`) are held to it. As a quota rest it lasts until the time the refusal names (`resetsAfter`: "resets in 0h 19min", "resets in 2d 3h", held to `longestWritten`), else the window's end, else `quotaRest` or the provider's own `quota-rest`, and is lifted once a reading finds the window renewed (`renewed`), which a credit rest never was (`TestFactoryStandardOutLeavesCore`, `TestPaymentRequiredReadByItsWords`, `TestResetsInWords`). An account whose windows weren't read yet (a start, or a reading that failed) can't tell the model's pool from the account's, so it rests whole, and `narrowWhenRead` waits off the request for a reading (`AwaitAllowance`, up to `allowanceWait`) and moves the rest to the model alone (`restID`, the same end) when the reading says its pool is the one used up; no reading in time, or one that says the whole account, leaves the account's rest, as does a rest lifted or made again meanwhile (#2036; `TestFactoryStandardOutLeavesCore`'s unread case, `TestResetsInWords`). The request itself isn't held for the reading: it has failed over already, and the rest's scope only matters to later requests.
- Codex offers its image tool (the `image_gen` namespace, #870) to every magpie provider. When a provider other than a ChatGPT sign-in or openai.com answers a bare 400, `attempt` asks it once more without the tool (`image_tool.go`). This happens both passed through and translated. If that works, the provider is marked unfit for the tool (`tool:image_gen`) and isn't offered it again. If it doesn't, the tool is restored and the 400 stands (#949).
- A turn the agent stopped with its tool call still out is replayed by the client with the call and no result, and Google's APIs turn a history whose call turn goes unanswered away with 400, every later turn of the conversation with it. The Gemini build (`buildCodeAssistSent`, which serves a Google sign-in, Vertex AI and a custom provider's Gemini API alike) answers such a call with the synthetic result the chat build pairs an interrupted turn's call with (`pairToolMessages`, #1275), right after the calls (`pairFunctionTurns`, #1560): beside the answers the client sent there, or in a user turn of its own before what the user said, since a turn mixing a `functionResponse` with text is turned away too (#1533); a Claude model on Antigravity gets it in the one tool_result turn. A call with no id counts as answered by a response of its name. A second answer to one call is left out, with the images it carried, before the turns are built (`dropSecondAnswers`, #2262): Google refuses a call turn whose `functionResponse` count differs from its `functionCall` count, and the Chat and Messages builds already keep the first answer. The Responses build (`buildResponses`) leaves it out the same way. A call made again under the same id takes an answer of its own. A Gemini client that keeps no ids answers two calls of one function by name in their order, and each answer goes to its own call (`parseGemini`), not both to the latest. `TestGeminiDropsASecondAnswer`, `TestSecondAnswerDroppedWithItsImages`, `TestGeminiClientUnnamedCallsAnsweredInOrder`.
diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.go
index 505cf26f8..87b90e910 100644
--- a/internal/gateway/gateway.go
+++ b/internal/gateway/gateway.go
@@ -2868,9 +2868,14 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
if hw.slow {
// no first content in the group's FirstToken, and nothing of
// it sent: the next member is asked, and this one doesn't
- // rest, a long prompt being slow to start anywhere
+ // rest the first time, a long prompt being slow to start
+ // anywhere; slow again and again, its model rests on this
+ // account (restSlow)
call.Status, call.Error = http.StatusGatewayTimeout, fmt.Sprintf("%s: no first token in %ds", c.p.Name, firstToken)
try.Status, try.Error, try.Fail = call.Status, call.Error, failSlow
+ if rest, ok := restSlow(c, time.Now()); ok {
+ try.Rest = &rest
+ }
call.TTFT, call.FirstText, call.Flow, call.FirstReasoning = 0, 0, 0, 0
watched.attempt(call, try, c.p.ID, sent, hw, capture)
s.trace.update(tr, func(t *Route) { t.Tries[len(t.Tries)-1] = try })
diff --git a/internal/gateway/group_test.go b/internal/gateway/group_test.go
index 7432a31b5..c9b110211 100644
--- a/internal/gateway/group_test.go
+++ b/internal/gateway/group_test.go
@@ -74,7 +74,7 @@ func fresh(t *testing.T) {
// rather than a turn on, or with a failure already counted.
func forgetRouting() {
routed.Lock()
- routed.used, routed.failures = map[string]tokenUse{}, map[string]int{}
+ routed.used, routed.failures, routed.slow = map[string]tokenUse{}, map[string]int{}, map[string]slowRun{}
routed.Unlock()
routes.Forget()
}
diff --git a/internal/gateway/routing.go b/internal/gateway/routing.go
index 97ac13608..b534a7bcb 100644
--- a/internal/gateway/routing.go
+++ b/internal/gateway/routing.go
@@ -38,7 +38,15 @@ var routed = struct {
sync.Mutex
used map[string]tokenUse // candidate restKey → tokens it served
failures map[string]int // candidate restKey → failures since it last answered
-}{used: map[string]tokenUse{}, failures: map[string]int{}}
+ slow map[string]slowRun // candidate restID → its slow starts since it last answered (restSlow)
+}{used: map[string]tokenUse{}, failures: map[string]int{}, slow: map[string]slowRun{}}
+
+// slowRun is a model's slow starts in a row on one account or key, and
+// when the last of them was.
+type slowRun struct {
+ n int
+ at time.Time
+}
type tokenUse struct {
n float64
@@ -74,6 +82,7 @@ func servedCandidate(c candidate, tokens int) {
if id := c.restID(); id != c.restKey() {
routed.Lock()
delete(routed.failures, id)
+ delete(routed.slow, id)
routed.Unlock()
clearRest(id)
}
@@ -183,6 +192,11 @@ const (
// afternoon without anyone acting; the Routing page lifts it at once
// (Unrest) for a user whose appeal went through
restrictedRest = 2 * time.Hour
+ // slowRests is how many slow starts in a row (restSlow) rest a model
+ // on an account or key, and slowForget how long after the last one
+ // the next starts a new run
+ slowRests = 2
+ slowForget = 30 * time.Minute
)
var (
@@ -298,8 +312,9 @@ const (
// left, and nobody rests
failRoom = "room"
// failSlow: the member hadn't begun its reply when its group's
- // FirstToken ran out, nothing of it sent — the next is asked, and
- // nobody rests, as a long prompt is slow anywhere
+ // FirstToken ran out, nothing of it sent — the next is asked. Once,
+ // nobody rests, as a long prompt is slow anywhere; again and again on
+ // one account's model, that model rests there (restSlow)
failSlow = "slow"
// failLoop: the reply was ended stuck in a loop (#1359) — the agent
// is told, and nobody rests, as the model, not the account, looped
@@ -1072,6 +1087,61 @@ func remoteModelRest(c candidate, why string, header http.Header) bool {
return why == failQuota || why == failCredit || why == failRate || why == failRestricted || why == failVerify
}
+// restSlow counts a slow start — let go for no first token in its group's
+// FirstToken — against the candidate's model on its account or key
+// (restID), and rests that model there from the slowRests'th in a row: a
+// minute, twice as long each time after, up to the provider's rest cap.
+// One alone rests nothing, a long prompt being slow to start anywhere; a
+// run ends when the model answers there (servedCandidate) or slowForget
+// passes with none. Its other models and the account's other mates are
+// asked as before (小八 on Discord: WorkBuddy's kimi-k3-1 and glm-5.3,
+// out of a per-model token limit the vendor never named, stalled 15 times
+// in 17 minutes, each request waiting 30s for it before the fallback
+// answered). A slow start while it rests for one keeps the count where it
+// is, so one request's tries don't stretch it. A provider set never to
+// rest after a failure that says nothing of why (RestCapOff) isn't
+// rested, nor its slow starts counted.
+func restSlow(c candidate, now time.Time) (Rest, bool) {
+ if c.p.NoFailRest() {
+ return Rest{}, false
+ }
+ id := c.restID()
+ prev, resting := restOf(id)
+ routed.Lock()
+ run := routed.slow[id]
+ if now.Sub(run.at) >= slowForget {
+ run.n = 0
+ }
+ if !resting || prev.Why != failSlow || run.n == 0 {
+ run.n++
+ }
+ run.at = now
+ routed.slow[id] = run
+ routed.Unlock()
+ if run.n < slowRests {
+ return Rest{}, false
+ }
+ d, by := min(fallbackCooldown<<min(run.n-slowRests, 10), c.p.RestCapOr(longestRetry)), "cooldown"
+ if run.n > slowRests {
+ by = "backoff"
+ }
+ r := Rest{Why: failSlow, Status: http.StatusGatewayTimeout, Until: now.Add(d), By: by, Failures: run.n, Key: id, Model: c.model}
+ if a := c.p.Account; a != nil {
+ r.agent, r.user = a.UsageAgent(), a.User
+ }
+ restingUntil.Lock()
+ defer restingUntil.Unlock()
+ if until, ok := restingUntil.m[id]; ok && until.After(r.Until) {
+ // resting longer already, for what another failure said
+ kept := restingUntil.note[id]
+ kept.Until = until
+ return kept, true
+ }
+ restingUntil.m[id] = r.Until
+ restingUntil.note[id] = r
+ return r, true
+}
+
// rateRest is how long a rate-limited candidate sits out: as long as the
// vendor says (Retry-After, a reset header or a reset time in its error),
// and a minute when it says nothing. One limited again as soon as its rest
diff --git a/internal/gateway/slow_rest_test.go b/internal/gateway/slow_rest_test.go
new file mode 100644
index 000000000..edcbe4301
--- /dev/null
+++ b/internal/gateway/slow_rest_test.go
@@ -0,0 +1,183 @@
+package gateway
+
+import (
+ "context"
+ "io"
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "sync"
+ "sync/atomic"
+ "testing"
+ "time"
+
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// keyedSlow is slow to start on one key's one model while slow is set,
+// and answers at once otherwise. It says who it was asked as, key/model.
+type keyedSlow struct {
+ key, model string
+ slow atomic.Bool
+ arrived chan struct{} // told of each slow request, when set
+ mu sync.Mutex
+ asked []string
+}
+
+func (k *keyedSlow) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+ body, _ := io.ReadAll(r.Body)
+ key, model := strings.TrimPrefix(r.Header.Get("Authorization"), "Bearer "), modelOf(body)
+ k.mu.Lock()
+ k.asked = append(k.asked, key+"/"+model)
+ k.mu.Unlock()
+ w.Header().Set("Content-Type", "text/event-stream")
+ w.WriteHeader(200)
+ if key == k.key && model == k.model && k.slow.Load() {
+ io.WriteString(w, `data: {"id":"s","choices":[{"index":0,"delta":{"role":"assistant"}}]}`+"\n\n")
+ w.(http.Flusher).Flush()
+ if k.arrived != nil {
+ k.arrived <- struct{}{}
+ }
+ <-r.Context().Done()
+ return
+ }
+ io.WriteString(w, chatStreamOK)
+}
+
+func (k *keyedSlow) first() string {
+ k.mu.Lock()
+ defer k.mu.Unlock()
+ if len(k.asked) == 0 {
+ return ""
+ }
+ return k.asked[0]
+}
+
+func (k *keyedSlow) reset() {
+ k.mu.Lock()
+ k.asked = nil
+ k.mu.Unlock()
+}
+
+// A model that keeps failing to start on one account rests there (小八 on
+// Discord: a WorkBuddy account's kimi-k3-1 stalled 58 times, past the
+// group's first-token time, and magpie asked it first again each time, 15
+// times in 17 minutes). Once nobody rests; twice in a row, that model on
+// that account rests, and a success in between starts the count again.
+// The same model on another key and another model on the same key are
+// asked as before.
+func TestSlowStartsRestTheModelOnItsAccount(t *testing.T) {
+ fresh(t)
+ up := &keyedSlow{key: "k1", model: "m"}
+ up.slow.Store(true)
+ serveOn(t, "a", "k1", []string{"m", "n"}, up, "k2")
+ if err := provider.SaveGroup(provider.Group{Name: "G", Members: []string{"a/m"}, Routing: provider.Ordered, FirstToken: 1}); err != nil {
+ t.Fatal(err)
+ }
+ s := New()
+ k1m, k1, k2m := "a#"+provider.KeyID("k1")+"/m", "a#"+provider.KeyID("k1"), "a#"+provider.KeyID("k2")+"/m"
+ ask := func(model string) Try {
+ t.Helper()
+ up.reset()
+ code, body := postAs(t, s, "", `{"model":"`+model+`","stream":true,"messages":[{"role":"user","content":"hi"}]}`)
+ if code != 200 || !strings.Contains(body, "from b") {
+ t.Fatalf("%s: %d %s", model, code, body)
+ }
+ r := s.trace.routes[len(s.trace.routes)-1]
+ return r.Tries[0]
+ }
+
+ if tr := ask("group/g"); up.first() != "k1/m" || tr.Fail != failSlow || tr.Rest != nil {
+ t.Fatalf("once: first %s, %+v", up.first(), tr)
+ }
+ up.slow.Store(false)
+ if tr := ask("group/g"); up.first() != "k1/m" || tr.Status != 200 {
+ t.Fatalf("answered: first %s, %+v", up.first(), tr)
+ }
+ up.slow.Store(true)
+ if tr := ask("group/g"); tr.Fail != failSlow || tr.Rest != nil {
+ t.Fatalf("an answer in between starts the count again: %+v", tr)
+ }
+ tr := ask("group/g")
+ if tr.Fail != failSlow || tr.Rest == nil || tr.Rest.Why != failSlow || tr.Rest.Model != "m" || tr.Rest.Key != k1m || tr.Rest.Failures != 2 {
+ t.Fatalf("twice in a row: %+v (rest %+v)", tr, tr.Rest)
+ }
+ if d := time.Until(tr.Rest.Until); d <= 0 || d > fallbackCooldown {
+ t.Fatalf("rests %s, want up to %s", d, fallbackCooldown)
+ }
+ if _, ok := restOf(k1); ok {
+ t.Fatal("the whole key rests")
+ }
+ if _, ok := restOf(k2m); ok {
+ t.Fatal("the model rests on the other key")
+ }
+ if ask("group/g"); up.first() != "k2/m" {
+ t.Fatalf("the resting model was asked first: %s", up.first())
+ }
+ if ask("a/n"); up.first() != "k1/n" {
+ t.Fatalf("another model on the key: first %s", up.first())
+ }
+
+ // a third slow start (asked again once its rest is over) rests longer
+ restingUntil.Lock()
+ restingUntil.m[k1m] = time.Now().Add(-time.Second)
+ restingUntil.Unlock()
+ tr = ask("group/g")
+ if tr.Rest == nil || tr.Rest.By != "backoff" || tr.Rest.Failures != 3 || time.Until(tr.Rest.Until) <= fallbackCooldown {
+ t.Fatalf("again: %+v (rest %+v)", tr, tr.Rest)
+ }
+ // and an answer ends it
+ restingUntil.Lock()
+ restingUntil.m[k1m] = time.Now().Add(-time.Second)
+ restingUntil.Unlock()
+ up.slow.Store(false)
+ ask("group/g")
+ up.slow.Store(true)
+ if tr := ask("group/g"); tr.Rest != nil {
+ t.Fatalf("after an answer: rest %+v", tr.Rest)
+ }
+}
+
+// A request its client gives up on while the model hasn't begun counts no
+// slow start, however often.
+func TestCanceledSlowStartRestsNothing(t *testing.T) {
+ fresh(t)
+ up := &keyedSlow{key: "k1", model: "m", arrived: make(chan struct{}, 1)}
+ up.slow.Store(true)
+ serveOn(t, "a", "k1", []string{"m"}, up, "k2")
+ if err := provider.SaveGroup(provider.Group{Name: "G", Members: []string{"a/m"}, Routing: provider.Ordered, FirstToken: 60}); err != nil {
+ t.Fatal(err)
+ }
+ s := New()
+ for i := range 3 {
+ ctx, cancel := context.WithCancel(context.Background())
+ done := make(chan struct{})
+ go func() {
+ defer close(done)
+ req := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(`{"model":"group/g","stream":true,"messages":[{"role":"user","content":"hi"}]}`)).WithContext(ctx)
+ s.Handler().ServeHTTP(httptest.NewRecorder(), req)
+ }()
+ select {
+ case <-up.arrived:
+ case <-done: // answered by another: the slow one wasn't asked
+ cancel()
+ t.Fatalf("cancel %d: k1/m wasn't asked (rested?)", i)
+ }
+ cancel()
+ <-done
+ up.mu.Lock()
+ asked := append([]string(nil), up.asked...)
+ up.mu.Unlock()
+ if len(asked) != i+1 {
+ t.Fatalf("cancel %d: asked %v", i, asked)
+ }
+ }
+ if _, ok := restOf("a#" + provider.KeyID("k1") + "/m"); ok {
+ t.Fatal("a canceled request rests the model")
+ }
+ routed.Lock()
+ defer routed.Unlock()
+ if len(routed.slow) != 0 {
+ t.Fatalf("counted: %v", routed.slow)
+ }
+}
diff --git a/internal/gui/assets/plugins/routing/client.js b/internal/gui/assets/plugins/routing/client.js
index e1b5ac3be..d3e92cc28 100644
--- a/internal/gui/assets/plugins/routing/client.js
+++ b/internal/gui/assets/plugins/routing/client.js
@@ -1045,6 +1045,10 @@ function routingPage(part) {
if (tr.fail === "slow")
return !next && r.done
? t("{who} hadn't begun answering after {ms}, and nobody is left to try, so {agent} gets the error.", { who: name, ms: took(tr.ms), agent })
+ // slow again and again on this account: its model rests here
+ : tr.rest
+ ? t("{who} hadn't begun answering after {ms}, {n} times in a row, so the request went on to the next before any of it reached {agent}. It rests {d} for {model} alone, longer each time it is slow again; its other models answer as before.",
+ { who: name, ms: took(tr.ms), n: tr.rest.failures, model: tr.rest.model || tr.model, d: dur(at(tr.rest.until) - at(tr.start) - (tr.ms || 0)), agent })
: t("{who} hadn't begun answering after {ms}, so the request went on to the next before any of it reached {agent}. Nothing is wrong with {who}, so it doesn't rest.", { who: name, ms: took(tr.ms), agent });
if (tr.fail === "proxy")
return r.tries[i + 1]
@@ -5096,7 +5100,7 @@ function routingPage(part) {
firstIn.hidden = firstUnit.hidden = firstMode !== "custom";
firstIn.setAttribute("aria-invalid", String(!!firstErr));
fHint.textContent = firstErr || (d.firstToken
- ? t("A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. It doesn't rest; the last one left is always waited for. Only for streamed requests.", { n: took(d.firstToken * 1000) })
+ ? t("A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. Let go once, it doesn't rest; let go twice in a row, its model rests on that account, a minute and longer each time, until it answers again. The last one left is always waited for. Only for streamed requests.", { n: took(d.firstToken * 1000) })
: t("Every member is waited for, however long it takes to begin answering."));
};
fw.append(segs([...FIRST_OPTS.map(([n, l]) => [n, t(l)]), ["custom", t("Custom")]], firstMode, (v) => {
diff --git a/internal/gui/assets/plugins/routing/i18n.js b/internal/gui/assets/plugins/routing/i18n.js
index 3b65c83cf..2cb7dd3cb 100644
--- a/internal/gui/assets/plugins/routing/i18n.js
+++ b/internal/gui/assets/plugins/routing/i18n.js
@@ -172,9 +172,10 @@ magpie.strings({
"Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.": "告诉 agent 组里最大模型的上下文;某个成员放不下的对话会交给放得下的成员。",
"{v} is not a length of tokens like 200k or 1m": "{v} 不是有效的 token 长度,例如 200k 或 1m",
"Every member is waited for, however long it takes to begin answering.": "每个成员都会一直等到它开始回答,不论多久。",
- "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. It doesn't rest; the last one left is always waited for. Only for streamed requests.": "成员超过 {n} 仍未开始回答(无文字、思考或工具调用)时,改问下一个,此时尚无内容发给 agent。它不会休息;最后一个总会等到底。仅对流式请求生效。",
+ "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. Let go once, it doesn't rest; let go twice in a row, its model rests on that account, a minute and longer each time, until it answers again. The last one left is always waited for. Only for streamed requests.": "成员超过 {n} 仍未开始回答(无文字、思考或工具调用)时,改问下一个,此时尚无内容发给 agent。偶尔一次不休息;连续两次,该模型在这个账号上休息,从一分钟起每次更久,直到它再次回答。最后一个总会等到底。仅对流式请求生效。",
"Next after {n} without a first token": "{n} 无首字换下一个",
"{who} hadn't begun answering after {ms}, so the request went on to the next before any of it reached {agent}. Nothing is wrong with {who}, so it doesn't rest.": "{who} 超过 {ms} 未开始回答,请求在发给 {agent} 任何内容前转给下一个。{who} 本身没有出错,不休息。",
+ "{who} hadn't begun answering after {ms}, {n} times in a row, so the request went on to the next before any of it reached {agent}. It rests {d} for {model} alone, longer each time it is slow again; its other models answer as before.": "{who} 连续 {n} 次超过 {ms} 未开始回答,请求在发给 {agent} 任何内容前转给下一个。它仅就 {model} 休息 {d},再慢则休息更久;它的其他模型照常回答。",
"{who} was rate limited at {time} while it had quota left, so it went to the back: it comes round again once those ahead of it are rate limited in turn.": "{who} 在 {time} 仍有额度时被限流,已排到最后:前面的账号依次被限流后才再轮到它。",
"rate limited at {time} · at the back": "{time} 被限流 · 排在最后",
"A group needs a model in it": "组里至少要有一个模型",
@@ -714,9 +715,10 @@ magpie.strings({
"Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.": "告訴 agent 組裡最大模型的上下文;某個成員放不下的對話會交給放得下的成員。",
"{v} is not a length of tokens like 200k or 1m": "{v} 不是有效的 token 長度,例如 200k 或 1m",
"Every member is waited for, however long it takes to begin answering.": "每個成員都會一直等到它開始回答,不論多久。",
- "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. It doesn't rest; the last one left is always waited for. Only for streamed requests.": "成員超過 {n} 仍未開始回答(無文字、思考或工具呼叫)時,改問下一個,此時尚無內容發給 agent。它不會休息;最後一個總會等到底。僅對流式請求生效。",
+ "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. Let go once, it doesn't rest; let go twice in a row, its model rests on that account, a minute and longer each time, until it answers again. The last one left is always waited for. Only for streamed requests.": "成員超過 {n} 仍未開始回答(無文字、思考或工具呼叫)時,改問下一個,此時尚無內容發給 agent。偶爾一次不休息;連續兩次,該模型在這個帳號上休息,從一分鐘起每次更久,直到它再次回答。最後一個總會等到底。僅對流式請求生效。",
"Next after {n} without a first token": "{n} 無首字換下一個",
"{who} hadn't begun answering after {ms}, so the request went on to the next before any of it reached {agent}. Nothing is wrong with {who}, so it doesn't rest.": "{who} 超過 {ms} 未開始回答,請求在發給 {agent} 任何內容前轉給下一個。{who} 本身沒有出錯,不休息。",
+ "{who} hadn't begun answering after {ms}, {n} times in a row, so the request went on to the next before any of it reached {agent}. It rests {d} for {model} alone, longer each time it is slow again; its other models answer as before.": "{who} 連續 {n} 次超過 {ms} 未開始回答,請求在發給 {agent} 任何內容前轉給下一個。它僅就 {model} 休息 {d},再慢則休息更久;它的其他模型照常回答。",
"{who} was rate limited at {time} while it had quota left, so it went to the back: it comes round again once those ahead of it are rate limited in turn.": "{who} 在 {time} 仍有額度時被限流,已排到最後:前面的帳號依次被限流後才再輪到它。",
"rate limited at {time} · at the back": "{time} 被限流 · 排在最後",
"A group needs a model in it": "組裡至少要有一個模型",
@@ -1236,9 +1238,10 @@ magpie.strings({
"Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.": "エージェントには最大モデルのコンテキストを伝えます。あるメンバーに長すぎる会話は収まるメンバーに回されます。",
"{v} is not a length of tokens like 200k or 1m": "{v} は 200k や 1m のようなトークン数ではありません",
"Every member is waited for, however long it takes to begin answering.": "どのメンバーも、応答を始めるまでどれだけかかっても待ちます。",
- "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. It doesn't rest; the last one left is always waited for. Only for streamed requests.": "{n} 経っても応答を始めない(テキスト、推論、ツール呼び出しがまだない)メンバーは、何もエージェントに届く前に打ち切られ、次のメンバーに問い合わせます。そのメンバーは休止しません。最後に残ったメンバーは常に待ちます。ストリーミングリクエストのみ対象です。",
+ "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. Let go once, it doesn't rest; let go twice in a row, its model rests on that account, a minute and longer each time, until it answers again. The last one left is always waited for. Only for streamed requests.": "{n} 経っても応答を始めない(テキスト、推論、ツール呼び出しがまだない)メンバーは、何もエージェントに届く前に打ち切られ、次のメンバーに問い合わせます。1 回なら休止しません。2 回続けば、そのアカウントでそのモデルが休止します。1 分から始まり、再び応答するまで毎回長くなります。最後に残ったメンバーは常に待ちます。ストリーミングリクエストのみ対象です。",
"Next after {n} without a first token": "最初のトークンが {n} なければ次へ",
"{who} hadn't begun answering after {ms}, so the request went on to the next before any of it reached {agent}. Nothing is wrong with {who}, so it doesn't rest.": "{who} が {ms} 経っても応答を始めなかったため、{agent} に何も届く前にリクエストは次へ進みました。{who} に問題はないので、休止しません。",
+ "{who} hadn't begun answering after {ms}, {n} times in a row, so the request went on to the next before any of it reached {agent}. It rests {d} for {model} alone, longer each time it is slow again; its other models answer as before.": "{who} が {ms} 経っても応答を始めないことが {n} 回続いたため、{agent} に何も届く前にリクエストは次へ進みました。{model} についてだけ {d} 休止し、また遅ければ毎回長くなります。ほかのモデルはこれまでどおり応答します。",
"{who} was rate limited at {time} while it had quota left, so it went to the back: it comes round again once those ahead of it are rate limited in turn.": "{who} は {time} にクォータが残っているのにレート制限されたため、最後尾に回りました。前にあるものが順にレート制限されると再び回ってきます。",
"rate limited at {time} · at the back": "{time} にレート制限 · 最後尾",
"A group needs a model in it": "グループには少なくとも 1 つのモデルが必要です",
@@ -1769,9 +1772,10 @@ magpie.strings({
"Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.": "Agenten wird das Fenster des größten Modells gesagt; ein für ein Mitglied zu langes Gespräch geht an eines mit Platz dafür.",
"{v} is not a length of tokens like 200k or 1m": "{v} ist keine Tokenlänge wie 200k oder 1m",
"Every member is waited for, however long it takes to begin answering.": "Auf jedes Mitglied wird gewartet, egal wie lange es bis zum Antwortbeginn braucht.",
- "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. It doesn't rest; the last one left is always waited for. Only for streamed requests.": "Ein Mitglied, das nach {n} noch nicht zu antworten begonnen hat — noch kein Text, kein Reasoning, kein Tool-Aufruf —, wird aufgegeben und das nächste gefragt, bevor etwas davon den Agenten erreicht. Es pausiert nicht; auf das letzte verbleibende wird immer gewartet. Nur für gestreamte Anfragen.",
+ "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. Let go once, it doesn't rest; let go twice in a row, its model rests on that account, a minute and longer each time, until it answers again. The last one left is always waited for. Only for streamed requests.": "Ein Mitglied, das nach {n} noch nicht zu antworten begonnen hat — noch kein Text, kein Reasoning, kein Tool-Aufruf —, wird aufgegeben und das nächste gefragt, bevor etwas davon den Agenten erreicht. Einmal aufgegeben, pausiert es nicht; zweimal hintereinander, pausiert sein Modell auf diesem Konto, eine Minute und jedes Mal länger, bis es wieder antwortet. Auf das letzte verbleibende wird immer gewartet. Nur für gestreamte Anfragen.",
"Next after {n} without a first token": "Nächstes nach {n} ohne erstes Token",
"{who} hadn't begun answering after {ms}, so the request went on to the next before any of it reached {agent}. Nothing is wrong with {who}, so it doesn't rest.": "{who} hatte nach {ms} noch nicht zu antworten begonnen, daher ging die Anfrage an das nächste, bevor etwas davon {agent} erreichte. Mit {who} ist alles in Ordnung, daher pausiert es nicht.",
+ "{who} hadn't begun answering after {ms}, {n} times in a row, so the request went on to the next before any of it reached {agent}. It rests {d} for {model} alone, longer each time it is slow again; its other models answer as before.": "{who} hatte {n}-mal hintereinander nach {ms} noch nicht zu antworten begonnen, daher ging die Anfrage an das nächste, bevor etwas davon {agent} erreichte. Es pausiert {d} nur für {model}, jedes Mal länger, wenn es wieder langsam ist; seine anderen Modelle antworten wie bisher.",
"{who} was rate limited at {time} while it had quota left, so it went to the back: it comes round again once those ahead of it are rate limited in turn.": "{who} erhielt um {time} trotz verbleibendem Kontingent ein Rate-Limit und wanderte daher nach hinten: Es kommt wieder an die Reihe, sobald auch die vor ihm ein Rate-Limit erhalten.",
"rate limited at {time} · at the back": "Rate-Limit um {time} · hinten",
"A group needs a model in it": "Eine Gruppe braucht mindestens ein Modell",
diff --git a/internal/gui/tests/group-first-token.test.cjs b/internal/gui/tests/group-first-token.test.cjs
index 588e0592f..2b1a0321d 100644
--- a/internal/gui/tests/group-first-token.test.cjs
+++ b/internal/gui/tests/group-first-token.test.cjs
@@ -169,8 +169,9 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
const missing = await page.evaluate(() => [
"Wait", "30 s", "1 min", "2 min", "Slow to start", "slow to start", "Next after {n} without a first token",
"Every member is waited for, however long it takes to begin answering.",
- "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. It doesn't rest; the last one left is always waited for. Only for streamed requests.",
+ "A member that hasn't begun answering after {n} — no text, reasoning or tool call yet — is let go and the next one asked, before any of it reaches the agent. Let go once, it doesn't rest; let go twice in a row, its model rests on that account, a minute and longer each time, until it answers again. The last one left is always waited for. Only for streamed requests.",
"{who} hadn't begun answering after {ms}, so the request went on to the next before any of it reached {agent}. Nothing is wrong with {who}, so it doesn't rest.",
+ "{who} hadn't begun answering after {ms}, {n} times in a row, so the request went on to the next before any of it reached {agent}. It rests {d} for {model} alone, longer each time it is slow again; its other models answer as before.",
"In magpie", "Vendor's first token",
"Custom", "e.g. 5", "Minutes", "min", "{v} is not a number of minutes",
].filter((k) => !["zh", "zh-TW", "ja", "de"].every((l) => I18N[l][k])));
diff --git a/internal/gui/tests/routing-went-on.test.cjs b/internal/gui/tests/routing-went-on.test.cjs
index 4b7c73cb9..85de84b90 100644
--- a/internal/gui/tests/routing-went-on.test.cjs
+++ b/internal/gui/tests/routing-went-on.test.cjs
@@ -8,7 +8,8 @@
// with none, the agent got the error. And the last one left that rests
// (a key, #1772), or was slow to begin, says nobody is left, not that the
// request goes on; the last one out of its allowance after another failed
-// says the agent got the earlier error (Try.Earlier).
+// says the agent got the earlier error (Try.Earlier). A member slow to
+// begin again and again (小八 on Discord) says its model rests there.
const assert = require("node:assert/strict");
const fs = require("node:fs/promises");
const path = require("node:path");
@@ -58,6 +59,15 @@ const routes = [{
{ id: codex.id, model: codex.model, start: at(3), done: true, status: 429, ms: 200, fail: "quota", earlier: true },
],
done: true, status: 503, error: "Grok: upstream busy", ms: 500,
+}, {
+ // slow to begin twice in a row: its model rests on that account
+ id: 100, seq: 100, time: at(4), agent: "claude", model: "group/s", provider: "antigravity", order: [agA, agB],
+ tries: [
+ { id: agA.id, model: agA.model, start: at(4), done: true, status: 504, ms: 30000, fail: "slow", error: "Antigravity: no first token in 30s",
+ rest: { status: 504, why: "slow", by: "cooldown", failures: 2, model: agA.model, until: new Date(Date.parse(at(4)) + 90e3).toISOString() } },
+ { id: agB.id, model: agB.model, start: at(4), done: true, status: 200, ms: 4000 },
+ ],
+ done: true, status: 200, ms: 34000,
}];
function serve(lang) {
@@ -91,15 +101,20 @@ function serve(lang) {
// the next; earlier: the agent got the earlier error
const want = {
en: { went: "and the request goes on to [email protected] before any of the reply reaches Claude Code", gets: ["wouldn't fix", "so Claude Code gets"], passed: "passed to", row: "went on to the next",
- left: "nobody is left to try, so Claude Code gets the error", on: "goes on to", earlier: "Claude Code gets the error an earlier try gave" },
+ left: "nobody is left to try, so Claude Code gets the error", on: "goes on to", earlier: "Claude Code gets the error an earlier try gave",
+ rests: "It rests 1 min for claude-sonnet-4-6 alone", calm: "so it doesn't rest" },
zh: { went: "请求在回复到达 Claude Code 前转给 [email protected]", gets: ["换账号也无法解决", "Claude Code 收到此错误"], passed: "已交给", row: "已转给下一个",
- left: "且无其他可试,Claude Code 收到此错误", on: "转给", earlier: "Claude Code 收到的是此前某次尝试的错误" },
+ left: "且无其他可试,Claude Code 收到此错误", on: "转给", earlier: "Claude Code 收到的是此前某次尝试的错误",
+ rests: "连续 2 次", calm: "不休息" },
"zh-TW": { went: "請求在回覆到達 Claude Code 前轉給 [email protected]", gets: ["換帳號也無法解決", "Claude Code 收到此錯誤"], passed: "已交給", row: "已轉給下一個",
- left: "且無其他可試,Claude Code 收到此錯誤", on: "轉給", earlier: "Claude Code 收到的是先前某次嘗試的錯誤" },
+ left: "且無其他可試,Claude Code 收到此錯誤", on: "轉給", earlier: "Claude Code 收到的是先前某次嘗試的錯誤",
+ rests: "連續 2 次", calm: "不休息" },
ja: { went: "リクエストは [email protected] に回ります", gets: ["別のアカウントでも解決しない", "Claude Code に届きます"], passed: "渡しました", row: "次に回しました",
- left: "他に試せる相手がいないため、Claude Code にエラーが届きます", on: "に回ります", earlier: "それより前の試行のエラー" },
+ left: "他に試せる相手がいないため、Claude Code にエラーが届きます", on: "に回ります", earlier: "それより前の試行のエラー",
+ rests: "2 回続いた", calm: "休止しません" },
de: { went: "die Anfrage geht an [email protected], bevor etwas von der Antwort Claude Code erreicht", gets: ["nicht beheben würde", "daher erhält Claude Code ihn"], passed: "übergeben", row: "an den nächsten weitergegeben",
- left: "es bleibt niemand zum Versuchen, daher erhält Claude Code den Fehler", on: "geht an", earlier: "Claude Code erhält den Fehler eines früheren Versuchs" },
+ left: "es bleibt niemand zum Versuchen, daher erhält Claude Code den Fehler", on: "geht an", earlier: "Claude Code erhält den Fehler eines früheren Versuchs",
+ rests: "2-mal hintereinander", calm: "daher pausiert es nicht" },
};
for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
@@ -161,6 +176,12 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
assert(c, JSON.stringify(steps));
assert(c.includes(w.earlier), c);
+ // slow to begin twice in a row: its model rests on that account
+ steps = await stepsOf(4);
+ const sr = steps.find((s) => s.includes("[email protected] (") && s.includes("30"));
+ assert(sr, JSON.stringify(steps));
+ assert(sr.includes(w.rests) && sr.includes("claude-sonnet-4-6") && !sr.includes(w.calm), sr);
+
assert.equal(await page.evaluate(() => document.documentElement.scrollWidth <= innerWidth), true, "the page scrolls sideways");
assert.deepEqual(errors, []);
});