Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
routing: an intent's effort survives the group's failover (#3664)
yetonee863f520c339
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.mdindex 990886d3b..6672026a7 100644--- a/docs/subsystems/gateway-routing.md+++ b/docs/subsystems/gateway-routing.md@@ -165,7 +165,7 @@ recorded as a prediction failure (`noPrediction`).- The classifier's requests are magpie's own (#1839). Asked in words, they go out with `RouterAgent`'s User-Agent (`magpie-router/1`), so their records' agent is magpie and `callKind` marks them `routing`; a decision model's (Jev, System One) record is written with `Kind: "routing"` too. They go to the same key or account as any other request and count in its totals, while `usage.Group.Own` (`Record.ByMagpie`: agent `magpie` or one of its `magpie-` User-Agents, with search's and image description's) says how many of an account's, key's or model's calls, tokens and cost were magpie's own. A classifier set to a subscription's model (`modelRef.Sub`, or a group whose first member is one) asks that subscription as every turn begins, twice when it picks both intent and effort in words; the Routing page says so beside it (`classifierSub` in `plugins/routing/client.js`). `TestIntentRoutesATurn`, `TestJevPicksTheIntent`, `TestSummaryCountsMagpiesOwnCallsApart`, `TestByMagpie`, `TestServedRefSaysASubscription`, `own-requests.test.cjs`.- A rule may name several members (`Rule.More` after `Rule.Use`, `more` in the group's JSON, `use=a,b` typed; monkeyin_92 on X: one rule, several models, not a rule each). They go first in that order, ahead of the later matching rules' members (`ThenUses` leaves out every one the rule names) and the group's others: when the first fails or rests the rule's next takes the request, and the hit says so (`RuleHit.More`; `Instead` is that next one with `Unready` false, where an `Instead` from a later rule comes with `Unready` true). `Uses`, `SetUses`, `MapUses`, `Without` and `Names` in [`grouprule.go`](../../internal/provider/grouprule.go) are how every reader goes through them: `cleanRules` (each must be the group's, each named once, a pause names one), the turn's hold, the tool rounds and compaction (`ruleFor`, the rule's first that takes the summary), `nestedRules` and `ruleLed` (the one of them a nested group's turn stays on), `ruleEffort` (the rule's `useEffort` asked of each), `ruledEntry` (what a group takes is what every one of them takes), a model's rename and removal (a rule goes with its last), `ShareGroup` and its import (one this magpie hasn't is left out of the rule), the CLI's `groups rules` and the TUI. The editor draws a chip per member with a + for the next and an × on each while there is more than one (`rUses`/`setUses` in [`plugins/routing/client.js`](../../internal/gui/assets/plugins/routing/client.js)). A rule of one member writes no `more`, so a group saved before it reads and saves as it was. A magpie from before it reads `use` alone: its requests go to the first, and the others are plain members in the group's order; if it saves the group, `more` is dropped. `TestRuleSeveralModelsInOrder`, `TestCleanRulesSeveralModels`, `routing-rule-models.test.cjs`.- The editor's rules are checked top first, so their order is theirs to set: a rule's number is its handle (`ruleHandle`, `.rt-rhandle`), dragged (`dragRows`, as the members are) or moved a place with Alt+↑/↓, the rules renumbered at once; the grip beside it shows on hover only, and Move up and down stay under its ⋯ (`moveRule`). Saved as the group's `rules` in their new order. `routing-rule-drag.test.cjs`.- - A rule can set the reasoning its member is asked for (`Rule.UseEffort`, `useEffort` in the group's JSON, `use-effort=low` typed; #2633, dengtongcai: an intent picks the model and the effort as a pair). It is an output like `Use`, not a condition like `Effort`. `ruleFor` copies it onto the hit (`RuleHit.UseEffort`) from the rule that put a member first, outer or nested; `ruleOver` keeps that rule's members, and `ruleEffort` gives each candidate of them the level fitted to its model's own (`fitLevel`), which `withFixedEffort` writes into the request's field for its protocol (Chat's `reasoning_effort`, Responses' `reasoning.effort`, Anthropic's thinking budget or `output_config.effort`) even when the agent asked none. A member fixed at an effort keeps its own; otherwise the rule's beats the model's suffix, the turn's pick and the agent's pick, and isn't bounded by the group's levels. A model whose levels aren't known (`Efforts` empty) is asked as the agent asked, as with the auto verdict. The members the group fails over to are asked as the agent asked. A Gemini-protocol client's request is left as it came, as for every effort magpie sets. `cleanRules` refuses a value outside `provider.Efforts` and a pause with one. The try records it (`Try.RuleEffort`), and the Routing page says the rule set it. `rule_effort_test.go`, `grouprule_useeffort_test.go`, `routing-rule-effort.test.cjs`.+ - A rule can set the reasoning its member is asked for (`Rule.UseEffort`, `useEffort` in the group's JSON, `use-effort=low` typed; #2633, dengtongcai: an intent picks the model and the effort as a pair). It is an output like `Use`, not a condition like `Effort`. `ruleFor` copies it onto the hit (`RuleHit.UseEffort`) from the rule that put a member first, outer or nested; `ruleOver` keeps that rule's members, and `ruleEffort` gives each candidate of them the level fitted to its model's own (`fitLevel`), which `withFixedEffort` writes into the request's field for its protocol (Chat's `reasoning_effort`, Responses' `reasoning.effort`, Anthropic's thinking budget or `output_config.effort`) even when the agent asked none. A member fixed at an effort keeps its own; otherwise the rule's beats the model's suffix, the turn's pick and the agent's pick, and isn't bounded by the group's levels. A model whose levels aren't known (`Efforts` empty) is asked as the agent asked, as with the auto verdict. The members the group fails over to are asked it too (#3664): the effort is the turn's decision, so when the rule's own member is turned away the member that takes the request over is asked the same level, fitted to its model, rather than the agent's. A rule that names the candidate goes before one it only fails over from, and a nested group's rule decides for that group's members only: when the whole nested group fails, the outer group's member outside it is asked as the agent asked, or by the outer group's rule (`TestIntentEffortSurvivesTheGroupsFailover`, `TestNestedRuleEffortStaysInItsGroup`). A Gemini-protocol client's request is left as it came, as for every effort magpie sets. `cleanRules` refuses a value outside `provider.Efforts` and a pause with one. The try records it (`Try.RuleEffort`), and the Routing page says the rule set it. `rule_effort_test.go`, `grouprule_useeffort_test.go`, `routing-rule-effort.test.cjs`.- A rule by folder (`Rule.Folders`, `folders=~/work/acme,~/oss`; Cario0x on X: routing by the project being worked on) matches when the folder the agent works in is one of them or inside one (`provider.FolderUnder`, [`rulefolder.go`](../../internal/provider/rulefolder.go)): at a path's boundary, so `/a/b` is not under `/a/bc`; `~` is the home folder; a Windows drive is one with its WSL mount (`C:\work` is `/mnt/c/work`) and a `\\wsl$\<distro>` or `\\wsl.localhost\<distro>` share is the distro's own path; case doesn't count on macOS and Windows, nor ever for a drive. A rule outside folders (`Rule.ExceptFolders`, `except-folders=~/big-task`; #1606, Ti800) matches when the folder is in none of them, and an agent that says none isn't known to be in one, so it matches; a rule may have both (`Rule.ByFolder`, `inFolders`). The folder is read from the request only when a rule of the group has folders or except-folders (`workDir` in [`workdir.go`](../../internal/gateway/workdir.go)), in each agent's own words, taken from captured requests: Claude Code's ` - Primary working directory: <path>` (its `<`/`>` as entities; ` (was <old>)` after a change dropped), Codex's `<environment_context>` `<cwd>`, Pi's `<cwd>` in the system prompt, and OpenCode's `Working directory:` inside its `<env>` block. Only the system prompt and the user's and system messages' text count, never the model's words or a tool's result, and the last said wins. An agent that says none has no folder, and no `folders=` rule matches it (unknown, never empty). The same reading tells a plugin's fetch where its agent works ([provider-plugins](provider-plugins.md#where-the-agent-works-for-a-plugins-fetch)). A pause rule may hold by folder too (below). The trace's `RuleHit.Folder` is the folder read, and the Routing page tells it beside the rule's `folder …` condition. `TestFolderUnder`, `TestRuleMatchesByFolder`, `TestParseRuleFolders`, `TestWorkDirOfEachAgent`, `TestRuleByFolder`, `TestRuleMatchesExceptFolders`, `routing-folder.test.cjs`. Rules by folder are never in a shared group (`ShareGroup` drops every `ByFolder` rule).- A rule by agent (`Rule.Agents`, `agents=codex,sillytavern`) matches the agent the gateway names the request's (`agentOf` in [`gateway.go`](../../internal/gateway/gateway.go): its `magpie-<name>` key, else its User-Agent), in lower case (`cleanRules`, `MatchesBesidesIntent`, `Pauses`). A client magpie doesn't know is named by its User-Agent's product name, before the first `/` or space (`usage.AgentOf`), so a rule can hold SillyTavern or one's own app (raikyou on Discord). The editor's agent picker lists the agents magpie knows, then those `/api/usage?period=30d` heard from that it doesn't (not `other`, nor `magpie`), and takes a name typed in its filter as the gateway reads a User-Agent (`uaName` in [`plugins/routing/client.js`](../../internal/gui/assets/plugins/routing/client.js), through the picker's `field.typed`): "MyApp/2.0 (Linux)" is `myapp`. `routing-rule-agent-typed.test.cjs`.- A pause rule (`Rule.Pause`, `pause=<model>`; John on Discord: a model of the group paused in its vendor's peak hours) leaves the model it names out of the group while it holds (`provider.PausedOut`, on `ruleClock`): the group's own rules and each nested group's on the way down to a member, a group named taking all its models, and a model named taking it wherever a group in the group has it (#625). It runs before planning, on every request rather than once a turn, so the model is not first, not a failover, and not kept by affinity, and comes back as soon as the hours end. It holds by hours, days, agents and folders only (`cleanRules` refuses another condition), and is not a send-to rule: `Matches`, `MatchRule`, `ThenUses`, `Ruled` and the compaction rules leave it out. A send-to rule naming a paused model finds it unready and the group routes as without it. The trace keeps what was paused (`GroupRef.Paused`), and the Routing page says so. Every model paused answers 503 (`pausedError`) naming each rule. Embeddings go by the same pauses; manual groups have no rules (`Live`), so none pause there. A pause by folder (#1606, Ti800: a subscription's quota kept for the long task in one folder, `pause=<model> except-folders=~/big-task`) reads the folder only when such a pause is on the way (`provider.PausesByFolder`, then `workDir`): outside its except-folders, which includes a request that says no folder (a subagent's or a small model's that doesn't), the model is left out; with `folders=`, only requests known to be in them. Embeddings and System One's decision calls have no folder. `Paused.Folder`/`ByFolder` carry what was read, and the Routing page says the folder or that none was said. `rules_pause_test.go`, `TestPauseByFolder`, `TestPauseByFolderRule`, `TestParsePauseByFolder`, `routing-pause-folder.test.cjs`.diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.goindex 710b520ed..a86d0a479 100644--- a/internal/gateway/gateway.go+++ b/internal/gateway/gateway.go@@ -2852,7 +2852,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro// member fixed at an effort keeps its ownruled := ""if c.effort == "" {- if ruled = ruleEffort(c, hit, nested); ruled != "" {+ if ruled = ruleEffort(c, hit, nested, ms); ruled != "" {fixed = ""}}diff --git a/internal/gateway/rule_effort_test.go b/internal/gateway/rule_effort_test.goindex 3dc55614a..5a1ff6eb2 100644--- a/internal/gateway/rule_effort_test.go+++ b/internal/gateway/rule_effort_test.go@@ -84,28 +84,105 @@ func TestRuleEffortUnsetOrUnknownLevels(t *testing.T) {}}-// Only the member the rule names: once it fails, the next member is asked-// as the agent asked.-func TestRuleEffortStaysWithItsMember(t *testing.T) {- s, a, b := ruled(t)- for _, m := range []string{"a/small", "b/big"} {- if err := provider.SetModelEfforts(m, []string{"low", "medium", "high"}); err != nil {+// #3664: an intent names the model and the effort it is asked for, and+// that model's provider turns the request away. The group fails over to+// another of its members, and the turn keeps the intent's effort there,+// fitted to that model's own levels, rather than the agent's. A model whose+// levels aren't known is asked as the agent asked, and a member the user+// fixed at an effort keeps its own (#189).+func TestIntentEffortSurvivesTheGroupsFailover(t *testing.T) {+ for _, x := range []struct {+ name, big string+ levels []string // b/big's, nil for unknown+ want, ruled string // what b is sent, and Try.RuleEffort+ fixed string+ }{+ {"unknown levels", "b/big", nil, "high", "", ""},+ {"the intent's level", "b/big", []string{"low", "medium", "high"}, "low", "low", ""},+ {"fitted to its own", "b/big", []string{"medium", "high"}, "medium", "medium", ""},+ {"fixed at its own", "b/big:high", []string{"low", "medium", "high"}, "high", "", "high"},+ } {+ // each its own magpie: the first failure rests a/small's key+ s, a, b, c := intented(t, provider.Rule{Use: "a/small", Intent: "simple questions", UseEffort: "low"})+ if err := provider.SetModelEfforts("a/small", []string{"low", "medium", "high"}); err != nil {t.Fatal(err)}+ if x.levels != nil {+ if err := provider.SetModelEfforts("b/big", x.levels); err != nil {+ t.Fatal(err)+ }+ }+ g, _, _ := provider.FindGroup("group/r")+ g.Members = []string{"a/small", x.big}+ if err := provider.SaveGroup(g); err != nil {+ t.Fatal(err)+ }+ c.set("1", 0)+ a.mu.Lock()+ a.fail = 500+ a.mu.Unlock()+ out, r := postOK(t, s, "", chat(long(2000), nil, 0, `,"reasoning_effort":"high"`))+ if sent := sentBody(t, b); !strings.Contains(out, "from kb") || sent["reasoning_effort"] != x.want {+ t.Fatalf("%s: the member the group failed over to sent %v (%s)", x.name, sent, out)+ }+ if sent := sentBody(t, a); sent["reasoning_effort"] != "low" {+ t.Fatalf("%s: the intent's member sent %v", x.name, sent)+ }+ if len(r.Tries) != 2 || r.Tries[0].RuleEffort != "low" || r.Tries[1].RuleEffort != x.ruled || r.Tries[1].Effort != x.want || r.Tries[1].Fixed != x.fixed {+ t.Fatalf("%s: traced %+v", x.name, r.Tries)+ }}- setMembers(t, []string{"a/small", "b/big"}, provider.Rule{Use: "a/small", Tokens: 1000, UseEffort: "low"})- a.mu.Lock()- a.fail = 500- a.mu.Unlock()- out, r := postOK(t, s, "", chat(long(2000), nil, 0, `,"reasoning_effort":"medium"`))- if sent := sentBody(t, b); !strings.Contains(out, "from kb") || sent["reasoning_effort"] != "medium" {- t.Fatalf("fallback sent %v (%s)", sent, out)+}++// #3664 within a group of groups: the inner group's rule asks its member+// for low reasoning. When that member fails, the inner group's other is+// asked low too; when the whole inner group fails, the outer group's+// member outside it is asked as the agent asked, since the inner rule+// decides for its own group's members only.+func TestNestedRuleEffortStaysInItsGroup(t *testing.T) {+ fresh(t)+ k := &keyed{fail: map[string]int{}}+ for _, id := range []string{"f1", "q1", "q2"} {+ serveOn(t, id, "k"+id, []string{"m"}, k)+ if err := provider.SetModelEfforts(id+"/m", []string{"low", "medium", "high"}); err != nil {+ t.Fatal(err)+ }+ }+ for _, g := range []provider.Group{+ {Name: "Quality", Members: []string{"q1/m", "q2/m"}, Routing: provider.Ordered,+ Rules: []provider.Rule{{Use: "q1/m", Tokens: 1, UseEffort: "low"}}},+ {Name: "Coding", Members: []string{"group/quality", "f1/m"}, Routing: provider.Ordered},+ } {+ if err := provider.SaveGroup(g); err != nil {+ t.Fatal(err)+ }+ }+ s := New()+ body := `{"model":"group/coding","reasoning_effort":"high","messages":[{"role":"user","content":"fix the bug"}]}`+ send := func(want string) Route {+ t.Helper()+ k.tried = nil+ code, out := postAs(t, s, "", body)+ if code != 200 || !strings.Contains(out, "from k"+want) {+ t.Fatalf("%d %s, tried %v", code, out, k.tried)+ }+ return lastRoute(s)+ }+ efforts := func(r Route) string {+ var out []string+ for _, tr := range r.Tries {+ out = append(out, tr.ID+"="+tr.Effort+"/"+tr.RuleEffort)+ }+ return strings.Join(out, " ")}- if sent := sentBody(t, a); sent["reasoning_effort"] != "low" {- t.Fatalf("ruled member sent %v", sent)+ k.fail["kq1"] = 500+ if r := send("q2"); efforts(r) != "q1=low/low q2=low/low" {+ t.Fatalf("within the inner group: %s", efforts(r))}- if len(r.Tries) != 2 || r.Tries[0].RuleEffort != "low" || r.Tries[1].RuleEffort != "" {- t.Fatalf("traced %+v", r.Tries)+ k.fail["kq2"] = 500+ s.UnrestAll("q1")+ if r := send("f1"); efforts(r) != "q1=low/low q2=low/low f1=high/" {+ t.Fatalf("outside it: %s", efforts(r))}}diff --git a/internal/gateway/rules.go b/internal/gateway/rules.goindex efe58f524..ad390b87c 100644--- a/internal/gateway/rules.go+++ b/internal/gateway/rules.go@@ -756,28 +756,49 @@ func ruleOver(hit *RuleHit, ms []provider.Member, cs []candidate, pl planned, afreturn cs, pl}-// ruleEffort is the reasoning a rule asks its member for (#2633), when c-// is one of that member's keys or accounts — the outermost group's rule-// that names one, as the turn's pick is the outermost group's — fitted to-// the levels c's model has. A model whose levels aren't known is asked as-// it would be without the rule: magpie can't tell it thinks. "" for none.-// A rule after it that only fails over (Then) asks nothing of its member.-func ruleEffort(c candidate, hit *RuleHit, nested []NestedRule) string {+// ruleEffort is the reasoning a rule asks its request for (#2633), fitted+// to the levels c's model has: that of the outermost group's rule naming+// one of c's members, as the turn's pick is the outermost group's. A+// member the group fails over to is asked it too (#3664): the effort is+// the turn's decision, so a provider that turns the request away doesn't+// leave the member it went to instead at the agent's own level — and a+// rule that names c wins over one it only fails over from. A nested+// group's rule decides for that group's members only, so a member of the+// outer group outside it isn't asked its effort (ms are the outer+// group's, each with the groups it is reached through). A model whose+// levels aren't known is asked as it would be without the rule: magpie+// can't tell it thinks. "" for none.+// A rule after it that only fails over (Then) asks nothing of its own:+// its member is asked the first rule's, as every other member is.+func ruleEffort(c candidate, hit *RuleHit, nested []NestedRule, ms []provider.Member) string {hits := []*RuleHit{hit}for _, n := range nested {hits = append(hits, n.Rule)}- for _, h := range hits {- if h == nil || h.UseEffort == "" || !slices.ContainsFunc(h.members, func(m provider.Member) bool { return ofMember(c, m) }) {+ within := func(group string) bool {+ return slices.ContainsFunc(ms, func(m provider.Member) bool { return ofMember(c, m) && slices.Contains(m.Groups(), group) })+ }+ asked := "" // the effort of the first rule with one, unless one names c+ for i, h := range hits {+ if h == nil || h.UseEffort == "" || i > 0 && !within(nested[i-1].Group) {continue}- levels := c.p.Efforts(c.model)- if len(levels) == 0 {- return ""+ if asked == "" {+ asked = h.UseEffort+ }+ if slices.ContainsFunc(h.members, func(m provider.Member) bool { return ofMember(c, m) }) {+ asked = h.UseEffort+ break}- return fitLevel(h.UseEffort, levels)}- return ""+ if asked == "" {+ return ""+ }+ levels := c.p.Efforts(c.model)+ if len(levels) == 0 {+ return ""+ }+ return fitLevel(asked, levels)}// homeCands sends the keys or accounts of the rule's members as the