Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

gateway: every model in /v1/models says it takes tools, a routing group too

yetone6e080ca0440e
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.md
index 65772061d..97e674bb9 100644
--- a/docs/subsystems/gateway-routing.md
+++ b/docs/subsystems/gateway-routing.md
@@ -285,6 +285,7 @@ recorded as a prediction failure (`noPrediction`).
- A relayed Responses stream gives each output item the one id its `response.output_item.added` gave it, in every event after that carries an `item_id` or an `item`, and in the `output` of `response.completed`, `response.incomplete` or `response.failed` (`idTidy` in [`itemids.go`](../../internal/gateway/itemids.go), applied in `passthrough` before `phaseTidy`). GitHub Copilot's `/responses` gives an item another id in every event (checked 2026-10-19 on a real account, gpt-5.5 with web_search), and Codex starts an item by the added's id and ends one by the done's, so a commentary message was shown twice and a web search never ended (echo_ts on Discord, `TestCopilotItemKeepsItsID`). An upstream whose ids agree goes through byte for byte (`TestItemIDsThatAgreeGoAsTheyCame`); a translated reply's ids are magpie's own.
- A model's reply limit is published within the window beside it (#1438, `provider.OutputWithin`, `Entry.PublishedOutput`): `/v1/models`' `max_output_tokens` (`modelObject`), Cursor Private Inference's `capabilities.max_output_tokens` (`cursorLocalModel`), Muse Code's `limit.output` (`museModels`), the GUI's model and group rows (`providerInfo`, `providersState`), `magpie model output`, and every agent's file (`agent.maxTokens`, #338). The window and the output can come from different sources: Grok's backend lists grok-4.7's 256000 window, and models.dev fills its 500000 output (`catalog.Decorate` and `Provider.replyLimit` fill an output only where the vendor gives none). An unknown window or output is published as it is. The catalog keeps the unclamped `Entry.Output`, and a group's member is still asked for no more than that (`outputLimit`, `withMaxOutput`), so a request isn't lowered by the clamp. A limit the user set stays in their settings as they set it (`TestModelListsKeepOutputWithinTheWindow`). Outside a group, only a Max output the user set for the model (`Entry.OutputSet`) lowers a request (`userOutput`): Claude Code asks a claude-* model for its own full limit (128000 on Opus 5.5) and is told no figure of magpie's for it (`claudeOutput`), and a key whose quota is held per request as max_tokens times the price turned every turn away (YaCo on Discord, `TestUserMaxOutputLowersAPlainProvidersRequest`). A vendor's or models.dev's figure doesn't lower it, since it may be stale.
- `/v1/models` and `/v1/models/{id}` give each model magpie knows a price of its `pricing` in OpenRouter's shape, USD a token as decimal strings: `prompt` and `completion` always (a price set at 0 included), `input_cache_read` and `input_cache_write` only when that part has a price (`catalog.OpenRouterPricing`, `catalog.PerToken` in [`openrouter.go`](../../internal/catalog/openrouter.go), which an agent plugin's `connect` is given too; cleary5779 on Discord, an agent showing what a call costs). The price is the one the Usage page counts the model at (`provider.EntryPricer`, as `EntryPriceIn`: the user's own, else the provider's list price or its maker's at its price rate; a subscription's model at its list price; a group only when every member costs the same), the base price only (no `Tiers`, no 1-hour write). A model no price is known for has no `pricing` key, never a 0. It is told to the callers `magpie_price` is (`local` or `sharedWith`), whether they ask or not. The list reads the settings, groups and providers once (`EntryPricer`), and `catalog.PricedBy` and `AtEndpoint` look a model up in indexes built once a catalog load (`priceIndexOf`, `endpoints`), not by walking every models.dev model a look-up: a 755-model list went from about 38ms to 18ms with pricing on, against about 10ms without, at load 65 (`TestModelsListPricing`, `TestPerToken`). Claude Desktop's, Claude Science's, Muse Code's, Codex's and Ollama's lists, the decision, image and video models carry no `pricing`: their clients read a shape of their own, and an image's price isn't a token's.
+- `/v1/models` and `/v1/models/{id}` say each chat model takes tools, a routing group included (`toolsTold`): models.dev's `tool_call: true`, beside its `reasoning`, and OpenRouter's `supported_parameters` (`tools`, `tool_choice`, and `reasoning` when it thinks), which Kilo's and Cline's lists are read by. A client that judges a model by what its list says found nothing for a group, whose id is the user's own name (psikent on Discord: an SSH client), and gave it no tools. magpie has no model that takes none (a call written as text is read back as one, `textcalls.go`), so every member of a group takes them and the group says so. No `capabilities` key is added: Cursor Private Inference reads it as an object (`cursorLocalModel`), Ollama and LM Studio as arrays of their own words, and no reader of an OpenAI-shaped list agrees on one (`TestGroupListedTakesTools`). A group's window and output stay its largest member's, its images any member's (#712, #756): the gateway sends a longer conversation, a longer reply or an image only to the members that hold it (`roomy`, `withRoom`, `withMaxOutput`), and the editor's Context hint names the member a longer conversation skips when the windows differ (windy on Discord).
- A request that names no reply limit, translated to Anthropic's Messages API, which needs `max_tokens` (`forwardTranslated`), is sent the model's published limit (`askedOutput` in [`maxoutput.go`](../../internal/gateway/maxoutput.go)): the user's Max output, else the vendor's list or models.dev, within the window. `anthropicBody`'s `defaultMaxTokens` (16384) is left only for a model whose limit isn't known. Every Codex turn on a Claude went at 16384 before, and a long thinking and tool turn ran out of it: `stop_reason: max_tokens` reaches a Responses client as `incomplete_details.reason: "max_output_tokens"` (#1792, tensornull). A limit the client names goes as it is. The thinking budget fits under the filled limit as under one the client named (`TestAnthropicMaxTokensIsTheModels`).
- A restart to update on a Mac or Linux hands the listening socket to the
new version ([GUI shell](gui-shell.md#a-restart-to-update-hands-the-gateway-over),
diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.go
index 229cbf02e..c9e7fcb23 100644
--- a/internal/gateway/gateway.go
+++ b/internal/gateway/gateway.go
@@ -1028,10 +1028,32 @@ func modelObject(e provider.Entry) map[string]any {
return m
}
+// toolsTold says, in an OpenAI-compatible list, that a chat model takes
+// tools, by the names clients read it under: models.dev's tool_call
+// (beside its reasoning, which modelObject gives) and OpenRouter's
+// supported_parameters, whose "tools" Cline, Kilo and kin read before they
+// offer a model their tools. Without either, a client that matches what a
+// model can do by its id (psikent on Discord: an SSH client) found none
+// for a routing group, whose id is the user's own name, and wouldn't hand
+// it a tool. Every chat model magpie lists takes tools — the gateway reads
+// one written as text back as a call (textcalls.go) — so a group, all of
+// whose members do, says so as well. The decision models and Claude
+// Desktop's and Claude Science's lists, shaped for their own readers, are
+// left as they were.
+func toolsTold(m map[string]any, e provider.Entry) {
+ m["tool_call"] = true
+ params := []string{"tools", "tool_choice"}
+ if e.Thinks() {
+ params = append(params, "reasoning")
+ }
+ m["supported_parameters"] = params
+}
+
// pricedModelObject is modelObject with the model's pricing, as the list
// gives it, for the callers the list tells it to.
func pricedModelObject(r *http.Request, e provider.Entry) map[string]any {
m := modelObject(e)
+ toolsTold(m, e)
if local(r) || sharedWith(r) {
if pr, ok := provider.EntryPriceIn(settings.Load(), provider.GroupFinder(), e); ok {
m["pricing"] = catalog.OpenRouterPricing(pr)
@@ -1196,6 +1218,7 @@ func (s *Server) models(w http.ResponseWriter, r *http.Request) {
}
for i, e := range rows {
m := modelObject(e)
+ toolsTold(m, e)
if id, ok := ids[i]; ok {
m["id"] = id
}
diff --git a/internal/gateway/group_tools_listed_test.go b/internal/gateway/group_tools_listed_test.go
new file mode 100644
index 000000000..4bc493d6d
--- /dev/null
+++ b/internal/gateway/group_tools_listed_test.go
@@ -0,0 +1,69 @@
+package gateway
+
+import (
+ "encoding/json"
+ "net/http/httptest"
+ "slices"
+ "testing"
+
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// psikent on Discord: an SSH client matches what a model can do by what
+// its list says, and a routing group, whose id is the user's own name,
+// said nothing of tools, so the client wouldn't hand it one. Every chat
+// row of /v1/models — a provider's model and a group alike — says it
+// takes tools, as models.dev (tool_call) and OpenRouter
+// (supported_parameters) say it, and so does the one model asked by id.
+// A group of a 1M and a 128K member is told its largest window, as
+// agents have been since #712: a longer conversation skips the 128K one
+// (withRoom), and the group's Context setting tells its smallest instead
+// (windy on Discord).
+func TestGroupListedTakesTools(t *testing.T) {
+ fresh(t)
+ for id, n := range map[string]int{"a": 1000000, "b": 128000} {
+ if err := provider.Save(provider.Provider{ID: id, Name: id, Key: "k", Chat: "http://127.0.0.1:1/v1", Models: []string{"m"}, Contexts: map[string]int{"m": n}}); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if err := provider.SaveGroup(provider.Group{Name: "G", Members: []string{"a/m", "b/m"}, Routing: provider.Ordered}); err != nil {
+ t.Fatal(err)
+ }
+ type row struct {
+ ID string `json:"id"`
+ ToolCall bool `json:"tool_call"`
+ Params []string `json:"supported_parameters"`
+ Window int `json:"context_window"`
+ }
+ get := func(path string, into any) {
+ t.Helper()
+ rec := httptest.NewRecorder()
+ New().Handler().ServeHTTP(rec, httptest.NewRequest("GET", path, nil))
+ if err := json.Unmarshal(rec.Body.Bytes(), into); err != nil {
+ t.Fatalf("%s: %d %s", path, rec.Code, rec.Body.String())
+ }
+ }
+ var list struct{ Data []row }
+ get("/v1/models", &list)
+ seen := map[string]row{}
+ for _, r := range list.Data {
+ seen[r.ID] = r
+ }
+ for _, id := range []string{"a/m", "b/m", "group/g"} {
+ r, ok := seen[id]
+ if !ok {
+ t.Fatalf("%s not listed: %+v", id, list.Data)
+ }
+ if !r.ToolCall || !slices.Contains(r.Params, "tools") || !slices.Contains(r.Params, "tool_choice") {
+ t.Errorf("%s doesn't say it takes tools: %+v", id, r)
+ }
+ }
+ if w := seen["group/g"].Window; w != 1000000 {
+ t.Errorf("group window %d, want its largest member's 1000000", w)
+ }
+ var one row
+ get("/v1/models/group/g", &one)
+ if !one.ToolCall || !slices.Contains(one.Params, "tools") {
+ t.Errorf("the group asked by id doesn't say it takes tools: %+v", one)
+ }
+}
diff --git a/internal/gui/assets/plugins/routing/client.js b/internal/gui/assets/plugins/routing/client.js
index 866f6ec5e..22353cc82 100644
--- a/internal/gui/assets/plugins/routing/client.js
+++ b/internal/gui/assets/plugins/routing/client.js
@@ -6058,7 +6058,8 @@ function routingPage(part) {
ctxIn.setAttribute("aria-label", t("Context"));
ctxHint.id = "groupContextHint";
ctxIn.setAttribute("aria-describedby", ctxHint.id);
- const ctxs = () => d.members.map((id) => groups.models.find((x) => x.id === id)?.context || infoOf(id)?.context || 0).filter((n) => n > 0);
+ const ctxOf = (id) => groups.models.find((x) => x.id === id)?.context || infoOf(id)?.context || 0;
+ const ctxs = () => d.members.map(ctxOf).filter((n) => n > 0);
const ctxMode = () => d.context === -1 ? "smallest" : d.context > 0 ? "custom" : "largest";
let ctxErr = "";
const drawCtx = () => {
@@ -6070,6 +6071,10 @@ function routingPage(part) {
? t("Agents are told the group takes {n} tokens; a longer conversation still goes on to a member with room for it.", { n: (d.context || 0).toLocaleString(intlLang()) })
: mode === "smallest"
? (small ? t("Agents are told {n} tokens, its smallest model's, so they compact before any member would turn the conversation away.", { n: small.toLocaleString(intlLang()) }) : t("Agents are told its smallest model's window, so they compact before any member would turn the conversation away."))
+ // members whose windows differ (windy on Discord: a 1M group
+ // with a 128K member) are named with the length that skips them,
+ // by provider too, as one model's members differ by it alone
+ : small && small < big ? t("Agents are told {n} tokens, its largest model's; a conversation longer than {m} skips {who} for a member with room for it.", { n: big.toLocaleString(intlLang()), m: small.toLocaleString(intlLang()), who: d.members.filter((id) => ctxOf(id) === small).map((id) => memberLabel(g || {}, id)).join(", ") })
: (big ? t("Agents are told {n} tokens, its largest model's; a conversation too long for one member goes on to one with room for it.", { n: big.toLocaleString(intlLang()) }) : t("Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.")));
};
const CTX_OPTS = [["largest", "Largest model's"], ["smallest", "Smallest model's"], ["custom", "Custom"]];
diff --git a/internal/gui/assets/plugins/routing/i18n.js b/internal/gui/assets/plugins/routing/i18n.js
index 4718f63e9..62b620ae9 100644
--- a/internal/gui/assets/plugins/routing/i18n.js
+++ b/internal/gui/assets/plugins/routing/i18n.js
@@ -171,6 +171,7 @@ magpie.strings({
"Agents are told {n} tokens, its smallest model's, so they compact before any member would turn the conversation away.": "告诉 agent {n} tokens(组里最小模型的),这样 agent 会在任何成员拒收之前先压缩对话。",
"Agents are told its smallest model's window, so they compact before any member would turn the conversation away.": "告诉 agent 组里最小模型的上下文,这样 agent 会在任何成员拒收之前先压缩对话。",
"Agents are told {n} tokens, its largest model's; a conversation too long for one member goes on to one with room for it.": "告诉 agent {n} tokens(组里最大模型的);某个成员放不下的对话会交给放得下的成员。",
+ "Agents are told {n} tokens, its largest model's; a conversation longer than {m} skips {who} for a member with room for it.": "告诉 agent {n} tokens(组里最大模型的);超过 {m} 的对话会跳过 {who},交给放得下的成员。",
"Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.": "告诉 agent 组里最大模型的上下文;某个成员放不下的对话会交给放得下的成员。",
"{v} is not a length of tokens like 200k or 1m": "{v} 不是有效的 token 长度,例如 200k 或 1m",
"Every member is waited for, however long it takes to begin answering.": "每个成员都会一直等到它开始回答,不论多久。",
@@ -752,6 +753,7 @@ magpie.strings({
"Agents are told {n} tokens, its smallest model's, so they compact before any member would turn the conversation away.": "告訴 agent {n} tokens(組裡最小模型的),這樣 agent 會在任何成員拒收之前先壓縮對話。",
"Agents are told its smallest model's window, so they compact before any member would turn the conversation away.": "告訴 agent 組裡最小模型的上下文,這樣 agent 會在任何成員拒收之前先壓縮對話。",
"Agents are told {n} tokens, its largest model's; a conversation too long for one member goes on to one with room for it.": "告訴 agent {n} tokens(組裡最大模型的);某個成員放不下的對話會交給放得下的成員。",
+ "Agents are told {n} tokens, its largest model's; a conversation longer than {m} skips {who} for a member with room for it.": "告訴 agent {n} tokens(組裡最大模型的);超過 {m} 的對話會跳過 {who},交給放得下的成員。",
"Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.": "告訴 agent 組裡最大模型的上下文;某個成員放不下的對話會交給放得下的成員。",
"{v} is not a length of tokens like 200k or 1m": "{v} 不是有效的 token 長度,例如 200k 或 1m",
"Every member is waited for, however long it takes to begin answering.": "每個成員都會一直等到它開始回答,不論多久。",
@@ -1311,6 +1313,7 @@ magpie.strings({
"Agents are told {n} tokens, its smallest model's, so they compact before any member would turn the conversation away.": "エージェントには最小モデルの {n} トークンを伝えるので、どのメンバーにも断られる前に会話を圧縮します。",
"Agents are told its smallest model's window, so they compact before any member would turn the conversation away.": "エージェントには最小モデルのコンテキストを伝えるので、どのメンバーにも断られる前に会話を圧縮します。",
"Agents are told {n} tokens, its largest model's; a conversation too long for one member goes on to one with room for it.": "エージェントには最大モデルの {n} トークンを伝えます。あるメンバーに長すぎる会話は収まるメンバーに回されます。",
+ "Agents are told {n} tokens, its largest model's; a conversation longer than {m} skips {who} for a member with room for it.": "エージェントには最大モデルの {n} トークンを伝えます。{m} を超える会話は {who} を飛ばし、収まるメンバーに回されます。",
"Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.": "エージェントには最大モデルのコンテキストを伝えます。あるメンバーに長すぎる会話は収まるメンバーに回されます。",
"{v} is not a length of tokens like 200k or 1m": "{v} は 200k や 1m のようなトークン数ではありません",
"Every member is waited for, however long it takes to begin answering.": "どのメンバーも、応答を始めるまでどれだけかかっても待ちます。",
@@ -1882,6 +1885,7 @@ magpie.strings({
"Agents are told {n} tokens, its smallest model's, so they compact before any member would turn the conversation away.": "Agenten wird {n} Tokens gesagt, das kleinste Modell, sodass sie verdichten, bevor ein Mitglied das Gespräch ablehnt.",
"Agents are told its smallest model's window, so they compact before any member would turn the conversation away.": "Agenten wird das Fenster des kleinsten Modells gesagt, sodass sie verdichten, bevor ein Mitglied das Gespräch ablehnt.",
"Agents are told {n} tokens, its largest model's; a conversation too long for one member goes on to one with room for it.": "Agenten wird {n} Tokens gesagt, das größte Modell; ein für ein Mitglied zu langes Gespräch geht an eines mit Platz dafür.",
+ "Agents are told {n} tokens, its largest model's; a conversation longer than {m} skips {who} for a member with room for it.": "Agenten wird {n} Tokens gesagt, das größte Modell; ein Gespräch über {m} überspringt {who} und geht an ein Mitglied mit Platz dafür.",
"Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.": "Agenten wird das Fenster des größten Modells gesagt; ein für ein Mitglied zu langes Gespräch geht an eines mit Platz dafür.",
"{v} is not a length of tokens like 200k or 1m": "{v} ist keine Tokenlänge wie 200k oder 1m",
"Every member is waited for, however long it takes to begin answering.": "Auf jedes Mitglied wird gewartet, egal wie lange es bis zum Antwortbeginn braucht.",
diff --git a/internal/gui/tests/group-context.test.cjs b/internal/gui/tests/group-context.test.cjs
index 7ea72e2ed..7d0f1e761 100644
--- a/internal/gui/tests/group-context.test.cjs
+++ b/internal/gui/tests/group-context.test.cjs
@@ -3,7 +3,9 @@
// Discord: a custom length, or the shortest or longest member's). The
// group editor's "Context" row picks Largest model's (0, as before),
// Smallest model's (-1, followed as the members change) or Custom, with a
-// box that takes 300k or 1m; its hint says the length agents are told.
+// box that takes 300k or 1m; its hint says the length agents are told,
+// and, when the members' windows differ, which member a longer
+// conversation skips (windy on Discord).
// In English and Chinese, Chromium and WebKit.
const assert = require("node:assert/strict");
const fs = require("node:fs/promises");
@@ -95,6 +97,10 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
assert.equal(await row.locator(".opt.on").textContent(), w.largest);
assert(await row.locator("input").isHidden(), "no box for the largest");
assert((await hint()).includes(w.n1m), await hint());
+ // the members' windows differ (windy on Discord: a 1M group with a
+ // 128K member): the hint names the smallest and the length past
+ // which a conversation skips it
+ assert((await hint()).includes(w.n128) && (await hint()).includes("DeepSeek · deepseek-flash"), await hint());
await row.locator(".opt", { hasText: w.smallest }).click();
assert((await hint()).includes(w.n128), await hint());
assert.equal((await save()).context, -1);
@@ -127,6 +133,7 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
"Agents are told its smallest model's window, so they compact before any member would turn the conversation away.",
"Agents are told {n} tokens, its largest model's; a conversation too long for one member goes on to one with room for it.",
"Agents are told its largest model's window; a conversation too long for one member goes on to one with room for it.",
+ "Agents are told {n} tokens, its largest model's; a conversation longer than {m} skips {who} for a member with room for it.",
"{v} is not a length of tokens like 200k or 1m",
].filter((k) => !I18N.zh[k]));
assert.deepEqual(missing, [], "every string has its Chinese");