Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.
gateway: a history call to a tool the request doesn't declare gets a stub where it is refused (pi, T3 Code tool_search)
yetone31275d0734fe
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.mdindex 9f9f35226..e4590cc4b 100644--- a/docs/subsystems/gateway-routing.md+++ b/docs/subsystems/gateway-routing.md@@ -190,6 +190,7 @@ recorded as a prediction failure (`noPrediction`).- Kiro is told where to cache the prompt (plugins#110: a Kiro subscription read 0 from its cache). Its API takes Bedrock's `cachePoint: {"type":"default"}` on a `userInputMessage`, an `assistantResponseMessage` or as a tool entry. The built-in (`buildKiro` in [`kiro.go`](../../internal/gateway/kiro.go)) marks the first message when there is a system prompt, which it folds into that message, and the current message, as `buildAnthropic` marks Anthropic's. A moved Kiro's plugin (opencode-kiro-auth 0.1.10, the mover's `min`) turns each `cache_control` in the Anthropic body it is handed into one, at most 4, the last kept.- A model that refuses a conversation ending with the assistant's message (`refusesPrefill`: OpenCode Go's Claude models over Messages, "This model does not support assistant message prefill"; Volcengine's Agent Plan, "…for a model that does not support prefill"; Gemini's, "Requests ending with a model turn are not supported") is marked for that provider, model and API (`prefillRefused`, #1447); the provider's other models keep their prefill ([`prefill.go`](../../internal/gateway/prefill.go)). A Chat or Responses client's conversation ending with the assistant's message, which those APIs read as a turn already said (`Request.LastIsTurn`), is asked again with a user turn after it (`continueText`), translated (`forwardTranslated`) or relayed (`passthrough`, `userLastBody`), and so from then on. An Anthropic client's prefill is its own ask and gets the refusal as sent. magpie's own continuation of a cut reply (`continuation.go`) isn't asked of that model again; the try the model refused ends the reply with what cut it, not with the refusal. A request magpie builds in Gemini's own shape (`buildCodeAssistSent`: Vertex AI, Gemini CLI's and Antigravity's Code Assist, Factory) never ends with the model's turn, which all of them turn away: a user turn goes after it, `continueText`, or `compactLastText` when an agent is compacting with its ask in the system prompt (`modelLastAsk`), as one goes before a history that opens with the model's. Nor does it send a user turn that mixes a tool's `functionResponse` with text, which Vertex AI answers with the same "ending with a model turn" (#1533: Codex's compaction prompt after a tool's output): what the user said goes in a user turn of its own after the tools' responses (`hasResponse`), except for a Claude model on Antigravity, whose tool_result turn holds both.- A tool's `strict` that an upstream turns away (`toolStrictRefusal` in [`strict_refused.go`](../../internal/gateway/strict_refused.go): "Unsupported Chat Completions API field: tools[0].function.strict", OpenAI-style unknown_parameter, "Extra inputs are not permitted" and pydantic's extra_forbidden on it) is asked once more without it, in Chat's `tools[].function` and Responses' `tools[]` alike, and marked for that provider and API (`toolStrictRefused`): its next requests go without it at once. OpenCode 2 sends `"strict": false` on every tool through `@ai-sdk/openai-compatible`, and a custom provider that refused it ended the session (#1980). An upstream that hasn't refused it still gets it, as the ChatGPT backend needs it (#383). `TestToolStrictLeftOutWhereRefused` (real OpenCode 2.0.25 requests), `TestToolStrictRefusalWordings`.+- A history call to a tool the request doesn't declare, which an agent that loads its tools on demand sends (pi's and T3 Code's `tool_search`: the tools a search found are called in the history but not declared in the next request's `tools`, and pi sends `"tools": []` with no tools left; Gearclickist on Discord), goes as sent where it is taken: OpenAI's Chat and Responses, the ChatGPT backend and Gemini take it, and Anthropic takes a call to a tool it isn't given ([`undeclared_tools.go`](../../internal/gateway/undeclared_tools.go)). Anthropic refuses calls with no tools at all ("Requests which include `tool_use` or `tool_result` blocks must define tools"), so `anthropicBody` declares a stub for each tool the history called when the request has none (description "Called earlier in this conversation; not available now.", any object as its input) with `tool_choice` `none`, as the Kiro plugin declares its history's tools for Kiro. An upstream that turns the request away naming such a tool as not found, unknown or not defined, or that says tools must be defined beside the calls (`refusesHistoryTools`: a relay in front of Kiro or Bedrock, a vendor checking each name), is asked once more with a stub declared for each tool the history called and `tools` lacks (`withHistoryTools`, Chat, Responses and Messages bodies, relayed or translated), `tool_choice` `none` only where the client offered no tools, and marked for that provider and API (`historyToolsRefused`): its next requests carry the stubs at once. A Responses call under a namespace is the namespace's, and a request whose calls are all declared goes as it was. `TestUndeclaredHistoryToolRefusedThenStubbed`, `TestToollessHistoryToAnthropicDeclaresStubs`, `TestToollessHistoryRefusedOnChat`, `TestUndeclaredHistoryToolTakenAsSent`, `TestRefusesHistoryToolsMatch`.- A Codex custom tool (exec, apply_patch) that a Responses upstream turns away (`customToolRefusal` in [`custom_tool.go`](../../internal/gateway/custom_tool.go): DeepSeek's 400 "Unsupported custom tool: 'exec'. Only 'apply_patch' is supported.", #2656) ends the relay: `passthrough` marks the model (`customToolRefused`) and returns not done, and `attempt` translates the request on the same API (`ownAPI`), where each custom tool is offered as a function taking its `input` and a call comes back as the `custom_tool_call` Codex runs, its name kept. A marked model's next requests with a custom tool are translated at once; one without stays relayed, and an upstream that never refused them still gets them as sent. A compaction (`keepsCompaction`) isn't translated, and gets the refusal. Where the refusal reaches a group untranslated it is a shape refusal (`shapeRefused`): the next member is asked. `TestCustomToolTranslatedWhereRefused`, `TestCustomToolRefusalInGroup`, `TestCustomToolRefusalWordings`.- A tool call's arguments reach the client as the vendor writes them, translated too (echo_ts on Discord: Codex on Copilot about 10x slower, an apply_patch shown only once whole). The Responses decoder hands each `function_call_arguments.delta` and `custom_tool_call_input.delta` on as a `KToolArgs` delta (`argsDelta`), holding back only the one that would make the JSON whole until `.done`, which `endCall` checks against the final arguments; the Responses encoder streams a custom tool's input as `custom_tool_call_input.delta` (`customStream` in [`custom_tool.go`](../../internal/gateway/custom_tool.go)) before its `.done`. A Copilot account's Responses request is relayed with a `web_search` tool in it (`searchesItself`; checked on a real account 2026-10-17), and Copilot is never picked to search for another provider's model (`searchRank`). It is translated instead when a tool marks a parameter `"encrypted": true` (`asksSealing`, MultiAgentV2's `spawn_agent`): Copilot seals such an argument and can't read its own seal on the subagent's turn. `TestCallArgsStreamWhenTranslated`, `TestCustomInputStreams`, `TestResponsesCallArgsStream`, `TestCopilotRelayedUnlessAskedToSeal`, `TestCopilotUltraSubagentWithWebSearch`.- A relayed Responses stream gives each output item the one id its `response.output_item.added` gave it, in every event after that carries an `item_id` or an `item`, and in the `output` of `response.completed`, `response.incomplete` or `response.failed` (`idTidy` in [`itemids.go`](../../internal/gateway/itemids.go), applied in `passthrough` before `phaseTidy`). GitHub Copilot's `/responses` gives an item another id in every event (checked 2026-10-19 on a real account, gpt-5.5 with web_search), and Codex starts an item by the added's id and ends one by the done's, so a commentary message was shown twice and a web search never ended (echo_ts on Discord, `TestCopilotItemKeepsItsID`). An upstream whose ids agree goes through byte for byte (`TestItemIDsThatAgreeGoAsTheyCame`); a translated reply's ids are magpie's own.diff --git a/internal/gateway/anthropic.go b/internal/gateway/anthropic.goindex 0bd150611..db7ed5c9b 100644--- a/internal/gateway/anthropic.go+++ b/internal/gateway/anthropic.go@@ -997,9 +997,20 @@ func anthropicBody(r *Request, model, host string, replay bool) []byte {// auto mode's review, which Claude alone does (automode.go)out["safeguards"] = r.Safeguards}- if len(r.Tools) > 0 || r.WebSearch {+ // a history of calls with no tools to go with it, which Anthropic+ // turns away ("must define tools", undeclared_tools.go): each tool+ // it called is declared, and none of them may be called+ rTools := r.Tools+ stubbed := false+ if len(rTools) == 0 && !r.WebSearch {+ for _, n := range historyCallNames(r) {+ rTools = append(rTools, Tool{Name: n, Description: stubToolDescription})+ stubbed = true+ }+ }+ if len(rTools) > 0 || r.WebSearch {var tools []map[string]any- for _, t := range r.Tools {+ for _, t := range rTools {schema := t.Schemaif len(schema) == 0 {schema = json.RawMessage(`{"type":"object","properties":{}}`)@@ -1017,14 +1028,14 @@ func anthropicBody(r *Request, model, host string, replay bool) []byte {out["tools"] = toolsvar tc map[string]anyswitch {+ case stubbed || r.ToolChoice == "none":+ tc = map[string]any{"type": "none"}case r.ToolChoice == "auto" || r.ToolChoice == "":if r.Parallel != nil && !*r.Parallel {tc = map[string]any{"type": "auto"}}case r.ToolChoice == "required":tc = map[string]any{"type": "any"}- case r.ToolChoice == "none":- tc = map[string]any{"type": "none"}case strings.HasPrefix(r.ToolChoice, "name:"):tc = map[string]any{"type": "tool", "name": strings.TrimPrefix(r.ToolChoice, "name:")}}diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.goindex 43826a6e6..3e7b8c4ed 100644--- a/internal/gateway/gateway.go+++ b/internal/gateway/gateway.go@@ -4419,6 +4419,11 @@ func (s *Server) passthrough(w http.ResponseWriter, r *http.Request, p provider.if !s.fits(p.ID, toolStrictRefused, proto) {body, _ = withoutToolStrict(proto, body)}+ // a stub for each tool the history called and tools lacks, to an+ // upstream that turned such a call away before (undeclared_tools.go)+ if !s.fits(p.ID, historyToolsRefused, proto) {+ body, _, _ = withHistoryTools(proto, body)+ }// structured output an upstream turned away before (#2126): a schema// as any JSON object, or the format told in the system prompt aloneif !s.fits(p.ID, formatRefused(model), proto) {@@ -4469,6 +4474,24 @@ func (s *Server) passthrough(w http.ResponseWriter, r *http.Request, p provider.s.markUnfit(p.ID, mark, proto)}}+ // a history call to a tool the request doesn't declare, or calls with+ // no tools, turned away (pi's and T3 Code's tool_search,+ // undeclared_tools.go): asked once more with a stub declared for each+ // such tool, and so from then on+ if badRequest(res.StatusCode) && s.fits(p.ID, historyToolsRefused, proto) {+ b, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))+ res.Body.Close()+ res.Body = io.NopCloser(bytes.NewReader(b))+ if nb, names, toolless := withHistoryTools(proto, body); len(names) > 0 && refusesHistoryTools(b, names, toolless) {+ body = nb+ if res, err = s.forward(r.Context(), p, proto, path, p.Prepare(body), r.Header); err != nil {+ return writeError(w, proto, 502, p.Name+": "+err.Error()), err.Error(), true+ }+ if res.StatusCode < 400 {+ s.markUnfit(p.ID, historyToolsRefused, proto)+ }+ }+ }// a vendor that turns away the fields it doesn't know (Mistral's// extra_forbidden for OpenCode's store, #393) is asked again without// the optional ones it named, and not sent them again once that works@@ -5258,6 +5281,12 @@ func (s *Server) forwardTranslated(ctx context.Context, p provider.Provider, to// the other magpie searches for it (search_remote.go)body = withFields(body, map[string]any{"web_search_options": map[string]any{}})}+ if !s.fits(p.ID, historyToolsRefused, to) {+ // a stub for each tool the history called and tools lacks,+ // to an upstream that turned such a call away before+ // (undeclared_tools.go)+ body, _, _ = withHistoryTools(to, body)+ }if to == provider.CodeAssist && p.Account != nil {// the envelope is named in there, where the id it carries is// known: on Antigravity that is the variant the effort picked,@@ -5354,6 +5383,16 @@ func (s *Server) forwardTranslated(ctx context.Context, p provider.Provider, tos.markUnfit(p.ID, thinkingConfigField, to)continue}+ if badRequest(res.StatusCode) && s.fits(p.ID, historyToolsRefused, to) {+ if _, names, toolless := withHistoryTools(to, body); len(names) > 0 && refusesHistoryTools(b, names, toolless) {+ // a history call to a tool the request doesn't declare,+ // or calls with no tools, turned away (pi's and T3 Code's+ // tool_search): asked again with a stub declared for each+ // such tool, and so from then on+ s.markUnfit(p.ID, historyToolsRefused, to)+ continue+ }+ }if badRequest(res.StatusCode) && rootUnionRefusal.Match(b) && s.fits(p.ID, rootUnionRefused(model), to) {if _, changed := objectRootsReq(req); changed {// a tool's parameters refused for a union at their rootdiff --git a/internal/gateway/undeclared_tools.go b/internal/gateway/undeclared_tools.gonew file mode 100644index 000000000..a3069e799--- /dev/null+++ b/internal/gateway/undeclared_tools.go@@ -0,0 +1,225 @@+package gateway++import (+ "bytes"+ "encoding/json"+ "regexp"++ "github.com/yetone/magpie/internal/provider"+)++// An agent that loads its tools when it needs them (pi's and T3 Code's+// tool_search, Gearclickist on Discord) declares a few tools and+// tool_search in each request, while its history holds calls to tools a+// search found earlier and the request no longer declares; one that has+// no tools to offer sends "tools": [] beside a history of calls. OpenAI's+// Chat and Responses APIs and Gemini take such a history. Anthropic's+// Messages API takes a call to a tool it isn't given, but not a call with+// no tools at all ("Requests which include `tool_use` or `tool_result`+// blocks must define tools"), so anthropicBody declares a stub for each+// tool the history called when the request has none, with tool_choice+// none: the client offered nothing to call. A stricter upstream (a relay+// in front of Kiro or Bedrock, a vendor that checks each call's name+// against tools) turns the request away naming the tool; it is asked+// once more with a stub declared for each tool the history called and+// tools lacks, and so from then on. A request whose calls are all+// declared goes as it was.++// historyToolsRefused is how unfit remembers a provider refusing a+// history call to a tool its request didn't declare: the provider's+// validator, not a model, turns it away.+const historyToolsRefused = "tools[].history"++// stubToolDescription is a stub's description: the call it stands for+// was made earlier and the tool isn't offered now.+const stubToolDescription = "Called earlier in this conversation; not available now."++// anthropicToolName is a name Anthropic takes for a tool.+var anthropicToolName = regexp.MustCompile(`^[a-zA-Z0-9_-]{1,128}$`)++// toolsMissing is an upstream refusing tool calls in a request with no+// tools: Anthropic's "must define tools", Bedrock's "The toolConfig field+// must be defined when using toolUse and toolResult content blocks", or+// tools said required beside the calls.+var toolsMissing = regexp.MustCompile("(?i)must define tools|toolconfig field must be defined|tools?\\W{1,4}(?:param(?:eter)?|field)?\\W{0,4}(?:is |are )?(?:required|must be (?:defined|provided|specified))[^\\n]{0,80}(?:tool_calls?|tool_use|tool_result|toolUse|toolResult|function_call)")++// toolNotDeclared is what follows or leads a tool's name in a refusal of+// a call to it: not found, not defined, unknown.+const toolNotDeclared = `(?:not (?:found|defined|declared|provided|offered|available|present|exist|in (?:the )?(?:tools?|request|list))|unknown|undefined|undeclared|does ?n[o']t exist|no such|missing|invalid)`++// refusesHistoryTools reports whether b, a 4xx body, refuses a history+// call to one of names, the tools the history called that the request+// doesn't declare, or calls in a request with no tools (toolless).+func refusesHistoryTools(b []byte, names []string, toolless bool) bool {+ if toolless && toolsMissing.Match(b) {+ return true+ }+ for _, n := range names {+ q := regexp.QuoteMeta(n)+ name := `(?:^|[^A-Za-z0-9_.-])` + q + `(?:$|[^A-Za-z0-9_-])`+ re, err := regexp.Compile(`(?i)` + toolNotDeclared + `[^\n{}]{0,80}?` + name + `|` + name + `[^\n{}]{0,80}?` + toolNotDeclared)+ if err == nil && re.Match(b) {+ return true+ }+ }+ return false+}++// withHistoryTools is a Chat, Responses or Messages body with a stub+// declared for each tool its history calls and its tools don't, the+// names it declared, and whether tools was empty. tool_choice goes none+// where the client offered no tools, so the model calls none of them.+func withHistoryTools(proto provider.Protocol, body []byte) ([]byte, []string, bool) {+ if proto != provider.Chat && proto != provider.Responses && proto != provider.Anthropic {+ return body, nil, false+ }+ dec := json.NewDecoder(bytes.NewReader(body))+ dec.UseNumber()+ var m map[string]any+ if dec.Decode(&m) != nil || m == nil {+ return body, nil, false+ }+ tools, _ := m["tools"].([]any)+ declared := map[string]bool{}+ declaredNames(proto, tools, declared)+ var add []string+ for _, n := range calledNames(proto, m) {+ if !declared[n] && (proto != provider.Anthropic || anthropicToolName.MatchString(n)) {+ declared[n] = true+ add = append(add, n)+ }+ }+ if len(add) == 0 {+ return body, nil, false+ }+ toolless := len(tools) == 0+ for _, n := range add {+ tools = append(tools, stubTool(proto, n))+ }+ m["tools"] = tools+ if toolless {+ if _, set := m["tool_choice"]; !set {+ if proto == provider.Anthropic {+ m["tool_choice"] = map[string]any{"type": "none"}+ } else {+ m["tool_choice"] = "none"+ }+ }+ }+ out, err := marshalPlain(m)+ if err != nil {+ return body, nil, false+ }+ return out, add, toolless+}++// declaredNames adds the names tools declares to into: a function's, a+// custom tool's, and on Responses a namespace's own tools.+func declaredNames(proto provider.Protocol, tools []any, into map[string]bool) {+ for _, t := range tools {+ tm, _ := t.(map[string]any)+ if tm == nil {+ continue+ }+ if proto == provider.Responses && tm["type"] == "namespace" {+ nested, _ := tm["tools"].([]any)+ declaredNames(proto, nested, into)+ continue+ }+ if n, _ := tm["name"].(string); n != "" {+ into[n] = true+ }+ for _, k := range []string{"function", "custom"} {+ if fn, _ := tm[k].(map[string]any); fn != nil {+ if n, _ := fn["name"].(string); n != "" {+ into[n] = true+ }+ }+ }+ }+}++// calledNames is the names of the functions m's history calls, in order,+// each once. A Responses call under a namespace is the namespace's, which+// declares its own tools, and is left to it.+func calledNames(proto provider.Protocol, m map[string]any) []string {+ var names []string+ seen := map[string]bool{}+ add := func(n string) {+ if n != "" && !seen[n] {+ seen[n] = true+ names = append(names, n)+ }+ }+ switch proto {+ case provider.Chat:+ msgs, _ := m["messages"].([]any)+ for _, x := range msgs {+ msg, _ := x.(map[string]any)+ calls, _ := msg["tool_calls"].([]any)+ for _, c := range calls {+ cm, _ := c.(map[string]any)+ if fn, _ := cm["function"].(map[string]any); fn != nil {+ n, _ := fn["name"].(string)+ add(n)+ }+ }+ }+ case provider.Responses:+ items, _ := m["input"].([]any)+ for _, x := range items {+ it, _ := x.(map[string]any)+ if it == nil || it["type"] != "function_call" {+ continue+ }+ if ns, _ := it["namespace"].(string); ns != "" {+ continue+ }+ n, _ := it["name"].(string)+ add(n)+ }+ case provider.Anthropic:+ msgs, _ := m["messages"].([]any)+ for _, x := range msgs {+ msg, _ := x.(map[string]any)+ blocks, _ := msg["content"].([]any)+ for _, c := range blocks {+ cb, _ := c.(map[string]any)+ if cb != nil && cb["type"] == "tool_use" {+ n, _ := cb["name"].(string)+ add(n)+ }+ }+ }+ }+ return names+}++// stubTool is a function tool named n that takes any object, as proto+// declares one.+func stubTool(proto provider.Protocol, n string) map[string]any {+ params := map[string]any{"type": "object", "properties": map[string]any{}}+ switch proto {+ case provider.Chat:+ return map[string]any{"type": "function", "function": map[string]any{"name": n, "description": stubToolDescription, "parameters": params}}+ case provider.Responses:+ return map[string]any{"type": "function", "name": n, "description": stubToolDescription, "parameters": params}+ }+ return map[string]any{"name": n, "description": stubToolDescription, "input_schema": params}+}++// historyCallNames is the names of the tools r's history calls that+// Anthropic takes as a tool's name, in order, each once.+func historyCallNames(r *Request) []string {+ var names []string+ seen := map[string]bool{}+ for _, m := range r.Messages {+ for _, p := range m.Parts {+ if p.Kind == ToolCall && !seen[p.Name] && anthropicToolName.MatchString(p.Name) {+ seen[p.Name] = true+ names = append(names, p.Name)+ }+ }+ }+ return names+}diff --git a/internal/gateway/undeclared_tools_test.go b/internal/gateway/undeclared_tools_test.gonew file mode 100644index 000000000..3ee668cd1--- /dev/null+++ b/internal/gateway/undeclared_tools_test.go@@ -0,0 +1,222 @@+package gateway++import (+ "net/http"+ "net/http/httptest"+ "strings"+ "testing"++ "github.com/tidwall/gjson"+ "github.com/yetone/magpie/internal/provider"+)++// piDeferred is pi's request (openai-completions) after its tool_search+// loaded a tool: the history calls mcp__github__list_issues, and tools+// holds only read and tool_search (Gearclickist on Discord).+const piDeferred = `{"model":"fake/m1","stream":false,"messages":[+{"role":"system","content":"You are pi."},+{"role":"user","content":"list my open issues"},+{"role":"assistant","content":null,"tool_calls":[{"id":"call_1","type":"function","function":{"name":"tool_search","arguments":"{\"query\":\"github issues\"}"}}]},+{"role":"tool","tool_call_id":"call_1","content":"Loaded 1 tool. They are available from your next call:\n- mcp__github__list_issues: List issues"},+{"role":"assistant","content":null,"tool_calls":[{"id":"call_2","type":"function","function":{"name":"mcp__github__list_issues","arguments":"{\"state\":\"open\"}"}}]},+{"role":"tool","tool_call_id":"call_2","content":"#1 bug"},+{"role":"user","content":"thanks, and the closed ones?"}],+"tools":[{"type":"function","function":{"name":"read","description":"Read a file","parameters":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}}},+{"type":"function","function":{"name":"tool_search","description":"Search tools","parameters":{"type":"object","properties":{"query":{"type":"string"}},"required":["query"]}}}]}`++// piToolless is pi's request with no tools left to offer: openai-completions+// sends "tools": [] beside the history of calls.+var piToolless = strings.Replace(piDeferred, piDeferred[strings.Index(piDeferred, `"tools":[`):], `"tools":[]}`, 1)++const (+ histChatOK = `{"id":"c","object":"chat.completion","model":"m1","choices":[{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1,"total_tokens":2}}`+ histResponsesOK = "data: {\"type\":\"response.created\",\"response\":{\"id\":\"r1\",\"model\":\"m1\"}}\n\n" ++ "data: {\"type\":\"response.output_text.delta\",\"delta\":\"ok\"}\n\n" ++ "data: {\"type\":\"response.completed\",\"response\":{\"id\":\"r1\",\"status\":\"completed\",\"output\":[],\"usage\":{\"input_tokens\":1,\"output_tokens\":1}}}\n\n"+ histAnthropicOK = "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"msg_1\",\"type\":\"message\",\"role\":\"assistant\",\"model\":\"m1\",\"content\":[],\"usage\":{\"input_tokens\":1,\"output_tokens\":0}}}\n\n" ++ "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" ++ "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"ok\"}}\n\n" ++ "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" ++ "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"},\"usage\":{\"output_tokens\":1}}\n\n" ++ "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n"+)++// declaredIn is the tool names body's tools declare, on proto.+func declaredIn(proto provider.Protocol, body []byte) map[string]bool {+ out := map[string]bool{}+ for _, t := range gjson.GetBytes(body, "tools").Array() {+ n := t.Get("name").String()+ if proto == provider.Chat {+ n = t.Get("function.name").String()+ }+ out[n] = true+ }+ return out+}++func postChatTo(h http.Handler, body string) (int, string) {+ rec := httptest.NewRecorder()+ h.ServeHTTP(rec, httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(body)))+ return rec.Code, rec.Body.String()+}++// A Chat or Responses upstream that checks each history call's name+// against tools turns pi's request away naming the tool: it is asked once+// more with a stub declared for it, and the next request carries the+// stub from the start.+func TestUndeclaredHistoryToolRefusedThenStubbed(t *testing.T) {+ for _, tc := range []struct {+ proto provider.Protocol+ ok string+ refusal string+ }{+ {provider.Chat, histChatOK, `{"error":{"message":"Tool 'mcp__github__list_issues' not found in tools","type":"invalid_request_error","code":"invalid_tool"}}`},+ {provider.Responses, histResponsesOK, `{"error":{"message":"Invalid input[3]: function 'mcp__github__list_issues' is not defined in tools.","type":"invalid_request_error","param":"input"}}`},+ {provider.Anthropic, histAnthropicOK, `{"type":"error","error":{"type":"invalid_request_error","message":"Improperly formed request: tool mcp__github__list_issues not found in toolConfig"}}`},+ } {+ t.Run(string(tc.proto), func(t *testing.T) {+ f := &fake{t: t, ctype: ctypeOf(tc.proto), reply: tc.ok}+ var asked [][]byte+ f.refuse = func(b []byte) (int, string) {+ asked = append(asked, b)+ if !declaredIn(tc.proto, b)["mcp__github__list_issues"] {+ return 400, tc.refusal+ }+ return 0, ""+ }+ setup(t, tc.proto, f)+ h := New().Handler()+ code, out := postChatTo(h, piDeferred)+ if code != 200 || len(asked) != 2 {+ t.Fatalf("code %d after %d asks, want 200 after 2: %s", code, len(asked), out)+ }+ last := asked[1]+ d := declaredIn(tc.proto, last)+ if !d["read"] || !d["tool_search"] || len(d) != 3 {+ t.Fatalf("declared %v, want read, tool_search and the stub: %s", d, last)+ }+ if c := gjson.GetBytes(last, "tool_choice"); c.Exists() {+ t.Fatalf("tool_choice %s set where the client offered tools", c.Raw)+ }+ if strings.Contains(strings.ToLower(string(last)), "magpie") {+ t.Fatalf("the stub names magpie: %s", last)+ }+ asked = nil+ if code, out := postChatTo(h, piDeferred); code != 200 || len(asked) != 1 {+ t.Fatalf("next request: code %d after %d asks, want 200 after 1 (remembered): %s", code, len(asked), out)+ }+ })+ }+}++// Anthropic turns away tool_use blocks with no tools ("Requests which+// include `tool_use` or `tool_result` blocks must define tools"): pi's+// "tools": [] goes with a stub for each tool the history called, which+// none may be called.+func TestToollessHistoryToAnthropicDeclaresStubs(t *testing.T) {+ f := &fake{t: t, reply: histAnthropicOK}+ var asked [][]byte+ f.refuse = func(b []byte) (int, string) {+ asked = append(asked, b)+ if strings.Contains(string(b), `"tool_use"`) && len(gjson.GetBytes(b, "tools").Array()) == 0 {+ return 400, `{"type":"error","error":{"type":"invalid_request_error","message":"Requests which include ` + "`tool_use`" + ` or ` + "`tool_result`" + ` blocks must define tools."}}`+ }+ return 0, ""+ }+ setup(t, provider.Anthropic, f)+ code, out := post(t, "/v1/chat/completions", piToolless)+ if code != 200 || len(asked) != 1 {+ t.Fatalf("code %d after %d asks, want 200 at once: %s", code, len(asked), out)+ }+ d := declaredIn(provider.Anthropic, asked[0])+ if !d["tool_search"] || !d["mcp__github__list_issues"] || len(d) != 2 {+ t.Fatalf("declared %v, want the two tools the history called: %s", d, asked[0])+ }+ if c := gjson.GetBytes(asked[0], "tool_choice.type").String(); c != "none" {+ t.Fatalf("tool_choice %q, want none: the client offered no tools", c)+ }+}++// pi's "tools": [] to a Chat upstream that wants tools beside the calls+// (a relay in front of Anthropic passing its refusal on) is asked once+// more with the stubs, and tool_choice none.+func TestToollessHistoryRefusedOnChat(t *testing.T) {+ f := &fake{t: t, ctype: "application/json", reply: histChatOK}+ var asked [][]byte+ f.refuse = func(b []byte) (int, string) {+ asked = append(asked, b)+ if len(gjson.GetBytes(b, "tools").Array()) == 0 {+ return 400, `{"error":{"message":"litellm.BadRequestError: AnthropicException - {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"Requests which include ` + "`tool_use`" + ` or ` + "`tool_result`" + ` blocks must define tools.\"}}","type":null,"param":null,"code":"400"}}`+ }+ return 0, ""+ }+ setup(t, provider.Chat, f)+ code, out := post(t, "/v1/chat/completions", piToolless)+ if code != 200 || len(asked) != 2 {+ t.Fatalf("code %d after %d asks, want 200 after 2: %s", code, len(asked), out)+ }+ if d := declaredIn(provider.Chat, asked[1]); !d["tool_search"] || !d["mcp__github__list_issues"] {+ t.Fatalf("declared %v: %s", d, asked[1])+ }+ if c := gjson.GetBytes(asked[1], "tool_choice").String(); c != "none" {+ t.Fatalf("tool_choice %q, want none", c)+ }+}++// An upstream that takes the history as it is (OpenAI, Anthropic with+// tools, Gemini) is sent it as it was: no stub, and one ask.+func TestUndeclaredHistoryToolTakenAsSent(t *testing.T) {+ for _, tc := range []struct {+ proto provider.Protocol+ ok string+ }{{provider.Chat, histChatOK}, {provider.Responses, histResponsesOK}, {provider.Anthropic, histAnthropicOK}} {+ t.Run(string(tc.proto), func(t *testing.T) {+ f := &fake{t: t, ctype: ctypeOf(tc.proto), reply: tc.ok}+ setup(t, tc.proto, f)+ if code, out := post(t, "/v1/chat/completions", piDeferred); code != 200 || f.calls != 1 {+ t.Fatalf("code %d after %d asks: %s", code, f.calls, out)+ }+ if d := declaredIn(tc.proto, f.got); d["mcp__github__list_issues"] || len(d) != 2 {+ t.Fatalf("declared %v, want read and tool_search alone: %s", d, f.got)+ }+ })+ }+}++// A refusal over something else, or naming a tool the request declares,+// is not this one.+func TestRefusesHistoryToolsMatch(t *testing.T) {+ names := []string{"mcp__github__list_issues"}+ for _, b := range []string{+ `{"error":{"message":"Tool 'mcp__github__list_issues' not found in tools"}}`,+ `{"error":{"message":"unknown tool: mcp__github__list_issues"}}`,+ `{"error":{"message":"function \"mcp__github__list_issues\" does not exist"}}`,+ } {+ if !refusesHistoryTools([]byte(b), names, false) {+ t.Errorf("not matched: %s", b)+ }+ }+ for _, b := range []string{+ `{"error":{"message":"Tool 'read' not found in tools"}}`,+ `{"error":{"message":"Invalid value for 'temperature'"}}`,+ `{"error":{"message":"mcp__github__list_issues_extra not found"}}`,+ `{"error":{"message":"Requests which include tool_use or tool_result blocks must define tools."}}`,+ } {+ if refusesHistoryTools([]byte(b), names, false) {+ t.Errorf("matched: %s", b)+ }+ }+ if !refusesHistoryTools([]byte(`Requests which include tool_use or tool_result blocks must define tools.`), names, true) {+ t.Error("must define tools not matched with no tools")+ }+}++// ctypeOf is the fake's reply type: a Chat client's request is relayed to+// a Chat upstream as sent (not streamed), and translated, streamed, to+// the others.+func ctypeOf(proto provider.Protocol) string {+ if proto == provider.Chat {+ return "application/json"+ }+ return ""+}