Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

gateway: a history call to a tool the request doesn't declare gets a stub where it is refused (pi, T3 Code tool_search)

yetone31275d0734fe
diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.md
index 9f9f35226..e4590cc4b 100644
--- a/docs/subsystems/gateway-routing.md
+++ b/docs/subsystems/gateway-routing.md
@@ -190,6 +190,7 @@ recorded as a prediction failure (`noPrediction`).
- Kiro is told where to cache the prompt (plugins#110: a Kiro subscription read 0 from its cache). Its API takes Bedrock's `cachePoint: {"type":"default"}` on a `userInputMessage`, an `assistantResponseMessage` or as a tool entry. The built-in (`buildKiro` in [`kiro.go`](../../internal/gateway/kiro.go)) marks the first message when there is a system prompt, which it folds into that message, and the current message, as `buildAnthropic` marks Anthropic's. A moved Kiro's plugin (opencode-kiro-auth 0.1.10, the mover's `min`) turns each `cache_control` in the Anthropic body it is handed into one, at most 4, the last kept.
- A model that refuses a conversation ending with the assistant's message (`refusesPrefill`: OpenCode Go's Claude models over Messages, "This model does not support assistant message prefill"; Volcengine's Agent Plan, "…for a model that does not support prefill"; Gemini's, "Requests ending with a model turn are not supported") is marked for that provider, model and API (`prefillRefused`, #1447); the provider's other models keep their prefill ([`prefill.go`](../../internal/gateway/prefill.go)). A Chat or Responses client's conversation ending with the assistant's message, which those APIs read as a turn already said (`Request.LastIsTurn`), is asked again with a user turn after it (`continueText`), translated (`forwardTranslated`) or relayed (`passthrough`, `userLastBody`), and so from then on. An Anthropic client's prefill is its own ask and gets the refusal as sent. magpie's own continuation of a cut reply (`continuation.go`) isn't asked of that model again; the try the model refused ends the reply with what cut it, not with the refusal. A request magpie builds in Gemini's own shape (`buildCodeAssistSent`: Vertex AI, Gemini CLI's and Antigravity's Code Assist, Factory) never ends with the model's turn, which all of them turn away: a user turn goes after it, `continueText`, or `compactLastText` when an agent is compacting with its ask in the system prompt (`modelLastAsk`), as one goes before a history that opens with the model's. Nor does it send a user turn that mixes a tool's `functionResponse` with text, which Vertex AI answers with the same "ending with a model turn" (#1533: Codex's compaction prompt after a tool's output): what the user said goes in a user turn of its own after the tools' responses (`hasResponse`), except for a Claude model on Antigravity, whose tool_result turn holds both.
- A tool's `strict` that an upstream turns away (`toolStrictRefusal` in [`strict_refused.go`](../../internal/gateway/strict_refused.go): "Unsupported Chat Completions API field: tools[0].function.strict", OpenAI-style unknown_parameter, "Extra inputs are not permitted" and pydantic's extra_forbidden on it) is asked once more without it, in Chat's `tools[].function` and Responses' `tools[]` alike, and marked for that provider and API (`toolStrictRefused`): its next requests go without it at once. OpenCode 2 sends `"strict": false` on every tool through `@ai-sdk/openai-compatible`, and a custom provider that refused it ended the session (#1980). An upstream that hasn't refused it still gets it, as the ChatGPT backend needs it (#383). `TestToolStrictLeftOutWhereRefused` (real OpenCode 2.0.25 requests), `TestToolStrictRefusalWordings`.
+- A history call to a tool the request doesn't declare, which an agent that loads its tools on demand sends (pi's and T3 Code's `tool_search`: the tools a search found are called in the history but not declared in the next request's `tools`, and pi sends `"tools": []` with no tools left; Gearclickist on Discord), goes as sent where it is taken: OpenAI's Chat and Responses, the ChatGPT backend and Gemini take it, and Anthropic takes a call to a tool it isn't given ([`undeclared_tools.go`](../../internal/gateway/undeclared_tools.go)). Anthropic refuses calls with no tools at all ("Requests which include `tool_use` or `tool_result` blocks must define tools"), so `anthropicBody` declares a stub for each tool the history called when the request has none (description "Called earlier in this conversation; not available now.", any object as its input) with `tool_choice` `none`, as the Kiro plugin declares its history's tools for Kiro. An upstream that turns the request away naming such a tool as not found, unknown or not defined, or that says tools must be defined beside the calls (`refusesHistoryTools`: a relay in front of Kiro or Bedrock, a vendor checking each name), is asked once more with a stub declared for each tool the history called and `tools` lacks (`withHistoryTools`, Chat, Responses and Messages bodies, relayed or translated), `tool_choice` `none` only where the client offered no tools, and marked for that provider and API (`historyToolsRefused`): its next requests carry the stubs at once. A Responses call under a namespace is the namespace's, and a request whose calls are all declared goes as it was. `TestUndeclaredHistoryToolRefusedThenStubbed`, `TestToollessHistoryToAnthropicDeclaresStubs`, `TestToollessHistoryRefusedOnChat`, `TestUndeclaredHistoryToolTakenAsSent`, `TestRefusesHistoryToolsMatch`.
- A Codex custom tool (exec, apply_patch) that a Responses upstream turns away (`customToolRefusal` in [`custom_tool.go`](../../internal/gateway/custom_tool.go): DeepSeek's 400 "Unsupported custom tool: 'exec'. Only 'apply_patch' is supported.", #2656) ends the relay: `passthrough` marks the model (`customToolRefused`) and returns not done, and `attempt` translates the request on the same API (`ownAPI`), where each custom tool is offered as a function taking its `input` and a call comes back as the `custom_tool_call` Codex runs, its name kept. A marked model's next requests with a custom tool are translated at once; one without stays relayed, and an upstream that never refused them still gets them as sent. A compaction (`keepsCompaction`) isn't translated, and gets the refusal. Where the refusal reaches a group untranslated it is a shape refusal (`shapeRefused`): the next member is asked. `TestCustomToolTranslatedWhereRefused`, `TestCustomToolRefusalInGroup`, `TestCustomToolRefusalWordings`.
- A tool call's arguments reach the client as the vendor writes them, translated too (echo_ts on Discord: Codex on Copilot about 10x slower, an apply_patch shown only once whole). The Responses decoder hands each `function_call_arguments.delta` and `custom_tool_call_input.delta` on as a `KToolArgs` delta (`argsDelta`), holding back only the one that would make the JSON whole until `.done`, which `endCall` checks against the final arguments; the Responses encoder streams a custom tool's input as `custom_tool_call_input.delta` (`customStream` in [`custom_tool.go`](../../internal/gateway/custom_tool.go)) before its `.done`. A Copilot account's Responses request is relayed with a `web_search` tool in it (`searchesItself`; checked on a real account 2026-10-17), and Copilot is never picked to search for another provider's model (`searchRank`). It is translated instead when a tool marks a parameter `"encrypted": true` (`asksSealing`, MultiAgentV2's `spawn_agent`): Copilot seals such an argument and can't read its own seal on the subagent's turn. `TestCallArgsStreamWhenTranslated`, `TestCustomInputStreams`, `TestResponsesCallArgsStream`, `TestCopilotRelayedUnlessAskedToSeal`, `TestCopilotUltraSubagentWithWebSearch`.
- A relayed Responses stream gives each output item the one id its `response.output_item.added` gave it, in every event after that carries an `item_id` or an `item`, and in the `output` of `response.completed`, `response.incomplete` or `response.failed` (`idTidy` in [`itemids.go`](../../internal/gateway/itemids.go), applied in `passthrough` before `phaseTidy`). GitHub Copilot's `/responses` gives an item another id in every event (checked 2026-10-19 on a real account, gpt-5.5 with web_search), and Codex starts an item by the added's id and ends one by the done's, so a commentary message was shown twice and a web search never ended (echo_ts on Discord, `TestCopilotItemKeepsItsID`). An upstream whose ids agree goes through byte for byte (`TestItemIDsThatAgreeGoAsTheyCame`); a translated reply's ids are magpie's own.
diff --git a/internal/gateway/anthropic.go b/internal/gateway/anthropic.go
index 0bd150611..db7ed5c9b 100644
--- a/internal/gateway/anthropic.go
+++ b/internal/gateway/anthropic.go
@@ -997,9 +997,20 @@ func anthropicBody(r *Request, model, host string, replay bool) []byte {
// auto mode's review, which Claude alone does (automode.go)
out["safeguards"] = r.Safeguards
}
- if len(r.Tools) > 0 || r.WebSearch {
+ // a history of calls with no tools to go with it, which Anthropic
+ // turns away ("must define tools", undeclared_tools.go): each tool
+ // it called is declared, and none of them may be called
+ rTools := r.Tools
+ stubbed := false
+ if len(rTools) == 0 && !r.WebSearch {
+ for _, n := range historyCallNames(r) {
+ rTools = append(rTools, Tool{Name: n, Description: stubToolDescription})
+ stubbed = true
+ }
+ }
+ if len(rTools) > 0 || r.WebSearch {
var tools []map[string]any
- for _, t := range r.Tools {
+ for _, t := range rTools {
schema := t.Schema
if len(schema) == 0 {
schema = json.RawMessage(`{"type":"object","properties":{}}`)
@@ -1017,14 +1028,14 @@ func anthropicBody(r *Request, model, host string, replay bool) []byte {
out["tools"] = tools
var tc map[string]any
switch {
+ case stubbed || r.ToolChoice == "none":
+ tc = map[string]any{"type": "none"}
case r.ToolChoice == "auto" || r.ToolChoice == "":
if r.Parallel != nil && !*r.Parallel {
tc = map[string]any{"type": "auto"}
}
case r.ToolChoice == "required":
tc = map[string]any{"type": "any"}
- case r.ToolChoice == "none":
- tc = map[string]any{"type": "none"}
case strings.HasPrefix(r.ToolChoice, "name:"):
tc = map[string]any{"type": "tool", "name": strings.TrimPrefix(r.ToolChoice, "name:")}
}
diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.go
index 43826a6e6..3e7b8c4ed 100644
--- a/internal/gateway/gateway.go
+++ b/internal/gateway/gateway.go
@@ -4419,6 +4419,11 @@ func (s *Server) passthrough(w http.ResponseWriter, r *http.Request, p provider.
if !s.fits(p.ID, toolStrictRefused, proto) {
body, _ = withoutToolStrict(proto, body)
}
+ // a stub for each tool the history called and tools lacks, to an
+ // upstream that turned such a call away before (undeclared_tools.go)
+ if !s.fits(p.ID, historyToolsRefused, proto) {
+ body, _, _ = withHistoryTools(proto, body)
+ }
// structured output an upstream turned away before (#2126): a schema
// as any JSON object, or the format told in the system prompt alone
if !s.fits(p.ID, formatRefused(model), proto) {
@@ -4469,6 +4474,24 @@ func (s *Server) passthrough(w http.ResponseWriter, r *http.Request, p provider.
s.markUnfit(p.ID, mark, proto)
}
}
+ // a history call to a tool the request doesn't declare, or calls with
+ // no tools, turned away (pi's and T3 Code's tool_search,
+ // undeclared_tools.go): asked once more with a stub declared for each
+ // such tool, and so from then on
+ if badRequest(res.StatusCode) && s.fits(p.ID, historyToolsRefused, proto) {
+ b, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
+ res.Body.Close()
+ res.Body = io.NopCloser(bytes.NewReader(b))
+ if nb, names, toolless := withHistoryTools(proto, body); len(names) > 0 && refusesHistoryTools(b, names, toolless) {
+ body = nb
+ if res, err = s.forward(r.Context(), p, proto, path, p.Prepare(body), r.Header); err != nil {
+ return writeError(w, proto, 502, p.Name+": "+err.Error()), err.Error(), true
+ }
+ if res.StatusCode < 400 {
+ s.markUnfit(p.ID, historyToolsRefused, proto)
+ }
+ }
+ }
// a vendor that turns away the fields it doesn't know (Mistral's
// extra_forbidden for OpenCode's store, #393) is asked again without
// the optional ones it named, and not sent them again once that works
@@ -5258,6 +5281,12 @@ func (s *Server) forwardTranslated(ctx context.Context, p provider.Provider, to
// the other magpie searches for it (search_remote.go)
body = withFields(body, map[string]any{"web_search_options": map[string]any{}})
}
+ if !s.fits(p.ID, historyToolsRefused, to) {
+ // a stub for each tool the history called and tools lacks,
+ // to an upstream that turned such a call away before
+ // (undeclared_tools.go)
+ body, _, _ = withHistoryTools(to, body)
+ }
if to == provider.CodeAssist && p.Account != nil {
// the envelope is named in there, where the id it carries is
// known: on Antigravity that is the variant the effort picked,
@@ -5354,6 +5383,16 @@ func (s *Server) forwardTranslated(ctx context.Context, p provider.Provider, to
s.markUnfit(p.ID, thinkingConfigField, to)
continue
}
+ if badRequest(res.StatusCode) && s.fits(p.ID, historyToolsRefused, to) {
+ if _, names, toolless := withHistoryTools(to, body); len(names) > 0 && refusesHistoryTools(b, names, toolless) {
+ // a history call to a tool the request doesn't declare,
+ // or calls with no tools, turned away (pi's and T3 Code's
+ // tool_search): asked again with a stub declared for each
+ // such tool, and so from then on
+ s.markUnfit(p.ID, historyToolsRefused, to)
+ continue
+ }
+ }
if badRequest(res.StatusCode) && rootUnionRefusal.Match(b) && s.fits(p.ID, rootUnionRefused(model), to) {
if _, changed := objectRootsReq(req); changed {
// a tool's parameters refused for a union at their root
diff --git a/internal/gateway/undeclared_tools.go b/internal/gateway/undeclared_tools.go
new file mode 100644
index 000000000..a3069e799
--- /dev/null
+++ b/internal/gateway/undeclared_tools.go
@@ -0,0 +1,225 @@
+package gateway
+
+import (
+ "bytes"
+ "encoding/json"
+ "regexp"
+
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// An agent that loads its tools when it needs them (pi's and T3 Code's
+// tool_search, Gearclickist on Discord) declares a few tools and
+// tool_search in each request, while its history holds calls to tools a
+// search found earlier and the request no longer declares; one that has
+// no tools to offer sends "tools": [] beside a history of calls. OpenAI's
+// Chat and Responses APIs and Gemini take such a history. Anthropic's
+// Messages API takes a call to a tool it isn't given, but not a call with
+// no tools at all ("Requests which include `tool_use` or `tool_result`
+// blocks must define tools"), so anthropicBody declares a stub for each
+// tool the history called when the request has none, with tool_choice
+// none: the client offered nothing to call. A stricter upstream (a relay
+// in front of Kiro or Bedrock, a vendor that checks each call's name
+// against tools) turns the request away naming the tool; it is asked
+// once more with a stub declared for each tool the history called and
+// tools lacks, and so from then on. A request whose calls are all
+// declared goes as it was.
+
+// historyToolsRefused is how unfit remembers a provider refusing a
+// history call to a tool its request didn't declare: the provider's
+// validator, not a model, turns it away.
+const historyToolsRefused = "tools[].history"
+
+// stubToolDescription is a stub's description: the call it stands for
+// was made earlier and the tool isn't offered now.
+const stubToolDescription = "Called earlier in this conversation; not available now."
+
+// anthropicToolName is a name Anthropic takes for a tool.
+var anthropicToolName = regexp.MustCompile(`^[a-zA-Z0-9_-]{1,128}$`)
+
+// toolsMissing is an upstream refusing tool calls in a request with no
+// tools: Anthropic's "must define tools", Bedrock's "The toolConfig field
+// must be defined when using toolUse and toolResult content blocks", or
+// tools said required beside the calls.
+var toolsMissing = regexp.MustCompile("(?i)must define tools|toolconfig field must be defined|tools?\\W{1,4}(?:param(?:eter)?|field)?\\W{0,4}(?:is |are )?(?:required|must be (?:defined|provided|specified))[^\\n]{0,80}(?:tool_calls?|tool_use|tool_result|toolUse|toolResult|function_call)")
+
+// toolNotDeclared is what follows or leads a tool's name in a refusal of
+// a call to it: not found, not defined, unknown.
+const toolNotDeclared = `(?:not (?:found|defined|declared|provided|offered|available|present|exist|in (?:the )?(?:tools?|request|list))|unknown|undefined|undeclared|does ?n[o']t exist|no such|missing|invalid)`
+
+// refusesHistoryTools reports whether b, a 4xx body, refuses a history
+// call to one of names, the tools the history called that the request
+// doesn't declare, or calls in a request with no tools (toolless).
+func refusesHistoryTools(b []byte, names []string, toolless bool) bool {
+ if toolless && toolsMissing.Match(b) {
+ return true
+ }
+ for _, n := range names {
+ q := regexp.QuoteMeta(n)
+ name := `(?:^|[^A-Za-z0-9_.-])` + q + `(?:$|[^A-Za-z0-9_-])`
+ re, err := regexp.Compile(`(?i)` + toolNotDeclared + `[^\n{}]{0,80}?` + name + `|` + name + `[^\n{}]{0,80}?` + toolNotDeclared)
+ if err == nil && re.Match(b) {
+ return true
+ }
+ }
+ return false
+}
+
+// withHistoryTools is a Chat, Responses or Messages body with a stub
+// declared for each tool its history calls and its tools don't, the
+// names it declared, and whether tools was empty. tool_choice goes none
+// where the client offered no tools, so the model calls none of them.
+func withHistoryTools(proto provider.Protocol, body []byte) ([]byte, []string, bool) {
+ if proto != provider.Chat && proto != provider.Responses && proto != provider.Anthropic {
+ return body, nil, false
+ }
+ dec := json.NewDecoder(bytes.NewReader(body))
+ dec.UseNumber()
+ var m map[string]any
+ if dec.Decode(&m) != nil || m == nil {
+ return body, nil, false
+ }
+ tools, _ := m["tools"].([]any)
+ declared := map[string]bool{}
+ declaredNames(proto, tools, declared)
+ var add []string
+ for _, n := range calledNames(proto, m) {
+ if !declared[n] && (proto != provider.Anthropic || anthropicToolName.MatchString(n)) {
+ declared[n] = true
+ add = append(add, n)
+ }
+ }
+ if len(add) == 0 {
+ return body, nil, false
+ }
+ toolless := len(tools) == 0
+ for _, n := range add {
+ tools = append(tools, stubTool(proto, n))
+ }
+ m["tools"] = tools
+ if toolless {
+ if _, set := m["tool_choice"]; !set {
+ if proto == provider.Anthropic {
+ m["tool_choice"] = map[string]any{"type": "none"}
+ } else {
+ m["tool_choice"] = "none"
+ }
+ }
+ }
+ out, err := marshalPlain(m)
+ if err != nil {
+ return body, nil, false
+ }
+ return out, add, toolless
+}
+
+// declaredNames adds the names tools declares to into: a function's, a
+// custom tool's, and on Responses a namespace's own tools.
+func declaredNames(proto provider.Protocol, tools []any, into map[string]bool) {
+ for _, t := range tools {
+ tm, _ := t.(map[string]any)
+ if tm == nil {
+ continue
+ }
+ if proto == provider.Responses && tm["type"] == "namespace" {
+ nested, _ := tm["tools"].([]any)
+ declaredNames(proto, nested, into)
+ continue
+ }
+ if n, _ := tm["name"].(string); n != "" {
+ into[n] = true
+ }
+ for _, k := range []string{"function", "custom"} {
+ if fn, _ := tm[k].(map[string]any); fn != nil {
+ if n, _ := fn["name"].(string); n != "" {
+ into[n] = true
+ }
+ }
+ }
+ }
+}
+
+// calledNames is the names of the functions m's history calls, in order,
+// each once. A Responses call under a namespace is the namespace's, which
+// declares its own tools, and is left to it.
+func calledNames(proto provider.Protocol, m map[string]any) []string {
+ var names []string
+ seen := map[string]bool{}
+ add := func(n string) {
+ if n != "" && !seen[n] {
+ seen[n] = true
+ names = append(names, n)
+ }
+ }
+ switch proto {
+ case provider.Chat:
+ msgs, _ := m["messages"].([]any)
+ for _, x := range msgs {
+ msg, _ := x.(map[string]any)
+ calls, _ := msg["tool_calls"].([]any)
+ for _, c := range calls {
+ cm, _ := c.(map[string]any)
+ if fn, _ := cm["function"].(map[string]any); fn != nil {
+ n, _ := fn["name"].(string)
+ add(n)
+ }
+ }
+ }
+ case provider.Responses:
+ items, _ := m["input"].([]any)
+ for _, x := range items {
+ it, _ := x.(map[string]any)
+ if it == nil || it["type"] != "function_call" {
+ continue
+ }
+ if ns, _ := it["namespace"].(string); ns != "" {
+ continue
+ }
+ n, _ := it["name"].(string)
+ add(n)
+ }
+ case provider.Anthropic:
+ msgs, _ := m["messages"].([]any)
+ for _, x := range msgs {
+ msg, _ := x.(map[string]any)
+ blocks, _ := msg["content"].([]any)
+ for _, c := range blocks {
+ cb, _ := c.(map[string]any)
+ if cb != nil && cb["type"] == "tool_use" {
+ n, _ := cb["name"].(string)
+ add(n)
+ }
+ }
+ }
+ }
+ return names
+}
+
+// stubTool is a function tool named n that takes any object, as proto
+// declares one.
+func stubTool(proto provider.Protocol, n string) map[string]any {
+ params := map[string]any{"type": "object", "properties": map[string]any{}}
+ switch proto {
+ case provider.Chat:
+ return map[string]any{"type": "function", "function": map[string]any{"name": n, "description": stubToolDescription, "parameters": params}}
+ case provider.Responses:
+ return map[string]any{"type": "function", "name": n, "description": stubToolDescription, "parameters": params}
+ }
+ return map[string]any{"name": n, "description": stubToolDescription, "input_schema": params}
+}
+
+// historyCallNames is the names of the tools r's history calls that
+// Anthropic takes as a tool's name, in order, each once.
+func historyCallNames(r *Request) []string {
+ var names []string
+ seen := map[string]bool{}
+ for _, m := range r.Messages {
+ for _, p := range m.Parts {
+ if p.Kind == ToolCall && !seen[p.Name] && anthropicToolName.MatchString(p.Name) {
+ seen[p.Name] = true
+ names = append(names, p.Name)
+ }
+ }
+ }
+ return names
+}
diff --git a/internal/gateway/undeclared_tools_test.go b/internal/gateway/undeclared_tools_test.go
new file mode 100644
index 000000000..3ee668cd1
--- /dev/null
+++ b/internal/gateway/undeclared_tools_test.go
@@ -0,0 +1,222 @@
+package gateway
+
+import (
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "testing"
+
+ "github.com/tidwall/gjson"
+ "github.com/yetone/magpie/internal/provider"
+)
+
+// piDeferred is pi's request (openai-completions) after its tool_search
+// loaded a tool: the history calls mcp__github__list_issues, and tools
+// holds only read and tool_search (Gearclickist on Discord).
+const piDeferred = `{"model":"fake/m1","stream":false,"messages":[
+{"role":"system","content":"You are pi."},
+{"role":"user","content":"list my open issues"},
+{"role":"assistant","content":null,"tool_calls":[{"id":"call_1","type":"function","function":{"name":"tool_search","arguments":"{\"query\":\"github issues\"}"}}]},
+{"role":"tool","tool_call_id":"call_1","content":"Loaded 1 tool. They are available from your next call:\n- mcp__github__list_issues: List issues"},
+{"role":"assistant","content":null,"tool_calls":[{"id":"call_2","type":"function","function":{"name":"mcp__github__list_issues","arguments":"{\"state\":\"open\"}"}}]},
+{"role":"tool","tool_call_id":"call_2","content":"#1 bug"},
+{"role":"user","content":"thanks, and the closed ones?"}],
+"tools":[{"type":"function","function":{"name":"read","description":"Read a file","parameters":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}}},
+{"type":"function","function":{"name":"tool_search","description":"Search tools","parameters":{"type":"object","properties":{"query":{"type":"string"}},"required":["query"]}}}]}`
+
+// piToolless is pi's request with no tools left to offer: openai-completions
+// sends "tools": [] beside the history of calls.
+var piToolless = strings.Replace(piDeferred, piDeferred[strings.Index(piDeferred, `"tools":[`):], `"tools":[]}`, 1)
+
+const (
+ histChatOK = `{"id":"c","object":"chat.completion","model":"m1","choices":[{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1,"total_tokens":2}}`
+ histResponsesOK = "data: {\"type\":\"response.created\",\"response\":{\"id\":\"r1\",\"model\":\"m1\"}}\n\n" +
+ "data: {\"type\":\"response.output_text.delta\",\"delta\":\"ok\"}\n\n" +
+ "data: {\"type\":\"response.completed\",\"response\":{\"id\":\"r1\",\"status\":\"completed\",\"output\":[],\"usage\":{\"input_tokens\":1,\"output_tokens\":1}}}\n\n"
+ histAnthropicOK = "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"msg_1\",\"type\":\"message\",\"role\":\"assistant\",\"model\":\"m1\",\"content\":[],\"usage\":{\"input_tokens\":1,\"output_tokens\":0}}}\n\n" +
+ "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" +
+ "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"ok\"}}\n\n" +
+ "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +
+ "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"},\"usage\":{\"output_tokens\":1}}\n\n" +
+ "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n"
+)
+
+// declaredIn is the tool names body's tools declare, on proto.
+func declaredIn(proto provider.Protocol, body []byte) map[string]bool {
+ out := map[string]bool{}
+ for _, t := range gjson.GetBytes(body, "tools").Array() {
+ n := t.Get("name").String()
+ if proto == provider.Chat {
+ n = t.Get("function.name").String()
+ }
+ out[n] = true
+ }
+ return out
+}
+
+func postChatTo(h http.Handler, body string) (int, string) {
+ rec := httptest.NewRecorder()
+ h.ServeHTTP(rec, httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(body)))
+ return rec.Code, rec.Body.String()
+}
+
+// A Chat or Responses upstream that checks each history call's name
+// against tools turns pi's request away naming the tool: it is asked once
+// more with a stub declared for it, and the next request carries the
+// stub from the start.
+func TestUndeclaredHistoryToolRefusedThenStubbed(t *testing.T) {
+ for _, tc := range []struct {
+ proto provider.Protocol
+ ok string
+ refusal string
+ }{
+ {provider.Chat, histChatOK, `{"error":{"message":"Tool 'mcp__github__list_issues' not found in tools","type":"invalid_request_error","code":"invalid_tool"}}`},
+ {provider.Responses, histResponsesOK, `{"error":{"message":"Invalid input[3]: function 'mcp__github__list_issues' is not defined in tools.","type":"invalid_request_error","param":"input"}}`},
+ {provider.Anthropic, histAnthropicOK, `{"type":"error","error":{"type":"invalid_request_error","message":"Improperly formed request: tool mcp__github__list_issues not found in toolConfig"}}`},
+ } {
+ t.Run(string(tc.proto), func(t *testing.T) {
+ f := &fake{t: t, ctype: ctypeOf(tc.proto), reply: tc.ok}
+ var asked [][]byte
+ f.refuse = func(b []byte) (int, string) {
+ asked = append(asked, b)
+ if !declaredIn(tc.proto, b)["mcp__github__list_issues"] {
+ return 400, tc.refusal
+ }
+ return 0, ""
+ }
+ setup(t, tc.proto, f)
+ h := New().Handler()
+ code, out := postChatTo(h, piDeferred)
+ if code != 200 || len(asked) != 2 {
+ t.Fatalf("code %d after %d asks, want 200 after 2: %s", code, len(asked), out)
+ }
+ last := asked[1]
+ d := declaredIn(tc.proto, last)
+ if !d["read"] || !d["tool_search"] || len(d) != 3 {
+ t.Fatalf("declared %v, want read, tool_search and the stub: %s", d, last)
+ }
+ if c := gjson.GetBytes(last, "tool_choice"); c.Exists() {
+ t.Fatalf("tool_choice %s set where the client offered tools", c.Raw)
+ }
+ if strings.Contains(strings.ToLower(string(last)), "magpie") {
+ t.Fatalf("the stub names magpie: %s", last)
+ }
+ asked = nil
+ if code, out := postChatTo(h, piDeferred); code != 200 || len(asked) != 1 {
+ t.Fatalf("next request: code %d after %d asks, want 200 after 1 (remembered): %s", code, len(asked), out)
+ }
+ })
+ }
+}
+
+// Anthropic turns away tool_use blocks with no tools ("Requests which
+// include `tool_use` or `tool_result` blocks must define tools"): pi's
+// "tools": [] goes with a stub for each tool the history called, which
+// none may be called.
+func TestToollessHistoryToAnthropicDeclaresStubs(t *testing.T) {
+ f := &fake{t: t, reply: histAnthropicOK}
+ var asked [][]byte
+ f.refuse = func(b []byte) (int, string) {
+ asked = append(asked, b)
+ if strings.Contains(string(b), `"tool_use"`) && len(gjson.GetBytes(b, "tools").Array()) == 0 {
+ return 400, `{"type":"error","error":{"type":"invalid_request_error","message":"Requests which include ` + "`tool_use`" + ` or ` + "`tool_result`" + ` blocks must define tools."}}`
+ }
+ return 0, ""
+ }
+ setup(t, provider.Anthropic, f)
+ code, out := post(t, "/v1/chat/completions", piToolless)
+ if code != 200 || len(asked) != 1 {
+ t.Fatalf("code %d after %d asks, want 200 at once: %s", code, len(asked), out)
+ }
+ d := declaredIn(provider.Anthropic, asked[0])
+ if !d["tool_search"] || !d["mcp__github__list_issues"] || len(d) != 2 {
+ t.Fatalf("declared %v, want the two tools the history called: %s", d, asked[0])
+ }
+ if c := gjson.GetBytes(asked[0], "tool_choice.type").String(); c != "none" {
+ t.Fatalf("tool_choice %q, want none: the client offered no tools", c)
+ }
+}
+
+// pi's "tools": [] to a Chat upstream that wants tools beside the calls
+// (a relay in front of Anthropic passing its refusal on) is asked once
+// more with the stubs, and tool_choice none.
+func TestToollessHistoryRefusedOnChat(t *testing.T) {
+ f := &fake{t: t, ctype: "application/json", reply: histChatOK}
+ var asked [][]byte
+ f.refuse = func(b []byte) (int, string) {
+ asked = append(asked, b)
+ if len(gjson.GetBytes(b, "tools").Array()) == 0 {
+ return 400, `{"error":{"message":"litellm.BadRequestError: AnthropicException - {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"Requests which include ` + "`tool_use`" + ` or ` + "`tool_result`" + ` blocks must define tools.\"}}","type":null,"param":null,"code":"400"}}`
+ }
+ return 0, ""
+ }
+ setup(t, provider.Chat, f)
+ code, out := post(t, "/v1/chat/completions", piToolless)
+ if code != 200 || len(asked) != 2 {
+ t.Fatalf("code %d after %d asks, want 200 after 2: %s", code, len(asked), out)
+ }
+ if d := declaredIn(provider.Chat, asked[1]); !d["tool_search"] || !d["mcp__github__list_issues"] {
+ t.Fatalf("declared %v: %s", d, asked[1])
+ }
+ if c := gjson.GetBytes(asked[1], "tool_choice").String(); c != "none" {
+ t.Fatalf("tool_choice %q, want none", c)
+ }
+}
+
+// An upstream that takes the history as it is (OpenAI, Anthropic with
+// tools, Gemini) is sent it as it was: no stub, and one ask.
+func TestUndeclaredHistoryToolTakenAsSent(t *testing.T) {
+ for _, tc := range []struct {
+ proto provider.Protocol
+ ok string
+ }{{provider.Chat, histChatOK}, {provider.Responses, histResponsesOK}, {provider.Anthropic, histAnthropicOK}} {
+ t.Run(string(tc.proto), func(t *testing.T) {
+ f := &fake{t: t, ctype: ctypeOf(tc.proto), reply: tc.ok}
+ setup(t, tc.proto, f)
+ if code, out := post(t, "/v1/chat/completions", piDeferred); code != 200 || f.calls != 1 {
+ t.Fatalf("code %d after %d asks: %s", code, f.calls, out)
+ }
+ if d := declaredIn(tc.proto, f.got); d["mcp__github__list_issues"] || len(d) != 2 {
+ t.Fatalf("declared %v, want read and tool_search alone: %s", d, f.got)
+ }
+ })
+ }
+}
+
+// A refusal over something else, or naming a tool the request declares,
+// is not this one.
+func TestRefusesHistoryToolsMatch(t *testing.T) {
+ names := []string{"mcp__github__list_issues"}
+ for _, b := range []string{
+ `{"error":{"message":"Tool 'mcp__github__list_issues' not found in tools"}}`,
+ `{"error":{"message":"unknown tool: mcp__github__list_issues"}}`,
+ `{"error":{"message":"function \"mcp__github__list_issues\" does not exist"}}`,
+ } {
+ if !refusesHistoryTools([]byte(b), names, false) {
+ t.Errorf("not matched: %s", b)
+ }
+ }
+ for _, b := range []string{
+ `{"error":{"message":"Tool 'read' not found in tools"}}`,
+ `{"error":{"message":"Invalid value for 'temperature'"}}`,
+ `{"error":{"message":"mcp__github__list_issues_extra not found"}}`,
+ `{"error":{"message":"Requests which include tool_use or tool_result blocks must define tools."}}`,
+ } {
+ if refusesHistoryTools([]byte(b), names, false) {
+ t.Errorf("matched: %s", b)
+ }
+ }
+ if !refusesHistoryTools([]byte(`Requests which include tool_use or tool_result blocks must define tools.`), names, true) {
+ t.Error("must define tools not matched with no tools")
+ }
+}
+
+// ctypeOf is the fake's reply type: a Chat client's request is relayed to
+// a Chat upstream as sent (not streamed), and translated, streamed, to
+// the others.
+func ctypeOf(proto provider.Protocol) string {
+ if proto == provider.Chat {
+ return "application/json"
+ }
+ return ""
+}