Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

feat(eve): add native Gateway web search for Anthropic and Gemini (#4580)

Ben Pankowf8ea556b508c
diff --git a/.changeset/gateway-native-web-search.md b/.changeset/gateway-native-web-search.md
new file mode 100644
index 000000000..979c7b64d
--- /dev/null
+++ b/.changeset/gateway-native-web-search.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+`webSearch({ provider: "native" })` uses the model vendor's own hosted search for AI Gateway models: OpenAI web search for OpenAI models, Anthropic web search for Anthropic models, and Google Search grounding for Gemini 3 and later, instead of Exa. Other Gateway models don't get `web_search` with this setting unless you set `fallback`, for example `webSearch({ provider: "native", fallback: "exa" })`. Gemini models before Gemini 3, including direct ones, no longer get Google Search, which made the model drop every other tool.
diff --git a/.changeset/gateway-openai-web-search.md b/.changeset/gateway-openai-web-search.md
deleted file mode 100644
index 38de51c2f..000000000
--- a/.changeset/gateway-openai-web-search.md
+++ /dev/null
@@ -1,5 +0,0 @@
----
-"eve": patch
----
-
-`webSearch({ provider: "openai" })` selects OpenAI's hosted web search for OpenAI models routed through AI Gateway, so search runs server-side with URL citations instead of going to Exa. Other Gateway models don't get `web_search` with this setting unless you set `fallback`, for example `webSearch({ provider: "openai", fallback: "exa" })`.
diff --git a/docs/concepts/built-in-tools.md b/docs/concepts/built-in-tools.md
index 06e367167..401f08a98 100644
--- a/docs/concepts/built-in-tools.md
+++ b/docs/concepts/built-in-tools.md
@@ -218,23 +218,27 @@ export default webSearch({ provider: "browserbase" });
Use a [Gateway model ID](../agent-config#set-the-model) to route searches through Browserbase. AI Gateway executes the search using `AI_GATEWAY_API_KEY` or Vercel project OIDC credentials; no `BROWSERBASE_API_KEY` is needed. See [Browserbase Search on AI Gateway](https://vercel.com/docs/ai-gateway/models-and-providers/web-search#using-browserbase-search).
-Select OpenAI's hosted web search for an OpenAI model routed through AI Gateway:
+Select the model vendor's own hosted search for a model routed through AI Gateway:
```ts title="agent/tools/web_search.ts"
import { webSearch } from "eve/tools/web_search";
-export default webSearch({ provider: "openai" });
+export default webSearch({ provider: "native" });
```
-OpenAI runs the search server-side and returns URL citations, and AI Gateway uses its usual credentials, so no `OPENAI_API_KEY` is needed. OpenAI hosted search serves only OpenAI models, so any other Gateway model, including one picked by a dynamic model resolver, doesn't get `web_search`. eve picks the search tool from the requested model, so when AI Gateway's own `models` fallback serves a call with a non-OpenAI model, that call runs without search. Set `fallback` to give those models a provider that works with any Gateway model (`"exa"`, `"parallel"`, or `"browserbase"`):
+The vendor runs the search server-side and returns citations: OpenAI models use OpenAI web search, Anthropic models use Anthropic web search, and Gemini 3 and later models use Google Search grounding. AI Gateway uses its usual credentials, so no `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or Google key is needed. Every other Gateway model, including one picked by a dynamic model resolver, doesn't get `web_search`. That includes Gemini models before Gemini 3, which can't use Google Search alongside other tools.
+
+eve picks the search tool from the requested model, so when AI Gateway's own `models` fallback serves a call with a model from another vendor, that call can run without search. Anthropic web search runs only on Anthropic's own API. AI Gateway tries another host when Amazon Bedrock or Google Vertex AI rejects the tool. If no host accepts it, eve retries the step without `web_search`.
+
+Set `fallback` to give models without native search a provider that works with any Gateway model (`"exa"`, `"parallel"`, or `"browserbase"`):
```ts title="agent/tools/web_search.ts"
import { webSearch } from "eve/tools/web_search";
-export default webSearch({ provider: "openai", fallback: "exa" });
+export default webSearch({ provider: "native", fallback: "exa" });
```
-The `provider` setting applies only to AI Gateway models. Unsupported direct providers omit `web_search`.
+The `provider` setting applies only to AI Gateway models. Direct OpenAI, Anthropic, and Gemini 3 and later models always use their native search; other direct models omit `web_search`.
Replace provider-managed search with an authored implementation:
diff --git a/docs/reference/typescript-api.md b/docs/reference/typescript-api.md
index 165cff77b..159dcaba2 100644
--- a/docs/reference/typescript-api.md
+++ b/docs/reference/typescript-api.md
@@ -61,7 +61,7 @@ Tool-wide authoring helpers and types such as `defineTool`, `defineWorkflowTool`
`AgentReasoningDefinition` is exported from `eve` for the top-level `defineAgent({ reasoning })` setting. `AgentLimitsDefinition` is exported for `defineAgent({ limits })`. `AgentWorkflowDefinition`, `AgentWorkflowRetentionDefinition`, and `AgentWorkflowWorldDefinition` are exported from `eve` for the `defineAgent({ experimental: { workflow } })` config shape. `WebSearchToolInput` and `WebSearchProvider` are exported from `eve/tools/web_search`.
-`webSearch({ provider })` selects `"exa"`, `"parallel"`, `"browserbase"`, or `"openai"` for AI Gateway search using the Gateway model's credentials. `"openai"` applies only to OpenAI models; set `fallback` to `"exa"`, `"parallel"`, or `"browserbase"` to give other Gateway models search, which they otherwise don't get. Direct provider models keep their native search implementation when supported. See [Built-in tools](../concepts/built-in-tools#web_search) for setup and provider behavior.
+`webSearch({ provider })` selects `"exa"`, `"parallel"`, `"browserbase"`, or `"native"` for AI Gateway search using the Gateway model's credentials. `"native"` uses the model vendor's own search for OpenAI, Anthropic, and Gemini 3 and later models; set `fallback` to `"exa"`, `"parallel"`, or `"browserbase"` to give other Gateway models search, which they otherwise don't get. Direct provider models keep their native search implementation when supported. See [Built-in tools](../concepts/built-in-tools#web_search) for setup and provider behavior.
`defineInstructions` accepts `{ content: string, role?: "system" | "user" }`; omitted `role` means `"system"`. Its `eve/instructions` version of `defineDynamic` accepts only `session.started` and `turn.started` handlers returning `defineInstructions(...)` or `null`. The legacy `{ markdown: string }` definition remains available as a deprecated system-role form.
diff --git a/e2e/fixtures/agent-native-web-search/agent/agent.ts b/e2e/fixtures/agent-native-web-search/agent/agent.ts
new file mode 100644
index 000000000..4bf35c1ee
--- /dev/null
+++ b/e2e/fixtures/agent-native-web-search/agent/agent.ts
@@ -0,0 +1,7 @@
+import { e2eAgentConfig } from "@eve-e2e/config";
+import { defineAgent } from "eve";
+
+export default defineAgent({
+ ...e2eAgentConfig(),
+ defaultTools: false,
+});
diff --git a/e2e/fixtures/agent-native-web-search/agent/instructions.md b/e2e/fixtures/agent-native-web-search/agent/instructions.md
new file mode 100644
index 000000000..1ee9356f5
--- /dev/null
+++ b/e2e/fixtures/agent-native-web-search/agent/instructions.md
@@ -0,0 +1 @@
+Help Alice and Bob research public web pages. Use web_search to discover sources. Ground your replies in the search results and include source URLs when relevant.
diff --git a/e2e/fixtures/agent-native-web-search/agent/tools/web_search.ts b/e2e/fixtures/agent-native-web-search/agent/tools/web_search.ts
new file mode 100644
index 000000000..af9bc536c
--- /dev/null
+++ b/e2e/fixtures/agent-native-web-search/agent/tools/web_search.ts
@@ -0,0 +1,3 @@
+import { webSearch } from "eve/tools/web_search";
+
+export default webSearch({ provider: "native" });
diff --git a/e2e/fixtures/agent-native-web-search/evals/evals.config.ts b/e2e/fixtures/agent-native-web-search/evals/evals.config.ts
new file mode 100644
index 000000000..3a1a10d6f
--- /dev/null
+++ b/e2e/fixtures/agent-native-web-search/evals/evals.config.ts
@@ -0,0 +1,3 @@
+import { defineEvalConfig } from "eve/evals";
+
+export default defineEvalConfig({});
diff --git a/e2e/fixtures/agent-native-web-search/evals/search.eval.ts b/e2e/fixtures/agent-native-web-search/evals/search.eval.ts
new file mode 100644
index 000000000..6a517ac46
--- /dev/null
+++ b/e2e/fixtures/agent-native-web-search/evals/search.eval.ts
@@ -0,0 +1,37 @@
+import { defineEval } from "eve/evals";
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+ return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+
+/** Anthropic web search returns result pages; OpenAI web search returns the action it took. */
+function isNativeSearchOutput(value: unknown): boolean {
+ if (Array.isArray(value)) {
+ return value.some(
+ (result) =>
+ isRecord(result) && result.type === "web_search_result" && typeof result.url === "string",
+ );
+ }
+ return isRecord(value) && isRecord(value.action) && typeof value.action.type === "string";
+}
+
+export default defineEval({
+ tags: ["real-model"],
+ description: "Native Gateway search runs the model vendor's hosted search and cites a source.",
+ async test(t) {
+ const turn = await t.send(
+ [
+ "Alice is learning how to map array elements and flatten the results by one level.",
+ "Use web_search to find MDN's English reference for Array.prototype.flatMap.",
+ "Reply with the reference's title and its URL so she can read it.",
+ ].join("\n"),
+ );
+
+ turn.expectOk();
+ turn
+ .calledTool("web_search", { output: isNativeSearchOutput })
+ .label("web_search runs the model vendor's hosted search");
+ turn.noFailedActions();
+ t.messageIncludes(/developer\.mozilla\.org\/[^\s]*flatMap/iu);
+ },
+});
diff --git a/e2e/fixtures/agent-native-web-search/package.json b/e2e/fixtures/agent-native-web-search/package.json
new file mode 100644
index 000000000..187c318fa
--- /dev/null
+++ b/e2e/fixtures/agent-native-web-search/package.json
@@ -0,0 +1,26 @@
+{
+ "name": "agent-native-web-search",
+ "version": "0.0.0",
+ "private": true,
+ "type": "module",
+ "scripts": {
+ "build": "eve build",
+ "dev": "eve dev",
+ "start": "eve start",
+ "typecheck": "eve build --skip-sandbox-prewarm && tsc",
+ "test:e2e": "eve eval --strict"
+ },
+ "dependencies": {
+ "@eve-e2e/config": "workspace:*",
+ "@workflow/world-postgres": "catalog:",
+ "eve": "workspace:*"
+ },
+ "devDependencies": {
+ "@types/node": "catalog:",
+ "typescript": "catalog:"
+ },
+ "e2e": {
+ "modelMatrix": "full",
+ "worlds": []
+ }
+}
diff --git a/e2e/fixtures/agent-native-web-search/tsconfig.json b/e2e/fixtures/agent-native-web-search/tsconfig.json
new file mode 100644
index 000000000..85e3eceb8
--- /dev/null
+++ b/e2e/fixtures/agent-native-web-search/tsconfig.json
@@ -0,0 +1,17 @@
+{
+ "compilerOptions": {
+ "target": "ES2022",
+ "module": "esnext",
+ "moduleResolution": "bundler",
+ "outDir": "dist",
+ "rootDir": ".",
+ "strict": true,
+ "esModuleInterop": true,
+ "skipLibCheck": true,
+ "forceConsistentCasingInFileNames": true,
+ "declaration": true,
+ "noEmit": true,
+ "types": ["node"]
+ },
+ "include": ["agent/**/*.ts", "evals/**/*.ts"]
+}
diff --git a/packages/eve/extension-contracts/reports/tool/v84.json b/packages/eve/extension-contracts/reports/tool/v84.json
new file mode 100644
index 000000000..cdd5f0894
--- /dev/null
+++ b/packages/eve/extension-contracts/reports/tool/v84.json
@@ -0,0 +1,29 @@
+{
+ "kind": "eve-extension-capability-contract",
+ "capability": "tool",
+ "epoch": 84,
+ "sha256": "7af94003cd65bfae073afe9e4146322e8d544ec283b29ced8c5a8ff25897b886",
+ "exports": [
+ "AgentRouterInput",
+ "AgentRouterTool",
+ "WorkflowStepToolContext",
+ "WorkflowTool",
+ "WorkflowToolDefinition",
+ "WorkflowToolInput",
+ "WorkflowToolOptions",
+ "agentRouter",
+ "decide",
+ "defaultWebSearch",
+ "defineTool",
+ "defineWorkflowTool",
+ "disableTool",
+ "isDisabledToolSentinel",
+ "isWebSearchToolDefinition",
+ "serializeModelInputSchema",
+ "toolOutput",
+ "toolOutputPart",
+ "toolResultFrom",
+ "webSearch",
+ "workflow"
+ ]
+}
diff --git a/packages/eve/src/compiler/extension-compatibility.ts b/packages/eve/src/compiler/extension-compatibility.ts
index ed5551a62..d144cb999 100644
--- a/packages/eve/src/compiler/extension-compatibility.ts
+++ b/packages/eve/src/compiler/extension-compatibility.ts
@@ -22,9 +22,9 @@ interface ExtensionCapabilityContract {
const EXTENSION_CAPABILITY_CONTRACTS = {
extension: { current: 1, supported: [1], dropped: {} },
tool: {
- current: 83,
+ current: 84,
supported: [
- 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 29, 30, 31, 32, 34, 35, 76, 77, 78, 80, 81, 82, 83,
+ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 29, 30, 31, 32, 34, 35, 76, 77, 78, 80, 81, 82, 84,
],
dropped: {
14: "TaskExec.delegated was removed; migrate to workflow-backed background tools",
@@ -84,6 +84,7 @@ const EXTENSION_CAPABILITY_CONTRACTS = {
74: "eve/ai evaluate was renamed to decide and AI SDK evaluation models became decision models; import decide from eve/ai",
75: "eve/ai evaluate was renamed to decide and AI SDK evaluation models became decision models; import decide from eve/ai",
79: "Sandbox provider handle hooks were renamed to onSandboxStop and onSandboxDelete.",
+ 83: 'webSearch provider "openai" was replaced by "native" before it shipped in a release',
},
},
dynamicTool: {
diff --git a/packages/eve/src/compiler/normalize-manifest.test.ts b/packages/eve/src/compiler/normalize-manifest.test.ts
index 6343d1e76..0fd8e592b 100644
--- a/packages/eve/src/compiler/normalize-manifest.test.ts
+++ b/packages/eve/src/compiler/normalize-manifest.test.ts
@@ -323,7 +323,7 @@ describe("compileAgentManifest source graph", () => {
it.each<WebSearchToolInput>([
{ provider: "parallel" },
{ provider: "browserbase" },
- { fallback: "exa", provider: "openai" },
+ { fallback: "exa", provider: "native" },
])(
"preserves the %o search selection through serialization and runtime preparation",
async (selection) => {
diff --git a/packages/eve/src/harness/model-call/errors.test.ts b/packages/eve/src/harness/model-call/errors.test.ts
index 575e64509..b30dba629 100644
--- a/packages/eve/src/harness/model-call/errors.test.ts
+++ b/packages/eve/src/harness/model-call/errors.test.ts
@@ -511,6 +511,26 @@ describe("extractUnsupportedProviderToolTypes", () => {
expect(extractUnsupportedProviderToolTypes(error)).toEqual(["web_search_20250305"]);
});
+ it("returns the upstream tool type from Bedrock's raw rejection when every host failed", () => {
+ const upstream = Object.assign(new Error("Bad Request"), {
+ data: {
+ providerAttempts: [
+ {
+ error:
+ "tools.0: Input tag 'web_search_20250305' found using 'type' does not match any of the expected tags: 'bash_20250124', 'custom'",
+ provider: "bedrock",
+ },
+ { error: "Bad Request", provider: "vertexAnthropic" },
+ ],
+ },
+ });
+ const error = Object.assign(new Error("gateway", { cause: upstream }), {
+ name: "GatewayInternalServerError",
+ });
+
+ expect(extractUnsupportedProviderToolTypes(error)).toEqual(["web_search_20250305"]);
+ });
+
it("deduplicates when multiple providerAttempts reference the same tool type", () => {
const error = gatewayProviderToolFailure({
unsupportedTypes: ["web_search_20250305", "web_search_20250305"],
diff --git a/packages/eve/src/harness/model-call/errors.ts b/packages/eve/src/harness/model-call/errors.ts
index f70b28dcd..2a06a03f8 100644
--- a/packages/eve/src/harness/model-call/errors.ts
+++ b/packages/eve/src/harness/model-call/errors.ts
@@ -14,12 +14,14 @@ const GATEWAY_MODEL_REQUEST_REJECTED_MESSAGE =
* that AI Gateway returns when a fallback provider cannot serve a
* provider-specific tool (e.g. Bedrock rejecting `web_search_20250305`).
*
- * The phrasing comes from the gateway's own provider attempt projection
- * and is stable across the Bedrock and Vertex Anthropic backends. We
- * anchor the match on the literal `tool type` prefix to avoid sweeping
- * in unrelated "not supported" errors.
+ * The gateway reports a single rejecting host as `tool type 'X' is not
+ * supported`. When every host fails, it surfaces each host's raw error, and
+ * Bedrock's reads `Input tag 'X' found using 'type' does not match`. Both
+ * alternatives are anchored on their literal phrasing to avoid sweeping in
+ * unrelated "not supported" errors.
*/
-const UNSUPPORTED_TOOL_TYPE_REGEX = /tool type ['"]([\w.-]+)['"] is not supported/i;
+const UNSUPPORTED_TOOL_TYPE_REGEX =
+ /tool type ['"]([\w.-]+)['"] is not supported|Input tag ['"]([\w.-]+)['"] found using ['"]type['"] does not match/i;
/**
* The most informative human-readable rejection a model-call error
@@ -122,10 +124,8 @@ export function extractUnsupportedProviderToolTypes(error: unknown): readonly st
// includes a large request snapshot. Fall back to a raw string
// scan so we still surface the tool name when the regex match
// lies before the truncation boundary.
- const match = UNSUPPORTED_TOOL_TYPE_REGEX.exec(responseBody);
- if (match?.[1] !== undefined) {
- found.add(match[1]);
- }
+ const type = matchUnsupportedToolType(responseBody);
+ if (type !== undefined) found.add(type);
}
}
}
@@ -133,14 +133,17 @@ export function extractUnsupportedProviderToolTypes(error: unknown): readonly st
return [...found];
}
+function matchUnsupportedToolType(text: string): string | undefined {
+ const match = UNSUPPORTED_TOOL_TYPE_REGEX.exec(text);
+ return match?.[1] ?? match?.[2];
+}
+
function collectUnsupportedToolTypesFromValue(value: unknown, out: Set<string>): void {
if (value === null || value === undefined) return;
if (typeof value === "string") {
- const match = UNSUPPORTED_TOOL_TYPE_REGEX.exec(value);
- if (match?.[1] !== undefined) {
- out.add(match[1]);
- }
+ const type = matchUnsupportedToolType(value);
+ if (type !== undefined) out.add(type);
return;
}
diff --git a/packages/eve/src/harness/model-profile.test.ts b/packages/eve/src/harness/model-profile.test.ts
index abe5921a9..932141076 100644
--- a/packages/eve/src/harness/model-profile.test.ts
+++ b/packages/eve/src/harness/model-profile.test.ts
@@ -8,7 +8,12 @@ function model(provider: string, modelId = "test-model"): LanguageModel {
return new MockLanguageModelV3({ modelId, provider });
}
-const direct = { anthropicCache: false, filesOutsideToolResults: false, gateway: false };
+const direct = {
+ anthropicCache: false,
+ filesOutsideToolResults: false,
+ gateway: false,
+ googleSearchDropsTools: false,
+};
describe("resolveModelProfile", () => {
it.each<[string, LanguageModel, ModelProfile]>([
@@ -50,6 +55,21 @@ describe("resolveModelProfile", () => {
{ ...direct, filesOutsideToolResults: true, provider: "openai" },
],
["the ChatGPT subscription", model("codex.responses"), { ...direct, provider: "codex" }],
+ [
+ "a Gateway Gemini model older than Gemini 3",
+ "google/gemini-2.5-flash",
+ { ...direct, gateway: true, googleSearchDropsTools: true, provider: "google" },
+ ],
+ [
+ "a direct Gemini model older than Gemini 3",
+ model("google.generative-ai", "models/gemini-2.5-pro"),
+ { ...direct, googleSearchDropsTools: true, provider: "google" },
+ ],
+ [
+ "Gemini Robotics-ER 1.5, which the SDK treats as pre-Gemini 2",
+ "google/gemini-robotics-er-1.5-preview",
+ { ...direct, gateway: true, googleSearchDropsTools: true, provider: "google" },
+ ],
["a provider-less test double", {} as LanguageModel, { ...direct, provider: "" }],
])("describes %s", (_, input, expected) => {
expect(resolveModelProfile(input)).toEqual(expected);
diff --git a/packages/eve/src/harness/model-profile.ts b/packages/eve/src/harness/model-profile.ts
index 0bc97527d..fc33dd5d9 100644
--- a/packages/eve/src/harness/model-profile.ts
+++ b/packages/eve/src/harness/model-profile.ts
@@ -20,6 +20,11 @@ export interface ModelProfile {
readonly anthropicCache: boolean;
/** Chat Completions APIs can't carry files in tool results, so they move to user messages. */
readonly filesOutsideToolResults: boolean;
+ /**
+ * A Gemini model older than Gemini 3, which can't combine Google Search with function tools:
+ * the request keeps the search and drops every other tool.
+ */
+ readonly googleSearchDropsTools: boolean;
}
export function resolveModelProfile(model: LanguageModel): ModelProfile {
@@ -31,6 +36,7 @@ export function resolveModelProfile(model: LanguageModel): ModelProfile {
anthropicCache: false,
filesOutsideToolResults: false,
gateway: true,
+ googleSearchDropsTools: PRE_GEMINI_3_MODEL.test(modelId),
provider: modelId.split("/")[0]!,
};
}
@@ -42,10 +48,15 @@ export function resolveModelProfile(model: LanguageModel): ModelProfile {
(provider.includes("bedrock") && modelId.includes("anthropic")),
filesOutsideToolResults: provider.endsWith(".chat"),
gateway: false,
+ googleSearchDropsTools: PRE_GEMINI_3_MODEL.test(modelId),
provider: topLevelProvider,
};
}
+// Mirrors the Gemini generations `@ai-sdk/google` treats as pre-Gemini 3.
+const PRE_GEMINI_3_MODEL =
+ /(^|\/)gemini-(?:[12](?:[.-]|$)|pro(?:-vision)?$|robotics-er-1\.5(?:[.-]|$))/i;
+
/** A test double may omit `provider` or `modelId`; it reads as a direct model with no known provider. */
function lowerCaseOrEmpty(value: unknown): string {
return typeof value === "string" ? value.toLowerCase() : "";
diff --git a/packages/eve/src/harness/provider-tools.test.ts b/packages/eve/src/harness/provider-tools.test.ts
index 88be9f42f..c84ebc912 100644
--- a/packages/eve/src/harness/provider-tools.test.ts
+++ b/packages/eve/src/harness/provider-tools.test.ts
@@ -93,11 +93,15 @@ describe("resolveWebSearchBackend", () => {
it.each<[string, WebSearchSelection | undefined, string | null]>([
["openai/gpt-5.4", undefined, "exa"],
["openai/gpt-5.4", { provider: "parallel" }, "parallel"],
- ["openai/gpt-5.4", { provider: "openai" }, "openai"],
["anthropic/claude-opus-4.6", undefined, "exa"],
- // OpenAI's hosted search serves only OpenAI models: other models use the fallback, if any.
- ["anthropic/claude-opus-4.6", { provider: "openai" }, null],
- ["anthropic/claude-opus-4.6", { fallback: "parallel", provider: "openai" }, "parallel"],
+ ["openai/gpt-5.4", { provider: "native" }, "openai"],
+ ["anthropic/claude-opus-4.6", { provider: "native" }, "anthropic"],
+ ["google/gemini-3-flash", { provider: "native" }, "google"],
+ // Native search would drop a pre-Gemini 3 model's other tools; models without one use the
+ // fallback, if any.
+ ["google/gemini-2.5-flash", { provider: "native" }, null],
+ ["xai/grok-4", { provider: "native" }, null],
+ ["xai/grok-4", { fallback: "parallel", provider: "native" }, "parallel"],
])("uses Gateway search for Gateway model %s with %o", (model, selection, expected) => {
expect(resolveWebSearchBackend(resolveModelProfile(model), selection)).toBe(expected);
});
@@ -106,12 +110,16 @@ describe("resolveWebSearchBackend", () => {
["openai.responses", "openai"],
["anthropic.messages", "anthropic"],
["google.generative-ai", "google"],
+ ["google.generative-ai", null, "gemini-2.5-pro"],
["openrouter.chat", null],
["some-provider", null],
- ] as const)("uses native search for direct provider %s when it has one", (provider, expected) => {
- const model = new MockLanguageModelV3({ provider });
- expect(resolveWebSearchBackend(resolveModelProfile(model))).toBe(expected);
- });
+ ] as const)(
+ "uses native search for direct provider %s when it has one",
+ (provider, expected, modelId?: string) => {
+ const model = new MockLanguageModelV3({ modelId, provider });
+ expect(resolveWebSearchBackend(resolveModelProfile(model))).toBe(expected);
+ },
+ );
it("uses Anthropic webSearch_20250305 to avoid the unsupported beta header", async () => {
const tool = await resolveWebSearchProviderTool("anthropic");
diff --git a/packages/eve/src/harness/provider-tools.ts b/packages/eve/src/harness/provider-tools.ts
index 57a4876f2..bcf9e151d 100644
--- a/packages/eve/src/harness/provider-tools.ts
+++ b/packages/eve/src/harness/provider-tools.ts
@@ -71,26 +71,33 @@ export function resolveWebSearchOutputSchema(
}
/**
- * Determines the web search backend for a model: the selected search provider on AI Gateway
- * (Exa by default), or its fallback when the provider can't serve the model; the provider's
- * native search for direct OpenAI, Anthropic, and Google models; and none (`null`) otherwise.
+ * Determines the web search backend for a model. On AI Gateway it is the selected search
+ * provider (Exa by default); `native` selects the model vendor's own search, or the fallback
+ * when the model has none. Direct models always use their provider's native search. A model
+ * left without a backend gets none (`null`).
*/
export function resolveWebSearchBackend(
profile: ModelProfile,
selection: WebSearchSelection = { provider: "exa" },
): WebSearchBackend | null {
- if (profile.gateway) {
- // OpenAI's hosted search serves only OpenAI models.
- return selection.provider === "openai" && profile.provider !== "openai"
- ? (selection.fallback ?? null)
- : selection.provider;
- }
- return NATIVE_WEB_SEARCH_PROVIDERS.has(profile.provider)
- ? (profile.provider as WebSearchBackend)
- : null;
+ if (!profile.gateway) return resolveNativeWebSearchBackend(profile);
+ if (selection.provider !== "native") return selection.provider;
+ return resolveNativeWebSearchBackend(profile) ?? selection.fallback ?? null;
+}
+
+function resolveNativeWebSearchBackend(profile: ModelProfile): NativeWebSearchBackend | null {
+ if (!NATIVE_WEB_SEARCH_BACKENDS.has(profile.provider)) return null;
+ if (profile.provider === "google" && profile.googleSearchDropsTools) return null;
+ return profile.provider as NativeWebSearchBackend;
}
-const NATIVE_WEB_SEARCH_PROVIDERS: ReadonlySet<string> = new Set(["anthropic", "google", "openai"]);
+type NativeWebSearchBackend = "anthropic" | "google" | "openai";
+
+const NATIVE_WEB_SEARCH_BACKENDS: ReadonlySet<string> = new Set<NativeWebSearchBackend>([
+ "anthropic",
+ "google",
+ "openai",
+]);
/**
* Constructs the AI SDK provider tool for web search based on the resolved
diff --git a/packages/eve/src/internal/authored-definition/schema-backed.test.ts b/packages/eve/src/internal/authored-definition/schema-backed.test.ts
index 0033c3260..9f6213514 100644
--- a/packages/eve/src/internal/authored-definition/schema-backed.test.ts
+++ b/packages/eve/src/internal/authored-definition/schema-backed.test.ts
@@ -115,7 +115,7 @@ describe("normalizeToolDefinition", () => {
expect(entry).toEqual({ kind: "disabled" });
});
- it.each(["exa", "parallel", "browserbase", "openai"] as const)(
+ it.each(["exa", "parallel", "browserbase", "native"] as const)(
"normalizes the %s web search provider",
(provider) => {
expect(normalizeToolDefinition(webSearch({ provider }), FAILURE_MESSAGE)).toEqual({
@@ -127,14 +127,14 @@ describe("normalizeToolDefinition", () => {
it("normalizes a web search fallback provider", () => {
expect(
- normalizeToolDefinition(webSearch({ fallback: "exa", provider: "openai" }), FAILURE_MESSAGE),
- ).toEqual({ kind: "web-search-tool", selection: { fallback: "exa", provider: "openai" } });
+ normalizeToolDefinition(webSearch({ fallback: "exa", provider: "native" }), FAILURE_MESSAGE),
+ ).toEqual({ kind: "web-search-tool", selection: { fallback: "exa", provider: "native" } });
});
it("rejects a fallback that can't serve every Gateway model", () => {
expect(() =>
normalizeToolDefinition(
- { fallback: "openai", kind: "eve:web-search-tool", provider: "openai" },
+ { fallback: "native", kind: "eve:web-search-tool", provider: "native" },
FAILURE_MESSAGE,
),
).toThrow('Expected "fallback" to be one of: exa, parallel, browserbase');
@@ -146,7 +146,7 @@ describe("normalizeToolDefinition", () => {
{ fallback: "parallel", kind: "eve:web-search-tool", provider: "exa" },
FAILURE_MESSAGE,
),
- ).toThrow('"fallback" applies only to provider "openai"');
+ ).toThrow('"fallback" applies only to provider "native"');
});
it("rejects an unsupported web search provider", () => {
diff --git a/packages/eve/src/internal/authored-definition/schema-backed.ts b/packages/eve/src/internal/authored-definition/schema-backed.ts
index 6ec2c3933..1e27a93cf 100644
--- a/packages/eve/src/internal/authored-definition/schema-backed.ts
+++ b/packages/eve/src/internal/authored-definition/schema-backed.ts
@@ -120,9 +120,9 @@ export function normalizeToolDefinition(value: unknown, message: string): Normal
if (record.fallback === undefined) {
return { kind: "web-search-tool", selection: { provider } };
}
- if (provider !== "openai") {
+ if (provider !== "native") {
throw new Error(
- `${message} "fallback" applies only to provider "openai"; provider "${provider}" serves every AI Gateway model.`,
+ `${message} "fallback" applies only to provider "native"; provider "${provider}" serves every AI Gateway model.`,
);
}
const fallback = expectOneOf(
diff --git a/packages/eve/src/shared/web-search.ts b/packages/eve/src/shared/web-search.ts
index 721ad31ac..ba1d86222 100644
--- a/packages/eve/src/shared/web-search.ts
+++ b/packages/eve/src/shared/web-search.ts
@@ -2,10 +2,10 @@
export const WEB_SEARCH_FALLBACK_PROVIDERS = ["exa", "parallel", "browserbase"] as const;
/**
- * Web search providers available through Vercel AI Gateway. `openai` is OpenAI's hosted search,
- * which serves only OpenAI models.
+ * Web search providers available through Vercel AI Gateway. `native` is the model vendor's own
+ * hosted search, which only some vendors' models have.
*/
-export const WEB_SEARCH_PROVIDERS = [...WEB_SEARCH_FALLBACK_PROVIDERS, "openai"] as const;
+export const WEB_SEARCH_PROVIDERS = [...WEB_SEARCH_FALLBACK_PROVIDERS, "native"] as const;
export type WebSearchProvider = (typeof WEB_SEARCH_PROVIDERS)[number];
diff --git a/packages/eve/src/tools/provided/web-search.ts b/packages/eve/src/tools/provided/web-search.ts
index 1c92ac740..6ce1b8bc8 100644
--- a/packages/eve/src/tools/provided/web-search.ts
+++ b/packages/eve/src/tools/provided/web-search.ts
@@ -11,9 +11,12 @@ const WEB_SEARCH_TOOL_KIND = "eve:web-search-tool";
*/
export type WebSearchToolInput =
| {
- /** OpenAI's hosted search, which serves only OpenAI models. */
- readonly provider: "openai";
- /** Provider for non-OpenAI Gateway models. Without it, those models don't get `web_search`. */
+ /**
+ * The model vendor's own hosted search: OpenAI's for OpenAI models, Anthropic's for Claude,
+ * and Google Search grounding for Gemini 3 and later.
+ */
+ readonly provider: "native";
+ /** Provider for Gateway models without native search. Without it, they don't get `web_search`. */
readonly fallback?: WebSearchFallbackProvider;
}
| {
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 6b88b0e26..1709a5574 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -1100,6 +1100,25 @@ importers:
specifier: 'catalog:'
version: 7.0.2
+ e2e/fixtures/agent-native-web-search:
+ dependencies:
+ '@eve-e2e/config':
+ specifier: workspace:*
+ version: link:../e2e-config
+ '@workflow/world-postgres':
+ specifier: 'catalog:'
+ eve:
+ specifier: workspace:*
+ version: link:../../../packages/eve
+ devDependencies:
+ '@types/node':
+ specifier: 'catalog:'
+ version: 24.13.3
+ typescript:
+ specifier: 'catalog:'
+ version: 7.0.2
+
e2e/fixtures/agent-openapi-swagger:
dependencies:
'@eve-e2e/config':