Skip to content

An AI-powered search engine with a generative UI

docs: clarify fetch tool PDF support and type usage

Yoshiki Miuraadc987fd18ec
diff --git a/lib/agents/prompts/search-mode-prompts.ts b/lib/agents/prompts/search-mode-prompts.ts
index 7bd72a05..ba33f279 100644
--- a/lib/agents/prompts/search-mode-prompts.ts
+++ b/lib/agents/prompts/search-mode-prompts.ts
@@ -46,6 +46,8 @@ Fetch tool usage:
- **ONLY use fetch tool when a URL is directly provided by the user in their query**
- Do NOT use fetch to get more details from search results
- This keeps responses fast and efficient
+- **For PDF URLs (ending in .pdf)**: ALWAYS use \`type: "api"\` - regular type will fail on PDFs
+- **For regular web pages**: Use default \`type: "regular"\` for fast HTML fetching
Citation Format (MANDATORY):
[number](#toolCallId) - Always use this EXACT format
@@ -196,6 +198,9 @@ Fetch tool usage:
- Use when you need deeper content analysis beyond search snippets
- Fetch the top 2-3 most relevant/recent URLs for comprehensive coverage
- Especially important for news, current events, and time-sensitive information
+- **For PDF URLs (ending in .pdf)**: ALWAYS use \`type: "api"\` - regular type will fail on PDFs
+- **For complex JavaScript-rendered pages**: Use \`type: "api"\` for better extraction
+- **For regular web pages**: Use default \`type: "regular"\` for fast HTML fetching
When using the ask_question tool:
- Create clear, concise questions
diff --git a/lib/schema/fetch.tsx b/lib/schema/fetch.tsx
index 04d1d0bc..c0d78843 100644
--- a/lib/schema/fetch.tsx
+++ b/lib/schema/fetch.tsx
@@ -7,7 +7,7 @@ export const fetchSchema = z.object({
.enum(['regular', 'api'])
.default('regular')
.describe(
- 'Fetch method: "regular" (default) = fast direct HTML fetch, "api" = advanced extraction for PDFs or complex pages (requires API keys)'
+ 'Fetch method: "regular" (default) = fast direct HTML fetch for simple web pages (does NOT support PDFs), "api" = advanced extraction for PDFs and complex JavaScript-rendered pages (requires Jina or Tavily API keys)'
)
})
diff --git a/lib/tools/fetch.ts b/lib/tools/fetch.ts
index bbaa6270..46fbca0f 100644
--- a/lib/tools/fetch.ts
+++ b/lib/tools/fetch.ts
@@ -82,7 +82,7 @@ async function fetchRegularData(url: string): Promise<SearchResultsType> {
if (error instanceof Error && error.name === 'AbortError') {
throw new Error('Request timeout after 10 seconds')
}
- console.error('Regular fetch error:', error)
+ console.error('Fetch error:', error)
throw error instanceof Error ? error : new Error('Unknown fetch error')
}
}
@@ -115,7 +115,7 @@ async function fetchJinaReaderData(url: string): Promise<SearchResultsType> {
images: []
}
} catch (error) {
- console.error('Jina Reader API error:', error)
+ console.error('API Error:', error)
throw error instanceof Error ? error : new Error('Jina Reader API failed')
}
}
@@ -150,7 +150,7 @@ async function fetchTavilyExtractData(url: string): Promise<SearchResultsType> {
images: []
}
} catch (error) {
- console.error('Tavily Extract API error:', error)
+ console.error('API Error:', error)
throw error instanceof Error
? error
: new Error('Tavily Extract API failed')
@@ -159,7 +159,7 @@ async function fetchTavilyExtractData(url: string): Promise<SearchResultsType> {
export const fetchTool = tool({
description:
- 'Fetch content from any URL. By default uses "regular" type which performs fast, direct HTML fetching without external APIs - ideal for most websites. Only use "api" type when you need: 1) PDF content extraction, 2) Complex JavaScript-rendered pages, 3) Better markdown formatting, 4) Table extraction. The "api" type requires Jina or Tavily API keys.',
+ 'Fetch content from any URL. By default uses "regular" type which performs fast, direct HTML fetching without external APIs - ideal for most websites. IMPORTANT: "regular" type does NOT support PDFs and will fail on PDF URLs. Use "api" type when you need: 1) PDF content extraction (required for .pdf URLs), 2) Complex JavaScript-rendered pages, 3) Better markdown formatting, 4) Table extraction. The "api" type requires Jina or Tavily API keys and uses Jina Reader if available, otherwise falls back to Tavily Extract.',
inputSchema: fetchSchema,
async *execute({ url, type = 'regular' }) {
// Yield initial fetching state