Personal AI, On Personal Devices
Python
feat(tools): add Firecrawl as a web_search engine
Juampif2ccefa7fb27
diff --git a/docs/getting-started/configuration.md b/docs/getting-started/configuration.mdindex 45aaebf1..c7a39c14 100644--- a/docs/getting-started/configuration.md+++ b/docs/getting-started/configuration.md@@ -1132,7 +1132,9 @@ OpenJarvis respects the following environment variables:| `YOUDOTCOM_API_KEY` | API key for the You.com web search engine. Optional — raises the keyless free-tier limits and enables You.com Contents extraction for URL queries. || `SERPLY_API_KEY` | API key for the Serply web search engine, which proxies Google. Optional. When set and no other search key is, `auto` engine selection prefers Serply over the keyless tier. Keys: [serply.io](https://serply.io). || `SERPLY_PROXY_LOCATION` | Two-letter country code asking Serply for that country's Google result set, for example `DE`. Optional. Unset means the API answers from its own default region. See [serply.io/docs](https://serply.io/docs). |-| `OPENJARVIS_WEB_SEARCH_ENGINE` | Web search engine for the `web_search` tool: `auto` (default), `youcom`, `tavily`, `serply`, or `duckduckgo`. |+| `FIRECRAWL_API_KEY` | API key for the Firecrawl web search engine. Optional. When set and no other search key is, `auto` engine selection prefers Firecrawl over the keyless tier, and URL queries return the page or PDF as markdown. Keys: [firecrawl.dev](https://www.firecrawl.dev/app/api-keys?utm_source=openjarvis&utm_medium=integration). |+| `FIRECRAWL_API_URL` | Base URL of the Firecrawl API, for example a [self-hosted Firecrawl](https://docs.firecrawl.dev/contributing/self-host). Optional. Defaults to `https://api.firecrawl.dev`. Any other URL enables the Firecrawl engine without a key, for instances that run without auth. |+| `OPENJARVIS_WEB_SEARCH_ENGINE` | Web search engine for the `web_search` tool: `auto` (default), `youcom`, `tavily`, `serply`, `firecrawl`, or `duckduckgo`. |In the desktop app, you can save `ATLASCLOUD_API_KEY` in **Cloud Models** or**Settings → API Keys** instead of setting an environment variable.diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.mdindex c44e8c26..10075d39 100644--- a/docs/getting-started/quickstart.md+++ b/docs/getting-started/quickstart.md@@ -137,22 +137,24 @@ You get a ChatGPT-like interface with streaming responses, tool use, energy moniWeb search works with no configuration: queries go to the You.com keylessfree tier, which is rate limited per IP and needs no signup. To raise those-limits, or to use Tavily or Serply instead, add `YOUDOTCOM_API_KEY`,-`TAVILY_API_KEY` or `SERPLY_API_KEY` under **Settings → Tools → Web Search**-after the app starts, or export one before starting quickstart:+limits, or to use Tavily, Serply or Firecrawl instead, add `YOUDOTCOM_API_KEY`,+`TAVILY_API_KEY`, `SERPLY_API_KEY` or `FIRECRAWL_API_KEY` under+**Settings → Tools → Web Search** after the app starts, or export one before+starting quickstart:```bashexport YOUDOTCOM_API_KEY="..." # free key: https://you.com/platformexport SERPLY_API_KEY="..." # free tier: https://serply.io+export FIRECRAWL_API_KEY="..." # free tier: https://www.firecrawl.dev/app/api-keys?utm_source=openjarvis&utm_medium=integration./scripts/quickstart.sh```Set `OPENJARVIS_WEB_SEARCH_ENGINE` to pick an engine explicitly — `youcom`,-`tavily`, `serply`, or `duckduckgo`. The default, `auto`, uses Tavily when-`TAVILY_API_KEY` is set, then You.com, then Serply, and settles on the You.com-keyless tier when no key is set at all. DuckDuckGo remains the fallback-for every engine; dropping to it is logged at `WARNING`, since its scraped-results are unranked.+`tavily`, `serply`, `firecrawl`, or `duckduckgo`. The default, `auto`, uses+Tavily when `TAVILY_API_KEY` is set, then You.com, then Serply, then Firecrawl,+and settles on the You.com keyless tier when no key is set at all. DuckDuckGo+remains the fallback for every engine; dropping to it is logged at `WARNING`,+since its scraped results are unranked.Serply proxies Google itself, so Google search operators reach the indexunchanged: `site:arxiv.org` restricts the corpus and `after:2026-01-01` bounds@@ -162,6 +164,15 @@ that country's result set instead of the default region. Keys and requestoptions are documented at [serply.io](https://serply.io) and[serply.io/docs](https://serply.io/docs).+Firecrawl also changes what happens when a query is a URL: instead of the local+HTML strip, the page comes back as markdown, including JavaScript-rendered+pages and PDFs. Set `FIRECRAWL_API_URL` to use a+[self-hosted Firecrawl](https://docs.firecrawl.dev/contributing/self-host)+instead of the hosted API; no key is needed if your instance runs without+auth. Page fetches then run on your instance, and search queries go on to the+search backend it is configured with. Keys:+[firecrawl.dev](https://www.firecrawl.dev/app/api-keys?utm_source=openjarvis&utm_medium=integration).+The script does not automatically source `.env` files. Run `source .env`first if that is where you keep the key. Stop any existing OpenJarvis serverbefore restarting so it inherits the updated environment.@@ -290,7 +301,7 @@ Agents add multi-turn reasoning and tool-calling capabilities. The `orchestrator| `retrieval` | Search the memory store for relevant context. || `llm` | Make sub-queries to another model. || `file_read` | Read files with path validation. |-| `web_search` | Web search via You.com (keyless by default), Tavily, or Serply, with a DuckDuckGo fallback. |+| `web_search` | Web search via You.com (keyless by default), Tavily, Serply, or Firecrawl, with a DuckDuckGo fallback. |### CLI Examplediff --git a/docs/user-guide/deep-research.md b/docs/user-guide/deep-research.mdindex b8803f62..c9d6d988 100644--- a/docs/user-guide/deep-research.md+++ b/docs/user-guide/deep-research.md@@ -160,6 +160,6 @@ Smaller chunk sizes work better for code, where each function or class is a natu**Slow responses** -- The agent makes multiple search passes. Each turn involves a model call. Reduce `max_turns` or use a smaller model (`qwen3.5:4b`) for faster but less thorough results.-**Web search not working** -- The `web_search` tool needs no API key by default; it uses the You.com keyless free tier, which is rate limited per IP. Set `YOUDOTCOM_API_KEY` (free key at [you.com/platform](https://you.com/platform?utm_source=open-jarvis-openjarvis&utm_medium=oss_integration&utm_campaign=2026-09-oss-integrations&utm_content=docs)) for higher limits, or install Tavily with `uv sync --extra tools-search` and set `TAVILY_API_KEY`. Set `SERPLY_API_KEY` ([serply.io](https://serply.io)) to search through Google itself, which is the only engine here that honours Google operators such as `site:` and `after:` in a research query. A `WARNING` naming the failed engine is logged whenever search drops to the DuckDuckGo fallback.+**Web search not working** -- The `web_search` tool needs no API key by default; it uses the You.com keyless free tier, which is rate limited per IP. Set `YOUDOTCOM_API_KEY` (free key at [you.com/platform](https://you.com/platform?utm_source=open-jarvis-openjarvis&utm_medium=oss_integration&utm_campaign=2026-09-oss-integrations&utm_content=docs)) for higher limits, or install Tavily with `uv sync --extra tools-search` and set `TAVILY_API_KEY`. Set `SERPLY_API_KEY` ([serply.io](https://serply.io)) to search through Google itself, which is the only engine here that honours Google operators such as `site:` and `after:` in a research query. Set `FIRECRAWL_API_KEY` ([firecrawl.dev](https://www.firecrawl.dev/app/api-keys?utm_source=openjarvis&utm_medium=integration)) to have URL lookups, including PDFs, return markdown. A `WARNING` naming the failed engine is logged whenever search drops to the DuckDuckGo fallback.**Wrong chunks retrieved** -- Try re-indexing with different chunk sizes. For technical documents, smaller chunks (`256`) often retrieve more precisely. For narrative text, larger chunks (`1024`) preserve more context.diff --git a/src/openjarvis/core/credentials.py b/src/openjarvis/core/credentials.pyindex c81aa1a9..c65f9bd9 100644--- a/src/openjarvis/core/credentials.py+++ b/src/openjarvis/core/credentials.py@@ -33,7 +33,12 @@ def _default_path() -> Path:TOOL_CREDENTIALS: dict[str, list[str]] = {- "web_search": ["TAVILY_API_KEY", "YOUDOTCOM_API_KEY", "SERPLY_API_KEY"],+ "web_search": [+ "TAVILY_API_KEY",+ "YOUDOTCOM_API_KEY",+ "SERPLY_API_KEY",+ "FIRECRAWL_API_KEY",+ ],"get_weather": ["OPENWEATHERMAP_API_KEY"],"image_generate": ["OPENAI_API_KEY"],"slack": ["SLACK_BOT_TOKEN", "SLACK_APP_TOKEN"],@@ -74,7 +79,14 @@ TOOL_CREDENTIALS: dict[str, list[str]] = {# and every listed key only raises limits, changes the engine, or improves# result quality.OPTIONAL_TOOL_CREDENTIALS: dict[str, frozenset[str]] = {- "web_search": frozenset({"TAVILY_API_KEY", "YOUDOTCOM_API_KEY", "SERPLY_API_KEY"}),+ "web_search": frozenset(+ {+ "TAVILY_API_KEY",+ "YOUDOTCOM_API_KEY",+ "SERPLY_API_KEY",+ "FIRECRAWL_API_KEY",+ }+ ),}diff --git a/src/openjarvis/security/data_boundary_audit.py b/src/openjarvis/security/data_boundary_audit.pyindex 82177274..64a502d6 100644--- a/src/openjarvis/security/data_boundary_audit.py+++ b/src/openjarvis/security/data_boundary_audit.py@@ -13,6 +13,7 @@ import osfrom dataclasses import dataclassfrom pathlib import Pathfrom typing import Any, Iterable, Literal+from urllib.parse import urlparsefrom openjarvis.core.credentials import TOOL_CREDENTIALS@@ -70,6 +71,7 @@ API_KEY_ENV_VARS = {"TAVILY_API_KEY": ("Tavily web search", {"tavily", "web_search"}),"YOUDOTCOM_API_KEY": ("You.com web search", {"youcom", "web_search"}),"SERPLY_API_KEY": ("Serply web search", {"serply", "web_search"}),+ "FIRECRAWL_API_KEY": ("Firecrawl web search", {"firecrawl", "web_search"}),}CHANNEL_SECRET_FIELDS: tuple[tuple[str, str, str], ...] = (@@ -953,11 +955,15 @@ def _audit_tool_surfaces(config: Any, builder: _FindingBuilder) -> None:if tools & WEB_SEARCH_TOOLS:destination = _web_search_destination()+ data = "user or agent search query"+ if destination.startswith("Firecrawl"):+ # URL queries are scraped by Firecrawl too, not fetched locally.+ data += " and URL lookups"builder.add(finding_id="web-search-tool-configured",status="warn",title="Web search tool is configured",- potential_data_path=(f"user or agent search query -> {destination}"),+ potential_data_path=(f"{data} -> {destination}"),evidence=(f"configured tool(s) = {_format_tools(tools & WEB_SEARCH_TOOLS)}"),@@ -1077,6 +1083,14 @@ def _audit_tool_surfaces(config: Any, builder: _FindingBuilder) -> None:)+def _firecrawl_self_hosted_url() -> str | None:+ """Return ``FIRECRAWL_API_URL`` when it points away from the hosted API."""+ api_url = (os.environ.get("FIRECRAWL_API_URL") or "").strip().rstrip("/")+ if api_url and api_url != "https://api.firecrawl.dev":+ return api_url+ return None++def _web_search_destination() -> str:"""Name the service ``web_search`` sends queries to, from env keys alone.@@ -1086,7 +1100,7 @@ def _web_search_destination() -> str:source of truth; ``test_data_boundary_audit`` asserts the two agree."""engine = (os.environ.get("OPENJARVIS_WEB_SEARCH_ENGINE") or "auto").strip().lower()- if engine not in {"auto", "youcom", "tavily", "duckduckgo", "serply"}:+ if engine not in {"auto", "youcom", "tavily", "duckduckgo", "serply", "firecrawl"}:engine = "auto"if engine == "auto":if os.environ.get("TAVILY_API_KEY"):@@ -1095,6 +1109,8 @@ def _web_search_destination() -> str:engine = "youcom"elif os.environ.get("SERPLY_API_KEY"):engine = "serply"+ elif os.environ.get("FIRECRAWL_API_KEY") or _firecrawl_self_hosted_url():+ engine = "firecrawl"else:engine = "youcom"@@ -1107,6 +1123,16 @@ def _web_search_destination() -> str:if location:return f"Serply web search API (Google SERP, region {location})"return "Serply web search API (Google SERP)"+ if engine == "firecrawl":+ api_url = _firecrawl_self_hosted_url()+ if api_url:+ # Name the host only: a self-hosted URL can carry credentials.+ host = urlparse(api_url).hostname or "custom URL"+ return (+ f"Firecrawl search API (self-hosted at {host}; search queries go "+ "on to its configured search backend)"+ )+ return "Firecrawl search API"tier = "keyed" if os.environ.get("YOUDOTCOM_API_KEY") else "keyless free tier"return f"You.com web search API ({tier})"diff --git a/src/openjarvis/tools/web_search.py b/src/openjarvis/tools/web_search.pyindex 75558e37..c6abea8d 100644--- a/src/openjarvis/tools/web_search.py+++ b/src/openjarvis/tools/web_search.py@@ -8,6 +8,9 @@ API-backed engine over the DuckDuckGo HTML scrape:* ``TAVILY_API_KEY`` set → Tavily (unchanged for existing installs)* ``YOUDOTCOM_API_KEY`` set → You.com, keyed* ``SERPLY_API_KEY`` set → Serply, a Google SERP proxy+* ``FIRECRAWL_API_KEY`` set, or ``FIRECRAWL_API_URL`` pointing at a+ self-hosted instance → Firecrawl, which also returns URL lookups (including+ PDFs) as markdown* none of them → You.com, keyless free tier (no signup, rate limited per IP)The keyless tier is what makes a fresh install API-backed with zero config.@@ -43,8 +46,29 @@ SERPLY_API_KEY_ENV = "SERPLY_API_KEY"# set. Unset means the API answers from its own default region.SERPLY_LOCATION_ENV = "SERPLY_PROXY_LOCATION"+FIRECRAWL_API_KEY_ENV = "FIRECRAWL_API_KEY"+# Base URL of the Firecrawl API. Point it at a self-hosted Firecrawl to run+# page fetches on an instance you operate; that instance still sends search+# queries to its own configured search backend.+FIRECRAWL_API_URL_ENV = "FIRECRAWL_API_URL"+FIRECRAWL_DEFAULT_API_URL = "https://api.firecrawl.dev"+FIRECRAWL_ORIGIN = "openjarvis"+# PDFs are billed per page, and only the first max_chars of the markdown are+# kept, so parsing stops after a few pages.+FIRECRAWL_PDF_MAX_PAGES = 3+# Server-side scrape timeout, below the client timeout so a slow page fails on+# Firecrawl's side first instead of being abandoned mid-scrape.+FIRECRAWL_SCRAPE_TIMEOUT_MS = 45000+# Search descriptions can run to several thousand characters of page text, so+# each one is cut to a snippet that a small local model can take five of.+FIRECRAWL_SNIPPET_CHARS = 500+FIRECRAWL_KEYS_URL = (+ "https://www.firecrawl.dev/app/api-keys"+ "?utm_source=openjarvis&utm_medium=integration"+)+ENGINE_ENV = "OPENJARVIS_WEB_SEARCH_ENGINE"-ENGINES = ("auto", "youcom", "tavily", "duckduckgo", "serply")+ENGINES = ("auto", "youcom", "tavily", "duckduckgo", "serply", "firecrawl")# Identifies OpenJarvis to You.com. The keyless tier carries no API key, so the# User-Agent is the only attribution signal; sent to You.com hosts only.@@ -89,6 +113,8 @@ class WebSearchTool(BaseTool):youcom_api_key: str | None = None,serply_api_key: str | None = None,serply_location: str | None = None,+ firecrawl_api_key: str | None = None,+ firecrawl_api_url: str | None = None,):"""Configure the search engine.@@ -102,6 +128,19 @@ class WebSearchTool(BaseTool):self._youcom_api_key = youcom_api_key or os.environ.get(YOUCOM_API_KEY_ENV)self._serply_api_key = serply_api_key or os.environ.get(SERPLY_API_KEY_ENV)self._serply_location = serply_location or os.environ.get(SERPLY_LOCATION_ENV)+ self._firecrawl_api_key = firecrawl_api_key or os.environ.get(+ FIRECRAWL_API_KEY_ENV+ )+ self._firecrawl_api_url = (+ firecrawl_api_url+ or os.environ.get(FIRECRAWL_API_URL_ENV)+ or FIRECRAWL_DEFAULT_API_URL+ ).rstrip("/")+ # Self-hosted instances usually run without auth, so a custom URL+ # enables the engine even when no key is set.+ self._firecrawl_self_hosted = (+ self._firecrawl_api_url != FIRECRAWL_DEFAULT_API_URL+ )self._max_results = max_resultsrequested = (engine or os.environ.get(ENGINE_ENV) or "auto").strip().lower()@@ -119,9 +158,10 @@ class WebSearchTool(BaseTool):Keyed engines come first, in the order they were added, so everyinstall that already had a key resolves exactly as it did before:- Tavily, then keyed You.com, then Serply. The keyless You.com tier is- last and remains the zero-config API-backed path, which means a Serply- key only ever wins over having no key at all.+ Tavily, then keyed You.com, then Serply, then Firecrawl. The keyless+ You.com tier is last and remains the zero-config API-backed path, which+ means a Serply or Firecrawl key only ever wins over having no key at+ all."""if self._engine != "auto":return self._engine@@ -131,6 +171,8 @@ class WebSearchTool(BaseTool):return "youcom"if self._serply_api_key:return "serply"+ if self._firecrawl_api_key or self._firecrawl_self_hosted:+ return "firecrawl"return "youcom"@property@@ -164,6 +206,7 @@ class WebSearchTool(BaseTool):"TAVILY_API_KEY",YOUCOM_API_KEY_ENV,SERPLY_API_KEY_ENV,+ FIRECRAWL_API_KEY_ENV,],"engine": engine,"engines": list(ENGINES),@@ -435,6 +478,139 @@ class WebSearchTool(BaseTool):metadata=metadata,)+ def _firecrawl_headers(self) -> dict[str, str]:+ headers = {"Accept": "application/json", "Content-Type": "application/json"}+ if self._firecrawl_api_key:+ headers["Authorization"] = f"Bearer {self._firecrawl_api_key}"+ return headers++ @staticmethod+ def _format_firecrawl_results(payload: dict[str, Any]) -> tuple[str, int]:+ """Render a Firecrawl search payload in the shared result format.++ Only snippets are returned, each description cut to+ :data:`FIRECRAWL_SNIPPET_CHARS`. Search can also scrape every result+ page, but full pages per result would crowd out a small local model's+ context, so URL lookups are the place to read a page in full.+ """+ data = payload.get("data") if isinstance(payload, dict) else None+ if not isinstance(data, dict):+ data = {}+ parts: list[str] = []+ count = 0+ for item in data.get("web") or []:+ title = item.get("title") or "Untitled"+ url = item.get("url", "")+ content = item.get("description") or ""+ if len(content) > FIRECRAWL_SNIPPET_CHARS:+ content = content[:FIRECRAWL_SNIPPET_CHARS].rstrip() + "…"+ parts.append(f"### {title}\nSource: {url}\nSummary: {content}")+ count += 1+ return "\n\n---\n\n".join(parts), count++ def _firecrawl_search(self, query: str, max_results: int) -> ToolResult:+ """Search via Firecrawl, or the instance at ``FIRECRAWL_API_URL``."""+ import httpx++ if not self._firecrawl_api_key and not self._firecrawl_self_hosted:+ raise _WebSearchEngineError(f"{FIRECRAWL_API_KEY_ENV} is not set")+ try:+ response = httpx.post(+ f"{self._firecrawl_api_url}/v2/search",+ json={+ "query": query,+ "limit": max_results,+ "origin": FIRECRAWL_ORIGIN,+ },+ headers=self._firecrawl_headers(),+ timeout=30.0,+ )+ response.raise_for_status()+ payload = response.json()+ except httpx.HTTPStatusError as exc:+ status = exc.response.status_code+ if status == 401:+ raise _WebSearchEngineError(+ f"Firecrawl rejected the credential in {FIRECRAWL_API_KEY_ENV}"+ ) from exc+ if status == 402:+ raise _WebSearchEngineError(+ "Firecrawl credits are exhausted for this key. Check the "+ f"plan or create a new key at {FIRECRAWL_KEYS_URL}"+ ) from exc+ if status == 429:+ raise _WebSearchEngineError(+ "Firecrawl rate limit reached (HTTP 429)"+ ) from exc+ raise _WebSearchEngineError(+ f"Firecrawl search failed with HTTP {status}"+ ) from exc+ except (httpx.HTTPError, ValueError) as exc:+ raise _WebSearchEngineError(f"Firecrawl search failed: {exc}") from exc++ formatted, count = self._format_firecrawl_results(payload)+ metadata: dict[str, Any] = {"num_results": count, "engine": "firecrawl"}+ if isinstance(payload, dict) and "creditsUsed" in payload:+ metadata["credits"] = payload["creditsUsed"]+ return ToolResult(+ tool_name="web_search",+ content=formatted or "No results found.",+ success=True,+ metadata=metadata,+ )++ def _firecrawl_extract(self, url: str, max_chars: int = 6000) -> str | None:+ """Return Firecrawl-scraped markdown for ``url``, or ``None``.++ Firecrawl renders JavaScript and parses PDFs, which the regex HTML+ strip in :meth:`_fetch_url` cannot. Output is truncated at the same+ ``max_chars`` as the local path so a long PDF cannot flood the+ context, and PDF parsing stops after :data:`FIRECRAWL_PDF_MAX_PAGES`+ pages since PDFs are billed per page. Any failure returns ``None`` and+ the caller falls back to the local fetch.+ """+ if not self._firecrawl_api_key and not self._firecrawl_self_hosted:+ return None++ import httpx++ try:+ response = httpx.post(+ f"{self._firecrawl_api_url}/v2/scrape",+ json={+ "url": url,+ "formats": ["markdown"],+ "onlyMainContent": True,+ "parsers": [{"type": "pdf", "maxPages": FIRECRAWL_PDF_MAX_PAGES}],+ "timeout": FIRECRAWL_SCRAPE_TIMEOUT_MS,+ "origin": FIRECRAWL_ORIGIN,+ },+ headers=self._firecrawl_headers(),+ timeout=60.0,+ )+ response.raise_for_status()+ payload = response.json()+ except (httpx.HTTPError, ValueError) as exc:+ reason = type(exc).__name__+ if isinstance(exc, httpx.HTTPStatusError):+ reason = f"HTTP {exc.response.status_code}"+ logger.warning(+ "Firecrawl scrape failed for %s (%s); falling back to local "+ "HTML extraction",+ url,+ reason,+ )+ return None++ data = payload.get("data") if isinstance(payload, dict) else None+ markdown = data.get("markdown") if isinstance(data, dict) else None+ if not markdown:+ return None+ text = str(markdown)+ if len(text) > max_chars:+ text = text[:max_chars] + "\n\n[Content truncated]"+ return text+def _duckduckgo_search(self, query: str, max_results: int) -> str:"""Search using DuckDuckGo as fallback."""from ddgs import DDGS@@ -463,6 +639,26 @@ class WebSearchTool(BaseTool):# If the query contains a URL, fetch it directly instead of searchingurl = self._extract_url(query) if not self._is_url(query) else query.strip()if url:+ if self._resolve_engine() == "firecrawl":+ ssrf_error = check_ssrf(self._normalize_url(url))+ if ssrf_error:+ return ToolResult(+ tool_name="web_search",+ content=f"Failed to fetch URL: {ssrf_error}",+ success=False,+ )+ extracted = self._firecrawl_extract(url)+ if extracted:+ return ToolResult(+ tool_name="web_search",+ content=extracted,+ success=True,+ metadata={+ "url": url,+ "mode": "fetch",+ "extractor": "firecrawl_scrape",+ },+ )if self._resolve_engine() == "youcom":ssrf_error = check_ssrf(self._normalize_url(url))if ssrf_error:@@ -509,6 +705,8 @@ class WebSearchTool(BaseTool):return self._youcom_search(query, max_results)if engine == "serply":return self._serply_search(query, max_results)+ if engine == "firecrawl":+ return self._firecrawl_search(query, max_results)return self._tavily_search(query, max_results)except _WebSearchEngineError as exc:reason = str(exc)diff --git a/tests/core/test_credentials.py b/tests/core/test_credentials.pyindex 603aaed6..fbd4e4cd 100644--- a/tests/core/test_credentials.py+++ b/tests/core/test_credentials.py@@ -187,6 +187,7 @@ class TestOptionalCredentials:assert is_credential_optional("web_search", "TAVILY_API_KEY") is Trueassert is_credential_optional("web_search", "YOUDOTCOM_API_KEY") is Trueassert is_credential_optional("web_search", "SERPLY_API_KEY") is True+ assert is_credential_optional("web_search", "FIRECRAWL_API_KEY") is Truedef test_other_tool_keys_stay_required(self):from openjarvis.core.credentials import is_credential_optional@@ -226,3 +227,15 @@ class TestOptionalCredentials:assert load_credentials(path=path)["web_search"]["SERPLY_API_KEY"] == ("srp-key")++ def test_firecrawl_key_can_be_persisted(self, tmp_path, monkeypatch):+ """Same guard for the Firecrawl key: declared, so the Settings UI can+ store it, and still not required for web_search to run."""+ from openjarvis.core.credentials import load_credentials, save_credential++ path = tmp_path / "credentials.toml"+ monkeypatch.delenv("FIRECRAWL_API_KEY", raising=False)+ save_credential("web_search", "FIRECRAWL_API_KEY", "fc-test", path=path)+ assert load_credentials(path=path)["web_search"]["FIRECRAWL_API_KEY"] == (+ "fc-test"+ )diff --git a/tests/security/test_data_boundary_audit.py b/tests/security/test_data_boundary_audit.pyindex 2214b4da..c700d1c2 100644--- a/tests/security/test_data_boundary_audit.py+++ b/tests/security/test_data_boundary_audit.py@@ -1,5 +1,6 @@from __future__ import annotations+import itertoolsimport osimport pytest@@ -337,6 +338,23 @@ def test_web_search_tool_and_tavily_env_are_correlated(assert "secret-value" not in str(report.to_dict(show_paths=True))+def test_firecrawl_web_search_data_path_includes_url_lookups(tmp_path, monkeypatch):+ """With Firecrawl, URL queries are scraped by the service rather than+ fetched locally, so the finding has to say URLs leave the machine too."""+ config = _low_noise_config()+ config.tools.enabled = "web_search"+ for key in ("TAVILY_API_KEY", "YOUDOTCOM_API_KEY", "SERPLY_API_KEY"):+ monkeypatch.delenv(key, raising=False)+ monkeypatch.delenv("OPENJARVIS_WEB_SEARCH_ENGINE", raising=False)+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")++ report = build_data_boundary_report(config, tmp_path)++ findings = {finding.id: finding for finding in report.findings}+ path = findings["web-search-tool-configured"].potential_data_path+ assert path == "user or agent search query and URL lookups -> Firecrawl search API"++def test_mcp_servers_are_flagged_when_non_empty(tmp_path):config = _low_noise_config()config.tools.mcp.enabled = True@@ -1354,6 +1372,8 @@ class TestWebSearchDestination:"YOUDOTCOM_API_KEY","SERPLY_API_KEY","SERPLY_PROXY_LOCATION",+ "FIRECRAWL_API_KEY",+ "FIRECRAWL_API_URL","OPENJARVIS_WEB_SEARCH_ENGINE",):monkeypatch.delenv(key, raising=False)@@ -1403,6 +1423,33 @@ class TestWebSearchDestination:monkeypatch.setenv("SERPLY_PROXY_LOCATION", "DE")assert "region DE" in _web_search_destination()+ def test_firecrawl_is_named(self, monkeypatch):+ from openjarvis.security.data_boundary_audit import _web_search_destination++ self._clear(monkeypatch)+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ assert _web_search_destination() == "Firecrawl search API"++ def test_firecrawl_self_hosted_names_only_the_host(self, monkeypatch):+ """A self-hosted instance keeps queries off the hosted API, so the+ audit says so, without echoing a URL that may carry credentials."""+ from openjarvis.security.data_boundary_audit import _web_search_destination++ self._clear(monkeypatch)+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ monkeypatch.setenv("FIRECRAWL_API_URL", "https://user:[email protected]:3002/")+ destination = _web_search_destination()+ assert destination.startswith("Firecrawl search API (self-hosted at ")+ assert "fc.internal;" in destination+ assert "pw" not in destination++ def test_keyless_self_hosted_firecrawl_is_named(self, monkeypatch):+ from openjarvis.security.data_boundary_audit import _web_search_destination++ self._clear(monkeypatch)+ monkeypatch.setenv("FIRECRAWL_API_URL", "http://localhost:3002")+ assert "self-hosted at localhost" in _web_search_destination()+def test_matches_the_tool_resolution(self, monkeypatch):"""Guard against the audit's copy of the precedence rule drifting fromWebSearchTool._resolve_engine, which is the source of truth."""@@ -1414,23 +1461,34 @@ class TestWebSearchDestination:"tavily": "Tavily","duckduckgo": "DuckDuckGo","serply": "Serply",+ "firecrawl": "Firecrawl",}- engines = (None, "auto", "youcom", "tavily", "duckduckgo", "serply")- for tavily in (None, "tvly-key"):- for youcom in (None, "ydc-key"):- for serply in (None, "srp-key"):- for engine in engines:- self._clear(monkeypatch)- if tavily:- monkeypatch.setenv("TAVILY_API_KEY", tavily)- if youcom:- monkeypatch.setenv("YOUDOTCOM_API_KEY", youcom)- if serply:- monkeypatch.setenv("SERPLY_API_KEY", serply)- if engine:- monkeypatch.setenv("OPENJARVIS_WEB_SEARCH_ENGINE", engine)- resolved = WebSearchTool()._resolve_engine()- assert labels[resolved] in _web_search_destination(), (- f"tavily={tavily} youcom={youcom} serply={serply} "- f"engine={engine}"- )+ engines = (+ None,+ "auto",+ "youcom",+ "tavily",+ "duckduckgo",+ "serply",+ "firecrawl",+ )+ keys = (+ ("TAVILY_API_KEY", "tvly-key"),+ ("YOUDOTCOM_API_KEY", "ydc-key"),+ ("SERPLY_API_KEY", "srp-key"),+ ("FIRECRAWL_API_KEY", "fc-test"),+ ("FIRECRAWL_API_URL", "http://localhost:3002"),+ )+ for present in itertools.product((False, True), repeat=len(keys)):+ for engine in engines:+ self._clear(monkeypatch)+ for (name, value), is_set in zip(keys, present):+ if is_set:+ monkeypatch.setenv(name, value)+ if engine:+ monkeypatch.setenv("OPENJARVIS_WEB_SEARCH_ENGINE", engine)+ resolved = WebSearchTool()._resolve_engine()+ set_vars = [name for (name, _), is_set in zip(keys, present) if is_set]+ assert labels[resolved] in _web_search_destination(), (+ f"set={set_vars} engine={engine}"+ )diff --git a/tests/server/test_tools_endpoint.py b/tests/server/test_tools_endpoint.pyindex 697dce3d..bd1ff32d 100644--- a/tests/server/test_tools_endpoint.py+++ b/tests/server/test_tools_endpoint.py@@ -86,8 +86,8 @@ def test_weather_tool_picker_recognizes_connector_credential(tmp_path, monkeypatdef test_tool_credentials_browser_lifecycle(tmp_path, monkeypatch):"""The browser API can save, report, and remove a Tavily key.- ``web_search`` declares You.com and Serply keys alongside it, so status- reports all three.+ ``web_search`` declares You.com, Serply and Firecrawl keys alongside it, so+ status reports all four."""from fastapi import FastAPIfrom fastapi.testclient import TestClient@@ -98,6 +98,7 @@ def test_tool_credentials_browser_lifecycle(tmp_path, monkeypatch):monkeypatch.delenv("TAVILY_API_KEY", raising=False)monkeypatch.delenv("YOUDOTCOM_API_KEY", raising=False)monkeypatch.delenv("SERPLY_API_KEY", raising=False)+ monkeypatch.delenv("FIRECRAWL_API_KEY", raising=False)app = FastAPI()tools_router = create_agent_manager_router(MagicMock())[3]app.include_router(tools_router)@@ -113,6 +114,7 @@ def test_tool_credentials_browser_lifecycle(tmp_path, monkeypatch):"TAVILY_API_KEY": True,"YOUDOTCOM_API_KEY": False,"SERPLY_API_KEY": False,+ "FIRECRAWL_API_KEY": False,}deleted = client.delete(@@ -124,6 +126,7 @@ def test_tool_credentials_browser_lifecycle(tmp_path, monkeypatch):"TAVILY_API_KEY": False,"YOUDOTCOM_API_KEY": False,"SERPLY_API_KEY": False,+ "FIRECRAWL_API_KEY": False,}diff --git a/tests/tools/test_web_search.py b/tests/tools/test_web_search.pyindex abd219ae..9754b58b 100644--- a/tests/tools/test_web_search.py+++ b/tests/tools/test_web_search.py@@ -24,6 +24,13 @@ def _no_ambient_serply_key(monkeypatch):monkeypatch.delenv("SERPLY_PROXY_LOCATION", raising=False)[email protected](autouse=True)+def _no_ambient_firecrawl_key(monkeypatch):+ """Same guard as above for the Firecrawl env, for the same reason (#972)."""+ monkeypatch.delenv("FIRECRAWL_API_KEY", raising=False)+ monkeypatch.delenv("FIRECRAWL_API_URL", raising=False)++class TestWebSearchTool:def test_spec_name_and_category(self):tool = WebSearchTool(api_key="test-key")@@ -805,6 +812,39 @@ class TestEngineSelection:monkeypatch.delenv("TAVILY_API_KEY", raising=False)assert WebSearchTool()._resolve_engine() == "youcom"+ def test_auto_prefers_firecrawl_over_the_keyless_tier(self, monkeypatch):+ """A Firecrawl key only ever wins over having no key at all."""+ monkeypatch.delenv("TAVILY_API_KEY", raising=False)+ monkeypatch.delenv("YOUDOTCOM_API_KEY", raising=False)+ monkeypatch.delenv("OPENJARVIS_WEB_SEARCH_ENGINE", raising=False)+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ assert WebSearchTool()._resolve_engine() == "firecrawl"++ def test_firecrawl_never_displaces_an_existing_keyed_engine(self, monkeypatch):+ """Adding a Firecrawl key must not change where an install already goes."""+ monkeypatch.delenv("OPENJARVIS_WEB_SEARCH_ENGINE", raising=False)+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")++ monkeypatch.setenv("TAVILY_API_KEY", "tvly-x")+ monkeypatch.setenv("YOUDOTCOM_API_KEY", "ydc-key")+ monkeypatch.setenv("SERPLY_API_KEY", "srp-x")+ assert WebSearchTool()._resolve_engine() == "tavily"++ monkeypatch.delenv("TAVILY_API_KEY", raising=False)+ assert WebSearchTool()._resolve_engine() == "youcom"++ monkeypatch.delenv("YOUDOTCOM_API_KEY", raising=False)+ assert WebSearchTool()._resolve_engine() == "serply"++ def test_auto_uses_a_keyless_self_hosted_firecrawl(self, monkeypatch):+ """Self-hosted instances usually run without auth, so pointing+ FIRECRAWL_API_URL away from the hosted API is enough to opt in."""+ monkeypatch.delenv("TAVILY_API_KEY", raising=False)+ monkeypatch.delenv("YOUDOTCOM_API_KEY", raising=False)+ monkeypatch.delenv("OPENJARVIS_WEB_SEARCH_ENGINE", raising=False)+ monkeypatch.setenv("FIRECRAWL_API_URL", "http://localhost:3002")+ assert WebSearchTool()._resolve_engine() == "firecrawl"+def test_explicit_engine_overrides_key_presence(self, monkeypatch):monkeypatch.setenv("TAVILY_API_KEY", "tvly-x")assert WebSearchTool(engine="youcom")._resolve_engine() == "youcom"@@ -833,6 +873,8 @@ class TestEngineSelection:assert "YOUDOTCOM_API_KEY" in spec.metadata["optional_api_keys"]assert "SERPLY_API_KEY" in spec.metadata["optional_api_keys"]assert "serply" in spec.metadata["engines"]+ assert "FIRECRAWL_API_KEY" in spec.metadata["optional_api_keys"]+ assert "firecrawl" in spec.metadata["engines"]# Backwards compatible for readers of the old single-key field.assert spec.metadata["requires_api_key"] == "TAVILY_API_KEY"@@ -1141,6 +1183,332 @@ class TestSerplySearch:assert "SERPLY_API_KEY" not in result.metadata["fallback_reason"]+class TestFirecrawlSearch:+ """The Firecrawl engine, exercised against mocked HTTP."""++ PAYLOAD = {+ "success": True,+ "data": {+ "web": [+ {+ "title": "Result A",+ "url": "https://example.com/a",+ "description": "Desc A",+ "position": 1,+ },+ {+ "title": "Result B",+ "url": "https://example.com/b",+ "description": "Desc B",+ "position": 2,+ },+ ]+ },+ }++ def _mock_post(self, monkeypatch, payload=None, status=200):+ import httpx++ calls = {}++ def _post(url, **kwargs):+ calls["url"] = url+ calls["json"] = kwargs.get("json")+ calls["headers"] = kwargs.get("headers")+ resp = MagicMock()+ resp.status_code = status+ resp.json.return_value = payload if payload is not None else self.PAYLOAD+ if status >= 400:+ resp.text = "error body"+ resp.raise_for_status.side_effect = httpx.HTTPStatusError(+ f"HTTP {status}", request=MagicMock(), response=resp+ )+ else:+ resp.raise_for_status = MagicMock()+ return resp++ monkeypatch.setattr(httpx, "post", _post)+ return calls++ def _mock_ddgs(self, monkeypatch):+ mock_ddgs = MagicMock()+ mock_ddgs.text.return_value = []+ mock_module = MagicMock()+ mock_module.DDGS.return_value = mock_ddgs+ monkeypatch.setitem(sys.modules, "ddgs", mock_module)++ def test_endpoint_auth_and_origin(self, monkeypatch):+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ calls = self._mock_post(monkeypatch)++ result = WebSearchTool(engine="firecrawl").execute(query="test query")++ assert result.success is True+ assert calls["url"] == "https://api.firecrawl.dev/v2/search"+ assert calls["headers"]["Authorization"] == "Bearer fc-test"+ assert calls["json"]["origin"] == "openjarvis"+ assert result.metadata["engine"] == "firecrawl"++ def test_result_format_matches_the_other_engines(self, monkeypatch):+ """Same labeled Source/Summary shape agents already parse (#390)."""+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ self._mock_post(monkeypatch)++ result = WebSearchTool(engine="firecrawl").execute(query="test query")++ assert "### Result A" in result.content+ assert "Source: https://example.com/a" in result.content+ assert "Summary: Desc A" in result.content+ assert "### Result B" in result.content+ assert result.metadata["num_results"] == 2++ def test_max_results_passed_as_limit(self, monkeypatch):+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ calls = self._mock_post(monkeypatch)+ WebSearchTool(engine="firecrawl", max_results=3).execute(+ query="q", max_results=7+ )+ assert calls["json"] == {"query": "q", "limit": 7, "origin": "openjarvis"}++ def test_self_hosted_api_url(self, monkeypatch):+ """FIRECRAWL_API_URL points search at a self-hosted instance."""+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ monkeypatch.setenv("FIRECRAWL_API_URL", "http://localhost:3002/")+ calls = self._mock_post(monkeypatch)++ WebSearchTool(engine="firecrawl").execute(query="q")++ assert calls["url"] == "http://localhost:3002/v2/search"++ def test_keyless_self_hosted_sends_no_auth_header(self, monkeypatch):+ monkeypatch.setenv("FIRECRAWL_API_URL", "http://localhost:3002")+ calls = self._mock_post(monkeypatch)++ result = WebSearchTool(engine="firecrawl").execute(query="q")++ assert result.metadata["engine"] == "firecrawl"+ assert "Authorization" not in calls["headers"]++ def test_long_descriptions_are_capped(self, monkeypatch):+ """Descriptions can carry thousands of characters of page text; five+ of them must still fit a small model's context."""+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ payload = {+ "data": {+ "web": [+ {+ "title": "Long",+ "url": "https://e.com",+ "description": "x" * 10000,+ }+ ]+ }+ }+ self._mock_post(monkeypatch, payload=payload)++ result = WebSearchTool(engine="firecrawl").execute(query="q")++ assert "x" * 500 + "…" in result.content+ assert "x" * 501 not in result.content++ def test_credits_are_reported(self, monkeypatch):+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ self._mock_post(monkeypatch, payload={**self.PAYLOAD, "creditsUsed": 2})+ result = WebSearchTool(engine="firecrawl").execute(query="q")+ assert result.metadata["credits"] == 2++ def test_missing_key_is_named_without_a_request(self, monkeypatch):+ """An explicit firecrawl engine with no key against the hosted API+ should say the key is missing, not that it was rejected."""+ import httpx++ monkeypatch.setattr(+ httpx, "post", MagicMock(side_effect=AssertionError("must not be called"))+ )+ self._mock_ddgs(monkeypatch)++ result = WebSearchTool(engine="firecrawl").execute(query="q")++ assert result.metadata["fallback_reason"] == "FIRECRAWL_API_KEY is not set"++ def test_empty_results(self, monkeypatch):+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ self._mock_post(monkeypatch, payload={"success": True, "data": {"web": []}})+ result = WebSearchTool(engine="firecrawl").execute(query="q")+ assert result.success is True+ assert result.content == "No results found."++ def test_rejected_key_names_the_env_var(self, monkeypatch):+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-bad")+ self._mock_post(monkeypatch, status=401)+ self._mock_ddgs(monkeypatch)++ result = WebSearchTool(engine="firecrawl").execute(query="q")++ assert result.metadata["fallback_from"] == "firecrawl"+ assert result.metadata["degraded"] is True+ assert "FIRECRAWL_API_KEY" in result.metadata["fallback_reason"]++ def test_exhausted_credits_point_to_the_key_page(self, monkeypatch):+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ self._mock_post(monkeypatch, status=402)+ self._mock_ddgs(monkeypatch)++ result = WebSearchTool(engine="firecrawl").execute(query="q")++ reason = result.metadata["fallback_reason"]+ assert "credits" in reason+ assert "firecrawl.dev/app/api-keys" in reason++ def test_rate_limit_is_named(self, monkeypatch):+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ self._mock_post(monkeypatch, status=429)+ self._mock_ddgs(monkeypatch)++ result = WebSearchTool(engine="firecrawl").execute(query="q")++ assert "rate limit" in result.metadata["fallback_reason"]++ def test_other_http_error_reports_the_status(self, monkeypatch):+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ self._mock_post(monkeypatch, status=500)+ self._mock_ddgs(monkeypatch)++ result = WebSearchTool(engine="firecrawl").execute(query="q")++ assert "HTTP 500" in result.metadata["fallback_reason"]+ assert "FIRECRAWL_API_KEY" not in result.metadata["fallback_reason"]+++class TestFirecrawlScrapeExtraction:+ def _mock_ssrf(self, monkeypatch):+ import openjarvis.tools.web_search as _ws++ monkeypatch.setattr(_ws, "check_ssrf", lambda url: None)++ def _mock_scrape(self, monkeypatch, markdown):+ import httpx++ resp = MagicMock()+ resp.json.return_value = {"success": True, "data": {"markdown": markdown}}+ resp.raise_for_status = MagicMock()+ post = MagicMock(return_value=resp)+ monkeypatch.setattr(httpx, "post", post)+ return post++ def test_scrape_used_for_url_queries(self, monkeypatch):+ """URL queries come back as markdown instead of the regex HTML strip."""+ self._mock_ssrf(monkeypatch)+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ post = self._mock_scrape(monkeypatch, "# Heading\n\nBody text.")++ result = WebSearchTool(engine="firecrawl").execute(+ query="https://example.com/a.pdf"+ )++ assert result.success is True+ assert result.content == "# Heading\n\nBody text."+ assert result.metadata["extractor"] == "firecrawl_scrape"+ assert post.call_args.args[0] == "https://api.firecrawl.dev/v2/scrape"+ assert post.call_args.kwargs["json"] == {+ "url": "https://example.com/a.pdf",+ "formats": ["markdown"],+ "onlyMainContent": True,+ "parsers": [{"type": "pdf", "maxPages": 3}],+ "timeout": 45000,+ "origin": "openjarvis",+ }++ def test_scrape_output_is_truncated_like_the_local_fetch(self, monkeypatch):+ """A long PDF must not flood a small model's context."""+ self._mock_ssrf(monkeypatch)+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ self._mock_scrape(monkeypatch, "x" * 10000)++ result = WebSearchTool(engine="firecrawl").execute(+ query="https://example.com/a"+ )++ assert result.content.startswith("x" * 6000)+ assert result.content.endswith("[Content truncated]")+ assert len(result.content) < 6100++ def test_keyless_self_hosted_scrape(self, monkeypatch):+ self._mock_ssrf(monkeypatch)+ monkeypatch.setenv("FIRECRAWL_API_URL", "http://localhost:3002")+ post = self._mock_scrape(monkeypatch, "# Self-hosted")++ result = WebSearchTool(engine="firecrawl").execute(+ query="https://example.com/a"+ )++ assert result.content == "# Self-hosted"+ assert post.call_args.args[0] == "http://localhost:3002/v2/scrape"+ assert "Authorization" not in post.call_args.kwargs["headers"]++ def test_url_query_without_key_uses_local_fetch(self, monkeypatch):+ """Explicit firecrawl with no key and the hosted URL never calls the+ API for URL queries; the local fetch handles them as before."""+ self._mock_ssrf(monkeypatch)+ import httpx++ monkeypatch.setattr(+ httpx, "post", MagicMock(side_effect=AssertionError("must not be called"))+ )+ resp = MagicMock()+ resp.text = "<html><body>Local text</body></html>"+ resp.headers = {"content-type": "text/html"}+ resp.raise_for_status = MagicMock()+ monkeypatch.setattr(httpx.Client, "get", MagicMock(return_value=resp))++ result = WebSearchTool(engine="firecrawl").execute(+ query="https://example.com/a"+ )++ assert result.metadata["extractor"] == "local"++ def test_scrape_failure_falls_back_to_local_fetch(self, monkeypatch):+ self._mock_ssrf(monkeypatch)+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ import httpx++ monkeypatch.setattr(+ httpx, "post", MagicMock(side_effect=httpx.ConnectError("down"))+ )+ resp = MagicMock()+ resp.text = "<html><body>Local text</body></html>"+ resp.headers = {"content-type": "text/html"}+ resp.raise_for_status = MagicMock()+ monkeypatch.setattr(httpx.Client, "get", MagicMock(return_value=resp))++ result = WebSearchTool(engine="firecrawl").execute(+ query="https://example.com/a"+ )++ assert result.success is True+ assert "Local text" in result.content+ assert result.metadata["extractor"] == "local"++ def test_url_branch_ssrf_blocked_before_scrape_call(self, monkeypatch):+ """The scrape upgrade must not become an SSRF bypass."""+ monkeypatch.setenv("FIRECRAWL_API_KEY", "fc-test")+ import openjarvis.tools.web_search as _ws++ monkeypatch.setattr(_ws, "check_ssrf", lambda url: "private IP blocked")+ import httpx++ monkeypatch.setattr(+ httpx, "post", MagicMock(side_effect=AssertionError("must not be called"))+ )++ result = WebSearchTool(engine="firecrawl").execute(+ query="http://169.254.169.254/metadata"+ )++ assert result.success is False+ assert "private IP blocked" in result.content++class TestFallbackVisibility:def test_fallback_is_logged_at_warning(self, monkeypatch, caplog):"""Regression for the silent-degradation half of #923: a typo'd key