~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
refactor(search): remove direct DuckDuckGo integration (#7310)
LearningCircuit8f2cfce760e3
diff --git a/changelog.d/7310.removal.md b/changelog.d/7310.removal.mdnew file mode 100644index 0000000000..fc4fe82a74--- /dev/null+++ b/changelog.d/7310.removal.md@@ -0,0 +1 @@+Remove LDR's dormant DuckDuckGo adapter and direct search dependency. Wayback no longer silently queries another provider to discover URLs. It keeps its lookup of the host of each lowercase `http://` or `https://` URL in a query, or of a query that is only a bare domain, and tells research agents to discover URLs through a configured provider first. Any other query now returns no results, including a dotted topic, a domain with a path but no scheme, a URL with an uppercase scheme or a bracketed IPv6 host, and a bare domain with a punycode top-level domain such as `.xn--p1ai` (give it as a full URL); Wayback used to look a dotted query up as a URL when discovery found no URLs. A URL with a user name or password before the host is not supported: the start of the user name is looked up instead of the host. SearXNG's upstream engine configuration remains available.diff --git a/community_benchmark_results/benchmark_template.yaml b/community_benchmark_results/benchmark_template.yamlindex 11b21e758b..08700ac73e 100644--- a/community_benchmark_results/benchmark_template.yaml+++ b/community_benchmark_results/benchmark_template.yaml@@ -8,7 +8,7 @@ model_provider: # ollama, llamacpp, openai_endpoint, etc.quantization: # e.g., Q4_K_M, Q5_K_S, FP16, etc.# Search Engine (critical for benchmark reproducibility)-search_engine: # e.g., searxng, tavily, duckduckgo, google+search_engine: # e.g., searxng, tavily, wikipedia, google_psesearch_provider_version: # if known, e.g., "latest", "2024.1.0"average_results_per_query: # optional, e.g., 10diff --git a/docs/architecture.md b/docs/architecture.mdindex 67d81d1053..60714c99f6 100644--- a/docs/architecture.md+++ b/docs/architecture.md@@ -44,7 +44,6 @@ flowchart TBWEB_SEARCH[/"🌐 Web Search"\]TAVILY[Tavily]BRAVE[Brave]- DDG[DuckDuckGo]ACADEMIC[/"📚 Academic"\]ARXIV[ArXiv]PUBMED[PubMed]@@ -78,7 +77,7 @@ flowchart TBEXEC <--> OPENAI & ANTHROPIC & GEMINI & OPENROUTEREXEC <--> SEARXNG & ELASTIC & LIBRARY- EXEC <--> TAVILY & BRAVE & DDG+ EXEC <--> TAVILY & BRAVEEXEC <--> ARXIV & PUBMED & SEMANTICSYNTH --> SQLCIPHER@@ -184,7 +183,6 @@ mindmapSearXNGTavilyBrave- DuckDuckGoAcademicArXivPubMeddiff --git a/docs/search-engines.md b/docs/search-engines.mdindex 7e7e9b2a7f..32fb0c434b 100644--- a/docs/search-engines.md+++ b/docs/search-engines.md@@ -75,12 +75,6 @@ result = quick_summary(`LDR_SEARCH_ALLOW_PRIVATE_ENGINE_URLS=true` — only one is needed — see[SearXNG-Setup](SearXNG-Setup.md)-#### DuckDuckGo-- **Specialization**: Privacy-focused web search-- **Best for**: General web queries without tracking-- **Results**: Web pages, instant answers-- **Rate Limit**: Strict - use SearXNG for better reliability-### Technical Search#### GitHub@@ -101,6 +95,23 @@ result = quick_summary(- **Best for**: Finding deleted content, tracking changes over time- **Results**: Archived web pages with timestamps- **Rate Limit**: Moderate+- **Input**: A query that contains full `http://` or `https://` URLs, or a+ query that is only a bare domain, such as `example.com`. For these, only+ the host is looked up; the port, path, query string and fragment are+ ignored. Not looked up: a domain inside a sentence, a domain with a path+ but without `http://` or `https://`, a URL with an uppercase scheme such+ as `HTTPS://`, a URL with a bracketed IPv6 host, and a bare domain with a+ punycode top-level domain such as `.xn--p1ai` (give it as a full+ `http://` or `https://` URL instead). A URL with a user name or password+ before the host is not supported: Wayback looks up the start of the user+ name instead of the host, so give the plain URL. A host name is read only+ as far as letters, digits, hyphens, underscores, dots and percent-escapes+ run, so one written with other characters, such as the combining marks of+ some scripts, is cut short (as a bare domain it is not recognised); use+ its `xn--` form in a full URL. For a topic query, first find page URLs+ with your configured search provider, such as SearXNG, then pass those+ URLs to Wayback, which looks up their hosts. Wayback does not send+ queries to another search provider.### News Searchdiff --git a/docs/troubleshooting.md b/docs/troubleshooting.mdindex 89f281b6ed..25f603a9e5 100644--- a/docs/troubleshooting.md+++ b/docs/troubleshooting.md@@ -97,26 +97,21 @@ This guide covers common issues and their solutions.## Search Engine Issues-### DuckDuckGo Returning No Results--**Symptoms:**-- Empty search results-- "No results found" consistently--**Solutions:**--1. **Rate limiting:** DuckDuckGo aggressively rate limits. Solutions:- - Switch to SearXNG or another engine- - Increase wait time in rate limiting settings- - Use `rate_limiting.profile = conservative`--2. **Check network:** Verify you can access DuckDuckGo directly--3. **Try alternative engines:**- ```python- # In settings- search.tool = "searxng" # or "brave", "tavily", etc.- ```+### Wayback Returning No Results++Wayback looks up archived versions of the host of each `http://` or+`https://` URL, or of a bare domain. It does not discover page URLs through+another search provider. Search for the topic with your configured provider,+such as SearXNG, and pass a resulting URL to Wayback, which looks up its+host. An empty archive result can also mean that the archive has no capture+of that host.++Wayback also returns nothing for a URL with an uppercase scheme or a+bracketed IPv6 host, and for a bare domain with a punycode top-level domain+such as `.xn--p1ai`; give a punycode domain as a full `http://` or `https://`+URL. A URL with a user name or password before the host is not supported:+Wayback looks up the start of the user name instead of the host, so remove+it.### SearXNG Setup Issuesdiff --git a/pdm.lock b/pdm.lockindex e03a2a0311..49a382c437 100644--- a/pdm.lock+++ b/pdm.lock@@ -5,7 +5,7 @@groups = ["default", "dev", "mcp"]strategy = ["inherit_metadata"]lock_version = "4.5.0"-content_hash = "sha256:4fead3891d3376d1334013c74a30c63d9f084aa68bdba90cd7e80765a5cb768f"+content_hash = "sha256:6c63ba0723772a8ca46b78c8a4dbe1ef828657edcf8c07cd259b27b9eeaba4a6"[[metadata.targets]]requires_python = ">=3.12,<3.15"@@ -1277,22 +1277,6 @@ files = [{file = "docstring_parser-0.18.0.tar.gz", hash = "sha256:292510982205c12b1248696f44959db3cdd1740237a968ea1e2e7a900eeb2015"},]-[[package]]-name = "duckduckgo-search"-version = "8.1.1"-requires_python = ">=3.9"-summary = "Search for words, documents, images, news, maps and text translation using the DuckDuckGo.com search engine."-groups = ["default"]-dependencies = [- "click>=8.1.8",- "lxml>=5.3.0",- "primp>=0.15.0",-]-files = [- {file = "duckduckgo_search-8.1.1-py3-none-any.whl", hash = "sha256:f48adbb06626ee05918f7e0cef3a45639e9939805c4fc179e68c48a12f1b5062"},- {file = "duckduckgo_search-8.1.1.tar.gz", hash = "sha256:9da91c9eb26a17e016ea1da26235d40404b46b0565ea86d75a9f78cc9441f935"},-]-[[package]]name = "dynaconf"version = "3.3.5"@@ -4319,46 +4303,6 @@ files = [{file = "pre_commit-4.6.2.tar.gz", hash = "sha256:8f5d7bfb021ecdbcd9d49d89847082dd24172ccde534390081a679ad046e2441"},]-[[package]]-name = "primp"-version = "2.0.1"-requires_python = ">=3.10"-summary = "HTTP client that can impersonate web browsers"-groups = ["default"]-files = [- {file = "primp-2.0.1-cp310-abi3-macosx_10_12_x86_64.whl", hash = "sha256:4296ae43a8660bcb1dfc1570ed07dc9f8ab64f11fdebf3272532411b7fe321ef"},- {file = "primp-2.0.1-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:ca21c764f17ba42dde38c29d6a1f01970d39fa5f4793b0fe702006bd71b7a16a"},- {file = "primp-2.0.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:aa85a55b1c53ef8c2d14f1c61f0b7ab0ff300b349b2768a52d6c2f3f3f9ca80c"},- {file = "primp-2.0.1-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:9edc6d2f2fd30d2ddf8c093bf533a3e99ae1385a1d2af15baaa299fb55310fb8"},- {file = "primp-2.0.1-cp310-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:24d3ffeb054c23c7588d0240b5488d960f669398d1ab4cb3873158322bdf821d"},- {file = "primp-2.0.1-cp310-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:0beec080cb61044bd8b2eb0e3ca60f15f2df8ced9f2fbc6f1760855c20aed42a"},- {file = "primp-2.0.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9a7be373adfded677a9092ae2743873d5c8a9573148d617a189d26715f7d8ea5"},- {file = "primp-2.0.1-cp310-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:acc8b31f7fcc241b8ecb94069efea611f4f451d90f1798684c379d05e108b2c0"},- {file = "primp-2.0.1-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:e5f8d170c69b4afbe61d854b3f0cf27a0c0e557f0e54ebe595dad4db0f72127d"},- {file = "primp-2.0.1-cp310-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:706c843c86162d431c5a2051b8b41e16c7c51bca6bd60d139d7614609463a6c5"},- {file = "primp-2.0.1-cp310-abi3-musllinux_1_2_i686.whl", hash = "sha256:415aa6bb1b998ace5a456734df95755b5342b73fa0a6babbcb3ca471c83b9dbe"},- {file = "primp-2.0.1-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:7f494686da2991212f5607d125c32cb8da72a92e179d497c7d2ffeba21abff1e"},- {file = "primp-2.0.1-cp310-abi3-win32.whl", hash = "sha256:1cb429afd3a5ea98c625292f3c6581b0ccc0d398d3307603cce25ec140dfd671"},- {file = "primp-2.0.1-cp310-abi3-win_amd64.whl", hash = "sha256:0e27f3e233cf34cae6cbc8158af58b93fb0a87ceb1802b4611c7a14a2d822cdc"},- {file = "primp-2.0.1-cp310-abi3-win_arm64.whl", hash = "sha256:dea9370fcf6624725f564f9b9cf4fec3c127f86b7494196d03343a835fe3dee4"},- {file = "primp-2.0.1-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:656ea4ff0d45bbd119394a6834a367e34192d75cb7a5945cfc2f1cbb266b1be4"},- {file = "primp-2.0.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:31b3cd957f02dc49e5e9d9cbadd809747872caebfaf093dc34775a7f866a3e65"},- {file = "primp-2.0.1-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0b8af35eb64d61291b105479245c89ed1231a5fff1e9b75870515892ffabf054"},- {file = "primp-2.0.1-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:4f659e6479073c4693195f0f43b7a96ae0afd353f6f057754f80be016eea6f20"},- {file = "primp-2.0.1-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:9ba278a9af63981f2aece0d98fec9d6cc5a918943eb56e4f4df2b3ca90dab787"},- {file = "primp-2.0.1-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bedb42ea9188dd571db46d93f0b994d2e8e9271d2ddd55ee0ab7ac2844742bed"},- {file = "primp-2.0.1-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:888ba0708e518acad84bfc0a8bb274ec7ae48655a95afdb9c8184bd88cc6d7ab"},- {file = "primp-2.0.1-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:71fa07394c0084940d86c9f441bc2f05d7f8951a944bde315bebb6eec9b718d7"},- {file = "primp-2.0.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:c8f2b2eaacddf2bdff7b1b5f4219249d06246e577432fcb9b9babbc65146ff32"},- {file = "primp-2.0.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:934696c74b8a88a7dbb40b3bb54a182b0c9782427f036035cb21bfc0cf7e24a9"},- {file = "primp-2.0.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:e5f2e7b9fe557a440a929746a318074fd9989be318ce75411d01f1f3ed7bc85d"},- {file = "primp-2.0.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:2b4970ab274deaa13224777fb52b8745523293c23566a6c44fa3fbd61e47c183"},- {file = "primp-2.0.1-cp314-cp314t-win32.whl", hash = "sha256:0440d84854d1f9218277eef2c688a1774408e0d8a3805077eb6db432a4fa7970"},- {file = "primp-2.0.1-cp314-cp314t-win_amd64.whl", hash = "sha256:f0806c7653ee05bbe3c7b28f97f19bd5d763d7da1366026d5cb85cd8713f5c33"},- {file = "primp-2.0.1-cp314-cp314t-win_arm64.whl", hash = "sha256:a26651747b21efdff1ff986ee3e98ec3b349ce84b01b22226ce0b7a967041f01"},- {file = "primp-2.0.1.tar.gz", hash = "sha256:82ba17b077bef19a189d9ec8d77ca632496cb444e0f4fa37e27e90041cf0da8f"},-]-[[package]]name = "propcache"version = "0.5.4"diff --git a/pyproject.toml b/pyproject.tomlindex 2d613ea788..eb2c4de289 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -28,7 +28,6 @@ dependencies = ["langchain-anthropic~=1.3", # lock pulls 1.4.6: GHSA-gr75-jv2w-4656 fix"openai>=2", # direct dep: error_handling/openai_compat_errors.py imports exception classes"httpx>=0.27", # direct dep: error_handling/openai_compat_errors.py imports ConnectError/ReadTimeout- "duckduckgo-search~=8.1","python-dateutil~=2.9","typing-extensions~=4.15","justext~=3.0",diff --git a/src/local_deep_research/advanced_search_system/strategies/langgraph_agent_strategy.py b/src/local_deep_research/advanced_search_system/strategies/langgraph_agent_strategy.pyindex 5da1ed6874..16b58fd1e9 100644--- a/src/local_deep_research/advanced_search_system/strategies/langgraph_agent_strategy.py+++ b/src/local_deep_research/advanced_search_system/strategies/langgraph_agent_strategy.py@@ -1345,7 +1345,6 @@ _TOOL_DISPLAY_NAMES = {"search_searxng": "the web (SearXNG)","search_google_scholar": "Google Scholar","search_brave": "Brave Search",- "search_duckduckgo": "DuckDuckGo","search_serper": "Google (Serper)","search_scaleserp": "Google (ScaleSERP)","search_wikipedia": "Wikipedia",@@ -1436,6 +1435,20 @@ def _format_results(results: list[dict], start_idx: int) -> str:# ``_make_fetch_progress_adapter`` for backwards compatibility with# existing tests.+# Appended to both Wayback tool descriptions. The engine looks up the host+# of each lowercase http(s) URL in its input, or an input that is only a bare+# domain; it does not discover URLs or pick a domain out of a sentence. The+# "Not supported" sentence names the inputs its patterns miss or misread.+_WAYBACK_INPUT_HINT = (+ " Input must contain a full http:// or https:// URL, or be only a bare"+ " domain such as example.com. Only the host is looked up, so a URL's"+ " port, path, query and fragment are ignored. Not supported: an"+ " uppercase scheme, a bracketed IPv6 host, a user name or password"+ " before the host, and a bare domain with a punycode (xn--) top-level"+ " domain. Use a configured search provider to discover URLs before"+ " looking up their archives."+)+def _make_web_search_tool(search_engine_name: str,@@ -1449,6 +1462,8 @@ def _make_web_search_tool():"""Create a primary search tool with research-scoped availability."""availability = availability or SearchEngineAvailability()+ if search_engine_name == "wayback":+ description += _WAYBACK_INPUT_HINT@tooldef web_search(query: str) -> str:@@ -1517,6 +1532,8 @@ def _make_specialized_search_tool():"""Create a specialized search tool with research-scoped availability."""availability = availability or SearchEngineAvailability()+ if engine_name == "wayback":+ description += _WAYBACK_INPUT_HINT@tooldef specialized_search(query: str) -> str:@@ -2462,9 +2479,8 @@ class LangGraphAgentStrategy(BaseSearchStrategy):``semantic_scholar`` rather than the class-derived``semanticscholar``). The class-name fallback historically returneda lowercased, ``SearchEngine``-stripped variant that was almost- but not quite canonical — e.g. ``DuckDuckGoSearchEngine`` →- ``"duckduckgo"`` instead of the registry key ``"ddg"``, and- ``SemanticScholarSearchEngine`` → ``"semanticscholar"`` instead of+ but not quite canonical — e.g. ``SemanticScholarSearchEngine`` →+ ``"semanticscholar"`` instead of``"semantic_scholar"``. That mismatch leaked the configured primaryengine back into the specialised-tools loop as a redundant``search_<engine>`` tool alongside ``web_search``. Reverse-lookup@@ -2484,7 +2500,7 @@ class LangGraphAgentStrategy(BaseSearchStrategy):return tool_setting# Fall back to the engine CLASS, but resolve via the registry so# we land on the canonical id (``semantic_scholar``,- # ``ddg``, ...) rather than the class-derived heuristic. The+ # ``google_pse``, ...) rather than the class-derived heuristic. The# registry is the single source of truth for which Python class# implements which canonical id.if self.search is not None and hasattr(self.search, "__class__"):@@ -2508,7 +2524,7 @@ class LangGraphAgentStrategy(BaseSearchStrategy):``web_search`` is a generic wrapper around the user's configuredengine. Resolve it through the same curated ``_TOOL_DISPLAY_NAMES``map as the specialized search tools (keyed by ``search_<engine>``)- so the UI shows brand-correct names like "DuckDuckGo" or+ so the UI shows brand-correct names like "Brave Search" or"the web (SearXNG)" instead of the raw lowercase engine id(e.g. "searxng"). ``search_collection_<id>`` tools (including``web_search`` remapped onto a collection primary engine) resolvediff --git a/src/local_deep_research/error_handling/report_generator.py b/src/local_deep_research/error_handling/report_generator.pyindex 218ed018a9..5d86371301 100644--- a/src/local_deep_research/error_handling/report_generator.py+++ b/src/local_deep_research/error_handling/report_generator.py@@ -378,7 +378,7 @@ We're here to help you get this working:"Your search query is too long for GitHub's API (max 256 characters).\n\n""**Try this:**\n""- Shorten your research query\n"- "- Use a different search engine (DuckDuckGo, Searx, etc.)\n"+ "- Use a different search engine (SearXNG, Wikipedia, etc.)\n""- Break your research into smaller, focused queries"),"No module named.*local_deep_research": (@@ -402,7 +402,7 @@ We're here to help you get this working:"- **Start SearXNG:** `docker run -d -p 8080:8080 searxng/searxng` then set URL to `http://localhost:8080`\n""- **Try different search terms:** Use broader, more general keywords\n""- **Check network connection:** Ensure you can access the internet\n"- "- **Switch search engines:** Try DuckDuckGo, Brave, or Google (if API key configured)\n"+ "- **Switch search engines:** Try SearXNG, Brave, or Google (if API key configured)\n""- **Check for typos** in your research query"),"TypeError.*Context.*Size|'<' not supported between instances of .* and 'NoneType'": (diff --git a/src/local_deep_research/mcp/server.py b/src/local_deep_research/mcp/server.pyindex b9f6fa7799..0693191752 100644--- a/src/local_deep_research/mcp/server.py+++ b/src/local_deep_research/mcp/server.py@@ -688,6 +688,11 @@ def search(query: The search query string.engine: Search engine to use (e.g., "arxiv", "wikipedia", "searxng", "brave").This is required — use list_search_engines() to see available options.+ For "wayback", the query must contain a full http:// or https://+ URL, or be only a bare domain such as example.com; only the+ host is looked up. Not supported: an uppercase scheme, a+ bracketed IPv6 host, a user name or password before the host,+ and a bare domain with a punycode (xn--) top-level domain.max_results: Maximum number of results to return (1-100). Default is 10.Returns:diff --git a/src/local_deep_research/security/module_whitelist.py b/src/local_deep_research/security/module_whitelist.pyindex 3186ec1767..39e0cf1cb0 100644--- a/src/local_deep_research/security/module_whitelist.py+++ b/src/local_deep_research/security/module_whitelist.py@@ -31,7 +31,6 @@ ALLOWED_MODULE_PATHS: frozenset[str] = frozenset(".engines.search_engine_arxiv",".engines.search_engine_brave",".engines.search_engine_collection",- ".engines.search_engine_ddg",".engines.search_engine_elasticsearch",".engines.search_engine_exa",".engines.search_engine_federal_register",@@ -81,7 +80,6 @@ ALLOWED_CLASS_NAMES: frozenset[str] = frozenset("BaseSearchEngine","BraveSearchEngine","CollectionSearchEngine",- "DuckDuckGoSearchEngine","ElasticsearchSearchEngine","GutenbergSearchEngine","ExaSearchEngine",diff --git a/src/local_deep_research/web_search_engines/engine_registry.py b/src/local_deep_research/web_search_engines/engine_registry.pyindex e885b300d0..a37454c55f 100644--- a/src/local_deep_research/web_search_engines/engine_registry.py+++ b/src/local_deep_research/web_search_engines/engine_registry.py@@ -168,12 +168,6 @@ ENGINE_REGISTRY: Dict[str, EngineEntry] = {class_name="ZenodoSearchEngine",),# --- Engines implemented but without settings files ---- "ddg": EngineEntry(- module_path=".engines.search_engine_ddg",- class_name="DuckDuckGoSearchEngine",- full_search_module=".engines.full_search",- full_search_class="FullSearchResults",- ),"guardian": EngineEntry(module_path=".engines.search_engine_guardian",class_name="GuardianSearchEngine",diff --git a/src/local_deep_research/web_search_engines/engines/search_engine_ddg.py b/src/local_deep_research/web_search_engines/engines/search_engine_ddg.pydeleted file mode 100644index 6b5a9db132..0000000000--- a/src/local_deep_research/web_search_engines/engines/search_engine_ddg.py+++ /dev/null@@ -1,133 +0,0 @@-from typing import Any, Dict, List, Optional--from langchain_community.utilities import DuckDuckGoSearchAPIWrapper-from langchain_core.language_models import BaseLLM--from ...security.secure_logging import logger-from ..rate_limiting import RateLimitError-from ..search_engine_base import BaseSearchEngine, Exposure, Sensitivity---class DuckDuckGoSearchEngine(BaseSearchEngine):- """DuckDuckGo search engine implementation with two-phase retrieval"""-- # Mark as public search engine- is_public = True- egress_sensitivity = Sensitivity.NON_SENSITIVE- egress_exposure = Exposure.EXPOSING- # Mark as generic search engine (general web search)- is_generic = True-- def __init__(- self,- max_results: int = 10,- region: str = "us",- safe_search: bool = True,- llm: Optional[BaseLLM] = None,- language: str = "English",- include_full_content: bool = False,- max_filtered_results: Optional[int] = None,- settings_snapshot: Optional[Dict[str, Any]] = None,- ):- """- Initialize the DuckDuckGo search engine.-- Args:- max_results: Maximum number of search results- region: Region code for search results- safe_search: Whether to enable safe search- llm: Language model for relevance filtering- language: Language for content processing- include_full_content: Retained for backward compatibility; full-content- retrieval is handled via the factory wrapper.- """- max_results = min(max_results, 50)- # Initialize the BaseSearchEngine with LLM, max_filtered_results, and max_results- super().__init__(- llm=llm,- max_filtered_results=max_filtered_results,- max_results=max_results,- include_full_content=include_full_content,- settings_snapshot=settings_snapshot,- )- self.region = region- self.safe_search = safe_search- self.language = language-- # Initialize the DuckDuckGo wrapper- self.engine = DuckDuckGoSearchAPIWrapper(- region=region,- max_results=max_results,- safesearch="moderate" if safe_search else "off",- )-- # Note: full-content retrieval is the factory wrapper's job.- # The wrapper fetches pages via ``batch_fetch_and_extract`` and- # populates ``full_content`` on each result; the inner engine- # only emits snippets from ``_get_previews``.-- def _get_previews(self, query: str) -> List[Dict[str, Any]]:- """- Get preview information (titles and snippets) for initial search results.-- Args:- query: The search query-- Returns:- List of preview dictionaries with 'id', 'title', and 'snippet' keys- """- try:- # Get search results from DuckDuckGo- results = self.engine.results(query, max_results=self.max_results)-- if not isinstance(results, list):- return []-- # Process results to get previews- previews = []- for i, result in enumerate(results):- preview = {- "id": result.get("link"), # Use URL as ID for DDG- "title": result.get("title", ""),- "snippet": result.get("snippet", ""),- "link": result.get("link", ""),- }-- previews.append(preview)-- return previews-- except Exception as e:- error_msg = str(e)- sanitized = self._sanitize_error_message(error_msg)- logger.exception(- f"Error getting DuckDuckGo previews ({type(e).__name__}): {sanitized}"- )-- # Check for known rate limit patterns- if "202 Ratelimit" in error_msg or "ratelimit" in error_msg.lower():- raise RateLimitError(f"DuckDuckGo rate limit hit: {sanitized}")- if "403" in error_msg or "forbidden" in error_msg.lower():- raise RateLimitError(- f"DuckDuckGo access forbidden (possible rate limit): {sanitized}"- )- if (- "timeout" in error_msg.lower()- or "timed out" in error_msg.lower()- ):- # Timeouts can sometimes indicate rate limiting- raise RateLimitError(- f"DuckDuckGo timeout (possible rate limit): {sanitized}"- )-- return []-- def _get_full_content(- self, relevant_items: List[Dict[str, Any]]- ) -> List[Dict[str, Any]]:- """Return items unchanged.-- The factory wrapper populates ``full_content`` after running the- engine; the inner engine never fetches pages itself.- """- return relevant_itemsdiff --git a/src/local_deep_research/web_search_engines/engines/search_engine_wayback.py b/src/local_deep_research/web_search_engines/engines/search_engine_wayback.pyindex f4c2fd4bd9..ea80086078 100644--- a/src/local_deep_research/web_search_engines/engines/search_engine_wayback.py+++ b/src/local_deep_research/web_search_engines/engines/search_engine_wayback.py@@ -15,8 +15,9 @@ from ..search_engine_base import BaseSearchEngine, Exposure, Sensitivityclass WaybackSearchEngine(BaseSearchEngine):"""- Internet Archive Wayback Machine search engine implementation- Provides access to historical versions of web pages+ Look up historical versions of supplied URLs or domains in the+ Internet Archive Wayback Machine. This engine does not discover URLs+ through another search provider."""# Mark as public search engine@@ -62,8 +63,23 @@ class WaybackSearchEngine(BaseSearchEngine):def _extract_urls_from_query(self, query: str) -> List[str]:"""- Extract URLs from a query string or interpret as an URL if possible.- For non-URL queries, use a DuckDuckGo search to find relevant URLs.+ Extract the scheme and host of each lowercase ``http://`` or+ ``https://`` URL in a query, or read a query that is only a bare+ domain as a URL. After the scheme, a URL match keeps letters and+ digits of any script, underscores, hyphens, dots and percent-escapes+ and ends at the first other character, so a port, path, query+ string or fragment is dropped and only the host is looked up. The+ same rule cuts short a host written with other characters, such as+ combining marks, and keeps a bare domain written with them from+ matching. Not read: a URL with an uppercase scheme or a bracketed+ IPv6 host, which does not match, and the host of a URL with a user+ name or password before it, where the match ends at or before the+ ``@`` and so looks up the start of the user name instead. The last+ label of a bare domain may hold only letters, digits and+ underscores, so a punycode top-level domain such as ``.xn--p1ai`` is+ not read as a bare domain, though the same domain in a full URL is.+ Topic queries need URL discovery through a configured search+ provider before they can be looked up in the Wayback Machine.Args:query: The search query or URL@@ -85,47 +101,11 @@ class WaybackSearchEngine(BaseSearchEngine):logger.info("Query appears to be a domain")return [f"http://{query}"]- # For non-URL queries, use DuckDuckGo to find relevant URLs- logger.info(- "Query is not a URL, using DuckDuckGo to find relevant URLs"- )- try:- # Import DuckDuckGo search engine- from langchain_community.utilities import DuckDuckGoSearchAPIWrapper-- # Use max_results from parent class, but limit to 5 for URL discovery- url_search_limit = min(5, self.max_results)- ddg = DuckDuckGoSearchAPIWrapper(max_results=url_search_limit)- # Pass max_results as a positional argument- results = ddg.results(query, url_search_limit)-- # Extract URLs from results- ddg_urls = [- str(result.get("link"))- for result in results- if result.get("link")- ]- if ddg_urls:- logger.info(- f"Found {len(ddg_urls)} URLs from DuckDuckGo search"- )- return ddg_urls- except Exception as e:- safe_msg = self._scrub_error(e)- logger.exception(- f"Error using DuckDuckGo for URL discovery ({type(e).__name__}): {safe_msg}"- )-- # Fallback: treat the query as a potential domain or path- if "/" in query and "." in query:- logger.info("Treating query as a partial URL")- return [f"http://{query}"]- if "." in query:- logger.info("Treating query as a domain")- return [f"http://{query}"]-# Return empty list if nothing worked- logger.warning("Could not extract any URLs from the query")+ logger.warning(+ "Wayback needs an http(s) URL, or a query that is only a bare "+ "domain; discover URLs with a configured search provider first"+ )return []def _format_timestamp(self, timestamp: str) -> str:diff --git a/src/local_deep_research/web_search_engines/search_engine_factory.py b/src/local_deep_research/web_search_engines/search_engine_factory.pyindex 477042b09e..2dbb98edf5 100644--- a/src/local_deep_research/web_search_engines/search_engine_factory.py+++ b/src/local_deep_research/web_search_engines/search_engine_factory.py@@ -26,7 +26,7 @@ TIME_PERIOD_ENGINES = ("serpapi", "tavily", "wikinews")_NON_LOCATION_REGIONS = frozenset({"wt-wt", "all", "none", "worldwide", ""})# Engines that accept a language code (ISO 639-1) in their ``language`` parameter.-# Engines like OpenLibrary (MARC code or None) or Wayback/DDG (language name+# Engines like OpenLibrary (MARC code or None) or Wayback (language name# for Justext) are excluded so that global language settings do not corrupt them.LANGUAGE_CODE_ENGINES = frozenset({"searxng", "mojeek", "wikipedia", "tinyfish", "scaleserp"}@@ -443,7 +443,7 @@ def create_search_engine(# MCP server never set that flag (langgraph-agent tools explicitly set# use_full_search=False), so without this fallback the engine fetches# full content (search_snippets_only=False) but the wrapper path that- # actually hydrates ``full_content`` for searxng/ddg/serpapi/etc. never+ # actually hydrates ``full_content`` for searxng/serpapi/etc. never# runs.if "use_full_search" not in kwargs and "search_snippets_only" in kwargs:kwargs["use_full_search"] = not kwargs["search_snippets_only"]@@ -1079,7 +1079,6 @@ def get_search(# Add engine-specific parametersif search_tool in [- "duckduckgo","serpapi","google_pse","brave",diff --git a/tests/api_tests/populate_search_engines.py b/tests/api_tests/populate_search_engines.pyindex 3c42543d36..f63470ffca 100644--- a/tests/api_tests/populate_search_engines.py+++ b/tests/api_tests/populate_search_engines.py@@ -36,11 +36,6 @@ def populate_search_engines():"description": "Microsoft Bing search engine","strengths": ["Good for technical queries", "Image search", "News"],},- "duckduckgo": {- "display_name": "DuckDuckGo",- "description": "Privacy-focused search engine",- "strengths": ["Privacy", "No tracking", "Instant answers"],- },"wikipedia": {"display_name": "Wikipedia","description": "Wikipedia search",diff --git a/tests/api_tests/test_search_engines_api.py b/tests/api_tests/test_search_engines_api.pyindex a8fec49e13..331a5961cf 100644--- a/tests/api_tests/test_search_engines_api.py+++ b/tests/api_tests/test_search_engines_api.py@@ -53,7 +53,7 @@ class TestSearchEnginesAPI:# Check if search engine settings existendpoints = ["/settings/api/search.engine.web.searxng.display_name",- "/settings/api/search.engine.web.duckduckgo.display_name",+ "/settings/api/search.engine.web.wikipedia.display_name","/settings/api/search.engine.web.google.display_name","/settings/api/search.engine.web.bing.display_name",]diff --git a/tests/fixtures/search_engine_mocks.py b/tests/fixtures/search_engine_mocks.pyindex df72cff125..5f49d424af 100644--- a/tests/fixtures/search_engine_mocks.py+++ b/tests/fixtures/search_engine_mocks.py@@ -4,7 +4,7 @@ Search engine mock fixtures - Based on scottvr's patterns but updated for currenThis module provides mock responses for various search engines."""-from typing import Any, Dict, List+from typing import Any, Dictfrom urllib.parse import urlparseimport pytest@@ -135,22 +135,6 @@ class SearchEngineMocks:],}- @staticmethod- def ddg_response() -> List[Dict[str, Any]]:- """Mock DuckDuckGo search response."""- return [- {- "title": "Artificial Intelligence - Wikipedia",- "link": "https://en.wikipedia.org/wiki/Artificial_intelligence",- "snippet": "Artificial intelligence (AI) is intelligence demonstrated by machines...",- },- {- "title": "What is AI? Artificial Intelligence Explained",- "link": "https://example.com/what-is-ai",- "snippet": "AI refers to the simulation of human intelligence in machines...",- },- ]-@staticmethoddef error_responses() -> Dict[str, Any]:"""Collection of error responses for testing error handling."""diff --git a/tests/mcp/test_mcp_protocol.py b/tests/mcp/test_mcp_protocol.pyindex 7f3313139c..345ff7a27b 100644--- a/tests/mcp/test_mcp_protocol.py+++ b/tests/mcp/test_mcp_protocol.py@@ -121,6 +121,20 @@ class TestMCPToolRegistration:assert get_configuration.__doc__ is not Noneassert "config" in get_configuration.__doc__.lower()+ def test_search_docstring_states_wayback_input_contract(self):+ """The wayback hint names the inputs the engine cannot read."""+ from local_deep_research.mcp.server import search++ doc = " ".join(search.__doc__.split())+ assert "full http:// or https:// URL" in doc+ for unsupported in (+ "an uppercase scheme",+ "a bracketed IPv6 host",+ "a user name or password before the host",+ "a bare domain with a punycode (xn--) top-level domain",+ ):+ assert unsupported in doc+class TestMCPLogging:"""Tests for MCP logging configuration (critical for STDIO)."""diff --git a/tests/search_engines/conftest.py b/tests/search_engines/conftest.pyindex 385d38a09a..a4e8a5d3a5 100644--- a/tests/search_engines/conftest.py+++ b/tests/search_engines/conftest.py@@ -351,33 +351,6 @@ def mock_github_code_search_response(mock_response_factory):)-# ============================================================================-# DuckDuckGo-specific fixtures-# ============================================================================---def mock_ddg_response():- """Mock DuckDuckGo search results (from duckduckgo_search library format)."""- return [- {- "title": "Introduction to Machine Learning | Example Site",- "href": "https://example.com/ml-intro",- "body": "A comprehensive introduction to machine learning concepts and algorithms.",- },- {- "title": "Machine Learning Tutorial - Learn ML",- "href": "https://learn-ml.example.org/tutorial",- "body": "Step-by-step tutorial on building your first machine learning model.",- },- {- "title": "ML Best Practices",- "href": "https://blog.example.com/ml-best-practices",- "body": "Learn the best practices for implementing machine learning in production.",- },- ]--# ============================================================================# Brave Search-specific fixtures# ============================================================================diff --git a/tests/search_engines/test_search_engine_ddg.py b/tests/search_engines/test_search_engine_ddg.pydeleted file mode 100644index 645744e324..0000000000--- a/tests/search_engines/test_search_engine_ddg.py+++ /dev/null@@ -1,358 +0,0 @@-"""-Comprehensive tests for the DuckDuckGo search engine.-Tests initialization, search functionality, error handling, and result parsing.--Note: These tests require the 'duckduckgo_search' package to be installed.-"""--import pytest-from unittest.mock import Mock, patch--# Check if DuckDuckGo search is actually importable (requires both-# langchain_community and the ddgs package to be installed)-try:- from langchain_community.utilities import DuckDuckGoSearchAPIWrapper-- DuckDuckGoSearchAPIWrapper()- DDGS_AVAILABLE = True-except (ImportError, Exception):- DDGS_AVAILABLE = False--[email protected](not DDGS_AVAILABLE, reason="ddgs package not installed")-class TestDuckDuckGoSearchEngineInit:- """Tests for DuckDuckGo search engine initialization."""-- def test_init_default_parameters(self):- """Test initialization with default parameters."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- engine = DuckDuckGoSearchEngine()-- assert engine.max_results >= 10- assert engine.is_public is True-- def test_init_custom_max_results(self):- """Test initialization with custom max results."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- engine = DuckDuckGoSearchEngine(max_results=25)- assert engine.max_results >= 25-- def test_init_with_region(self):- """Test initialization with specific region."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- engine = DuckDuckGoSearchEngine(region="us-en")- assert engine.region == "us-en"--[email protected](not DDGS_AVAILABLE, reason="ddgs package not installed")-class TestDuckDuckGoSearchExecution:- """Tests for DuckDuckGo search execution."""-- @pytest.fixture- def ddg_engine(self):- """Create a DuckDuckGo engine instance."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- return DuckDuckGoSearchEngine(max_results=10)-- def test_engine_initialization(self, ddg_engine):- """Test that engine is properly initialized."""- assert ddg_engine is not None- assert ddg_engine.max_results >= 10--[email protected](not DDGS_AVAILABLE, reason="ddgs package not installed")-class TestDuckDuckGoRegionSupport:- """Tests for DuckDuckGo region/locale support."""-- def test_default_region(self):- """Test default region configuration."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- engine = DuckDuckGoSearchEngine()- # Default region should be set or None- assert hasattr(engine, "region")-- def test_custom_region(self):- """Test custom region configuration."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- engine = DuckDuckGoSearchEngine(region="de-de")- assert engine.region == "de-de"--[email protected](not DDGS_AVAILABLE, reason="ddgs package not installed")-class TestDuckDuckGoEngineType:- """Tests for DuckDuckGo engine type identification."""-- def test_engine_type_set(self):- """Test that engine type is properly set."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- engine = DuckDuckGoSearchEngine()- assert "duckduckgo" in engine.engine_type.lower()-- def test_engine_is_public(self):- """Test that engine is marked as public."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- engine = DuckDuckGoSearchEngine()- assert engine.is_public is True-- def test_engine_is_generic(self):- """Test that engine is marked as generic (not specialized)."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- engine = DuckDuckGoSearchEngine()- assert engine.is_generic is True---# Tests that can run without the ddgs package (just testing mock structure)-class TestDDGResponseFixtures:- """Tests for DuckDuckGo response fixture structure."""-- def test_mock_response_structure(self, mock_ddg_response):- """Test that mock response has correct structure."""- assert isinstance(mock_ddg_response, list)- assert len(mock_ddg_response) == 3-- def test_mock_response_has_required_fields(self, mock_ddg_response):- """Test that mock response items have required fields."""- for result in mock_ddg_response:- assert "title" in result- assert "href" in result- assert "body" in result-- def test_mock_response_urls_valid(self, mock_ddg_response):- """Test that URLs in mock response are valid."""- for result in mock_ddg_response:- assert result["href"].startswith("http")--[email protected](not DDGS_AVAILABLE, reason="ddgs package not installed")-class TestDuckDuckGoGetPreviews:- """Tests for the _get_previews method and error handling."""-- @pytest.fixture- def ddg_engine(self):- """Create a DuckDuckGo engine instance."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- return DuckDuckGoSearchEngine(max_results=10)-- def test_get_previews_returns_correct_format(self, ddg_engine):- """Test that _get_previews returns properly formatted previews."""- with patch.object(ddg_engine.engine, "results") as mock_results:- mock_results.return_value = [- {- "title": "Test Title",- "link": "https://example.com",- "snippet": "Test snippet",- },- {- "title": "Title 2",- "link": "https://example2.com",- "snippet": "Snippet 2",- },- ]-- previews = ddg_engine._get_previews("test query")-- assert len(previews) == 2- assert previews[0]["title"] == "Test Title"- assert previews[0]["link"] == "https://example.com"- assert previews[0]["snippet"] == "Test snippet"- assert previews[0]["id"] == "https://example.com"-- def test_get_previews_handles_empty_results(self, ddg_engine):- """Test that _get_previews handles empty results list."""- with patch.object(ddg_engine.engine, "results") as mock_results:- mock_results.return_value = []-- previews = ddg_engine._get_previews("test query")-- assert previews == []-- def test_get_previews_handles_non_list_results(self, ddg_engine):- """Test that _get_previews handles non-list results."""- with patch.object(ddg_engine.engine, "results") as mock_results:- mock_results.return_value = None-- previews = ddg_engine._get_previews("test query")-- assert previews == []-- def test_get_previews_rate_limit_202_raises_error(self, ddg_engine):- """Test that 202 Ratelimit error raises RateLimitError."""- from local_deep_research.web_search_engines.rate_limiting import (- RateLimitError,- )-- with patch.object(ddg_engine.engine, "results") as mock_results:- mock_results.side_effect = Exception("202 Ratelimit hit")-- with pytest.raises(RateLimitError) as exc_info:- ddg_engine._get_previews("test query")-- assert "rate limit" in str(exc_info.value).lower()-- def test_get_previews_403_forbidden_raises_error(self, ddg_engine):- """Test that 403 forbidden error raises RateLimitError."""- from local_deep_research.web_search_engines.rate_limiting import (- RateLimitError,- )-- with patch.object(ddg_engine.engine, "results") as mock_results:- mock_results.side_effect = Exception("403 Forbidden response")-- with pytest.raises(RateLimitError) as exc_info:- ddg_engine._get_previews("test query")-- assert "forbidden" in str(exc_info.value).lower()-- def test_get_previews_timeout_raises_error(self, ddg_engine):- """Test that timeout error raises RateLimitError."""- from local_deep_research.web_search_engines.rate_limiting import (- RateLimitError,- )-- with patch.object(ddg_engine.engine, "results") as mock_results:- mock_results.side_effect = Exception("Connection timeout")-- with pytest.raises(RateLimitError) as exc_info:- ddg_engine._get_previews("test query")-- assert "timeout" in str(exc_info.value).lower()-- def test_get_previews_generic_exception_returns_empty(self, ddg_engine):- """Test that generic exceptions return empty list."""- with patch.object(ddg_engine.engine, "results") as mock_results:- mock_results.side_effect = ValueError("Some generic error")-- previews = ddg_engine._get_previews("test query")-- assert previews == []-- def test_get_previews_handles_missing_fields(self, ddg_engine):- """Test that _get_previews handles results with missing fields."""- with patch.object(ddg_engine.engine, "results") as mock_results:- mock_results.return_value = [- {"title": "Title Only"}, # Missing link and snippet- {"link": "https://example.com"}, # Missing title and snippet- ]-- previews = ddg_engine._get_previews("test query")-- assert len(previews) == 2- assert previews[0]["title"] == "Title Only"- assert previews[0]["snippet"] == ""- assert previews[1]["title"] == ""--[email protected](not DDGS_AVAILABLE, reason="ddgs package not installed")-class TestDuckDuckGoGetFullContent:- """Tests for the _get_full_content method."""-- @pytest.fixture- def ddg_engine_with_full_content(self):- """Create a DuckDuckGo engine with full content enabled."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- mock_llm = Mock()- return DuckDuckGoSearchEngine(- max_results=10,- llm=mock_llm,- include_full_content=True,- )-- @pytest.fixture- def ddg_engine_snippets_only(self):- """Create a DuckDuckGo engine without full content."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- return DuckDuckGoSearchEngine(- max_results=10,- include_full_content=False,- )-- def test_get_full_content_passes_through(- self, ddg_engine_with_full_content- ):- """DDG no longer self-wraps, so ``_get_full_content`` is a- pass-through. The factory wrapper populates ``full_content``- after the inner engine runs.- """- items = [{"title": "Test", "snippet": "Snippet"}]- assert not hasattr(ddg_engine_with_full_content, "full_search")- assert ddg_engine_with_full_content._get_full_content(items) is items--[email protected](not DDGS_AVAILABLE, reason="ddgs package not installed")-class TestDuckDuckGoRunMethod:- """Tests for the run() method which uses the two-phase approach."""-- @pytest.fixture- def ddg_engine(self):- """Create a DuckDuckGo engine instance."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- return DuckDuckGoSearchEngine(max_results=5)-- def test_run_returns_results(self, ddg_engine):- """Test that run() returns search results."""- with patch.object(ddg_engine, "_get_previews") as mock_previews:- mock_previews.return_value = [- {- "id": "1",- "title": "Test",- "snippet": "Test snippet",- "link": "https://example.com",- },- ]-- results = ddg_engine.run("test query")-- # Should return filtered results (or all if no filtering)- assert isinstance(results, list)-- def test_run_accepts_research_context(self, ddg_engine):- """Test that run() accepts research_context parameter."""- with patch.object(ddg_engine, "_get_previews") as mock_previews:- mock_previews.return_value = []-- # Should not raise- results = ddg_engine.run(- "test query",- research_context={"previous_findings": "some info"},- )-- assert isinstance(results, list)diff --git a/tests/search_engines/test_search_engine_wayback.py b/tests/search_engines/test_search_engine_wayback.pyindex 9c94cb3a38..5c87930017 100644--- a/tests/search_engines/test_search_engine_wayback.py+++ b/tests/search_engines/test_search_engine_wayback.py@@ -122,8 +122,7 @@ class TestWaybackURLExtraction:"""Test extraction of full HTTP URL (domain portion)."""urls = engine._extract_urls_from_query("https://example.com/page")# The regex extracts the domain portion of the URL- assert len(urls) >= 1- assert any("example.com" in url for url in urls)+ assert urls == ["https://example.com"]def test_extract_url_from_multiple_urls(self, engine):"""Test extraction of multiple URLs."""@@ -138,10 +137,10 @@ class TestWaybackURLExtraction:urls = engine._extract_urls_from_query("example.com")assert "http://example.com" in urls- def test_extract_url_from_partial_path(self, engine):- """Test extraction from partial URL path."""+ def test_partial_path_without_scheme_is_not_extracted(self, engine):+ """A domain with a path but no scheme is not an archive target."""urls = engine._extract_urls_from_query("example.com/page/test")- assert "http://example.com/page/test" in urls+ assert urls == []class TestWaybackTimestampFormatting:diff --git a/tests/search_engines/test_wayback_archive_only.py b/tests/search_engines/test_wayback_archive_only.pynew file mode 100644index 0000000000..aaec84be7a--- /dev/null+++ b/tests/search_engines/test_wayback_archive_only.py@@ -0,0 +1,226 @@+"""Wayback looks up the hosts of supplied targets without another provider."""++from unittest.mock import Mock++import pytest+import requests++from local_deep_research.web_search_engines.engines.search_engine_wayback import (+ WaybackSearchEngine,+)+++def discovery_guards(monkeypatch):+ import langchain_community.utilities as utilities++ discovery = Mock()+ discovery.return_value.results.return_value = [+ {"link": "https://unexpected.example"}+ ]+ monkeypatch.setattr(utilities, "DuckDuckGoSearchAPIWrapper", discovery)+ request = Mock(side_effect=AssertionError("Unexpected outbound request"))+ monkeypatch.setattr(requests.sessions.Session, "request", request)+ return discovery, request+++ ("query", "expected"),+ [+ ("https://example.com", ["https://example.com"]),+ ("example.com", ["http://example.com"]),+ ("sub.example.co.uk", ["http://sub.example.co.uk"]),+ pytest.param(+ "Check https://example.com/page archive",+ ["https://example.com"],+ id="running_text_path",+ ),+ pytest.param(+ "https://example.com:8443/page?token=secret#access_token=x",+ ["https://example.com"],+ id="port_query_and_fragment_dropped",+ ),+ pytest.param(+ "[source](https://example.com:8443/page?x=1).",+ ["https://example.com"],+ id="markdown_link",+ ),+ pytest.param(+ "[source: https://example.com] for details",+ ["https://example.com"],+ id="bracket_closer_after_host",+ ),+ pytest.param(+ "Citations: [1: https://example.com], [2: https://example.org]",+ ["https://example.com", "https://example.org"],+ id="bracketed_citations",+ ),+ pytest.param(+ "Archived versions of [https://a.example, https://b.example]",+ ["https://a.example", "https://b.example"],+ id="bracketed_list",+ ),+ pytest.param(+ "Pages (https://a.example/about, https://b.example/about) in 2010",+ ["https://a.example", "https://b.example"],+ id="parenthesised_list",+ ),+ pytest.param(+ "What did https://example.com's homepage say in 2005?",+ ["https://example.com"],+ id="ascii_possessive",+ ),+ pytest.param(+ "The bot said 'see https://example.com' twice",+ ["https://example.com"],+ id="closing_quote_without_adjacent_opener",+ ),+ pytest.param(+ "**https://example.com**'s homepage",+ ["https://example.com"],+ id="emphasis_then_possessive",+ ),+ pytest.param(+ "https://example.com(see note)",+ ["https://example.com"],+ id="parenthesis_after_host",+ ),+ pytest.param(+ "see https://example.com/a(x)https://example.org/b now",+ ["https://example.com", "https://example.org"],+ id="urls_jammed_without_space",+ ),+ pytest.param(+ "[read https://example.com/label-url][1]",+ ["https://example.com"],+ id="reference_style_label",+ ),+ pytest.param(+ "请查看https://example.com/about的历史版本",+ ["https://example.com"],+ id="cjk_text_around_url",+ ),+ pytest.param(+ "参见https://example.com。",+ ["https://example.com"],+ id="cjk_full_stop_after_host",+ ),+ pytest.param(+ "https://example.com,https://example.org",+ ["https://example.com", "https://example.org"],+ id="fullwidth_comma_list",+ ),+ ("https://", []),+ ("example.com/page", []),+ ("Find example.com", []),+ ("This sentence has a period.", []),+ ("topic.with.dots and words", []),+ ("topic/with.a.path and words", []),+ ("historical information about a topic", []),+ ("", []),+ # Inputs the tool descriptions and docs name as unsupported: userinfo+ # reads the user name instead of the host, and an uppercase scheme, a+ # bracketed IPv6 host and a punycode TLD in a bare domain match+ # nothing, while the same punycode domain in a full URL is read. A+ # combining mark is not a host character and ends the host early.+ ("https://user:[email protected]/?api_key=x", ["https://user"]),+ ("HTTPS://EXAMPLE.COM/page", []),+ ("https://[2001:db8::1]/page", []),+ ("xn--e1afmkfd.xn--p1ai", []),+ ("http://xn--e1afmkfd.xn--p1ai/", ["http://xn--e1afmkfd.xn--p1ai"]),+ ("https://bu\u0308cher.de/x", ["https://bu"]),+ ("bu\u0308cher.de", []),+ ],+)+def test_extract_archive_hosts_without_discovery(+ query, expected, discovery_guards+):+ discovery, request = discovery_guards+ engine = WaybackSearchEngine()++ assert engine._extract_urls_from_query(query) == expected+ discovery.assert_not_called()+ request.assert_not_called()+++ "query",+ [+ "historical information about a topic",+ "This sentence has a period.",+ "topic/with.a.path and words",+ "example.com/page",+ "",+ ],+)+def test_plain_text_previews_do_not_search_or_request_archives(+ query, discovery_guards+):+ discovery, request = discovery_guards+ engine = WaybackSearchEngine()+ engine._get_wayback_snapshots = Mock(return_value=[])++ assert engine._get_previews(query) == []+ discovery.assert_not_called()+ engine._get_wayback_snapshots.assert_not_called()+ request.assert_not_called()++[email protected]("closest_only", [False, True])+def test_archive_previews_look_up_only_the_host(+ closest_only, discovery_guards, monkeypatch+):+ from local_deep_research.web_search_engines.engines import (+ search_engine_wayback as wayback,+ )++ discovery, request = discovery_guards+ query = (+ "See https://example.com:8443/page?token=secret#access_token=x "+ "and https://example.org/a"+ )+ hosts = ["https://example.com", "https://example.org"]+ timestamp = "20240101000000"++ def archive_response(_endpoint, *, params):+ target = params["url"]+ response = Mock(status_code=200, headers={})+ if closest_only:+ response.json.return_value = {+ "archived_snapshots": {+ "closest": {+ "available": True,+ "timestamp": timestamp,+ "url": f"https://web.archive.org/web/{timestamp}/{target}",+ }+ }+ }+ else:+ response.json.return_value = [+ ["timestamp", "original", "statuscode", "mimetype"],+ [timestamp, target, "200", "text/html"],+ ]+ return response++ archive_lookup = Mock(side_effect=archive_response)+ monkeypatch.setattr(wayback, "safe_get", archive_lookup)+ engine = WaybackSearchEngine(closest_only=closest_only)+ engine.rate_tracker = Mock()++ previews = engine._get_previews(query)++ endpoint = engine.available_api if closest_only else engine.cdx_api+ calls = archive_lookup.call_args_list+ assert [call.args[0] for call in calls] == [endpoint] * len(hosts)+ assert [call.kwargs["params"]["url"] for call in calls] == hosts+ assert [preview["original_url"] for preview in previews] == hosts+ assert [preview["link"] for preview in previews] == [+ f"https://web.archive.org/web/{timestamp}/{host}" for host in hosts+ ]+ # Nothing after a host reaches the archive or the stored results.+ for preview in previews:+ for value in (preview["title"], preview["link"], preview["id"]):+ assert "8443" not in value+ assert "token" not in value+ discovery.assert_not_called()+ request.assert_not_called()diff --git a/tests/security/test_egress_policy.py b/tests/security/test_egress_policy.pyindex d55a67a2b1..8035cf8586 100644--- a/tests/security/test_egress_policy.py+++ b/tests/security/test_egress_policy.py@@ -353,11 +353,16 @@ def test_evaluate_engine_no_snapshot_fails_closed():assert decision.reason == "no_snapshot"-def test_evaluate_engine_unknown_name_fails_closed():- ctx = make_ctx()- decision = evaluate_engine(- "totally_made_up_engine", ctx, settings_snapshot={}- )+ "engine_name", ["totally_made_up_engine", "ddg", "duckduckgo"]+)+ "scope",+ [EgressScope.PRIVATE_ONLY, EgressScope.PUBLIC_ONLY, EgressScope.BOTH],+)+def test_evaluate_engine_unknown_name_fails_closed(engine_name, scope):+ ctx = make_ctx(scope=scope)+ decision = evaluate_engine(engine_name, ctx, settings_snapshot={})assert not decision.allowedassert decision.reason == "engine_unknown"@@ -434,7 +439,6 @@ def test_evaluate_engine_searxng_remote_classified_public():# Engines that can be loaded in CI (no optional deps needed)._PUBLIC_ENGINES = ["brave",- "ddg","exa","federal_register","github",diff --git a/tests/security/test_module_whitelist.py b/tests/security/test_module_whitelist.pyindex 52e6d5863e..af1dc9a371 100644--- a/tests/security/test_module_whitelist.py+++ b/tests/security/test_module_whitelist.py@@ -44,7 +44,6 @@ class TestAllowedModulePaths:"""Whitelist should contain expected search engine modules."""expected_engines = [".engines.search_engine_brave",- ".engines.search_engine_ddg",".engines.search_engine_tavily",".engines.search_engine_wikipedia",]@@ -65,7 +64,6 @@ class TestAllowedClassNames:"""Whitelist should contain expected search engine classes."""expected_classes = ["BraveSearchEngine",- "DuckDuckGoSearchEngine","TavilySearchEngine","WikipediaSearchEngine","BaseSearchEngine",@@ -101,6 +99,25 @@ class TestValidateModuleImport:)assert result is True+ @pytest.mark.parametrize(+ "module_path,class_name",+ [+ (".engines.search_engine_ddg", "BraveSearchEngine"),+ (".engines.search_engine_brave", "DuckDuckGoSearchEngine"),+ (".engines.search_engine_ddg", "DuckDuckGoSearchEngine"),+ ],+ )+ def test_removed_duckduckgo_adapter_cannot_be_imported(+ self, module_path, class_name+ ):+ assert validate_module_import(module_path, class_name) is False+ with patch(+ "local_deep_research.security.module_whitelist.importlib.import_module"+ ) as import_module:+ with pytest.raises(SecurityError):+ get_safe_module_class(module_path, class_name)+ import_module.assert_not_called()+def test_rejects_absolute_path(self):"""Should reject absolute module paths."""# Absolute path attemptdiff --git a/tests/security/test_module_whitelist_high_value.py b/tests/security/test_module_whitelist_high_value.pyindex cb1eaf9543..5b1fd9345b 100644--- a/tests/security/test_module_whitelist_high_value.py+++ b/tests/security/test_module_whitelist_high_value.py@@ -38,7 +38,6 @@ class TestAllowedModulePaths:def test_known_engine_paths_present(self):expected = [".engines.search_engine_brave",- ".engines.search_engine_ddg",".engines.search_engine_arxiv",".engines.search_engine_wikipedia",".engines.search_engine_tavily",@@ -64,7 +63,6 @@ class TestAllowedClassNames:def test_known_classes_present(self):expected = ["BraveSearchEngine",- "DuckDuckGoSearchEngine","ArXivSearchEngine","WikipediaSearchEngine","TavilySearchEngine",diff --git a/tests/strategies/test_langgraph_agent_strategy.py b/tests/strategies/test_langgraph_agent_strategy.pyindex e2959323ad..8b78ff797d 100644--- a/tests/strategies/test_langgraph_agent_strategy.py+++ b/tests/strategies/test_langgraph_agent_strategy.py@@ -1845,7 +1845,7 @@ class TestSearchToolMakers:return_value=mock_engine,):tool_fn = _make_web_search_tool(- search_engine_name="duckduckgo",+ search_engine_name="brave",model=MagicMock(),settings_snapshot={},collector=collector,@@ -1908,7 +1908,7 @@ class TestSearchToolMakers:return_value=mock_engine,) as mock_create:tool_fn = _make_web_search_tool(- search_engine_name="duckduckgo",+ search_engine_name="brave",model=MagicMock(),settings_snapshot=snapshot,collector=collector,@@ -2236,7 +2236,7 @@ class TestFetchProgressAdapter:return_value=mock_engine,) as mock_create:tool_fn = _make_web_search_tool(- search_engine_name="duckduckgo",+ search_engine_name="brave",model=MagicMock(),settings_snapshot={},collector=collector,@@ -2453,7 +2453,7 @@ class TestLangGraphAgentStrategy:"model": MagicMock(),"search": MagicMock(),"all_links_of_system": [],- "settings_snapshot": {"search.tool": {"value": "duckduckgo"}},+ "settings_snapshot": {"search.tool": {"value": "brave"}},}defaults.update(overrides)return LangGraphAgentStrategy(**defaults)@@ -2784,14 +2784,9 @@ class TestLangGraphAgentStrategy:def test_engine_name_fallback_to_class(self):mock_search = MagicMock()- mock_search.__class__.__name__ = "DuckDuckGoSearchEngine"+ mock_search.__class__.__name__ = "GooglePSESearchEngine"strategy = self._make_strategy(search=mock_search, settings_snapshot={})- # Registry reverse-lookup yields the canonical id (``ddg``),- # NOT the class-derived ``duckduckgo`` the previous heuristic- # produced — that mismatch let ``search_duckduckgo`` slip into- # the agent's specialized tool list alongside ``web_search`` when- # DuckDuckGo was the configured primary.- assert strategy._search_engine_name == "ddg"+ assert strategy._search_engine_name == "google_pse"def test_engine_name_semantic_scholar_resolves_to_canonical(self):"""``SemanticScholarSearchEngine`` -> ``semantic_scholar`` (canonical),@@ -2809,9 +2804,9 @@ class TestLangGraphAgentStrategy:display-name map, with brand-correct casing rather than the rawlowercase id."""strategy = self._make_strategy(- settings_snapshot={"search.tool": {"value": "duckduckgo"}}+ settings_snapshot={"search.tool": {"value": "brave"}})- assert strategy._display_tool_name("web_search") == "DuckDuckGo"+ assert strategy._display_tool_name("web_search") == "Brave Search"def test_display_tool_name_web_search_searxng(self):strategy = self._make_strategy(@@ -3493,7 +3488,7 @@ class TestBuildLibraryResolver:"model": MagicMock(),"search": MagicMock(),"all_links_of_system": [],- "settings_snapshot": {"search.tool": {"value": "duckduckgo"}},+ "settings_snapshot": {"search.tool": {"value": "brave"}},}defaults.update(overrides)return LangGraphAgentStrategy(**defaults)@@ -3508,7 +3503,7 @@ class TestBuildLibraryResolver:strategy = self._make_strategy(settings_snapshot={- "search.tool": {"value": "duckduckgo"},+ "search.tool": {"value": "brave"},"_username": "alice",})@@ -3523,7 +3518,7 @@ class TestBuildLibraryResolver:the pre-A3 behaviour so the egress policy rejects library URLsunchanged."""strategy = self._make_strategy(- settings_snapshot={"search.tool": {"value": "duckduckgo"}}+ settings_snapshot={"search.tool": {"value": "brave"}})assert strategy._build_library_resolver() is None@@ -3564,7 +3559,7 @@ class TestToolCallProgressFormatting:model=MagicMock(),search=MagicMock(),all_links_of_system=[],- settings_snapshot={"search.tool": {"value": "duckduckgo"}},+ settings_snapshot={"search.tool": {"value": "brave"}},)def _tc(self, name, **args):@@ -3685,9 +3680,9 @@ class TestToolCallProgressFormatting:strategy = self._make_strategy()out = strategy._format_tool_call_progress(self._tc("web_search", url="https://example.org"),- "DuckDuckGo",+ "Brave Search",)- assert out == '🔍 Searching DuckDuckGo: "https://example.org"'+ assert out == '🔍 Searching Brave Search: "https://example.org"'def test_unknown_tool_uses_search_prefix(self):"""A tool that isn't fetch_content or research_subtopic gets the@@ -5150,7 +5145,7 @@ class TestFetchModeSettingResolution:"model": MagicMock(),"search": MagicMock(),"all_links_of_system": [],- "settings_snapshot": {"search.tool": "duckduckgo"},+ "settings_snapshot": {"search.tool": "brave"},}defaults.update(overrides)return LangGraphAgentStrategy(**defaults)@@ -5161,7 +5156,7 @@ class TestFetchModeSettingResolution:"""strategy = self._make_strategy(settings_snapshot={- "search.tool": "duckduckgo",+ "search.tool": "brave","search.fetch.mode": "summary_focus",})@@ -5179,7 +5174,7 @@ class TestFetchModeSettingResolution:with loguru_caplog.at_level("WARNING"):strategy = self._make_strategy(settings_snapshot={- "search.tool": "duckduckgo",+ "search.tool": "brave","search.fetch.mode": "definitely-not-a-real-mode",})@@ -5196,7 +5191,7 @@ class TestFetchModeSettingResolution:"""strategy = self._make_strategy(settings_snapshot={- "search.tool": "duckduckgo",+ "search.tool": "brave","search.fetch.mode": "disabled",})@@ -5513,7 +5508,7 @@ class TestProgressMetadataKeepsStableId:model=MagicMock(),search=MagicMock(),all_links_of_system=[],- settings_snapshot={"search.tool": {"value": "duckduckgo"}},+ settings_snapshot={"search.tool": {"value": "brave"}},)def test_tool_call_metadata_keeps_id_label_in_message(self):@@ -5578,7 +5573,7 @@ class TestProgressMetadataKeepsStableId:# metadata keeps the stable id ...assert metadata["tool"] == "web_search"# ... while the user sees the brand label in the message text.- assert "DuckDuckGo" in message+ assert "Brave Search" in message# ---------------------------------------------------------------------------@@ -5610,7 +5605,7 @@ class TestEgressScopeFiltering:# Available-engines fixture. ``arxiv`` and ``pubmed`` are registered# public engines (``is_public = True`` on their classes); ``library``# is hardcoded local in ``evaluate_engine`` (line 322-326).- # ``duckduckgo`` is the current primary — already added as+ # ``brave`` is the current primary — already added as# ``web_search`` and explicitly skipped at line 618._FIXTURE_AVAILABLE = {"arxiv": {@@ -5638,19 +5633,19 @@ class TestEgressScopeFiltering:"description": "My research papers (Collection)","strengths": ["curated documents"],},- "duckduckgo": {+ "brave": {"is_local": False,- "description": "DuckDuckGo",+ "description": "Brave Search",},}- def _make_strategy(self, scope, primary_engine="duckduckgo"):+ def _make_strategy(self, scope, primary_engine="brave"):from local_deep_research.advanced_search_system.strategies.langgraph_agent_strategy import (LangGraphAgentStrategy,)mock_search = MagicMock()- mock_search.__class__.__name__ = "DuckDuckGoSearchEngine"+ mock_search.__class__.__name__ = "BraveSearchEngine"return LangGraphAgentStrategy(model=MagicMock(),search=mock_search,@@ -5785,17 +5780,20 @@ class TestEgressScopeFiltering:)# ------------------------------------------------------------------- # BOTH (default) — every classified engine is registered+ # Legacy BOTH resolves through adaptive primary classification# ------------------------------------------------------------------- def test_both_scope_registers_every_classified_engine(self):- """The default scope BOTH must register every classified engine- in the available dict. The current primary is excluded by the- explicit ``continue`` at line 618 — NOT by the scope filter — so- a regression that moved it into the scope-mismatch path would- still be caught by the assertion that it's absent.+ @pytest.mark.parametrize(+ "primary,include_local",+ [("brave", False), ("unclassified_test_primary", True)],+ )+ def test_legacy_both_scope_follows_primary_classification(+ self, primary, include_local+ ):+ """Known public primaries resolve to PUBLIC_ONLY; an unknown+ primary retains the resolved BOTH fallback."""- strategy = self._make_strategy(scope="both")+ strategy = self._make_strategy(scope="both", primary_engine=primary)with patch("local_deep_research.web_search_engines.search_engines_config.list_eligible_engine_configs",return_value=self._FIXTURE_AVAILABLE,@@ -5803,13 +5801,15 @@ class TestEgressScopeFiltering:tools = strategy._build_tools(overall_query="q")names = self._tool_names(tools)- for expected in ("search_arxiv", "search_pubmed", "search_library"):+ for expected in ("search_arxiv", "search_pubmed"):assert expected in names, (- f"Expected {expected} under BOTH but got: {sorted(names)}"+ f"Expected {expected} but got: {sorted(names)}")# The current engine is NEVER added as a specialized tool# regardless of scope.- assert "search_duckduckgo" not in names+ assert ("search_library" in names) is include_local+ assert ("search_collection_abc123" in names) is include_local+ assert ("search_brave" in names) is (primary != "brave")# ------------------------------------------------------------------# Fail-closed: corrupted scope value@@ -5883,9 +5883,8 @@ class TestEgressScopeFiltering:# Filtering the tool LIST closes the latency-leak half of the timing# attack. The other half is the prompt addendum: the LLM is *told* which# tools exist so it doesn't waste tokens probing for forbidden engines.-# These tests pin that the addendum text varies by scope and is empty-# under BOTH (we don't want to bleed policy state into the LLM for the-# default scope).+# These tests pin that the addendum text varies by resolved scope and is+# empty under the unclassified-primary BOTH fallback.class TestEgressScopePolicyAddendum:@@ -5894,13 +5893,13 @@ class TestEgressScopePolicyAddendum:wording must reflect the active scope."""- def _make_strategy(self, scope, primary_engine="duckduckgo"):+ def _make_strategy(self, scope, primary_engine="brave"):from local_deep_research.advanced_search_system.strategies.langgraph_agent_strategy import (LangGraphAgentStrategy,)mock_search = MagicMock()- mock_search.__class__.__name__ = "DuckDuckGoSearchEngine"+ mock_search.__class__.__name__ = "BraveSearchEngine"return LangGraphAgentStrategy(model=MagicMock(),search=mock_search,@@ -5911,7 +5910,7 @@ class TestEgressScopePolicyAddendum:},)- def _capture_prompt(self, scope, primary="duckduckgo"):+ def _capture_prompt(self, scope, primary="brave"):"""Run analyze_topic in a heavily-mocked harness and return the``system_prompt`` string passed to ``create_agent``. There's nosmaller public hook for the addendum — the prompt-string is@@ -5960,7 +5959,7 @@ class TestEgressScopePolicyAddendum:prompt = self._capture_prompt("strict")assert "RESTRICTED MODE" in prompt# The primary engine name must be cited.- assert "duckduckgo" in prompt.lower()+ assert "brave" in prompt.lower()def test_private_only_addendum_names_public_engines_as_unavailable(self):"""PRIVATE-ONLY addendum must explicitly warn the LLM that@@ -5984,17 +5983,17 @@ class TestEgressScopePolicyAddendum:# Names at least one canonical local tool.assert "library" in prompt.lower()- def test_both_scope_injects_no_policy_addendum(self):- """Under BOTH (default), the strategy MUST NOT inject any of the- three scope-specific marker phrases. Bleeding scope state into- every prompt would (a) bloat the default-case prompt for no- reason and (b) leak which scope the user picked even when they- didn't restrict anything.- """- prompt = self._capture_prompt("both")+ @pytest.mark.parametrize(+ "primary,public_addendum",+ [("brave", True), ("unclassified_test_primary", False)],+ )+ def test_legacy_both_scope_addendum_follows_primary(+ self, primary, public_addendum+ ):+ prompt = self._capture_prompt("both", primary=primary)assert "RESTRICTED MODE" not in promptassert "PRIVATE-ONLY MODE" not in prompt- assert "PUBLIC-ONLY MODE" not in prompt+ assert ("PUBLIC-ONLY MODE" in prompt) is public_addendum# ---------------------------------------------------------------------------@@ -6028,9 +6027,9 @@ class TestResearchSubtopicToolOverflow:else {})tool = _make_research_subtopic_tool(- search_engine_name="duckduckgo",+ search_engine_name="brave",model=MagicMock(),- settings_snapshot={"search.tool": {"value": "duckduckgo"}},+ settings_snapshot={"search.tool": {"value": "brave"}},collector=collector,max_sub_iterations=8,progress_callback=progress_callback,@@ -6468,7 +6467,7 @@ class TestFinalizeCitationLogging:model=MagicMock(),search=MagicMock(),all_links_of_system=all_links if all_links is not None else [],- settings_snapshot={"search.tool": {"value": "duckduckgo"}},+ settings_snapshot={"search.tool": {"value": "brave"}},citation_handler=citation_handler,)@@ -7184,7 +7183,7 @@ class TestFinalizeSourcesPayload:model=MagicMock(),search=MagicMock(),all_links_of_system=all_links if all_links is not None else [],- settings_snapshot={"search.tool": {"value": "duckduckgo"}},+ settings_snapshot={"search.tool": {"value": "brave"}},citation_handler=MagicMock(),)@@ -8532,3 +8531,47 @@ class TestSearchToolThreadCleanup:assert "t" in resultmock_cleanup.assert_not_called()++[email protected]("specialized", [False, True])+def test_wayback_tool_description_states_accepted_input(specialized):+ from local_deep_research.advanced_search_system.strategies.langgraph_agent_strategy import (+ _make_specialized_search_tool,+ _make_web_search_tool,+ )+ from local_deep_research.web_search_engines.engines.search_engine_wayback import (+ WaybackSearchEngine,+ )++ if specialized:+ search = _make_specialized_search_tool(+ "wayback", "Search archives.", MagicMock(), {}, MagicMock()+ )+ else:+ search = _make_web_search_tool("wayback", MagicMock(), {}, MagicMock())++ assert "full http:// or https:// URL" in search.description+ assert "or be only a bare domain" in search.description+ assert "Only the host is looked up" in search.description+ assert "configured search provider" in search.description+ # It also names the inputs the patterns miss or misread; each form is+ # pinned in test_wayback_archive_only.py.+ for unsupported in (+ "an uppercase scheme",+ "a bracketed IPv6 host",+ "a user name or password before the host",+ "a bare domain with a punycode (xn--) top-level domain",+ ):+ assert unsupported in search.description+ # The description is the parser's contract: the host of a full URL+ # anywhere in the input, but a bare domain only when it is the whole+ # input.+ engine = WaybackSearchEngine()+ assert engine._extract_urls_from_query("example.com") == [+ "http://example.com"+ ]+ assert engine._extract_urls_from_query("Find example.com") == []+ assert engine._extract_urls_from_query("example.com/page") == []+ assert engine._extract_urls_from_query("Find https://example.com/page") == [+ "https://example.com"+ ]diff --git a/tests/test_advertised_but_dead_sweep.py b/tests/test_advertised_but_dead_sweep.pyindex e44a8d0c59..dd8743e6bb 100644--- a/tests/test_advertised_but_dead_sweep.py+++ b/tests/test_advertised_but_dead_sweep.py@@ -1073,9 +1073,9 @@ def test_fallback_guard_rejects_unreadable_provider_id(monkeypatch):reason=("KNOWN DEFECT, pre-existing. ENGINE_REGISTRY calls itself the ""single source of truth for which class implements each engine, "- "but 'ddg' and 'guardian' have no settings block, so "+ "but 'guardian' has no settings block, so ""search_config()'s 'if name in search_engines' guard never fires "- "and the factory rejects both names. The classes are correct and "+ "and the factory rejects that name. The class is correct and ""importable; it is the offer that cannot be taken up. "),)@@ -1083,10 +1083,10 @@ def test_engine_registry_entries_are_reachable():"""Registry entries that name a real class but can never be selected.engine_registry.py calls itself "the single source of truth for which- Python module/class implements each search engine". Two of its entries —- ``ddg`` and ``guardian`` — have no ``search.engine.web.<name>.*`` settings,+ Python module/class implements each search engine". Its ``guardian`` entry+ has no ``search.engine.web.guardian.*`` settings,so ``search_engines_config``'s ``if name in search_engines`` guard never- fires for them and the factory rejects both names.+ fires for it and the factory rejects that name.Category: caller-facing dead registry entry. The classes themselves arecorrect and importable; it is the offer that cannot be taken up.diff --git a/tests/ui_tests/NO_CI_executes_research_full_research_completion.js b/tests/ui_tests/NO_CI_executes_research_full_research_completion.jsindex 3776815404..191cc14b7f 100644--- a/tests/ui_tests/NO_CI_executes_research_full_research_completion.js+++ b/tests/ui_tests/NO_CI_executes_research_full_research_completion.js@@ -69,7 +69,7 @@ async function testCompleteResearch() {model_provider: 'OLLAMA',model: 'llama3.1:8b', // Use available modelcustom_endpoint: '',- search_engine: 'duckduckgo', // Use search engine that doesn't need API key+ search_engine: 'wikipedia', // Use search engine that doesn't need API keyiterations: 1,questions_per_iteration: 2,strategy: 'source-based',diff --git a/tests/ui_tests/test_check_search_engines.js b/tests/ui_tests/test_check_search_engines.jsindex 7948cc27e8..80dea6a0d6 100644--- a/tests/ui_tests/test_check_search_engines.js+++ b/tests/ui_tests/test_check_search_engines.js@@ -70,8 +70,8 @@ async function checkSearchEngines() {});}- // Try to use duckduckgo which should work without API key- console.log('\n🧪 Testing research with DuckDuckGo...');+ // Wikipedia works without an API key.+ console.log('\n🧪 Testing research with Wikipedia...');const testResult = await page.evaluate(async () => {const response = await fetch('/api/start_research', {@@ -82,7 +82,7 @@ async function checkSearchEngines() {mode: 'quick',model_provider: 'OLLAMA',model: 'llama3.1:8b',- search_engine: 'duckduckgo', // Try duckduckgo+ search_engine: 'wikipedia',iterations: 1,questions_per_iteration: 1})diff --git a/tests/web_search_engines/engines/test_search_engine_ddg.py b/tests/web_search_engines/engines/test_search_engine_ddg.pydeleted file mode 100644index d382f059fb..0000000000--- a/tests/web_search_engines/engines/test_search_engine_ddg.py+++ /dev/null@@ -1,381 +0,0 @@-"""-Tests for the DuckDuckGoSearchEngine class.--Tests cover:-- Initialization and configuration-- Preview generation-- Full content retrieval-- Rate limit error handling-- Run method-"""--from unittest.mock import Mock, patch-import pytest---class TestDuckDuckGoSearchEngineInit:- """Tests for DuckDuckGoSearchEngine initialization."""-- def test_init_with_defaults(self):- """Initialize with default values."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper:- engine = DuckDuckGoSearchEngine()-- assert engine.max_results == 10- assert engine.region == "us"- assert engine.safe_search is True- assert engine.language == "English"- assert engine.include_full_content is False- mock_wrapper.assert_called_once()-- def test_init_with_custom_max_results(self):- """Initialize with custom max_results."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ):- engine = DuckDuckGoSearchEngine(max_results=25)-- assert engine.max_results == 25-- def test_init_with_custom_region(self):- """Initialize with custom region."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper:- engine = DuckDuckGoSearchEngine(region="uk")-- assert engine.region == "uk"- mock_wrapper.assert_called_with(- region="uk", max_results=10, safesearch="moderate"- )-- def test_init_with_safe_search_disabled(self):- """Initialize with safe_search disabled."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper:- engine = DuckDuckGoSearchEngine(safe_search=False)-- assert engine.safe_search is False- mock_wrapper.assert_called_with(- region="us", max_results=10, safesearch="off"- )-- def test_init_with_llm(self):- """Initialize with LLM."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- mock_llm = Mock()- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ):- engine = DuckDuckGoSearchEngine(llm=mock_llm)-- assert engine.llm is mock_llm-- def test_init_does_not_create_full_search(self):- """DDG no longer self-wraps in ``__init__``. The factory wrapper- is the canonical fetcher; the inner engine only emits snippets."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- mock_llm = Mock()- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ):- with patch(- "local_deep_research.web_search_engines.engines.full_search.FullSearchResults"- ) as mock_full_search:- engine = DuckDuckGoSearchEngine(- llm=mock_llm, include_full_content=True- )-- assert not hasattr(engine, "full_search")- mock_full_search.assert_not_called()---class TestGetPreviews:- """Tests for _get_previews method."""-- def test_get_previews_returns_results(self):- """Get previews returns formatted results."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- mock_results = [- {- "title": "Result 1",- "snippet": "Snippet 1",- "link": "https://example1.com",- },- {- "title": "Result 2",- "snippet": "Snippet 2",- "link": "https://example2.com",- },- ]-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper_class:- mock_wrapper = Mock()- mock_wrapper.results.return_value = mock_results- mock_wrapper_class.return_value = mock_wrapper-- engine = DuckDuckGoSearchEngine()- previews = engine._get_previews("test query")-- assert len(previews) == 2- assert previews[0]["title"] == "Result 1"- assert previews[0]["snippet"] == "Snippet 1"- assert previews[0]["link"] == "https://example1.com"- assert previews[0]["id"] == "https://example1.com"-- def test_get_previews_empty_results(self):- """Get previews handles empty results."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper_class:- mock_wrapper = Mock()- mock_wrapper.results.return_value = []- mock_wrapper_class.return_value = mock_wrapper-- engine = DuckDuckGoSearchEngine()- previews = engine._get_previews("test query")-- assert previews == []-- def test_get_previews_non_list_results(self):- """Get previews handles non-list results."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper_class:- mock_wrapper = Mock()- mock_wrapper.results.return_value = None- mock_wrapper_class.return_value = mock_wrapper-- engine = DuckDuckGoSearchEngine()- previews = engine._get_previews("test query")-- assert previews == []-- def test_get_previews_rate_limit_error(self):- """Get previews raises RateLimitError on rate limit."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )- from local_deep_research.web_search_engines.rate_limiting import (- RateLimitError,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper_class:- mock_wrapper = Mock()- mock_wrapper.results.side_effect = Exception(- "202 Ratelimit exceeded"- )- mock_wrapper_class.return_value = mock_wrapper-- engine = DuckDuckGoSearchEngine()-- with pytest.raises(RateLimitError):- engine._get_previews("test query")-- def test_get_previews_forbidden_error(self):- """Get previews raises RateLimitError on 403 forbidden."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )- from local_deep_research.web_search_engines.rate_limiting import (- RateLimitError,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper_class:- mock_wrapper = Mock()- mock_wrapper.results.side_effect = Exception("403 Forbidden")- mock_wrapper_class.return_value = mock_wrapper-- engine = DuckDuckGoSearchEngine()-- with pytest.raises(RateLimitError):- engine._get_previews("test query")-- def test_get_previews_timeout_error(self):- """Get previews raises RateLimitError on timeout."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )- from local_deep_research.web_search_engines.rate_limiting import (- RateLimitError,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper_class:- mock_wrapper = Mock()- mock_wrapper.results.side_effect = Exception("Connection timed out")- mock_wrapper_class.return_value = mock_wrapper-- engine = DuckDuckGoSearchEngine()-- with pytest.raises(RateLimitError):- engine._get_previews("test query")-- def test_get_previews_general_error(self):- """Get previews returns empty on general error."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ) as mock_wrapper_class:- mock_wrapper = Mock()- mock_wrapper.results.side_effect = Exception("Some other error")- mock_wrapper_class.return_value = mock_wrapper-- engine = DuckDuckGoSearchEngine()- previews = engine._get_previews("test query")-- assert previews == []---class TestGetFullContent:- """Tests for _get_full_content method.-- Engines no longer self-wrap, so ``_get_full_content`` is a- pass-through. The factory wrapper populates ``full_content`` after- the inner engine runs.- """-- def test_get_full_content_passes_through(self):- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ):- engine = DuckDuckGoSearchEngine()-- items = [{"title": "Result", "link": "https://example.com"}]- assert engine._get_full_content(items) is items-- def test_engine_does_not_self_wrap(self):- """DDG must not carry a ``full_search`` attribute; the factory- wrapper handles full-content retrieval."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ):- engine = DuckDuckGoSearchEngine(include_full_content=True)-- assert not hasattr(engine, "full_search")---class TestRun:- """Tests for run method."""-- def test_run_calls_parent(self):- """Run calls parent class run method."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ):- engine = DuckDuckGoSearchEngine()-- with patch.object(- engine,- "_get_previews",- return_value=[- {- "id": "https://example.com",- "title": "Result",- "snippet": "Snippet",- "link": "https://example.com",- }- ],- ):- with patch.object(- engine,- "_get_full_content",- return_value=[{"title": "Result"}],- ):- results = engine.run("test query")-- assert len(results) == 1-- def test_run_handles_empty_results(self):- """Run handles empty results."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- with patch(- "local_deep_research.web_search_engines.engines.search_engine_ddg.DuckDuckGoSearchAPIWrapper"- ):- engine = DuckDuckGoSearchEngine()-- with patch.object(engine, "_get_previews", return_value=[]):- results = engine.run("test query")-- assert results == []---class TestClassAttributes:- """Tests for class attributes."""-- def test_is_public(self):- """DuckDuckGoSearchEngine is marked as public."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- assert DuckDuckGoSearchEngine.is_public is True-- def test_is_generic(self):- """DuckDuckGoSearchEngine is marked as generic."""- from local_deep_research.web_search_engines.engines.search_engine_ddg import (- DuckDuckGoSearchEngine,- )-- assert DuckDuckGoSearchEngine.is_generic is Truediff --git a/tests/web_search_engines/engines/test_search_engine_wayback.py b/tests/web_search_engines/engines/test_search_engine_wayback.pyindex a6d38b19a9..47be632d94 100644--- a/tests/web_search_engines/engines/test_search_engine_wayback.py+++ b/tests/web_search_engines/engines/test_search_engine_wayback.py@@ -89,7 +89,7 @@ class TestExtractUrlsFromQuery:# Note: The regex only captures domain, not pathurls = engine._extract_urls_from_query(- "Check https://example.com archive"+ "Check https://example.com/page archive")assert len(urls) == 1@@ -122,8 +122,8 @@ class TestExtractUrlsFromQuery:assert len(urls) == 1assert urls[0] == "http://example.com"- def test_extract_partial_url(self):- """Extract partial URL with path."""+ def test_partial_url_without_scheme_is_not_extracted(self):+ """A domain with a path but no scheme is not an archive target."""from local_deep_research.web_search_engines.engines.search_engine_wayback import (WaybackSearchEngine,)@@ -132,8 +132,7 @@ class TestExtractUrlsFromQuery:urls = engine._extract_urls_from_query("example.com/page")- assert len(urls) == 1- assert urls[0] == "http://example.com/page"+ assert urls == []def test_extract_no_urls_found(self):"""Return empty when no URLs found."""@@ -143,12 +142,7 @@ class TestExtractUrlsFromQuery:engine = WaybackSearchEngine()- # Patch at the langchain_community location since it's imported inside the method- with patch(- "langchain_community.utilities.DuckDuckGoSearchAPIWrapper"- ) as mock_ddg:- mock_ddg.return_value.results.return_value = []- urls = engine._extract_urls_from_query("just some text")+ urls = engine._extract_urls_from_query("just some text")assert urls == []diff --git a/tests/web_search_engines/engines/test_search_engine_wayback_coverage.py b/tests/web_search_engines/engines/test_search_engine_wayback_coverage.pyindex 8f67d5e470..0d1f9663ad 100644--- a/tests/web_search_engines/engines/test_search_engine_wayback_coverage.py+++ b/tests/web_search_engines/engines/test_search_engine_wayback_coverage.py@@ -3,8 +3,8 @@ Coverage tests for WaybackSearchEngine targeting statements not hit bythe existing test_search_engine_wayback.py.Covers:-- _extract_urls_from_query: DuckDuckGo returns URLs; DuckDuckGo raises; query- contains dot+slash (partial URL branch); query has dot but no slash (domain branch)+- _extract_urls_from_query: plain text and partial URLs without a scheme+ return no targets; a bare domain is accepted without provider discovery- _format_timestamp: exception path returns raw timestamp- _get_wayback_snapshots: CDX rate-limit (429); CDX response with only header row;generic exception is swallowed and returns []@@ -51,51 +51,16 @@ def _mock_response(status=200, json_body=None, text=""):class TestExtractUrlsFromQueryExtra:- def test_ddg_returns_urls(self):- engine = _engine()- ddg_results = [- {"link": "https://found.com"},- {"link": "https://also.com"},+ def test_plain_text_has_no_archive_target(self):+ assert _engine()._extract_urls_from_query("some plain text query") == []++ def test_partial_url_without_scheme_has_no_archive_target(self):+ assert _engine()._extract_urls_from_query("example.com/some/path") == []++ def test_domain_without_protocol(self):+ assert _engine()._extract_urls_from_query("example.org") == [+ "http://example.org"]- with patch(- "langchain_community.utilities.DuckDuckGoSearchAPIWrapper"- ) as MockDDG:- MockDDG.return_value.results.return_value = ddg_results- urls = engine._extract_urls_from_query("some plain text query")- assert "https://found.com" in urls- assert "https://also.com" in urls-- def test_ddg_raises_falls_through_to_empty(self):- engine = _engine()- with patch(- "langchain_community.utilities.DuckDuckGoSearchAPIWrapper"- ) as MockDDG:- MockDDG.return_value.results.side_effect = RuntimeError("ddg down")- urls = engine._extract_urls_from_query("no dot here just words")- assert urls == []-- def test_partial_url_dot_and_slash_branch(self):- """query with '.' and '/' but no http prefix hits the dot+slash branch."""- engine = _engine()- with patch(- "langchain_community.utilities.DuckDuckGoSearchAPIWrapper"- ) as MockDDG:- MockDDG.return_value.results.return_value = []- urls = engine._extract_urls_from_query("example.com/some/path")- assert urls == ["http://example.com/some/path"]-- def test_domain_only_dot_no_slash_branch(self):- """query with '.' but no '/' hits the domain branch after DuckDuckGo fails."""- engine = _engine()- with patch(- "langchain_community.utilities.DuckDuckGoSearchAPIWrapper"- ) as MockDDG:- # Results list has no 'link' values → ddg_urls will be empty- MockDDG.return_value.results.return_value = [{"snippet": "no link"}]- urls = engine._extract_urls_from_query("example.org")- # The domain-without-http regex should have matched first and returned early- # OR the dot-no-slash fallback fires. Either way the result includes the domain.- assert any("example.org" in u for u in urls)# ---------------------------------------------------------------------------diff --git a/tests/web_search_engines/test_engine_success_path.py b/tests/web_search_engines/test_engine_success_path.pyindex 777de16fb0..378a4dfa10 100644--- a/tests/web_search_engines/test_engine_success_path.py+++ b/tests/web_search_engines/test_engine_success_path.py@@ -18,10 +18,10 @@ What this file pins that the per-engine unit tests do not:1. **The whole registry, not a favourite subset.** Every entry in``ENGINE_REGISTRY`` must resolve to an importable ``BaseSearchEngine``- subclass, and must be constructible without a live backend. Three- entries are not (``ddg``, ``elasticsearch``, ``google_pse``), one more- constructs into a permanently empty engine (``searxng``), and two are- unreachable through the factory at all (``ddg``, ``guardian``) --+ subclass, and must be constructible without a live backend. Two+ entries are not (``elasticsearch``, ``google_pse``), one more+ constructs into a permanently empty engine (``searxng``), and one is+ unreachable through the factory at all (``guardian``) --each pinned below.2. **"Usable", not "a list".** For each engine the first preview isasserted field by field against the stubbed upstream document: the@@ -77,9 +77,6 @@ from local_deep_research.web_search_engines.engines import (from local_deep_research.web_search_engines.engines import (search_engine_collection as collection_engine,)-from local_deep_research.web_search_engines.engines import (- search_engine_ddg as ddg_engine,-)from local_deep_research.web_search_engines.engines import (search_engine_elasticsearch as es_engine,)@@ -1924,9 +1921,9 @@ CASES_BY_NAME = {case.name: case for case in CASES}# ``search.engine.web.*`` settings and only *decorates* the entries it finds# with the registry's module/class, so an engine with no settings never# enters the dict and ``create_search_engine`` fails closed on it with-# "Unknown search engine". These two are therefore unreachable from the UI,+# "Unknown search engine". This entry is therefore unreachable from the UI,# the API and the LangGraph tool list alike.-REGISTRY_ENGINES_WITHOUT_SETTINGS = {"ddg", "guardian"}+REGISTRY_ENGINES_WITHOUT_SETTINGS = {"guardian"}def _shipped_engine_names():@@ -1984,15 +1981,15 @@ class TestRegistryIsWiredUp:"case in this file")- def test_ddg_and_guardian_are_unreachable_through_the_factory(self):- """DEFECT: two registry entries no settings file defines.+ def test_guardian_is_unreachable_through_the_factory(self):+ """DEFECT: a registry entry no settings file defines.``search_config()`` is settings-derived: it reads``search.engine.web.*`` and only *decorates* the names it findswith the registry's module/class. An engine with no shippedsettings therefore never enters the dict, and``create_search_engine`` fails closed on it with "Unknown search- engine". ``ddg`` and ``guardian`` are dead registry rows --+ engine". ``guardian`` is a dead registry row --unreachable from the UI, the API and the agent's tool list. A fixeither ships settings for them or drops the rows."""@@ -2008,23 +2005,6 @@ class TestRegistryIsWiredUp:class TestEnginesThatCannotBeConstructed:"""Registry rows whose class cannot be built without a live backend."""- def test_ddg_cannot_be_constructed_at_all(self):- """DEFECT: the DuckDuckGo engine is dead on a stock install.-- ``pyproject.toml`` pins ``duckduckgo-search~=8.1``, but- ``langchain-community~=0.4`` (0.4.2 here) migrated- ``DuckDuckGoSearchAPIWrapper`` onto the renamed ``ddgs``- distribution, which nothing declares. Constructing the wrapper --- which ``DuckDuckGoSearchEngine.__init__`` does eagerly -- raises- ImportError, so the engine can never run. The blast radius is- wider than the ``ddg`` row itself: ``WaybackSearchEngine`` uses- the same wrapper to turn a plain question into URLs (see below).- """- with pytest.raises(ImportError, match="ddgs"):- ddg_engine.DuckDuckGoSearchEngine(- max_results=3, llm=None, settings_snapshot=SNAPSHOT- )-def test_elasticsearch_raises_when_the_cluster_is_down(self, inert_tracker):"""Elasticsearch probes the cluster inside ``__init__``.@@ -2372,15 +2352,10 @@ class TestDownstreamSilentDrops:def test_wayback_returns_nothing_for_a_plain_research_question(self, inert_tracker):- """DEFECT: Wayback is unusable for ordinary queries.-- ``_extract_urls_from_query`` needs a URL. For a normal question- it falls back to ``DuckDuckGoSearchAPIWrapper`` to discover- URLs -- the wrapper that cannot even be constructed on this- install (see ``test_ddg_cannot_be_constructed_at_all``). The- remaining heuristics only fire on strings containing a dot, so a- research question yields zero snapshots. Wayback ships with- settings and is offered in the UI regardless.+ """Wayback looks up hosts of supplied URLs; topics need discovery.++ The same adapter and archive payload produce usable results for+ explicit URLs, while plain topics make no archive request."""engine = _wire(wayback_engine.WaybackSearchEngine(diff --git a/tests/web_search_engines/test_search_engine_factory.py b/tests/web_search_engines/test_search_engine_factory.pyindex fb3d0fc554..1f8cb302a2 100644--- a/tests/web_search_engines/test_search_engine_factory.py+++ b/tests/web_search_engines/test_search_engine_factory.py@@ -342,7 +342,7 @@ class TestGetSearch:)result = get_search(- search_tool="duckduckgo",+ search_tool="brave",llm_instance=mock_llm,max_results=10,settings_snapshot={"test": "value"},@@ -351,8 +351,8 @@ class TestGetSearch:mock_create.assert_called_once()assert result == mock_engine- def test_get_search_with_duckduckgo_params(self):- """Test get_search with DuckDuckGo specific params."""+ def test_get_search_with_brave_params(self):+ """Test get_search with Brave specific params."""mock_llm = Mock()mock_engine = Mock()@@ -366,7 +366,7 @@ class TestGetSearch:)get_search(- search_tool="duckduckgo",+ search_tool="brave",llm_instance=mock_llm,max_results=20,region="uk",@@ -699,7 +699,7 @@ class TestGetSearch:mock_create.return_value = Mock()get_search(- search_tool="duckduckgo",+ search_tool="brave",llm_instance=mock_llm,max_results=10,region="uk",