~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
fix(settings): add agent_enabled and use_in_auto_search flags for specialized and key-gated search engines (#6519)
Sa7a8d2b109c0
diff --git a/changelog.d/6519.bugfix.md b/changelog.d/6519.bugfix.mdnew file mode 100644index 0000000000..0955f748a5--- /dev/null+++ b/changelog.d/6519.bugfix.md@@ -0,0 +1,5 @@+Add explicit `agent_enabled` and `use_in_auto_search` configuration settings for specialized domain and key-gated search engines.++Specialized domain engines (Project Gutenberg, Open Library, PubChem, Stack Exchange, Zenodo) and private or domain-specific key-gated engines (NASA ADS, Paperless) now default to disabled for the research agent (`agent_enabled: false`). Because unconfigured engines previously fell back to enabled in `search_engines_config.py`, these engines were silently available as specialized agent tools; users who rely on them can explicitly re-enable them via the **Available to Research Agent** checkbox in Settings → Search Engines.++General web scraper engines (Scale SERP, Serper) now expose explicit configuration controls while preserving their prior default behavior (`agent_enabled: true`, `use_in_auto_search: false`).diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.mdindex c33484796a..177fd5eeec 100644--- a/docs/CONFIGURATION.md+++ b/docs/CONFIGURATION.md@@ -313,12 +313,14 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.google_pse.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_GOOGLE_PSE_USE_IN_AUTO_SEARCH` | `false` | Include Google PSE in auto search mode | SEARCH || `search.engine.web.google_pse.weaknesses` | `LDR_SEARCH_ENGINE_WEB_GOOGLE_PSE_WEAKNESSES` | ``["requires API key with usage limits", "limited to 10,000 queries/day on free tier", "requires search engine configuration in Google Control Panel"]`` | Limitations: Requires API key, limited to 10,000 queries/day on free tier, requires search engine configuration in Google Control Panel. | SEARCH || `search.engine.web.gutenberg` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG` | ``{"requires_api_key": false, "requires_llm": true, "description": "Search 70,000+ free public domain books with full text via Gutendex.", "reliability": 0.95}`` | Project Gutenberg search engine configuration | SEARCH |+| `search.engine.web.gutenberg.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG_AGENT_ENABLED` | `false` | Include Project Gutenberg as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.gutenberg.default_params.max_results` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG_DEFAULT_PARAMS_MAX_RESULTS` | `20` | Maximum number of results to retrieve from Project Gutenberg. | SEARCH || `search.engine.web.gutenberg.description` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG_DESCRIPTION` | `Search 70,000+ free public domain books with full text via Gutendex.` | Human-readable description of the search engine. | SEARCH || `search.engine.web.gutenberg.display_name` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG_DISPLAY_NAME` | `Project Gutenberg` | Display name for Project Gutenberg search engine. | SEARCH || `search.engine.web.gutenberg.reliability` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG_RELIABILITY` | `0.95` | Reliability score for Project Gutenberg. | SEARCH || `search.engine.web.gutenberg.requires_api_key` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG_REQUIRES_API_KEY` | `false` | Project Gutenberg is free and does not require an API key. | SEARCH || `search.engine.web.gutenberg.strengths` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG_STRENGTHS` | ``["public domain", "full text", "classic literature", "free ebooks", "multiple formats"]`` | Key strengths of Project Gutenberg. | SEARCH |+| `search.engine.web.gutenberg.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG_USE_IN_AUTO_SEARCH` | `false` | Include Project Gutenberg in auto search mode | SEARCH || `search.engine.web.gutenberg.weaknesses` | `LDR_SEARCH_ENGINE_WEB_GUTENBERG_WEAKNESSES` | ``["public domain only", "older works", "limited modern titles"]`` | Limitations of Project Gutenberg. | SEARCH || `search.engine.web.mojeek.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_MOJEEK_AGENT_ENABLED` | `true` | Include Mojeek as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.mojeek.api_key` | `LDR_SEARCH_ENGINE_WEB_MOJEEK_API_KEY` | `` | The Mojeek API key to use. Get one at mojeek.com/services/api. | SEARCH |@@ -334,6 +336,7 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.mojeek.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_MOJEEK_USE_IN_AUTO_SEARCH` | `false` | Include Mojeek in auto search mode | SEARCH || `search.engine.web.mojeek.weaknesses` | `LDR_SEARCH_ENGINE_WEB_MOJEEK_WEAKNESSES` | ``["smaller index than Google/Bing", "requires paid API key"]`` | Limitations of Mojeek: Smaller index than Google/Bing, requires paid API key. | SEARCH || `search.engine.web.nasa_ads` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS` | ``{"requires_api_key": true, "requires_llm": true, "description": "Search millions of astronomy and physics papers with natural language queries", "reliability": 0.95}`` | NASA Astrophysics Data System search engine configuration | SEARCH |+| `search.engine.web.nasa_ads.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS_AGENT_ENABLED` | `false` | Include NASA ADS as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.nasa_ads.api_key` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS_API_KEY` | `` | NASA ADS API key (required). Get one at https://ui.adsabs.harvard.edu/user/settings/token | SEARCH || `search.engine.web.nasa_ads.default_params.from_publication_date` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS_DEFAULT_PARAMS_FROM_PUBLICATION_DATE` | `` | Only return papers published after this date (YYYY-MM-DD format, leave empty for all dates) | SEARCH || `search.engine.web.nasa_ads.default_params.include_arxiv` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS_DEFAULT_PARAMS_INCLUDE_ARXIV` | `true` | Include ArXiv preprints in search results | SEARCH |@@ -346,6 +349,7 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.nasa_ads.reliability` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS_RELIABILITY` | `0.95` | Reliability score for NASA ADS search results (0.0-1.0, higher is better) | SEARCH || `search.engine.web.nasa_ads.requires_api_key` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS_REQUIRES_API_KEY` | `true` | Whether NASA ADS requires an API key (always true for NASA ADS) | SEARCH || `search.engine.web.nasa_ads.strengths` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS_STRENGTHS` | ``["astronomy papers", "astrophysics research", "physics papers", "space science", "planetary science", "cosmology", "high energy physics", "historical papers", "citation tracking", "natural language queries"]`` | List of research areas where NASA ADS excels | SEARCH |+| `search.engine.web.nasa_ads.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS_USE_IN_AUTO_SEARCH` | `false` | Include NASA ADS in auto search mode | SEARCH || `search.engine.web.nasa_ads.weaknesses` | `LDR_SEARCH_ENGINE_WEB_NASA_ADS_WEAKNESSES` | ``["biomedical content", "computer science papers", "social sciences", "humanities", "requires API key for good performance"]`` | List of research areas where NASA ADS has limited coverage | SEARCH || `search.engine.web.openalex` | `LDR_SEARCH_ENGINE_WEB_OPENALEX` | ``{"requires_api_key": false, "requires_llm": true, "description": "Search 250+ million academic papers with natural language queries", "reliability": 0.95}`` | OpenAlex academic search engine configuration | SEARCH || `search.engine.web.openalex.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_OPENALEX_AGENT_ENABLED` | `true` | Include OpenAlex as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH |@@ -364,13 +368,16 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.openalex.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_OPENALEX_USE_IN_AUTO_SEARCH` | `true` | Include OpenAlex in auto search mode | SEARCH || `search.engine.web.openalex.weaknesses` | `LDR_SEARCH_ENGINE_WEB_OPENALEX_WEAKNESSES` | ``["non-academic content", "news articles", "general web content", "real-time information"]`` | List of content types where OpenAlex has limited coverage | SEARCH || `search.engine.web.openlibrary` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY` | ``{"requires_api_key": false, "requires_llm": true, "description": "Search 2M+ books with metadata, covers, and full text from Internet Archive.", "reliability": 0.9}`` | Open Library search engine configuration | SEARCH |+| `search.engine.web.openlibrary.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY_AGENT_ENABLED` | `false` | Include Open Library as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.openlibrary.default_params.max_results` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY_DEFAULT_PARAMS_MAX_RESULTS` | `20` | Maximum number of results to retrieve from Open Library. | SEARCH || `search.engine.web.openlibrary.description` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY_DESCRIPTION` | `Search 2M+ books with metadata, covers, and full text from Internet Archive.` | Human-readable description of the search engine. | SEARCH || `search.engine.web.openlibrary.display_name` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY_DISPLAY_NAME` | `Open Library` | Display name for Open Library search engine. | SEARCH || `search.engine.web.openlibrary.reliability` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY_RELIABILITY` | `0.9` | Reliability score for Open Library. | SEARCH || `search.engine.web.openlibrary.requires_api_key` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY_REQUIRES_API_KEY` | `false` | Open Library is free and does not require an API key. | SEARCH || `search.engine.web.openlibrary.strengths` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY_STRENGTHS` | ``["books", "literature", "metadata", "covers", "full text", "Internet Archive"]`` | Key strengths of Open Library. | SEARCH |+| `search.engine.web.openlibrary.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY_USE_IN_AUTO_SEARCH` | `false` | Include Open Library in auto search mode | SEARCH || `search.engine.web.openlibrary.weaknesses` | `LDR_SEARCH_ENGINE_WEB_OPENLIBRARY_WEAKNESSES` | ``["books only", "variable metadata quality", "English focus"]`` | Limitations of Open Library. | SEARCH |+| `search.engine.web.paperless.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_PAPERLESS_AGENT_ENABLED` | `false` | Include Paperless as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.paperless.api_key` | `PAPERLESS_API_TOKEN` | `` | API key for authentication (get from Paperless admin panel) | SEARCH || `search.engine.web.paperless.default_params.api_url` | `LDR_SEARCH_ENGINE_WEB_PAPERLESS_DEFAULT_PARAMS_API_URL` | `http://localhost:8000` | URL of your Paperless-ngx instance (e.g., http://localhost:8000) | SEARCH || `search.engine.web.paperless.default_params.include_content` | `LDR_SEARCH_ENGINE_WEB_PAPERLESS_DEFAULT_PARAMS_INCLUDE_CONTENT` | `true` | Include full document content (not just snippets) | SEARCH |@@ -385,8 +392,10 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.paperless.requires_llm` | `LDR_SEARCH_ENGINE_WEB_PAPERLESS_REQUIRES_LLM` | `true` | Whether this engine requires an LLM for operation | SEARCH || `search.engine.web.paperless.strengths` | `LDR_SEARCH_ENGINE_WEB_PAPERLESS_STRENGTHS` | ``["personal documents", "OCR-processed content", "tagged and organized", "full-text search", "no rate limits", "private data"]`` | Strengths of this search engine | SEARCH || `search.engine.web.paperless.supports_full_search` | `LDR_SEARCH_ENGINE_WEB_PAPERLESS_SUPPORTS_FULL_SEARCH` | `true` | Whether this engine supports full document search | SEARCH |+| `search.engine.web.paperless.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_PAPERLESS_USE_IN_AUTO_SEARCH` | `false` | Include Paperless in auto search mode | SEARCH || `search.engine.web.paperless.weaknesses` | `LDR_SEARCH_ENGINE_WEB_PAPERLESS_WEAKNESSES` | ``["keyword-based search only", "no semantic understanding", "limited query operators"]`` | Weaknesses of this search engine | SEARCH || `search.engine.web.pubchem` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM` | ``{"requires_api_key": false, "requires_llm": true, "description": "Search millions of chemical compounds with structures, properties, and bioactivity data.", "reliability": 0.95}`` | PubChem search engine configuration | SEARCH |+| `search.engine.web.pubchem.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM_AGENT_ENABLED` | `false` | Include PubChem as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.pubchem.default_params.include_synonyms` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM_DEFAULT_PARAMS_INCLUDE_SYNONYMS` | `true` | Include compound synonyms in full content results. | SEARCH || `search.engine.web.pubchem.default_params.max_results` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM_DEFAULT_PARAMS_MAX_RESULTS` | `10` | Maximum number of results to retrieve from PubChem. | SEARCH || `search.engine.web.pubchem.description` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM_DESCRIPTION` | `Search millions of chemical compounds with structures, properties, and bioactivity data.` | Human-readable description of the search engine. | SEARCH |@@ -394,6 +403,7 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.pubchem.reliability` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM_RELIABILITY` | `0.95` | Reliability score for PubChem. | SEARCH || `search.engine.web.pubchem.requires_api_key` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM_REQUIRES_API_KEY` | `false` | PubChem is free and does not require an API key. | SEARCH || `search.engine.web.pubchem.strengths` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM_STRENGTHS` | ``["chemistry", "compounds", "drug information", "molecular properties", "bioactivity"]`` | Key strengths of PubChem. | SEARCH |+| `search.engine.web.pubchem.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM_USE_IN_AUTO_SEARCH` | `false` | Include PubChem in auto search mode | SEARCH || `search.engine.web.pubchem.weaknesses` | `LDR_SEARCH_ENGINE_WEB_PUBCHEM_WEAKNESSES` | ``["chemistry only", "technical data", "requires chemical knowledge"]`` | Limitations of PubChem. | SEARCH || `search.engine.web.pubmed.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_PUBMED_AGENT_ENABLED` | `true` | Include PubMed as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.pubmed.api_key` | `LDR_SEARCH_ENGINE_WEB_PUBMED_API_KEY` | `` | The PubMed API key to use. | SEARCH |@@ -425,6 +435,7 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.pubmed.strengths` | `LDR_SEARCH_ENGINE_WEB_PUBMED_STRENGTHS` | ``["biomedical literature", "medical research", "clinical studies", "life sciences", "health information", "scientific papers"]`` | Key advantages of PubMed: Peer-reviewed biomedical literature, MeSH indexing, abstracts, citation data, free access. | SEARCH || `search.engine.web.pubmed.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_PUBMED_USE_IN_AUTO_SEARCH` | `true` | Include PubMed in auto search mode | SEARCH || `search.engine.web.pubmed.weaknesses` | `LDR_SEARCH_ENGINE_WEB_PUBMED_WEAKNESSES` | ``["non-medical topics", "very recent papers may be missing", "limited to published research"]`` | Limitations of PubMed: Limited to biomedical literature, keyword-based search, full-text only for open-access papers. | SEARCH |+| `search.engine.web.scaleserp.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_SCALESERP_AGENT_ENABLED` | `true` | Include Scale SERP as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.scaleserp.api_key` | `LDR_SEARCH_ENGINE_WEB_SCALESERP_API_KEY` | `` | The ScaleSerp API key to use. Get yours at https://scaleserp.com | SEARCH || `search.engine.web.scaleserp.default_params.device` | `LDR_SEARCH_ENGINE_WEB_SCALESERP_DEFAULT_PARAMS_DEVICE` | `desktop` | Device type for search results | SEARCH || `search.engine.web.scaleserp.default_params.enable_cache` | `LDR_SEARCH_ENGINE_WEB_SCALESERP_DEFAULT_PARAMS_ENABLE_CACHE` | `true` | Enable 1-hour caching to save API credits (cached searches are FREE within 1 hour) | SEARCH |@@ -439,6 +450,7 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.scaleserp.requires_llm` | `LDR_SEARCH_ENGINE_WEB_SCALESERP_REQUIRES_LLM` | `false` | Whether this search engine requires an LLM for relevance filtering | SEARCH || `search.engine.web.scaleserp.strengths` | `LDR_SEARCH_ENGINE_WEB_SCALESERP_STRENGTHS` | ``["1-hour free caching for repeated searches", "High-quality Google search results", "Real-time results with knowledge graph", "Cached searches don't count against quota", "Rich snippets and related questions"]`` | Strengths of this search engine | SEARCH || `search.engine.web.scaleserp.supports_full_search` | `LDR_SEARCH_ENGINE_WEB_SCALESERP_SUPPORTS_FULL_SEARCH` | `true` | Whether this search engine supports full document search | SEARCH |+| `search.engine.web.scaleserp.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_SCALESERP_USE_IN_AUTO_SEARCH` | `false` | Include Scale SERP in auto search mode | SEARCH || `search.engine.web.scaleserp.weaknesses` | `LDR_SEARCH_ENGINE_WEB_SCALESERP_WEAKNESSES` | ``["Requires API key with usage limits", "Not specialized for academic content", "Cache only lasts 1 hour"]`` | Weaknesses of this search engine | SEARCH || `search.engine.web.searxng.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_SEARXNG_AGENT_ENABLED` | `true` | Include SearXNG as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.searxng.default_params.categories` | `LDR_SEARCH_ENGINE_WEB_SEARXNG_DEFAULT_PARAMS_CATEGORIES` | ``["general"]`` | SearXNG search categories to use (general, images, videos, news, music, files, etc.). | SEARCH |@@ -491,6 +503,7 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.serpapi.supports_full_search` | `LDR_SEARCH_ENGINE_WEB_SERPAPI_SUPPORTS_FULL_SEARCH` | `true` | Indicates this engine can fetch full page content after initial search, not just snippets. | SEARCH || `search.engine.web.serpapi.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_SERPAPI_USE_IN_AUTO_SEARCH` | `false` | Include SerpAPI in auto search mode | SEARCH || `search.engine.web.serpapi.weaknesses` | `LDR_SEARCH_ENGINE_WEB_SERPAPI_WEAKNESSES` | ``["requires API key with usage limits", "not specialized for academic content"]`` | Limitations: Requires paid API key with usage limits, not specialized for academic or technical content. | SEARCH |+| `search.engine.web.serper.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_SERPER_AGENT_ENABLED` | `true` | Include Serper as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.serper.api_key` | `LDR_SEARCH_ENGINE_WEB_SERPER_API_KEY` | `` | The Serper API key to use. Get yours at https://serper.dev | SEARCH || `search.engine.web.serper.default_params.include_full_content` | `LDR_SEARCH_ENGINE_WEB_SERPER_DEFAULT_PARAMS_INCLUDE_FULL_CONTENT` | `false` | Include full webpage content in results (may slow down search) | SEARCH || `search.engine.web.serper.default_params.region` | `LDR_SEARCH_ENGINE_WEB_SERPER_DEFAULT_PARAMS_REGION` | `us` | Country code for localized results (e.g., 'us', 'gb', 'fr') | SEARCH |@@ -504,6 +517,7 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.serper.requires_llm` | `LDR_SEARCH_ENGINE_WEB_SERPER_REQUIRES_LLM` | `false` | Whether this search engine requires an LLM for relevance filtering | SEARCH || `search.engine.web.serper.strengths` | `LDR_SEARCH_ENGINE_WEB_SERPER_STRENGTHS` | ``["Lightning-fast response times (1-2 seconds)", "Very affordable pricing", "High-quality Google search results", "Real-time results", "Knowledge graph and related searches", "People Also Ask data", "Simple pay-per-use pricing"]`` | Strengths of this search engine | SEARCH || `search.engine.web.serper.supports_full_search` | `LDR_SEARCH_ENGINE_WEB_SERPER_SUPPORTS_FULL_SEARCH` | `true` | Whether this search engine supports full document search | SEARCH |+| `search.engine.web.serper.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_SERPER_USE_IN_AUTO_SEARCH` | `false` | Include Serper in auto search mode | SEARCH || `search.engine.web.serper.weaknesses` | `LDR_SEARCH_ENGINE_WEB_SERPER_WEAKNESSES` | ``["Requires API key with usage limits", "Not specialized for academic content"]`` | Weaknesses of this search engine | SEARCH || `search.engine.web.serply.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_SERPLY_AGENT_ENABLED` | `true` | Include Serply as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool; this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.serply.api_key` | `LDR_SEARCH_ENGINE_WEB_SERPLY_API_KEY` | `` | The Serply API key to use. Get yours at https://serply.io | SEARCH |@@ -532,6 +546,7 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.sofya.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_SOFYA_USE_IN_AUTO_SEARCH` | `false` | Whether to include this engine in auto/meta search selection | SEARCH || `search.engine.web.sofya.weaknesses` | `LDR_SEARCH_ENGINE_WEB_SOFYA_WEAKNESSES` | ``["Requires API key with credit-based usage limits", "Not specialized for academic content"]`` | Weaknesses of this search engine | SEARCH || `search.engine.web.stackexchange` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE` | ``{"requires_api_key": false, "requires_llm": true, "description": "Search Stack Overflow and other Q&A sites for programming answers.", "reliability": 0.9}`` | Stack Exchange search engine configuration | SEARCH |+| `search.engine.web.stackexchange.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE_AGENT_ENABLED` | `false` | Include Stack Exchange as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.stackexchange.default_params.max_results` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE_DEFAULT_PARAMS_MAX_RESULTS` | `20` | Maximum number of results to retrieve from Stack Exchange. | SEARCH || `search.engine.web.stackexchange.default_params.site` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE_DEFAULT_PARAMS_SITE` | `stackoverflow` | Stack Exchange site to search (stackoverflow, serverfault, superuser, etc.). | SEARCH || `search.engine.web.stackexchange.description` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE_DESCRIPTION` | `Search Stack Overflow and other Q&A sites for programming answers.` | Human-readable description of the search engine. | SEARCH |@@ -539,6 +554,7 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.stackexchange.reliability` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE_RELIABILITY` | `0.9` | Reliability score for Stack Exchange. | SEARCH || `search.engine.web.stackexchange.requires_api_key` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE_REQUIRES_API_KEY` | `false` | Stack Exchange is free without an API key (300 requests/day). | SEARCH || `search.engine.web.stackexchange.strengths` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE_STRENGTHS` | ``["programming", "Q&A", "code examples", "community moderated", "multiple sites"]`` | Key strengths of Stack Exchange. | SEARCH |+| `search.engine.web.stackexchange.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE_USE_IN_AUTO_SEARCH` | `false` | Include Stack Exchange in auto search mode | SEARCH || `search.engine.web.stackexchange.weaknesses` | `LDR_SEARCH_ENGINE_WEB_STACKEXCHANGE_WEAKNESSES` | ``["technical focus", "rate limited", "Q&A format only"]`` | Limitations of Stack Exchange. | SEARCH || `search.engine.web.tavily.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_TAVILY_AGENT_ENABLED` | `true` | Include Tavily as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.tavily.api_key` | `LDR_SEARCH_ENGINE_WEB_TAVILY_API_KEY` | `` | The Tavily API key to use. | SEARCH |@@ -601,12 +617,14 @@ They are not available in the Web UI because they are needed to start the applic| `search.engine.web.wikipedia.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_WIKIPEDIA_USE_IN_AUTO_SEARCH` | `true` | Include Wikipedia in auto search mode | SEARCH || `search.engine.web.wikipedia.weaknesses` | `LDR_SEARCH_ENGINE_WEB_WIKIPEDIA_WEAKNESSES` | ``["recent events", "specialized academic topics", "product comparisons"]`` | Limitations: May lag on recent events, less depth on specialized academic topics, no product comparisons. | SEARCH || `search.engine.web.zenodo` | `LDR_SEARCH_ENGINE_WEB_ZENODO` | ``{"requires_api_key": false, "requires_llm": true, "description": "Search millions of research datasets, software, and publications with DOIs.", "reliability": 0.95}`` | Zenodo search engine configuration | SEARCH |+| `search.engine.web.zenodo.agent_enabled` | `LDR_SEARCH_ENGINE_WEB_ZENODO_AGENT_ENABLED` | `false` | Include Zenodo as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections. | SEARCH || `search.engine.web.zenodo.default_params.max_results` | `LDR_SEARCH_ENGINE_WEB_ZENODO_DEFAULT_PARAMS_MAX_RESULTS` | `20` | Maximum number of results to retrieve from Zenodo. | SEARCH || `search.engine.web.zenodo.description` | `LDR_SEARCH_ENGINE_WEB_ZENODO_DESCRIPTION` | `Search millions of research datasets, software, and publications with DOIs.` | Human-readable description of the search engine. | SEARCH || `search.engine.web.zenodo.display_name` | `LDR_SEARCH_ENGINE_WEB_ZENODO_DISPLAY_NAME` | `Zenodo` | Display name for Zenodo search engine. | SEARCH || `search.engine.web.zenodo.reliability` | `LDR_SEARCH_ENGINE_WEB_ZENODO_RELIABILITY` | `0.95` | Reliability score for Zenodo. | SEARCH || `search.engine.web.zenodo.requires_api_key` | `LDR_SEARCH_ENGINE_WEB_ZENODO_REQUIRES_API_KEY` | `false` | Zenodo is free and does not require an API key for search. | SEARCH || `search.engine.web.zenodo.strengths` | `LDR_SEARCH_ENGINE_WEB_ZENODO_STRENGTHS` | ``["research data", "datasets", "DOIs", "software", "open access", "CERN"]`` | Key strengths of Zenodo. | SEARCH |+| `search.engine.web.zenodo.use_in_auto_search` | `LDR_SEARCH_ENGINE_WEB_ZENODO_USE_IN_AUTO_SEARCH` | `false` | Include Zenodo in auto search mode | SEARCH || `search.engine.web.zenodo.weaknesses` | `LDR_SEARCH_ENGINE_WEB_ZENODO_WEAKNESSES` | ``["research focus", "variable quality", "metadata only"]`` | Limitations of Zenodo. | SEARCH || `search.favorites` | `LDR_SEARCH_FAVORITES` | ``["arxiv", "searxng", "library", "openalex"]`` | List of favorite search engine IDs that appear at the top of the dropdown. Edit as JSON array, e.g. ["arxiv", "brave"] | SEARCH || `search.fetch.mode` | `LDR_SEARCH_FETCH_MODE` | `summary_focus_query` | Controls the LangGraph agent's fetch_content tool. 'summary_focus_query' (default) asks the LLM to extract only spans relevant to a focus question the agent supplies per call AND the original research query, so vague focus can be disambiguated — keeps small-model context clean. 'summary_focus' is the same but uses only the per-call focus question. 'full' returns the entire extracted page text (legacy behavior; can flood small-model context with boilerplate/metadata and degrade benchmark accuracy). 'disabled' removes the tool entirely so the agent works from search snippets only — useful for diagnosing whether fetching is helping or hurting accuracy. Note: This setting applies exclusively to the langgraph-agent strategy; pipeline strategies retrieve full page text upfront during search when search.snippets_only is disabled. | SEARCH |diff --git a/src/local_deep_research/defaults/settings/search_engines/paperless.json b/src/local_deep_research/defaults/settings/search_engines/paperless.jsonindex 3fc1a380bb..86f51ef2ad 100644--- a/src/local_deep_research/defaults/settings/search_engines/paperless.json+++ b/src/local_deep_research/defaults/settings/search_engines/paperless.json@@ -220,5 +220,33 @@"limited query operators"],"visible": true+ },+ "search.engine.web.paperless.use_in_auto_search": {+ "category": "paperless",+ "description": "Include Paperless in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },+ "search.engine.web.paperless.agent_enabled": {+ "category": "paperless",+ "description": "Include Paperless as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true}}diff --git a/src/local_deep_research/defaults/settings/search_engines/scaleserp.json b/src/local_deep_research/defaults/settings/search_engines/scaleserp.jsonindex f29916af4c..f9392e42c5 100644--- a/src/local_deep_research/defaults/settings/search_engines/scaleserp.json+++ b/src/local_deep_research/defaults/settings/search_engines/scaleserp.json@@ -227,5 +227,33 @@"Cache only lasts 1 hour"],"visible": false+ },+ "search.engine.web.scaleserp.use_in_auto_search": {+ "category": "scaleserp",+ "description": "Include Scale SERP in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },+ "search.engine.web.scaleserp.agent_enabled": {+ "category": "scaleserp",+ "description": "Include Scale SERP as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": true,+ "visible": true}}diff --git a/src/local_deep_research/defaults/settings/search_engines/serper.json b/src/local_deep_research/defaults/settings/search_engines/serper.jsonindex 11fea1ca7a..75303ebd08 100644--- a/src/local_deep_research/defaults/settings/search_engines/serper.json+++ b/src/local_deep_research/defaults/settings/search_engines/serper.json@@ -226,5 +226,33 @@"Not specialized for academic content"],"visible": false+ },+ "search.engine.web.serper.use_in_auto_search": {+ "category": "serper",+ "description": "Include Serper in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },+ "search.engine.web.serper.agent_enabled": {+ "category": "serper",+ "description": "Include Serper as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": true,+ "visible": true}}diff --git a/src/local_deep_research/defaults/settings_gutenberg.json b/src/local_deep_research/defaults/settings_gutenberg.jsonindex 6ffee853bc..a22f2168bf 100644--- a/src/local_deep_research/defaults/settings_gutenberg.json+++ b/src/local_deep_research/defaults/settings_gutenberg.json@@ -92,5 +92,33 @@"ui_element": "number","value": 20,"visible": true+ },+ "search.engine.web.gutenberg.use_in_auto_search": {+ "category": "gutenberg",+ "description": "Include Project Gutenberg in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },+ "search.engine.web.gutenberg.agent_enabled": {+ "category": "gutenberg",+ "description": "Include Project Gutenberg as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true}}diff --git a/src/local_deep_research/defaults/settings_nasa_ads.json b/src/local_deep_research/defaults/settings_nasa_ads.jsonindex 179cf0ee08..6d37c2acdb 100644--- a/src/local_deep_research/defaults/settings_nasa_ads.json+++ b/src/local_deep_research/defaults/settings_nasa_ads.json@@ -229,5 +229,33 @@"requires API key for good performance"],"visible": true+ },+ "search.engine.web.nasa_ads.use_in_auto_search": {+ "category": "nasa_ads",+ "description": "Include NASA ADS in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },+ "search.engine.web.nasa_ads.agent_enabled": {+ "category": "nasa_ads",+ "description": "Include NASA ADS as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true}}diff --git a/src/local_deep_research/defaults/settings_openlibrary.json b/src/local_deep_research/defaults/settings_openlibrary.jsonindex 849fe59db7..7e55160f37 100644--- a/src/local_deep_research/defaults/settings_openlibrary.json+++ b/src/local_deep_research/defaults/settings_openlibrary.json@@ -92,5 +92,33 @@"ui_element": "number","value": 20,"visible": true+ },+ "search.engine.web.openlibrary.use_in_auto_search": {+ "category": "openlibrary",+ "description": "Include Open Library in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },+ "search.engine.web.openlibrary.agent_enabled": {+ "category": "openlibrary",+ "description": "Include Open Library as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true}}diff --git a/src/local_deep_research/defaults/settings_pubchem.json b/src/local_deep_research/defaults/settings_pubchem.jsonindex cd48934786..230a15ba61 100644--- a/src/local_deep_research/defaults/settings_pubchem.json+++ b/src/local_deep_research/defaults/settings_pubchem.json@@ -102,5 +102,33 @@"ui_element": "checkbox","value": true,"visible": true+ },+ "search.engine.web.pubchem.use_in_auto_search": {+ "category": "pubchem",+ "description": "Include PubChem in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },+ "search.engine.web.pubchem.agent_enabled": {+ "category": "pubchem",+ "description": "Include PubChem as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true}}diff --git a/src/local_deep_research/defaults/settings_stackexchange.json b/src/local_deep_research/defaults/settings_stackexchange.jsonindex 4614bfd8c9..3014c01fca 100644--- a/src/local_deep_research/defaults/settings_stackexchange.json+++ b/src/local_deep_research/defaults/settings_stackexchange.json@@ -103,5 +103,33 @@"value": "stackoverflow","options": ["stackoverflow", "serverfault", "superuser", "askubuntu", "unix", "math", "physics", "stats", "security", "dba"],"visible": true+ },+ "search.engine.web.stackexchange.use_in_auto_search": {+ "category": "stackexchange",+ "description": "Include Stack Exchange in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },+ "search.engine.web.stackexchange.agent_enabled": {+ "category": "stackexchange",+ "description": "Include Stack Exchange as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true}}diff --git a/src/local_deep_research/defaults/settings_zenodo.json b/src/local_deep_research/defaults/settings_zenodo.jsonindex c03f5fe10e..c486478ecd 100644--- a/src/local_deep_research/defaults/settings_zenodo.json+++ b/src/local_deep_research/defaults/settings_zenodo.json@@ -92,5 +92,33 @@"ui_element": "number","value": 20,"visible": true+ },+ "search.engine.web.zenodo.use_in_auto_search": {+ "category": "zenodo",+ "description": "Include Zenodo in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },+ "search.engine.web.zenodo.agent_enabled": {+ "category": "zenodo",+ "description": "Include Zenodo as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true}}diff --git a/tests/security/test_noncollection_agent_enabled.py b/tests/security/test_noncollection_agent_enabled.pyindex dcba5d3b69..68fd25a83d 100644--- a/tests/security/test_noncollection_agent_enabled.py+++ b/tests/security/test_noncollection_agent_enabled.py@@ -511,3 +511,178 @@ def test_canonical_primary_skip_uses_settings_value_not_class_heuristic():# And the factory was never invoked for it either -- the skip# happens before construction.assert "semantic_scholar" not in {c.args[0] for c in spec.call_args_list}+++# -- Specialized domain adapters default disabled (#1540 / parity) ---------+++ "engine_name",+ ["gutenberg", "openlibrary", "pubchem", "stackexchange", "zenodo"],+)+def test_specialized_domain_engines_default_disabled_for_agent(engine_name):+ """Specialized domain search engines (Project Gutenberg, Open Library,+ PubChem, Stack Exchange, Zenodo) added in #1540 must default to+ agent_enabled=False and use_in_auto_search=False so they do not silently+ leak into general research agent toolsets or auto searches."""+ from local_deep_research.settings.manager import SettingsManager+ from local_deep_research.web_search_engines.search_engines_config import (+ list_eligible_engine_configs,+ search_config,+ )++ manager = SettingsManager(db_session=None)+ defaults = manager.default_settings+ assert (+ defaults[f"search.engine.web.{engine_name}.agent_enabled"]["value"]+ is False+ )+ assert (+ defaults[f"search.engine.web.{engine_name}.use_in_auto_search"]["value"]+ is False+ )++ # In search_config built from default snapshot, agent_enabled is False+ snapshot = manager.get_settings_snapshot()+ cfg = search_config(settings_snapshot=snapshot)+ assert cfg[engine_name].get("agent_enabled") is False+ assert cfg[engine_name].get("use_in_auto_search") is False++ # In eligible engine discovery for the research agent, it is excluded+ eligible = list_eligible_engine_configs(+ settings_snapshot=snapshot, check_agent_enabled=True+ )+ assert engine_name not in eligible++ # LangGraph agent strategy filters it out by default+ names, spec = _build({engine_name: cfg[engine_name]})+ assert engine_name not in names+ assert engine_name not in {c.args[0] for c in spec.call_args_list}+++ "engine_name",+ ["gutenberg", "openlibrary", "pubchem", "stackexchange", "zenodo"],+)+def test_specialized_domain_engines_can_be_enabled_for_agent(engine_name):+ """When a user explicitly enables one of the specialized domain engines,+ it must be included in the agent's specialized tools."""+ from local_deep_research.settings.manager import SettingsManager+ from local_deep_research.web_search_engines.search_engines_config import (+ list_eligible_engine_configs,+ search_config,+ )++ manager = SettingsManager(db_session=None)+ snapshot = manager.get_settings_snapshot()+ snapshot[f"search.engine.web.{engine_name}.agent_enabled"] = True++ cfg = search_config(settings_snapshot=snapshot)+ assert cfg[engine_name].get("agent_enabled") is True++ eligible = list_eligible_engine_configs(+ settings_snapshot=snapshot, check_agent_enabled=True+ )+ assert engine_name in eligible++ names, spec = _build(+ {engine_name: cfg[engine_name]}, snapshot_extra=snapshot+ )+ assert engine_name in names+ assert engine_name in {c.args[0] for c in spec.call_args_list}+++# -- Key-gated search engine agent controls ----------------------------------+++ "engine_name,expected_agent_default",+ [+ ("nasa_ads", False),+ ("paperless", False),+ ("scaleserp", True),+ ("serper", True),+ ],+)+def test_key_gated_engines_default_flags(engine_name, expected_agent_default):+ """Key-gated engines must define both agent_enabled and use_in_auto_search.+ Specialized/private ones (NASA ADS, Paperless) default agent_enabled=False;+ general web scrapers (ScaleSERP, Serper) default agent_enabled=True.+ All default use_in_auto_search=False to avoid unbudgeted quota consumption."""+ from local_deep_research.settings.manager import SettingsManager+ from local_deep_research.web_search_engines.search_engines_config import (+ search_config,+ )++ manager = SettingsManager(db_session=None)+ defaults = manager.default_settings++ assert (+ defaults[f"search.engine.web.{engine_name}.agent_enabled"]["value"]+ is expected_agent_default+ )+ assert (+ defaults[f"search.engine.web.{engine_name}.use_in_auto_search"]["value"]+ is False+ )++ snapshot = manager.get_settings_snapshot()+ cfg = search_config(settings_snapshot=snapshot)+ assert cfg[engine_name].get("agent_enabled") is expected_agent_default+ assert cfg[engine_name].get("use_in_auto_search") is False++[email protected]("engine_name", ["nasa_ads", "paperless"])+def test_key_gated_specialized_engines_require_key_and_explicit_opt_in(+ engine_name,+):+ """NASA ADS and Paperless do not become agent tools simply because an API+ key is configured; they require explicit agent_enabled=True opt-in."""+ from local_deep_research.settings.manager import SettingsManager+ from local_deep_research.web_search_engines.search_engines_config import (+ list_eligible_engine_configs,+ )++ manager = SettingsManager(db_session=None)+ snapshot = manager.get_settings_snapshot()++ # Step 1: Add API key only. agent_enabled is still False by default.+ snapshot[f"search.engine.web.{engine_name}.api_key"] = "test-token"+ eligible = list_eligible_engine_configs(+ settings_snapshot=snapshot, check_agent_enabled=True+ )+ assert engine_name not in eligible++ # Step 2: Explicitly enable for agent.+ snapshot[f"search.engine.web.{engine_name}.agent_enabled"] = True+ eligible = list_eligible_engine_configs(+ settings_snapshot=snapshot, check_agent_enabled=True+ )+ assert engine_name in eligible++[email protected]("engine_name", ["scaleserp", "serper"])+def test_key_gated_general_engines_can_be_disabled_for_agent(engine_name):+ """ScaleSERP and Serper are available to the agent once an API key is configured,+ and can be toggled off via agent_enabled=False."""+ from local_deep_research.settings.manager import SettingsManager+ from local_deep_research.web_search_engines.search_engines_config import (+ list_eligible_engine_configs,+ )++ manager = SettingsManager(db_session=None)+ snapshot = manager.get_settings_snapshot()++ # Step 1: Add API key. Defaults agent_enabled=True.+ snapshot[f"search.engine.web.{engine_name}.api_key"] = "test-token"+ eligible = list_eligible_engine_configs(+ settings_snapshot=snapshot, check_agent_enabled=True+ )+ assert engine_name in eligible++ # Step 2: Toggle off.+ snapshot[f"search.engine.web.{engine_name}.agent_enabled"] = False+ eligible = list_eligible_engine_configs(+ settings_snapshot=snapshot, check_agent_enabled=True+ )+ assert engine_name not in eligiblediff --git a/tests/settings/golden_master_settings.json b/tests/settings/golden_master_settings.jsonindex 4145ef4e26..4edafbf123 100644--- a/tests/settings/golden_master_settings.json+++ b/tests/settings/golden_master_settings.json@@ -4149,6 +4149,20 @@},"visible": false},+ "search.engine.web.gutenberg.agent_enabled": {+ "category": "gutenberg",+ "description": "Include Project Gutenberg as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.gutenberg.default_params.max_results": {"category": "gutenberg","description": "Maximum number of results to retrieve from Project Gutenberg.",@@ -4220,6 +4234,20 @@],"visible": true},+ "search.engine.web.gutenberg.use_in_auto_search": {+ "category": "gutenberg",+ "description": "Include Project Gutenberg in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.gutenberg.weaknesses": {"category": "gutenberg","description": "Limitations of Project Gutenberg.",@@ -4443,6 +4471,20 @@},"visible": false},+ "search.engine.web.nasa_ads.agent_enabled": {+ "category": "nasa_ads",+ "description": "Include NASA ADS as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.nasa_ads.api_key": {"category": "nasa_ads","description": "NASA ADS API key (required). Get one at https://ui.adsabs.harvard.edu/user/settings/token",@@ -4635,6 +4677,20 @@],"visible": true},+ "search.engine.web.nasa_ads.use_in_auto_search": {+ "category": "nasa_ads",+ "description": "Include NASA ADS in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.nasa_ads.weaknesses": {"category": "nasa_ads","description": "List of research areas where NASA ADS has limited coverage",@@ -4931,6 +4987,20 @@},"visible": false},+ "search.engine.web.openlibrary.agent_enabled": {+ "category": "openlibrary",+ "description": "Include Open Library as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.openlibrary.default_params.max_results": {"category": "openlibrary","description": "Maximum number of results to retrieve from Open Library.",@@ -5003,6 +5073,20 @@],"visible": true},+ "search.engine.web.openlibrary.use_in_auto_search": {+ "category": "openlibrary",+ "description": "Include Open Library in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.openlibrary.weaknesses": {"category": "openlibrary","description": "Limitations of Open Library.",@@ -5017,6 +5101,20 @@],"visible": true},+ "search.engine.web.paperless.agent_enabled": {+ "category": "paperless",+ "description": "Include Paperless as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.paperless.api_key": {"category": "paperless","description": "API key for authentication (get from Paperless admin panel)",@@ -5221,6 +5319,20 @@"value": true,"visible": true},+ "search.engine.web.paperless.use_in_auto_search": {+ "category": "paperless",+ "description": "Include Paperless in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.paperless.weaknesses": {"category": "paperless","description": "Weaknesses of this search engine",@@ -5258,6 +5370,20 @@},"visible": false},+ "search.engine.web.pubchem.agent_enabled": {+ "category": "pubchem",+ "description": "Include PubChem as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.pubchem.default_params.include_synonyms": {"category": "pubchem","description": "Include compound synonyms in full content results.",@@ -5339,6 +5465,20 @@],"visible": true},+ "search.engine.web.pubchem.use_in_auto_search": {+ "category": "pubchem",+ "description": "Include PubChem in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.pubchem.weaknesses": {"category": "pubchem","description": "Limitations of PubChem.",@@ -5784,6 +5924,20 @@],"visible": true},+ "search.engine.web.scaleserp.agent_enabled": {+ "category": "scaleserp",+ "description": "Include Scale SERP as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": true,+ "visible": true+ },"search.engine.web.scaleserp.api_key": {"category": "scaleserp","description": "The ScaleSerp API key to use. Get yours at https://scaleserp.com",@@ -5995,6 +6149,20 @@"value": true,"visible": false},+ "search.engine.web.scaleserp.use_in_auto_search": {+ "category": "scaleserp",+ "description": "Include Scale SERP in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.scaleserp.weaknesses": {"category": "scaleserp","description": "Weaknesses of this search engine",@@ -6796,6 +6964,20 @@],"visible": true},+ "search.engine.web.serper.agent_enabled": {+ "category": "serper",+ "description": "Include Serper as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": true,+ "visible": true+ },"search.engine.web.serper.api_key": {"category": "serper","description": "The Serper API key to use. Get yours at https://serper.dev",@@ -7007,6 +7189,20 @@"value": true,"visible": false},+ "search.engine.web.serper.use_in_auto_search": {+ "category": "serper",+ "description": "Include Serper in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.serper.weaknesses": {"category": "serper","description": "Weaknesses of this search engine",@@ -7466,6 +7662,20 @@},"visible": false},+ "search.engine.web.stackexchange.agent_enabled": {+ "category": "stackexchange",+ "description": "Include Stack Exchange as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.stackexchange.default_params.max_results": {"category": "stackexchange","description": "Maximum number of results to retrieve from Stack Exchange.",@@ -7559,6 +7769,20 @@],"visible": true},+ "search.engine.web.stackexchange.use_in_auto_search": {+ "category": "stackexchange",+ "description": "Include Stack Exchange in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.stackexchange.weaknesses": {"category": "stackexchange","description": "Limitations of Stack Exchange.",@@ -8509,6 +8733,20 @@},"visible": false},+ "search.engine.web.zenodo.agent_enabled": {+ "category": "zenodo",+ "description": "Include Zenodo as a specialized tool for the langgraph research agent. The engine selected as `search.tool` always keeps its generic web_search tool — this flag only controls additional specialized tools. Independent of egress; the per-collection 'available to the research agent' toggle still wins for collections.",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Available to Research Agent",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.zenodo.default_params.max_results": {"category": "zenodo","description": "Maximum number of results to retrieve from Zenodo.",@@ -8581,6 +8819,20 @@],"visible": true},+ "search.engine.web.zenodo.use_in_auto_search": {+ "category": "zenodo",+ "description": "Include Zenodo in auto search mode",+ "editable": true,+ "max_value": null,+ "min_value": null,+ "name": "Include in Auto Search",+ "options": null,+ "step": null,+ "type": "SEARCH",+ "ui_element": "checkbox",+ "value": false,+ "visible": true+ },"search.engine.web.zenodo.weaknesses": {"category": "zenodo","description": "Limitations of Zenodo.",diff --git a/tests/settings/test_settings_defaults_integrity.py b/tests/settings/test_settings_defaults_integrity.pyindex 1c346ac367..13818f3cec 100644--- a/tests/settings/test_settings_defaults_integrity.py+++ b/tests/settings/test_settings_defaults_integrity.py@@ -706,6 +706,31 @@ class TestSettingsDefaultsIntegrity:"engines with agent_enabled but missing use_in_auto_search: "+ ", ".join(sorted(only_agent_set)))+ missing_flags = engine_names - (set(only_in_auto) & set(only_agent))+ if missing_flags:+ problems.append(+ "engines with settings in defaults missing paired flags: "+ + ", ".join(sorted(missing_flags))+ )+ expected_domain_engines = {+ "gutenberg",+ "nasa_ads",+ "openlibrary",+ "paperless",+ "pubchem",+ "scaleserp",+ "serper",+ "stackexchange",+ "zenodo",+ }+ missing_domain = expected_domain_engines - (+ set(only_in_auto) & set(only_agent)+ )+ if missing_domain:+ problems.append(+ "specialized domain engines missing paired flags: "+ + ", ".join(sorted(missing_domain))+ )assert not problems, "\n".join(problems)# -- Test 13: Snapshot values match typed get_setting -------------------