~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
fix(news): store subscription models without the provider prefix (#7862)
LearningCircuit1d98e231fcb4
diff --git a/changelog.d/7860.bugfix.md b/changelog.d/7860.bugfix.mdnew file mode 100644index 0000000000..1f2b266799--- /dev/null+++ b/changelog.d/7860.bugfix.md@@ -0,0 +1,5 @@+Scheduled news subscriptions now run, and generate their headline and topics, with the subscription's own provider and model instead of the default provider; when that provider is `openai_endpoint`, they use the custom endpoint saved with the subscription, as run-now already did. Before starting, a scheduled run now builds its egress context and applies the two-axis egress audit, the checks the research worker applies to run-now runs, and an `openai_endpoint` run needs an endpoint URL (the subscription's, or else the saved `llm.openai_endpoint.url`) that is set and passes the custom-endpoint check run-now applies; a run these checks refuse is skipped and logged. A subscription saved with an empty model (`''`) now uses the model from the user's settings (`llm.model`) instead of failing, as a NULL model already did; it still runs on its own saved provider if it has one. Under the strict egress scope, a subscription whose search engine is not the saved primary engine runs on the saved primary, and the log notes it; run-now refuses such a request instead.++The news page's "Create Subscription" now saves a `PROVIDER:model` selection as that provider and the bare model (`OLLAMA:llama3.1:8b` as Ollama with `llama3.1:8b`). The model field is free text, so only an uppercase provider key, the form the dropdown's options use, is split off: "Create Subscription" and "Run Once" keep any other value whole and use the default provider for it. That deliberately includes a lowercase key such as `ollama:llama3.1:8b`, so that a typed tag such as `qwen:7b` stays a model name instead of being read as a provider prefix.++After upgrading, scheduled runs of existing subscriptions use the provider saved with them, and one with a blank model uses the model from the user's settings (`llm.model`), with its saved provider if it has one. A model saved by earlier versions with a dropdown prefix such as `OLLAMA:` has the prefix dropped, and a typed tag whose first segment was saved as the provider (`qwen:7b` under `qwen`) runs whole on the default provider. The subscription form's Test Run resolves such rows the same way.diff --git a/src/local_deep_research/news/subscription_runner.py b/src/local_deep_research/news/subscription_runner.pyindex fa58dbd9c0..19dd276280 100644--- a/src/local_deep_research/news/subscription_runner.py+++ b/src/local_deep_research/news/subscription_runner.py@@ -18,7 +18,63 @@ skipped entirely)."""from datetime import datetime, timedelta, timezone-from typing import Any, Dict, Optional+from typing import Any, Dict, Optional, Tuple+++def resolve_subscription_model(+ model_provider: Any, model: Any+) -> Tuple[Optional[str], Optional[str]]:+ """Return the ``(provider, model)`` a saved subscription runs with.++ ``None`` in either place means "use the user's setting" (``llm.provider``+ / ``llm.model``). The scheduler and the run-now/overdue payload both+ resolve a saved row here, so every path runs it the same way.++ * An empty model is unset, and so is a provider that is not a built-in+ provider key. A kept provider is returned in lowercase canonical form.+ * Before #7860 the news page's "Create Subscription" saved the whole+ selection as the model and its first colon segment as the provider+ (lowercased on save). Such a row is recognised by the model's first+ segment naming the stored provider (compared ignoring case). The+ model field is free text, so the case of that segment is a heuristic+ for whether it was picked from the dropdown, whose options use the+ uppercase key, or typed:++ - exactly an uppercase built-in key: the ``KEY:`` prefix is stripped+ once and the provider kept, so+ ``("ollama", "OLLAMA:llama3.1:8b")`` -> ``("ollama", "llama3.1:8b")``;+ - anything else is taken as a model tag typed by hand, not a provider+ choice: the provider is unset and the model kept whole, so+ ``("qwen", "qwen:7b")`` -> ``(None, "qwen:7b")`` and+ ``("llama3.1", "llama3.1:8b")`` -> ``(None, "llama3.1:8b")``. A+ lowercase key is deliberately treated the same way, so+ ``("ollama", "ollama:llama3.1:8b")`` -> ``(None,+ "ollama:llama3.1:8b")``, so that a typed tag such as ``qwen:7b``+ stays a model name instead of being read as a provider prefix. That+ is how scheduled runs have run these rows so far.++ * Any other model is kept as saved: ``("ollama", "llama3.1:8b")`` and+ ``("openai", "OPENAI_ENDPOINT:org/model")`` are unchanged.+ """+ from ..llm.providers import discover_providers+ from ..llm.providers.base import normalize_provider++ if not isinstance(model, str) or not model.strip():+ model = None+ provider = model_provider.strip() if isinstance(model_provider, str) else ""+ keys = discover_providers() # built-in keys, uppercase+ resolved = (+ normalize_provider(provider) if provider.upper() in keys else None+ )++ head, separator, rest = (model or "").partition(":")+ if provider and separator and head.lower() == provider.lower():+ # Saved by the old Create Subscription; ``head in keys`` is a+ # case-sensitive match against the uppercase keys.+ if resolved and head in keys:+ return resolved, rest if rest.strip() else None+ return None, model+ return resolved, modeldef build_subscription_request_data(@@ -42,14 +98,16 @@ def build_subscription_request_data(metadata block. Shared by the manual run-now route and the overdue sweep sotheir payloads cannot diverge.- ``model_provider``/``model`` are passed through as-is, including falsy- values: an unset provider/model is intentional and makes the backend fall- back to the user's ``llm.provider`` / ``llm.model`` settings (see- ``web/routers/research.py::_extract_research_params``). Hardcoding a provider here- would override the user's configured default for subscriptions created- without an explicit model.+ ``model_provider``/``model`` are resolved with+ ``resolve_subscription_model`` (as the scheduler does) and sent even when+ unset (``None``): an unset provider/model is intentional and makes the+ backend fall back to the user's ``llm.provider`` / ``llm.model`` settings+ (see ``web/routers/research.py::_extract_research_params``). Hardcoding a+ provider here would override the user's configured default for+ subscriptions created without an explicit model."""query = query_template.replace("YYYY-MM-DD", current_date)+ model_provider, model = resolve_subscription_model(model_provider, model)request_data: Dict[str, Any] = {"query": query,diff --git a/src/local_deep_research/scheduler/background.py b/src/local_deep_research/scheduler/background.pyindex 418fbd6f29..dd338c08d2 100644--- a/src/local_deep_research/scheduler/background.py+++ b/src/local_deep_research/scheduler/background.py@@ -8,7 +8,7 @@ import threadingfrom dataclasses import dataclassfrom datetime import datetime, timedelta, UTCfrom functools import wraps-from typing import Any, Callable, Dict, List+from typing import Any, Callable, Dict, List, Optionalfrom cachetools import TTLCachefrom loguru import logger@@ -30,7 +30,6 @@ from ..security.log_sanitizer import failure_reason_for_log, redact_secrets# concrete ``LibraryRAGService`` is no longer imported at module scope.from ..database.library_init import get_default_library_idfrom ..database.models.library import Document, DocumentCollection-from ..constants import DEFAULT_SEARCH_TOOLSCHEDULER_AVAILABLE = True # Always available since it's a required dependency@@ -70,6 +69,161 @@ def _log_invalid_refresh_interval(logger.debug(message, subscription_id)+class _SubscriptionRunRefused(ValueError):+ """A scheduled subscription run that must not start (already logged)."""+++def _subscription_run_snapshot(+ settings_snapshot: Any, subscription: Dict[str, Any], username: str+) -> Any:+ """Return the settings snapshot a scheduled subscription run uses.++ A copy of the user's ``settings_snapshot`` with the subscription's+ provider, model and search engine applied through the shared runtime+ overlay that run-now also uses (an unset value keeps the user's setting),+ scoped to ``username``. ``subscription`` carries the provider and model+ as ``resolve_subscription_model`` returned them.++ When the run's provider is ``openai_endpoint``, the subscription's+ ``custom_endpoint`` is applied as ``llm.openai_endpoint.url``, as+ ``_extract_research_params`` in ``web/routers/research.py`` does for+ run-now; when it is blank, the user's saved URL stays. The URL the run+ then uses, from either source, must be set and pass+ ``is_safe_custom_llm_endpoint``, as run-now requires; otherwise a+ policy-audit line is logged and ``_SubscriptionRunRefused`` (a+ ``ValueError``) is raised, so the run is not started. For any other+ provider the endpoint is not used and not checked.+ """+ from ..llm.providers.base import normalize_provider+ from ..search_system import ensure_snapshot_username+ from ..utilities.type_utils import overlay_runtime_settings, unwrap_setting+ from ..utilities.url_utils import is_safe_custom_llm_endpoint++ if not isinstance(settings_snapshot, dict):+ return settings_snapshot+ search_engine = subscription.get("search_engine") or None+ snapshot = dict(settings_snapshot)+ overlay_runtime_settings(+ snapshot,+ model_provider=subscription.get("model_provider") or None,+ model=subscription.get("model") or None,+ search_engine=search_engine,+ )++ run_engine = unwrap_setting(snapshot.get("search.tool"))+ if search_engine and run_engine != search_engine:+ # Under the strict egress scope the overlay keeps the saved primary.+ logger.info(+ "Strict egress scope: subscription {} searches with the saved "+ "primary engine '{}' instead of its own '{}'",+ subscription.get("id"),+ run_engine,+ search_engine,+ )++ provider = unwrap_setting(snapshot.get("llm.provider"))+ if (+ isinstance(provider, str)+ and normalize_provider(provider) == "openai_endpoint"+ ):+ custom_endpoint = subscription.get("custom_endpoint")+ if isinstance(custom_endpoint, str):+ custom_endpoint = custom_endpoint.strip()+ if custom_endpoint:+ overlay_runtime_settings(snapshot, custom_endpoint=custom_endpoint)+ # Check the URL the run uses, whether it is the subscription's or+ # the user's saved one: the provider's own base-URL check admits+ # link-local addresses that run-now refuses.+ endpoint = unwrap_setting(snapshot.get("llm.openai_endpoint.url"))+ refusal = None+ if endpoint is None or (+ isinstance(endpoint, str) and not endpoint.strip()+ ):+ refusal = (+ "Custom endpoint URL is required for OpenAI endpoint provider"+ )+ elif not is_safe_custom_llm_endpoint(endpoint):+ refusal = "Invalid custom endpoint URL"+ if refusal:+ logger.bind(policy_audit=True).warning(+ "scheduled subscription run refused: custom endpoint",+ subscription_id=subscription.get("id"),+ reason=refusal,+ )+ raise _SubscriptionRunRefused(refusal)+ return ensure_snapshot_username(snapshot, username)+++def _subscription_run_admitted(+ run_snapshot: Any, username: str, subscription_id: Any+) -> bool:+ """Build a scheduled run's egress context and apply the two-axis audit.++ These are the two checks of the research worker's chokepoint+ (``run_research_process`` in ``web/services/research_service.py``):+ the run's egress context is built from ``run_snapshot`` for its primary+ engine (``resolve_run_primary_engine`` / ``context_from_snapshot``), and+ the two-axis (sensitivity x exposure) rule is evaluated with+ ``audit_run_from_snapshot``, which fails closed on an internal error.+ Returns False, after a policy-audit log line, when the rule refuses the+ run or the context cannot be built (``ValueError`` /+ ``PolicyDeniedError``); the caller then runs no research and generates+ nothing. As in the worker, an unavailable security module keeps the+ legacy behaviour. The custom-endpoint check is applied before this, by+ ``_subscription_run_snapshot``.++ Run-now's request-time engine check (``_precheck_engine_policy`` in+ ``web/routers/research.py``) is not applied. Under the strict egress+ scope, run-now refuses a request whose search engine is not the saved+ primary; a scheduled subscription with such an engine runs on the saved+ primary instead, and ``_subscription_run_snapshot`` logs that. The+ search-engine factory still applies ``evaluate_engine`` to each engine+ the run creates.+ """+ try:+ from ..security.egress.policy import (+ PolicyDeniedError,+ context_from_snapshot,+ resolve_run_primary_engine,+ )+ from ..security.egress.run_classification import (+ audit_run_from_snapshot,+ )++ try:+ primary = resolve_run_primary_engine(run_snapshot)+ ctx = context_from_snapshot(+ run_snapshot, primary, username=username+ )+ except (PolicyDeniedError, ValueError) as err:+ # The exception class name only. The repo's failure-path+ # logging rule allows no exception text at all here (and no+ # new chained-logger violations): str() of a caught exception+ # can carry user-controlled content (e.g. ValueError detail),+ # and PolicyDeniedError also keeps the raw target in an+ # attribute, so nothing richer than the class name is logged.+ logger.bind(policy_audit=True).warning(+ "scheduled subscription run refused: egress policy unevaluable",+ subscription_id=subscription_id,+ reason=type(err).__name__,+ )+ return False++ decision = audit_run_from_snapshot(run_snapshot, ctx, primary)+ if not decision.allowed:+ logger.bind(policy_audit=True).warning(+ "scheduled subscription run refused: two-axis egress",+ subscription_id=subscription_id,+ reason=decision.reason,+ offending=decision.offending,+ )+ return False+ except ImportError:+ # security module unavailable — preserve legacy behaviour.+ logger.debug("egress audit hook unavailable for scheduled runs")+ return True++def _one_document_pass_per_user(method: Callable) -> Callable:"""Skip a document pass while another one for the same user is running.@@ -2109,7 +2263,10 @@ class BackgroundJobScheduler:NewsSubscription,SubscriptionStatus,)- from ..news.subscription_runner import advance_refresh_schedule+ from ..news.subscription_runner import (+ advance_refresh_schedule,+ resolve_subscription_model,+ )from ..news.constants import is_runnable_refresh_intervalwith get_user_db_session(username, password) as db:@@ -2155,15 +2312,19 @@ class BackgroundJobScheduler:advance_refresh_schedule(sub, datetime.now(UTC))db.commit()+ model_provider, model = resolve_subscription_model(+ sub.model_provider, sub.model+ )subscription_data = {"id": sub.id,"name": sub.name,"query": query,"original_query": sub.query_or_topic,- "model_provider": sub.model_provider,- "model": sub.model,+ "model_provider": model_provider,+ "model": model,"search_strategy": sub.search_strategy,"search_engine": sub.search_engine,+ "custom_endpoint": sub.custom_endpoint,# Capture before scope-exit rollback expires the ORM# object, avoiding a lazy refresh while rescheduling."refresh_interval_minutes": sub.refresh_interval_minutes,@@ -2248,27 +2409,19 @@ class BackgroundJobScheduler:with get_user_db_session(username, password) as db:settings_manager = SettingsManager(db)- settings_snapshot = settings_manager.get_settings_snapshot()-- # Use the search engine from the subscription if specified- search_engine = subscription.get("search_engine")-- if search_engine:- settings_snapshot["search.tool"] = {- "value": search_engine,- "ui_element": "select",- }- logger.info(- f"Using subscription's search engine: '{search_engine}' for {subscription['id']}"- )- else:- # Use the user's default search tool from their settings- default_search_tool = settings_snapshot.get(- "search.tool", DEFAULT_SEARCH_TOOL- )- logger.info(- f"Using user's default search tool: '{default_search_tool}' for {subscription['id']}"+ # The research and the headline/topic calls after it all run+ # on this snapshot: the user's settings with the+ # subscription's provider, model, search engine and custom+ # endpoint on top.+ try:+ settings_snapshot = _subscription_run_snapshot(+ settings_manager.get_settings_snapshot(),+ subscription,+ username,)+ except _SubscriptionRunRefused:+ # The custom-endpoint check refused the run (logged).+ returnlogger.debug(f"Settings snapshot has {len(settings_snapshot)} settings"@@ -2278,6 +2431,18 @@ class BackgroundJobScheduler:f"Key settings: llm.model={settings_snapshot.get('llm.model')}, llm.provider={settings_snapshot.get('llm.provider')}, search.tool={settings_snapshot.get('search.tool')}")+ # Pre-run egress checks: the custom-endpoint check above, then+ # building the run's egress context and the two-axis audit. A+ # refused run researches and generates nothing. Run-now's+ # request-time engine check is not applied: under the strict+ # scope a subscription whose engine is not the saved primary+ # runs on the saved primary (logged); run-now refuses such a+ # request.+ if not _subscription_run_admitted(+ settings_snapshot, username, subscription["id"]+ ):+ return+# Set up research parametersquery = subscription["query"]@@ -2306,15 +2471,15 @@ class BackgroundJobScheduler:search_strategy = subscription.get("search_strategy")# Build kwargs for quick_summary, only including- # search_strategy if the subscription specifies one.+ # search_strategy if the subscription specifies one. The+ # provider and model travel in settings_snapshot: quick_summary+ # reads its ``provider`` argument only when no snapshot is given.quick_summary_kwargs = {"query": query,"research_id": research_id,"username": username,"user_password": password,"settings_snapshot": settings_snapshot,- "model_name": subscription.get("model"),- "provider": subscription.get("model_provider"),"metadata": metadata,"search_original_query": False, # Don't send long subscription prompts to search engines}@@ -2335,6 +2500,7 @@ class BackgroundJobScheduler:subscription["id"],result,subscription,+ run_snapshot=settings_snapshot,)except Exception as e:@@ -2358,8 +2524,14 @@ class BackgroundJobScheduler:subscription_id: int,result: Dict[str, Any],subscription: Dict[str, Any],+ run_snapshot: Optional[Dict[str, Any]] = None,):- """Store research result in database for news display."""+ """Store research result in database for news display.++ ``run_snapshot`` is the settings snapshot the research ran with (see+ ``_subscription_run_snapshot``); the headline and topic calls use it+ so they run on the same provider and model as the research.+ """try:from ..database.session_context import get_user_db_sessionfrom ..database.models import ResearchHistory@@ -2400,6 +2572,14 @@ class BackgroundJobScheduler:settings_snapshot = ensure_snapshot_username(settings_snapshot, username)+ # Headline and topics are generated with the provider and+ # model the research ran with (#7860). The run snapshot also+ # carries ``_username`` for the reason above.+ llm_snapshot = (+ run_snapshot+ if isinstance(run_snapshot, dict)+ else settings_snapshot+ )# Get the report content - check both 'report' and 'summary' fieldsreport_content = serializable_result.get(@@ -2465,7 +2645,7 @@ class BackgroundJobScheduler:query=query_text,findings=report_content,max_length=200, # Allow longer headlines for news- settings_snapshot=settings_snapshot,+ settings_snapshot=llm_snapshot,)# Generate topics from the findings@@ -2477,7 +2657,7 @@ class BackgroundJobScheduler:findings=report_content,category=subscription.get("name", "News"),max_topics=6,- settings_snapshot=settings_snapshot,+ settings_snapshot=llm_snapshot,)logger.info(diff --git a/src/local_deep_research/web/static/js/pages/news.js b/src/local_deep_research/web/static/js/pages/news.jsindex 14fda13251..bd94baa127 100644--- a/src/local_deep_research/web/static/js/pages/news.js+++ b/src/local_deep_research/web/static/js/pages/news.js@@ -570,15 +570,15 @@ function getNewsSubscriptionModelConfig() {const model = document.getElementById('news-subscription-model')?.value ?? '';if (!model) return null;- // Every built-in provider key (a test pins it to the registry). A model ID- // can contain colons (llama3.1:8b), so a colon alone is not a delimiter.+ // Every built-in provider key (a test pins it to the registry). The field+ // is free text, so only an uppercase key is taken as a provider prefix.const knownProviders = new Set(['ANTHROPIC', 'ANTHROPIC_ENDPOINT', 'ATLASCLOUD', 'DEEPSEEK', 'GOOGLE','IONOS', 'LLAMACPP', 'LMSTUDIO', 'OLLAMA', 'OPENAI', 'OPENAI_ENDPOINT','OPENROUTER', 'ORCAROUTER', 'QWEN', 'REQUESTY', 'XAI',]);const separator = model.indexOf(':');- const provider = model.slice(0, separator).toUpperCase();+ const provider = model.slice(0, separator);if (separator > 0 && knownProviders.has(provider)) {return { provider, model: model.slice(separator + 1) };}@@ -4216,16 +4216,15 @@ async function handleNewsSubscriptionSubmit(e) {const isActive = document.getElementById('news-subscription-active').checked;const runNow = document.getElementById('news-subscription-run-now').checked;- // Model configuration - get from custom dropdown- const modelInput = document.getElementById('news-subscription-model');- const model = modelInput ? modelInput.value : '';-- // Extract provider from the model selection (e.g., "OLLAMA:llama3" -> "OLLAMA")- let modelProvider = '';- if (model && model.includes(':')) {- const parts = model.split(':');- modelProvider = parts[0];- }+ // Split the model selection exactly as Run Once does. The model field is+ // free text, so this is a heuristic: only an uppercase provider key, the+ // form the dropdown's options use, is split off ("OLLAMA:llama3.1:8b" ->+ // OLLAMA + "llama3.1:8b"). Any other value, a typed tag such as qwen:7b+ // or ollama:llama3.1:8b included, is all model ID, with the provider left+ // to the user's default.+ const modelConfig = getNewsSubscriptionModelConfig();+ const modelProvider = modelConfig?.provider ?? '';+ const model = modelConfig?.model ?? '';const searchStrategy = document.getElementById('news-subscription-strategy').value;const submitButton = document.querySelector(diff --git a/src/local_deep_research/web/templates/pages/news-subscription-form.html b/src/local_deep_research/web/templates/pages/news-subscription-form.htmlindex 68a0bda19f..5129875dda 100644--- a/src/local_deep_research/web/templates/pages/news-subscription-form.html+++ b/src/local_deep_research/web/templates/pages/news-subscription-form.html@@ -468,7 +468,7 @@ async function loadProviders() {: provider.label;// Handle case-insensitive comparison for provider namesoption.selected = !isDisabled && (- provider.value.toUpperCase() === currentProvider.toUpperCase() ||+ provider.value.toUpperCase() === (currentProvider || '').toUpperCase() ||(!currentProvider && provider.default));providerSelect.appendChild(option);@@ -794,6 +794,41 @@ async function handleSubscriptionSubmit(e) {}}+// A subscription saved from the news page before #7860 can hold the whole+// "PROVIDER:model" selection as its model, with the first segment as its+// provider. Resolve it the way scheduled runs do (resolve_subscription_model+// in news/subscription_runner.py). The news page's model field is free text,+// so this is a heuristic: a first segment that is exactly an uppercase+// built-in provider key, the form the dropdown's options use, is stripped;+// any other first segment naming the provider is a typed model tag, which+// runs whole on the default provider.+function resolveSavedSubscriptionModel(provider, model) {+ // The same keys as knownProviders in news.js (a test pins both to the+ // registry), so this works before the provider list has loaded.+ const knownProviders = new Set([+ 'ANTHROPIC', 'ANTHROPIC_ENDPOINT', 'ATLASCLOUD', 'DEEPSEEK', 'GOOGLE',+ 'IONOS', 'LLAMACPP', 'LMSTUDIO', 'OLLAMA', 'OPENAI', 'OPENAI_ENDPOINT',+ 'OPENROUTER', 'ORCAROUTER', 'QWEN', 'REQUESTY', 'XAI',+ ]);+ // resolve_subscription_model keeps a provider only when it names a+ // built-in provider key (compared after trimming, as it strips the+ // saved value first). Any other provider is not a provider choice and+ // is dropped, so the run falls back to the user's default provider+ // with the model kept whole.+ if (provider && !knownProviders.has(provider.trim().toUpperCase())) {+ provider = '';+ }+ const separator = model.indexOf(':');+ const head = model.slice(0, separator);+ if (!provider || separator <= 0 || head.toUpperCase() !== provider.toUpperCase()) {+ return { provider, model };+ }+ if (knownProviders.has(head)) {+ return { provider, model: model.slice(separator + 1) };+ }+ return { provider: '', model };+}+// Handle test runasync function handleTestRun(event, actionAlreadyOwned = false) {if (event) {@@ -813,9 +848,11 @@ async function handleTestRun(event, actionAlreadyOwned = false) {}// Collect all configuration- const provider = document.getElementById('subscription-provider').value || currentProvider || '';const modelHidden = document.getElementById('subscription-model_hidden');- const model = modelHidden ? modelHidden.value : '';+ const { provider, model } = resolveSavedSubscriptionModel(+ document.getElementById('subscription-provider').value || currentProvider || '',+ modelHidden ? modelHidden.value : '',+ );const customEndpoint = document.getElementById('subscription-custom-endpoint').value;const searchEngineHidden = document.getElementById('subscription-search-engine_hidden');const searchEngineInput = document.getElementById('subscription-search-engine');diff --git a/tests/js/pages/news-actions-direct-runtime.test.js b/tests/js/pages/news-actions-direct-runtime.test.jsindex b2551a93f8..eab0d118a4 100644--- a/tests/js/pages/news-actions-direct-runtime.test.js+++ b/tests/js/pages/news-actions-direct-runtime.test.js@@ -1084,7 +1084,7 @@ it('loads safe folder options and submits the template modal through the migratefolder_id: 'folder-3299',is_active: true,model_provider: 'OLLAMA',- model: 'OLLAMA:llama3',+ model: 'llama3',search_strategy: 'source-based',});expect(document.getElementById('newsSubscriptionModal').dataset.hidden)@@ -1177,7 +1177,10 @@ it.each([['OPENAI:gpt-4.1-mini', 'OPENAI', 'gpt-4.1-mini'],['QWEN:qwen3.8-max', 'QWEN', 'qwen3.8-max'],['OLLAMA:llama3.1:8b', 'OLLAMA', 'llama3.1:8b'],- ['ollama:llama3.1:8b', 'OLLAMA', 'llama3.1:8b'],+ // The field is free text: only an uppercase key (the form the dropdown's+ // options use) is taken as a provider; a lowercase one is part of a tag.+ ['ollama:llama3.1:8b', undefined, 'ollama:llama3.1:8b'],+ ['qwen:7b', undefined, 'qwen:7b'],['llama3.1:8b', undefined, 'llama3.1:8b'],['custom:model:tag', undefined, 'custom:model:tag'],['OLLAMA:OPENAI:model:tag', 'OLLAMA', 'OPENAI:model:tag'],diff --git a/tests/js/pages/news-subscription-models-runtime.test.js b/tests/js/pages/news-subscription-models-runtime.test.jsindex 7101d7b5ab..5ea09a1797 100644--- a/tests/js/pages/news-subscription-models-runtime.test.js+++ b/tests/js/pages/news-subscription-models-runtime.test.js@@ -24,13 +24,13 @@ const MODEL_PAYLOAD = {},};-function compileModelsLoader(modelDropdownInstance) {+function compileModelsLoader(modelDropdownInstance, currentProvider = 'ollama') {return compileTemplateHarness({templatePath: TEMPLATE_PATH,functionNames: ['getCSRFToken', 'loadModelsForProvider', 'loadProviders'],dependencies: { modelDropdownInstance },preamble: `- let currentProvider = 'ollama';+ let currentProvider = ${JSON.stringify(currentProvider)};const defaultModel = 'baseline-model';let modelOptions = [];`,@@ -116,3 +116,32 @@ it('uses the current CSRF token when a provider change loads its models', asyncexpect(document.getElementById('custom-endpoint-container').style.display).toBe('block');expect(dropdown.setValue).not.toHaveBeenCalled();});++// A subscription saved without a provider renders currentProvider as null.+// Matching the options against it used to throw, which left the provider+// list on "Error loading providers".+it('selects the default provider when the subscription has none', async () => {+ const payload = {+ ...MODEL_PAYLOAD,+ provider_options: [+ { value: 'OLLAMA', label: 'Ollama', default: true },+ { value: 'OPENAI_ENDPOINT', label: 'Custom endpoint' },+ ],+ };+ const fetchMock = vi.fn().mockResolvedValue({+ ok: true, json: async () => payload,+ });+ vi.stubGlobal('fetch', fetchMock);+ const consoleError = vi.spyOn(console, 'error').mockImplementation(() => {});++ await compileModelsLoader({ setValue: vi.fn() }, null).loadProviders();++ expect(consoleError).not.toHaveBeenCalled();+ const provider = document.getElementById('subscription-provider');+ expect(Array.from(provider.options, option => option.value))+ .toEqual(['', 'OLLAMA', 'OPENAI_ENDPOINT']);+ expect(provider.value).toBe('OLLAMA');+ expect(window.updateDropdownOptions).toHaveBeenLastCalledWith(+ document.getElementById('subscription-model'), payload.providers.ollama_models,+ );+});diff --git a/tests/js/pages/news-subscription-submit-runtime.test.js b/tests/js/pages/news-subscription-submit-runtime.test.jsindex b710f2926c..f6cce552f3 100644--- a/tests/js/pages/news-subscription-submit-runtime.test.js+++ b/tests/js/pages/news-subscription-submit-runtime.test.js@@ -74,6 +74,7 @@ function compileSubscriptionRuntime({'endNewsResearchPollWithFeed','completeNewsResearchPoll','pollForNewsResearchResults',+ 'getNewsSubscriptionModelConfig','handleNewsSubscriptionSubmit',],dependencies: {@@ -96,6 +97,7 @@ function compileSubscriptionRuntime({let activeNewsSubscriptionSubmit = null;`,returnExpression: `({+ getNewsSubscriptionModelConfig,handleNewsSubscriptionSubmit,pollForNewsResearchResults,})`,@@ -197,7 +199,7 @@ it('creates, runs, polls, and completes a subscription using the returned researfolder_id: 'folder-3299',is_active: true,model_provider: 'OLLAMA',- model: 'OLLAMA:llama3',+ model: 'llama3',search_strategy: 'source-based',}),});@@ -250,6 +252,52 @@ it('creates, runs, polls, and completes a subscription using the returned researexpect(runtime.loadNewsFeed).toHaveBeenCalledOnce();});+// Create Subscription used to split on the first colon for the provider but+// save the whole selection as the model, so OLLAMA:llama3.1:8b was saved as+// model "OLLAMA:llama3.1:8b" and an unprefixed llama3.1:8b got provider+// "llama3.1" (#7860). It now goes through the parser Run Once uses.+it.each([+ ['OLLAMA:llama3.1:8b', 'OLLAMA', 'llama3.1:8b'],+ // A typed model tag is not split, even when its first segment is a+ // provider key in another case (qwen:7b is an Ollama tag, not QWEN).+ ['ollama:llama3.1:8b', '', 'ollama:llama3.1:8b'],+ ['qwen:7b', '', 'qwen:7b'],+ ['OPENAI:gpt-4.1-mini', 'OPENAI', 'gpt-4.1-mini'],+ ['OLLAMA:OPENAI:model:tag', 'OLLAMA', 'OPENAI:model:tag'],+ ['llama3.1:8b', '', 'llama3.1:8b'],+ ['custom:model:tag', '', 'custom:model:tag'],+ ['gpt-4.1-mini', '', 'gpt-4.1-mini'],+ ['', '', ''],+])('saves selection %j with the provider split Run Once uses', async (+ selected,+ provider,+ model,+) => {+ document.getElementById('news-subscription-model').value = selected;+ document.getElementById('news-subscription-run-now').checked = false;+ const fetchMock = vi.fn().mockResolvedValue(response({+ status: 'success',+ subscription_id: 'subscription-model-split',+ }));+ vi.stubGlobal('fetch', fetchMock);+ const runtime = compileSubscriptionRuntime();++ // The shared parser: only an uppercase provider key (the form the+ // dropdown's options use) is split off the free-text model field.+ expect(runtime.getNewsSubscriptionModelConfig())+ .toEqual(selected ? { provider, model } : null);++ await runtime.handleNewsSubscriptionSubmit({ preventDefault: vi.fn() });++ expect(fetchMock).toHaveBeenCalledOnce();+ const [url, options] = fetchMock.mock.calls[0];+ expect(url).toBe('/news/api/subscribe');+ const payload = JSON.parse(options.body);+ expect(payload.model_provider).toBe(provider);+ expect(payload.model).toBe(model);+ expect(runtime.loadSubscriptions).toHaveBeenCalledOnce();+});+it('gates a second submit until the owned create-and-run workflow settles', async () => {const pendingCreate = deferred();const fetchMock = vi.fn()diff --git a/tests/js/pages/news-subscription-test-run-runtime.test.js b/tests/js/pages/news-subscription-test-run-runtime.test.jsindex d066cc0579..bc792875b4 100644--- a/tests/js/pages/news-subscription-test-run-runtime.test.js+++ b/tests/js/pages/news-subscription-test-run-runtime.test.js@@ -55,6 +55,7 @@ function compileSubscriptionRunner(dependencies) {'isValidResearchId','scheduleSubscriptionRedirect','handleSubscriptionSubmit',+ 'resolveSavedSubscriptionModel','handleTestRun','handleCreateAndRun',]@@ -544,3 +545,91 @@ it.each(['handleSubscriptionSubmit', 'handleTestRun', 'handleCreateAndRun'])(expect(body.model).toBe('gpt-4.1-mini');},);++// A subscription saved from the news page before #7860 holds the whole+// "PROVIDER:model" selection as its model. Test Run resolves it the way the+// scheduled run does (resolve_subscription_model): an uppercase provider key+// (the form the dropdown's options use) is stripped, and a typed model tag+// whose first segment names the provider runs whole on the default provider.+// A provider that is not a built-in key is dropped the same way, so the run+// uses the user's default provider with the model kept whole.+it.each([+ ['OLLAMA', 'OLLAMA:llama3', 'OLLAMA', 'llama3'],+ ['OLLAMA', 'OLLAMA:llama3.1:8b', 'OLLAMA', 'llama3.1:8b'],+ ['QWEN', 'qwen:7b', undefined, 'qwen:7b'],+ ['OLLAMA', 'ollama:llama3', undefined, 'ollama:llama3'],+ ['OLLAMA', 'llama3.1:8b', 'OLLAMA', 'llama3.1:8b'],+ ['OPENAI', 'OPENAI_ENDPOINT:org/model', 'OPENAI', 'OPENAI_ENDPOINT:org/model'],+ ['CUSTOM', 'my-model', undefined, 'my-model'],+ ['CUSTOM', 'CUSTOM:my-model', undefined, 'CUSTOM:my-model'],+ ['CUSTOM', 'OLLAMA:llama3', undefined, 'OLLAMA:llama3'],+])('test-runs provider %s with saved model %s as %s + %s', async (+ selectedProvider,+ savedModel,+ provider,+ model,+) => {+ const select = document.getElementById('subscription-provider');+ select.innerHTML = `+ <option value="">Select a provider</option>+ <option value="OLLAMA">Ollama</option>+ <option value="OPENAI">OpenAI</option>+ <option value="OPENAI_ENDPOINT">Custom endpoint</option>+ <option value="QWEN">Qwen</option>+ <option value="CUSTOM">Custom</option>+ `;+ select.value = selectedProvider;+ document.getElementById('subscription-model_hidden').value = savedModel;+ const fetchMock = vi.fn().mockResolvedValue({+ ok: true,+ status: 200,+ json: vi.fn().mockResolvedValue({+ status: 'success',+ research_id: 'research-legacy-model',+ }),+ });+ vi.stubGlobal('fetch', fetchMock);++ await compileSubscriptionRunner(createDependencies()).handleTestRun();++ expect(fetchMock).toHaveBeenCalledOnce();+ const [url, options] = fetchMock.mock.calls[0];+ expect(url).toBe('/api/start_research');+ const body = JSON.parse(options.body);+ expect(body.model_provider).toBe(provider);+ expect(body.model).toBe(model);+});++// The provider keys are a fixed set, not the provider list the form loads, so+// a saved prefixed model resolves the same way before that list has loaded.+it.each([+ ['OLLAMA', 'OLLAMA:llama3.1:8b', 'OLLAMA', 'llama3.1:8b'],+ ['QWEN', 'qwen:7b', undefined, 'qwen:7b'],+])('test-runs saved provider %s with model %s as %s + %s before the provider list loads', async (+ savedProvider,+ savedModel,+ provider,+ model,+) => {+ document.getElementById('subscription-provider').innerHTML = '';+ document.getElementById('subscription-model_hidden').value = savedModel;+ const fetchMock = vi.fn().mockResolvedValue({+ ok: true,+ status: 200,+ json: vi.fn().mockResolvedValue({+ status: 'success',+ research_id: 'research-unloaded-providers',+ }),+ });+ vi.stubGlobal('fetch', fetchMock);++ await compileSubscriptionRunner({+ ...createDependencies(),+ currentProvider: savedProvider,+ }).handleTestRun();++ expect(fetchMock).toHaveBeenCalledOnce();+ const body = JSON.parse(fetchMock.mock.calls[0][1].body);+ expect(body.model_provider).toBe(provider);+ expect(body.model).toBe(model);+});diff --git a/tests/news/subscription_manager/test_scheduler_coverage.py b/tests/news/subscription_manager/test_scheduler_coverage.pyindex d7f466a735..e093b8d3f2 100644--- a/tests/news/subscription_manager/test_scheduler_coverage.py+++ b/tests/news/subscription_manager/test_scheduler_coverage.py@@ -942,6 +942,8 @@ class TestCheckSubscriptionReschedule:class TestTriggerResearchSync:+ # Registered public engines: a scheduled run applies the two-axis egress+ # check first, which refuses an unknown engine with a cloud provider.def _base_sub(self, search_engine="searxng"):return {"id": 1,@@ -959,7 +961,7 @@ class TestTriggerResearchSync:mock_db = _ctx(MagicMock())mock_sm = MagicMock()mock_sm.get_settings_snapshot.return_value = {- "search.tool": {"value": "google", "ui_element": "select"}+ "search.tool": {"value": "wikipedia", "ui_element": "select"}}with (@@ -978,20 +980,23 @@ class TestTriggerResearchSync:_setup_user(sched, "u")mock_db = _ctx(MagicMock())mock_sm = MagicMock()- snapshot = {}+ snapshot = {"search.tool": {"value": "searxng", "ui_element": "select"}}mock_sm.get_settings_snapshot.return_value = snapshotwith (patch(DB_SESSION, return_value=mock_db),patch(SETTINGS_MGR, return_value=mock_sm),- patch(QUICK_SUMMARY, return_value={"report": "r"}),+ patch(QUICK_SUMMARY, return_value={"report": "r"}) as qs,patch(SET_CTX),patch.object(sched, "_store_research_result"),):sched._trigger_subscription_research_sync(- "u", self._base_sub(search_engine="bing")+ "u", self._base_sub(search_engine="wikipedia"))- assert snapshot["search.tool"]["value"] == "bing"+ run_snapshot = qs.call_args.kwargs["settings_snapshot"]+ assert run_snapshot["search.tool"]["value"] == "wikipedia"+ # The run gets an overlaid copy; the user's snapshot is untouched.+ assert snapshot["search.tool"]["value"] == "searxng"def test_exception_handled(self, sched):_setup_user(sched, "u")diff --git a/tests/news/test_subscription_model_prefix.py b/tests/news/test_subscription_model_prefix.pynew file mode 100644index 0000000000..349c1fb1ab--- /dev/null+++ b/tests/news/test_subscription_model_prefix.py@@ -0,0 +1,613 @@+"""Saved news subscriptions run with their own provider and model (#7860).++* ``resolve_subscription_model`` turns a saved row into the+ ``(provider, model)`` it runs with, including rows the news page's+ "Create Subscription" saved before #7860 with the whole ``PROVIDER:model``+ selection as the model and its first segment as the provider.+* A scheduled run hands quick_summary one settings snapshot carrying the+ subscription's provider, model, search engine and (for ``openai_endpoint``)+ custom endpoint (quick_summary reads its ``provider`` argument only when no+ snapshot is given), and generates the headline and topics from that same+ snapshot.+* A scheduled run is refused, and researches and generates nothing, when+ its egress context cannot be built, when the two-axis rule refuses it, or+ when it would use an ``openai_endpoint`` URL that is unset or fails the+ custom-endpoint check run-now applies.+* The run-now/overdue payload resolves saved rows the same way.+"""++from contextlib import contextmanager+from datetime import datetime, UTC+from unittest.mock import MagicMock, Mock, patch++import pytest++from local_deep_research.news.subscription_runner import (+ build_subscription_request_data,+ resolve_subscription_model,+)++DB_SESSION = "local_deep_research.database.session_context.get_user_db_session"+SETTINGS_MGR = "local_deep_research.settings.manager.SettingsManager"+SET_CTX = "local_deep_research.config.thread_settings.set_settings_context"+GET_LLM = "local_deep_research.api.research_functions.get_llm"+QUICK_SUMMARY = "local_deep_research.api.research_functions.quick_summary"+HEADLINE_GEN = (+ "local_deep_research.news.utils.headline_generator.generate_headline"+)+TOPIC_GEN = "local_deep_research.news.utils.topic_generator.generate_topics"+REPORT_STORAGE = "local_deep_research.storage.get_report_storage"+CITATION_FMT = (+ "local_deep_research.text_optimization.citation_formatter.CitationFormatter"+)+GET_SETTING_SNAP = (+ "local_deep_research.config.search_config.get_setting_from_snapshot"+)++# The user's own settings in every scheduler test below.+DEFAULT_PROVIDER = "openai"+DEFAULT_MODEL = "gpt-4.1-mini"+PRIMARY_ENGINE = "searxng"+USER_ENDPOINT = "http://10.0.0.5:8080/v1"+++ "provider,model,expected",+ [+ # Saved from the model dropdown before #7860: the uppercase key is+ # stripped once, whatever case the provider was stored in.+ ("ollama", "OLLAMA:llama3", ("ollama", "llama3")),+ ("OLLAMA", "OLLAMA:llama3", ("ollama", "llama3")),+ ("ollama", "OLLAMA:llama3.1:8b", ("ollama", "llama3.1:8b")),+ ("openai", "OPENAI:gpt-4.1-mini", ("openai", "gpt-4.1-mini")),+ (+ "openai_endpoint",+ "OPENAI_ENDPOINT:org/model",+ ("openai_endpoint", "org/model"),+ ),+ ("ollama", "OLLAMA:OLLAMA:model", ("ollama", "OLLAMA:model")),+ ("ollama", "OLLAMA:", ("ollama", None)),+ # A model tag typed by hand, its first segment saved as the+ # provider: the provider is unset and the model kept whole, even+ # when that segment is a provider key in another case.+ ("qwen", "qwen:7b", (None, "qwen:7b")),+ ("llama3.1", "llama3.1:8b", (None, "llama3.1:8b")),+ ("ollama", "ollama:llama3.1:8b", (None, "ollama:llama3.1:8b")),+ # Any other row is kept as saved; OPENAI is not a prefix of+ # OPENAI_ENDPOINT:, and another provider's prefix is not stripped.+ ("ollama", "llama3.1:8b", ("ollama", "llama3.1:8b")),+ (+ "openai",+ "OPENAI_ENDPOINT:org/model",+ ("openai", "OPENAI_ENDPOINT:org/model"),+ ),+ ("ollama", "OPENAI:model:tag", ("ollama", "OPENAI:model:tag")),+ (+ "openrouter",+ "vendor/model:free",+ ("openrouter", "vendor/model:free"),+ ),+ # Not a built-in provider key: unset.+ ("custom", "my-model", (None, "my-model")),+ # NULL/'' provider: unset, and the model is not touched.+ (None, "OLLAMA:llama3", (None, "OLLAMA:llama3")),+ ("", "OLLAMA:llama3", (None, "OLLAMA:llama3")),+ (None, "llama3.1:8b", (None, "llama3.1:8b")),+ # NULL/'' model: unset.+ ("ollama", None, ("ollama", None)),+ ("ollama", "", ("ollama", None)),+ (None, None, (None, None)),+ ("", "", (None, None)),+ ],+)+def test_resolve_subscription_model(provider, model, expected):+ assert resolve_subscription_model(provider, model) == expected+++ "provider,model,expected",+ [+ ("ollama", "OLLAMA:llama3.1:8b", ("ollama", "llama3.1:8b")),+ ("qwen", "qwen:7b", (None, "qwen:7b")),+ ("llama3.1", "llama3.1:8b", (None, "llama3.1:8b")),+ ("ollama", "", ("ollama", None)),+ ],+)+def test_run_now_payload_resolves_the_saved_row(provider, model, expected):+ """The run-now route and the overdue sweep build their payload here."""+ data = build_subscription_request_data(+ query_template="q",+ current_date="2026-10-24",+ triggered_by="manual",+ subscription_id="s",+ model_provider=provider,+ model=model,+ )+ assert (data["model_provider"], data["model"]) == expected+++def scheduler(monkeypatch):+ from local_deep_research.scheduler.background import (+ BackgroundJobScheduler,+ )+ import local_deep_research.scheduler.background as mod++ # The egress scope comes from the test snapshots, not the environment.+ monkeypatch.delenv("LDR_POLICY_EGRESS_SCOPE", raising=False)+ BackgroundJobScheduler._instance = None+ mod._scheduler_instance = None+ with patch(f"{mod.__name__}.BackgroundScheduler") as cls:+ cls.return_value = MagicMock()+ sched = BackgroundJobScheduler()+ sched.user_sessions["u"] = {+ "scheduled_jobs": set(),+ "last_activity": datetime.now(UTC),+ }+ sched._credential_store.store("u", "pw")+ sched.scheduler.get_job.return_value = None+ yield sched+ BackgroundJobScheduler._instance = None+ mod._scheduler_instance = None+++def _ctx(session):+ session.__enter__ = Mock(return_value=session)+ session.__exit__ = Mock(return_value=False)+ return session+++def _value(snapshot, key):+ entry = snapshot[key]+ return entry["value"] if isinstance(entry, dict) else entry+++def _user_snapshot(+ egress_scope=None,+ primary_engine=PRIMARY_ENGINE,+ user_provider=DEFAULT_PROVIDER,+ user_endpoint=USER_ENDPOINT,+):+ snapshot = {+ "llm.provider": {"value": user_provider, "ui_element": "select"},+ "llm.model": {"value": DEFAULT_MODEL, "ui_element": "text"},+ }+ if user_endpoint is not None:+ snapshot["llm.openai_endpoint.url"] = {+ "value": user_endpoint,+ "ui_element": "text",+ }+ if primary_engine:+ snapshot["search.tool"] = {+ "value": primary_engine,+ "ui_element": "select",+ }+ if egress_scope:+ snapshot["policy.egress_scope"] = {+ "value": egress_scope,+ "ui_element": "select",+ }+ return snapshot+++@contextmanager+def _saved_subscription(+ provider,+ model,+ search_engine=None,+ egress_scope=None,+ custom_endpoint=None,+ primary_engine=PRIMARY_ENGINE,+ user_provider=DEFAULT_PROVIDER,+ user_endpoint=USER_ENDPOINT,+):+ """Patch the user database to hold one active subscription row."""+ sub = MagicMock()+ sub.status = "active"+ sub.id = 1+ sub.name = "Sub"+ sub.query_or_topic = "fusion news"+ sub.refresh_interval_minutes = 60+ sub.model_provider = provider+ sub.model = model+ sub.search_strategy = "news_aggregation"+ sub.search_engine = search_engine+ sub.custom_endpoint = custom_endpoint+ db = _ctx(MagicMock())+ db.get.return_value = sub+ settings_manager = MagicMock()+ # A fresh copy per call, as SettingsManager builds one.+ settings_manager.get_settings_snapshot.side_effect = (+ lambda *args, **kwargs: _user_snapshot(+ egress_scope, primary_engine, user_provider, user_endpoint+ )+ )+ with (+ patch(DB_SESSION, return_value=db),+ patch(SETTINGS_MGR, return_value=settings_manager),+ patch(SET_CTX),+ ):+ yield db+++class _StopAtLLM(Exception):+ """Raised by the patched get_llm to end the run once it is reached."""+++def _research_llm_call(scheduler, provider, model, **kwargs):+ """Run the saved row through the scheduler and the real quick_summary up+ to get_llm; return the provider, model and snapshot get_llm resolves."""+ with (+ _saved_subscription(provider, model, **kwargs),+ patch(GET_LLM, side_effect=_StopAtLLM) as get_llm,+ patch.object(scheduler, "_store_research_result") as store,+ ):+ scheduler._check_subscription("u", 1)++ get_llm.assert_called_once()+ store.assert_not_called()+ call = get_llm.call_args.kwargs+ snapshot = call["settings_snapshot"]+ # get_llm uses an explicit argument and otherwise the snapshot.+ used_provider = call["provider"]+ if used_provider is None:+ used_provider = _value(snapshot, "llm.provider")+ used_model = call["model_name"]+ if used_model is None:+ used_model = _value(snapshot, "llm.model")+ return used_provider, used_model, snapshot+++SCHEDULED_ROWS = [+ # The subscription's own provider and model.+ ("ollama", "llama3.1:8b", ("ollama", "llama3.1:8b")),+ ("anthropic", "claude-model", ("anthropic", "claude-model")),+ # Saved from the dropdown before #7860.+ ("ollama", "OLLAMA:llama3.1:8b", ("ollama", "llama3.1:8b")),+ # A typed model tag: the user's provider with the whole tag.+ ("qwen", "qwen:7b", (DEFAULT_PROVIDER, "qwen:7b")),+ ("llama3.1", "llama3.1:8b", (DEFAULT_PROVIDER, "llama3.1:8b")),+ # Unset: the user's provider and model.+ (None, None, (DEFAULT_PROVIDER, DEFAULT_MODEL)),+ ("", "", (DEFAULT_PROVIDER, DEFAULT_MODEL)),+ ("ollama", "", ("ollama", DEFAULT_MODEL)),+]++[email protected]("provider,model,expected", SCHEDULED_ROWS)+def test_scheduled_research_runs_on_the_subscription_provider_and_model(+ scheduler, provider, model, expected+):+ used_provider, used_model, _ = _research_llm_call(+ scheduler, provider, model+ )+ assert (used_provider, used_model) == expected+++ "egress_scope,expected_engine",+ [+ (None, "wikipedia"),+ # Under STRICT the shared overlay keeps the saved primary engine.+ ("strict", PRIMARY_ENGINE),+ ],+)+def test_scheduled_search_engine_goes_through_the_runtime_overlay(+ scheduler, egress_scope, expected_engine+):+ used_provider, used_model, snapshot = _research_llm_call(+ scheduler,+ "ollama",+ "llama3.1:8b",+ search_engine="wikipedia",+ egress_scope=egress_scope,+ )+ assert _value(snapshot, "search.tool") == expected_engine+ assert (used_provider, used_model) == ("ollama", "llama3.1:8b")++[email protected]("provider,model,expected", SCHEDULED_ROWS)+def test_headline_and_topics_use_the_research_provider_and_model(+ scheduler, provider, model, expected+):+ formatter = MagicMock()+ formatter.return_value.format_document.return_value = "R"+ with (+ _saved_subscription(provider, model) as db,+ patch(+ QUICK_SUMMARY,+ return_value={"report": "R", "query": "q", "sources": []},+ ) as quick_summary,+ patch(HEADLINE_GEN, return_value="H") as headline,+ patch(TOPIC_GEN, return_value=["t"]) as topics,+ patch(REPORT_STORAGE),+ patch(CITATION_FMT, formatter),+ patch(GET_SETTING_SNAP, return_value="no_hyperlinks"),+ ):+ scheduler._check_subscription("u", 1)++ quick_summary.assert_called_once()+ headline.assert_called_once()+ topics.assert_called_once()+ db.add.assert_called_once()+ research_snapshot = quick_summary.call_args.kwargs["settings_snapshot"]+ for snapshot in (+ research_snapshot,+ headline.call_args.kwargs["settings_snapshot"],+ topics.call_args.kwargs["settings_snapshot"],+ ):+ used = (+ _value(snapshot, "llm.provider"),+ _value(snapshot, "llm.model"),+ )+ assert used == expected+ assert snapshot["_username"] == "u"+++BACKGROUND_LOGGER = "local_deep_research.scheduler.background.logger"+# A self-hosted store, classified sensitive by the two-axis rule (ADR-0007).+SENSITIVE_PRIMARY = "elasticsearch"+++@contextmanager+def _scheduled_news_generation():+ """Patch quick_summary and the headline/topic generation after it."""+ formatter = MagicMock()+ formatter.return_value.format_document.return_value = "R"+ with (+ patch(+ QUICK_SUMMARY,+ return_value={"report": "R", "query": "q", "sources": []},+ ) as quick_summary,+ patch(HEADLINE_GEN, return_value="H") as headline,+ patch(TOPIC_GEN, return_value=["t"]) as topics,+ patch(REPORT_STORAGE),+ patch(CITATION_FMT, formatter),+ patch(GET_SETTING_SNAP, return_value="no_hyperlinks"),+ ):+ yield quick_summary, headline, topics+++ "primary_engine,search_engine,provider,admitted,reason,offending",+ [+ # Configurations the two-axis rule refuses.+ (+ SENSITIVE_PRIMARY,+ None,+ "anthropic",+ False,+ "sensitive_to_exposing_inference",+ ("llm:anthropic",),+ ),+ (+ SENSITIVE_PRIMARY,+ None,+ "openai",+ False,+ "sensitive_to_exposing_inference",+ ("llm:openai",),+ ),+ # The subscription's own search engine is the one audited, not the+ # user's saved primary.+ (+ PRIMARY_ENGINE,+ SENSITIVE_PRIMARY,+ "anthropic",+ False,+ "sensitive_to_exposing_inference",+ ("llm:anthropic",),+ ),+ # Configurations it allows.+ (SENSITIVE_PRIMARY, None, "ollama", True, None, ()),+ (PRIMARY_ENGINE, None, "anthropic", True, None, ()),+ (SENSITIVE_PRIMARY, PRIMARY_ENGINE, "anthropic", True, None, ()),+ ],+)+def test_scheduled_run_applies_the_two_axis_egress_check(+ scheduler,+ primary_engine,+ search_engine,+ provider,+ admitted,+ reason,+ offending,+):+ with (+ _saved_subscription(+ provider,+ "some-model",+ search_engine=search_engine,+ primary_engine=primary_engine,+ user_provider="ollama",+ ) as db,+ _scheduled_news_generation() as (quick_summary, headline, topics),+ patch(BACKGROUND_LOGGER) as logger,+ ):+ scheduler._check_subscription("u", 1)++ for generated in (quick_summary, headline, topics, db.add):+ assert generated.called is admitted+ refusals = [+ call.args[0] for call in logger.bind.return_value.warning.call_args_list+ ]+ if admitted:+ snapshot = quick_summary.call_args.kwargs["settings_snapshot"]+ assert _value(snapshot, "llm.provider") == provider+ assert _value(snapshot, "search.tool") == (+ search_engine or primary_engine+ )+ assert not any(+ message.startswith("scheduled subscription run refused")+ for message in refusals+ ), refusals+ else:+ # Pin the refusal line AND its stable fields. The call-suppression+ # asserts above pass whether the run was refused by the rule or+ # blew up on the log path (the error is swallowed by+ # _check_subscription's catch-all and the run generates nothing+ # either way), so without this the line could silently vanish.+ logger.bind.assert_any_call(policy_audit=True)+ logger.bind.return_value.warning.assert_called_once_with(+ "scheduled subscription run refused: two-axis egress",+ subscription_id=1,+ reason=reason,+ offending=offending,+ )+++ "primary_engine,egress_scope,exc_name",+ [+ # No primary engine: the run's egress scope cannot be resolved+ # (ValueError).+ (None, None, "ValueError"),+ # An unknown saved scope (PolicyDeniedError).+ (PRIMARY_ENGINE, "not-a-scope", "PolicyDeniedError"),+ ],+)+def test_scheduled_run_is_refused_when_its_egress_context_cannot_be_built(+ scheduler, primary_engine, egress_scope, exc_name+):+ with (+ _saved_subscription(+ "anthropic",+ "some-model",+ primary_engine=primary_engine,+ egress_scope=egress_scope,+ ) as db,+ _scheduled_news_generation() as (quick_summary, headline, topics),+ patch(BACKGROUND_LOGGER) as logger,+ ):+ scheduler._check_subscription("u", 1)++ for generated in (quick_summary, headline, topics, db.add):+ generated.assert_not_called()+ logger.bind.assert_any_call(policy_audit=True)+ # Pin the refusal line AND its stable fields. ``reason`` is the+ # exception CLASS NAME only: the exception text can carry the refused+ # target (PolicyDeniedError holds ``target=str(raw)``) and must never+ # reach the log (the security suites enforce this repo-wide).+ logger.bind.return_value.warning.assert_called_once_with(+ "scheduled subscription run refused: egress policy unevaluable",+ subscription_id=1,+ reason=exc_name,+ )+++SUBSCRIPTION_ENDPOINT = "http://192.168.1.20:8000/v1"+++ "provider,custom_endpoint,expected_endpoint",+ [+ # An openai_endpoint row runs against the endpoint saved with it.+ ("openai_endpoint", SUBSCRIPTION_ENDPOINT, SUBSCRIPTION_ENDPOINT),+ # Without one, the user's endpoint setting.+ ("openai_endpoint", None, USER_ENDPOINT),+ ("openai_endpoint", "", USER_ENDPOINT),+ # Another provider ignores the saved endpoint, as run-now does.+ ("ollama", SUBSCRIPTION_ENDPOINT, USER_ENDPOINT),+ ],+)+def test_scheduled_openai_endpoint_run_uses_the_subscription_endpoint(+ scheduler, provider, custom_endpoint, expected_endpoint+):+ used_provider, used_model, snapshot = _research_llm_call(+ scheduler, provider, "org/model", custom_endpoint=custom_endpoint+ )+ assert (used_provider, used_model) == (provider, "org/model")+ assert _value(snapshot, "llm.openai_endpoint.url") == expected_endpoint+++# Link-local endpoints the openai_endpoint provider's own base-URL check+# admits but the custom-endpoint check run-now applies refuses.+LINK_LOCAL_ENDPOINTS = [+ "http://169.254.1.2:11434/v1",+ "169.254.1.2:11434",+ "http://[fe80::1]:8000/v1",+]+LINK_LOCAL_ENDPOINT = LINK_LOCAL_ENDPOINTS[0]+++# run_endpoint: the URL in the snapshot the run gets, or None when the run is+# refused.+ "provider,custom_endpoint,user_provider,user_endpoint,run_endpoint",+ [+ # The subscription's own endpoint fails the check.+ *[+ ("openai_endpoint", endpoint, "ollama", USER_ENDPOINT, None)+ for endpoint in LINK_LOCAL_ENDPOINTS+ ],+ (+ "openai_endpoint",+ "http://169.254.169.254/latest/meta-data/",+ "ollama",+ USER_ENDPOINT,+ None,+ ),+ # No endpoint on the subscription: the user's saved URL is the one+ # checked, also when openai_endpoint is the user's own provider.+ ("openai_endpoint", None, "ollama", LINK_LOCAL_ENDPOINT, None),+ ("openai_endpoint", "", "ollama", LINK_LOCAL_ENDPOINT, None),+ (None, None, "openai_endpoint", LINK_LOCAL_ENDPOINT, None),+ # No endpoint anywhere (user URL blank or not saved).+ ("openai_endpoint", None, "ollama", "", None),+ ("openai_endpoint", " ", "ollama", None, None),+ (None, None, "openai_endpoint", "", None),+ # A safe endpoint runs, and the subscription's endpoint replaces an+ # unsafe saved URL. Another provider does not use the endpoint, so+ # neither an unsafe nor a blank one is checked.+ (+ "openai_endpoint",+ SUBSCRIPTION_ENDPOINT,+ "ollama",+ LINK_LOCAL_ENDPOINT,+ SUBSCRIPTION_ENDPOINT,+ ),+ ("openai_endpoint", None, "ollama", USER_ENDPOINT, USER_ENDPOINT),+ ("ollama", LINK_LOCAL_ENDPOINT, "ollama", "", ""),+ ],+)+def test_scheduled_run_checks_the_endpoint_it_would_use(+ scheduler,+ provider,+ custom_endpoint,+ user_provider,+ user_endpoint,+ run_endpoint,+):+ """An openai_endpoint run needs a URL that passes the check run-now+ applies, whether it comes from the subscription or the user's settings;+ otherwise nothing runs and a policy-audit line is logged."""+ with (+ _saved_subscription(+ provider,+ "org/model",+ custom_endpoint=custom_endpoint,+ user_provider=user_provider,+ user_endpoint=user_endpoint,+ ) as db,+ _scheduled_news_generation() as (quick_summary, headline, topics),+ patch(BACKGROUND_LOGGER) as logger,+ ):+ scheduler._check_subscription("u", 1)++ admitted = run_endpoint is not None+ for generated in (quick_summary, headline, topics, db.add):+ assert generated.called is admitted+ refusals = [+ call.args[0] for call in logger.bind.return_value.warning.call_args_list+ ]+ if admitted:+ snapshot = quick_summary.call_args.kwargs["settings_snapshot"]+ assert _value(snapshot, "llm.openai_endpoint.url") == run_endpoint+ assert "scheduled subscription run refused: custom endpoint" not in (+ refusals+ )+ else:+ logger.bind.assert_any_call(policy_audit=True)+ assert refusals == [+ "scheduled subscription run refused: custom endpoint"+ ], refusalsdiff --git a/tests/web/test_news_run_once_model_dispatch.py b/tests/web/test_news_run_once_model_dispatch.pyindex d7c54902f7..fc5d21e5ce 100644--- a/tests/web/test_news_run_once_model_dispatch.py+++ b/tests/web/test_news_run_once_model_dispatch.py@@ -12,12 +12,20 @@ from langchain_core.language_models.fake_chat_models import FakeListChatModelfrom local_deep_research.config.llm_config import get_llmfrom local_deep_research.llm.providers import discover_providers+from local_deep_research.llm.providers.base import normalize_provider+from local_deep_research.news.subscription_runner import (+ resolve_subscription_model,+)from local_deep_research.web.routers.research import _extract_research_paramsNEWS_JS = (Path(__file__).resolve().parents[2]/ "src/local_deep_research/web/static/js/pages/news.js")+SUBSCRIPTION_FORM = (+ Path(__file__).resolve().parents[2]+ / "src/local_deep_research/web/templates/pages/news-subscription-form.html"+)PAYLOAD_PROBE = r"""const fs = require('node:fs');const vm = require('node:vm');@@ -56,7 +64,10 @@ vm.runInContext(("OLLAMA:llama3.1:8b", "ollama", "llama3.1:8b"),("llama3.1:8b", "ollama", "llama3.1:8b"),("custom:model:tag", "ollama", "custom:model:tag"),- ("ollama:llama3.1:8b", "ollama", "llama3.1:8b"),+ # A typed model tag stays whole on the default provider, even when+ # its first segment is a provider key in lowercase.+ ("ollama:llama3.1:8b", "ollama", "ollama:llama3.1:8b"),+ ("qwen:7b", "ollama", "qwen:7b"),("OLLAMA:OPENAI:model:tag", "ollama", "OPENAI:model:tag"),("gpt-4.1-mini", "ollama", "gpt-4.1-mini"),("", "ollama", "saved-model:tag"),@@ -110,24 +121,154 @@ def test_news_run_once_model_reaches_provider_constructor(assert llm.base_llm is fake-def test_news_run_once_provider_set_matches_the_provider_registry():+ "source", [NEWS_JS, SUBSCRIPTION_FORM], ids=lambda path: path.name+)+def test_news_run_once_provider_set_matches_the_provider_registry(source):"""Run Once splits a ``PROVIDER:model`` selection only when the prefixis in news.js's ``knownProviders`` set; any other prefix stays inside- the model id and the run silently uses the default provider. That set- is hand-maintained, so pin it to the auto-discovered provider registry:- a provider added to the registry (or removed from it) must be added to- (or removed from) the set too."""+ the model id and the run silently uses the default provider. The+ subscription form's Test Run resolves saved rows with the same set.+ Both sets are hand-maintained, so pin them to the auto-discovered+ provider registry: a provider added to the registry (or removed from+ it) must be added to (or removed from) both sets too."""match = re.search(r"const knownProviders = new Set\(\[(.*?)\]\);",- NEWS_JS.read_text(encoding="utf-8"),+ source.read_text(encoding="utf-8"),re.DOTALL,)- assert match is not None, "knownProviders set not found in news.js"+ assert match is not None, f"knownProviders set not found in {source.name}"listed = re.findall(r"'([^']*)'", match.group(1))assert len(listed) == len(set(listed)), f"duplicate keys: {listed}"registry = set(discover_providers(force_refresh=True))assert registry, "provider auto-discovery returned nothing"assert set(listed) == registry, (- f"news.js knownProviders is missing {sorted(registry - set(listed))} "+ f"{source.name} knownProviders is missing "+ f"{sorted(registry - set(listed))} "f"and lists unregistered {sorted(set(listed) - registry)}")+++def _extract_form_resolver(source: str) -> str:+ """The ``resolveSavedSubscriptionModel`` body out of the template."""+ start = source.index("function resolveSavedSubscriptionModel")+ open_brace = source.index("{", start)+ depth = 0+ for index in range(open_brace, len(source)):+ if source[index] == "{":+ depth += 1+ elif source[index] == "}":+ depth -= 1+ if depth == 0:+ return source[start : index + 1]+ raise AssertionError(+ "resolveSavedSubscriptionModel has an unterminated body"+ )+++def test_form_test_run_resolver_drops_non_registry_providers():+ """Test Run must resolve a saved row the way scheduled runs do.++ ``resolve_subscription_model`` keeps a provider only when it names a+ built-in provider key (compared after stripping); a non-registry+ provider is dropped — the run then uses the user's default provider —+ and the model is kept whole. Pinned from the template source so the+ mirror holds without a JS runtime; the behavioural parity matrix runs+ under node in the test below.+ """+ body = _extract_form_resolver(SUBSCRIPTION_FORM.read_text(encoding="utf-8"))+ assert re.search(+ r"if\s*\(\s*provider\s*&&\s*!\s*knownProviders\.has\("+ r"\s*provider\.trim\(\)\.toUpperCase\(\)\s*\)\s*\)"+ r"\s*\{\s*provider\s*=\s*''\s*;",+ body,+ ), body+++FORM_RESOLVER_PROBE = r"""+const fs = require('node:fs');+const source = fs.readFileSync(process.argv[1], 'utf8');+const start = source.indexOf('function resolveSavedSubscriptionModel');+const openBrace = source.indexOf('{', start);+let depth = 0;+let end = -1;+for (let i = openBrace; i < source.length; i += 1) {+ if (source[i] === '{') depth += 1;+ else if (source[i] === '}') {+ depth -= 1;+ if (depth === 0) { end = i + 1; break; }+ }+}+const resolve = eval('(' + source.slice(start, end) + ')');+const rows = JSON.parse(process.argv[2]);+const out = rows.map(([provider, model]) => {+ const resolved = resolve(provider, model);+ return [resolved.provider || null, resolved.model || null];+});+process.stdout.write(JSON.stringify(out));+"""+++#: ``(provider, model)`` rows covering the resolver matrix's shapes:+#: the old-form uppercase prefix, typed model tags, plain rows, unset+#: values, and non-registry providers (where the Test Run resolver used to+#: keep the provider the scheduled run drops).+FORM_RESOLVER_MATRIX = [+ ("ollama", "OLLAMA:llama3"),+ ("OLLAMA", "OLLAMA:llama3.1:8b"),+ ("ollama", "OLLAMA:llama3.1:8b"),+ ("openai", "OPENAI:gpt-4.1-mini"),+ ("openai_endpoint", "OPENAI_ENDPOINT:org/model"),+ ("ollama", "OLLAMA:OLLAMA:model"),+ ("ollama", "OLLAMA:"),+ ("qwen", "qwen:7b"),+ ("llama3.1", "llama3.1:8b"),+ ("ollama", "ollama:llama3.1:8b"),+ ("ollama", "llama3.1:8b"),+ ("openai", "OPENAI_ENDPOINT:org/model"),+ ("ollama", "OPENAI:model:tag"),+ ("openrouter", "vendor/model:free"),+ ("custom", "my-model"),+ ("custom", "custom:tag"),+ ("custom", "CUSTOM:tag"),+ ("custom", "OLLAMA:llama3"),+ ("", "OLLAMA:llama3"),+ ("ollama", ""),+ ("", ""),+]+++def test_form_test_run_resolver_parity_with_the_python_resolver():+ """The template's Test Run resolver must resolve every row to the run+ ``resolve_subscription_model`` describes. An empty resolved field is+ what the request omits (``if (provider) requestData.model_provider =+ provider``), the API spelling of the Python resolver's ``None`` ("use+ the user's setting"); the provider is compared in canonical form,+ because the request carries the raw saved casing and the backend+ normalizes both sides with ``normalize_provider``."""+ node = shutil.which("node")+ if node is None:+ pytest.skip("Node.js is required to execute the template's resolver")+ result = subprocess.run(+ [+ node,+ "-e",+ FORM_RESOLVER_PROBE,+ str(SUBSCRIPTION_FORM),+ json.dumps(FORM_RESOLVER_MATRIX),+ ],+ capture_output=True,+ text=True,+ check=True,+ timeout=10,+ )+ resolved = json.loads(result.stdout)+ assert len(resolved) == len(FORM_RESOLVER_MATRIX)+ for (provider, model), (js_provider, js_model) in zip(+ FORM_RESOLVER_MATRIX, resolved+ ):+ py_provider, py_model = resolve_subscription_model(provider, model)+ assert js_model == py_model, (provider, model)+ assert normalize_provider(js_provider) == normalize_provider(+ py_provider+ ), (provider, model)