~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
"""Security coverage for ``web/services/research_service.py`` (FastAPI port).
WHY THIS FILE EXISTS
--------------------
The historical ADR-0010 review found that predecessor tests for
``research_service.py`` had been removed and that some successors did not call
the service directly. This file supplies behavior-level regression evidence by
calling the production functions. Nothing below re-implements service logic;
every assertion is on a value produced by ``research_service`` itself.
WHAT IS COVERED HERE
--------------------
1. ``run_research_process`` username gate (research_service.py:935). Both
predecessor tests that executed it were deleted; this file restores direct
coverage. Without the gate a worker runs with ``username=None`` and every
encrypted-DB write in the run is silently dropped or misattributed.
2. The ``username`` KEYWORD-ONLY barrier on ``save_research_strategy`` /
``get_research_strategy`` (research_service.py:345 / :393). Making it
positional-or-optional reopens the ``get_user_db_session(None)`` fallback
from #4526 --- work silently attributed to no user. At the review snapshot,
``save_*`` had no direct behavioral test (``grep`` found only call sites
that patched it out plus a hand-rolled ORM insert in
``tests/database/test_research_strategy_fk_regression.py``); this file now
pins it.
PARTIAL OVERLAP, declared: ``get_research_strategy``'s per-user DB
selection *is* genuinely covered by
``tests/security/test_metrics_hostile_input_fastapi.py::
test_get_research_strategy_selects_the_db_by_username``. What is NOT
covered there --- and is covered here --- is the write side and the
keyword-only signature itself (that file pins the kwarg at one *route*
call site via AST, which a signature change would not trip).
3. ``cancel_research`` cross-user isolation (research_service.py:3194). All
8 predecessor tests of ``TestCancelResearch`` were deleted; at the review
snapshot the one surviving caller
(``tests/web/queue/test_queued_research_lifecycle_races.py``) patched the
database session out entirely. The cases below now prove that ``username``
selects the database that gets written.
4. Per-user LLM-provider scoping of news headline/topic generation
(research_service.py:2195/:2227/:2238) --- the
``settings_snapshot["_username"]`` threading via the real
``ensure_snapshot_username``. Without it the LLM policy-enforcement
point cannot resolve a per-user provider.
5. ``_generate_report_path`` (research_service.py:518) had no direct coverage
at the review snapshot. The cases below pin the module's path-containment
guard: the md5 filename stops ``../`` or a NUL byte in a user-controlled
query from reaching the filesystem.
6. CWE-209 error sanitisation (research_service.py:2757/:2765/:2779), 10
deleted predecessor tests. At the review snapshot, ``grep -rn`` for each
genericised literal returned no hits. The cases below now assert all three.
VACUITY
-------
Every negative assertion is paired with a positive control asserted FIRST.
The isolation tests seed the SAME ``research_id`` into BOTH users' encrypted
databases with different content, so "user B saw nothing" cannot pass
against an empty table, and the caller's own row is always asserted before
the other user's is asserted unchanged. The worker-driven tests assert that
the code under test was actually reached (the generator was called, the
error update was queued) before asserting what it must not contain.
"""
from __future__ import annotations
import inspect
import itertools
import os
import re
import uuid
from contextlib import contextmanager
from unittest.mock import MagicMock, Mock, create_autospec, patch
import pytest
from loguru import logger
# Imported so this file fails loudly if a symbol is renamed rather than
# silently asserting against a stub.
from local_deep_research.constants import ResearchStatus
from local_deep_research.database.models import (
ResearchHistory,
ResearchStrategy,
)
from local_deep_research.database.session_context import get_user_db_session
from local_deep_research.web import research_state
from local_deep_research.web.services import research_service
from local_deep_research.web.services.research_service import (
_generate_report_path,
cancel_research,
get_research_strategy,
run_research_process,
save_research_strategy,
)
# The app's docs_url toggle reads PYTEST_CURRENT_TEST at app-import time;
# pytest only sets that per-test, not at collection.
os.environ.setdefault("TESTING", "1")
MODULE = "local_deep_research.web.services.research_service"
STATE_MODULE = "local_deep_research.web.research_state"
QUEUE_MODULE = "local_deep_research.web.queue.processor_v2"
PASSWORD = "ResearchIso!Pass123" # noqa: S105 — test-only credential
# The production progress callback emits at the custom "MILESTONE" level
# registered by log_utils.init_loguru, which tests do not run.
try:
logger.level("MILESTONE", no=26)
except (ValueError, TypeError):
pass
# ---------------------------------------------------------------------------
# Two-real-user harness (real registration, real encrypted per-user DBs)
# ---------------------------------------------------------------------------
# MONOTONIC, not random: rate limiting is keyed per client IP and random
# octets collide, producing 429s from /auth/register's per-IP bucket
# that have nothing to do with the guard under test.
_IP_COUNTER = itertools.count(1)
def _next_forwarded_for() -> str:
n = next(_IP_COUNTER)
return f"10.212.{(n // 250) % 250}.{(n % 250) + 1}"
def _fresh_client(app):
from fastapi.testclient import TestClient
client = TestClient(app, raise_server_exceptions=False)
client.headers.update({"X-Forwarded-For": _next_forwarded_for()})
return client
def _csrf(client) -> str:
"""CSRF is enforced by ASGI middleware — fetch a real token."""
client.get("/auth/login")
return client.get("/auth/csrf-token").json()["csrf_token"]
def _register_and_login(app, username: str):
client = _fresh_client(app)
resp = client.post(
"/auth/register",
data={
"username": username,
"password": PASSWORD,
"confirm_password": PASSWORD,
"acknowledge": "true",
"csrf_token": _csrf(client),
},
follow_redirects=False,
)
assert resp.status_code in (200, 302), (
f"registration of {username!r} failed: "
f"{resp.status_code} / {resp.text[:400]}"
)
resp = client.post(
"/auth/login",
data={
"username": username,
"password": PASSWORD,
"csrf_token": _csrf(client),
},
follow_redirects=False,
)
assert resp.status_code in (200, 302), (
f"login of {username!r} failed: {resp.status_code} / {resp.text[:400]}"
)
return client
def _user(prefix: str) -> str:
return f"{prefix}_{uuid.uuid4().hex[:10]}"
def _seed_research(username, research_id, query, status):
"""Insert one ResearchHistory row into *username*'s encrypted DB."""
with get_user_db_session(username, PASSWORD) as session:
session.add(
ResearchHistory(
id=research_id,
query=query,
mode="quick_summary",
status=status,
created_at="2024-01-01T00:00:00",
)
)
session.commit()
def _status_of(username, research_id):
with get_user_db_session(username, PASSWORD) as session:
row = session.query(ResearchHistory).filter_by(id=research_id).first()
return None if row is None else row.status
def _strategy_rows(username, research_id):
with get_user_db_session(username, PASSWORD) as session:
return [
row.strategy_name
for row in session.query(ResearchStrategy).filter_by(
research_id=research_id
)
]
@pytest.fixture
def two_users(app):
"""Two registered, logged-in users with live encrypted databases.
The clients are held for the lifetime of the test: logging out (or
dropping them) would clear the session password the service's
``get_user_db_session(username)`` call resolves through, and every
service call below deliberately runs WITHOUT an explicit password so
that the username really is what selects the database.
``session_password_store`` is a process-wide singleton that
``reset_all_singletons`` does not touch, so the entries are dropped on
teardown — a leaked one would be visible to every later test in the
same worker.
"""
from local_deep_research.database.session_passwords import (
session_password_store,
)
alice, bob = _user("rsiso_alice"), _user("rsiso_bob")
clients = (
_register_and_login(app, alice),
_register_and_login(app, bob),
)
try:
yield {"alice": alice, "bob": bob, "_clients": clients}
finally:
for username in (alice, bob):
session_password_store.clear_all_for_user(username)
# ---------------------------------------------------------------------------
# Real-worker harness
# ---------------------------------------------------------------------------
@contextmanager
def _mock_db_session(session=None):
if session is None:
session = MagicMock()
yield session
def _worker_patches(db_session=None, extra=None):
"""Patches that keep the REAL ``run_research_process`` body running.
Nothing between the entry point and the code under test is replaced:
the username gate, the egress context build, the LLM/search override
branches, the error classifier and the news-snapshot threading all
execute for real. Only the process boundaries (sockets, queue
processor, encrypted DB, thread-local cleanup) are stubbed.
"""
session = MagicMock() if db_session is None else db_session
patches = {
f"{MODULE}.get_user_db_session": lambda *a, **kw: _mock_db_session(
session
),
f"{MODULE}.cleanup_research_resources": MagicMock(),
f"{MODULE}.handle_termination": MagicMock(),
f"{MODULE}.set_search_context": MagicMock(),
f"{MODULE}._sio_emit": create_autospec(
research_service._sio_emit, spec_set=True
),
f"{MODULE}._sio_remove": create_autospec(
research_service._sio_remove, spec_set=True
),
f"{MODULE}._socket_emitter": create_autospec(
research_service._socket_emitter, spec_set=True
),
f"{STATE_MODULE}.is_termination_requested": MagicMock(
return_value=False
),
f"{STATE_MODULE}.is_research_active": MagicMock(return_value=False),
f"{STATE_MODULE}.update_progress_and_check_active": MagicMock(
return_value=(5, True)
),
"local_deep_research.web.routes.globals.is_termination_requested": (
MagicMock(return_value=False)
),
"local_deep_research.settings.logger.log_settings": MagicMock(),
"local_deep_research.config.thread_settings.set_settings_context": (
MagicMock()
),
f"{QUEUE_MODULE}.queue_processor": MagicMock(),
}
if extra:
patches.update(extra)
return patches
@contextmanager
def _worker(patches):
"""Apply a {target: replacement} mapping and yield the mocks by target."""
started = {}
ctxs = []
try:
for target, replacement in patches.items():
ctx = patch(target, replacement)
ctxs.append(ctx)
started[target] = ctx.__enter__()
yield started
finally:
for ctx in reversed(ctxs):
ctx.__exit__(None, None, None)
# A run always carries a matching ``search_engine`` kwarg and snapshot entry:
# ``build_run_egress_context`` resolves the primary from the kwarg, and a
# mismatch makes the worker refuse the run (fail-closed) before any of the
# code under test is reached.
BASE_RUN_KWARGS = {
"mode": "quick",
"search_engine": "searxng",
}
def _snapshot():
return {"search.tool": "searxng"}
# ===========================================================================
# 1. run_research_process username gate (research_service.py:935)
# ===========================================================================
class TestRunResearchProcessUsernameGate:
"""``raise ValueError("Username is required for research process")``.
The worker writes token metrics, research logs and the report into the
*caller's* encrypted database. With no username those writes have no
database to go to; the guard turns that into a loud failure at the top
of the thread instead of a run whose every persistence call is swallowed
by a bare ``except``.
"""
def test_missing_username_raises_before_any_work(self):
"""No ``username`` kwarg at all -> ValueError, nothing started."""
patches = _worker_patches()
with _worker(patches) as mocks:
with pytest.raises(
ValueError, match="Username is required for research process"
):
run_research_process(
research_id=str(uuid.uuid4()),
query="gate probe",
settings_snapshot=_snapshot(),
**BASE_RUN_KWARGS,
)
# The guard must fire BEFORE the thread context is established;
# a set_search_context({"username": None, ...}) would poison
# every log line and DB lookup made from this thread.
assert not mocks[f"{MODULE}.set_search_context"].called
assert not mocks[f"{MODULE}.cleanup_research_resources"].called
@pytest.mark.parametrize("username", ["", None])
def test_falsy_username_raises(self, username):
"""Empty string and explicit ``None`` are refused, not accepted."""
patches = _worker_patches()
with _worker(patches):
with pytest.raises(
ValueError, match="Username is required for research process"
):
run_research_process(
research_id=str(uuid.uuid4()),
query="gate probe",
username=username,
settings_snapshot=_snapshot(),
**BASE_RUN_KWARGS,
)
def test_valid_username_passes_the_gate_and_is_threaded_downstream(self):
"""POSITIVE CONTROL.
Without this every assertion above would also hold for a worker that
raised ``ValueError`` unconditionally. Termination is requested up
front so the run takes the shortest real path past the gate; what is
asserted is that the gate let it through AND that the username it
accepted is the one handed to the thread context and to cleanup.
"""
research_id = str(uuid.uuid4())
patches = _worker_patches(
extra={
f"{STATE_MODULE}.is_termination_requested": MagicMock(
return_value=True
),
}
)
with _worker(patches) as mocks:
run_research_process(
research_id=research_id,
query="gate probe",
username="alice_gate",
settings_snapshot=_snapshot(),
**BASE_RUN_KWARGS,
)
ctx = mocks[f"{MODULE}.set_search_context"]
assert ctx.called, "the gate refused a valid username"
assert ctx.call_args.args[0]["username"] == "alice_gate"
assert ctx.call_args.args[0]["research_id"] == research_id
cleanup = mocks[f"{MODULE}.cleanup_research_resources"]
assert cleanup.called
assert cleanup.call_args.args[1] == "alice_gate", (
"cleanup must be scoped to the run owner"
)
# ===========================================================================
# 2. username keyword-only barrier on the strategy accessors
# (research_service.py:345 / :393)
# ===========================================================================
class TestStrategyUsernameBarrier:
"""``save_research_strategy(rid, name, *, username)`` and its reader.
The ``*`` is the guard. Demoting ``username`` to a positional-or-default
parameter is exactly the shape of the #4526 regression: call sites that
forget it fall back to ``get_user_db_session(None)``, and the strategy
is written to (or read from) whatever database the ambient session
happens to resolve.
"""
@pytest.mark.parametrize(
"func", [save_research_strategy, get_research_strategy]
)
def test_username_is_keyword_only_and_required(self, func):
param = inspect.signature(func).parameters["username"]
assert param.kind is inspect.Parameter.KEYWORD_ONLY, (
f"{func.__name__}: username must stay keyword-only so no caller "
"can drift into the get_user_db_session(None) fallback (#4526)"
)
assert param.default is inspect.Parameter.empty, (
f"{func.__name__}: username must have no default; a default of "
"None reopens the implicit-session fallback"
)
def test_positional_username_is_rejected(self):
"""The barrier at runtime, not only in the signature object."""
with pytest.raises(TypeError):
save_research_strategy("rid", "source-based", "alice")
with pytest.raises(TypeError):
get_research_strategy("rid", "alice")
def test_save_and_get_use_the_callers_own_database(self, two_users):
"""Round trip through the real functions, two users, ONE id.
Both users hold the same ``research_id``. A save that reached the
wrong database would overwrite the other user's strategy; a read
that reached the wrong database would return it. Either way the
assertions below flip.
"""
alice, bob = two_users["alice"], two_users["bob"]
research_id = str(uuid.uuid4())
# ResearchStrategy.research_id is a FK onto research_history.id with
# PRAGMA foreign_keys on, so the parent row must exist in each DB.
_seed_research(
alice, research_id, "alice query", ResearchStatus.COMPLETED
)
_seed_research(bob, research_id, "bob query", ResearchStatus.COMPLETED)
# POSITIVE CONTROL first: the write is really performed.
save_research_strategy(research_id, "alice-strategy", username=alice)
assert _strategy_rows(alice, research_id) == ["alice-strategy"]
assert get_research_strategy(research_id, username=alice) == (
"alice-strategy"
)
# Alice's write must not have landed in Bob's database.
assert _strategy_rows(bob, research_id) == []
assert get_research_strategy(research_id, username=bob) is None
# Bob writes his own value for the same id; alice's must survive.
save_research_strategy(research_id, "bob-strategy", username=bob)
assert get_research_strategy(research_id, username=bob) == (
"bob-strategy"
)
assert get_research_strategy(research_id, username=alice) == (
"alice-strategy"
)
# The update branch (existing row) must also stay in-database.
save_research_strategy(research_id, "alice-strategy-2", username=alice)
assert _strategy_rows(alice, research_id) == ["alice-strategy-2"]
assert _strategy_rows(bob, research_id) == ["bob-strategy"]
# ===========================================================================
# 3. cancel_research cross-user isolation (research_service.py:3194)
# ===========================================================================
@pytest.fixture(autouse=True)
def _clear_termination_flags():
"""``cancel_research`` sets a PROCESS-GLOBAL termination flag.
``research_state._termination_flags`` is module state that outlives the
test; leaving entries behind would make a later worker test take the
"terminated before starting" branch. Recorded ids are cleared after each
test in this module.
"""
before = set(research_state._termination_flags)
yield
for rid in set(research_state._termination_flags) - before:
research_state.clear_termination_flag(rid)
class TestCancelResearchIsolation:
"""``cancel_research(research_id, username)``.
The ORM query carries no owner predicate — the ``username`` argument is
the ONLY thing that decides which encrypted database is opened and
written. These tests seed the same ``research_id`` into both users'
databases so a cancel that reached the wrong one is visible as a status
change on the wrong row, not merely as an absent row.
"""
def test_owner_can_cancel_their_own_research(self, two_users):
"""POSITIVE CONTROL. Without it, "B cannot cancel A's research"
would pass against a function that cancelled nothing at all."""
alice = two_users["alice"]
research_id = str(uuid.uuid4())
_seed_research(
alice, research_id, "alice query", ResearchStatus.IN_PROGRESS
)
assert cancel_research(research_id, alice) is True
assert _status_of(alice, research_id) == ResearchStatus.SUSPENDED
def test_other_user_cannot_cancel_a_research_they_do_not_have(
self, two_users
):
"""Bob has no such row: refusal, and Alice's row is untouched."""
alice, bob = two_users["alice"], two_users["bob"]
research_id = str(uuid.uuid4())
_seed_research(
alice, research_id, "alice query", ResearchStatus.IN_PROGRESS
)
# Bob's database is NOT empty — it holds a different in-progress
# research — so "nothing to cancel" cannot be an artefact of an
# empty table.
bob_own = str(uuid.uuid4())
_seed_research(bob, bob_own, "bob query", ResearchStatus.IN_PROGRESS)
# POSITIVE CONTROL: bob's own cancel works, so the refusal below is
# about ownership and not about bob's database being unreachable.
assert cancel_research(bob_own, bob) is True
assert _status_of(bob, bob_own) == ResearchStatus.SUSPENDED
assert cancel_research(research_id, bob) is False, (
"bob must not be able to cancel a research he does not own"
)
assert _status_of(alice, research_id) == ResearchStatus.IN_PROGRESS
# ...and alice is still able to cancel it herself afterwards, so the
# refusal above did not come from the row/database being unusable.
assert cancel_research(research_id, alice) is True
assert _status_of(alice, research_id) == ResearchStatus.SUSPENDED
def test_colliding_research_id_cancels_only_the_callers_row(
self, two_users
):
"""The sharpest form: both users hold the SAME id, in progress.
Bob cancels. His own row must flip (positive control, in the same
assertion block) and Alice's must not (isolation).
"""
alice, bob = two_users["alice"], two_users["bob"]
research_id = str(uuid.uuid4())
_seed_research(
alice, research_id, "alice query", ResearchStatus.IN_PROGRESS
)
_seed_research(
bob, research_id, "bob query", ResearchStatus.IN_PROGRESS
)
assert cancel_research(research_id, bob) is True
assert _status_of(bob, research_id) == ResearchStatus.SUSPENDED
assert _status_of(alice, research_id) == ResearchStatus.IN_PROGRESS, (
"cancelling bob's research must not suspend alice's research "
"with the same id"
)
# And the reverse direction, so the test cannot pass by always
# writing to whichever database was opened first.
assert cancel_research(research_id, alice) is True
assert _status_of(alice, research_id) == ResearchStatus.SUSPENDED
def test_terminal_state_is_reported_from_the_callers_own_row(
self, two_users
):
"""``True`` for an already-terminal research is decided by the
caller's own row, not by any other user's copy of the id."""
alice, bob = two_users["alice"], two_users["bob"]
research_id = str(uuid.uuid4())
_seed_research(
alice, research_id, "alice query", ResearchStatus.COMPLETED
)
_seed_research(
bob, research_id, "bob query", ResearchStatus.IN_PROGRESS
)
# Alice's row is terminal -> True, and nothing is rewritten.
assert cancel_research(research_id, alice) is True
assert _status_of(alice, research_id) == ResearchStatus.COMPLETED
# Bob's live row was not dragged into a terminal state by it.
assert _status_of(bob, research_id) == ResearchStatus.IN_PROGRESS
# Bob's own call must still act on HIS row (the live one), not on
# alice's terminal copy of the same id.
assert cancel_research(research_id, bob) is True
assert _status_of(bob, research_id) == ResearchStatus.SUSPENDED
assert _status_of(alice, research_id) == ResearchStatus.COMPLETED
# ===========================================================================
# 4. Per-user LLM-provider scoping of news headline/topic generation
# (research_service.py:2195 / :2227 / :2238)
# ===========================================================================
HEADLINE_TARGET = (
"local_deep_research.news.utils.headline_generator.generate_headline"
)
TOPIC_TARGET = "local_deep_research.news.utils.topic_generator.generate_topics"
def _news_research_row():
"""A completed news research whose metadata triggers the news branch."""
row = MagicMock()
row.status = ResearchStatus.IN_PROGRESS
row.created_at = "2024-01-01T00:00:00"
row.report_content = "# Report body"
row.research_meta = {"is_news_search": True, "category": "Markets"}
return row
def _run_quick_news_worker(username, snapshot):
"""Drive the REAL worker to quick-mode completion on a news search.
Returns ``(generate_headline_mock, generate_topics_mock, research_row)``.
``ensure_snapshot_username`` is deliberately NOT patched — the property
under test is that the real one runs and that its output is what reaches
the generators.
"""
session = MagicMock()
research_row = _news_research_row()
session.query.return_value.filter_by.return_value.first.return_value = (
research_row
)
system = Mock()
system.all_links_of_system = []
system.analyze_topic.return_value = {
"findings": [
{"phase": "Final synthesis", "content": "Synthesized answer."}
],
"formatted_findings": "# Summary\n\nSynthesized answer.",
"iterations": 1,
"current_knowledge": "Synthesized answer.",
}
headline = MagicMock(return_value="A Generated Headline")
topics = MagicMock(return_value=["alpha", "beta"])
storage = MagicMock()
storage.save_report.return_value = True
patches = _worker_patches(
db_session=session,
extra={
f"{MODULE}.AdvancedSearchSystem": MagicMock(return_value=system),
f"{MODULE}.get_llm": MagicMock(),
f"{MODULE}.get_search": MagicMock(),
"local_deep_research.storage.get_report_storage": MagicMock(
return_value=storage
),
HEADLINE_TARGET: headline,
TOPIC_TARGET: topics,
},
)
with _worker(patches):
run_research_process(
research_id=str(uuid.uuid4()),
query="market news today",
username=username,
settings_snapshot=snapshot,
**BASE_RUN_KWARGS,
)
return headline, topics, research_row
class TestNewsSnapshotOwnerScoping:
"""``headline_topic_snapshot = ensure_snapshot_username(snapshot, user)``.
Headline/topic generation resolves an LLM through the snapshot. The
per-user policy-enforcement point looks the owner up under
``settings_snapshot["_username"]``; without it the run silently resolves
the shared/built-in namespace instead of the user's own provider (and,
for a user whose policy is local-only, cannot enforce it).
"""
def test_generators_receive_a_snapshot_scoped_to_the_run_owner(self):
snapshot = {"search.tool": "searxng", "llm.provider": "ollama"}
headline, topics, research_row = _run_quick_news_worker(
"alice_news", snapshot
)
# POSITIVE CONTROL: the branch was actually reached. Every
# assertion below is vacuous if the generators were never called.
assert headline.call_count == 1, (
"the news headline branch was never reached"
)
assert topics.call_count == 1
h_snapshot = headline.call_args.kwargs["settings_snapshot"]
t_snapshot = topics.call_args.kwargs["settings_snapshot"]
assert h_snapshot["_username"] == "alice_news"
assert t_snapshot["_username"] == "alice_news"
# One snapshot object is built once and shared by both call sites.
assert h_snapshot is t_snapshot
# The run's own settings must still be carried, i.e. the snapshot is
# the run's snapshot with the owner added — not a bare {"_username"}.
assert h_snapshot["llm.provider"] == "ollama"
# ...and the caller's dict is not mutated (no cross-run bleed).
assert "_username" not in snapshot
# The generated values really came back through the worker.
assert research_row.research_meta["generated_headline"] == (
"A Generated Headline"
)
assert research_row.research_meta["generated_topics"] == [
"alpha",
"beta",
]
def test_the_username_tracks_the_run_owner(self):
"""Anti-constant control: a second run under a different owner must
carry that owner, so the assertion above cannot be satisfied by a
hard-coded or leaked value."""
_, topics_a, _ = _run_quick_news_worker(
"alice_news", {"search.tool": "searxng"}
)
_, topics_b, _ = _run_quick_news_worker(
"bob_news", {"search.tool": "searxng"}
)
assert (
topics_a.call_args.kwargs["settings_snapshot"]["_username"]
== "alice_news"
)
assert (
topics_b.call_args.kwargs["settings_snapshot"]["_username"]
== "bob_news"
)
def test_an_explicit_snapshot_owner_is_never_overwritten(self):
"""A snapshot that already names its owner (e.g. a scheduler-built
one) must be passed through unchanged rather than re-stamped."""
snapshot = {"search.tool": "searxng", "_username": "explicit_owner"}
headline, _, _ = _run_quick_news_worker("alice_news", snapshot)
assert headline.call_count == 1
assert headline.call_args.kwargs["settings_snapshot"]["_username"] == (
"explicit_owner"
)
# ===========================================================================
# 5. _generate_report_path path containment (research_service.py:518)
# ===========================================================================
# The only characters the generated basename may contain. The trailing
# 8-hex random segment keeps same-query, same-second calls from colliding
# in the install-shared directory (preventive; the helper has no caller).
_REPORT_NAME_RE = re.compile(
r"^research_report_[0-9a-f]{10}_\d+_[0-9a-f]{8}\.md$"
)
HOSTILE_QUERIES = [
"../../../etc/passwd",
"..\\..\\..\\windows\\system32\\config\\sam",
"/etc/shadow",
"....//....//etc/passwd",
"%2e%2e%2f%2e%2e%2fetc%2fpasswd",
"nul\x00byte",
"trailing\x00/../../root/.ssh/authorized_keys",
"line\nbreak\r\ninjection",
"semi;colon && rm -rf /",
"~/.ssh/id_rsa",
"\u202eevil",
pytest.param("a" * 5000, id="long-ascii-query-5k"),
"",
]
class TestGenerateReportPathContainment:
"""``_generate_report_path(query)`` — the module's only path guard.
The query is fully user-controlled. The function's containment property
is that the basename is derived ONLY from an md5 hex digest, a unix
timestamp, and a random uuid segment, so no byte of the query can
influence the path. These tests
re-anchor ``OUTPUT_DIR`` to a temp directory so the assertion proves the
result is anchored to that constant rather than merely being "some path
that happens to exist".
"""
def test_benign_query_lands_in_output_dir(self, tmp_path, monkeypatch):
"""POSITIVE CONTROL: the function does produce a usable path under
OUTPUT_DIR, so the containment assertions below are not passing
because the function returns something inert."""
monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)
path = _generate_report_path("a perfectly ordinary question")
assert path.parent == tmp_path
assert _REPORT_NAME_RE.match(path.name), path.name
assert path.suffix == ".md"
# The path is writable as-is — proving it is a real filename and not
# a directory traversal that would fail at open() time.
path.write_text("ok")
assert path.read_text() == "ok"
assert [p.name for p in tmp_path.iterdir()] == [path.name]
@pytest.mark.parametrize("query", HOSTILE_QUERIES)
def test_hostile_query_cannot_escape_output_dir(
self, query, tmp_path, monkeypatch
):
monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)
path = _generate_report_path(query)
assert path.parent == tmp_path, (
f"{query!r} moved the report out of OUTPUT_DIR: {path}"
)
assert path.resolve().parent == tmp_path.resolve(), (
f"{query!r} escaped OUTPUT_DIR after resolution: {path}"
)
assert _REPORT_NAME_RE.match(path.name), (
f"{query!r} reached the filename: {path.name!r}"
)
# Neither a NUL byte nor a separator survives into the name.
assert "\x00" not in str(path)
assert "/" not in path.name and "\\" not in path.name
assert ".." not in path.name
@pytest.mark.parametrize("query", HOSTILE_QUERIES)
def test_no_fragment_of_the_query_reaches_the_filename(
self, query, tmp_path, monkeypatch
):
"""The md5 property: the basename carries no user text at all."""
monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)
name = _generate_report_path(query).name
for token in re.findall(r"[A-Za-z0-9_.-]{4,}", query):
assert token not in name, (
f"query fragment {token!r} leaked into {name!r}"
)
def test_hash_segment_is_a_function_of_the_query(
self, tmp_path, monkeypatch
):
"""Distinct queries must not collapse onto one filename (which would
make one user's report overwrite another's), and the same query must
hash stably (proving the segment is the digest, not a random blob)."""
monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)
def _hash_of(query):
return _generate_report_path(query).name.split("_")[2]
assert _hash_of("query one") == _hash_of("query one")
assert _hash_of("query one") != _hash_of("query two")
assert _hash_of("../../etc/passwd") != _hash_of("../../etc/shadow")
def test_non_ascii_query_is_encoded_not_raised(self, tmp_path, monkeypatch):
"""``query.encode("utf-8")`` — a non-ASCII query must hash, not
raise, and must not reach the name."""
monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)
path = _generate_report_path("Grüße 你好 🙂 ../..")
assert _REPORT_NAME_RE.match(path.name), path.name
assert path.parent == tmp_path
# ===========================================================================
# 6. CWE-209 error sanitisation
# (research_service.py:2757 / :2765 / :2779)
# ===========================================================================
LLM_CONFIG_MSG = "There was a problem with the LLM configuration."
SEARCH_CONFIG_MSG = "There was a problem with the search engine configuration."
GENERIC_MSG = (
"Research failed due to an unexpected error. Contact your "
"administrator or check the server logs for details."
)
# Server-side detail that must never reach the caller. Each string is
# planted in the raw exception the worker's classifier sees.
LEAK_LLM = "/srv/ldr/models/tenant-private-llama.gguf"
LEAK_SEARCH = "https://searx.internal.corp:8888/?api_key=sk-live-DEADBEEFCAFE"
LEAK_GENERIC = (
'File "/opt/ldr/src/secret_module.py", line 42: '
"postgresql://ldr:[email protected]:5432/ldr_prod"
)
def _run_worker_into_error(*, llm_error=None, search_error=None, username):
"""Drive the REAL worker into its outer error handler.
The exception is raised from the real LLM / search-engine construction
branches, so the *classifier* under test (the ``LLM Configuration
Error:`` / ``Search Engine Configuration Error`` wrapping at
research_service.py:1487/:1541) runs for real too — this is not a
hand-built message handed straight to the handler.
Returns ``(queue_processor_mock, error_report_generator_mock,
sio_emit_mock)``.
"""
queue_processor = MagicMock()
report_generator = MagicMock(
return_value=MagicMock(
generate_error_report=MagicMock(return_value="error report")
)
)
storage = MagicMock()
storage.save_report.return_value = True
extra = {
f"{MODULE}.ErrorReportGenerator": report_generator,
f"{QUEUE_MODULE}.queue_processor": queue_processor,
"local_deep_research.storage.get_report_storage": MagicMock(
return_value=storage
),
f"{MODULE}.AdvancedSearchSystem": MagicMock(),
f"{MODULE}.get_llm": MagicMock(
side_effect=RuntimeError(llm_error) if llm_error else None
),
f"{MODULE}.get_search": MagicMock(
side_effect=RuntimeError(search_error) if search_error else None
),
}
patches = _worker_patches(extra=extra)
with _worker(patches) as mocks:
run_research_process(
research_id=str(uuid.uuid4()),
query="cwe209 probe",
username=username,
# ``model`` is what takes the get_llm override branch; the
# search branch is taken by ``search_engine`` (always set).
model="tenant-model",
settings_snapshot=_snapshot(),
**BASE_RUN_KWARGS,
)
return queue_processor, report_generator, mocks[f"{MODULE}._sio_emit"]
def _client_visible_text(queue_processor, sio_emit):
"""Everything the worker hands to a client-visible sink, as one string.
``queue_error_update`` writes ``error_message`` + ``metadata`` onto the
research row (rendered in the history UI) and ``_sio_emit`` pushes the
message to the browser. Both are asserted together so a leak surviving
on either sink fails.
"""
parts = []
for call in queue_processor.queue_error_update.call_args_list:
parts.append(repr(call.kwargs))
for call in sio_emit.call_args_list:
parts.append(repr(call.args) + repr(call.kwargs))
return "\n".join(parts)
class TestErrorMessageSanitisation:
"""CWE-209. Three genericised strings, all three unasserted on branch.
Verified before writing: ``grep -rn`` over ``tests/`` for each literal
below returned zero hits, so this file is their only coverage.
"""
def test_llm_configuration_error_is_genericised(self):
qp, report_gen, sio = _run_worker_into_error(
llm_error=f"failed to load {LEAK_LLM}", username="alice_err"
)
# POSITIVE CONTROL: the handler ran and produced an update. Without
# this, "the path does not appear" would pass for a worker that
# crashed before reaching the classifier.
assert qp.queue_error_update.call_count == 1, (
"the outer error handler never queued an update"
)
kwargs = qp.queue_error_update.call_args.kwargs
assert kwargs["status"] == ResearchStatus.FAILED
assert kwargs["error_message"] == LLM_CONFIG_MSG
assert kwargs["metadata"]["error"] == LLM_CONFIG_MSG
assert kwargs["metadata"]["solution"].startswith(
"Review your LLM model settings"
)
visible = _client_visible_text(qp, sio)
assert LEAK_LLM not in visible, visible
assert ".gguf" not in visible
assert "/srv/" not in visible
# The persisted error report is retrievable through the report
# routes, so the sanitised message — not the raw exception — must be
# what the generator embeds.
report_kwargs = (
report_gen.return_value.generate_error_report.call_args.kwargs
)
assert report_kwargs["error_message"] == (
f"Research failed: {LLM_CONFIG_MSG}"
)
assert LEAK_LLM not in repr(report_kwargs)
def test_search_engine_configuration_error_is_genericised(self):
qp, report_gen, sio = _run_worker_into_error(
search_error=f"searxng unreachable at {LEAK_SEARCH}",
username="alice_err",
)
assert qp.queue_error_update.call_count == 1
kwargs = qp.queue_error_update.call_args.kwargs
assert kwargs["error_message"] == SEARCH_CONFIG_MSG
assert kwargs["metadata"]["error"] == SEARCH_CONFIG_MSG
assert kwargs["metadata"]["solution"].startswith(
"Review your search engine settings"
)
visible = _client_visible_text(qp, sio)
assert LEAK_SEARCH not in visible, visible
assert "sk-live-DEADBEEFCAFE" not in visible
assert "searx.internal.corp" not in visible
report_kwargs = (
report_gen.return_value.generate_error_report.call_args.kwargs
)
assert report_kwargs["error_message"] == (
f"Research failed: {SEARCH_CONFIG_MSG}"
)
assert LEAK_SEARCH not in repr(report_kwargs)
def test_unclassified_error_is_replaced_wholesale(self):
"""The ``else`` branch: anything the classifier does not recognise
must be replaced, not echoed. This is the branch that would leak a
traceback, a filesystem path or a connection string."""
qp, report_gen, sio = _run_worker_into_error(
llm_error=LEAK_GENERIC, username="alice_err"
)
assert qp.queue_error_update.call_count == 1
kwargs = qp.queue_error_update.call_args.kwargs
assert kwargs["error_message"] == GENERIC_MSG
assert kwargs["metadata"]["error"] == GENERIC_MSG
# The unclassified branch carries no ``solution`` hint — a leak
# would most plausibly reappear there.
assert "solution" not in kwargs["metadata"]
visible = _client_visible_text(qp, sio)
assert LEAK_GENERIC not in visible, visible
assert "hunter2" not in visible
assert "postgresql://" not in visible
assert "secret_module.py" not in visible
report_kwargs = (
report_gen.return_value.generate_error_report.call_args.kwargs
)
assert report_kwargs["error_message"] == (
f"Research failed: {GENERIC_MSG}"
)
assert "hunter2" not in repr(report_kwargs)
def test_the_three_messages_are_distinguishable(self):
"""Anti-collapse control.
A regression that replaced every branch with one string would still
satisfy each "no leak" assertion above. The three classifications
must remain distinct, so a user can tell an LLM misconfiguration
from a search-engine one.
"""
messages = set()
for kwargs in (
{"llm_error": f"failed to load {LEAK_LLM}"},
{"search_error": f"searxng unreachable at {LEAK_SEARCH}"},
{"llm_error": LEAK_GENERIC},
):
qp, _, _ = _run_worker_into_error(username="alice_err", **kwargs)
messages.add(
qp.queue_error_update.call_args.kwargs["error_message"]
)
assert messages == {LLM_CONFIG_MSG, SEARCH_CONFIG_MSG, GENERIC_MSG}