Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

43.6 KB
Raw Blame
"""Security coverage for ``web/services/research_service.py`` (FastAPI port).

WHY THIS FILE EXISTS
--------------------
The historical ADR-0010 review found that predecessor tests for
``research_service.py`` had been removed and that some successors did not call
the service directly. This file supplies behavior-level regression evidence by
calling the production functions. Nothing below re-implements service logic;
every assertion is on a value produced by ``research_service`` itself.

WHAT IS COVERED HERE
--------------------
1. ``run_research_process`` username gate (research_service.py:935). Both
   predecessor tests that executed it were deleted; this file restores direct
   coverage. Without the gate a worker runs with ``username=None`` and every
   encrypted-DB write in the run is silently dropped or misattributed.

2. The ``username`` KEYWORD-ONLY barrier on ``save_research_strategy`` /
   ``get_research_strategy`` (research_service.py:345 / :393). Making it
   positional-or-optional reopens the ``get_user_db_session(None)`` fallback
   from #4526 --- work silently attributed to no user. At the review snapshot,
   ``save_*`` had no direct behavioral test (``grep`` found only call sites
   that patched it out plus a hand-rolled ORM insert in
   ``tests/database/test_research_strategy_fk_regression.py``); this file now
   pins it.
   PARTIAL OVERLAP, declared: ``get_research_strategy``'s per-user DB
   selection *is* genuinely covered by
   ``tests/security/test_metrics_hostile_input_fastapi.py::
   test_get_research_strategy_selects_the_db_by_username``. What is NOT
   covered there --- and is covered here --- is the write side and the
   keyword-only signature itself (that file pins the kwarg at one *route*
   call site via AST, which a signature change would not trip).

3. ``cancel_research`` cross-user isolation (research_service.py:3194). All
   8 predecessor tests of ``TestCancelResearch`` were deleted; at the review
   snapshot the one surviving caller
   (``tests/web/queue/test_queued_research_lifecycle_races.py``) patched the
   database session out entirely. The cases below now prove that ``username``
   selects the database that gets written.

4. Per-user LLM-provider scoping of news headline/topic generation
   (research_service.py:2195/:2227/:2238) --- the
   ``settings_snapshot["_username"]`` threading via the real
   ``ensure_snapshot_username``. Without it the LLM policy-enforcement
   point cannot resolve a per-user provider.

5. ``_generate_report_path`` (research_service.py:518) had no direct coverage
   at the review snapshot. The cases below pin the module's path-containment
   guard: the md5 filename stops ``../`` or a NUL byte in a user-controlled
   query from reaching the filesystem.

6. CWE-209 error sanitisation (research_service.py:2757/:2765/:2779), 10
   deleted predecessor tests. At the review snapshot, ``grep -rn`` for each
   genericised literal returned no hits. The cases below now assert all three.

VACUITY
-------
Every negative assertion is paired with a positive control asserted FIRST.
The isolation tests seed the SAME ``research_id`` into BOTH users' encrypted
databases with different content, so "user B saw nothing" cannot pass
against an empty table, and the caller's own row is always asserted before
the other user's is asserted unchanged. The worker-driven tests assert that
the code under test was actually reached (the generator was called, the
error update was queued) before asserting what it must not contain.
"""

from __future__ import annotations

import inspect
import itertools
import os
import re
import uuid
from contextlib import contextmanager
from unittest.mock import MagicMock, Mock, create_autospec, patch

import pytest
from loguru import logger

# Imported so this file fails loudly if a symbol is renamed rather than
# silently asserting against a stub.
from local_deep_research.constants import ResearchStatus
from local_deep_research.database.models import (
    ResearchHistory,
    ResearchStrategy,
)
from local_deep_research.database.session_context import get_user_db_session
from local_deep_research.web import research_state
from local_deep_research.web.services import research_service
from local_deep_research.web.services.research_service import (
    _generate_report_path,
    cancel_research,
    get_research_strategy,
    run_research_process,
    save_research_strategy,
)

# The app's docs_url toggle reads PYTEST_CURRENT_TEST at app-import time;
# pytest only sets that per-test, not at collection.
os.environ.setdefault("TESTING", "1")

MODULE = "local_deep_research.web.services.research_service"
STATE_MODULE = "local_deep_research.web.research_state"
QUEUE_MODULE = "local_deep_research.web.queue.processor_v2"

PASSWORD = "ResearchIso!Pass123"  # noqa: S105 — test-only credential

# The production progress callback emits at the custom "MILESTONE" level
# registered by log_utils.init_loguru, which tests do not run.
try:
    logger.level("MILESTONE", no=26)
except (ValueError, TypeError):
    pass


# ---------------------------------------------------------------------------
# Two-real-user harness (real registration, real encrypted per-user DBs)
# ---------------------------------------------------------------------------

# MONOTONIC, not random: rate limiting is keyed per client IP and random
# octets collide, producing 429s from /auth/register's per-IP bucket
# that have nothing to do with the guard under test.
_IP_COUNTER = itertools.count(1)


def _next_forwarded_for() -> str:
    n = next(_IP_COUNTER)
    return f"10.212.{(n // 250) % 250}.{(n % 250) + 1}"


def _fresh_client(app):
    from fastapi.testclient import TestClient

    client = TestClient(app, raise_server_exceptions=False)
    client.headers.update({"X-Forwarded-For": _next_forwarded_for()})
    return client


def _csrf(client) -> str:
    """CSRF is enforced by ASGI middleware — fetch a real token."""
    client.get("/auth/login")
    return client.get("/auth/csrf-token").json()["csrf_token"]


def _register_and_login(app, username: str):
    client = _fresh_client(app)

    resp = client.post(
        "/auth/register",
        data={
            "username": username,
            "password": PASSWORD,
            "confirm_password": PASSWORD,
            "acknowledge": "true",
            "csrf_token": _csrf(client),
        },
        follow_redirects=False,
    )
    assert resp.status_code in (200, 302), (
        f"registration of {username!r} failed: "
        f"{resp.status_code} / {resp.text[:400]}"
    )

    resp = client.post(
        "/auth/login",
        data={
            "username": username,
            "password": PASSWORD,
            "csrf_token": _csrf(client),
        },
        follow_redirects=False,
    )
    assert resp.status_code in (200, 302), (
        f"login of {username!r} failed: {resp.status_code} / {resp.text[:400]}"
    )
    return client


def _user(prefix: str) -> str:
    return f"{prefix}_{uuid.uuid4().hex[:10]}"


def _seed_research(username, research_id, query, status):
    """Insert one ResearchHistory row into *username*'s encrypted DB."""
    with get_user_db_session(username, PASSWORD) as session:
        session.add(
            ResearchHistory(
                id=research_id,
                query=query,
                mode="quick_summary",
                status=status,
                created_at="2024-01-01T00:00:00",
            )
        )
        session.commit()


def _status_of(username, research_id):
    with get_user_db_session(username, PASSWORD) as session:
        row = session.query(ResearchHistory).filter_by(id=research_id).first()
        return None if row is None else row.status


def _strategy_rows(username, research_id):
    with get_user_db_session(username, PASSWORD) as session:
        return [
            row.strategy_name
            for row in session.query(ResearchStrategy).filter_by(
                research_id=research_id
            )
        ]


@pytest.fixture
def two_users(app):
    """Two registered, logged-in users with live encrypted databases.

    The clients are held for the lifetime of the test: logging out (or
    dropping them) would clear the session password the service's
    ``get_user_db_session(username)`` call resolves through, and every
    service call below deliberately runs WITHOUT an explicit password so
    that the username really is what selects the database.

    ``session_password_store`` is a process-wide singleton that
    ``reset_all_singletons`` does not touch, so the entries are dropped on
    teardown — a leaked one would be visible to every later test in the
    same worker.
    """
    from local_deep_research.database.session_passwords import (
        session_password_store,
    )

    alice, bob = _user("rsiso_alice"), _user("rsiso_bob")
    clients = (
        _register_and_login(app, alice),
        _register_and_login(app, bob),
    )
    try:
        yield {"alice": alice, "bob": bob, "_clients": clients}
    finally:
        for username in (alice, bob):
            session_password_store.clear_all_for_user(username)


# ---------------------------------------------------------------------------
# Real-worker harness
# ---------------------------------------------------------------------------


@contextmanager
def _mock_db_session(session=None):
    if session is None:
        session = MagicMock()
    yield session


def _worker_patches(db_session=None, extra=None):
    """Patches that keep the REAL ``run_research_process`` body running.

    Nothing between the entry point and the code under test is replaced:
    the username gate, the egress context build, the LLM/search override
    branches, the error classifier and the news-snapshot threading all
    execute for real. Only the process boundaries (sockets, queue
    processor, encrypted DB, thread-local cleanup) are stubbed.
    """
    session = MagicMock() if db_session is None else db_session
    patches = {
        f"{MODULE}.get_user_db_session": lambda *a, **kw: _mock_db_session(
            session
        ),
        f"{MODULE}.cleanup_research_resources": MagicMock(),
        f"{MODULE}.handle_termination": MagicMock(),
        f"{MODULE}.set_search_context": MagicMock(),
        f"{MODULE}._sio_emit": create_autospec(
            research_service._sio_emit, spec_set=True
        ),
        f"{MODULE}._sio_remove": create_autospec(
            research_service._sio_remove, spec_set=True
        ),
        f"{MODULE}._socket_emitter": create_autospec(
            research_service._socket_emitter, spec_set=True
        ),
        f"{STATE_MODULE}.is_termination_requested": MagicMock(
            return_value=False
        ),
        f"{STATE_MODULE}.is_research_active": MagicMock(return_value=False),
        f"{STATE_MODULE}.update_progress_and_check_active": MagicMock(
            return_value=(5, True)
        ),
        "local_deep_research.web.routes.globals.is_termination_requested": (
            MagicMock(return_value=False)
        ),
        "local_deep_research.settings.logger.log_settings": MagicMock(),
        "local_deep_research.config.thread_settings.set_settings_context": (
            MagicMock()
        ),
        f"{QUEUE_MODULE}.queue_processor": MagicMock(),
    }
    if extra:
        patches.update(extra)
    return patches


@contextmanager
def _worker(patches):
    """Apply a {target: replacement} mapping and yield the mocks by target."""
    started = {}
    ctxs = []
    try:
        for target, replacement in patches.items():
            ctx = patch(target, replacement)
            ctxs.append(ctx)
            started[target] = ctx.__enter__()
        yield started
    finally:
        for ctx in reversed(ctxs):
            ctx.__exit__(None, None, None)


# A run always carries a matching ``search_engine`` kwarg and snapshot entry:
# ``build_run_egress_context`` resolves the primary from the kwarg, and a
# mismatch makes the worker refuse the run (fail-closed) before any of the
# code under test is reached.
BASE_RUN_KWARGS = {
    "mode": "quick",
    "search_engine": "searxng",
}


def _snapshot():
    return {"search.tool": "searxng"}


# ===========================================================================
# 1. run_research_process username gate  (research_service.py:935)
# ===========================================================================


class TestRunResearchProcessUsernameGate:
    """``raise ValueError("Username is required for research process")``.

    The worker writes token metrics, research logs and the report into the
    *caller's* encrypted database. With no username those writes have no
    database to go to; the guard turns that into a loud failure at the top
    of the thread instead of a run whose every persistence call is swallowed
    by a bare ``except``.
    """

    def test_missing_username_raises_before_any_work(self):
        """No ``username`` kwarg at all -> ValueError, nothing started."""
        patches = _worker_patches()
        with _worker(patches) as mocks:
            with pytest.raises(
                ValueError, match="Username is required for research process"
            ):
                run_research_process(
                    research_id=str(uuid.uuid4()),
                    query="gate probe",
                    settings_snapshot=_snapshot(),
                    **BASE_RUN_KWARGS,
                )
            # The guard must fire BEFORE the thread context is established;
            # a set_search_context({"username": None, ...}) would poison
            # every log line and DB lookup made from this thread.
            assert not mocks[f"{MODULE}.set_search_context"].called
            assert not mocks[f"{MODULE}.cleanup_research_resources"].called

    @pytest.mark.parametrize("username", ["", None])
    def test_falsy_username_raises(self, username):
        """Empty string and explicit ``None`` are refused, not accepted."""
        patches = _worker_patches()
        with _worker(patches):
            with pytest.raises(
                ValueError, match="Username is required for research process"
            ):
                run_research_process(
                    research_id=str(uuid.uuid4()),
                    query="gate probe",
                    username=username,
                    settings_snapshot=_snapshot(),
                    **BASE_RUN_KWARGS,
                )

    def test_valid_username_passes_the_gate_and_is_threaded_downstream(self):
        """POSITIVE CONTROL.

        Without this every assertion above would also hold for a worker that
        raised ``ValueError`` unconditionally. Termination is requested up
        front so the run takes the shortest real path past the gate; what is
        asserted is that the gate let it through AND that the username it
        accepted is the one handed to the thread context and to cleanup.
        """
        research_id = str(uuid.uuid4())
        patches = _worker_patches(
            extra={
                f"{STATE_MODULE}.is_termination_requested": MagicMock(
                    return_value=True
                ),
            }
        )
        with _worker(patches) as mocks:
            run_research_process(
                research_id=research_id,
                query="gate probe",
                username="alice_gate",
                settings_snapshot=_snapshot(),
                **BASE_RUN_KWARGS,
            )

        ctx = mocks[f"{MODULE}.set_search_context"]
        assert ctx.called, "the gate refused a valid username"
        assert ctx.call_args.args[0]["username"] == "alice_gate"
        assert ctx.call_args.args[0]["research_id"] == research_id

        cleanup = mocks[f"{MODULE}.cleanup_research_resources"]
        assert cleanup.called
        assert cleanup.call_args.args[1] == "alice_gate", (
            "cleanup must be scoped to the run owner"
        )


# ===========================================================================
# 2. username keyword-only barrier on the strategy accessors
#    (research_service.py:345 / :393)
# ===========================================================================


class TestStrategyUsernameBarrier:
    """``save_research_strategy(rid, name, *, username)`` and its reader.

    The ``*`` is the guard. Demoting ``username`` to a positional-or-default
    parameter is exactly the shape of the #4526 regression: call sites that
    forget it fall back to ``get_user_db_session(None)``, and the strategy
    is written to (or read from) whatever database the ambient session
    happens to resolve.
    """

    @pytest.mark.parametrize(
        "func", [save_research_strategy, get_research_strategy]
    )
    def test_username_is_keyword_only_and_required(self, func):
        param = inspect.signature(func).parameters["username"]
        assert param.kind is inspect.Parameter.KEYWORD_ONLY, (
            f"{func.__name__}: username must stay keyword-only so no caller "
            "can drift into the get_user_db_session(None) fallback (#4526)"
        )
        assert param.default is inspect.Parameter.empty, (
            f"{func.__name__}: username must have no default; a default of "
            "None reopens the implicit-session fallback"
        )

    def test_positional_username_is_rejected(self):
        """The barrier at runtime, not only in the signature object."""
        with pytest.raises(TypeError):
            save_research_strategy("rid", "source-based", "alice")
        with pytest.raises(TypeError):
            get_research_strategy("rid", "alice")

    def test_save_and_get_use_the_callers_own_database(self, two_users):
        """Round trip through the real functions, two users, ONE id.

        Both users hold the same ``research_id``. A save that reached the
        wrong database would overwrite the other user's strategy; a read
        that reached the wrong database would return it. Either way the
        assertions below flip.
        """
        alice, bob = two_users["alice"], two_users["bob"]
        research_id = str(uuid.uuid4())

        # ResearchStrategy.research_id is a FK onto research_history.id with
        # PRAGMA foreign_keys on, so the parent row must exist in each DB.
        _seed_research(
            alice, research_id, "alice query", ResearchStatus.COMPLETED
        )
        _seed_research(bob, research_id, "bob query", ResearchStatus.COMPLETED)

        # POSITIVE CONTROL first: the write is really performed.
        save_research_strategy(research_id, "alice-strategy", username=alice)
        assert _strategy_rows(alice, research_id) == ["alice-strategy"]
        assert get_research_strategy(research_id, username=alice) == (
            "alice-strategy"
        )

        # Alice's write must not have landed in Bob's database.
        assert _strategy_rows(bob, research_id) == []
        assert get_research_strategy(research_id, username=bob) is None

        # Bob writes his own value for the same id; alice's must survive.
        save_research_strategy(research_id, "bob-strategy", username=bob)
        assert get_research_strategy(research_id, username=bob) == (
            "bob-strategy"
        )
        assert get_research_strategy(research_id, username=alice) == (
            "alice-strategy"
        )

        # The update branch (existing row) must also stay in-database.
        save_research_strategy(research_id, "alice-strategy-2", username=alice)
        assert _strategy_rows(alice, research_id) == ["alice-strategy-2"]
        assert _strategy_rows(bob, research_id) == ["bob-strategy"]


# ===========================================================================
# 3. cancel_research cross-user isolation  (research_service.py:3194)
# ===========================================================================


@pytest.fixture(autouse=True)
def _clear_termination_flags():
    """``cancel_research`` sets a PROCESS-GLOBAL termination flag.

    ``research_state._termination_flags`` is module state that outlives the
    test; leaving entries behind would make a later worker test take the
    "terminated before starting" branch. Recorded ids are cleared after each
    test in this module.
    """
    before = set(research_state._termination_flags)
    yield
    for rid in set(research_state._termination_flags) - before:
        research_state.clear_termination_flag(rid)


class TestCancelResearchIsolation:
    """``cancel_research(research_id, username)``.

    The ORM query carries no owner predicate — the ``username`` argument is
    the ONLY thing that decides which encrypted database is opened and
    written. These tests seed the same ``research_id`` into both users'
    databases so a cancel that reached the wrong one is visible as a status
    change on the wrong row, not merely as an absent row.
    """

    def test_owner_can_cancel_their_own_research(self, two_users):
        """POSITIVE CONTROL. Without it, "B cannot cancel A's research"
        would pass against a function that cancelled nothing at all."""
        alice = two_users["alice"]
        research_id = str(uuid.uuid4())
        _seed_research(
            alice, research_id, "alice query", ResearchStatus.IN_PROGRESS
        )

        assert cancel_research(research_id, alice) is True
        assert _status_of(alice, research_id) == ResearchStatus.SUSPENDED

    def test_other_user_cannot_cancel_a_research_they_do_not_have(
        self, two_users
    ):
        """Bob has no such row: refusal, and Alice's row is untouched."""
        alice, bob = two_users["alice"], two_users["bob"]
        research_id = str(uuid.uuid4())
        _seed_research(
            alice, research_id, "alice query", ResearchStatus.IN_PROGRESS
        )
        # Bob's database is NOT empty — it holds a different in-progress
        # research — so "nothing to cancel" cannot be an artefact of an
        # empty table.
        bob_own = str(uuid.uuid4())
        _seed_research(bob, bob_own, "bob query", ResearchStatus.IN_PROGRESS)

        # POSITIVE CONTROL: bob's own cancel works, so the refusal below is
        # about ownership and not about bob's database being unreachable.
        assert cancel_research(bob_own, bob) is True
        assert _status_of(bob, bob_own) == ResearchStatus.SUSPENDED

        assert cancel_research(research_id, bob) is False, (
            "bob must not be able to cancel a research he does not own"
        )
        assert _status_of(alice, research_id) == ResearchStatus.IN_PROGRESS

        # ...and alice is still able to cancel it herself afterwards, so the
        # refusal above did not come from the row/database being unusable.
        assert cancel_research(research_id, alice) is True
        assert _status_of(alice, research_id) == ResearchStatus.SUSPENDED

    def test_colliding_research_id_cancels_only_the_callers_row(
        self, two_users
    ):
        """The sharpest form: both users hold the SAME id, in progress.

        Bob cancels. His own row must flip (positive control, in the same
        assertion block) and Alice's must not (isolation).
        """
        alice, bob = two_users["alice"], two_users["bob"]
        research_id = str(uuid.uuid4())
        _seed_research(
            alice, research_id, "alice query", ResearchStatus.IN_PROGRESS
        )
        _seed_research(
            bob, research_id, "bob query", ResearchStatus.IN_PROGRESS
        )

        assert cancel_research(research_id, bob) is True
        assert _status_of(bob, research_id) == ResearchStatus.SUSPENDED
        assert _status_of(alice, research_id) == ResearchStatus.IN_PROGRESS, (
            "cancelling bob's research must not suspend alice's research "
            "with the same id"
        )

        # And the reverse direction, so the test cannot pass by always
        # writing to whichever database was opened first.
        assert cancel_research(research_id, alice) is True
        assert _status_of(alice, research_id) == ResearchStatus.SUSPENDED

    def test_terminal_state_is_reported_from_the_callers_own_row(
        self, two_users
    ):
        """``True`` for an already-terminal research is decided by the
        caller's own row, not by any other user's copy of the id."""
        alice, bob = two_users["alice"], two_users["bob"]
        research_id = str(uuid.uuid4())
        _seed_research(
            alice, research_id, "alice query", ResearchStatus.COMPLETED
        )
        _seed_research(
            bob, research_id, "bob query", ResearchStatus.IN_PROGRESS
        )

        # Alice's row is terminal -> True, and nothing is rewritten.
        assert cancel_research(research_id, alice) is True
        assert _status_of(alice, research_id) == ResearchStatus.COMPLETED
        # Bob's live row was not dragged into a terminal state by it.
        assert _status_of(bob, research_id) == ResearchStatus.IN_PROGRESS

        # Bob's own call must still act on HIS row (the live one), not on
        # alice's terminal copy of the same id.
        assert cancel_research(research_id, bob) is True
        assert _status_of(bob, research_id) == ResearchStatus.SUSPENDED
        assert _status_of(alice, research_id) == ResearchStatus.COMPLETED


# ===========================================================================
# 4. Per-user LLM-provider scoping of news headline/topic generation
#    (research_service.py:2195 / :2227 / :2238)
# ===========================================================================

HEADLINE_TARGET = (
    "local_deep_research.news.utils.headline_generator.generate_headline"
)
TOPIC_TARGET = "local_deep_research.news.utils.topic_generator.generate_topics"


def _news_research_row():
    """A completed news research whose metadata triggers the news branch."""
    row = MagicMock()
    row.status = ResearchStatus.IN_PROGRESS
    row.created_at = "2024-01-01T00:00:00"
    row.report_content = "# Report body"
    row.research_meta = {"is_news_search": True, "category": "Markets"}
    return row


def _run_quick_news_worker(username, snapshot):
    """Drive the REAL worker to quick-mode completion on a news search.

    Returns ``(generate_headline_mock, generate_topics_mock, research_row)``.
    ``ensure_snapshot_username`` is deliberately NOT patched — the property
    under test is that the real one runs and that its output is what reaches
    the generators.
    """
    session = MagicMock()
    research_row = _news_research_row()
    session.query.return_value.filter_by.return_value.first.return_value = (
        research_row
    )

    system = Mock()
    system.all_links_of_system = []
    system.analyze_topic.return_value = {
        "findings": [
            {"phase": "Final synthesis", "content": "Synthesized answer."}
        ],
        "formatted_findings": "# Summary\n\nSynthesized answer.",
        "iterations": 1,
        "current_knowledge": "Synthesized answer.",
    }

    headline = MagicMock(return_value="A Generated Headline")
    topics = MagicMock(return_value=["alpha", "beta"])
    storage = MagicMock()
    storage.save_report.return_value = True

    patches = _worker_patches(
        db_session=session,
        extra={
            f"{MODULE}.AdvancedSearchSystem": MagicMock(return_value=system),
            f"{MODULE}.get_llm": MagicMock(),
            f"{MODULE}.get_search": MagicMock(),
            "local_deep_research.storage.get_report_storage": MagicMock(
                return_value=storage
            ),
            HEADLINE_TARGET: headline,
            TOPIC_TARGET: topics,
        },
    )
    with _worker(patches):
        run_research_process(
            research_id=str(uuid.uuid4()),
            query="market news today",
            username=username,
            settings_snapshot=snapshot,
            **BASE_RUN_KWARGS,
        )
    return headline, topics, research_row


class TestNewsSnapshotOwnerScoping:
    """``headline_topic_snapshot = ensure_snapshot_username(snapshot, user)``.

    Headline/topic generation resolves an LLM through the snapshot. The
    per-user policy-enforcement point looks the owner up under
    ``settings_snapshot["_username"]``; without it the run silently resolves
    the shared/built-in namespace instead of the user's own provider (and,
    for a user whose policy is local-only, cannot enforce it).
    """

    def test_generators_receive_a_snapshot_scoped_to_the_run_owner(self):
        snapshot = {"search.tool": "searxng", "llm.provider": "ollama"}
        headline, topics, research_row = _run_quick_news_worker(
            "alice_news", snapshot
        )

        # POSITIVE CONTROL: the branch was actually reached. Every
        # assertion below is vacuous if the generators were never called.
        assert headline.call_count == 1, (
            "the news headline branch was never reached"
        )
        assert topics.call_count == 1

        h_snapshot = headline.call_args.kwargs["settings_snapshot"]
        t_snapshot = topics.call_args.kwargs["settings_snapshot"]
        assert h_snapshot["_username"] == "alice_news"
        assert t_snapshot["_username"] == "alice_news"
        # One snapshot object is built once and shared by both call sites.
        assert h_snapshot is t_snapshot
        # The run's own settings must still be carried, i.e. the snapshot is
        # the run's snapshot with the owner added — not a bare {"_username"}.
        assert h_snapshot["llm.provider"] == "ollama"
        # ...and the caller's dict is not mutated (no cross-run bleed).
        assert "_username" not in snapshot

        # The generated values really came back through the worker.
        assert research_row.research_meta["generated_headline"] == (
            "A Generated Headline"
        )
        assert research_row.research_meta["generated_topics"] == [
            "alpha",
            "beta",
        ]

    def test_the_username_tracks_the_run_owner(self):
        """Anti-constant control: a second run under a different owner must
        carry that owner, so the assertion above cannot be satisfied by a
        hard-coded or leaked value."""
        _, topics_a, _ = _run_quick_news_worker(
            "alice_news", {"search.tool": "searxng"}
        )
        _, topics_b, _ = _run_quick_news_worker(
            "bob_news", {"search.tool": "searxng"}
        )
        assert (
            topics_a.call_args.kwargs["settings_snapshot"]["_username"]
            == "alice_news"
        )
        assert (
            topics_b.call_args.kwargs["settings_snapshot"]["_username"]
            == "bob_news"
        )

    def test_an_explicit_snapshot_owner_is_never_overwritten(self):
        """A snapshot that already names its owner (e.g. a scheduler-built
        one) must be passed through unchanged rather than re-stamped."""
        snapshot = {"search.tool": "searxng", "_username": "explicit_owner"}
        headline, _, _ = _run_quick_news_worker("alice_news", snapshot)
        assert headline.call_count == 1
        assert headline.call_args.kwargs["settings_snapshot"]["_username"] == (
            "explicit_owner"
        )


# ===========================================================================
# 5. _generate_report_path path containment  (research_service.py:518)
# ===========================================================================

# The only characters the generated basename may contain. The trailing
# 8-hex random segment keeps same-query, same-second calls from colliding
# in the install-shared directory (preventive; the helper has no caller).
_REPORT_NAME_RE = re.compile(
    r"^research_report_[0-9a-f]{10}_\d+_[0-9a-f]{8}\.md$"
)

HOSTILE_QUERIES = [
    "../../../etc/passwd",
    "..\\..\\..\\windows\\system32\\config\\sam",
    "/etc/shadow",
    "....//....//etc/passwd",
    "%2e%2e%2f%2e%2e%2fetc%2fpasswd",
    "nul\x00byte",
    "trailing\x00/../../root/.ssh/authorized_keys",
    "line\nbreak\r\ninjection",
    "semi;colon && rm -rf /",
    "~/.ssh/id_rsa",
    "\u202eevil",
    pytest.param("a" * 5000, id="long-ascii-query-5k"),
    "",
]


class TestGenerateReportPathContainment:
    """``_generate_report_path(query)`` — the module's only path guard.

    The query is fully user-controlled. The function's containment property
    is that the basename is derived ONLY from an md5 hex digest, a unix
    timestamp, and a random uuid segment, so no byte of the query can
    influence the path. These tests
    re-anchor ``OUTPUT_DIR`` to a temp directory so the assertion proves the
    result is anchored to that constant rather than merely being "some path
    that happens to exist".
    """

    def test_benign_query_lands_in_output_dir(self, tmp_path, monkeypatch):
        """POSITIVE CONTROL: the function does produce a usable path under
        OUTPUT_DIR, so the containment assertions below are not passing
        because the function returns something inert."""
        monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)
        path = _generate_report_path("a perfectly ordinary question")

        assert path.parent == tmp_path
        assert _REPORT_NAME_RE.match(path.name), path.name
        assert path.suffix == ".md"
        # The path is writable as-is — proving it is a real filename and not
        # a directory traversal that would fail at open() time.
        path.write_text("ok")
        assert path.read_text() == "ok"
        assert [p.name for p in tmp_path.iterdir()] == [path.name]

    @pytest.mark.parametrize("query", HOSTILE_QUERIES)
    def test_hostile_query_cannot_escape_output_dir(
        self, query, tmp_path, monkeypatch
    ):
        monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)
        path = _generate_report_path(query)

        assert path.parent == tmp_path, (
            f"{query!r} moved the report out of OUTPUT_DIR: {path}"
        )
        assert path.resolve().parent == tmp_path.resolve(), (
            f"{query!r} escaped OUTPUT_DIR after resolution: {path}"
        )
        assert _REPORT_NAME_RE.match(path.name), (
            f"{query!r} reached the filename: {path.name!r}"
        )
        # Neither a NUL byte nor a separator survives into the name.
        assert "\x00" not in str(path)
        assert "/" not in path.name and "\\" not in path.name
        assert ".." not in path.name

    @pytest.mark.parametrize("query", HOSTILE_QUERIES)
    def test_no_fragment_of_the_query_reaches_the_filename(
        self, query, tmp_path, monkeypatch
    ):
        """The md5 property: the basename carries no user text at all."""
        monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)
        name = _generate_report_path(query).name
        for token in re.findall(r"[A-Za-z0-9_.-]{4,}", query):
            assert token not in name, (
                f"query fragment {token!r} leaked into {name!r}"
            )

    def test_hash_segment_is_a_function_of_the_query(
        self, tmp_path, monkeypatch
    ):
        """Distinct queries must not collapse onto one filename (which would
        make one user's report overwrite another's), and the same query must
        hash stably (proving the segment is the digest, not a random blob)."""
        monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)

        def _hash_of(query):
            return _generate_report_path(query).name.split("_")[2]

        assert _hash_of("query one") == _hash_of("query one")
        assert _hash_of("query one") != _hash_of("query two")
        assert _hash_of("../../etc/passwd") != _hash_of("../../etc/shadow")

    def test_non_ascii_query_is_encoded_not_raised(self, tmp_path, monkeypatch):
        """``query.encode("utf-8")`` — a non-ASCII query must hash, not
        raise, and must not reach the name."""
        monkeypatch.setattr(research_service, "OUTPUT_DIR", tmp_path)
        path = _generate_report_path("Grüße 你好 🙂 ../..")
        assert _REPORT_NAME_RE.match(path.name), path.name
        assert path.parent == tmp_path


# ===========================================================================
# 6. CWE-209 error sanitisation
#    (research_service.py:2757 / :2765 / :2779)
# ===========================================================================

LLM_CONFIG_MSG = "There was a problem with the LLM configuration."
SEARCH_CONFIG_MSG = "There was a problem with the search engine configuration."
GENERIC_MSG = (
    "Research failed due to an unexpected error. Contact your "
    "administrator or check the server logs for details."
)

# Server-side detail that must never reach the caller. Each string is
# planted in the raw exception the worker's classifier sees.
LEAK_LLM = "/srv/ldr/models/tenant-private-llama.gguf"
LEAK_SEARCH = "https://searx.internal.corp:8888/?api_key=sk-live-DEADBEEFCAFE"
LEAK_GENERIC = (
    'File "/opt/ldr/src/secret_module.py", line 42: '
    "postgresql://ldr:[email protected]:5432/ldr_prod"
)


def _run_worker_into_error(*, llm_error=None, search_error=None, username):
    """Drive the REAL worker into its outer error handler.

    The exception is raised from the real LLM / search-engine construction
    branches, so the *classifier* under test (the ``LLM Configuration
    Error:`` / ``Search Engine Configuration Error`` wrapping at
    research_service.py:1487/:1541) runs for real too — this is not a
    hand-built message handed straight to the handler.

    Returns ``(queue_processor_mock, error_report_generator_mock,
    sio_emit_mock)``.
    """
    queue_processor = MagicMock()
    report_generator = MagicMock(
        return_value=MagicMock(
            generate_error_report=MagicMock(return_value="error report")
        )
    )
    storage = MagicMock()
    storage.save_report.return_value = True

    extra = {
        f"{MODULE}.ErrorReportGenerator": report_generator,
        f"{QUEUE_MODULE}.queue_processor": queue_processor,
        "local_deep_research.storage.get_report_storage": MagicMock(
            return_value=storage
        ),
        f"{MODULE}.AdvancedSearchSystem": MagicMock(),
        f"{MODULE}.get_llm": MagicMock(
            side_effect=RuntimeError(llm_error) if llm_error else None
        ),
        f"{MODULE}.get_search": MagicMock(
            side_effect=RuntimeError(search_error) if search_error else None
        ),
    }
    patches = _worker_patches(extra=extra)
    with _worker(patches) as mocks:
        run_research_process(
            research_id=str(uuid.uuid4()),
            query="cwe209 probe",
            username=username,
            # ``model`` is what takes the get_llm override branch; the
            # search branch is taken by ``search_engine`` (always set).
            model="tenant-model",
            settings_snapshot=_snapshot(),
            **BASE_RUN_KWARGS,
        )
    return queue_processor, report_generator, mocks[f"{MODULE}._sio_emit"]


def _client_visible_text(queue_processor, sio_emit):
    """Everything the worker hands to a client-visible sink, as one string.

    ``queue_error_update`` writes ``error_message`` + ``metadata`` onto the
    research row (rendered in the history UI) and ``_sio_emit`` pushes the
    message to the browser. Both are asserted together so a leak surviving
    on either sink fails.
    """
    parts = []
    for call in queue_processor.queue_error_update.call_args_list:
        parts.append(repr(call.kwargs))
    for call in sio_emit.call_args_list:
        parts.append(repr(call.args) + repr(call.kwargs))
    return "\n".join(parts)


class TestErrorMessageSanitisation:
    """CWE-209. Three genericised strings, all three unasserted on branch.

    Verified before writing: ``grep -rn`` over ``tests/`` for each literal
    below returned zero hits, so this file is their only coverage.
    """

    def test_llm_configuration_error_is_genericised(self):
        qp, report_gen, sio = _run_worker_into_error(
            llm_error=f"failed to load {LEAK_LLM}", username="alice_err"
        )

        # POSITIVE CONTROL: the handler ran and produced an update. Without
        # this, "the path does not appear" would pass for a worker that
        # crashed before reaching the classifier.
        assert qp.queue_error_update.call_count == 1, (
            "the outer error handler never queued an update"
        )
        kwargs = qp.queue_error_update.call_args.kwargs
        assert kwargs["status"] == ResearchStatus.FAILED
        assert kwargs["error_message"] == LLM_CONFIG_MSG
        assert kwargs["metadata"]["error"] == LLM_CONFIG_MSG
        assert kwargs["metadata"]["solution"].startswith(
            "Review your LLM model settings"
        )

        visible = _client_visible_text(qp, sio)
        assert LEAK_LLM not in visible, visible
        assert ".gguf" not in visible
        assert "/srv/" not in visible

        # The persisted error report is retrievable through the report
        # routes, so the sanitised message — not the raw exception — must be
        # what the generator embeds.
        report_kwargs = (
            report_gen.return_value.generate_error_report.call_args.kwargs
        )
        assert report_kwargs["error_message"] == (
            f"Research failed: {LLM_CONFIG_MSG}"
        )
        assert LEAK_LLM not in repr(report_kwargs)

    def test_search_engine_configuration_error_is_genericised(self):
        qp, report_gen, sio = _run_worker_into_error(
            search_error=f"searxng unreachable at {LEAK_SEARCH}",
            username="alice_err",
        )

        assert qp.queue_error_update.call_count == 1
        kwargs = qp.queue_error_update.call_args.kwargs
        assert kwargs["error_message"] == SEARCH_CONFIG_MSG
        assert kwargs["metadata"]["error"] == SEARCH_CONFIG_MSG
        assert kwargs["metadata"]["solution"].startswith(
            "Review your search engine settings"
        )

        visible = _client_visible_text(qp, sio)
        assert LEAK_SEARCH not in visible, visible
        assert "sk-live-DEADBEEFCAFE" not in visible
        assert "searx.internal.corp" not in visible

        report_kwargs = (
            report_gen.return_value.generate_error_report.call_args.kwargs
        )
        assert report_kwargs["error_message"] == (
            f"Research failed: {SEARCH_CONFIG_MSG}"
        )
        assert LEAK_SEARCH not in repr(report_kwargs)

    def test_unclassified_error_is_replaced_wholesale(self):
        """The ``else`` branch: anything the classifier does not recognise
        must be replaced, not echoed. This is the branch that would leak a
        traceback, a filesystem path or a connection string."""
        qp, report_gen, sio = _run_worker_into_error(
            llm_error=LEAK_GENERIC, username="alice_err"
        )

        assert qp.queue_error_update.call_count == 1
        kwargs = qp.queue_error_update.call_args.kwargs
        assert kwargs["error_message"] == GENERIC_MSG
        assert kwargs["metadata"]["error"] == GENERIC_MSG
        # The unclassified branch carries no ``solution`` hint — a leak
        # would most plausibly reappear there.
        assert "solution" not in kwargs["metadata"]

        visible = _client_visible_text(qp, sio)
        assert LEAK_GENERIC not in visible, visible
        assert "hunter2" not in visible
        assert "postgresql://" not in visible
        assert "secret_module.py" not in visible

        report_kwargs = (
            report_gen.return_value.generate_error_report.call_args.kwargs
        )
        assert report_kwargs["error_message"] == (
            f"Research failed: {GENERIC_MSG}"
        )
        assert "hunter2" not in repr(report_kwargs)

    def test_the_three_messages_are_distinguishable(self):
        """Anti-collapse control.

        A regression that replaced every branch with one string would still
        satisfy each "no leak" assertion above. The three classifications
        must remain distinct, so a user can tell an LLM misconfiguration
        from a search-engine one.
        """
        messages = set()
        for kwargs in (
            {"llm_error": f"failed to load {LEAK_LLM}"},
            {"search_error": f"searxng unreachable at {LEAK_SEARCH}"},
            {"llm_error": LEAK_GENERIC},
        ):
            qp, _, _ = _run_worker_into_error(username="alice_err", **kwargs)
            messages.add(
                qp.queue_error_update.call_args.kwargs["error_message"]
            )
        assert messages == {LLM_CONFIG_MSG, SEARCH_CONFIG_MSG, GENERIC_MSG}