~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
"""Route-level proof of the three security boundaries on ``/api/v1``.
The endpoints live in ``src/local_deep_research/web/routers/api_v1.py``.
Every guard below survived the Flask -> FastAPI migration in ``src/`` but
lost its test when ``tests/web/test_api_coverage.py`` was deleted. ADR-0010
records the historical measurement; this file is the committed regression
evidence.
COVERAGE AREA 1 - fail-closed settings snapshot
-----------------------------------------------
``_load_user_context_into_params`` threads the caller's ``username`` and the
snapshot of *their* encrypted settings database into every research call. If
that snapshot cannot be read it returns a 503 instead of continuing with an
empty one, because an empty snapshot resolves to the PERMISSIVE default
egress scope: a user who configured PRIVATE_ONLY would silently have their
data sent to the cloud by a request that looks like it succeeded. The opt-out
(``allow_default_settings``) is a strict ``is True`` identity check, so a
truthy-but-not-true value such as the string ``"false"`` must NOT disable the
refusal, and taking it emits a ``policy_audit`` warning so the downgrade is
never silent.
COVERAGE AREA 2 - hostile body values
-------------------------------------
``quick_summary`` requires ``query`` to be a real string (a dict or list
would otherwise flow into the research pipeline as a type confusion), and
``analyze_documents`` - which has no ``**kwargs`` - validates body keys
against its real signature, with ``username`` and ``settings_snapshot``
deliberately excluded so a caller cannot name the user whose settings and
credentials the run executes with.
``quick_summary``/``generate_report`` DO have ``**kwargs``, so an unknown
body key doesn't TypeError - it is silently forwarded to the research
function instead. ``_REJECTED_BODY_PARAMS`` denylists the specific keys
that are unsafe to forward: ``retrievers``/``llms`` (registry poisoning),
``progress_callback`` (Callable-typed - a string assigned to it blows up
mid-research as an opaque 500 the moment something tries to call it),
``openai_endpoint_url`` (live credential/endpoint steering - overlays the
account's stored ``llm.openai_endpoint`` URL and API key with a
caller-chosen host), ``research_id``/``programmatic_mode``/
``research_context`` (identity/audit plumbing the REST path already
manages itself), and
``settings``/``settings_override``/``api_key``/``user_password``/
``metadata``/``provider``/``max_search_results`` (silent no-ops on the
REST path, kept out so a future refactor can't quietly turn "no-op" into
"lever"). ``temperature`` is the one exception in that no-op family: it is
a PUBLICLY DOCUMENTED parameter (``GET /api/v1`` and release notes 1.8.1
both tell callers to pass it to ``/quick_summary``), so hard-400ing it
would break existing callers. It is accepted (200), stripped from
``params`` before the research call (so it still can't reach it), and the
response carries a ``warnings`` entry naming it. See
``_ACCEPTED_BUT_INEFFECTIVE_PARAMS`` in ``api_v1.py`` and
``TestIneffectiveButAcceptedParams`` below.
COVERAGE AREA 3 - CWE-209 scrub wiring
--------------------------------------
``tests/web/routers/test_api_v1_error_scrub.py`` covers ``_scrub_error_fields``
as a unit and its docstring claims the endpoint wiring is covered elsewhere;
it is not. A helper that is never called protects nothing - that is exactly
the shape of the original news SSRF in this repository - so these tests plant
the credential below the endpoint and read the HTTP response.
Test design notes
-----------------
* Nothing between the client and the guard is stubbed. Registration, login,
session cookies, CSRF, ``require_api_access`` and the encrypted per-user
database all run for real. Only the research function *below* the endpoint
is replaced, so no LLM or search engine is contacted.
* The snapshot failure is induced by making the real ``SettingsManager``
instance's ``get_settings_snapshot`` raise - the ``try/except`` under test
is untouched, only the operation it is there to catch is made to fail.
* Every negative assertion is paired with a positive control in the same
test: the identical request is proven to reach the research function first,
so "the function was not called" / "the secret is absent" can never pass
because the route was unreachable for an unrelated reason (bad body, CSRF,
rate limit, 500).
* CSRF is middleware-enforced for ``/api/v1``: the migration deliberately
removed Flask's blanket ``csrf.exempt`` for this blueprint, so a bare POST
is rejected with 403 before any dependency runs. Every mutating request
here carries a real session-bound token.
* Rate limiting is keyed per client IP, so each client gets a distinct peer
address from a monotonic counter. Random addresses collide over a long
session and produce 429s unrelated to the guard under test.
Scope note observed while writing these: the ``isinstance(query, str)`` guard
exists ONLY in ``api_quick_summary``. ``api_generate_report`` and
``api_analyze_documents`` read ``data.get("query")`` without a type check.
That asymmetry is source behaviour, not something these tests pin.
"""
from __future__ import annotations
import itertools
import uuid
from contextlib import contextmanager
from unittest.mock import patch
import pytest
from loguru import logger
from local_deep_research.web.routers import api_v1
# These tests log in for real, so opt out of the autouse
# ``_legacy_bare_username_auth`` shim that relaxes the server-side
# session-id check for the legacy bare-username test idiom.
pytestmark = pytest.mark.real_session_check
ACCOUNT_PASSWORD = "Api-Boundary-Probe-1!" # noqa: S105
QUICK_SUMMARY_PATH = "/api/v1/quick_summary"
GENERATE_REPORT_PATH = "/api/v1/generate_report"
ANALYZE_DOCUMENTS_PATH = "/api/v1/analyze_documents"
QUERY = "api v1 boundary probe"
COLLECTION = "boundary-probe-collection"
# Patch targets differ because the routers import differently:
# quick_summary/generate_report are imported inside the handler body,
# analyze_documents at module import time (its signature seeds
# _ANALYZE_DOCUMENTS_PARAMS).
QUICK_SUMMARY_TARGET = (
"local_deep_research.api.research_functions.quick_summary"
)
GENERATE_REPORT_TARGET = (
"local_deep_research.api.research_functions.generate_report"
)
ANALYZE_DOCUMENTS_TARGET = (
"local_deep_research.web.routers.api_v1.analyze_documents"
)
# The three endpoints share _load_user_context_into_params, so gaps 17 and 19
# are parametrized across all of them. Each entry is
# (label, path, minimal body, patch target, mocked return payload).
_ENDPOINTS = [
(
"quick_summary",
QUICK_SUMMARY_PATH,
{"query": QUERY},
QUICK_SUMMARY_TARGET,
{"summary": "fine", "findings": [], "research_id": "probe-1"},
),
(
"generate_report",
GENERATE_REPORT_PATH,
{"query": QUERY},
GENERATE_REPORT_TARGET,
{"content": "fine", "metadata": {}},
),
(
"analyze_documents",
ANALYZE_DOCUMENTS_PATH,
{"query": QUERY, "collection_name": COLLECTION},
ANALYZE_DOCUMENTS_TARGET,
{"summary": "fine", "findings": []},
),
]
_ENDPOINT_IDS = [entry[0] for entry in _ENDPOINTS]
# Planted below the endpoint so the response can be searched for it. The
# "Error:" prefix is what arms _scrub_error_fields; the credential shape is
# what sanitize_error_for_client redacts.
_PLANTED_CREDENTIAL = "sk-BOUNDARYWIRING1234567890"
_PLANTED_DB_PASSWORD = "boundary-wiring-db-pw-9182" # noqa: S105
_BENIGN_MARKER = "benign-passthrough-marker"
# Raised in place of the settings-snapshot read. Carries a credential shape
# so the 503 body can be checked for exception-text echo at the same time.
_SNAPSHOT_FAILURE_TEXT = (
f"SQLCipher decrypt failed for settings db ({_PLANTED_CREDENTIAL})"
)
# Rate limiting is keyed per client IP and the limiter's enabled flag is
# resolved at import time, so a fixture-set env var cannot turn it off. A
# monotonic counter gives every client a unique peer; random addresses
# collide over a long session and yield 429s unrelated to the guard.
_peer_counter = itertools.count(1)
@pytest.fixture
def live_app(tmp_path, monkeypatch):
"""The real assembled FastAPI app on a temp data dir.
Same shape as ``tests/security/test_research_password_gate_fastapi.py``:
the routes read module-level singletons (``db_manager``,
``session_password_store``), so the app must run against those exact
instances and the data dir has to be repointed on the singleton itself.
Usernames created here are tracked and their password-store entries
dropped afterwards - the store is process-wide and ``reset_all_singletons``
does not touch it, so a leaked entry would be visible to later tests in
the same worker.
"""
monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))
monkeypatch.setenv("LDR_TESTING_WITH_MOCKS", "true")
monkeypatch.setenv("LDR_DISABLE_RATE_LIMITING", "true")
monkeypatch.setenv("LDR_DB_CONFIG_KDF_ITERATIONS", "1000")
from local_deep_research.database.auth_db import init_auth_database
from local_deep_research.database.encrypted_db import db_manager
from local_deep_research.database.session_passwords import (
session_password_store,
)
from local_deep_research.web.fastapi_app import app as fastapi_app
import local_deep_research.web.routers.auth as auth_routes
original_data_dir = db_manager.data_dir
created_users: list[str] = []
try:
db_manager.data_dir = tmp_path / "encrypted_databases"
init_auth_database()
# Keep the synchronous test off the real post-login worker threads.
monkeypatch.setattr(
auth_routes,
"_perform_post_login_tasks",
lambda _u, _p, _sid=None: None,
)
yield _Harness(fastapi_app, created_users)
finally:
for username in created_users:
session_password_store.clear_all_for_user(username)
db_manager.close_all_databases()
db_manager.data_dir = original_data_dir
class _Harness:
"""The app plus the bookkeeping ``_api_user`` needs."""
def __init__(self, app, created_users):
self.app = app
self.created_users = created_users
def _client(app):
"""A TestClient with its own, monotonically assigned peer address."""
from fastapi.testclient import TestClient
peer = next(_peer_counter)
client = TestClient(app, raise_server_exceptions=False)
client.headers.update(
{"X-Forwarded-For": f"10.{peer // 254 % 254 + 1}.{peer % 254 + 1}.9"}
)
return client
def _csrf(client):
"""A CSRF token bound to this client's session (middleware-enforced)."""
client.get("/auth/login")
resp = client.get("/auth/csrf-token")
return resp.json().get("csrf_token", "") if resp.status_code == 200 else ""
def _api_user(harness):
"""Register a fresh user and return an authenticated client.
Registration (not a bare session poke) is what puts the account's
password in ``session_password_store``, which is how
``get_user_db_session(username)`` opens the encrypted settings database
that both ``require_api_access`` and the snapshot load depend on.
"""
username = f"apiv1b_{uuid.uuid4().hex[:8]}"
harness.created_users.append(username)
client = _client(harness.app)
token = _csrf(client)
resp = client.post(
"/auth/register",
data={
"username": username,
"password": ACCOUNT_PASSWORD,
"confirm_password": ACCOUNT_PASSWORD,
"acknowledge": "true",
"csrf_token": token,
},
headers={"X-CSRFToken": token},
follow_redirects=False,
)
assert resp.status_code in (200, 302), (
f"registration failed: {resp.status_code} / {resp.text[:400]}"
)
# Consume the one-shot post-login temp-auth token so the session is
# fully settled before the request under test.
assert client.get("/auth/check").status_code == 200, (
"the client must be authenticated after registration"
)
return client, username
def _post(client, path, body):
token = _csrf(client)
return client.post(path, json=body, headers={"X-CSRFToken": token})
def _store_setting(username, key, value):
"""Write a setting into the user's own encrypted database."""
from local_deep_research.database.session_context import get_user_db_session
from local_deep_research.settings import SettingsManager
with get_user_db_session(username, password=ACCOUNT_PASSWORD) as db_session:
assert SettingsManager(db_session).set_setting(key, value), (
f"could not seed {key} for {username}"
)
@contextmanager
def _snapshot_load_fails():
"""Make the settings-snapshot read fail for real.
The guard itself (``_load_user_context_into_params``'s ``try/except``) is
NOT touched: only ``get_settings_snapshot`` on the real manager instance
is made to raise, which is what a SQLCipher decrypt failure or a corrupt
settings table produces. ``get_setting`` is left intact so
``require_api_access`` - which runs first and reads ``app.enable_api``
through the same factory - still behaves normally and the request really
does reach the boundary under test.
"""
real_factory = api_v1.get_settings_manager
def _raise(*_args, **_kwargs):
raise RuntimeError(_SNAPSHOT_FAILURE_TEXT)
def _factory(db_session=None, username=None):
manager = real_factory(db_session, username)
manager.get_settings_snapshot = _raise
return manager
with patch.object(api_v1, "get_settings_manager", _factory):
yield
@contextmanager
def _captured_log_records():
"""Collect loguru records (with their ``extra`` dict) during the block."""
records: list[dict] = []
# The package disables its own loguru namespace in __init__; enable it so
# in-module audit warnings reach the sink. Restored in finally.
logger.enable("local_deep_research")
sink_id = logger.add(lambda message: records.append(message.record))
try:
yield records
finally:
logger.remove(sink_id)
logger.disable("local_deep_research")
def _policy_audit_downgrade_records(records):
return [
record
for record in records
if record["extra"].get("policy_audit")
and "NOT bound by the user's egress policy" in record["message"]
]
class TestSettingsSnapshotBoundary:
"""Coverage area 1: ``_load_user_context_into_params``."""
@pytest.mark.parametrize(
"label,path,body,target,payload", _ENDPOINTS, ids=_ENDPOINT_IDS
)
def test_the_callers_own_settings_are_threaded_into_the_run(
self, live_app, label, path, body, target, payload
):
"""The run must use the caller's STORED settings, not app defaults.
This is the #3661 bug class: with the snapshot missing, the research
runs on application defaults plus ``LDR_*`` env vars, so the user's
provider, model and - critically - their egress policy do not apply.
A sentinel written into the user's own encrypted database proves the
snapshot is theirs and not a generic default set.
"""
client, username = _api_user(live_app)
sentinel = f"sentinel-model-{uuid.uuid4().hex[:8]}"
_store_setting(username, "llm.model", sentinel)
with patch(target) as research_fn:
research_fn.return_value = payload
resp = _post(client, path, body)
assert resp.status_code == 200, f"{label}: {resp.text[:400]}"
assert research_fn.call_count == 1
kwargs = research_fn.call_args.kwargs
assert kwargs["username"] == username, (
f"{label}: the run must be attributed to the authenticated user"
)
assert kwargs["programmatic_mode"] is False, (
f"{label}: authenticated REST calls persist DB-backed metrics"
)
snapshot = kwargs["settings_snapshot"]
assert isinstance(snapshot, dict) and snapshot, (
f"{label}: an empty snapshot resolves to the permissive egress "
f"scope; got {snapshot!r}"
)
assert snapshot.get("llm.model") == sentinel, (
f"{label}: the snapshot is not the caller's stored settings"
)
@pytest.mark.parametrize(
"label,path,body,target,payload", _ENDPOINTS, ids=_ENDPOINT_IDS
)
def test_unreadable_settings_refuse_the_run_instead_of_downgrading_it(
self, live_app, label, path, body, target, payload
):
"""Fail CLOSED: 503, not a run with an empty (permissive) snapshot."""
client, username = _api_user(live_app)
# Positive control: the identical request, in the identical session,
# reaches the research function when the snapshot loads. Without it
# "the function was not called" would also pass for a route that
# refused everything, or that 403'd on CSRF.
with patch(target) as research_fn:
research_fn.return_value = payload
ok = _post(client, path, body)
assert ok.status_code == 200, (
f"{label}: positive control failed: {ok.text[:400]}"
)
assert research_fn.call_count == 1
with patch(target) as research_fn:
research_fn.return_value = payload
with _snapshot_load_fails():
refused = _post(client, path, body)
assert refused.status_code == 503, (
f"{label}: expected a fail-closed refusal, got "
f"{refused.status_code} / {refused.text[:400]}"
)
assert research_fn.call_count == 0, (
f"{label}: the research must not run without the user's settings"
)
data = refused.json()
assert data["reason"] == "settings_unavailable"
assert "REFUSED" in data["error"]
assert "allow_default_settings" in data["how_to_fix"], (
f"{label}: the refusal must name the documented opt-in"
)
# The refusal is a fixed message: the underlying exception text (here
# carrying a credential shape) must not be echoed to the client.
assert _PLANTED_CREDENTIAL not in refused.text
assert "SQLCipher" not in refused.text
def test_a_literal_json_true_opts_in_to_defaults_and_an_empty_snapshot(
self, live_app
):
"""The documented escape hatch: opt in, run, empty snapshot."""
client, _username = _api_user(live_app)
with patch(QUICK_SUMMARY_TARGET) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
with _snapshot_load_fails():
resp = _post(
client,
QUICK_SUMMARY_PATH,
{"query": QUERY, "allow_default_settings": True},
)
assert resp.status_code == 200, resp.text[:400]
assert research_fn.call_count == 1
kwargs = research_fn.call_args.kwargs
assert kwargs["settings_snapshot"] == {}, (
"the opt-in path runs with defaults, i.e. an empty snapshot"
)
assert "allow_default_settings" not in kwargs, (
"the flag is an API-boundary control and must not be forwarded "
"to the research function, which has no such parameter"
)
@pytest.mark.parametrize(
"truthy",
["true", "True", "false", "0", 1, [1]],
ids=["str-true", "str-True", "str-false", "str-0", "int-1", "list"],
)
def test_only_a_literal_json_true_disables_the_refusal(
self, live_app, truthy
):
"""``is True``, not ``bool()``.
Every value here is truthy under ``bool()`` except the strings
``"false"`` and ``"0"``, which are truthy *as strings* - the classic
way a boolean-ish flag from an untyped client accidentally disables a
security boundary. Only a real JSON ``true`` may.
"""
client, _username = _api_user(live_app)
with patch(QUICK_SUMMARY_TARGET) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
with _snapshot_load_fails():
resp = _post(
client,
QUICK_SUMMARY_PATH,
{"query": QUERY, "allow_default_settings": truthy},
)
assert resp.status_code == 503, (
f"allow_default_settings={truthy!r} must not opt out of the "
f"fail-closed refusal, got {resp.status_code} / {resp.text[:300]}"
)
assert research_fn.call_count == 0
def test_the_opt_in_downgrade_is_logged_as_a_policy_audit_event(
self, live_app
):
"""A silent downgrade is the failure mode; the warning is the fix.
Paired controls in one test: the refusal path must emit NO downgrade
record (otherwise "a record exists" proves nothing about the opt-in),
and the opt-in path must emit exactly one, bound with
``policy_audit=True`` and carrying the user it applies to.
"""
client, username = _api_user(live_app)
body = {"query": QUERY, "allow_default_settings": True}
with patch(QUICK_SUMMARY_TARGET) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
with _captured_log_records() as records:
with _snapshot_load_fails():
refused = _post(
client, QUICK_SUMMARY_PATH, {"query": QUERY}
)
refusal_records = _policy_audit_downgrade_records(records)
records.clear()
with _snapshot_load_fails():
opted_in = _post(client, QUICK_SUMMARY_PATH, body)
opt_in_records = _policy_audit_downgrade_records(records)
assert refused.status_code == 503
assert opted_in.status_code == 200, opted_in.text[:400]
assert refusal_records == [], (
"the fail-closed path must not claim a policy downgrade happened"
)
assert len(opt_in_records) == 1, (
"the opt-in downgrade must be recorded exactly once, got "
f"{len(opt_in_records)}"
)
record = opt_in_records[0]
assert record["level"].name == "WARNING"
assert record["extra"].get("user") == username, (
"the audit record must name the user whose policy was bypassed"
)
assert "allow_default_settings=true" in record["message"]
class TestQueryTypeValidation:
"""Coverage area 2: request type checks on ``quick_summary``."""
def test_a_string_query_reaches_the_research_function(self, live_app):
"""Positive control for the whole class."""
client, _username = _api_user(live_app)
with patch(QUICK_SUMMARY_TARGET) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(client, QUICK_SUMMARY_PATH, {"query": QUERY})
assert resp.status_code == 200, resp.text[:400]
assert research_fn.call_args.args[0] == QUERY
@pytest.mark.parametrize(
"query",
[
123,
12.5,
["injected", "list"],
{"nested": "dict"},
None,
True,
],
ids=["int", "float", "list", "dict", "null", "bool"],
)
def test_a_non_string_query_is_rejected_with_400(self, live_app, query):
"""Type confusion on the public programmatic API.
``{"query": <non-string>}`` passes the presence check above the guard,
so without ``isinstance`` a dict or list flows straight into the
research pipeline.
"""
client, _username = _api_user(live_app)
with patch(QUICK_SUMMARY_TARGET) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(client, QUICK_SUMMARY_PATH, {"query": query})
assert resp.status_code == 400, (
f"query={query!r} must be rejected, got {resp.status_code} / "
f"{resp.text[:300]}"
)
assert resp.json()["error"] == "Query must be a string"
assert research_fn.call_count == 0, (
f"query={query!r} reached the research pipeline"
)
class TestRegistryParamsRejection:
"""Coverage area 2: ``retrievers``/``llms`` are rejected, not forwarded.
Ported from PR #5533, which originally landed this guard on the Flask
blueprint at ``src/local_deep_research/web/api.py``. That blueprint was
unmounted by the FastAPI migration (#3299) and later deleted outright,
so the guard never reached the endpoints that actually serve
``/api/v1/quick_summary`` and ``/api/v1/generate_report`` on current
``main`` - this class exercises the live ``api_v1.py`` implementation
instead.
``retrievers``/``llms`` are declared parameters of ``quick_summary``/
``generate_report`` (see ``QUICK_SUMMARY_TARGET``/``GENERATE_REPORT_TARGET``
in ``api/research_functions.py``) that get registered into the retriever
registry / LLM registry the moment they reach the research function. A
JSON body can never carry a live ``BaseRetriever``/``BaseChatModel``, so
any value posted on these keys is necessarily the wrong type - this
rejects it at the HTTP boundary with a 400 instead of letting it reach
``register_multiple`` (an opaque 500) or silently poison the caller's own
registry namespace. ``analyze_documents`` has no ``retrievers``/``llms``
parameter, so it is already covered by
``TestAnalyzeDocumentsParameterInjection``'s signature-derived allowlist
and is not repeated here.
"""
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
@pytest.mark.parametrize(
"body_extra",
[
{"retrievers": {"poison": "not-a-retriever"}},
{"llms": {"poison": "not-an-llm"}},
{
"retrievers": {"poison": "not-a-retriever"},
"llms": {"poison": "not-an-llm"},
},
],
ids=["retrievers", "llms", "both"],
)
def test_rejects_registry_params_with_400(
self, live_app, label, path, target, body_extra
):
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(client, path, {"query": QUERY, **body_extra})
assert resp.status_code == 400, (
f"{label}: expected a 400, got {resp.status_code} / "
f"{resp.text[:300]}"
)
# The research function (and thus registration) must never run.
assert research_fn.call_count == 0, (
f"{label}: a registry param reached the research function"
)
# The static message TAIL ("They either require a live Python
# object...") lists every _REJECTED_BODY_PARAMS key, so asserting
# against it alone would pass for ANY key in the deny-list
# regardless of which one was actually posted. But the message's
# dynamic PREFIX ("...REST API: {sorted, comma-joined present
# keys}.") is built from exactly the keys _reject_unsafe_body_params
# found in this request's body (see api_v1.py), so asserting that
# prefix does discriminate a reverted/broken guard from a working
# one, unlike a bare "error" in resp.json() check.
expected_prefix = f"REST API: {', '.join(sorted(body_extra))}."
assert expected_prefix in resp.json()["error"], (
f"{label}: error must name exactly {sorted(body_extra)}: "
f"{resp.json()!r}"
)
@pytest.mark.parametrize(
"label,path,body,target",
[
(
"quick_summary",
QUICK_SUMMARY_PATH,
{"query": QUERY},
QUICK_SUMMARY_TARGET,
),
(
"generate_report",
GENERATE_REPORT_PATH,
{"query": QUERY},
GENERATE_REPORT_TARGET,
),
],
ids=["quick_summary", "generate_report"],
)
def test_clean_request_without_registry_params_still_succeeds(
self, live_app, label, path, body, target
):
"""Positive control: the guard doesn't reject legitimate requests."""
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(client, path, body)
assert resp.status_code == 200, f"{label}: {resp.text[:400]}"
assert "retrievers" not in research_fn.call_args.kwargs
assert "llms" not in research_fn.call_args.kwargs
class TestUnsafeForwardParamsRejection:
"""Coverage area 2: the rest of the unvalidated-forwarding class.
``TestRegistryParamsRejection`` above covers ``retrievers``/``llms``.
Rejecting only those two left the same *class* of bug open: ``params``
for ``quick_summary``/``generate_report`` is still built as
``{k: v for k, v in body.items() if k not in (...)}``, forwarding every
other body key straight to the research function.
``progress_callback`` is the sharpest instance: it is Callable-typed,
a JSON body can never carry one, and unlike retrievers/llms it doesn't
fail loudly at the registration site — the string gets assigned to
``system.progress_callback`` and only blows up deep inside the strategy
loop, mid-research, as an opaque 500 (POSTing
``{"query": "x", "progress_callback": "boom"}`` reaches
``_init_search_system`` -> ``system.set_progress_callback("boom")``).
``settings``/``settings_override``/``api_key``/``user_password``/
``metadata`` round out the class: declared parameters (or bare
**kwargs reads) that are silent no-ops over the REST path today, kept
out anyway so no future refactor turns "no-op" back into "lever" the
way it already is for username/settings_snapshot. See
``_REJECTED_BODY_PARAMS`` in ``api_v1.py`` for the full rationale.
``provider``/``max_search_results`` are the same no-op shape as
``api_key`` above: named parameters of ``quick_summary``/
``generate_report`` consumed only inside the
``if "settings_snapshot" not in kwargs:`` branch that the REST path
never takes (``_load_user_context_into_params`` always injects a
snapshot). See the ``_DEAD_OR_CONFUSING_PARAMS`` comment in
``api_v1.py`` for why these cannot simply be threaded into the
settings snapshot instead. ``temperature`` is the same dead shape but
is handled differently (200 + a ``warnings`` entry, not a 400) because
it is a publicly documented parameter — see
``TestIneffectiveButAcceptedParams`` below, not this class.
``openai_endpoint_url`` is the opposite of a no-op: it is forwarded
unconditionally (never gated behind the settings_snapshot branch
above), and when the caller's stored ``llm.provider`` is
``openai_endpoint`` it OVERLAYS ``settings_snapshot["llm.openai_endpoint.
url"]`` inside ``get_llm`` — steering that run's prompts, and the
account's already-configured endpoint API key, to any host the caller
names. See ``_CREDENTIAL_STEERING_PARAMS`` in ``api_v1.py``.
``research_id``/``programmatic_mode``/``research_context`` are
identity/audit plumbing the REST path already manages itself: a
caller-supplied ``research_id`` can split-brain a run's
SearchCall/TokenUsage metrics rows on ``generate_report`` (which mints
its own fresh id for one but not the other), ``programmatic_mode=true``
lets a caller opt their own calls out of the DB-backed
persistence/audit trail that ``_load_user_context_into_params``
deliberately turns on for authenticated REST calls, and a
caller-supplied ``research_context`` reaches
``get_llm(research_context=...)`` on ``generate_report`` (unlike
``quick_summary``, which always overwrites it with its own
server-built metrics dict before calling ``_init_search_system``) -
a non-dict value 500s where the code mutates it
(``research_context["context_limit"] = ...``), and a dict value feeds
caller-controlled ``username``/``user_password``/query metadata into
``TokenCounter``. See ``_IDENTITY_PLUMBING_PARAMS`` in ``api_v1.py``.
``output_file`` is a different shape again: a declared parameter of
``generate_report`` that names a server-side filesystem path to write
the finished report to. ``api.allow_file_output`` has no default
anywhere under ``defaults/``, so it fails closed today - but only
AFTER the full research-and-report run completes, surfacing as an
opaque 500 (``FileWriteSecurityError`` isn't special-cased by the
endpoint's catch-all). And ``file_write_verifier.py``'s write path is a
bare ``open(filepath, mode)`` with no containment, so an operator who
has explicitly turned the setting on gets no traversal/symlink
protection either. A server-side path posted in a public JSON body is
not a REST-shaped parameter, so it is rejected outright rather than
validated early. See ``_SERVER_PATH_PARAMS`` in ``api_v1.py``. This
was NOT covered by the original ``_REJECTED_BODY_PARAMS`` audit
(which was scoped to keys reaching ``_init_search_system``, and
``output_file`` never does) - found and closed in post-merge review of
PR #5533.
"""
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
@pytest.mark.parametrize(
"key,value",
[
("progress_callback", "boom"),
("settings", {"llm.provider": "openai"}),
("settings_override", {"llm.provider": "openai"}),
("api_key", "sk-not-really-a-key"),
("user_password", "not-really-a-password"),
("metadata", {"anything": "here"}),
("provider", "anthropic"),
("max_search_results", 5),
("openai_endpoint_url", "https://attacker.example.com/v1"),
("research_id", "attacker-chosen-research-id"),
("programmatic_mode", True),
(
"research_context",
{"username": "attacker", "user_password": "not-a-password"},
),
("research_mode", "attacker-chosen-mode-label"),
# output_file: a server-side filesystem path is not a
# REST-shaped parameter (see _SERVER_PATH_PARAMS in api_v1.py).
# Regression test for the completeness-claim gap found in
# post-merge review of #5533: the reject-list's original
# docstring claimed to be "the full class" for
# quick_summary/generate_report but this key, forwarded
# straight to generate_report's own write-to-disk path, was
# never in it.
("output_file", "/tmp/pwned-by-api-boundary-test.md"),
],
)
def test_rejects_unsafe_forward_params_with_400(
self, live_app, label, path, target, key, value
):
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(client, path, {"query": QUERY, key: value})
assert resp.status_code == 400, (
f"{label}/{key}: expected a 400, got {resp.status_code} / "
f"{resp.text[:300]}"
)
# The research function must never run with the unsafe key present.
assert research_fn.call_count == 0, (
f"{label}/{key}: an unsafe param reached the research function"
)
# The static message TAIL lists every _REJECTED_BODY_PARAMS key
# (so a bare "key in error" substring check would pass regardless
# of which key was actually posted), but the dynamic PREFIX names
# exactly the keys _reject_unsafe_body_params found present in this
# request — asserting it does discriminate a reverted/broken guard.
assert f"REST API: {key}." in resp.json()["error"], (
f"{label}/{key}: error must name {key!r}: {resp.json()!r}"
)
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
def test_clean_request_with_documented_params_still_succeeds(
self, live_app, label, path, target
):
"""Positive control: legitimate documented params still pass.
Proves the new denylist doesn't collaterally reject ordinary
request-configuration params that were never part of this bug
class (they aren't Callable-typed and aren't dead no-ops).
``temperature`` is deliberately NOT included here either: it is a
dead no-op on this REST path (see
``TestUnsafeForwardParamsRejection``'s docstring), just no longer a
400 — it belongs in ``TestIneffectiveButAcceptedParams`` below,
which asserts the 200 + ``warnings`` + stripped-before-the-call
behavior specifically.
``search_tool``/``model_name``/``search_strategy`` are the
audited-allowed set from the ``_REJECTED_BODY_PARAMS`` comment
block in ``api_v1.py``: traced into
``_init_search_system``/``get_llm``/``get_search`` and found to
only select among the caller's OWN already-configured
provider/model/strategy — no host or credential steering, unlike
``openai_endpoint_url``. These three are also type-validated (see
``TestForwardedParamTypeValidation``).
``iterations``/``questions_per_iteration`` are real kwargs
forwarded through quick_summary's/generate_report's ``**kwargs``.
This test only proves the boundary doesn't strip or reject them.
"""
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(
client,
path,
{
"query": QUERY,
"iterations": 2,
"search_tool": "wikipedia",
"model_name": "gpt-4o-mini",
"search_strategy": "source_based",
"questions_per_iteration": 2,
},
)
assert resp.status_code == 200, f"{label}: {resp.text[:400]}"
assert research_fn.call_args.kwargs["iterations"] == 2
assert research_fn.call_args.kwargs["search_tool"] == "wikipedia"
assert research_fn.call_args.kwargs["model_name"] == "gpt-4o-mini"
assert research_fn.call_args.kwargs["search_strategy"] == "source_based"
assert research_fn.call_args.kwargs["questions_per_iteration"] == 2
class TestForwardedParamTypeValidation:
"""Coverage area 2: wrong-scalar-type guard for confirmed footguns.
``model_name``/``search_strategy``/``search_tool`` are forwarded, live,
documented knobs (see ``test_clean_request_with_documented_params_still_
succeeds`` above) - correctly not rejected outright. But each one is
handed, with no type check, to an unconditional operation on its first
consumer: ``model_name.strip()`` in ``config/llm_config.py``,
``search_strategy.lower()`` in ``search_system.py``, and (once a
truthy value survives the ``search_tool or get_setting_from_snapshot``
fallback) ``retriever_registry.get(name, ...)`` -> a plain dict
``.get()`` in ``web_search_engines/retriever_registry.py``. A JSON body
sending the wrong scalar type for any of these previously reached that
operation and raised an unhandled ``AttributeError``/``TypeError`` - an
opaque 500. ``_validate_param_types`` in ``api_v1.py`` catches the
wrong type at the boundary instead, with a 400 naming the offending
key(s).
``iterations`` is NOT covered here: when omitted it is not forwarded
(snapshot ``search.iterations`` is used). When present it is passed
into ``AdvancedSearchSystem``. See
``TestUnsafeForwardParamsRejection.
test_clean_request_with_documented_params_still_succeeds`` for proof
an explicit value is still accepted and forwarded.
"""
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
@pytest.mark.parametrize(
"key,value",
[
("model_name", 123),
("search_strategy", 5),
("search_tool", ["searxng"]),
("search_tool", {"tool": "searxng"}),
("iterations", 0),
("iterations", -1),
("iterations", 101),
("iterations", "3"),
("iterations", 1.5),
("iterations", True),
("questions_per_iteration", 0),
("questions_per_iteration", -1),
("questions_per_iteration", 21),
("questions_per_iteration", "3"),
("questions_per_iteration", 1.5),
("questions_per_iteration", False),
],
ids=[
"model_name-int",
"search_strategy-int",
"search_tool-list",
"search_tool-dict",
"iterations-0",
"iterations-neg",
"iterations-101",
"iterations-str",
"iterations-float",
"iterations-bool",
"questions_per_iteration-0",
"questions_per_iteration-neg",
"questions_per_iteration-21",
"questions_per_iteration-str",
"questions_per_iteration-float",
"questions_per_iteration-bool",
],
)
def test_wrong_type_is_rejected_with_400(
self, live_app, label, path, target, key, value
):
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(client, path, {"query": QUERY, key: value})
assert resp.status_code == 400, (
f"{label}/{key}: expected a 400, got {resp.status_code} / "
f"{resp.text[:300]}"
)
assert research_fn.call_count == 0, (
f"{label}/{key}: a wrong-typed param reached the research function"
)
assert key in resp.json()["error"], (
f"{label}/{key}: error must name {key!r}: {resp.json()!r}"
)
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
def test_correctly_typed_values_still_succeed(
self, live_app, label, path, target
):
"""Positive control: the type guard doesn't collaterally reject
correctly-typed values for the same keys (``iterations`` included,
even though it is not type-validated - see the class docstring)."""
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(
client,
path,
{
"query": QUERY,
"model_name": "gpt-4o-mini",
"search_strategy": "source_based",
"search_tool": "wikipedia",
"iterations": 3,
"questions_per_iteration": 2,
},
)
assert resp.status_code == 200, f"{label}: {resp.text[:400]}"
assert research_fn.call_args.kwargs["model_name"] == "gpt-4o-mini"
assert research_fn.call_args.kwargs["search_strategy"] == "source_based"
assert research_fn.call_args.kwargs["search_tool"] == "wikipedia"
assert research_fn.call_args.kwargs["iterations"] == 3
assert research_fn.call_args.kwargs["questions_per_iteration"] == 2
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
@pytest.mark.parametrize(
"key",
["model_name", "search_tool", "iterations", "questions_per_iteration"],
)
def test_null_is_let_through_for_none_as_sentinel_params(
self, live_app, label, path, target, key
):
"""``{"model_name": null}`` / ``{"search_tool": null}`` were 200
before this module's type validation existed - both reach a
``param or <fallback>`` / ``if param is None: <fallback>`` branch
in their consumer and resolve from the settings snapshot exactly as
if the key had been omitted (``config/llm_config.get_llm`` for
``model_name``; ``config/search_config.get_search`` for
``search_tool``, reached via ``research_functions.py``'s own
``if search_tool:`` guard). A blanket ``isinstance`` check would
have turned that documented no-op spelling into a new 400; see
``_NULL_IS_SENTINEL_PARAMS`` in api_v1.py.
"""
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(client, path, {"query": QUERY, key: None})
assert resp.status_code == 200, (
f"{label}/{key}: null must be let through as the sentinel "
f"for 'use the stored setting', got {resp.status_code} / "
f"{resp.text[:300]}"
)
assert research_fn.call_args.kwargs[key] is None
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
def test_null_search_strategy_is_still_rejected(
self, live_app, label, path, target
):
"""``search_strategy`` gets NO null carve-out, unlike ``model_name``/
``search_tool`` above: its consumer default is the non-None string
``"source_based"`` (``_init_search_system``), so an EXPLICIT
``null`` overrides that default with a real ``None`` and reaches
``strategy_name.lower()`` (search_system.py) unconditionally. That
was a genuine pre-existing 500 - not a documented no-op - so this
module's 400 for it is a real fix, not an over-rejection (the one
case in this whole audit where the null-related behavior change is
an improvement rather than a regression).
"""
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(
client, path, {"query": QUERY, "search_strategy": None}
)
assert resp.status_code == 400, (
f"{label}: expected a 400 for null search_strategy, got "
f"{resp.status_code} / {resp.text[:300]}"
)
assert research_fn.call_count == 0
class TestIneffectiveButAcceptedParams:
"""Coverage area 2: ``temperature`` is a documented no-op, not a 400.
Unlike the dead no-ops in ``TestUnsafeForwardParamsRejection``,
``temperature`` was a PUBLICLY DOCUMENTED REST parameter (``GET
/api/v1``'s own ``parameters`` dict, and release notes 1.8.1 telling
callers migrating off the removed ``quick_summary_test`` endpoint to
"call /quick_summary with search_tool, iterations, and temperature set
explicitly"). Hard-400ing it would break existing callers who followed
that documentation, so it takes the honest-and-non-breaking path
instead: accepted (200), popped out of ``params`` before the research
function is called so it genuinely cannot reach it, and the response
carries a ``warnings`` entry naming it. See
``_ACCEPTED_BUT_INEFFECTIVE_PARAMS``/``_pop_ineffective_params`` in
``api_v1.py``.
"""
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
def test_temperature_succeeds_with_a_warning_and_is_stripped(
self, live_app, label, path, target
):
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(client, path, {"query": QUERY, "temperature": 0.1})
assert resp.status_code == 200, (
f"{label}: temperature must not be hard-rejected (it is a "
f"documented parameter), got {resp.status_code} / "
f"{resp.text[:300]}"
)
# It must still never reach the research function.
assert "temperature" not in research_fn.call_args.kwargs, (
f"{label}: temperature was forwarded to the research function "
"despite being an accepted-but-ineffective param"
)
body = resp.json()
assert any("temperature" in w for w in body.get("warnings", [])), (
f"{label}: response must warn that temperature had no effect: "
f"{body!r}"
)
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
def test_no_warnings_key_when_nothing_ineffective_was_posted(
self, live_app, label, path, target
):
"""Negative control: a clean request gets no ``warnings`` noise."""
client, _username = _api_user(live_app)
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(client, path, {"query": QUERY})
assert resp.status_code == 200, f"{label}: {resp.text[:300]}"
assert "warnings" not in resp.json()
class TestAnalyzeDocumentsParameterInjection:
"""Coverage area 2: the ``analyze_documents`` body-key allowlist."""
def test_a_documented_parameter_is_accepted_and_forwarded(self, live_app):
"""Positive control: the allowlist admits what the signature has."""
client, _username = _api_user(live_app)
with patch(ANALYZE_DOCUMENTS_TARGET) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(
client,
ANALYZE_DOCUMENTS_PATH,
{
"query": QUERY,
"collection_name": COLLECTION,
"max_results": 3,
"force_reindex": False,
},
)
assert resp.status_code == 200, resp.text[:400]
assert research_fn.call_args.args[:2] == (QUERY, COLLECTION)
assert research_fn.call_args.kwargs["max_results"] == 3
assert research_fn.call_args.kwargs["force_reindex"] is False
def test_an_unknown_body_key_is_rejected_with_400(self, live_app):
"""``analyze_documents`` has no ``**kwargs``.
Without the allowlist an unknown key becomes a ``TypeError`` at call
time and surfaces as an opaque 500; with it, arbitrary body keys never
reach the function at all.
"""
client, _username = _api_user(live_app)
with patch(ANALYZE_DOCUMENTS_TARGET) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(
client,
ANALYZE_DOCUMENTS_PATH,
{
"query": QUERY,
"collection_name": COLLECTION,
"not_a_real_parameter": "x",
},
)
assert resp.status_code == 400, (
f"expected a 400, got {resp.status_code} / {resp.text[:300]}"
)
assert research_fn.call_count == 0
data = resp.json()
assert "not_a_real_parameter" in data["error"], (
"the error must name the offending key"
)
allowed = data["allowed_parameters"]
assert allowed == sorted(allowed) and allowed, allowed
assert "max_results" in allowed, (
"the advertised allowlist must be the real signature"
)
@pytest.mark.parametrize(
"injected,value",
[
("username", "victim"),
("settings_snapshot", {"llm.provider": "openai"}),
("programmatic_mode", True),
],
)
def test_server_set_identity_parameters_cannot_be_supplied_by_the_caller(
self, live_app, injected, value
):
"""Parameter injection against the per-user isolation boundary.
``username`` and ``settings_snapshot`` are derived by the server from
the authenticated session. If the body could supply them, a caller
could name the user whose stored settings and API credentials the run
executes with - and, via ``settings_snapshot``, hand itself a
permissive egress policy. They are excluded from the allowlist, so
they are rejected rather than honoured.
``programmatic_mode`` is different: it IS a genuine declared
parameter of ``analyze_documents``, so it would otherwise survive
the signature-derived allowlist untouched.
``_load_user_context_into_params`` sets it via ``setdefault``
specifically so authenticated REST calls default to DB-backed
metrics/rate-limit persistence, and ``setdefault`` means an
explicit body value would be respected rather than overridden -
letting a caller opt their own ``analyze_documents`` calls out of
the audit trail and DB-backed rate-limit accounting, the same gap
already closed for ``quick_summary``/``generate_report``. It must
be explicitly subtracted from ``_ANALYZE_DOCUMENTS_PARAMS`` (see
``api_v1.py``) rather than merely excluded like
``username``/``settings_snapshot`` above.
"""
client, _username = _api_user(live_app)
with patch(ANALYZE_DOCUMENTS_TARGET) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(
client,
ANALYZE_DOCUMENTS_PATH,
{
"query": QUERY,
"collection_name": COLLECTION,
injected: value,
},
)
assert resp.status_code == 400, (
f"{injected} in the body must be rejected, got "
f"{resp.status_code} / {resp.text[:300]}"
)
assert research_fn.call_count == 0, (
f"{injected} from the request body reached analyze_documents"
)
data = resp.json()
assert injected in data["error"]
assert injected not in data["allowed_parameters"], (
f"{injected} must never be advertised as caller-settable"
)
def test_output_file_is_rejected_like_the_other_two_endpoints(
self, live_app
):
"""``output_file`` IS a genuine declared parameter of
``analyze_documents`` (unlike ``username``/``settings_snapshot``
above), so - like ``programmatic_mode`` - it would otherwise
survive the signature-derived allowlist untouched.
``analyze_documents`` (research_functions.py) reaches the same
``write_file_verified`` -> bare ``open(filepath, mode)`` sink as
``generate_report`` (see ``_SERVER_PATH_PARAMS`` in api_v1.py),
behind the same unset-by-default ``api.allow_file_output`` gate and
the same opaque catch-all 500 when that gate is closed. A prior
version of this module reported that gap without closing it,
reasoning it was "separately-scoped"; that reasoning did not
survive review - the fix is the same one-token subtraction from
``_ANALYZE_DOCUMENTS_PARAMS`` already used for
``programmatic_mode``. Regression test for that fix.
"""
client, _username = _api_user(live_app)
with patch(ANALYZE_DOCUMENTS_TARGET) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(
client,
ANALYZE_DOCUMENTS_PATH,
{
"query": QUERY,
"collection_name": COLLECTION,
"output_file": "/tmp/pwned-by-analyze-documents-test.md",
},
)
assert resp.status_code == 400, (
f"output_file in the body must be rejected, got "
f"{resp.status_code} / {resp.text[:300]}"
)
assert research_fn.call_count == 0, (
"output_file from the request body reached analyze_documents"
)
data = resp.json()
assert "output_file" in data["error"]
assert "output_file" not in data["allowed_parameters"], (
"output_file must never be advertised as caller-settable on "
"analyze_documents"
)
class TestErrorScrubWiring:
"""Coverage area 3: the endpoint invokes the CWE-209 scrub."""
@pytest.mark.parametrize(
"label,path,body,target",
[
(
"quick_summary",
QUICK_SUMMARY_PATH,
{"query": QUERY},
QUICK_SUMMARY_TARGET,
),
(
"generate_report",
GENERATE_REPORT_PATH,
{"query": QUERY},
GENERATE_REPORT_TARGET,
),
(
"analyze_documents",
ANALYZE_DOCUMENTS_PATH,
{"query": QUERY, "collection_name": COLLECTION},
ANALYZE_DOCUMENTS_TARGET,
),
],
ids=_ENDPOINT_IDS,
)
def test_exception_text_returned_by_the_research_call_leaves_scrubbed(
self, live_app, label, path, body, target
):
"""Credentials planted below the endpoint must not reach the client.
Strategies surface a failed provider call as ``"Error: ..."`` in the
result payload, which is how an API key or DB password ends up in an
HTTP 200 body. The helper that removes it is unit-tested; this proves
the endpoint calls it on the object it actually returns.
"""
client, _username = _api_user(live_app)
leaked = (
"Error: LLM call failed: "
f"https://api.example.com/v1?api_key={_PLANTED_CREDENTIAL}"
)
payload = {
"research_id": _BENIGN_MARKER,
"content": "report body",
"summary": leaked,
"current_knowledge": leaked,
"formatted_findings": leaked,
"findings": [
{
"phase": "Error",
"content": (
"Error: connection to "
f"https://db.example.com:5432/db?password={_PLANTED_DB_PASSWORD}"
),
}
],
}
with patch(target) as research_fn:
research_fn.return_value = payload
resp = _post(client, path, body)
# Positive control: the response really is the payload above, at 200.
# Without this, both "not in" assertions would pass on a 500 or an
# empty body - the classic vacuous scrub test.
assert resp.status_code == 200, f"{label}: {resp.text[:400]}"
data = resp.json()
assert data["research_id"] == _BENIGN_MARKER, (
f"{label}: the mocked payload did not reach the client"
)
assert data["summary"].startswith("Error:"), (
f"{label}: the error marker must survive the scrub so clients can "
f"still classify the failure"
)
assert _PLANTED_CREDENTIAL not in resp.text, (
f"{label}: the API key reached the client - _scrub_error_fields "
f"is not wired into this endpoint"
)
assert _PLANTED_DB_PASSWORD not in resp.text, (
f"{label}: findings[].content is not scrubbed at this endpoint"
)
@pytest.mark.parametrize(
"label,path,body,target",
[
(
"quick_summary",
QUICK_SUMMARY_PATH,
{"query": QUERY},
QUICK_SUMMARY_TARGET,
),
(
"generate_report",
GENERATE_REPORT_PATH,
{"query": QUERY},
GENERATE_REPORT_TARGET,
),
(
"analyze_documents",
ANALYZE_DOCUMENTS_PATH,
{"query": QUERY, "collection_name": COLLECTION},
ANALYZE_DOCUMENTS_TARGET,
),
],
ids=_ENDPOINT_IDS,
)
def test_an_unhandled_exception_below_the_endpoint_does_not_leak(
self, live_app, label, path, body, target
):
"""The 500 arm returns a fixed message, never the exception text."""
client, _username = _api_user(live_app)
# Positive control: the same request succeeds when nothing raises, so
# "the credential is absent" is not just "the route is unreachable".
with patch(target) as research_fn:
research_fn.return_value = {"research_id": _BENIGN_MARKER}
ok = _post(client, path, body)
assert ok.status_code == 200, f"{label}: {ok.text[:400]}"
assert ok.json()["research_id"] == _BENIGN_MARKER
with patch(target) as research_fn:
research_fn.side_effect = RuntimeError(
f"provider auth failed with key {_PLANTED_CREDENTIAL}"
)
resp = _post(client, path, body)
assert resp.status_code == 500, (
f"{label}: expected the generic 500 arm, got {resp.status_code}"
)
assert resp.json()["error"] == (
"An internal error has occurred. Please try again later."
)
assert _PLANTED_CREDENTIAL not in resp.text, (
f"{label}: the exception text reached the client"
)
class TestStrictEgressScopeRespectedOnRestPath:
"""The /api/v1 quick_summary/generate_report endpoints feed runtime
overrides through ``overlay_runtime_settings`` on the way to
``_init_search_system``. Under STRICT egress scope, that overlay
MUST NOT overwrite the saved primary ``search.tool`` — otherwise a
STRICT user with a saved ``library`` primary could widen the run to
``searxng`` purely by sending a JSON body.
"""
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
],
)
def test_search_tool_override_does_not_widen_run_under_strict(
self, live_app, label, path, target
):
client, username = _api_user(live_app)
_store_setting(username, "policy.egress_scope", "strict")
_store_setting(username, "search.tool", "library")
with patch(target) as research_fn:
research_fn.return_value = {"summary": "fine", "findings": []}
resp = _post(
client,
path,
{"query": QUERY, "search_tool": "searxng"},
)
assert resp.status_code == 200, f"{label}: {resp.text[:400]}"
forwarded_snapshot = research_fn.call_args.kwargs["settings_snapshot"]
# REST snapshot is flat (SettingsManager.get_settings_snapshot
# strips the ``{"value": ...}`` envelope) — the saved primary
# is just the string under ``search.tool``.
assert forwarded_snapshot["search.tool"] == "library", (
f"{label}: STRICT user must keep saved primary; got "
f"{forwarded_snapshot['search.tool']!r}"
)
@pytest.mark.parametrize(
"label,path,target",
[
("quick_summary", QUICK_SUMMARY_PATH, QUICK_SUMMARY_TARGET),
("generate_report", GENERATE_REPORT_PATH, GENERATE_REPORT_TARGET),
(
"analyze_documents",
ANALYZE_DOCUMENTS_PATH,
ANALYZE_DOCUMENTS_TARGET,
),
],
)
def test_strict_blank_saved_primary_fails_closed_as_400_not_500(
self, live_app, label, path, target
):
"""STRICT + blank saved primary + explicit engine on REST must be
a curated 400, not the generic 500.
The route-level ``except Exception`` used to swallow the
``PolicyDeniedError`` from ``_init_search_system``/factory before
the global handler could render its 400. Pins the
``except PolicyDeniedError`` arm in ``api_v1.py``.
"""
from local_deep_research.security.egress.policy import (
Decision,
PolicyDeniedError,
)
client, _username = _api_user(live_app)
body = {"query": QUERY, "search_tool": "searxng"}
if label == "analyze_documents":
# search_tool is not an analyze_documents param (unknown-param
# 400); the STRICT blank-primary denial there comes from
# get_llm/get_search, so pin the except arm with a valid body.
body = {"query": QUERY, "collection_name": COLLECTION}
with patch(target) as research_fn:
research_fn.side_effect = PolicyDeniedError(
Decision(False, "invalid_policy_config"),
target="searxng",
)
resp = _post(
client,
path,
body,
)
assert resp.status_code == 400, (
f"{label}: expected curated 400, got {resp.status_code} / "
f"{resp.text[:400]}"
)
assert "Egress policy refused this request" in resp.json()["error"]
assert "invalid_policy_config" in resp.json()["error"]