Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

84.8 KB
Raw Blame
"""Security coverage for the library / RAG surface lost in the Flask ->
FastAPI migration (PR #3299).

The migration deleted ``tests/research_library/`` (21 paths, 682 tests) while
porting all four blueprints 1:1 into ``web/routers/{library,rag,
library_delete,library_search}.py``. A provenance audit of those deletions
found five security-flagged behaviours that survived in ``src/`` but lost
every assertion that pinned them. This file re-pins exactly those five, at
the layer where the contract actually lives.

COVERAGE AREA 1 -- note-protection 403 on ``DELETE /library/api/document/{id}``.
    Guard: ``research_library/deletion/services/document_deletion.py``
    ``_delete_document_locked`` -> ``_is_note_document`` -> ``is_note: True``,
    mapped to 403 at ``web/routers/library_delete.py::delete_document``.
    At the ADR-0010 snapshot, no direct assertion covered this route: the
    branch's service test
    (``tests/research_library/deletion/services/test_document_deletion.py::
    TestDocumentDeletionServiceNoteRefusal``) covers only ``delete_blob_only``
    and ``remove_from_collection``, and its own docstring defers the
    ``delete_document`` case to the route suite this PR deleted. A regression
    lets a note Document be hard-deleted (and bulk-amplified) through the
    generic document API, bypassing NoteService's version/link cleanup.

COVERAGE AREA 2 -- the 403-vs-404 mapping on the two sibling delete routes,
    ``DELETE /library/api/document/{id}/blob`` and
    ``DELETE /library/api/collection/{cid}/document/{did}``. The service-level
    refusals were covered by the branch test named above; the status mapping
    was not. The tests below now pin that mapping because a refusal surfaced as
    404 instead of 403 is a different contract (the frontend uses the
    distinction to route the user to the notes API).

COVERAGE AREA 3 -- ``web/routers/rag.py::_format_test_embedding_error``.
    At the review snapshot it had no direct assertions (11 predecessor tests
    were deleted); this file now pins its branches. This is the CWE-209 error
    boundary on
    ``POST /library/api/rag/test-embedding``: the function's output goes
    straight into the response body. Three branches matter -- internal LDR,
    upstream provider, and unexpected failures. Every branch returns a fixed
    category message because exception text can carry filesystem paths, SQL,
    response bodies, or credentials.
    ``tests/web/routers/test_rag_hostile_input.py::
    test_test_embedding_does_not_leak_decoder_message`` covers only the
    malformed-body 400, which returns before this function is ever reached.

COVERAGE AREA 4 -- ``web/routers/rag.py::_sanitized_indexing_errors``. Its
output is
    stored in ``TaskMetadata.result_metadata`` by both terminal paths of
    ``_background_index_worker`` and read back by the index-status / SSE
    endpoints, i.e. it reaches the browser. Unscrubbed per-document errors are
    a disclosure; an unbounded list is a response-size problem.
    ``TestIndexingErrorAllowlistMatchesService`` checks the allowlist behind it
    against the error text ``LibraryRAGService`` returns, so a fixed message
    the service adds cannot be withheld unnoticed and an allowlisted one
    cannot outlive the code that returns it.
    ``TestIndexingTaskErrorPassThroughs`` does the same for the stored task
    error the index-status endpoint serves, against the text
    ``web/routers/rag.py`` writes to indexing task rows.

COVERAGE AREA 5 -- ``research_library/utils/is_downloadable_domain``. It had no
    direct coverage at the review snapshot; the tests below and
    ``tests/web/routers/test_library_download_outcomes.py`` now cover it. It is
    the
    allowlist gate deciding which remote URLs
    the server will fetch (``web/routers/library.py:879``, ``:1539`` and
    ``:1766``), so it is SSRF-adjacent. The tests below pin the policy the implementation and
    its docstring actually describe -- "academic domain OR direct PDF link" --
    and deliberately pin the remaining CURRENT-BEHAVIOUR weaknesses (plus the
    regressions closed by ``harden: reject private hosts in
    is_downloadable_domain, anchor the PubMed rule``) rather than inventing a
    stricter policy (see ``TestIsDownloadableDomainKnownWeaknesses``).
    ``TestIsDownloadableDomainPrivateHostNormalization`` separately pins the
    call-site-only IP-literal normalization (trailing dot, decimal/hex,
    shorthand-dotted, per-octet octal, RFC 6598 CGNAT) added on top of that
    hardening -- see that class's docstring for why the shared
    ``is_private_ip()`` helper itself is untouched.

Harness: the ``auth_client`` idiom from
``tests/web/routers/test_library_delete_hostile_input.py`` -- a real,
in-process ``TestClient(app, raise_server_exceptions=False)`` against the live
FastAPI app, a freshly registered and auto-authenticated throwaway user, a real
CSRF token (CSRF is ASGI-middleware-enforced, not a config flag), a per-test
``LDR_DATA_DIR``, and a distinct ``X-Forwarded-For`` so the per-IP limiter
cannot bucket these clients together. No network, no LLM.

Non-vacuity: every "the forbidden thing is absent" assertion is paired with a
positive assertion that the *expected* content is present. An empty library or
a 500 would otherwise satisfy "no filesystem path in the response" trivially.
"""

import os
import uuid
from contextlib import suppress
from unittest.mock import patch

import pytest
from fastapi.testclient import TestClient

# The app's docs_url toggle reads PYTEST_CURRENT_TEST at app-import time;
# pytest only sets that per-test, not at collection.
os.environ.setdefault("TESTING", "1")


# ---------------------------------------------------------------------------
# Harness
# ---------------------------------------------------------------------------


def _cleanup_client(client, username):
    try:
        with suppress(Exception):
            client.post("/auth/logout", follow_redirects=False)
        with suppress(Exception):
            from local_deep_research.web.auth.session_manager import (
                session_manager,
            )

            session_manager.destroy_all_user_sessions(username)
        with suppress(Exception):
            from local_deep_research.database.session_passwords import (
                session_password_store,
            )

            session_password_store.clear_all_for_user(username)
        with suppress(Exception):
            from local_deep_research.database.thread_local_session import (
                clear_user_credentials,
            )

            clear_user_credentials(username)
    finally:
        client.close()


@pytest.fixture
def library_client(temp_data_dir, monkeypatch, request):
    """Authenticated TestClient for a throwaway user against a per-test
    ``LDR_DATA_DIR``. Returns ``(client, username, password)`` so tests can
    open a direct DB session for the same user to seed rows and to verify
    post-conditions in the database rather than only in the response body.
    """
    monkeypatch.setenv("LDR_DATA_DIR", str(temp_data_dir))
    # Production PBKDF2 iteration count would dominate wall-clock in a
    # per-test fixture (same reason test_library_delete_hostile_input.py
    # lowers it).
    monkeypatch.setenv("LDR_DB_CONFIG_KDF_ITERATIONS", "1000")

    from local_deep_research.web.fastapi_app import app

    client = TestClient(app, raise_server_exceptions=False)
    username = f"test_lib_sec_{uuid.uuid4().hex[:8]}"
    password = "TestPassword123!"  # noqa: S105
    request.addfinalizer(lambda: _cleanup_client(client, username))
    # Distinct peer per client: the auth/registration routes are per-IP rate
    # limited, and several of these fixtures can run inside one session.
    client.headers.update(
        {"X-Forwarded-For": f"10.{uuid.uuid4().int % 254 + 1}.13.7"}
    )

    def _csrf():
        client.get("/auth/login")
        resp = client.get("/auth/csrf-token")
        return (
            resp.json().get("csrf_token", "") if resp.status_code == 200 else ""
        )

    reg = client.post(
        "/auth/register",
        data={
            "username": username,
            "password": password,
            "confirm_password": password,
            "acknowledge": "true",
            "csrf_token": _csrf(),
        },
        follow_redirects=False,
    )
    if reg.status_code != 302:
        pytest.fail(
            f"Auth bootstrap broken: registration returned {reg.status_code} "
            f"(expected 302): {reg.text[:300]}"
        )

    who = client.get("/auth/check")
    if (
        who.status_code != 200
        or who.json().get("authenticated") is not True
        or who.json().get("username") != username
    ):
        pytest.fail(
            "Auth bootstrap broken: registration did not establish the "
            f"expected session: {who.status_code} {who.text[:300]}"
        )

    csrf_resp = client.get("/auth/csrf-token")
    if csrf_resp.status_code == 200:
        token = csrf_resp.json().get("csrf_token")
        if token:
            client.headers.update({"X-CSRFToken": token})

    return client, username, password


def _seed_document(
    username, password, *, source_type_name, title, storage_mode
):
    """Insert one real ``Document`` row owned by *username*.

    ``source_type_name`` selects the ``SourceType`` seeded by
    ``initialize_library_for_user`` on register/login -- "note" produces a
    document that ``_is_note_document`` recognises, "user_upload" an ordinary
    one. Returns the new document id.
    """
    from local_deep_research.database.models.library import Document, SourceType
    from local_deep_research.database.session_context import get_user_db_session

    doc_id = str(uuid.uuid4())
    with get_user_db_session(username, password) as session:
        source_type = (
            session.query(SourceType).filter_by(name=source_type_name).first()
        )
        assert source_type is not None, (
            f"{source_type_name!r} SourceType should have been seeded by "
            "initialize_library_for_user on register/login"
        )
        session.add(
            Document(
                id=doc_id,
                source_type_id=source_type.id,
                document_hash=uuid.uuid4().hex,
                filename=f"{source_type_name}-fixture.md",
                file_size=11,
                file_type="md",
                text_content="hello world",
                title=title,
                storage_mode=storage_mode,
            )
        )
        session.commit()
    return doc_id


def _seed_collection(username, password, *, collection_type, name):
    """Insert one real ``Collection`` row and return its id."""
    from local_deep_research.database.models.library import Collection
    from local_deep_research.database.session_context import get_user_db_session

    collection_id = str(uuid.uuid4())
    with get_user_db_session(username, password) as session:
        session.add(
            Collection(
                id=collection_id, name=name, collection_type=collection_type
            )
        )
        session.commit()
    return collection_id


def _link(username, password, document_id, collection_id):
    """Link a document into a collection (the ``document_collections`` join)."""
    from local_deep_research.database.models.library import DocumentCollection
    from local_deep_research.database.session_context import get_user_db_session

    with get_user_db_session(username, password) as session:
        session.add(
            DocumentCollection(
                document_id=document_id, collection_id=collection_id
            )
        )
        session.commit()


def _document_exists(username, password, document_id):
    from local_deep_research.database.models.library import Document
    from local_deep_research.database.session_context import get_user_db_session

    with get_user_db_session(username, password) as session:
        return session.get(Document, document_id) is not None


def _link_exists(username, password, document_id, collection_id):
    from local_deep_research.database.models.library import DocumentCollection
    from local_deep_research.database.session_context import get_user_db_session

    with get_user_db_session(username, password) as session:
        return (
            session.query(DocumentCollection)
            .filter_by(document_id=document_id, collection_id=collection_id)
            .first()
            is not None
        )


# ===========================================================================
# COVERAGE AREA 1 -- DELETE /library/api/document/{id} note protection
# and the note must survive.
# ===========================================================================


class TestDeleteDocumentNoteRefusal:
    """``DELETE /library/api/document/{id}`` on a note Document.

    This is the direct route-level regression evidence. The guard lives in
    ``document_deletion.py::_delete_document_locked`` (``_is_note_document``
    -> ``{"is_note": True}``) and is mapped to 403 in
    ``library_delete.py::delete_document``. Deleting the guard, or downgrading
    the 403 to the generic 404 branch below it, flips these assertions.
    """

    def test_deleting_a_note_is_refused_with_403_and_the_note_survives(
        self, library_client
    ):
        client, username, password = library_client
        note_id = _seed_document(
            username,
            password,
            source_type_name="note",
            title="Protected note",
            storage_mode="database",
        )

        resp = client.delete(f"/library/api/document/{note_id}")

        assert resp.status_code == 403, (
            "note deletion via the generic document API must be refused with "
            f"403 (not 404, not 200): got {resp.status_code}: {resp.text[:300]}"
        )
        body = resp.json()
        assert body["success"] is False
        assert body["deleted"] is False
        assert body["is_note"] is True, (
            f"response must carry the is_note discriminator: {body!r}"
        )
        assert "/api/notes/" in body["error"], (
            f"refusal must route the caller to the notes API: {body!r}"
        )

        # The point of the guard: the row is still there.
        assert _document_exists(username, password, note_id), (
            "the note Document must still exist after the refused delete"
        )

    def test_bulk_delete_cannot_amplify_past_the_note_guard(
        self, library_client
    ):
        """The bulk endpoint loops over the same service method, so the guard
        (not the route's 403 mapping) is what protects it. Bulk returns 200
        with a per-item failure -- the note must still be in the database.
        """
        client, username, password = library_client
        note_id = _seed_document(
            username,
            password,
            source_type_name="note",
            title="Protected note",
            storage_mode="database",
        )

        resp = client.request(
            "DELETE",
            "/library/api/documents/bulk",
            json={"document_ids": [note_id]},
        )

        assert resp.status_code == 200, resp.text
        body = resp.json()
        assert body["deleted"] == 0, (
            f"the note must not be counted as deleted: {body!r}"
        )
        assert _document_exists(username, password, note_id), (
            "the note Document must survive the bulk delete path too"
        )

    def test_an_ordinary_document_is_still_deleted_with_200(
        self, library_client
    ):
        """Positive control (non-vacuity). Proves the 403 above is the note
        guard firing and not a blanket refusal, a broken harness, or an empty
        library: the same route, same client, same request shape deletes a
        non-note document and the row really goes away.
        """
        client, username, password = library_client
        doc_id = _seed_document(
            username,
            password,
            source_type_name="user_upload",
            title="Ordinary document",
            storage_mode="database",
        )

        resp = client.delete(f"/library/api/document/{doc_id}")

        assert resp.status_code == 200, resp.text
        body = resp.json()
        assert body["success"] is True
        assert body["deleted"] is True
        assert "is_note" not in body
        assert not _document_exists(username, password, doc_id), (
            "an ordinary document must actually be gone from the DB"
        )


# ===========================================================================
# COVERAGE AREA 2 -- 403-vs-404 mapping on sibling delete routes
# ===========================================================================


class TestSiblingDeleteRoutes403VsNotFound:
    """The refusals themselves are covered at the service layer; what is
    untested is that ``library_delete.py`` maps them to 403 and NOT to the
    404/400 branch sitting immediately below in the same handler. Each test
    pairs the 403 case with the 404 case on the same route so a mapping that
    collapsed the two would fail.
    """

    def test_blob_delete_of_a_note_returns_403_not_404(self, library_client):
        client, username, password = library_client
        note_id = _seed_document(
            username,
            password,
            source_type_name="note",
            title="Protected note",
            storage_mode="database",
        )

        resp = client.delete(f"/library/api/document/{note_id}/blob")

        assert resp.status_code == 403, (
            "a note blob-delete refusal must map to 403, not the sibling "
            f"404/400 branch: got {resp.status_code}: {resp.text[:300]}"
        )
        body = resp.json()
        assert body["success"] is False
        assert body["is_note"] is True
        assert body["bytes_freed"] == 0

        assert _document_exists(username, password, note_id)

    def test_blob_delete_of_an_unknown_document_returns_404(
        self, library_client
    ):
        """Discriminator for the test above: the same route's not-found case
        must stay 404, so "everything is 403" would not pass.
        """
        client, _, _ = library_client

        resp = client.delete("/library/api/document/no-such-document/blob")

        assert resp.status_code == 404, resp.text
        body = resp.json()
        assert body["success"] is False
        assert "not found" in body["error"].lower()
        assert "is_note" not in body

    def test_blob_delete_of_an_ordinary_document_returns_200(
        self, library_client
    ):
        """Positive control: the 403 and 404 above are real branch outcomes,
        not a route that refuses everything.
        """
        client, username, password = library_client
        doc_id = _seed_document(
            username,
            password,
            source_type_name="user_upload",
            title="Ordinary document",
            storage_mode="database",
        )

        resp = client.delete(f"/library/api/document/{doc_id}/blob")

        assert resp.status_code == 200, resp.text
        body = resp.json()
        assert body["success"] is True
        assert body["deleted"] is True

    def test_remove_note_from_its_notes_collection_returns_403_not_404(
        self, library_client
    ):
        client, username, password = library_client
        note_id = _seed_document(
            username,
            password,
            source_type_name="note",
            title="Protected note",
            storage_mode="database",
        )
        notes_collection_id = _seed_collection(
            username, password, collection_type="notes", name="Notes"
        )
        _link(username, password, note_id, notes_collection_id)

        resp = client.delete(
            f"/library/api/collection/{notes_collection_id}/document/{note_id}"
        )

        assert resp.status_code == 403, (
            "unlinking a note from its permanent notes-collection home must "
            f"map to 403, not 404: got {resp.status_code}: {resp.text[:300]}"
        )
        body = resp.json()
        assert body["success"] is False
        assert body["unlinked"] is False
        assert body["protected"] is True, (
            f"response must carry the protected discriminator: {body!r}"
        )

        # Both the link and the note itself must survive.
        assert _link_exists(username, password, note_id, notes_collection_id)
        assert _document_exists(username, password, note_id)

    def test_remove_unknown_document_from_collection_returns_404(
        self, library_client
    ):
        """Discriminator for the test above."""
        client, username, password = library_client
        notes_collection_id = _seed_collection(
            username, password, collection_type="notes", name="Notes"
        )

        resp = client.delete(
            f"/library/api/collection/{notes_collection_id}/document/nope"
        )

        assert resp.status_code == 404, resp.text
        body = resp.json()
        assert body["success"] is False
        assert "not found" in body["error"].lower()
        assert "protected" not in body

    def test_remove_ordinary_document_from_collection_returns_200(
        self, library_client
    ):
        """Positive control for the two above: the route can and does return
        200 for a permitted unlink.
        """
        client, username, password = library_client
        doc_id = _seed_document(
            username,
            password,
            source_type_name="user_upload",
            title="Ordinary document",
            storage_mode="database",
        )
        collection_id = _seed_collection(
            username,
            password,
            collection_type="user_collection",
            name="Ordinary collection",
        )
        _link(username, password, doc_id, collection_id)

        resp = client.delete(
            f"/library/api/collection/{collection_id}/document/{doc_id}"
        )

        assert resp.status_code == 200, resp.text
        body = resp.json()
        assert body["success"] is True
        assert body["unlinked"] is True
        assert not _link_exists(username, password, doc_id, collection_id)


# ===========================================================================
# COVERAGE AREA 3 -- _format_test_embedding_error (CWE-209)
# ===========================================================================


# A realistic OpenAI-style key. Written split so the literal is not itself a
# credential-looking blob in the repo, and long enough to trip the
# ``sk-[A-Za-z0-9-]{20,}`` pattern in security/log_sanitizer.py.
_FAKE_API_KEY = "sk-proj-" + "A1b2C3d4E5f6G7h8" * 3
# A server-side filesystem path of the shape internal LDR exceptions really
# carry (encrypted per-user database files).
_SERVER_PATH = "/srv/ldr/data/encrypted_databases/victim_user.db"


def _internal_exception(message):
    """A genuine LDR-internal exception (``type(exc).__module__`` starts with
    ``local_deep_research``), which is what the suppression branch keys on.
    """
    from local_deep_research.config.thread_settings import (
        NoSettingsContextError,
    )

    return NoSettingsContextError(message)


def _upstream_exception(message):
    """A genuine ``openai``-module exception, matching
    ``_UPSTREAM_MODULE_PREFIXES``.
    """
    import httpx
    import openai

    return openai.AuthenticationError(
        message,
        response=httpx.Response(
            401, request=httpx.Request("GET", "http://provider.invalid/v1")
        ),
        body=None,
    )


class TestFormatTestEmbeddingErrorUnit:
    """Direct coverage of ``rag.py::_format_test_embedding_error``. Nothing on
    the branch referenced this function at all.
    """

    def test_internal_exception_detail_is_suppressed_entirely(self):
        """The rag.py:334-338 branch. Its comment states the reason outright:
        internal exceptions "can carry filesystem paths / SQL fragments that
        sanitize_error_message() does not pattern-match", so their detail is
        withheld from the browser. The exception below carries exactly such a
        path.
        """
        from local_deep_research.web.routers.rag import (
            _format_test_embedding_error,
        )

        message = _format_test_embedding_error(
            _internal_exception(f"could not open {_SERVER_PATH}"),
            "nomic-embed-text",
        )

        # Positive first: the caller still gets a useful, correctly
        # categorised message (so the absence checks below are not vacuous).
        assert "internal LDR error" in message
        assert "NoSettingsContextError" not in message
        assert "nomic-embed-text" in message
        assert "report it on GitHub" in message
        # And the detail -- including the server filesystem path -- is gone.
        assert _SERVER_PATH not in message
        assert "encrypted_databases" not in message
        assert "could not open" not in message

    def test_internal_exception_does_not_leak_a_key_either(self):
        from local_deep_research.web.routers.rag import (
            _format_test_embedding_error,
        )

        message = _format_test_embedding_error(
            _internal_exception(f"auth header was {_FAKE_API_KEY}"), "m"
        )

        assert "internal LDR error" in message
        assert _FAKE_API_KEY not in message

    def test_upstream_provider_error_withholds_exception_text(self):
        """Provider failures retain guidance without reflecting detail."""
        from local_deep_research.web.routers.rag import (
            _format_test_embedding_error,
        )

        message = _format_test_embedding_error(
            _upstream_exception(f"Incorrect API key provided: {_FAKE_API_KEY}"),
            "text-embedding-3-small",
        )

        assert "The request to the provider failed" in message
        assert "text-embedding-3-small" in message
        assert "Incorrect API key provided" not in message
        assert _FAKE_API_KEY not in message
        assert "[REDACTED_KEY]" not in message

    def test_stdlib_exception_no_longer_echoes_verbatim_detail(self):
        """The final fallback. An exception from neither the internal nor an
        upstream module prefix now gets a fixed category message -- neither
        ``str(exc)`` nor exception type metadata is echoed.

        HARDENED for CodeQL alert 8001 (CWE-209). This branch previously
        returned ``f"Embedding test failed for model '{model}': {detail}"``
        where ``detail = sanitize_error_message(str(exc))`` -- credential
        shapes only, no path/SQL suppression and no length cap. Full leak-path
        coverage now lives in
        ``tests/web/routers/test_rag_embedding_error_sanitisation.py``.
        """
        from local_deep_research.web.routers.rag import (
            _format_test_embedding_error,
        )

        message = _format_test_embedding_error(
            RuntimeError("Connection refused by embedding backend"), "m"
        )

        # Positive: still correctly categorised and names the model.
        assert "Embedding test failed for model 'm'" in message
        assert "unexpected error" in message
        assert "RuntimeError" not in message
        # Negative: the exception text itself no longer reaches the caller.
        assert "Connection refused by embedding backend" not in message
        # Not miscategorised as an LDR bug -- the #4208 regression.
        assert "bug in LDR" not in message
        assert "The request to the provider failed" not in message

    def test_stdlib_exception_branch_withholds_the_detail_entirely(self):
        """Credential redaction used to be the only mitigation on this branch.
        Now the detail is withheld outright, so a credential in a stdlib
        exception cannot reach the browser even if a scrubber gap let its
        shape through.
        """
        from local_deep_research.web.routers.rag import (
            _format_test_embedding_error,
        )

        message = _format_test_embedding_error(
            RuntimeError(f"auth failed for {_FAKE_API_KEY}"), "m"
        )

        assert "unexpected error" in message
        assert "RuntimeError" not in message
        assert "auth failed for" not in message
        assert _FAKE_API_KEY not in message

    def test_stdlib_exception_branch_no_longer_carries_a_server_path(self):
        """The hardening the previous version of this test anticipated.

        ``sanitize_error_message`` is a credential-shape scrubber and does not
        touch filesystem paths, so a stdlib ``OSError``/``FileNotFoundError``
        raised anywhere under ``get_embedding_function`` used to put a server
        path in the HTTP 500 body -- exactly the disclosure the
        internal-module branch exists to prevent, one branch further down.
        The fallback now returns a fixed category message.
        """
        from local_deep_research.web.routers.rag import (
            _format_test_embedding_error,
        )

        message = _format_test_embedding_error(
            FileNotFoundError(f"No such file or directory: {_SERVER_PATH}"),
            "m",
        )

        # Positive control first.
        assert "Embedding test failed for model 'm'" in message
        assert "unexpected error" in message
        assert "FileNotFoundError" not in message
        # The disclosure.
        assert _SERVER_PATH not in message
        assert "encrypted_databases" not in message
        assert "No such file or directory" not in message

    def test_empty_exception_message_still_returns_a_category(self):
        """An exception with no message still produces useful guidance."""
        from local_deep_research.web.routers.rag import (
            _format_test_embedding_error,
        )

        message = _format_test_embedding_error(ValueError(""), "m")

        assert "Embedding test failed for model 'm'" in message
        assert "unexpected error" in message
        assert "ValueError" not in message

    def test_module_prefix_match_is_boundary_anchored(self):
        """``_module_matches`` must not treat ``local_deep_research_evil`` (or
        ``openai_shim``) as the real prefix -- a substring match here would
        misroute an unrelated third-party exception into either the
        suppression branch or the "provider error" wording.
        """
        from local_deep_research.web.routers.rag import _module_matches

        assert _module_matches("local_deep_research", "local_deep_research")
        assert _module_matches(
            "local_deep_research.config.thread_settings", "local_deep_research"
        )
        assert not _module_matches(
            "local_deep_research_evil", "local_deep_research"
        )
        assert not _module_matches("openai_shim", "openai")
        assert not _module_matches("", "local_deep_research")


class TestTestEmbeddingEndpointDoesNotLeak:
    """End-to-end proof over real HTTP that the categorisation above is what
    the browser actually receives from
    ``POST /library/api/rag/test-embedding``. The deleted suite's
    ``TestTestEmbeddingErrorCategorization`` had no HTTP half at the historical
    snapshot; the branch's ``test_rag_hostile_input.py`` covered only the
    malformed-body 400, which returns before ``_format_test_embedding_error``
    is reached. This class now pins the HTTP wiring.

    ``get_embeddings`` (not ``get_embedding_function``) is patched at its
    definition site (``embeddings.embeddings_config``) because the route
    imports it inside the handler body, so the patch is picked up per
    request. The route calls ``get_embeddings`` directly -- rather than the
    ``get_embedding_function`` wrapper -- so the returned instance is
    reachable for ``_close_base_llm`` cleanup after the test embed; see
    ``local_deep_research.web.routers.rag.test_embedding``. Nothing else is
    mocked: real auth, real CSRF, real settings load, real response.
    """

    _PATCH_TARGET = (
        "local_deep_research.embeddings.embeddings_config.get_embeddings"
    )

    def test_internal_error_does_not_put_a_server_path_in_the_response(
        self, library_client
    ):
        client, _, _ = library_client

        with patch(
            self._PATCH_TARGET,
            side_effect=_internal_exception(f"could not open {_SERVER_PATH}"),
        ):
            resp = client.post(
                "/library/api/rag/test-embedding",
                json={"provider": "ollama", "model": "nomic-embed-text"},
            )

        assert resp.status_code == 500, resp.text
        body = resp.json()
        assert body["success"] is False
        # Positive control FIRST: the response really is the formatted
        # internal-error message, so the absence assertions below cannot be
        # satisfied by an empty body, a 401, or an unrelated error page.
        assert "internal LDR error" in body["error"], (
            f"expected the internal-error wording, got: {body!r}"
        )
        assert "NoSettingsContextError" not in body["error"]
        # Now the disclosure check.
        assert _SERVER_PATH not in resp.text
        assert "encrypted_databases" not in resp.text
        assert "could not open" not in resp.text

    def test_provider_error_reaches_the_browser_without_exception_text(
        self, library_client
    ):
        client, _, _ = library_client

        with patch(
            self._PATCH_TARGET,
            side_effect=_upstream_exception(
                f"Incorrect API key provided: {_FAKE_API_KEY}"
            ),
        ):
            resp = client.post(
                "/library/api/rag/test-embedding",
                json={
                    "provider": "openai",
                    "model": "text-embedding-3-small",
                },
            )

        assert resp.status_code == 500, resp.text
        body = resp.json()
        assert "The request to the provider failed" in body["error"]
        assert "Incorrect API key provided" not in body["error"]
        assert _FAKE_API_KEY not in resp.text
        assert "[REDACTED_KEY]" not in body["error"]

    def test_successful_embedding_still_returns_200(self, library_client):
        """Non-vacuity control for the two tests above: with the same patch
        seam returning a working embedding function, the route reaches its
        success path. This proves the 500s above come from the exception
        under test and not from the route failing earlier (settings load,
        auth, threadpool) for unrelated reasons.
        """
        client, _, _ = library_client

        class _FakeEmbeddings:
            def embed_documents(self, texts):
                return [[0.1, 0.2, 0.3]]

        with patch(self._PATCH_TARGET, return_value=_FakeEmbeddings()):
            resp = client.post(
                "/library/api/rag/test-embedding",
                json={"provider": "ollama", "model": "nomic-embed-text"},
            )

        assert resp.status_code == 200, resp.text
        body = resp.json()
        assert body["success"] is True
        assert body["dimension"] == 3


# ===========================================================================
# COVERAGE AREA 4 -- _sanitized_indexing_errors and the 50-item bound
# ===========================================================================


class TestSanitizedIndexingErrors:
    """``rag.py::_sanitized_indexing_errors`` is the client-message gate
    for the per-document error list that both terminal paths of
    ``_background_index_worker`` write into ``TaskMetadata.result_metadata``,
    which the index-status / SSE endpoints hand to the browser. The tests below
    pin its redaction and length-bound contracts.
    """

    def test_exception_detail_is_replaced_without_losing_document_identity(
        self,
    ):
        from local_deep_research.web.routers.rag import (
            _sanitized_indexing_errors,
        )

        out = _sanitized_indexing_errors(
            {
                "errors": [
                    {
                        "doc_id": "doc-1",
                        "title": "Quarterly report",
                        "error": f"embedding call rejected key {_FAKE_API_KEY}",
                    }
                ]
            }
        )

        assert len(out) == 1
        entry = out[0]
        # The UI still identifies the failed document.
        assert entry["doc_id"] == "doc-1"
        assert entry["title"] == "Quarterly report"
        assert (
            entry["error"]
            == "Indexing failed. Check the server logs for details."
        )
        assert _FAKE_API_KEY not in entry["error"]

    def test_path_sql_and_provider_detail_are_not_returned(self):
        from local_deep_research.web.routers.rag import (
            _sanitized_indexing_errors,
        )

        out = _sanitized_indexing_errors(
            {
                "errors": [
                    {
                        "doc_id": "doc-1",
                        "error": (
                            "DB at /srv/private/user.db failed [SQL: SELECT * "
                            "FROM secrets] via http://internal.example:11434"
                        ),
                    }
                ]
            }
        )
        assert (
            out[0]["error"]
            == "Indexing failed. Check the server logs for details."
        )

    def test_output_is_bounded_to_fifty_entries_by_default(self):
        """A failed index of a large collection can produce one error per
        document. The list is embedded in task metadata that is serialised
        into an index-status response, so the bound is what keeps a failed
        1000-document job from becoming a multi-megabyte response.
        """
        from local_deep_research.web.routers.rag import (
            _sanitized_indexing_errors,
        )

        results = {
            "errors": [
                {"doc_id": f"doc-{i}", "title": f"T{i}", "error": f"boom {i}"}
                for i in range(500)
            ]
        }

        out = _sanitized_indexing_errors(results)

        assert len(out) == 50, (
            f"the per-document error list must be bounded, got {len(out)}"
        )
        # It is the FIRST 50, in order -- not a random or trailing slice.
        assert out[0]["doc_id"] == "doc-0"
        assert out[-1]["doc_id"] == "doc-49"
        assert all(
            entry["error"]
            == "Indexing failed. Check the server logs for details."
            for entry in out
        )

    def test_limit_is_overridable_and_still_applied(self):
        from local_deep_research.web.routers.rag import (
            _sanitized_indexing_errors,
        )

        results = {"errors": [{"doc_id": str(i)} for i in range(20)]}

        assert len(_sanitized_indexing_errors(results, limit=3)) == 3
        assert len(_sanitized_indexing_errors(results, limit=0)) == 0

    def test_shorter_lists_are_passed_through_unbounded(self):
        """Non-vacuity control for the bound: a list under the limit must not
        be truncated, so "always returns <= 50" is not achieved by returning
        nothing.
        """
        from local_deep_research.web.routers.rag import (
            _sanitized_indexing_errors,
        )

        results = {
            "errors": [
                {"doc_id": f"doc-{i}", "title": None, "error": "boom"}
                for i in range(7)
            ]
        }

        assert len(_sanitized_indexing_errors(results)) == 7

    def test_missing_or_empty_error_fields_get_a_neutral_placeholder(self):
        """``str(item.get("error") or "Indexing failed")`` -- a missing/None/
        empty error must not serialise as the literal string "None".
        """
        from local_deep_research.web.routers.rag import (
            _sanitized_indexing_errors,
        )

        out = _sanitized_indexing_errors(
            {
                "errors": [
                    {"doc_id": "a"},
                    {"doc_id": "b", "error": None},
                    {"doc_id": "c", "error": ""},
                ]
            }
        )

        assert [entry["error"] for entry in out] == ["Indexing failed"] * 3
        assert [entry["doc_id"] for entry in out] == ["a", "b", "c"]
        # doc_id/title are always present keys, even when the input omitted
        # them -- the index-status consumer indexes them directly.
        assert all(entry["title"] is None for entry in out)

    def test_no_errors_key_yields_an_empty_list(self):
        from local_deep_research.web.routers.rag import (
            _sanitized_indexing_errors,
        )

        assert _sanitized_indexing_errors({}) == []
        assert _sanitized_indexing_errors({"errors": []}) == []

    def test_non_string_error_values_use_the_generic_message(self):
        """An exception object must never be rendered for the client."""
        from local_deep_research.web.routers.rag import (
            _sanitized_indexing_errors,
        )

        out = _sanitized_indexing_errors(
            {
                "errors": [
                    {
                        "doc_id": "x",
                        "error": RuntimeError(f"bad key {_FAKE_API_KEY}"),
                    }
                ]
            }
        )

        assert (
            out[0]["error"]
            == "Indexing failed. Check the server logs for details."
        )
        assert _FAKE_API_KEY not in out[0]["error"]

    def test_only_known_service_messages_are_preserved(self):
        from local_deep_research.web.routers.rag import (
            _sanitized_indexing_errors,
        )

        out = _sanitized_indexing_errors(
            {
                "errors": [
                    {"error": "Document deleted during indexing"},
                    {"error": "Document deleted during indexing /srv/private"},
                ]
            }
        )
        assert out[0]["error"] == "Document deleted during indexing"
        assert (
            out[1]["error"]
            == "Indexing failed. Check the server logs for details."
        )


#: Text ``LibraryRAGService`` returns as a result's ``"error"`` that clients
#: must not see verbatim: ``_client_indexing_error`` gives them the generic
#: message instead. ``{}`` marks each f-string placeholder.
_WITHHELD_SERVICE_ERRORS = frozenset(
    {
        "Operation failed: {}",
        "Operation failed: VectorStoreIntegrityError",
        "Indexing failed: {}",
        "Worker crashed: {}",
        "Indexing must run outside an existing database session",
    }
)

#: ``"error"`` values that copy the error of a result the service's own
#: methods returned, as ``(enclosing function, source)``. The text they copy
#: is classified by ``_SAFE_INDEXING_ERRORS`` and the set above.
_FORWARDED_SERVICE_ERRORS = frozenset(
    {
        ("index_all_documents", "result.get('error')"),
        ("index_documents_parallel", "result.get('error')"),
        ("index_documents_parallel", "scope_error['error']"),
    }
)

#: Reconciliation reasons the service returns that clients must not see
#: verbatim. The status endpoint shows only ``_EMPTY_VECTOR_STORE_REASON``.
_WITHHELD_SERVICE_REASONS = frozenset()


def _library_rag_service_values(key):
    """Every value the ``LibraryRAGService`` module builds under ``key``.

    A value counts when it is a dict-literal entry, a ``dict(...)`` or
    ``.update(...)`` keyword, or an ``x[key] = ...`` assignment; a
    conditional expression counts as both of its branches. Returns
    ``(texts, others)``: ``texts`` holds each string and f-string, with
    ``{}`` for each placeholder, and ``others`` holds every other value as
    ``(enclosing function, source)``.
    """
    import ast
    import inspect

    from local_deep_research.research_library.services import (
        library_rag_service,
    )

    texts, others = set(), set()

    def record(value, function):
        if isinstance(value, ast.IfExp):
            record(value.body, function)
            record(value.orelse, function)
        elif isinstance(value, ast.Constant) and isinstance(value.value, str):
            texts.add(value.value)
        elif isinstance(value, ast.JoinedStr):
            texts.add(
                "".join(
                    part.value if isinstance(part, ast.Constant) else "{}"
                    for part in value.values
                )
            )
        else:
            others.add((function, ast.unparse(value)))

    def visit(node, function):
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
            function = node.name
        if isinstance(node, ast.Dict):
            for dict_key, value in zip(node.keys, node.values):
                if isinstance(dict_key, ast.Constant) and dict_key.value == key:
                    record(value, function)
        elif isinstance(node, ast.Call) and (
            getattr(node.func, "id", None) == "dict"
            or getattr(node.func, "attr", None) == "update"
        ):
            for keyword in node.keywords:
                if keyword.arg == key:
                    record(keyword.value, function)
        elif (
            isinstance(node, (ast.Assign, ast.AnnAssign))
            and node.value is not None
        ):
            targets = (
                node.targets if isinstance(node, ast.Assign) else [node.target]
            )
            for target in targets:
                if (
                    isinstance(target, ast.Subscript)
                    and isinstance(target.slice, ast.Constant)
                    and target.slice.value == key
                ):
                    record(node.value, function)
        for child in ast.iter_child_nodes(node):
            visit(child, function)

    visit(ast.parse(inspect.getsource(library_rag_service)), "<module>")
    return texts, others


class TestIndexingErrorAllowlistMatchesService:
    """``_SAFE_INDEXING_ERRORS`` must match the error text that
    ``LibraryRAGService`` returns.

    The RAG document index and remove routes, the note index route, both
    indexing SSE streams and the index-status endpoint pass each
    per-document ``"error"`` through ``_client_indexing_error``. An exact
    match in ``_SAFE_INDEXING_ERRORS`` reaches the client, and any other text
    becomes the generic message. Every such value comes from
    ``LibraryRAGService``, so these tests collect the values its module
    builds under ``"error"``, in the forms ``_library_rag_service_values``
    lists. Each string or f-string must be allowlisted or
    listed in ``_WITHHELD_SERVICE_ERRORS``, and each allowlisted string must
    still be returned. Reconciliation ``"reason"`` values get the same check
    against ``_EMPTY_VECTOR_STORE_REASON``, the one reason the status
    endpoint shows. New or reworded text fails here until someone decides
    whether clients may see it.
    """

    def test_every_error_the_service_returns_is_classified(self):
        from local_deep_research.web.routers.rag import _SAFE_INDEXING_ERRORS

        texts, others = _library_rag_service_values("error")

        unclassified = sorted(
            texts - _SAFE_INDEXING_ERRORS - _WITHHELD_SERVICE_ERRORS
        )
        assert unclassified == [], (
            f"LibraryRAGService returns unclassified error text {unclassified}."
            " Add fixed guidance a user can act on to _SAFE_INDEXING_ERRORS"
            " in web/routers/rag.py, and anything else to"
            " _WITHHELD_SERVICE_ERRORS in this file."
        )
        assert others == _FORWARDED_SERVICE_ERRORS, (
            "LibraryRAGService builds an error value that is neither text nor"
            f" a known copy of another result's error: {sorted(others)}"
        )

    def test_every_classified_error_is_still_returned(self):
        """An allowlisted string that nothing returns hides drift: a test
        that pins it passes while the service's real message is withheld."""
        from local_deep_research.web.routers.rag import _SAFE_INDEXING_ERRORS

        texts, _ = _library_rag_service_values("error")

        assert not _SAFE_INDEXING_ERRORS & _WITHHELD_SERVICE_ERRORS
        assert sorted(_SAFE_INDEXING_ERRORS - texts) == [], (
            "allowlisted error text that LibraryRAGService no longer returns"
        )
        assert sorted(_WITHHELD_SERVICE_ERRORS - texts) == [], (
            "withheld error text that LibraryRAGService no longer returns"
        )

    def test_every_reconciliation_reason_is_classified(self):
        from local_deep_research.web.routers.rag import (
            _EMPTY_VECTOR_STORE_REASON,
        )

        texts, others = _library_rag_service_values("reason")

        unclassified = sorted(
            texts - {_EMPTY_VECTOR_STORE_REASON} - _WITHHELD_SERVICE_REASONS
        )
        assert unclassified == [], (
            f"LibraryRAGService returns unclassified reason {unclassified}."
            " Make it _EMPTY_VECTOR_STORE_REASON in web/routers/rag.py if"
            " clients should see it, or add it to _WITHHELD_SERVICE_REASONS"
            " in this file."
        )
        assert others == set()
        assert _EMPTY_VECTOR_STORE_REASON in texts
        assert _WITHHELD_SERVICE_REASONS <= texts

    def test_clients_receive_each_text_as_classified(self):
        from local_deep_research.web.routers.rag import (
            _EMPTY_VECTOR_STORE_REASON,
            _SAFE_INDEXING_ERRORS,
            _client_indexing_error,
            _client_reconciliation_reason,
        )

        for text in _SAFE_INDEXING_ERRORS:
            assert _client_indexing_error(text) == text
            for detail in (
                f"{text}: /srv/private.db",
                f"/srv/private.db: {text}",
            ):
                assert _client_indexing_error(detail) == (
                    "Indexing failed. Check the server logs for details."
                )
        for template in _WITHHELD_SERVICE_ERRORS:
            assert _client_indexing_error(
                template.replace("{}", "OperationalError")
            ) == ("Indexing failed. Check the server logs for details.")
        assert (
            _client_reconciliation_reason(_EMPTY_VECTOR_STORE_REASON)
            == _EMPTY_VECTOR_STORE_REASON
        )


#: Fixed text that ``web/routers/rag.py`` writes to an indexing task's
#: ``error_message`` without a failure record, and that the index-status
#: endpoint must not serve as stored.
_WITHHELD_TASK_ERRORS = frozenset()


def _indexing_task_error_writes():
    """The fixed texts ``web/routers/rag.py`` writes to an indexing task's
    ``error_message``, as ``(with_failure, without_failure)``.

    That module creates every indexing ``TaskMetadata`` row and is the only
    one that writes them. A write is an ``error_message=`` keyword of any
    call, such as ``_update_task_status(...)`` or ``TaskMetadata(...)``, or
    an ``x.error_message = ...`` assignment. Its text counts when it is a
    string, an f-string (``{}`` for each placeholder) or a module-level
    string constant; a conditional expression counts as both of its
    branches. A write whose call also passes a ``failure`` record goes in
    ``with_failure``: for a record it recognises, the status endpoint
    rebuilds the text instead of matching it.
    """
    import ast
    import inspect

    from local_deep_research.web.routers import rag

    namespace = vars(rag)
    with_failure, without_failure = set(), set()

    def texts(value):
        if isinstance(value, ast.IfExp):
            return texts(value.body) | texts(value.orelse)
        if isinstance(value, ast.Constant) and isinstance(value.value, str):
            return {value.value}
        if isinstance(value, ast.JoinedStr):
            return {
                "".join(
                    part.value if isinstance(part, ast.Constant) else "{}"
                    for part in value.values
                )
            }
        if isinstance(value, ast.Name) and isinstance(
            namespace.get(value.id), str
        ):
            return {namespace[value.id]}
        return set()

    for node in ast.walk(ast.parse(inspect.getsource(rag))):
        if isinstance(node, ast.Call):
            keywords = {keyword.arg: keyword.value for keyword in node.keywords}
            if "error_message" in keywords:
                failure = keywords.get("failure")
                records = failure is not None and not (
                    isinstance(failure, ast.Constant) and failure.value is None
                )
                (with_failure if records else without_failure).update(
                    texts(keywords["error_message"])
                )
        elif isinstance(node, ast.Assign) and any(
            isinstance(target, ast.Attribute) and target.attr == "error_message"
            for target in node.targets
        ):
            without_failure.update(texts(node.value))
    return with_failure, without_failure


def _stored_message_pass_throughs(source, namespace):
    """The stored texts the task-error function in ``source`` serves
    unchanged.

    The function's first parameter is the stored ``error_message``. Its
    pass-throughs are the texts it compares that parameter with exactly:
    ``==`` with a string, or ``in`` with a collection of strings, each
    written out or a name bound in ``namespace``. Returns
    ``(texts, others)``. ``others`` holds the source of every other use of
    the parameter, so a new shape fails the guard instead of passing
    unseen. Three uses are allowed: an ``is None`` or ``is not None`` check,
    an ``isinstance`` check, and a ``return`` of the parameter directly in
    the body of an ``if`` whose test is such a comparison, alone or in an
    ``and``.
    """
    import ast
    import textwrap

    function = ast.parse(textwrap.dedent(source)).body[0]
    param = function.args.args[0].arg
    parents = {
        child: node
        for node in ast.walk(function)
        for child in ast.iter_child_nodes(node)
    }

    def is_param(node):
        return isinstance(node, ast.Name) and node.id == param

    def text_of(node):
        if isinstance(node, ast.Constant) and isinstance(node.value, str):
            return node.value
        if isinstance(node, ast.Name) and isinstance(
            namespace.get(node.id), str
        ):
            return namespace[node.id]
        return None

    def texts_in(node):
        if isinstance(node, (ast.Set, ast.Tuple, ast.List)):
            items = [text_of(element) for element in node.elts]
        elif isinstance(node, ast.Name) and isinstance(
            namespace.get(node.id), (frozenset, set, tuple, list)
        ):
            items = list(namespace[node.id])
        else:
            return None
        if all(isinstance(item, str) for item in items):
            return set(items)
        return None

    def exact_match(node):
        if not (isinstance(node, ast.Compare) and len(node.ops) == 1):
            return None
        left, op, right = node.left, node.ops[0], node.comparators[0]
        if isinstance(op, ast.Eq):
            if is_param(left) and text_of(right) is not None:
                return {text_of(right)}
            if is_param(right) and text_of(left) is not None:
                return {text_of(left)}
        if isinstance(op, ast.In) and is_param(left):
            return texts_in(right)
        return None

    def guarded(statement):
        branch = parents.get(statement)
        if not (isinstance(branch, ast.If) and statement in branch.body):
            return False
        test = branch.test
        if isinstance(test, ast.BoolOp) and isinstance(test.op, ast.And):
            tests = test.values
        else:
            tests = [test]
        return any(exact_match(each) is not None for each in tests)

    def allowed(use, parent):
        if isinstance(parent, ast.Compare):
            return (
                parent.left is use
                and len(parent.ops) == 1
                and isinstance(parent.ops[0], (ast.Is, ast.IsNot))
                and isinstance(parent.comparators[0], ast.Constant)
                and parent.comparators[0].value is None
            )
        if isinstance(parent, ast.Call):
            return (
                getattr(parent.func, "id", None) == "isinstance"
                and bool(parent.args)
                and parent.args[0] is use
            )
        return isinstance(parent, ast.Return) and guarded(parent)

    texts, others = set(), set()
    for node in ast.walk(function):
        if not (is_param(node) and isinstance(node.ctx, ast.Load)):
            continue
        parent = parents[node]
        matched = exact_match(parent)
        if matched is not None:
            texts |= matched
        elif not allowed(node, parent):
            others.add(ast.unparse(parent))
    return texts, others


def _task_error_pass_throughs():
    """``_stored_message_pass_throughs`` for ``_client_indexing_task_error``."""
    import inspect

    from local_deep_research.web.routers import rag

    return _stored_message_pass_throughs(
        inspect.getsource(rag._client_indexing_task_error), vars(rag)
    )


class TestIndexingTaskErrorPassThroughs:
    """Stored task errors reach clients unchanged only for text the indexing
    code writes.

    ``get_index_status`` passes each task's stored ``error_message`` through
    ``_client_indexing_task_error``. That function rebuilds the client text
    from the failure record or the result counts, and serves the stored text
    itself only on an exact match, a pass-through. These tests check the
    pass-throughs against the text ``web/routers/rag.py`` writes to indexing
    task rows. A pass-through that nothing writes is dead: a test that seeds
    its text into a row passes while no code path produces it. Fixed text
    written without a failure record must be passed through or listed in
    ``_WITHHELD_TASK_ERRORS``, so it cannot be withheld unnoticed.
    """

    def test_every_pass_through_is_written_to_a_task_row(self):
        from local_deep_research.web.routers.rag import (
            _client_indexing_task_error,
        )

        with_failure, without_failure = _indexing_task_error_writes()
        pass_throughs, _ = _task_error_pass_throughs()

        dead = sorted(pass_throughs - with_failure - without_failure)
        assert dead == [], (
            f"_client_indexing_task_error serves stored text {dead} unchanged,"
            " but web/routers/rag.py never writes it to an indexing task row."
            " Remove the pass-through, or add the code that writes the text."
        )
        for text in pass_throughs:
            assert _client_indexing_task_error(text, None) == text

    def test_every_fixed_text_written_without_a_record_is_classified(self):
        from local_deep_research.web.routers.rag import (
            _COLLECTION_NOT_FOUND,
            _client_indexing_task_error,
        )

        with_failure, without_failure = _indexing_task_error_writes()
        pass_throughs, _ = _task_error_pass_throughs()

        unclassified = sorted(
            without_failure - pass_throughs - _WITHHELD_TASK_ERRORS
        )
        assert unclassified == [], (
            f"web/routers/rag.py writes task error text {unclassified} with no"
            " failure record, and the index-status endpoint does not serve it."
            " Add an exact-match pass-through to _client_indexing_task_error"
            " if clients should see it, or add it to _WITHHELD_TASK_ERRORS in"
            " this file."
        )
        assert not pass_throughs & _WITHHELD_TASK_ERRORS
        assert sorted(_WITHHELD_TASK_ERRORS - without_failure) == [], (
            "withheld task error text that web/routers/rag.py no longer writes"
        )
        for text in _WITHHELD_TASK_ERRORS:
            assert _client_indexing_task_error(text, None) == (
                "Indexing failed. Check the server logs for details."
            )
        # Control: the scan sees the worker's writes.
        assert _COLLECTION_NOT_FOUND in with_failure

    def test_stored_text_is_served_only_on_an_exact_match(self):
        _, others = _task_error_pass_throughs()

        assert sorted(others) == [], (
            "_client_indexing_task_error uses the stored message in a way"
            f" these checks do not classify: {sorted(others)}"
        )

    def test_the_scan_recognises_each_pass_through_shape(self):
        """Control for the checks above, which find no pass-through in the
        current code."""
        source = """
            def task_error(message, result, failure=None):
                if message is None:
                    return None
                if message == "Literal text":
                    return message
                if isinstance(message, str) and message in NAMED:
                    return message
                if message in {"Set text"}:
                    return message
                if message.startswith("Prefix"):
                    return message
                return "generic"
        """

        texts, others = _stored_message_pass_throughs(
            source, {"NAMED": frozenset({"Named text"})}
        )

        assert texts == {"Literal text", "Named text", "Set text"}
        assert others == {"message.startswith", "return message"}


# ===========================================================================
# COVERAGE AREA 5 -- is_downloadable_domain URL allowlist
# ===========================================================================


class TestIsDownloadableDomainAllowlist:
    """``research_library/utils/is_downloadable_domain`` decides which remote
    URLs the server will fetch (``library.py:879`` and ``:1539``, and the
    ``POST /library/api/download-source`` gate at ``:1766``). This class is the
    direct policy regression evidence that was absent at the review snapshot.

    The policy these tests pin is the one the implementation and the
    ``is_downloadable_url`` docstring describe: "from a downloadable academic
    domain OR a direct PDF link". Host matching is exact-or-dot-suffix against
    a fixed list, after ``urlparse(url.lower()).hostname``.
    """

    def test_allowlisted_hosts_pass(self):
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "https://arxiv.org/abs/2401.12345",
            "https://www.nature.com/articles/s41586-024-00000-0",
            "https://pubmed.ncbi.nlm.nih.gov/12345678/",
            "https://doi.org/10.1000/xyz123",
            "https://openreview.net/forum?id=abc",
        ]:
            assert is_downloadable_domain(url) is True, url

    def test_subdomains_of_an_allowlisted_host_pass(self):
        """``hostname.endswith("." + domain)`` -- a real subdomain is in."""
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        assert is_downloadable_domain("https://export.arxiv.org/abs/1") is True

    def test_matching_is_case_insensitive(self):
        """``urlparse(url.lower())`` -- an upper/mixed-case host must not be
        able to slip past the allowlist, and must not be wrongly rejected
        either.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        assert is_downloadable_domain("https://ARXIV.ORG/abs/1") is True
        assert is_downloadable_domain("https://ArXiv.Org/abs/1") is True
        assert is_downloadable_domain("https://EVIL.TEST/index.html") is False

    def test_non_allowlisted_hosts_are_refused(self):
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "https://evil.test/index.html",
            "https://example.com/paper",
            # SSRF classics -- no allowlisted host, no PDF-shaped path.
            "http://169.254.169.254/latest/meta-data/",
            "http://127.0.0.1:8080/admin",
            "http://[::1]/admin",
            "http://localhost/admin",
        ]:
            assert is_downloadable_domain(url) is False, url

    def test_userinfo_cannot_forge_an_allowlisted_host(self):
        """``https://[email protected]/`` -- the allowlisted name appears in
        the userinfo, not the host. The check reads ``parsed.hostname`` (which
        strips userinfo), never the raw netloc, so this is refused. A rewrite
        to string matching on the URL or the netloc would flip this.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "https://[email protected]/",
            "https://arxiv.org:[email protected]/",
            "https://[email protected]:[email protected]/paper",
        ]:
            assert is_downloadable_domain(url) is False, url

    def test_suffix_and_prefix_lookalike_hosts_are_refused(self):
        """The dot-anchored suffix check must not admit
        ``evil-arxiv.org``/``arxivorg.test``, and the exact check must not
        admit ``arxiv.org.evil.test``.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "https://evil-arxiv.org/abs/1",
            "https://arxivorg.test/abs/1",
            "https://arxiv.org.evil.test/abs/1",
            "https://notarxiv.org/abs/1",
            "https://nature.com.evil.test/paper",
        ]:
            assert is_downloadable_domain(url) is False, url

    def test_trailing_dot_fqdn_is_refused(self):
        """CURRENT BEHAVIOUR: ``arxiv.org.`` (root-anchored FQDN) is not
        normalised, so it does not match the allowlist and the URL is
        REFUSED. That is the fail-closed direction, so it is pinned as-is --
        but it is the one place where an allowlisted resource is turned away.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        assert is_downloadable_domain("https://arxiv.org./abs/1") is False

    def test_falsy_and_unparseable_input_is_refused(self):
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in ["", None, "not a url", "://", "https://"]:
            assert is_downloadable_domain(url) is False, repr(url)

    def test_non_http_schemes_without_a_pdf_shape_are_refused(self):
        """The deleted suite's ``file://``/``ftp://`` tests were vacuous
        (``assert result is True or result is False``). These assert the
        outcome.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        assert is_downloadable_domain("file:///etc/passwd") is False
        assert is_downloadable_domain("ftp://evil.test/x") is False

    def test_is_downloadable_url_delegates_unchanged(self):
        """``is_downloadable_url`` is the documented "single source of truth"
        wrapper used elsewhere; it must not diverge from the gate.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
            is_downloadable_url,
        )

        for url in [
            "https://arxiv.org/abs/1",
            "https://evil.test/index.html",
            "https://[email protected]/",
        ]:
            assert is_downloadable_url(url) is is_downloadable_domain(url), url


class TestIsDownloadableDomainKnownWeaknesses:
    """CURRENT BEHAVIOUR, pinned deliberately -- these are NOT assertions that
    the remaining behaviour is correct.

    Private-host PDF candidates and PubMed hostname lookalikes are rejected.
    PDF-shaped URLs on public hosts and the host-independent PubMed-path
    heuristic remain accepted for compatibility.
    """

    def test_pdf_shaped_paths_still_bypass_the_allowlist_for_public_hosts(
        self,
    ):
        """The ``.pdf`` / ``/pdf/`` / ``type=pdf`` / ``format=pdf`` checks run
        independently of the host allowlist: the direct-PDF check precedes
        it, while the path/query checks follow it. They accept hosts that
        pass the private/local exclusion. This remains the documented
        "or is a direct PDF link" half of the policy (``is_downloadable_url``'s
        docstring): the allowlist still does not constrain the fetch target
        when the attacker controls the path or query on a public host --
        e.g. ``https://evil.test/x?type=pdf``.

        (Internal/metadata hosts are covered separately below -- those are
        now rejected regardless of PDF shape.)
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "https://evil.test/payload.pdf",
            "https://evil.test/pdf/anything",
            "https://evil.test/x?type=pdf",
            "https://evil.test/x?format=pdf",
        ]:
            assert is_downloadable_domain(url) is True, url

    def test_private_host_pdf_shaped_paths_are_now_rejected(self):
        """Regression pin for ``harden: reject private hosts in
        is_downloadable_domain``. A private/loopback/link-local hostname is
        now rejected FIRST, before any PDF-shape heuristic can short-circuit
        past it -- closing the misleading appearance that the PDF checks
        bypass the allowlist for internal or cloud-metadata targets such as
        ``http://127.0.0.1:8080/x?type=pdf`` or
        ``http://169.254.169.254/...?type=pdf``.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "http://127.0.0.1:8080/internal?type=pdf",
            "http://169.254.169.254/latest/meta-data/x?type=pdf",
        ]:
            assert is_downloadable_domain(url) is False, url

    def test_the_pubmed_hostname_check_is_now_anchored(self):
        """Regression pin for ``harden: ... anchor the PubMed rule``. The
        unanchored ``"pubmed" in hostname`` substring arm was removed, unlike
        the exact/dot-suffix matching used for every entry in
        ``downloadable_domains``. An attacker-registered host that merely
        contains "pubmed" as a substring -- and is not itself allowlisted or
        a dot-suffix match of an allowlisted domain -- is no longer admitted.

        (The real PubMed domains are unaffected: they are already in the
        allowlist above and matched there by exact/suffix comparison.)

        The same anchoring applies to the ``pubmed.gov`` /
        ``pubmedcentral.nih.gov`` entries added alongside this test: a host
        that merely starts or ends with those strings -- without being an
        exact match or dot-suffix -- does not qualify either.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "https://pubmed.evil.test/anything",
            "https://evil-pubmed-mirror.test/x",
            "https://pubmed.gov.evil.test/anything",
            "https://evilpubmed.gov/x",
            "https://pubmedcentral.nih.gov.attacker.com/x",
        ]:
            assert is_downloadable_domain(url) is False, url

    def test_the_pubmed_path_check_remains_an_unanchored_substring_match(
        self,
    ):
        """CURRENT BEHAVIOUR, pinned deliberately -- the hardening commit
        intentionally kept the PATH arm of the PubMed special case
        (``"/pubmed/" in path``), and it is still an unanchored substring
        match: any public host serving a ``/pubmed/`` path is admitted. This
        is narrower than the removed hostname arm (an attacker needs a path
        segment, not just a hostname substring) but is still not
        allowlist-anchored.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        assert is_downloadable_domain("https://evil.test/pubmed/x") is True

    def test_pubmed_and_pmc_real_host_forms_still_pass(self):
        """The hardening commit removed only the unanchored ``"pubmed" in
        hostname`` substring arm; the exact/dot-suffix allowlist match
        above it (untouched by that removal) is what actually admits every
        real PubMed/PMC/NCBI host, so none of them regressed.

        ``pubmed.gov`` and ``pubmedcentral.nih.gov`` are the official
        short-form PubMed/PMC hosts and are separate entries from
        ``pubmed.ncbi.nlm.nih.gov`` / ``ncbi.nlm.nih.gov`` -- removing the
        unanchored hostname substring arm dropped them (they are not a
        dot-suffix of any allowlisted domain), so they are pinned here as
        their own explicit allowlist entries rather than folded into the
        broader ``nih.gov``.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "https://pubmed.ncbi.nlm.nih.gov/12345678/",
            "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1234567/",
            "https://europepmc.org/article/MED/12345678",
            "https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_bulk/",
            "https://pubmed.gov/12345678",
            "https://www.pubmed.gov/12345678",
            "https://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=123456",
        ]:
            assert is_downloadable_domain(url) is True, url


class TestIsDownloadableDomainPrivateHostNormalization:
    """Regression pins for the call-site-only normalization layered on top
    of ``is_private_ip()`` inside ``is_downloadable_domain`` (trailing dot,
    single-token decimal/hex IPv4, RFC 6598 CGNAT).

    ``is_private_ip()`` itself is unmodified: it is a SHARED helper (also
    used by ``utilities/url_utils.py`` for outbound-request scheme
    selection and ``web/dependencies/rate_limit.py`` for the local-network
    rate-limit exemption), so changing what it considers private is out of
    scope for this filter and could change behaviour for those unrelated
    callers. The normalization instead lives in
    ``research_library/utils/__init__.py`` (``_decode_ipv4_literal``,
    ``_is_blocked_private_host``), applied only at this call site.
    """

    def test_alternate_ip_encodings_of_a_private_host_are_now_rejected(self):
        """``is_private_ip()`` only parses a hostname already written as a
        dotted-quad / colon-hex literal, so a private-host URL that is
        PDF-shaped used to sail straight past the guard (the PDF check
        returns True before the allowlist, and the private-host check
        could not recognise the host). These four textual variants of
        ``127.0.0.1`` / ``192.168.1.10`` / ``printer.local`` -- all
        resolved by glibc/requests to their private target -- are now
        normalized and rejected.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "http://2130706433/x.pdf",  # decimal for 127.0.0.1
            "http://0x7f000001/x.pdf",  # hex for 127.0.0.1
            "http://192.168.1.10./x.pdf",  # trailing-dot RFC1918
            "http://printer.local./x.pdf",  # trailing-dot mDNS
        ]:
            assert is_downloadable_domain(url) is False, url

    def test_idna_private_host_spellings_are_rejected(self):
        """Normalize Unicode numeric labels and separators before filtering."""
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for host in [
            "127.0.0.1",
            "127。0。0。1",
            "127.0.0.1",
            "127。0。0。1",
            "2130706433",
            "0x7f000001",
            "printer。local。",
            "100.64.0.1",
        ]:
            for suffix in ["/x.pdf", "/pdf/x", "/x?type=pdf", "/pubmed/x"]:
                url = f"http://{host}{suffix}"
                assert is_downloadable_domain(url) is False, url

        for host in [
            "8.8.8.8",
            "100.63.255.255",
            "bücher.example",
        ]:
            url = f"http://{host}/x.pdf"
            assert is_downloadable_domain(url) is True, url

    def test_mapped_ipv6_cgnat_hosts_are_rejected(self):
        """Apply the CGNAT exclusion to the embedded IPv4 address."""
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for host in [
            "::ffff:100.64.0.0",
            "::ffff:100.64.0.1",
            "::ffff:100.127.255.255",
            "::ffff:6440:1",
        ]:
            for suffix in ["/x.pdf", "/x?format=pdf", "/pubmed/x"]:
                url = f"http://[{host}]{suffix}"
                assert is_downloadable_domain(url) is False, url

        for host in [
            "::ffff:100.63.255.255",
            "::ffff:100.128.0.0",
            "::ffff:8.8.8.8",
        ]:
            url = f"http://[{host}]/x.pdf"
            assert is_downloadable_domain(url) is True, url

    def test_wrapped_ipv6_private_spellings_are_rejected(self):
        """The candidate loop's ``ipv4_mapped`` unwrap used to feed ONLY the
        CGNAT check, so IPv6-wrapped spellings of exactly the private
        targets this filter claims to reject were admitted: the
        IPv4-compatible (``::169.254.169.254``), IPv4-translated
        (``::ffff:0:169.254.169.254``) and NAT64 (``64:ff9b::a9fe:a9fe``)
        forms all parse to IPv6 addresses whose ``ipv4_mapped`` is None,
        and ``is_private_ip`` does not classify the transition prefixes as
        private. The shared recognition table
        ``security.ip_ranges.PRIVATE_IP_RANGES`` -- already the source of
        truth for the fetch-time chokepoint -- recognizes every one of
        these as private, so the candidate loop now checks each parsed
        candidate (and its mapped unwrap) against that table.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for host in [
            "::169.254.169.254",  # IPv4-compatible wrap of IMDS
            "::ffff:0:169.254.169.254",  # IPv4-translated wrap of IMDS
            "64:ff9b::a9fe:a9fe",  # NAT64 well-known prefix wrap of IMDS
            "::127.0.0.1",  # IPv4-compatible wrap of loopback
            "::ffff:0:100.64.0.1",  # translated wrap of the pinned CGNAT vector
        ]:
            for suffix in ["/x.pdf", "/latest/meta-data/x?type=pdf"]:
                url = f"http://[{host}]{suffix}"
                assert is_downloadable_domain(url) is False, url

        # WHAT-HELD controls: public IPv6 literals (native and mapped) and
        # the CGNAT boundary publics keep their exact head behaviour.
        for url in [
            "http://[2600::]/x.pdf",
            "http://[::ffff:8.8.8.8]/x.pdf",
            "http://100.63.255.255/x.pdf",
            "http://100.128.0.1/x.pdf",
        ]:
            assert is_downloadable_domain(url) is True, url

    def test_rfc6598_cgnat_hosts_are_now_rejected(self):
        """``is_private_ip()`` does not classify ``100.64.0.0/10`` (RFC 6598
        shared/CGNAT address space) as private; this module checks the
        range itself at the call site, without touching the shared helper.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        assert is_downloadable_domain("http://100.64.0.5/x.pdf") is False
        # Sanity: just outside the /10 range is unaffected either way.
        assert is_downloadable_domain("http://100.63.255.255/x.pdf") is True
        assert is_downloadable_domain("http://100.128.0.1/x.pdf") is True

    def test_dotted_ipv4_literal_shorthand_and_octal_are_now_rejected(self):
        """Regression pin: the shorthand-dotted (``127.1``) and per-octet
        octal (``0177.0.0.1``) gaps that ``test_is_downloadable_domain_known_
        private_ip_gaps`` used to pin as open are now closed.
        ``_decode_ipv4_literal`` decodes via ``socket.inet_aton`` (glibc's
        own IPv4-literal grammar), not a hand-rolled single-token parser, so
        it recognizes every dotted/shorthand/octal/hex form a resolver
        does -- not just a bare integer with no dots.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            "http://127.1/x.pdf",  # shorthand for 127.0.0.1
            "http://0177.0.0.1/x.pdf",  # per-octet octal for 127.0.0.1
        ]:
            assert is_downloadable_domain(url) is False, url

    def test_single_token_leading_zero_octal_is_now_rejected(self):
        """CONFIRMED bypass, now fixed: the previous decoder used
        ``int(hostname, 0)`` -- Python integer-literal syntax -- which
        raises ``ValueError`` on a single-token leading-zero octal string
        like ``017700000001`` (Python requires an explicit ``0o`` prefix
        for octal; a bare leading zero is a ``SyntaxError``/``ValueError``
        in modern Python). glibc's ``inet_aton`` has no such requirement:
        ``017700000001`` and ``0177`` are valid octal literals to it and
        resolve to ``127.0.0.1`` / ``0.0.0.127`` respectively, so a URL
        built from either sailed straight past the old guard. Verified via
        ``socket.getaddrinfo`` that this is exactly what a real resolver
        does with these strings too.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        assert is_downloadable_domain("http://017700000001/x.pdf") is False
        assert is_downloadable_domain("http://0177/x.pdf") is False

    def test_python_literal_only_forms_are_no_longer_misdecoded(self):
        """The old ``int(hostname, 0)`` decoder also over-decoded: Python
        accepts ``0b``-binary and underscore-grouped digits as integer
        literals, but no resolver treats either as an IP address --
        ``socket.inet_aton`` (and ``socket.getaddrinfo``) reject both
        outright. These hostnames both encode ``127.0.0.1`` under Python's
        literal grammar, but since a resolver never decodes them that way,
        they must NOT be treated as the private loopback address by this
        filter either -- confirmed here via a PDF-shaped path, which would
        be forced to False if (wrongly) decoded as private.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for url in [
            # 0b01111111... is 127.0.0.1 in binary -- a Python int literal,
            # not anything a resolver treats as an IPv4 address.
            "http://0b01111111000000000000000000000001/x.pdf",
            # Underscore-grouped decimal for 2130706433 (127.0.0.1) -- valid
            # to Python's int(x, 0), meaningless to a resolver.
            "http://2_130_706_433/x.pdf",
        ]:
            assert is_downloadable_domain(url) is True, url

    def test_percent_encoded_private_hosts_are_now_rejected(self):
        """CONFIRMED bypass, now fixed: the private-host check ran against
        the raw, still-percent-encoded ``urlparse().hostname`` string, so an
        encoded loopback/local host never matched any recognized literal
        form and sailed straight past the guard as an opaque non-matching
        hostname -- exactly the same way an unrecognized public hostname
        would. ``%31%32%37.0.0.1`` percent-decodes to ``127.0.0.1``;
        ``%6c%6f%63%61%6c%68%6f%73%74`` decodes to ``localhost``;
        ``%31%30%2e%30%2e%30%2e%31`` decodes to ``10.0.0.1``. The hostname
        is now percent-decoded (one pass, via ``urllib.parse.unquote``)
        before IDNA normalization and the legacy-IPv4 decode, so these are
        recognized the same as their plain-text spellings.
        """
        from local_deep_research.research_library.utils import (
            is_downloadable_domain,
        )

        for host in [
            "%31%32%37.0.0.1",  # percent-encoded 127.0.0.1
            "127.0.0.1%00",  # trailing percent-encoded NUL after loopback
            "127.0.0.1%2e",  # percent-encoded trailing dot
            "%6c%6f%63%61%6c%68%6f%73%74",  # percent-encoded "localhost"
            "%31%30%2e%30%2e%30%2e%31",  # percent-encoded 10.0.0.1
        ]:
            for suffix in ["/x.pdf", "/pubmed/x", "/x?type=pdf"]:
                url = f"http://{host}{suffix}"
                assert is_downloadable_domain(url) is False, url

        # WHAT-HELD control: an ordinary allowlisted host is unaffected by
        # percent-decoding (there is nothing to decode).
        assert is_downloadable_domain("https://arxiv.org/abs/1") is True


class TestDownloadSourceEnforcesTheAllowlist:
    """The HTTP enforcement point (``library.py:1766``). Pairs the refusal
    with a positive control that an allowlisted URL gets PAST the gate --
    otherwise a route that 400s on everything would satisfy the refusal test.
    Neither case performs any network IO: the refusal returns before the
    threadpool call, and the allowlisted case stops at "Resource not found"
    because no matching ``ResearchResource`` row exists.
    """

    def test_non_allowlisted_url_is_refused_with_400(self, library_client):
        client, _, _ = library_client

        resp = client.post(
            "/library/api/download-source",
            json={"research_id": 1, "url": "https://evil.test/index.html"},
        )

        assert resp.status_code == 400, resp.text
        assert resp.json()["error"] == "URL is not from a downloadable domain"

    def test_allowlisted_url_passes_the_gate(self, library_client):
        """Positive control: the same request shape with an allowlisted host
        reaches the resource lookup (404), proving the 400 above is the
        allowlist gate and not generic request rejection.
        """
        client, _, _ = library_client

        resp = client.post(
            "/library/api/download-source",
            json={"research_id": 1, "url": "https://arxiv.org/abs/2401.12345"},
        )

        assert resp.status_code == 404, resp.text
        assert resp.json()["error"] == "Resource not found"