~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
"""Route-layer authorization / input-bound coverage for the library and notes
surfaces lost in the Flask -> FastAPI migration (PR #3299).
Companion to ``tests/security/test_library_rag_security_fastapi.py``, which
provides evidence for the single-document note-protection 403 and
``_format_test_embedding_error``. This file covers four additional areas
identified by the historical ADR-0010 review. Nothing here duplicates it.
COVERAGE AREA 1 -- note protection on bulk paths and the protected-collection
flag. ``DELETE /library/api/documents/bulk`` is already covered next door;
its two mutating siblings are not:
``DELETE /library/api/documents/blobs`` (loops ``delete_blob_only``) and
``DELETE /library/api/collection/{cid}/documents/bulk`` (loops
``remove_from_collection``). Neither route has the single-document route's
403 mapping -- ``BulkDeletionService`` returns 200 with a per-item failure --
so the SERVICE guard is the only thing standing between a "select all ->
delete" in the library UI and mass note damage. Also here:
``rag.py::_is_protected_collection``, whose result the collection-details
route serializes so the UI can hide destructive affordances on the three
system collection types.
COVERAGE AREA 2 -- ``POST /library/api/open-folder`` must HARD-403 (the branch's smoke
sweeps only assert ``< 500``, which a re-enabled 200 satisfies); the upload
EXTENSION allowlist at the route (size and count caps are already covered by
``tests/web/routers/test_rag_upload_limits_source_of_truth.py``, extension is
not); ``library.py::get_authenticated_user_password`` fail-closed; and the
SSE auth-failure fail-closed events in ``download_all_text`` /
``download_bulk``.
COVERAGE AREA 3 -- notes: version-read cross-note scoping (the branch covers
the analogous guard for RESTORE only), the fact-check grade route's
cross-resource authorization ("research is not linked to this note") and its
claim sanitisation, the ``MAX_SEARCH_LEN`` / ``MAX_CLAIM_LEN`` /
``MAX_PASSAGE_LEN`` input-size clamps, the ``MAX_LINK_TEXT_LEN`` clamp on
``POST /notes/api/notes/resolve-link``, and the annotation-delete
anchor/ownership check on ``DELETE .../research/{id}/annotations/{note_id}``
and ``.../documents/{id}/annotations/{note_id}``.
COVERAGE AREA 4 -- the collection-type allowlist (``rag.py:1761``) and the
system-collection rename guard (``rag.py:1885``) AT THE ROUTE, plus
``library_delete.py:288``'s protected-type -> 409 mapping. That last one is
helper-covered (``tests/deletion/test_collection_deletion.py`` proves the
service refuses), but the route mapping lacked direct coverage at the
review snapshot. This file now pins that call site and status contract.
Harness (matching the sibling security files):
* ``TestClient(app, raise_server_exceptions=False)`` against the live FastAPI
app -- no dependency overrides, because every route here opens the caller's
ENCRYPTED per-user database and that only works after a real login.
* CSRF is ASGI-middleware-enforced, so each client mints a real token. A bare
mutating request 403s at CSRF before any dependency runs, which would test
CSRF rather than the guard under test.
* Rate limiting is keyed per client IP (registration is limited per IP) AND, for the
notes AI buckets, per authenticated user (``_notes_factcheck_limit`` is
3/minute). Every test therefore gets a FRESH user on a FRESH IP drawn from a
MONOTONIC counter -- random addresses collide in a long session and surface
as unrelated 429s.
* Rows are seeded into the caller's own encrypted DB through
``get_user_db_session(username, password)`` so the guards run against real
data, not mocks.
VACUITY: every "the destructive thing did not happen" assertion is paired with a
positive control asserted in the same test or an adjacent one -- an empty
library, a 500, or a blanket refusal would otherwise satisfy the negative half
on its own. Each protected case is also paired with its 404 / 200 discriminator
so a route that collapsed every outcome into one status could not pass.
"""
import itertools
import json
import os
import uuid
from contextlib import contextmanager, suppress
from datetime import datetime, timezone
from unittest.mock import patch
import pytest
from fastapi.testclient import TestClient
# The app's docs_url toggle reads PYTEST_CURRENT_TEST at app-import time;
# pytest only sets that per-test, not at collection.
os.environ.setdefault("TESTING", "1")
PASSWORD = "TestPassword123!" # noqa: S105 -- test-only credential
# Monotonic, never-reused client IPs. Rate limiting buckets per IP, so a random
# address can collide with an earlier client in a large session and produce a
# 429 that has nothing to do with the assertion under test. 172.31/16 also keeps
# these clear of the 10/8 addresses the sibling security files draw from.
_IP_COUNTER = itertools.count(1)
def _next_forwarded_for():
n = next(_IP_COUNTER)
return f"172.31.{(n // 250) % 250}.{(n % 250) + 1}"
# ---------------------------------------------------------------------------
# Harness
# ---------------------------------------------------------------------------
def _csrf(client):
"""CSRF is enforced by ASGI middleware -- fetch a real token."""
client.get("/auth/login")
resp = client.get("/auth/csrf-token")
return resp.json().get("csrf_token", "") if resp.status_code == 200 else ""
def _cleanup_client(client, username):
try:
with suppress(Exception):
client.post("/auth/logout", follow_redirects=False)
with suppress(Exception):
from local_deep_research.web.auth.session_manager import (
session_manager,
)
session_manager.destroy_all_user_sessions(username)
with suppress(Exception):
from local_deep_research.database.session_passwords import (
session_password_store,
)
session_password_store.clear_all_for_user(username)
with suppress(Exception):
from local_deep_research.database.thread_local_session import (
clear_user_credentials,
)
clear_user_credentials(username)
finally:
client.close()
@pytest.fixture
def authed(temp_data_dir, monkeypatch, request):
"""One real registered and auto-authenticated user per test.
Returns ``(client, username, password)``. A per-test user (not a
module-scoped one) is required because the notes AI rate limits key on the
authenticated user: ``_notes_factcheck_limit`` is 3/minute, which a shared
user would exhaust part-way through this file.
"""
monkeypatch.setenv("LDR_DATA_DIR", str(temp_data_dir))
# Production PBKDF2 iterations would dominate wall-clock in a per-test
# fixture (same reason tests/conftest.py's `app` fixture lowers it).
monkeypatch.setenv("LDR_DB_CONFIG_KDF_ITERATIONS", "1000")
from local_deep_research.web.fastapi_app import app
client = TestClient(app, raise_server_exceptions=False)
username = f"lna_{uuid.uuid4().hex[:10]}"
request.addfinalizer(lambda: _cleanup_client(client, username))
client.headers.update({"X-Forwarded-For": _next_forwarded_for()})
reg = client.post(
"/auth/register",
data={
"username": username,
"password": PASSWORD,
"confirm_password": PASSWORD,
"acknowledge": "true",
"csrf_token": _csrf(client),
},
follow_redirects=False,
)
if reg.status_code != 302:
pytest.fail(
f"Auth bootstrap broken: registration returned {reg.status_code} "
f"(expected 302): {reg.text[:300]}"
)
who = client.get("/auth/check")
if (
who.status_code != 200
or who.json().get("authenticated") is not True
or who.json().get("username") != username
):
pytest.fail(
"Auth bootstrap broken: registration did not establish the "
f"expected session: {who.status_code} {who.text[:300]}"
)
token = client.get("/auth/csrf-token")
if token.status_code == 200 and token.json().get("csrf_token"):
client.headers.update({"X-CSRFToken": token.json()["csrf_token"]})
return client, username, PASSWORD
# ---------------------------------------------------------------------------
# Direct-DB seeding helpers (the caller's own encrypted database)
# ---------------------------------------------------------------------------
def _session(username, password):
from local_deep_research.database.session_context import get_user_db_session
return get_user_db_session(username, password)
def _seed_document(
username, password, *, source_type_name, title, storage_mode="database"
):
"""Insert one real ``Document`` row and return its id.
``source_type_name`` selects a ``SourceType`` seeded by
``initialize_library_for_user`` on register/login: "note" produces a
document ``_is_note_document`` recognises, "user_upload" an ordinary one.
"""
from local_deep_research.database.models.library import Document, SourceType
doc_id = str(uuid.uuid4())
with _session(username, password) as session:
source_type = (
session.query(SourceType).filter_by(name=source_type_name).first()
)
assert source_type is not None, (
f"{source_type_name!r} SourceType should have been seeded by "
"initialize_library_for_user on register/login"
)
session.add(
Document(
id=doc_id,
source_type_id=source_type.id,
document_hash=uuid.uuid4().hex,
filename=f"{source_type_name}-fixture.md",
file_size=11,
file_type="md",
text_content="hello world",
title=title,
storage_mode=storage_mode,
)
)
session.commit()
return doc_id
def _seed_collection(username, password, *, collection_type, name=None):
"""Insert one real ``Collection`` row and return its id."""
from local_deep_research.database.models.library import Collection
collection_id = str(uuid.uuid4())
with _session(username, password) as session:
session.add(
Collection(
id=collection_id,
name=name or f"{collection_type}-{uuid.uuid4().hex[:6]}",
collection_type=collection_type,
)
)
session.commit()
return collection_id
def _link(username, password, document_id, collection_id):
from local_deep_research.database.models.library import DocumentCollection
with _session(username, password) as session:
session.add(
DocumentCollection(
document_id=document_id, collection_id=collection_id
)
)
session.commit()
def _document_row(username, password, document_id):
"""Return ``(exists, storage_mode, file_path)`` for a document."""
from local_deep_research.database.models.library import Document
with _session(username, password) as session:
doc = session.get(Document, document_id)
if doc is None:
return (False, None, None)
return (True, doc.storage_mode, doc.file_path)
def _link_exists(username, password, document_id, collection_id):
from local_deep_research.database.models.library import DocumentCollection
with _session(username, password) as session:
return (
session.query(DocumentCollection)
.filter_by(document_id=document_id, collection_id=collection_id)
.first()
is not None
)
def _collection_exists(username, password, collection_id):
from local_deep_research.database.models.library import Collection
with _session(username, password) as session:
return session.get(Collection, collection_id) is not None
def _seed_research(username, password, *, status="completed"):
"""Insert one ``ResearchHistory`` row and return its id."""
from local_deep_research.database.models import ResearchHistory
research_id = str(uuid.uuid4())
with _session(username, password) as session:
session.add(
ResearchHistory(
id=research_id,
query="does the sky exist",
mode="quick_summary",
status=status,
created_at=datetime.now(timezone.utc).isoformat(),
title="fixture research",
)
)
session.commit()
return research_id
def _link_research_to_note(username, password, note_id, research_id):
from local_deep_research.database.models import NoteResearch
with _session(username, password) as session:
session.add(NoteResearch(document_id=note_id, research_id=research_id))
session.commit()
# ---------------------------------------------------------------------------
# HTTP helpers
# ---------------------------------------------------------------------------
@contextmanager
def _second_authed_client(prefix="lna2"):
"""A second, independent registered and auto-authenticated user.
Same registration bootstrap as the ``authed`` fixture, callable a second
time from inside a single test that already holds an ``authed`` client
(which -- via ``monkeypatch``/``temp_data_dir`` -- has already pointed
``LDR_DATA_DIR`` at the shared per-test data directory both users need to
land in). Used only for the cross-user (ownership) case; every other test
in this file needs exactly one user and uses ``authed``.
"""
from local_deep_research.web.fastapi_app import app
client = TestClient(app, raise_server_exceptions=False)
username = f"{prefix}_{uuid.uuid4().hex[:10]}"
try:
client.headers.update({"X-Forwarded-For": _next_forwarded_for()})
reg = client.post(
"/auth/register",
data={
"username": username,
"password": PASSWORD,
"confirm_password": PASSWORD,
"acknowledge": "true",
"csrf_token": _csrf(client),
},
follow_redirects=False,
)
if reg.status_code != 302:
pytest.fail(
"Auth bootstrap broken for second user: registration returned "
f"{reg.status_code} (expected 302): {reg.text[:300]}"
)
who = client.get("/auth/check")
if (
who.status_code != 200
or who.json().get("authenticated") is not True
or who.json().get("username") != username
):
pytest.fail(
"Auth bootstrap broken for second user: registration did not "
f"establish the expected session: {who.status_code} "
f"{who.text[:300]}"
)
token = client.get("/auth/csrf-token")
if token.status_code == 200 and token.json().get("csrf_token"):
client.headers.update({"X-CSRFToken": token.json()["csrf_token"]})
yield client, username, PASSWORD
finally:
_cleanup_client(client, username)
def _create_note(client, *, title=None, content="hello world"):
resp = client.post(
"/notes/api/notes",
json={
"title": title or f"Note {uuid.uuid4().hex[:8]}",
"content": content,
},
)
assert resp.status_code == 201, f"note create failed: {resp.text[:400]}"
return resp.json()["id"]
def _create_collection_via_api(client, *, collection_type="user_collection"):
resp = client.post(
"/library/api/collections",
json={
"name": f"coll-{uuid.uuid4().hex[:8]}",
"type": collection_type,
},
)
assert resp.status_code == 200, resp.text
return resp.json()["collection"]["id"]
def _sse_events(text):
"""Parse an SSE body into the list of decoded ``data:`` payloads."""
events = []
for line in text.splitlines():
if line.startswith("data: "):
events.append(json.loads(line[len("data: ") :]))
return events
def _no_db_password():
"""Context manager making ``get_authenticated_user_password`` fail.
Patches the password STORE rather than the resolver, so the real
``library.get_authenticated_user_password`` executes and its real
``AuthenticationRequiredError`` is what the routes have to cope with.
``require_auth`` does not consult the store (it checks the session plus
``db_manager.is_user_connected``), so the request still reaches the handler
-- which is precisely the state these guards exist for: an authenticated
session whose DB password is no longer resolvable.
"""
from local_deep_research.database.session_passwords import (
session_password_store,
)
class _Both:
def __enter__(self):
self._a = patch.object(
session_password_store,
"get_session_password",
return_value=None,
)
self._b = patch.object(
session_password_store,
"get_any_session_password",
return_value=None,
)
self._a.start()
self._b.start()
return self
def __exit__(self, *exc):
self._b.stop()
self._a.stop()
return False
return _Both()
# ===========================================================================
# COVERAGE AREA 1 -- note protection on bulk delete paths
#
# The single-document routes map the service's refusal to a 403; the bulk
# routes do NOT (they return 200 with a per-item failure), so the route layer
# offers these paths no protection at all. Only the service guard
# (`_is_note_document`) stands between the library UI's "select all -> delete"
# and mass note damage. `DELETE /library/api/documents/bulk` is covered in
# test_library_rag_security_fastapi.py; these are its two mutating siblings.
# ===========================================================================
class TestBulkBlobDeleteNoteProtection:
"""``DELETE /library/api/documents/blobs`` loops ``delete_blob_only``.
Without the guard a note's ``storage_mode`` / ``file_path`` are overwritten
with the blob-deleted sentinel, which is how the library serves and
re-indexes it.
"""
def test_bulk_blob_delete_refuses_a_note_and_leaves_it_untouched(
self, authed
):
client, username, password = authed
note_id = _seed_document(
username,
password,
source_type_name="note",
title="Protected note",
)
resp = client.request(
"DELETE",
"/library/api/documents/blobs",
json={"document_ids": [note_id]},
)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["deleted"] == 0, (
f"the note's blob must not be deleted: {body!r}"
)
# "skipped" is the no-stored-PDF bucket; a refused note is a FAILURE,
# so a guard that degraded into the skip branch would fail here.
assert body["skipped"] == 0, body
assert body["failed"] == 1, body
assert body["errors"][0]["document_id"] == note_id, body
assert "notes api" in body["errors"][0]["error"].lower(), (
f"the refusal must route the caller to the notes API: {body!r}"
)
exists, storage_mode, file_path = _document_row(
username, password, note_id
)
assert exists, "the note row must survive"
assert storage_mode == "database", (
"the note's storage_mode must not be rewritten to the "
f"blob-deleted sentinel: {storage_mode!r}"
)
assert file_path is None, (
f"the note's file_path must not be stamped: {file_path!r}"
)
def test_bulk_blob_delete_still_works_on_an_ordinary_document(self, authed):
"""Positive control (non-vacuity): same route, same request shape, a
non-note document -- the blob really is deleted and the row really is
rewritten. Proves the refusal above is the note guard firing rather
than a broken harness, an empty library, or a blanket refusal.
"""
client, username, password = authed
doc_id = _seed_document(
username,
password,
source_type_name="user_upload",
title="Ordinary document",
)
resp = client.request(
"DELETE",
"/library/api/documents/blobs",
json={"document_ids": [doc_id]},
)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["deleted"] == 1, body
assert body["failed"] == 0, body
exists, storage_mode, file_path = _document_row(
username, password, doc_id
)
assert exists, "blob-only delete must keep the document row"
assert storage_mode == "none", (
f"an ordinary document's blob delete must land: {storage_mode!r}"
)
assert file_path is not None, (
"an ordinary document must get the blob-deleted sentinel path"
)
def test_bulk_blob_delete_of_a_mixed_batch_protects_only_the_note(
self, authed
):
"""The amplification shape that matters: one "select all" request
carrying both kinds. The ordinary document must be processed AND the
note must be refused, in the same response.
"""
client, username, password = authed
note_id = _seed_document(
username, password, source_type_name="note", title="Note in batch"
)
doc_id = _seed_document(
username,
password,
source_type_name="user_upload",
title="Document in batch",
)
resp = client.request(
"DELETE",
"/library/api/documents/blobs",
json={"document_ids": [note_id, doc_id]},
)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["total"] == 2, body
assert body["deleted"] == 1, f"only the ordinary document: {body!r}"
assert body["failed"] == 1, f"only the note refused: {body!r}"
assert [e["document_id"] for e in body["errors"]] == [note_id], body
assert _document_row(username, password, note_id)[1] == "database"
assert _document_row(username, password, doc_id)[1] == "none"
class TestBulkCollectionRemovalNoteProtection:
"""``DELETE /library/api/collection/{cid}/documents/bulk`` loops
``remove_from_collection``.
The Notes collection is every note's permanent home
(``_get_or_create_notes_collection``) and nothing re-links a note after an
unlink, so removal there silently drops the note out of semantic search,
the collection view and the auto-reindex worker while ``list_notes`` still
shows it.
"""
def test_bulk_removal_refuses_a_note_from_its_notes_collection(
self, authed
):
client, username, password = authed
notes_collection = _seed_collection(
username, password, collection_type="notes", name="Notes"
)
note_id = _seed_document(
username, password, source_type_name="note", title="Homed note"
)
_link(username, password, note_id, notes_collection)
resp = client.request(
"DELETE",
f"/library/api/collection/{notes_collection}/documents/bulk",
json={"document_ids": [note_id]},
)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["unlinked"] == 0, body
assert body["deleted"] == 0, body
assert body["failed"] == 1, body
assert "permanent home" in body["errors"][0]["error"].lower(), body
assert _link_exists(username, password, note_id, notes_collection), (
"the note must still be linked to its notes collection"
)
assert _document_row(username, password, note_id)[0], (
"the note document itself must survive"
)
def test_bulk_removal_still_unlinks_an_ordinary_document_from_a_notes_collection(
self, authed
):
"""Positive control #1 (discriminates on the DOCUMENT half of the
guard): the guard is ``notes collection AND note document``. A plain
document sitting in a notes collection is not protected.
"""
client, username, password = authed
notes_collection = _seed_collection(
username, password, collection_type="notes", name="Notes"
)
doc_id = _seed_document(
username,
password,
source_type_name="user_upload",
title="Ordinary in notes collection",
)
_link(username, password, doc_id, notes_collection)
resp = client.request(
"DELETE",
f"/library/api/collection/{notes_collection}/documents/bulk",
json={"document_ids": [doc_id]},
)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["failed"] == 0, body
assert body["unlinked"] == 1, body
assert not _link_exists(username, password, doc_id, notes_collection)
def test_bulk_removal_still_unlinks_a_note_from_an_ordinary_collection(
self, authed
):
"""Positive control #2 (discriminates on the COLLECTION half): a note
may be removed from an ordinary collection -- only its notes-collection
home is protected -- and the note itself survives that removal because
it is still linked to its home.
"""
client, username, password = authed
notes_collection = _seed_collection(
username, password, collection_type="notes", name="Notes"
)
user_collection = _seed_collection(
username, password, collection_type="user_collection"
)
note_id = _seed_document(
username, password, source_type_name="note", title="Shared note"
)
_link(username, password, note_id, notes_collection)
_link(username, password, note_id, user_collection)
resp = client.request(
"DELETE",
f"/library/api/collection/{user_collection}/documents/bulk",
json={"document_ids": [note_id]},
)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["failed"] == 0, body
assert body["unlinked"] == 1, body
assert not _link_exists(username, password, note_id, user_collection)
assert _link_exists(username, password, note_id, notes_collection), (
"removing a note from an ordinary collection must not touch its home"
)
assert _document_row(username, password, note_id)[0]
class TestProtectedCollectionFlagSerialization:
"""``rag.py::_is_protected_collection`` reaches the browser as
``collection.is_protected`` on
``GET /library/api/collections/{id}/documents``.
The UI hides destructive affordances on the strength of that flag; the
deletion service's refusal is covered elsewhere, this serialization is not.
(Audit note: the register calls the field "protected"; the shipped spelling
is ``is_protected`` -- these tests pin the shipped one.)
"""
@pytest.mark.parametrize(
"collection_type", ["notes", "research_history", "default_library"]
)
def test_system_collection_types_are_flagged_protected(
self, authed, collection_type
):
client, username, password = authed
collection_id = _seed_collection(
username, password, collection_type=collection_type
)
resp = client.get(f"/library/api/collections/{collection_id}/documents")
assert resp.status_code == 200, resp.text
collection = resp.json()["collection"]
# Positive control on the payload itself: the response really is about
# THIS collection, so `is_protected` cannot be passing by accident on
# an error body or a different row.
assert collection["id"] == collection_id
assert collection["collection_type"] == collection_type
assert collection["is_protected"] is True, (
f"{collection_type!r} must serialize as protected: {collection!r}"
)
@pytest.mark.parametrize(
"collection_type", ["user_collection", "user_uploads", "zotero"]
)
def test_user_collection_types_are_not_flagged_protected(
self, authed, collection_type
):
"""Non-vacuity for the test above: if ``is_protected`` were hardcoded
True (or the key merely present), this would fail. ``zotero`` is
included deliberately -- it is NOT in ``PROTECTED_COLLECTION_TYPES``
(users create and remove Zotero-synced collections), so a "protect
every system-ish type" regression is caught here.
"""
client, username, password = authed
collection_id = _seed_collection(
username, password, collection_type=collection_type
)
resp = client.get(f"/library/api/collections/{collection_id}/documents")
assert resp.status_code == 200, resp.text
collection = resp.json()["collection"]
assert collection["id"] == collection_id
assert collection["collection_type"] == collection_type
assert collection["is_protected"] is False, (
f"{collection_type!r} must NOT serialize as protected: "
f"{collection!r}"
)
# ===========================================================================
# COVERAGE AREA 2 -- library route validation and authorization
# ===========================================================================
class TestOpenFolderIsHardDisabled:
"""``POST /library/api/open-folder`` launches a file manager ON THE SERVER
HOST. It is disabled unconditionally.
The branch's only coverage is the generic mutating-route smoke sweep, which
asserts ``< 500`` -- a re-enabled handler returning 200 passes that sweep.
"""
@pytest.mark.parametrize(
"body",
[
{},
{"document_id": "some-document"},
{"path": "/etc"},
{"path": "../../.."},
],
ids=["empty", "document_id", "absolute-path", "traversal-path"],
)
def test_open_folder_always_403s(self, authed, body):
client, _, _ = authed
resp = client.post("/library/api/open-folder", json=body)
assert resp.status_code == 403, (
"open-folder must hard-403 for an authenticated caller; a 200 "
f"means the server-side file manager is reachable: "
f"{resp.status_code} {resp.text[:300]}"
)
payload = resp.json()
assert payload["status"] == "error", payload
assert "disabled" in payload["message"].lower(), payload
assert "desktop" in payload["message"].lower(), payload
def test_the_403_is_not_a_blanket_refusal_of_this_client(self, authed):
"""Non-vacuity: the same authenticated, CSRF-carrying client can drive
a sibling library mutation successfully, so the 403 above is the
open-folder guard and not a dead session or a missing token.
"""
client, _, _ = authed
ok = client.post(
"/library/api/collections",
json={
"name": f"probe-{uuid.uuid4().hex[:6]}",
"type": "user_collection",
},
)
assert ok.status_code == 200, (
f"the harness itself must be able to mutate: {ok.text[:300]}"
)
class TestUploadExtensionAllowlistAtTheRoute:
"""``POST /library/api/collections/{id}/upload`` must reject a
non-allowlisted extension (``is_extension_supported``).
The per-file size cap and the per-request file count cap are already
covered by ``tests/web/routers/test_rag_upload_limits_source_of_truth.py``;
the extension allowlist is covered only at the helper
(``tests/document_loaders/test_loader_registry.py``), never at the route
that actually stores the bytes.
"""
@staticmethod
def _upload(client, collection_id, filename, content=b"payload bytes"):
return client.post(
f"/library/api/collections/{collection_id}/upload",
files={"files": (filename, content, "application/octet-stream")},
data={"storage_mode": "database"},
)
def test_a_supported_extension_is_accepted(self, authed):
"""Positive control, asserted first: with an empty collection every
"the file was not stored" assertion below would pass trivially.
"""
client, _, _ = authed
collection_id = _create_collection_via_api(client)
resp = self._upload(
client, collection_id, "notes.txt", b"real extractable text\n"
)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["success"] is True, body
assert body["summary"]["successful"] == 1, body
assert body["summary"]["failed"] == 0, body
assert body["errors"] == [], body
@pytest.mark.parametrize(
"filename",
[
"payload.exe",
"payload.sh",
"payload.dll",
"payload.so",
"archive.zip",
# Double extension: only the LAST suffix decides, so a name that
# "looks like" an allowlisted type must still be rejected.
"payload.txt.exe",
],
)
def test_an_unsupported_extension_is_rejected_and_nothing_is_stored(
self, authed, filename
):
client, username, password = authed
collection_id = _create_collection_via_api(client)
resp = self._upload(client, collection_id, filename)
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["summary"]["successful"] == 0, (
f"{filename!r} must not be stored: {body!r}"
)
assert body["summary"]["failed"] == 1, body
assert body["uploaded"] == [], body
assert len(body["errors"]) == 1, body
assert body["errors"][0]["error"].startswith("Unsupported format:"), (
f"the rejection must be the extension allowlist, not extraction "
f"failure or a sanitiser: {body!r}"
)
from local_deep_research.database.models.library import Document
with _session(username, password) as session:
assert (
session.query(Document).filter_by(filename=filename).first()
is None
), f"no Document row may exist for {filename!r}"
class TestGetAuthenticatedUserPasswordFailsClosed:
"""``library.py::get_authenticated_user_password`` must RAISE
``AuthenticationRequiredError`` when no DB password can be resolved.
Every download route calls it before constructing a ``DownloadService``. A
regression that returned ``None`` (or ``""``) instead of raising would hand
that falsy value to the SQLCipher key derivation. At the review snapshot,
other tests only *patched* this function; this class now pins its contract.
"""
def test_it_returns_the_real_password_when_a_session_has_one(self, authed):
"""Positive control, asserted first -- otherwise "it raised" would be
satisfied by a function that always raises.
"""
from local_deep_research.web.routers.library import (
get_authenticated_user_password,
)
_, username, password = authed
assert get_authenticated_user_password(username) == password
def test_it_raises_when_no_session_password_exists(self, authed):
from local_deep_research.web.exceptions import (
AuthenticationRequiredError,
)
from local_deep_research.web.routers.library import (
get_authenticated_user_password,
)
_, username, _ = authed
with _no_db_password():
with pytest.raises(AuthenticationRequiredError) as excinfo:
get_authenticated_user_password(username)
assert "Authentication required" in str(excinfo.value)
def test_it_raises_rather_than_falling_back_when_the_store_errors(
self, authed
):
"""The session-scoped lookup is wrapped in ``except Exception``. That
swallow must fall through to the "no password" RAISE, never to a
silent ``None`` return.
"""
from local_deep_research.database.session_passwords import (
session_password_store,
)
from local_deep_research.web.exceptions import (
AuthenticationRequiredError,
)
from local_deep_research.web.routers.library import (
get_authenticated_user_password,
)
_, username, _ = authed
with (
patch.object(
session_password_store,
"get_session_password",
side_effect=RuntimeError("store down"),
),
patch.object(
session_password_store,
"get_any_session_password",
return_value=None,
),
):
with pytest.raises(AuthenticationRequiredError):
get_authenticated_user_password(username, session_id="sid")
class TestSseStreamsFailClosedOnAuthFailure:
"""``download_all_text`` and ``download_bulk`` build their SSE generator
AFTER the response has already been committed as 200 ``text/event-stream``,
so an unresolvable DB password cannot be signalled with a status code. Both
must emit a terminal error EVENT and stop -- never fall through and
construct a ``DownloadService`` with a missing password.
The branch's ``test_sse_response_headers.py`` asserts the headers only; it
patches the password resolver rather than exercising its failure.
"""
ALL_TEXT = "/library/api/download-all-text"
BULK = "/library/api/download-bulk"
def test_download_all_text_streams_normally_when_authenticated(
self, authed
):
"""Positive control, asserted first: an authenticated caller gets a
real stream that never mentions authentication.
"""
client, _, _ = authed
resp = client.post(self.ALL_TEXT)
assert resp.status_code == 200, resp.text
assert resp.headers["content-type"].startswith("text/event-stream")
events = _sse_events(resp.text)
assert events, f"the stream must carry events: {resp.text[:300]!r}"
assert events[-1]["complete"] is True, events
assert "error" not in events[-1], events
assert "Authentication required" not in resp.text
def test_download_all_text_emits_an_error_event_and_stops(self, authed):
client, _, _ = authed
with _no_db_password():
resp = client.post(self.ALL_TEXT)
assert resp.status_code == 200, resp.text
events = _sse_events(resp.text)
assert len(events) == 1, (
f"the stream must stop after the refusal, not keep going: {events!r}"
)
assert events[0]["error"] == "Authentication required", events
assert events[0]["complete"] is True, events
assert events[0]["total"] == 0, events
def test_download_bulk_streams_normally_when_authenticated(self, authed):
"""Positive control for the bulk stream. An unknown research id queues
nothing, so the stream reports "no new papers" -- the point is that it
gets PAST the credential gate and produces the normal two-event shape.
"""
client, _, _ = authed
resp = client.post(
self.BULK, json={"research_ids": [str(uuid.uuid4())], "mode": "pdf"}
)
assert resp.status_code == 200, resp.text
assert resp.headers["content-type"].startswith("text/event-stream")
events = _sse_events(resp.text)
assert len(events) == 2, events
assert events[0] == {"progress": 0, "current": 0, "total": 0}, events
assert events[1]["complete"] is True, events
assert "Authentication required" not in resp.text
def test_download_bulk_emits_an_error_event_and_stops(self, authed):
client, _, _ = authed
with _no_db_password():
resp = client.post(
self.BULK,
json={"research_ids": [str(uuid.uuid4())], "mode": "pdf"},
)
assert resp.status_code == 200, resp.text
events = _sse_events(resp.text)
assert len(events) == 1, (
f"the bulk stream must stop at the refusal: {events!r}"
)
assert events[0]["error"] == "Authentication required", events
assert events[0]["complete"] is True, events
assert events[0]["total"] == 0, events
# ===========================================================================
# COVERAGE AREA 3 -- notes router
# ===========================================================================
def _versions(client, note_id):
resp = client.get(f"/notes/api/notes/{note_id}/versions")
assert resp.status_code == 200, resp.text
return resp.json()["versions"]
class TestNoteVersionReadRoutesAreScopedToTheirOwnNote:
"""``GET .../notes/{note_id}/versions/{version_id}`` and
``.../versions/semantic-diff`` filter by ``document_id=note_id``.
The branch covers the analogous guard for RESTORE only
(``test_note_integration.py::
test_restore_with_bookends_rejects_version_from_another_note_same_user``).
At the review snapshot, the READ paths -- which return the version's full
content -- had no direct test. The cases below now pin them. Cross-note
here, not cross-user: both notes belong to the same caller, so per-user DB
isolation cannot mask a missing filter.
"""
def test_reading_another_notes_version_is_404(self, authed):
client, _, _ = authed
note_a = _create_note(client, content="alpha secret content")
note_b = _create_note(client, content="beta content")
version_a = _versions(client, note_a)[0]["id"]
# Positive control FIRST: the version is readable through its OWN note.
own = client.get(f"/notes/api/notes/{note_a}/versions/{version_a}")
assert own.status_code == 200, own.text
assert own.json()["version"]["id"] == version_a
assert "alpha secret content" in own.json()["version"]["content"]
cross = client.get(f"/notes/api/notes/{note_b}/versions/{version_a}")
assert cross.status_code == 404, (
"a version must not be readable through a sibling note's id: "
f"{cross.status_code} {cross.text[:300]}"
)
assert "alpha secret content" not in cross.text
def test_semantic_diff_rejects_a_version_from_another_note(
self, authed, monkeypatch
):
from local_deep_research.web.routers import notes as notes_router
class _FakeAI:
def __init__(self, *args, **kwargs):
pass
async def semantic_diff(self, a, b):
return {"summary": "fake diff", "a": a, "b": b}
monkeypatch.setattr(notes_router, "NoteAIService", _FakeAI)
client, _, _ = authed
note_a = _create_note(client, content="alpha secret content")
note_b = _create_note(client, content="beta content")
version_a = _versions(client, note_a)[0]["id"]
# Positive control FIRST.
own = client.get(
f"/notes/api/notes/{note_a}/versions/semantic-diff",
params={"version1": version_a, "version2": "current"},
)
assert own.status_code == 200, own.text
assert own.json()["diff"]["summary"] == "fake diff"
cross = client.get(
f"/notes/api/notes/{note_b}/versions/semantic-diff",
params={"version1": version_a, "version2": "current"},
)
assert cross.status_code == 404, (
"semantic-diff must not accept a version belonging to another "
f"note: {cross.status_code} {cross.text[:300]}"
)
assert "alpha secret content" not in cross.text
def test_semantic_diff_rejects_a_foreign_version2(
self, authed, monkeypatch
):
"""``version2`` goes through the same ``document_id=note_id`` filter as
``version1``; a filter dropped from only the second lookup would still
pass the test above.
"""
from local_deep_research.web.routers import notes as notes_router
class _FakeAI:
def __init__(self, *args, **kwargs):
pass
async def semantic_diff(self, a, b):
return {"summary": "fake diff"}
monkeypatch.setattr(notes_router, "NoteAIService", _FakeAI)
client, _, _ = authed
note_a = _create_note(client, content="alpha secret content")
note_b = _create_note(client, content="beta content")
version_a = _versions(client, note_a)[0]["id"]
version_b = _versions(client, note_b)[0]["id"]
# Positive control FIRST: two versions of the SAME note diff fine.
own = client.get(
f"/notes/api/notes/{note_b}/versions/semantic-diff",
params={"version1": version_b, "version2": "current"},
)
assert own.status_code == 200, own.text
cross = client.get(
f"/notes/api/notes/{note_b}/versions/semantic-diff",
params={"version1": version_b, "version2": version_a},
)
assert cross.status_code == 404, (
f"version2 must be scoped too: {cross.status_code} {cross.text[:300]}"
)
assert "alpha secret content" not in cross.text
@pytest.mark.parametrize("route", ["versions", "version", "semantic-diff"])
def test_version_routes_404_for_a_non_note_document(
self, authed, route, monkeypatch
):
"""``/versions*`` must not become a generic reader for ANY library
document's history. Each route re-checks ``_is_note`` on the parent, so
the notes API cannot be pointed at an arbitrary uploaded document.
"""
from local_deep_research.web.routers import notes as notes_router
class _FakeAI:
def __init__(self, *args, **kwargs):
pass
async def semantic_diff(self, a, b):
return {"summary": "fake diff"}
monkeypatch.setattr(notes_router, "NoteAIService", _FakeAI)
client, username, password = authed
note_id = _create_note(client, content="a real note")
version_id = _versions(client, note_id)[0]["id"]
document_id = _seed_document(
username,
password,
source_type_name="user_upload",
title="Not a note",
)
if route == "versions":
good = client.get(f"/notes/api/notes/{note_id}/versions")
bad = client.get(f"/notes/api/notes/{document_id}/versions")
elif route == "version":
good = client.get(
f"/notes/api/notes/{note_id}/versions/{version_id}"
)
bad = client.get(
f"/notes/api/notes/{document_id}/versions/{version_id}"
)
else:
# Both version params are required; supplying only one 400s at the
# arity check BEFORE the _is_note guard, which would make this
# test pass for the wrong reason.
params = {"version1": version_id, "version2": "current"}
good = client.get(
f"/notes/api/notes/{note_id}/versions/semantic-diff",
params=params,
)
bad = client.get(
f"/notes/api/notes/{document_id}/versions/semantic-diff",
params=params,
)
# Positive control: the same shape against a real note is NOT a 404.
assert good.status_code != 404, (
f"harness check failed for {route!r}: {good.status_code} "
f"{good.text[:300]}"
)
assert bad.status_code == 404, (
f"{route!r} must 404 for a non-note document id: "
f"{bad.status_code} {bad.text[:300]}"
)
assert bad.json()["error"] == "Note not found", bad.text
class TestFactCheckGradeCrossResourceAuthz:
"""``POST /notes/api/notes/{note_id}/fact-check/{research_id}/grade`` must
refuse a research run that is not linked to the note in the path.
Without it a client can grade an arbitrary completed research against an
arbitrary note and stamp a foreign ``note_id`` into that research's
``research_meta['fact_check']`` -- an IDOR-shaped cross-resource write.
"""
@staticmethod
def _grade(client, note_id, research_id, claims=None):
return client.post(
f"/notes/api/notes/{note_id}/fact-check/{research_id}/grade",
json={"claims": claims or ["the sky is blue"]},
)
def test_grading_a_research_linked_to_another_note_is_refused(self, authed):
client, username, password = authed
note_owner = _create_note(client, content="linked note")
note_other = _create_note(client, content="unlinked note")
research_id = _seed_research(username, password, status="completed")
_link_research_to_note(username, password, note_owner, research_id)
# Positive control FIRST: the LINKED note gets past the authz check.
# It stops later, at the report gate (502) -- proving the 404 below is
# the link check and not "this research is unusable".
linked = self._grade(client, note_owner, research_id)
assert linked.status_code == 502, (
"the linked note must pass the authz check and reach the report "
f"gate: {linked.status_code} {linked.text[:300]}"
)
cross = self._grade(client, note_other, research_id)
assert cross.status_code == 404, (
f"cross-note grading must be refused: {cross.status_code} "
f"{cross.text[:300]}"
)
assert cross.json()["error"] == "Research is not linked to this note"
def test_an_unknown_research_id_reports_not_found_not_not_linked(
self, authed
):
"""Discriminator for the test above: the two 404s carry DIFFERENT
errors, so a handler that collapsed "no such research" and "not linked"
into one branch could not pass both.
"""
client, _, _ = authed
note_id = _create_note(client)
resp = self._grade(client, note_id, str(uuid.uuid4()))
assert resp.status_code == 404, resp.text
assert resp.json()["error"] == "Research not found", resp.text
def test_an_incomplete_linked_research_is_409_not_graded(self, authed):
"""The status ladder below the authz check: a linked-but-unfinished
research must not be graded either.
"""
client, username, password = authed
note_id = _create_note(client)
research_id = _seed_research(username, password, status="in_progress")
_link_research_to_note(username, password, note_id, research_id)
resp = self._grade(client, note_id, research_id)
assert resp.status_code == 409, resp.text
assert resp.json()["status"] == "in_progress", resp.text
class TestFactCheckClaimSanitisation:
"""``claims`` is free client text that gets framed into an LLM prompt and
fanned out into research runs -- an input-bound AND cost-amplification
surface. ``notes.py:2319-2325`` strips, truncates to ``MAX_CLAIM_LEN`` and
caps at ``NoteAIService.MAX_CLAIMS_PER_NOTE``.
The sanitisation is observed by stubbing ``_grade_note_claims`` -- the seam
the route hands the sanitised list to -- so the assertion is on what the
ROUTE produced, not on downstream behaviour.
"""
@pytest.fixture
def captured(self, monkeypatch):
from local_deep_research.web.routers import notes as notes_router
box = {}
async def _fake(username, note_id, research_id, claims):
box["claims"] = claims
return {"success": True, "verdicts": []}, 200
monkeypatch.setattr(notes_router, "_grade_note_claims", _fake)
return box
@staticmethod
def _grade(client, note_id, claims):
return client.post(
f"/notes/api/notes/{note_id}/fact-check/{uuid.uuid4()}/grade",
json={"claims": claims},
)
def test_ordinary_claims_pass_through_unchanged(self, authed, captured):
"""Positive control, asserted first: without it every "the payload was
trimmed" assertion below would also pass on a route that dropped
claims entirely.
"""
client, _, _ = authed
note_id = _create_note(client)
resp = self._grade(client, note_id, ["one claim", " two claim "])
assert resp.status_code == 200, resp.text
assert captured["claims"] == ["one claim", "two claim"], captured
def test_each_claim_is_truncated_to_max_claim_len(self, authed, captured):
from local_deep_research.web.routers.notes import MAX_CLAIM_LEN
client, _, _ = authed
note_id = _create_note(client)
resp = self._grade(client, note_id, ["A" * (MAX_CLAIM_LEN * 3)])
assert resp.status_code == 200, resp.text
assert len(captured["claims"]) == 1
assert len(captured["claims"][0]) == MAX_CLAIM_LEN, (
f"claim must be truncated to {MAX_CLAIM_LEN}: "
f"{len(captured['claims'][0])}"
)
def test_the_claim_count_is_capped(self, authed, captured):
from local_deep_research.research_library.notes.services.note_ai_service import (
NoteAIService,
)
client, _, _ = authed
note_id = _create_note(client)
cap = NoteAIService.MAX_CLAIMS_PER_NOTE
resp = self._grade(
client, note_id, [f"claim {i}" for i in range(cap * 5)]
)
assert resp.status_code == 200, resp.text
assert len(captured["claims"]) == cap, (
f"claim count must be capped at {cap}: {len(captured['claims'])}"
)
@pytest.mark.parametrize(
"claims",
[
"a bare string",
{"claim": "an object"},
42,
None,
],
ids=["string", "object", "int", "null"],
)
def test_a_non_list_claims_field_is_400(self, authed, captured, claims):
client, _, _ = authed
note_id = _create_note(client)
resp = self._grade(client, note_id, claims)
assert resp.status_code == 400, resp.text
assert resp.json()["error"] == "claims required", resp.text
assert "claims" not in captured, (
"a rejected body must never reach the grader"
)
@pytest.mark.parametrize(
"claims",
[[], ["", " "], [None, 7, {"a": 1}]],
ids=["empty", "blank-strings", "non-strings"],
)
def test_a_list_with_no_usable_claims_is_400(
self, authed, captured, claims
):
"""Non-string / blank entries are filtered out, and an empty result
must 400 rather than starting a grade with zero claims.
"""
client, _, _ = authed
note_id = _create_note(client)
resp = self._grade(client, note_id, claims)
assert resp.status_code == 400, resp.text
assert resp.json()["error"] == "claims required", resp.text
assert "claims" not in captured
class TestNotesInputSizeClamps:
"""``_clamp_text_query`` / ``MAX_SEARCH_LEN`` and ``MAX_PASSAGE_LEN``.
Unbounded free text here flows into ``ILIKE %...%`` over unbounded TEXT
columns and into embedding calls. At the review snapshot both guards
survived in ``src/`` without direct tests; this class now pins them.
"""
def test_a_search_at_the_cap_is_accepted(self, authed):
"""Positive control, asserted first: the 400 below is the length cap,
not "long queries break the route".
"""
from local_deep_research.web.routers.notes import MAX_SEARCH_LEN
client, _, _ = authed
resp = client.get(
"/notes/api/notes", params={"search": "q" * MAX_SEARCH_LEN}
)
assert resp.status_code == 200, resp.text
assert resp.json()["success"] is True
def test_a_search_over_the_cap_is_400(self, authed):
from local_deep_research.web.routers.notes import MAX_SEARCH_LEN
client, _, _ = authed
resp = client.get(
"/notes/api/notes", params={"search": "q" * (MAX_SEARCH_LEN + 1)}
)
assert resp.status_code == 400, (
f"a search one char over MAX_SEARCH_LEN must 400: "
f"{resp.status_code} {resp.text[:300]}"
)
assert str(MAX_SEARCH_LEN) in resp.json()["error"], resp.text
def test_semantic_search_query_is_capped_too(self, authed, monkeypatch):
"""The same clamp guards the embedding-backed search, where an
unbounded query is also an embedding-cost amplifier.
"""
from local_deep_research.web.routers import notes as notes_router
from local_deep_research.web.routers.notes import MAX_SEARCH_LEN
class _FakeAI:
def __init__(self, *args, **kwargs):
pass
def semantic_search(self, query, **kwargs):
return []
monkeypatch.setattr(notes_router, "NoteAIService", _FakeAI)
client, _, _ = authed
# Positive control FIRST: a query at exactly the cap is served, so the
# 400 below is the length clamp and not "this route always 400s".
at_cap = client.get(
"/notes/api/notes/semantic-search",
params={"q": "q" * MAX_SEARCH_LEN},
)
assert at_cap.status_code == 200, at_cap.text
assert at_cap.json()["success"] is True
resp = client.get(
"/notes/api/notes/semantic-search",
params={"q": "q" * (MAX_SEARCH_LEN + 1)},
)
assert resp.status_code == 400, resp.text
assert str(MAX_SEARCH_LEN) in resp.json()["error"], resp.text
def test_similar_passages_truncates_to_max_passage_len(
self, authed, monkeypatch
):
from local_deep_research.web.routers import notes as notes_router
from local_deep_research.web.routers.notes import MAX_PASSAGE_LEN
seen = {}
class _RecordingAI:
def __init__(self, *args, **kwargs):
pass
def find_similar_passages(self, text, exclude_note_id=None):
seen["text"] = text
return []
monkeypatch.setattr(notes_router, "NoteAIService", _RecordingAI)
client, _, _ = authed
note_id = _create_note(client)
# Positive control FIRST: a short passage reaches the service verbatim,
# so the recorder is really wired and truncation is not "it never ran".
short = client.post(
f"/notes/api/notes/{note_id}/similar-passages",
json={"text": " a short selection "},
)
assert short.status_code == 200, short.text
assert seen["text"] == "a short selection", seen
resp = client.post(
f"/notes/api/notes/{note_id}/similar-passages",
json={"text": "P" * (MAX_PASSAGE_LEN * 3)},
)
assert resp.status_code == 200, resp.text
assert len(seen["text"]) == MAX_PASSAGE_LEN, (
f"the passage must be truncated to {MAX_PASSAGE_LEN} before it "
f"reaches the embedding call: {len(seen['text'])}"
)
class TestResolveLinkInputSizeClamp:
"""``POST /notes/api/notes/resolve-link`` clamps ``link_text`` at
``MAX_LINK_TEXT_LEN`` (``notes.py:56``) via ``_clamp_text_query`` before it
ever reaches ``NoteService.resolve_link``.
Note the route guard is not the only line of defence -- the service also
silently truncates at its own ``MAX_LINK_TEXT_LENGTH`` (same value, 500,
but a separate constant in ``note_service.py``) as a belt-and-braces
measure. That means the OBSERVABLE difference the route guard buys is the
status code: with the route guard, an oversized ``link_text`` 400s before
the service is even called; without it, the request would fall through to
the service's silent truncate-and-continue and come back 404 (or 200) --
never 400. The assertions below pin the 400 specifically so a disabled
route guard cannot hide behind the service's silent fallback.
"""
def test_link_text_at_the_cap_is_not_rejected_for_its_length(self, authed):
"""Positive control, asserted first: a link_text AT the cap must not
400. It will still 404 ("Note not found") because nothing matches
1000 copies of the letter q -- the point is that the failure is NOT
the length clamp.
"""
from local_deep_research.web.routers.notes import MAX_LINK_TEXT_LEN
client, _, _ = authed
resp = client.post(
"/notes/api/notes/resolve-link",
json={"link_text": "q" * MAX_LINK_TEXT_LEN},
)
assert resp.status_code != 400, (
f"a link_text exactly at the cap must not be rejected for its "
f"length: {resp.status_code} {resp.text[:300]}"
)
def test_link_text_over_the_cap_is_400(self, authed):
from local_deep_research.web.routers.notes import MAX_LINK_TEXT_LEN
client, _, _ = authed
resp = client.post(
"/notes/api/notes/resolve-link",
json={"link_text": "q" * (MAX_LINK_TEXT_LEN + 1)},
)
assert resp.status_code == 400, (
f"a link_text one char over MAX_LINK_TEXT_LEN must 400: "
f"{resp.status_code} {resp.text[:300]}"
)
assert str(MAX_LINK_TEXT_LEN) in resp.json()["error"], resp.text
def test_a_real_link_still_resolves_under_the_cap(self, authed):
"""Non-vacuity: the clamp does not interfere with an ordinary
resolvable link -- proves the 400 above is the length guard, not a
route that always refuses this endpoint.
"""
client, _, _ = authed
title = f"Linkable {uuid.uuid4().hex[:8]}"
_create_note(client, title=title)
resp = client.post(
"/notes/api/notes/resolve-link", json={"link_text": title}
)
assert resp.status_code == 200, resp.text
assert resp.json()["success"] is True
# ===========================================================================
# COVERAGE AREA 3 (continued) -- annotation-delete anchor/ownership check
#
# DELETE .../research/{id}/annotations/{note_id} and
# .../documents/{id}/annotations/{note_id} must refuse unless
# NoteService.has_annotation() finds a NoteReference row anchoring note_id to
# the EXACT resource named in the path (and that lookup runs exclusively
# inside the caller's own per-user encrypted database, which is the ownership
# half). The deleted predecessors were
# test_research_notes_routes.py::test_delete_requires_matching_anchor and
# ::test_delete_document_annotation_requires_anchor -- mocked-service unit
# tests with no FastAPI successor. These exercise the real route, the real
# NoteService and a real per-user database.
# ===========================================================================
class TestAnnotationDeleteAnchorCheck:
@staticmethod
def _create_research_annotation(client, research_id, quote="quoted text"):
resp = client.post(
f"/notes/api/research/{research_id}/annotations",
json={"comment": "a comment", "quote": quote},
)
assert resp.status_code == 201, resp.text
return resp.json()["annotation"]["note_id"]
@staticmethod
def _create_document_annotation(client, document_id, quote="quoted text"):
resp = client.post(
f"/notes/api/documents/{document_id}/annotations",
json={"comment": "a comment", "quote": quote},
)
assert resp.status_code == 201, resp.text
return resp.json()["annotation"]["note_id"]
@staticmethod
def _research_annotation_note_ids(client, research_id):
resp = client.get(f"/notes/api/research/{research_id}/annotations")
assert resp.status_code == 200, resp.text
return {a["note_id"] for a in resp.json()["annotations"]}
@staticmethod
def _document_annotation_note_ids(client, document_id):
resp = client.get(f"/notes/api/documents/{document_id}/annotations")
assert resp.status_code == 200, resp.text
return {a["note_id"] for a in resp.json()["annotations"]}
# -- research target -----------------------------------------------
def test_deleting_a_research_annotation_through_its_own_id_succeeds(
self, authed
):
"""Positive control, asserted first: otherwise every "the annotation
must still exist" assertion below would pass on a route that refused
every delete.
"""
client, username, password = authed
research_id = _seed_research(username, password)
note_id = self._create_research_annotation(client, research_id)
assert note_id in self._research_annotation_note_ids(
client, research_id
)
resp = client.delete(
f"/notes/api/research/{research_id}/annotations/{note_id}"
)
assert resp.status_code == 200, resp.text
assert note_id not in self._research_annotation_note_ids(
client, research_id
)
def test_deleting_a_research_annotation_via_a_different_research_id_is_404(
self, authed
):
client, username, password = authed
research_a = _seed_research(username, password)
research_b = _seed_research(username, password)
note_id = self._create_research_annotation(client, research_a)
resp = client.delete(
f"/notes/api/research/{research_b}/annotations/{note_id}"
)
assert resp.status_code == 404, (
"an annotation anchored to research A must not be deletable "
f"through research B's path: {resp.status_code} "
f"{resp.text[:300]}"
)
assert resp.json()["error"] == "Annotation not found", resp.text
assert note_id in self._research_annotation_note_ids(
client, research_a
), "the annotation must survive under its real research id"
def test_deleting_a_plain_note_via_the_research_annotation_route_is_404(
self, authed
):
"""A note with no NoteReference anchor at all (created through the
ordinary POST /notes/api/notes route) must not be deletable through
the annotation route -- ``has_annotation`` requires a real anchored
reference, not merely a note that exists.
"""
client, username, password = authed
research_id = _seed_research(username, password)
note_id = _create_note(client, content="not an annotation")
resp = client.delete(
f"/notes/api/research/{research_id}/annotations/{note_id}"
)
assert resp.status_code == 404, resp.text
# the plain note itself must survive
still_there = client.get(f"/notes/api/notes/{note_id}/versions")
assert still_there.status_code == 200, (
"the unrelated note must not have been deleted: "
f"{still_there.status_code}"
)
# -- document target -------------------------------------------------
def test_deleting_a_document_annotation_through_its_own_id_succeeds(
self, authed
):
"""Positive control for the document twin, asserted first."""
client, username, password = authed
document_id = _seed_document(
username, password, source_type_name="user_upload", title="Doc A"
)
note_id = self._create_document_annotation(client, document_id)
assert note_id in self._document_annotation_note_ids(
client, document_id
)
resp = client.delete(
f"/notes/api/documents/{document_id}/annotations/{note_id}"
)
assert resp.status_code == 200, resp.text
assert note_id not in self._document_annotation_note_ids(
client, document_id
)
def test_deleting_a_document_annotation_via_a_different_document_id_is_404(
self, authed
):
client, username, password = authed
document_a = _seed_document(
username, password, source_type_name="user_upload", title="Doc A"
)
document_b = _seed_document(
username, password, source_type_name="user_upload", title="Doc B"
)
note_id = self._create_document_annotation(client, document_a)
resp = client.delete(
f"/notes/api/documents/{document_b}/annotations/{note_id}"
)
assert resp.status_code == 404, (
"an annotation anchored to document A must not be deletable "
f"through document B's path: {resp.status_code} "
f"{resp.text[:300]}"
)
assert resp.json()["error"] == "Annotation not found", resp.text
assert note_id in self._document_annotation_note_ids(
client, document_a
), "the annotation must survive under its real document id"
# -- ownership (cross-user) ------------------------------------------
def test_a_second_user_cannot_delete_another_users_research_annotation(
self, authed
):
"""Ownership half of S13: ``has_annotation`` runs entirely inside
``get_user_db_session(self.username)`` -- the caller's OWN encrypted
database. Even a second authenticated user who somehow obtained the
exact ``note_id`` (e.g. it leaked through another channel) cannot
delete it, because that id simply does not exist in their database.
"""
client, username, password = authed
research_id = _seed_research(username, password)
note_id = self._create_research_annotation(client, research_id)
with _second_authed_client() as (
other_client,
other_username,
other_password,
):
other_research_id = _seed_research(other_username, other_password)
resp = other_client.delete(
f"/notes/api/research/{other_research_id}/annotations/{note_id}"
)
assert resp.status_code == 404, (
"a foreign user's note_id must not be deletable: "
f"{resp.status_code} {resp.text[:300]}"
)
assert note_id in self._research_annotation_note_ids(
client, research_id
), "the victim's annotation must survive the cross-user attempt"
# ===========================================================================
# COVERAGE AREA 4 -- collection type, rename, and protected-delete rules
# ===========================================================================
class TestCreateCollectionTypeAllowlist:
"""``POST /library/api/collections`` allowlists user-creatable types.
A user-crafted ``type:"notes"`` impostor is undeletable under
``PROTECTED_COLLECTION_TYPES`` and can nondeterministically win
``_get_or_create_notes_collection``'s unordered ``.first()`` lookup,
splitting the notes corpus into an attacker-named collection.
``tests/api_tests/test_collections_api.py`` covers name validation only.
"""
@pytest.mark.parametrize(
"collection_type", ["user_uploads", "user_collection"]
)
def test_allowlisted_types_are_accepted_and_persisted(
self, authed, collection_type
):
"""Positive control, asserted first -- otherwise "creation was
refused" would pass on a route that refused every type.
"""
client, username, password = authed
resp = client.post(
"/library/api/collections",
json={
"name": f"ok-{uuid.uuid4().hex[:6]}",
"type": collection_type,
},
)
assert resp.status_code == 200, resp.text
collection_id = resp.json()["collection"]["id"]
from local_deep_research.database.models.library import Collection
with _session(username, password) as session:
stored = session.get(Collection, collection_id)
assert stored is not None
assert stored.collection_type == collection_type
@pytest.mark.parametrize(
"collection_type",
[
"notes",
"default_library",
"research_history",
"zotero",
"anything_else",
"",
],
)
def test_non_allowlisted_types_are_refused_with_400(
self, authed, collection_type
):
client, username, password = authed
name = f"impostor-{uuid.uuid4().hex[:6]}"
resp = client.post(
"/library/api/collections",
json={"name": name, "type": collection_type},
)
assert resp.status_code == 400, (
f"type={collection_type!r} must be refused: {resp.status_code} "
f"{resp.text[:300]}"
)
body = resp.json()
assert body["success"] is False
assert "Invalid collection type" in body["error"], body
from local_deep_research.database.models.library import Collection
with _session(username, password) as session:
assert (
session.query(Collection).filter_by(name=name).first() is None
), "a refused collection must not be written to the database"
@pytest.mark.parametrize(
"collection_type",
[123, ["notes"], {"type": "notes"}, True],
ids=["int", "list", "dict", "bool"],
)
def test_a_non_string_type_is_refused_with_400(
self, authed, collection_type
):
"""An unhashable (or simply wrong-typed) value must be rejected by its
own branch rather than crashing the ``in`` test or the ``.strip()``.
"""
client, _, _ = authed
resp = client.post(
"/library/api/collections",
json={
"name": f"badtype-{uuid.uuid4().hex[:6]}",
"type": collection_type,
},
)
assert resp.status_code == 400, (
f"type={collection_type!r} must 400, not 500: {resp.status_code} "
f"{resp.text[:300]}"
)
assert resp.json()["error"] == "Collection type must be a string"
def test_the_default_type_is_an_allowlisted_one(self, authed):
"""Omitting ``type`` must not open a hole around the allowlist."""
client, username, password = authed
resp = client.post(
"/library/api/collections",
json={"name": f"default-{uuid.uuid4().hex[:6]}"},
)
assert resp.status_code == 200, resp.text
from local_deep_research.database.models.library import Collection
from local_deep_research.research_library.deletion.services.collection_deletion import (
PROTECTED_COLLECTION_TYPES,
)
with _session(username, password) as session:
stored = session.get(Collection, resp.json()["collection"]["id"])
assert stored.collection_type == "user_uploads"
assert stored.collection_type not in PROTECTED_COLLECTION_TYPES
class TestSystemCollectionRenameGuardAtTheRoute:
"""``PUT /library/api/collections/{id}`` refuses to rename or redescribe a
system collection (409), while deliberately leaving the per-collection
``is_public`` / ``agent_enabled`` toggles working.
"""
@pytest.mark.parametrize(
"collection_type", ["notes", "research_history", "default_library"]
)
@pytest.mark.parametrize(
"payload",
[
{"name": "hijacked"},
{"description": "hijacked"},
{"name": "hijacked", "description": "hijacked"},
],
ids=["name", "description", "both"],
)
def test_renaming_a_system_collection_is_409(
self, authed, collection_type, payload
):
client, username, password = authed
original = f"System {uuid.uuid4().hex[:6]}"
collection_id = _seed_collection(
username,
password,
collection_type=collection_type,
name=original,
)
resp = client.put(
f"/library/api/collections/{collection_id}", json=payload
)
assert resp.status_code == 409, (
f"{collection_type!r} must refuse {payload!r} with 409: "
f"{resp.status_code} {resp.text[:300]}"
)
assert "system collection" in resp.json()["error"].lower(), resp.text
from local_deep_research.database.models.library import Collection
with _session(username, password) as session:
stored = session.get(Collection, collection_id)
assert stored.name == original, "the name must be unchanged"
assert stored.description is None, (
"the description must be unchanged"
)
def test_renaming_a_user_collection_still_works(self, authed):
"""Positive control: the 409s above are the system-collection guard,
not a broken PUT route.
"""
client, username, password = authed
collection_id = _create_collection_via_api(client)
resp = client.put(
f"/library/api/collections/{collection_id}",
json={"name": "renamed-fine", "description": "also fine"},
)
assert resp.status_code == 200, resp.text
from local_deep_research.database.models.library import Collection
with _session(username, password) as session:
stored = session.get(Collection, collection_id)
assert stored.name == "renamed-fine"
assert stored.description == "also fine"
def test_non_identity_toggles_still_work_on_a_system_collection(
self, authed
):
"""The guard locks IDENTITY fields only. A "harden everything" change
that also froze ``is_public`` / ``agent_enabled`` would silently break
egress classification for the Notes collection -- fail loudly instead.
"""
client, username, password = authed
collection_id = _seed_collection(
username, password, collection_type="notes", name="Notes"
)
resp = client.put(
f"/library/api/collections/{collection_id}",
json={"is_public": True, "agent_enabled": False},
)
assert resp.status_code == 200, resp.text
from local_deep_research.database.models.library import Collection
with _session(username, password) as session:
stored = session.get(Collection, collection_id)
assert bool(stored.is_public) is True
assert stored.agent_enabled is False
class TestProtectedCollectionDeleteMapsTo409AtTheRoute:
"""``library_delete.py:288`` -- ``result["collection_type"] in
PROTECTED_COLLECTION_TYPES`` -> 409.
The service refusal was already covered
(``tests/deletion/test_collection_deletion.py::
TestProtectedCollectionTypes``); the route's status mapping was not. This
class pins the helper-covered/route-uncovered shape identified by the
review. The mapping is load-bearing,
because the frontend distinguishes 409 (refused by policy) from 404
(gone) and 400 (generic failure).
"""
@pytest.mark.parametrize(
"collection_type", ["notes", "research_history", "default_library"]
)
def test_deleting_a_protected_collection_is_409(
self, authed, collection_type
):
client, username, password = authed
collection_id = _seed_collection(
username, password, collection_type=collection_type
)
document_id = _seed_document(
username,
password,
source_type_name="user_upload",
title="Inside the system collection",
)
_link(username, password, document_id, collection_id)
resp = client.delete(f"/library/api/collections/{collection_id}")
assert resp.status_code == 409, (
f"a protected collection must map to 409 (not 400, not 404): "
f"{resp.status_code} {resp.text[:300]}"
)
body = resp.json()
assert body["success"] is False
assert body["collection_type"] == collection_type, body
assert "system collection" in body["error"].lower(), body
assert _collection_exists(username, password, collection_id), (
"the protected collection must still exist"
)
assert _document_row(username, password, document_id)[0], (
"its documents must not have been cascade-orphan-deleted"
)
def test_deleting_an_unknown_collection_is_404(self, authed):
"""Discriminator: the route's other refusal branch must stay 404, so
"everything is 409" could not pass.
"""
client, _, _ = authed
resp = client.delete(f"/library/api/collections/{uuid.uuid4()}")
assert resp.status_code == 404, resp.text
assert "not found" in resp.json()["error"].lower()
def test_deleting_an_ordinary_collection_still_returns_200(self, authed):
"""Positive control: the 409 above is the protected-type mapping, not
a DELETE route that refuses everything.
"""
client, username, password = authed
collection_id = _create_collection_via_api(client)
resp = client.delete(f"/library/api/collections/{collection_id}")
assert resp.status_code == 200, resp.text
assert resp.json()["success"] is True
assert not _collection_exists(username, password, collection_id), (
"an ordinary collection must really be gone"
)
class TestNotesListPayloadElementTypes:
"""The list-shaped notes payloads must reject non-string ELEMENTS.
Ported from ``tests/notes/test_notes_routes_review_fixes.py`` (deleted in
this migration), which asserted each of these 400s and their exact
messages. At the review snapshot nothing had replaced them; this class now
pins all five messages in ``web/routers/notes.py``.
These are not cosmetic validation. The routes' own comments say what the
guards are standing in front of -- ``synthesize_notes`` does
``dict.fromkeys(note_ids)`` and the reorder path dedups -- so an
unhashable element (a dict or a nested list) raises ``TypeError`` and
FastAPI turns that into an opaque 500. The guard is the difference between
a clean 400 telling the caller what is wrong and a request that looks like
a backend outage in logs and monitoring.
A shape that is merely *deep* rather than wrong is the interesting case:
``["a", ["b"]]`` passes ``isinstance(x, list)`` and only fails on the
element check, which is exactly the assertion a naive rewrite drops.
"""
#: (route, payload key, extra payload) for the three element-checked
#: bodies. ``reorder`` is parametrised separately: it needs a note id.
_AI_ROUTES = [
("/notes/api/notes/suggest-tags", "existing_tags"),
("/notes/api/notes/synthesize/preview", "note_ids"),
("/notes/api/notes/synthesize", "note_ids"),
]
@pytest.mark.parametrize("route,key", _AI_ROUTES)
@pytest.mark.parametrize(
"bad",
[
pytest.param([{"a": 1}], id="dict-element"),
pytest.param(["ok", ["nested"]], id="nested-list-element"),
pytest.param([1, 2], id="int-elements"),
],
)
def test_non_string_elements_are_400_not_500(self, authed, route, key, bad):
client, _, _ = authed
resp = client.post(route, json={key: bad, "content": "x"})
assert resp.status_code == 400, (
f"{route} must reject a non-string element in {key!r} with 400, "
f"not let it reach dict.fromkeys / dedup and 500: "
f"{resp.status_code} {resp.text[:300]}"
)
assert resp.json()["error"] == f"{key} must be a list of strings"
assert resp.json()["success"] is False
@pytest.mark.parametrize("route,key", _AI_ROUTES)
def test_a_well_formed_list_gets_past_the_element_guard(
self, authed, route, key
):
"""Positive control. Without this, a route that 400s unconditionally
would satisfy every assertion above.
The request may still fail for its own reasons (no such note, AI not
configured); what must NOT come back is the element-type error.
"""
client, _, _ = authed
resp = client.post(
route,
json={key: ["plain-string-id"], "content": "x"},
)
body = (
resp.json()
if resp.headers.get("content-type", "").startswith(
"application/json"
)
else {}
)
assert body.get("error") != f"{key} must be a list of strings", (
f"a list of plain strings must clear the element guard on {route}"
)
def test_reorder_rejects_a_non_list_and_an_empty_list(self, authed):
client, _, _ = authed
note_id = _create_note(client)
for payload in ({"research_ids": "not-a-list"}, {"research_ids": []}):
resp = client.post(
f"/notes/api/notes/{note_id}/research/reorder",
json=payload,
)
assert resp.status_code == 400, resp.text
assert (
resp.json()["error"] == "research_ids must be a non-empty list"
), resp.text
def test_reorder_rejects_non_string_elements(self, authed):
client, _, _ = authed
note_id = _create_note(client)
resp = client.post(
f"/notes/api/notes/{note_id}/research/reorder",
json={"research_ids": ["ok", {"nested": "object"}]},
)
assert resp.status_code == 400, resp.text
assert (
resp.json()["error"] == "research_ids must be a list of strings"
), resp.text
def test_a_well_formed_reorder_reaches_the_ownership_check(self, authed):
"""Positive control for the two reorder tests above, and coverage of
the third message in one shot.
A syntactically valid list of strings must clear BOTH shape guards and
fail later, on whether those ids are actually this note's linked
research. Getting that third error proves the shape guards passed
rather than the route rejecting everything.
"""
client, _, _ = authed
note_id = _create_note(client)
resp = client.post(
f"/notes/api/notes/{note_id}/research/reorder",
json={"research_ids": ["not-linked-to-this-note"]},
)
assert resp.status_code == 400, resp.text
assert resp.json()["error"] == (
"research_ids do not match the note's linked research"
), resp.text