Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

fix(security): gate unencrypted filesystem PDF storage behind an operator setting (#5537)

LearningCircuitfb49985aa8a5
diff --git a/changelog.d/+gate-filesystem-pdf-storage.security.md b/changelog.d/+gate-filesystem-pdf-storage.security.md
new file mode 100644
index 0000000000..43f7f722a4
--- /dev/null
+++ b/changelog.d/+gate-filesystem-pdf-storage.security.md
@@ -0,0 +1,7 @@
+Unencrypted **filesystem** storage for library-downloaded PDFs is now an environment-only operator gate (`LDR_RESEARCH_LIBRARY_ALLOW_FILESYSTEM_PDF_STORAGE`) and disabled by default. Previously any user could select it and have fetched third-party PDFs written as plaintext to a shared library directory. When the gate is off, the option is withheld from the settings UI and any stored or environment value of `filesystem` is coerced to the encrypted `database` mode before a PDF is written; a migration rewrites existing `filesystem` selections to `database`. Previously-written plaintext files remain readable. The default (`database`, encrypted) and `none` (text-only) modes are unchanged.
+
+Library-downloaded PDFs are now stored under a **per-user** subdirectory (`<library>/<username>/`) instead of a single shared directory (#5521). Previously two users' documents could collide on the same on-disk filename (per-user autoincrement resource ids) in the shared directory, letting one user's PDF overwrite or expose another's. Reads can fall back to the legacy shared location so PDFs downloaded before this change still load — no files are moved — but that fallback is now operator-gated and off by default (see below). Shared-library mode (`research_library.shared_library`) now **requires** the operator opt-in `LDR_RESEARCH_LIBRARY_ALLOW_SHARED_LIBRARY`: because both that flag and `storage_path` are user-editable, honoring a user's `shared_library=true` would let a tenant drop the per-user boundary and point storage at another user's directory, so the setting is ignored unless the operator enables the env gate. **Upgrade note:** existing shared-library deployments must set `LDR_RESEARCH_LIBRARY_ALLOW_SHARED_LIBRARY=true` to keep sharing a single directory; otherwise new downloads go to per-user subdirectories and previously-downloaded shared PDFs stop resolving until the operator *also* enables the legacy read fallback (`LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK=true`).
+
+The legacy shared-root fallback is now **read-only**: document deletion resolves only within the deleting user's own per-user directory and never follows the fallback into the shared root. Because pre-isolation files were never moved into per-user directories, an ordinary self-service delete of a legacy document could otherwise resolve a colliding relative path (e.g. `pdfs/5.pdf`) to another tenant's file and unlink it. Deletes now leave shared-root files untouched.
+
+The legacy shared-root **read** fallback is now operator-gated (`LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK`) and disabled by default, closing a cross-tenant read: because the fallback base is derived from the user-editable `research_library.storage_path`, a user could point their `storage_path` at another user's directory and — since per-user resource ids collide by construction — have a read of their own document id resolve to the other user's PDF. With the gate off (default), library PDF reads resolve strictly within the requesting user's own per-user root; operators who need pre-isolation files to keep loading can opt in on single-user or trusted deployments.
diff --git a/src/local_deep_research/database/migrations/versions/0029_gate_filesystem_pdf_storage.py b/src/local_deep_research/database/migrations/versions/0029_gate_filesystem_pdf_storage.py
new file mode 100644
index 0000000000..7948d2847e
--- /dev/null
+++ b/src/local_deep_research/database/migrations/versions/0029_gate_filesystem_pdf_storage.py
@@ -0,0 +1,152 @@
+"""Reset unencrypted filesystem PDF-storage selections to the encrypted default.
+
+The ``filesystem`` PDF storage mode writes library-downloaded third-party
+PDFs as plaintext to disk (cleartext storage of sensitive information,
+CWE-312). It is now an environment-only operator gate
+(``research_library.allow_filesystem_pdf_storage``) and disabled by default.
+
+Following migration 0027's re-enable semantics, this rewrites existing
+``filesystem`` selections UNCONDITIONALLY (it does not consult the env gate).
+The gate only re-exposes the option in the settings UI and re-permits the
+mode at write time; it deliberately does NOT reactivate a value stored
+before the capability boundary existed. An operator who later opts in must
+re-select ``filesystem`` explicitly. Previously-written plaintext files
+remain readable — ``PDFStorageManager.load_pdf`` checks the database first
+and then falls back to the filesystem regardless of the stored mode, so no
+read regression follows from the rewrite.
+
+What this migration does:
+1. settings: rows whose ``research_library.pdf_storage_mode`` decodes to a
+ ``filesystem`` value (any case/padding) are rewritten to ``database``.
+2. queued_researches: each ``settings_snapshot`` JSON blob is inspected for
+ the same key — both the wrapped submission shape (a nested
+ ``settings_snapshot`` dict) and the legacy flat shape, holding either a
+ bare scalar or a ``{"value": ...}`` metadata dict — and matching entries
+ are rewritten to ``database``. Only rows that actually change are
+ re-serialized; malformed or unrecognized snapshots are left untouched.
+
+Revision ID: 0029
+Revises: 0028
+"""
+
+from __future__ import annotations
+
+import json
+
+import sqlalchemy as sa
+from alembic import op
+from loguru import logger
+from sqlalchemy import inspect
+
+revision = "0029"
+down_revision = "0028"
+branch_labels = None
+depends_on = None
+
+PDF_STORAGE_MODE_KEY = "research_library.pdf_storage_mode"
+REPLACEMENT_MODE = "database"
+REPLACEMENT_MODE_JSON = json.dumps(REPLACEMENT_MODE)
+
+_settings = sa.table("settings", sa.column("key"), sa.column("value"))
+_queued = sa.table(
+ "queued_researches", sa.column("id"), sa.column("settings_snapshot")
+)
+
+
+def _is_filesystem(mode) -> bool:
+ return isinstance(mode, str) and mode.strip().lower() == "filesystem"
+
+
+def _rewrite_queued_research_snapshots(conn) -> None:
+ rows = conn.execute(
+ sa.select(_queued.c.id, _queued.c.settings_snapshot)
+ ).fetchall()
+
+ rewritten = 0
+ for row_id, raw in rows:
+ if not raw:
+ continue
+ try:
+ snapshot = json.loads(raw) if isinstance(raw, str) else raw
+ except (TypeError, json.JSONDecodeError):
+ continue
+ if not isinstance(snapshot, dict):
+ continue
+
+ settings = snapshot
+ if "settings_snapshot" in snapshot:
+ settings = snapshot["settings_snapshot"]
+ if not isinstance(settings, dict):
+ continue
+
+ entry = settings.get(PDF_STORAGE_MODE_KEY)
+ if isinstance(entry, dict):
+ if not _is_filesystem(entry.get("value")):
+ continue
+ entry["value"] = REPLACEMENT_MODE
+ elif _is_filesystem(entry):
+ settings[PDF_STORAGE_MODE_KEY] = REPLACEMENT_MODE
+ else:
+ continue
+
+ conn.execute(
+ sa.update(_queued)
+ .where(_queued.c.id == row_id)
+ .values(settings_snapshot=json.dumps(snapshot))
+ )
+ rewritten += 1
+
+ if rewritten:
+ logger.info(
+ "Reset filesystem PDF-storage mode in {} queued research row(s).",
+ rewritten,
+ )
+
+
+def upgrade() -> None:
+ conn = op.get_bind()
+ inspector = inspect(conn)
+ if inspector.has_table("settings"):
+ rows = conn.execute(
+ sa.select(_settings.c.value).where(
+ _settings.c.key == PDF_STORAGE_MODE_KEY
+ )
+ ).all()
+ for (raw_value,) in rows:
+ try:
+ decoded = (
+ json.loads(raw_value)
+ if isinstance(raw_value, str)
+ else raw_value
+ )
+ except (TypeError, json.JSONDecodeError):
+ continue
+ # Per-row decoded-value guard, mirroring the queued_researches
+ # path: rewrite only a row whose value actually decodes to a
+ # filesystem variant. settings.key is UNIQUE so there is at most
+ # one row here, but scoping the UPDATE to this row's exact stored
+ # value (rather than a blanket update-by-key gated on an aggregate
+ # "any row matched" flag) keeps a non-filesystem row untouched even
+ # if that invariant ever changes — making the write literally the
+ # per-row behavior the docstring describes.
+ if not _is_filesystem(decoded):
+ continue
+ result = conn.execute(
+ sa.update(_settings)
+ .where(
+ _settings.c.key == PDF_STORAGE_MODE_KEY,
+ _settings.c.value == raw_value,
+ )
+ .values(value=REPLACEMENT_MODE_JSON)
+ )
+ logger.info(
+ "Reset {} filesystem PDF-storage-mode row(s) to database.",
+ result.rowcount,
+ )
+
+ if inspector.has_table("queued_researches"):
+ _rewrite_queued_research_snapshots(conn)
+
+
+def downgrade() -> None:
+ """No-op: restoring unencrypted plaintext storage would weaken security."""
diff --git a/src/local_deep_research/research_library/deletion/services/document_deletion.py b/src/local_deep_research/research_library/deletion/services/document_deletion.py
index a6b1e89393..dd94d46754 100644
--- a/src/local_deep_research/research_library/deletion/services/document_deletion.py
+++ b/src/local_deep_research/research_library/deletion/services/document_deletion.py
@@ -233,8 +233,13 @@ class DocumentDeletionService:
from ...utils import get_absolute_path_from_settings
try:
+ # Delete only within this user's own root — never the
+ # legacy shared root, where a colliding relative path
+ # can resolve to another tenant's file (#5521).
file_path = get_absolute_path_from_settings(
- document.file_path
+ document.file_path,
+ self.username,
+ allow_legacy_fallback=False,
)
# get_absolute_path_from_settings("") returns the
# library root (its `if not relative_path` branch),
@@ -456,8 +461,12 @@ class DocumentDeletionService:
if document.file_path:
try:
+ # Own-root only: never unlink via the legacy
+ # shared root (cross-tenant collision, #5521).
file_path = get_absolute_path_from_settings(
- document.file_path
+ document.file_path,
+ self.username,
+ allow_legacy_fallback=False,
)
# See the fail-closed comment on the equivalent
# guard in _delete_document_locked() above:
@@ -710,8 +719,12 @@ class DocumentDeletionService:
from ...utils import get_absolute_path_from_settings
try:
+ # Own-root only: never unlink via the legacy
+ # shared root (cross-tenant collision, #5521).
file_path = get_absolute_path_from_settings(
- document.file_path
+ document.file_path,
+ self.username,
+ allow_legacy_fallback=False,
)
# See the fail-closed comment on the equivalent
# guard in _delete_document_locked() above:
diff --git a/src/local_deep_research/research_library/routes/library_routes.py b/src/local_deep_research/research_library/routes/library_routes.py
index 7ce4dde4f7..2a6e4fd180 100644
--- a/src/local_deep_research/research_library/routes/library_routes.py
+++ b/src/local_deep_research/research_library/routes/library_routes.py
@@ -9,6 +9,7 @@ Provides web endpoints for:
import json
import math
+import os
from datetime import UTC, datetime
from io import BytesIO
from pathlib import Path
@@ -40,6 +41,7 @@ from ..services.download_service import DownloadService
from ..services.library_service import LibraryService
from ..services.pdf_storage_manager import PDFStorageManager
from ..utils import (
+ apply_user_subdir,
get_document_for_resource,
handle_api_error,
is_downloadable_domain,
@@ -426,19 +428,29 @@ def view_pdf_page(document_id):
storage_mode = settings.get_setting(
"research_library.pdf_storage_mode", "none"
)
- library_root = (
+ base_root = (
Path(
- settings.get_setting(
- "research_library.storage_path",
- str(get_library_directory()),
+ os.path.expandvars(
+ settings.get_setting(
+ "research_library.storage_path",
+ str(get_library_directory()),
+ )
)
)
.expanduser()
.resolve()
)
+ shared_library = settings.get_setting(
+ "research_library.shared_library", False
+ )
+ # Serve from the per-user root, falling back to the legacy shared root
+ # for PDFs downloaded before per-user isolation (issue #5521).
+ library_root = apply_user_subdir(base_root, username, shared_library)
# Use PDFStorageManager to load PDF (handles database and filesystem)
- pdf_manager = PDFStorageManager(library_root, storage_mode)
+ pdf_manager = PDFStorageManager(
+ library_root, storage_mode, legacy_root=base_root
+ )
pdf_bytes = pdf_manager.load_pdf(document, db_session)
if pdf_bytes:
diff --git a/src/local_deep_research/research_library/services/download_service.py b/src/local_deep_research/research_library/services/download_service.py
index 85f4bfa112..7bbb045790 100644
--- a/src/local_deep_research/research_library/services/download_service.py
+++ b/src/local_deep_research/research_library/services/download_service.py
@@ -31,14 +31,17 @@ from ...database.models.download_tracker import (
DownloadTracker,
)
from ...security import safe_get, sanitize_error_for_client
-from ...security.path_validator import PathValidator
from ...database.models.library import (
Collection,
Document as Document,
DocumentStatus,
DownloadQueue as LibraryDownloadQueue,
)
-from .pdf_storage_manager import DEFAULT_MAX_PDF_SIZE_MB, PDFStorageManager
+from .pdf_storage_manager import (
+ DEFAULT_MAX_PDF_SIZE_MB,
+ PDFStorageManager,
+ resolve_pdf_storage_mode,
+)
from ...database.models.research import ResearchResource
from ...database.library_init import get_source_type_id, get_default_library_id
from ...database.session_context import get_user_db_session, safe_rollback
@@ -46,6 +49,7 @@ from ...utilities.db_utils import get_settings_manager
from ...library.download_management import RetryManager
from ...config.paths import get_library_directory
from ..utils import (
+ apply_user_subdir,
ensure_in_collection,
get_document_for_resource,
get_url_hash,
@@ -142,13 +146,31 @@ class DownloadService:
)
raise ValueError("Storage path setting cannot be None")
- self.library_root = str(
+ # Per-user library root (issue #5521). PDFs are written to a per-user
+ # subdirectory (<base>/<username>) instead of the shared base, so two
+ # users' per-user autoincrement resource ids can't collide on the same
+ # `pdfs/<resource_id>.pdf` path in one shared directory. This mirrors
+ # get_library_storage_path()'s layout but derives the base from this
+ # user's own settings manager (self.settings) so it stays correct in
+ # background/scheduler threads that have no request context.
+ # `legacy_library_root` is the old shared base: it is never written to,
+ # only used as a READ fallback so PDFs downloaded before per-user
+ # isolation still load (no blind bulk file move — see PR notes).
+ base_library_root = (
Path(os.path.expandvars(storage_path_setting))
.expanduser()
.resolve()
)
+ shared_library = self.settings.get_setting(
+ "research_library.shared_library", False
+ )
+ self.legacy_library_root = str(base_library_root)
+ self.library_root = str(
+ apply_user_subdir(base_library_root, self.username, shared_library)
+ )
logger.debug(
- f"[DOWNLOAD_SERVICE_INIT] Library root resolved to: {self.library_root}"
+ f"[DOWNLOAD_SERVICE_INIT] Library root resolved to: {self.library_root} "
+ f"(legacy shared root: {self.legacy_library_root})"
)
# Create directory structure
@@ -354,9 +376,22 @@ class DownloadService:
)
if tracker and tracker.file_path:
- # Compute absolute path and verify file still exists
+ # Compute absolute path and verify file still exists. Pass the
+ # username so per-user files resolve, with legacy-shared
+ # fallback for pre-isolation downloads (issue #5521). Pass
+ # self.settings so storage_path/shared_library are read from
+ # this user's captured manager — the ambient one is db-less in
+ # background/scheduler threads and would resolve the default
+ # root, missing a user's UI-customized storage_path. Use
+ # getattr: __init__ always sets self.settings in production
+ # (incl. scheduler threads), but test doubles that construct
+ # a DownloadService via __new__/partial mocks may not — fall
+ # back to None (the ambient get_settings_manager()) rather
+ # than raising AttributeError.
absolute_path = get_absolute_path_from_settings(
- tracker.file_path
+ tracker.file_path,
+ self.username,
+ settings_manager=getattr(self, "settings", None),
)
if absolute_path and absolute_path.is_file():
return True, str(absolute_path)
@@ -782,9 +817,18 @@ class DownloadService:
logger.info(f"Download failed with reason: {error_msg}")
return False, error_msg, status_code
- # Get PDF storage mode setting
- pdf_storage_mode = self.settings.get_setting(
- "research_library.pdf_storage_mode", "none"
+ # Get PDF storage mode setting. Resolve through the operator gate
+ # BEFORE any PDF is written: a stored value or an
+ # LDR_RESEARCH_LIBRARY_PDF_STORAGE_MODE env override could still
+ # read "filesystem" even though the settings UI withholds that
+ # option, so the unencrypted mode is coerced to the encrypted
+ # "database" default here unless the operator opted in. This is
+ # the true enforcement point — hiding the dropdown alone is not
+ # enough.
+ pdf_storage_mode = resolve_pdf_storage_mode(
+ self.settings.get_setting(
+ "research_library.pdf_storage_mode", "none"
+ )
)
max_pdf_size_mb = int(
self.settings.get_setting(
@@ -804,11 +848,13 @@ class DownloadService:
tracker.is_downloaded = True
tracker.downloaded_at = datetime.now(UTC)
- # Initialize PDF storage manager
+ # Initialize PDF storage manager. Writes go to the per-user root;
+ # legacy_root gives read fallback for pre-isolation files (#5521).
pdf_storage_manager = PDFStorageManager(
library_root=self.library_root,
storage_mode=pdf_storage_mode,
max_pdf_size_mb=max_pdf_size_mb,
+ legacy_root=self.legacy_library_root,
)
# Update attempt with success info
@@ -1166,6 +1212,7 @@ class DownloadService:
pdf_manager = PDFStorageManager(
library_root=self.library_root,
storage_mode=pdf_storage_mode,
+ legacy_root=self.legacy_library_root,
)
pdf_content = pdf_manager.load_pdf(doc, session)
@@ -1264,21 +1311,29 @@ class DownloadService:
if not pdf_document or pdf_document.status != "completed":
return None
- # Validate path to prevent path traversal attacks
+ # Validate path to prevent path traversal attacks. Resolve against the
+ # per-user root with legacy-shared fallback (issue #5521) so a PDF
+ # downloaded before per-user isolation is still found. Pass
+ # self.settings so storage_path/shared_library come from this user's
+ # captured manager — the ambient one is db-less in background/scheduler
+ # threads (this runs under the document scheduler) and would resolve
+ # the default root, missing a user's UI-customized storage_path. Use
+ # getattr: __init__ always sets self.settings in production (incl.
+ # scheduler threads), but test doubles that construct a
+ # DownloadService via __new__/partial mocks may not — fall back to
+ # None (the ambient get_settings_manager()) rather than raising
+ # AttributeError.
if (
not pdf_document.file_path
or pdf_document.file_path in FILE_PATH_SENTINELS
):
return None
- try:
- safe_path = PathValidator.validate_safe_path(
- pdf_document.file_path, str(self.library_root)
- )
- pdf_path = Path(safe_path)
- except ValueError:
- logger.warning(f"Path traversal blocked: {pdf_document.file_path}")
- return None
- if not pdf_path.is_file():
+ pdf_path = get_absolute_path_from_settings(
+ pdf_document.file_path,
+ self.username,
+ settings_manager=getattr(self, "settings", None),
+ )
+ if not pdf_path or not pdf_path.is_file():
return None
logger.info(f"Found existing PDF, extracting text from: {pdf_path}")
diff --git a/src/local_deep_research/research_library/services/library_service.py b/src/local_deep_research/research_library/services/library_service.py
index 94e246a568..cc1d223c29 100644
--- a/src/local_deep_research/research_library/services/library_service.py
+++ b/src/local_deep_research/research_library/services/library_service.py
@@ -28,7 +28,6 @@ from ...database.models.library import (
from ...database.models.metrics import ResearchRating
from ...database.models.research import ResearchHistory, ResearchResource
from ...database.session_context import get_user_db_session
-from ...security import PathValidator
from ...config.paths import get_library_directory
from ...utilities.sql_utils import escape_like as _escape_like
from ..utils import (
@@ -80,7 +79,7 @@ class LibraryService:
"""
if not file_path or file_path in FILE_PATH_SENTINELS:
return None
- abs_path = get_absolute_path_from_settings(file_path)
+ abs_path = get_absolute_path_from_settings(file_path, self.username)
return str(abs_path) if abs_path else None
def _is_arxiv_url(self, url: str) -> bool:
@@ -421,7 +420,9 @@ class LibraryService:
# Determine availability flags - use Document.file_path directly
file_absolute_path = None
if doc.file_path and doc.file_path not in FILE_PATH_SENTINELS:
- abs_path = get_absolute_path_from_settings(doc.file_path)
+ abs_path = get_absolute_path_from_settings(
+ doc.file_path, self.username
+ )
if abs_path:
file_absolute_path = str(abs_path)
@@ -1033,11 +1034,16 @@ class LibraryService:
.first()
)
- # Delete physical file
+ # Delete physical file. Own-root only: never resolve via the
+ # legacy shared root, where a colliding relative path can be
+ # another tenant's file (#5521), consistent with the delete
+ # sites in document_deletion.py / PDFStorageManager.delete_pdf.
if tracker and tracker.file_path:
try:
file_path = get_absolute_path_from_settings(
- tracker.file_path
+ tracker.file_path,
+ self.username,
+ allow_legacy_fallback=False,
)
if file_path and file_path.is_file():
file_path.unlink()
@@ -1104,20 +1110,15 @@ class LibraryService:
)
if tracker and tracker.file_path:
- # Validate path is within library root to prevent traversal attacks
- library_root = get_absolute_path_from_settings("")
- if not library_root:
- logger.warning("Could not determine library root")
- return False
- try:
- validated_path = PathValidator.validate_safe_path(
- tracker.file_path, library_root, allow_absolute=False
- )
- if validated_path and validated_path.is_file():
- return open_file_location(str(validated_path))
- except ValueError:
- logger.warning("Path validation failed")
- return False
+ # Resolve within the per-user root (traversal + symlink safe),
+ # with legacy-shared fallback for pre-isolation files (#5521).
+ validated_path = get_absolute_path_from_settings(
+ tracker.file_path, self.username
+ )
+ if validated_path and validated_path.is_file():
+ return open_file_location(str(validated_path))
+ logger.warning("Path validation failed or file missing")
+ return False
return False
@@ -1224,9 +1225,9 @@ class LibraryService:
)
if tracker and tracker.file_path:
- # Check if file exists
+ # Check if file exists (per-user, legacy-shared fallback)
file_path = get_absolute_path_from_settings(
- tracker.file_path
+ tracker.file_path, self.username
)
if file_path and file_path.is_file():
stats["files_found"] += 1
diff --git a/src/local_deep_research/research_library/services/pdf_storage_manager.py b/src/local_deep_research/research_library/services/pdf_storage_manager.py
index f284f50145..06b73cc1ea 100644
--- a/src/local_deep_research/research_library/services/pdf_storage_manager.py
+++ b/src/local_deep_research/research_library/services/pdf_storage_manager.py
@@ -32,6 +32,55 @@ from ...security.path_validator import PathValidator
DEFAULT_MAX_PDF_SIZE_MB = 3072 # 3 GB
+def filesystem_pdf_storage_allowed() -> bool:
+ """Return whether the operator enabled unencrypted filesystem PDF storage.
+
+ Environment-only operator gate mirroring
+ ``policy.allow_unprotected_egress``: the ``filesystem`` mode writes
+ library-downloaded third-party PDFs as PLAINTEXT to disk (cleartext
+ storage of sensitive information, CWE-312), so it is disabled by default
+ and cannot be turned on through the user-writable settings API — only
+ via the environment variable
+ ``LDR_RESEARCH_LIBRARY_ALLOW_FILESYSTEM_PDF_STORAGE=true``.
+ """
+ from ...settings.env_registry import get_env_setting
+
+ return bool(
+ get_env_setting("research_library.allow_filesystem_pdf_storage", False)
+ )
+
+
+def resolve_pdf_storage_mode(mode):
+ """Resolve the effective PDF storage mode, enforcing the operator gate.
+
+ This is the runtime enforcement point (PEP) for the unencrypted-storage
+ gate: hiding the ``filesystem`` option in the settings UI is not enough,
+ because a value stored before the gate existed — or an
+ ``LDR_RESEARCH_LIBRARY_PDF_STORAGE_MODE=filesystem`` env override —
+ could still resolve to ``filesystem``. Callers that are about to WRITE
+ a PDF must resolve the mode through here first.
+
+ When the mode is ``filesystem`` and the gate is off, it is coerced to
+ the encrypted ``database`` default. Reads are unaffected:
+ ``PDFStorageManager.load_pdf`` checks the database first and then falls
+ back to the filesystem regardless of ``storage_mode``, so any
+ previously-written plaintext files stay readable after coercion.
+ """
+ if (
+ isinstance(mode, str)
+ and mode.strip().lower() == "filesystem"
+ and not filesystem_pdf_storage_allowed()
+ ):
+ logger.warning(
+ "Unencrypted filesystem PDF storage is operator-gated and "
+ "disabled; coercing pdf_storage_mode 'filesystem' -> 'database' "
+ "(set LDR_RESEARCH_LIBRARY_ALLOW_FILESYSTEM_PDF_STORAGE=true to "
+ "opt in)."
+ )
+ return "database"
+ return mode
+
+
class PDFStorageManager:
"""Unified interface for PDF storage across all modes."""
@@ -40,21 +89,46 @@ class PDFStorageManager:
library_root: Path,
storage_mode: str,
max_pdf_size_mb: int = DEFAULT_MAX_PDF_SIZE_MB,
+ legacy_root: Optional[Path] = None,
+ username: Optional[str] = None,
):
"""
Initialize PDF storage manager.
Args:
- library_root: Base directory for filesystem storage
+ library_root: Base directory for filesystem storage. With per-user
+ library isolation (issue #5521) this is the per-user
+ directory; new PDFs are always WRITTEN here.
storage_mode: One of 'none', 'filesystem', 'database'
max_pdf_size_mb: Maximum PDF file size in MB. Should not
exceed `FileUploadValidator.MAX_FILE_SIZE` (the upload
validator's per-file cap, default 3 GB) — uploads above
that cap are rejected before they reach this layer.
+ legacy_root: Optional legacy shared root to fall back to when a
+ file is not found under ``library_root``. Lets PDFs
+ downloaded before per-user isolation still load; never
+ written to. Ignored when equal to ``library_root``.
+ username: Owning user for destructive operations. When falsy,
+ ``library_root`` cannot be guaranteed to be a per-user root
+ (``apply_user_subdir`` returns the bare shared root for an
+ empty username), so ``delete_pdf`` fails closed rather than
+ unlink within a possibly-shared directory.
"""
self.library_root = Path(library_root).resolve()
self.storage_mode = storage_mode
+ self.username = username
self.max_pdf_size_bytes = max_pdf_size_mb * 1024 * 1024
+ resolved_legacy = (
+ Path(legacy_root).resolve() if legacy_root is not None else None
+ )
+ # Only keep a distinct legacy root — a legacy root equal to the
+ # per-user root is a no-op and would double-check the same directory.
+ self.legacy_root = (
+ resolved_legacy
+ if resolved_legacy is not None
+ and resolved_legacy != self.library_root
+ else None
+ )
if storage_mode not in ("none", "filesystem", "database"):
logger.warning(
@@ -62,15 +136,58 @@ class PDFStorageManager:
)
self.storage_mode = "none"
- def _get_safe_file_path(self, relative_path: str) -> Optional[Path]:
+ def _safe_path_in_root(
+ self, relative_path: str, root: Path
+ ) -> Optional[Path]:
+ """Validate ``relative_path`` inside ``root`` (traversal + symlink
+ safe). Returns the validated Path (which may not exist), or None if
+ the path is invalid/unsafe."""
+ try:
+ # Use PathValidator to safely join and validate the path
+ safe_path = PathValidator.validate_safe_path(
+ relative_path, str(root)
+ )
+ safe_path = Path(safe_path)
+ # Block symbolic links to prevent symlink-based escapes
+ if safe_path.is_symlink():
+ logger.warning(f"Symlink blocked: {relative_path}")
+ return None
+ return safe_path
+ except ValueError:
+ logger.warning(f"Path traversal blocked: {relative_path}")
+ return None
+
+ def _get_safe_file_path(
+ self, relative_path: str, *, allow_legacy_fallback: bool = True
+ ) -> Optional[Path]:
"""
Safely resolve a relative path within the library root.
Prevents path traversal attacks by validating the path stays within
- the library root directory.
+ the library root directory. Resolves against the per-user root first
+ and, when the file is absent there, falls back to the legacy shared
+ root (issue #5521) so pre-isolation downloads still load. When
+ neither location has the file, the per-user path is returned so
+ callers' ``is_file()``/``None`` handling is unchanged.
+
+ ``allow_legacy_fallback`` must be ``False`` for destructive callers
+ (unlink): the legacy shared root is not per-user namespaced, so a
+ colliding relative path can resolve to another tenant's file and be
+ unlinked. Delete only within this manager's own ``library_root``.
+
+ Even for read-only callers the legacy shared-root fallback is
+ cross-tenant read primitive on a multi-tenant instance (the shared
+ root is derived from the user-editable ``research_library.storage_path``
+ and per-user resource ids collide by construction), so it only fires
+ when the operator opted into it via
+ ``research_library.allow_legacy_read_fallback`` — OFF by default.
Args:
relative_path: Relative path from database
+ allow_legacy_fallback: Permit the read-only legacy shared-root
+ fallback. Pass ``False`` from delete/unlink paths. Even when
+ ``True``, the fallback only fires if the operator enabled
+ ``research_library.allow_legacy_read_fallback``.
Returns:
Validated absolute Path or None if path is invalid/unsafe
@@ -78,20 +195,25 @@ class PDFStorageManager:
if not relative_path or relative_path in FILE_PATH_SENTINELS:
return None
- try:
- # Use PathValidator to safely join and validate the path
- safe_path = PathValidator.validate_safe_path(
- relative_path, str(self.library_root)
- )
- safe_path = Path(safe_path)
- # Block symbolic links to prevent symlink-based escapes
- if safe_path.is_symlink():
- logger.warning(f"Symlink blocked: {relative_path}")
- return None
- return safe_path
- except ValueError:
- logger.warning(f"Path traversal blocked: {relative_path}")
- return None
+ primary = self._safe_path_in_root(relative_path, self.library_root)
+ if primary is not None and primary.is_file():
+ return primary
+ # Legacy shared-root fallback for files downloaded before per-user
+ # isolation. Never written to; read-only fallback. Enforced at the
+ # fallback-USE site (view_pdf_page passes legacy_root=base_root
+ # explicitly) so the operator gate covers every caller: only follow
+ # the legacy root when the caller permits it AND the operator opted in.
+ from ..utils import _legacy_read_fallback_allowed
+
+ if (
+ allow_legacy_fallback
+ and self.legacy_root is not None
+ and _legacy_read_fallback_allowed()
+ ):
+ legacy = self._safe_path_in_root(relative_path, self.legacy_root)
+ if legacy is not None and legacy.is_file():
+ return legacy
+ return primary
def save_pdf(
self,
@@ -219,13 +341,15 @@ class PDFStorageManager:
return False
@classmethod
- def pdf_exists(cls, library_root, document, session):
+ def pdf_exists(cls, library_root, document, session, legacy_root=None):
"""Check if a PDF exists in any storage backend.
Use this when you need to check PDF availability without a specific
storage mode — e.g. generating document URLs in search results.
+ ``legacy_root`` enables the per-user -> legacy-shared read fallback
+ (issue #5521) so a pre-isolation filesystem PDF still reports present.
"""
- manager = cls(library_root, "none")
+ manager = cls(library_root, "none", legacy_root=legacy_root)
return manager.has_pdf(document, session)
def _infer_storage_mode(self, document: Document) -> str:
@@ -395,8 +519,25 @@ class PDFStorageManager:
try:
if storage_mode == "filesystem":
- # Use safe path resolution to prevent path traversal attacks
- file_path = self._get_safe_file_path(document.file_path)
+ # Fail closed without a user context: an empty username means
+ # library_root may be the bare shared root (apply_user_subdir
+ # returns the shared base for an empty username), where a
+ # colliding relative path can belong to another tenant. Refuse
+ # the unlink rather than risk deleting another user's file.
+ if not self.username:
+ logger.warning(
+ "Refusing filesystem PDF delete for document "
+ f"{document.id}: no user context, cannot confirm the "
+ "library root is per-user (would risk unlinking a "
+ "shared-root file)."
+ )
+ return False
+ # Use safe path resolution to prevent path traversal attacks.
+ # Delete within our own root only — never the shared legacy
+ # root, where a colliding path can be another tenant's file.
+ file_path = self._get_safe_file_path(
+ document.file_path, allow_legacy_fallback=False
+ )
if file_path and file_path.is_file():
file_path.unlink()
logger.info(f"Deleted PDF file: {file_path}")
diff --git a/src/local_deep_research/research_library/utils/__init__.py b/src/local_deep_research/research_library/utils/__init__.py
index e33a9e36bc..57452dd9b4 100644
--- a/src/local_deep_research/research_library/utils/__init__.py
+++ b/src/local_deep_research/research_library/utils/__init__.py
@@ -200,6 +200,106 @@ def ensure_in_collection(
return doc_collection
+def _reject_unsafe_username_component(username: str) -> None:
+ """Fail closed unless ``username`` is a single, safe path component.
+
+ ``apply_user_subdir`` joins the username directly into a filesystem path,
+ so any value that could let a downstream ``.resolve()`` escape the
+ library base would be an arbitrary-directory read/write primitive. This
+ mirrors registration's *exact* predicate
+ (``username.replace('_','').replace('-','').isalnum()``) so the two
+ checks can never diverge: any Unicode letters/digits registration accepts
+ (e.g. accented Latin, Cyrillic, CJK) are accepted here too, alongside
+ plain ``[A-Za-z0-9_-]``. ``str.isalnum()`` still rejects every
+ path-escape vector — separators (``/``, ``\\``), ``..``, ``:``, dots,
+ spaces, and control characters are never alphanumeric — so this remains a
+ fail-closed traversal guard; it does not attempt to additionally block
+ reserved device names or homoglyphs, since neither is meaningfully closed
+ by matching registration's charset. We reject rather than sanitize/hash
+ so a valid account keeps a human-readable per-user directory, and reject
+ rather than fall back to the shared base (which would risk cross-user
+ co-mingling).
+ """
+ stripped = username.replace("_", "").replace("-", "")
+ if not stripped or not stripped.isalnum():
+ raise ValueError("Unsafe username for per-user library path")
+
+
+def apply_user_subdir(
+ base_path, username: Optional[str], shared_library: bool = False
+) -> Path:
+ """Return the per-user library directory under ``base_path``.
+
+ Single source of truth for the per-user library layout (issue #5521):
+ downloaded PDFs live under ``<base>/<username>`` so two users' per-user
+ autoincrement ``resource_id`` filenames (e.g. ``pdfs/5.pdf``) can no
+ longer collide in one shared directory. In shared-library mode — or when
+ no username is available — the base directory itself is returned.
+
+ The ``username`` is validated as a single safe path component before it is
+ joined (see :func:`_reject_unsafe_username_component`); an unsafe value
+ raises ``ValueError`` rather than escaping the base or silently sharing it.
+
+ Unlike :func:`get_library_storage_path`, this works from an
+ already-resolved base path plus an explicit ``shared_library`` flag, so
+ background threads (the scheduler download path) and snapshot-based
+ callers (collection/library search engines) get the same layout without
+ depending on request-context settings resolution.
+ """
+ # Expand $VARs (parity with the write path in DownloadService.__init__);
+ # the read-side resolvers previously only expanduser()'d, so a storage_path
+ # containing an env-var token resolved to a different, non-existent path on
+ # read than on write (files became unfindable, tracker state corrupted).
+ base = Path(os.path.expandvars(str(base_path))).expanduser().resolve()
+ # ``shared_library`` removes the per-user directory boundary. Because both
+ # ``research_library.shared_library`` and ``research_library.storage_path``
+ # are user-editable, a multi-tenant attacker could otherwise set their own
+ # shared_library=true and point storage_path at a victim's directory to
+ # read/overwrite their library PDFs. So shared mode only takes effect when
+ # an operator has explicitly enabled it via the environment; otherwise the
+ # per-user subdirectory is always enforced.
+ if (shared_library and _shared_library_allowed()) or not username:
+ return base
+ _reject_unsafe_username_component(username)
+ return base / username
+
+
+def _shared_library_allowed() -> bool:
+ """Whether the operator enabled shared-library mode (env-only gate).
+
+ Environment-only, mirroring ``filesystem_pdf_storage_allowed`` and
+ ``policy.allow_unprotected_egress``: shared mode drops the per-user
+ directory isolation, so it cannot be turned on through the user-writable
+ settings API — only via
+ ``LDR_RESEARCH_LIBRARY_ALLOW_SHARED_LIBRARY=true``.
+ """
+ from ...settings.env_registry import get_env_setting
+
+ return bool(get_env_setting("research_library.allow_shared_library", False))
+
+
+def _legacy_read_fallback_allowed() -> bool:
+ """Whether the operator opted into the legacy shared-root READ fallback.
+
+ Environment-only, mirroring ``_shared_library_allowed`` /
+ ``filesystem_pdf_storage_allowed``. The legacy fallback resolves a
+ per-user read miss against the shared root derived from the user-editable
+ ``research_library.storage_path``. Because a user could point their own
+ ``storage_path`` at another user's directory, and per-user autoincrement
+ resource ids collide by construction, that fallback is a cross-tenant
+ read primitive on a multi-tenant instance. It is therefore OFF by default
+ and can only be enabled by the operator via
+ ``LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK=true`` — never through
+ the user-writable settings API. When off, reads resolve strictly within
+ the caller's own per-user root.
+ """
+ from ...settings.env_registry import get_env_setting
+
+ return bool(
+ get_env_setting("research_library.allow_legacy_read_fallback", False)
+ )
+
+
def get_library_storage_path(username: str) -> Path:
"""
Get the storage path for a user's library.
@@ -219,15 +319,9 @@ def get_library_storage_path(username: str) -> Path:
settings = get_settings_manager()
# Get the base path from settings (uses centralized path, respects LDR_DATA_DIR)
- base_path = (
- Path(
- settings.get_setting(
- "research_library.storage_path",
- str(get_library_directory()),
- )
- )
- .expanduser()
- .resolve()
+ base_path = settings.get_setting(
+ "research_library.storage_path",
+ str(get_library_directory()),
)
# Check if shared library mode is enabled
@@ -235,14 +329,9 @@ def get_library_storage_path(username: str) -> Path:
"research_library.shared_library", False
)
- if shared_library:
- # Shared mode: all users use the same directory
- base_path.mkdir(parents=True, exist_ok=True)
- return base_path
- # Default: user isolation with subdirectories
- user_dir = base_path / username
- user_dir.mkdir(parents=True, exist_ok=True)
- return user_dir
+ storage_path = apply_user_subdir(base_path, username, shared_library)
+ storage_path.mkdir(parents=True, exist_ok=True)
+ return storage_path
def open_file_location(file_path: str) -> bool:
@@ -317,50 +406,163 @@ def get_absolute_library_path(
return None
-def get_absolute_path_from_settings(relative_path: str) -> Optional[Path]:
+def _resolve_within_root(
+ relative_path: str, library_root: Path
+) -> Optional[Path]:
+ """Validate ``relative_path`` inside ``library_root`` (traversal + symlink
+ safe). Returns the absolute path, or ``None`` if unsafe."""
+ try:
+ safe_path = PathValidator.validate_safe_path(
+ relative_path, str(library_root)
+ )
+ if safe_path is None:
+ return None
+ result = Path(safe_path)
+ if result.is_symlink():
+ logger.warning(f"Symlink blocked: {relative_path}")
+ return None
+ return result
+ except ValueError:
+ logger.warning(f"Path traversal blocked: {relative_path}")
+ return None
+
+
+def get_absolute_path_from_settings(
+ relative_path: str,
+ username: Optional[str] = None,
+ *,
+ allow_legacy_fallback: bool = True,
+ settings_manager=None,
+) -> Optional[Path]:
"""
Get absolute path using settings manager for library root.
Uses PathValidator to prevent path traversal attacks.
+ ``settings_manager`` lets a caller supply an explicit, already-scoped
+ settings manager instead of the ambient ``get_settings_manager()``. This
+ matters in background/scheduler threads with no Flask request context:
+ there the ambient manager resolves to a *db-less* manager that reads only
+ env vars and the defaults file, so a user's UI-customized
+ ``research_library.storage_path`` / ``research_library.shared_library``
+ (stored only in their per-user encrypted DB) is invisible and the path
+ silently resolves against the default library location — the wrong root.
+ ``DownloadService`` passes its captured ``self.settings`` here so these
+ reads stay consistent with the per-user root it computed in ``__init__``
+ (issue #5521). When ``None`` the ambient manager is used, preserving the
+ request-context behavior for existing callers.
+
+ When ``username`` is provided the path is resolved against that user's
+ per-user library directory (issue #5521). If the file is not found there,
+ it falls back to the legacy shared root so PDFs downloaded before per-user
+ isolation still load correctly (no data loss). When ``username``
+ is ``None`` the legacy shared-root behavior is preserved unchanged for
+ callers that have no user context.
+
+ ``allow_legacy_fallback`` must be ``False`` for *destructive* callers
+ (delete / unlink). The legacy shared root is not per-user namespaced, and
+ per-user autoincrement resource ids collide by construction (each user has
+ an independently-numbered database), so filenames like ``pdfs/3.pdf`` can
+ map to *another* tenant's file in the shared root. Resolving-then-unlinking
+ that path would delete a different user's PDF. Because migration 0029 does
+ not move pre-isolation files into per-user subdirectories, the per-user
+ location is always empty for legacy documents and the fallback would fire
+ deterministically. Destructive callers therefore resolve only within the
+ caller's own per-user root; a missing file there means "nothing of mine to
+ delete" rather than reaching into shared storage. As a fail-closed guard,
+ a destructive call (``allow_legacy_fallback=False``) with a falsy
+ ``username`` raises ``ValueError`` rather than resolving to the bare
+ shared root: without a user context there is no per-user directory to
+ scope the unlink to, and silently resolving into the shared root could
+ unlink another tenant's file.
+
+ Even for read-only callers the legacy shared-root fallback is a
+ cross-tenant read primitive on a multi-tenant instance: the shared root is
+ derived from the user-editable ``research_library.storage_path``, so a user
+ can point their own ``storage_path`` at another user's directory and — via
+ the colliding-id fallback — read that user's PDFs. The read fallback is
+ therefore gated behind the operator-only
+ ``research_library.allow_legacy_read_fallback`` env setting and is OFF by
+ default; when off, reads resolve strictly within the caller's own per-user
+ root.
+
Args:
relative_path: The relative path from library root
+ username: Optional username for per-user resolution
+ allow_legacy_fallback: Permit the legacy shared-root fallback for
+ read-only callers. Pass ``False`` from delete/unlink paths. Even
+ when ``True``, the fallback only fires if the operator enabled
+ ``research_library.allow_legacy_read_fallback``.
+ settings_manager: Optional explicit settings manager to read
+ ``storage_path`` / ``shared_library`` from. Defaults to the
+ ambient ``get_settings_manager()``; pass a captured, user-scoped
+ manager from background/scheduler threads (see above).
Returns:
The absolute path, or None if the path is unsafe
+
+ Raises:
+ ValueError: If ``allow_legacy_fallback`` is ``False`` (destructive
+ caller) and ``username`` is falsy — fail closed rather than
+ resolve within the shared root.
"""
- from ...utilities.db_utils import get_settings_manager
+ # Fail closed: a destructive caller with no user context has no per-user
+ # directory to scope its unlink to. apply_user_subdir(base, "", ...) would
+ # otherwise return the bare shared root, so resolving+unlinking there could
+ # destroy another tenant's colliding file. Read-only callers
+ # (allow_legacy_fallback=True) keep the legacy no-username behavior.
+ if not allow_legacy_fallback and not username:
+ raise ValueError(
+ "Refusing to resolve a destructive library path without a user "
+ "context (would resolve within the shared root)."
+ )
- settings = get_settings_manager()
- library_root = (
+ if settings_manager is None:
+ from ...utilities.db_utils import get_settings_manager
+
+ settings = get_settings_manager()
+ else:
+ settings = settings_manager
+ base_path = (
Path(
- settings.get_setting(
- "research_library.storage_path",
- str(get_library_directory()),
+ os.path.expandvars(
+ settings.get_setting(
+ "research_library.storage_path",
+ str(get_library_directory()),
+ )
)
)
.expanduser()
.resolve()
)
+ shared_library = settings.get_setting(
+ "research_library.shared_library", False
+ )
+ per_user_root = apply_user_subdir(base_path, username, shared_library)
if not relative_path:
- return library_root
-
- try:
- # Use PathValidator to prevent path traversal attacks
- safe_path = PathValidator.validate_safe_path(
- relative_path, str(library_root)
- )
- if safe_path is None:
- return None
- result = Path(safe_path)
- if result.is_symlink():
- logger.warning(f"Symlink blocked: {relative_path}")
- return None
- return result
- except ValueError:
- logger.warning(f"Path traversal blocked: {relative_path}")
- return None
+ # An empty path asks for the (per-user) library root itself.
+ return per_user_root
+
+ primary = _resolve_within_root(relative_path, per_user_root)
+ # Legacy fallback: when the per-user location has no file, look in the
+ # legacy shared root where pre-isolation downloads still live. Gated three
+ # ways: (1) read-only callers only — destructive callers pass
+ # allow_legacy_fallback=False so they never unlink a colliding file that
+ # belongs to another tenant; (2) the operator must have opted into the
+ # shared-root read fallback, since it is a cross-tenant read primitive
+ # when a user points storage_path at another user's directory; (3) the
+ # per-user root must actually differ from the shared root.
+ if (
+ allow_legacy_fallback
+ and _legacy_read_fallback_allowed()
+ and per_user_root != base_path
+ and (primary is None or not primary.exists())
+ ):
+ legacy = _resolve_within_root(relative_path, base_path)
+ if legacy is not None and legacy.exists():
+ return legacy
+ return primary
def handle_api_error(operation: str, error: Exception, status_code: int = 500):
diff --git a/src/local_deep_research/research_library/zotero/sync_service.py b/src/local_deep_research/research_library/zotero/sync_service.py
index 8fc7ddf27c..60facd241c 100644
--- a/src/local_deep_research/research_library/zotero/sync_service.py
+++ b/src/local_deep_research/research_library/zotero/sync_service.py
@@ -65,8 +65,9 @@ from ..deletion.utils.cascade_helper import CascadeHelper
from ..services.pdf_storage_manager import (
DEFAULT_MAX_PDF_SIZE_MB,
PDFStorageManager,
+ resolve_pdf_storage_mode,
)
-from ..utils import ensure_in_collection
+from ..utils import apply_user_subdir, ensure_in_collection
from .client import (
ZoteroClient,
ZoteroError,
@@ -272,15 +273,27 @@ class ZoteroSyncService:
"zotero.org/settings/keys."
)
- def _library_root(self) -> str:
+ def _library_root(self) -> Path:
+ # Per-user library root (issue #5521), mirroring the DownloadService
+ # write path: the base storage directory from this user's settings,
+ # narrowed to the per-user subdirectory via ``apply_user_subdir``
+ # (shared mode is an operator-gated env opt-in). Today
+ # ``zotero.pdf_storage_mode`` only offers ``database`` (encrypted) and
+ # ``none`` (text-only), so this root is a formality for the DB-blob
+ # path — but building the PDFStorageManager on the per-user root (and
+ # routing the mode through ``resolve_pdf_storage_mode`` at the call
+ # site) keeps this write path from ever escaping per-user isolation or
+ # the unencrypted-filesystem gate if the mode set ever changes.
with get_user_db_session(self.username, self.password) as session:
sm = SettingsManager(session)
storage_path = sm.get_setting(
"research_library.storage_path", str(get_library_directory())
)
- return str(
- Path(os.path.expandvars(storage_path)).expanduser().resolve()
- )
+ shared_library = sm.get_setting(
+ "research_library.shared_library", False
+ )
+ base = Path(os.path.expandvars(storage_path)).expanduser().resolve()
+ return apply_user_subdir(base, self.username, shared_library)
# -- public operations -------------------------------------------------
@@ -610,9 +623,16 @@ class ZoteroSyncService:
)
]
+ # Route the mode through the operator gate and write to the
+ # per-user root — mirroring the DownloadService write path — so
+ # this construction cannot bypass the PR's own enforcement even
+ # if ``zotero.pdf_storage_mode`` ever gains a ``filesystem``
+ # option. ``resolve_pdf_storage_mode`` coerces a gated
+ # ``filesystem`` value to the encrypted ``database`` default;
+ # today it is a no-op for the ``database``/``none`` choices.
pdf_storage = PDFStorageManager(
library_root=self._library_root(),
- storage_mode=cfg.pdf_storage_mode,
+ storage_mode=resolve_pdf_storage_mode(cfg.pdf_storage_mode),
max_pdf_size_mb=DEFAULT_MAX_PDF_SIZE_MB,
)
diff --git a/src/local_deep_research/settings/env_definitions/security.py b/src/local_deep_research/settings/env_definitions/security.py
index 7d93ac37d9..ddfdd500e1 100644
--- a/src/local_deep_research/settings/env_definitions/security.py
+++ b/src/local_deep_research/settings/env_definitions/security.py
@@ -87,4 +87,67 @@ SECURITY_SETTINGS = [
),
default=False,
),
+ BooleanSetting(
+ key="research_library.allow_filesystem_pdf_storage",
+ description=(
+ "Allow users to select the UNENCRYPTED 'filesystem' PDF storage "
+ "mode for library-downloaded PDFs. Disabled by default; this is "
+ "an environment-only operator gate and cannot be changed through "
+ "the user-writable settings API. Filesystem mode writes fetched "
+ "third-party PDFs as PLAINTEXT to a shared library directory "
+ "(cleartext storage of sensitive information, CWE-312). When "
+ "off, the 'filesystem' option is withheld from the settings UI "
+ "and any stored or "
+ "environment value of 'filesystem' is coerced to the encrypted "
+ "'database' mode at write time. Previously-written plaintext "
+ "files remain readable. Enable only when the library directory "
+ "lives on an operator-controlled encrypted volume."
+ ),
+ default=False,
+ ),
+ BooleanSetting(
+ key="research_library.allow_shared_library",
+ description=(
+ "Allow shared-library mode, which drops the per-user library "
+ "directory boundary so all users' downloaded PDFs live in one "
+ "shared directory. Disabled by default; this is an "
+ "environment-only operator gate and cannot be enabled through the "
+ "user-writable settings API. Because both the shared_library flag "
+ "and the storage_path are otherwise user-editable, leaving shared "
+ "mode user-toggleable would let a multi-tenant user point their "
+ "own storage_path at another user's directory and read/overwrite "
+ "their PDFs. When off, the per-user subdirectory is always "
+ "enforced regardless of the user's shared_library setting. Enable "
+ "only on single-tenant or mutually-trusted deployments."
+ ),
+ default=False,
+ ),
+ BooleanSetting(
+ # NOTE: risk-bearing opt-in. Enabling this re-opens a cross-tenant
+ # library-PDF READ. The description below is the operator-facing
+ # warning; keep the two in sync. Env var (auto-derived from the key):
+ # LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK.
+ key="research_library.allow_legacy_read_fallback",
+ description=(
+ "SECURITY RISK — leave DISABLED (default) on any multi-user or "
+ "untrusted deployment. When ENABLED, library PDF reads that miss "
+ "in the caller's own per-user directory fall back to a shared "
+ "root derived from the user-editable "
+ "'research_library.storage_path' setting. On a multi-user "
+ "instance this re-opens a CROSS-TENANT READ: a user can edit "
+ "their own storage_path to point at another user's library "
+ "directory, and because per-user autoincrement resource ids "
+ "collide by construction (e.g. 'pdfs/5.pdf'), a read of their own "
+ "document id then resolves — via this fallback — to the OTHER "
+ "user's PDF and returns its contents. The safe state is OFF: with "
+ "the gate off, reads resolve strictly within the requesting "
+ "user's own per-user root and the fallback never fires. This is "
+ "an environment-only operator gate and cannot be enabled through "
+ "the user-writable settings API. Enable ONLY on a single-user or "
+ "fully-trusted deployment that needs PDFs downloaded before "
+ "per-user isolation (issue #5521) to keep loading from the legacy "
+ "shared location."
+ ),
+ default=False,
+ ),
]
diff --git a/src/local_deep_research/web/routes/settings_routes.py b/src/local_deep_research/web/routes/settings_routes.py
index a14fca630f..81401f9700 100644
--- a/src/local_deep_research/web/routes/settings_routes.py
+++ b/src/local_deep_research/web/routes/settings_routes.py
@@ -392,6 +392,47 @@ def _shape_egress_scope_metadata(settings):
return shaped
+def _shape_pdf_storage_mode_metadata(settings):
+ """Hide the operator-gated unencrypted 'filesystem' PDF storage option.
+
+ Mirrors ``_shape_egress_scope_metadata``: when the operator gate
+ ``research_library.allow_filesystem_pdf_storage`` is off (the default),
+ the ``filesystem`` choice is stripped from the
+ ``research_library.pdf_storage_mode`` options served to the settings UI,
+ leaving the encrypted ``database`` default and ``none``. Consumption-site
+ coercion (``resolve_pdf_storage_mode``) is the actual protection; this
+ just keeps a disabled option out of the dropdown.
+ """
+ if not isinstance(settings, dict):
+ return settings
+ metadata = settings.get("research_library.pdf_storage_mode")
+ if not isinstance(metadata, dict):
+ return settings
+ options = metadata.get("options")
+ if not isinstance(options, list):
+ return settings
+ # Lazy import to avoid importing the research_library package (which pulls
+ # in blueprints) at settings-routes module load time.
+ from ...research_library.services.pdf_storage_manager import (
+ filesystem_pdf_storage_allowed,
+ )
+
+ if filesystem_pdf_storage_allowed():
+ return settings
+ shaped = dict(settings)
+ mode = dict(metadata)
+ mode["options"] = [
+ option
+ for option in options
+ if str(option.get("value") if isinstance(option, dict) else option)
+ .strip()
+ .lower()
+ != "filesystem"
+ ]
+ shaped["research_library.pdf_storage_mode"] = mode
+ return shaped
+
+
def _shape_effective_setting_metadata(settings):
"""Apply deployment overrides to settings API response metadata."""
if not isinstance(settings, dict):
@@ -413,7 +454,9 @@ def _shape_effective_setting_metadata(settings):
)
current["editable"] = False
shaped[key] = current
- return _shape_egress_scope_metadata(shaped)
+ return _shape_pdf_storage_mode_metadata(
+ _shape_egress_scope_metadata(shaped)
+ )
def _shape_single_effective_metadata(key, setting_data):
@@ -930,6 +973,26 @@ def save_settings(db_session: Optional[Session] = None, settings_manager=None):
ui_element=db_setting.ui_element,
)
+ # Validate against the setting's constraints (options
+ # membership, numeric bounds, checkbox type) — the same
+ # check the JSON PUT (api_update_setting) and
+ # save_all_settings run. Without it this JS-disabled POST
+ # fallback wrote values unchecked, letting an authenticated
+ # client persist an out-of-options value (e.g. a withheld
+ # research_library.pdf_storage_mode) that the other two
+ # save paths reject.
+ if value is not None:
+ is_valid, error_message = validate_setting(
+ db_setting, value
+ )
+ if not is_valid:
+ logger.warning(
+ f"Validation failed for setting {key}: "
+ f"{error_message}"
+ )
+ failed_count += 1
+ continue
+
# Save the setting
if settings_manager.set_setting(key, value, commit=False):
updated_count += 1
diff --git a/src/local_deep_research/web_search_engines/engines/search_engine_collection.py b/src/local_deep_research/web_search_engines/engines/search_engine_collection.py
index 12213c231b..bcb5bbf7c0 100644
--- a/src/local_deep_research/web_search_engines/engines/search_engine_collection.py
+++ b/src/local_deep_research/web_search_engines/engines/search_engine_collection.py
@@ -329,18 +329,32 @@ class CollectionSearchEngine(LibraryRAGSearchEngine):
if document:
from pathlib import Path
+ from ...research_library.utils import apply_user_subdir
+
library_root = get_setting_from_snapshot(
"research_library.storage_path",
default=str(get_library_directory()),
settings_snapshot=self.settings_snapshot,
)
- library_root = (
+ base_root = (
Path(os.path.expandvars(library_root))
.expanduser()
.resolve()
)
+ shared_library = get_setting_from_snapshot(
+ "research_library.shared_library",
+ default=False,
+ settings_snapshot=self.settings_snapshot,
+ )
+ # Per-user root with legacy-shared fallback (issue #5521).
+ per_user_root = apply_user_subdir(
+ base_root, self.username, shared_library
+ )
if PDFStorageManager.pdf_exists(
- library_root, document, session
+ per_user_root,
+ document,
+ session,
+ legacy_root=base_root,
):
document_url = f"/library/document/{doc_id}/pdf"
except Exception as e:
diff --git a/src/local_deep_research/web_search_engines/engines/search_engine_library.py b/src/local_deep_research/web_search_engines/engines/search_engine_library.py
index 7a286fe3a8..4d3809a77f 100644
--- a/src/local_deep_research/web_search_engines/engines/search_engine_library.py
+++ b/src/local_deep_research/web_search_engines/engines/search_engine_library.py
@@ -377,19 +377,36 @@ class LibraryRAGSearchEngine(BaseSearchEngine):
)
if document:
from pathlib import Path
+ from ...research_library.utils import (
+ apply_user_subdir,
+ )
library_root = get_setting_from_snapshot(
"research_library.storage_path",
default=str(get_library_directory()),
settings_snapshot=self.settings_snapshot,
)
- library_root = (
+ base_root = (
Path(os.path.expandvars(library_root))
.expanduser()
.resolve()
)
+ shared_library = get_setting_from_snapshot(
+ "research_library.shared_library",
+ default=False,
+ settings_snapshot=self.settings_snapshot,
+ )
+ # Per-user root, legacy-shared fallback (#5521).
+ per_user_root = apply_user_subdir(
+ base_root,
+ self.username,
+ shared_library,
+ )
if PDFStorageManager.pdf_exists(
- library_root, document, session
+ per_user_root,
+ document,
+ session,
+ legacy_root=base_root,
):
document_url = (
f"/library/document/{doc_id}/pdf"
diff --git a/tests/database/test_alembic_migrations.py b/tests/database/test_alembic_migrations.py
index 90eb79c165..737076106e 100644
--- a/tests/database/test_alembic_migrations.py
+++ b/tests/database/test_alembic_migrations.py
@@ -6086,6 +6086,11 @@ class TestMigrationSafetyGuards:
# downgrade cannot know which rows to restore. Rewriting 'adaptive'
# back to 'both' would clobber the legitimate default. No-op by design.
"0019_retire_both_egress_scope.py",
+ # 0029: the 'filesystem' -> 'database' coercion is lossy — after upgrade
+ # a row is indistinguishable from an originally-'database' one, so a
+ # downgrade cannot know which rows were originally 'filesystem'. Same
+ # rationale as 0019/0027. No-op by design.
+ "0029_gate_filesystem_pdf_storage.py",
}
# Migrations whose downgrade is intentionally NotImplementedError
diff --git a/tests/database/test_migration_0028_expand_rag_index_identity.py b/tests/database/test_migration_0028_expand_rag_index_identity.py
index 10e9d064f4..864952791c 100644
--- a/tests/database/test_migration_0028_expand_rag_index_identity.py
+++ b/tests/database/test_migration_0028_expand_rag_index_identity.py
@@ -5,7 +5,6 @@ import pytest
from local_deep_research.database.alembic_runner import (
get_alembic_config,
- get_head_revision,
get_migrations_dir,
stamp_database,
)
@@ -90,7 +89,15 @@ def legacy_engine(tmp_path):
engine.dispose()
-def test_0028_is_head_and_chains_to_0027():
+def test_0028_chains_correctly_to_0027():
+ """0028 (expand_rag_index_identity) chains directly off 0027.
+
+ Originally this asserted ``get_head_revision() == "0028"``, but a later
+ migration added 0029 (gate filesystem PDF storage) on top, so head moved
+ past 0028. The substantive invariant the original test protected — that
+ 0028 is correctly anchored in the chain — survives by checking
+ down_revision instead. Mirrors ``TestMigration0009HeadAlignment``.
+ """
from alembic.config import Config
from alembic.script import ScriptDirectory
@@ -98,7 +105,6 @@ def test_0028_is_head_and_chains_to_0027():
config.set_main_option("script_location", str(get_migrations_dir()))
script = ScriptDirectory.from_config(config)
- assert get_head_revision() == "0028"
assert script.get_revision("0028").down_revision == "0027"
diff --git a/tests/database/test_migration_0029_gate_filesystem_pdf_storage.py b/tests/database/test_migration_0029_gate_filesystem_pdf_storage.py
new file mode 100644
index 0000000000..b02a5cf66f
--- /dev/null
+++ b/tests/database/test_migration_0029_gate_filesystem_pdf_storage.py
@@ -0,0 +1,276 @@
+"""Tests for migration 0029: reset unencrypted filesystem PDF-storage mode."""
+
+import json
+
+import pytest
+from alembic import command
+from sqlalchemy import create_engine, text
+
+from local_deep_research.database.alembic_runner import get_alembic_config
+
+KEY = "research_library.pdf_storage_mode"
+
+
+def _upgrade(engine, revision):
+ config = get_alembic_config(engine)
+ with engine.begin() as conn:
+ config.attributes["connection"] = conn
+ command.upgrade(config, revision)
+
+
+def migrated_to_0028_engine(tmp_path):
+ engine = create_engine(f"sqlite:///{tmp_path / 'test_0029.db'}")
+ _upgrade(engine, "0028")
+ yield engine
+ engine.dispose()
+
+
+def _seed(engine, value):
+ with engine.begin() as conn:
+ conn.execute(
+ text(
+ "INSERT INTO settings "
+ "(key, value, type, name, ui_element, visible, editable) "
+ "VALUES (:key, :value, 'app', "
+ "'PDF Storage Mode', 'select', 1, 1)"
+ ),
+ {"key": KEY, "value": json.dumps(value)},
+ )
+
+
+def _read(engine):
+ with engine.begin() as conn:
+ raw = conn.execute(
+ text("SELECT value FROM settings WHERE key=:key"),
+ {"key": KEY},
+ ).scalar_one()
+ return json.loads(raw)
+
+
+def _seed_queued(engine, research_id, snapshot):
+ with engine.begin() as conn:
+ conn.execute(
+ text(
+ "INSERT INTO queued_researches "
+ "(username, research_id, query, mode, settings_snapshot, "
+ "position) "
+ "VALUES ('tester', :research_id, 'q', 'quick', :snapshot, 1)"
+ ),
+ {"research_id": research_id, "snapshot": json.dumps(snapshot)},
+ )
+
+
+def _read_queued(engine, research_id):
+ with engine.begin() as conn:
+ raw = conn.execute(
+ text(
+ "SELECT settings_snapshot FROM queued_researches "
+ "WHERE research_id = :research_id"
+ ),
+ {"research_id": research_id},
+ ).scalar_one()
+ return json.loads(raw)
+
+
[email protected]("legacy", ["filesystem", "FILESYSTEM", " filesystem "])
+def test_filesystem_is_reset_to_database(migrated_to_0028_engine, legacy):
+ _seed(migrated_to_0028_engine, legacy)
+ _upgrade(migrated_to_0028_engine, "0029")
+ assert _read(migrated_to_0028_engine) == "database"
+
+
[email protected]("mode", ["database", "none"])
+def test_other_modes_are_preserved(migrated_to_0028_engine, mode):
+ _seed(migrated_to_0028_engine, mode)
+ _upgrade(migrated_to_0028_engine, "0029")
+ assert _read(migrated_to_0028_engine) == mode
+
+
[email protected]("value", [None, 123, True])
+def test_settings_non_string_value_is_left_untouched(
+ migrated_to_0028_engine, value
+):
+ """A corrupted non-string row is not 'filesystem' and is left byte-for-byte."""
+ _seed(migrated_to_0028_engine, value)
+
+ def _raw_stored():
+ with migrated_to_0028_engine.begin() as conn:
+ return conn.execute(
+ text("SELECT value FROM settings WHERE key=:key"),
+ {"key": KEY},
+ ).scalar_one()
+
+ before = _raw_stored()
+ _upgrade(migrated_to_0028_engine, "0029")
+ assert _raw_stored() == before
+
+
+ ("research_id", "snapshot", "expected"),
+ [
+ (
+ "wrapped-scalar",
+ {
+ "submission": {"query": "queued query", "mode": "deep"},
+ "settings_snapshot": {
+ KEY: " FileSystem ",
+ "search.tool": {"value": "searxng", "type": "select"},
+ },
+ "queue_metadata": {"attempt": 2},
+ },
+ {
+ "submission": {"query": "queued query", "mode": "deep"},
+ "settings_snapshot": {
+ KEY: "database",
+ "search.tool": {"value": "searxng", "type": "select"},
+ },
+ "queue_metadata": {"attempt": 2},
+ },
+ ),
+ (
+ "wrapped-metadata",
+ {
+ "submission": {"query": "queued query"},
+ "settings_snapshot": {
+ KEY: {
+ "value": "filesystem",
+ "type": "select",
+ "label": "PDF Storage Mode",
+ },
+ },
+ },
+ {
+ "submission": {"query": "queued query"},
+ "settings_snapshot": {
+ KEY: {
+ "value": "database",
+ "type": "select",
+ "label": "PDF Storage Mode",
+ },
+ },
+ },
+ ),
+ (
+ "flat-scalar",
+ {KEY: "filesystem", "query": "legacy queued query"},
+ {KEY: "database", "query": "legacy queued query"},
+ ),
+ (
+ "flat-metadata",
+ {
+ KEY: {"value": " filesystem ", "type": "select"},
+ "query": "legacy queued query",
+ },
+ {
+ KEY: {"value": "database", "type": "select"},
+ "query": "legacy queued query",
+ },
+ ),
+ ],
+)
+def test_queued_filesystem_modes_are_reset_to_database(
+ migrated_to_0028_engine, research_id, snapshot, expected
+):
+ _seed_queued(migrated_to_0028_engine, research_id, snapshot)
+ _upgrade(migrated_to_0028_engine, "0029")
+ assert _read_queued(migrated_to_0028_engine, research_id) == expected
+
+
+def test_queued_other_mode_is_preserved(migrated_to_0028_engine):
+ snapshot = {
+ "settings_snapshot": {
+ KEY: {"value": "none", "type": "select"},
+ "search.tool": "searxng",
+ },
+ }
+ _seed_queued(migrated_to_0028_engine, "preserved", snapshot)
+ _upgrade(migrated_to_0028_engine, "0029")
+ assert _read_queued(migrated_to_0028_engine, "preserved") == snapshot
+
+
+ ("research_id", "raw_snapshot"),
+ [
+ ("malformed-json", "{not valid json"),
+ ("non-dict-top-level", json.dumps(["filesystem"])),
+ (
+ "non-dict-nested-snapshot",
+ json.dumps(
+ {"submission": {"query": "q"}, "settings_snapshot": None}
+ ),
+ ),
+ ],
+)
+def test_unrecognized_snapshots_are_left_untouched(
+ migrated_to_0028_engine, research_id, raw_snapshot
+):
+ with migrated_to_0028_engine.begin() as conn:
+ conn.execute(
+ text(
+ "INSERT INTO queued_researches "
+ "(username, research_id, query, mode, settings_snapshot, "
+ "position) "
+ "VALUES ('tester', :research_id, 'q', 'quick', :snapshot, 1)"
+ ),
+ {"research_id": research_id, "snapshot": raw_snapshot},
+ )
+ _seed_queued(
+ migrated_to_0028_engine,
+ "still-migrated",
+ {KEY: "filesystem", "query": "q"},
+ )
+
+ _upgrade(migrated_to_0028_engine, "0029")
+
+ with migrated_to_0028_engine.begin() as conn:
+ untouched = conn.execute(
+ text(
+ "SELECT settings_snapshot FROM queued_researches "
+ "WHERE research_id = :research_id"
+ ),
+ {"research_id": research_id},
+ ).scalar_one()
+ assert untouched == raw_snapshot
+ assert _read_queued(migrated_to_0028_engine, "still-migrated") == {
+ KEY: "database",
+ "query": "q",
+ }
+
+
+def test_settings_are_migrated_without_queued_researches_table(
+ migrated_to_0028_engine,
+):
+ _seed(migrated_to_0028_engine, "filesystem")
+ with migrated_to_0028_engine.begin() as conn:
+ conn.execute(text("DROP TABLE queued_researches"))
+
+ _upgrade(migrated_to_0028_engine, "0029")
+
+ assert _read(migrated_to_0028_engine) == "database"
+
+
+def test_downgrade_is_noop(migrated_to_0028_engine):
+ _seed(migrated_to_0028_engine, "filesystem")
+ _upgrade(migrated_to_0028_engine, "0029")
+ assert _read(migrated_to_0028_engine) == "database"
+
+ config = get_alembic_config(migrated_to_0028_engine)
+ with migrated_to_0028_engine.begin() as conn:
+ config.attributes["connection"] = conn
+ command.downgrade(config, "0028")
+
+ # Downgrade must NOT restore the plaintext selection.
+ assert _read(migrated_to_0028_engine) == "database"
+
+
+def test_0029_chains_to_0028():
+ from alembic.config import Config
+ from alembic.script import ScriptDirectory
+
+ from local_deep_research.database.alembic_runner import get_migrations_dir
+
+ config = Config()
+ config.set_main_option("script_location", str(get_migrations_dir()))
+ script = ScriptDirectory.from_config(config)
+ assert script.get_revision("0029").down_revision == "0028"
diff --git a/tests/research_library/services/test_download_service.py b/tests/research_library/services/test_download_service.py
index 3a1185a7b7..9c3dc00988 100644
--- a/tests/research_library/services/test_download_service.py
+++ b/tests/research_library/services/test_download_service.py
@@ -36,7 +36,8 @@ class TestDownloadServiceInit:
assert service.username == "test_user"
assert len(service.downloaders) > 0
- assert service.library_root == "/tmp/test_library"
+ # Per-user isolation (default) appends the username subdir.
+ assert service.library_root == "/tmp/test_library/test_user"
class TestDownloadServiceUrlNormalization:
@@ -1439,8 +1440,8 @@ class TestSaveTextWithDb:
service = DownloadService(username="test_user")
- # Service is properly configured
- assert service.library_root == "/tmp/test_library"
+ # Service is properly configured (per-user isolation appends username)
+ assert service.library_root == "/tmp/test_library/test_user"
def test_save_text_with_db_handles_serialization(self, mocker):
"""Handles text content serialization."""
diff --git a/tests/research_library/services/test_download_service_coverage.py b/tests/research_library/services/test_download_service_coverage.py
index 8c06417a58..98cd8f568b 100644
--- a/tests/research_library/services/test_download_service_coverage.py
+++ b/tests/research_library/services/test_download_service_coverage.py
@@ -50,7 +50,8 @@ def svc():
service.downloaders = []
service.retry_manager = MagicMock()
service.settings = MagicMock()
- service.library_root = "/tmp/test_library"
+ service.library_root = "/tmp/test_library/test_user"
+ service.legacy_library_root = "/tmp/test_library"
service._pubmed_delay = 1.0
service._last_pubmed_request = 0.0
return service
@@ -1000,9 +1001,12 @@ class TestTryExistingPdfExtraction:
with (
patch(f"{MODULE}.FILE_PATH_SENTINELS", ()),
- patch(f"{MODULE}.PathValidator") as pv,
+ # Traversal is rejected inside get_absolute_path_from_settings,
+ # which returns None → method returns None.
+ patch(
+ f"{MODULE}.get_absolute_path_from_settings", return_value=None
+ ),
):
- pv.validate_safe_path.side_effect = ValueError("blocked")
assert (
svc._try_existing_pdf_extraction(session, resource, 1) is None
)
@@ -1019,9 +1023,11 @@ class TestTryExistingPdfExtraction:
with (
patch(f"{MODULE}.FILE_PATH_SENTINELS", ()),
- patch(f"{MODULE}.PathValidator") as pv,
+ patch(
+ f"{MODULE}.get_absolute_path_from_settings",
+ return_value=tmp_path / "missing.pdf",
+ ),
):
- pv.validate_safe_path.return_value = str(tmp_path / "missing.pdf")
assert (
svc._try_existing_pdf_extraction(session, resource, 1) is None
)
@@ -1042,13 +1048,15 @@ class TestTryExistingPdfExtraction:
with (
patch(f"{MODULE}.FILE_PATH_SENTINELS", ()),
- patch(f"{MODULE}.PathValidator") as pv,
+ patch(
+ f"{MODULE}.get_absolute_path_from_settings",
+ return_value=pdf_file,
+ ),
patch.object(
svc, "_extract_text_from_pdf", return_value="extracted text"
),
patch.object(svc, "_save_text_with_db"),
):
- pv.validate_safe_path.return_value = str(pdf_file)
result = svc._try_existing_pdf_extraction(session, resource, 1)
assert result == (True, None)
@@ -1067,10 +1075,12 @@ class TestTryExistingPdfExtraction:
with (
patch(f"{MODULE}.FILE_PATH_SENTINELS", ()),
- patch(f"{MODULE}.PathValidator") as pv,
+ patch(
+ f"{MODULE}.get_absolute_path_from_settings",
+ return_value=pdf_file,
+ ),
patch.object(svc, "_extract_text_from_pdf", return_value=None),
):
- pv.validate_safe_path.return_value = str(pdf_file)
assert (
svc._try_existing_pdf_extraction(session, resource, 1) is None
)
@@ -1090,10 +1100,12 @@ class TestTryExistingPdfExtraction:
with (
patch(f"{MODULE}.FILE_PATH_SENTINELS", ()),
- patch(f"{MODULE}.PathValidator") as pv,
+ patch(
+ f"{MODULE}.get_absolute_path_from_settings",
+ return_value=pdf_file,
+ ),
patch("builtins.open", side_effect=IOError("disk error")),
):
- pv.validate_safe_path.return_value = str(pdf_file)
assert (
svc._try_existing_pdf_extraction(session, resource, 1) is None
)
diff --git a/tests/research_library/services/test_download_service_deep_coverage.py b/tests/research_library/services/test_download_service_deep_coverage.py
index 3881f1c7e3..a2e2637a8a 100644
--- a/tests/research_library/services/test_download_service_deep_coverage.py
+++ b/tests/research_library/services/test_download_service_deep_coverage.py
@@ -32,7 +32,8 @@ def svc():
service.downloaders = []
service.retry_manager = MagicMock()
service.settings = MagicMock()
- service.library_root = "/tmp/test_library"
+ service.library_root = "/tmp/test_library/test_user"
+ service.legacy_library_root = "/tmp/test_library"
service._pubmed_delay = 1.0
service._last_pubmed_request = 0.0
return service
diff --git a/tests/research_library/services/test_download_service_logic.py b/tests/research_library/services/test_download_service_logic.py
index 5504b926a3..a6385d5f72 100644
--- a/tests/research_library/services/test_download_service_logic.py
+++ b/tests/research_library/services/test_download_service_logic.py
@@ -349,9 +349,10 @@ class TestInitialization:
service.close()
def test_library_root_set(self, tmp_path):
- """Library root is set from settings."""
+ """Library root is per-user (<base>/<username>); legacy root is base."""
service = _create_service(tmp_path)
- assert service.library_root == str(tmp_path)
+ assert service.library_root == str(tmp_path / "testuser")
+ assert service.legacy_library_root == str(tmp_path)
service.close()
def test_downloaders_initialized(self, tmp_path):
diff --git a/tests/research_library/services/test_download_service_strategy_coverage.py b/tests/research_library/services/test_download_service_strategy_coverage.py
index 695f1e1a75..6b009522d9 100644
--- a/tests/research_library/services/test_download_service_strategy_coverage.py
+++ b/tests/research_library/services/test_download_service_strategy_coverage.py
@@ -39,7 +39,8 @@ def svc():
service.downloaders = []
service.retry_manager = MagicMock()
service.settings = MagicMock()
- service.library_root = "/tmp/test_library"
+ service.library_root = "/tmp/test_library/test_user"
+ service.legacy_library_root = "/tmp/test_library"
service._pubmed_delay = 1.0
service._last_pubmed_request = 0.0
return service
diff --git a/tests/research_library/services/test_expandvars_path.py b/tests/research_library/services/test_expandvars_path.py
index 0df27e93bd..2a85430e80 100644
--- a/tests/research_library/services/test_expandvars_path.py
+++ b/tests/research_library/services/test_expandvars_path.py
@@ -36,7 +36,8 @@ class TestDownloadServiceExpandvars:
service = DownloadService(username="test_user")
- expected = str(Path(real_home) / "mylib")
+ # Per-user isolation (default) appends the username subdir.
+ expected = str(Path(real_home) / "mylib" / "test_user")
assert service.library_root == expected
assert "$" not in service.library_root
@@ -63,7 +64,8 @@ class TestDownloadServiceExpandvars:
service = DownloadService(username="test_user")
- expected = str(Path(data_dir) / "library")
+ # Per-user isolation (default) appends the username subdir.
+ expected = str(Path(data_dir) / "library" / "test_user")
assert service.library_root == expected
assert "$" not in service.library_root
diff --git a/tests/research_library/services/test_library_service_collections_stats.py b/tests/research_library/services/test_library_service_collections_stats.py
index a40b7c4cb5..072ded666d 100644
--- a/tests/research_library/services/test_library_service_collections_stats.py
+++ b/tests/research_library/services/test_library_service_collections_stats.py
@@ -931,10 +931,6 @@ class TestOpenFileLocation:
mocker.patch(
"local_deep_research.research_library.services.library_service.get_absolute_path_from_settings",
- return_value=MagicMock(),
- )
- mocker.patch(
- "local_deep_research.research_library.services.library_service.PathValidator.validate_safe_path",
return_value=mock_validated_path,
)
mock_open_fn = mocker.patch(
diff --git a/tests/research_library/services/test_library_service_coverage.py b/tests/research_library/services/test_library_service_coverage.py
index 4a44cdeda8..87f0c7b37c 100644
--- a/tests/research_library/services/test_library_service_coverage.py
+++ b/tests/research_library/services/test_library_service_coverage.py
@@ -388,8 +388,12 @@ class TestOpenFileLocationMissingBranches:
assert result is False
- def test_path_validation_raises_value_error_returns_false(self, mocker):
- """PathValidator.validate_safe_path raises ValueError → returns False."""
+ def test_unresolvable_or_missing_path_returns_false(self, mocker):
+ """Resolved path exists but is not a file → returns False.
+
+ Path traversal is now rejected inside get_absolute_path_from_settings
+ (it returns None); a resolution that is not a file also yields False.
+ """
service = _make_service()
mock_session = MagicMock()
@@ -397,7 +401,7 @@ class TestOpenFileLocationMissingBranches:
mock_doc.original_url = "https://example.com/doc.pdf"
mock_tracker = Mock()
- mock_tracker.file_path = "../../etc/passwd" # traversal attempt
+ mock_tracker.file_path = "pdfs/doc.pdf"
mock_doc_query = MagicMock()
mock_session.get.return_value = mock_doc
@@ -415,13 +419,11 @@ class TestOpenFileLocationMissingBranches:
mock_session.query.side_effect = query_router
_mock_session_cm(mocker, mock_session)
+ resolved = MagicMock()
+ resolved.is_file.return_value = False
mocker.patch(
f"{MODULE}.get_absolute_path_from_settings",
- return_value=MagicMock(), # non-None library root
- )
- mocker.patch(
- f"{MODULE}.PathValidator.validate_safe_path",
- side_effect=ValueError("Path traversal detected"),
+ return_value=resolved,
)
result = service.open_file_location("doc-123")
diff --git a/tests/research_library/services/test_library_service_crud_deep.py b/tests/research_library/services/test_library_service_crud_deep.py
index 20bd859426..b79440b9e6 100644
--- a/tests/research_library/services/test_library_service_crud_deep.py
+++ b/tests/research_library/services/test_library_service_crud_deep.py
@@ -704,7 +704,7 @@ class TestSyncLibraryWithFilesystem:
path_call_count = {"n": 0}
- def path_resolver(fp):
+ def path_resolver(fp, *args, **kwargs):
idx = path_call_count["n"]
path_call_count["n"] += 1
if idx < 3:
diff --git a/tests/research_library/services/test_pdf_storage_gate.py b/tests/research_library/services/test_pdf_storage_gate.py
new file mode 100644
index 0000000000..8fb074cf3a
--- /dev/null
+++ b/tests/research_library/services/test_pdf_storage_gate.py
@@ -0,0 +1,119 @@
+"""Tests for the operator gate on unencrypted filesystem PDF storage.
+
+The ``filesystem`` PDF storage mode writes library-downloaded third-party
+PDFs as plaintext to disk. It is now an environment-only operator gate
+(``research_library.allow_filesystem_pdf_storage`` /
+``LDR_RESEARCH_LIBRARY_ALLOW_FILESYSTEM_PDF_STORAGE``) and disabled by
+default. These tests cover the runtime enforcement point:
+``filesystem_pdf_storage_allowed`` and ``resolve_pdf_storage_mode``, plus the
+consumption behavior (a coerced mode must not reach ``_save_to_filesystem``).
+"""
+
+from unittest.mock import MagicMock, patch
+
+import pytest
+
+from local_deep_research.research_library.services.pdf_storage_manager import (
+ PDFStorageManager,
+ filesystem_pdf_storage_allowed,
+ resolve_pdf_storage_mode,
+)
+
+ENV_VAR = "LDR_RESEARCH_LIBRARY_ALLOW_FILESYSTEM_PDF_STORAGE"
+
+
+def gate_off(monkeypatch):
+ monkeypatch.delenv(ENV_VAR, raising=False)
+
+
+def gate_on(monkeypatch):
+ monkeypatch.setenv(ENV_VAR, "true")
+
+
+class TestFilesystemPdfStorageGate:
+ def test_gate_off_by_default(self, gate_off):
+ assert filesystem_pdf_storage_allowed() is False
+
+ def test_gate_on_when_env_set(self, gate_on):
+ assert filesystem_pdf_storage_allowed() is True
+
+ @pytest.mark.parametrize("truthy", ["true", "1", "yes", "on", "enabled"])
+ def test_gate_on_accepts_truthy_env_values(self, monkeypatch, truthy):
+ monkeypatch.setenv(ENV_VAR, truthy)
+ assert filesystem_pdf_storage_allowed() is True
+
+ def test_resolve_coerces_filesystem_when_gate_off(self, gate_off):
+ assert resolve_pdf_storage_mode("filesystem") == "database"
+
+ @pytest.mark.parametrize(
+ "value", ["FILESYSTEM", " filesystem ", "Filesystem"]
+ )
+ def test_resolve_coercion_is_case_and_padding_insensitive(
+ self, gate_off, value
+ ):
+ assert resolve_pdf_storage_mode(value) == "database"
+
+ def test_resolve_preserves_filesystem_when_gate_on(self, gate_on):
+ assert resolve_pdf_storage_mode("filesystem") == "filesystem"
+
+ @pytest.mark.parametrize("mode", ["database", "none", "", "unknown"])
+ def test_resolve_passthrough_for_non_filesystem_modes(self, gate_off, mode):
+ assert resolve_pdf_storage_mode(mode) == mode
+
+
+class TestGateEnforcedAtConsumption:
+ """A filesystem mode that survives to a PDFStorageManager must, once
+ resolved through the gate, write to the encrypted database rather than
+ plaintext on disk."""
+
+ def test_gate_off_coerced_mode_never_writes_plaintext(
+ self, gate_off, tmp_path, mock_pdf_content
+ ):
+ # Simulate a stored/env value of "filesystem" reaching the consumer.
+ effective_mode = resolve_pdf_storage_mode("filesystem")
+ assert effective_mode == "database"
+
+ manager = PDFStorageManager(tmp_path, effective_mode)
+ mock_doc = MagicMock()
+ mock_doc.id = "doc-123"
+ mock_session = MagicMock()
+ mock_session.query.return_value.filter_by.return_value.first.return_value = None
+
+ with patch.object(manager, "_save_to_filesystem") as save_to_fs:
+ result, _ = manager.save_pdf(
+ mock_pdf_content, mock_doc, mock_session, "test.pdf"
+ )
+
+ # No plaintext write happened; the encrypted DB path was taken.
+ save_to_fs.assert_not_called()
+ assert result == "database"
+ assert mock_doc.storage_mode == "database"
+ assert mock_session.add.called
+ # No plaintext file was created under the library root.
+ assert not list(tmp_path.rglob("*.pdf"))
+
+ def test_gate_on_honors_filesystem_write(
+ self, gate_on, tmp_path, mock_pdf_content
+ ):
+ effective_mode = resolve_pdf_storage_mode("filesystem")
+ assert effective_mode == "filesystem"
+
+ manager = PDFStorageManager(tmp_path, effective_mode)
+ mock_doc = MagicMock()
+ mock_doc.id = "doc-123"
+ mock_session = MagicMock()
+
+ with patch.object(
+ manager,
+ "_save_to_filesystem",
+ return_value=tmp_path / "pdfs" / "test.pdf",
+ ) as save_to_fs:
+ result, _ = manager.save_pdf(
+ mock_pdf_content, mock_doc, mock_session, "test.pdf"
+ )
+
+ save_to_fs.assert_called_once()
+ assert mock_doc.storage_mode == "filesystem"
+ assert result == "pdfs/test.pdf"
diff --git a/tests/research_library/services/test_pdf_storage_manager.py b/tests/research_library/services/test_pdf_storage_manager.py
index 05fc61af19..3644d10526 100644
--- a/tests/research_library/services/test_pdf_storage_manager.py
+++ b/tests/research_library/services/test_pdf_storage_manager.py
@@ -414,7 +414,9 @@ class TestPDFStorageManagerDeletePdf:
def test_deletes_filesystem_file(self, tmp_path, mock_pdf_content):
"""Should delete file from filesystem."""
- manager = PDFStorageManager(tmp_path, "filesystem")
+ # A user context is required for destructive filesystem ops (delete
+ # fails closed without one; see delete_pdf).
+ manager = PDFStorageManager(tmp_path, "filesystem", username="alice")
mock_doc = MagicMock()
mock_doc.id = "doc-123"
mock_doc.storage_mode = "filesystem"
diff --git a/tests/research_library/services/test_per_user_library_root.py b/tests/research_library/services/test_per_user_library_root.py
new file mode 100644
index 0000000000..02455f48f4
--- /dev/null
+++ b/tests/research_library/services/test_per_user_library_root.py
@@ -0,0 +1,609 @@
+"""Tests for per-user library-root isolation and legacy read fallback (#5521).
+
+Downloaded PDFs are written under a per-user subdirectory
+(``<base>/<username>``) so two users' per-user autoincrement ``resource_id``
+filenames can no longer collide in one shared directory. Reads fall back to
+the legacy shared root so PDFs downloaded before per-user isolation still
+load (no blind bulk file move).
+"""
+
+from unittest.mock import MagicMock
+
+import pytest
+
+from local_deep_research.research_library.services.pdf_storage_manager import (
+ PDFStorageManager,
+)
+from local_deep_research.research_library.utils import apply_user_subdir
+
+# Operator env gate for the legacy shared-root READ fallback (default OFF).
+# Reads only follow the fallback into the shared root when this is enabled;
+# tests that exercise the fallback opt in explicitly.
+LEGACY_READ_FALLBACK_ENV = "LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK"
+
+
+class TestApplyUserSubdir:
+ def test_per_user_subdir_when_not_shared(self, tmp_path):
+ assert (
+ apply_user_subdir(tmp_path, "alice", False)
+ == (tmp_path / "alice").resolve()
+ )
+
+ def test_shared_mode_returns_base(self, tmp_path, monkeypatch):
+ # Shared mode only takes effect when the operator env gate is on.
+ monkeypatch.setattr(
+ "local_deep_research.research_library.utils."
+ "_shared_library_allowed",
+ lambda: True,
+ )
+ assert apply_user_subdir(tmp_path, "alice", True) == tmp_path.resolve()
+
+ def test_shared_mode_gated_off_enforces_per_user(self, tmp_path):
+ # SECURITY: with the operator gate OFF (default), a user's own
+ # shared_library=true must NOT drop per-user isolation — otherwise a
+ # multi-tenant user could set shared_library=true and point
+ # storage_path at another user's directory to read their PDFs.
+ assert (
+ apply_user_subdir(tmp_path, "alice", True)
+ == (tmp_path / "alice").resolve()
+ )
+
+ def test_no_username_returns_base(self, tmp_path):
+ assert apply_user_subdir(tmp_path, None, False) == tmp_path.resolve()
+
+ def test_two_users_get_distinct_dirs(self, tmp_path):
+ alice = apply_user_subdir(tmp_path, "alice", False)
+ bob = apply_user_subdir(tmp_path, "bob", False)
+ assert alice != bob
+
+ def test_valid_registration_usernames_are_accepted(self, tmp_path):
+ # Values a valid registration can produce ([A-Za-z0-9_-]) must join.
+ for name in ("alice", "Bob-2", "user_name", "abc123"):
+ assert (
+ apply_user_subdir(tmp_path, name, False)
+ == (tmp_path / name).resolve()
+ )
+
+ def test_non_ascii_username_round_trips(self, tmp_path):
+ # Registration's ``username.replace("_","").replace("-","").isalnum()``
+ # check is Unicode-aware, so accented/Cyrillic/CJK usernames register
+ # successfully. The per-user path guard must accept the exact same
+ # usernames rather than raising for an already-provisioned account.
+ for name in ("álice", "аlice", "Müller", "田中太郎"):
+ result = apply_user_subdir(tmp_path, name, False)
+ assert result == (tmp_path / name).resolve()
+ assert result.name == name
+
+
+class TestUsernameSanitization:
+ """A traversal/separator username must be rejected at the chokepoint, not
+ join into a path that a downstream ``.resolve()`` can escape the base
+ with (defense-in-depth for future non-alnum username provisioning)."""
+
+ @pytest.mark.parametrize(
+ "bad",
+ [
+ # Path traversal / separators.
+ "../evil",
+ "../../etc",
+ "a/b",
+ "a\\b",
+ "..\\..\\x",
+ "..",
+ ".",
+ "foo/..",
+ "/etc/passwd",
+ "\\\\server\\share",
+ "a\x00b",
+ "..%2f..",
+ # Windows drive-relative ':' (base / "D:evil" discards base) and
+ # NTFS alternate-data-stream 'name:stream'.
+ "D:evil",
+ "a:b",
+ "alice:hidden",
+ # Trailing dot / space canonicalization.
+ "alice.",
+ "alice ",
+ " alice",
+ "a b",
+ # Reserved device name style punctuation (not actually a reserved
+ # device name check — just still non-alnum after stripping).
+ "CON.",
+ # All-punctuation usernames: empty after stripping '_'/'-'.
+ "-",
+ "__",
+ ],
+ )
+ def test_traversal_username_is_rejected(self, tmp_path, bad):
+ with pytest.raises(ValueError):
+ apply_user_subdir(tmp_path, bad, False)
+
+ def test_empty_username_short_circuits_to_base(self, tmp_path):
+ # Pre-existing shared-mode / no-username short-circuit is preserved:
+ # an empty username returns the base rather than raising.
+ assert apply_user_subdir(tmp_path, "", False) == tmp_path.resolve()
+
+ def test_traversal_username_cannot_escape_base(self, tmp_path):
+ # Even after the join+resolve that a consumer performs, no accepted
+ # username escapes the base: rejection happens before the join.
+ base = (tmp_path / "library").resolve()
+ base.mkdir()
+ with pytest.raises(ValueError):
+ apply_user_subdir(base, "../../../../tmp", False)
+
+ def test_shared_mode_skips_username_check(self, tmp_path, monkeypatch):
+ # With the operator gate on, shared mode never joins the username, so
+ # it is not validated.
+ monkeypatch.setattr(
+ "local_deep_research.research_library.utils."
+ "_shared_library_allowed",
+ lambda: True,
+ )
+ assert (
+ apply_user_subdir(tmp_path, "../../etc", True) == tmp_path.resolve()
+ )
+
+
+class TestDownloadWritesToPerUserPath:
+ """Simulates the download write path: a PDFStorageManager built with the
+ per-user root writes PDFs under ``<base>/<username>/pdfs/``."""
+
+ def test_write_lands_in_per_user_dir(self, tmp_path, mock_pdf_content):
+ base = tmp_path
+ per_user = apply_user_subdir(base, "alice", False)
+ manager = PDFStorageManager(per_user, "filesystem")
+ doc = MagicMock()
+ doc.id = "doc-1"
+ rel_path, _ = manager.save_pdf(
+ mock_pdf_content, doc, MagicMock(), "5.pdf"
+ )
+
+ assert rel_path == "pdfs/5.pdf"
+ written = base / "alice" / "pdfs" / "5.pdf"
+ assert written.is_file()
+ assert written.read_bytes() == mock_pdf_content
+ # Nothing was written to the shared base directly.
+ assert not (base / "pdfs" / "5.pdf").exists()
+
+ def test_two_users_same_resource_id_do_not_collide(
+ self, tmp_path, mock_pdf_content
+ ):
+ base = tmp_path
+ alice_mgr = PDFStorageManager(
+ apply_user_subdir(base, "alice", False), "filesystem"
+ )
+ bob_mgr = PDFStorageManager(
+ apply_user_subdir(base, "bob", False), "filesystem"
+ )
+ alice_bytes = mock_pdf_content + b"-alice"
+ bob_bytes = mock_pdf_content + b"-bob"
+
+ a_rel, _ = alice_mgr.save_pdf(
+ alice_bytes, MagicMock(id="a"), MagicMock(), "5.pdf"
+ )
+ b_rel, _ = bob_mgr.save_pdf(
+ bob_bytes, MagicMock(id="b"), MagicMock(), "5.pdf"
+ )
+
+ # Same RELATIVE path (per-user autoincrement collides at id 5)...
+ assert a_rel == b_rel == "pdfs/5.pdf"
+ # ...but distinct absolute files with each user's own bytes.
+ a_file = base / "alice" / "pdfs" / "5.pdf"
+ b_file = base / "bob" / "pdfs" / "5.pdf"
+ assert a_file.read_bytes() == alice_bytes
+ assert b_file.read_bytes() == bob_bytes
+
+
+class TestLegacyReadFallback:
+ """A PDF sitting in the legacy shared root still loads when the manager
+ reads from the per-user root with legacy_root fallback — but ONLY when the
+ operator has opted into the legacy read fallback. The gate is OFF by
+ default (cross-tenant read risk), so these tests enable it explicitly."""
+
+ def _pdf_doc(self, rel_path="pdfs/5.pdf"):
+ doc = MagicMock()
+ doc.id = "doc-1"
+ doc.file_type = "pdf"
+ doc.file_path = rel_path
+ return doc
+
+ def _session_without_blob(self):
+ session = MagicMock()
+ session.query.return_value.filter_by.return_value.first.return_value = (
+ None
+ )
+ return session
+
+ def test_legacy_file_loads_via_fallback(
+ self, tmp_path, mock_pdf_content, monkeypatch
+ ):
+ # Operator opted in: pre-isolation files keep loading (no functional
+ # loss for trusted/single-user deployments).
+ monkeypatch.setenv(LEGACY_READ_FALLBACK_ENV, "true")
+ base = tmp_path
+ per_user = apply_user_subdir(base, "alice", False)
+ per_user.mkdir(parents=True, exist_ok=True)
+ # Legacy file lives in the SHARED base (pre-isolation download).
+ legacy_pdf = base / "pdfs" / "5.pdf"
+ legacy_pdf.parent.mkdir(parents=True, exist_ok=True)
+ legacy_pdf.write_bytes(mock_pdf_content)
+
+ manager = PDFStorageManager(per_user, "none", legacy_root=base)
+ loaded = manager.load_pdf(self._pdf_doc(), self._session_without_blob())
+ assert loaded == mock_pdf_content
+ assert manager.has_pdf(self._pdf_doc(), self._session_without_blob())
+
+ def test_legacy_fallback_blocked_when_gate_off(
+ self, tmp_path, mock_pdf_content
+ ):
+ # SECURITY: with the operator gate OFF (default), the legacy shared
+ # root is NOT consulted even when a legacy_root is configured — a read
+ # miss in the per-user root returns None rather than reaching into the
+ # shared root (which, when storage_path is pointed at another user's
+ # directory, would be a cross-tenant read).
+ base = tmp_path
+ per_user = apply_user_subdir(base, "alice", False)
+ per_user.mkdir(parents=True, exist_ok=True)
+ legacy_pdf = base / "pdfs" / "5.pdf"
+ legacy_pdf.parent.mkdir(parents=True, exist_ok=True)
+ legacy_pdf.write_bytes(mock_pdf_content)
+
+ manager = PDFStorageManager(per_user, "none", legacy_root=base)
+ assert (
+ manager.load_pdf(self._pdf_doc(), self._session_without_blob())
+ is None
+ )
+ assert not manager.has_pdf(
+ self._pdf_doc(), self._session_without_blob()
+ )
+ # The shared-root file is untouched — only the read was blocked.
+ assert legacy_pdf.exists()
+
+ def test_per_user_file_preferred_over_legacy(
+ self, tmp_path, mock_pdf_content
+ ):
+ base = tmp_path
+ per_user = apply_user_subdir(base, "alice", False)
+ # Same relative path exists in BOTH roots with different content.
+ (per_user / "pdfs").mkdir(parents=True, exist_ok=True)
+ (per_user / "pdfs" / "5.pdf").write_bytes(mock_pdf_content + b"-user")
+ (base / "pdfs").mkdir(parents=True, exist_ok=True)
+ (base / "pdfs" / "5.pdf").write_bytes(mock_pdf_content + b"-legacy")
+
+ manager = PDFStorageManager(per_user, "none", legacy_root=base)
+ loaded = manager.load_pdf(self._pdf_doc(), self._session_without_blob())
+ assert loaded == mock_pdf_content + b"-user"
+
+ def test_missing_everywhere_returns_none(self, tmp_path):
+ base = tmp_path
+ per_user = apply_user_subdir(base, "alice", False)
+ per_user.mkdir(parents=True, exist_ok=True)
+ manager = PDFStorageManager(per_user, "none", legacy_root=base)
+ assert (
+ manager.load_pdf(self._pdf_doc(), self._session_without_blob())
+ is None
+ )
+
+ def test_no_legacy_root_does_not_fall_back(
+ self, tmp_path, mock_pdf_content
+ ):
+ base = tmp_path
+ per_user = apply_user_subdir(base, "alice", False)
+ per_user.mkdir(parents=True, exist_ok=True)
+ # File only in legacy location; manager has NO legacy_root.
+ (base / "pdfs").mkdir(parents=True, exist_ok=True)
+ (base / "pdfs" / "5.pdf").write_bytes(mock_pdf_content)
+
+ manager = PDFStorageManager(per_user, "none")
+ assert (
+ manager.load_pdf(self._pdf_doc(), self._session_without_blob())
+ is None
+ )
+
+ def test_equal_legacy_root_is_ignored(self, tmp_path):
+ # A legacy_root equal to library_root must be dropped (no-op).
+ manager = PDFStorageManager(tmp_path, "none", legacy_root=tmp_path)
+ assert manager.legacy_root is None
+
+
+class TestDestructiveDeleteDoesNotUseLegacyFallback:
+ """Deleting a user's own document must never unlink a colliding file in
+ the shared legacy root — per-user autoincrement ids collide by
+ construction, so ``pdfs/5.pdf`` there can belong to a different tenant
+ (#5521)."""
+
+ def _pdf_doc(self, rel_path="pdfs/5.pdf"):
+ doc = MagicMock()
+ doc.id = "doc-1"
+ doc.file_type = "pdf"
+ doc.file_path = rel_path
+ doc.storage_mode = "filesystem"
+ return doc
+
+ def test_delete_pdf_keeps_legacy_shared_file(
+ self, tmp_path, mock_pdf_content
+ ):
+ base = tmp_path
+ per_user = apply_user_subdir(base, "alice", False)
+ per_user.mkdir(parents=True, exist_ok=True)
+ # Another tenant's file at the same relative path in the shared root.
+ legacy_pdf = base / "pdfs" / "5.pdf"
+ legacy_pdf.parent.mkdir(parents=True, exist_ok=True)
+ legacy_pdf.write_bytes(mock_pdf_content)
+
+ manager = PDFStorageManager(
+ per_user, "none", legacy_root=base, username="alice"
+ )
+ manager.delete_pdf(self._pdf_doc(), MagicMock())
+
+ # The shared-root file must survive — it is not alice's to delete.
+ assert legacy_pdf.exists()
+
+ def test_delete_pdf_removes_own_per_user_file(
+ self, tmp_path, mock_pdf_content
+ ):
+ base = tmp_path
+ per_user = apply_user_subdir(base, "alice", False)
+ own_pdf = per_user / "pdfs" / "5.pdf"
+ own_pdf.parent.mkdir(parents=True, exist_ok=True)
+ own_pdf.write_bytes(mock_pdf_content)
+
+ manager = PDFStorageManager(
+ per_user, "none", legacy_root=base, username="alice"
+ )
+ manager.delete_pdf(self._pdf_doc(), MagicMock())
+
+ assert not own_pdf.exists()
+
+ def test_delete_pdf_fails_closed_without_username(
+ self, tmp_path, mock_pdf_content
+ ):
+ # Fail closed: an empty username means library_root may be the bare
+ # shared root, so delete_pdf must refuse the filesystem unlink rather
+ # than risk deleting another tenant's colliding file.
+ base = tmp_path
+ shared_pdf = base / "pdfs" / "5.pdf"
+ shared_pdf.parent.mkdir(parents=True, exist_ok=True)
+ shared_pdf.write_bytes(mock_pdf_content)
+
+ # library_root IS the shared base (username was empty upstream).
+ manager = PDFStorageManager(base, "none", username="")
+ result = manager.delete_pdf(self._pdf_doc(), MagicMock())
+
+ assert result is False
+ # The file the manager could have unlinked survives.
+ assert shared_pdf.exists()
+
+
+class TestResolverDeleteGate:
+ """``get_absolute_path_from_settings(..., allow_legacy_fallback=False)``
+ resolves only within the caller's own per-user root — the path the real
+ delete route (document_deletion.py) hands to ``unlink``."""
+
+ def _mock_settings(self, mocker, base):
+ mock_settings = MagicMock()
+
+ def _get_setting(key, default=None):
+ if key == "research_library.storage_path":
+ return str(base)
+ if key == "research_library.shared_library":
+ return False
+ return default
+
+ mock_settings.get_setting.side_effect = _get_setting
+ mocker.patch(
+ "local_deep_research.utilities.db_utils.get_settings_manager",
+ return_value=mock_settings,
+ )
+ return mock_settings
+
+ def test_read_resolves_to_legacy_but_delete_does_not(
+ self, mocker, tmp_path, mock_pdf_content, monkeypatch
+ ):
+ from local_deep_research.research_library.utils import (
+ get_absolute_path_from_settings,
+ )
+
+ # Operator opted into the read fallback so the read half exercises it.
+ monkeypatch.setenv(LEGACY_READ_FALLBACK_ENV, "true")
+ base = tmp_path
+ self._mock_settings(mocker, base)
+ legacy_pdf = base / "pdfs" / "5.pdf"
+ legacy_pdf.parent.mkdir(parents=True, exist_ok=True)
+ legacy_pdf.write_bytes(mock_pdf_content)
+
+ # Read-only callers still find the pre-isolation file (gate on).
+ read_path = get_absolute_path_from_settings("pdfs/5.pdf", "alice")
+ assert read_path == legacy_pdf
+
+ # Destructive callers resolve within alice's own root only, so the
+ # shared file is never the unlink target — regardless of the read gate.
+ del_path = get_absolute_path_from_settings(
+ "pdfs/5.pdf", "alice", allow_legacy_fallback=False
+ )
+ assert del_path != legacy_pdf
+ assert apply_user_subdir(base, "alice", False) in del_path.parents
+
+ def test_destructive_resolve_fails_closed_without_username(
+ self, mocker, tmp_path
+ ):
+ # Fail closed: a destructive resolve with no user context has no
+ # per-user root to scope the unlink to, and apply_user_subdir would
+ # otherwise return the bare shared root. Raise rather than hand a
+ # shared-root path to unlink.
+ from local_deep_research.research_library.utils import (
+ get_absolute_path_from_settings,
+ )
+
+ base = tmp_path
+ self._mock_settings(mocker, base)
+
+ for empty in ("", None):
+ with pytest.raises(ValueError):
+ get_absolute_path_from_settings(
+ "pdfs/5.pdf", empty, allow_legacy_fallback=False
+ )
+
+ # Read-only callers with no username keep the legacy shared-root
+ # behavior (no raise): they resolve against the shared base.
+ (base / "pdfs").mkdir(parents=True, exist_ok=True)
+ (base / "pdfs" / "5.pdf").touch()
+ read_path = get_absolute_path_from_settings("pdfs/5.pdf", None)
+ assert read_path == base / "pdfs" / "5.pdf"
+
+
+class TestGetAbsolutePathFallback:
+ """get_absolute_path_from_settings resolves per-user first, then legacy."""
+
+ def _patch_settings(self, monkeypatch, base):
+ import local_deep_research.utilities.db_utils as db_utils
+
+ sm = MagicMock()
+
+ def _get_setting(key, default=None):
+ if key == "research_library.storage_path":
+ return str(base)
+ if key == "research_library.shared_library":
+ return False
+ return default
+
+ sm.get_setting.side_effect = _get_setting
+ monkeypatch.setattr(
+ db_utils, "get_settings_manager", lambda *a, **k: sm
+ )
+
+ def test_resolves_per_user_then_legacy(
+ self, tmp_path, monkeypatch, mock_pdf_content
+ ):
+ from local_deep_research.research_library.utils import (
+ get_absolute_path_from_settings,
+ )
+
+ # Operator opted into the read fallback: a pre-isolation file in the
+ # shared root still loads (no functional loss for trusted operators).
+ monkeypatch.setenv(LEGACY_READ_FALLBACK_ENV, "true")
+ base = tmp_path
+ self._patch_settings(monkeypatch, base)
+
+ # Legacy file only.
+ (base / "pdfs").mkdir(parents=True, exist_ok=True)
+ (base / "pdfs" / "5.pdf").write_bytes(mock_pdf_content)
+
+ resolved = get_absolute_path_from_settings("pdfs/5.pdf", "alice")
+ assert resolved == (base / "pdfs" / "5.pdf")
+ assert resolved.is_file()
+
+ def test_prefers_per_user_file(
+ self, tmp_path, monkeypatch, mock_pdf_content
+ ):
+ from local_deep_research.research_library.utils import (
+ get_absolute_path_from_settings,
+ )
+
+ base = tmp_path
+ self._patch_settings(monkeypatch, base)
+
+ (base / "alice" / "pdfs").mkdir(parents=True, exist_ok=True)
+ (base / "alice" / "pdfs" / "5.pdf").write_bytes(mock_pdf_content)
+ (base / "pdfs").mkdir(parents=True, exist_ok=True)
+ (base / "pdfs" / "5.pdf").write_bytes(mock_pdf_content + b"-legacy")
+
+ resolved = get_absolute_path_from_settings("pdfs/5.pdf", "alice")
+ assert resolved == (base / "alice" / "pdfs" / "5.pdf")
+
+ def test_no_username_uses_shared_root(
+ self, tmp_path, monkeypatch, mock_pdf_content
+ ):
+ from local_deep_research.research_library.utils import (
+ get_absolute_path_from_settings,
+ )
+
+ base = tmp_path
+ self._patch_settings(monkeypatch, base)
+ (base / "pdfs").mkdir(parents=True, exist_ok=True)
+ (base / "pdfs" / "5.pdf").write_bytes(mock_pdf_content)
+
+ # Legacy behavior preserved: resolves against the shared base.
+ resolved = get_absolute_path_from_settings("pdfs/5.pdf")
+ assert resolved == (base / "pdfs" / "5.pdf")
+
+
+class TestCrossTenantReadFallbackGate:
+ """The legacy READ fallback is an operator gate (default OFF) because the
+ fallback base is derived from the user-editable
+ ``research_library.storage_path``.
+
+ Attack: the attacker edits their own ``storage_path`` to point at the
+ VICTIM's per-user directory (``<base>/<victim>``). Their per-user root
+ becomes ``<base>/<victim>/<attacker>`` (empty), but the legacy-fallback
+ base becomes ``<base>/<victim>``. Because per-user autoincrement resource
+ ids collide by construction, a read of the attacker's own ``pdfs/5.pdf``
+ would — via the legacy fallback — resolve to ``<base>/<victim>/pdfs/5.pdf``,
+ the VICTIM's file. The gate must keep that fallback from firing by default.
+ """
+
+ def _patch_settings(self, mocker, storage_path):
+ sm = MagicMock()
+
+ def _get_setting(key, default=None):
+ if key == "research_library.storage_path":
+ return str(storage_path)
+ if key == "research_library.shared_library":
+ return False
+ return default
+
+ sm.get_setting.side_effect = _get_setting
+ mocker.patch(
+ "local_deep_research.utilities.db_utils.get_settings_manager",
+ return_value=sm,
+ )
+
+ def test_attacker_cannot_read_victim_pdf_when_gate_off(
+ self, mocker, tmp_path, mock_pdf_content
+ ):
+ base = tmp_path
+ # Victim's per-user library and PDF (victim used storage_path=<base>).
+ victim_root = base / "victim"
+ victim_pdf = victim_root / "pdfs" / "5.pdf"
+ victim_pdf.parent.mkdir(parents=True, exist_ok=True)
+ victim_pdf.write_bytes(mock_pdf_content + b"-VICTIM-SECRET")
+
+ # Attacker points their own storage_path at the victim's directory.
+ self._patch_settings(mocker, victim_root)
+
+ # Gate OFF (default): the read must NOT resolve to the victim's file.
+ from local_deep_research.research_library.utils import (
+ get_absolute_path_from_settings,
+ )
+
+ resolved = get_absolute_path_from_settings("pdfs/5.pdf", "attacker")
+
+ assert resolved != victim_pdf
+ # Resolves strictly within the attacker's own per-user root...
+ attacker_root = apply_user_subdir(victim_root, "attacker", False)
+ assert attacker_root in resolved.parents
+ # ...where the attacker has no such file (nothing leaked).
+ assert not resolved.exists()
+
+ def test_attacker_read_leaks_only_when_operator_opts_in(
+ self, mocker, tmp_path, mock_pdf_content, monkeypatch
+ ):
+ # Symmetric control: with the operator gate explicitly ON, the legacy
+ # fallback fires again (this is the opted-in, single-user/trusted
+ # behavior the operator accepts). Confirms the gate — not something
+ # else — is what closes the vector.
+ monkeypatch.setenv(LEGACY_READ_FALLBACK_ENV, "true")
+ base = tmp_path
+ victim_root = base / "victim"
+ victim_pdf = victim_root / "pdfs" / "5.pdf"
+ victim_pdf.parent.mkdir(parents=True, exist_ok=True)
+ victim_pdf.write_bytes(mock_pdf_content)
+
+ self._patch_settings(mocker, victim_root)
+
+ from local_deep_research.research_library.utils import (
+ get_absolute_path_from_settings,
+ )
+
+ resolved = get_absolute_path_from_settings("pdfs/5.pdf", "attacker")
+ assert resolved == victim_pdf
diff --git a/tests/research_library/test_utils.py b/tests/research_library/test_utils.py
index 6227066bf9..54bdd9bbe4 100644
--- a/tests/research_library/test_utils.py
+++ b/tests/research_library/test_utils.py
@@ -301,7 +301,9 @@ class TestLibraryServiceSafeAbsolutePath:
assert result is not None
assert isinstance(result, str)
- assert result == str(tmp_path / "pdfs/test.pdf")
+ # Per-user isolation (#5521) puts library files under the
+ # user's own subdirectory, not the bare shared root.
+ assert result == str(tmp_path / "testuser" / "pdfs/test.pdf")
def test_path_traversal_returns_none(self, tmp_path):
"""Path traversal attempts should return None."""
@@ -485,9 +487,12 @@ class TestPathTraversalIntegration:
PDFStorageManager,
)
+ # A user context is required for destructive filesystem ops (delete
+ # fails closed without one); the traversal path must still be blocked.
manager = PDFStorageManager(
library_root=tmp_path,
storage_mode="filesystem",
+ username="alice",
)
# Create a mock document with malicious file_path
diff --git a/tests/research_library/utils/test_utils.py b/tests/research_library/utils/test_utils.py
index 9e251d0975..7619c67d7f 100644
--- a/tests/research_library/utils/test_utils.py
+++ b/tests/research_library/utils/test_utils.py
@@ -68,7 +68,16 @@ class TestGetLibraryStoragePath:
@patch("local_deep_research.utilities.db_utils.get_settings_manager")
@patch("local_deep_research.research_library.utils.get_library_directory")
- def test_shared_library_mode(self, mock_lib_dir, mock_get_sm, tmp_path):
+ def test_shared_library_mode(
+ self, mock_lib_dir, mock_get_sm, tmp_path, monkeypatch
+ ):
+ # Shared mode drops per-user isolation; it only takes effect when the
+ # operator has enabled the env gate.
+ monkeypatch.setattr(
+ "local_deep_research.research_library.utils."
+ "_shared_library_allowed",
+ lambda: True,
+ )
mock_lib_dir.return_value = tmp_path / "library"
mock_sm = MagicMock()
mock_sm.get_setting.side_effect = lambda key, default: {
diff --git a/tests/web/routes/test_settings_routes.py b/tests/web/routes/test_settings_routes.py
index ace8011f5c..696f314c66 100644
--- a/tests/web/routes/test_settings_routes.py
+++ b/tests/web/routes/test_settings_routes.py
@@ -269,6 +269,77 @@ class TestEgressScopeMetadataShaping:
assert shaped["search.iterations"]["editable"] is False
+class TestPdfStorageModeMetadataShaping:
+ """The unencrypted 'filesystem' PDF storage option is withheld from the
+ served ``research_library.pdf_storage_mode`` metadata unless the operator
+ gate ``LDR_RESEARCH_LIBRARY_ALLOW_FILESYSTEM_PDF_STORAGE`` is set."""
+
+ ENV_VAR = "LDR_RESEARCH_LIBRARY_ALLOW_FILESYSTEM_PDF_STORAGE"
+
+ @staticmethod
+ def _settings():
+ return {
+ "research_library.pdf_storage_mode": {
+ "value": "database",
+ "options": [
+ {"value": "database", "label": "Database (Encrypted)"},
+ {"value": "filesystem", "label": "Filesystem"},
+ {"value": "none", "label": "None (Text Only)"},
+ ],
+ }
+ }
+
+ def test_disabled_gate_hides_filesystem_option(self, monkeypatch):
+ from local_deep_research.web.routes.settings_routes import (
+ _shape_pdf_storage_mode_metadata,
+ )
+
+ monkeypatch.delenv(self.ENV_VAR, raising=False)
+ shaped = _shape_pdf_storage_mode_metadata(self._settings())
+ values = [
+ option["value"]
+ for option in shaped["research_library.pdf_storage_mode"]["options"]
+ ]
+ assert values == ["database", "none"]
+
+ def test_enabled_gate_keeps_filesystem_option(self, monkeypatch):
+ from local_deep_research.web.routes.settings_routes import (
+ _shape_pdf_storage_mode_metadata,
+ )
+
+ monkeypatch.setenv(self.ENV_VAR, "true")
+ settings = self._settings()
+ assert _shape_pdf_storage_mode_metadata(settings) == settings
+
+ def test_effective_metadata_filters_filesystem_option(self, monkeypatch):
+ from local_deep_research.web.routes.settings_routes import (
+ _shape_effective_setting_metadata,
+ )
+
+ monkeypatch.delenv(self.ENV_VAR, raising=False)
+ shaped = _shape_effective_setting_metadata(self._settings())
+ values = [
+ option["value"]
+ for option in shaped["research_library.pdf_storage_mode"]["options"]
+ ]
+ assert "filesystem" not in values
+
+ def test_single_effective_metadata_filters_filesystem_option(
+ self, monkeypatch
+ ):
+ """The single-setting GET (``/settings/api/<key>``) must hide the
+ gated ``filesystem`` option too, matching the bulk path."""
+ from local_deep_research.web.routes.settings_routes import (
+ _shape_single_effective_metadata,
+ )
+
+ monkeypatch.delenv(self.ENV_VAR, raising=False)
+ key = "research_library.pdf_storage_mode"
+ shaped = _shape_single_effective_metadata(key, self._settings()[key])
+ values = [option["value"] for option in shaped["options"]]
+ assert values == ["database", "none"]
+
+
class TestModelDiscoveryPolicy:
MODULE = "local_deep_research.web.routes.settings_routes"