Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

fix(library)!: remove ambiguous legacy PDF read fallback (#7268)

LearningCircuit0540ce16bcaf
diff --git a/changelog.d/+remove-legacy-pdf-read-fallback.breaking.md b/changelog.d/+remove-legacy-pdf-read-fallback.breaking.md
new file mode 100644
index 0000000000..b9c625a0b3
--- /dev/null
+++ b/changelog.d/+remove-legacy-pdf-read-fallback.breaking.md
@@ -0,0 +1 @@
+Library PDF reads no longer fall back to the old shared directory when a file is missing from the user's own library. The `LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK` option has been removed and is ignored if still set. Existing per-user files and encrypted database PDFs remain readable; legacy files are left untouched. Recover old PDFs through the Library upload flow only after verifying their owner, as described in the upgrade guide. Internal PDF storage helpers no longer accept `legacy_root` or `allow_legacy_fallback`, and resolving a nonempty library file path now requires a username.
diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md
index 0f62b0a819..3643a7c6c3 100644
--- a/docs/CONFIGURATION.md
+++ b/docs/CONFIGURATION.md
@@ -44,7 +44,6 @@ They are not available in the Web UI because they are needed to start the applic
| `LDR_NOTIFICATIONS_ALLOW_PRIVATE_IPS` | Boolean | `False` | No | | Allow notification webhooks to target private/local IP addresses. Environment-only to prevent SSRF bypass via the user-writable settings API. Only enable this if your notification endpoints are on a trusted local network. | Security | |
| `LDR_POLICY_ALLOW_UNPROTECTED_EGRESS` | Boolean | `False` | No | | Allow users to select the UNPROTECTED egress scope. Disabled by default; this is an environment-only operator gate and cannot be changed through the user-writable settings API. Hard SSRF and cloud-metadata protections remain active when enabled. | Security | |
| `LDR_RESEARCH_LIBRARY_ALLOW_FILESYSTEM_PDF_STORAGE` | Boolean | `False` | No | | Allow users to select the UNENCRYPTED 'filesystem' PDF storage mode for library-downloaded PDFs. Disabled by default; this is an environment-only operator gate and cannot be changed through the user-writable settings API. Filesystem mode writes fetched third-party PDFs as PLAINTEXT to a shared library directory (cleartext storage of sensitive information, CWE-312). When off, the 'filesystem' option is withheld from the settings UI and any stored or environment value of 'filesystem' is coerced to the encrypted 'database' mode at write time. Previously-written plaintext files remain readable. Enable only when the library directory lives on an operator-controlled encrypted volume. | Security | |
-| `LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK` | Boolean | `False` | No | | SECURITY RISK — leave DISABLED (default) on any multi-user or untrusted deployment. When ENABLED, library PDF reads that miss in the caller's own per-user directory fall back to a shared root derived from the operator-only 'research_library.storage_path' setting. Legacy saved paths remain in place, and per-user resource ids collide by design (e.g. 'pdfs/5.pdf'), so this fallback can still expose another user's PDF on a multi-user instance. The safe state is OFF: with the gate off, reads resolve strictly within the requesting user's own per-user root and the fallback never fires. This is an environment-only operator gate and cannot be enabled through the user-writable settings API. Enable ONLY on a single-user or fully-trusted deployment that needs PDFs downloaded before per-user isolation (issue #5521) to keep loading from the legacy shared location. | Security | |
| `LDR_RESEARCH_LIBRARY_ALLOW_SHARED_LIBRARY` | Boolean | `False` | No | | Allow shared-library mode, which drops the per-user library directory boundary so all users' downloaded PDFs live in one shared directory. Disabled by default; this is an environment-only operator gate and cannot be enabled through the user-writable settings API. The shared_library flag is otherwise user-editable; leaving shared mode user-toggleable would let an account drop its per-user directory boundary. The storage_path is now operator-only, but legacy saved paths remain in place. When off, the per-user subdirectory is always enforced regardless of the user's shared_library setting. Enable only on single-tenant or mutually-trusted deployments. | Security | |
| `LDR_SEARCH_ALLOW_PRIVATE_ENGINE_URLS` | Boolean | `False` | No | | Allow user-editable PUBLIC search-engine URLs (e.g. the SearXNG instance_url) to point at private / loopback / link-local addresses. Environment-only operator gate so it cannot be flipped through the user-writable settings API. Disabled by default: a public-nature engine (its data source is the public internet) pointed at an internal host lets any authenticated user turn a research run into an internal port scan / service probe, breaking the PUBLIC_ONLY egress promise. Enable only when you self-host SearXNG on localhost/LAN and accept that engine fetches may reach your private network. To approve only specific origins instead of all private addresses, use the finer-grained LDR_SEARCH_PRIVATE_ENGINE_URL_ALLOWLIST. Docker deployments instead pin the URL directly via LDR_SEARCH_ENGINE_WEB_SEARXNG_DEFAULT_PARAMS_INSTANCE_URL, which is trusted as operator-provisioned. Cloud-metadata endpoints (ALWAYS_BLOCKED_METADATA_IPS) stay blocked regardless of this flag. | Security | |
| `LDR_SEARCH_ALLOW_PRIVATE_RESULT_FETCH` | Boolean | `False` | No | | Allow full-content fetching (search.snippets_only off) to follow the RESULT URLs a PUBLIC search engine returns into private / loopback addresses, e.g. an internal wiki that a self-hosted SearXNG instance indexes. Distinct from, and required in addition to, the instance-URL approval (LDR_SEARCH_ALLOW_PRIVATE_ENGINE_URLS, LDR_SEARCH_PRIVATE_ENGINE_URL_ALLOWLIST or an env-locked instance URL): that approval only covers reaching the engine itself, and because a public engine proxies the public web, extending it to the URLs the engine returns would let a poisoned index turn result fetching into a scan of the LDR host's private network. Environment-only operator gate so it cannot be flipped through the user-writable settings API. Disabled by default and not set by the bundled docker-compose.yml. Link-local addresses (169.254.0.0/16, fe80::/10) and the cloud-metadata endpoints in ALWAYS_BLOCKED_METADATA_IPS stay blocked regardless of this flag. | Security | |
diff --git a/docs/deployment/upgrading.md b/docs/deployment/upgrading.md
index 1cde2b2a7b..a4195f2d5b 100644
--- a/docs/deployment/upgrading.md
+++ b/docs/deployment/upgrading.md
@@ -78,6 +78,28 @@ Review the release notes for custom API-client changes, including the removed
legacy news routes and CSRF requirements. See the
[configuration reference](../CONFIGURATION.md) for current settings.
+### PDFs in the old shared library directory
+
+Library reads no longer fall back from a missing per-user PDF to the old
+shared library directory. `LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK`
+has been removed; leaving it set does not restore that behavior. Existing
+per-user files and PDFs stored in the user's encrypted database remain
+readable. The upgrade does not move or delete legacy files.
+
+If an old PDF becomes unavailable, recover it explicitly:
+
+1. Retain the offline backup of the old library and user databases.
+2. Verify the file's owner from trusted source records or a known backup.
+ A name such as `pdfs/5.pdf` or a matching document/resource ID is not proof:
+ different users can have the same IDs. Leave files with uncertain ownership
+ untouched; do not distribute them to users or copy them into every library.
+3. Provide a verified copy only to its owner. The owner can sign in and use
+ the [Library upload flow](../library-and-rag.md#uploading-documents), selecting
+ database PDF storage to retain the bytes in their encrypted database.
+4. Verify that the imported PDF opens in that account before removing any
+ backup. Uploading does not automatically repair old research links or
+ document IDs; retain those records and the original backup separately.
+
### Automatic pre-migration backups
When the new release opens a user database that needs migration, normally at
diff --git a/src/local_deep_research/research_library/deletion/services/document_deletion.py b/src/local_deep_research/research_library/deletion/services/document_deletion.py
index dd94d46754..5760fee59a 100644
--- a/src/local_deep_research/research_library/deletion/services/document_deletion.py
+++ b/src/local_deep_research/research_library/deletion/services/document_deletion.py
@@ -239,7 +239,6 @@ class DocumentDeletionService:
file_path = get_absolute_path_from_settings(
document.file_path,
self.username,
- allow_legacy_fallback=False,
)
# get_absolute_path_from_settings("") returns the
# library root (its `if not relative_path` branch),
@@ -466,7 +465,6 @@ class DocumentDeletionService:
file_path = get_absolute_path_from_settings(
document.file_path,
self.username,
- allow_legacy_fallback=False,
)
# See the fail-closed comment on the equivalent
# guard in _delete_document_locked() above:
@@ -724,7 +722,6 @@ class DocumentDeletionService:
file_path = get_absolute_path_from_settings(
document.file_path,
self.username,
- allow_legacy_fallback=False,
)
# See the fail-closed comment on the equivalent
# guard in _delete_document_locked() above:
diff --git a/src/local_deep_research/research_library/services/download_service.py b/src/local_deep_research/research_library/services/download_service.py
index 924626e774..184d456349 100644
--- a/src/local_deep_research/research_library/services/download_service.py
+++ b/src/local_deep_research/research_library/services/download_service.py
@@ -215,9 +215,6 @@ class DownloadService:
# get_library_storage_path()'s layout but derives the base from this
# user's own settings manager (self.settings) so it stays correct in
# background/scheduler threads that have no request context.
- # `legacy_library_root` is the old shared base: it is never written to,
- # only used as a READ fallback so PDFs downloaded before per-user
- # isolation still load (no blind bulk file move — see PR notes).
base_library_root = (
Path(os.path.expandvars(storage_path_setting))
.expanduser()
@@ -226,13 +223,11 @@ class DownloadService:
shared_library = self.settings.get_setting(
"research_library.shared_library", False
)
- self.legacy_library_root = str(base_library_root)
self.library_root = str(
apply_user_subdir(base_library_root, self.username, shared_library)
)
logger.debug(
- f"[DOWNLOAD_SERVICE_INIT] Library root resolved to: {self.library_root} "
- f"(legacy shared root: {self.legacy_library_root})"
+ f"[DOWNLOAD_SERVICE_INIT] Library root resolved to: {self.library_root}"
)
# Create directory structure
@@ -1155,13 +1150,11 @@ class DownloadService:
tracker.is_downloaded = True
tracker.downloaded_at = downloaded_at_val
- # Initialize PDF storage manager. Writes go to the per-user root;
- # legacy_root gives read fallback for pre-isolation files (#5521).
+ # Read and write PDFs only within the per-user root.
pdf_storage_manager = PDFStorageManager(
library_root=self.library_root,
storage_mode=pdf_storage_mode,
max_pdf_size_mb=max_pdf_size_mb,
- legacy_root=self.legacy_library_root,
)
# Update attempt with success info
@@ -1896,7 +1889,6 @@ class DownloadService:
pdf_manager = PDFStorageManager(
library_root=self.library_root,
storage_mode=pdf_storage_mode,
- legacy_root=self.legacy_library_root,
)
pdf_content = pdf_manager.load_pdf(doc, session)
@@ -2163,7 +2155,6 @@ class DownloadService:
pdf_manager = PDFStorageManager(
library_root=Path(self.library_root),
storage_mode=pdf_storage_mode,
- legacy_root=Path(self.legacy_library_root),
)
pdf_content = pdf_manager.load_pdf(pdf_document, session)
if pdf_content is None:
@@ -2222,8 +2213,7 @@ class DownloadService:
return None
# Validate path to prevent path traversal attacks. Resolve against the
- # per-user root with legacy-shared fallback (issue #5521) so a PDF
- # downloaded before per-user isolation is still found. Pass
+ # per-user root only; shared legacy filenames cannot prove ownership. Pass
# self.settings so storage_path/shared_library come from this user's
# captured manager — the ambient one is db-less in background/scheduler
# threads (this runs under the document scheduler) and would resolve
@@ -2248,6 +2238,7 @@ class DownloadService:
logger.info(f"Found existing PDF, extracting text from: {pdf_path}")
try:
+ # nosemgrep: semgrep.rules.path-traversal-risk, reason: get_absolute_path_from_settings requires user context and confines this read to the selected library root; missing PDFs never fall back to another root
with open(pdf_path, "rb") as f:
pdf_content = f.read()
text = self._extract_text_from_pdf(pdf_content)
diff --git a/src/local_deep_research/research_library/services/library_service.py b/src/local_deep_research/research_library/services/library_service.py
index 00f5ba04b3..09a1dff648 100644
--- a/src/local_deep_research/research_library/services/library_service.py
+++ b/src/local_deep_research/research_library/services/library_service.py
@@ -1090,7 +1090,6 @@ class LibraryService:
file_path = get_absolute_path_from_settings(
tracker.file_path,
self.username,
- allow_legacy_fallback=False,
)
if file_path and file_path.is_file():
file_path.unlink()
@@ -1154,8 +1153,7 @@ class LibraryService:
tracker = find_tracker_by_url(session, doc.original_url)
if tracker and tracker.file_path:
- # Resolve within the per-user root (traversal + symlink safe),
- # with legacy-shared fallback for pre-isolation files (#5521).
+ # Resolve only within the per-user root (traversal + symlink safe).
validated_path = get_absolute_path_from_settings(
tracker.file_path, self.username
)
@@ -1311,7 +1309,7 @@ class LibraryService:
# Text-only documents (storage none) have no file
# either; leave them alone (see the no-path branch
# below for the tracker.file_path=None shape).
- # Check if file exists (per-user, legacy-shared fallback)
+ # Check if the file exists in the per-user root.
if settings_manager is None:
settings_manager = get_settings_manager(
username=self.username
diff --git a/src/local_deep_research/research_library/services/pdf_storage_manager.py b/src/local_deep_research/research_library/services/pdf_storage_manager.py
index 71913902ca..5e68cc640c 100644
--- a/src/local_deep_research/research_library/services/pdf_storage_manager.py
+++ b/src/local_deep_research/research_library/services/pdf_storage_manager.py
@@ -95,7 +95,7 @@ class PDFStorageManager:
library_root: Path,
storage_mode: str,
max_pdf_size_mb: int = DEFAULT_MAX_PDF_SIZE_MB,
- legacy_root: Optional[Path] = None,
+ *,
username: Optional[str] = None,
):
"""
@@ -110,10 +110,6 @@ class PDFStorageManager:
exceed `FileUploadValidator.MAX_FILE_SIZE` (the upload
validator's per-file cap, default 3 GB) — uploads above
that cap are rejected before they reach this layer.
- legacy_root: Optional legacy shared root to fall back to when a
- file is not found under ``library_root``. Lets PDFs
- downloaded before per-user isolation still load; never
- written to. Ignored when equal to ``library_root``.
username: Owning user for destructive operations. When falsy,
``library_root`` cannot be guaranteed to be a per-user root
(``apply_user_subdir`` returns the bare shared root for an
@@ -124,18 +120,6 @@ class PDFStorageManager:
self.storage_mode = storage_mode
self.username = username
self.max_pdf_size_bytes = max_pdf_size_mb * 1024 * 1024
- resolved_legacy = (
- Path(legacy_root).resolve() if legacy_root is not None else None
- )
- # Only keep a distinct legacy root — a legacy root equal to the
- # per-user root is a no-op and would double-check the same directory.
- self.legacy_root = (
- resolved_legacy
- if resolved_legacy is not None
- and resolved_legacy != self.library_root
- else None
- )
-
if storage_mode not in ("none", "filesystem", "database"):
logger.warning(
f"Unknown storage mode '{storage_mode}', defaulting to 'none'"
@@ -163,63 +147,16 @@ class PDFStorageManager:
logger.warning(f"Path traversal blocked: {relative_path}")
return None
- def _get_safe_file_path(
- self, relative_path: str, *, allow_legacy_fallback: bool = True
- ) -> Optional[Path]:
- """
- Safely resolve a relative path within the library root.
-
- Prevents path traversal attacks by validating the path stays within
- the library root directory. Resolves against the per-user root first
- and, when the file is absent there, falls back to the legacy shared
- root (issue #5521) so pre-isolation downloads still load. When
- neither location has the file, the per-user path is returned so
- callers' ``is_file()``/``None`` handling is unchanged.
-
- ``allow_legacy_fallback`` must be ``False`` for destructive callers
- (unlink): the legacy shared root is not per-user namespaced, so a
- colliding relative path can resolve to another tenant's file and be
- unlinked. Delete only within this manager's own ``library_root``.
-
- Even for read-only callers the legacy shared-root fallback is
- cross-tenant read primitive on a multi-tenant instance (the shared
- root is derived from the operator-only ``research_library.storage_path``
- and per-user resource ids collide by construction), so it only fires
- when the operator opted into it via
- ``research_library.allow_legacy_read_fallback`` — OFF by default.
-
- Args:
- relative_path: Relative path from database
- allow_legacy_fallback: Permit the read-only legacy shared-root
- fallback. Pass ``False`` from delete/unlink paths. Even when
- ``True``, the fallback only fires if the operator enabled
- ``research_library.allow_legacy_read_fallback``.
+ def _get_safe_file_path(self, relative_path: str) -> Optional[Path]:
+ """Resolve only inside this manager's library root.
- Returns:
- Validated absolute Path or None if path is invalid/unsafe
+ A missing per-user file must not fall back to a shared legacy root:
+ colliding relative filenames do not establish ownership. Legacy files
+ require explicit recovery after the operator verifies their owner.
"""
if not relative_path or relative_path in FILE_PATH_SENTINELS:
return None
-
- primary = self._safe_path_in_root(relative_path, self.library_root)
- if primary is not None and primary.is_file():
- return primary
- # Legacy shared-root fallback for files downloaded before per-user
- # isolation. Never written to; read-only fallback. Enforced at the
- # fallback-USE site (view_pdf_page passes legacy_root=base_root
- # explicitly) so the operator gate covers every caller: only follow
- # the legacy root when the caller permits it AND the operator opted in.
- from ..utils import _legacy_read_fallback_allowed
-
- if (
- allow_legacy_fallback
- and self.legacy_root is not None
- and _legacy_read_fallback_allowed()
- ):
- legacy = self._safe_path_in_root(relative_path, self.legacy_root)
- if legacy is not None and legacy.is_file():
- return legacy
- return primary
+ return self._safe_path_in_root(relative_path, self.library_root)
def save_pdf(
self,
@@ -347,15 +284,14 @@ class PDFStorageManager:
return False
@classmethod
- def pdf_exists(cls, library_root, document, session, legacy_root=None):
+ def pdf_exists(cls, library_root, document, session):
"""Check if a PDF exists in any storage backend.
Use this when you need to check PDF availability without a specific
storage mode — e.g. generating document URLs in search results.
- ``legacy_root`` enables the per-user -> legacy-shared read fallback
- (issue #5521) so a pre-isolation filesystem PDF still reports present.
+ Filesystem reads stay inside the supplied per-user library root.
"""
- manager = cls(library_root, "none", legacy_root=legacy_root)
+ manager = cls(library_root, "none")
return manager.has_pdf(document, session)
def _infer_storage_mode(self, document: Document) -> str:
@@ -552,9 +488,7 @@ class PDFStorageManager:
# Use safe path resolution to prevent path traversal attacks.
# Delete within our own root only — never the shared legacy
# root, where a colliding path can be another tenant's file.
- file_path = self._get_safe_file_path(
- document.file_path, allow_legacy_fallback=False
- )
+ file_path = self._get_safe_file_path(document.file_path)
if file_path and file_path.is_file():
file_path.unlink()
logger.info(f"Deleted PDF file: {file_path}")
diff --git a/src/local_deep_research/research_library/utils/__init__.py b/src/local_deep_research/research_library/utils/__init__.py
index 005e85dfea..48f5751c59 100644
--- a/src/local_deep_research/research_library/utils/__init__.py
+++ b/src/local_deep_research/research_library/utils/__init__.py
@@ -530,28 +530,6 @@ def _shared_library_allowed() -> bool:
return bool(get_env_setting("research_library.allow_shared_library", False))
-def _legacy_read_fallback_allowed() -> bool:
- """Whether the operator opted into the legacy shared-root READ fallback.
-
- Environment-only, mirroring ``_shared_library_allowed`` /
- ``filesystem_pdf_storage_allowed``. The legacy fallback resolves a
- per-user read miss against the shared root derived from
- ``research_library.storage_path``. Legacy saved paths remain in place, and
- per-user autoincrement resource ids collide by construction, so the fallback
- can still be a cross-tenant read primitive on a multi-tenant instance. It
- is therefore OFF by default
- and can only be enabled by the operator via
- ``LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK=true`` — never through
- the user-writable settings API. When off, reads resolve strictly within
- the caller's own per-user root.
- """
- from ...settings.env_registry import get_env_setting
-
- return bool(
- get_env_setting("research_library.allow_legacy_read_fallback", False)
- )
-
-
def get_library_storage_path(username: str) -> Path:
"""
Get the storage path for a user's library.
@@ -704,90 +682,30 @@ def get_absolute_path_from_settings(
relative_path: str,
username: Optional[str] = None,
*,
- allow_legacy_fallback: bool = True,
settings_manager=None,
) -> Optional[Path]:
- """
- Get absolute path using settings manager for library root.
+ """Resolve a path inside the user's configured library root.
- Uses PathValidator to prevent path traversal attacks.
-
- ``settings_manager`` lets a caller supply an explicit, already-scoped
- settings manager instead of the ambient ``get_settings_manager()``. This
- matters in background/scheduler threads with no Flask request context:
- there the ambient manager resolves to a *db-less* manager that reads only
- env vars and the defaults file, so a user's previously saved
- ``research_library.storage_path`` / ``research_library.shared_library``
- (stored only in their per-user encrypted DB) is invisible and the path
- silently resolves against the default library location — the wrong root.
- ``DownloadService`` passes its captured ``self.settings`` here so these
- reads stay consistent with the per-user root it computed in ``__init__``
- (issue #5521). When ``None`` the ambient manager is used, preserving the
- request-context behavior for existing callers.
-
- When ``username`` is provided the path is resolved against that user's
- per-user library directory (issue #5521). If the file is not found there,
- it falls back to the legacy shared root so PDFs downloaded before per-user
- isolation still load correctly (no data loss). When ``username``
- is ``None`` the legacy shared-root behavior is preserved unchanged for
- callers that have no user context.
-
- ``allow_legacy_fallback`` must be ``False`` for *destructive* callers
- (delete / unlink). The legacy shared root is not per-user namespaced, and
- per-user autoincrement resource ids collide by construction (each user has
- an independently-numbered database), so filenames like ``pdfs/3.pdf`` can
- map to *another* tenant's file in the shared root. Resolving-then-unlinking
- that path would delete a different user's PDF. Because migration 0029 does
- not move pre-isolation files into per-user subdirectories, the per-user
- location is always empty for legacy documents and the fallback would fire
- deterministically. Destructive callers therefore resolve only within the
- caller's own per-user root; a missing file there means "nothing of mine to
- delete" rather than reaching into shared storage. As a fail-closed guard,
- a destructive call (``allow_legacy_fallback=False``) with a falsy
- ``username`` raises ``ValueError`` rather than resolving to the bare
- shared root: without a user context there is no per-user directory to
- scope the unlink to, and silently resolving into the shared root could
- unlink another tenant's file.
-
- Even for read-only callers the legacy shared-root fallback is a
- cross-tenant read primitive on a multi-tenant instance: the shared root is
- derived from ``research_library.storage_path``. Legacy account-chosen
- paths remain in place, so a pre-existing path that points at another
- user's directory can still expose PDFs through colliding ids. The fallback
- is therefore gated behind the operator-only
- ``research_library.allow_legacy_read_fallback`` env setting and is OFF by
- default; when off, reads resolve strictly within the caller's own per-user
- root.
+ ``settings_manager`` supplies an already-scoped manager for background
+ threads; otherwise use the ambient manager. The returned path is checked
+ for traversal and symlink escapes, even when the file is missing.
- Args:
- relative_path: The relative path from library root
- username: Optional username for per-user resolution
- allow_legacy_fallback: Permit the legacy shared-root fallback for
- read-only callers. Pass ``False`` from delete/unlink paths. Even
- when ``True``, the fallback only fires if the operator enabled
- ``research_library.allow_legacy_read_fallback``.
- settings_manager: Optional explicit settings manager to read
- ``storage_path`` / ``shared_library`` from. Defaults to the
- ambient ``get_settings_manager()``; pass a captured, user-scoped
- manager from background/scheduler threads (see above).
+ Missing files never fall back to the old shared directory. Legacy file
+ names alone cannot establish ownership; recover those files explicitly
+ after verifying their owner. The retired legacy-read environment flag has
+ no effect. Operator-enabled shared-library mode remains an explicit root
+ selection, not a fallback from a missing per-user file.
- Returns:
- The absolute path, or None if the path is unsafe
+ A nonempty path requires a username for both reads and deletes. An empty
+ path may still return the configured base root without a user context for
+ trusted cleanup callers that need a containment boundary.
- Raises:
- ValueError: If ``allow_legacy_fallback`` is ``False`` (destructive
- caller) and ``username`` is falsy — fail closed rather than
- resolve within the shared root.
+ Returns the absolute path, or None if it is unsafe. Raises ValueError for
+ a nonempty path without a username.
"""
- # Fail closed: a destructive caller with no user context has no per-user
- # directory to scope its unlink to. apply_user_subdir(base, "", ...) would
- # otherwise return the bare shared root, so resolving+unlinking there could
- # destroy another tenant's colliding file. Read-only callers
- # (allow_legacy_fallback=True) keep the legacy no-username behavior.
- if not allow_legacy_fallback and not username:
+ if relative_path and not username:
raise ValueError(
- "Refusing to resolve a destructive library path without a user "
- "context (would resolve within the shared root)."
+ "Refusing to resolve a library file without a user context"
)
if settings_manager is None:
@@ -817,25 +735,7 @@ def get_absolute_path_from_settings(
# An empty path asks for the (per-user) library root itself.
return per_user_root
- primary = _resolve_within_root(relative_path, per_user_root)
- # Legacy fallback: when the per-user location has no file, look in the
- # legacy shared root where pre-isolation downloads still live. Gated three
- # ways: (1) read-only callers only — destructive callers pass
- # allow_legacy_fallback=False so they never unlink a colliding file that
- # belongs to another tenant; (2) the operator must have opted into the
- # shared-root read fallback, since it is a cross-tenant read primitive
- # when a legacy storage_path points at another user's directory; (3) the
- # per-user root must actually differ from the shared root.
- if (
- allow_legacy_fallback
- and _legacy_read_fallback_allowed()
- and per_user_root != base_path
- and (primary is None or not primary.exists())
- ):
- legacy = _resolve_within_root(relative_path, base_path)
- if legacy is not None and legacy.exists():
- return legacy
- return primary
+ return _resolve_within_root(relative_path, per_user_root)
def handle_api_error(operation: str, error: Exception, status_code: int = 500):
diff --git a/src/local_deep_research/settings/env_definitions/security.py b/src/local_deep_research/settings/env_definitions/security.py
index 4129b28a67..6ec3a3d38a 100644
--- a/src/local_deep_research/settings/env_definitions/security.py
+++ b/src/local_deep_research/settings/env_definitions/security.py
@@ -195,29 +195,4 @@ SECURITY_SETTINGS = [
),
default=False,
),
- BooleanSetting(
- # NOTE: risk-bearing opt-in. Enabling this re-opens a cross-tenant
- # library-PDF READ. The description below is the operator-facing
- # warning; keep the two in sync. Env var (auto-derived from the key):
- # LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK.
- key="research_library.allow_legacy_read_fallback",
- description=(
- "SECURITY RISK — leave DISABLED (default) on any multi-user or "
- "untrusted deployment. When ENABLED, library PDF reads that miss "
- "in the caller's own per-user directory fall back to a shared "
- "root derived from the operator-only "
- "'research_library.storage_path' setting. Legacy saved paths "
- "remain in place, and per-user resource ids collide by design "
- "(e.g. 'pdfs/5.pdf'), so this fallback can still expose another "
- "user's PDF on a multi-user instance. The safe state is OFF: with "
- "the gate off, reads resolve strictly within the requesting "
- "user's own per-user root and the fallback never fires. This is "
- "an environment-only operator gate and cannot be enabled through "
- "the user-writable settings API. Enable ONLY on a single-user or "
- "fully-trusted deployment that needs PDFs downloaded before "
- "per-user isolation (issue #5521) to keep loading from the legacy "
- "shared location."
- ),
- default=False,
- ),
]
diff --git a/src/local_deep_research/web/routers/library.py b/src/local_deep_research/web/routers/library.py
index 100495e976..983c3aec64 100644
--- a/src/local_deep_research/web/routers/library.py
+++ b/src/local_deep_research/web/routers/library.py
@@ -548,13 +548,8 @@ def _serve_pdf(
storage_mode = settings.get_setting(
"research_library.pdf_storage_mode", "none"
)
- # expandvars() matters here and was dropped in the port of main's
- # #5537. `library_root` below survives without it because
- # apply_user_subdir() re-expands whatever it is handed, but
- # `legacy_root` is passed this value RAW -- so with a storage_path
- # containing an env-var token the legacy fallback would search a
- # directory literally named e.g. "$HOME", and pre-isolation PDFs
- # would silently stop being findable.
+ # Expand configured environment variables before choosing the user's
+ # library directory, matching the download/storage path resolution.
base_root = (
Path(
os.path.expandvars(
@@ -570,21 +565,12 @@ def _serve_pdf(
shared_library = settings.get_setting(
"research_library.shared_library", False
)
- # Serve from the per-user root, falling back to the legacy shared root
- # for PDFs downloaded before per-user isolation (issue #5521).
- #
- # Ported from #5537, which landed on the Flask library_routes.py this
- # router replaced. Passing the bare shared root here (as this route
- # did) resolves a document id against a directory every user writes
- # into: two users' document ids collide by construction, so one user's
- # id could serve another's file, and the legacy-fallback gate was
- # bypassed rather than consulted.
+ # Serve only from the requesting user's root. A matching relative
+ # filename in the shared legacy directory does not prove ownership.
library_root = apply_user_subdir(base_root, username, shared_library)
# Use PDFStorageManager to load PDF (handles database and filesystem)
- pdf_manager = PDFStorageManager(
- library_root, storage_mode, legacy_root=base_root
- )
+ pdf_manager = PDFStorageManager(library_root, storage_mode)
pdf_bytes = pdf_manager.load_pdf(document, db_session)
if pdf_bytes:
diff --git a/src/local_deep_research/web_search_engines/engines/search_engine_library.py b/src/local_deep_research/web_search_engines/engines/search_engine_library.py
index dd92e880ea..71acb0a5ba 100644
--- a/src/local_deep_research/web_search_engines/engines/search_engine_library.py
+++ b/src/local_deep_research/web_search_engines/engines/search_engine_library.py
@@ -709,7 +709,7 @@ class LibraryRAGSearchEngine(BaseSearchEngine):
default=False,
settings_snapshot=self.settings_snapshot,
)
- # Per-user root with legacy-shared fallback (issue #5521).
+ # PDF availability uses the same per-user root as serving.
per_user_root = apply_user_subdir(
base_root, self.username, shared_library
)
@@ -717,7 +717,6 @@ class LibraryRAGSearchEngine(BaseSearchEngine):
per_user_root,
document,
session,
- legacy_root=base_root,
):
document_url = f"/library/document/{doc_id}/pdf"
except Exception as e:
diff --git a/tests/infrastructure_tests/test_documented_env_vars.py b/tests/infrastructure_tests/test_documented_env_vars.py
index 6c91678abd..6e7530ca08 100644
--- a/tests/infrastructure_tests/test_documented_env_vars.py
+++ b/tests/infrastructure_tests/test_documented_env_vars.py
@@ -17,6 +17,9 @@ This test scans the operator-facing docs (``docs/deployment/``,
tokens outside fenced code blocks, and requires each to appear somewhere
under ``src/`` — as a literal the code reads (``os.environ``/``os.getenv``
usage, env_registry definitions, or any other textual reference).
+An occurrence immediately followed by "has been removed." or "has been
+removed;" is an upgrade warning, not a claim that the variable still works.
+Only that occurrence is excluded; later usage advice still gets checked.
Deliberately one-directional: the reverse direction (code reads a variable
no doc mentions) is legitimate — internal and test-mode knobs outnumber
@@ -71,6 +74,10 @@ def _documented_env_vars() -> dict[str, list[str]]:
token = match.group(1).strip()
if not _ENV_VAR_RE.match(token):
continue
+ # Preserve explicit migration notices without exempting every
+ # mention of the variable or other variables in the paragraph.
+ if re.match(r"\s+has\s+been\s+removed[.;]", text[match.end() :]):
+ continue
line = text.count("\n", 0, match.start()) + 1
found.setdefault(token, []).append(
f"{doc.relative_to(REPO_ROOT)}:{line}"
@@ -214,5 +221,48 @@ def test_scanned_docs_exist():
)
+ ("text", "expected"),
+ [
+ ("`LDR_RETIRED_OPTION` has been removed.", set()),
+ ("`LDR_RETIRED_OPTION`\nhas been removed; do not set it.", set()),
+ (
+ "`LDR_RETIRED_OPTION` has been removed.\n"
+ "Set `LDR_RETIRED_OPTION` to enable this feature.",
+ {"LDR_RETIRED_OPTION"},
+ ),
+ (
+ "`LDR_RETIRED_OPTION` has been removed; set `LDR_UNKNOWN_OPTION`.",
+ {"LDR_UNKNOWN_OPTION"},
+ ),
+ (
+ "`LDR_RETIRED_OPTION` has not been removed.",
+ {"LDR_RETIRED_OPTION"},
+ ),
+ (
+ "`LDR_RETIRED_OPTION` will be removed in the next release.",
+ {"LDR_RETIRED_OPTION"},
+ ),
+ (
+ "`LDR_RETIRED_OPTION` has been removed from the examples.",
+ {"LDR_RETIRED_OPTION"},
+ ),
+ (
+ "Set `LDR_UNKNOWN_OPTION` to enable this feature.",
+ {"LDR_UNKNOWN_OPTION"},
+ ),
+ ],
+)
+def test_documented_env_extractor_distinguishes_removal_notices(
+ monkeypatch, tmp_path, text, expected
+):
+ doc = tmp_path / "upgrading.md"
+ doc.write_text(text, encoding="utf-8")
+ monkeypatch.setitem(globals(), "REPO_ROOT", tmp_path)
+ monkeypatch.setitem(globals(), "DOC_PATHS", [doc])
+
+ assert set(_documented_env_vars()) == expected
+
+
if __name__ == "__main__": # pragma: no cover
pytest.main([__file__, "-v"])
diff --git a/tests/research_library/services/test_download_service_arxiv_provenance.py b/tests/research_library/services/test_download_service_arxiv_provenance.py
index 293262e2e0..9517b7ff92 100644
--- a/tests/research_library/services/test_download_service_arxiv_provenance.py
+++ b/tests/research_library/services/test_download_service_arxiv_provenance.py
@@ -42,7 +42,6 @@ def service():
instance.username = "test_user"
instance.password = "test_password"
instance.library_root = "/tmp/test_library/test_user"
- instance.legacy_library_root = "/tmp/test_library"
instance.settings = MagicMock()
instance.settings.get_setting.side_effect = lambda key, default=None: {
"research_library.pdf_storage_mode": "none",
@@ -325,7 +324,6 @@ def test_arxiv_text_reuses_database_pdf_and_document_identity(service):
manager.assert_called_once_with(
library_root=Path(service.library_root),
storage_mode="none",
- legacy_root=Path(service.legacy_library_root),
)
storage_manager.load_pdf.assert_called_once_with(document, session)
assert (
diff --git a/tests/research_library/services/test_download_service_logic.py b/tests/research_library/services/test_download_service_logic.py
index c25b07f77d..e956d2417f 100644
--- a/tests/research_library/services/test_download_service_logic.py
+++ b/tests/research_library/services/test_download_service_logic.py
@@ -349,10 +349,9 @@ class TestInitialization:
service.close()
def test_library_root_set(self, tmp_path):
- """Library root is per-user (<base>/<username>); legacy root is base."""
+ """Library root is per-user (<base>/<username>)."""
service = _create_service(tmp_path)
assert service.library_root == str(tmp_path / "testuser")
- assert service.legacy_library_root == str(tmp_path)
service.close()
def test_downloaders_initialized(self, tmp_path):
diff --git a/tests/research_library/services/test_per_user_library_root.py b/tests/research_library/services/test_per_user_library_root.py
index 02455f48f4..adb88f147c 100644
--- a/tests/research_library/services/test_per_user_library_root.py
+++ b/tests/research_library/services/test_per_user_library_root.py
@@ -1,11 +1,4 @@
-"""Tests for per-user library-root isolation and legacy read fallback (#5521).
-
-Downloaded PDFs are written under a per-user subdirectory
-(``<base>/<username>``) so two users' per-user autoincrement ``resource_id``
-filenames can no longer collide in one shared directory. Reads fall back to
-the legacy shared root so PDFs downloaded before per-user isolation still
-load (no blind bulk file move).
-"""
+"""Per-user library isolation and refusal of ambiguous legacy PDF reads."""
from unittest.mock import MagicMock
@@ -16,9 +9,7 @@ from local_deep_research.research_library.services.pdf_storage_manager import (
)
from local_deep_research.research_library.utils import apply_user_subdir
-# Operator env gate for the legacy shared-root READ fallback (default OFF).
-# Reads only follow the fallback into the shared root when this is enabled;
-# tests that exercise the fallback opt in explicitly.
+# Retired flag: old deployment configuration must not restore the fallback.
LEGACY_READ_FALLBACK_ENV = "LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK"
@@ -194,11 +185,8 @@ class TestDownloadWritesToPerUserPath:
assert b_file.read_bytes() == bob_bytes
-class TestLegacyReadFallback:
- """A PDF sitting in the legacy shared root still loads when the manager
- reads from the per-user root with legacy_root fallback — but ONLY when the
- operator has opted into the legacy read fallback. The gate is OFF by
- default (cross-tenant read risk), so these tests enable it explicitly."""
+class TestNoLegacyReadFallback:
+ """Shared-root filenames cannot establish ownership, even with the old flag."""
def _pdf_doc(self, rel_path="pdfs/5.pdf"):
doc = MagicMock()
@@ -214,50 +202,22 @@ class TestLegacyReadFallback:
)
return session
- def test_legacy_file_loads_via_fallback(
- self, tmp_path, mock_pdf_content, monkeypatch
+ @pytest.mark.parametrize("old_flag", ["false", "true"])
+ def test_legacy_file_never_loads(
+ self, tmp_path, mock_pdf_content, monkeypatch, old_flag
):
- # Operator opted in: pre-isolation files keep loading (no functional
- # loss for trusted/single-user deployments).
- monkeypatch.setenv(LEGACY_READ_FALLBACK_ENV, "true")
- base = tmp_path
- per_user = apply_user_subdir(base, "alice", False)
- per_user.mkdir(parents=True, exist_ok=True)
- # Legacy file lives in the SHARED base (pre-isolation download).
- legacy_pdf = base / "pdfs" / "5.pdf"
- legacy_pdf.parent.mkdir(parents=True, exist_ok=True)
+ monkeypatch.setenv(LEGACY_READ_FALLBACK_ENV, old_flag)
+ per_user = apply_user_subdir(tmp_path, "alice", False)
+ legacy_pdf = tmp_path / "pdfs" / "5.pdf"
+ legacy_pdf.parent.mkdir(parents=True)
legacy_pdf.write_bytes(mock_pdf_content)
- manager = PDFStorageManager(per_user, "none", legacy_root=base)
- loaded = manager.load_pdf(self._pdf_doc(), self._session_without_blob())
- assert loaded == mock_pdf_content
- assert manager.has_pdf(self._pdf_doc(), self._session_without_blob())
-
- def test_legacy_fallback_blocked_when_gate_off(
- self, tmp_path, mock_pdf_content
- ):
- # SECURITY: with the operator gate OFF (default), the legacy shared
- # root is NOT consulted even when a legacy_root is configured — a read
- # miss in the per-user root returns None rather than reaching into the
- # shared root (which, when storage_path is pointed at another user's
- # directory, would be a cross-tenant read).
- base = tmp_path
- per_user = apply_user_subdir(base, "alice", False)
- per_user.mkdir(parents=True, exist_ok=True)
- legacy_pdf = base / "pdfs" / "5.pdf"
- legacy_pdf.parent.mkdir(parents=True, exist_ok=True)
- legacy_pdf.write_bytes(mock_pdf_content)
-
- manager = PDFStorageManager(per_user, "none", legacy_root=base)
- assert (
- manager.load_pdf(self._pdf_doc(), self._session_without_blob())
- is None
- )
- assert not manager.has_pdf(
- self._pdf_doc(), self._session_without_blob()
- )
- # The shared-root file is untouched — only the read was blocked.
- assert legacy_pdf.exists()
+ manager = PDFStorageManager(per_user, "none")
+ doc, session = self._pdf_doc(), self._session_without_blob()
+ assert manager.load_pdf(doc, session) is None
+ assert not manager.has_pdf(doc, session)
+ assert not PDFStorageManager.pdf_exists(per_user, doc, session)
+ assert legacy_pdf.read_bytes() == mock_pdf_content
def test_per_user_file_preferred_over_legacy(
self, tmp_path, mock_pdf_content
@@ -270,7 +230,7 @@ class TestLegacyReadFallback:
(base / "pdfs").mkdir(parents=True, exist_ok=True)
(base / "pdfs" / "5.pdf").write_bytes(mock_pdf_content + b"-legacy")
- manager = PDFStorageManager(per_user, "none", legacy_root=base)
+ manager = PDFStorageManager(per_user, "none")
loaded = manager.load_pdf(self._pdf_doc(), self._session_without_blob())
assert loaded == mock_pdf_content + b"-user"
@@ -278,7 +238,7 @@ class TestLegacyReadFallback:
base = tmp_path
per_user = apply_user_subdir(base, "alice", False)
per_user.mkdir(parents=True, exist_ok=True)
- manager = PDFStorageManager(per_user, "none", legacy_root=base)
+ manager = PDFStorageManager(per_user, "none")
assert (
manager.load_pdf(self._pdf_doc(), self._session_without_blob())
is None
@@ -300,11 +260,6 @@ class TestLegacyReadFallback:
is None
)
- def test_equal_legacy_root_is_ignored(self, tmp_path):
- # A legacy_root equal to library_root must be dropped (no-op).
- manager = PDFStorageManager(tmp_path, "none", legacy_root=tmp_path)
- assert manager.legacy_root is None
-
class TestDestructiveDeleteDoesNotUseLegacyFallback:
"""Deleting a user's own document must never unlink a colliding file in
@@ -331,9 +286,7 @@ class TestDestructiveDeleteDoesNotUseLegacyFallback:
legacy_pdf.parent.mkdir(parents=True, exist_ok=True)
legacy_pdf.write_bytes(mock_pdf_content)
- manager = PDFStorageManager(
- per_user, "none", legacy_root=base, username="alice"
- )
+ manager = PDFStorageManager(per_user, "none", username="alice")
manager.delete_pdf(self._pdf_doc(), MagicMock())
# The shared-root file must survive — it is not alice's to delete.
@@ -348,9 +301,7 @@ class TestDestructiveDeleteDoesNotUseLegacyFallback:
own_pdf.parent.mkdir(parents=True, exist_ok=True)
own_pdf.write_bytes(mock_pdf_content)
- manager = PDFStorageManager(
- per_user, "none", legacy_root=base, username="alice"
- )
+ manager = PDFStorageManager(per_user, "none", username="alice")
manager.delete_pdf(self._pdf_doc(), MagicMock())
assert not own_pdf.exists()
@@ -376,9 +327,7 @@ class TestDestructiveDeleteDoesNotUseLegacyFallback:
class TestResolverDeleteGate:
- """``get_absolute_path_from_settings(..., allow_legacy_fallback=False)``
- resolves only within the caller's own per-user root — the path the real
- delete route (document_deletion.py) hands to ``unlink``."""
+ """Reads and deletes use the same per-user-only path resolution."""
def _mock_settings(self, mocker, base):
mock_settings = MagicMock()
@@ -397,14 +346,14 @@ class TestResolverDeleteGate:
)
return mock_settings
- def test_read_resolves_to_legacy_but_delete_does_not(
+ def test_read_and_delete_ignore_legacy_file(
self, mocker, tmp_path, mock_pdf_content, monkeypatch
):
from local_deep_research.research_library.utils import (
get_absolute_path_from_settings,
)
- # Operator opted into the read fallback so the read half exercises it.
+ # A stale deployment flag must not change read or delete resolution.
monkeypatch.setenv(LEGACY_READ_FALLBACK_ENV, "true")
base = tmp_path
self._mock_settings(mocker, base)
@@ -412,21 +361,12 @@ class TestResolverDeleteGate:
legacy_pdf.parent.mkdir(parents=True, exist_ok=True)
legacy_pdf.write_bytes(mock_pdf_content)
- # Read-only callers still find the pre-isolation file (gate on).
- read_path = get_absolute_path_from_settings("pdfs/5.pdf", "alice")
- assert read_path == legacy_pdf
+ path = get_absolute_path_from_settings("pdfs/5.pdf", "alice")
+ assert path == apply_user_subdir(base, "alice", False) / "pdfs/5.pdf"
+ assert not path.exists()
+ assert legacy_pdf.read_bytes() == mock_pdf_content
- # Destructive callers resolve within alice's own root only, so the
- # shared file is never the unlink target — regardless of the read gate.
- del_path = get_absolute_path_from_settings(
- "pdfs/5.pdf", "alice", allow_legacy_fallback=False
- )
- assert del_path != legacy_pdf
- assert apply_user_subdir(base, "alice", False) in del_path.parents
-
- def test_destructive_resolve_fails_closed_without_username(
- self, mocker, tmp_path
- ):
+ def test_file_resolve_fails_closed_without_username(self, mocker, tmp_path):
# Fail closed: a destructive resolve with no user context has no
# per-user root to scope the unlink to, and apply_user_subdir would
# otherwise return the bare shared root. Raise rather than hand a
@@ -440,20 +380,15 @@ class TestResolverDeleteGate:
for empty in ("", None):
with pytest.raises(ValueError):
- get_absolute_path_from_settings(
- "pdfs/5.pdf", empty, allow_legacy_fallback=False
- )
+ get_absolute_path_from_settings("pdfs/5.pdf", empty)
- # Read-only callers with no username keep the legacy shared-root
- # behavior (no raise): they resolve against the shared base.
- (base / "pdfs").mkdir(parents=True, exist_ok=True)
- (base / "pdfs" / "5.pdf").touch()
- read_path = get_absolute_path_from_settings("pdfs/5.pdf", None)
- assert read_path == base / "pdfs" / "5.pdf"
+ # Trusted callers may still request only the configured root, e.g.
+ # as a containment boundary for cleanup; this does not return a file.
+ assert get_absolute_path_from_settings("") == base
-class TestGetAbsolutePathFallback:
- """get_absolute_path_from_settings resolves per-user first, then legacy."""
+class TestGetAbsolutePerUserPath:
+ """Missing per-user files never change the resolution root."""
def _patch_settings(self, monkeypatch, base):
import local_deep_research.utilities.db_utils as db_utils
@@ -472,15 +407,14 @@ class TestGetAbsolutePathFallback:
db_utils, "get_settings_manager", lambda *a, **k: sm
)
- def test_resolves_per_user_then_legacy(
+ def test_missing_file_stays_in_per_user_root(
self, tmp_path, monkeypatch, mock_pdf_content
):
from local_deep_research.research_library.utils import (
get_absolute_path_from_settings,
)
- # Operator opted into the read fallback: a pre-isolation file in the
- # shared root still loads (no functional loss for trusted operators).
+ # The retired flag cannot bring back implicit shared-root reads.
monkeypatch.setenv(LEGACY_READ_FALLBACK_ENV, "true")
base = tmp_path
self._patch_settings(monkeypatch, base)
@@ -490,8 +424,8 @@ class TestGetAbsolutePathFallback:
(base / "pdfs" / "5.pdf").write_bytes(mock_pdf_content)
resolved = get_absolute_path_from_settings("pdfs/5.pdf", "alice")
- assert resolved == (base / "pdfs" / "5.pdf")
- assert resolved.is_file()
+ assert resolved == (base / "alice" / "pdfs" / "5.pdf")
+ assert not resolved.is_file()
def test_prefers_per_user_file(
self, tmp_path, monkeypatch, mock_pdf_content
@@ -511,7 +445,7 @@ class TestGetAbsolutePathFallback:
resolved = get_absolute_path_from_settings("pdfs/5.pdf", "alice")
assert resolved == (base / "alice" / "pdfs" / "5.pdf")
- def test_no_username_uses_shared_root(
+ def test_nonempty_path_requires_username(
self, tmp_path, monkeypatch, mock_pdf_content
):
from local_deep_research.research_library.utils import (
@@ -523,24 +457,12 @@ class TestGetAbsolutePathFallback:
(base / "pdfs").mkdir(parents=True, exist_ok=True)
(base / "pdfs" / "5.pdf").write_bytes(mock_pdf_content)
- # Legacy behavior preserved: resolves against the shared base.
- resolved = get_absolute_path_from_settings("pdfs/5.pdf")
- assert resolved == (base / "pdfs" / "5.pdf")
+ with pytest.raises(ValueError, match="user context"):
+ get_absolute_path_from_settings("pdfs/5.pdf")
-class TestCrossTenantReadFallbackGate:
- """The legacy READ fallback is an operator gate (default OFF) because the
- fallback base is derived from the user-editable
- ``research_library.storage_path``.
-
- Attack: the attacker edits their own ``storage_path`` to point at the
- VICTIM's per-user directory (``<base>/<victim>``). Their per-user root
- becomes ``<base>/<victim>/<attacker>`` (empty), but the legacy-fallback
- base becomes ``<base>/<victim>``. Because per-user autoincrement resource
- ids collide by construction, a read of the attacker's own ``pdfs/5.pdf``
- would — via the legacy fallback — resolve to ``<base>/<victim>/pdfs/5.pdf``,
- the VICTIM's file. The gate must keep that fallback from firing by default.
- """
+class TestCrossTenantReadFallbackRemoved:
+ """A missing own file cannot turn into another user's colliding PDF."""
def _patch_settings(self, mocker, storage_path):
sm = MagicMock()
@@ -585,13 +507,10 @@ class TestCrossTenantReadFallbackGate:
# ...where the attacker has no such file (nothing leaked).
assert not resolved.exists()
- def test_attacker_read_leaks_only_when_operator_opts_in(
+ def test_retired_flag_cannot_enable_cross_user_read(
self, mocker, tmp_path, mock_pdf_content, monkeypatch
):
- # Symmetric control: with the operator gate explicitly ON, the legacy
- # fallback fires again (this is the opted-in, single-user/trusted
- # behavior the operator accepts). Confirms the gate — not something
- # else — is what closes the vector.
+ # Previously documented opt-in behavior is no longer available.
monkeypatch.setenv(LEGACY_READ_FALLBACK_ENV, "true")
base = tmp_path
victim_root = base / "victim"
@@ -606,4 +525,6 @@ class TestCrossTenantReadFallbackGate:
)
resolved = get_absolute_path_from_settings("pdfs/5.pdf", "attacker")
- assert resolved == victim_pdf
+ assert resolved == victim_root / "attacker" / "pdfs/5.pdf"
+ assert not resolved.exists()
+ assert victim_pdf.read_bytes() == mock_pdf_content
diff --git a/tests/research_library/test_utils.py b/tests/research_library/test_utils.py
index 54bdd9bbe4..381563afde 100644
--- a/tests/research_library/test_utils.py
+++ b/tests/research_library/test_utils.py
@@ -160,10 +160,10 @@ class TestGetAbsolutePathFromSettings:
"local_deep_research.utilities.db_utils.get_settings_manager",
return_value=mock_settings,
):
- result = get_absolute_path_from_settings("pdfs/test.pdf")
+ result = get_absolute_path_from_settings("pdfs/test.pdf", "alice")
assert result is not None
- assert result == tmp_path / "pdfs/test.pdf"
+ assert result == tmp_path / "alice" / "pdfs/test.pdf"
def test_empty_path_returns_library_root(self, tmp_path):
"""Empty path should return the library root."""
@@ -202,7 +202,7 @@ class TestGetAbsolutePathFromSettings:
]
for path in traversal_paths:
- result = get_absolute_path_from_settings(path)
+ result = get_absolute_path_from_settings(path, "alice")
assert result is None, f"Path '{path}' should have been blocked"
def test_symlink_blocked(self, tmp_path):
@@ -214,7 +214,8 @@ class TestGetAbsolutePathFromSettings:
)
# Create a symlink inside library root pointing outside
- link_path = tmp_path / "evil_link"
+ (tmp_path / "alice").mkdir()
+ link_path = tmp_path / "alice" / "evil_link"
try:
link_path.symlink_to("/etc/passwd")
except OSError:
@@ -227,7 +228,7 @@ class TestGetAbsolutePathFromSettings:
"local_deep_research.utilities.db_utils.get_settings_manager",
return_value=mock_settings,
):
- result = get_absolute_path_from_settings("evil_link")
+ result = get_absolute_path_from_settings("evil_link", "alice")
assert result is None
@@ -581,7 +582,7 @@ class TestPathTraversalAttackVectors:
"../../../../etc/shadow",
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
assert result is None, f"Attack '{attack}' should be blocked"
def test_nested_traversal_escaping_root(self, tmp_path):
@@ -604,7 +605,7 @@ class TestPathTraversalAttackVectors:
"a/b/c/d/../../../../../../../../etc/passwd",
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
assert result is None, f"Attack '{attack}' should be blocked"
def test_traversal_staying_in_root_is_allowed(self, tmp_path):
@@ -623,11 +624,11 @@ class TestPathTraversalAttackVectors:
# This path uses .. but stays within library root
# valid/path/to/../../../outside resolves to "outside" which is in root
result = get_absolute_path_from_settings(
- "valid/path/to/../../../outside"
+ "valid/path/to/../../../outside", "alice"
)
- # This is actually safe - resolves to tmp_path/outside
+ # This remains inside the requesting user's root.
assert result is not None
- assert result == tmp_path / "outside"
+ assert result == tmp_path / "alice" / "outside"
def test_absolute_path_injection(self, tmp_path):
"""Absolute paths should be blocked."""
@@ -649,7 +650,7 @@ class TestPathTraversalAttackVectors:
"/var/log/auth.log",
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
assert result is None, f"Attack '{attack}' should be blocked"
def test_dot_current_dir_traversal(self, tmp_path):
@@ -671,7 +672,7 @@ class TestPathTraversalAttackVectors:
"././../secret.txt",
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
assert result is None, f"Attack '{attack}' should be blocked"
def test_double_slash_traversal(self, tmp_path):
@@ -693,7 +694,7 @@ class TestPathTraversalAttackVectors:
"pdfs//..//..//secret.txt",
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
assert result is None, f"Attack '{attack}' should be blocked"
def test_backslash_traversal_windows_style(self, tmp_path):
@@ -721,7 +722,7 @@ class TestPathTraversalAttackVectors:
"..\\..\\windows\\win.ini",
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
if sys.platform == "win32":
# On Windows, these should be blocked
assert result is None, (
@@ -751,7 +752,7 @@ class TestPathTraversalAttackVectors:
"subdir/../../../../../../etc/passwd",
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
assert result is None, f"Attack '{attack}' should be blocked"
def test_mangled_patterns_stay_in_library(self, tmp_path):
@@ -777,7 +778,7 @@ class TestPathTraversalAttackVectors:
"..../file.pdf",
]
for pattern in patterns:
- result = get_absolute_path_from_settings(pattern)
+ result = get_absolute_path_from_settings(pattern, "alice")
# These are allowed because they stay in library
if result is not None:
assert str(tmp_path) in str(result)
@@ -806,7 +807,7 @@ class TestPathTraversalAttackVectors:
"../../../etc/passwd%00.pdf", # URL-encoded null (if decoded)
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
assert result is None, (
f"Null byte attack '{attack!r}' should be blocked"
)
@@ -834,7 +835,7 @@ class TestPathTraversalAttackVectors:
"//attacker.com/share/file.pdf",
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
# Should be blocked or stay within library
if result is not None:
assert str(tmp_path) in str(result), (
@@ -864,7 +865,7 @@ class TestPathTraversalAttackVectors:
"..;/..;/..;/etc/passwd", # Tomcat bypass
]
for attack in attacks:
- result = get_absolute_path_from_settings(attack)
+ result = get_absolute_path_from_settings(attack, "alice")
# These are unusual patterns - should either be blocked or safe
if result is not None:
assert str(tmp_path) in str(result), (
@@ -891,10 +892,10 @@ class TestPathTraversalEdgeCases:
# Whitespace-only paths should return None or be sanitized
whitespace_paths = [" ", "\t", "\n", " \t\n "]
for path in whitespace_paths:
- result = get_absolute_path_from_settings(path)
+ result = get_absolute_path_from_settings(path, "alice")
# After strip(), these become empty, which should return library_root
# or None depending on implementation
- assert result is None or result == tmp_path
+ assert result is None or result == tmp_path / "alice"
def test_path_with_spaces(self, tmp_path):
"""Valid paths with spaces should work."""
@@ -909,9 +910,11 @@ class TestPathTraversalEdgeCases:
"local_deep_research.utilities.db_utils.get_settings_manager",
return_value=mock_settings,
):
- result = get_absolute_path_from_settings("pdfs/my document.pdf")
+ result = get_absolute_path_from_settings(
+ "pdfs/my document.pdf", "alice"
+ )
assert result is not None
- assert result == tmp_path / "pdfs/my document.pdf"
+ assert result == tmp_path / "alice" / "pdfs/my document.pdf"
def test_path_with_special_characters(self, tmp_path):
"""Paths with allowed special characters should work."""
@@ -933,9 +936,9 @@ class TestPathTraversalEdgeCases:
"pdfs/test.file.pdf",
]
for path in valid_paths:
- result = get_absolute_path_from_settings(path)
+ result = get_absolute_path_from_settings(path, "alice")
assert result is not None, f"Path '{path}' should be allowed"
- assert result == tmp_path / path
+ assert result == tmp_path / "alice" / path
def test_very_long_path(self, tmp_path):
"""Very long paths should be handled without crashing."""
@@ -952,7 +955,7 @@ class TestPathTraversalEdgeCases:
):
# Create a very long but valid path
long_path = "/".join(["subdir"] * 50) + "/file.pdf"
- result = get_absolute_path_from_settings(long_path)
+ result = get_absolute_path_from_settings(long_path, "alice")
# Should either work or fail gracefully (no crash)
assert result is None or isinstance(result, Path)
@@ -977,7 +980,7 @@ class TestPathTraversalEdgeCases:
]
for path in unicode_paths:
# Should not crash - just verify no exception is raised
- _ = get_absolute_path_from_settings(path)
+ _ = get_absolute_path_from_settings(path, "alice")
# Result depends on filesystem support
diff --git a/tests/research_library/utils/test_utils.py b/tests/research_library/utils/test_utils.py
index 684ad7eb51..48c4396aed 100644
--- a/tests/research_library/utils/test_utils.py
+++ b/tests/research_library/utils/test_utils.py
@@ -343,8 +343,8 @@ class TestGetAbsolutePathFromSettings:
get_absolute_path_from_settings,
)
- result = get_absolute_path_from_settings("docs/file.pdf")
- assert result == tmp_path / "library" / "docs" / "file.pdf"
+ result = get_absolute_path_from_settings("docs/file.pdf", "alice")
+ assert result == tmp_path / "library" / "alice" / "docs" / "file.pdf"
@patch("local_deep_research.utilities.db_utils.get_settings_manager")
@patch("local_deep_research.research_library.utils.get_library_directory")
diff --git a/tests/security/test_operator_only_library_path.py b/tests/security/test_operator_only_library_path.py
index 7de56e9a53..8f4007c6fd 100644
--- a/tests/security/test_operator_only_library_path.py
+++ b/tests/security/test_operator_only_library_path.py
@@ -91,7 +91,6 @@ def test_legacy_location_survives_upgrade_and_operator_override(
get_absolute_path_from_settings(
"pdfs/5.pdf",
"alice",
- allow_legacy_fallback=False,
settings_manager=manager,
)
== stored_pdf
diff --git a/tests/security/test_pdf_path_traversal.py b/tests/security/test_pdf_path_traversal.py
index 8678735342..21997242fe 100644
--- a/tests/security/test_pdf_path_traversal.py
+++ b/tests/security/test_pdf_path_traversal.py
@@ -119,10 +119,15 @@ class TestGetAbsolutePathFromSettings:
)
mock_mgr = MagicMock()
- mock_mgr.get_setting.return_value = str(tmp_path)
+ mock_mgr.get_setting.side_effect = lambda key, default=None: {
+ "research_library.storage_path": str(tmp_path),
+ "research_library.shared_library": False,
+ }.get(key, default)
mock_settings_mgr.return_value = mock_mgr
- result = get_absolute_path_from_settings("../../etc/passwd")
+ result = get_absolute_path_from_settings(
+ "../../etc/passwd", username="alice"
+ )
assert result is None
@patch("local_deep_research.utilities.db_utils.get_settings_manager")
@@ -132,11 +137,25 @@ class TestGetAbsolutePathFromSettings:
)
mock_mgr = MagicMock()
- mock_mgr.get_setting.return_value = str(tmp_path)
+ mock_mgr.get_setting.side_effect = lambda key, default=None: {
+ "research_library.storage_path": str(tmp_path),
+ "research_library.shared_library": False,
+ }.get(key, default)
mock_settings_mgr.return_value = mock_mgr
- result = get_absolute_path_from_settings("docs/file.pdf")
- assert result.is_relative_to(tmp_path.resolve())
+ result = get_absolute_path_from_settings(
+ "docs/file.pdf", username="alice"
+ )
+ assert result == tmp_path.resolve() / "alice" / "docs" / "file.pdf"
+
+ @pytest.mark.parametrize("username", [None, ""])
+ def test_nonempty_path_requires_user_context(self, username):
+ from local_deep_research.research_library.utils import (
+ get_absolute_path_from_settings,
+ )
+
+ with pytest.raises(ValueError, match="without a user context"):
+ get_absolute_path_from_settings("docs/file.pdf", username=username)
@patch("local_deep_research.utilities.db_utils.get_settings_manager")
def test_empty_path_returns_library_root(self, mock_settings_mgr, tmp_path):
diff --git a/tests/settings/test_shipped_defaults_contracts.py b/tests/settings/test_shipped_defaults_contracts.py
index 4a6e074d80..77038ecd5b 100644
--- a/tests/settings/test_shipped_defaults_contracts.py
+++ b/tests/settings/test_shipped_defaults_contracts.py
@@ -568,7 +568,6 @@ class TestSecureByDefault:
"notifications.allow_private_ips",
"policy.allow_unprotected_egress",
"research_library.allow_filesystem_pdf_storage",
- "research_library.allow_legacy_read_fallback",
"research_library.allow_shared_library",
"search.allow_private_engine_urls",
"search.allow_private_result_fetch",
diff --git a/tests/test_config_env_var_contracts.py b/tests/test_config_env_var_contracts.py
index dfed2d65a5..619a214d31 100644
--- a/tests/test_config_env_var_contracts.py
+++ b/tests/test_config_env_var_contracts.py
@@ -206,11 +206,15 @@ def test_registry_inventory_is_populated_and_holds_the_security_gates():
"LDR_SECURITY_ALLOW_NAT64",
"LDR_SECURITY_CORS_ALLOWED_ORIGINS",
"LDR_SECURITY_WEBSOCKET_ALLOWED_ORIGINS",
- "LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK",
"LDR_BOOTSTRAP_ALLOW_UNENCRYPTED",
):
assert gate in declared, f"security gate {gate} left the registry"
+ # Shared-root PDF reads were removed, including their opt-in switch.
+ # Keep the retirement explicit instead of restoring a no-op setting to
+ # satisfy the inventory of security gates that still exist.
+ assert "LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK" not in declared
+
# ---------------------------------------------------------------------------
# 2. Documentation coverage
diff --git a/tests/web/routers/test_library_pdf_root_expandvars.py b/tests/web/routers/test_library_pdf_root_expandvars.py
index 418b9cbf67..d8b5d5a152 100644
--- a/tests/web/routers/test_library_pdf_root_expandvars.py
+++ b/tests/web/routers/test_library_pdf_root_expandvars.py
@@ -1,83 +1,101 @@
-"""`legacy_root` must get an env-var-expanded path, like `library_root` does.
+"""PDF routes expand storage roots and never read ambiguous legacy files."""
-main's #5537 wrapped the library storage path in ``os.path.expandvars()``
-before ``.expanduser().resolve()``. The FastAPI port of ``view_pdf_page``
-dropped that call.
-
-``library_root`` survived the omission by accident: ``apply_user_subdir()``
-re-expands whatever it is handed. ``legacy_root`` did not -- it is passed the
-raw ``base_root``. So with a ``research_library.storage_path`` containing an
-env-var token, the legacy fallback (which exists to find PDFs downloaded
-before per-user isolation, #5521) would search a directory literally named
-``$LDR_TEST_ROOT`` and silently find nothing.
-
-This pins both roots as expanded, so the two cannot drift apart again.
-"""
-
-import inspect
-import os
-from pathlib import Path
+from contextlib import contextmanager
+from types import SimpleNamespace
import pytest
-
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+from sqlalchemy import create_engine
+from sqlalchemy.orm import Session
+from sqlalchemy.pool import StaticPool
+
+from local_deep_research.database.models import Base
+from local_deep_research.database.models.library import (
+ Document,
+ DocumentBlob,
+ SourceType,
+)
from local_deep_research.web.routers import library as library_mod
-def env_token_root(tmp_path, monkeypatch):
- """A storage_path written with an env-var token, plus its real target."""
- real = tmp_path / "libroot"
- real.mkdir()
- monkeypatch.setenv("LDR_TEST_ROOT", str(real))
- return "$LDR_TEST_ROOT", real
-
-
-def test_expandvars_is_applied_to_the_storage_path(env_token_root):
- """The bug in one line: without expandvars the token survives literally."""
- token, real = env_token_root
-
- expanded = Path(os.path.expandvars(token)).expanduser().resolve()
- unexpanded = Path(token).expanduser().resolve()
-
- assert expanded == real.resolve()
- assert "$LDR_TEST_ROOT" in str(unexpanded), (
- "sanity check: an unexpanded token really does survive into the path"
[email protected]("route", ["document", "api/document"])
[email protected]("source", ["legacy", "per_user", "database"])
[email protected]("old_flag", ["false", "true"])
+def test_pdf_routes_keep_user_storage_boundary(
+ tmp_path, monkeypatch, route, source, old_flag
+):
+ base = tmp_path / "library"
+ monkeypatch.setenv("LDR_TEST_ROOT", str(base))
+ monkeypatch.setenv(
+ "LDR_RESEARCH_LIBRARY_ALLOW_LEGACY_READ_FALLBACK", old_flag
)
- assert expanded != unexpanded
-
-
-def test_view_pdf_page_expands_the_storage_path_before_use():
- """Pin the fix at its source.
-
- A route-level test cannot reach this cheaply: view_pdf_page looks the
- document up first, so a nonexistent id short-circuits before
- PDFStorageManager is ever constructed, and seeding a real document plus
- real PDF bytes to observe a path computation would test the fixture more
- than the fix.
-
- So assert the property directly on the source: the base_root computation
- must expand env vars. `library_root` is safe either way because
- apply_user_subdir() re-expands, but `legacy_root` receives base_root RAW,
- which is exactly how the omission became observable.
- """
- src = Path(inspect.getsourcefile(library_mod)).read_text()
- start = src.index("base_root = (")
- block = src[start : src.index(")", src.index("resolve()", start))]
-
- assert "expandvars" in block, (
- "base_root must expand env vars before .expanduser().resolve(); "
- "without it legacy_root searches a directory named after the literal "
- "token (main #5537 applied expandvars here)"
+ monkeypatch.setenv("LDR_RESEARCH_LIBRARY_ALLOW_SHARED_LIBRARY", "false")
+ legacy_pdf = base / "pdfs" / "5.pdf"
+ legacy_pdf.parent.mkdir(parents=True)
+ legacy_bytes = b"%PDF-1.4\nlegacy file with unverified ownership\n%%EOF"
+ legacy_pdf.write_bytes(legacy_bytes)
+ own_bytes = b"%PDF-1.4\nverified user's PDF\n%%EOF"
+ if source == "per_user":
+ own_pdf = base / "alice" / "pdfs" / "5.pdf"
+ own_pdf.parent.mkdir(parents=True)
+ own_pdf.write_bytes(own_bytes)
+
+ engine = create_engine(
+ "sqlite://",
+ poolclass=StaticPool,
+ connect_args={"check_same_thread": False},
)
-
-
-def test_legacy_root_is_the_same_base_root_that_was_expanded():
- """Guards the specific coupling: whatever base_root ends up being, it is
- what legacy_root gets. If someone later expands only the library_root
- branch, this fails."""
- src = Path(inspect.getsourcefile(library_mod)).read_text()
-
- assert "legacy_root=base_root" in src.replace(" ", ""), (
- "legacy_root is no longer fed from base_root; re-check that it still "
- "receives an env-var-expanded path"
+ Base.metadata.create_all(engine)
+ settings = {
+ "research_library.storage_path": "$LDR_TEST_ROOT",
+ "research_library.shared_library": False,
+ "research_library.pdf_storage_mode": "database",
+ }
+ monkeypatch.setattr(
+ library_mod,
+ "get_settings_manager",
+ lambda _session: SimpleNamespace(get_setting=settings.get),
)
+ with Session(engine) as session:
+ session.add(
+ SourceType(id="test-source", name="test", display_name="Test")
+ )
+ session.add(
+ Document(
+ id="doc-1",
+ source_type_id="test-source",
+ document_hash="test-hash",
+ file_type="pdf",
+ file_path="pdfs/5.pdf",
+ filename="test.pdf",
+ file_size=len(own_bytes),
+ title="Stored document",
+ status="completed",
+ )
+ )
+ session.flush()
+ if source == "database":
+ session.add(DocumentBlob(document_id="doc-1", pdf_binary=own_bytes))
+ session.commit()
+
+ @contextmanager
+ def user_session(username):
+ assert username == "alice"
+ yield session
+
+ monkeypatch.setattr(library_mod, "get_user_db_session", user_session)
+ app = FastAPI()
+ app.include_router(library_mod.router)
+ app.dependency_overrides[library_mod.require_auth] = lambda: "alice"
+ with TestClient(app) as client:
+ response = client.get(f"/library/{route}/doc-1/pdf")
+ if source == "legacy":
+ assert response.status_code == 404
+ assert legacy_bytes not in response.content
+ else:
+ assert response.status_code == 200
+ assert response.content == own_bytes
+ assert response.headers["content-type"] == "application/pdf"
+ engine.dispose()
+ assert legacy_pdf.read_bytes() == legacy_bytes