~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
fix(security): restrict optional report backup permissions (#7169)
LearningCircuite5cdabd26b6e
diff --git a/changelog.d/+report-backup-owner-only.security.md b/changelog.d/+report-backup-owner-only.security.mdnew file mode 100644index 0000000000..01f45225e3--- /dev/null+++ b/changelog.d/+report-backup-owner-only.security.md@@ -0,0 +1 @@+Optional plaintext report backups now use owner-only permissions on POSIX while keeping existing report paths. At startup the web application sets `research_outputs/` to mode 0700, and new or rewritten report and metadata files use mode 0600. Report file backups are refused when another OS account owns that directory or its filesystem does not keep mode 0700, for example on Docker Desktop bind mounts of Windows folders, WSL drives such as `/mnt/c`, CIFS/SMB shares and exFAT. Reports are still saved in the database, users who enabled the option see a warning on the research page, and the server log names the cause.diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.mdindex 118a313c85..9d23eef82c 100644--- a/docs/CONFIGURATION.md+++ b/docs/CONFIGURATION.md@@ -216,7 +216,7 @@ They are not available in the Web UI because they are needed to start the applic| `rate_limiting.memory_window` | `LDR_RATE_LIMITING_MEMORY_WINDOW` | `100` | Number of recent requests to remember per search engine for calculating optimal wait times. At least 3 successful requests are needed before wait time estimates start updating. | APP || `rate_limiting.profile` | `LDR_RATE_LIMITING_PROFILE` | `balanced` | Overall rate limiting behavior. 'Conservative' uses longer waits and adapts slowly (fewer errors). 'Balanced' is the default. 'Aggressive' uses shorter waits and adapts quickly (faster but riskier). | APP || `report.citation_format` | `LDR_REPORT_CITATION_FORMAT` | `number_hyperlinks` | Citation formatting style for reports. Options include numbered hyperlinks [1], domain-based hyperlinks [arxiv.org], smart domain numbering [arxiv.org-1], source-tagged citations with the global citation number preserved (e.g. [arxiv-1], [openai.com-2], [arxiv-3]), and plain numbered references without links. | REPORT |-| `report.enable_file_backup` | `LDR_REPORT_ENABLE_FILE_BACKUP` | `false` | Also save reports as files on disk for external access (reports are always stored in the database regardless). Enable this if you need to access reports from outside the web interface. | REPORT |+| `report.enable_file_backup` | `LDR_REPORT_ENABLE_FILE_BACKUP` | `false` | Also save plaintext report files in research_outputs/ under the data directory for external access (reports remain in the database). On POSIX, that directory is set to mode 0700 and new or rewritten report files to 0600; on Windows, configure equivalent access controls. If another OS account owns the directory or its filesystem does not keep mode 0700, as on Docker Desktop bind mounts of Windows folders, WSL drives such as /mnt/c, CIFS/SMB shares and exFAT, no files are written and the research page shows a warning. Keep the data directory on a Linux filesystem path or a Docker named volume, or turn this off. | REPORT || `report.export_formats` | `LDR_REPORT_EXPORT_FORMATS` | ``["markdown", "latex", "quarto", "ris"]`` | Available export format options displayed in the user interface. Supported formats include Markdown (.md), LaTeX (.tex), Quarto (.qmd), and RIS bibliographic format. | REPORT || `report.max_context_chars` | `LDR_REPORT_MAX_CONTEXT_CHARS` | `4000` | Maximum character limit for previous section context when generating report sections. If accumulated context exceeds this, it is truncated at sentence boundaries. Lower values are safer for smaller local models. Default is 4000. | REPORT || `report.max_context_sections` | `LDR_REPORT_MAX_CONTEXT_SECTIONS` | `3` | Number of previously generated report sections to include as context when writing new sections. Helps the LLM avoid repeating content. Higher values increase context size but may exceed smaller model limits. Default is 3. | REPORT |diff --git a/docs/deployment/upgrading.md b/docs/deployment/upgrading.mdindex 72768caaef..a679d72a5a 100644--- a/docs/deployment/upgrading.md+++ b/docs/deployment/upgrading.md@@ -44,6 +44,11 @@ See the [configuration reference](../CONFIGURATION.md).with the password the user had when it was taken. Tell users to keeptheir current passwords until a rollback is no longer possible; do notcollect users' passwords yourself.+ If `report.enable_file_backup` is enabled, this backup also includes+ plaintext reports in `research_outputs/`. Keep it, and any other offline+ copies, accessible only to trusted operators. See+ [Report file backups](#report-file-backups) for the new permissions on the+ live directory.4. Review the release's compatibility notes and your reverse-proxy setup.Socket.IO now uses `/ws/socket.io`. A trusted TLS-terminating proxy requires`TRUST_PROXY_HEADERS=true` and must be the only way to reach the backend.@@ -105,6 +110,30 @@ path parameters now return 422 instead of 404. `GET /settings` without atrailing slash now returns a 307 redirect to `/settings/`; request the latterdirectly if the client does not follow redirects.+### Report file backups++At startup, the web application now sets `research_outputs/` under the data+directory to mode `0700` on POSIX systems, whether or not+`report.enable_file_backup` is enabled. New or rewritten report and metadata+files get mode `0600`. Reports saved before the upgrade keep their own modes+until they are rewritten, and from then on other OS accounts cannot open them+through the directory. Access gained earlier is not revoked: a hard link made+while the directory was readable still opens a report that keeps its old+mode, and a file descriptor or working directory that another process+already holds inside the directory keeps working. If other accounts could+read the directory before, check for such links and processes.++This needs a directory that the account running Local Deep Research owns, on+a filesystem that keeps the mode. Docker Desktop bind mounts of Windows+folders, WSL drives such as `/mnt/c`, CIFS/SMB shares and exFAT often accept+`chmod` without keeping it, or show another owner. In those cases the+application writes no report files. Reports are still saved in the database,+the server log names the cause, and users who enabled the option see a+warning on the research page. Keep the data directory (`LDR_DATA_DIR`) on a+Linux filesystem path or a Docker named volume owned by that account, or turn+the option off. On Windows, configure access controls on the data directory+for the application account.+### PDFs in the old shared library directoryLibrary reads no longer fall back from a missing per-user PDF to the olddiff --git a/src/local_deep_research/config/paths.py b/src/local_deep_research/config/paths.pyindex 27c0d1ac1f..939736a1af 100644--- a/src/local_deep_research/config/paths.py+++ b/src/local_deep_research/config/paths.py@@ -3,8 +3,10 @@ Centralized path configuration for Local Deep Research.Handles database location using platformdirs for proper user data storage."""+import errnoimport hashlibimport os+import statfrom pathlib import Pathimport platformdirs@@ -61,7 +63,9 @@ def get_data_directory() -> Path:return data_dir-def _ensure_dir(subdir: str, label: str | None = None) -> Path:+def _ensure_dir(+ subdir: str, label: str | None = None, mode: int | None = None+) -> Path:"""Create (if needed) and return ``<data_dir>/<subdir>``.Args:@@ -69,25 +73,143 @@ def _ensure_dir(subdir: str, label: str | None = None) -> Path:(e.g. ``"encrypted_databases/backups"``).label: If provided, emit a debug log ``"Using {label} directory:{path}"``. ``None`` suppresses logging.+ mode: Optional creation mode for a new directory.Returns:The ensured directory path."""path = get_data_directory() / subdir- path.mkdir(parents=True, exist_ok=True) # Safe: bootstrap data subdirectory+ if mode is None:+ path.mkdir( # Safe: bootstrap data subdirectory+ parents=True, exist_ok=True+ )+ else:+ path.mkdir( # Safe: bootstrap data subdirectory+ parents=True, exist_ok=True, mode=mode+ )if label is not None:logger.debug(f"Using {label} directory: {path}")return path-def get_research_outputs_directory() -> Path:+#: ``DirectoryNotPrivateError.reason`` values.+OWNED_BY_ANOTHER_ACCOUNT = "owned_by_another_account"+MODE_NOT_KEPT = "mode_not_kept"++#: What an operator can do when report file backups are refused.+REPORT_BACKUP_REMEDY = (+ "Keep the data directory (LDR_DATA_DIR) on a Linux filesystem path or a "+ "Docker named volume owned by the account that runs Local Deep Research, "+ "or turn off report.enable_file_backup"+)+++class DirectoryNotPrivateError(PermissionError):+ """A directory cannot be restricted to the current OS account.++ ``reason`` is ``OWNED_BY_ANOTHER_ACCOUNT`` or ``MODE_NOT_KEPT``."""- Get the directory for storing research outputs (reports, etc.).++ def __init__(self, message: str, reason: str):+ super().__init__(message)+ self.reason = reason+++def restrict_directory_to_owner(path: Path) -> None:+ """Set ``path`` to mode 0700 on POSIX and verify the result.++ Other systems have no POSIX mode bits, so this does nothing there; the+ directory's access-control list decides who can read it.++ The result is checked because some mounts accept chmod without keeping+ it, or show another owner: Docker Desktop bind mounts of Windows+ folders, WSL drives such as /mnt/c, CIFS/SMB shares and exFAT.++ Raises:+ DirectoryNotPrivateError: another OS account owns the directory, or+ its mode is not exactly 0700 after chmod.+ OSError: chmod or stat failed for another reason.+ """+ if os.name != "posix":+ return+ try:+ path.chmod(0o700)+ except PermissionError:+ # Only the owner may chmod. Report a foreign owner as such below;+ # any other cause propagates unchanged.+ if path.stat().st_uid == os.geteuid():+ raise+ info = path.stat()+ if info.st_uid != os.geteuid():+ raise DirectoryNotPrivateError(+ f"{path} is not private: it is owned by uid {info.st_uid}, "+ f"but this process runs as uid {os.geteuid()}",+ OWNED_BY_ANOTHER_ACCOUNT,+ )+ mode = stat.S_IMODE(info.st_mode)+ if mode != 0o700:+ raise DirectoryNotPrivateError(+ f"{path} is not private: its filesystem kept mode {mode:04o} "+ "after chmod 0700",+ MODE_NOT_KEPT,+ )+++def describe_private_directory_refusal(error: OSError) -> str:+ """Say why ``restrict_directory_to_owner`` refused a directory.++ Fixed text for log lines: it never echoes the error message, only its+ category and, for other OS errors, the symbolic errno.+ """+ reason = getattr(error, "reason", None)+ if reason == OWNED_BY_ANOTHER_ACCOUNT:+ return "another OS account owns it"+ if reason == MODE_NOT_KEPT:+ return "its filesystem did not keep mode 0700"+ code = errno.errorcode.get(getattr(error, "errno", None) or 0)+ return "it could not be created or set to mode 0700" + (+ f" ({code})" if code else ""+ )+++def get_research_outputs_directory() -> Path:+ """Get the directory for research outputs and optional report backups.++ A missing directory is created with mode 0700 (subject to the umask).+ An existing directory is returned unchanged: the web application+ restricts it at startup (``restrict_research_outputs_directory``), and+ ``FileReportStorage`` restricts and verifies it before writing.Returns:Path to research outputs directory"""- return _ensure_dir("research_outputs", label="research outputs")+ return _ensure_dir("research_outputs", label="research outputs", mode=0o700)+++def restrict_research_outputs_directory() -> bool:+ """Restrict the research outputs directory to this OS account.++ The web application calls this once at startup, so a directory left at+ 0755 by an older release becomes 0700 even when no user has report file+ backups enabled. An ``OSError`` is logged with its remedy, not raised:+ ``FileReportStorage`` then refuses to write report backups there, and+ database report storage is unaffected.++ Returns:+ False when the directory could not be made private, otherwise True+ (always True on non-POSIX systems, where nothing is changed).+ """+ try:+ restrict_directory_to_owner(get_research_outputs_directory())+ except OSError as exc:+ cause = describe_private_directory_refusal(exc)+ logger.warning(+ f"Could not make the research outputs directory private: {cause}. "+ "Report file backups will be refused until this is fixed; "+ f"reports are still saved in the database. {REPORT_BACKUP_REMEDY}."+ )+ return False+ return Truedef get_journal_data_directory() -> Path:diff --git a/src/local_deep_research/defaults/default_settings.json b/src/local_deep_research/defaults/default_settings.jsonindex 6716a0a1a6..ee6920f26e 100644--- a/src/local_deep_research/defaults/default_settings.json+++ b/src/local_deep_research/defaults/default_settings.json@@ -951,7 +951,7 @@},"report.enable_file_backup": {"category": "report_parameters",- "description": "Also save reports as files on disk for external access (reports are always stored in the database regardless). Enable this if you need to access reports from outside the web interface.",+ "description": "Also save plaintext report files in research_outputs/ under the data directory for external access (reports remain in the database). On POSIX, that directory is set to mode 0700 and new or rewritten report files to 0600; on Windows, configure equivalent access controls. If another OS account owns the directory or its filesystem does not keep mode 0700, as on Docker Desktop bind mounts of Windows folders, WSL drives such as /mnt/c, CIFS/SMB shares and exFAT, no files are written and the research page shows a warning. Keep the data directory on a Linux filesystem path or a Docker named volume, or turn this off.","editable": true,"max_value": null,"min_value": null,diff --git a/src/local_deep_research/security/file_write_verifier.py b/src/local_deep_research/security/file_write_verifier.pyindex 52e294b0df..a271421c52 100644--- a/src/local_deep_research/security/file_write_verifier.py+++ b/src/local_deep_research/security/file_write_verifier.py@@ -20,6 +20,22 @@ def _open_nofollow(path: str, flags: int) -> int:return os.open(path, flags | _O_NOFOLLOW, 0o666)+def _open_nofollow_with_permissions(+ path: str, flags: int, permissions: int+) -> int:+ """Tighten an existing file before truncation, or create it privately."""+ truncate = bool(flags & os.O_TRUNC)+ fd = os.open(path, (flags & ~os.O_TRUNC) | _O_NOFOLLOW, permissions)+ try:+ os.fchmod(fd, permissions)+ if truncate:+ os.ftruncate(fd, 0)+ except BaseException:+ os.close(fd)+ raise+ return fd++# Keys that should never be written to disk in clear textSENSITIVE_KEYS = frozenset({@@ -80,6 +96,7 @@ def write_file_verified(mode: str = "w",encoding: str = "utf-8",settings_snapshot: dict = None,+ permissions: int | None = None,) -> None:"""Write content to a file only if security settings allow it.@@ -92,6 +109,8 @@ def write_file_verified(mode: File open mode (default: "w")encoding: File encoding (default: "utf-8")settings_snapshot: Optional settings snapshot for programmatic mode+ permissions: POSIX file mode for private output. Existing files are+ tightened before their contents are replaced.Raises:FileWriteSecurityError: If the security setting doesn't match required value@@ -128,15 +147,22 @@ def write_file_verified(# component. Parent-directory symlinks and a leaf already dereferenced# by the caller's Path.resolve() are outside this check.# Let open() validate modes, own the descriptor and handle text/binary- # translation. Python supplies platform flags; creation honors the umask.- # This remains an in-place write, not an atomic file replacement.+ # translation. This remains an in-place write, not an atomic replacement.+ opener = _open_nofollow+ if permissions is not None:+ if os.name != "posix" or not 0 <= permissions <= 0o777:+ raise ValueError("Private POSIX file permissions are unavailable")++ def private_opener(path: str, flags: int) -> int:+ return _open_nofollow_with_permissions(path, flags, permissions)++ opener = private_opener+if "b" in mode:- with open(filepath, mode, opener=_open_nofollow) as f: # nosec B603+ with open(filepath, mode, opener=opener) as f: # nosec B603f.write(content)else:- with open(- filepath, mode, encoding=encoding, opener=_open_nofollow- ) as f: # nosec B603+ with open(filepath, mode, encoding=encoding, opener=opener) as f: # nosec B603f.write(content)logger.debug(@@ -151,6 +177,7 @@ def write_json_verified(required_value: Any = True,context: str = "",settings_snapshot: dict = None,+ permissions: int | None = None,**json_kwargs,) -> None:"""Write JSON data to a file only if security settings allow it.@@ -162,6 +189,7 @@ def write_json_verified(required_value: Required value for the setting (default: True)context: Description of what's being written (for error messages)settings_snapshot: Optional settings snapshot for programmatic mode+ permissions: POSIX file mode for private output.**json_kwargs: Additional keyword arguments to pass to json.dumps()(e.g., indent=2, ensure_ascii=False, sort_keys=True, default=custom_serializer)@@ -194,4 +222,5 @@ def write_json_verified(mode="w",encoding="utf-8",settings_snapshot=settings_snapshot,+ permissions=permissions,)diff --git a/src/local_deep_research/storage/database_with_file_backup.py b/src/local_deep_research/storage/database_with_file_backup.pyindex dba8c26b51..d994f5a851 100644--- a/src/local_deep_research/storage/database_with_file_backup.py+++ b/src/local_deep_research/storage/database_with_file_backup.py@@ -7,6 +7,10 @@ from sqlalchemy.orm import Sessionfrom .base import ReportStoragefrom .database import DatabaseReportStoragefrom .file import FileReportStorage+from ..config.paths import (+ REPORT_BACKUP_REMEDY,+ describe_private_directory_refusal,+)class DatabaseWithFileBackupStorage(ReportStorage):@@ -23,10 +27,24 @@ class DatabaseWithFileBackupStorage(ReportStorage):Args:session: SQLAlchemy database session- enable_file_storage: Whether to also save reports to file system+ enable_file_storage: Whether to also save reports to file system.+ If FileReportStorage refuses its directory with an OSError+ (for example one that is not private), ``file_storage``+ stays None, a warning names the cause, and reports are saved+ to the database only."""self.db_storage = DatabaseReportStorage(session)- self.file_storage = FileReportStorage() if enable_file_storage else None+ self.file_storage = None+ if enable_file_storage:+ try:+ self.file_storage = FileReportStorage()+ except OSError as exc:+ cause = describe_private_directory_refusal(exc)+ logger.warning(+ "Report file backup is disabled because the research "+ f"outputs directory was refused: {cause}. Reports are "+ f"still saved in the database. {REPORT_BACKUP_REMEDY}."+ )self.enable_file_storage = enable_file_storagedef save_report(diff --git a/src/local_deep_research/storage/file.py b/src/local_deep_research/storage/file.pyindex 02f16f5343..b8ce6bcd0d 100644--- a/src/local_deep_research/storage/file.py+++ b/src/local_deep_research/storage/file.py@@ -1,11 +1,15 @@"""File-based report storage implementation."""+import osfrom pathlib import Pathfrom typing import Dict, Any, List, Optionalfrom loguru import loggerfrom .base import ReportStorage-from ..config.paths import get_research_outputs_directory+from ..config.paths import (+ get_research_outputs_directory,+ restrict_directory_to_owner,+)class FileReportStorage(ReportStorage):@@ -17,17 +21,28 @@ class FileReportStorage(ReportStorage):Args:base_dir: Base directory for storing reports.If None, uses default research outputs directory.++ Raises:+ DirectoryNotPrivateError: On POSIX, another OS account owns the+ directory or its filesystem did not keep mode 0700.+ OSError: The directory could not be created or restricted.+ DirectoryCreationSecurityError: The path contains a null byte or+ a ``..`` segment."""# Lazy import: mirrors the other lazy security imports in this# module (e.g. file_write_verifier) and avoids pulling security# into this module's top-level import graph.from ..security.directory_creation import create_directory- self.base_dir = base_dir or get_research_outputs_directory()- create_directory(- self.base_dir,+ self.base_dir = create_directory(+ base_dir or get_research_outputs_directory(),context="file report storage base directory",+ mode=0o700,)+ # The directory may already exist at 0755 from an older install.+ # Protect legacy backups, and refuse the directory unless it is+ # private, before any new write.+ restrict_directory_to_owner(self.base_dir)def _get_report_path(self, research_id: str) -> Path:"""Get the file path for a report."""@@ -69,6 +84,7 @@ class FileReportStorage(ReportStorage):content,"report.enable_file_backup",context="file storage backup",+ permissions=0o600 if os.name == "posix" else None,)# Save metadata if provided@@ -79,6 +95,7 @@ class FileReportStorage(ReportStorage):metadata,"report.enable_file_backup",context="file storage metadata",+ permissions=0o600 if os.name == "posix" else None,)logger.info(diff --git a/src/local_deep_research/web/fastapi_app.py b/src/local_deep_research/web/fastapi_app.pyindex cdedca1928..17f92782b3 100644--- a/src/local_deep_research/web/fastapi_app.py+++ b/src/local_deep_research/web/fastapi_app.py@@ -381,6 +381,19 @@ async def lifespan(app: FastAPI):)logger.info("=" * 60)+ # Restrict research_outputs/ (optional plaintext report backups) to this+ # OS account, so a directory left at 0755 by an older release becomes+ # 0700 whether or not any user has report file backups enabled. The+ # function logs its own failure and remedy; FileReportStorage refuses a+ # directory that is not private, so a failure here only disables file+ # backups. Wrapped so it can never block server boot.+ try:+ from ..config.paths import restrict_research_outputs_directory++ restrict_research_outputs_directory()+ except Exception:+ logger.exception("Research outputs directory startup check failed")+# Surface a cipher misconfiguration that otherwise only shows up as# affected users getting "Invalid username or password": a relaxed# SQLCipher KDF (test mode) on a deployment that already holds real userdiff --git a/src/local_deep_research/web/warning_checks/__init__.py b/src/local_deep_research/web/warning_checks/__init__.pyindex 30bf10c0f3..c7de68f58a 100644--- a/src/local_deep_research/web/warning_checks/__init__.py+++ b/src/local_deep_research/web/warning_checks/__init__.py@@ -18,6 +18,7 @@ from .backup import (check_backup_disabled,check_backup_healthy,check_no_backups_exist,+ check_report_backup_unavailable,)from .hardware import (LOCAL_PROVIDERS,@@ -426,6 +427,43 @@ def calculate_warnings(username: Optional[str] = None) -> List[dict]:except Exception:logger.debug("Backup status check skipped")+ # --- Report file backup check ---+ # Runs the same owner and mode check that FileReportStorage runs+ # before writing, so the banner shows while that check would+ # refuse the default directory. Own guard: a failure here must+ # not hide the history banners below.+ try:+ report_backup_enabled = bool(+ settings_manager.get_setting(+ "report.enable_file_backup", False+ )+ )+ report_backup_problem = None+ if report_backup_enabled:+ from ...config.paths import (+ DirectoryNotPrivateError,+ get_research_outputs_directory,+ restrict_directory_to_owner,+ )++ try:+ restrict_directory_to_owner(+ get_research_outputs_directory()+ )+ except DirectoryNotPrivateError as exc:+ report_backup_problem = exc.reason+ except OSError:+ report_backup_problem = "unavailable"+ w = _safe_check(+ check_report_backup_unavailable,+ report_backup_enabled,+ report_backup_problem,+ )+ if w:+ warnings.append(w)+ except Exception:+ logger.exception("Report file backup warning check skipped")+# --- History-based checks (need DB queries) ---if is_local and not dismiss_context_below_history:w = _safe_check(diff --git a/src/local_deep_research/web/warning_checks/backup.py b/src/local_deep_research/web/warning_checks/backup.pyindex e265e1f7d8..9f6440a306 100644--- a/src/local_deep_research/web/warning_checks/backup.py+++ b/src/local_deep_research/web/warning_checks/backup.py@@ -5,6 +5,55 @@ Pure functions that take primitive values and return warning dicts (or None).from typing import Optional+from ...config.paths import MODE_NOT_KEPT, OWNED_BY_ANOTHER_ACCOUNT++_REPORT_BACKUP_CAUSES = {+ OWNED_BY_ANOTHER_ACCOUNT: (+ "another OS account owns the server's research_outputs folder, as "+ "with Docker Desktop bind mounts of Windows folders or after the app "+ "ran as a different user"+ ),+ MODE_NOT_KEPT: (+ "the filesystem holding the server's research_outputs folder does not "+ "keep owner-only permissions, as on WSL drives such as /mnt/c, "+ "CIFS/SMB shares and exFAT"+ ),+}+++def check_report_backup_unavailable(+ backup_enabled: bool, problem: Optional[str]+) -> Optional[dict]:+ """Warn that report file backups are refused although they are enabled.++ ``problem`` is None when the folder is private, a+ ``DirectoryNotPrivateError.reason``, or any other string when the folder+ could not be prepared at all. Not dismissible: it shows while the+ setting is on and FileReportStorage would refuse the folder.+ """+ if not backup_enabled or problem is None:+ return None+ cause = _REPORT_BACKUP_CAUSES.get(+ problem,+ "the server could not make its research_outputs folder private "+ "(the server log names the cause when a report is saved)",+ )+ return {+ "type": "report_backup_unavailable",+ "icon": "⚠️",+ "title": "Report File Backups Disabled",+ "message": (+ "Report file backup is on, but no report files are being "+ f"written because {cause}. Reports are still saved in the "+ "database. Keep the data directory (LDR_DATA_DIR) on a Linux "+ "filesystem path or a Docker named volume owned by the account "+ "that runs the app, or turn off report file backup."+ ),+ "dismissKey": None,+ "actionUrl": "/settings#setting-report-enable_file_backup",+ "actionLabel": "Open report backup setting",+ }+def check_backup_disabled(backup_enabled: bool, dismissed: booldiff --git a/tests/config/test_paths.py b/tests/config/test_paths.pyindex 60bd7dcac2..4dc7502e81 100644--- a/tests/config/test_paths.py+++ b/tests/config/test_paths.py@@ -1,11 +1,15 @@"""Tests for paths module."""+import errnoimport hashlib+import os+import statfrom pathlib import Pathfrom unittest.mock import patchimport pytest+from local_deep_research.config import pathsfrom local_deep_research.config.paths import (get_data_directory,get_research_outputs_directory,@@ -78,6 +82,80 @@ class TestGetResearchOutputsDirectory:assert result.exists()assert result.is_dir()+ @pytest.mark.skipif(os.name != "posix", reason="POSIX mode bits")+ def test_lookup_leaves_an_existing_directory_unchanged(+ self, tmp_path, monkeypatch+ ):+ """Only startup and FileReportStorage restrict it, never a lookup."""+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))+ root = tmp_path / "research_outputs"+ root.mkdir()+ root.chmod(0o755)++ assert get_research_outputs_directory() == root+ assert stat.S_IMODE(root.stat().st_mode) == 0o755++[email protected](os.name != "posix", reason="POSIX mode bits")+class TestRestrictResearchOutputsDirectory:+ """The startup step that makes research_outputs/ private."""++ @pytest.mark.parametrize("already_exists", [False, True])+ def test_new_and_legacy_directories_become_private(+ self, tmp_path, monkeypatch, already_exists+ ):+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))+ root = tmp_path / "research_outputs"+ if already_exists:+ root.mkdir()+ root.chmod(0o755)+ (root / "legacy.md").write_text("old report", encoding="utf-8")++ assert paths.restrict_research_outputs_directory() is True++ assert stat.S_IMODE(root.stat().st_mode) == 0o700+ if already_exists:+ assert (root / "legacy.md").read_text(+ encoding="utf-8"+ ) == "old report"++ def test_chmod_failure_is_logged_and_lookup_still_works(+ self, tmp_path, monkeypatch, loguru_caplog+ ):+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))+ root = tmp_path / "research_outputs"+ root.mkdir()++ with patch.object(+ Path,+ "chmod",+ side_effect=PermissionError(errno.EPERM, "chmod refused"),+ ):+ with loguru_caplog.at_level("WARNING"):+ assert paths.restrict_research_outputs_directory() is False+ assert get_research_outputs_directory() == root++ assert "set to mode 0700 (EPERM)" in loguru_caplog.text+ assert "report.enable_file_backup" in loguru_caplog.text+ # Fixed text only: the error's own message never reaches the log.+ assert "chmod refused" not in loguru_caplog.text++ def test_mode_the_filesystem_does_not_keep_is_logged_with_remedy(+ self, tmp_path, monkeypatch, loguru_caplog+ ):+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))+ root = tmp_path / "research_outputs"+ root.mkdir()+ root.chmod(0o777)+ # A drvfs or CIFS mount: chmod succeeds and changes nothing.+ monkeypatch.setattr(Path, "chmod", lambda self, mode, **kwargs: None)++ with loguru_caplog.at_level("WARNING"):+ assert paths.restrict_research_outputs_directory() is False++ assert "did not keep mode 0700" in loguru_caplog.text+ assert "Docker named volume" in loguru_caplog.text+class TestGetCacheDirectory:"""Tests for get_cache_directory function."""diff --git a/tests/settings/golden_master_settings.json b/tests/settings/golden_master_settings.jsonindex 3db66dab5b..88f5baef7c 100644--- a/tests/settings/golden_master_settings.json+++ b/tests/settings/golden_master_settings.json@@ -2602,7 +2602,7 @@},"report.enable_file_backup": {"category": "report_parameters",- "description": "Also save reports as files on disk for external access (reports are always stored in the database regardless). Enable this if you need to access reports from outside the web interface.",+ "description": "Also save plaintext report files in research_outputs/ under the data directory for external access (reports remain in the database). On POSIX, that directory is set to mode 0700 and new or rewritten report files to 0600; on Windows, configure equivalent access controls. If another OS account owns the directory or its filesystem does not keep mode 0700, as on Docker Desktop bind mounts of Windows folders, WSL drives such as /mnt/c, CIFS/SMB shares and exFAT, no files are written and the research page shows a warning. Keep the data directory on a Linux filesystem path or a Docker named volume, or turn this off.","editable": true,"max_value": null,"min_value": null,diff --git a/tests/storage/test_report_backup_permissions.py b/tests/storage/test_report_backup_permissions.pynew file mode 100644index 0000000000..9a815cdcb3--- /dev/null+++ b/tests/storage/test_report_backup_permissions.py@@ -0,0 +1,226 @@+"""The optional plaintext report backup must not be exposed by a wide umask."""++import errno+import os+import stat+from pathlib import Path+from unittest.mock import MagicMock, patch++import pytest++from local_deep_research.storage.database_with_file_backup import (+ DatabaseWithFileBackupStorage,+)+from local_deep_research.storage.file import FileReportStorage+from local_deep_research.security.file_write_verifier import write_file_verified++[email protected](os.name != "posix", reason="POSIX mode bits")+def test_backup_keeps_flat_paths_but_restricts_directory_and_files(+ tmp_path, monkeypatch+):+ root = tmp_path / "research_outputs"+ root.mkdir()+ root.chmod(0o755)+ existing = root / "old.md"+ existing.write_text(+ "old content that is longer than the replacement", encoding="utf-8"+ )+ existing.chmod(0o644)+ monkeypatch.setattr(+ "local_deep_research.config.search_config.get_setting_from_snapshot",+ lambda *args, **kwargs: True,+ )++ old_umask = os.umask(0o022)+ try:+ storage = FileReportStorage(base_dir=root)+ assert stat.S_IMODE(root.stat().st_mode) == 0o700+ assert storage.save_report("old", "new content", {"query": "private"})+ assert storage.save_report("new", "another report")+ finally:+ os.umask(old_umask)++ assert existing.read_text(encoding="utf-8") == "new content"+ for filename in ("old.md", "old_metadata.json", "new.md"):+ assert stat.S_IMODE((root / filename).stat().st_mode) == 0o600+++def test_backup_permission_failure_does_not_prevent_database_save(mock_session):+ with patch(+ "local_deep_research.storage.database_with_file_backup.FileReportStorage",+ side_effect=PermissionError("private mode unavailable"),+ ):+ storage = DatabaseWithFileBackupStorage(+ mock_session, enable_file_storage=True+ )+ storage.db_storage = MagicMock()+ storage.db_storage.save_report.return_value = True++ assert storage.file_storage is None+ assert storage.save_report("research", "saved in encrypted database")+ storage.db_storage.save_report.assert_called_once()++[email protected](os.name != "posix", reason="POSIX mode bits")+def test_chmod_that_does_not_stick_refuses_directory_and_degrades_to_database(+ tmp_path, monkeypatch, mock_session, loguru_caplog+):+ root = tmp_path / "research_outputs"+ root.mkdir()+ root.chmod(0o755)+ monkeypatch.setattr(Path, "chmod", lambda self, mode: None)++ with pytest.raises(PermissionError, match="not private"):+ FileReportStorage(base_dir=root)++ monkeypatch.setattr(+ "local_deep_research.storage.file.get_research_outputs_directory",+ lambda: root,+ )+ with loguru_caplog.at_level("WARNING"):+ storage = DatabaseWithFileBackupStorage(+ mock_session, enable_file_storage=True+ )+ # The log names the cause and the remedy, not just "unavailable".+ assert "did not keep mode 0700" in loguru_caplog.text+ assert "Docker named volume" in loguru_caplog.text+ storage.db_storage = MagicMock()+ storage.db_storage.save_report.return_value = True++ assert storage.file_storage is None+ assert storage.save_report("research", "saved in encrypted database")+ storage.db_storage.save_report.assert_called_once()+++def _refuse_chmod(message):+ def refuse(self, mode, **kwargs):+ raise PermissionError(errno.EPERM, message)++ return refuse++[email protected](os.name != "posix", reason="POSIX ownership")+ "chmod_refused", [False, True], ids=["chmod-allowed", "chmod-refused"]+)+def test_directory_owned_by_another_account_is_refused(+ tmp_path, monkeypatch, mock_session, chmod_refused+):+ """A 0700 directory owned by another account stays readable by it.++ Root can chmod such a directory and an unprivileged process cannot;+ either way the refusal names the owner rather than a raw chmod error.+ """+ root = tmp_path / "research_outputs"+ root.mkdir()+ root.chmod(0o755)+ other_uid = root.stat().st_uid + 1+ monkeypatch.setattr(os, "geteuid", lambda: other_uid)+ if chmod_refused:+ monkeypatch.setattr(+ Path, "chmod", _refuse_chmod("Operation not permitted")+ )++ with pytest.raises(PermissionError, match="owned by uid"):+ FileReportStorage(base_dir=root)++ monkeypatch.setattr(+ "local_deep_research.storage.file.get_research_outputs_directory",+ lambda: root,+ )+ storage = DatabaseWithFileBackupStorage(+ mock_session, enable_file_storage=True+ )+ assert storage.file_storage is None++[email protected](os.name != "posix", reason="POSIX mode bits")+def test_chmod_error_on_own_directory_keeps_its_cause(tmp_path, monkeypatch):+ root = tmp_path / "research_outputs"+ root.mkdir()+ monkeypatch.setattr(Path, "chmod", _refuse_chmod("immutable directory"))++ with pytest.raises(PermissionError, match="immutable directory"):+ FileReportStorage(base_dir=root)++[email protected](os.name != "posix", reason="POSIX mode bits")+def test_mode_failure_preserves_existing_report_contents(tmp_path, monkeypatch):+ report = tmp_path / "existing.md"+ report.write_text("old report", encoding="utf-8")+ monkeypatch.setattr(+ "local_deep_research.config.search_config.get_setting_from_snapshot",+ lambda *args, **kwargs: True,+ )++ with patch(+ "local_deep_research.security.file_write_verifier.os.fchmod",+ side_effect=PermissionError("cannot restrict file mode"),+ ):+ with pytest.raises(PermissionError):+ write_file_verified(+ report,+ "replacement",+ "report.enable_file_backup",+ permissions=0o600,+ )++ assert report.read_text(encoding="utf-8") == "old report"++[email protected](os.name != "posix", reason="POSIX mode bits")+def test_mode_failure_closes_the_descriptor(tmp_path, monkeypatch):+ report = tmp_path / "existing.md"+ report.write_text("old report", encoding="utf-8")+ monkeypatch.setattr(+ "local_deep_research.config.search_config.get_setting_from_snapshot",+ lambda *args, **kwargs: True,+ )+ given, closed = [], []+ real_close = os.close++ def failing_fchmod(fd, mode):+ given.append(fd)+ raise PermissionError("cannot restrict file mode")++ def recording_close(fd):+ closed.append(fd)+ real_close(fd)++ monkeypatch.setattr(os, "fchmod", failing_fchmod)+ monkeypatch.setattr(os, "close", recording_close)+ with pytest.raises(PermissionError):+ write_file_verified(+ report,+ "replacement",+ "report.enable_file_backup",+ permissions=0o600,+ )++ assert len(given) == 1+ assert given[0] in closed, "the descriptor of a failed write was leaked"+++ not hasattr(os, "O_NOFOLLOW"), reason="O_NOFOLLOW unavailable"+)+def test_private_write_refuses_a_leaf_symlink(tmp_path, monkeypatch):+ """permissions= keeps O_NOFOLLOW, so fchmod never reaches a link target."""+ monkeypatch.setattr(+ "local_deep_research.config.search_config.get_setting_from_snapshot",+ lambda *args, **kwargs: True,+ )+ victim = tmp_path / "victim.txt"+ victim.write_text("victim", encoding="utf-8")+ victim.chmod(0o644)+ link = tmp_path / "report.md"+ link.symlink_to(victim)++ with pytest.raises(OSError):+ write_file_verified(+ link, "replacement", "report.enable_file_backup", permissions=0o600+ )++ assert victim.read_text(encoding="utf-8") == "victim"+ assert stat.S_IMODE(victim.stat().st_mode) == 0o644diff --git a/tests/web/test_lifespan_startup_shutdown.py b/tests/web/test_lifespan_startup_shutdown.pyindex 3141a9ad87..e1f05aa76c 100644--- a/tests/web/test_lifespan_startup_shutdown.py+++ b/tests/web/test_lifespan_startup_shutdown.py@@ -107,9 +107,11 @@ argv: <mode> <output json path>"""import json+import osimport sysimport threadingfrom pathlib import Path+from stat import S_IMODEMODE = sys.argv[1]OUT = Path(sys.argv[2])@@ -251,7 +253,25 @@ def _css_stat():return {"mtime_ns": stat.st_mtime_ns, "size": stat.st_size}-result = {"mode": MODE, "css_before": _css_stat()}+# An upgraded install: research_outputs/ already holds a report at 0755.+# Loosened here, after every import, so only the lifespan startup itself can+# make it private again.+outputs_dir = Path(os.environ["LDR_DATA_DIR"]) / "research_outputs"+outputs_dir.mkdir(exist_ok=True)+legacy_report = outputs_dir / "legacy.md"+legacy_report.write_text("old report", encoding="utf-8")+outputs_dir.chmod(0o755)+++def _outputs_mode():+ return S_IMODE(outputs_dir.stat().st_mode)+++result = {+ "mode": MODE,+ "css_before": _css_stat(),+ "outputs_mode_before": _outputs_mode(),+}result["threads_before"] = snapshot()@@ -265,6 +285,10 @@ with TestClient(fastapi_app.app) as client:drain = log_utils._queue_processor_threadresult["log_queue_alive_during"] = drain is not None and drain.is_alive()result["css_after_startup"] = _css_stat()+ result["outputs_mode_during"] = _outputs_mode()+ result["legacy_report_after_startup"] = legacy_report.read_text(+ encoding="utf-8"+ )# Immediately after __exit__, then again after giving anything still alive a# bounded join. The join absorbs the exit race of a thread that WAS signalled@@ -446,6 +470,26 @@ def test_startup_regenerates_themes_css(clean_boot):)[email protected](os.name != "posix", reason="POSIX mode bits")+def test_startup_makes_an_existing_research_outputs_directory_private(+ clean_boot,+):+ """An upgraded install's 0755 research_outputs/ is 0700 once serving.++ The probe loosens the directory after importing the app, so neither an+ import nor an earlier lookup can be what restricted it.+ """+ assert clean_boot["outputs_mode_before"] == 0o755, (+ "premise broken: the probe did not start from a 0755 directory"+ )+ assert clean_boot["outputs_mode_during"] == 0o700, (+ "startup left research_outputs/ at "+ f"{clean_boot['outputs_mode_during']:04o}; plaintext report backups "+ "saved by an older release stay reachable by other OS accounts"+ )+ assert clean_boot["legacy_report_after_startup"] == "old report"++# ---------------------------------------------------------------------------# 2. Shutdown released what startup acquired.# ---------------------------------------------------------------------------diff --git a/tests/web/test_startup_ordering_contracts.py b/tests/web/test_startup_ordering_contracts.pyindex dbafedb99c..8a7d5b4e4b 100644--- a/tests/web/test_startup_ordering_contracts.py+++ b/tests/web/test_startup_ordering_contracts.py@@ -585,6 +585,12 @@ _GUARDED_STARTUP_STEPS = {"warn_if_threadpool_exceeds_db_pool": "AnyIO worker pool sizing","theme_registry.get_combined_css": "themes.css regeneration","warn_if_weak_kdf_with_existing_databases": "weak-SQLCipher-KDF check",+ # ADDED BY #7169, deliberately. Sets research_outputs/ (optional plaintext+ # report backups) to 0700. Degraded mode is "an older install's directory+ # keeps its old mode", and FileReportStorage independently refuses a+ # directory that is not private, so no new backup lands there. Not worth+ # refusing to boot over; the guard uses logger.exception.+ "restrict_research_outputs_directory": "research_outputs/ privacy","get_background_job_scheduler": "news scheduler","start_connection_cleanup_scheduler": "connection-cleanup scheduler",# MOVED HERE FROM _UNGUARDED_STARTUP_STEPS BY #5982, deliberately.diff --git a/tests/web/warning_checks/test_orchestrator.py b/tests/web/warning_checks/test_orchestrator.pyindex 312e30a7e1..73a9123b98 100644--- a/tests/web/warning_checks/test_orchestrator.py+++ b/tests/web/warning_checks/test_orchestrator.py@@ -12,8 +12,14 @@ these tests call it directly with a ``username=`` kwarg instead of patchinga fake session."""+import errno+import os+import stat+from pathlib import Pathfrom unittest.mock import Mock, patch+import pytest+def _make_settings_manager(overrides=None):"""Build a mock SettingsManager with sensible defaults covering every@@ -913,3 +919,93 @@ class TestCalculateWarningsReadsItsSettings:"app.warnings.dismiss_legacy_config",):assert key in called_keys, f"{key} is never read"+++class TestReportBackupUnavailableBanner:+ """Report file backups that are on but refused get a visible banner."""++ @staticmethod+ def _warnings(tmp_path, monkeypatch, enabled):+ from local_deep_research.web.warning_checks import calculate_warnings++ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))+ mgr = _make_settings_manager({"report.enable_file_backup": enabled})+ with _patch_orchestrator(mgr):+ with patch(+ "local_deep_research.config.paths.get_user_backup_directory",+ return_value=tmp_path,+ ):+ return calculate_warnings(username="test")++ @staticmethod+ def _banners(warnings):+ return [w for w in warnings if w["type"] == "report_backup_unavailable"]++ @pytest.mark.skipif(os.name != "posix", reason="POSIX ownership and modes")+ @pytest.mark.parametrize(+ "breakage, cause",+ [+ ("mode_not_kept", "does not keep owner-only permissions"),+ ("foreign_owner", "another OS account owns"),+ ("chmod_error", "could not make its research_outputs folder"),+ ],+ )+ def test_refused_backup_directory_shows_cause_and_remedy(+ self, tmp_path, monkeypatch, breakage, cause+ ):+ root = tmp_path / "research_outputs"+ root.mkdir()+ root.chmod(0o755)+ if breakage == "mode_not_kept":+ # A drvfs or CIFS mount: chmod succeeds and changes nothing.+ monkeypatch.setattr(+ Path, "chmod", lambda self, mode, **kwargs: None+ )+ elif breakage == "foreign_owner":+ other_uid = root.stat().st_uid + 1+ monkeypatch.setattr(os, "geteuid", lambda: other_uid)+ else:++ def read_only(self, mode, **kwargs):+ raise OSError(errno.EROFS, "Read-only file system")++ monkeypatch.setattr(Path, "chmod", read_only)++ banners = self._banners(+ self._warnings(tmp_path, monkeypatch, enabled=True)+ )++ assert len(banners) == 1, "refused report backups raised no banner"+ banner = banners[0]+ assert cause in banner["message"]+ assert "Docker named volume" in banner["message"]+ assert "turn off report file backup" in banner["message"]+ assert banner["dismissKey"] is None+ assert (+ banner["actionUrl"] == "/settings#setting-report-enable_file_backup"+ )++ @pytest.mark.skipif(os.name != "posix", reason="POSIX mode bits")+ def test_no_banner_for_a_directory_that_can_be_made_private(+ self, tmp_path, monkeypatch+ ):+ root = tmp_path / "research_outputs"+ root.mkdir()+ root.chmod(0o755)++ warnings = self._warnings(tmp_path, monkeypatch, enabled=True)++ assert self._banners(warnings) == []++ @pytest.mark.skipif(os.name != "posix", reason="POSIX mode bits")+ def test_backups_off_means_no_banner_and_no_directory_change(+ self, tmp_path, monkeypatch+ ):+ root = tmp_path / "research_outputs"+ root.mkdir()+ root.chmod(0o755)++ warnings = self._warnings(tmp_path, monkeypatch, enabled=False)++ assert self._banners(warnings) == []+ assert stat.S_IMODE(root.stat().st_mode) == 0o755