~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
harden: contain benchmark output to the data dir and re-check backup purge entries (#5769)
LearningCircuit9bb18394026f
diff --git a/changelog.d/+benchmark-output-containment.security.md b/changelog.d/+benchmark-output-containment.security.mdnew file mode 100644index 0000000000..52203ffa58--- /dev/null+++ b/changelog.d/+benchmark-output-containment.security.md@@ -0,0 +1 @@+Benchmark files written by `run_benchmark`, and the reports of both `compare_configurations` helpers, now stay below the data directory. This guards CLI and Python API callers only: no code reads the `benchmark.output_dir` setting, and web benchmark runs, which keep their results in the database, never call these functions, so #7417's exposure was not reachable from the web UI. `run_benchmark`, the `run_*_benchmark` and `evaluate_*` helpers that call it, and the `compare_configurations` helpers in `local_deep_research.api.benchmark_functions` and `local_deep_research.benchmarks.benchmark_functions` resolve their output directory there: a relative path is taken relative to the data directory instead of the working directory, and a path outside it, or the data directory itself, is replaced, with a logged warning, by `benchmark_results` or `benchmark_comparisons` below it. If that default is not usable either, for example because it is a symlink to another disk, the call raises `BenchmarkOutputDirectoryError` before writing anything. A comparison resolves its root once, so the per-configuration results follow its report, unless a configuration name leads out of that root (for example an absolute name or `..` components); those results then land elsewhere below the data directory. `evaluate_benchmark_quality` keeps its temporary directory inside `benchmark_results`. The benchmark CLI defaults were already below the data directory. Other benchmark writers still use the directory their caller gives: the parameter optimizer (`OptunaOptimizer` and the `optimize_*` helpers), the benchmark evaluators (the SimpleQA evaluator's default path writes its files there, and both evaluators create their subdirectory there), `local_deep_research.benchmarks.comparison.compare_configurations`, and writers that take a file path (`grade_results`, `generate_report`, `Benchmark_results`). When such a caller passes `run_benchmark` a relative directory or one outside the data directory, as the BrowseComp evaluator, the SimpleQA evaluator with `use_direct_dataset=False` and several scripts under `examples/` do, those benchmark files land below the data directory while the caller's own files and the directory it prints stay where they were; the paths `run_benchmark` returns are the real ones. The backup purge after a password change now also re-checks each listed backup and skips any that is a symlink or resolves outside the user's backup directory. These checks are snapshots and do not protect against concurrent filesystem changes.diff --git a/docs/cli-tools.md b/docs/cli-tools.mdindex 297e01d35c..a65e6ad9e3 100644--- a/docs/cli-tools.md+++ b/docs/cli-tools.md@@ -42,11 +42,13 @@ python -m local_deep_research.benchmarks.cli.benchmark_commands simpleqa [option| `--search-provider` | (default) | LLM provider || `--eval-model` | (default) | Model for answer evaluation || `--eval-provider` | (default) | Provider for evaluation |-| `--output-dir` | ~/.local-deep-research/benchmark_results | Results directory |+| `--output-dir` | data directory/benchmark_results | Results directory, kept below the data directory (see **Output location**) || `--human-eval` | false | Use human evaluation || `--no-eval` | false | Skip evaluation phase || `--custom-dataset` | - | Path to custom dataset |+**Output location:** benchmark result, evaluation and report files are written below the data directory (`LDR_DATA_DIR`, or the platform default). A relative `--output-dir` is taken relative to the data directory, not the current directory. A path outside the data directory, or the data directory itself, is replaced by `benchmark_results` there, with a warning in the log. If that default is not usable either, for example because it is a symlink to another disk, the command stops with an error before running. The report and results paths it prints are the real locations.+**Example:**```bash@@ -91,7 +93,7 @@ python -m local_deep_research.benchmarks.cli.benchmark_commands compare [options|--------|---------|-------------|| `--dataset` | simpleqa | Dataset to use (simpleqa, browsecomp) || `--examples` | 20 | Examples per configuration |-| `--output-dir` | ~/.local-deep-research/benchmark_results/comparison | Results directory |+| `--output-dir` | data directory/benchmark_results/comparison | Results directory, kept below the data directory (see **Output location** under `simpleqa`; the default replacement here is `benchmark_comparisons`) |**Example:**diff --git a/examples/benchmarks/README.md b/examples/benchmarks/README.mdindex 61099e0c4a..8c4992dcef 100644--- a/examples/benchmarks/README.md+++ b/examples/benchmarks/README.md@@ -17,7 +17,7 @@ Options:- `--iterations`: Number of search iterations (default: 3)- `--questions`: Questions per iteration (default: 3)- `--search-tool`: Search tool to use (default: "searxng")-- `--output-dir`: Directory to save results (default: "benchmark_results")+- `--output-dir`: Directory to save results (default: "benchmark_results", relative to the LDR data directory; see [Where results are written](#where-results-are-written))- `--no-eval`: Skip evaluation- `--human-eval`: Use human evaluation- `--eval-model`: Model to use for evaluation@@ -36,7 +36,7 @@ Options:- `--iterations`: Number of search iterations (default: 1)- `--questions`: Questions per iteration (default: 1)- `--search-tool`: Search tool to use (default: "searxng")-- `--output-dir`: Directory to save results (default: "browsecomp_results")+- `--output-dir`: Directory to save results (default: "browsecomp_results", relative to the current directory; this script writes its files itself)See `browsecomp_benchmark_readme.md` for more information on how BrowseComp works.@@ -52,9 +52,50 @@ python run_simpleqa.pypython run_browsecomp.py --iterations 3 --questions 3```+## Where results are written++Scripts that hand an output directory to the library's benchmark runner+(`evaluate_simpleqa`, `evaluate_browsecomp`, `run_simpleqa_benchmark`,+`run_browsecomp_benchmark` or `compare_configurations`) get those benchmark+files below the LDR data directory (`LDR_DATA_DIR`, or the platform+default). A relative directory is placed relative to the data directory,+not the current directory. A directory outside the data directory, such as+the project-root `benchmark_results/<timestamp>` paths some scripts build,+is replaced by `benchmark_results` there, with a warning in the log. If that+default is not usable either, for example because it is a symlink to another+disk, the run stops with an error before it starts.++This applies to `run_simpleqa.py`, `run_gemini_benchmark.py` and+`gemini/run_gemini_benchmark_fixed.py`, which call `evaluate_simpleqa` and+`evaluate_browsecomp`, and to `../run_benchmark.py`,+`../optimization/run_gemini_benchmark.py` and+`../optimization/run_parallel_benchmark.py`. The report and results paths+these functions return are the real locations. A script's own summary JSON+and its "Results saved to" line still use the directory the script created,+which may then hold only that summary.++`claude_grading/benchmark.py`, `scripts/run_benchmark_with_claude_grading.py`+and `scripts/run_focused_benchmark_fixed.py` call the benchmark evaluators+directly, on their own and through `CompositeBenchmarkEvaluator`, so their+output is split. The SimpleQA evaluator's default path writes its results,+evaluation and report files itself, into a `simpleqa` subdirectory of the+directory it is given, so they stay in the script's own directory tree with+its summary JSON (for example+`./benchmark_results/claude_grading_<timestamp>/simpleqa/simpleqa`). The+BrowseComp evaluator creates its `browsecomp` subdirectory there too, but+`run_browsecomp_benchmark` writes its benchmark files below the data+directory, so that subdirectory stays empty. Parameter optimizer runs+(`../optimization/`) split the same way and keep the optimizer's own study+files in the directory they are given.++`run_browsecomp.py`, `run_resumable_parallel_benchmark.py`,+`browsecomp/run_browsecomp_fixed_v2.py` and `scripts/run_grader_only.py`+call `quick_summary`, `grade_results` or `generate_report` themselves and+write every file to the paths they are given.+## Evaluating Results-Results are saved in the specified output directories and include:+Results are saved in the output directories described above and include:- Raw results (JSONL format)- Evaluation results (JSONL format)- Summary reports (Markdown format)diff --git a/src/local_deep_research/api/benchmark_functions.py b/src/local_deep_research/api/benchmark_functions.pyindex 573a3f04fc..ad9fb5509a 100644--- a/src/local_deep_research/api/benchmark_functions.py+++ b/src/local_deep_research/api/benchmark_functions.py@@ -16,6 +16,7 @@ from ..benchmarks import (run_simpleqa_benchmark,run_xbench_deepsearch_benchmark,)+from ..benchmarks.runners import resolve_output_dirdef evaluate_simpleqa(@@ -224,7 +225,11 @@ def compare_configurations(dataset_type: Type of dataset to usenum_examples: Number of examples to evaluateconfigurations: List of search configurations to compare- output_dir: Directory to save results+ output_dir: Directory to save results, resolved below the data+ directory like ``run_benchmark``'s: a value outside it (or the+ data directory itself) is replaced by ``benchmark_comparisons``+ there, and ``BenchmarkOutputDirectoryError`` is raised if that+ is not usable either.Returns:Dictionary with comparison results@@ -252,7 +257,16 @@ def compare_configurations(},]- # Create output directory+ # Resolve once, so the per-configuration results and the comparison+ # report share one root below the data directory, also when the+ # requested root is replaced. The exception is a configuration name+ # that leads out of that root (for example an absolute name or ".."+ # components): run_benchmark then resolves that subdirectory on its+ # own, so those results land elsewhere below the data directory (in+ # benchmark_results when the path leaves the data directory).+ output_dir = str(+ resolve_output_dir(output_dir, fallback="benchmark_comparisons")+ )import osos.makedirs(output_dir, exist_ok=True)diff --git a/src/local_deep_research/benchmarks/benchmark_functions.py b/src/local_deep_research/benchmarks/benchmark_functions.pyindex e879b1f72b..b2a48f895c 100644--- a/src/local_deep_research/benchmarks/benchmark_functions.py+++ b/src/local_deep_research/benchmarks/benchmark_functions.py@@ -20,6 +20,7 @@ from ..benchmarks import (run_simpleqa_benchmark,run_xbench_deepsearch_benchmark,)+from .runners import resolve_output_dirdef evaluate_simpleqa(@@ -327,7 +328,11 @@ def compare_configurations(dataset_type: Type of dataset to usenum_examples: Number of examples to evaluateconfigurations: List of search configurations to compare- output_dir: Directory to save results+ output_dir: Directory to save results, resolved below the data+ directory like ``run_benchmark``'s: a value outside it (or the+ data directory itself) is replaced by ``benchmark_comparisons``+ there, and ``BenchmarkOutputDirectoryError`` is raised if that+ is not usable either.Returns:Dictionary with comparison results@@ -355,8 +360,16 @@ def compare_configurations(},]- # Create output directory-+ # Resolve once, so the per-configuration results and the comparison+ # report share one root below the data directory, also when the+ # requested root is replaced. The exception is a configuration name+ # that leads out of that root (for example an absolute name or ".."+ # components): run_benchmark then resolves that subdirectory on its+ # own, so those results land elsewhere below the data directory (in+ # benchmark_results when the path leaves the data directory).+ output_dir = str(+ resolve_output_dir(output_dir, fallback="benchmark_comparisons")+ )# nosemgrep: semgrep.rules.path-traversal-risk, reason: CLI/Python benchmark API intentionally accepts an output directory; the web runner does not call this functionPath(output_dir).mkdir(parents=True, exist_ok=True)diff --git a/src/local_deep_research/benchmarks/comparison/evaluator.py b/src/local_deep_research/benchmarks/comparison/evaluator.pyindex b56f442119..fb82d3bf9f 100644--- a/src/local_deep_research/benchmarks/comparison/evaluator.py+++ b/src/local_deep_research/benchmarks/comparison/evaluator.py@@ -75,7 +75,8 @@ def compare_configurations(Args:query: Research query to use for evaluationconfigurations: List of parameter configurations to compare- output_dir: Directory to save comparison results+ output_dir: Directory to save comparison results, used as given+ (not resolved below the data directory)model_name: Name of the LLM model to useprovider: LLM providersearch_tool: Search engine to usediff --git a/src/local_deep_research/benchmarks/evaluators/base.py b/src/local_deep_research/benchmarks/evaluators/base.pyindex eabb20715b..338fb3ed1c 100644--- a/src/local_deep_research/benchmarks/evaluators/base.py+++ b/src/local_deep_research/benchmarks/evaluators/base.py@@ -60,6 +60,15 @@ class BaseBenchmarkEvaluator(ABC):"""Create a benchmark-specific subdirectory for output.+ The subdirectory is created where the caller says (a relative+ path is relative to the working directory); it is not resolved+ below the data directory. Files that ``run_benchmark`` writes for+ it (the BrowseComp evaluator and the SimpleQA evaluator with+ ``use_direct_dataset=False``) follow ``runners.resolve_output_dir``+ instead, so unless ``output_dir`` is an absolute path below the+ data directory they land elsewhere below the data directory and+ this subdirectory stays empty.+Args:output_dir: Parent directory for outputdiff --git a/src/local_deep_research/benchmarks/evaluators/simpleqa.py b/src/local_deep_research/benchmarks/evaluators/simpleqa.pyindex 1e196ff598..ccfecdde66 100644--- a/src/local_deep_research/benchmarks/evaluators/simpleqa.py+++ b/src/local_deep_research/benchmarks/evaluators/simpleqa.py@@ -112,7 +112,10 @@ class SimpleQAEvaluator(BaseBenchmarkEvaluator):Args:system_config: Search and LLM configuration parametersnum_examples: Number of benchmark examples to run- output_dir: Directory to save evaluation results+ output_dir: Directory to save evaluation results. The files+ are written there directly; unlike ``run_benchmark``, this+ path does not resolve the directory below the data+ directory.Returns:Dictionary with benchmark resultsdiff --git a/src/local_deep_research/benchmarks/metrics/calculation.py b/src/local_deep_research/benchmarks/metrics/calculation.pyindex 83c1aeb18c..f102b359e7 100644--- a/src/local_deep_research/benchmarks/metrics/calculation.py+++ b/src/local_deep_research/benchmarks/metrics/calculation.py@@ -131,17 +131,27 @@ def evaluate_benchmark_quality(Args:system_config: Configuration parameters to evaluatenum_examples: Number of benchmark examples to use- output_dir: Directory to save results (temporary if None)+ output_dir: Directory to save results. If None, a temporary+ directory is created inside the default ``benchmark_results``+ output directory and removed afterwards.Returns:Dictionary with benchmark metrics"""- from ..runners import run_simpleqa_benchmark+ from ..runners import resolve_output_dir, run_simpleqa_benchmark- # Create temporary directory if not provided+ # Create a temporary directory if none was provided. It must be below+ # the data directory: run_benchmark replaces an output directory outside+ # it, so results for a system temp directory would land elsewhere and+ # outlive the cleanup below. Putting it inside run_benchmark's own+ # default root keeps both under the same rule.temp_dir = Noneif output_dir is None:- temp_dir = tempfile.mkdtemp(prefix="ldr_benchmark_")+ scratch_root = resolve_output_dir(None, fallback="benchmark_results")+ scratch_root.mkdir(parents=True, exist_ok=True)+ temp_dir = tempfile.mkdtemp(+ prefix="ldr_benchmark_", dir=str(scratch_root)+ )output_dir = temp_dirtry:diff --git a/src/local_deep_research/benchmarks/optimization/optuna_optimizer.py b/src/local_deep_research/benchmarks/optimization/optuna_optimizer.pyindex 88f55c4cdf..867051ba4e 100644--- a/src/local_deep_research/benchmarks/optimization/optuna_optimizer.py+++ b/src/local_deep_research/benchmarks/optimization/optuna_optimizer.py@@ -117,7 +117,10 @@ class OptunaOptimizer:Args:base_query: The research query to use for all experiments- output_dir: Directory to save optimization results+ output_dir: Directory to save optimization results. It is used+ as given, not resolved below the data directory; benchmark+ files that ``run_benchmark`` writes for a trial follow+ ``runners.resolve_output_dir`` instead.model_name: Name of the LLM model to useprovider: LLM providersearch_tool: Search engine to usediff --git a/src/local_deep_research/benchmarks/runners.py b/src/local_deep_research/benchmarks/runners.pyindex 5baaf2b8ae..73185151e6 100644--- a/src/local_deep_research/benchmarks/runners.py+++ b/src/local_deep_research/benchmarks/runners.py@@ -38,6 +38,80 @@ def format_query(question: str, dataset_type: str = "simpleqa") -> str:return question+class BenchmarkOutputDirectoryError(ValueError):+ """No benchmark output directory below the data directory is usable."""+++def _below_data_directory(value: str, data_dir: Path) -> Optional[Path]:+ """Resolve ``value`` and return it if it is strictly below ``data_dir``.++ Returns None for a value that resolves to ``data_dir`` itself, outside+ it, or cannot be resolved.+ """+ from ..security.path_validator import resolve_allow_missing++ try:+ path = Path(os.path.expandvars(str(value))).expanduser()+ if not path.is_absolute():+ path = data_dir / path+ resolved = resolve_allow_missing(path)+ except (OSError, RuntimeError, ValueError):+ return None+ if resolved != data_dir and resolved.is_relative_to(data_dir):+ return resolved+ return None+++def resolve_output_dir(output_dir: Optional[str], *, fallback: str) -> Path:+ """Return the benchmark output directory, resolved below the data dir.++ ``$VAR`` and ``~`` are expanded, and a relative value is taken relative+ to the data directory (``LDR_DATA_DIR`` or the platform default), not+ the working directory. ``None``, ``""`` and ``"."`` select ``fallback``.+ Any other value that resolves outside the data directory, to the data+ directory itself, or cannot be resolved, is replaced by ``fallback``+ (relative to the data directory) and a warning is logged.++ Raises:+ BenchmarkOutputDirectoryError: ``fallback`` is not usable either, for+ example because ``<data dir>/<fallback>`` is a symlink to+ another disk. Benchmark files are never written into the data+ directory root.+ ValueError, OSError: ``LDR_DATA_DIR`` is invalid or the data+ directory cannot be resolved.++ Symlinks are followed with ``resolve_allow_missing`` before the+ comparison, so a link that leaves the data directory, a symlink loop and+ a chain whose resolved form outgrows PATH_MAX are not accepted. The+ check is a snapshot: it does not protect against a concurrent change to+ the filesystem.+ """+ # Lazy imports: path_validator imports config.paths at module level.+ from ..config.paths import get_data_directory+ from ..security.path_validator import resolve_allow_missing++ data_dir = resolve_allow_missing(get_data_directory())+ if output_dir is not None and Path(str(output_dir)) != Path("."):+ resolved = _below_data_directory(output_dir, data_dir)+ if resolved is not None:+ return resolved+ logger.warning(+ "Benchmark output directory is not below the data directory or "+ f"cannot be resolved; using the default {fallback!r} there"+ )+ resolved = _below_data_directory(fallback, data_dir)+ if resolved is not None:+ return resolved+ raise BenchmarkOutputDirectoryError(+ "No benchmark output directory below the data directory is usable: "+ f"neither the requested directory nor the default {fallback!r} "+ "resolves to a directory inside it. Choose an output directory "+ "inside the data directory, set LDR_DATA_DIR to a data directory "+ "that contains the location you want, or remove the symlink that "+ f"points {fallback!r} outside the data directory."+ )++def run_benchmark(dataset_type: str,dataset_path: Optional[str] = None,@@ -58,7 +132,12 @@ def run_benchmark(dataset_type: Type of dataset ("simpleqa" or "browsecomp")dataset_path: Optional custom dataset pathnum_examples: Number of examples to use- output_dir: Directory to save results+ output_dir: Directory to save results. It is resolved below the+ data directory by :func:`resolve_output_dir`: a relative value+ is relative to the data directory, and a value outside it (or+ the data directory itself) is replaced by ``benchmark_results``+ there. Raises ``BenchmarkOutputDirectoryError`` before any+ work when ``benchmark_results`` is not usable either.run_evaluation: Whether to evaluate resultsevaluation_config: Custom LLM config for evaluationsearch_config: Custom search parameters@@ -67,8 +146,13 @@ def run_benchmark(seed: Random seed for reproducibilityReturns:- Dictionary with benchmark results and metrics+ Dictionary with benchmark results and metrics. The returned file+ paths are inside the resolved output directory."""+ output_dir = str(+ resolve_output_dir(output_dir, fallback="benchmark_results")+ )+# Ensure output directory existsos.makedirs(output_dir, exist_ok=True)diff --git a/src/local_deep_research/database/backup/backup_service.py b/src/local_deep_research/database/backup/backup_service.pyindex 8814763e1e..45d3806007 100644--- a/src/local_deep_research/database/backup/backup_service.py+++ b/src/local_deep_research/database/backup/backup_service.py@@ -1113,8 +1113,21 @@ class BackupService:# purge cannot delete is checked again with the new one._get_backup_verifications(self.username).clear()- # Delete all existing backup files+ # Delete all existing backup files, including quarantined+ # damaged copies (they are still under the old key).+ # list_backups() reports only bare names from a filtered glob,+ # but re-check each joined path before removing it, so an entry+ # that is not a bare in-root name (a poisoned listing or a future+ # caller) cannot redirect the unlink outside backup_dir.+ # is_safe_glob_result also skips a symlink entry. The check is a+ # snapshot, not atomic with the unlink.for backup_path in self._managed_backup_paths():+ if not is_safe_glob_result(backup_path, self.backup_dir):+ logger.warning(+ f"Refusing to delete backup {backup_path.name}: it "+ "is a symlink or resolves outside the backup directory"+ )+ continuetry:self._remove_backup_files(backup_path)logger.debug(f"Purged old-key backup: {backup_path.name}")diff --git a/tests/api/test_benchmark_functions.py b/tests/api/test_benchmark_functions.pyindex 7e064ece6a..289defe0f9 100644--- a/tests/api/test_benchmark_functions.py+++ b/tests/api/test_benchmark_functions.py@@ -2,6 +2,20 @@from unittest.mock import patch+import pytest++[email protected](autouse=True)+def _pin_data_directory(tmp_path, monkeypatch):+ """Keep comparison output out of the real user data directory.++ compare_configurations resolves its output root inside LDR_DATA_DIR and+ replaces any other location with ``benchmark_comparisons`` there.+ Pinning the data directory to this test's tmp_path keeps that root per+ test.+ """+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))+class TestGetAvailableBenchmarks:def test_returns_list(self):diff --git a/tests/api/test_benchmark_functions_deep_coverage.py b/tests/api/test_benchmark_functions_deep_coverage.pyindex d8331614c8..4a9e224371 100644--- a/tests/api/test_benchmark_functions_deep_coverage.py+++ b/tests/api/test_benchmark_functions_deep_coverage.py@@ -10,10 +10,24 @@ Targets:from unittest.mock import patch+import pytest+MODULE = "local_deep_research.api.benchmark_functions"[email protected](autouse=True)+def _pin_data_directory(tmp_path, monkeypatch):+ """Keep comparison output out of the real user data directory.++ compare_configurations resolves its output root inside LDR_DATA_DIR and+ replaces any other location with ``benchmark_comparisons`` there.+ Pinning the data directory to this test's tmp_path keeps that root per+ test.+ """+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))++class TestEvaluateSimpleqa:@patch(f"{MODULE}.run_simpleqa_benchmark")def test_evaluation_config_with_model_only(self, mock_run):diff --git a/tests/benchmarks/test_benchmark_functions_extended.py b/tests/benchmarks/test_benchmark_functions_extended.pyindex 6738172927..b1c799bb62 100644--- a/tests/benchmarks/test_benchmark_functions_extended.py+++ b/tests/benchmarks/test_benchmark_functions_extended.py@@ -15,6 +15,18 @@ from local_deep_research.benchmarks.benchmark_functions import ()[email protected](autouse=True)+def _pin_data_directory(tmp_path, monkeypatch):+ """Keep comparison output out of the real user data directory.++ compare_configurations resolves its output root inside LDR_DATA_DIR and+ replaces any other location with ``benchmark_comparisons`` there.+ Pinning the data directory to this test's tmp_path keeps that root per+ test.+ """+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))++# ── get_available_benchmarks ───────────────────────────────────────diff --git a/tests/benchmarks/test_benchmark_functions_extra_coverage.py b/tests/benchmarks/test_benchmark_functions_extra_coverage.pyindex d878859498..801491cec3 100644--- a/tests/benchmarks/test_benchmark_functions_extra_coverage.py+++ b/tests/benchmarks/test_benchmark_functions_extra_coverage.py@@ -13,9 +13,23 @@ Targets the 36 missing lines not covered by test_benchmark_functions_coverage.pyfrom unittest.mock import patch+import pytest+MODULE = "local_deep_research.benchmarks.benchmark_functions"[email protected](autouse=True)+def _pin_data_directory(tmp_path, monkeypatch):+ """Keep comparison output out of the real user data directory.++ compare_configurations resolves its output root inside LDR_DATA_DIR and+ replaces any other location with ``benchmark_comparisons`` there.+ Pinning the data directory to this test's tmp_path keeps that root per+ test.+ """+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))++def _no_settings(key, *args, **kwargs):return Nonediff --git a/tests/benchmarks/test_calculation_coverage.py b/tests/benchmarks/test_calculation_coverage.pyindex dcfbfcfdcf..f85ad9c478 100644--- a/tests/benchmarks/test_calculation_coverage.py+++ b/tests/benchmarks/test_calculation_coverage.py@@ -255,11 +255,14 @@ class TestEvaluateBenchmarkQuality:assert result["quality_score"] == 0.75mock_run.assert_called_once()- def test_uses_temp_dir_when_output_dir_none(self):+ def test_uses_temp_dir_when_output_dir_none(self, tmp_path, monkeypatch):from local_deep_research.benchmarks.metrics.calculation import (evaluate_benchmark_quality,)+ # The temporary directory is created inside the default+ # benchmark_results directory below the data directory.+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))mock_results = {"metrics": {"accuracy": 0.5}}with patch("local_deep_research.benchmarks.runners.run_simpleqa_benchmark",@@ -343,12 +346,18 @@ class TestEvaluateBenchmarkQuality:assert search_config["model_name"] == "gpt-4"assert search_config["provider"] == "openai"- def test_temp_dir_cleaned_up_on_success(self):- """When output_dir is None, a temp dir is created and cleaned up."""+ def test_temp_dir_cleaned_up_on_success(self, tmp_path, monkeypatch):+ """When output_dir is None, a temp dir is created and cleaned up.++ It is created inside the default benchmark_results directory below+ the data directory, so run_benchmark keeps its results there and the+ cleanup removes them too.+ """from local_deep_research.benchmarks.metrics.calculation import (evaluate_benchmark_quality,)+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))created_dirs = []original_mkdtemp = __import__("tempfile").mkdtemp@@ -375,6 +384,10 @@ class TestEvaluateBenchmarkQuality:)assert len(created_dirs) == 1+ assert (+ Path(created_dirs[0]).parent+ == (tmp_path / "benchmark_results").resolve()+ )# The temp dir should have been cleaned upassert not Path(created_dirs[0]).exists()diff --git a/tests/benchmarks/test_runners.py b/tests/benchmarks/test_runners.pyindex 31611cd87b..dff96e933c 100644--- a/tests/benchmarks/test_runners.py+++ b/tests/benchmarks/test_runners.py@@ -12,6 +12,19 @@ from unittest.mock import Mock, patchfrom pathlib import Pathimport tempfile+import pytest++[email protected](autouse=True)+def _pin_data_directory(tmp_path, monkeypatch):+ """Keep benchmark output out of the real user data directory.++ run_benchmark resolves its output directory inside LDR_DATA_DIR and replaces+ any other location with a default directory there. Pinning the data+ directory to this test's tmp_path keeps that default per test.+ """+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))+class TestFormatQuery:"""Tests for the format_query function."""@@ -61,11 +74,15 @@ class TestFormatQuery:class TestRunBenchmark:"""Tests for the run_benchmark function."""- def test_run_benchmark_creates_output_dir(self):+ def test_run_benchmark_creates_output_dir(self, monkeypatch):"""run_benchmark creates output directory."""from local_deep_research.benchmarks.runners import run_benchmarkwith tempfile.TemporaryDirectory() as tmpdir:+ # run_benchmark output_dir is now confined to the LDR data directory,+ # so point the data dir at tmpdir to keep this out-of-tree output+ # location honoured rather than clamped to the default.+ monkeypatch.setenv("LDR_DATA_DIR", tmpdir)output_dir = Path(tmpdir) / "new_dir"# Mock the dataset loading and searchdiff --git a/tests/benchmarks/test_runners_coverage.py b/tests/benchmarks/test_runners_coverage.pyindex 8816b1befc..f4e07c105a 100644--- a/tests/benchmarks/test_runners_coverage.py+++ b/tests/benchmarks/test_runners_coverage.py@@ -12,10 +12,23 @@ import tempfilefrom pathlib import Pathfrom unittest.mock import Mock, patch+import pytest+MODULE = "local_deep_research.benchmarks.runners"[email protected](autouse=True)+def _pin_data_directory(tmp_path, monkeypatch):+ """Keep benchmark output out of the real user data directory.++ run_benchmark resolves its output directory inside LDR_DATA_DIR and replaces+ any other location with a default directory there. Pinning the data+ directory to this test's tmp_path keeps that default per test.+ """+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))++# ---------------------------------------------------------------------------# Helpers# ---------------------------------------------------------------------------@@ -923,7 +936,9 @@ class TestOutputFileCleanup:@patch(f"{MODULE}.DatasetRegistry")@patch(f"{MODULE}.time")- def test_existing_files_are_removed(self, mock_time, mock_registry):+ def test_existing_files_are_removed(+ self, mock_time, mock_registry, monkeypatch+ ):from local_deep_research.benchmarks.runners import run_benchmarkmock_time.strftime.return_value = "20260101_000000"@@ -934,19 +949,23 @@ class TestOutputFileCleanup:mock_registry.create_dataset.return_value = mock_datasetwith tempfile.TemporaryDirectory() as tmpdir:+ # run_benchmark output_dir is now confined below the LDR data+ # directory (never its root); point the data dir at tmpdir so+ # cleanup operates on a directory inside it.+ monkeypatch.setenv("LDR_DATA_DIR", tmpdir)+ out = Path(tmpdir) / "results"+ out.mkdir()for suffix in ["_results.jsonl", "_evaluation.jsonl", "_report.md"]:- p = Path(tmpdir) / f"simpleqa_20260101_000000{suffix}"+ p = out / f"simpleqa_20260101_000000{suffix}"p.write_text("old content")run_benchmark(dataset_type="simpleqa",- output_dir=tmpdir,+ output_dir=str(out),run_evaluation=False,)- results_file = (- Path(tmpdir) / "simpleqa_20260101_000000_results.jsonl"- )+ results_file = out / "simpleqa_20260101_000000_results.jsonl"assert not results_file.exists() or results_file.read_text() == ""diff --git a/tests/security/test_benchmark_and_temp_lifecycle.py b/tests/security/test_benchmark_and_temp_lifecycle.pyindex 0f03ce8952..436c7ffe52 100644--- a/tests/security/test_benchmark_and_temp_lifecycle.py+++ b/tests/security/test_benchmark_and_temp_lifecycle.py@@ -109,6 +109,11 @@ def test_runner_cleanup_only_unlinks_its_timestamped_outputs():from local_deep_research.benchmarks import runners+ # run_benchmark keeps its output directory below the data directory and+ # never writes into the data directory root itself.+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))+ out = tmp_path / "benchmark-output"+ out.mkdir()timestamp = "20000101_000000"monkeypatch.setattr(runners,@@ -121,19 +126,17 @@ def test_runner_cleanup_only_unlinks_its_timestamped_outputs(runners.DatasetRegistry, "create_dataset", lambda **_kwargs: dataset)for suffix in ("results.jsonl", "evaluation.jsonl", "report.md"):- (tmp_path / f"simpleqa_{timestamp}_{suffix}").write_text("stale")- sibling = tmp_path / f"simpleqa_{timestamp}_results-backup.jsonl"- nested = tmp_path / "nested" / f"simpleqa_{timestamp}_results.jsonl"+ (out / f"simpleqa_{timestamp}_{suffix}").write_text("stale")+ sibling = out / f"simpleqa_{timestamp}_results-backup.jsonl"+ nested = out / "nested" / f"simpleqa_{timestamp}_results.jsonl"sibling.write_text("canary")nested.parent.mkdir()nested.write_text("nested canary")- runners.run_benchmark(- "simpleqa", output_dir=str(tmp_path), run_evaluation=False- )+ runners.run_benchmark("simpleqa", output_dir=str(out), run_evaluation=False)for suffix in ("results.jsonl", "evaluation.jsonl", "report.md"):- assert not (tmp_path / f"simpleqa_{timestamp}_{suffix}").exists()+ assert not (out / f"simpleqa_{timestamp}_{suffix}").exists()assert sibling.read_text() == "canary"assert nested.read_text() == "nested canary"@@ -235,23 +238,29 @@ def test_reporting_writer_keeps_adjacent_canary(tmp_path, monkeypatch):def test_comparison_api_writer_keeps_adjacent_canary(tmp_path, monkeypatch):from local_deep_research.api import benchmark_functions+ # compare_configurations keeps its output root below the data directory+ # and never writes into the data directory root itself.+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))+ out = tmp_path / "comparison-output"+ out.mkdir()_allow_benchmark_writes(monkeypatch)monkeypatch.setattr(benchmark_functions,"run_benchmark",lambda **_kwargs: {"metrics": {}, "total_examples": 0},)- canary = tmp_path / "comparison_simpleqa_backup.md"- nested = tmp_path / "nested" / "comparison_simpleqa.md"+ canary = out / "comparison_simpleqa_backup.md"+ nested = out / "nested" / "comparison_simpleqa.md"canary.write_text("canary")nested.parent.mkdir()nested.write_text("nested canary")result = benchmark_functions.compare_configurations(- configurations=[{"name": "only"}], output_dir=str(tmp_path)+ configurations=[{"name": "only"}], output_dir=str(out))assert Path(result["report_path"]).is_file()+ assert Path(result["report_path"]).parent == out.resolve()assert canary.read_text() == "canary"assert nested.read_text() == "nested canary"@@ -261,12 +270,15 @@ def test_quality_evaluation_removes_real_mkdtemp_directory():from local_deep_research.benchmarks.metrics import calculation+ # The temporary directory is created inside the default+ # benchmark_results directory below the data directory.+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))temp_dir = tmp_path / "ldr_benchmark_created"temp_dir.mkdir()monkeypatch.setattr(calculation,"tempfile",- SimpleNamespace(mkdtemp=lambda prefix: str(temp_dir)),+ SimpleNamespace(mkdtemp=lambda prefix, dir: str(temp_dir)),)monkeypatch.setattr("local_deep_research.benchmarks.runners.run_simpleqa_benchmark",@@ -282,13 +294,16 @@ def test_quality_evaluation_rmtree_targets_only_mkdtemp_dir_on_error():from local_deep_research.benchmarks.metrics import calculation+ # The temporary directory is created inside the default+ # benchmark_results directory below the data directory.+ monkeypatch.setenv("LDR_DATA_DIR", str(tmp_path))temp_dir = tmp_path / "ldr_benchmark_created"temp_dir.mkdir()captured = []monkeypatch.setattr(calculation,"tempfile",- SimpleNamespace(mkdtemp=lambda prefix: str(temp_dir)),+ SimpleNamespace(mkdtemp=lambda prefix, dir: str(temp_dir)),)monkeypatch.setattr("local_deep_research.benchmarks.runners.run_simpleqa_benchmark",diff --git a/tests/security/test_benchmark_output_containment.py b/tests/security/test_benchmark_output_containment.pynew file mode 100644index 0000000000..888508762e--- /dev/null+++ b/tests/security/test_benchmark_output_containment.py@@ -0,0 +1,334 @@+"""Benchmark output directories stay below the LDR data directory.++``resolve_output_dir`` (benchmarks/runners.py) resolves the output root of+``run_benchmark``, of both ``compare_configurations`` helpers+(``api.benchmark_functions`` and ``benchmarks.benchmark_functions``) and of+``evaluate_benchmark_quality``'s temporary directory. A relative value is+taken relative to the data directory. A value outside it, the data directory+itself, or one that cannot be resolved is replaced by a default directory+below it; if that default is not usable either, the resolver raises rather+than writing into the data directory root.+``tests/security/test_settings_path_escape.py`` drives the real runner with+an outside directory (the #7417 tripwire).++Library storage paths follow a different, operator-managed policy and are+not covered here. Every hostile destination stays beneath ``tmp_path``.+"""++from pathlib import Path+from unittest.mock import patch++import pytest++from local_deep_research.benchmarks.runners import (+ BenchmarkOutputDirectoryError,+ resolve_output_dir,+)++from tests.security.path_max_fixtures import build_path_max_escape+++def data_dir(tmp_path, monkeypatch):+ path = tmp_path / "ldr-data"+ path.mkdir()+ monkeypatch.setenv("LDR_DATA_DIR", str(path))+ return path.resolve()+++def outside(tmp_path):+ path = tmp_path / "outside"+ path.mkdir()+ (path / "CANARY").write_text("do not alter")+ return path+++def _files_below(directory: Path) -> dict:+ return {+ path.relative_to(directory): path.read_bytes()+ for path in directory.rglob("*")+ if path.is_file()+ }+++# ---------------------------------------------------------------------------+# resolve_output_dir+# ---------------------------------------------------------------------------+++def test_relative_output_dir_resolves_under_the_data_directory(+ data_dir, tmp_path, monkeypatch+):+ workspace = tmp_path / "workspace"+ workspace.mkdir()+ monkeypatch.chdir(workspace)++ resolved = resolve_output_dir("nested/results", fallback="default")++ assert resolved == data_dir / "nested" / "results"+++def test_absolute_output_dir_inside_the_data_directory_is_kept(data_dir):+ requested = data_dir / "kept" / "results"++ assert resolve_output_dir(str(requested), fallback="default") == requested++[email protected]("kind", ["absolute", "relative", "expandvars"])+def test_outside_output_dir_is_replaced_by_the_fallback(+ data_dir, outside, monkeypatch, kind+):+ monkeypatch.setenv("LDR_TEST_OUTSIDE", str(outside))+ requested = {+ "absolute": str(outside / "results"),+ "relative": "../outside/results",+ "expandvars": "$LDR_TEST_OUTSIDE/results",+ }[kind]++ resolved = resolve_output_dir(requested, fallback="benchmark_results")++ assert resolved == data_dir / "benchmark_results"+++def test_symlink_out_of_the_data_directory_is_replaced(data_dir, outside):+ link = data_dir / "link"+ try:+ link.symlink_to(outside, target_is_directory=True)+ except (OSError, NotImplementedError):+ pytest.skip("Cannot create symlinks in this environment")++ resolved = resolve_output_dir("link/results", fallback="benchmark_results")++ assert resolved == data_dir / "benchmark_results"+++def test_path_max_symlink_chain_out_of_the_data_directory_is_replaced(+ data_dir, outside+):+ """A bare non-strict resolve() reports this escape as contained."""+ escape = build_path_max_escape(data_dir, outside)++ resolved = resolve_output_dir(+ f"{escape}/results", fallback="benchmark_results"+ )++ assert resolved == data_dir / "benchmark_results"++[email protected]("unset", [None, "", ".", "./"])+def test_unset_output_dir_selects_the_fallback(data_dir, unset):+ assert (+ resolve_output_dir(unset, fallback="benchmark_results")+ == data_dir / "benchmark_results"+ )++[email protected]("kind", ["absolute", "dotdot"])+def test_data_directory_root_is_replaced_by_the_fallback(data_dir, kind):+ requested = str(data_dir) if kind == "absolute" else "sub/.."++ assert (+ resolve_output_dir(requested, fallback="benchmark_results")+ == data_dir / "benchmark_results"+ )+++def test_malformed_output_dir_is_replaced_by_the_fallback(data_dir):+ assert (+ resolve_output_dir("bad\x00path", fallback="benchmark_results")+ == data_dir / "benchmark_results"+ )+++def test_unusable_fallback_raises_instead_of_using_the_data_root(+ data_dir, outside+):+ with pytest.raises(BenchmarkOutputDirectoryError, match="LDR_DATA_DIR"):+ resolve_output_dir(str(outside), fallback="bad\x00fallback")+++def test_symlinked_default_fails_closed(data_dir, outside):+ """An operator symlinked the default directory to another disk."""+ link = data_dir / "benchmark_results"+ try:+ link.symlink_to(outside, target_is_directory=True)+ except (OSError, NotImplementedError):+ pytest.skip("Cannot create symlinks in this environment")++ for requested in (str(link), "benchmark_results", None):+ with pytest.raises(+ BenchmarkOutputDirectoryError, match="remove the symlink"+ ):+ resolve_output_dir(requested, fallback="benchmark_results")+++def test_unavailable_home_expansion_uses_the_fallback(data_dir, monkeypatch):+ original = Path.expanduser++ def expanduser(path):+ if str(path) == "~unavailable":+ raise RuntimeError("home unavailable")+ return original(path)++ monkeypatch.setattr(Path, "expanduser", expanduser)++ assert (+ resolve_output_dir("~unavailable", fallback="benchmark_results")+ == data_dir / "benchmark_results"+ )+++def test_invalid_operator_data_root_is_not_hidden():+ with patch(+ "local_deep_research.config.paths.get_data_directory",+ side_effect=ValueError("invalid operator root"),+ ):+ with pytest.raises(ValueError, match="invalid operator root"):+ resolve_output_dir("results", fallback="benchmark_results")+++# ---------------------------------------------------------------------------+# Writers+# ---------------------------------------------------------------------------+++def test_run_benchmark_writes_a_relative_output_dir_under_the_data_directory(+ data_dir, tmp_path, monkeypatch+):+ from local_deep_research.benchmarks import runners++ workspace = tmp_path / "workspace"+ workspace.mkdir()+ monkeypatch.chdir(workspace)++ with (+ patch.object(+ runners.DatasetRegistry, "create_dataset", side_effect=RuntimeError+ ),+ patch.object(+ runners,+ "load_dataset",+ return_value=[{"id": "1", "problem": "q", "answer": "a"}],+ ),+ patch.object(+ runners,+ "quick_summary",+ return_value={"summary": "a", "sources": []},+ ),+ ):+ result = runners.run_benchmark(+ "simpleqa", output_dir="nested/results", run_evaluation=False+ )++ results = Path(result["results_path"])+ assert results.parent == data_dir / "nested" / "results"+ assert results.is_file()+ assert list(workspace.iterdir()) == []+++def test_run_benchmark_fails_closed_without_writing(+ data_dir, outside, monkeypatch+):+ """No result file lands in the data root or behind the symlink."""+ from local_deep_research.benchmarks import runners++ link = data_dir / "benchmark_results"+ try:+ link.symlink_to(outside, target_is_directory=True)+ except (OSError, NotImplementedError):+ pytest.skip("Cannot create symlinks in this environment")+ quick_summary = []+ monkeypatch.setattr(+ runners, "quick_summary", lambda **kwargs: quick_summary.append(1)+ )+ before_outside = _files_below(outside)++ with pytest.raises(BenchmarkOutputDirectoryError):+ runners.run_benchmark(+ "simpleqa", output_dir=str(link), run_evaluation=False+ )++ assert quick_summary == []+ assert sorted(path.name for path in data_dir.iterdir()) == [+ "benchmark_results"+ ]+ assert _files_below(outside) == before_outside+++ "module_name",+ [+ "local_deep_research.api.benchmark_functions",+ "local_deep_research.benchmarks.benchmark_functions",+ ],+)[email protected]("output_kind", ["absolute", "relative"])+def test_comparison_keeps_results_and_report_under_one_root(+ data_dir, tmp_path, module_name, output_kind+):+ import importlib++ module = importlib.import_module(module_name)+ outside = tmp_path / "outside-comparison"+ requested = str(outside) if output_kind == "absolute" else "comparisons"+ expected = data_dir / (+ "benchmark_comparisons" if output_kind == "absolute" else "comparisons"+ )+ output_paths = []++ def run_benchmark(**kwargs):+ output_paths.append(Path(kwargs["output_dir"]))+ return {"metrics": {}, "total_examples": 1}++ with (+ patch.object(module, "run_benchmark", side_effect=run_benchmark),+ patch(+ "local_deep_research.security.file_write_verifier.write_file_verified"+ ) as write,+ ):+ result = module.compare_configurations(+ output_dir=requested,+ configurations=[{"name": "First Setup"}, {"name": "Second Setup"}],+ )++ assert output_paths == [expected / "First_Setup", expected / "Second_Setup"]+ report = Path(result["report_path"])+ assert report.parent == expected+ write.assert_called_once()+ assert write.call_args.args[0] == str(report)+ assert expected.is_dir()+ assert not outside.exists()+++def test_quality_evaluation_keeps_its_temporary_results_contained(data_dir):+ """The runner keeps the temporary directory, so the cleanup removes it.++ A system temporary directory is outside the data directory, so the+ runner would write to its fallback instead and the results would+ outlive the cleanup.+ """+ from local_deep_research.benchmarks.metrics import calculation++ used = []++ def run_simpleqa_benchmark(**kwargs):+ directory = resolve_output_dir(+ kwargs["output_dir"], fallback="benchmark_results"+ )+ directory.mkdir(parents=True, exist_ok=True)+ (directory / "results.jsonl").write_text("{}\n")+ used.append(directory)+ return {"metrics": {"accuracy": 1.0}}++ with patch(+ "local_deep_research.benchmarks.runners.run_simpleqa_benchmark",+ side_effect=run_simpleqa_benchmark,+ ):+ result = calculation.evaluate_benchmark_quality({}, output_dir=None)++ assert result["accuracy"] == 1.0+ assert len(used) == 1+ assert used[0].parent == data_dir / "benchmark_results"+ assert not used[0].exists()+ assert list((data_dir / "benchmark_results").iterdir()) == []diff --git a/tests/security/test_identity_paths_and_host_effects.py b/tests/security/test_identity_paths_and_host_effects.pyindex 4b208b9786..b4f44c1204 100644--- a/tests/security/test_identity_paths_and_host_effects.py+++ b/tests/security/test_identity_paths_and_host_effects.py@@ -270,26 +270,18 @@ def backup_service_in_tmpdir(tmp_path, monkeypatch):return service- strict=True,- reason=(- "DEFENCE IN DEPTH: purge_and_refresh deletes every entry returned "- "by list_backups without resolving it and checking containment in "- "self.backup_dir (backup_service.py purge_and_refresh). The value "- "is server-derived today -- list_backups enumerates backup_dir -- "- "so this is NOT a reachable attack, only a missing guard that a "- "future caller or a poisoned directory listing could turn into an "- "out-of-root unlink. This test monkeypatches list_backups to "- "inject the hostile path. Fix: resolve each entry against "- "backup_dir and skip anything outside it. Flip the marker when "- "the containment check lands. The sibling symlink test below "- "passes already -- bare names are resolved against backup_dir."- ),-)@pytest.mark.parametrize("forged_path", ["../outside.db", "/tmp/not-used"])def test_purge_does_not_delete_backup_entries_outside_root(tmp_path, monkeypatch, backup_service_in_tmpdir, forged_path):+ """Defence in depth: a listing entry outside backup_dir is not deleted.++ list_backups only reports bare names from backup_dir, so this is not a+ reachable attack today; the test monkeypatches list_backups to inject+ the hostile path. purge_and_refresh re-checks each joined path with+ is_safe_glob_result and skips anything outside backup_dir. This was a+ strict xfail until that check landed.+ """service = backup_service_in_tmpdiroutside_canary = tmp_path / "outside.db"outside_canary.write_bytes(b"do not delete")diff --git a/tests/security/test_report_cleanup_only_for_deleted_rows.py b/tests/security/test_report_cleanup_only_for_deleted_rows.pynew file mode 100644index 0000000000..77a1547a4f--- /dev/null+++ b/tests/security/test_report_cleanup_only_for_deleted_rows.py@@ -0,0 +1,132 @@+"""Report files are removed only for research rows the route really deleted.++Both FastAPI cleanup routes (``delete_research`` and ``clear_history`` in+web/routers/research.py) share ``_unlink_report_file_if_inside_root`` for+the containment checks; ``tests/web/routers/test_report_unlink_guard.py``+and ``tests/security/test_report_path_deletion.py`` pin those. These tests+pin the ordering: a row whose guarded delete was refused (claimed by a+worker, or in progress) keeps its report file, and cleanup runs only for+the rows that were deleted.++Queue-state cleanup is scoped to the requesting user: since #7606 both routes+pass ``acting_user=username`` to ``cleanup_queued_research_state`` so another+user's queue row (and the task metadata it still needs) is left alone. The+assertions below pin that keyword as well as the deleted-row IDs.+"""++from types import SimpleNamespace+from unittest.mock import MagicMock, patch++import pytest++from local_deep_research.constants import ResearchStatus+from local_deep_research.web.routers import research as research_router+++def _session_context(db):+ context = MagicMock()+ context.__enter__.return_value = db+ # Never swallow an exception raised inside the route's session block.+ context.__exit__.return_value = False+ return context++[email protected]("deleted", [0, 1])+def test_delete_route_removes_the_report_only_after_row_deletion(deleted):+ db = MagicMock()+ db.query.return_value.scalar.return_value = False+ db.query.return_value.filter_by.return_value.first.return_value = (+ SimpleNamespace(+ status=ResearchStatus.COMPLETED, report_path="report.md"+ )+ )+ with (+ patch.object(+ research_router,+ "get_user_db_session",+ return_value=_session_context(db),+ ),+ patch.object(+ research_router, "_unlink_report_file_if_inside_root"+ ) as unlink,+ patch.object(+ research_router,+ "_delete_research_with_dependents",+ return_value=deleted,+ ),+ patch(+ "local_deep_research.web.queue.lifecycle_cleanup.cleanup_queued_research_state"+ ) as cleanup,+ ):+ result = research_router.delete_research(+ MagicMock(), "research-id", "alice"+ )++ if deleted:+ assert result == {"status": "success"}+ unlink.assert_called_once_with("report.md")+ cleanup.assert_called_once_with(+ db, ["research-id"], acting_user="alice"+ )+ db.commit.assert_called_once()+ else:+ # The row still exists, so a worker claimed it after the pre-check.+ assert result.status_code == 400+ unlink.assert_not_called()+ cleanup.assert_not_called()+ db.commit.assert_not_called()+++def test_clear_history_removes_reports_only_for_deleted_rows():+ rows = [+ SimpleNamespace(+ id="finished",+ status=ResearchStatus.COMPLETED,+ report_path="finished.md",+ ),+ SimpleNamespace(+ id="claimed",+ status=ResearchStatus.COMPLETED,+ report_path="claimed.md",+ ),+ SimpleNamespace(+ id="active",+ status=ResearchStatus.IN_PROGRESS,+ report_path="active.md",+ ),+ ]+ db = MagicMock()+ db.query.return_value.scalar.return_value = False+ db.query.return_value.all.return_value = rows+ with (+ patch.object(+ research_router,+ "get_user_db_session",+ return_value=_session_context(db),+ ),+ patch.object(+ research_router, "get_active_research_ids", return_value=[]+ ),+ patch.object(+ research_router, "_unlink_report_file_if_inside_root"+ ) as unlink,+ patch.object(+ research_router,+ "_delete_research_with_dependents",+ side_effect=[1, 0],+ ) as cascade,+ patch(+ "local_deep_research.web.queue.lifecycle_cleanup.cleanup_queued_research_state"+ ) as cleanup,+ ):+ result = research_router.clear_history(MagicMock(), "alice")++ assert result == {"status": "success"}+ # "active" is protected before the cascade; "claimed" is refused by it.+ assert [call.args[1] for call in cascade.call_args_list] == [+ "finished",+ "claimed",+ ]+ unlink.assert_called_once_with("finished.md")+ cleanup.assert_called_once_with(db, ["finished"], acting_user="alice")+ db.commit.assert_called_once()diff --git a/tests/security/test_settings_bulk_import.py b/tests/security/test_settings_bulk_import.pyindex a4f92da6c4..1656d04d17 100644--- a/tests/security/test_settings_bulk_import.py+++ b/tests/security/test_settings_bulk_import.py@@ -7,8 +7,11 @@ locations and environment overrides remain supported, including locationsoutside LDR_DATA_DIR, but user imports cannot redirect them. These checksexercise the real manager and download-service path with isolated roots.-The separate benchmark-output containment gap remains a strict xfail.-Select validation and valid user-editable imports are ordinary controls.+Benchmark output is contained where it is written instead: run_benchmark+resolves its output directory inside LDR_DATA_DIR (test_settings_path_escape).+The import itself still stores an outside benchmark.output_dir value, which+remains a strict xfail. Select validation and valid user-editable imports are+ordinary controls."""from pathlib import Path@@ -83,9 +86,11 @@ def test_import_cannot_redirect_saved_library_root(settings_manager, value):@pytest.mark.xfail(strict=True,reason=(- "OPEN EXPOSURE: benchmark.output_dir still accepts paths outside "- "LDR_DATA_DIR; the library's operator-only policy does not cover "- "benchmark output. Remove this marker when that policy is fixed."+ "OPEN: the settings import still stores a benchmark.output_dir "+ "outside LDR_DATA_DIR; the library's operator-only policy does not "+ "cover it. No code reads this key, and run_benchmark contains the "+ "output directory it is given when writing. Remove this marker if "+ "the import gains a path policy for this key."),)def test_import_rejects_benchmark_path_escape(settings_manager):@@ -123,7 +128,8 @@ def test_import_enforces_same_value_validation_as_save_routes(``app.timezone`` is used because select is one of the classes thevalidator can actually judge. Library paths have a separate operator-only- write policy; benchmark path containment remains a separate expected failure.+ write policy; benchmark output paths are contained by run_benchmark, and+ the import's acceptance of them remains a separate expected failure.Note the two validators read their constraints from different places:the save route judges the stored ``Setting`` row, while the importdiff --git a/tests/security/test_settings_path_escape.py b/tests/security/test_settings_path_escape.pyindex 1f705acc9d..39b51651f4 100644--- a/tests/security/test_settings_path_escape.py+++ b/tests/security/test_settings_path_escape.py@@ -1,4 +1,4 @@-"""Configured library-root compatibility and benchmark containment tripwires.+"""Configured library-root compatibility and benchmark output containment.The selected library policy (#7152/#7224, closing #6335) preserves existingsaved roots and operator overrides outside LDR_DATA_DIR. User imports cannot@@ -6,8 +6,9 @@ redirect them; test_settings_bulk_import covers that write boundary with thereal settings manager. These resolver tests ensure configured roots stillwork and document paths remain scoped to the user's library.-Benchmark output has a separate unresolved containment policy and retains-its strict xfails. Every configured root and canary lives under tmp_path.+Benchmark output follows a different rule (#7417): run_benchmark resolves+its output directory inside LDR_DATA_DIR and replaces any other location.+Every configured root and canary lives under tmp_path."""import os@@ -164,23 +165,16 @@ def test_relative_storage_path_still_creates_a_library(tmp_path, monkeypatch):assert library_root.is_dir()- strict=True,- reason=(- "OPEN EXPOSURE: benchmark.output_dir is a user-editable setting "- "and run_benchmark passes it straight to os.makedirs "- "(benchmarks/runners.py:73) and then writes the results/report "- "files under it (:113-119), with no containment check against "- "LDR_DATA_DIR. Fix: resolve output_dir and reject/clamp anything "- "outside the data directory. Remove this marker when the check "- "lands."- ),-)@pytest.mark.parametrize("output_dir", ["absolute", "relative"])def test_benchmark_writer_cannot_escape_data_directory(isolated_data_dir, tmp_path, monkeypatch, output_dir):- """The real runner must not create or append results outside LDR_DATA_DIR."""+ """The real runner must not create or append results outside LDR_DATA_DIR.++ This was a strict xfail until #7417's fix: run_benchmark now resolves+ output_dir inside the data directory and replaces an outside value with+ ``benchmark_results`` there.+ """from local_deep_research.benchmarks import runnersoutside = tmp_path / "outside"