Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

5.1 KB
Raw Blame

CLI Tools Reference

Local Deep Research includes command-line tools for benchmarking and running the MCP server.

Table of Contents


Benchmarking CLI

Run benchmarks to evaluate search quality and compare configurations.

Basic Usage

python -m local_deep_research.benchmarks.cli.benchmark_commands <command> [options]

Commands

simpleqa - Run SimpleQA Benchmark

Tests factual question answering accuracy.

python -m local_deep_research.benchmarks.cli.benchmark_commands simpleqa [options]

Options:

Option Default Description
--examples 100 Number of questions to test
--iterations 3 Search iterations per question
--questions 3 Questions per iteration
--search-tool searxng Search engine to use
--search-strategy source_based Strategy (source-based, focused-iteration, focused-iteration-standard, topic-organization)
--search-model (default) LLM model for research
--search-provider (default) LLM provider
--eval-model (default) Model for answer evaluation
--eval-provider (default) Provider for evaluation
--output-dir data directory/benchmark_results Results directory, kept below the data directory (see Output location)
--human-eval false Use human evaluation
--no-eval false Skip evaluation phase
--custom-dataset - Path to custom dataset

Output location: benchmark result, evaluation and report files are written below the data directory (LDR_DATA_DIR, or the platform default). A relative --output-dir is taken relative to the data directory, not the current directory. A path outside the data directory, or the data directory itself, is replaced by benchmark_results there, with a warning in the log. If that default is not usable either, for example because it is a symlink to another disk, the command stops with an error before running. The report and results paths it prints are the real locations.

Example:

# Run 50 examples with Ollama
python -m local_deep_research.benchmarks.cli.benchmark_commands simpleqa \
  --examples 50 \
  --search-provider ollama \
  --search-model llama3.2

browsecomp - Run BrowseComp Benchmark

Tests complex reasoning and multi-step research.

python -m local_deep_research.benchmarks.cli.benchmark_commands browsecomp [options]

Same options as simpleqa.

Example:

# Run BrowseComp with focused-iteration strategy
python -m local_deep_research.benchmarks.cli.benchmark_commands browsecomp \
  --examples 20 \
  --search-strategy focused-iteration \
  --iterations 5

compare - Compare Configurations

Compare multiple search configurations on the same dataset.

python -m local_deep_research.benchmarks.cli.benchmark_commands compare [options]

Options:

Option Default Description
--dataset simpleqa Dataset to use (simpleqa, browsecomp)
--examples 20 Examples per configuration
--output-dir data directory/benchmark_results/comparison Results directory, kept below the data directory (see Output location under simpleqa; the default replacement here is benchmark_comparisons)

Example:

# Compare configurations
python -m local_deep_research.benchmarks.cli.benchmark_commands compare \
  --dataset simpleqa \
  --examples 30

list - List Available Benchmarks

python -m local_deep_research.benchmarks.cli.benchmark_commands list

Shows available benchmark datasets and their descriptions.


MCP Server CLI

Run the MCP server for Claude Desktop integration.

Basic Usage

# Via entry point
ldr-mcp

# Via module
python -m local_deep_research.mcp

The server communicates over STDIO (stdin/stdout for JSON-RPC, stderr for logs). It is designed to be launched by Claude Desktop, not run interactively.

Environment Variables

Set via Claude Desktop config env block or shell environment:

Variable Description Example
LDR_LLM_PROVIDER LLM provider openai, ollama, anthropic
LDR_LLM_MODEL Model name gpt-4, llama3:8b
LDR_LLM_OPENAI_API_KEY OpenAI API key sk-...
LDR_SEARCH_TOOL Default search engine searxng, arxiv, wikipedia
LDR_SEARCH_SEARCH_STRATEGY Default strategy source-based, focused-iteration

See MCP Server Guide for full documentation.


Troubleshooting

Benchmark Not Starting

  • Verify LLM provider is configured
  • Check search engine is available
  • Ensure sufficient disk space for results

Export Permission Error

  • Check write permissions on output directory
  • Use a different output directory

See Also