Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

1.7 KB
Raw Blame

LLM Integration Examples

This directory contains examples of integrating custom LangChain LLMs with Local Deep Research.

Examples

1. Basic Custom LLM (basic_custom_llm.py)

Shows the simplest way to create and use a custom LLM with LDR.

2. Advanced Custom LLM (advanced_custom_llm.py)

Demonstrates advanced features like:

  • Factory functions with configuration
  • Multiple LLM registration
  • Combining with custom retrievers
  • Error handling and retry logic

3. Fine-tuned Model Integration (finetuned_model_example.py)

Example of using a fine-tuned model for domain-specific research.

4. Mock LLM for Testing (mock_llm_example.py)

Shows how to create mock LLMs for testing your research pipelines without API costs.

5. Rate-Limited Wrapper (rate_limited_llm.py)

Demonstrates wrapping any LLM with rate limiting to avoid API limits.

Running the Examples

  1. Install Local Deep Research:
pip install local-deep-research
  1. Run an example:
python examples/llm_integration/basic_custom_llm.py

Key Concepts

  • BaseChatModel: All custom LLMs must inherit from langchain_core.language_models.BaseChatModel
  • Factory Functions: Can be used to create LLMs with dynamic configuration
  • Registration: LLMs are registered via the llms parameter in API functions
  • Provider Selection: Use the registered name as the provider parameter

Common Use Cases

  1. Fine-tuned Models: Use models trained on your specific domain
  2. Custom Wrappers: Add logging, retry logic, or preprocessing
  3. Mock Testing: Test research flows without real LLM calls
  4. Rate Limiting: Manage API quotas effectively
  5. Multi-Model Pipelines: Use different models for different research phases