~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
"""
Document loaders module.
Provides centralized document loading functionality for both:
- Collection uploads (bytes from HTTP requests)
- Local search engine (file paths on disk)
Supported formats (35+ formats):
Documents:
- PDF (.pdf)
- Text (.txt)
- Markdown (.md, .markdown)
- Word (.doc, .docx)
- RTF (.rtf) - Rich Text Format
- RST (.rst) - reStructuredText documentation
Presentations:
- PowerPoint (.ppt, .pptx)
Spreadsheets:
- Excel (.xls, .xlsx)
- CSV (.csv), TSV (.tsv)
- ODT (.odt) - OpenDocument text
Data formats:
- JSON (.json)
- YAML (.yaml, .yml)
- XML (.xml) - important for USPTO patent data
- TOML (.toml) - config files
Web content:
- HTML (.html, .htm)
- MHTML (.mhtml, .mht) - saved web pages
Images (OCR):
- PNG, JPG, JPEG, TIFF, BMP, HEIC
Research/Notes:
- Jupyter Notebooks (.ipynb)
- Evernote exports (.enex)
- EPUB (.epub) - ebooks (requires pandoc)
- Org (.org) - Emacs org-mode files
- Email (.eml) - email messages
"""
from .bytes_loader import extract_text_from_bytes, load_from_bytes
from .json_loader import SimpleJSONLoader
from .loader_registry import (
get_loader_class_for_extension,
get_loader_for_path,
get_supported_extensions,
is_extension_supported,
)
from .yaml_loader import YAMLLoader
__all__ = [
# Bytes loading (for uploads)
"load_from_bytes",
"extract_text_from_bytes",
# Path loading (for local files)
"get_loader_for_path",
# Registry functions
"get_supported_extensions",
"is_extension_supported",
"get_loader_class_for_extension",
# Custom loaders
"YAMLLoader",
"SimpleJSONLoader",
]