~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
CI/CD Integration for Enhanced Test Framework
Overview
The enhanced test framework from PR #361 has been integrated into the CI/CD pipeline with the following components:
1. Main Test Pipeline Integration
Updated tests/run_all_tests.py
The main test runner now includes our new tests in the unit test suite:
tests/test_wikipedia_url_security.py- Security validation teststests/test_search_engines_enhanced.py- Enhanced search engine teststests/test_utils.py- Test utilities validation
These tests run in all profiles:
- fast - Quick feedback (< 30s)
- standard - Development testing (< 5min)
- full - Comprehensive testing (< 15min)
- unit-only - Unit tests only (< 10s)
2. CI Workflow Integration
.github/workflows/pytest-tests.yml
The main pytest workflow runs all tests including the enhanced test framework:
- Runs on all PRs and pushes to main/dev
- Includes URL validation security tests
- Includes enhanced search engine tests
- Uploads coverage reports to GitHub Pages
3. Test Organization
New Test Structure
tests/
├── fixtures/ # Reusable test fixtures
│ ├── search_engine_mocks.py
│ └── README.md
├── mock_fixtures.py # Mock data functions
├── mock_modules.py # Dynamic module mocking
├── test_utils.py # Common test utilities
├── test_search_engines_enhanced.py
└── test_wikipedia_url_security.py
4. CI Environment Variables
The tests use these environment variables in CI:
USE_FALLBACK_LLM=true- Uses mock LLM to avoid API callsCI=true- Indicates CI environment for headless testing
5. Benefits
- Security: URL validation tests run on every PR
- Fast Feedback: Enhanced tests run in parallel workflow
- Coverage: New tests included in coverage reports
- Reliability: Mock fixtures ensure consistent test results
- Modularity: Fixtures can be reused across test suites
6. Running Tests Locally
To run the new tests locally:
# Run all new tests
pdm run pytest tests/test_wikipedia_url_security.py tests/test_search_engines_enhanced.py -v
# Run with coverage
pdm run pytest tests/test_wikipedia_url_security.py tests/test_search_engines_enhanced.py --cov=src --cov-report=html
# Run via the test runner
cd tests && pdm run python run_all_tests.py unit-only
7. Future Improvements
- Add more search engine mock fixtures
- Create integration tests for all search engines
- Add performance benchmarks
- Expand security validation to other engines
- Add fuzzing tests for URL validation