~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
UI Tests
Browser-based testing using Puppeteer to verify that web pages load correctly and JavaScript executes without errors.
Setup
Install Puppeteer (if not already installed):
npm install puppeteer
Running Tests
Local environment notes
The auth endpoints rate-limit at 5 logins per 15 min per IP, and registrations are rate limited per IP as well (see REGISTRATION_RATE_LIMIT in web/dependencies/rate_limit.py). The full suite re-authenticates dozens of times, so a local run against a default-configured server gets blocked after the first few tests and shows a wave of Login failed failures that are not real test failures.
Two options for local runs:
Disable rate limiting (simplest):
LDR_DISABLE_RATE_LIMITING=true python -m local_deep_research.web.appUse the CI test-user pattern: pre-create
test_adminonce, then run the suite withCI=truesoauth_helper.jslogs in with that user instead of registering fresh users each time.python scripts/ci/init_test_database.py # one-time setup CI=true node tests/ui_tests/run_all_tests.js --shard=<shard>Note:
CI=truerequires--shard=<name>(the runner fails fast otherwise to catch matrix misconfiguration). Iterate shards manually for a full local run.
Either approach is fine; option 1 is closer to how CI runs the suite (CI launches the server with rate limiting off).
Run All UI Tests (Recommended)
cd /path/to/local-deep-research
node tests/ui_tests/run_all_tests.js
Run a Single Shard
Tests are partitioned into shards so CI can run them in parallel matrix
cells. Each entry in run_all_tests.js carries a shard: property; the
authoritative shard list is VALID_SHARDS at the top of that file (kept in
sync with strategy.matrix.shard in .github/workflows/docker-tests.yml).
Local dev rarely needs this, but you can reproduce a CI cell with:
node tests/ui_tests/run_all_tests.js --shard=settings-pages
node tests/ui_tests/run_all_tests.js --shard=library
node tests/ui_tests/run_all_tests.js --shard=accessibility
Valid shards (source of truth: VALID_SHARDS in run_all_tests.js):
| Shard | Scope |
|---|---|
auth-login |
Login/auth flow |
auth-register |
Registration (isolated — heavy SQLCipher DB creation) |
auth-pages |
Page browsing, navigation, comprehensive auth |
research-workflow |
Core research lifecycle |
research-form |
Research form interactions + results |
research-metrics |
Metrics charts, dashboard, progress |
settings-core |
Settings page, errors, save, interactions |
settings-pages |
Settings tabs, star reviews, journal quality |
library |
Collections, documents, download manager |
history-news |
History page, news subscriptions |
mobile |
Mobile navigation, interactions, UI |
api-crud |
API endpoints, CRUD operations, rate limiting |
error-benchmark |
Error handling/recovery, benchmark, context overflow |
accessibility |
Keyboard navigation & ARIA |
chat-core |
Chat ARIA, CSRF, suggestion chips |
chat-lifecycle |
Chat sessions, export, edit title |
Passing an unknown shard, or leaving --shard off while CI=true is set,
fails fast with an error — this catches matrix misconfiguration before a cell
wastes compute running the wrong slice.
Run All Page Tests
cd /path/to/local-deep-research
node tests/ui_tests/test_pages_browser.js
Run Individual Component Tests
# Test metrics charts functionality
node tests/ui_tests/test_metrics_charts.js
# Test research results page loading
node tests/ui_tests/test_research_results.js
# Test settings page functionality
node tests/ui_tests/test_settings_page.js
# Test settings error detection
node tests/ui_tests/test_settings_errors.js
# Test settings save functionality
node tests/ui_tests/test_settings_save.js
# Test star reviews analytics page
node tests/ui_tests/test_star_reviews.js
Prerequisites
- Web server must be running on http://127.0.0.1:5000
- Start the server with:
python -m local_deep_research.web.app
What These Tests Check
All Pages
- Page loads without timeout
- Has title and body elements
- No critical JavaScript errors
- Basic DOM structure is present
Metrics Dashboard (/metrics/)
- Loading, content, and error elements exist
- Metrics data loads (token counts, research counts)
- JavaScript executes without syntax errors
- Takes screenshot for visual debugging
Research Page (/)
- Query input field exists and is enabled
- Submit button exists and is enabled
- Mode selection dropdown exists
History Page (/history/)
- History container elements exist
- Search functionality is present
- Content is visible
Settings Page (/settings/)
- Forms and input elements exist
- Per-setting auto-save controls are available
- Configuration interface loads
Individual Component Tests
Metrics Charts Test (test_metrics_charts.js)
- Verifies both token consumption and search activity charts render correctly
- Checks for Canvas elements (Chart.js charts)
- Scrolls through the page to test dynamic loading
- Takes screenshot for visual verification
- Validates chart data loading from API endpoints
Research Results Test (test_research_results.js)
- Tests loading of specific research report (research ID 67)
- Monitors network requests to identify API failures
- Checks for error messages vs. actual content
- Validates research report file loading functionality
Settings Page Test (test_settings_page.js)
- Comprehensive test of settings page loading
- Monitors all API calls (available models, search engines, settings)
- Counts setting form elements to verify proper loading
- Validates 300+ setting elements are present on the page
- Tests settings synchronization and form functionality
Settings Error Detection Test (test_settings_errors.js)
- Tests for error messages when changing setting values
- Monitors browser console for JavaScript errors
- Detects network errors (4xx/5xx HTTP responses)
- Searches for error DOM elements (.error, .alert-danger, etc.)
- Simulates user interaction with dropdowns and input fields
- Validates error handling and user feedback mechanisms
Settings Save Test (test_settings_save.js)
- Tests the per-setting auto-save workflow
- Monitors network requests to
/research/settings/save_all_settings - Confirms the bulk-submit button is absent
- Validates checkbox-triggered auto-save
- Checks for proper API response handling (success/error states)
- Monitors console logging during save operations
- Verifies success/error message display to users
- Tests CSRF token handling and form validation
Star Reviews Test (test_star_reviews.js)
- Tests the star reviews analytics page and navigation
- Monitors API endpoint
/metrics/api/star-reviewsfunctionality - Validates Chart.js rendering for bar charts and line charts
- Tests period selector and data filtering
- Checks overall statistics display and rating distribution
- Verifies recent ratings list population
- Tests navigation between metrics dashboard and star reviews page
Testing Strategy
The UI test suite follows a layered approach:
1. Page Load Tests (test_pages_browser.js)
- Purpose: Verify basic page functionality works across all routes
- Coverage: Home, metrics, history, settings pages
- Validates: Page loads, DOM structure, basic JavaScript execution
2. Component-Specific Tests
- Purpose: Deep dive into specific UI components and workflows
- Coverage: Charts, forms, API integrations, file loading
- Validates: Complex interactions, data flow, error handling
3. Error Detection Tests
- Purpose: Catch JavaScript errors and API failures that could break user experience
- Coverage: Console errors, network failures, validation errors
- Validates: Proper error handling and user feedback
4. Integration Tests
- Purpose: Test complete user workflows end-to-end
- Coverage: Settings save, research report loading, chart rendering
- Validates: Data persistence, file operations, API coordination
What Problems These Tests Solve
- Regression Detection: Catch when code changes break existing functionality
- JavaScript Error Monitoring: Identify runtime errors that users would encounter
- API Integration Validation: Ensure frontend properly communicates with backend
- File Path Resolution: Verify reports and assets load correctly regardless of server setup
- Cross-Browser Compatibility: Puppeteer testing ensures consistent behavior
- Performance Monitoring: Detect slow loading or hanging operations
Output
- Console logs show detailed test progress
- Screenshots saved to
tests/ui_tests/screenshots/ - Summary report shows pass/fail status for each page
Why These Tests Are Useful
- Catch JavaScript Errors: Unlike unit tests, these catch runtime JS errors in the browser
- End-to-End Validation: Verify the complete request → response → render cycle
- Regression Prevention: Detect when changes break the UI
- Cross-Page Coverage: Test all major application pages consistently
Example Output
🚀 Starting browser test session...
📄 Testing Metrics Dashboard: http://127.0.0.1:5000/metrics/
📝 [LOG] === METRICS SCRIPT STARTED ===
📝 [LOG] === STARTING LOADMETRICS ===
📝 [LOG] Basic data success, setting metricsData
✅ Metrics Dashboard loaded successfully
🔍 Basic checks for Metrics Dashboard:
Has title: true
Has body: true
Body visible: true
📊 Metrics page checks:
Content visible: true
Total tokens: 41,801
Total researches: 11
==================================================
📋 TEST SUMMARY
==================================================
✅ PASS Home/Research Page
✅ PASS Metrics Dashboard
✅ PASS History Page
✅ PASS Settings Page
Total: 4 tests
Passed: 4
Failed: 0
🎉 All tests passed!
Shared helpers in test_lib/
Common patterns live in tests/ui_tests/test_lib/ and are re-exported from ./test_lib. Use these rather than inlining the equivalent logic — duplicated logic is how the same bug class hides in multiple files (#4069 / #4127).
findActionButton(page, { selectors, keywords, click })— locate a button by text content with word-boundary matching. Defaults toselectors='button, a.btn, .btn'andkeywords=['create', 'new', 'add']. Returns{ found, text }.- Why this exists:
text.includes('new')matches the substring "new" inside "News" (e.g. an unrelated<a>Back to News Feed</a>button). The helper wraps\b(?:create|new|add)\bso "Create Subscription" / "New Folder" / "Add Item" still match but "News Feed" doesn't. - Example:
const { found } = await findActionButton(page, { click: true });
- Why this exists:
navigateTo(page, url, options)— robust navigation with retries and CI-tuned timeouts.setupTest({ authenticate: true }) / teardownTest(ctx)— launches a browser, optionally authenticates viaauth_helper.js(uses the CI test user whenCI=true, registers a fresh user otherwise), and returns a context withpage,browser, and config.TestResults(suiteName)— collector with.run(group, name, fn),.skip(...),.print(),.save()(writes JSON + JUnit XML totest-results/).
Prefer asserting browser-level contracts (HTML5 :invalid, navigation target, computed style) over heuristic scraping for .error / .invalid-feedback classes — heuristic selectors mask regressions when app markup drifts.
Adding New Page Tests
To test a new page, add an entry to the testCases array in test_pages_browser.js:
{ path: '/new-page/', name: 'New Page', tests: newPageTests }
Then create a custom test function:
const newPageTests = async (page) => {
console.log('🧪 Running new page tests...');
const checks = await page.evaluate(() => {
// Check for page-specific elements
return {
hasSpecificElement: !!document.getElementById('specific-element')
};
});
console.log(`Specific element present: ${checks.hasSpecificElement}`);
};