Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

8.8 KB
Raw Blame

Test Files Analysis - Updated

Generated: 2025-09-21T12:15:00.000Z

Test File Organization

After analysis and reorganization, test files have been categorized with clear naming conventions:

File Naming Conventions:

  • test_*.js - Regular tests that can potentially run in CI
  • NO_CI_*.js - Tests that execute actual research (should NOT run in CI)
  • DEBUG_*.js - Debug/fix versions that are temporary or experimental
  • *_helper.js / *_config.js - Shared utilities and configurations

Current Test Statistics

Category Count Description
CI Tests 8 Currently configured in CI workflows
Regular Tests ~60 Can potentially be added to CI
NO_CI Tests 13 Execute actual research, manual only
DEBUG Tests 5 Temporary debug/fix versions
Helper Files 4 Shared utilities

Tests Currently in CI

File CI Workflow Status Purpose
test_auth_flow.js critical-ui-tests ✅ Active Authentication flow testing
test_concurrent_limit.js critical-ui-tests ✅ Active Concurrent request limits
test_export_functionality.js critical-ui-tests ✅ Active Export features
test_followup_research.js followup-research-tests ⚠️ Research Follow-up research (may fail)
test_metrics.js performance-tests ✅ Active Metrics functionality
test_research_simple.js critical-ui-tests ⚠️ Research Simple research submission
test_research_submit.js critical-ui-tests ⚠️ Research Research submission
test_responsive_ui_comprehensive.js responsive-ui-tests-enhanced ✅ Active UI responsiveness (includes mobile)
mobile/test_mobile_navigation_ci.js mobile-ui-tests ✅ Active Mobile navigation testing

Mobile Tests Organization

Mobile tests are organized in a separate mobile/ subdirectory:

Mobile-Specific Tests (mobile/ directory)

File Purpose In CI
test_mobile_navigation_ci.js Mobile navigation for CI ✅ Yes (mobile-ui-tests.yml)
test_mobile_nav_all_pages.js Tests mobile nav across all pages ❌ No
test_mobile_navigation_authenticated.js Authenticated mobile navigation ❌ No
test_mobile_ui_comprehensive.js Comprehensive mobile UI testing ❌ No

Mobile-Related Tests (main directory)

File Purpose In CI
test_mobile_metrics.js Mobile metrics testing ❌ No
test_responsive_ui_comprehensive.js Tests all viewports including mobile ✅ Yes
DEBUG_mobile_debug.js Mobile debugging (temporary) ❌ No
DEBUG_settings_mobile_fix.js Mobile settings fixes (temporary) ❌ No

NO_CI Tests (Execute Research)

These tests actually submit and wait for research to complete, so they cannot run in CI:

Original Name New Name Purpose
test_ajax_submit.js NO_CI_executes_research_ajax_research_submission.js AJAX research submission
test_research_complete.js NO_CI_executes_research_full_research_completion.js Full research completion flow
test_followup_simple.js NO_CI_executes_research_followup_simple.js Simple follow-up research
test_multi_research.js NO_CI_executes_research_multiple_research.js Multiple research submissions
test_complete_workflow.js NO_CI_executes_research_complete_workflow.js End-to-end workflow
test_simple_research.js NO_CI_executes_research_basic_research_flow.js Basic research flow
test_research_with_available_model.js NO_CI_executes_research_research_with_model.js Research with specific model

DEBUG Tests (Temporary/Experimental)

Original Name New Name Purpose
test_metrics_thread_fix.js DEBUG_metrics_thread_fix.js Threading issue fix
test_mobile_debug.js DEBUG_mobile_debug.js Mobile UI debugging
test_news_features_fixed.js DEBUG_news_features_fixed.js News feature fixes
test_research_submit_debug.js DEBUG_research_submit_debug.js Research submission debugging
test_settings_mobile_fix.js DEBUG_settings_mobile_fix.js Mobile settings fixes

Useful "Simple" Tests (Should Keep)

These tests were labeled "simple" but are actually valuable focused integration tests:

Test File Purpose Can Run in CI
test_simple_auth.js Basic authentication flow ✅ Yes
test_simple_metrics.js Metrics dashboard access ✅ Yes
test_simple_cost.js Cost analytics page ✅ Yes
test_api_key_simple_verify.js API key setting/retrieval ✅ Yes
test_settings_simple.js Settings save functionality ✅ Yes
test_queue_simple.js Research queue submission ⚠️ Maybe (if mocked)

Tests to Add to CI

Based on analysis, these tests provide good coverage without executing research:

High Priority (Core Functionality)

  1. test_api_key_comprehensive.js - API key management
  2. test_simple_auth.js - Authentication basics
  3. test_simple_metrics.js - Metrics dashboard
  4. test_settings_page.js - Settings functionality

Medium Priority (Feature Testing)

  1. test_history_page.js - History functionality
  2. test_news_subscription_form.js - News features
  3. test_autocomplete_selection.js - UI autocomplete
  4. test_benchmark_settings.js - Benchmark features

Low Priority (Edge Cases)

  1. test_context_overflow_standalone.js - Edge case handling
  2. test_rate_limiting_settings.js - Rate limit testing

Recommendations

1. Immediate Actions

  • ✅ Keep all test_simple_*.js files - they're valuable focused tests
  • ✅ Renamed research-executing tests with NO_CI_ prefix
  • ✅ Renamed debug tests with DEBUG_ prefix

2. CI Improvements

  • Add the high-priority tests to CI workflows
  • Consider creating a new "quick-tests" workflow for simple tests
  • Mock research submission for tests that need it

3. Test Maintenance

  • Delete DEBUG files after fixes are merged
  • Consolidate duplicate test coverage (16 metrics tests → 3-4 comprehensive ones)
  • Add proper test descriptions to all files

4. Test Categories for CI Workflows

Create separate CI workflows:

  • critical-auth-tests.yml - Authentication and user management
  • api-tests.yml - API endpoints and settings
  • ui-tests.yml - UI components and responsiveness
  • metrics-tests.yml - Metrics and analytics features

Helper Files

Essential shared utilities that all tests depend on:

File Purpose
auth_helper.js Authentication utilities
puppeteer_config.js Browser configuration
model_helper.js Model setup utilities
browser_config.js Browser settings

CI Workflow Test Distribution

No Duplication Found! ✅

Each test file is run in exactly ONE workflow - no tests are duplicated across workflows.

Test to Workflow Mapping:

Test File Workflow Purpose
test_auth_flow.js critical-ui-tests.yml Authentication flow (foundation for other tests)
test_concurrent_limit.js critical-ui-tests.yml Tests concurrent request limits
test_export_functionality.js critical-ui-tests.yml Export feature testing
test_research_simple.js critical-ui-tests.yml Simple research submission
test_research_submit.js critical-ui-tests.yml Research submission testing
test_followup_research.js followup-research-tests.yml Follow-up research features
test_metrics.js performance-tests.yml Metrics and performance
test_responsive_ui_comprehensive.js responsive-ui-tests-enhanced.yml UI responsiveness across viewports

Workflow Coverage:

Workflow Tests Run Focus Area
critical-ui-tests.yml 5 tests Core functionality: auth, research, export
followup-research-tests.yml 1 test Follow-up research feature
performance-tests.yml 1 test Performance metrics
responsive-ui-tests-enhanced.yml 1 test UI responsiveness (all viewports)
mobile-ui-tests.yml 1 test Mobile-specific navigation

Analysis:

  • Good separation of concerns - Each workflow has a clear purpose
  • No redundancy - Tests are not duplicated
  • Room for expansion - Most workflows only run 1-2 tests, could add more

Summary

  • Total test files: 86 (after reorganization)
    • 68 regular tests (can run in CI)
    • 13 NO_CI tests (execute research)
    • 5 DEBUG tests (temporary)
  • Files currently in CI: 8 (no duplication)
  • Potential CI additions: 20+ tests identified
  • Research-executing tests: 13 (clearly marked NO_CI)
  • Debug files: 5 (clearly marked DEBUG)

The test suite is now better organized with clear naming conventions that indicate which tests can run in CI and which require manual execution.