Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

8.1 KB
Raw Blame

Metrics & Analytics Dashboard

This guide covers the Metrics Dashboard for monitoring research performance, costs, and usage analytics.

Table of Contents


Overview

The Metrics Dashboard provides insights into your research activity:

  • Token usage and LLM costs
  • Search performance and rate limiting
  • Research quality ratings
  • Link analytics and domain classification

Access the dashboard at: http://localhost:5000/metrics


Research Metrics

Overview Page

The main metrics page shows:

Metric Description
Total Researches Number of research sessions
Total Tokens Tokens consumed across all research
Total Cost Estimated LLM API costs
Average Duration Mean research completion time

Incomplete Metrics Warning

If token, search, or rate-limit updates cannot be recorded, the main metrics page shows a warning for the signed-in user. It counts final write failures after retries, missing database credentials, and search or token updates skipped to preserve pending caller work or when no matching owner session is available. Research can continue while recording is unavailable. The warning is a snapshot taken when the page is rendered: changing the time range or research mode reloads the metrics data but does not refresh the warning. Reload the full page to see the latest count and reason categories; the server logs provide the existing diagnostic details.

The count describes failed or skipped recording attempts, not a verified number of missing rows: an error during session cleanup can occur after a write.

These counters are held in memory by the current server process, separately for each user. They reset on restart and retain at most 512 recent writers; older entries can be evicted. Counts are therefore lower bounds, not a durable audit or a count restricted to the dashboard's selected date range. An absent warning does not prove historical metrics are complete. Calls without an identified user and intentional programmatic mode do not create a user's warning. The counter stores no passwords, queries, or exception messages.

Per-Research Metrics

Click on any research to see detailed metrics:

Timeline

View the research execution timeline:

  • Start and end times
  • Iteration progress
  • Question generation timing
  • Search execution timing

Search Metrics

Details about search operations:

  • Queries executed
  • Results retrieved
  • Search engine usage
  • Response times

Links

All links discovered during research:

  • Source URLs
  • Domain classification
  • Relevance scores

Cost Analytics

Token Tracking

The system tracks token usage for cost estimation:

Metric Description
Input Tokens Tokens sent to LLM
Output Tokens Tokens received from LLM
Total Tokens Combined usage
Estimated Cost Based on model pricing

Cost Breakdown

View costs broken down by:

  • Model - Which LLM models were used
  • Provider - OpenAI, Anthropic, etc.
  • Time Period - Daily, weekly, monthly
  • Research - Per-research costs

Accessing Cost Analytics

  1. Navigate to Metrics → Costs
  2. View summary statistics
  3. Filter by date range
  4. Export data as needed

Pricing Information

The system includes pricing data for common models:

GET /metrics/api/pricing
GET /metrics/api/pricing/<model_name>

Rate Limiting

Monitoring Rate Limits

View current rate limiting status for search engines:

  1. Navigate to Metrics → Rate Limiting
  2. See per-engine statistics
Column Description
Engine Search engine name
Current Wait Active wait time
Success Rate Percentage of successful requests
Total Requests Request count
Last Updated Most recent activity

Rate Limit Management

From the dashboard you can:

  • View current rate limit estimates
  • See learned optimal wait times
  • Monitor success rates

Link Analytics

Domain Classification

The system classifies domains found in research:

Category Examples
Academic arxiv.org, pubmed.gov
News bbc.com, reuters.com
Government .gov, .edu
Commercial company websites
Reference wikipedia.org

Link Statistics

View aggregate statistics:

  • Total links discovered
  • Unique domains
  • Category distribution
  • Top domains by frequency

Accessing Link Analytics

  1. Navigate to Metrics → Links
  2. View domain breakdown
  3. Filter by category
  4. See classification summary

Classification API

Classify domains programmatically:

POST /metrics/api/domain-classifications/classify
{
  "domains": ["example.com", "arxiv.org"]
}

Star Reviews

Rating Research Quality

Rate your research results to track quality over time:

  1. Open a completed research
  2. Click the star rating (1-5)
  3. Optionally add feedback text

Viewing Ratings

Navigate to Metrics → Star Reviews to see:

  • Average rating across all research
  • Rating distribution
  • Recent ratings with feedback
  • Trends over time

Rating API

# Get ratings for a research
GET /metrics/api/ratings/<research_id>

# Submit a rating
POST /metrics/api/ratings/<research_id>
{
  "rating": 5,
  "feedback": "Excellent results"
}

API Reference

Metrics Endpoints

Endpoint Method Description
/metrics/api/metrics GET Overview metrics
/metrics/api/metrics/enhanced GET Detailed metrics
/metrics/api/metrics/research/<id> GET Per-research metrics
/metrics/api/metrics/research/<id>/timeline GET Research timeline
/metrics/api/metrics/research/<id>/search GET Search metrics
/metrics/api/metrics/research/<id>/links GET Research links

Cost Endpoints

Endpoint Method Description
/metrics/api/pricing GET Model pricing data
/metrics/api/pricing/<model> GET Specific model pricing
/metrics/api/cost-calculation POST Calculate costs
/metrics/api/research-costs/<id> GET Research cost breakdown
/metrics/api/cost-analytics GET Cost analytics summary

Rate Limiting Endpoints

Endpoint Method Description
/metrics/api/rate-limiting GET Rate limit statistics
/metrics/api/rate-limiting/current GET Current rate limits

Rating Endpoints

Endpoint Method Description
/metrics/api/ratings/<id> GET Get research rating
/metrics/api/ratings/<id> POST Submit rating
/metrics/api/star-reviews GET All ratings summary

Domain Classification Endpoints

Endpoint Method Description
/metrics/api/domain-classifications GET All classifications
/metrics/api/domain-classifications/summary GET Classification summary
/metrics/api/domain-classifications/classify POST Classify domains
/metrics/api/domain-classifications/progress GET Classification progress
/metrics/api/link-analytics GET Link analytics data

Troubleshooting

No Metrics Showing

  • Ensure you have completed at least one research
  • Check that token tracking is enabled
  • Refresh the page

Cost Estimates Incorrect

  • Verify model pricing is up to date
  • Check provider is correctly identified
  • Token counts may vary by model

Rate Limiting Data Missing

  • Run some searches to generate data
  • Verify rate limiting is enabled in Settings, then refresh the dashboard

See Also