Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Add comprehensive evaluation framework for compression accuracy
chopratejas92e4a24ea7b5
diff --git a/README.md b/README.mdindex 48331e482..99c6afb7d 100644--- a/README.md+++ b/README.md@@ -283,6 +283,63 @@ See the full [Agno Integration Guide](docs/agno.md) for hooks, multi-provider su| **Text Utilities** | Opt-in compression for search/logs | [Text Compression](docs/text-compression.md) || **LLMLingua-2** | ML-based 20x compression (opt-in) | [LLMLingua](docs/llmlingua.md) || **Code-Aware** | AST-based code compression (tree-sitter) | [Transforms](docs/transforms.md) |+| **Evals Framework** | Prove compression preserves accuracy (12+ datasets) | [Evals](headroom/evals/README.md) |++---++## Evaluation Framework: Prove It Works++Skeptical? Good. We built a comprehensive evaluation framework to **prove** compression preserves accuracy.++```bash+# Install evals+pip install "headroom-ai[evals]"++# Quick sanity check (5 samples)+python -m headroom.evals quick++# Run on real datasets+python -m headroom.evals benchmark --dataset hotpotqa -n 100+```++### How Evals Work++```+Original Context ───► LLM ───► Response A+ │+Compressed Context ─► LLM ───► Response B+ │+ Compare A vs B │+ ─────────────────+ F1 Score: 0.95+ Semantic Similarity: 0.97+ Ground Truth Match: ✓+ ─────────────────+ PASS: Accuracy preserved+```++### Available Datasets (12+)++| Category | Datasets |+|----------|----------|+| **RAG** | HotpotQA, Natural Questions, TriviaQA, MS MARCO, SQuAD |+| **Long Context** | LongBench (4K-128K tokens), NarrativeQA |+| **Tool Use** | BFCL (function calling), ToolBench, Built-in samples |+| **Code** | CodeSearchNet, HumanEval |++### CI Integration++```yaml+# GitHub Actions+- name: Run Compression Evals+ run: python -m headroom.evals quick -n 20+ env:+ ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}+```++Exit code 0 if accuracy ≥ 90%, 1 otherwise.++See the full [Evals Documentation](headroom/evals/README.md) for datasets, metrics, and programmatic API.---@@ -334,6 +391,7 @@ pip install headroom-ai # SDK onlypip install "headroom-ai[proxy]" # Proxy serverpip install "headroom-ai[langchain]" # LangChain integrationpip install "headroom-ai[agno]" # Agno agent framework+pip install "headroom-ai[evals]" # Evaluation frameworkpip install "headroom-ai[code]" # AST-based code compressionpip install "headroom-ai[llmlingua]" # ML-based compressionpip install "headroom-ai[all]" # Everything@@ -349,6 +407,7 @@ pip install "headroom-ai[all]" # Everything|-------|-------------|| [Memory Guide](docs/memory.md) | Persistent memory for LLMs || [Compression Guide](docs/compression.md) | Universal compression with ML detection |+| [Evals Framework](headroom/evals/README.md) | Prove compression preserves accuracy || [LangChain Integration](docs/langchain.md) | Full LangChain support || [Agno Integration](docs/agno.md) | Full Agno agent framework support || [SDK Guide](docs/sdk.md) | Fine-grained control |diff --git a/headroom/evals/README.md b/headroom/evals/README.mdnew file mode 100644index 000000000..4f67bef8b--- /dev/null+++ b/headroom/evals/README.md@@ -0,0 +1,333 @@+# Headroom Evaluation Framework++**Prove that compression preserves LLM accuracy through rigorous before/after testing.**++## Installation++```bash+pip install headroom-ai[evals]+```++This installs:+- `datasets` - HuggingFace datasets for benchmark data+- `sentence-transformers` - Semantic similarity computation+- `numpy` & `scikit-learn` - ML metrics+- `anthropic` & `openai` - LLM API clients++## Quick Start++### CLI Usage++```bash+# Quick sanity check (5 samples, uses Anthropic Claude)+python -m headroom.evals quick++# List all available datasets+python -m headroom.evals list++# Run benchmark on specific dataset+python -m headroom.evals benchmark --dataset hotpotqa -n 50++# Run all RAG datasets+python -m headroom.evals benchmark --dataset rag++# Run ALL datasets (comprehensive)+python -m headroom.evals benchmark --dataset all -o results.json++# Generate HTML report+python -m headroom.evals report -i results.json+```++### Python API++```python+from headroom.evals import run_quick_eval, BeforeAfterRunner, LLMConfig+from headroom.evals import load_hotpotqa, load_squad, load_tool_output_samples++# Quick evaluation+results = run_quick_eval(n_samples=5)+print(results.summary())++# Custom evaluation+runner = BeforeAfterRunner(+ llm_config=LLMConfig(provider="anthropic", model="claude-sonnet-4-20250514"),+ use_semantic_similarity=True,+)++# Load a dataset+suite = load_hotpotqa(n=100)++# Run evaluation+results = runner.run(suite, progress_callback=lambda cur, tot, r: print(f"{cur}/{tot}"))+print(results.summary())+```++## The Before/After Evaluation Pattern++This framework proves compression accuracy through a simple but rigorous pattern:++```+┌─────────────────────────────────────────────────────────────┐+│ BEFORE/AFTER EVAL │+├─────────────────────────────────────────────────────────────┤+│ │+│ Original Context ────────┐ │+│ (e.g., 10,000 tokens) │ │+│ ▼ │+│ ┌─────────┐ │+│ │ LLM │───► Response A │+│ └─────────┘ │+│ ▲ │+│ Compressed Context ──────┘ │+│ (e.g., 3,000 tokens) │ │+│ ▼ │+│ ┌─────────┐ │+│ │ LLM │───► Response B │+│ └─────────┘ │+│ │+│ Compare A vs B: │+│ • F1 Score (token overlap) │+│ • Semantic Similarity (embedding cosine) │+│ • Ground Truth Match (if available) │+│ • Exact Match (normalized) │+│ │+│ Result: PASS if responses are functionally equivalent │+│ FAIL if compression lost critical information │+│ │+└─────────────────────────────────────────────────────────────┘+```++## Available Datasets++### RAG / Retrieval (5 datasets)++| Dataset | Description | Default N |+|---------|-------------|-----------|+| `hotpotqa` | Multi-hop QA requiring reasoning over multiple Wikipedia passages | 100 |+| `natural_questions` | Real Google search questions with Wikipedia answers | 100 |+| `triviaqa` | Large-scale trivia QA with evidence documents | 100 |+| `msmarco` | Real Bing search queries with relevant passages | 100 |+| `squad` | SQuAD v2 reading comprehension with extractive answers | 100 |++### Long Context (2 datasets)++| Dataset | Description | Default N |+|---------|-------------|-----------|+| `longbench` | Long context understanding (4K-128K tokens) | 50 |+| `narrativeqa` | Story comprehension requiring narrative understanding | 100 |++### Tool Use (3 datasets)++| Dataset | Description | Default N |+|---------|-------------|-----------|+| `bfcl` | Berkeley Function Calling Leaderboard - API schemas | 100 |+| `toolbench` | Real-world API tool usage scenarios | 100 |+| `tool_outputs` | Built-in realistic tool outputs (JSON, logs, etc.) | 8 |++### Code (2 datasets)++| Dataset | Description | Default N |+|---------|-------------|-----------|+| `codesearchnet` | Code snippets with natural language descriptions | 100 |+| `humaneval` | Hand-crafted programming problems (OpenAI) | 164 |++## Metrics++### F1 Score+Token-level overlap between original and compressed responses.+```+F1 = 2 * (precision * recall) / (precision + recall)+```+A score of 1.0 means identical tokens, 0.0 means no overlap.++### Semantic Similarity+Cosine similarity between sentence embeddings (using `all-MiniLM-L6-v2`).+Range: 0.0 to 1.0, where 1.0 = semantically identical.++### Exact Match+Boolean indicating if responses are identical after normalization.++### Ground Truth Match+Checks if the compressed response contains the known correct answer.++### Accuracy Preserved+Overall verdict: `True` if ANY of:+- F1 > 0.7+- Semantic Similarity > 0.85+- Contains Ground Truth++## Programmatic Usage++### Loading Datasets++```python+from headroom.evals import (+ load_hotpotqa,+ load_natural_questions,+ load_triviaqa,+ load_msmarco,+ load_squad,+ load_longbench,+ load_narrativeqa,+ load_bfcl,+ load_toolbench,+ load_codesearchnet,+ load_humaneval,+ load_tool_output_samples,+ load_dataset_by_name,+ list_available_datasets,+)++# Load specific dataset+suite = load_hotpotqa(n=50)++# Load by name (useful for dynamic loading)+suite = load_dataset_by_name("msmarco", n=100)++# List all datasets by category+datasets = list_available_datasets()+# {'rag': ['hotpotqa', 'natural_questions', ...], 'tool_use': [...], ...}+```++### Custom Datasets++```python+from headroom.evals import EvalCase, EvalSuite, load_custom_dataset++# Create custom cases+cases = [+ EvalCase(+ id="my_case_001",+ context="Long document content here...",+ query="What is the main point?",+ ground_truth="The main point is X",+ metadata={"source": "my_data"},+ ),+]+suite = EvalSuite(name="MyDataset", cases=cases)++# Or load from JSONL file+suite = load_custom_dataset("my_data.jsonl")+```++JSONL format:+```json+{"id": "case_001", "context": "...", "query": "...", "ground_truth": "..."}+{"id": "case_002", "context": "...", "query": "...", "ground_truth": "..."}+```++### Configuring the Runner++```python+from headroom.evals import BeforeAfterRunner, LLMConfig+from headroom.transforms import SmartCrusherConfig++runner = BeforeAfterRunner(+ llm_config=LLMConfig(+ provider="anthropic", # or "openai", "ollama"+ model="claude-sonnet-4-20250514",+ temperature=0.0, # Deterministic for reproducibility+ max_tokens=1024,+ ),+ crusher_config=SmartCrusherConfig(+ target_ratio=0.5,+ preserve_structure=True,+ ),+ use_semantic_similarity=True, # Requires sentence-transformers+)+```++### Computing Metrics Directly++```python+from headroom.evals import (+ compute_f1,+ compute_exact_match,+ compute_semantic_similarity,+ compute_answer_equivalence,+ compute_rouge_l,+ compute_information_recall,+)++# Compare two responses+f1 = compute_f1("The answer is 42", "The answer is forty-two")+exact = compute_exact_match("yes", "YES") # True (normalized)+semantic = compute_semantic_similarity("The cat sat", "A feline was sitting")++# Comprehensive equivalence check+result = compute_answer_equivalence(+ response_a="The capital is Paris",+ response_b="Paris is the capital",+ ground_truth="Paris",+)+# {'exact_match': False, 'f1_score': 0.8, 'semantic_similarity': 0.95, 'equivalent': True}++# Information recall (check if facts survived compression)+recall = compute_information_recall(+ original_context="John Smith was born in 1990 in New York",+ compressed_context="J. Smith born 1990 NY",+ probe_facts=["John Smith", "1990", "New York"],+)+# {'recall': 0.67, 'facts_lost': ['John Smith']}+```++## CI/CD Integration++```bash+# Run quick eval and fail if accuracy < 90%+python -m headroom.evals quick -n 10++# Exit code: 0 if accuracy >= 90%, 1 otherwise+```++```yaml+# GitHub Actions example+- name: Run Compression Evals+ run: |+ pip install headroom-ai[evals]+ python -m headroom.evals quick -n 20+ env:+ ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}+```++## Environment Variables++- `ANTHROPIC_API_KEY` - Required for Anthropic Claude models+- `OPENAI_API_KEY` - Required for OpenAI models+- No key needed for Ollama (local models)++## Interpretation Guide++### Ideal Results+```+Accuracy Preservation: 95%++Average F1: 0.85++Average Semantic Similarity: 0.90++Compression Ratio: 40-70%+```++### Warning Signs+- Accuracy < 90% → Compression may be too aggressive+- F1 < 0.7 → Significant token loss+- Semantic Similarity < 0.85 → Meaning drift++### Troubleshooting+1. **Low accuracy on specific dataset**: Try adjusting `target_ratio` in SmartCrusherConfig+2. **Semantic similarity unavailable**: Install `sentence-transformers`+3. **Dataset loading fails**: Ensure `datasets` package is installed++## Architecture++```+headroom/evals/+├── __init__.py # Public API exports+├── __main__.py # CLI entry point+├── core.py # EvalCase, EvalResult, EvalSuite+├── datasets.py # Dataset loaders (HF + built-in)+├── metrics.py # F1, semantic similarity, etc.+├── runners/+│ ├── __init__.py+│ └── before_after.py # BeforeAfterRunner+└── reports/+ └── __init__.py+```diff --git a/headroom/evals/__init__.py b/headroom/evals/__init__.pynew file mode 100644index 000000000..801068f04--- /dev/null+++ b/headroom/evals/__init__.py@@ -0,0 +1,101 @@+"""Headroom Evaluation Framework.++Prove that compression doesn't impact LLM accuracy through:+1. Before/After comparisons on identical queries+2. Ground truth benchmarks (HotpotQA, BFCL, SQuAD, etc.)+3. Information retrieval probes+4. Statistical significance testing++Install with: pip install headroom-ai[evals]++Quick start:+ from headroom.evals import run_quick_eval+ results = run_quick_eval(n_samples=5)+ print(results.summary())++Available datasets:+ - RAG: hotpotqa, natural_questions, triviaqa, msmarco, squad+ - Long Context: longbench, narrativeqa+ - Tool Use: bfcl, toolbench, tool_outputs+ - Code: codesearchnet, humaneval+"""++from headroom.evals.core import (+ CompressionEvaluator,+ EvalCase,+ EvalMode,+ EvalResult,+ EvalSuite,+ EvalSuiteResult,+)+from headroom.evals.datasets import (+ DATASET_REGISTRY,+ list_available_datasets,+ load_bfcl,+ load_codesearchnet,+ load_custom_dataset,+ load_dataset_by_name,+ load_hotpotqa,+ load_humaneval,+ load_longbench,+ load_msmarco,+ load_narrativeqa,+ load_natural_questions,+ load_squad,+ load_tool_output_samples,+ load_toolbench,+ load_triviaqa,+)+from headroom.evals.metrics import (+ compute_answer_equivalence,+ compute_exact_match,+ compute_f1,+ compute_information_recall,+ compute_rouge_l,+ compute_semantic_similarity,+)+from headroom.evals.runners.before_after import (+ BeforeAfterRunner,+ LLMConfig,+ run_quick_eval,+)+from headroom.transforms.content_router import ContentRouterConfig++__all__ = [+ # Core classes+ "EvalCase",+ "EvalResult",+ "EvalSuite",+ "EvalSuiteResult",+ "EvalMode",+ "CompressionEvaluator",+ # Runner+ "BeforeAfterRunner",+ "LLMConfig",+ "ContentRouterConfig",+ "run_quick_eval",+ # Metrics+ "compute_f1",+ "compute_exact_match",+ "compute_semantic_similarity",+ "compute_answer_equivalence",+ "compute_rouge_l",+ "compute_information_recall",+ # Dataset loaders+ "load_hotpotqa",+ "load_natural_questions",+ "load_triviaqa",+ "load_msmarco",+ "load_squad",+ "load_longbench",+ "load_narrativeqa",+ "load_bfcl",+ "load_toolbench",+ "load_codesearchnet",+ "load_humaneval",+ "load_tool_output_samples",+ "load_custom_dataset",+ "load_dataset_by_name",+ "list_available_datasets",+ "DATASET_REGISTRY",+]diff --git a/headroom/evals/__main__.py b/headroom/evals/__main__.pynew file mode 100644index 000000000..99d108d94--- /dev/null+++ b/headroom/evals/__main__.py@@ -0,0 +1,393 @@+"""CLI entry point for Headroom evaluation framework.++Usage:+ python -m headroom.evals quick # Quick sanity check (5 samples)+ python -m headroom.evals benchmark # Full benchmark suite+ python -m headroom.evals list # List available datasets+ python -m headroom.evals --help # Show help++Install dependencies:+ pip install headroom-ai[evals]+"""++from __future__ import annotations++import argparse+import sys+from pathlib import Path+from typing import TYPE_CHECKING++if TYPE_CHECKING:+ from headroom.evals.core import EvalResult+++def cmd_quick(args: argparse.Namespace) -> None:+ """Run quick sanity check."""+ from headroom.evals.runners.before_after import run_quick_eval++ results = run_quick_eval(+ n_samples=args.n,+ provider=args.provider,+ model=args.model,+ )++ if args.output:+ results.save(args.output)+ print(f"\nResults saved to: {args.output}")++ # Exit with error if accuracy not preserved+ if results.accuracy_preservation_rate < 0.9:+ print("\nWARNING: Accuracy preservation rate below 90%!")+ sys.exit(1)+++def cmd_list(args: argparse.Namespace) -> None:+ """List available datasets."""+ from headroom.evals.datasets import DATASET_REGISTRY, list_available_datasets++ print("\n" + "=" * 60)+ print("AVAILABLE EVALUATION DATASETS")+ print("=" * 60)++ by_category = list_available_datasets()++ for category, datasets in sorted(by_category.items()):+ category_title = category.replace("_", " ").title()+ print(f"\n{category_title}:")+ print("-" * 40)+ for name in sorted(datasets):+ info = DATASET_REGISTRY[name]+ default_n = info.get("default_n", "N/A")+ print(f" {name:20} (n={default_n})")+ print(f" {info['description']}")++ print("\n" + "=" * 60)+ print("Usage: python -m headroom.evals benchmark --dataset <name> -n <samples>")+ print("=" * 60 + "\n")+++def cmd_benchmark(args: argparse.Namespace) -> None:+ """Run full benchmark suite."""+ from headroom.evals.datasets import (+ DATASET_REGISTRY,+ load_dataset_by_name,+ load_tool_output_samples,+ )+ from headroom.evals.runners.before_after import BeforeAfterRunner, LLMConfig++ print("=" * 60)+ print("HEADROOM COMPRESSION ACCURACY BENCHMARK")+ print("=" * 60)++ runner = BeforeAfterRunner(+ llm_config=LLMConfig(provider=args.provider, model=args.model),+ use_semantic_similarity=args.semantic,+ )++ all_results = []++ # Determine which datasets to run+ if args.dataset == "all":+ # Run all datasets+ dataset_names = list(DATASET_REGISTRY.keys())+ elif args.dataset == "quick":+ # Quick subset: tool_outputs + a few samples from each category+ dataset_names = ["tool_outputs", "squad", "bfcl"]+ elif args.dataset in DATASET_REGISTRY:+ dataset_names = [args.dataset]+ else:+ # Check if it's a category+ from headroom.evals.datasets import list_available_datasets++ by_category = list_available_datasets()+ if args.dataset in by_category:+ dataset_names = by_category[args.dataset]+ else:+ print(f"Unknown dataset or category: {args.dataset}")+ print(f"Available: {', '.join(DATASET_REGISTRY.keys())}")+ print(f"Categories: {', '.join(by_category.keys())}")+ sys.exit(1)++ for name in dataset_names:+ print(f"\n--- Loading {name} ---")+ try:+ if name == "tool_outputs":+ suite = load_tool_output_samples()+ else:+ suite = load_dataset_by_name(name, n=args.n)++ print(f"Loaded {len(suite)} cases")++ def progress(current: int, total: int, result: EvalResult) -> None:+ status = "PASS" if result.accuracy_preserved else "FAIL"+ compression = f"{result.compression_ratio:.0%}"+ print(f" [{current}/{total}] {status} F1={result.f1_score:.2f} Comp={compression}")++ result = runner.run(suite, progress_callback=progress)+ all_results.append(result)+ print(f"\n{result.summary()}")++ except ImportError as e:+ print(f"Skipping {name}: {e}")+ print(" (Install with: pip install headroom-ai[evals])")+ except Exception as e:+ print(f"Error loading {name}: {e}")++ if not all_results:+ print("\nNo datasets were successfully loaded.")+ sys.exit(1)++ # Aggregate all results+ print("\n" + "=" * 60)+ print("OVERALL RESULTS")+ print("=" * 60)++ total_cases = sum(r.total_cases for r in all_results)+ total_passed = sum(r.passed_cases for r in all_results)+ total_original = sum(r.total_original_tokens for r in all_results)+ total_compressed = sum(r.total_compressed_tokens for r in all_results)++ if total_cases > 0:+ print(f"""+Total Cases: {total_cases}+Passed: {total_passed} ({total_passed / total_cases:.1%})+Failed: {total_cases - total_passed}+Total Original Tokens: {total_original:,}+Total Compressed: {total_compressed:,}+Tokens Saved: {total_original - total_compressed:,} ({(total_original - total_compressed) / total_original:.1%})+""")++ if args.output:+ # Save combined results+ import json++ output = {+ "suites": [r.to_dict() for r in all_results],+ "totals": {+ "cases": total_cases,+ "passed": total_passed,+ "accuracy_rate": total_passed / total_cases if total_cases else 0,+ "tokens_original": total_original,+ "tokens_compressed": total_compressed,+ },+ }+ with open(args.output, "w") as f:+ json.dump(output, f, indent=2)+ print(f"Results saved to: {args.output}")+++def cmd_report(args: argparse.Namespace) -> None:+ """Generate HTML report from results."""+ import json++ if not Path(args.input).exists():+ print(f"Error: Input file not found: {args.input}")+ sys.exit(1)++ with open(args.input) as f:+ data = json.load(f)++ # Generate HTML report+ html = f"""<!DOCTYPE html>+<html>+<head>+ <title>Headroom Evaluation Report</title>+ <style>+ body {{+ font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;+ max-width: 1000px;+ margin: 40px auto;+ padding: 20px;+ background: #f8f9fa;+ }}+ h1 {{ color: #1a1a2e; margin-bottom: 10px; }}+ h2 {{ color: #16213e; margin-top: 30px; }}+ .summary {{+ background: white;+ padding: 24px;+ border-radius: 12px;+ margin: 20px 0;+ box-shadow: 0 2px 8px rgba(0,0,0,0.08);+ }}+ .summary h2 {{ margin-top: 0; }}+ .metric {{+ display: inline-block;+ margin: 10px 20px 10px 0;+ padding: 12px 20px;+ background: #f0f0f0;+ border-radius: 8px;+ }}+ .metric .value {{ font-size: 24px; font-weight: bold; }}+ .metric .label {{ font-size: 12px; color: #666; }}+ .pass {{ color: #22c55e; }}+ .fail {{ color: #ef4444; }}+ table {{+ width: 100%;+ border-collapse: collapse;+ margin: 20px 0;+ background: white;+ border-radius: 8px;+ overflow: hidden;+ box-shadow: 0 2px 8px rgba(0,0,0,0.08);+ }}+ th, td {{+ padding: 14px 16px;+ text-align: left;+ border-bottom: 1px solid #eee;+ }}+ th {{ background: #f8f9fa; font-weight: 600; }}+ tr:hover {{ background: #f8f9fa; }}+ .badge {{+ display: inline-block;+ padding: 4px 10px;+ border-radius: 20px;+ font-size: 12px;+ font-weight: 600;+ }}+ .badge.pass {{ background: #dcfce7; color: #166534; }}+ .badge.fail {{ background: #fee2e2; color: #991b1b; }}+ footer {{+ margin-top: 40px;+ padding-top: 20px;+ border-top: 1px solid #ddd;+ color: #666;+ font-size: 14px;+ }}+ </style>+</head>+<body>+ <h1>Headroom Compression Accuracy Report</h1>+ <p>Proving compression preserves LLM accuracy through before/after comparison.</p>++ <div class="summary">+ <h2>Overall Results</h2>+ <div class="metric">+ <div class="value">{data["totals"]["cases"]}</div>+ <div class="label">Total Cases</div>+ </div>+ <div class="metric">+ <div class="value {("pass" if data["totals"]["accuracy_rate"] > 0.9 else "fail")}">{data["totals"]["accuracy_rate"]:.1%}</div>+ <div class="label">Accuracy Preserved</div>+ </div>+ <div class="metric">+ <div class="value">{data["totals"]["tokens_original"] - data["totals"]["tokens_compressed"]:,}</div>+ <div class="label">Tokens Saved</div>+ </div>+ <div class="metric">+ <div class="value">{(data["totals"]["tokens_original"] - data["totals"]["tokens_compressed"]) / data["totals"]["tokens_original"]:.1%}</div>+ <div class="label">Compression Rate</div>+ </div>+ </div>+"""++ for suite in data.get("suites", []):+ pass_rate = suite["passed_cases"] / suite["total_cases"] if suite["total_cases"] > 0 else 0+ html += f"""+ <h2>{suite["suite_name"]}</h2>+ <p>+ <span class="badge {"pass" if pass_rate > 0.9 else "fail"}">{suite["passed_cases"]}/{suite["total_cases"]} passed</span>+ | F1: {suite["avg_f1_score"]:.3f}+ | Compression: {suite["avg_compression_ratio"]:.1%}+ </p>++ <table>+ <tr><th>Case ID</th><th>Status</th><th>F1 Score</th><th>Compression</th></tr>+"""+ for result in suite.get("results", [])[:20]: # Limit to 20 per suite+ status = "PASS" if result["accuracy_preserved"] else "FAIL"+ status_class = "pass" if result["accuracy_preserved"] else "fail"+ html += f""" <tr>+ <td>{result["case_id"]}</td>+ <td><span class="badge {status_class}">{status}</span></td>+ <td>{result["f1_score"]:.3f}</td>+ <td>{result["compression_ratio"]:.1%}</td>+ </tr>+"""+ html += " </table>\n"++ html += """+ <footer>+ <p>Generated by <strong>Headroom Evaluation Framework</strong></p>+ <p>Install: <code>pip install headroom-ai[evals]</code></p>+ </footer>+</body>+</html>+"""++ output_path = args.output or args.input.replace(".json", ".html")+ with open(output_path, "w") as f:+ f.write(html)+ print(f"Report generated: {output_path}")+++def main() -> None:+ parser = argparse.ArgumentParser(+ description="Headroom Evaluation Framework - Prove compression preserves accuracy",+ formatter_class=argparse.RawDescriptionHelpFormatter,+ epilog="""+Examples:+ python -m headroom.evals quick # Quick 5-sample check+ python -m headroom.evals quick -n 10 # Quick 10-sample check+ python -m headroom.evals list # List available datasets+ python -m headroom.evals benchmark # Run tool_outputs benchmark+ python -m headroom.evals benchmark --dataset hotpotqa -n 50+ python -m headroom.evals benchmark --dataset rag # Run all RAG datasets+ python -m headroom.evals benchmark --dataset all # Run ALL datasets+ python -m headroom.evals report -i results.json # Generate HTML report++Available datasets by category:+ RAG: hotpotqa, natural_questions, triviaqa, msmarco, squad+ Long Context: longbench, narrativeqa+ Tool Use: bfcl, toolbench, tool_outputs+ Code: codesearchnet, humaneval++Install dependencies:+ pip install headroom-ai[evals]+""",+ )++ subparsers = parser.add_subparsers(dest="command", help="Command to run")++ # Quick command+ quick_parser = subparsers.add_parser("quick", help="Run quick sanity check")+ quick_parser.add_argument("-n", type=int, default=5, help="Number of samples")+ quick_parser.add_argument("--provider", default="anthropic", help="LLM provider")+ quick_parser.add_argument("--model", default="claude-sonnet-4-20250514", help="Model to use")+ quick_parser.add_argument("-o", "--output", help="Output file for results (JSON)")+ quick_parser.set_defaults(func=cmd_quick)++ # List command+ list_parser = subparsers.add_parser("list", help="List available datasets")+ list_parser.set_defaults(func=cmd_list)++ # Benchmark command+ bench_parser = subparsers.add_parser("benchmark", help="Run full benchmark suite")+ bench_parser.add_argument("-n", type=int, default=20, help="Samples per dataset")+ bench_parser.add_argument(+ "--dataset",+ default="tool_outputs",+ help="Dataset name, category (rag, tool_use, code, long_context), 'quick', or 'all'",+ )+ bench_parser.add_argument("--provider", default="anthropic", help="LLM provider")+ bench_parser.add_argument("--model", default="claude-sonnet-4-20250514", help="Model to use")+ bench_parser.add_argument("--semantic", action="store_true", help="Enable semantic similarity")+ bench_parser.add_argument("-o", "--output", help="Output file for results (JSON)")+ bench_parser.set_defaults(func=cmd_benchmark)++ # Report command+ report_parser = subparsers.add_parser("report", help="Generate HTML report")+ report_parser.add_argument("-i", "--input", required=True, help="Input JSON results file")+ report_parser.add_argument("-o", "--output", help="Output HTML file")+ report_parser.set_defaults(func=cmd_report)++ args = parser.parse_args()++ if not args.command:+ parser.print_help()+ sys.exit(1)++ args.func(args)+++if __name__ == "__main__":+ main()diff --git a/headroom/evals/core.py b/headroom/evals/core.pynew file mode 100644index 000000000..dca3cdc85--- /dev/null+++ b/headroom/evals/core.py@@ -0,0 +1,412 @@+"""Core evaluation infrastructure for Headroom.++This module provides the foundation for proving that compression+preserves LLM accuracy through rigorous A/B testing.+"""++from __future__ import annotations++import json+import time+from collections.abc import Callable, Iterator+from dataclasses import dataclass, field+from datetime import datetime+from enum import Enum+from pathlib import Path+from typing import Any++from headroom.transforms.smart_crusher import SmartCrusher, SmartCrusherConfig+++class EvalMode(Enum):+ """Evaluation mode."""++ BEFORE_AFTER = "before_after" # Compare original vs compressed+ GROUND_TRUTH = "ground_truth" # Compare against known answer+ RETRIEVAL = "retrieval" # Test information retrieval+ AGENTIC = "agentic" # Test agent task completion+++@dataclass+class EvalCase:+ """A single evaluation case.++ Attributes:+ id: Unique identifier for this case+ context: The original context (tool output, document, etc.)+ query: The question or task to perform+ ground_truth: Optional known correct answer+ metadata: Additional metadata (source, category, etc.)+ """++ id: str+ context: str+ query: str+ ground_truth: str | None = None+ metadata: dict[str, Any] = field(default_factory=dict)++ @classmethod+ def from_dict(cls, d: dict) -> EvalCase:+ return cls(+ id=d["id"],+ context=d["context"],+ query=d["query"],+ ground_truth=d.get("ground_truth"),+ metadata=d.get("metadata", {}),+ )++ def to_dict(self) -> dict:+ return {+ "id": self.id,+ "context": self.context,+ "query": self.query,+ "ground_truth": self.ground_truth,+ "metadata": self.metadata,+ }+++@dataclass+class EvalResult:+ """Result of a single evaluation.++ Captures both the original and compressed responses,+ along with metrics comparing them.+ """++ case_id: str+ mode: EvalMode++ # Context stats+ original_tokens: int+ compressed_tokens: int+ compression_ratio: float++ # Responses+ response_original: str+ response_compressed: str++ # Metrics+ exact_match: bool+ f1_score: float+ semantic_similarity: float | None = None+ contains_ground_truth: bool | None = None++ # Timing+ latency_original_ms: float = 0.0+ latency_compressed_ms: float = 0.0++ # Overall verdict+ accuracy_preserved: bool = True++ # Metadata+ error: str | None = None+ timestamp: str = field(default_factory=lambda: datetime.now().isoformat())++ def to_dict(self) -> dict:+ return {+ "case_id": self.case_id,+ "mode": self.mode.value,+ "original_tokens": self.original_tokens,+ "compressed_tokens": self.compressed_tokens,+ "compression_ratio": self.compression_ratio,+ "response_original": self.response_original[:500], # Truncate for storage+ "response_compressed": self.response_compressed[:500],+ "exact_match": self.exact_match,+ "f1_score": self.f1_score,+ "semantic_similarity": self.semantic_similarity,+ "contains_ground_truth": self.contains_ground_truth,+ "latency_original_ms": self.latency_original_ms,+ "latency_compressed_ms": self.latency_compressed_ms,+ "accuracy_preserved": self.accuracy_preserved,+ "error": self.error,+ "timestamp": self.timestamp,+ }+++@dataclass+class EvalSuiteResult:+ """Aggregated results from an evaluation suite."""++ suite_name: str+ total_cases: int+ passed_cases: int+ failed_cases: int++ # Aggregate metrics+ avg_compression_ratio: float+ avg_f1_score: float+ avg_semantic_similarity: float | None+ accuracy_preservation_rate: float++ # Token savings+ total_original_tokens: int+ total_compressed_tokens: int+ total_tokens_saved: int++ # Individual results+ results: list[EvalResult] = field(default_factory=list)++ # Metadata+ timestamp: str = field(default_factory=lambda: datetime.now().isoformat())+ duration_seconds: float = 0.0++ def to_dict(self) -> dict:+ return {+ "suite_name": self.suite_name,+ "total_cases": self.total_cases,+ "passed_cases": self.passed_cases,+ "failed_cases": self.failed_cases,+ "avg_compression_ratio": self.avg_compression_ratio,+ "avg_f1_score": self.avg_f1_score,+ "avg_semantic_similarity": self.avg_semantic_similarity,+ "accuracy_preservation_rate": self.accuracy_preservation_rate,+ "total_original_tokens": self.total_original_tokens,+ "total_compressed_tokens": self.total_compressed_tokens,+ "total_tokens_saved": self.total_tokens_saved,+ "timestamp": self.timestamp,+ "duration_seconds": self.duration_seconds,+ "results": [r.to_dict() for r in self.results],+ }++ def save(self, path: Path | str) -> None:+ """Save results to JSON file."""+ path = Path(path)+ path.parent.mkdir(parents=True, exist_ok=True)+ with open(path, "w") as f:+ json.dump(self.to_dict(), f, indent=2)++ def summary(self) -> str:+ """Generate human-readable summary."""+ lines = [+ f"=== {self.suite_name} ===",+ f"Cases: {self.passed_cases}/{self.total_cases} passed ({self.accuracy_preservation_rate:.1%})",+ f"Compression: {self.avg_compression_ratio:.1%} average",+ f"F1 Score: {self.avg_f1_score:.3f}",+ ]+ if self.avg_semantic_similarity is not None:+ lines.append(f"Semantic Similarity: {self.avg_semantic_similarity:.3f}")+ lines.append(+ f"Tokens: {self.total_original_tokens:,} → {self.total_compressed_tokens:,} "+ f"({self.total_tokens_saved:,} saved)"+ )+ return "\n".join(lines)+++class EvalSuite:+ """A collection of evaluation cases."""++ def __init__(self, name: str, cases: list[EvalCase] | None = None):+ self.name = name+ self.cases: list[EvalCase] = cases or []++ def add_case(self, case: EvalCase) -> None:+ self.cases.append(case)++ def __len__(self) -> int:+ return len(self.cases)++ def __iter__(self) -> Iterator[EvalCase]:+ return iter(self.cases)++ @classmethod+ def from_jsonl(cls, path: Path | str, name: str | None = None) -> EvalSuite:+ """Load suite from JSONL file."""+ path = Path(path)+ cases = []+ with open(path) as f:+ for line in f:+ if line.strip():+ cases.append(EvalCase.from_dict(json.loads(line)))+ return cls(name=name or path.stem, cases=cases)++ def to_jsonl(self, path: Path | str) -> None:+ """Save suite to JSONL file."""+ path = Path(path)+ path.parent.mkdir(parents=True, exist_ok=True)+ with open(path, "w") as f:+ for case in self.cases:+ f.write(json.dumps(case.to_dict()) + "\n")+++class CompressionEvaluator:+ """Main evaluator for compression accuracy.++ This is the core class that runs A/B comparisons between+ original and compressed contexts.+ """++ def __init__(+ self,+ llm_fn: Callable[[str, str], str] | None = None,+ crusher_config: SmartCrusherConfig | None = None,+ semantic_similarity_fn: Callable[[str, str], float] | None = None,+ ):+ """Initialize evaluator.++ Args:+ llm_fn: Function that takes (context, query) and returns response.+ If None, uses a mock for testing.+ crusher_config: Configuration for SmartCrusher.+ semantic_similarity_fn: Optional function to compute semantic similarity.+ """+ self.llm_fn = llm_fn or self._mock_llm+ self.crusher = SmartCrusher(config=crusher_config or SmartCrusherConfig())+ self.semantic_similarity_fn = semantic_similarity_fn++ def _mock_llm(self, context: str, query: str) -> str:+ """Mock LLM for testing without API calls."""+ # Return a response that includes key terms from context+ words = context.split()[:50]+ return f"Based on the context, {' '.join(words[:20])}..."++ def _estimate_tokens(self, text: str) -> int:+ """Estimate token count (roughly 4 chars per token)."""+ return len(text) // 4++ def evaluate_case(+ self,+ case: EvalCase,+ mode: EvalMode = EvalMode.BEFORE_AFTER,+ ) -> EvalResult:+ """Evaluate a single case.++ Runs the query against both original and compressed context,+ then compares the responses.+ """+ from headroom.evals.metrics import (+ compute_exact_match,+ compute_f1,+ )++ original_tokens = self._estimate_tokens(case.context)++ # Compress the context+ try:+ compressed_result = self.crusher.crush(case.context)+ compressed_context = compressed_result.compressed+ compressed_tokens = self._estimate_tokens(compressed_context)+ except Exception:+ # If compression fails, use original+ compressed_context = case.context+ compressed_tokens = original_tokens++ compression_ratio = 1 - (compressed_tokens / original_tokens) if original_tokens > 0 else 0++ # Run LLM with original context+ start = time.time()+ try:+ response_original = self.llm_fn(case.context, case.query)+ except Exception as e:+ response_original = f"ERROR: {e}"+ latency_original = (time.time() - start) * 1000++ # Run LLM with compressed context+ start = time.time()+ try:+ response_compressed = self.llm_fn(compressed_context, case.query)+ except Exception as e:+ response_compressed = f"ERROR: {e}"+ latency_compressed = (time.time() - start) * 1000++ # Compute metrics+ exact_match = compute_exact_match(response_original, response_compressed)+ f1_score = compute_f1(response_original, response_compressed)++ # Semantic similarity if available+ semantic_sim = None+ if self.semantic_similarity_fn:+ try:+ semantic_sim = self.semantic_similarity_fn(response_original, response_compressed)+ except Exception:+ pass++ # Check ground truth if available+ contains_ground_truth = None+ if case.ground_truth:+ gt_lower = case.ground_truth.lower()+ contains_ground_truth = (+ gt_lower in response_compressed.lower()+ or compute_f1(response_compressed, case.ground_truth) > 0.5+ )++ # Determine if accuracy is preserved+ # Criteria: F1 > 0.8 OR semantic similarity > 0.9 OR contains ground truth+ accuracy_preserved = (+ f1_score > 0.8+ or (semantic_sim is not None and semantic_sim > 0.9)+ or (contains_ground_truth is True)+ )++ return EvalResult(+ case_id=case.id,+ mode=mode,+ original_tokens=original_tokens,+ compressed_tokens=compressed_tokens,+ compression_ratio=compression_ratio,+ response_original=response_original,+ response_compressed=response_compressed,+ exact_match=exact_match,+ f1_score=f1_score,+ semantic_similarity=semantic_sim,+ contains_ground_truth=contains_ground_truth,+ latency_original_ms=latency_original,+ latency_compressed_ms=latency_compressed,+ accuracy_preserved=accuracy_preserved,+ )++ def evaluate_suite(+ self,+ suite: EvalSuite,+ mode: EvalMode = EvalMode.BEFORE_AFTER,+ progress_callback: Callable[[int, int], None] | None = None,+ ) -> EvalSuiteResult:+ """Evaluate an entire suite of cases.++ Args:+ suite: The evaluation suite to run+ mode: Evaluation mode+ progress_callback: Optional callback(current, total) for progress++ Returns:+ Aggregated results with statistics+ """+ start_time = time.time()+ results: list[EvalResult] = []++ for i, case in enumerate(suite):+ result = self.evaluate_case(case, mode)+ results.append(result)++ if progress_callback:+ progress_callback(i + 1, len(suite))++ # Aggregate metrics+ passed = sum(1 for r in results if r.accuracy_preserved)+ failed = len(results) - passed++ total_original = sum(r.original_tokens for r in results)+ total_compressed = sum(r.compressed_tokens for r in results)++ avg_compression = sum(r.compression_ratio for r in results) / len(results) if results else 0+ avg_f1 = sum(r.f1_score for r in results) / len(results) if results else 0++ semantic_sims = [+ r.semantic_similarity for r in results if r.semantic_similarity is not None+ ]+ avg_semantic = sum(semantic_sims) / len(semantic_sims) if semantic_sims else None++ return EvalSuiteResult(+ suite_name=suite.name,+ total_cases=len(results),+ passed_cases=passed,+ failed_cases=failed,+ avg_compression_ratio=avg_compression,+ avg_f1_score=avg_f1,+ avg_semantic_similarity=avg_semantic,+ accuracy_preservation_rate=passed / len(results) if results else 0,+ total_original_tokens=total_original,+ total_compressed_tokens=total_compressed,+ total_tokens_saved=total_original - total_compressed,+ results=results,+ duration_seconds=time.time() - start_time,+ )diff --git a/headroom/evals/datasets.py b/headroom/evals/datasets.pynew file mode 100644index 000000000..ab2a0e68c--- /dev/null+++ b/headroom/evals/datasets.py@@ -0,0 +1,1260 @@+"""Dataset loaders for evaluation benchmarks.++Loads real data from established sources for comprehensive compression evaluation:++RAG/Retrieval:+- HotpotQA: Multi-hop QA with Wikipedia passages+- Natural Questions: Google's real search questions+- TriviaQA: Large-scale trivia QA+- MS MARCO: Microsoft's real search queries+- SQuAD: Reading comprehension++Long Context:+- LongBench: Long context understanding benchmark+- NarrativeQA: Story comprehension++Tool Use:+- BFCL: Berkeley Function Calling Leaderboard+- ToolBench: API tool usage benchmark++Code:+- CodeSearchNet: Code search and understanding+- HumanEval: Code generation benchmark++Custom:+- Tool output samples: Built-in realistic tool outputs+"""++from __future__ import annotations++import json+from pathlib import Path+from typing import Any++from headroom.evals.core import EvalCase, EvalSuite+++def _check_datasets_installed() -> None:+ """Check if HuggingFace datasets is installed."""+ try:+ import datasets # noqa: F401+ except ImportError as e:+ raise ImportError(+ "HuggingFace datasets required for this loader. "+ "Install with: pip install headroom-ai[evals]"+ ) from e+++# =============================================================================+# RAG / RETRIEVAL DATASETS+# =============================================================================+++def load_hotpotqa(+ n: int = 100,+ split: str = "validation",+) -> EvalSuite:+ """Load HotpotQA dataset for multi-hop QA evaluation.++ HotpotQA contains questions requiring reasoning over multiple+ Wikipedia passages, with verified ground truth answers.++ Dataset: https://huggingface.co/datasets/hotpotqa/hotpot_qa++ Args:+ n: Number of samples to load+ split: Dataset split ("train", "validation")++ Returns:+ EvalSuite with HotpotQA cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ ds = load_dataset("hotpotqa/hotpot_qa", "fullwiki", split=split)++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if i >= n:+ break++ # Build context from supporting facts+ context_parts = []+ for title, sentences in zip(item["context"]["title"], item["context"]["sentences"]):+ context_parts.append(f"## {title}\n" + "\n".join(sentences))++ context = "\n\n".join(context_parts)++ cases.append(+ EvalCase(+ id=f"hotpot_{i}",+ context=context,+ query=item["question"],+ ground_truth=item["answer"],+ metadata={+ "source": "HotpotQA",+ "type": item.get("type", "unknown"),+ "level": item.get("level", "unknown"),+ },+ )+ )++ return EvalSuite(name="HotpotQA", cases=cases)+++def load_natural_questions(+ n: int = 100,+ split: str = "validation",+) -> EvalSuite:+ """Load Google's Natural Questions dataset.++ Real questions from Google search with long-form Wikipedia answers.+ Excellent for testing compression on factual retrieval.++ Dataset: https://huggingface.co/datasets/google-research-datasets/natural_questions++ Args:+ n: Number of samples to load+ split: Dataset split ("train", "validation")++ Returns:+ EvalSuite with Natural Questions cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ ds = load_dataset("google-research-datasets/natural_questions", "default", split=split)++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if len(cases) >= n:+ break++ # Get document text (can be very long)+ doc_tokens = item.get("document", {}).get("tokens", {})+ if not doc_tokens:+ continue++ tokens = doc_tokens.get("token", [])+ is_html = doc_tokens.get("is_html", [])++ # Filter out HTML tokens, keep text only+ text_tokens = [t for t, h in zip(tokens, is_html) if not h]+ context = " ".join(text_tokens[:2000]) # Limit context size++ if not context.strip():+ continue++ # Get question+ question = item.get("question", {}).get("text", "")+ if not question:+ continue++ # Get short answer if available+ annotations = item.get("annotations", {})+ short_answers = annotations.get("short_answers", [[]])+ ground_truth = None+ if short_answers and short_answers[0]:+ first_answer = short_answers[0][0]+ start = first_answer.get("start_token", 0)+ end = first_answer.get("end_token", 0)+ if end > start:+ ground_truth = " ".join(tokens[start:end])++ cases.append(+ EvalCase(+ id=f"nq_{i}",+ context=context,+ query=question,+ ground_truth=ground_truth,+ metadata={+ "source": "Natural Questions",+ "has_short_answer": ground_truth is not None,+ },+ )+ )++ return EvalSuite(name="Natural_Questions", cases=cases)+++def load_triviaqa(+ n: int = 100,+ split: str = "validation",+ subset: str = "rc",+) -> EvalSuite:+ """Load TriviaQA dataset.++ Large-scale trivia QA with evidence documents.+ Good for testing factoid question answering.++ Dataset: https://huggingface.co/datasets/trivia_qa++ Args:+ n: Number of samples to load+ split: Dataset split ("train", "validation")+ subset: Dataset subset ("rc" for reading comprehension, "unfiltered")++ Returns:+ EvalSuite with TriviaQA cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ ds = load_dataset("trivia_qa", subset, split=split)++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if len(cases) >= n:+ break++ # Get search results as context+ search_results = item.get("search_results", {})+ search_contexts = search_results.get("search_context", [])++ if not search_contexts:+ # Try entity pages+ entity_pages = item.get("entity_pages", {})+ wiki_contexts = entity_pages.get("wiki_context", [])+ if wiki_contexts:+ context = "\n\n".join(wiki_contexts[:3]) # Top 3 wiki pages+ else:+ continue+ else:+ context = "\n\n".join(search_contexts[:5]) # Top 5 search results++ if not context.strip():+ continue++ question = item.get("question", "")+ if not question:+ continue++ # Ground truth answer+ answer = item.get("answer", {})+ ground_truth = answer.get("value") or answer.get("normalized_value")++ cases.append(+ EvalCase(+ id=f"triviaqa_{i}",+ context=context[:10000], # Limit context size+ query=question,+ ground_truth=ground_truth,+ metadata={+ "source": "TriviaQA",+ "subset": subset,+ "aliases": answer.get("aliases", []),+ },+ )+ )++ return EvalSuite(name=f"TriviaQA_{subset}", cases=cases)+++def load_msmarco(+ n: int = 100,+ split: str = "validation",+) -> EvalSuite:+ """Load MS MARCO passage ranking dataset.++ Real Bing search queries with relevant passages.+ Excellent for testing RAG context compression.++ Dataset: https://huggingface.co/datasets/microsoft/ms_marco++ Args:+ n: Number of samples to load+ split: Dataset split ("train", "validation", "test")++ Returns:+ EvalSuite with MS MARCO cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ ds = load_dataset("microsoft/ms_marco", "v2.1", split=split)++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if len(cases) >= n:+ break++ # Build context from passages+ passages = item.get("passages", {})+ passage_texts = passages.get("passage_text", [])+ is_selected = passages.get("is_selected", [])++ if not passage_texts:+ continue++ # Combine passages as context+ context_parts = []+ for j, (text, selected) in enumerate(zip(passage_texts, is_selected)):+ prefix = "[RELEVANT] " if selected else ""+ context_parts.append(f"{prefix}Passage {j + 1}: {text}")++ context = "\n\n".join(context_parts)++ query = item.get("query", "")+ if not query:+ continue++ # Get answers+ answers = item.get("answers", [])+ ground_truth = answers[0] if answers else None++ cases.append(+ EvalCase(+ id=f"msmarco_{i}",+ context=context,+ query=query,+ ground_truth=ground_truth,+ metadata={+ "source": "MS_MARCO",+ "query_type": item.get("query_type", "unknown"),+ "num_passages": len(passage_texts),+ },+ )+ )++ return EvalSuite(name="MS_MARCO", cases=cases)+++def load_squad(+ n: int = 100,+ split: str = "validation",+) -> EvalSuite:+ """Load SQuAD v2 dataset for reading comprehension.++ SQuAD contains paragraphs with questions and extractive answers.++ Dataset: https://huggingface.co/datasets/rajpurkar/squad_v2++ Args:+ n: Number of samples to load+ split: Dataset split++ Returns:+ EvalSuite with SQuAD cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ ds = load_dataset("rajpurkar/squad_v2", split=split)++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if len(cases) >= n:+ break++ # Skip unanswerable questions+ if not item["answers"]["text"]:+ continue++ cases.append(+ EvalCase(+ id=f"squad_{i}",+ context=item["context"],+ query=item["question"],+ ground_truth=item["answers"]["text"][0], # First answer+ metadata={+ "source": "SQuAD_v2",+ "title": item.get("title", ""),+ },+ )+ )++ return EvalSuite(name="SQuAD_v2", cases=cases)+++# =============================================================================+# LONG CONTEXT DATASETS+# =============================================================================+++def load_longbench(+ n: int = 50,+ task: str = "qasper",+) -> EvalSuite:+ """Load LongBench dataset for long context evaluation.++ LongBench tests understanding of very long documents (4K-128K tokens).+ Critical for testing compression on long contexts.++ Dataset: https://huggingface.co/datasets/THUDM/LongBench++ Available tasks:+ - qasper: Scientific paper QA+ - multifieldqa_en: Multi-field QA+ - narrativeqa: Story comprehension+ - gov_report: Government report summarization+ - qmsum: Meeting summarization+ - multi_news: Multi-document summarization++ Args:+ n: Number of samples to load+ task: LongBench task name++ Returns:+ EvalSuite with LongBench cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ try:+ ds = load_dataset("THUDM/LongBench", task, split="test")+ except Exception as e:+ raise ValueError(f"Failed to load LongBench task '{task}': {e}") from e++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if i >= n:+ break++ context = item.get("context", "")+ if not context:+ continue++ query = item.get("input", "")+ if not query:+ continue++ # Ground truth (list of answers for some tasks)+ answers = item.get("answers", [])+ ground_truth = answers[0] if answers else None++ cases.append(+ EvalCase(+ id=f"longbench_{task}_{i}",+ context=context,+ query=query,+ ground_truth=ground_truth,+ metadata={+ "source": "LongBench",+ "task": task,+ "context_length": len(context),+ },+ )+ )++ return EvalSuite(name=f"LongBench_{task}", cases=cases)+++def load_narrativeqa(+ n: int = 100,+ split: str = "test",+) -> EvalSuite:+ """Load NarrativeQA dataset for story comprehension.++ Questions about books and movie scripts requiring understanding+ of narrative structure and long-range dependencies.++ Dataset: https://huggingface.co/datasets/deepmind/narrativeqa++ Args:+ n: Number of samples to load+ split: Dataset split++ Returns:+ EvalSuite with NarrativeQA cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ ds = load_dataset("deepmind/narrativeqa", split=split)++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if len(cases) >= n:+ break++ # Get summary as context (full text is very long)+ document = item.get("document", {})+ summary = document.get("summary", {})+ context = summary.get("text", "")++ if not context:+ continue++ question = item.get("question", {}).get("text", "")+ if not question:+ continue++ # Multiple reference answers+ answers = item.get("answers", [])+ answer_texts = [a.get("text", "") for a in answers if a.get("text")]+ ground_truth = answer_texts[0] if answer_texts else None++ cases.append(+ EvalCase(+ id=f"narrativeqa_{i}",+ context=context,+ query=question,+ ground_truth=ground_truth,+ metadata={+ "source": "NarrativeQA",+ "document_kind": document.get("kind", "unknown"),+ "all_answers": answer_texts,+ },+ )+ )++ return EvalSuite(name="NarrativeQA", cases=cases)+++# =============================================================================+# TOOL USE / FUNCTION CALLING DATASETS+# =============================================================================+++def load_bfcl(+ n: int = 100,+ category: str = "simple",+) -> EvalSuite:+ """Load Berkeley Function Calling Leaderboard dataset.++ BFCL contains real API schemas with ground truth function calls,+ ideal for testing tool output compression.++ Dataset: https://huggingface.co/datasets/gorilla-llm/Berkeley-Function-Calling-Leaderboard++ Available categories:+ - simple: Single function calls+ - multiple: Multiple function selection+ - parallel: Parallel function calls+ - exec_simple: Executable simple functions+ - exec_multiple: Executable multiple functions+ - exec_parallel: Executable parallel functions++ Args:+ n: Number of samples to load+ category: BFCL category++ Returns:+ EvalSuite with BFCL cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ dataset_name = f"BFCL_v3_live_{category}"++ try:+ ds = load_dataset(+ "gorilla-llm/Berkeley-Function-Calling-Leaderboard",+ dataset_name,+ split="train",+ trust_remote_code=True,+ )+ except Exception as e:+ raise ValueError(f"Failed to load BFCL dataset '{dataset_name}': {e}") from e++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if i >= n:+ break++ # Extract question from nested structure+ question = ""+ if item.get("question"):+ try:+ question = item["question"][0][0].get("content", "")+ except (IndexError, KeyError, TypeError):+ question = str(item.get("question", ""))++ # Functions as context (this is what we'd compress)+ functions = item.get("function", [])+ context = json.dumps(functions, indent=2) if functions else ""++ # Ground truth function call+ ground_truth = item.get("ground_truth", [])+ gt_str = json.dumps(ground_truth) if ground_truth else None++ cases.append(+ EvalCase(+ id=f"bfcl_{category}_{i}",+ context=context,+ query=question,+ ground_truth=gt_str,+ metadata={+ "source": "BFCL",+ "category": category,+ "num_functions": len(functions),+ },+ )+ )++ return EvalSuite(name=f"BFCL_{category}", cases=cases)+++def load_toolbench(+ n: int = 100,+ category: str = "G1",+) -> EvalSuite:+ """Load ToolBench dataset for API tool usage.++ Real-world API scenarios with multiple tools and complex reasoning.++ Dataset: https://huggingface.co/datasets/ToolBench/ToolBench++ Categories:+ - G1: Single-tool single-step+ - G2: Single-tool multi-step+ - G3: Multi-tool single-step++ Args:+ n: Number of samples to load+ category: ToolBench category (G1, G2, G3)++ Returns:+ EvalSuite with ToolBench cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ try:+ ds = load_dataset("ToolBench/ToolBench", category, split="test")+ except Exception as e:+ raise ValueError(f"Failed to load ToolBench category '{category}': {e}") from e++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if len(cases) >= n:+ break++ # Get tool definitions as context+ tools = item.get("api_list", [])+ if not tools:+ continue++ # Format tools as JSON context+ tool_defs = []+ for tool in tools:+ tool_defs.append(+ {+ "name": tool.get("api_name", ""),+ "description": tool.get("api_description", ""),+ "parameters": tool.get("required_parameters", [])+ + tool.get("optional_parameters", []),+ }+ )++ context = json.dumps(tool_defs, indent=2)++ query = item.get("query", "")+ if not query:+ continue++ # Expected answer/trajectory+ answer = item.get("answer", "")++ cases.append(+ EvalCase(+ id=f"toolbench_{category}_{i}",+ context=context,+ query=query,+ ground_truth=answer if answer else None,+ metadata={+ "source": "ToolBench",+ "category": category,+ "num_tools": len(tools),+ },+ )+ )++ return EvalSuite(name=f"ToolBench_{category}", cases=cases)+++# =============================================================================+# CODE DATASETS+# =============================================================================+++def load_codesearchnet(+ n: int = 100,+ language: str = "python",+ split: str = "test",+) -> EvalSuite:+ """Load CodeSearchNet dataset for code understanding.++ Code snippets with natural language descriptions.+ Tests compression on code without losing semantic meaning.++ Dataset: https://huggingface.co/datasets/code_search_net++ Languages: python, java, javascript, go, ruby, php++ Args:+ n: Number of samples to load+ language: Programming language+ split: Dataset split++ Returns:+ EvalSuite with CodeSearchNet cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ try:+ ds = load_dataset("code_search_net", language, split=split)+ except Exception as e:+ raise ValueError(f"Failed to load CodeSearchNet for '{language}': {e}") from e++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if len(cases) >= n:+ break++ code = item.get("func_code_string", "") or item.get("whole_func_string", "")+ if not code:+ continue++ # Use docstring as query (find code from description)+ docstring = item.get("func_documentation_string", "")+ if not docstring:+ continue++ cases.append(+ EvalCase(+ id=f"codesearchnet_{language}_{i}",+ context=code,+ query="What does this code do? Describe its functionality.",+ ground_truth=docstring,+ metadata={+ "source": "CodeSearchNet",+ "language": language,+ "func_name": item.get("func_name", ""),+ "repo": item.get("repository_name", ""),+ },+ )+ )++ return EvalSuite(name=f"CodeSearchNet_{language}", cases=cases)+++def load_humaneval(+ n: int = 164, # Total size is 164+) -> EvalSuite:+ """Load HumanEval dataset for code generation.++ Hand-crafted programming problems with test cases.+ Tests if compression preserves enough info for code generation.++ Dataset: https://huggingface.co/datasets/openai_humaneval++ Args:+ n: Number of samples to load (max 164)++ Returns:+ EvalSuite with HumanEval cases+ """+ _check_datasets_installed()+ from datasets import load_dataset++ ds = load_dataset("openai_humaneval", split="test")++ cases: list[EvalCase] = []+ for i, item in enumerate(ds):+ if i >= n:+ break++ # Prompt contains function signature and docstring+ prompt = item.get("prompt", "")+ if not prompt:+ continue++ # Canonical solution+ canonical = item.get("canonical_solution", "")++ # Test cases for verification+ test = item.get("test", "")++ # Use the prompt as context, ask to complete+ cases.append(+ EvalCase(+ id=f"humaneval_{item.get('task_id', i)}",+ context=prompt,+ query="Complete this function implementation.",+ ground_truth=canonical,+ metadata={+ "source": "HumanEval",+ "task_id": item.get("task_id", ""),+ "entry_point": item.get("entry_point", ""),+ "test": test,+ },+ )+ )++ return EvalSuite(name="HumanEval", cases=cases)+++# =============================================================================+# BUILT-IN TOOL OUTPUT SAMPLES+# =============================================================================+++def load_tool_output_samples() -> EvalSuite:+ """Load built-in tool output samples for testing.++ These are realistic tool outputs that headroom is designed to compress:+ - API responses (JSON)+ - Log outputs+ - Code files+ - Database query results+ - Kubernetes events+ - Error tracebacks+ """+ cases = [+ # GitHub API response+ EvalCase(+ id="github_search_001",+ context=json.dumps(+ {+ "total_count": 3,+ "items": [+ {+ "id": 12345,+ "name": "headroom",+ "full_name": "anthropic/headroom",+ "description": "Context optimization for LLM applications",+ "stargazers_count": 1250,+ "language": "Python",+ "topics": ["llm", "compression"],+ },+ {+ "id": 23456,+ "name": "llm-cache",+ "full_name": "openai/llm-cache",+ "description": "High-performance caching for LLMs",+ "stargazers_count": 890,+ "language": "Python",+ "topics": ["caching", "llm"],+ },+ {+ "id": 34567,+ "name": "prompt-optimizer",+ "full_name": "google/prompt-optimizer",+ "description": "Automatic prompt optimization with RL",+ "stargazers_count": 2100,+ "language": "Python",+ "topics": ["prompt-engineering", "rlhf"],+ },+ ],+ },+ indent=2,+ ),+ query="Which repository has the most stars?",+ ground_truth="prompt-optimizer",+ metadata={"source": "tool_output", "tool": "github_search"},+ ),+ # Kubernetes events+ EvalCase(+ id="k8s_events_001",+ context="""NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE+default 2m Warning FailedScheduling pod/nginx-deployment-5d8b9c7f4-x2k9j 0/3 nodes are available: 3 Insufficient memory+default 5m Normal Scheduled pod/redis-master-0 Successfully assigned default/redis-master-0 to node-2+kube-system 1h Warning NodeNotReady node/node-3 Node node-3 status is now: NodeNotReady+default 30s Normal Pulled pod/api-server-7f8d9c8b5-m4n2p Container image "api-server:v2.1.0" already present""",+ query="What is the error with the nginx deployment?",+ ground_truth="Insufficient memory",+ metadata={"source": "tool_output", "tool": "kubectl_events"},+ ),+ # Python traceback+ EvalCase(+ id="traceback_001",+ context="""Traceback (most recent call last):+ File "/app/services/payment.py", line 127, in process_payment+ result = stripe.PaymentIntent.create(+ File "/usr/local/lib/python3.11/site-packages/stripe/api_resources/payment_intent.py", line 87, in create+ return cls._static_request("post", url, params=params)+stripe.error.CardError: Your card was declined. This transaction requires authentication.+Request ID: req_a1b2c3d4e5f6g7h8+Error Code: card_declined+Decline Code: authentication_required""",+ query="What is the error code?",+ ground_truth="card_declined",+ metadata={"source": "tool_output", "tool": "error_logs"},+ ),+ # Database query result+ EvalCase(+ id="db_query_001",+ context=json.dumps(+ [+ {+ "user_id": 1,+ "name": "Alice",+ "email": "[email protected]",+ "role": "admin",+ "created_at": "2024-01-15",+ },+ {+ "user_id": 2,+ "name": "Bob",+ "email": "[email protected]",+ "role": "user",+ "created_at": "2024-02-20",+ },+ {+ "user_id": 3,+ "name": "Charlie",+ "email": "[email protected]",+ "role": "user",+ "created_at": "2024-03-10",+ },+ {+ "user_id": 4,+ "name": "Diana",+ "email": "[email protected]",+ "role": "moderator",+ "created_at": "2024-04-05",+ },+ {+ "user_id": 5,+ "name": "Eve",+ "email": "[email protected]",+ "role": "user",+ "created_at": "2024-05-01",+ },+ ],+ indent=2,+ ),+ query="Who is the admin user?",+ ground_truth="Alice",+ metadata={"source": "tool_output", "tool": "database_query"},+ ),+ # Metrics data+ EvalCase(+ id="metrics_001",+ context=json.dumps(+ {+ "service": "api-gateway",+ "period": "last_hour",+ "metrics": {+ "requests_total": 125432,+ "requests_success": 124890,+ "requests_failed": 542,+ "latency_p50_ms": 45,+ "latency_p99_ms": 230,+ "error_rate_percent": 0.43,+ "cpu_usage_percent": 67.5,+ "memory_usage_mb": 2048,+ },+ },+ indent=2,+ ),+ query="What is the p99 latency?",+ ground_truth="230",+ metadata={"source": "tool_output", "tool": "metrics_api"},+ ),+ # Git log output+ EvalCase(+ id="git_log_001",+ context="""commit a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0+Author: Alice Developer <[email protected]>+Date: Mon Jan 15 10:30:00 2024 -0800++ Fix critical security vulnerability in authentication++ - Patched SQL injection in login endpoint+ - Added input sanitization+ - Updated tests++commit b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0u1+Author: Bob Engineer <[email protected]>+Date: Sun Jan 14 15:45:00 2024 -0800++ Add user profile feature++ - New profile page component+ - Avatar upload functionality+ - Bio field with markdown support++commit c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0u1v2+Author: Alice Developer <[email protected]>+Date: Sat Jan 13 09:00:00 2024 -0800++ Refactor database connection pooling""",+ query="Who fixed the security vulnerability?",+ ground_truth="Alice Developer",+ metadata={"source": "tool_output", "tool": "git_log"},+ ),+ # AWS CLI output+ EvalCase(+ id="aws_ec2_001",+ context=json.dumps(+ {+ "Reservations": [+ {+ "Instances": [+ {+ "InstanceId": "i-0abc123def456789a",+ "InstanceType": "t3.large",+ "State": {"Name": "running"},+ "PrivateIpAddress": "10.0.1.100",+ "Tags": [{"Key": "Name", "Value": "web-server-1"}],+ },+ {+ "InstanceId": "i-0def456ghi789012b",+ "InstanceType": "t3.xlarge",+ "State": {"Name": "stopped"},+ "PrivateIpAddress": "10.0.1.101",+ "Tags": [{"Key": "Name", "Value": "web-server-2"}],+ },+ {+ "InstanceId": "i-0ghi789jkl012345c",+ "InstanceType": "r5.2xlarge",+ "State": {"Name": "running"},+ "PrivateIpAddress": "10.0.2.50",+ "Tags": [{"Key": "Name", "Value": "database-primary"}],+ },+ ]+ }+ ]+ },+ indent=2,+ ),+ query="Which instance is stopped?",+ ground_truth="web-server-2",+ metadata={"source": "tool_output", "tool": "aws_ec2_describe"},+ ),+ # Large JSON API response with nested data+ EvalCase(+ id="complex_api_001",+ context=json.dumps(+ {+ "status": "success",+ "data": {+ "organization": {+ "id": "org_123",+ "name": "Acme Corp",+ "plan": "enterprise",+ },+ "projects": [+ {+ "id": "proj_001",+ "name": "Backend API",+ "status": "active",+ "team_size": 5,+ "budget_remaining": 15000,+ },+ {+ "id": "proj_002",+ "name": "Mobile App",+ "status": "active",+ "team_size": 8,+ "budget_remaining": 28500,+ },+ {+ "id": "proj_003",+ "name": "Data Pipeline",+ "status": "paused",+ "team_size": 3,+ "budget_remaining": 5000,+ },+ ],+ "total_budget": 100000,+ "spent": 51500,+ },+ },+ indent=2,+ ),+ query="Which project has the highest budget remaining?",+ ground_truth="Mobile App",+ metadata={"source": "tool_output", "tool": "project_api"},+ ),+ ]++ return EvalSuite(name="ToolOutputSamples", cases=cases)+++# =============================================================================+# CUSTOM DATASET LOADERS+# =============================================================================+++def load_custom_dataset(path: Path | str) -> EvalSuite:+ """Load a custom evaluation dataset from JSONL file.++ Expected format (one JSON object per line):+ {"id": "case_001", "context": "...", "query": "...", "ground_truth": "..."}++ Args:+ path: Path to JSONL file++ Returns:+ EvalSuite with loaded cases+ """+ return EvalSuite.from_jsonl(path)+++def generate_retrieval_probes(+ context: str,+ n_probes: int = 5,+) -> list[str]:+ """Generate retrieval probes from a context.++ Extracts key facts/entities that should be retrievable+ after compression.++ Args:+ context: The context to analyze+ n_probes: Number of probes to generate++ Returns:+ List of fact strings to probe for+ """+ import re++ probes = []++ # Look for specific patterns+ patterns = [+ r"\b[A-Z][a-z]+ [A-Z][a-z]+\b", # Names (e.g., "John Smith")+ r"\b\d{4}-\d{2}-\d{2}\b", # Dates (e.g., "2024-01-15")+ r"\b[A-Z]{2,}\b", # Acronyms (e.g., "API", "HTTP")+ r"\b\d+\.\d+%?\b", # Numbers (e.g., "99.9%", "123.45")+ r'"[^"]{5,50}"', # Quoted strings+ r"\b[a-z_]+_[a-z_]+\b", # Snake case identifiers+ ]++ for pattern in patterns:+ matches = re.findall(pattern, context)+ for match in matches[:2]: # Take up to 2 per pattern+ if match not in probes:+ probes.append(match.strip('"'))+ if len(probes) >= n_probes:+ return probes++ return probes+++# =============================================================================+# DATASET REGISTRY & UTILITIES+# =============================================================================+++DATASET_REGISTRY: dict[str, dict[str, Any]] = {+ # RAG/Retrieval+ "hotpotqa": {+ "loader": load_hotpotqa,+ "description": "Multi-hop QA requiring reasoning over multiple Wikipedia passages",+ "category": "rag",+ "default_n": 100,+ },+ "natural_questions": {+ "loader": load_natural_questions,+ "description": "Real Google search questions with Wikipedia answers",+ "category": "rag",+ "default_n": 100,+ },+ "triviaqa": {+ "loader": load_triviaqa,+ "description": "Large-scale trivia QA with evidence documents",+ "category": "rag",+ "default_n": 100,+ },+ "msmarco": {+ "loader": load_msmarco,+ "description": "Real Bing search queries with relevant passages",+ "category": "rag",+ "default_n": 100,+ },+ "squad": {+ "loader": load_squad,+ "description": "Reading comprehension with extractive answers",+ "category": "rag",+ "default_n": 100,+ },+ # Long Context+ "longbench": {+ "loader": load_longbench,+ "description": "Long context understanding (4K-128K tokens)",+ "category": "long_context",+ "default_n": 50,+ },+ "narrativeqa": {+ "loader": load_narrativeqa,+ "description": "Story comprehension requiring narrative understanding",+ "category": "long_context",+ "default_n": 100,+ },+ # Tool Use+ "bfcl": {+ "loader": load_bfcl,+ "description": "Berkeley Function Calling Leaderboard - API schemas",+ "category": "tool_use",+ "default_n": 100,+ },+ "toolbench": {+ "loader": load_toolbench,+ "description": "Real-world API tool usage scenarios",+ "category": "tool_use",+ "default_n": 100,+ },+ # Code+ "codesearchnet": {+ "loader": load_codesearchnet,+ "description": "Code snippets with natural language descriptions",+ "category": "code",+ "default_n": 100,+ },+ "humaneval": {+ "loader": load_humaneval,+ "description": "Hand-crafted programming problems",+ "category": "code",+ "default_n": 164,+ },+ # Built-in+ "tool_outputs": {+ "loader": load_tool_output_samples,+ "description": "Built-in realistic tool outputs (JSON, logs, etc.)",+ "category": "tool_use",+ "default_n": None, # Fixed size+ },+}+++def list_available_datasets() -> dict[str, list[str]]:+ """List all available datasets by category.++ Returns:+ Dictionary mapping category to list of dataset names+ """+ by_category: dict[str, list[str]] = {}+ for name, info in DATASET_REGISTRY.items():+ category = info["category"]+ if category not in by_category:+ by_category[category] = []+ by_category[category].append(name)+ return by_category+++def load_dataset_by_name(+ name: str,+ n: int | None = None,+ **kwargs: Any,+) -> EvalSuite:+ """Load a dataset by name from the registry.++ Args:+ name: Dataset name from registry+ n: Number of samples (uses default if not specified)+ **kwargs: Additional arguments for the loader++ Returns:+ EvalSuite with loaded cases+ """+ if name not in DATASET_REGISTRY:+ available = ", ".join(DATASET_REGISTRY.keys())+ raise ValueError(f"Unknown dataset '{name}'. Available: {available}")++ info = DATASET_REGISTRY[name]+ loader = info["loader"]++ # Use provided n or default+ if n is None:+ n = info.get("default_n")++ if n is not None:+ result: EvalSuite = loader(n=n, **kwargs)+ else:+ result = loader(**kwargs)+ return resultdiff --git a/headroom/evals/metrics.py b/headroom/evals/metrics.pynew file mode 100644index 000000000..07b1d75a6--- /dev/null+++ b/headroom/evals/metrics.py@@ -0,0 +1,294 @@+"""Evaluation metrics for comparing LLM outputs.++These metrics determine whether compression preserved accuracy+by comparing responses from original vs compressed context.+"""++from __future__ import annotations++import re+from collections import Counter+++def normalize_text(text: str) -> str:+ """Normalize text for comparison.++ - Lowercase+ - Remove extra whitespace+ - Remove punctuation (optional)+ """+ text = text.lower().strip()+ text = re.sub(r"\s+", " ", text)+ return text+++def tokenize(text: str) -> list[str]:+ """Simple word tokenization."""+ # Split on whitespace and punctuation+ tokens = re.findall(r"\b\w+\b", text.lower())+ return tokens+++def compute_exact_match(response_a: str, response_b: str) -> bool:+ """Check if two responses are exactly the same (after normalization)."""+ return normalize_text(response_a) == normalize_text(response_b)+++def compute_f1(response_a: str, response_b: str) -> float:+ """Compute token-level F1 score between two responses.++ F1 = 2 * (precision * recall) / (precision + recall)++ This measures how much overlap there is between the two responses.+ A score of 1.0 means identical tokens, 0.0 means no overlap.+ """+ tokens_a = tokenize(response_a)+ tokens_b = tokenize(response_b)++ if not tokens_a or not tokens_b:+ return 0.0++ # Count token occurrences+ counter_a = Counter(tokens_a)+ counter_b = Counter(tokens_b)++ # Find common tokens (considering counts)+ common = sum((counter_a & counter_b).values())++ if common == 0:+ return 0.0++ precision = common / len(tokens_a)+ recall = common / len(tokens_b)++ return 2 * precision * recall / (precision + recall)+++def compute_bleu(response_a: str, response_b: str, max_n: int = 4) -> float:+ """Compute BLEU-like score between two responses.++ Uses n-gram precision up to max_n.+ """+ tokens_a = tokenize(response_a)+ tokens_b = tokenize(response_b)++ if not tokens_a or not tokens_b:+ return 0.0++ precisions = []++ for n in range(1, max_n + 1):+ # Get n-grams+ ngrams_a = [tuple(tokens_a[i : i + n]) for i in range(len(tokens_a) - n + 1)]+ ngrams_b = [tuple(tokens_b[i : i + n]) for i in range(len(tokens_b) - n + 1)]++ if not ngrams_a:+ break++ counter_a = Counter(ngrams_a)+ counter_b = Counter(ngrams_b)++ # Clipped count+ clipped = sum((counter_a & counter_b).values())+ total = len(ngrams_a)++ if total > 0:+ precisions.append(clipped / total)+ else:+ precisions.append(0.0)++ if not precisions or all(p == 0 for p in precisions):+ return 0.0++ # Geometric mean of precisions+ import math++ log_sum = sum(math.log(p) for p in precisions if p > 0) / len(precisions)+ return math.exp(log_sum)+++def compute_rouge_l(response_a: str, response_b: str) -> float:+ """Compute ROUGE-L score (longest common subsequence).++ Measures the longest common subsequence between two responses.+ """+ tokens_a = tokenize(response_a)+ tokens_b = tokenize(response_b)++ if not tokens_a or not tokens_b:+ return 0.0++ # LCS using dynamic programming+ m, n = len(tokens_a), len(tokens_b)+ dp = [[0] * (n + 1) for _ in range(m + 1)]++ for i in range(1, m + 1):+ for j in range(1, n + 1):+ if tokens_a[i - 1] == tokens_b[j - 1]:+ dp[i][j] = dp[i - 1][j - 1] + 1+ else:+ dp[i][j] = max(dp[i - 1][j], dp[i][j - 1])++ lcs_length = dp[m][n]++ if lcs_length == 0:+ return 0.0++ precision = lcs_length / m+ recall = lcs_length / n++ return 2 * precision * recall / (precision + recall)+++def compute_semantic_similarity(+ response_a: str,+ response_b: str,+ model_name: str = "all-MiniLM-L6-v2",+) -> float:+ """Compute semantic similarity using sentence embeddings.++ Requires sentence-transformers package.++ Args:+ response_a: First response+ response_b: Second response+ model_name: Sentence transformer model to use++ Returns:+ Cosine similarity between embeddings (0.0 to 1.0)+ """+ try:+ import numpy as np+ from sentence_transformers import SentenceTransformer+ except ImportError as e:+ raise ImportError(+ "sentence-transformers required for semantic similarity. "+ "Install with: pip install sentence-transformers"+ ) from e++ model = SentenceTransformer(model_name)++ embeddings = model.encode([response_a, response_b])+ embedding_a, embedding_b = embeddings[0], embeddings[1]++ # Cosine similarity+ dot_product = np.dot(embedding_a, embedding_b)+ norm_a = np.linalg.norm(embedding_a)+ norm_b = np.linalg.norm(embedding_b)++ if norm_a == 0 or norm_b == 0:+ return 0.0++ return float(dot_product / (norm_a * norm_b))+++def compute_answer_equivalence(+ response_a: str,+ response_b: str,+ ground_truth: str | None = None,+ semantic_threshold: float = 0.85,+ f1_threshold: float = 0.7,+) -> dict[str, float | bool | None]:+ """Comprehensive answer equivalence check.++ Combines multiple metrics to determine if two responses+ are functionally equivalent.++ Args:+ response_a: Original response+ response_b: Compressed response+ ground_truth: Optional ground truth answer+ semantic_threshold: Threshold for semantic similarity+ f1_threshold: Threshold for F1 score++ Returns:+ Dictionary with metrics and overall verdict+ """+ exact_match = compute_exact_match(response_a, response_b)+ f1_score = compute_f1(response_a, response_b)+ rouge_l = compute_rouge_l(response_a, response_b)+ semantic_similarity: float | None = None+ ground_truth_in_a: bool | None = None+ ground_truth_in_b: bool | None = None++ # Try semantic similarity+ try:+ semantic_similarity = compute_semantic_similarity(response_a, response_b)+ except ImportError:+ pass++ # Check ground truth+ if ground_truth:+ gt_lower = ground_truth.lower()+ ground_truth_in_a = gt_lower in response_a.lower()+ ground_truth_in_b = gt_lower in response_b.lower()++ # Determine equivalence+ # Equivalent if ANY of:+ # 1. Exact match+ # 2. High F1 score+ # 3. High semantic similarity+ # 4. Both contain ground truth+ equivalent = (+ exact_match+ or f1_score >= f1_threshold+ or (semantic_similarity is not None and semantic_similarity >= semantic_threshold)+ or (ground_truth_in_a is True and ground_truth_in_b is True)+ )++ result: dict[str, float | bool | None] = {+ "exact_match": exact_match,+ "f1_score": f1_score,+ "rouge_l": rouge_l,+ "semantic_similarity": semantic_similarity,+ "ground_truth_in_a": ground_truth_in_a,+ "ground_truth_in_b": ground_truth_in_b,+ "equivalent": equivalent,+ }++ return result+++def compute_information_recall(+ original_context: str,+ compressed_context: str,+ probe_facts: list[str],+) -> dict:+ """Test if specific facts are preserved after compression.++ Args:+ original_context: Original context before compression+ compressed_context: Context after compression+ probe_facts: List of facts/strings that should be retrievable++ Returns:+ Dictionary with recall metrics+ """+ original_lower = original_context.lower()+ compressed_lower = compressed_context.lower()++ facts_in_original = []+ facts_in_compressed = []+ facts_lost = []++ for fact in probe_facts:+ fact_lower = fact.lower()+ in_original = fact_lower in original_lower+ in_compressed = fact_lower in compressed_lower++ if in_original:+ facts_in_original.append(fact)+ if in_compressed:+ facts_in_compressed.append(fact)+ else:+ facts_lost.append(fact)++ recall = len(facts_in_compressed) / len(facts_in_original) if facts_in_original else 1.0++ return {+ "total_probes": len(probe_facts),+ "facts_in_original": len(facts_in_original),+ "facts_preserved": len(facts_in_compressed),+ "facts_lost": facts_lost,+ "recall": recall,+ }diff --git a/headroom/evals/reports/__init__.py b/headroom/evals/reports/__init__.pynew file mode 100644index 000000000..a265a26bb--- /dev/null+++ b/headroom/evals/reports/__init__.py@@ -0,0 +1 @@+"""Report generation for evaluation results."""diff --git a/headroom/evals/runners/__init__.py b/headroom/evals/runners/__init__.pynew file mode 100644index 000000000..4eb8de7dc--- /dev/null+++ b/headroom/evals/runners/__init__.py@@ -0,0 +1,5 @@+"""Evaluation runners for different scenarios."""++from headroom.evals.runners.before_after import BeforeAfterRunner++__all__ = ["BeforeAfterRunner"]diff --git a/headroom/evals/runners/before_after.py b/headroom/evals/runners/before_after.pynew file mode 100644index 000000000..199fe701a--- /dev/null+++ b/headroom/evals/runners/before_after.py@@ -0,0 +1,342 @@+"""Before/After evaluation runner.++This is the core evaluation pattern for proving compression accuracy:+1. Run query with ORIGINAL context → Response A+2. Run query with COMPRESSED context → Response B+3. Compare A and B using multiple metrics+4. Report if accuracy is preserved++This is the gold standard for proving compression doesn't break anything.+"""++from __future__ import annotations++import time+from collections.abc import Callable+from dataclasses import dataclass+from typing import Any, Literal++from headroom.evals.core import (+ EvalCase,+ EvalMode,+ EvalResult,+ EvalSuite,+ EvalSuiteResult,+)+from headroom.evals.metrics import compute_semantic_similarity+from headroom.transforms.content_router import ContentRouter, ContentRouterConfig+from headroom.transforms.smart_crusher import SmartCrusherConfig+++@dataclass+class LLMConfig:+ """Configuration for LLM calls."""++ provider: Literal["anthropic", "openai", "ollama"] = "anthropic"+ model: str = "claude-sonnet-4-20250514"+ temperature: float = 0.0 # Deterministic for reproducibility+ max_tokens: int = 1024+++class BeforeAfterRunner:+ """Runner for before/after compression evaluation.++ This runner:+ 1. Takes an evaluation suite+ 2. For each case, runs the same query against original and compressed context+ 3. Compares responses to determine if accuracy is preserved+ 4. Generates comprehensive report++ Example:+ ```python+ runner = BeforeAfterRunner(llm_config=LLMConfig(model="claude-sonnet-4-20250514"))+ results = runner.run(suite)+ print(results.summary())+ ```+ """++ def __init__(+ self,+ llm_config: LLMConfig | None = None,+ crusher_config: SmartCrusherConfig | None = None,+ router_config: ContentRouterConfig | None = None,+ use_semantic_similarity: bool = True,+ ):+ """Initialize runner.++ Args:+ llm_config: Configuration for LLM calls+ crusher_config: Configuration for SmartCrusher (deprecated, use router_config)+ router_config: Configuration for ContentRouter (handles all content types)+ use_semantic_similarity: Whether to compute semantic similarity+ (requires sentence-transformers)+ """+ self.llm_config = llm_config or LLMConfig()+ self.crusher_config = crusher_config or SmartCrusherConfig()+ self.router_config = router_config or ContentRouterConfig()+ self.use_semantic_similarity = use_semantic_similarity++ # Initialize LLM client+ self._llm_client = self._init_llm_client()++ # Use ContentRouter for intelligent routing to appropriate compressor+ # This handles: JSON → SmartCrusher, plain text → LLMLingua/TextCompressor,+ # code → CodeAwareCompressor, etc.+ self._router = ContentRouter(config=self.router_config)++ def _init_llm_client(self) -> Any:+ """Initialize the appropriate LLM client."""+ if self.llm_config.provider == "anthropic":+ try:+ import anthropic++ return anthropic.Anthropic()+ except ImportError as e:+ raise ImportError(+ "anthropic package required. Install with: pip install anthropic"+ ) from e+ elif self.llm_config.provider == "openai":+ try:+ import openai++ return openai.OpenAI()+ except ImportError as e:+ raise ImportError(+ "openai package required. Install with: pip install openai"+ ) from e+ elif self.llm_config.provider == "ollama":+ try:+ import ollama++ return ollama.Client()+ except ImportError as e:+ raise ImportError(+ "ollama package required. Install with: pip install ollama"+ ) from e+ else:+ raise ValueError(f"Unknown provider: {self.llm_config.provider}")++ def _call_llm(self, context: str, query: str) -> str:+ """Call the LLM with context and query."""+ prompt = f"""Based on the following context, answer the question.++Context:+{context}++Question: {query}++Answer:"""++ if self.llm_config.provider == "anthropic":+ response = self._llm_client.messages.create(+ model=self.llm_config.model,+ max_tokens=self.llm_config.max_tokens,+ temperature=self.llm_config.temperature,+ messages=[{"role": "user", "content": prompt}],+ )+ return str(response.content[0].text)+ elif self.llm_config.provider == "openai":+ response = self._llm_client.chat.completions.create(+ model=self.llm_config.model,+ max_tokens=self.llm_config.max_tokens,+ temperature=self.llm_config.temperature,+ messages=[{"role": "user", "content": prompt}],+ )+ content = response.choices[0].message.content+ return str(content) if content else ""+ elif self.llm_config.provider == "ollama":+ response = self._llm_client.chat(+ model=self.llm_config.model,+ messages=[{"role": "user", "content": prompt}],+ options={"temperature": self.llm_config.temperature},+ )+ return str(response["message"]["content"])++ return ""++ def _estimate_tokens(self, text: str) -> int:+ """Estimate token count."""+ return len(text) // 4++ def evaluate_case(self, case: EvalCase) -> EvalResult:+ """Evaluate a single case with before/after comparison."""+ from headroom.evals.metrics import compute_exact_match, compute_f1++ original_tokens = self._estimate_tokens(case.context)++ # Compress the context using ContentRouter+ # This automatically routes to the appropriate compressor:+ # - JSON arrays → SmartCrusher+ # - Plain text → LLMLingua (if available) or TextCompressor+ # - Code → CodeAwareCompressor+ # - Search results → SearchCompressor+ # - etc.+ try:+ compressed_result = self._router.compress(+ case.context,+ context=case.query, # Pass query as context for relevance-aware compression+ )+ compressed_context = compressed_result.compressed+ compressed_tokens = self._estimate_tokens(compressed_context)+ except Exception:+ compressed_context = case.context+ compressed_tokens = original_tokens++ compression_ratio = 1 - (compressed_tokens / original_tokens) if original_tokens > 0 else 0++ # Run LLM with ORIGINAL context+ start = time.time()+ try:+ response_original = self._call_llm(case.context, case.query)+ except Exception as e:+ response_original = f"ERROR: {e}"+ latency_original = (time.time() - start) * 1000++ # Run LLM with COMPRESSED context+ start = time.time()+ try:+ response_compressed = self._call_llm(compressed_context, case.query)+ except Exception as e:+ response_compressed = f"ERROR: {e}"+ latency_compressed = (time.time() - start) * 1000++ # Compute metrics+ exact_match = compute_exact_match(response_original, response_compressed)+ f1_score = compute_f1(response_original, response_compressed)++ # Semantic similarity+ semantic_sim = None+ if self.use_semantic_similarity:+ try:+ semantic_sim = compute_semantic_similarity(response_original, response_compressed)+ except (ImportError, Exception):+ pass++ # Check ground truth+ contains_ground_truth = None+ if case.ground_truth:+ gt_lower = case.ground_truth.lower()+ contains_ground_truth = (+ gt_lower in response_compressed.lower()+ or compute_f1(response_compressed, case.ground_truth) > 0.5+ )++ # Determine accuracy preservation+ accuracy_preserved = (+ f1_score > 0.7+ or (semantic_sim is not None and semantic_sim > 0.85)+ or contains_ground_truth is True+ )++ return EvalResult(+ case_id=case.id,+ mode=EvalMode.BEFORE_AFTER,+ original_tokens=original_tokens,+ compressed_tokens=compressed_tokens,+ compression_ratio=compression_ratio,+ response_original=response_original,+ response_compressed=response_compressed,+ exact_match=exact_match,+ f1_score=f1_score,+ semantic_similarity=semantic_sim,+ contains_ground_truth=contains_ground_truth,+ latency_original_ms=latency_original,+ latency_compressed_ms=latency_compressed,+ accuracy_preserved=accuracy_preserved,+ )++ def run(+ self,+ suite: EvalSuite,+ progress_callback: Callable[[int, int, EvalResult], None] | None = None,+ ) -> EvalSuiteResult:+ """Run evaluation on entire suite.++ Args:+ suite: Evaluation suite to run+ progress_callback: Optional callback(current, total, result)++ Returns:+ Aggregated results+ """+ start_time = time.time()+ results: list[EvalResult] = []++ for i, case in enumerate(suite):+ result = self.evaluate_case(case)+ results.append(result)++ if progress_callback:+ progress_callback(i + 1, len(suite), result)++ # Aggregate+ passed = sum(1 for r in results if r.accuracy_preserved)+ total_original = sum(r.original_tokens for r in results)+ total_compressed = sum(r.compressed_tokens for r in results)++ avg_compression = sum(r.compression_ratio for r in results) / len(results) if results else 0+ avg_f1 = sum(r.f1_score for r in results) / len(results) if results else 0++ semantic_sims = [+ r.semantic_similarity for r in results if r.semantic_similarity is not None+ ]+ avg_semantic = sum(semantic_sims) / len(semantic_sims) if semantic_sims else None++ return EvalSuiteResult(+ suite_name=suite.name,+ total_cases=len(results),+ passed_cases=passed,+ failed_cases=len(results) - passed,+ avg_compression_ratio=avg_compression,+ avg_f1_score=avg_f1,+ avg_semantic_similarity=avg_semantic,+ accuracy_preservation_rate=passed / len(results) if results else 0,+ total_original_tokens=total_original,+ total_compressed_tokens=total_compressed,+ total_tokens_saved=total_original - total_compressed,+ results=results,+ duration_seconds=time.time() - start_time,+ )+++def run_quick_eval(+ n_samples: int = 5,+ provider: Literal["anthropic", "openai", "ollama"] = "anthropic",+ model: str = "claude-sonnet-4-20250514",+) -> EvalSuiteResult:+ """Run a quick evaluation with built-in samples.++ This is a fast sanity check to verify compression isn't breaking things.++ Args:+ n_samples: Number of samples to test+ provider: LLM provider+ model: Model to use++ Returns:+ Evaluation results+ """+ from headroom.evals.datasets import load_tool_output_samples++ suite = load_tool_output_samples()+ suite.cases = suite.cases[:n_samples]++ runner = BeforeAfterRunner(+ llm_config=LLMConfig(provider=provider, model=model),+ use_semantic_similarity=False, # Faster without embeddings+ )++ def progress(current: int, total: int, result: EvalResult) -> None:+ status = "PASS" if result.accuracy_preserved else "FAIL"+ print(f" [{current}/{total}] {result.case_id}: {status} (F1={result.f1_score:.2f})")++ print(f"\nRunning quick eval with {n_samples} samples...")+ results = runner.run(suite, progress_callback=progress)++ print(f"\n{results.summary()}")+ return results+++if __name__ == "__main__":+ # Quick test+ run_quick_eval(n_samples=3)diff --git a/headroom/transforms/content_router.py b/headroom/transforms/content_router.pyindex a3e776b8e..1bfb448cd 100644--- a/headroom/transforms/content_router.py+++ b/headroom/transforms/content_router.py@@ -740,10 +740,9 @@ class ContentRouter(Transform):return self._try_llmlingua(content, context)elif strategy == CompressionStrategy.TEXT:- compressor = self._get_text_compressor()- if compressor:- result = compressor.compress(content, context=context)- return result.compressed, result.compressed_line_count+ # Prefer LLMLingua for text if available (ML-based, better compression)+ # Falls back to heuristic TextCompressor if LLMLingua unavailable+ return self._try_llmlingua(content, context)except Exception as e:logger.warning("Compression with %s failed: %s", strategy.value, e)diff --git a/headroom/transforms/llmlingua_compressor.py b/headroom/transforms/llmlingua_compressor.pyindex 13511094e..165f6ad53 100644--- a/headroom/transforms/llmlingua_compressor.py+++ b/headroom/transforms/llmlingua_compressor.py@@ -215,9 +215,9 @@ class LLMLinguaConfig:ccr_ttl: int = 300 # 5 minutes# Content type specific settings- code_compression_rate: float = 0.4 # More conservative for code- json_compression_rate: float = 0.35 # Slightly conservative for JSON- text_compression_rate: float = 0.25 # More aggressive for plain text+ code_compression_rate: float = 0.5 # Conservative for code+ json_compression_rate: float = 0.4 # Somewhat conservative for JSON+ text_compression_rate: float = 0.5 # Balanced for plain text (higher = more accurate)@dataclassdiff --git a/pyproject.toml b/pyproject.tomlindex 265bda1c4..058790be5 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -81,6 +81,15 @@ code = [agno = ["agno>=1.0.0",]+# Evaluation framework for testing compression accuracy+evals = [+ "datasets>=2.14.0", # HuggingFace datasets+ "sentence-transformers>=2.2.0", # Semantic similarity+ "numpy>=1.24.0", # Numerical operations+ "scikit-learn>=1.3.0", # ML metrics+ "anthropic>=0.18.0", # Anthropic API for evals+ "openai>=1.0.0", # OpenAI API for evals+]# Development dependenciesdev = ["pytest>=7.0.0",@@ -96,7 +105,7 @@ dev = []# All optional dependenciesall = [- "headroom-ai[relevance,proxy,reports,llmlingua,code]",+ "headroom-ai[relevance,proxy,reports,llmlingua,code,evals]",][project.scripts]diff --git a/tests/test_toin.py b/tests/test_toin.pyindex 983642ef5..c4d42fcd9 100644--- a/tests/test_toin.py+++ b/tests/test_toin.py@@ -18,8 +18,15 @@ from headroom.telemetry import (@pytest.fixture(autouse=True)-def reset_globals():- """Reset global state before each test."""+def reset_globals(monkeypatch, tmp_path):+ """Reset global state before each test.++ Also disables disk persistence by setting HEADROOM_TOIN_PATH to a temp file+ to avoid loading stale data from ~/.headroom/toin.json.+ """+ # Use a unique temp file for each test to avoid cross-test contamination+ temp_toin_path = str(tmp_path / "toin_test.json")+ monkeypatch.setenv("HEADROOM_TOIN_PATH", temp_toin_path)reset_toin()yieldreset_toin()@@ -178,14 +185,13 @@ class TestTOINConfig:def test_default_values(self):"""Default config values."""- from pathlib import Path-config = TOINConfig()assert config.enabled is True- # Default storage path is ~/.headroom/toin.json- expected_path = str(Path.home() / ".headroom" / "toin.json")- assert config.storage_path == expected_path+ # Storage path comes from HEADROOM_TOIN_PATH env var (set by fixture) or default+ # Just verify it's a non-empty string+ assert isinstance(config.storage_path, str)+ assert len(config.storage_path) > 0assert config.auto_save_interval == 600assert config.min_samples_for_recommendation == 10assert config.min_users_for_network_effect == 3diff --git a/tests/test_transforms/test_llmlingua_compressor.py b/tests/test_transforms/test_llmlingua_compressor.pyindex 1c111dd7f..1b432e8a3 100644--- a/tests/test_transforms/test_llmlingua_compressor.py+++ b/tests/test_transforms/test_llmlingua_compressor.py@@ -168,11 +168,11 @@ class TestLLMLinguaConfig:"""Different content types have appropriate compression rates."""config = LLMLinguaConfig()- # Code should be more conservative- assert config.code_compression_rate > config.text_compression_rate- # JSON should be between code and text- assert config.json_compression_rate > config.text_compression_rate- assert config.json_compression_rate < config.code_compression_rate+ # Code and text are equally conservative for accuracy+ assert config.code_compression_rate >= config.text_compression_rate+ # JSON can be slightly more aggressive since structure is preserved+ assert config.json_compression_rate <= config.code_compression_rate+ assert config.json_compression_rate <= config.text_compression_rate# =============================================================================diff --git a/uv.lock b/uv.lockindex 255c8ce11..794154840 100644--- a/uv.lock+++ b/uv.lock@@ -1,9 +1,15 @@version = 1requires-python = ">=3.10"resolution-markers = [- "python_full_version >= '3.14'",- "python_full_version >= '3.12' and python_full_version < '3.14'",- "python_full_version == '3.11.*'",+ "python_full_version >= '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'emscripten'",+ "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'","python_full_version < '3.11'",]@@ -502,6 +508,42 @@ toml = [{ name = "tomli", marker = "python_full_version <= '3.11'" },]+[[package]]+name = "datasets"+version = "4.5.0"+source = { registry = "https://pypi.netflix.net/simple" }+dependencies = [+ { name = "dill" },+ { name = "filelock" },+ { name = "fsspec", extra = ["http"] },+ { name = "httpx" },+ { name = "huggingface-hub" },+ { name = "multiprocess" },+ { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },+ { name = "numpy", version = "2.4.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },+ { name = "packaging" },+ { name = "pandas", version = "2.3.3", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },+ { name = "pandas", version = "3.0.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },+ { name = "pyarrow" },+ { name = "pyyaml" },+ { name = "requests" },+ { name = "tqdm" },+ { name = "xxhash" },+]+sdist = { url = "https://pypi.netflix.net/packages/19604776764/datasets-4.5.0.tar.gz", hash = "sha256:00c698ce1c2452e646cc5fad47fef39d3fe78dd650a8a6eb205bb45eb63cd500", size = 588384 }+wheels = [+ { url = "https://pypi.netflix.net/packages/19604776763/datasets-4.5.0-py3-none-any.whl", hash = "sha256:b5d7e08096ffa407dd69e58b1c0271c9b2506140839b8d99af07375ad31b6726", size = 515196 },+]++[[package]]+name = "dill"+version = "0.4.0"+source = { registry = "https://pypi.netflix.net/simple" }+sdist = { url = "https://pypi.netflix.net/packages/18612031426/dill-0.4.0.tar.gz", hash = "sha256:0633f1d2df477324f53a895b02c901fb961bdbf65a17122586ea7019292cbcf0", size = 186976 }+wheels = [+ { url = "https://pypi.netflix.net/packages/18612031425/dill-0.4.0-py3-none-any.whl", hash = "sha256:44f54bf6412c2c8464c14e8243eb163690a9800dbe2c367330883b19c7561049", size = 119668 },+]+[[package]]name = "distlib"version = "0.4.0"@@ -751,11 +793,16 @@ wheels = [[[package]]name = "fsspec"-version = "2025.12.0"+version = "2025.10.0"source = { registry = "https://pypi.netflix.net/simple" }-sdist = { url = "https://pypi.netflix.net/packages/19417522985/fsspec-2025.12.0.tar.gz", hash = "sha256:c505de011584597b1060ff778bb664c1bc022e87921b0e4f10cc9c44f9635973", size = 309748 }+sdist = { url = "https://pypi.netflix.net/packages/19267811096/fsspec-2025.10.0.tar.gz", hash = "sha256:b6789427626f068f9a83ca4e8a3cc050850b6c0f71f99ddb4f542b8266a26a59", size = 309285 }wheels = [- { url = "https://pypi.netflix.net/packages/19417522984/fsspec-2025.12.0-py3-none-any.whl", hash = "sha256:8bf1fe301b7d8acfa6e8571e3b1c3d158f909666642431cc78a1b7b4dbc5ec5b", size = 201422 },+ { url = "https://pypi.netflix.net/packages/19267811095/fsspec-2025.10.0-py3-none-any.whl", hash = "sha256:7c7712353ae7d875407f97715f0e1ffcc21e33d5b24556cb1e090ae9409ec61d", size = 200966 },+]++[package.optional-dependencies]+http = [+ { name = "aiohttp" },][[package]]@@ -882,12 +929,17 @@ agno = [{ name = "agno" },]all = [+ { name = "anthropic" },+ { name = "datasets" },{ name = "fastapi" },{ name = "httpx" },{ name = "jinja2" },{ name = "llmlingua" },{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },{ name = "numpy", version = "2.4.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },+ { name = "openai" },+ { name = "scikit-learn", version = "1.7.2", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },+ { name = "scikit-learn", version = "1.8.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },{ name = "sentence-transformers" },{ name = "torch" },{ name = "transformers" },@@ -909,6 +961,16 @@ dev = [{ name = "pytest-cov" },{ name = "ruff" },]+evals = [+ { name = "anthropic" },+ { name = "datasets" },+ { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },+ { name = "numpy", version = "2.4.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },+ { name = "openai" },+ { name = "scikit-learn", version = "1.7.2", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },+ { name = "scikit-learn", version = "1.8.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },+ { name = "sentence-transformers" },+]llmlingua = [{ name = "llmlingua" },{ name = "torch" },@@ -932,25 +994,31 @@ reports = [requires-dist = [{ name = "agno", marker = "extra == 'agno'", specifier = ">=1.0.0" },{ name = "anthropic", marker = "extra == 'dev'", specifier = ">=0.18.0" },+ { name = "anthropic", marker = "extra == 'evals'", specifier = ">=0.18.0" },+ { name = "datasets", marker = "extra == 'evals'", specifier = ">=2.14.0" },{ name = "fastapi", marker = "extra == 'proxy'", specifier = ">=0.100.0" },- { name = "headroom-ai", extras = ["relevance", "proxy", "reports", "llmlingua", "code"], marker = "extra == 'all'" },+ { name = "headroom-ai", extras = ["relevance", "proxy", "reports", "llmlingua", "code", "evals"], marker = "extra == 'all'" },{ name = "httpx", marker = "extra == 'proxy'", specifier = ">=0.24.0" },{ name = "jinja2", marker = "extra == 'reports'", specifier = ">=3.0.0" },{ name = "langchain-ollama", marker = "extra == 'dev'", specifier = ">=0.2.0" },{ name = "litellm", specifier = ">=1.0.0" },{ name = "llmlingua", marker = "extra == 'llmlingua'", specifier = ">=0.2.0" },{ name = "mypy", marker = "extra == 'dev'", specifier = ">=1.0.0" },+ { name = "numpy", marker = "extra == 'evals'", specifier = ">=1.24.0" },{ name = "numpy", marker = "extra == 'relevance'", specifier = ">=1.24.0" },{ name = "ollama", marker = "extra == 'dev'", specifier = ">=0.4.0" },{ name = "openai", specifier = ">=2.14.0" },{ name = "openai", marker = "extra == 'dev'", specifier = ">=1.0.0" },+ { name = "openai", marker = "extra == 'evals'", specifier = ">=1.0.0" },{ name = "pre-commit", marker = "extra == 'dev'", specifier = ">=3.0.0" },{ name = "pydantic", specifier = ">=2.0.0" },{ name = "pytest", marker = "extra == 'dev'", specifier = ">=7.0.0" },{ name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.21.0" },{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.0.0" },{ name = "ruff", marker = "extra == 'dev'", specifier = ">=0.1.0" },+ { name = "scikit-learn", marker = "extra == 'evals'", specifier = ">=1.3.0" },{ name = "sentence-transformers", specifier = ">=5.2.0" },+ { name = "sentence-transformers", marker = "extra == 'evals'", specifier = ">=2.2.0" },{ name = "sentence-transformers", marker = "extra == 'relevance'", specifier = ">=2.2.0" },{ name = "tiktoken", specifier = ">=0.5.0" },{ name = "torch", marker = "extra == 'llmlingua'", specifier = ">=2.0.0" },@@ -1682,6 +1750,29 @@ wheels = [{ url = "https://pypi.netflix.net/packages/19171438431/multidict-6.7.0-py3-none-any.whl", hash = "sha256:394fc5c42a333c9ffc3e421a4c85e08580d990e08b99f6bf35b4132114c5dcb3", size = 12317 },]+[[package]]+name = "multiprocess"+version = "0.70.18"+source = { registry = "https://pypi.netflix.net/simple" }+dependencies = [+ { name = "dill" },+]+sdist = { url = "https://pypi.netflix.net/packages/18615643461/multiprocess-0.70.18.tar.gz", hash = "sha256:f9597128e6b3e67b23956da07cf3d2e5cba79e2f4e0fba8d7903636663ec6d0d", size = 1798503 }+wheels = [+ { url = "https://pypi.netflix.net/packages/18615643269/multiprocess-0.70.18-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:25d4012dcaaf66b9e8e955f58482b42910c2ee526d532844d8bcf661bbc604df", size = 135083 },+ { url = "https://pypi.netflix.net/packages/18615643270/multiprocess-0.70.18-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:06b19433de0d02afe5869aec8931dd5c01d99074664f806c73896b0d9e527213", size = 135128 },+ { url = "https://pypi.netflix.net/packages/18615643271/multiprocess-0.70.18-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:6fa1366f994373aaf2d4738b0f56e707caeaa05486e97a7f71ee0853823180c2", size = 135132 },+ { url = "https://pypi.netflix.net/packages/18615643272/multiprocess-0.70.18-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:8b8940ae30139e04b076da6c5b83e9398585ebdf0f2ad3250673fef5b2ff06d6", size = 144695 },+ { url = "https://pypi.netflix.net/packages/18615643273/multiprocess-0.70.18-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:0929ba95831adb938edbd5fb801ac45e705ecad9d100b3e653946b7716cb6bd3", size = 144742 },+ { url = "https://pypi.netflix.net/packages/18615643274/multiprocess-0.70.18-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:4d77f8e4bfe6c6e2e661925bbf9aed4d5ade9a1c6502d5dfc10129b9d1141797", size = 144745 },+ { url = "https://pypi.netflix.net/packages/18615643281/multiprocess-0.70.18-py310-none-any.whl", hash = "sha256:60c194974c31784019c1f459d984e8f33ee48f10fcf42c309ba97b30d9bd53ea", size = 134948 },+ { url = "https://pypi.netflix.net/packages/18615643282/multiprocess-0.70.18-py311-none-any.whl", hash = "sha256:5aa6eef98e691281b3ad923be2832bf1c55dd2c859acd73e5ec53a66aae06a1d", size = 144462 },+ { url = "https://pypi.netflix.net/packages/18615643283/multiprocess-0.70.18-py312-none-any.whl", hash = "sha256:9b78f8e5024b573730bfb654783a13800c2c0f2dfc0c25e70b40d184d64adaa2", size = 150287 },+ { url = "https://pypi.netflix.net/packages/18615643458/multiprocess-0.70.18-py313-none-any.whl", hash = "sha256:871743755f43ef57d7910a38433cfe41319e72be1bbd90b79c7a5ac523eb9334", size = 151917 },+ { url = "https://pypi.netflix.net/packages/18615643459/multiprocess-0.70.18-py38-none-any.whl", hash = "sha256:dbf705e52a154fe5e90fb17b38f02556169557c2dd8bb084f2e06c2784d8279b", size = 132636 },+ { url = "https://pypi.netflix.net/packages/18615643460/multiprocess-0.70.18-py39-none-any.whl", hash = "sha256:e78ca805a72b1b810c690b6b4cc32579eba34f403094bbbae962b7b5bf9dfcb8", size = 133478 },+]+[[package]]name = "mypy"version = "1.19.1"@@ -1754,9 +1845,15 @@ name = "networkx"version = "3.6.1"source = { registry = "https://pypi.netflix.net/simple" }resolution-markers = [- "python_full_version >= '3.14'",- "python_full_version >= '3.12' and python_full_version < '3.14'",- "python_full_version == '3.11.*'",+ "python_full_version >= '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'emscripten'",+ "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'",]sdist = { url = "https://pypi.netflix.net/packages/19441125159/networkx-3.6.1.tar.gz", hash = "sha256:26b7c357accc0c8cde558ad486283728b65b6a95d85ee1cd66bafab4c8168509", size = 2517025 }wheels = [@@ -1857,9 +1954,15 @@ name = "numpy"version = "2.4.0"source = { registry = "https://pypi.netflix.net/simple" }resolution-markers = [- "python_full_version >= '3.14'",- "python_full_version >= '3.12' and python_full_version < '3.14'",- "python_full_version == '3.11.*'",+ "python_full_version >= '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'emscripten'",+ "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'",]sdist = { url = "https://pypi.netflix.net/packages/19506581406/numpy-2.4.0.tar.gz", hash = "sha256:6e504f7b16118198f138ef31ba24d985b124c2c469fe8467007cf30fd992f934", size = 20685720 }wheels = [@@ -2207,6 +2310,141 @@ wheels = [{ url = "https://pypi.netflix.net/packages/18622031467/packaging-25.0-py3-none-any.whl", hash = "sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484", size = 66469 },]+[[package]]+name = "pandas"+version = "2.3.3"+source = { registry = "https://pypi.netflix.net/simple" }+resolution-markers = [+ "python_full_version < '3.11'",+]+dependencies = [+ { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },+ { name = "python-dateutil", marker = "python_full_version < '3.11'" },+ { name = "pytz", marker = "python_full_version < '3.11'" },+ { name = "tzdata", marker = "python_full_version < '3.11'" },+]+sdist = { url = "https://pypi.netflix.net/packages/19146550405/pandas-2.3.3.tar.gz", hash = "sha256:e05e1af93b977f7eafa636d043f9f94c7ee3ac81af99c13508215942e64c993b", size = 4495223 }+wheels = [+ { url = "https://pypi.netflix.net/packages/19146420125/pandas-2.3.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:376c6446ae31770764215a6c937f72d917f214b43560603cd60da6408f183b6c", size = 11555763 },+ { url = "https://pypi.netflix.net/packages/19146420126/pandas-2.3.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e19d192383eab2f4ceb30b412b22ea30690c9e618f78870357ae1d682912015a", size = 10801217 },+ { url = "https://pypi.netflix.net/packages/19146422192/pandas-2.3.3-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5caf26f64126b6c7aec964f74266f435afef1c1b13da3b0636c7518a1fa3e2b1", size = 12148791 },+ { url = "https://pypi.netflix.net/packages/19146428335/pandas-2.3.3-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:dd7478f1463441ae4ca7308a70e90b33470fa593429f9d4c578dd00d1fa78838", size = 12769373 },+ { url = "https://pypi.netflix.net/packages/19146430403/pandas-2.3.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:4793891684806ae50d1288c9bae9330293ab4e083ccd1c5e383c34549c6e4250", size = 13200444 },+ { url = "https://pypi.netflix.net/packages/19146432510/pandas-2.3.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:28083c648d9a99a5dd035ec125d42439c6c1c525098c58af0fc38dd1a7a1b3d4", size = 13858459 },+ { url = "https://pypi.netflix.net/packages/19146434580/pandas-2.3.3-cp310-cp310-win_amd64.whl", hash = "sha256:503cf027cf9940d2ceaa1a93cfb5f8c8c7e6e90720a2850378f0b3f3b1e06826", size = 11346086 },+ { url = "https://pypi.netflix.net/packages/19146436651/pandas-2.3.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:602b8615ebcc4a0c1751e71840428ddebeb142ec02c786e8ad6b1ce3c8dec523", size = 11578790 },+ { url = "https://pypi.netflix.net/packages/19146564789/pandas-2.3.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:8fe25fc7b623b0ef6b5009149627e34d2a4657e880948ec3c840e9402e5c1b45", size = 10833831 },+ { url = "https://pypi.netflix.net/packages/19146438723/pandas-2.3.3-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b468d3dad6ff947df92dcb32ede5b7bd41a9b3cceef0a30ed925f6d01fb8fa66", size = 12199267 },+ { url = "https://pypi.netflix.net/packages/19146440796/pandas-2.3.3-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b98560e98cb334799c0b07ca7967ac361a47326e9b4e5a7dfb5ab2b1c9d35a1b", size = 12789281 },+ { url = "https://pypi.netflix.net/packages/19146440797/pandas-2.3.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37b5848ba49824e5c30bedb9c830ab9b7751fd049bc7914533e01c65f79791", size = 13240453 },+ { url = "https://pypi.netflix.net/packages/19146442879/pandas-2.3.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:db4301b2d1f926ae677a751eb2bd0e8c5f5319c9cb3f88b0becbbb0b07b34151", size = 13890361 },+ { url = "https://pypi.netflix.net/packages/19146442880/pandas-2.3.3-cp311-cp311-win_amd64.whl", hash = "sha256:f086f6fe114e19d92014a1966f43a3e62285109afe874f067f5abbdcbb10e59c", size = 11348702 },+ { url = "https://pypi.netflix.net/packages/19146444957/pandas-2.3.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:6d21f6d74eb1725c2efaa71a2bfc661a0689579b58e9c0ca58a739ff0b002b53", size = 11597846 },+ { url = "https://pypi.netflix.net/packages/19146564796/pandas-2.3.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:3fd2f887589c7aa868e02632612ba39acb0b8948faf5cc58f0850e165bd46f35", size = 10729618 },+ { url = "https://pypi.netflix.net/packages/19146447035/pandas-2.3.3-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ecaf1e12bdc03c86ad4a7ea848d66c685cb6851d807a26aa245ca3d2017a1908", size = 11737212 },+ { url = "https://pypi.netflix.net/packages/19146449114/pandas-2.3.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b3d11d2fda7eb164ef27ffc14b4fcab16a80e1ce67e9f57e19ec0afaf715ba89", size = 12362693 },+ { url = "https://pypi.netflix.net/packages/19146451194/pandas-2.3.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:a68e15f780eddf2b07d242e17a04aa187a7ee12b40b930bfdd78070556550e98", size = 12771002 },+ { url = "https://pypi.netflix.net/packages/19146453275/pandas-2.3.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:371a4ab48e950033bcf52b6527eccb564f52dc826c02afd9a1bc0ab731bba084", size = 13450971 },+ { url = "https://pypi.netflix.net/packages/19146455357/pandas-2.3.3-cp312-cp312-win_amd64.whl", hash = "sha256:a16dcec078a01eeef8ee61bf64074b4e524a2a3f4b3be9326420cabe59c4778b", size = 10992722 },+ { url = "https://pypi.netflix.net/packages/19146455358/pandas-2.3.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:56851a737e3470de7fa88e6131f41281ed440d29a9268dcbf0002da5ac366713", size = 11544671 },+ { url = "https://pypi.netflix.net/packages/19146457442/pandas-2.3.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bdcd9d1167f4885211e401b3036c0c8d9e274eee67ea8d0758a256d60704cfe8", size = 10680807 },+ { url = "https://pypi.netflix.net/packages/19146459527/pandas-2.3.3-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e32e7cc9af0f1cc15548288a51a3b681cc2a219faa838e995f7dc53dbab1062d", size = 11709872 },+ { url = "https://pypi.netflix.net/packages/19146462847/pandas-2.3.3-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:318d77e0e42a628c04dc56bcef4b40de67918f7041c2b061af1da41dcff670ac", size = 12306371 },+ { url = "https://pypi.netflix.net/packages/19146464934/pandas-2.3.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4e0a175408804d566144e170d0476b15d78458795bb18f1304fb94160cabf40c", size = 12765333 },+ { url = "https://pypi.netflix.net/packages/19146467022/pandas-2.3.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:93c2d9ab0fc11822b5eece72ec9587e172f63cff87c00b062f6e37448ced4493", size = 13418120 },+ { url = "https://pypi.netflix.net/packages/19146486223/pandas-2.3.3-cp313-cp313-win_amd64.whl", hash = "sha256:f8bfc0e12dc78f777f323f55c58649591b2cd0c43534e8355c51d3fede5f4dee", size = 10993991 },+ { url = "https://pypi.netflix.net/packages/19146469111/pandas-2.3.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:75ea25f9529fdec2d2e93a42c523962261e567d250b0013b16210e1d40d7c2e5", size = 12048227 },+ { url = "https://pypi.netflix.net/packages/19146469112/pandas-2.3.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:74ecdf1d301e812db96a465a525952f4dde225fdb6d8e5a521d47e1f42041e21", size = 11411056 },+ { url = "https://pypi.netflix.net/packages/19146471203/pandas-2.3.3-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6435cb949cb34ec11cc9860246ccb2fdc9ecd742c12d3304989017d53f039a78", size = 11645189 },+ { url = "https://pypi.netflix.net/packages/19146476329/pandas-2.3.3-cp313-cp313t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:900f47d8f20860de523a1ac881c4c36d65efcb2eb850e6948140fa781736e110", size = 12121912 },+ { url = "https://pypi.netflix.net/packages/19146478950/pandas-2.3.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:a45c765238e2ed7d7c608fc5bc4a6f88b642f2f01e70c0c23d2224dd21829d86", size = 12712160 },+ { url = "https://pypi.netflix.net/packages/19146483505/pandas-2.3.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:c4fc4c21971a1a9f4bdb4c73978c7f7256caa3e62b323f70d6cb80db583350bc", size = 13199233 },+ { url = "https://pypi.netflix.net/packages/19146502664/pandas-2.3.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:ee15f284898e7b246df8087fc82b87b01686f98ee67d85a17b7ab44143a3a9a0", size = 11540635 },+ { url = "https://pypi.netflix.net/packages/19146506512/pandas-2.3.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1611aedd912e1ff81ff41c745822980c49ce4a7907537be8692c8dbc31924593", size = 10759079 },+ { url = "https://pypi.netflix.net/packages/19146508610/pandas-2.3.3-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6d2cefc361461662ac48810cb14365a365ce864afe85ef1f447ff5a1e99ea81c", size = 11814049 },+ { url = "https://pypi.netflix.net/packages/19146510709/pandas-2.3.3-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ee67acbbf05014ea6c763beb097e03cd629961c8a632075eeb34247120abcb4b", size = 12332638 },+ { url = "https://pypi.netflix.net/packages/19146512809/pandas-2.3.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c46467899aaa4da076d5abc11084634e2d197e9460643dd455ac3db5856b24d6", size = 12886834 },+ { url = "https://pypi.netflix.net/packages/19146515308/pandas-2.3.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6253c72c6a1d990a410bc7de641d34053364ef8bcd3126f7e7450125887dffe3", size = 13409925 },+ { url = "https://pypi.netflix.net/packages/19146531255/pandas-2.3.3-cp314-cp314-win_amd64.whl", hash = "sha256:1b07204a219b3b7350abaae088f451860223a52cfb8a6c53358e7948735158e5", size = 11109071 },+ { url = "https://pypi.netflix.net/packages/19146517410/pandas-2.3.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:2462b1a365b6109d275250baaae7b760fd25c726aaca0054649286bcfbb3e8ec", size = 12048504 },+ { url = "https://pypi.netflix.net/packages/19146519513/pandas-2.3.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:0242fe9a49aa8b4d78a4fa03acb397a58833ef6199e9aa40a95f027bb3a1b6e7", size = 11410702 },+ { url = "https://pypi.netflix.net/packages/19146521617/pandas-2.3.3-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a21d830e78df0a515db2b3d2f5570610f5e6bd2e27749770e8bb7b524b89b450", size = 11634535 },+ { url = "https://pypi.netflix.net/packages/19146524940/pandas-2.3.3-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2e3ebdb170b5ef78f19bfb71b0dc5dc58775032361fa188e814959b74d726dd5", size = 12121582 },+ { url = "https://pypi.netflix.net/packages/19146527046/pandas-2.3.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:d051c0e065b94b7a3cea50eb1ec32e912cd96dba41647eb24104b6c6c14c5788", size = 12699963 },+ { url = "https://pypi.netflix.net/packages/19146529153/pandas-2.3.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:3869faf4bd07b3b66a9f462417d0ca3a9df29a9f6abd5d0d0dbab15dac7abe87", size = 13202175 },+]++[[package]]+name = "pandas"+version = "3.0.0"+source = { registry = "https://pypi.netflix.net/simple" }+resolution-markers = [+ "python_full_version >= '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'emscripten'",+ "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'",+]+dependencies = [+ { name = "numpy", version = "2.4.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },+ { name = "python-dateutil", marker = "python_full_version >= '3.11'" },+ { name = "tzdata", marker = "(python_full_version >= '3.11' and sys_platform == 'emscripten') or (python_full_version >= '3.11' and sys_platform == 'win32')" },+]+sdist = { url = "https://pypi.netflix.net/packages/19645949718/pandas-3.0.0.tar.gz", hash = "sha256:0facf7e87d38f721f0af46fe70d97373a37701b1c09f7ed7aeeb292ade5c050f", size = 4633005 }+wheels = [+ { url = "https://pypi.netflix.net/packages/19645921316/pandas-3.0.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:d64ce01eb9cdca96a15266aa679ae50212ec52757c79204dbc7701a222401850", size = 10313247 },+ { url = "https://pypi.netflix.net/packages/19645921317/pandas-3.0.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:613e13426069793aa1ec53bdcc3b86e8d32071daea138bbcf4fa959c9cdaa2e2", size = 9913131 },+ { url = "https://pypi.netflix.net/packages/19645921318/pandas-3.0.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0192fee1f1a8e743b464a6607858ee4b071deb0b118eb143d71c2a1d170996d5", size = 10741925 },+ { url = "https://pypi.netflix.net/packages/19645927138/pandas-3.0.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f0b853319dec8d5e0c8b875374c078ef17f2269986a78168d9bd57e49bf650ae", size = 11245979 },+ { url = "https://pypi.netflix.net/packages/19645927139/pandas-3.0.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:707a9a877a876c326ae2cb640fbdc4ef63b0a7b9e2ef55c6df9942dcee8e2af9", size = 11756337 },+ { url = "https://pypi.netflix.net/packages/19645927140/pandas-3.0.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:afd0aa3d0b5cda6e0b8ffc10dbcca3b09ef3cbcd3fe2b27364f85fdc04e1989d", size = 12325517 },+ { url = "https://pypi.netflix.net/packages/19645927141/pandas-3.0.0-cp311-cp311-win_amd64.whl", hash = "sha256:113b4cca2614ff7e5b9fee9b6f066618fe73c5a83e99d721ffc41217b2bf57dd", size = 9881576 },+ { url = "https://pypi.netflix.net/packages/19645927142/pandas-3.0.0-cp311-cp311-win_arm64.whl", hash = "sha256:c14837eba8e99a8da1527c0280bba29b0eb842f64aa94982c5e21227966e164b", size = 9140807 },+ { url = "https://pypi.netflix.net/packages/19645927143/pandas-3.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:9803b31f5039b3c3b10cc858c5e40054adb4b29b4d81cb2fd789f4121c8efbcd", size = 10334013 },+ { url = "https://pypi.netflix.net/packages/19645927144/pandas-3.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:14c2a4099cd38a1d18ff108168ea417909b2dea3bd1ebff2ccf28ddb6a74d740", size = 9874154 },+ { url = "https://pypi.netflix.net/packages/19645927145/pandas-3.0.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d257699b9a9960e6125686098d5714ac59d05222bef7a5e6af7a7fd87c650801", size = 10384433 },+ { url = "https://pypi.netflix.net/packages/19645927146/pandas-3.0.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:69780c98f286076dcafca38d8b8eee1676adf220199c0a39f0ecbf976b68151a", size = 10864519 },+ { url = "https://pypi.netflix.net/packages/19645938905/pandas-3.0.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:4a66384f017240f3858a4c8a7cf21b0591c3ac885cddb7758a589f0f71e87ebb", size = 11394124 },+ { url = "https://pypi.netflix.net/packages/19645938906/pandas-3.0.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:be8c515c9bc33989d97b89db66ea0cececb0f6e3c2a87fcc8b69443a6923e95f", size = 11920444 },+ { url = "https://pypi.netflix.net/packages/19645938907/pandas-3.0.0-cp312-cp312-win_amd64.whl", hash = "sha256:a453aad8c4f4e9f166436994a33884442ea62aa8b27d007311e87521b97246e1", size = 9730970 },+ { url = "https://pypi.netflix.net/packages/19645938908/pandas-3.0.0-cp312-cp312-win_arm64.whl", hash = "sha256:da768007b5a33057f6d9053563d6b74dd6d029c337d93c6d0d22a763a5c2ecc0", size = 9041950 },+ { url = "https://pypi.netflix.net/packages/19645938909/pandas-3.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:b78d646249b9a2bc191040988c7bb524c92fa8534fb0898a0741d7e6f2ffafa6", size = 10307129 },+ { url = "https://pypi.netflix.net/packages/19645938910/pandas-3.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bc9cba7b355cb4162442a88ce495e01cb605f17ac1e27d6596ac963504e0305f", size = 9850201 },+ { url = "https://pypi.netflix.net/packages/19645938911/pandas-3.0.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c9a1a149aed3b6c9bf246033ff91e1b02d529546c5d6fb6b74a28fea0cf4c70", size = 10354031 },+ { url = "https://pypi.netflix.net/packages/19645941055/pandas-3.0.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:95683af6175d884ee89471842acfca29172a85031fccdabc35e50c0984470a0e", size = 10861165 },+ { url = "https://pypi.netflix.net/packages/19645941056/pandas-3.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:1fbbb5a7288719e36b76b4f18d46ede46e7f916b6c8d9915b756b0a6c3f792b3", size = 11359359 },+ { url = "https://pypi.netflix.net/packages/19645941057/pandas-3.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8e8b9808590fa364416b49b2a35c1f4cf2785a6c156935879e57f826df22038e", size = 11912907 },+ { url = "https://pypi.netflix.net/packages/19645941058/pandas-3.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:98212a38a709feb90ae658cb6227ea3657c22ba8157d4b8f913cd4c950de5e7e", size = 9732138 },+ { url = "https://pypi.netflix.net/packages/19645941059/pandas-3.0.0-cp313-cp313-win_arm64.whl", hash = "sha256:177d9df10b3f43b70307a149d7ec49a1229a653f907aa60a48f1877d0e6be3be", size = 9033568 },+ { url = "https://pypi.netflix.net/packages/19645941060/pandas-3.0.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:2713810ad3806767b89ad3b7b69ba153e1c6ff6d9c20f9c2140379b2a98b6c98", size = 10741936 },+ { url = "https://pypi.netflix.net/packages/19645943210/pandas-3.0.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:15d59f885ee5011daf8335dff47dcb8a912a27b4ad7826dc6cbe809fd145d327", size = 10393884 },+ { url = "https://pypi.netflix.net/packages/19645943211/pandas-3.0.0-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:24e6547fb64d2c92665dd2adbfa4e85fa4fd70a9c070e7cfb03b629a0bbab5eb", size = 10310740 },+ { url = "https://pypi.netflix.net/packages/19645943212/pandas-3.0.0-cp313-cp313t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:48ee04b90e2505c693d3f8e8f524dab8cb8aaf7ddcab52c92afa535e717c4812", size = 10700014 },+ { url = "https://pypi.netflix.net/packages/19645943213/pandas-3.0.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:66f72fb172959af42a459e27a8d8d2c7e311ff4c1f7db6deb3b643dbc382ae08", size = 11323737 },+ { url = "https://pypi.netflix.net/packages/19645943214/pandas-3.0.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:4a4a400ca18230976724a5066f20878af785f36c6756e498e94c2a5e5d57779c", size = 11771558 },+ { url = "https://pypi.netflix.net/packages/19645943215/pandas-3.0.0-cp313-cp313t-win_amd64.whl", hash = "sha256:940eebffe55528074341a5a36515f3e4c5e25e958ebbc764c9502cfc35ba3faa", size = 10473771 },+ { url = "https://pypi.netflix.net/packages/19645945371/pandas-3.0.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:597c08fb9fef0edf1e4fa2f9828dd27f3d78f9b8c9b4a748d435ffc55732310b", size = 10312075 },+ { url = "https://pypi.netflix.net/packages/19645945372/pandas-3.0.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:447b2d68ac5edcbf94655fe909113a6dba6ef09ad7f9f60c80477825b6c489fe", size = 9900213 },+ { url = "https://pypi.netflix.net/packages/19645945373/pandas-3.0.0-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:debb95c77ff3ed3ba0d9aa20c3a2f19165cc7956362f9873fce1ba0a53819d70", size = 10428768 },+ { url = "https://pypi.netflix.net/packages/19645945374/pandas-3.0.0-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fedabf175e7cd82b69b74c30adbaa616de301291a5231138d7242596fc296a8d", size = 10882954 },+ { url = "https://pypi.netflix.net/packages/19645945375/pandas-3.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:412d1a89aab46889f3033a386912efcdfa0f1131c5705ff5b668dda88305e986", size = 11430293 },+ { url = "https://pypi.netflix.net/packages/19645945376/pandas-3.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:e979d22316f9350c516479dd3a92252be2937a9531ed3a26ec324198a99cdd49", size = 11952452 },+ { url = "https://pypi.netflix.net/packages/19645945377/pandas-3.0.0-cp314-cp314-win_amd64.whl", hash = "sha256:083b11415b9970b6e7888800c43c82e81a06cd6b06755d84804444f0007d6bb7", size = 9851081 },+ { url = "https://pypi.netflix.net/packages/19645947540/pandas-3.0.0-cp314-cp314-win_arm64.whl", hash = "sha256:5db1e62cb99e739fa78a28047e861b256d17f88463c76b8dafc7c1338086dca8", size = 9174610 },+ { url = "https://pypi.netflix.net/packages/19645947541/pandas-3.0.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:697b8f7d346c68274b1b93a170a70974cdc7d7354429894d5927c1effdcccd73", size = 10761999 },+ { url = "https://pypi.netflix.net/packages/19645947542/pandas-3.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:8cb3120f0d9467ed95e77f67a75e030b67545bcfa08964e349252d674171def2", size = 10410279 },+ { url = "https://pypi.netflix.net/packages/19645947543/pandas-3.0.0-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:33fd3e6baa72899746b820c31e4b9688c8e1b7864d7aec2de7ab5035c285277a", size = 10330198 },+ { url = "https://pypi.netflix.net/packages/19645947544/pandas-3.0.0-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a8942e333dc67ceda1095227ad0febb05a3b36535e520154085db632c40ad084", size = 10728513 },+ { url = "https://pypi.netflix.net/packages/19645947545/pandas-3.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:783ac35c4d0fe0effdb0d67161859078618b1b6587a1af15928137525217a721", size = 11345550 },+ { url = "https://pypi.netflix.net/packages/19645947546/pandas-3.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:125eb901e233f155b268bbef9abd9afb5819db74f0e677e89a61b246228c71ac", size = 11799386 },+ { url = "https://pypi.netflix.net/packages/19645949716/pandas-3.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:b86d113b6c109df3ce0ad5abbc259fe86a1bd4adfd4a31a89da42f84f65509bb", size = 10873041 },+ { url = "https://pypi.netflix.net/packages/19645949717/pandas-3.0.0-cp314-cp314t-win_arm64.whl", hash = "sha256:1c39eab3ad38f2d7a249095f0a3d8f8c22cc0f847e98ccf5bbe732b272e2d9fa", size = 9441003 },+]+[[package]]name = "pathspec"version = "1.0.2"@@ -2392,6 +2630,63 @@ wheels = [{ url = "https://pypi.netflix.net/packages/19533565350/psutil-7.2.1-cp37-abi3-win_arm64.whl", hash = "sha256:0d67c1822c355aa6f7314d92018fb4268a76668a536f133599b91edd48759442", size = 133836 },]+[[package]]+name = "pyarrow"+version = "23.0.0"+source = { registry = "https://pypi.netflix.net/simple" }+sdist = { url = "https://pypi.netflix.net/packages/19630424053/pyarrow-23.0.0.tar.gz", hash = "sha256:180e3150e7edfcd182d3d9afba72f7cf19839a497cc76555a8dce998a8f67615", size = 1167185 }+wheels = [+ { url = "https://pypi.netflix.net/packages/19630364393/pyarrow-23.0.0-cp310-cp310-macosx_12_0_arm64.whl", hash = "sha256:cbdc2bf5947aa4d462adcf8453cf04aee2f7932653cb67a27acd96e5e8528a67", size = 34281861 },+ { url = "https://pypi.netflix.net/packages/19630364394/pyarrow-23.0.0-cp310-cp310-macosx_12_0_x86_64.whl", hash = "sha256:4d38c836930ce15cd31dce20114b21ba082da231c884bdc0a7b53e1477fe7f07", size = 35825067 },+ { url = "https://pypi.netflix.net/packages/19630367838/pyarrow-23.0.0-cp310-cp310-manylinux_2_28_aarch64.whl", hash = "sha256:4222ff8f76919ecf6c716175a0e5fddb5599faeed4c56d9ea41a2c42be4998b2", size = 44458539 },+ { url = "https://pypi.netflix.net/packages/19630367839/pyarrow-23.0.0-cp310-cp310-manylinux_2_28_x86_64.whl", hash = "sha256:87f06159cbe38125852657716889296c83c37b4d09a5e58f3d10245fd1f69795", size = 47535889 },+ { url = "https://pypi.netflix.net/packages/19630370974/pyarrow-23.0.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:1675c374570d8b91ea6d4edd4608fa55951acd44e0c31bd146e091b4005de24f", size = 48157777 },+ { url = "https://pypi.netflix.net/packages/19630370975/pyarrow-23.0.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:247374428fde4f668f138b04031a7e7077ba5fa0b5b1722fdf89a017bf0b7ee0", size = 50580441 },+ { url = "https://pypi.netflix.net/packages/19630373893/pyarrow-23.0.0-cp310-cp310-win_amd64.whl", hash = "sha256:de53b1bd3b88a2ee93c9af412c903e57e738c083be4f6392288294513cd8b2c1", size = 27530028 },+ { url = "https://pypi.netflix.net/packages/19630376806/pyarrow-23.0.0-cp311-cp311-macosx_12_0_arm64.whl", hash = "sha256:5574d541923efcbfdf1294a2746ae3b8c2498a2dc6cd477882f6f4e7b1ac08d3", size = 34276762 },+ { url = "https://pypi.netflix.net/packages/19630376807/pyarrow-23.0.0-cp311-cp311-macosx_12_0_x86_64.whl", hash = "sha256:2ef0075c2488932e9d3c2eb3482f9459c4be629aa673b725d5e3cf18f777f8e4", size = 35821420 },+ { url = "https://pypi.netflix.net/packages/19630380351/pyarrow-23.0.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:65666fc269669af1ef1c14478c52222a2aa5c907f28b68fb50a203c777e4f60c", size = 44457412 },+ { url = "https://pypi.netflix.net/packages/19630384303/pyarrow-23.0.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:4d85cb6177198f3812db4788e394b757223f60d9a9f5ad6634b3e32be1525803", size = 47534285 },+ { url = "https://pypi.netflix.net/packages/19630384304/pyarrow-23.0.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1a9ff6fa4141c24a03a1a434c63c8fa97ce70f8f36bccabc18ebba905ddf0f17", size = 48157913 },+ { url = "https://pypi.netflix.net/packages/19630385873/pyarrow-23.0.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:84839d060a54ae734eb60a756aeacb62885244aaa282f3c968f5972ecc7b1ecc", size = 50582529 },+ { url = "https://pypi.netflix.net/packages/19630385874/pyarrow-23.0.0-cp311-cp311-win_amd64.whl", hash = "sha256:a149a647dbfe928ce8830a713612aa0b16e22c64feac9d1761529778e4d4eaa5", size = 27542646 },+ { url = "https://pypi.netflix.net/packages/19630388797/pyarrow-23.0.0-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:5961a9f646c232697c24f54d3419e69b4261ba8a8b66b0ac54a1851faffcbab8", size = 34189116 },+ { url = "https://pypi.netflix.net/packages/19630388798/pyarrow-23.0.0-cp312-cp312-macosx_12_0_x86_64.whl", hash = "sha256:632b3e7c3d232f41d64e1a4a043fb82d44f8a349f339a1188c6a0dd9d2d47d8a", size = 35803831 },+ { url = "https://pypi.netflix.net/packages/19630393960/pyarrow-23.0.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:76242c846db1411f1d6c2cc3823be6b86b40567ee24493344f8226ba34a81333", size = 44436452 },+ { url = "https://pypi.netflix.net/packages/19630395490/pyarrow-23.0.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:b73519f8b52ae28127000986bf228fda781e81d3095cd2d3ece76eb5cf760e1b", size = 47557396 },+ { url = "https://pypi.netflix.net/packages/19630395491/pyarrow-23.0.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:068701f6823449b1b6469120f399a1239766b117d211c5d2519d4ed5861f75de", size = 48147129 },+ { url = "https://pypi.netflix.net/packages/19630395492/pyarrow-23.0.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:1801ba947015d10e23bca9dd6ef5d0e9064a81569a89b6e9a63b59224fd060df", size = 50596642 },+ { url = "https://pypi.netflix.net/packages/19630399368/pyarrow-23.0.0-cp312-cp312-win_amd64.whl", hash = "sha256:52265266201ec25b6839bf6bd4ea918ca6d50f31d13e1cf200b4261cd11dc25c", size = 27660628 },+ { url = "https://pypi.netflix.net/packages/19630399369/pyarrow-23.0.0-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:ad96a597547af7827342ffb3c503c8316e5043bb09b47a84885ce39394c96e00", size = 34184630 },+ { url = "https://pypi.netflix.net/packages/19630401982/pyarrow-23.0.0-cp313-cp313-macosx_12_0_x86_64.whl", hash = "sha256:b9edf990df77c2901e79608f08c13fbde60202334a4fcadb15c1f57bf7afee43", size = 35796820 },+ { url = "https://pypi.netflix.net/packages/19630401983/pyarrow-23.0.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:36d1b5bc6ddcaff0083ceec7e2561ed61a51f49cce8be079ee8ed406acb6fdef", size = 44438735 },+ { url = "https://pypi.netflix.net/packages/19630403423/pyarrow-23.0.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:4292b889cd224f403304ddda8b63a36e60f92911f89927ec8d98021845ea21be", size = 47557263 },+ { url = "https://pypi.netflix.net/packages/19630403424/pyarrow-23.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:dfd9e133e60eaa847fd80530a1b89a052f09f695d0b9c34c235ea6b2e0924cf7", size = 48153529 },+ { url = "https://pypi.netflix.net/packages/19630404975/pyarrow-23.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:832141cc09fac6aab1cd3719951d23301396968de87080c57c9a7634e0ecd068", size = 50598851 },+ { url = "https://pypi.netflix.net/packages/19630413938/pyarrow-23.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:7a7d067c9a88faca655c71bcc30ee2782038d59c802d57950826a07f60d83c4c", size = 27577747 },+ { url = "https://pypi.netflix.net/packages/19630404976/pyarrow-23.0.0-cp313-cp313t-macosx_12_0_arm64.whl", hash = "sha256:ce9486e0535a843cf85d990e2ec5820a47918235183a5c7b8b97ed7e92c2d47d", size = 34446038 },+ { url = "https://pypi.netflix.net/packages/19630409645/pyarrow-23.0.0-cp313-cp313t-macosx_12_0_x86_64.whl", hash = "sha256:075c29aeaa685fd1182992a9ed2499c66f084ee54eea47da3eb76e125e06064c", size = 35921142 },+ { url = "https://pypi.netflix.net/packages/19630409646/pyarrow-23.0.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:799965a5379589510d888be3094c2296efd186a17ca1cef5b77703d4d5121f53", size = 44490374 },+ { url = "https://pypi.netflix.net/packages/19630411077/pyarrow-23.0.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:ef7cac8fe6fccd8b9e7617bfac785b0371a7fe26af59463074e4882747145d40", size = 47527896 },+ { url = "https://pypi.netflix.net/packages/19630412509/pyarrow-23.0.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:15a414f710dc927132dd67c361f78c194447479555af57317066ee5116b90e9e", size = 48210401 },+ { url = "https://pypi.netflix.net/packages/19630412510/pyarrow-23.0.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:3e0d2e6915eca7d786be6a77bf227fbc06d825a75b5b5fe9bcbef121dec32685", size = 50579677 },+ { url = "https://pypi.netflix.net/packages/19630413945/pyarrow-23.0.0-cp313-cp313t-win_amd64.whl", hash = "sha256:4b317ea6e800b5704e5e5929acb6e2dc13e9276b708ea97a39eb8b345aa2658b", size = 27631889 },+ { url = "https://pypi.netflix.net/packages/19630415381/pyarrow-23.0.0-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:20b187ed9550d233a872074159f765f52f9d92973191cd4b93f293a19efbe377", size = 34213265 },+ { url = "https://pypi.netflix.net/packages/19630415382/pyarrow-23.0.0-cp314-cp314-macosx_12_0_x86_64.whl", hash = "sha256:18ec84e839b493c3886b9b5e06861962ab4adfaeb79b81c76afbd8d84c7d5fda", size = 35819211 },+ { url = "https://pypi.netflix.net/packages/19630415383/pyarrow-23.0.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:e438dd3f33894e34fd02b26bd12a32d30d006f5852315f611aa4add6c7fab4bc", size = 44502313 },+ { url = "https://pypi.netflix.net/packages/19630416822/pyarrow-23.0.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:a244279f240c81f135631be91146d7fa0e9e840e1dfed2aba8483eba25cd98e6", size = 47585886 },+ { url = "https://pypi.netflix.net/packages/19630416823/pyarrow-23.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c4692e83e42438dba512a570c6eaa42be2f8b6c0f492aea27dec54bdc495103a", size = 48207055 },+ { url = "https://pypi.netflix.net/packages/19630418264/pyarrow-23.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae7f30f898dfe44ea69654a35c93e8da4cef6606dc4c72394068fd95f8e9f54a", size = 50619812 },+ { url = "https://pypi.netflix.net/packages/19630424045/pyarrow-23.0.0-cp314-cp314-win_amd64.whl", hash = "sha256:5b86bb649e4112fb0614294b7d0a175c7513738876b89655605ebb87c804f861", size = 28263851 },+ { url = "https://pypi.netflix.net/packages/19630418265/pyarrow-23.0.0-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:ebc017d765d71d80a3f8584ca0566b53e40464586585ac64176115baa0ada7d3", size = 34463240 },+ { url = "https://pypi.netflix.net/packages/19630419708/pyarrow-23.0.0-cp314-cp314t-macosx_12_0_x86_64.whl", hash = "sha256:0800cc58a6d17d159df823f87ad66cefebf105b982493d4bad03ee7fab84b993", size = 35935712 },+ { url = "https://pypi.netflix.net/packages/19630419709/pyarrow-23.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:3a7c68c722da9bb5b0f8c10e3eae71d9825a4b429b40b32709df5d1fa55beb3d", size = 44503523 },+ { url = "https://pypi.netflix.net/packages/19630421154/pyarrow-23.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:bd5556c24622df90551063ea41f559b714aa63ca953db884cfb958559087a14e", size = 47542490 },+ { url = "https://pypi.netflix.net/packages/19630421155/pyarrow-23.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:54810f6e6afc4ffee7c2e0051b61722fbea9a4961b46192dcfae8ea12fa09059", size = 48233605 },+ { url = "https://pypi.netflix.net/packages/19630422602/pyarrow-23.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:14de7d48052cf4b0ed174533eafa3cfe0711b8076ad70bede32cf59f744f0d7c", size = 50603979 },+ { url = "https://pypi.netflix.net/packages/19630422603/pyarrow-23.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:427deac1f535830a744a4f04a6ac183a64fcac4341b3f618e693c41b7b98d2b0", size = 28338905 },+]+[[package]]name = "pydantic"version = "2.12.5"@@ -2594,6 +2889,18 @@ wheels = [{ url = "https://pypi.netflix.net/packages/19062775857/pytest_cov-7.0.0-py3-none-any.whl", hash = "sha256:3b8e9558b16cc1479da72058bdecf8073661c7f57f7d3c5f22a1c23507f2d861", size = 22424 },]+[[package]]+name = "python-dateutil"+version = "2.9.0.post0"+source = { registry = "https://pypi.netflix.net/simple" }+dependencies = [+ { name = "six" },+]+sdist = { url = "https://pypi.netflix.net/packages/10065929740/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432 }+wheels = [+ { url = "https://pypi.netflix.net/packages/10065929741/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892 },+]+[[package]]name = "python-dotenv"version = "1.2.1"@@ -2612,6 +2919,15 @@ wheels = [{ url = "https://pypi.netflix.net/packages/19493710604/python_multipart-0.0.21-py3-none-any.whl", hash = "sha256:cf7a6713e01c87aa35387f4774e812c4361150938d20d232800f75ffcf266090", size = 24541 },]+[[package]]+name = "pytz"+version = "2025.2"+source = { registry = "https://pypi.netflix.net/simple" }+sdist = { url = "https://pypi.netflix.net/packages/18551616030/pytz-2025.2.tar.gz", hash = "sha256:360b9e3dbb49a209c21ad61809c7fb453643e048b38924c765813546746e81c3", size = 320884 }+wheels = [+ { url = "https://pypi.netflix.net/packages/18551616029/pytz-2025.2-py2.py3-none-any.whl", hash = "sha256:5ddf76296dd8c44c26eb8f4b6f35488f3ccbf6fbbd7adee0b7262d43f0ec2f00", size = 509225 },+]+[[package]]name = "pyyaml"version = "6.0.3"@@ -3063,9 +3379,15 @@ name = "scikit-learn"version = "1.8.0"source = { registry = "https://pypi.netflix.net/simple" }resolution-markers = [- "python_full_version >= '3.14'",- "python_full_version >= '3.12' and python_full_version < '3.14'",- "python_full_version == '3.11.*'",+ "python_full_version >= '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'emscripten'",+ "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'",]dependencies = [{ name = "joblib", marker = "python_full_version >= '3.11'" },@@ -3177,9 +3499,15 @@ name = "scipy"version = "1.16.3"source = { registry = "https://pypi.netflix.net/simple" }resolution-markers = [- "python_full_version >= '3.14'",- "python_full_version >= '3.12' and python_full_version < '3.14'",- "python_full_version == '3.11.*'",+ "python_full_version >= '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'win32'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform == 'emscripten'",+ "python_full_version >= '3.12' and python_full_version < '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'win32'",+ "python_full_version == '3.11.*' and sys_platform == 'emscripten'",+ "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'",]dependencies = [{ name = "numpy", version = "2.4.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },@@ -3286,6 +3614,15 @@ wheels = [{ url = "https://pypi.netflix.net/packages/431600727/shellingham-1.5.4-py2.py3-none-any.whl", hash = "sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", size = 9755 },]+[[package]]+name = "six"+version = "1.17.0"+source = { registry = "https://pypi.netflix.net/simple" }+sdist = { url = "https://pypi.netflix.net/packages/17694775990/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031 }+wheels = [+ { url = "https://pypi.netflix.net/packages/17694775989/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050 },+]+[[package]]name = "smmap"version = "5.0.2"@@ -3748,6 +4085,15 @@ wheels = [{ url = "https://pypi.netflix.net/packages/19151033439/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611 },]+[[package]]+name = "tzdata"+version = "2025.3"+source = { registry = "https://pypi.netflix.net/simple" }+sdist = { url = "https://pypi.netflix.net/packages/19473811492/tzdata-2025.3.tar.gz", hash = "sha256:de39c2ca5dc7b0344f2eba86f49d614019d29f060fc4ebc8a417896a620b56a7", size = 196772 }+wheels = [+ { url = "https://pypi.netflix.net/packages/19473811491/tzdata-2025.3-py2.py3-none-any.whl", hash = "sha256:06a47e5700f3081aab02b2e513160914ff0694bce9947d6b76ebd6bf57cfc5d1", size = 348521 },+]+[[package]]name = "urllib3"version = "2.6.3"@@ -3815,6 +4161,124 @@ wheels = [{ url = "https://pypi.netflix.net/packages/19577951672/virtualenv-20.36.1-py3-none-any.whl", hash = "sha256:575a8d6b124ef88f6f51d56d656132389f961062a9177016a50e4f507bbcc19f", size = 6008258 },]+[[package]]+name = "xxhash"+version = "3.6.0"+source = { registry = "https://pypi.netflix.net/simple" }+sdist = { url = "https://pypi.netflix.net/packages/19157650572/xxhash-3.6.0.tar.gz", hash = "sha256:f0162a78b13a0d7617b2845b90c763339d1f1d82bb04a4b07f4ab535cc5e05d6", size = 85160 }+wheels = [+ { url = "https://pypi.netflix.net/packages/19157619962/xxhash-3.6.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:87ff03d7e35c61435976554477a7f4cd1704c3596a89a8300d5ce7fc83874a71", size = 32845 },+ { url = "https://pypi.netflix.net/packages/19157619963/xxhash-3.6.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:f572dfd3d0e2eb1a57511831cf6341242f5a9f8298a45862d085f5b93394a27d", size = 30807 },+ { url = "https://pypi.netflix.net/packages/19157619964/xxhash-3.6.0-cp310-cp310-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:89952ea539566b9fed2bbd94e589672794b4286f342254fad28b149f9615fef8", size = 193786 },+ { url = "https://pypi.netflix.net/packages/19157619965/xxhash-3.6.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:48e6f2ffb07a50b52465a1032c3cf1f4a5683f944acaca8a134a2f23674c2058", size = 212830 },+ { url = "https://pypi.netflix.net/packages/19157619966/xxhash-3.6.0-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b5b848ad6c16d308c3ac7ad4ba6bede80ed5df2ba8ed382f8932df63158dd4b2", size = 211606 },+ { url = "https://pypi.netflix.net/packages/19157619967/xxhash-3.6.0-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a034590a727b44dd8ac5914236a7b8504144447a9682586c3327e935f33ec8cc", size = 444872 },+ { url = "https://pypi.netflix.net/packages/19157619968/xxhash-3.6.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8a8f1972e75ebdd161d7896743122834fe87378160c20e97f8b09166213bf8cc", size = 193217 },+ { url = "https://pypi.netflix.net/packages/19157619969/xxhash-3.6.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:ee34327b187f002a596d7b167ebc59a1b729e963ce645964bbc050d2f1b73d07", size = 210139 },+ { url = "https://pypi.netflix.net/packages/19157619970/xxhash-3.6.0-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:339f518c3c7a850dd033ab416ea25a692759dc7478a71131fe8869010d2b75e4", size = 197669 },+ { url = "https://pypi.netflix.net/packages/19157621270/xxhash-3.6.0-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:bf48889c9630542d4709192578aebbd836177c9f7a4a2778a7d6340107c65f06", size = 210018 },+ { url = "https://pypi.netflix.net/packages/19157621271/xxhash-3.6.0-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:5576b002a56207f640636056b4160a378fe36a58db73ae5c27a7ec8db35f71d4", size = 413058 },+ { url = "https://pypi.netflix.net/packages/19157621272/xxhash-3.6.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:af1f3278bd02814d6dedc5dec397993b549d6f16c19379721e5a1d31e132c49b", size = 190628 },+ { url = "https://pypi.netflix.net/packages/19157621273/xxhash-3.6.0-cp310-cp310-win32.whl", hash = "sha256:aed058764db109dc9052720da65fafe84873b05eb8b07e5e653597951af57c3b", size = 30577 },+ { url = "https://pypi.netflix.net/packages/19157621274/xxhash-3.6.0-cp310-cp310-win_amd64.whl", hash = "sha256:e82da5670f2d0d98950317f82a0e4a0197150ff19a6df2ba40399c2a3b9ae5fb", size = 31487 },+ { url = "https://pypi.netflix.net/packages/19157621275/xxhash-3.6.0-cp310-cp310-win_arm64.whl", hash = "sha256:4a082ffff8c6ac07707fb6b671caf7c6e020c75226c561830b73d862060f281d", size = 27863 },+ { url = "https://pypi.netflix.net/packages/19157621276/xxhash-3.6.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:b47bbd8cf2d72797f3c2772eaaac0ded3d3af26481a26d7d7d41dc2d3c46b04a", size = 32844 },+ { url = "https://pypi.netflix.net/packages/19157621277/xxhash-3.6.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2b6821e94346f96db75abaa6e255706fb06ebd530899ed76d32cd99f20dc52fa", size = 30809 },+ { url = "https://pypi.netflix.net/packages/19157621278/xxhash-3.6.0-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:d0a9751f71a1a65ce3584e9cae4467651c7e70c9d31017fa57574583a4540248", size = 194665 },+ { url = "https://pypi.netflix.net/packages/19157621279/xxhash-3.6.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8b29ee68625ab37b04c0b40c3fafdf24d2f75ccd778333cfb698f65f6c463f62", size = 213550 },+ { url = "https://pypi.netflix.net/packages/19157621280/xxhash-3.6.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:6812c25fe0d6c36a46ccb002f40f27ac903bf18af9f6dd8f9669cb4d176ab18f", size = 212384 },+ { url = "https://pypi.netflix.net/packages/19157621281/xxhash-3.6.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:4ccbff013972390b51a18ef1255ef5ac125c92dc9143b2d1909f59abc765540e", size = 445749 },+ { url = "https://pypi.netflix.net/packages/19157621282/xxhash-3.6.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:297b7fbf86c82c550e12e8fb71968b3f033d27b874276ba3624ea868c11165a8", size = 193880 },+ { url = "https://pypi.netflix.net/packages/19157621283/xxhash-3.6.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:dea26ae1eb293db089798d3973a5fc928a18fdd97cc8801226fae705b02b14b0", size = 210912 },+ { url = "https://pypi.netflix.net/packages/19157621284/xxhash-3.6.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:7a0b169aafb98f4284f73635a8e93f0735f9cbde17bd5ec332480484241aaa77", size = 198654 },+ { url = "https://pypi.netflix.net/packages/19157621285/xxhash-3.6.0-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:08d45aef063a4531b785cd72de4887766d01dc8f362a515693df349fdb825e0c", size = 210867 },+ { url = "https://pypi.netflix.net/packages/19157621286/xxhash-3.6.0-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:929142361a48ee07f09121fe9e96a84950e8d4df3bb298ca5d88061969f34d7b", size = 414012 },+ { url = "https://pypi.netflix.net/packages/19157626540/xxhash-3.6.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:51312c768403d8540487dbbfb557454cfc55589bbde6424456951f7fcd4facb3", size = 191409 },+ { url = "https://pypi.netflix.net/packages/19157626541/xxhash-3.6.0-cp311-cp311-win32.whl", hash = "sha256:d1927a69feddc24c987b337ce81ac15c4720955b667fe9b588e02254b80446fd", size = 30574 },+ { url = "https://pypi.netflix.net/packages/19157626542/xxhash-3.6.0-cp311-cp311-win_amd64.whl", hash = "sha256:26734cdc2d4ffe449b41d186bbeac416f704a482ed835d375a5c0cb02bc63fef", size = 31481 },+ { url = "https://pypi.netflix.net/packages/19157626543/xxhash-3.6.0-cp311-cp311-win_arm64.whl", hash = "sha256:d72f67ef8bf36e05f5b6c65e8524f265bd61071471cd4cf1d36743ebeeeb06b7", size = 27861 },+ { url = "https://pypi.netflix.net/packages/19157626544/xxhash-3.6.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:01362c4331775398e7bb34e3ab403bc9ee9f7c497bc7dee6272114055277dd3c", size = 32744 },+ { url = "https://pypi.netflix.net/packages/19157626545/xxhash-3.6.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b7b2df81a23f8cb99656378e72501b2cb41b1827c0f5a86f87d6b06b69f9f204", size = 30816 },+ { url = "https://pypi.netflix.net/packages/19157626546/xxhash-3.6.0-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:dc94790144e66b14f67b10ac8ed75b39ca47536bf8800eb7c24b50271ea0c490", size = 194035 },+ { url = "https://pypi.netflix.net/packages/19157626547/xxhash-3.6.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:93f107c673bccf0d592cdba077dedaf52fe7f42dcd7676eba1f6d6f0c3efffd2", size = 212914 },+ { url = "https://pypi.netflix.net/packages/19157626548/xxhash-3.6.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2aa5ee3444c25b69813663c9f8067dcfaa2e126dc55e8dddf40f4d1c25d7effa", size = 212163 },+ { url = "https://pypi.netflix.net/packages/19157626549/xxhash-3.6.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f7f99123f0e1194fa59cc69ad46dbae2e07becec5df50a0509a808f90a0f03f0", size = 445411 },+ { url = "https://pypi.netflix.net/packages/19157626550/xxhash-3.6.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:49e03e6fe2cac4a1bc64952dd250cf0dbc5ef4ebb7b8d96bce82e2de163c82a2", size = 193883 },+ { url = "https://pypi.netflix.net/packages/19157627869/xxhash-3.6.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:bd17fede52a17a4f9a7bc4472a5867cb0b160deeb431795c0e4abe158bc784e9", size = 210392 },+ { url = "https://pypi.netflix.net/packages/19157627870/xxhash-3.6.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:6fb5f5476bef678f69db04f2bd1efbed3030d2aba305b0fc1773645f187d6a4e", size = 197898 },+ { url = "https://pypi.netflix.net/packages/19157627871/xxhash-3.6.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:843b52f6d88071f87eba1631b684fcb4b2068cd2180a0224122fe4ef011a9374", size = 210655 },+ { url = "https://pypi.netflix.net/packages/19157627872/xxhash-3.6.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:7d14a6cfaf03b1b6f5f9790f76880601ccc7896aff7ab9cd8978a939c1eb7e0d", size = 414001 },+ { url = "https://pypi.netflix.net/packages/19157627873/xxhash-3.6.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:418daf3db71e1413cfe211c2f9a528456936645c17f46b5204705581a45390ae", size = 191431 },+ { url = "https://pypi.netflix.net/packages/19157627874/xxhash-3.6.0-cp312-cp312-win32.whl", hash = "sha256:50fc255f39428a27299c20e280d6193d8b63b8ef8028995323bf834a026b4fbb", size = 30617 },+ { url = "https://pypi.netflix.net/packages/19157627875/xxhash-3.6.0-cp312-cp312-win_amd64.whl", hash = "sha256:c0f2ab8c715630565ab8991b536ecded9416d615538be8ecddce43ccf26cbc7c", size = 31534 },+ { url = "https://pypi.netflix.net/packages/19157627876/xxhash-3.6.0-cp312-cp312-win_arm64.whl", hash = "sha256:eae5c13f3bc455a3bbb68bdc513912dc7356de7e2280363ea235f71f54064829", size = 27876 },+ { url = "https://pypi.netflix.net/packages/19157627877/xxhash-3.6.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:599e64ba7f67472481ceb6ee80fa3bd828fd61ba59fb11475572cc5ee52b89ec", size = 32738 },+ { url = "https://pypi.netflix.net/packages/19157627878/xxhash-3.6.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:7d8b8aaa30fca4f16f0c84a5c8d7ddee0e25250ec2796c973775373257dde8f1", size = 30821 },+ { url = "https://pypi.netflix.net/packages/19157629752/xxhash-3.6.0-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:d597acf8506d6e7101a4a44a5e428977a51c0fadbbfd3c39650cca9253f6e5a6", size = 194127 },+ { url = "https://pypi.netflix.net/packages/19157632008/xxhash-3.6.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:858dc935963a33bc33490128edc1c12b0c14d9c7ebaa4e387a7869ecc4f3e263", size = 212975 },+ { url = "https://pypi.netflix.net/packages/19157632009/xxhash-3.6.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ba284920194615cb8edf73bf52236ce2e1664ccd4a38fdb543506413529cc546", size = 212241 },+ { url = "https://pypi.netflix.net/packages/19157632010/xxhash-3.6.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:4b54219177f6c6674d5378bd862c6aedf64725f70dd29c472eaae154df1a2e89", size = 445471 },+ { url = "https://pypi.netflix.net/packages/19157632011/xxhash-3.6.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:42c36dd7dbad2f5238950c377fcbf6811b1cdb1c444fab447960030cea60504d", size = 193936 },+ { url = "https://pypi.netflix.net/packages/19157632012/xxhash-3.6.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f22927652cba98c44639ffdc7aaf35828dccf679b10b31c4ad72a5b530a18eb7", size = 210440 },+ { url = "https://pypi.netflix.net/packages/19157632013/xxhash-3.6.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:b45fad44d9c5c119e9c6fbf2e1c656a46dc68e280275007bbfd3d572b21426db", size = 197990 },+ { url = "https://pypi.netflix.net/packages/19157632014/xxhash-3.6.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:6f2580ffab1a8b68ef2b901cde7e55fa8da5e4be0977c68f78fc80f3c143de42", size = 210689 },+ { url = "https://pypi.netflix.net/packages/19157632015/xxhash-3.6.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:40c391dd3cd041ebc3ffe6f2c862f402e306eb571422e0aa918d8070ba31da11", size = 414068 },+ { url = "https://pypi.netflix.net/packages/19157632016/xxhash-3.6.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:f205badabde7aafd1a31e8ca2a3e5a763107a71c397c4481d6a804eb5063d8bd", size = 191495 },+ { url = "https://pypi.netflix.net/packages/19157632017/xxhash-3.6.0-cp313-cp313-win32.whl", hash = "sha256:2577b276e060b73b73a53042ea5bd5203d3e6347ce0d09f98500f418a9fcf799", size = 30620 },+ { url = "https://pypi.netflix.net/packages/19157634102/xxhash-3.6.0-cp313-cp313-win_amd64.whl", hash = "sha256:757320d45d2fbcce8f30c42a6b2f47862967aea7bf458b9625b4bbe7ee390392", size = 31542 },+ { url = "https://pypi.netflix.net/packages/19157634103/xxhash-3.6.0-cp313-cp313-win_arm64.whl", hash = "sha256:457b8f85dec5825eed7b69c11ae86834a018b8e3df5e77783c999663da2f96d6", size = 27880 },+ { url = "https://pypi.netflix.net/packages/19157634104/xxhash-3.6.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:a42e633d75cdad6d625434e3468126c73f13f7584545a9cf34e883aa1710e702", size = 32956 },+ { url = "https://pypi.netflix.net/packages/19157634105/xxhash-3.6.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:568a6d743219e717b07b4e03b0a828ce593833e498c3b64752e0f5df6bfe84db", size = 31072 },+ { url = "https://pypi.netflix.net/packages/19157634106/xxhash-3.6.0-cp313-cp313t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:bec91b562d8012dae276af8025a55811b875baace6af510412a5e58e3121bc54", size = 196409 },+ { url = "https://pypi.netflix.net/packages/19157634107/xxhash-3.6.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:78e7f2f4c521c30ad5e786fdd6bae89d47a32672a80195467b5de0480aa97b1f", size = 215736 },+ { url = "https://pypi.netflix.net/packages/19157634108/xxhash-3.6.0-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3ed0df1b11a79856df5ffcab572cbd6b9627034c1c748c5566fa79df9048a7c5", size = 214833 },+ { url = "https://pypi.netflix.net/packages/19157634109/xxhash-3.6.0-cp313-cp313t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:0e4edbfc7d420925b0dd5e792478ed393d6e75ff8fc219a6546fb446b6a417b1", size = 448348 },+ { url = "https://pypi.netflix.net/packages/19157634110/xxhash-3.6.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fba27a198363a7ef87f8c0f6b171ec36b674fe9053742c58dd7e3201c1ab30ee", size = 196070 },+ { url = "https://pypi.netflix.net/packages/19157634111/xxhash-3.6.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:794fe9145fe60191c6532fa95063765529770edcdd67b3d537793e8004cabbfd", size = 212907 },+ { url = "https://pypi.netflix.net/packages/19157634112/xxhash-3.6.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:6105ef7e62b5ac73a837778efc331a591d8442f8ef5c7e102376506cb4ae2729", size = 200839 },+ { url = "https://pypi.netflix.net/packages/19157635585/xxhash-3.6.0-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:f01375c0e55395b814a679b3eea205db7919ac2af213f4a6682e01220e5fe292", size = 213304 },+ { url = "https://pypi.netflix.net/packages/19157635586/xxhash-3.6.0-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:d706dca2d24d834a4661619dcacf51a75c16d65985718d6a7d73c1eeeb903ddf", size = 416930 },+ { url = "https://pypi.netflix.net/packages/19157635587/xxhash-3.6.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:5f059d9faeacd49c0215d66f4056e1326c80503f51a1532ca336a385edadd033", size = 193787 },+ { url = "https://pypi.netflix.net/packages/19157635588/xxhash-3.6.0-cp313-cp313t-win32.whl", hash = "sha256:1244460adc3a9be84731d72b8e80625788e5815b68da3da8b83f78115a40a7ec", size = 30916 },+ { url = "https://pypi.netflix.net/packages/19157635589/xxhash-3.6.0-cp313-cp313t-win_amd64.whl", hash = "sha256:b1e420ef35c503869c4064f4a2f2b08ad6431ab7b229a05cce39d74268bca6b8", size = 31799 },+ { url = "https://pypi.netflix.net/packages/19157635590/xxhash-3.6.0-cp313-cp313t-win_arm64.whl", hash = "sha256:ec44b73a4220623235f67a996c862049f375df3b1052d9899f40a6382c32d746", size = 28044 },+ { url = "https://pypi.netflix.net/packages/19157635591/xxhash-3.6.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:a40a3d35b204b7cc7643cbcf8c9976d818cb47befcfac8bbefec8038ac363f3e", size = 32754 },+ { url = "https://pypi.netflix.net/packages/19157635592/xxhash-3.6.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:a54844be970d3fc22630b32d515e79a90d0a3ddb2644d8d7402e3c4c8da61405", size = 30846 },+ { url = "https://pypi.netflix.net/packages/19157635593/xxhash-3.6.0-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:016e9190af8f0a4e3741343777710e3d5717427f175adfdc3e72508f59e2a7f3", size = 194343 },+ { url = "https://pypi.netflix.net/packages/19157635594/xxhash-3.6.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4f6f72232f849eb9d0141e2ebe2677ece15adfd0fa599bc058aad83c714bb2c6", size = 213074 },+ { url = "https://pypi.netflix.net/packages/19157635595/xxhash-3.6.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:63275a8aba7865e44b1813d2177e0f5ea7eadad3dd063a21f7cf9afdc7054063", size = 212388 },+ { url = "https://pypi.netflix.net/packages/19157636989/xxhash-3.6.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3cd01fa2aa00d8b017c97eb46b9a794fbdca53fc14f845f5a328c71254b0abb7", size = 445614 },+ { url = "https://pypi.netflix.net/packages/19157636990/xxhash-3.6.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0226aa89035b62b6a86d3c68df4d7c1f47a342b8683da2b60cedcddb46c4d95b", size = 194024 },+ { url = "https://pypi.netflix.net/packages/19157636991/xxhash-3.6.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c6e193e9f56e4ca4923c61238cdaced324f0feac782544eb4c6d55ad5cc99ddd", size = 210541 },+ { url = "https://pypi.netflix.net/packages/19157636992/xxhash-3.6.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:9176dcaddf4ca963d4deb93866d739a343c01c969231dbe21680e13a5d1a5bf0", size = 198305 },+ { url = "https://pypi.netflix.net/packages/19157636993/xxhash-3.6.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:c1ce4009c97a752e682b897aa99aef84191077a9433eb237774689f14f8ec152", size = 210848 },+ { url = "https://pypi.netflix.net/packages/19157636994/xxhash-3.6.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:8cb2f4f679b01513b7adbb9b1b2f0f9cdc31b70007eaf9d59d0878809f385b11", size = 414142 },+ { url = "https://pypi.netflix.net/packages/19157636995/xxhash-3.6.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:653a91d7c2ab54a92c19ccf43508b6a555440b9be1bc8be553376778be7f20b5", size = 191547 },+ { url = "https://pypi.netflix.net/packages/19157636996/xxhash-3.6.0-cp314-cp314-win32.whl", hash = "sha256:a756fe893389483ee8c394d06b5ab765d96e68fbbfe6fde7aa17e11f5720559f", size = 31214 },+ { url = "https://pypi.netflix.net/packages/19157636997/xxhash-3.6.0-cp314-cp314-win_amd64.whl", hash = "sha256:39be8e4e142550ef69629c9cd71b88c90e9a5db703fecbcf265546d9536ca4ad", size = 32290 },+ { url = "https://pypi.netflix.net/packages/19157636998/xxhash-3.6.0-cp314-cp314-win_arm64.whl", hash = "sha256:25915e6000338999236f1eb68a02a32c3275ac338628a7eaa5a269c401995679", size = 28795 },+ { url = "https://pypi.netflix.net/packages/19157636999/xxhash-3.6.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:c5294f596a9017ca5a3e3f8884c00b91ab2ad2933cf288f4923c3fd4346cf3d4", size = 32955 },+ { url = "https://pypi.netflix.net/packages/19157637000/xxhash-3.6.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1cf9dcc4ab9cff01dfbba78544297a3a01dafd60f3bde4e2bfd016cf7e4ddc67", size = 31072 },+ { url = "https://pypi.netflix.net/packages/19157639343/xxhash-3.6.0-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:01262da8798422d0685f7cef03b2bd3f4f46511b02830861df548d7def4402ad", size = 196579 },+ { url = "https://pypi.netflix.net/packages/19157639344/xxhash-3.6.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:51a73fb7cb3a3ead9f7a8b583ffd9b8038e277cdb8cb87cf890e88b3456afa0b", size = 215854 },+ { url = "https://pypi.netflix.net/packages/19157639345/xxhash-3.6.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b9c6df83594f7df8f7f708ce5ebeacfc69f72c9fbaaababf6cf4758eaada0c9b", size = 214965 },+ { url = "https://pypi.netflix.net/packages/19157639346/xxhash-3.6.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:627f0af069b0ea56f312fd5189001c24578868643203bca1abbc2c52d3a6f3ca", size = 448484 },+ { url = "https://pypi.netflix.net/packages/19157639347/xxhash-3.6.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:aa912c62f842dfd013c5f21a642c9c10cd9f4c4e943e0af83618b4a404d9091a", size = 196162 },+ { url = "https://pypi.netflix.net/packages/19157639348/xxhash-3.6.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:b465afd7909db30168ab62afe40b2fcf79eedc0b89a6c0ab3123515dc0df8b99", size = 213007 },+ { url = "https://pypi.netflix.net/packages/19157639349/xxhash-3.6.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:a881851cf38b0a70e7c4d3ce81fc7afd86fbc2a024f4cfb2a97cf49ce04b75d3", size = 200956 },+ { url = "https://pypi.netflix.net/packages/19157639350/xxhash-3.6.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:9b3222c686a919a0f3253cfc12bb118b8b103506612253b5baeaac10d8027cf6", size = 213401 },+ { url = "https://pypi.netflix.net/packages/19157639351/xxhash-3.6.0-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:c5aa639bc113e9286137cec8fadc20e9cd732b2cc385c0b7fa673b84fc1f2a93", size = 417083 },+ { url = "https://pypi.netflix.net/packages/19157639352/xxhash-3.6.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5c1343d49ac102799905e115aee590183c3921d475356cb24b4de29a4bc56518", size = 193913 },+ { url = "https://pypi.netflix.net/packages/19157639353/xxhash-3.6.0-cp314-cp314t-win32.whl", hash = "sha256:5851f033c3030dd95c086b4a36a2683c2ff4a799b23af60977188b057e467119", size = 31586 },+ { url = "https://pypi.netflix.net/packages/19157646443/xxhash-3.6.0-cp314-cp314t-win_amd64.whl", hash = "sha256:0444e7967dac37569052d2409b00a8860c2135cff05502df4da80267d384849f", size = 32526 },+ { url = "https://pypi.netflix.net/packages/19157646444/xxhash-3.6.0-cp314-cp314t-win_arm64.whl", hash = "sha256:bb79b1e63f6fd84ec778a4b1916dfe0a7c3fdb986c06addd5db3a0d413819d95", size = 28898 },+ { url = "https://pypi.netflix.net/packages/19157650567/xxhash-3.6.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:0f7b7e2ec26c1666ad5fc9dbfa426a6a3367ceaf79db5dd76264659d509d73b0", size = 30662 },+ { url = "https://pypi.netflix.net/packages/19157650568/xxhash-3.6.0-pp311-pypy311_pp73-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:5dc1e14d14fa0f5789ec29a7062004b5933964bb9b02aae6622b8f530dc40296", size = 41056 },+ { url = "https://pypi.netflix.net/packages/19157650569/xxhash-3.6.0-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:881b47fc47e051b37d94d13e7455131054b56749b91b508b0907eb07900d1c13", size = 36251 },+ { url = "https://pypi.netflix.net/packages/19157650570/xxhash-3.6.0-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c6dc31591899f5e5666f04cc2e529e69b4072827085c1ef15294d91a004bc1bd", size = 32481 },+ { url = "https://pypi.netflix.net/packages/19157650571/xxhash-3.6.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:15e0dac10eb9309508bfc41f7f9deaa7755c69e35af835db9cb10751adebc35d", size = 31565 },+]+[[package]]name = "yarl"version = "1.22.0"