Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Rewrite README + add Integration Guide
chopratejas3ebcd89d4663
diff --git a/README.md b/README.mdindex 51d9a0028..c8130f857 100644--- a/README.md+++ b/README.md@@ -29,7 +29,6 @@</a></p>----## Demo@@ -40,570 +39,164 @@----## Does It Actually Work? A Real Test--**The setup:** 100 production log entries. One critical error buried at position 67.--<details>-<summary><b>BEFORE:</b> 100 log entries (18,952 chars) - click to expand</summary>+## Quick Start-```json-[- {"timestamp": "2024-12-15T00:00:00Z", "level": "INFO", "service": "api-gateway", "message": "Request processed successfully - latency=50ms", "request_id": "req-000000", "status_code": 200},- {"timestamp": "2024-12-15T01:01:00Z", "level": "INFO", "service": "user-service", "message": "Request processed successfully - latency=51ms", "request_id": "req-000001", "status_code": 200},- {"timestamp": "2024-12-15T02:02:00Z", "level": "INFO", "service": "inventory", "message": "Request processed successfully - latency=52ms", "request_id": "req-000002", "status_code": 200},- // ... 64 more INFO entries ...- {"timestamp": "2024-12-15T03:47:23Z", "level": "FATAL", "service": "payment-gateway", "message": "Connection pool exhausted", "error_code": "PG-5523", "resolution": "Increase max_connections to 500 in config/database.yml", "affected_transactions": 1847},- // ... 32 more INFO entries ...-]-```-</details>--**AFTER:** Headroom compresses to 6 entries (1,155 chars):--```json-[- {"timestamp": "2024-12-15T00:00:00Z", "level": "INFO", "service": "api-gateway", ...},- {"timestamp": "2024-12-15T01:01:00Z", "level": "INFO", "service": "user-service", ...},- {"timestamp": "2024-12-15T02:02:00Z", "level": "INFO", "service": "inventory", ...},- {"timestamp": "2024-12-15T03:47:23Z", "level": "FATAL", "service": "payment-gateway", "error_code": "PG-5523", "resolution": "Increase max_connections to 500 in config/database.yml", "affected_transactions": 1847},- {"timestamp": "2024-12-15T02:38:00Z", "level": "INFO", "service": "inventory", ...},- {"timestamp": "2024-12-15T03:39:00Z", "level": "INFO", "service": "auth", ...}-]+```bash+pip install "headroom-ai[all]"```-**What happened:** First 3 items + the FATAL error + last 2 items. The critical error at position 67 was automatically preserved.-------**The question we asked Claude:** "What caused the outage? What's the error code? What's the fix?"--| | Baseline | Headroom |-|--|----------|----------|-| Input tokens | 10,144 | 1,260 |-| Correct answers | **4/4** | **4/4** |--Both responses: *"payment-gateway service, error PG-5523, fix: Increase max_connections to 500, 1,847 transactions affected"*--**87.6% fewer tokens. Same answer.**--Run it yourself: `python examples/needle_in_haystack_test.py`-------## Accuracy Benchmarks--> **Headroom's guarantee: compress without losing accuracy.**--We validate against established open-source benchmarks. Full methodology and reproducible tests: [Benchmarks Documentation](https://chopratejas.github.io/headroom/benchmarks/)--| Benchmark | Metric | Result | Status |-|-----------|--------|--------|--------|-| [Scrapinghub Article Extraction](https://huggingface.co/datasets/allenai/scrapinghub-article-extraction-benchmark) | F1 Score | **0.919** (baseline: 0.958) | :white_check_mark: |-| [Scrapinghub Article Extraction](https://huggingface.co/datasets/allenai/scrapinghub-article-extraction-benchmark) | Recall | **98.2%** | :white_check_mark: |-| [Scrapinghub Article Extraction](https://huggingface.co/datasets/allenai/scrapinghub-article-extraction-benchmark) | Compression | **94.9%** | :white_check_mark: |-| SmartCrusher (JSON) | Accuracy | **100%** (4/4 correct) | :white_check_mark: |-| SmartCrusher (JSON) | Compression | **87.6%** | :white_check_mark: |-| Multi-Tool Agent | Accuracy | **100%** (all findings) | :white_check_mark: |-| Multi-Tool Agent | Compression | **76.3%** | :white_check_mark: |--**Why recall matters most**: For LLM applications, capturing all relevant information is critical. 98.2% recall means nearly all content is preserved — LLMs can answer questions accurately from compressed context.--<details>-<summary><b>Run benchmarks yourself</b></summary>+```python+from headroom import compress-```bash-# Install with benchmark dependencies-pip install "headroom-ai[evals,html]" datasets+messages = [+ {"role": "user", "content": "What caused the outage?"},+ {"role": "tool", "content": huge_log_output, "tool_call_id": "call_1"},+]-# Run HTML extraction benchmark (no API key needed)-pytest tests/test_evals/test_html_oss_benchmarks.py::TestExtractionBenchmark -v -s+result = compress(messages, model="claude-sonnet-4-5-20250929")+# result.messages → same format, 50-90% fewer tokens+# result.tokens_saved → 8,000+# result.compression_ratio → 0.87-# Run QA accuracy tests (requires OPENAI_API_KEY)-pytest tests/test_evals/test_html_oss_benchmarks.py::TestQAAccuracyPreservation -v -s+response = client.messages.create(model="claude-sonnet-4-5-20250929", messages=result.messages)```-</details>+**Same answer. 87% fewer tokens.**----## Multi-Tool Agent Test: Real Function Calling--**The setup:** An Agno agent with 4 tools (GitHub Issues, ArXiv Papers, Code Search, Database Logs) investigating a memory leak. Total tool output: 62,323 chars (~15,580 tokens).--```python-from agno.agent import Agent-from agno.models.anthropic import Claude-from headroom.integrations.agno import HeadroomAgnoModel--# Wrap your model - that's it!-base_model = Claude(id="claude-sonnet-4-20250514")-model = HeadroomAgnoModel(wrapped_model=base_model)--agent = Agent(model=model, tools=[search_github, search_arxiv, search_code, query_db])-response = agent.run("Investigate the memory leak and recommend a fix")-```--**Results with Claude Sonnet:**--| | Baseline | Headroom |-|--|----------|----------|-| Tokens sent to API | 15,662 | 6,100 |-| API requests | 2 | 2 |-| Tool calls | 4 | 4 |-| Duration | 26.5s | 27.0s |+## How to Use Headroom-**76.3% fewer tokens. Same comprehensive answer.**+Headroom is a compression library, not just a proxy. Use whichever integration fits your stack:-Both found: Issue #42 (memory leak), the `cleanup_worker()` fix, OutOfMemoryError logs (7.8GB/8GB, 847 threads), and relevant research papers.+| You have... | Use this | Code |+|-------------|----------|------|+| Any Python app | `compress()` | `result = compress(messages, model="gpt-4o")` |+| LiteLLM | Callback | `litellm.callbacks = [HeadroomCallback()]` |+| Python proxy (FastAPI) | ASGI Middleware | `app.add_middleware(CompressionMiddleware)` |+| Claude Code / Cursor | Proxy | `ANTHROPIC_BASE_URL=http://localhost:8787 claude` |+| Agno agents | Wrap model | `HeadroomAgnoModel(your_model)` |+| LangChain | Wrap model | `HeadroomChatModel(your_llm)` *(experimental)* |-Run it yourself: `python examples/multi_tool_agent_test.py`+**Already have a proxy?** You don't need another one. See the **[Integration Guide](docs/integration-guide.md)** for detailed setup with LiteLLM, ASGI middleware, and direct `compress()` usage.---## How It Works-> Headroom optimizes LLM context *before* it hits the provider —-> without changing your agent logic or tools.--```mermaid-flowchart LR- User["Your App"]- Entry["Headroom"]- Transform["Context<br/>Optimization"]- LLM["LLM Provider"]- Response["Response"]-- User --> Entry --> Transform --> LLM --> Response```--### Inside Headroom--```mermaid-flowchart TB--subgraph Pipeline["Transform Pipeline"]- CA["Cache Aligner<br/><i>Stabilizes dynamic tokens</i>"]- SC["Smart Crusher<br/><i>Removes redundant tool output</i>"]- CM["Intelligent Context<br/><i>Score-based token fitting</i>"]- CA --> SC --> CM-end--subgraph CCR["CCR: Compress-Cache-Retrieve"]- Store[("Compressed<br/>Store")]- Tool["Retrieve Tool"]- Tool <--> Store-end--LLM["LLM Provider"]--CM --> LLM-SC -. "Stores originals" .-> Store-LLM -. "Requests full context<br/>if needed" .-> Tool+Your App → Headroom → LLM Provider+ ↓+ CacheAligner: stabilizes prefix for KV cache hits+ ContentRouter: routes to optimal compressor per content type+ → SmartCrusher (JSON) | CodeCompressor (code) | LLMLingua (text)+ IntelligentContext: score-based token fitting+ CCR: stores originals for retrieval if LLM needs more```-> Headroom never throws data away.-> It compresses aggressively and retrieves precisely.--### What actually happens--1. **Headroom intercepts context** — Tool outputs, logs, search results, and intermediate agent steps.--2. **Dynamic content is stabilized** — Timestamps, UUIDs, request IDs are normalized so prompts cache cleanly.--3. **Low-signal content is removed** — Repetitive or redundant data is crushed, not truncated.--4. **Original data is preserved** — Full content is stored separately and retrieved *only if the LLM asks*.--5. **Provider caches finally work** — Headroom aligns prompts so OpenAI, Anthropic, and Google caches actually hit.--For deep technical details, see [Architecture Documentation](docs/ARCHITECTURE.md).+Headroom never throws data away. It compresses aggressively and retrieves precisely.----## Why Headroom?--- **Zero code changes** - works as a transparent proxy-- **47-92% savings** - depends on your workload (tool-heavy = more savings)-- **Image compression** - 40-90% reduction via trained ML router (OpenAI, Anthropic, Google)-- **Reversible compression** - LLM retrieves original data via CCR-- **Content-aware** - code, logs, JSON, images each handled optimally-- **Provider caching** - automatic prefix optimization for cache hits-- **Framework native** - LangChain, Agno, MCP, agents supported-------## 30-Second Quickstart--### Option 1: Proxy (Zero Code Changes)--```bash-pip install "headroom-ai[all]" # Recommended for best performance-headroom proxy --port 8787-```--> **Note:** First startup downloads ML models (~500MB) for optimal compression. This is a one-time download.--**Dashboard:** Open http://localhost:8787/dashboard to see real-time stats, token savings, and request history.--Point your tools at the proxy:--```bash-# Claude Code-ANTHROPIC_BASE_URL=http://localhost:8787 claude--# Any OpenAI-compatible client-OPENAI_BASE_URL=http://localhost:8787/v1 cursor-```--**Enable Persistent Memory** - Claude remembers across conversations:--```bash-headroom proxy --memory-```--Memory auto-detects your provider (Anthropic, OpenAI, Gemini) and uses the appropriate format:-- **Anthropic**: Uses native memory tool (`memory_20250818`) - works with Claude Code subscriptions-- **OpenAI/Gemini/Others**: Uses function calling format-- All providers share the same semantic vector store for search--Set `x-headroom-user-id` header for per-user memory isolation (defaults to 'default').--**Claude Code Subscription Users** - Use MCP for CCR (Compress-Cache-Retrieve):--If you use Claude Code with a subscription (not API key), you need MCP to enable the `headroom_retrieve` tool:--```bash-# One-time setup-pip install "headroom-ai[mcp]"-headroom mcp install--# Every time you code-headroom proxy # Terminal 1-claude # Terminal 2 - now has headroom_retrieve!-```--What this does:-- Configures Claude Code to use Headroom's MCP server (`~/.claude/mcp.json`)-- When the proxy compresses large tool outputs, Claude sees markers like `[47 items compressed... hash=abc123]`-- Claude can call `headroom_retrieve` to get the full original content when needed--Check your setup:-```bash-headroom mcp status-```--<details>-<summary><b>Why MCP for subscriptions?</b></summary>--- **API users** can inject custom tools directly via the Messages API-- **Subscription users** use Claude Code's built-in tool set and can't inject tools programmatically-- **MCP** (Model Context Protocol) is Claude's official way to extend tools - it works with subscriptions--The MCP server exposes `headroom_retrieve` so Claude can request uncompressed content when the compressed summary isn't enough.-</details>--**Using AWS Bedrock, Google Vertex, or Azure?** Route through Headroom:--```bash-# AWS Bedrock - Terminal 1: Start proxy-export AWS_ACCESS_KEY_ID="AKIA..."-export AWS_SECRET_ACCESS_KEY="..."-export AWS_REGION="us-east-1"-headroom proxy --backend bedrock --region us-east-1--# AWS Bedrock - Terminal 2: Run Claude Code-export ANTHROPIC_API_KEY="sk-ant-dummy" # Any value works! Headroom ignores it.-export ANTHROPIC_BASE_URL="http://localhost:8787"-# IMPORTANT: Do NOT set CLAUDE_CODE_USE_BEDROCK=1 (Headroom handles Bedrock routing)-claude-```--<details>-<summary><b>VS Code settings.json for Bedrock</b> (click to expand)</summary>--```json-{- "claudeCode.environmentVariables": [- { "name": "ANTHROPIC_API_KEY", "value": "sk-ant-dummy" },- { "name": "ANTHROPIC_BASE_URL", "value": "http://localhost:8787" },- { "name": "AWS_ACCESS_KEY_ID", "value": "AKIA..." },- { "name": "AWS_SECRET_ACCESS_KEY", "value": "..." },- { "name": "AWS_REGION", "value": "us-east-1" }- ]-}-```--**Do NOT include** `CLAUDE_CODE_USE_BEDROCK` - Headroom handles the Bedrock routing.-</details>--**Using OpenRouter?** Access 400+ models through a single API:--```bash-# OpenRouter - Terminal 1: Start proxy-export OPENROUTER_API_KEY="sk-or-v1-..."-headroom proxy --backend openrouter--# OpenRouter - Terminal 2: Run your client-export ANTHROPIC_API_KEY="sk-ant-dummy" # Any value works! Headroom ignores it.-export ANTHROPIC_BASE_URL="http://localhost:8787"-# Use OpenRouter model names in your requests:-# - anthropic/claude-3.5-sonnet-# - openai/gpt-4o-# - google/gemini-pro-# - meta-llama/llama-3-70b-instruct-# See all models: https://openrouter.ai/models-```--```bash-# Google Vertex AI-headroom proxy --backend vertex_ai --region us-central1--# Azure OpenAI-headroom proxy --backend azure --region eastus-```--### Option 2: LangChain Integration--```bash-pip install "headroom-ai[langchain]"-```--```python-from langchain_openai import ChatOpenAI-from headroom.integrations import HeadroomChatModel--# Wrap your model - that's it!-llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))--# Use exactly like before-response = llm.invoke("Hello!")-```--See the full [LangChain Integration Guide](docs/langchain.md) for memory, retrievers, agents, and more.--### Option 3: Agno Integration--```bash-pip install "headroom-ai[agno]"-```--```python-from agno.agent import Agent-from agno.models.openai import OpenAIChat-from headroom.integrations.agno import HeadroomAgnoModel--# Wrap your model - that's it!-model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))-agent = Agent(model=model)--# Use exactly like before-response = agent.run("Hello!")+## Verified Performance-# Check savings-print(f"Tokens saved: {model.total_tokens_saved}")-```+| Scenario | Tokens Before | Tokens After | Savings |+|----------|--------------|-------------|---------|+| Code search (100 results) | 17,765 | 1,408 | **92%** |+| SRE incident debugging | 65,694 | 5,118 | **92%** |+| Codebase exploration | 78,502 | 41,254 | **47%** |+| GitHub issue triage | 54,174 | 14,761 | **73%** |-See the full [Agno Integration Guide](docs/agno.md) for hooks, multi-provider support, and more.+**Overhead**: 1-5ms. **Accuracy**: [benchmarked](docs/benchmarks.md) across 12+ datasets.----## Framework Integrations+## Integrations-| Framework | Integration | Docs |-|-----------|-------------|------|-| **LangChain** | `HeadroomChatModel`, memory, retrievers, agents | [Guide](docs/langchain.md) |-| **Agno** | `HeadroomAgnoModel`, hooks, multi-provider | [Guide](docs/agno.md) |-| **MCP** | Claude Code subscription support via `headroom mcp install` | [Guide](docs/mcp.md) |-| **Any OpenAI Client** | Proxy server | [Guide](docs/proxy.md) |+| Integration | Status | Docs |+|-------------|--------|------|+| `compress()` — one function | **Stable** | [Integration Guide](docs/integration-guide.md) |+| LiteLLM callback | **Stable** | [Integration Guide](docs/integration-guide.md#litellm) |+| ASGI middleware | **Stable** | [Integration Guide](docs/integration-guide.md#asgi-middleware) |+| Proxy server | **Stable** | [Proxy Docs](docs/proxy.md) |+| Agno | **Stable** | [Agno Guide](docs/agno.md) |+| MCP (Claude Code) | **Stable** | [MCP Guide](docs/mcp.md) |+| Strands | **Stable** | [Strands Guide](docs/strands.md) |+| LangChain | **Experimental** | [LangChain Guide](docs/langchain.md) |---## Features-| Feature | Description | Docs |-|---------|-------------|------|-| **Image Compression** | 40-90% token reduction for images via trained ML router | [Image Compression](docs/image-compression.md) |-| **Memory** | Persistent memory across conversations (zero-latency inline extraction) | [Memory](docs/memory.md) |-| **Universal Compression** | ML-based content detection + structure-preserving compression | [Compression](docs/compression.md) |-| **SmartCrusher** | Compresses JSON tool outputs statistically | [Transforms](docs/transforms.md) |-| **CacheAligner** | Stabilizes prefixes for provider caching | [Transforms](docs/transforms.md) |-| **IntelligentContext** | Score-based context dropping with TOIN-learned importance | [Transforms](docs/transforms.md) |-| **CCR** | Reversible compression with automatic retrieval | [CCR Guide](docs/ccr.md) |-| **MCP Server** | Claude Code subscription support via `headroom mcp install` | [MCP Guide](docs/mcp.md) |-| **LangChain** | Memory, retrievers, agents, streaming | [LangChain](docs/langchain.md) |-| **Agno** | Agent framework integration with hooks | [Agno](docs/agno.md) |-| **Text Utilities** | Opt-in compression for search/logs | [Text Compression](docs/text-compression.md) |-| **LLMLingua-2** | ML-based 20x compression (opt-in) | [LLMLingua](docs/llmlingua.md) |-| **Code-Aware** | AST-based code compression (tree-sitter) | [Transforms](docs/transforms.md) |-| **Evals Framework** | Prove compression preserves accuracy (12+ datasets) | [Evals](headroom/evals/README.md) |+| Feature | What it does |+|---------|-------------|+| **Content Router** | Auto-detects content type, routes to optimal compressor |+| **SmartCrusher** | Statistically compresses JSON arrays (tool outputs, API responses) |+| **CodeCompressor** | AST-aware code compression (Python, JS, Go, Rust, Java) |+| **LLMLingua-2** | ML-based 20x text compression |+| **CCR** | Reversible compression — LLM retrieves originals when needed |+| **CacheAligner** | Stabilizes prefixes for provider KV cache hits |+| **IntelligentContext** | Score-based context management with learned importance |+| **Image Compression** | 40-90% token reduction via trained ML router |+| **Memory** | Persistent memory across conversations |+| **Compression Hooks** | Customize compression with pre/post hooks |+| **Query Echo** | Re-injects user question after compressed data for better attention |----## Evaluation Framework: Prove It Works--Skeptical? Good. We built a comprehensive evaluation framework to **prove** compression preserves accuracy.+## Cloud Providers```bash-# Install evals-pip install "headroom-ai[evals]"--# Quick sanity check (5 samples)-python -m headroom.evals quick--# Run on real datasets-python -m headroom.evals benchmark --dataset hotpotqa -n 100-```--### How Evals Work--```-Original Context ───► LLM ───► Response A- │-Compressed Context ─► LLM ───► Response B- │- Compare A vs B │- ─────────────────- F1 Score: 0.95- Semantic Similarity: 0.97- Ground Truth Match: ✓- ─────────────────- PASS: Accuracy preserved+headroom proxy --backend bedrock --region us-east-1 # AWS Bedrock+headroom proxy --backend vertex_ai --region us-central1 # Google Vertex+headroom proxy --backend azure # Azure OpenAI+headroom proxy --backend openrouter # OpenRouter (400+ models)```-### Available Datasets (12+)--| Category | Datasets |-|----------|----------|-| **RAG** | HotpotQA, Natural Questions, TriviaQA, MS MARCO, SQuAD |-| **Long Context** | LongBench (4K-128K tokens), NarrativeQA |-| **Tool Use** | BFCL (function calling), ToolBench, Built-in samples |-| **Code** | CodeSearchNet, HumanEval |--### CI Integration--```yaml-# GitHub Actions-- name: Run Compression Evals- run: python -m headroom.evals quick -n 20- env:- ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}-```--Exit code 0 if accuracy ≥ 90%, 1 otherwise.--See the full [Evals Documentation](headroom/evals/README.md) for datasets, metrics, and programmatic API.-------## Verified Performance--These numbers are from actual API calls, not estimates:--| Scenario | Before | After | Savings | Verified |-|----------|--------|-------|---------|----------|-| Code search (100 results) | 17,765 tokens | 1,408 tokens | 92% | Claude Sonnet |-| SRE incident debugging | 65,694 tokens | 5,118 tokens | 92% | GPT-4o |-| Codebase exploration | 78,502 tokens | 41,254 tokens | 47% | GPT-4o |-| GitHub issue triage | 54,174 tokens | 14,761 tokens | 73% | GPT-4o |--**Overhead**: ~1-5ms compression latency--**When savings are highest**: Tool-heavy workloads (search, logs, database queries)-**When savings are lowest**: Conversation-heavy workloads with minimal tool use-------## Providers--| Provider | Token Counting | Cache Optimization |-|----------|----------------|-------------------|-| OpenAI | tiktoken (exact) | Automatic prefix caching |-| Anthropic | Official API | cache_control blocks |-| Google | Official API | Context caching |-| Cohere | Official API | - |-| Mistral | Official tokenizer | - |--New models auto-supported via naming pattern detection.-------## Safety Guarantees--- **Never removes human content** - user/assistant messages preserved-- **Never breaks tool ordering** - tool calls and responses stay paired-- **Parse failures are no-ops** - malformed content passes through unchanged-- **Compression is reversible** - LLM retrieves original data via CCR----## Installation```bash-# Recommended: Install everything for best compression performance-pip install "headroom-ai[all]"--# Or install specific components-pip install headroom-ai # SDK only-pip install "headroom-ai[proxy]" # Proxy server-pip install "headroom-ai[mcp]" # MCP server for Claude Code subscriptions-pip install "headroom-ai[langchain]" # LangChain integration-pip install "headroom-ai[agno]" # Agno agent framework-pip install "headroom-ai[evals]" # Evaluation framework-pip install "headroom-ai[code]" # AST-based code compression-pip install "headroom-ai[llmlingua]" # ML-based compression+pip install headroom-ai # Core library+pip install "headroom-ai[all]" # Everything (recommended)+pip install "headroom-ai[proxy]" # Proxy server+pip install "headroom-ai[mcp]" # MCP for Claude Code+pip install "headroom-ai[agno]" # Agno integration+pip install "headroom-ai[langchain]" # LangChain (experimental)+pip install "headroom-ai[evals]" # Evaluation framework```-**Requirements**: Python 3.10+--> **First-time startup:** Headroom downloads ML models (~500MB) on first run for optimal compression. This is cached locally and only happens once.+Python 3.10+---## Documentation-| Guide | Description |-|-------|-------------|-| [Memory Guide](docs/memory.md) | Persistent memory for LLMs |-| [Compression Guide](docs/compression.md) | Universal compression with ML detection |+| | |+|---|---|+| [Integration Guide](docs/integration-guide.md) | LiteLLM, ASGI, compress(), proxy |+| [Proxy Docs](docs/proxy.md) | Proxy server configuration |+| [Architecture](docs/ARCHITECTURE.md) | How the pipeline works |+| [CCR Guide](docs/ccr.md) | Reversible compression |+| [Benchmarks](docs/benchmarks.md) | Accuracy validation || [Evals Framework](headroom/evals/README.md) | Prove compression preserves accuracy |-| [LangChain Integration](docs/langchain.md) | Full LangChain support |-| [Agno Integration](docs/agno.md) | Full Agno agent framework support |-| [SDK Guide](docs/sdk.md) | Fine-grained control |-| [Proxy Guide](docs/proxy.md) | Production deployment |+| [Memory](docs/memory.md) | Persistent memory |+| [Agno](docs/agno.md) | Agno agent framework |+| [MCP](docs/mcp.md) | Claude Code subscriptions || [Configuration](docs/configuration.md) | All options |-| [CCR Guide](docs/ccr.md) | Reversible compression |-| [MCP Guide](docs/mcp.md) | Claude Code subscription support |-| [Metrics](docs/metrics.md) | Monitoring |-| [Troubleshooting](docs/troubleshooting.md) | Common issues |-------## Who's Using Headroom?--> Add your project here! [Open a PR](https://github.com/chopratejas/headroom/pulls) or [start a discussion](https://github.com/chopratejas/headroom/discussions).---## Contributing```bash-git clone https://github.com/chopratejas/headroom.git-cd headroom-pip install -e ".[dev]"-pytest+git clone https://github.com/chopratejas/headroom.git && cd headroom+pip install -e ".[dev]" && pytest```-See [CONTRIBUTING.md](CONTRIBUTING.md) for details.----## License-Apache License 2.0 - see [LICENSE](LICENSE).-------<p align="center">- <sub>Built for the AI developer community</sub>-</p>+Apache License 2.0 — see [LICENSE](LICENSE).diff --git a/docs/integration-guide.md b/docs/integration-guide.mdnew file mode 100644index 000000000..659649c30--- /dev/null+++ b/docs/integration-guide.md@@ -0,0 +1,292 @@+# Integration Guide++You don't need to run the Headroom proxy. Headroom is a compression library that works with **any** LLM client, proxy, or framework.++## Pick Your Path++| You have... | Use this | Setup |+|-------------|----------|-------|+| Any Python app | [`compress()`](#compress-function) | 2 lines |+| LiteLLM | [LiteLLM callback](#litellm) | 1 line |+| A Python proxy (FastAPI, custom) | [ASGI middleware](#asgi-middleware) | 1 line |+| Claude Code / Cursor | [Headroom proxy](#proxy) | 1 env var |+| Agno agents | [Agno integration](#agno) | Wrap model |+| LangChain | [LangChain integration](#langchain) | Wrap model |+| Non-Python app | [Headroom proxy](#proxy) | HTTP |++---++## compress() Function++The simplest integration. Works with any LLM client.++```python+from headroom import compress++# Before sending to your LLM:+result = compress(messages, model="claude-sonnet-4-5-20250929")+response = your_client.create(messages=result.messages) # Fewer tokens, same answer++print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")+```++### With Anthropic SDK++```python+from anthropic import Anthropic+from headroom import compress++client = Anthropic()+messages = [+ {"role": "user", "content": "What went wrong?"},+ {"role": "assistant", "content": "Let me check.", "tool_use": [...]},+ {"role": "user", "content": [{"type": "tool_result", "content": huge_json}]},+]++compressed = compress(messages, model="claude-sonnet-4-5-20250929")+response = client.messages.create(+ model="claude-sonnet-4-5-20250929",+ messages=compressed.messages,+ max_tokens=1000,+)+```++### With OpenAI SDK++```python+from openai import OpenAI+from headroom import compress++client = OpenAI()+messages = [+ {"role": "user", "content": "Analyze these results"},+ {"role": "tool", "content": big_json_output, "tool_call_id": "call_1"},+]++compressed = compress(messages, model="gpt-4o")+response = client.chat.completions.create(+ model="gpt-4o",+ messages=compressed.messages,+)+```++### With LiteLLM (direct)++```python+import litellm+from headroom import compress++messages = [...]+compressed = compress(messages, model="bedrock/claude-sonnet")+response = litellm.completion(model="bedrock/claude-sonnet", messages=compressed.messages)+```++### With any HTTP client++```python+import httpx+from headroom import compress++compressed = compress(messages, model="claude-sonnet-4-5-20250929")+httpx.post("https://api.anthropic.com/v1/messages", json={+ "model": "claude-sonnet-4-5-20250929",+ "messages": compressed.messages,+}, headers={"X-Api-Key": api_key, "anthropic-version": "2023-06-01"})+```++### What compress() returns++```python+result = compress(messages, model="gpt-4o")+result.messages # list[dict] — compressed messages, same format as input+result.tokens_before # int — original token count+result.tokens_after # int — compressed token count+result.tokens_saved # int — tokens removed+result.compression_ratio # float — 0.0 (no savings) to 1.0 (100% removed)+result.transforms_applied # list[str] — what ran (e.g., ["router:smart_crusher:0.35"])+```++---++## LiteLLM++If you're already using LiteLLM as your LLM gateway, add Headroom as a callback:++```python+import litellm+from headroom.integrations.litellm_callback import HeadroomCallback++litellm.callbacks = [HeadroomCallback()]++# All calls now compressed automatically+response = litellm.completion(model="gpt-4o", messages=[...])+response = litellm.completion(model="bedrock/claude-sonnet", messages=[...])+response = litellm.completion(model="azure/gpt-4o", messages=[...])+```++The callback compresses messages in LiteLLM's `pre_call_hook` before they're sent to the provider. Works with all 100+ LiteLLM-supported providers.++### With LiteLLM Proxy++If you run LiteLLM as a proxy server, use the ASGI middleware instead:++```python+# In your LiteLLM proxy startup+from litellm.proxy.proxy_server import app+from headroom.integrations.asgi import CompressionMiddleware++app.add_middleware(CompressionMiddleware)+```++Or use the callback in your LiteLLM config:++```yaml+# litellm_config.yaml+litellm_settings:+ callbacks: ["headroom.integrations.litellm_callback.HeadroomCallback"]+```++---++## ASGI Middleware++Drop-in middleware for any ASGI application (FastAPI, Starlette, LiteLLM proxy, custom proxies).++```python+from headroom.integrations.asgi import CompressionMiddleware++# FastAPI+app = FastAPI()+app.add_middleware(CompressionMiddleware)++# Starlette+app = Starlette(routes=[...])+app.add_middleware(CompressionMiddleware)++# LiteLLM proxy+from litellm.proxy.proxy_server import app+app.add_middleware(CompressionMiddleware)+```++The middleware intercepts POST requests to `/v1/messages`, `/v1/chat/completions`, `/v1/responses`, and `/chat/completions`. All other requests pass through untouched.++Response headers include:+- `x-headroom-compressed: true` — compression was applied+- `x-headroom-tokens-saved: 1234` — tokens removed++---++## Proxy++The Headroom proxy is a standalone HTTP server. Best for non-Python apps or tools that only support base URL configuration (Claude Code, Cursor).++```bash+pip install "headroom-ai[all]"+headroom proxy --port 8787+```++```bash+# Claude Code+ANTHROPIC_BASE_URL=http://localhost:8787 claude++# Cursor / Any OpenAI client+OPENAI_BASE_URL=http://localhost:8787/v1 cursor+```++### With Cloud Providers++```bash+# AWS Bedrock+headroom proxy --backend bedrock --region us-east-1++# Google Vertex AI+headroom proxy --backend vertex_ai --region us-central1++# Azure OpenAI+headroom proxy --backend azure++# OpenRouter (400+ models)+OPENROUTER_API_KEY=sk-or-... headroom proxy --backend openrouter+```++See [Proxy Documentation](proxy.md) for all options.++---++## Agno++Full integration with the Agno agent framework.++```python+from agno.agent import Agent+from agno.models.anthropic import Claude+from headroom.integrations.agno import HeadroomAgnoModel++model = HeadroomAgnoModel(Claude(id="claude-sonnet-4-20250514"))+agent = Agent(model=model, tools=[your_tools])+response = agent.run("Investigate the issue")++print(f"Tokens saved: {model.total_tokens_saved}")+```++See [Agno Guide](agno.md) for hooks, multi-provider, and streaming.++---++## LangChain++> **Experimental.** Core compression works. Streaming callbacks and async chains are still being tested.++```python+from langchain_openai import ChatOpenAI+from headroom.integrations import HeadroomChatModel++llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))+response = llm.invoke("Hello!")+```++See [LangChain Guide](langchain.md) for details and known limitations.++---++## Compression Hooks (Advanced)++Customize compression behavior without modifying Headroom's code:++```python+from headroom import compress, CompressionHooks, CompressContext++class MyHooks(CompressionHooks):+ def pre_compress(self, messages, ctx):+ # Modify messages before compression (dedup, filter, inject)+ return messages++ def compute_biases(self, messages, ctx):+ # Per-message compression aggressiveness+ # >1.0 = keep more, <1.0 = compress more+ return {5: 1.5, 6: 0.5} # Keep message 5, compress message 6++ def post_compress(self, event):+ # Observe results (logging, analytics, learning)+ print(f"Saved {event.tokens_saved} tokens")++result = compress(messages, model="gpt-4o", hooks=MyHooks())+```++See [Architecture](ARCHITECTURE.md) for how hooks integrate with the pipeline.++---++## FAQ++**Q: Does Headroom change the response format?**+No. Your LLM returns the same response format. Headroom only modifies the input messages.++**Q: What if compression removes something the LLM needs?**+Headroom stores originals in CCR (Compress-Cache-Retrieve). The LLM can call `headroom_retrieve` to get full uncompressed content. Compression summaries tell the LLM what's available.++**Q: Does it work with streaming?**+Yes. Compression happens before the request is sent. Streaming responses are unaffected.++**Q: How much latency does it add?**+1-5ms for compression. The token savings typically save more time on the LLM side than compression adds.diff --git a/headroom/compress.py b/headroom/compress.pyindex 187c9ac31..bec533f73 100644--- a/headroom/compress.py+++ b/headroom/compress.py@@ -183,17 +183,13 @@ def _get_pipeline() -> Any:if _pipeline is not None:return _pipeline- from headroom.transforms import ContentRouter, SmartCrusher, TransformPipeline-- _pipeline = TransformPipeline(- transforms=[- ContentRouter(),- SmartCrusher(),- ],- # No provider needed — pipeline uses tokenizer registry which- # auto-detects the right tokenizer per model:- # OpenAI → tiktoken (exact), Anthropic → calibrated estimation,- # Open models → HuggingFace (if installed)- )+ from headroom.transforms import TransformPipeline++ # Default pipeline: CacheAligner → ContentRouter → IntelligentContext+ # CacheAligner: stabilizes prefix for provider KV cache hits+ # ContentRouter: routes to the right compressor per content type+ # (SmartCrusher for JSON, CodeCompressor for code, LLMLingua for text)+ # IntelligentContext: enforces token limits with score-based dropping+ _pipeline = TransformPipeline()logger.debug("Headroom compression pipeline initialized")return _pipeline