Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Revamp docs site: new theme, comprehensive landing page, complete nav
chopratejas0db650fd720c
diff --git a/docs/index.md b/docs/index.mdindex 0f51a9cce..08f9307d9 100644--- a/docs/index.md+++ b/docs/index.md@@ -1,176 +1,372 @@# Headroom+<div class="hero" markdown>+**The Context Optimization Layer for LLM Applications**-Tool outputs are 70-95% redundant. Headroom compresses that away—without losing information.+Compress everything your AI agent reads. Same answers, fraction of the tokens.++</div>++<div class="badges" markdown>++[](https://pypi.org/project/headroom-ai/)+[](https://pypi.org/project/headroom-ai/)+[](https://github.com/chopratejas/headroom/blob/main/LICENSE)+[](https://discord.gg/QEyNhaGA)++</div>++<div class="stats-bar" markdown>++<div class="stat">+<span class="number">87%</span>+<span class="label">Avg Token Reduction</span>+</div>+<div class="stat">+<span class="number">100%</span>+<span class="label">Answer Accuracy</span>+</div>+<div class="stat">+<span class="number">6</span>+<span class="label">Compression Algorithms</span>+</div>+<div class="stat">+<span class="number">100+</span>+<span class="label">LLM Providers</span>+</div>++</div>----## Quick Install+## What It Does++Every tool call, DB query, file read, and RAG retrieval your agent makes is 70-95% boilerplate. Headroom compresses it away before it hits the model. The LLM sees less noise, responds faster, and costs less.-```bash-pip install headroom-ai[all]```+Your Agent / App+ │+ │ tool outputs, logs, DB reads, RAG results, file reads, API responses+ ▼+ Headroom ← proxy, Python library, or framework integration+ │+ ▼+ LLM Provider (OpenAI, Anthropic, Google, Bedrock, 100+ via LiteLLM)+```++Headroom works as a **transparent proxy** (zero code changes), a **Python function** (`compress()`), or a **framework integration** (LangChain, Agno, Strands, LiteLLM, MCP).++---## Quick Start-### Option 1: Proxy (Zero Code Changes)+=== "Proxy (Zero Code Changes)"-Start the proxy:+ ```bash+ pip install "headroom-ai[all]"+ headroom proxy+ ```-```bash-headroom proxy-```+ ```bash+ # Point any tool at the proxy+ ANTHROPIC_BASE_URL=http://localhost:8787 claude+ OPENAI_BASE_URL=http://localhost:8787/v1 your-app+ ```-Point your tools at it:+ That's it. Your existing code works unchanged, with 40-90% fewer tokens.-```bash-ANTHROPIC_BASE_URL=http://localhost:8787 claude+=== "Python SDK"++ ```python+ from headroom import compress++ result = compress(messages, model="claude-sonnet-4-5-20250929")+ response = client.messages.create(+ model="claude-sonnet-4-5-20250929",+ messages=result.messages,+ )+ print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")+ ```++ Works with any Python LLM client. [Full SDK guide →](sdk.md)++=== "Coding Agents"++ ```bash+ headroom wrap claude # Claude Code+ headroom wrap codex # OpenAI Codex CLI+ headroom wrap aider # Aider+ headroom wrap cursor # Cursor+ ```++ Starts the proxy, points your tool at it, compresses everything automatically.++=== "LiteLLM Callback"++ ```python+ import litellm+ from headroom.integrations.litellm_callback import HeadroomCallback++ litellm.callbacks = [HeadroomCallback()]+ # All 100+ providers now compressed automatically+ ```++---++## Framework Integrations++<div class="grid-container" markdown>++<div class="grid-item" markdown>++### LangChain++Wrap any chat model. Supports memory, retrievers, tools, streaming, async.++```python+from headroom.integrations import HeadroomChatModel++llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))```-That's it. Your existing code works unchanged, with 40-90% fewer tokens.+[LangChain Guide →](langchain.md)++</div>-### Option 2: Python SDK+<div class="grid-item" markdown>++### Agno++Full agent framework integration with observability hooks.```python-from headroom import compress+from headroom.integrations.agno import HeadroomAgnoModel-# Compress messages before sending to LLM-result = compress(messages, model="claude-sonnet-4-5-20250929")-response = client.messages.create(- model="claude-sonnet-4-5-20250929",- messages=result.messages,-)+model = HeadroomAgnoModel(Claude(id="claude-sonnet-4-20250514"))+agent = Agent(model=model)+```-print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")+[Agno Guide →](agno.md)++</div>++<div class="grid-item" markdown>++### Strands++Model wrapping + tool output hook provider for Strands Agents.++```python+from headroom.integrations.strands import HeadroomStrandsModel++model = HeadroomStrandsModel(wrapped_model=bedrock_model)+agent = Agent(model=model)```----+[Strands Guide →](strands.md)-## Why Headroom?+</div>-| Problem | Solution |-|---------|----------|-| Tool outputs bloat context with repetitive JSON | Statistical compression removes redundancy |-| Dynamic content breaks provider caching | Cache alignment stabilizes prefixes |-| Long conversations exceed context limits | Intelligent scoring drops low-value messages |-| Compressed data might be needed later | CCR stores originals for on-demand retrieval |+<div class="grid-item" markdown>----+### MCP Tools-## Results+Three tools for Claude Code, Cursor, or any MCP client: `headroom_compress`, `headroom_retrieve`, `headroom_stats`.++```bash+headroom mcp install && claude+```-**100 log entries. One critical error buried at position 67.**+[MCP Guide →](mcp.md)-| Metric | Baseline | Headroom |-|--------|----------|----------|-| Input tokens | 10,144 | 1,260 |-| Correct answers | 4/4 | 4/4 |+</div>-**87.6% fewer tokens. Same answer.**+</div>-The FATAL error was automatically preserved—no configuration needed.+[All integration patterns →](integration-guide.md){ .md-button }---## How It Works+Headroom runs a three-stage pipeline on every request:++```mermaid+graph LR+ A[Your Prompt] --> B[CacheAligner]+ B --> C[ContentRouter]+ C --> D[IntelligentContext]+ D --> E[LLM Provider]++ C -->|JSON| F[SmartCrusher]+ C -->|Code| G[CodeCompressor]+ C -->|Text| H[Kompress]+ C -->|Logs| I[LogCompressor]++ F --> D+ G --> D+ H --> D+ I --> D```-Your App → Headroom → LLM Provider- ↓- Compression- Caching- Retrieval-```-1. **Intercepts context** — Tool outputs, logs, search results-2. **Compresses intelligently** — Keeps errors, outliers, boundaries-3. **Stores originals** — Full data available if LLM requests it-4. **Aligns for caching** — Provider caches actually hit+**Stage 1: CacheAligner** — Stabilizes message prefixes so the provider's KV cache actually hits. Claude offers a 90% read discount on cached prefixes; CacheAligner makes that work.++**Stage 2: ContentRouter** — Auto-detects content type (JSON, code, logs, search results, diffs, HTML, plain text) and routes each to the optimal compressor:++| Content Type | Compressor | How It Works |+|-------------|-----------|-------------|+| JSON arrays | **SmartCrusher** | Statistical analysis: keeps errors, anomalies, boundaries. No hardcoded rules. |+| Source code | **CodeCompressor** | AST-aware (tree-sitter). Preserves function signatures, collapses bodies. |+| Plain text | **Kompress** | ModernBERT token classification. Removes redundant tokens while preserving meaning. |+| Build/test logs | **LogCompressor** | Keeps failures, errors, warnings. Drops passing noise. |+| Search results | **SearchCompressor** | Ranks by relevance to user query, keeps top matches. |+| Git diffs | **DiffCompressor** | Preserves change hunks, drops unchanged context. |+| HTML | **HTMLExtractor** | Strips markup, extracts readable content. |++**Stage 3: IntelligentContext** — If the conversation still exceeds the model's context limit, scores each message by importance (recency, references, density) and drops the lowest-value ones.++**Nothing is lost.** Compressed content goes into the CCR store (Compress-Cache-Retrieve). The LLM gets a `headroom_retrieve` tool and can fetch full originals when it needs more detail.++[Full architecture deep dive →](ARCHITECTURE.md)----## Integrations+## Results-=== "LangChain"+**100 production log entries. One critical error buried at position 67.**- ```python- from langchain_openai import ChatOpenAI- from headroom.integrations import HeadroomChatModel+| Metric | Baseline | Headroom |+|--------|----------|----------|+| Input tokens | 10,144 | 1,260 |+| Correct answers | **4/4** | **4/4** |- llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))- response = llm.invoke("Hello!")- ```+**87.6% fewer tokens. Same answer.** The FATAL error was automatically preserved — not by keyword matching, but by statistical analysis of field variance.-=== "Agno"+### Real Workloads- ```python- from agno.agent import Agent- from agno.models.openai import OpenAIChat- from headroom.integrations.agno import HeadroomAgnoModel+| Scenario | Before | After | Savings |+|----------|--------|-------|---------|+| Code search (100 results) | 17,765 | 1,408 | **92%** |+| SRE incident debugging | 65,694 | 5,118 | **92%** |+| Codebase exploration | 78,502 | 41,254 | **47%** |+| GitHub issue triage | 54,174 | 14,761 | **73%** |- model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))- agent = Agent(model=model)- ```+### Accuracy Benchmarks-=== "Strands"+| Benchmark | Category | N | Accuracy | Compression |+|-----------|----------|---|----------|-------------|+| GSM8K | Math | 100 | 0.870 | 0.000 delta |+| TruthfulQA | Factual | 100 | 0.560 | +0.030 delta |+| SQuAD v2 | QA | 100 | **97%** | 19% reduction |+| BFCL | Tool/Function | 100 | **97%** | 32% reduction |+| CCR Needle | Lossless | 50 | **100%** | 77% reduction |- ```python- from strands import Agent- from strands.models.bedrock import BedrockModel- from headroom.integrations.strands import HeadroomStrandsModel-- model = HeadroomStrandsModel(wrapped_model=BedrockModel(- model_id="us.anthropic.claude-sonnet-4-20250514-v1:0"- ))- agent = Agent(model=model)- ```+[Full benchmark methodology →](benchmarks.md) | [Known limitations →](LIMITATIONS.md)++----=== "AWS Bedrock"+## Key Features++<div class="grid-container" markdown>++<div class="grid-item" markdown>+### Lossless Compression (CCR)+Compresses aggressively, stores originals, gives the LLM a tool to retrieve full details. Nothing is thrown away.+[Learn more →](ccr.md)+</div>++<div class="grid-item" markdown>+### Smart Content Detection+Auto-detects JSON, code, logs, text, diffs, HTML. Routes each to the best compressor. Zero configuration needed.+[Learn more →](compression.md)+</div>++<div class="grid-item" markdown>+### Cache Optimization+Stabilizes prefixes so provider KV caches hit. Tracks frozen messages to preserve the 90% read discount.+[Learn more →](ccr.md)+</div>++<div class="grid-item" markdown>+### Image Compression+40-90% token reduction via trained ML router. Automatically selects resize/quality tradeoff per image.+[Learn more →](image-compression.md)+</div>++<div class="grid-item" markdown>+### Persistent Memory+Hierarchical memory (user/session/agent/turn) with SQLite + HNSW backends. Survives across conversations.+[Learn more →](memory.md)+</div>++<div class="grid-item" markdown>+### Failure Learning+Reads past sessions, finds failed tool calls, correlates with what succeeded, writes learnings to CLAUDE.md.+[Learn more →](learn.md)+</div>++<div class="grid-item" markdown>+### Multi-Agent Context+Compress what moves between agents. Any framework.+```python+ctx = SharedContext()+ctx.put("research", big_output)+summary = ctx.get("research") # ~80% smaller+```+[Learn more →](shared-context.md)+</div>- ```bash- # Start proxy with Bedrock backend- headroom proxy --backend bedrock --region us-east-1+<div class="grid-item" markdown>+### Metrics & Observability+Prometheus endpoint, per-request logging, cost tracking, budget limits, pipeline timing breakdowns.+[Learn more →](metrics.md)+</div>- # Point Claude Code at it- ANTHROPIC_API_KEY="sk-ant-dummy" \- ANTHROPIC_BASE_URL=http://localhost:8787 \- claude- ```+</div>----## Features+## Cloud Providers++Works with any LLM provider out of the box:-**Compression**+```bash+headroom proxy # Direct Anthropic/OpenAI+headroom proxy --backend bedrock --region us-east-1 # AWS Bedrock+headroom proxy --backend vertex_ai --region us-central1 # Google Vertex AI+headroom proxy --backend azure # Azure OpenAI+headroom proxy --backend openrouter # OpenRouter (400+ models)+```-- Statistical JSON array compression (no hardcoded rules)-- ML-based text compression via LLMLingua-- AST-aware code compression-- Image optimization (40-90% reduction)+Or via LiteLLM for 100+ providers (Together, Groq, Fireworks, Ollama, vLLM, etc.).-**Context Management**+----- Intelligent message scoring and dropping-- Compress-Cache-Retrieve (CCR) for lossless compression-- Provider cache alignment for better hit rates+## Installation-**Operations**+```bash+pip install headroom-ai # Core library+pip install "headroom-ai[all]" # Everything (recommended)+pip install "headroom-ai[proxy]" # Proxy server + MCP tools+pip install "headroom-ai[ml]" # ML compression (Kompress, requires torch)+pip install "headroom-ai[langchain]" # LangChain integration+pip install "headroom-ai[agno]" # Agno integration+pip install "headroom-ai[evals]" # Evaluation framework+```-- Prometheus metrics endpoint-- Request logging and cost tracking-- Budget limits and rate limiting+Requires Python 3.10+.---## Next Steps-- [Quickstart Guide](quickstart.md) — Get running in 5 minutes-- [Proxy Documentation](proxy.md) — Configure the optimization proxy-- [Architecture](ARCHITECTURE.md) — Deep dive into how it works+- **[Quickstart](quickstart.md)** — Running in 5 minutes+- **[Integration Guide](integration-guide.md)** — Every way to add Headroom to your stack+- **[Architecture](ARCHITECTURE.md)** — How the pipeline works under the hood+- **[Benchmarks](benchmarks.md)** — Accuracy and latency data+- **[Limitations](LIMITATIONS.md)** — When compression helps and when it doesn't----## License--Apache 2.0 — Free for commercial use.+Apache 2.0 — Free for commercial use. [GitHub](https://github.com/chopratejas/headroom) | [PyPI](https://pypi.org/project/headroom-ai/) | [Discord](https://discord.gg/QEyNhaGA)diff --git a/docs/overrides/.gitkeep b/docs/overrides/.gitkeepnew file mode 100644index 000000000..e69de29bbdiff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.cssnew file mode 100644index 000000000..79a9c7e04--- /dev/null+++ b/docs/stylesheets/extra.css@@ -0,0 +1,103 @@+/* Headroom docs custom styles */++/* Hero section on landing page */+.md-typeset .hero {+ text-align: center;+ padding: 2rem 0 1rem;+}++.md-typeset .hero h1 {+ font-size: 2.5rem;+ font-weight: 700;+ margin-bottom: 0.5rem;+}++.md-typeset .hero .tagline {+ font-size: 1.25rem;+ color: var(--md-default-fg-color--light);+ margin-bottom: 1.5rem;+}++/* Feature grid on landing page */+.grid-container {+ display: grid;+ grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));+ gap: 1rem;+ margin: 1.5rem 0;+}++.grid-item {+ border: 1px solid var(--md-default-fg-color--lightest);+ border-radius: 0.5rem;+ padding: 1.25rem;+ transition: border-color 0.2s, box-shadow 0.2s;+}++.grid-item:hover {+ border-color: var(--md-accent-fg-color);+ box-shadow: 0 2px 8px rgba(0, 0, 0, 0.08);+}++.grid-item h3 {+ margin-top: 0 !important;+ font-size: 1rem;+}++.grid-item p {+ font-size: 0.85rem;+ color: var(--md-default-fg-color--light);+ margin-bottom: 0;+}++/* Tighter code blocks */+.md-typeset pre > code {+ font-size: 0.8rem;+}++/* Stats bar */+.stats-bar {+ display: flex;+ justify-content: center;+ gap: 3rem;+ padding: 1rem 0;+ flex-wrap: wrap;+}++.stat {+ text-align: center;+}++.stat .number {+ display: block;+ font-size: 2rem;+ font-weight: 700;+ color: var(--md-primary-fg-color);+}++.stat .label {+ font-size: 0.8rem;+ color: var(--md-default-fg-color--light);+ text-transform: uppercase;+ letter-spacing: 0.05em;+}++/* Badge row */+.badges {+ text-align: center;+ margin-bottom: 1.5rem;+}++.badges img {+ margin: 0 0.15rem;+}++/* Better admonition for "how it works" */+.md-typeset .admonition.pipeline,+.md-typeset details.pipeline {+ border-color: var(--md-accent-fg-color);+}++/* Hide title on home page (we use hero instead) */+.md-typeset h1:first-child {+ /* keep visible but reduce gap */+}diff --git a/mkdocs.yml b/mkdocs.ymlindex 717a723a8..5642a553c 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -1,5 +1,5 @@site_name: Headroom-site_description: The Context Optimization Layer for LLM Applications+site_description: "The Context Optimization Layer for LLM Applications — compress everything your AI agent reads."site_url: https://chopratejas.github.io/headroomrepo_url: https://github.com/chopratejas/headroomrepo_name: chopratejas/headroom@@ -7,35 +7,52 @@ edit_uri: edit/main/docs/theme:name: material+ custom_dir: docs/overridespalette:- - scheme: default- primary: indigo- accent: indigo+ - media: "(prefers-color-scheme: light)"+ scheme: default+ primary: deep purple+ accent: ambertoggle:icon: material/brightness-7name: Switch to dark mode- - scheme: slate- primary: indigo- accent: indigo+ - media: "(prefers-color-scheme: dark)"+ scheme: slate+ primary: deep purple+ accent: ambertoggle:icon: material/brightness-4name: Switch to light mode+ font:+ text: Inter+ code: JetBrains Monofeatures:- content.code.copy+ - content.code.annotate- content.tabs.link- navigation.instant+ - navigation.instant.progress- navigation.tabs+ - navigation.tabs.sticky- navigation.sections+ - navigation.expand- navigation.top+ - navigation.indexes- search.highlight- search.suggest+ - search.share- toc.follow+ - header.autohideicon:repo: fontawesome/brands/github+ logo: material/layers-outlineplugins:- search+extra_css:+ - stylesheets/extra.css+markdown_extensions:- admonition- attr_list@@ -50,44 +67,57 @@ markdown_extensions:pygments_lang_class: true- pymdownx.inlinehilite- pymdownx.snippets- - pymdownx.superfences+ - pymdownx.superfences:+ custom_fences:+ - name: mermaid+ class: mermaid+ format: !!python/name:pymdownx.superfences.fence_code_format- pymdownx.tabbed:alternate_style: true- pymdownx.details+ - pymdownx.emoji:+ emoji_index: !!python/name:material.extensions.emoji.twemoji+ emoji_generator: !!python/name:material.extensions.emoji.to_svg+ - pymdownx.tasklist:+ custom_checkbox: truenav:- Home: index.md- Getting Started:- Quickstart: quickstart.md- Installation: getting-started.md- - Benchmarks: benchmarks.md+ - Configuration: configuration.md- User Guide:- Proxy Server: proxy.md- - Compression: compression.md+ - Compression Pipeline: compression.md+ - CCR (Lossless Retrieval): ccr.md- Image Compression: image-compression.md- - CCR (Compress-Cache-Retrieve): ccr.md- - Configuration: configuration.md+ - Memory System: memory.md+ - Shared Context: shared-context.md+ - Failure Learning: learn.md- Integrations:- - Integration Guide: integration-guide.md+ - Overview: integration-guide.md- LangChain: langchain.md- Agno: agno.md- Strands: strands.md- MCP Tools: mcp.md- - Advanced:- - LLMLingua: llmlingua.md- - Text Compression: text-compression.md- - Transforms: transforms.md+ - How It Works:- Architecture: ARCHITECTURE.md+ - Transforms Pipeline: transforms.md+ - Text Compression: text-compression.md+ - LLMLingua: llmlingua.md+ - Benchmarks:+ - Accuracy: benchmarks.md+ - Latency: LATENCY_BENCHMARKS.md+ - Limitations: LIMITATIONS.md- Reference:- API: api.md- SDK: sdk.md- - Metrics: metrics.md- - Errors: errors.md+ - Metrics & Observability: metrics.md+ - Error Codes: errors.md- Troubleshooting: troubleshooting.md- Deployment:- - macOS: macos-deployment.md- - Memory:- - Overview: memory.md+ - macOS LaunchAgent: macos-deployment.mdextra:social:@@ -95,3 +125,6 @@ extra:link: https://github.com/chopratejas/headroom- icon: fontawesome/brands/pythonlink: https://pypi.org/project/headroom-ai/+ - icon: fontawesome/brands/discord+ link: https://discord.gg/QEyNhaGA+ generator: false