Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Add architecture diagrams to README and docs
chopratejascd5ea2ea1d50
diff --git a/README.md b/README.mdindex a5d62f060..dac9ddd58 100644--- a/README.md+++ b/README.md@@ -122,14 +122,61 @@ Run it yourself: `python examples/multi_tool_agent_test.py`## How It Works-Headroom doesn't summarize or truncate blindly. It uses **statistical analysis**:+> Headroom optimizes LLM context *before* it hits the provider —+> without changing your agent logic or tools.++```mermaid+flowchart LR+ User["Your App"]+ Entry["Headroom"]+ Transform["Context<br/>Optimization"]+ LLM["LLM Provider"]+ Response["Response"]++ User --> Entry --> Transform --> LLM --> Response+```++### Inside Headroom++```mermaid+flowchart TB++subgraph Pipeline["Transform Pipeline"]+ CA["Cache Aligner<br/><i>Stabilizes dynamic tokens</i>"]+ SC["Smart Crusher<br/><i>Removes redundant tool output</i>"]+ CM["Context Manager<br/><i>Fits token budget</i>"]+ CA --> SC --> CM+end++subgraph CCR["CCR: Compress-Cache-Retrieve"]+ Store[("Compressed<br/>Store")]+ Tool["Retrieve Tool"]+ Tool <--> Store+end++LLM["LLM Provider"]++CM --> LLM+SC -. "Stores originals" .-> Store+LLM -. "Requests full context<br/>if needed" .-> Tool+```++> Headroom never throws data away.+> It compresses aggressively and retrieves precisely.++### What actually happens++1. **Headroom intercepts context** — Tool outputs, logs, search results, and intermediate agent steps.++2. **Dynamic content is stabilized** — Timestamps, UUIDs, request IDs are normalized so prompts cache cleanly.++3. **Low-signal content is removed** — Repetitive or redundant data is crushed, not truncated.++4. **Original data is preserved** — Full content is stored separately and retrieved *only if the LLM asks*.-1. **Detects redundancy** - Repeated fields like `"language": "typescript"` across 100 items-2. **Keeps what matters** - First items, last items, query-relevant matches, anomalies-3. **Preserves errors** - Never drops items containing "error", "exception", "failed"-4. **Maintains schema** - Output JSON structure stays identical+5. **Provider caches finally work** — Headroom aligns prompts so OpenAI, Anthropic, and Google caches actually hit.-The compression is **reversible** via CCR (Compress-Cache-Retrieve). If the LLM needs more data, it can request the original.+For deep technical details, see [Architecture Documentation](docs/ARCHITECTURE.md).---diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex 117d61aa7..938021c98 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -1,5 +1,51 @@# Headroom SDK: A Complete Explanation+## Architecture Overview++```mermaid+flowchart TB+ subgraph Entry["Entry Points"]+ Proxy["Proxy Mode<br/><i>Zero code changes</i>"]+ SDK["SDK Mode<br/><i>HeadroomClient</i>"]+ Integrations["Integrations<br/><i>LangChain / Agno</i>"]+ end++ subgraph Pipeline["Transform Pipeline"]+ direction TB++ CA["Cache Aligner<br/>━━━━━━━━━━━━━━━<br/>Extracts dynamic content<br/>(dates, UUIDs, tokens)<br/>Stable prefix for caching"]++ SC["Smart Crusher<br/>━━━━━━━━━━━━━━━<br/>Analyzes tool outputs<br/>Keeps: first, last, errors, outliers<br/>70-95% reduction"]++ CM["Context Manager<br/>━━━━━━━━━━━━━━━<br/>Enforces token limits<br/>Scores by recency and relevance<br/>Fits context window"]++ CA --> SC --> CM+ end++ subgraph Cache["Provider Cache Optimization"]+ direction LR+ Anthropic["Anthropic<br/><i>cache_control blocks</i><br/>90% savings"]+ OpenAI["OpenAI<br/><i>Prefix alignment</i><br/>50% savings"]+ Google["Google<br/><i>CachedContent API</i><br/>75% savings"]+ end++ subgraph CCR["CCR: Compress-Cache-Retrieve"]+ Store[("Compression<br/>Store")]+ Tool["Retrieve Tool<br/><i>LLM requests original</i>"]+ Store <--> Tool+ end++ LLM["LLM API<br/><i>OpenAI / Anthropic / Google</i>"]++ Entry --> Pipeline+ Pipeline --> Cache+ Cache --> LLM+ SC -.->|"Stores original"| Store+ LLM -.->|"If needed"| Tool+```++---+## What Problem Does Headroom Solve?When you use AI models like GPT-4 or Claude, you pay for **tokens** - the pieces of text you send (input) and receive (output). The problem is: