Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Clarify token vs cache run modes in docs
JerrettDavis5a2f1aeb42a2
diff --git a/README.md b/README.mdindex 49e03ca3f..06c645672 100644--- a/README.md+++ b/README.md@@ -110,12 +110,21 @@ Works with any LLM client — Anthropic, OpenAI, LiteLLM, Bedrock, Vercel AI SDKheadroom proxy --port 8787```+```bash+# Run mode (default: token)+headroom proxy --mode token # maximize compression+headroom proxy --mode cache # preserve Anthropic/OpenAI prefix cache stability+```+```bash# Point any LLM client at the proxyANTHROPIC_BASE_URL=http://localhost:8787 your-appOPENAI_BASE_URL=http://localhost:8787/v1 your-app```+Use `token` mode for short/medium sessions where raw compression savings matter most.+Use `cache` mode for long-running chats where preserving prior-turn bytes improves provider cache reuse.+Works with any language, any tool, any framework. **[Proxy docs](docs/proxy.md)**### Coding agents — one commanddiff --git a/docs/benchmarks.md b/docs/benchmarks.mdindex 4db1d0303..227c30945 100644--- a/docs/benchmarks.md+++ b/docs/benchmarks.md@@ -207,3 +207,10 @@ python -c "from headroom import compress; print(compress([{'role':'user','conten# Run local proxy mode benchmark (no API calls)python benchmarks/proxy_mode_benchmark.py --turns 12 --show-real-harness```++This benchmark compares `token` vs `cache` proxy modes on the same synthetic conversation:++- `token` should show higher compression.+- `cache` should preserve prior-turn stability and can win in long sessions with strong prefix-cache reuse.++`--show-real-harness` prints optional steps for running the same comparison with Claude Code, but does not call APIs by default.diff --git a/docs/proxy.md b/docs/proxy.mdindex 125caf8b5..244e9bde6 100644--- a/docs/proxy.md+++ b/docs/proxy.md@@ -53,6 +53,11 @@ headroom proxy --mode tokenHEADROOM_MODE=cache headroom proxy```+When to pick each:++- `token`: best for maximizing immediate compression savings.+- `cache`: best for long conversations where preserving prior-turn bytes improves prefix-cache reuse.+Legacy values (`token_headroom`, `cost_savings`) are still accepted as aliases.### Context Management Options