Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

Clarify token vs cache run modes in docs

JerrettDavis5a2f1aeb42a2
diff --git a/README.md b/README.md
index 49e03ca3f..06c645672 100644
--- a/README.md
+++ b/README.md
@@ -110,12 +110,21 @@ Works with any LLM client — Anthropic, OpenAI, LiteLLM, Bedrock, Vercel AI SDK
headroom proxy --port 8787
```
+```bash
+# Run mode (default: token)
+headroom proxy --mode token # maximize compression
+headroom proxy --mode cache # preserve Anthropic/OpenAI prefix cache stability
+```
+
```bash
# Point any LLM client at the proxy
ANTHROPIC_BASE_URL=http://localhost:8787 your-app
OPENAI_BASE_URL=http://localhost:8787/v1 your-app
```
+Use `token` mode for short/medium sessions where raw compression savings matter most.
+Use `cache` mode for long-running chats where preserving prior-turn bytes improves provider cache reuse.
+
Works with any language, any tool, any framework. **[Proxy docs](docs/proxy.md)**
### Coding agents — one command
diff --git a/docs/benchmarks.md b/docs/benchmarks.md
index 4db1d0303..227c30945 100644
--- a/docs/benchmarks.md
+++ b/docs/benchmarks.md
@@ -207,3 +207,10 @@ python -c "from headroom import compress; print(compress([{'role':'user','conten
# Run local proxy mode benchmark (no API calls)
python benchmarks/proxy_mode_benchmark.py --turns 12 --show-real-harness
```
+
+This benchmark compares `token` vs `cache` proxy modes on the same synthetic conversation:
+
+- `token` should show higher compression.
+- `cache` should preserve prior-turn stability and can win in long sessions with strong prefix-cache reuse.
+
+`--show-real-harness` prints optional steps for running the same comparison with Claude Code, but does not call APIs by default.
diff --git a/docs/proxy.md b/docs/proxy.md
index 125caf8b5..244e9bde6 100644
--- a/docs/proxy.md
+++ b/docs/proxy.md
@@ -53,6 +53,11 @@ headroom proxy --mode token
HEADROOM_MODE=cache headroom proxy
```
+When to pick each:
+
+- `token`: best for maximizing immediate compression savings.
+- `cache`: best for long conversations where preserving prior-turn bytes improves prefix-cache reuse.
+
Legacy values (`token_headroom`, `cost_savings`) are still accepted as aliases.
### Context Management Options