Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Fix inflated token savings stats from cache hits
chopratejas1aa312b92300
diff --git a/headroom/proxy/server.py b/headroom/proxy/server.pyindex 72e9e4c75..03392b44d 100644--- a/headroom/proxy/server.py+++ b/headroom/proxy/server.py@@ -1546,7 +1546,7 @@ class HeadroomProxy:model=model,input_tokens=0,output_tokens=0,- tokens_saved=cached.tokens_saved_per_hit,+ tokens_saved=0, # Savings already counted when response was cachedlatency_ms=optimization_latency,cached=True,)@@ -3957,7 +3957,7 @@ class HeadroomProxy:model=model,input_tokens=0,output_tokens=0,- tokens_saved=cached.tokens_saved_per_hit,+ tokens_saved=0, # Savings already counted when response was cachedlatency_ms=(time.time() - start_time) * 1000,cached=True,)