Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
fix(savings): don't bill free models at the $3/M fallback in the ledger (#2147)
Abhay Singhfb17156bfae1
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex cb5a7f9d7..aef7bae19 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -118,6 +118,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0* **proxy:** run a cold-start fast pass before background-compression deferral so byte-identical freeze doesn't lock sessions to the uncompressed transcript. Since #1850, a session's provider-cached prefix is frozen in whatever form its cold start forwarded; deferring the WHOLE pipeline (`HEADROOM_BACKGROUND_COMPRESSION=1`, frozen=0, ≥50k tokens) therefore cached the raw transcript and forfeited the session's compression savings for its lifetime — including sub-second lossless wins like `read_lifecycle` stale-read drops, observed in the field as sessions permanently stuck at 0 savings. The deferral branch now runs the pipeline synchronously with the new `skip_kompress=True` kwarg (everything except the Kompress ML stage — the only stage that can blow the request budget per #1171) under a bounded fast-pass budget (`HEADROOM_COLD_START_FAST_PASS_TIMEOUT_SECONDS`, default 10s), forwards the pruned form, and defers only Kompress to the background job (tagged `deferred:kompress_background`). Fail-open: on fast-pass timeout/error the request forwards uncompressed exactly as before. Units routed to Kompress under `skip_kompress` take the same fallback as when the model isn't ready.* **init/codex:** don't overwrite the user's `hooks.json`. `_ensure_codex_hooks` wrote a fresh payload containing only Headroom's two hooks, wholesale-replacing `~/.codex/hooks.json` — so any user-managed Codex hooks (and other top-level keys) were silently destroyed on `headroom init codex`. It now read-merges: existing entries are preserved, Headroom's are deduped on the `headroom-init-codex` marker and appended, matching `_ensure_claude_hooks` / `_ensure_copilot_hooks`.* **ccr:** detect `read_lifecycle` stale/superseded markers in the retrieve-tool injector so they stay redeemable. Those markers (`[Read content stale: … Retrieve original: hash=<hash>]`) store the original bytes in the CCR store under a valid hash, but none of `CCRToolInjector`'s patterns matched them — every pattern required the word "compressed" or the `<<ccr:` form. So on a frozen-prefix turn (both `read_lifecycle` and prefix freezing are on by default) the injector reported no compressed content, the `headroom_retrieve` tool was not injected, and the model was handed a marker advertising `Retrieve original: hash=X` with no tool to redeem it — silent data loss for stale reads, where retrieval is the only way to recover the original-at-read-time content (the exact case the #1006 guard exists to prevent). Added a pattern matching the load-bearing `Retrieve original: hash=` phrase, aligning the injector with the sibling `read_maturation` marker that was already (incidentally) detected.+* **savings:** stop the durable savings ledger from billing free (0-priced) models at the `$3/M` fallback. `estimate_cost_usd` guarded the litellm estimate with `if priced > 0`, so a genuinely free model — where `_estimate_compression_savings_usd` correctly returns `0.0` — was treated as "unpriced" and fell through to the blended fallback rate, writing phantom cost-avoided into the JSONL ledger and surfacing it in `headroom savings`. The ledger now trusts the estimate verbatim for known models (it already falls back internally for models litellm can't price and returns `0.0` for free ones), fixing the same defect at this call site that was already fixed inside the helper.* **init/codex:** stop `headroom init codex` from deleting per-profile provider settings. `_ensure_codex_provider` removed the root-level `model_provider`/`openai_base_url` (which init owns) with a multiline regex that matched those keys in **every** table, so a user's `[profiles.*]` overrides (e.g. `[profiles.work] model_provider = "azure"`) were silently stripped and those profiles fell through to the injected `"headroom"` default — config corruption. The strip is now scoped to the document root (everything before the first table header), so per-profile overrides are preserved while init still replaces a root-level assignment.* **proxy:** strip output-only content blocks from request messages before forwarding. Anthropic's server-side refusal-fallback feature (`server-side-fallback-2026-06-01`) emits a `{"type":"fallback","from":{...},"to":{...}}` block inside the assistant response to signal that a refused request was re-served by the fallback model. That block is valid on the *response* path but rejected on the *request* path, so when a client replays the assistant turn the next request 400s (`invalid_request_error: messages.N.content.0: Input tag 'fallback' ...`) and the conversation gets permanently stuck through the proxy. `read_request_json_with_bytes` (Anthropic/OpenAI/Bedrock) and `_read_request_json` (Gemini) now drop such blocks — re-encoding the raw bytes so byte-faithful passthrough cannot leak the pre-strip body, backfilling a benign text block if a turn is emptied, and leaving requests without such blocks byte-identical (no cache churn).* **wrap/doctor:** make the Claude Remote Control gate warning accurate and stop it firing for users who never had the feature ([#1779](https://github.com/headroomlabs-ai/headroom/issues/1779)). Claude Code 2.1.196 added a client-side check that **deterministically** disables first-party Remote Control (`/remote-control` / `/rc`) whenever `ANTHROPIC_BASE_URL` points at a non-`api.anthropic.com` host — which Headroom always does. The old notice hedged ("may hide the Remote Control menu"); it now states the disable as fact, names the `/rc` command, and detects the installed Claude Code version so the wording is exact (`2.1.196` when known, `2.1.196+` when not). The gate is upstream and RC's control-plane talks to `claude.ai` (not the API host), so Headroom cannot restore it — the warning tells you to run Claude without Headroom for RC sessions. The warning is suppressed for auth modes that never had Remote Control (API-key/PAYG via `ANTHROPIC_API_KEY`/`ANTHROPIC_AUTH_TOKEN`, and Bedrock/Vertex/Foundry cloud IAM) and on Claude Code builds older than 2.1.196 where RC is unaffected by a custom base URL. Both the `headroom wrap claude` launch banner and `headroom doctor` co-report the sibling base-URL gates Headroom *does* restore — on-demand tool loading (#746, automatic) and the 1M context window (#1158, via `--1m`) — and the wrap-side co-report is session-accurate: it says "already restored via --1m" when the flag is in effect and reports tool deferral OFF (not falsely "kept on") when the user chose `--tool-search false`/`ENABLE_TOOL_SEARCH=false`; the `ENABLE_TOOL_SEARCH=...` banner line got the same accuracy fix. `is_custom_anthropic_base_url` now recognizes scheme-less values (`myproxy.local:8080`, `127.0.0.1:8787`) as custom hosts and degrades gracefully on malformed URLs instead of crashing `doctor`. `doctor` resolves the Claude Code version lazily, so runs with no custom base URL never pay the `claude --version` subprocess. No request bytes are touched (cache-safe); this is UX/notice-only.diff --git a/headroom/savings_ledger.py b/headroom/savings_ledger.pyindex 0b0454ee6..7cb33c2b6 100644--- a/headroom/savings_ledger.py+++ b/headroom/savings_ledger.py@@ -106,9 +106,13 @@ def estimate_cost_usd(# noisy "Provider List" warnings, and the MCP path is unknown-model by far# the most often. Go straight to the blended fallback.if model and model != UNKNOWN:- priced = _estimate_compression_savings_usd(model, tokens_saved)- if priced > 0:- return round(priced, 6)+ # Trust _estimate_compression_savings_usd's return verbatim: it already+ # falls back to the blended rate for models litellm can't price, and+ # returns a legitimate 0.0 for genuinely free (0-priced) models. A+ # `priced > 0` gate here re-billed those free models at the $3/M fallback+ # -- phantom cost-avoided -- undoing the same fix already applied inside+ # that helper.+ return round(_estimate_compression_savings_usd(model, tokens_saved), 6)return round(float(tokens_saved) * float(fallback_rate), 6)diff --git a/tests/test_savings_ledger.py b/tests/test_savings_ledger.pyindex 83782f0b5..b454946d2 100644--- a/tests/test_savings_ledger.py+++ b/tests/test_savings_ledger.py@@ -38,6 +38,23 @@ def test_estimate_cost_unknown_short_circuits_to_fallback():assert L.estimate_cost_usd(L.UNKNOWN, 0) == 0.0+def test_free_model_is_not_billed_at_fallback(monkeypatch):+ """A known but 0-priced (free) model must cost $0, not the $3/M fallback.++ _estimate_compression_savings_usd returns a legitimate 0.0 for free models;+ the ledger must trust that rather than re-billing it at the blended rate.+ """+ monkeypatch.setattr(L, "_estimate_compression_savings_usd", lambda model, tokens: 0.0)++ assert L.estimate_cost_usd("free-local-model", 1_000_000) == 0.0+++def test_priced_model_uses_litellm_estimate(monkeypatch):+ monkeypatch.setattr(L, "_estimate_compression_savings_usd", lambda model, tokens: tokens * 2e-6)++ assert L.estimate_cost_usd("some-model", 1_000_000) == pytest.approx(2.0)++def test_explicit_cost_is_honored(monkeypatch, tmp_path):_events_env(monkeypatch, tmp_path)L.record_savings_event(tokens_before=100, tokens_after=10, model="x", client="c", cost_usd=1.25)