Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
fix(telemetry): only advance usage-report baseline after a 200 (#2149)
Abhay Singh0cddac632d8a
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex aef7bae19..2dde03b58 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -118,6 +118,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0* **proxy:** run a cold-start fast pass before background-compression deferral so byte-identical freeze doesn't lock sessions to the uncompressed transcript. Since #1850, a session's provider-cached prefix is frozen in whatever form its cold start forwarded; deferring the WHOLE pipeline (`HEADROOM_BACKGROUND_COMPRESSION=1`, frozen=0, ≥50k tokens) therefore cached the raw transcript and forfeited the session's compression savings for its lifetime — including sub-second lossless wins like `read_lifecycle` stale-read drops, observed in the field as sessions permanently stuck at 0 savings. The deferral branch now runs the pipeline synchronously with the new `skip_kompress=True` kwarg (everything except the Kompress ML stage — the only stage that can blow the request budget per #1171) under a bounded fast-pass budget (`HEADROOM_COLD_START_FAST_PASS_TIMEOUT_SECONDS`, default 10s), forwards the pruned form, and defers only Kompress to the background job (tagged `deferred:kompress_background`). Fail-open: on fast-pass timeout/error the request forwards uncompressed exactly as before. Units routed to Kompress under `skip_kompress` take the same fallback as when the model isn't ready.* **init/codex:** don't overwrite the user's `hooks.json`. `_ensure_codex_hooks` wrote a fresh payload containing only Headroom's two hooks, wholesale-replacing `~/.codex/hooks.json` — so any user-managed Codex hooks (and other top-level keys) were silently destroyed on `headroom init codex`. It now read-merges: existing entries are preserved, Headroom's are deduped on the `headroom-init-codex` marker and appended, matching `_ensure_claude_hooks` / `_ensure_copilot_hooks`.* **ccr:** detect `read_lifecycle` stale/superseded markers in the retrieve-tool injector so they stay redeemable. Those markers (`[Read content stale: … Retrieve original: hash=<hash>]`) store the original bytes in the CCR store under a valid hash, but none of `CCRToolInjector`'s patterns matched them — every pattern required the word "compressed" or the `<<ccr:` form. So on a frozen-prefix turn (both `read_lifecycle` and prefix freezing are on by default) the injector reported no compressed content, the `headroom_retrieve` tool was not injected, and the model was handed a marker advertising `Retrieve original: hash=X` with no tool to redeem it — silent data loss for stale reads, where retrieval is the only way to recover the original-at-read-time content (the exact case the #1006 guard exists to prevent). Added a pattern matching the load-bearing `Retrieve original: hash=` phrase, aligning the injector with the sibling `read_maturation` marker that was already (incidentally) detected.+* **telemetry:** only advance the usage-report baseline after a confirmed 200. `UsageReporter._report_usage` sends usage as a delta against the last snapshot, but it called `_snapshot_metrics()` (and advanced `_last_report_time`) unconditionally after the POST — including when the send returned non-200 or raised. So a report that failed to reach the cloud (which the module is explicitly designed to tolerate) still rebased the baseline, permanently dropping that window's requests/tokens from usage-based billing/quota; the next report started from the advanced baseline and never re-included them. The baseline now advances only on a 200, so a failed send leaves the window intact for the next report to retry.* **savings:** stop the durable savings ledger from billing free (0-priced) models at the `$3/M` fallback. `estimate_cost_usd` guarded the litellm estimate with `if priced > 0`, so a genuinely free model — where `_estimate_compression_savings_usd` correctly returns `0.0` — was treated as "unpriced" and fell through to the blended fallback rate, writing phantom cost-avoided into the JSONL ledger and surfacing it in `headroom savings`. The ledger now trusts the estimate verbatim for known models (it already falls back internally for models litellm can't price and returns `0.0` for free ones), fixing the same defect at this call site that was already fixed inside the helper.* **init/codex:** stop `headroom init codex` from deleting per-profile provider settings. `_ensure_codex_provider` removed the root-level `model_provider`/`openai_base_url` (which init owns) with a multiline regex that matched those keys in **every** table, so a user's `[profiles.*]` overrides (e.g. `[profiles.work] model_provider = "azure"`) were silently stripped and those profiles fell through to the injected `"headroom"` default — config corruption. The strip is now scoped to the document root (everything before the first table header), so per-profile overrides are preserved while init still replaces a root-level assignment.* **proxy:** strip output-only content blocks from request messages before forwarding. Anthropic's server-side refusal-fallback feature (`server-side-fallback-2026-06-01`) emits a `{"type":"fallback","from":{...},"to":{...}}` block inside the assistant response to signal that a refused request was re-served by the fallback model. That block is valid on the *response* path but rejected on the *request* path, so when a client replays the assistant turn the next request 400s (`invalid_request_error: messages.N.content.0: Input tag 'fallback' ...`) and the conversation gets permanently stuck through the proxy. `read_request_json_with_bytes` (Anthropic/OpenAI/Bedrock) and `_read_request_json` (Gemini) now drop such blocks — re-encoding the raw bytes so byte-faithful passthrough cannot leak the pre-strip body, backfilling a benign text block if a turn is emptied, and leaving requests without such blocks byte-identical (no cache churn).diff --git a/headroom/telemetry/reporter.py b/headroom/telemetry/reporter.pyindex eb23afd66..15217d334 100644--- a/headroom/telemetry/reporter.py+++ b/headroom/telemetry/reporter.py@@ -332,15 +332,21 @@ class UsageReporter:total_requests,total_tokens_saved,)+ # Only advance the baseline after a confirmed 200. Usage is a+ # delta against the last snapshot, so snapshotting on a failed+ # send (non-200 or exception) would permanently drop this+ # window's requests/tokens from billing — the next report starts+ # from the advanced baseline and never re-includes them. Leaving+ # both baselines intact makes the next report retry the full+ # period. (The empty-report early-return above only advances the+ # time, since there is nothing to lose.)+ self._snapshot_metrics()+ self._last_report_time = nowelse:logger.warning("Usage report returned status %d", resp.status_code)except Exception:logger.warning("Failed to send usage report", exc_info=True)- # Update snapshot- self._snapshot_metrics()- self._last_report_time = now-def _snapshot_metrics(self) -> None:"""Take a snapshot of current proxy metrics for delta computation."""if self._proxy is None or self._proxy.cost_tracker is None:diff --git a/tests/test_usage_reporter_snapshot.py b/tests/test_usage_reporter_snapshot.pynew file mode 100644index 000000000..314ebded1--- /dev/null+++ b/tests/test_usage_reporter_snapshot.py@@ -0,0 +1,72 @@+"""UsageReporter must only advance its delta baseline after a confirmed 200.++Snapshotting on a failed send permanently drops that window's usage from the+delta-based usage report (billing/quota under-count)."""++from __future__ import annotations++from types import SimpleNamespace++import anyio++from headroom.telemetry.reporter import UsageReporter+++class _Resp:+ def __init__(self, status_code: int) -> None:+ self.status_code = status_code++ def json(self) -> dict:+ return {}+++def _make_reporter(post_outcome) -> UsageReporter:+ r = object.__new__(UsageReporter)+ ct = SimpleNamespace(+ _tokens_saved_by_model={"m": 100},+ _tokens_sent_by_model={"m": 400},+ _requests_by_model={"m": 5},+ )+ r._proxy = SimpleNamespace(cost_tracker=ct)+ r._last_report_time = None+ r._last_tokens_saved_by_model = {}+ r._last_tokens_sent_by_model = {}+ r._last_requests_by_model = {}+ r._license_key = "k"+ r._cloud_url = "https://cloud.example"+ r._license_info = None++ class _Client:+ async def post(self, *args, **kwargs):+ if isinstance(post_outcome, Exception):+ raise post_outcome+ return _Resp(post_outcome)++ async def _get_client():+ return _Client()++ r._get_client = _get_client+ return r+++def test_baseline_advances_only_on_success():+ r = _make_reporter(200)+ anyio.run(r._report_usage)+ # Success -> baseline rebased to the current cumulative counters.+ assert r._last_tokens_saved_by_model == {"m": 100}+ assert r._last_requests_by_model == {"m": 5}+++def test_baseline_not_advanced_on_non_200():+ r = _make_reporter(500)+ anyio.run(r._report_usage)+ # Failed send -> baseline untouched so the window is retried next report.+ assert r._last_tokens_saved_by_model == {}+ assert r._last_requests_by_model == {}+++def test_baseline_not_advanced_on_exception():+ r = _make_reporter(RuntimeError("network down"))+ anyio.run(r._report_usage)+ assert r._last_tokens_saved_by_model == {}+ assert r._last_requests_by_model == {}