Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix: restore token-mode compression on frozen prefixes (#1489)

Omar Garcia8e0dadfe02da
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b0e87771a..0a3fc4d1f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -34,6 +34,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Bug Fixes
+* **proxy/anthropic:** restore token-mode compression on continued Claude Code turns with a frozen prefix and deferred CCR tool injection. Token mode now runs request-side compression even when the client did not pre-register `headroom_retrieve`, relying on the existing marker-triggered injection override to keep emitted CCR markers redeemable ([#1487](https://github.com/headroomlabs-ai/headroom/issues/1487)).
* **subscription:** stop zeroing the 5-hour headroom contribution counters on every poll. The rollover check compared `five_hour.resets_at` with a bare `!=`, but the usage API reports that timestamp with second-level jitter (observed flapping between `01:59:59Z` and `02:00:00Z` on consecutive polls within the same window), so a spurious "5h window rolled over" reset fired every poll interval (~5 min) and the dashboard's per-window savings stuck near 0%. Only a forward jump larger than `_ROLLOVER_MIN_ADVANCE` (1 minute) now counts as a real rollover.
* **transforms/content_router:** stop replacing `role="tool"` output with a lossy-unrecoverable summary on the live compression path (refs [#1307](https://github.com/chopratejas/headroom/issues/1307)). `ContentRouter.apply()` routed OpenAI-style `role="tool"` string messages — `Bash`/`grep`/`ls`/`cat` output — through the ML/word-drop summarizers; when the result carried no CCR retrieve marker (CCR off, ratio >= 0.8, or the size-gate fallback) the original was unrecoverable and the agent acted on a fabricated summary. Tool-role string content is now kept verbatim unless the compressed form is CCR-recoverable. Assistant/user text is unaffected, and structurally-lossless passes (SmartCrusher/Log/Search) still apply. The Anthropic `tool_result` block path is tracked separately.
* **rtk:** stop `rtk` hook registration from spuriously timing out during `headroom wrap`. Output is captured to a temp file instead of pipes, and `stdin` is closed, so a background process forked by `rtk init` can no longer hold the pipe open and block `subprocess.run` past its 10s timeout after the hooks were already registered.
diff --git a/headroom/proxy/handlers/anthropic.py b/headroom/proxy/handlers/anthropic.py
index 46d9a9f26..a01c9a5a1 100644
--- a/headroom/proxy/handlers/anthropic.py
+++ b/headroom/proxy/handlers/anthropic.py
@@ -1050,6 +1050,8 @@ class AnthropicHandlerMixin:
def should_skip_ccr_request_compression(
current_frozen_message_count: int,
) -> bool:
+ if is_token_mode(self.config.mode):
+ return False
# If the tool is already present, CCR stays reversible even on frozen turns.
return (
self.config.ccr_inject_tool
diff --git a/tests/test_proxy/test_anthropic_ccr_deferred_injection.py b/tests/test_proxy/test_anthropic_ccr_deferred_injection.py
index ee9adea26..60cea90ed 100644
--- a/tests/test_proxy/test_anthropic_ccr_deferred_injection.py
+++ b/tests/test_proxy/test_anthropic_ccr_deferred_injection.py
@@ -115,6 +115,7 @@ def test_frozen_prefix_skips_marker_emission_when_tool_injection_is_deferred(mon
proxy.config.optimize = True
proxy.config.image_optimize = False
proxy.config.ccr_inject_tool = True
+ proxy.config.mode = "cache"
_disable_pipeline_extensions(proxy)
fake_tracker = _FakePrefixTracker(frozen_count=1)
@@ -333,6 +334,83 @@ def test_token_mode_reclamp_keeps_reversible_ccr_path_when_effective_prefix_drop
assert any(tool.get("name") == "headroom_retrieve" for tool in forwarded["tools"])
+def test_token_mode_compresses_frozen_prefix_turns_when_tool_is_not_already_present(
+ monkeypatch,
+) -> None:
+ captured: dict[str, object] = {}
+ marker_message = {
+ "role": "user",
+ "content": "[100 items compressed to 10. Retrieve more: hash=abc123def456abc123def456]",
+ }
+ _force_compression(monkeypatch)
+
+ with _make_proxy_client() as client:
+ proxy = client.app.state.proxy
+ proxy.config.optimize = True
+ proxy.config.image_optimize = False
+ proxy.config.ccr_inject_tool = True
+ proxy.config.mode = "token"
+ _disable_pipeline_extensions(proxy)
+
+ fake_tracker = _FakePrefixTracker(frozen_count=1)
+ proxy.session_tracker_store.compute_session_id = lambda request, model, messages: (
+ "stable-session"
+ )
+ proxy.session_tracker_store.get_or_create = lambda session_id, provider: fake_tracker
+ proxy._get_compression_cache = lambda session_id: _FakeCompressionCache(frozen_count=1)
+
+ def _fake_apply(**kwargs):
+ captured.setdefault("compression_calls", []).append(kwargs["messages"])
+ captured["frozen_message_count"] = kwargs["frozen_message_count"]
+ return SimpleNamespace(
+ messages=[marker_message],
+ transforms_applied=["fake:ccr"],
+ timing={},
+ tokens_before=40,
+ tokens_after=10,
+ waste_signals=None,
+ )
+
+ proxy.anthropic_pipeline.apply = _fake_apply
+
+ async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
+ captured["body"] = body
+ return httpx.Response(
+ 200,
+ json={
+ "id": "msg_ccr_token_frozen",
+ "type": "message",
+ "role": "assistant",
+ "content": [{"type": "text", "text": "ok"}],
+ "usage": {
+ "input_tokens": 20,
+ "output_tokens": 3,
+ "cache_read_input_tokens": 0,
+ "cache_creation_input_tokens": 0,
+ },
+ },
+ )
+
+ proxy._retry_request = _fake_retry
+
+ response = client.post(
+ "/v1/messages",
+ headers={"x-api-key": "test-key", "anthropic-version": "2023-06-01"},
+ json={
+ "model": "claude-sonnet-4-6",
+ "max_tokens": 64,
+ "messages": [{"role": "user", "content": _RAW_TRANSCRIPT}],
+ },
+ )
+
+ assert response.status_code == 200
+ assert captured.get("frozen_message_count") == 1
+ assert len(captured.get("compression_calls", [])) == 1
+ forwarded = captured["body"]
+ assert forwarded["messages"] == [marker_message]
+ assert any(tool.get("name") == "headroom_retrieve" for tool in forwarded["tools"])
+
+
def test_existing_retrieve_tool_keeps_reversible_ccr_path_when_prefix_is_frozen(
monkeypatch,
) -> None: