Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
fix(proxy): honor force_kompress routing profile (#996)
weijie_chenb4682d6f91c7
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 464d93728..5b60e3d97 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -34,6 +34,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0* **proxy:** route native Bedrock `/model/{id}/converse` requests to the upstream Converse endpoint instead of the hard-coded `/invoke` action — the non-streaming handler now resolves the action from the inbound path, matching the streaming handler ([#999](https://github.com/chopratejas/headroom/pull/999)).* **proxy:** preserve byte-faithful `/v1/messages` forwarding when Anthropic tool arrays are already canonical, and only canonicalize-and-mutate tool lists when sorting changes ordering ([#1042](https://github.com/chopratejas/headroom/issues/1042)).* **ccr:** make retrieval store TTL configurable with `HEADROOM_CCR_TTL_SECONDS`, expose the effective TTL in `/v1/retrieve/stats`, and distinguish expired retrievals from missing hashes.+* **proxy:** make `force_kompress` skip ContentRouter auto-detection during compression and pass savings-profile kwargs through Anthropic batch requests.* **proxy:** add native Bedrock `/model/{id}/converse-stream` route and forward it through the existing streaming EventStream/SSE pipeline.* **wrap (codex):** fix `headroom wrap codex` producing a `config.toml` with duplicate top-level `model_provider` / `openai_base_url` keys (TOML-spec error) when the user had already configured their own provider. The injector now rewrites pre-existing top-level `model_provider` and `openai_base_url` lines in place — the previous value is kept in a `# was: …` trailing comment — instead of unconditionally prepending a duplicate, so `codex` can start against the proxy. The pre-wrap snapshot mechanism continues to byte-for-byte restore the original file on `headroom unwrap codex`.* **wrap:** isolate wrapped proxy subprocess stdout/stderr into `proxy-stdio.log`, so `proxy.log` remains the canonical rotating runtime log and Windows rollover failures from `RotatingFileHandler` are no longer blocked by wrapper stdio handles ([#1184](https://github.com/chopratejas/headroom/issues/1184)).diff --git a/headroom/proxy/handlers/anthropic.py b/headroom/proxy/handlers/anthropic.pyindex 19ac622fa..a366592c9 100644--- a/headroom/proxy/handlers/anthropic.py+++ b/headroom/proxy/handlers/anthropic.py@@ -2695,6 +2695,7 @@ class AnthropicHandlerMixin:context=extract_user_query(messages),frozen_message_count=frozen_message_count,request_id=request_id,+ **proxy_pipeline_kwargs(self.config),)optimized_messages = result.messagesdiff --git a/headroom/transforms/content_router.py b/headroom/transforms/content_router.pyindex e68856f4b..c2f62f448 100644--- a/headroom/transforms/content_router.py+++ b/headroom/transforms/content_router.py@@ -1104,15 +1104,18 @@ class ContentRouter(Transform):routing_log=[],)else:- # Determine strategy from content analysis- mixed = is_mixed_content(content)- detection = _detect_content(content)+ # Determine strategy from content analysis. When runtime settings+ # force Kompress, skip the full router detection path so large+ # proxy payloads do not pay for an unused strategy decision.force_kompress = bool(getattr(self, "_runtime_force_kompress", False))- strategy = (- CompressionStrategy.KOMPRESS- if force_kompress- else self._determine_strategy(content)- )+ if force_kompress:+ mixed = False+ detection = DetectionResult(ContentType.PLAIN_TEXT, 1.0, {})+ strategy = CompressionStrategy.KOMPRESS+ else:+ mixed = is_mixed_content(content)+ detection = _detect_content(content)+ strategy = self._determine_strategy(content)if debug_enabled:_log_router_debug("content_router_input",@@ -2569,8 +2572,14 @@ class ContentRouter(Transform):route_counts["error_protected"] += 1continue- # Detect content type for protection decisions- detection = _detect_content(content)+ # Detect content type for protection decisions. Even when the+ # runtime strategy is forced to Kompress, keep code-protection+ # checks but use the lightweight regex detector instead of the+ # full router chain.+ force_kompress = bool(getattr(self, "_runtime_force_kompress", False))+ detection = (+ _regex_detect_content_type(content) if force_kompress else _detect_content(content)+ )is_code = detection.content_type == ContentType.SOURCE_CODE# Protection 2: Don't compress recent CODEdiff --git a/tests/test_proxy_anthropic_cache_stability.py b/tests/test_proxy_anthropic_cache_stability.pyindex a085c326b..fb5d92187 100644--- a/tests/test_proxy_anthropic_cache_stability.py+++ b/tests/test_proxy_anthropic_cache_stability.py@@ -685,6 +685,71 @@ def test_batch_optimization_freezes_previous_turns_only() -> None:]+def test_batch_optimization_passes_savings_profile_kwargs() -> None:+ captured = {}+ with _make_proxy_client() as client:+ proxy = client.app.state.proxy+ proxy.config.optimize = True+ proxy.config.mode = "token"+ proxy.config.savings_profile = "agent-90"+ proxy.config.ccr_inject_tool = False++ def _fake_apply(**kwargs):+ captured["pipeline_kwargs"] = kwargs+ return SimpleNamespace(+ messages=kwargs["messages"],+ transforms_applied=[],+ timing={},+ tokens_before=100,+ tokens_after=80,+ waste_signals=None,+ )++ proxy.anthropic_pipeline.apply = _fake_apply++ async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001+ captured["body"] = body+ return httpx.Response(+ 200,+ json={+ "id": "msgbatch_profile",+ "type": "message_batch",+ "processing_status": "in_progress",+ "request_counts": {+ "processing": 1,+ "succeeded": 0,+ "errored": 0,+ "canceled": 0,+ },+ },+ )++ proxy._retry_request = _fake_retry++ response = client.post(+ "/v1/messages/batches",+ headers={"x-api-key": "test-key", "anthropic-version": "2023-06-01"},+ json={+ "requests": [+ {+ "custom_id": "req-1",+ "params": {+ "model": "claude-sonnet-4-6",+ "max_tokens": 128,+ "messages": [{"role": "user", "content": "compress me"}],+ },+ }+ ]+ },+ )++ assert response.status_code == 200+ pipeline_kwargs = captured["pipeline_kwargs"]+ assert pipeline_kwargs["force_kompress"] is True+ assert pipeline_kwargs["target_ratio"] == 0.10+ assert pipeline_kwargs["compress_user_messages"] is True++def test_token_mode_does_not_force_freeze_all_previous_turns() -> None:captured = {}with _make_proxy_client() as client:diff --git a/tests/test_transforms_content_router.py b/tests/test_transforms_content_router.pyindex 7aba2e3b6..9a60d7dc7 100644--- a/tests/test_transforms_content_router.py+++ b/tests/test_transforms_content_router.py@@ -277,6 +277,150 @@ def test_content_router_strategy_and_compress_paths(monkeypatch: pytest.MonkeyPaassert router.compress(" ").strategy_used is CompressionStrategy.PASSTHROUGH+def test_force_kompress_bypasses_content_detection(monkeypatch: pytest.MonkeyPatch) -> None:+ router = ContentRouter()+ router._runtime_force_kompress = True+ pure_result = RouterCompressionResult(+ compressed="pure",+ original="pure",+ strategy_used=CompressionStrategy.KOMPRESS,+ )++ monkeypatch.setattr(+ content_router_module,+ "is_mixed_content",+ lambda content: (_ for _ in ()).throw(AssertionError("mixed detection called")),+ )+ monkeypatch.setattr(+ content_router_module,+ "_detect_content",+ lambda content: (_ for _ in ()).throw(AssertionError("content detection called")),+ )+ monkeypatch.setattr(router, "_determine_strategy", lambda content: CompressionStrategy.MIXED)+ monkeypatch.setattr(router, "_compress_pure", lambda *args, **kwargs: pure_result)++ assert router.compress("large tool output") is pure_result+++def test_normal_compress_path_still_uses_content_detection(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ router = ContentRouter()+ calls = {"mixed": 0, "detect": 0}+ pure_result = RouterCompressionResult(+ compressed="pure",+ original="pure",+ strategy_used=CompressionStrategy.TEXT,+ )++ def _fake_mixed(content: str) -> bool:+ calls["mixed"] += 1+ return False++ def _fake_detect(content: str) -> DetectionResult:+ calls["detect"] += 1+ return DetectionResult(ContentType.PLAIN_TEXT, 1.0, {})++ monkeypatch.setattr(content_router_module, "is_mixed_content", _fake_mixed)+ monkeypatch.setattr(content_router_module, "_detect_content", _fake_detect)+ monkeypatch.setattr(router, "_compress_pure", lambda *args, **kwargs: pure_result)++ assert router.compress("plain text") is pure_result+ assert calls["mixed"] > 0+ assert calls["detect"] > 0+++def test_force_kompress_apply_uses_lightweight_detection(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ class FakeTokenizer:+ def count_text(self, text: str) -> int:+ return len(text.split())++ router = ContentRouter(ContentRouterConfig(protect_recent_code=2))+ content = " ".join(["plain text payload"] * 80)++ monkeypatch.setattr(+ content_router_module,+ "_detect_content",+ lambda content: (_ for _ in ()).throw(AssertionError("content detection called")),+ )+ monkeypatch.setattr(+ content_router_module,+ "_regex_detect_content_type",+ lambda content: DetectionResult(ContentType.PLAIN_TEXT, 1.0, {}),+ )+ monkeypatch.setattr(+ router,+ "compress",+ lambda content, context="", bias=1.0: RouterCompressionResult(+ compressed="compressed",+ original=content,+ strategy_used=CompressionStrategy.KOMPRESS,+ routing_log=[+ RoutingDecision(+ content_type=ContentType.PLAIN_TEXT,+ strategy=CompressionStrategy.KOMPRESS,+ original_tokens=len(content.split()),+ compressed_tokens=1,+ )+ ],+ ),+ )++ result = router.apply(+ [{"role": "tool", "content": content}],+ FakeTokenizer(),+ force_kompress=True,+ min_tokens_to_compress=10,+ protect_recent=2,+ )++ assert result.messages[0]["content"] == "compressed"+++def test_force_kompress_apply_lightweight_detection_protects_recent_code(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ class FakeTokenizer:+ def count_text(self, text: str) -> int:+ return len(text.split())++ router = ContentRouter(ContentRouterConfig(protect_recent_code=2))+ content = "\n".join(+ [+ "def generated_function(value):",+ " if value:",+ " return str(value)",+ ]+ * 40+ )++ monkeypatch.setattr(+ content_router_module,+ "_detect_content",+ lambda content: (_ for _ in ()).throw(AssertionError("content detection called")),+ )+ monkeypatch.setattr(+ router,+ "compress",+ lambda *args, **kwargs: (_ for _ in ()).throw(+ AssertionError("recent code should be protected")+ ),+ )++ result = router.apply(+ [{"role": "tool", "content": content}],+ FakeTokenizer(),+ force_kompress=True,+ min_tokens_to_compress=10,+ protect_recent=2,+ )++ assert result.messages[0]["content"] == content+ assert result.transforms_applied == ["router:protected:recent_code"]++def test_content_router_mixed_pure_apply_and_toin(monkeypatch: pytest.MonkeyPatch) -> None:router = ContentRouter()mixed_content = "\n".join(["before", "```python", "print('x')", "```", "after"])