Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
fix(transforms): guard Log fallback against invalid JSON + fix MIXED false-positive on source code (#1347)
monkeygold02c77640a95e
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 44d033136..2028457f0 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -233,6 +233,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0* **proxy:** register `ToolResultInterceptorTransform` in explicit transforms list when `HEADROOM_INTERCEPT_ENABLED` is set — closes [#829](https://github.com/headroomlabs-ai/headroom/issues/829).* **opencode:** write Headroom MCP config as a local stdio server instead of a remote `/mcp` URL, keep provider-only installs from adding MCP config, and allow `install apply --target opencode` ([#1380](https://github.com/headroomlabs-ai/headroom/issues/1380)).* **code:** keep Python `from __future__` imports before executable code during AST compression and validate compressed Python with `compile(..., "exec")` so compile-time syntax rules are enforced ([#1233](https://github.com/chopratejas/headroom/issues/1233)).+* **proxy/transforms:** stop the SMART_CRUSHER → Log fallback from collapsing truncated/invalid JSON tool outputs to a single CCR-retrieval marker (99.9% data loss when CCR retrieval isn't configured). The native magika detector classifies content by shape, not parseability, so a mid-stream-truncated JSON payload is tagged `json_array` and routed to SmartCrusher, which returns it unchanged; Kompress passes it through; the LogCompressor then treated the broken JSON as a multi-thousand-line "log" and reduced it to a retrieval marker. A JSON-validity guard now skips the Log fallback for content that fails `json.loads`, so invalid JSON passes through verbatim. Valid JSON arrays still reach the Log fallback (LogCompressor is a no-op on them). The guard also catches `RecursionError` from deeply nested JSON (e.g. `[[[[...]]]]` with 10k+ levels) so the router falls through to a safe strategy instead of crashing ([#1306](https://github.com/chopratejas/headroom/issues/1306)).+* **proxy/transforms:** fix MIXED false-positive on source code. `is_mixed_content` uses regex heuristics that misclassify Python code with dict/list literals (`{`, `[` at line start → `has_json_blocks`) and docstrings/comments (`has_prose`) as mixed content, routing it through `_compress_mixed` which splits it into sections and dispatches each to KOMPRESS — wasting 1–1.4s of latency with 0% compression. When the native magika detector confidently says `SOURCE_CODE` (confidence ≥ 0.8), `_determine_strategy` now trusts it over the regex heuristics. Additionally, when `prefer_code_aware_for_code=False` (the default), source code now uses `PASSTHROUGH` instead of falling back to `KOMPRESS`, which can destroy code semantics (98% compression, 11% fact recall on large blobs). This honours the config's stated intent ("let code pass through unmangled") and reduces latency on code blobs by 23–33×.+* **proxy/transforms:** catch `RecursionError` in `_try_detect_json` so deeply nested JSON arrays (10k+ nesting levels) no longer crash the content detector. The router falls through to a safe strategy (`TEXT` or `PASSTHROUGH`) instead of raising an unhandled exception.* **proxy:** report real input tokens on the streaming `message_start` event for LiteLLM/Bedrock-backed requests. LiteLLM streaming never surfaces prompt tokens mid-stream, so `message_start.usage.input_tokens` was always `0`; Anthropic clients (e.g. Claude Code) read input-token metrics from that event, underreporting token usage by ~99% in OTel/CloudWatch dashboards. The Bedrock streamer now backfills `input_tokens` with the count Headroom actually sent upstream when the backend leaves it unset, preserving any non-zero value the backend genuinely reports ([#1132](https://github.com/chopratejas/headroom/issues/1132)).* **proxy:** give buffered Anthropic request paths their own longer read timeout, so long `/v1/messages` turns and Anthropic batch or passthrough reads no longer trip the generic proxy cap while unrelated request timeouts stay unchanged.* **proxy:** retry upstream 429 rate limits honoring `Retry-After` instead of passing them straight to the client. Both the non-streaming (`_retry_request`) and streaming (`_stream_response`) forwarders returned an upstream 429 verbatim, so a parallel agent fan-out that exceeded the per-minute limit aborted every run; 429s are now retried with backoff (honoring the upstream `Retry-After`, capped at `retry_max_delay_ms`), surfacing only the exhausted 429 to the client ([#1221](https://github.com/chopratejas/headroom/issues/1221)).diff --git a/headroom/transforms/content_detector.py b/headroom/transforms/content_detector.pyindex 92783627b..48a969a8a 100644--- a/headroom/transforms/content_detector.py+++ b/headroom/transforms/content_detector.py@@ -293,24 +293,33 @@ def _try_detect_json(content: str) -> DetectionResult | None:try:value = json.loads(stripped)+ except RecursionError:+ # Deeply nested JSON (e.g. ``[[[[...]]]]`` with 10k+ levels) can+ # exceed Python's recursion limit. Treat as non-JSON so the router+ # falls through to a safe strategy instead of crashing.+ return Noneexcept ValueError:# Not pure JSON. First: a run of whitespace-separated top-level JSON# objects (web_search output, #1741) -> JSON_ARRAY.if stripped.startswith("{"):- items = _decode_concatenated_json(stripped)+ try:+ items = _decode_concatenated_json(stripped)+ except RecursionError:+ return Noneif items and len(items) >= 2 and all(isinstance(item, dict) for item in items):return DetectionResult(ContentType.JSON_ARRAY,1.0,{"item_count": len(items), "is_dict_array": True, "concatenated": True},)+# Otherwise decode one JSON value out of a small wrapped payload.start = min((i for i in (stripped.find("{"), stripped.find("[")) if i >= 0), default=-1)if start < 0:return Nonetry:value, end = _JSON_DECODER.raw_decode(stripped, start)- except ValueError:+ except (ValueError, RecursionError):return None# Accept only when the decoded JSON is the BULK of the content (see# _JSON_MIN_BULK_FRACTION) — a small structural wrapper around a JSON body,@@ -319,7 +328,7 @@ def _try_detect_json(content: str) -> DetectionResult | None:return None# A bare scalar (42, "s", true) is not structured data worth routing as JSON.- if not isinstance(value, (dict, list)):+ if not isinstance(value, dict | list):return Noneif isinstance(value, list):diff --git a/headroom/transforms/content_router.py b/headroom/transforms/content_router.pyindex 965183656..03ef0492e 100644--- a/headroom/transforms/content_router.py+++ b/headroom/transforms/content_router.py@@ -360,6 +360,36 @@ def _json_shape(content: str) -> dict[str, Any]:return {"is_json": True, "kind": type(parsed).__name__}+def _content_is_valid_json(content: str) -> bool:+ """Return True iff ``content`` parses as valid JSON.++ Used by the SMART_CRUSHER → Log fallback guard (#1306): the native+ magika detector tags content by shape, so truncated/mid-stream JSON+ tool outputs are misclassified as ``json_array``. SmartCrusher can't+ parse them and returns no savings; without this guard the LogCompressor+ would then collapse the broken JSON to a single CCR-retrieval marker+ (99.9% data loss). Valid JSON arrays still reach the Log fallback —+ LogCompressor is a no-op on them, so the guard is safe for the+ intended "repetitive JSONL" case.+ """+ try:+ json.loads(content)+ except (json.JSONDecodeError, ValueError):+ return False+ except RecursionError:+ # Deeply nested JSON (e.g. ``[[[[...]]]]`` with 10k+ levels) can+ # exceed Python's recursion limit inside ``json.loads``. Treat+ # it as invalid so the Log fallback is skipped — the content+ # passes through verbatim instead of being collapsed.+ logger.warning(+ "json.loads hit recursion limit on deeply nested JSON "+ "(%d chars); treating as invalid for Log-fallback guard",+ len(content),+ )+ return False+ return True++def _mixed_indicators(content: str) -> dict[str, bool]:return mixed_content_indicators(content)@@ -1829,6 +1859,19 @@ class ContentRouter(Transform):"""# 1. Check for mixed contentif is_mixed_content(content):+ # 2. Verify with the native detector: ``is_mixed_content`` uses+ # cheap regex heuristics that produce false positives on source+ # code. Python files with dict/list literals (``{``, ``[`` at+ # line start) trigger ``has_json_blocks``, and docstrings/comments+ # trigger ``has_prose`` — so a pure Python blob is misclassified+ # as MIXED, routed through ``_compress_mixed`` which splits it+ # into sections and dispatches each to KOMPRESS (no-op on code),+ # wasting latency on splitting without any compression.+ # When the native magika detector confidently says SOURCE_CODE,+ # trust it over the regex heuristics.+ detection = _detect_content(content)+ if detection.content_type == ContentType.SOURCE_CODE and detection.confidence >= 0.8:+ return self._strategy_from_detection(detection)return CompressionStrategy.MIXED# 2. Detect content type from content itself@@ -1863,7 +1906,19 @@ class ContentRouter(Transform):strategy == CompressionStrategy.CODE_AWAREand not self.config.prefer_code_aware_for_code):- strategy = CompressionStrategy.KOMPRESS+ # When CodeAware is not preferred, the intent is "let code pass+ # through unmangled" (per the config comment). Previously this+ # fell back to KOMPRESS, which is an ML compressor that can+ # destroy code semantics — on a 45K-token Python blob it+ # compresses to 912 tokens with 11% fact recall, making the+ # code useless for an agent. The MIXED path accidentally+ # protected code by splitting it into small sections that+ # KOMPRESS passes through, but that was a side-effect, not+ # intent. Now that the MIXED false-positive on source code is+ # fixed (``_determine_strategy`` trusts the native detector),+ # code reaches this path directly — so use PASSTHROUGH to+ # honour the "unmangled" intent explicitly.+ strategy = CompressionStrategy.PASSTHROUGHreturn strategy@@ -2387,9 +2442,24 @@ class ContentRouter(Transform):# Kompress can't shrink but the log compressor can).# Only attempted when the strategy was SMART_CRUSHER so# we don't reroute genuine code/diff content.+ #+ # JSON-validity guard (#1306): the native magika detector+ # classifies content by shape, not parseability, so a+ # truncated/mid-stream JSON tool output is tagged+ # ``json_array`` and routed to SMART_CRUSHER. SmartCrusher+ # returns it unchanged (it can't parse the broken JSON),+ # Kompress passes it through, and then the LogCompressor+ # treats the whole thing as a multi-thousand-line "log"+ # and collapses it to a single CCR-retrieval marker —+ # 99.9% data loss when CCR retrieval isn't configured.+ # Skip the Log fallback when the content isn't actually+ # valid JSON; the passthrough at the bottom of this+ # function preserves it verbatim. Valid JSON arrays still+ # get the Log fallback (LogCompressor is a no-op on them).if (strategy == CompressionStrategy.SMART_CRUSHERand self.config.enable_log_compressor+ and _content_is_valid_json(content)):log_compressor = self._get_log_compressor()if log_compressor is not None:diff --git a/tests/test_transforms_content_router.py b/tests/test_transforms_content_router.pyindex 2f333e911..3e478fc74 100644--- a/tests/test_transforms_content_router.py+++ b/tests/test_transforms_content_router.py@@ -299,7 +299,7 @@ def test_content_router_strategy_and_compress_paths(monkeypatch: pytest.MonkeyPa"_detect_content",lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),)- assert router._determine_strategy("code") is CompressionStrategy.KOMPRESS+ assert router._determine_strategy("code") is CompressionStrategy.PASSTHROUGHassert (router._strategy_from_detection(DetectionResult(ContentType.SEARCH_RESULTS, 1.0, {}))is CompressionStrategy.SEARCH@@ -623,6 +623,250 @@ def test_log_strategy_does_not_fallback_to_kompress_when_log_is_noop(assert strategy_chain == ["log"]+def test_smart_crusher_log_fallback_skipped_for_invalid_json(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """Truncated/invalid JSON must not reach the Log fallback (#1306).++ The native magika detector tags content by shape, so a truncated JSON+ tool output is classified as ``json_array`` and routed to SMART_CRUSHER.+ SmartCrusher returns it unchanged (can't parse broken JSON); Kompress+ passes it through. Without the JSON-validity guard, the LogCompressor+ would then collapse the whole thing to a single CCR-retrieval marker —+ 99.9% data loss when CCR retrieval isn't configured. The guard skips+ the Log fallback for invalid JSON so the content passes through verbatim.+ """+ router = ContentRouter(ContentRouterConfig())+ # Truncated JSON object: starts valid, cut mid-string so json.loads fails.+ truncated = '{"rows": [{"address": "Addr1", "name": "PoolA"}, {"address": "Addr2", "nam'++ # SmartCrusher returns no savings (can't parse) → Kompress is a no-op+ # (model not loaded in tests) → Log fallback must be SKIPPED.+ class CollapsingLogCompressor:+ def compress(self, content: str, bias: float = 1.0) -> SimpleNamespace:+ # Simulate the collapse: LogCompressor treats the broken JSON as+ # a multi-line "log" and reduces it to a retrieval marker.+ return SimpleNamespace(+ compressed="\n[2 lines compressed to 0. Retrieve more: hash=deadbeef]"+ )++ monkeypatch.setattr(router, "_get_log_compressor", lambda: CollapsingLogCompressor())+ # Kompress not ready → returns content unchanged (real behavior in tests).+ monkeypatch.setattr(+ router,+ "_try_ml_compressor",+ lambda content, context, question=None: (content, len(content.split())),+ )++ compressed, compressed_tokens, strategy_chain = router._apply_strategy_to_content(+ truncated,+ CompressionStrategy.SMART_CRUSHER,+ context="",+ )++ # Content preserved verbatim — no collapse.+ assert compressed == truncated+ assert CompressionStrategy.LOG.value not in strategy_chain+ assert CompressionStrategy.KOMPRESS.value in strategy_chain+++def test_smart_crusher_log_fallback_runs_for_valid_json(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """Valid JSON arrays still reach the Log fallback when SmartCrusher no-ops.++ The JSON-validity guard (#1306) only blocks the Log fallback for+ *invalid* JSON. Valid JSON arrays that SmartCrusher can't shrink (e.g.+ repetitive JSONL the proxy's own log dumps) still get the Log fallback,+ which is the intended "repetitive JSONL" path.+ """+ router = ContentRouter(ContentRouterConfig())+ valid_json = '[{"a": 1}, {"a": 1}, {"a": 1}, {"a": 1}]'++ class NoopSmartCrusher:+ def crush(self, content: str, query: str = "", bias: float = 1.0) -> SimpleNamespace:+ return SimpleNamespace(compressed=content)++ class ShrinkingLogCompressor:+ def compress(self, content: str, bias: float = 1.0) -> SimpleNamespace:+ return SimpleNamespace(compressed="[compressed]")++ monkeypatch.setattr(router, "_get_smart_crusher", lambda: NoopSmartCrusher())+ monkeypatch.setattr(router, "_get_log_compressor", lambda: ShrinkingLogCompressor())+ # Kompress no-op → Log fallback fires.+ monkeypatch.setattr(+ router,+ "_try_ml_compressor",+ lambda content, context, question=None: (content, len(content.split())),+ )++ compressed, _compressed_tokens, strategy_chain = router._apply_strategy_to_content(+ valid_json,+ CompressionStrategy.SMART_CRUSHER,+ context="",+ )++ assert CompressionStrategy.LOG.value in strategy_chain+ assert compressed == "[compressed]"+++# ---------------------------------------------------------------------------+# MIXED false-positive on source code: Python with dict literals + docstrings+# triggers ``has_json_blocks`` and ``has_prose`` in ``is_mixed_content``,+# misclassifying pure code as MIXED. The native detector (magika) correctly+# identifies it as SOURCE_CODE. ``_determine_strategy`` must trust the+# detector over the regex heuristics when confidence is high.+# ---------------------------------------------------------------------------+++def test_mixed_false_positive_on_source_code_overridden_by_detector(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """Python code with dicts + docstrings must not be routed to MIXED.++ ``is_mixed_content`` returns True (dict literals → has_json_blocks,+ docstrings → has_prose), but the native detector says SOURCE_CODE+ with high confidence. The detector must win.+ """+ router = ContentRouter(ContentRouterConfig())++ # Simulate the false positive: is_mixed_content says True, detector+ # says SOURCE_CODE with confidence 1.0 (real magika behaviour).+ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: True)+ monkeypatch.setattr(+ content_router_module,+ "_detect_content",+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),+ )++ strategy = router._determine_strategy("python code with dicts")+ # With prefer_code_aware_for_code=True (default), source code routes to CodeAware.+ assert strategy is CompressionStrategy.CODE_AWARE+ assert strategy is not CompressionStrategy.MIXED+++def test_mixed_still_used_when_detector_says_plain_text(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """Genuine mixed content (detector says PLAIN_TEXT) still uses MIXED.++ The override only fires when the detector says SOURCE_CODE. For+ actual mixed content (prose + JSON + code fences), the detector+ returns PLAIN_TEXT or another non-code type, and MIXED is correct.+ """+ router = ContentRouter(ContentRouterConfig())++ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: True)+ monkeypatch.setattr(+ content_router_module,+ "_detect_content",+ lambda content: DetectionResult(ContentType.PLAIN_TEXT, 0.9, {}),+ )++ strategy = router._determine_strategy("genuinely mixed content")+ assert strategy is CompressionStrategy.MIXED+++def test_mixed_still_used_when_detector_confidence_below_threshold(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """Low-confidence SOURCE_CODE detection does not override MIXED.++ If the detector is uncertain (confidence < 0.8), we keep the MIXED+ path rather than risking a false override.+ """+ router = ContentRouter(ContentRouterConfig())++ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: True)+ monkeypatch.setattr(+ content_router_module,+ "_detect_content",+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 0.5, {}),+ )++ strategy = router._determine_strategy("uncertain code")+ assert strategy is CompressionStrategy.MIXED+++def test_source_code_passthrough_preserves_content_verbatim(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """Code detected as SOURCE_CODE with CodeAware disabled → passthrough.++ When ``prefer_code_aware_for_code=False`` (default), code must pass+ through unmangled. Previously it fell to KOMPRESS which can destroy+ code semantics (98% compression, 11% recall on large blobs). Now+ it explicitly uses PASSTHROUGH.+ """+ router = ContentRouter(ContentRouterConfig(prefer_code_aware_for_code=False))++ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: False)+ monkeypatch.setattr(+ content_router_module,+ "_detect_content",+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),+ )++ code = "def hello():\n print('world')\n"+ result = router.compress(code)+ assert result.compressed == code+ assert result.strategy_used is CompressionStrategy.PASSTHROUGH+++def test_source_code_code_aware_enabled_uses_code_aware(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """When CodeAware is explicitly enabled, it is used for source code.++ The PASSTHROUGH fallback only applies when ``prefer_code_aware_for_code``+ is False. If a user explicitly enables it, CODE_AWARE is used.+ """+ router = ContentRouter(+ ContentRouterConfig(+ enable_code_aware=True,+ prefer_code_aware_for_code=True,+ )+ )++ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: False)+ monkeypatch.setattr(+ content_router_module,+ "_detect_content",+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),+ )++ strategy = router._determine_strategy("def foo(): pass")+ assert strategy is CompressionStrategy.CODE_AWARE+++def test_source_code_passthrough_does_not_invoke_kompress(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ """PASSTHROUGH for code must never call _try_ml_compressor.++ KOMPRESS on large code blobs can destroy semantics. The PASSTHROUGH+ path must be a pure passthrough — no ML compressor invocation.+ """+ router = ContentRouter(ContentRouterConfig(prefer_code_aware_for_code=False))++ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: False)+ monkeypatch.setattr(+ content_router_module,+ "_detect_content",+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),+ )++ def fail_kompress(*_args: object, **_kwargs: object) -> tuple[str, int]:+ raise AssertionError("PASSTHROUGH must not invoke Kompress")++ monkeypatch.setattr(router, "_try_ml_compressor", fail_kompress)++ code = "def hello():\n return 42\n"+ result = router.compress(code)+ assert result.compressed == code+ assert result.strategy_used is CompressionStrategy.PASSTHROUGH++# ---------------------------------------------------------------------------# Cache-safety tests for _process_content_blocks. These pin down the# block-level invariants that protect upstream prefix caches: