Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix(transforms): guard Log fallback against invalid JSON + fix MIXED false-positive on source code (#1347)

monkeygold02c77640a95e
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 44d033136..2028457f0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -233,6 +233,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* **proxy:** register `ToolResultInterceptorTransform` in explicit transforms list when `HEADROOM_INTERCEPT_ENABLED` is set — closes [#829](https://github.com/headroomlabs-ai/headroom/issues/829).
* **opencode:** write Headroom MCP config as a local stdio server instead of a remote `/mcp` URL, keep provider-only installs from adding MCP config, and allow `install apply --target opencode` ([#1380](https://github.com/headroomlabs-ai/headroom/issues/1380)).
* **code:** keep Python `from __future__` imports before executable code during AST compression and validate compressed Python with `compile(..., "exec")` so compile-time syntax rules are enforced ([#1233](https://github.com/chopratejas/headroom/issues/1233)).
+* **proxy/transforms:** stop the SMART_CRUSHER → Log fallback from collapsing truncated/invalid JSON tool outputs to a single CCR-retrieval marker (99.9% data loss when CCR retrieval isn't configured). The native magika detector classifies content by shape, not parseability, so a mid-stream-truncated JSON payload is tagged `json_array` and routed to SmartCrusher, which returns it unchanged; Kompress passes it through; the LogCompressor then treated the broken JSON as a multi-thousand-line "log" and reduced it to a retrieval marker. A JSON-validity guard now skips the Log fallback for content that fails `json.loads`, so invalid JSON passes through verbatim. Valid JSON arrays still reach the Log fallback (LogCompressor is a no-op on them). The guard also catches `RecursionError` from deeply nested JSON (e.g. `[[[[...]]]]` with 10k+ levels) so the router falls through to a safe strategy instead of crashing ([#1306](https://github.com/chopratejas/headroom/issues/1306)).
+* **proxy/transforms:** fix MIXED false-positive on source code. `is_mixed_content` uses regex heuristics that misclassify Python code with dict/list literals (`{`, `[` at line start → `has_json_blocks`) and docstrings/comments (`has_prose`) as mixed content, routing it through `_compress_mixed` which splits it into sections and dispatches each to KOMPRESS — wasting 1–1.4s of latency with 0% compression. When the native magika detector confidently says `SOURCE_CODE` (confidence ≥ 0.8), `_determine_strategy` now trusts it over the regex heuristics. Additionally, when `prefer_code_aware_for_code=False` (the default), source code now uses `PASSTHROUGH` instead of falling back to `KOMPRESS`, which can destroy code semantics (98% compression, 11% fact recall on large blobs). This honours the config's stated intent ("let code pass through unmangled") and reduces latency on code blobs by 23–33×.
+* **proxy/transforms:** catch `RecursionError` in `_try_detect_json` so deeply nested JSON arrays (10k+ nesting levels) no longer crash the content detector. The router falls through to a safe strategy (`TEXT` or `PASSTHROUGH`) instead of raising an unhandled exception.
* **proxy:** report real input tokens on the streaming `message_start` event for LiteLLM/Bedrock-backed requests. LiteLLM streaming never surfaces prompt tokens mid-stream, so `message_start.usage.input_tokens` was always `0`; Anthropic clients (e.g. Claude Code) read input-token metrics from that event, underreporting token usage by ~99% in OTel/CloudWatch dashboards. The Bedrock streamer now backfills `input_tokens` with the count Headroom actually sent upstream when the backend leaves it unset, preserving any non-zero value the backend genuinely reports ([#1132](https://github.com/chopratejas/headroom/issues/1132)).
* **proxy:** give buffered Anthropic request paths their own longer read timeout, so long `/v1/messages` turns and Anthropic batch or passthrough reads no longer trip the generic proxy cap while unrelated request timeouts stay unchanged.
* **proxy:** retry upstream 429 rate limits honoring `Retry-After` instead of passing them straight to the client. Both the non-streaming (`_retry_request`) and streaming (`_stream_response`) forwarders returned an upstream 429 verbatim, so a parallel agent fan-out that exceeded the per-minute limit aborted every run; 429s are now retried with backoff (honoring the upstream `Retry-After`, capped at `retry_max_delay_ms`), surfacing only the exhausted 429 to the client ([#1221](https://github.com/chopratejas/headroom/issues/1221)).
diff --git a/headroom/transforms/content_detector.py b/headroom/transforms/content_detector.py
index 92783627b..48a969a8a 100644
--- a/headroom/transforms/content_detector.py
+++ b/headroom/transforms/content_detector.py
@@ -293,24 +293,33 @@ def _try_detect_json(content: str) -> DetectionResult | None:
try:
value = json.loads(stripped)
+ except RecursionError:
+ # Deeply nested JSON (e.g. ``[[[[...]]]]`` with 10k+ levels) can
+ # exceed Python's recursion limit. Treat as non-JSON so the router
+ # falls through to a safe strategy instead of crashing.
+ return None
except ValueError:
# Not pure JSON. First: a run of whitespace-separated top-level JSON
# objects (web_search output, #1741) -> JSON_ARRAY.
if stripped.startswith("{"):
- items = _decode_concatenated_json(stripped)
+ try:
+ items = _decode_concatenated_json(stripped)
+ except RecursionError:
+ return None
if items and len(items) >= 2 and all(isinstance(item, dict) for item in items):
return DetectionResult(
ContentType.JSON_ARRAY,
1.0,
{"item_count": len(items), "is_dict_array": True, "concatenated": True},
)
+
# Otherwise decode one JSON value out of a small wrapped payload.
start = min((i for i in (stripped.find("{"), stripped.find("[")) if i >= 0), default=-1)
if start < 0:
return None
try:
value, end = _JSON_DECODER.raw_decode(stripped, start)
- except ValueError:
+ except (ValueError, RecursionError):
return None
# Accept only when the decoded JSON is the BULK of the content (see
# _JSON_MIN_BULK_FRACTION) — a small structural wrapper around a JSON body,
@@ -319,7 +328,7 @@ def _try_detect_json(content: str) -> DetectionResult | None:
return None
# A bare scalar (42, "s", true) is not structured data worth routing as JSON.
- if not isinstance(value, (dict, list)):
+ if not isinstance(value, dict | list):
return None
if isinstance(value, list):
diff --git a/headroom/transforms/content_router.py b/headroom/transforms/content_router.py
index 965183656..03ef0492e 100644
--- a/headroom/transforms/content_router.py
+++ b/headroom/transforms/content_router.py
@@ -360,6 +360,36 @@ def _json_shape(content: str) -> dict[str, Any]:
return {"is_json": True, "kind": type(parsed).__name__}
+def _content_is_valid_json(content: str) -> bool:
+ """Return True iff ``content`` parses as valid JSON.
+
+ Used by the SMART_CRUSHER → Log fallback guard (#1306): the native
+ magika detector tags content by shape, so truncated/mid-stream JSON
+ tool outputs are misclassified as ``json_array``. SmartCrusher can't
+ parse them and returns no savings; without this guard the LogCompressor
+ would then collapse the broken JSON to a single CCR-retrieval marker
+ (99.9% data loss). Valid JSON arrays still reach the Log fallback —
+ LogCompressor is a no-op on them, so the guard is safe for the
+ intended "repetitive JSONL" case.
+ """
+ try:
+ json.loads(content)
+ except (json.JSONDecodeError, ValueError):
+ return False
+ except RecursionError:
+ # Deeply nested JSON (e.g. ``[[[[...]]]]`` with 10k+ levels) can
+ # exceed Python's recursion limit inside ``json.loads``. Treat
+ # it as invalid so the Log fallback is skipped — the content
+ # passes through verbatim instead of being collapsed.
+ logger.warning(
+ "json.loads hit recursion limit on deeply nested JSON "
+ "(%d chars); treating as invalid for Log-fallback guard",
+ len(content),
+ )
+ return False
+ return True
+
+
def _mixed_indicators(content: str) -> dict[str, bool]:
return mixed_content_indicators(content)
@@ -1829,6 +1859,19 @@ class ContentRouter(Transform):
"""
# 1. Check for mixed content
if is_mixed_content(content):
+ # 2. Verify with the native detector: ``is_mixed_content`` uses
+ # cheap regex heuristics that produce false positives on source
+ # code. Python files with dict/list literals (``{``, ``[`` at
+ # line start) trigger ``has_json_blocks``, and docstrings/comments
+ # trigger ``has_prose`` — so a pure Python blob is misclassified
+ # as MIXED, routed through ``_compress_mixed`` which splits it
+ # into sections and dispatches each to KOMPRESS (no-op on code),
+ # wasting latency on splitting without any compression.
+ # When the native magika detector confidently says SOURCE_CODE,
+ # trust it over the regex heuristics.
+ detection = _detect_content(content)
+ if detection.content_type == ContentType.SOURCE_CODE and detection.confidence >= 0.8:
+ return self._strategy_from_detection(detection)
return CompressionStrategy.MIXED
# 2. Detect content type from content itself
@@ -1863,7 +1906,19 @@ class ContentRouter(Transform):
strategy == CompressionStrategy.CODE_AWARE
and not self.config.prefer_code_aware_for_code
):
- strategy = CompressionStrategy.KOMPRESS
+ # When CodeAware is not preferred, the intent is "let code pass
+ # through unmangled" (per the config comment). Previously this
+ # fell back to KOMPRESS, which is an ML compressor that can
+ # destroy code semantics — on a 45K-token Python blob it
+ # compresses to 912 tokens with 11% fact recall, making the
+ # code useless for an agent. The MIXED path accidentally
+ # protected code by splitting it into small sections that
+ # KOMPRESS passes through, but that was a side-effect, not
+ # intent. Now that the MIXED false-positive on source code is
+ # fixed (``_determine_strategy`` trusts the native detector),
+ # code reaches this path directly — so use PASSTHROUGH to
+ # honour the "unmangled" intent explicitly.
+ strategy = CompressionStrategy.PASSTHROUGH
return strategy
@@ -2387,9 +2442,24 @@ class ContentRouter(Transform):
# Kompress can't shrink but the log compressor can).
# Only attempted when the strategy was SMART_CRUSHER so
# we don't reroute genuine code/diff content.
+ #
+ # JSON-validity guard (#1306): the native magika detector
+ # classifies content by shape, not parseability, so a
+ # truncated/mid-stream JSON tool output is tagged
+ # ``json_array`` and routed to SMART_CRUSHER. SmartCrusher
+ # returns it unchanged (it can't parse the broken JSON),
+ # Kompress passes it through, and then the LogCompressor
+ # treats the whole thing as a multi-thousand-line "log"
+ # and collapses it to a single CCR-retrieval marker —
+ # 99.9% data loss when CCR retrieval isn't configured.
+ # Skip the Log fallback when the content isn't actually
+ # valid JSON; the passthrough at the bottom of this
+ # function preserves it verbatim. Valid JSON arrays still
+ # get the Log fallback (LogCompressor is a no-op on them).
if (
strategy == CompressionStrategy.SMART_CRUSHER
and self.config.enable_log_compressor
+ and _content_is_valid_json(content)
):
log_compressor = self._get_log_compressor()
if log_compressor is not None:
diff --git a/tests/test_transforms_content_router.py b/tests/test_transforms_content_router.py
index 2f333e911..3e478fc74 100644
--- a/tests/test_transforms_content_router.py
+++ b/tests/test_transforms_content_router.py
@@ -299,7 +299,7 @@ def test_content_router_strategy_and_compress_paths(monkeypatch: pytest.MonkeyPa
"_detect_content",
lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),
)
- assert router._determine_strategy("code") is CompressionStrategy.KOMPRESS
+ assert router._determine_strategy("code") is CompressionStrategy.PASSTHROUGH
assert (
router._strategy_from_detection(DetectionResult(ContentType.SEARCH_RESULTS, 1.0, {}))
is CompressionStrategy.SEARCH
@@ -623,6 +623,250 @@ def test_log_strategy_does_not_fallback_to_kompress_when_log_is_noop(
assert strategy_chain == ["log"]
+def test_smart_crusher_log_fallback_skipped_for_invalid_json(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Truncated/invalid JSON must not reach the Log fallback (#1306).
+
+ The native magika detector tags content by shape, so a truncated JSON
+ tool output is classified as ``json_array`` and routed to SMART_CRUSHER.
+ SmartCrusher returns it unchanged (can't parse broken JSON); Kompress
+ passes it through. Without the JSON-validity guard, the LogCompressor
+ would then collapse the whole thing to a single CCR-retrieval marker —
+ 99.9% data loss when CCR retrieval isn't configured. The guard skips
+ the Log fallback for invalid JSON so the content passes through verbatim.
+ """
+ router = ContentRouter(ContentRouterConfig())
+ # Truncated JSON object: starts valid, cut mid-string so json.loads fails.
+ truncated = '{"rows": [{"address": "Addr1", "name": "PoolA"}, {"address": "Addr2", "nam'
+
+ # SmartCrusher returns no savings (can't parse) → Kompress is a no-op
+ # (model not loaded in tests) → Log fallback must be SKIPPED.
+ class CollapsingLogCompressor:
+ def compress(self, content: str, bias: float = 1.0) -> SimpleNamespace:
+ # Simulate the collapse: LogCompressor treats the broken JSON as
+ # a multi-line "log" and reduces it to a retrieval marker.
+ return SimpleNamespace(
+ compressed="\n[2 lines compressed to 0. Retrieve more: hash=deadbeef]"
+ )
+
+ monkeypatch.setattr(router, "_get_log_compressor", lambda: CollapsingLogCompressor())
+ # Kompress not ready → returns content unchanged (real behavior in tests).
+ monkeypatch.setattr(
+ router,
+ "_try_ml_compressor",
+ lambda content, context, question=None: (content, len(content.split())),
+ )
+
+ compressed, compressed_tokens, strategy_chain = router._apply_strategy_to_content(
+ truncated,
+ CompressionStrategy.SMART_CRUSHER,
+ context="",
+ )
+
+ # Content preserved verbatim — no collapse.
+ assert compressed == truncated
+ assert CompressionStrategy.LOG.value not in strategy_chain
+ assert CompressionStrategy.KOMPRESS.value in strategy_chain
+
+
+def test_smart_crusher_log_fallback_runs_for_valid_json(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Valid JSON arrays still reach the Log fallback when SmartCrusher no-ops.
+
+ The JSON-validity guard (#1306) only blocks the Log fallback for
+ *invalid* JSON. Valid JSON arrays that SmartCrusher can't shrink (e.g.
+ repetitive JSONL the proxy's own log dumps) still get the Log fallback,
+ which is the intended "repetitive JSONL" path.
+ """
+ router = ContentRouter(ContentRouterConfig())
+ valid_json = '[{"a": 1}, {"a": 1}, {"a": 1}, {"a": 1}]'
+
+ class NoopSmartCrusher:
+ def crush(self, content: str, query: str = "", bias: float = 1.0) -> SimpleNamespace:
+ return SimpleNamespace(compressed=content)
+
+ class ShrinkingLogCompressor:
+ def compress(self, content: str, bias: float = 1.0) -> SimpleNamespace:
+ return SimpleNamespace(compressed="[compressed]")
+
+ monkeypatch.setattr(router, "_get_smart_crusher", lambda: NoopSmartCrusher())
+ monkeypatch.setattr(router, "_get_log_compressor", lambda: ShrinkingLogCompressor())
+ # Kompress no-op → Log fallback fires.
+ monkeypatch.setattr(
+ router,
+ "_try_ml_compressor",
+ lambda content, context, question=None: (content, len(content.split())),
+ )
+
+ compressed, _compressed_tokens, strategy_chain = router._apply_strategy_to_content(
+ valid_json,
+ CompressionStrategy.SMART_CRUSHER,
+ context="",
+ )
+
+ assert CompressionStrategy.LOG.value in strategy_chain
+ assert compressed == "[compressed]"
+
+
+# ---------------------------------------------------------------------------
+# MIXED false-positive on source code: Python with dict literals + docstrings
+# triggers ``has_json_blocks`` and ``has_prose`` in ``is_mixed_content``,
+# misclassifying pure code as MIXED. The native detector (magika) correctly
+# identifies it as SOURCE_CODE. ``_determine_strategy`` must trust the
+# detector over the regex heuristics when confidence is high.
+# ---------------------------------------------------------------------------
+
+
+def test_mixed_false_positive_on_source_code_overridden_by_detector(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Python code with dicts + docstrings must not be routed to MIXED.
+
+ ``is_mixed_content`` returns True (dict literals → has_json_blocks,
+ docstrings → has_prose), but the native detector says SOURCE_CODE
+ with high confidence. The detector must win.
+ """
+ router = ContentRouter(ContentRouterConfig())
+
+ # Simulate the false positive: is_mixed_content says True, detector
+ # says SOURCE_CODE with confidence 1.0 (real magika behaviour).
+ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: True)
+ monkeypatch.setattr(
+ content_router_module,
+ "_detect_content",
+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),
+ )
+
+ strategy = router._determine_strategy("python code with dicts")
+ # With prefer_code_aware_for_code=True (default), source code routes to CodeAware.
+ assert strategy is CompressionStrategy.CODE_AWARE
+ assert strategy is not CompressionStrategy.MIXED
+
+
+def test_mixed_still_used_when_detector_says_plain_text(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Genuine mixed content (detector says PLAIN_TEXT) still uses MIXED.
+
+ The override only fires when the detector says SOURCE_CODE. For
+ actual mixed content (prose + JSON + code fences), the detector
+ returns PLAIN_TEXT or another non-code type, and MIXED is correct.
+ """
+ router = ContentRouter(ContentRouterConfig())
+
+ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: True)
+ monkeypatch.setattr(
+ content_router_module,
+ "_detect_content",
+ lambda content: DetectionResult(ContentType.PLAIN_TEXT, 0.9, {}),
+ )
+
+ strategy = router._determine_strategy("genuinely mixed content")
+ assert strategy is CompressionStrategy.MIXED
+
+
+def test_mixed_still_used_when_detector_confidence_below_threshold(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Low-confidence SOURCE_CODE detection does not override MIXED.
+
+ If the detector is uncertain (confidence < 0.8), we keep the MIXED
+ path rather than risking a false override.
+ """
+ router = ContentRouter(ContentRouterConfig())
+
+ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: True)
+ monkeypatch.setattr(
+ content_router_module,
+ "_detect_content",
+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 0.5, {}),
+ )
+
+ strategy = router._determine_strategy("uncertain code")
+ assert strategy is CompressionStrategy.MIXED
+
+
+def test_source_code_passthrough_preserves_content_verbatim(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Code detected as SOURCE_CODE with CodeAware disabled → passthrough.
+
+ When ``prefer_code_aware_for_code=False`` (default), code must pass
+ through unmangled. Previously it fell to KOMPRESS which can destroy
+ code semantics (98% compression, 11% recall on large blobs). Now
+ it explicitly uses PASSTHROUGH.
+ """
+ router = ContentRouter(ContentRouterConfig(prefer_code_aware_for_code=False))
+
+ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: False)
+ monkeypatch.setattr(
+ content_router_module,
+ "_detect_content",
+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),
+ )
+
+ code = "def hello():\n print('world')\n"
+ result = router.compress(code)
+ assert result.compressed == code
+ assert result.strategy_used is CompressionStrategy.PASSTHROUGH
+
+
+def test_source_code_code_aware_enabled_uses_code_aware(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """When CodeAware is explicitly enabled, it is used for source code.
+
+ The PASSTHROUGH fallback only applies when ``prefer_code_aware_for_code``
+ is False. If a user explicitly enables it, CODE_AWARE is used.
+ """
+ router = ContentRouter(
+ ContentRouterConfig(
+ enable_code_aware=True,
+ prefer_code_aware_for_code=True,
+ )
+ )
+
+ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: False)
+ monkeypatch.setattr(
+ content_router_module,
+ "_detect_content",
+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),
+ )
+
+ strategy = router._determine_strategy("def foo(): pass")
+ assert strategy is CompressionStrategy.CODE_AWARE
+
+
+def test_source_code_passthrough_does_not_invoke_kompress(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """PASSTHROUGH for code must never call _try_ml_compressor.
+
+ KOMPRESS on large code blobs can destroy semantics. The PASSTHROUGH
+ path must be a pure passthrough — no ML compressor invocation.
+ """
+ router = ContentRouter(ContentRouterConfig(prefer_code_aware_for_code=False))
+
+ monkeypatch.setattr(content_router_module, "is_mixed_content", lambda content: False)
+ monkeypatch.setattr(
+ content_router_module,
+ "_detect_content",
+ lambda content: DetectionResult(ContentType.SOURCE_CODE, 1.0, {}),
+ )
+
+ def fail_kompress(*_args: object, **_kwargs: object) -> tuple[str, int]:
+ raise AssertionError("PASSTHROUGH must not invoke Kompress")
+
+ monkeypatch.setattr(router, "_try_ml_compressor", fail_kompress)
+
+ code = "def hello():\n return 42\n"
+ result = router.compress(code)
+ assert result.compressed == code
+ assert result.strategy_used is CompressionStrategy.PASSTHROUGH
+
+
# ---------------------------------------------------------------------------
# Cache-safety tests for _process_content_blocks. These pin down the
# block-level invariants that protect upstream prefix caches: