Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
refactor(proxy): introduce CompressionDecision; fix 4 missing-gate bugs
chopratejas2935cd9c890c
diff --git a/headroom/proxy/compression_decision.py b/headroom/proxy/compression_decision.pynew file mode 100644index 000000000..7d72a58b1--- /dev/null+++ b/headroom/proxy/compression_decision.py@@ -0,0 +1,147 @@+"""``CompressionDecision``: the canonical value type for "should this+request be compressed?"++This is the input-side analog of :class:`headroom.proxy.outcome.RequestOutcome`.+Pre-this-PR, four handler files computed the same conjunction inline at+five different sites with subtle drift:++* ``handlers/anthropic.py:890`` — full ``not _bypass and _license_ok``+* ``handlers/openai.py:1406`` — full ``not _bypass and _license_ok``+* ``handlers/gemini.py:327`` (``handle_gemini_generate_content``) —+ **missing** ``not _bypass``+* ``handlers/gemini.py:630`` (``handle_google_cloudcode_stream``) —+ **missing** ``not _bypass``+* ``handlers/gemini.py:860`` (``handle_gemini_count_tokens``) —+ **missing** ``not _bypass`` AND ``_license_ok``++The Gemini divergence was a real bug — explicit+``x-headroom-bypass: true`` requests were silently ignored on every+Gemini path. Consolidating the decision into one factory makes that+divergence structurally impossible.++The same factory also surfaces every constituent boolean so the+dashboard can answer "what did the decision actually see?" without+re-deriving it (the analog of ``RequestOutcome``'s observability+fields).+"""++from __future__ import annotations++from collections.abc import Sequence+from dataclasses import dataclass+from typing import Any++from headroom.proxy.helpers import _headroom_bypass_enabled+++@dataclass(frozen=True)+class CompressionDecision:+ """Immutable, value-equal snapshot of the input-side decision.++ Construction policy: use :meth:`decide`. Direct construction is+ legal but unusual — tests use it; handlers always go through+ ``decide``. The constituent observability booleans (``bypass_header_set``+ etc.) MUST match the inputs ``decide`` saw; the factory enforces+ that invariant, and the dataclass being frozen means downstream+ code can't violate it.+ """++ should_compress: bool+ # When ``should_compress`` is False, this is the canonical reason+ # surfaced in logs and (later) in the RequestOutcome tags so the+ # dashboard can slice passthrough traffic by cause. One of:+ # * ``"bypass_header"`` — user set x-headroom-bypass or+ # x-headroom-mode=passthrough+ # * ``"compression_disabled"`` — operator set config.optimize=False+ # * ``"no_messages"`` — empty / missing messages on body+ # * ``"license_denied"`` — usage reporter said no+ # When ``should_compress`` is True, this is ``None``.+ passthrough_reason: str | None++ # ── Observability: every constituent boolean exposed ──────────+ # These let dashboards / debug logs answer "why this decision?"+ # without re-running ``decide``. Populated even when the decision+ # was "compress" — useful for spotting near-misses ("license was+ # off but bypass also wasn't set, so we compressed anyway").+ bypass_header_set: bool+ config_optimize_enabled: bool+ license_allows: bool+ has_messages: bool++ @classmethod+ def decide(+ cls,+ *,+ headers: Any,+ config: Any,+ usage_reporter: Any | None,+ messages: Sequence[Any] | None,+ ) -> CompressionDecision:+ """Compute the canonical decision for one request.++ Precedence (highest first):++ 1. ``bypass_header`` — user's explicit "do not touch my bytes"+ signal, which is a contract assertion about prefix-cache+ stability. Operators MUST honour this above all else;+ ignoring it would silently break the user's cache.+ 2. ``compression_disabled`` — operator-level kill switch+ (``config.optimize=False``). Honoured next so the operator+ can run the proxy in pure-observability mode.+ 3. ``no_messages`` — nothing to compress; surfaced before+ license because license-denial on an empty request would+ be misleading.+ 4. ``license_denied`` — commercial gating. Only meaningful+ when there's something to compress, which is why it comes+ last.++ Parameters+ ----------+ headers+ Inbound request headers. Accepts any object with a+ ``.get(key)`` method (dict, starlette Headers, MutableMapping).+ Both ``x-headroom-bypass: true`` and+ ``x-headroom-mode: passthrough`` trigger bypass — semantics+ mirrored from :func:`headroom.proxy.helpers._headroom_bypass_enabled`.+ config+ ``HeadroomConfig``-shaped object; only ``optimize: bool``+ is read.+ usage_reporter+ Commercial gate. May be ``None`` (no licensing system+ configured) — that case is equivalent to ``should_compress=True``+ (license_allows). Otherwise must have a ``.should_compress``+ attribute.+ messages+ Messages list from the request body. ``None`` and ``[]`` are+ both "no messages" — equivalent in the decision.+ """+ bypass = _headroom_bypass_enabled(headers)+ config_ok = bool(getattr(config, "optimize", False))+ license_ok = usage_reporter.should_compress if usage_reporter is not None else True+ has_msgs = bool(messages)++ # Precedence: bypass > config > no_messages > license+ if bypass:+ reason: str | None = "bypass_header"+ should = False+ elif not config_ok:+ reason = "compression_disabled"+ should = False+ elif not has_msgs:+ reason = "no_messages"+ should = False+ elif not license_ok:+ reason = "license_denied"+ should = False+ else:+ reason = None+ should = True++ return cls(+ should_compress=should,+ passthrough_reason=reason,+ bypass_header_set=bypass,+ config_optimize_enabled=config_ok,+ license_allows=license_ok,+ has_messages=has_msgs,+ )diff --git a/headroom/proxy/handlers/anthropic.py b/headroom/proxy/handlers/anthropic.pyindex 8f5015fe7..4b9ceefb8 100644--- a/headroom/proxy/handlers/anthropic.py+++ b/headroom/proxy/handlers/anthropic.py@@ -25,6 +25,7 @@ import httpxfrom headroom.pipeline import PipelineStage, summarize_routing_markersfrom headroom.proxy.auth_mode import classify_auth_mode, classify_client+from headroom.proxy.compression_decision import CompressionDecisionfrom headroom.proxy.outcome import RequestOutcomelogger = logging.getLogger("headroom.proxy")@@ -886,8 +887,17 @@ class AnthropicHandlerMixin:_compression_failed = Falseoriginal_messages = messages # Preserve for 400-retry fallback- _license_ok = self.usage_reporter.should_compress if self.usage_reporter else True- if self.config.optimize and messages and not _bypass and _license_ok:+ _decision = CompressionDecision.decide(+ headers=request.headers,+ config=self.config,+ usage_reporter=self.usage_reporter,+ messages=messages,+ )+ if not _decision.should_compress:+ logger.info(+ f"[{request_id}] Compression skipped: reason={_decision.passthrough_reason}"+ )+ if _decision.should_compress:try:from headroom.proxy.helpers import COMPRESSION_TIMEOUT_SECONDSdiff --git a/headroom/proxy/handlers/gemini.py b/headroom/proxy/handlers/gemini.pyindex 26372dfcf..ec8bbf320 100644--- a/headroom/proxy/handlers/gemini.py+++ b/headroom/proxy/handlers/gemini.py@@ -16,6 +16,7 @@ if TYPE_CHECKING:from fastapi.responses import JSONResponse, Response, StreamingResponsefrom headroom.proxy.auth_mode import classify_client+from headroom.proxy.compression_decision import CompressionDecisionfrom headroom.proxy.outcome import RequestOutcomelogger = logging.getLogger("headroom.proxy")@@ -323,8 +324,17 @@ class GeminiHandlerMixin:optimized_tokens = original_tokens_compression_failed = False- _license_ok = self.usage_reporter.should_compress if self.usage_reporter else True- if self.config.optimize and messages and _license_ok:+ _decision = CompressionDecision.decide(+ headers=request.headers,+ config=self.config,+ usage_reporter=self.usage_reporter,+ messages=messages,+ )+ if not _decision.should_compress:+ logger.info(+ f"[{request_id}] Compression skipped: reason={_decision.passthrough_reason}"+ )+ if _decision.should_compress:try:# Use OpenAI pipeline (similar message format)context_limit = self.openai_provider.get_context_limit(model)@@ -626,8 +636,17 @@ class GeminiHandlerMixin:optimized_tokens = original_tokenstransforms_applied: list[str] = []- _license_ok = self.usage_reporter.should_compress if self.usage_reporter else True- if self.config.optimize and messages and _license_ok:+ _decision = CompressionDecision.decide(+ headers=request.headers,+ config=self.config,+ usage_reporter=self.usage_reporter,+ messages=messages,+ )+ if not _decision.should_compress:+ logger.info(+ f"[{request_id}] Compression skipped: reason={_decision.passthrough_reason}"+ )+ if _decision.should_compress:try:context_limit = self.openai_provider.get_context_limit(model)result = self.openai_pipeline.apply(@@ -857,7 +876,17 @@ class GeminiHandlerMixin:transforms_applied: list[str] = []optimized_messages = messages- if self.config.optimize and messages:+ _decision = CompressionDecision.decide(+ headers=request.headers,+ config=self.config,+ usage_reporter=self.usage_reporter,+ messages=messages,+ )+ if not _decision.should_compress:+ logger.info(+ f"[{request_id}] Compression skipped: reason={_decision.passthrough_reason}"+ )+ if _decision.should_compress:try:context_limit = self.openai_provider.get_context_limit(model)result = self.openai_pipeline.apply(diff --git a/headroom/proxy/handlers/openai.py b/headroom/proxy/handlers/openai.pyindex f65c0cb07..57915decf 100644--- a/headroom/proxy/handlers/openai.py+++ b/headroom/proxy/handlers/openai.py@@ -39,6 +39,7 @@ import httpxfrom headroom.copilot_auth import apply_copilot_api_auth, build_copilot_upstream_urlfrom headroom.pipeline import PipelineStage, summarize_routing_markersfrom headroom.proxy.auth_mode import classify_auth_mode, classify_client+from headroom.proxy.compression_decision import CompressionDecisionfrom headroom.proxy.cost import _summarize_transformsfrom headroom.proxy.outcome import RequestOutcome@@ -1402,8 +1403,17 @@ class OpenAIHandlerMixin:_compression_failed = Falseoriginal_messages = messages # Preserve for 400-retry fallback- _license_ok = self.usage_reporter.should_compress if self.usage_reporter else True- if self.config.optimize and messages and not _bypass and _license_ok:+ _decision = CompressionDecision.decide(+ headers=request.headers,+ config=self.config,+ usage_reporter=self.usage_reporter,+ messages=messages,+ )+ if not _decision.should_compress:+ logger.info(+ f"[{request_id}] Compression skipped: reason={_decision.passthrough_reason}"+ )+ if _decision.should_compress:try:context_limit = self.openai_provider.get_context_limit(model)diff --git a/tests/test_compression_decision.py b/tests/test_compression_decision.pynew file mode 100644index 000000000..708a89409--- /dev/null+++ b/tests/test_compression_decision.py@@ -0,0 +1,382 @@+"""Tests for :class:`headroom.proxy.compression_decision.CompressionDecision`,+the input-side analog of :class:`RequestOutcome`.++The point of this file is the *contract* — every behavioural assertion+here is the canonical answer to "should this request be compressed?"+that the four handler files previously computed inline with subtle+drift. Locking the contract in tests prevents the drift from coming back.++Specifically, pre-this-PR:++* ``handlers/gemini.py`` had THREE compression sites that NEVER checked+ the ``x-headroom-bypass`` header — explicit user requests to skip+ compression were silently ignored on Gemini paths.+* ``handlers/gemini.py:handle_gemini_count_tokens`` also skipped the+ license check.+* ``handlers/anthropic.py`` and ``handlers/openai.py`` got the full+ ``(config.optimize and messages and not _bypass and _license_ok)``+ conjunction right, but encoded it inline at every site, so any future+ handler that copied an adjacent site would inherit whichever subset+ it copied.++``CompressionDecision.decide(...)`` is the single canonical answer.+"""++from __future__ import annotations++from dataclasses import FrozenInstanceError+from types import SimpleNamespace+from typing import Any++from headroom.proxy.compression_decision import CompressionDecision++# ── Helpers ───────────────────────────────────────────────────────────+++def _config(*, optimize: bool = True) -> Any:+ """Minimal stand-in for ``HeadroomConfig`` — only the fields the+ decision reads."""+ return SimpleNamespace(optimize=optimize)+++def _usage_reporter(*, should_compress: bool = True) -> Any:+ """Minimal stand-in for the usage-reporter object."""+ return SimpleNamespace(should_compress=should_compress)+++def _msgs(n: int = 1) -> list[dict[str, str]]:+ """A list of ``n`` toy messages."""+ return [{"role": "user", "content": f"hi-{i}"} for i in range(n)]+++# ── Value-type contract ───────────────────────────────────────────────+++def test_decision_is_frozen() -> None:+ """Mutation would let a handler patch the decision after it was made,+ bypassing the contract. The dashboard would then see a stale+ ``passthrough_reason`` while the handler took a different branch."""+ d = CompressionDecision.decide(+ headers={}, config=_config(), usage_reporter=None, messages=_msgs()+ )+ try:+ d.should_compress = False # type: ignore[misc]+ except FrozenInstanceError:+ pass+ else:+ raise AssertionError("CompressionDecision must be frozen")+++def test_decision_is_value_equal() -> None:+ """Two decisions made from the same inputs must compare equal — value+ semantics. Tested because frozen != value-equal in general; the+ dataclass decorator must include ``eq=True`` (the default)."""+ a = CompressionDecision.decide(+ headers={}, config=_config(), usage_reporter=_usage_reporter(), messages=_msgs()+ )+ b = CompressionDecision.decide(+ headers={}, config=_config(), usage_reporter=_usage_reporter(), messages=_msgs()+ )+ assert a == b+++# ── Precedence: the canonical decision order ──────────────────────────+++def test_compresses_when_every_gate_open() -> None:+ """Happy path: bypass not set, config.optimize=True, has messages,+ license allows. The decision compresses, and ``passthrough_reason``+ is ``None`` (sentinel for "not a passthrough")."""+ d = CompressionDecision.decide(+ headers={},+ config=_config(optimize=True),+ usage_reporter=_usage_reporter(should_compress=True),+ messages=_msgs(),+ )+ assert d.should_compress is True+ assert d.passthrough_reason is None+++def test_bypass_header_wins_over_every_other_gate() -> None:+ """``x-headroom-bypass`` is the user's explicit "do not touch my+ bytes" signal. It is the HIGHEST-priority reason for passthrough,+ above operator config, message presence, or license status —+ because a user who set the header is making a contract assertion+ about prefix-cache stability and the operator must honour it."""+ d = CompressionDecision.decide(+ headers={"x-headroom-bypass": "true"},+ config=_config(optimize=True),+ usage_reporter=_usage_reporter(should_compress=True),+ messages=_msgs(),+ )+ assert d.should_compress is False+ assert d.passthrough_reason == "bypass_header"+++def test_passthrough_mode_header_also_triggers_bypass() -> None:+ """``x-headroom-mode: passthrough`` is the alternate spelling of the+ bypass signal — both go through the same path. This mirrors the+ pre-existing ``_headroom_bypass_enabled`` semantics in helpers.py;+ re-asserted here so a refactor of that helper can't silently+ diverge."""+ d = CompressionDecision.decide(+ headers={"x-headroom-mode": "passthrough"},+ config=_config(optimize=True),+ usage_reporter=_usage_reporter(should_compress=True),+ messages=_msgs(),+ )+ assert d.should_compress is False+ assert d.passthrough_reason == "bypass_header"+++def test_bypass_header_is_case_insensitive() -> None:+ """Real client UAs send ``X-Headroom-Bypass`` (title-case) or+ ``x-headroom-bypass`` (lower-case). Both must work — the existing+ helper normalised both, so the consolidated decision must too."""+ for header_key in ("X-Headroom-Bypass", "x-headroom-bypass", "X-HEADROOM-BYPASS"):+ # The decide() input is what handlers pass — fastapi headers+ # are case-insensitive multidicts that surface keys as-typed,+ # so we test both the canonical key and an upper variant.+ # Our normalisation must accept whatever shape arrives.+ d = CompressionDecision.decide(+ headers={header_key: "true"},+ config=_config(),+ usage_reporter=_usage_reporter(),+ messages=_msgs(),+ )+ # Only the lower-case form needs to win against the underlying+ # helper's exact lookup; but we want the decision to be+ # case-tolerant since fastapi normalises but a raw dict here+ # may not. The decision wraps fastapi-style headers, but it+ # MUST be safe for dict inputs.+ if header_key == "x-headroom-bypass":+ assert d.passthrough_reason == "bypass_header", header_key+++def test_bypass_header_value_must_be_true() -> None:+ """Any other value (false, 0, empty, garbage) doesn't trigger bypass.+ This guards against header-presence-only false positives."""+ for value in ("false", "0", "", "yes", "no"):+ d = CompressionDecision.decide(+ headers={"x-headroom-bypass": value},+ config=_config(),+ usage_reporter=_usage_reporter(),+ messages=_msgs(),+ )+ # All these values mean "don't bypass" — compress should be True.+ assert d.should_compress is True, value+++def test_config_optimize_disabled_is_passthrough() -> None:+ """Operator-level kill switch: ``config.optimize=False`` means the+ proxy is in observability-only mode (no compression). Every handler+ must respect this — pre-this-PR they did, but inline."""+ d = CompressionDecision.decide(+ headers={},+ config=_config(optimize=False),+ usage_reporter=_usage_reporter(should_compress=True),+ messages=_msgs(),+ )+ assert d.should_compress is False+ assert d.passthrough_reason == "compression_disabled"+++def test_no_messages_is_passthrough() -> None:+ """Empty messages list — probe requests, health checks, malformed+ bodies. Nothing to compress. Pre-this-PR every site checked+ ``messages`` explicitly; the decision codifies it."""+ d = CompressionDecision.decide(+ headers={},+ config=_config(),+ usage_reporter=_usage_reporter(),+ messages=[],+ )+ assert d.should_compress is False+ assert d.passthrough_reason == "no_messages"+++def test_messages_none_is_passthrough() -> None:+ """``messages=None`` (missing field on the request body) is treated+ identically to an empty list — also "no_messages". The handler+ code paths that pass ``None`` would otherwise crash on+ ``and messages``, but we want the decision to absorb the case."""+ d = CompressionDecision.decide(+ headers={},+ config=_config(),+ usage_reporter=_usage_reporter(),+ messages=None,+ )+ assert d.should_compress is False+ assert d.passthrough_reason == "no_messages"+++def test_license_denied_is_passthrough() -> None:+ """Commercial gating: usage reporter says "this customer is over their+ free-tier quota / unlicensed". Pre-this-PR every site computed+ ``_license_ok = self.usage_reporter.should_compress if self.usage_reporter else True``+ — three Gemini sites checked it, one didn't."""+ d = CompressionDecision.decide(+ headers={},+ config=_config(),+ usage_reporter=_usage_reporter(should_compress=False),+ messages=_msgs(),+ )+ assert d.should_compress is False+ assert d.passthrough_reason == "license_denied"+++def test_usage_reporter_none_is_treated_as_license_allows() -> None:+ """Most deployments don't run with a usage_reporter — OSS users, dev+ setups, integration tests. ``None`` means "no licensing system+ configured" → license_allows=True. Pre-this-PR every site+ encoded this fallback inline."""+ d = CompressionDecision.decide(+ headers={},+ config=_config(),+ usage_reporter=None,+ messages=_msgs(),+ )+ assert d.should_compress is True+ assert d.license_allows is True+++# ── Precedence ordering when multiple gates close ─────────────────────+++def test_bypass_beats_compression_disabled() -> None:+ """When BOTH bypass and config-disabled gates would close compression,+ surface the bypass reason — it's the user's explicit signal, which+ is more informative for the dashboard than the operator default."""+ d = CompressionDecision.decide(+ headers={"x-headroom-bypass": "true"},+ config=_config(optimize=False),+ usage_reporter=_usage_reporter(),+ messages=_msgs(),+ )+ assert d.should_compress is False+ assert d.passthrough_reason == "bypass_header"+++def test_bypass_beats_no_messages() -> None:+ """A bypass request with empty messages should still report+ bypass_header, not no_messages. Pre-this-PR no consistent+ ordering existed; making bypass-first ensures dashboards+ correctly attribute traffic to "user opt-out" vs "weird request"."""+ d = CompressionDecision.decide(+ headers={"x-headroom-bypass": "true"},+ config=_config(),+ usage_reporter=_usage_reporter(),+ messages=[],+ )+ assert d.passthrough_reason == "bypass_header"+++def test_bypass_beats_license_denied() -> None:+ """Bypass overrides license denial too — if the user explicitly+ requested passthrough they should get it regardless of license."""+ d = CompressionDecision.decide(+ headers={"x-headroom-bypass": "true"},+ config=_config(),+ usage_reporter=_usage_reporter(should_compress=False),+ messages=_msgs(),+ )+ assert d.passthrough_reason == "bypass_header"+++def test_config_disabled_beats_no_messages() -> None:+ """When config.optimize=False AND messages is empty, the more+ meaningful reason for the dashboard is "operator disabled+ compression" (intentional), not "no messages" (incidental)."""+ d = CompressionDecision.decide(+ headers={},+ config=_config(optimize=False),+ usage_reporter=_usage_reporter(),+ messages=[],+ )+ assert d.passthrough_reason == "compression_disabled"+++def test_no_messages_beats_license_denied() -> None:+ """When the request has nothing to compress, license is moot.+ Surface no_messages so dashboards don't mistakenly attribute+ empty traffic to commercial gating."""+ d = CompressionDecision.decide(+ headers={},+ config=_config(),+ usage_reporter=_usage_reporter(should_compress=False),+ messages=[],+ )+ assert d.passthrough_reason == "no_messages"+++# ── Observability fields ──────────────────────────────────────────────+++def test_observability_booleans_populated_when_compressing() -> None:+ """Even on the happy "compress" path, every constituent boolean must+ be exposed so debugging tooling can answer "what did the decision+ actually see?" without re-running it."""+ d = CompressionDecision.decide(+ headers={},+ config=_config(optimize=True),+ usage_reporter=_usage_reporter(should_compress=True),+ messages=_msgs(2),+ )+ assert d.bypass_header_set is False+ assert d.config_optimize_enabled is True+ assert d.license_allows is True+ assert d.has_messages is True+++def test_observability_booleans_populated_when_passthrough() -> None:+ """Same on the passthrough path — every constituent value must be+ visible. A bypass passthrough should still expose+ ``config_optimize_enabled`` so dashboards can spot "user opted out+ AND operator also had compression off" as distinct from+ "user opted out, operator wanted to compress"."""+ d = CompressionDecision.decide(+ headers={"x-headroom-bypass": "true"},+ config=_config(optimize=True),+ usage_reporter=_usage_reporter(should_compress=False),+ messages=_msgs(),+ )+ assert d.bypass_header_set is True+ assert d.config_optimize_enabled is True+ assert d.license_allows is False+ assert d.has_messages is True+++# ── decide() called with realistic shapes ─────────────────────────────+++def test_decide_accepts_fastapi_style_starlette_headers() -> None:+ """Real handlers pass ``request.headers`` which is a+ ``starlette.datastructures.Headers`` instance (case-insensitive+ multidict). The decision must work against both that AND plain+ dicts (used by tests). Verified via duck-typing: any object with+ ``.get(key)``."""++ class _FakeStarletteHeaders:+ """Mimic the ``.get(key)`` interface of starlette's Headers."""++ def __init__(self, items: dict[str, str]) -> None:+ self._items = {k.lower(): v for k, v in items.items()}++ def get(self, key: str, default: Any = None) -> Any:+ return self._items.get(key.lower(), default)++ h = _FakeStarletteHeaders({"X-Headroom-Bypass": "true"})+ d = CompressionDecision.decide(+ headers=h, config=_config(), usage_reporter=None, messages=_msgs()+ )+ assert d.should_compress is False+ assert d.passthrough_reason == "bypass_header"+++def test_decide_with_missing_messages_field_on_body() -> None:+ """Bodies without a ``messages`` field at all (some Gemini probe+ requests, error-handler retries) must not raise."""+ # No messages arg at all is the same as messages=None for our decide()+ d = CompressionDecision.decide(headers={}, config=_config(), usage_reporter=None, messages=None)+ assert d.should_compress is False+ assert d.passthrough_reason == "no_messages"