Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
feat(proxy): add provider-only HTTP proxy (#1807)
Andrew McFagueebe0a3bd7bbc
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 71b686432..9438e55f5 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -68,6 +68,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Features+* **proxy:** add provider-only HTTP proxy routing via `--http-proxy` and+ `HEADROOM_HTTP_PROXY`. Upstream LLM provider calls can now use an HTTP proxy+ without setting process-wide `HTTP_PROXY`/`HTTPS_PROXY` variables that are+ inherited by tool executions; proxied provider clients use HTTP/1.1 so HTTPS+ provider APIs can tunnel through CONNECT.* **proxy:** add output shaping for OpenAI Responses traffic on `/v1/responses` HTTP requests and Codex WebSocket `response.create` frames, with stable output-savings holdout keys and counted WS token strata for the experiment.* **wrap:** `headroom wrap claude --1m` preserves the 1M context window. Behind a custom `ANTHROPIC_BASE_URL` (the proxy) Claude Code drops the `context-1m` beta header and caps the window at 200k for entitled subscription users; the opt-in flag sets `ANTHROPIC_MODEL=<opus>[1m]` on the launched process so the 1M window activates through Headroom. A model already selected via `ANTHROPIC_MODEL` is preserved (only the `[1m]` suffix is appended) ([#1158](https://github.com/chopratejas/headroom/issues/1158)).* **learn:** weight loops in `headroom learn`. A new loop detector (`headroom/learn/loops.py`) recognizes repeated tool-call patterns — including RTK re-fetch loops, where RTK's output truncation makes the agent re-run larger-limit variants of a *successful* command — collapses output-limit variants to one signature, measures the wasted tokens, surfaces loops as a highest-priority digest section, and weights loop guardrails above one-off rules by their measured waste. Previously loops had no special weight and a no-failure re-fetch loop was skipped entirely. Adds an RTK-loop eval (`benchmarks/rtk_loop_learn_eval.py`) that reproduces a loop, runs it through Learn, and asserts the generated guardrail ranks first and prevents re-triggering.diff --git a/docs/content/docs/configuration.mdx b/docs/content/docs/configuration.mdxindex 2c53089fc..0fa0050dc 100644--- a/docs/content/docs/configuration.mdx+++ b/docs/content/docs/configuration.mdx@@ -228,6 +228,7 @@ headroom proxy --learn --min-evidence 3| `HEADROOM_MAX_CONNECTIONS` | Maximum upstream HTTP connections | `500` || `HEADROOM_MAX_KEEPALIVE` | Maximum upstream keep-alive connections | `100` || `HEADROOM_KEEPALIVE_EXPIRY` | Seconds an idle upstream keep-alive connection is kept open | `90` |+| `HEADROOM_HTTP_PROXY` | HTTP proxy URL for upstream provider requests only; HTTPS provider APIs use CONNECT | -- || `HEADROOM_BUDGET` | Daily budget limit in USD | -- || `HEADROOM_TELEMETRY` | Set to `on` to opt in to anonymous telemetry | `off` (opt-in) || `HEADROOM_STATELESS` | Set to `true` to disable filesystem writes | `false` |@@ -252,6 +253,8 @@ headroom proxy --learn --min-evidence 3| `HEADROOM_BETA_HEADER_STICKY` | Controls per-session `anthropic-beta` / `OpenAI-Beta` re-echo. `enabled` (default): the proxy unions beta tokens across turns within a session — if the client sends a token in turn N and omits it in turn N+1, the proxy re-injects it to preserve prefix-cache stability. `disabled`: the client's value is forwarded verbatim with no accumulation. Any other value raises at request time. See [Session Beta Header Tracking](/docs/configuration#session-beta-header-tracking). | `enabled` || `HEADROOM_BETA_TRACKER_MAX_SESSIONS` | LRU capacity of the in-memory session beta tracker. Once full, the oldest session entry is evicted. | `1000` |+For provider-only proxying, prefer `HEADROOM_HTTP_PROXY` over process-wide variables such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY`. HTTPX reads those global variables, but Headroom also passes them through to tool executions.+### Session Beta Header TrackingWhen running as a proxy, Headroom maintains a per-session union of `anthropic-beta` (and `OpenAI-Beta`) tokens via `SessionBetaTracker`. The session key is derived from the `x-headroom-session-id` header if present, otherwise from `md5(model + system_prompt[:500])[:16]` — stable across turns of the same conversation.diff --git a/docs/content/docs/proxy.mdx b/docs/content/docs/proxy.mdxindex 4bd675c1a..2f619d5b9 100644--- a/docs/content/docs/proxy.mdx+++ b/docs/content/docs/proxy.mdx@@ -34,6 +34,7 @@ Telemetry is **off by default** (opt-in). Opt in with `HEADROOM_TELEMETRY=on` or| `--limit-concurrency` | `1000` | Maximum concurrent connections before Uvicorn returns 503 || `--max-connections` | `500` | Maximum upstream HTTP connections || `--max-keepalive` | `100` | Maximum upstream keep-alive connections |+| `--http-proxy` | None | HTTP proxy URL for upstream provider requests only; HTTPS provider APIs use CONNECT || `--mode` | `token` | Optimization mode: `token` prioritizes compression, `cache` preserves provider prefix-cache stability || `--no-optimize` | `false` | Disable optimization (passthrough mode) || `--no-cache` | `false` | Disable semantic caching |@@ -50,6 +51,14 @@ Telemetry is **off by default** (opt-in). Opt in with `HEADROOM_TELEMETRY=on` or| `--no-telemetry` | `false` | Force anonymous telemetry off (already the default) || `--stateless` | `false` | Disable filesystem writes and keep runtime state in memory |+Use `--http-proxy` or `HEADROOM_HTTP_PROXY` when only provider API traffic should go through a proxy:++```bash+headroom proxy --http-proxy http://proxy.internal:8080+```++Avoid setting process-wide variables such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY` for this use case. HTTPX reads those variables too, but Headroom also inherits them into tool executions, so they can proxy unrelated tool traffic.+### Context management| Option | Default | Description |diff --git a/headroom/cli/proxy.py b/headroom/cli/proxy.pyindex c537309ae..eab35a381 100644--- a/headroom/cli/proxy.py+++ b/headroom/cli/proxy.py@@ -198,6 +198,15 @@ def dashboard(port: int, no_open: bool) -> None:"(SSLV3_ALERT_BAD_RECORD_MAC) when many concurrent streams are cancelled."),)+ "--http-proxy",+ default=None,+ envvar="HEADROOM_HTTP_PROXY",+ help=(+ "HTTP proxy URL for upstream provider requests only "+ "(HTTPS uses CONNECT; env: HEADROOM_HTTP_PROXY)."+ ),+)@click.option("--keepalive-expiry","keepalive_expiry",@@ -845,6 +854,7 @@ def proxy(max_keepalive_connections: int,keepalive_expiry: float,http2: bool,+ http_proxy: str | None,intercept_tool_results: bool,no_optimize: bool,no_cache: bool,@@ -1126,6 +1136,7 @@ def proxy(max_keepalive_connections=max_keepalive_connections,keepalive_expiry=keepalive_expiry,http2=http2,+ http_proxy=http_proxy,log_file=None if is_stateless else log_file,log_full_messages=log_messagesor os.environ.get("HEADROOM_LOG_MESSAGES", "").lower() in ("true", "1", "yes", "on"),diff --git a/headroom/proxy/models.py b/headroom/proxy/models.pyindex 34e9ec974..53088756b 100644--- a/headroom/proxy/models.py+++ b/headroom/proxy/models.py@@ -287,6 +287,7 @@ class ProxyConfig:max_keepalive_connections: int = 100keepalive_expiry: float = 90.0http2: bool = True+ http_proxy: str | None = None# Memory Systemmemory_enabled: bool = Falsediff --git a/headroom/proxy/server.py b/headroom/proxy/server.pyindex 9bb228f58..a771360c9 100644--- a/headroom/proxy/server.py+++ b/headroom/proxy/server.py@@ -597,6 +597,29 @@ def _apply_stateless_persistence(config: ProxyConfig) -> None:get_toin(TOINConfig(storage_path=""))+def _provider_httpx_client_options(+ config: ProxyConfig,+ verify: Any,+) -> tuple[bool, dict[str, Any]]:+ client_kwargs: dict[str, Any] = {+ "timeout": httpx.Timeout(+ connect=config.connect_timeout_seconds,+ read=config.request_timeout_seconds,+ write=config.request_timeout_seconds,+ pool=config.connect_timeout_seconds,+ ),+ "limits": httpx.Limits(+ max_connections=config.max_connections,+ max_keepalive_connections=config.max_keepalive_connections,+ keepalive_expiry=config.keepalive_expiry,+ ),+ "verify": verify,+ }+ if config.http_proxy:+ client_kwargs["proxy"] = config.http_proxy+ return config.http2 and not config.http_proxy, client_kwargs++class HeadroomProxy(StreamingMixin,AnthropicHandlerMixin,@@ -1372,27 +1395,12 @@ class HeadroomProxy(# is configured, else a strict-relaxed default context when# HEADROOM_TLS_STRICT=0, else httpx's default strict verification._verify = build_httpx_verify()- _client_kwargs: dict[str, Any] = {- "timeout": httpx.Timeout(- connect=self.config.connect_timeout_seconds,- read=self.config.request_timeout_seconds,- write=self.config.request_timeout_seconds,- pool=self.config.connect_timeout_seconds,- ),- "limits": httpx.Limits(- max_connections=self.config.max_connections,- max_keepalive_connections=self.config.max_keepalive_connections,- keepalive_expiry=self.config.keepalive_expiry,- ),- "verify": _verify,- }- self.http_client = httpx.AsyncClient(http2=self.config.http2, **_client_kwargs)+ _http2, _client_kwargs = _provider_httpx_client_options(self.config, _verify)+ self.http_client = httpx.AsyncClient(http2=_http2, **_client_kwargs)# Reuse the primary client when HTTP/2 is already off; otherwise keep a# dedicated HTTP/1.1 client for ChatGPT passthrough.self.http_client_h1 = (- self.http_client- if not self.config.http2- else httpx.AsyncClient(http2=False, **_client_kwargs)+ self.http_client if not _http2 else httpx.AsyncClient(http2=False, **_client_kwargs))logger.info("Headroom Proxy started")logger.info(f"Optimization: {'ENABLED' if self.config.optimize else 'DISABLED'}")@@ -1411,7 +1419,7 @@ class HeadroomProxy(logger.info(f"Connection Pool: max_connections={self.config.max_connections}, "f"max_keepalive={self.config.max_keepalive_connections}, "- f"http2={'ENABLED' if self.config.http2 else 'DISABLED'}"+ f"http2={'ENABLED' if _http2 else 'DISABLED'}")# Unit 4 pre-upstream concurrency announcement. Report the resolved@@ -4189,6 +4197,7 @@ def _proxy_config_from_env() -> ProxyConfig:max_keepalive_connections=_get_env_int("HEADROOM_MAX_KEEPALIVE", 100),keepalive_expiry=_get_env_float("HEADROOM_KEEPALIVE_EXPIRY", 90.0),http2=_get_env_bool("HEADROOM_HTTP2", True),+ http_proxy=os.environ.get("HEADROOM_HTTP_PROXY") or None,periodic_toin_stats_enabled=_get_env_bool("HEADROOM_PERIODIC_TOIN_STATS", True),proxy_token=os.environ.get("HEADROOM_PROXY_TOKEN") or None,offline=_get_env_bool("HEADROOM_OFFLINE", False),@@ -4246,7 +4255,7 @@ def run_server(# Format connection pool infopool_info = f"max={config.max_connections}, keepalive={config.max_keepalive_connections}"- http2_status = "ENABLED" if config.http2 else "DISABLED"+ http2_status = "ENABLED" if (config.http2 and not config.http_proxy) else "DISABLED"backend_status = format_backend_status(backend=config.backend,@@ -4580,6 +4589,10 @@ if __name__ == "__main__":action="store_true",help="Disable HTTP/2 (enabled by default for better throughput)",)+ parser.add_argument(+ "--http-proxy",+ help=("HTTP proxy URL for upstream provider requests only (env: HEADROOM_HTTP_PROXY)"),+ )parser.add_argument("--workers",type=int,@@ -4832,6 +4845,7 @@ if __name__ == "__main__":max_keepalive_connections=_get_env_int("HEADROOM_MAX_KEEPALIVE", args.max_keepalive),keepalive_expiry=_get_env_float("HEADROOM_KEEPALIVE_EXPIRY", args.keepalive_expiry),http2=not args.no_http2 and _get_env_bool("HEADROOM_HTTP2", True),+ http_proxy=_get_env_str("HEADROOM_HTTP_PROXY", args.http_proxy or "") or None,read_maturation=_get_env_bool("HEADROOM_READ_MATURATION", False),read_maturation_quiesce_turns=_get_env_int("HEADROOM_READ_MATURATION_QUIESCE_TURNS", 5),read_maturation_max_hold_turns=_get_env_int("HEADROOM_READ_MATURATION_MAX_HOLD_TURNS", 25),diff --git a/tests/test_cli_proxy_improvements.py b/tests/test_cli_proxy_improvements.pyindex 927088710..c10263d26 100644--- a/tests/test_cli_proxy_improvements.py+++ b/tests/test_cli_proxy_improvements.py@@ -89,6 +89,38 @@ class TestLearnNoLearnConflict:assert cfg.traffic_learning_enabled is False+class TestHttpProxyOption:+ """--http-proxy should configure only the provider HTTPX clients."""++ def test_http_proxy_cli_flag(self, runner: CliRunner, mock_run_server: dict) -> None:+ result = runner.invoke(+ main,+ ["proxy", "--http-proxy", "http://proxy.local:8080"],+ catch_exceptions=False,+ )+ assert result.exit_code == 0, result.output+ assert mock_run_server["config"].http_proxy == "http://proxy.local:8080"++ def test_http_proxy_env_var(self, runner: CliRunner, mock_run_server: dict) -> None:+ result = runner.invoke(+ main,+ ["proxy"],+ env={"HEADROOM_HTTP_PROXY": "http://proxy.local:8080"},+ catch_exceptions=False,+ )+ assert result.exit_code == 0, result.output+ assert mock_run_server["config"].http_proxy == "http://proxy.local:8080"++ def test_direct_server_env_http_proxy(self, monkeypatch: pytest.MonkeyPatch) -> None:+ import headroom.proxy.server as server_mod++ monkeypatch.delenv(server_mod._MULTI_WORKER_CONFIG_ENV, raising=False)+ monkeypatch.setenv("HEADROOM_HTTP_PROXY", "http://proxy.local:8080")++ config = server_mod._proxy_config_from_env()+ assert config.http_proxy == "http://proxy.local:8080"++class TestSubscriptionPollIntervalValidation:"""--subscription-poll-interval should reject values outside 1-3600."""diff --git a/tests/test_proxy_scalability.py b/tests/test_proxy_scalability.pyindex 864386f7e..1cd6a44a7 100644--- a/tests/test_proxy_scalability.py+++ b/tests/test_proxy_scalability.py@@ -250,7 +250,12 @@ class TestWorkerConfiguration:from headroom.proxy.server import _MULTI_WORKER_CONFIG_ENV, run_servercaptured = {}- config = ProxyConfig(host="0.0.0.0", port=8787, max_connections=200)+ config = ProxyConfig(+ host="0.0.0.0",+ port=8787,+ max_connections=200,+ http_proxy="http://proxy.local:8080",+ )def fake_run(app, **kwargs):captured["app"] = app@@ -270,6 +275,7 @@ class TestWorkerConfiguration:assert payload["host"] == "0.0.0.0"assert payload["port"] == 8787assert payload["max_connections"] == 200+ assert payload["http_proxy"] == "http://proxy.local:8080"finally:# run_server sets this via raw os.environ. Pop it directly rather# than via monkeypatch.delenv: delenv records the current (JSON)@@ -312,3 +318,28 @@ class TestWorkerConfiguration:server_mod.run_server(ProxyConfig(), print_banner=False)assert "loop" not in captured["kwargs"]+++class TestProviderHttpClientOptions:+ """Provider HTTPX options should keep proxy settings scoped to provider clients."""++ def test_default_http2_preserved_without_proxy(self):+ from headroom.proxy.models import ProxyConfig+ from headroom.proxy.server import _provider_httpx_client_options++ http2, kwargs = _provider_httpx_client_options(ProxyConfig(http2=True), verify=True)++ assert http2 is True+ assert "proxy" not in kwargs++ def test_http_proxy_sets_proxy_and_forces_http1(self):+ from headroom.proxy.models import ProxyConfig+ from headroom.proxy.server import _provider_httpx_client_options++ http2, kwargs = _provider_httpx_client_options(+ ProxyConfig(http2=True, http_proxy="http://proxy.local:8080"),+ verify=True,+ )++ assert http2 is False+ assert kwargs["proxy"] == "http://proxy.local:8080"