Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

feat(proxy): add provider-only HTTP proxy (#1807)

Andrew McFagueebe0a3bd7bbc
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 71b686432..9438e55f5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -68,6 +68,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Features
+* **proxy:** add provider-only HTTP proxy routing via `--http-proxy` and
+ `HEADROOM_HTTP_PROXY`. Upstream LLM provider calls can now use an HTTP proxy
+ without setting process-wide `HTTP_PROXY`/`HTTPS_PROXY` variables that are
+ inherited by tool executions; proxied provider clients use HTTP/1.1 so HTTPS
+ provider APIs can tunnel through CONNECT.
* **proxy:** add output shaping for OpenAI Responses traffic on `/v1/responses` HTTP requests and Codex WebSocket `response.create` frames, with stable output-savings holdout keys and counted WS token strata for the experiment.
* **wrap:** `headroom wrap claude --1m` preserves the 1M context window. Behind a custom `ANTHROPIC_BASE_URL` (the proxy) Claude Code drops the `context-1m` beta header and caps the window at 200k for entitled subscription users; the opt-in flag sets `ANTHROPIC_MODEL=<opus>[1m]` on the launched process so the 1M window activates through Headroom. A model already selected via `ANTHROPIC_MODEL` is preserved (only the `[1m]` suffix is appended) ([#1158](https://github.com/chopratejas/headroom/issues/1158)).
* **learn:** weight loops in `headroom learn`. A new loop detector (`headroom/learn/loops.py`) recognizes repeated tool-call patterns — including RTK re-fetch loops, where RTK's output truncation makes the agent re-run larger-limit variants of a *successful* command — collapses output-limit variants to one signature, measures the wasted tokens, surfaces loops as a highest-priority digest section, and weights loop guardrails above one-off rules by their measured waste. Previously loops had no special weight and a no-failure re-fetch loop was skipped entirely. Adds an RTK-loop eval (`benchmarks/rtk_loop_learn_eval.py`) that reproduces a loop, runs it through Learn, and asserts the generated guardrail ranks first and prevents re-triggering.
diff --git a/docs/content/docs/configuration.mdx b/docs/content/docs/configuration.mdx
index 2c53089fc..0fa0050dc 100644
--- a/docs/content/docs/configuration.mdx
+++ b/docs/content/docs/configuration.mdx
@@ -228,6 +228,7 @@ headroom proxy --learn --min-evidence 3
| `HEADROOM_MAX_CONNECTIONS` | Maximum upstream HTTP connections | `500` |
| `HEADROOM_MAX_KEEPALIVE` | Maximum upstream keep-alive connections | `100` |
| `HEADROOM_KEEPALIVE_EXPIRY` | Seconds an idle upstream keep-alive connection is kept open | `90` |
+| `HEADROOM_HTTP_PROXY` | HTTP proxy URL for upstream provider requests only; HTTPS provider APIs use CONNECT | -- |
| `HEADROOM_BUDGET` | Daily budget limit in USD | -- |
| `HEADROOM_TELEMETRY` | Set to `on` to opt in to anonymous telemetry | `off` (opt-in) |
| `HEADROOM_STATELESS` | Set to `true` to disable filesystem writes | `false` |
@@ -252,6 +253,8 @@ headroom proxy --learn --min-evidence 3
| `HEADROOM_BETA_HEADER_STICKY` | Controls per-session `anthropic-beta` / `OpenAI-Beta` re-echo. `enabled` (default): the proxy unions beta tokens across turns within a session — if the client sends a token in turn N and omits it in turn N+1, the proxy re-injects it to preserve prefix-cache stability. `disabled`: the client's value is forwarded verbatim with no accumulation. Any other value raises at request time. See [Session Beta Header Tracking](/docs/configuration#session-beta-header-tracking). | `enabled` |
| `HEADROOM_BETA_TRACKER_MAX_SESSIONS` | LRU capacity of the in-memory session beta tracker. Once full, the oldest session entry is evicted. | `1000` |
+For provider-only proxying, prefer `HEADROOM_HTTP_PROXY` over process-wide variables such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY`. HTTPX reads those global variables, but Headroom also passes them through to tool executions.
+
### Session Beta Header Tracking
When running as a proxy, Headroom maintains a per-session union of `anthropic-beta` (and `OpenAI-Beta`) tokens via `SessionBetaTracker`. The session key is derived from the `x-headroom-session-id` header if present, otherwise from `md5(model + system_prompt[:500])[:16]` — stable across turns of the same conversation.
diff --git a/docs/content/docs/proxy.mdx b/docs/content/docs/proxy.mdx
index 4bd675c1a..2f619d5b9 100644
--- a/docs/content/docs/proxy.mdx
+++ b/docs/content/docs/proxy.mdx
@@ -34,6 +34,7 @@ Telemetry is **off by default** (opt-in). Opt in with `HEADROOM_TELEMETRY=on` or
| `--limit-concurrency` | `1000` | Maximum concurrent connections before Uvicorn returns 503 |
| `--max-connections` | `500` | Maximum upstream HTTP connections |
| `--max-keepalive` | `100` | Maximum upstream keep-alive connections |
+| `--http-proxy` | None | HTTP proxy URL for upstream provider requests only; HTTPS provider APIs use CONNECT |
| `--mode` | `token` | Optimization mode: `token` prioritizes compression, `cache` preserves provider prefix-cache stability |
| `--no-optimize` | `false` | Disable optimization (passthrough mode) |
| `--no-cache` | `false` | Disable semantic caching |
@@ -50,6 +51,14 @@ Telemetry is **off by default** (opt-in). Opt in with `HEADROOM_TELEMETRY=on` or
| `--no-telemetry` | `false` | Force anonymous telemetry off (already the default) |
| `--stateless` | `false` | Disable filesystem writes and keep runtime state in memory |
+Use `--http-proxy` or `HEADROOM_HTTP_PROXY` when only provider API traffic should go through a proxy:
+
+```bash
+headroom proxy --http-proxy http://proxy.internal:8080
+```
+
+Avoid setting process-wide variables such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY` for this use case. HTTPX reads those variables too, but Headroom also inherits them into tool executions, so they can proxy unrelated tool traffic.
+
### Context management
| Option | Default | Description |
diff --git a/headroom/cli/proxy.py b/headroom/cli/proxy.py
index c537309ae..eab35a381 100644
--- a/headroom/cli/proxy.py
+++ b/headroom/cli/proxy.py
@@ -198,6 +198,15 @@ def dashboard(port: int, no_open: bool) -> None:
"(SSLV3_ALERT_BAD_RECORD_MAC) when many concurrent streams are cancelled."
),
)
+ "--http-proxy",
+ default=None,
+ envvar="HEADROOM_HTTP_PROXY",
+ help=(
+ "HTTP proxy URL for upstream provider requests only "
+ "(HTTPS uses CONNECT; env: HEADROOM_HTTP_PROXY)."
+ ),
+)
@click.option(
"--keepalive-expiry",
"keepalive_expiry",
@@ -845,6 +854,7 @@ def proxy(
max_keepalive_connections: int,
keepalive_expiry: float,
http2: bool,
+ http_proxy: str | None,
intercept_tool_results: bool,
no_optimize: bool,
no_cache: bool,
@@ -1126,6 +1136,7 @@ def proxy(
max_keepalive_connections=max_keepalive_connections,
keepalive_expiry=keepalive_expiry,
http2=http2,
+ http_proxy=http_proxy,
log_file=None if is_stateless else log_file,
log_full_messages=log_messages
or os.environ.get("HEADROOM_LOG_MESSAGES", "").lower() in ("true", "1", "yes", "on"),
diff --git a/headroom/proxy/models.py b/headroom/proxy/models.py
index 34e9ec974..53088756b 100644
--- a/headroom/proxy/models.py
+++ b/headroom/proxy/models.py
@@ -287,6 +287,7 @@ class ProxyConfig:
max_keepalive_connections: int = 100
keepalive_expiry: float = 90.0
http2: bool = True
+ http_proxy: str | None = None
# Memory System
memory_enabled: bool = False
diff --git a/headroom/proxy/server.py b/headroom/proxy/server.py
index 9bb228f58..a771360c9 100644
--- a/headroom/proxy/server.py
+++ b/headroom/proxy/server.py
@@ -597,6 +597,29 @@ def _apply_stateless_persistence(config: ProxyConfig) -> None:
get_toin(TOINConfig(storage_path=""))
+def _provider_httpx_client_options(
+ config: ProxyConfig,
+ verify: Any,
+) -> tuple[bool, dict[str, Any]]:
+ client_kwargs: dict[str, Any] = {
+ "timeout": httpx.Timeout(
+ connect=config.connect_timeout_seconds,
+ read=config.request_timeout_seconds,
+ write=config.request_timeout_seconds,
+ pool=config.connect_timeout_seconds,
+ ),
+ "limits": httpx.Limits(
+ max_connections=config.max_connections,
+ max_keepalive_connections=config.max_keepalive_connections,
+ keepalive_expiry=config.keepalive_expiry,
+ ),
+ "verify": verify,
+ }
+ if config.http_proxy:
+ client_kwargs["proxy"] = config.http_proxy
+ return config.http2 and not config.http_proxy, client_kwargs
+
+
class HeadroomProxy(
StreamingMixin,
AnthropicHandlerMixin,
@@ -1372,27 +1395,12 @@ class HeadroomProxy(
# is configured, else a strict-relaxed default context when
# HEADROOM_TLS_STRICT=0, else httpx's default strict verification.
_verify = build_httpx_verify()
- _client_kwargs: dict[str, Any] = {
- "timeout": httpx.Timeout(
- connect=self.config.connect_timeout_seconds,
- read=self.config.request_timeout_seconds,
- write=self.config.request_timeout_seconds,
- pool=self.config.connect_timeout_seconds,
- ),
- "limits": httpx.Limits(
- max_connections=self.config.max_connections,
- max_keepalive_connections=self.config.max_keepalive_connections,
- keepalive_expiry=self.config.keepalive_expiry,
- ),
- "verify": _verify,
- }
- self.http_client = httpx.AsyncClient(http2=self.config.http2, **_client_kwargs)
+ _http2, _client_kwargs = _provider_httpx_client_options(self.config, _verify)
+ self.http_client = httpx.AsyncClient(http2=_http2, **_client_kwargs)
# Reuse the primary client when HTTP/2 is already off; otherwise keep a
# dedicated HTTP/1.1 client for ChatGPT passthrough.
self.http_client_h1 = (
- self.http_client
- if not self.config.http2
- else httpx.AsyncClient(http2=False, **_client_kwargs)
+ self.http_client if not _http2 else httpx.AsyncClient(http2=False, **_client_kwargs)
)
logger.info("Headroom Proxy started")
logger.info(f"Optimization: {'ENABLED' if self.config.optimize else 'DISABLED'}")
@@ -1411,7 +1419,7 @@ class HeadroomProxy(
logger.info(
f"Connection Pool: max_connections={self.config.max_connections}, "
f"max_keepalive={self.config.max_keepalive_connections}, "
- f"http2={'ENABLED' if self.config.http2 else 'DISABLED'}"
+ f"http2={'ENABLED' if _http2 else 'DISABLED'}"
)
# Unit 4 pre-upstream concurrency announcement. Report the resolved
@@ -4189,6 +4197,7 @@ def _proxy_config_from_env() -> ProxyConfig:
max_keepalive_connections=_get_env_int("HEADROOM_MAX_KEEPALIVE", 100),
keepalive_expiry=_get_env_float("HEADROOM_KEEPALIVE_EXPIRY", 90.0),
http2=_get_env_bool("HEADROOM_HTTP2", True),
+ http_proxy=os.environ.get("HEADROOM_HTTP_PROXY") or None,
periodic_toin_stats_enabled=_get_env_bool("HEADROOM_PERIODIC_TOIN_STATS", True),
proxy_token=os.environ.get("HEADROOM_PROXY_TOKEN") or None,
offline=_get_env_bool("HEADROOM_OFFLINE", False),
@@ -4246,7 +4255,7 @@ def run_server(
# Format connection pool info
pool_info = f"max={config.max_connections}, keepalive={config.max_keepalive_connections}"
- http2_status = "ENABLED" if config.http2 else "DISABLED"
+ http2_status = "ENABLED" if (config.http2 and not config.http_proxy) else "DISABLED"
backend_status = format_backend_status(
backend=config.backend,
@@ -4580,6 +4589,10 @@ if __name__ == "__main__":
action="store_true",
help="Disable HTTP/2 (enabled by default for better throughput)",
)
+ parser.add_argument(
+ "--http-proxy",
+ help=("HTTP proxy URL for upstream provider requests only (env: HEADROOM_HTTP_PROXY)"),
+ )
parser.add_argument(
"--workers",
type=int,
@@ -4832,6 +4845,7 @@ if __name__ == "__main__":
max_keepalive_connections=_get_env_int("HEADROOM_MAX_KEEPALIVE", args.max_keepalive),
keepalive_expiry=_get_env_float("HEADROOM_KEEPALIVE_EXPIRY", args.keepalive_expiry),
http2=not args.no_http2 and _get_env_bool("HEADROOM_HTTP2", True),
+ http_proxy=_get_env_str("HEADROOM_HTTP_PROXY", args.http_proxy or "") or None,
read_maturation=_get_env_bool("HEADROOM_READ_MATURATION", False),
read_maturation_quiesce_turns=_get_env_int("HEADROOM_READ_MATURATION_QUIESCE_TURNS", 5),
read_maturation_max_hold_turns=_get_env_int("HEADROOM_READ_MATURATION_MAX_HOLD_TURNS", 25),
diff --git a/tests/test_cli_proxy_improvements.py b/tests/test_cli_proxy_improvements.py
index 927088710..c10263d26 100644
--- a/tests/test_cli_proxy_improvements.py
+++ b/tests/test_cli_proxy_improvements.py
@@ -89,6 +89,38 @@ class TestLearnNoLearnConflict:
assert cfg.traffic_learning_enabled is False
+class TestHttpProxyOption:
+ """--http-proxy should configure only the provider HTTPX clients."""
+
+ def test_http_proxy_cli_flag(self, runner: CliRunner, mock_run_server: dict) -> None:
+ result = runner.invoke(
+ main,
+ ["proxy", "--http-proxy", "http://proxy.local:8080"],
+ catch_exceptions=False,
+ )
+ assert result.exit_code == 0, result.output
+ assert mock_run_server["config"].http_proxy == "http://proxy.local:8080"
+
+ def test_http_proxy_env_var(self, runner: CliRunner, mock_run_server: dict) -> None:
+ result = runner.invoke(
+ main,
+ ["proxy"],
+ env={"HEADROOM_HTTP_PROXY": "http://proxy.local:8080"},
+ catch_exceptions=False,
+ )
+ assert result.exit_code == 0, result.output
+ assert mock_run_server["config"].http_proxy == "http://proxy.local:8080"
+
+ def test_direct_server_env_http_proxy(self, monkeypatch: pytest.MonkeyPatch) -> None:
+ import headroom.proxy.server as server_mod
+
+ monkeypatch.delenv(server_mod._MULTI_WORKER_CONFIG_ENV, raising=False)
+ monkeypatch.setenv("HEADROOM_HTTP_PROXY", "http://proxy.local:8080")
+
+ config = server_mod._proxy_config_from_env()
+ assert config.http_proxy == "http://proxy.local:8080"
+
+
class TestSubscriptionPollIntervalValidation:
"""--subscription-poll-interval should reject values outside 1-3600."""
diff --git a/tests/test_proxy_scalability.py b/tests/test_proxy_scalability.py
index 864386f7e..1cd6a44a7 100644
--- a/tests/test_proxy_scalability.py
+++ b/tests/test_proxy_scalability.py
@@ -250,7 +250,12 @@ class TestWorkerConfiguration:
from headroom.proxy.server import _MULTI_WORKER_CONFIG_ENV, run_server
captured = {}
- config = ProxyConfig(host="0.0.0.0", port=8787, max_connections=200)
+ config = ProxyConfig(
+ host="0.0.0.0",
+ port=8787,
+ max_connections=200,
+ http_proxy="http://proxy.local:8080",
+ )
def fake_run(app, **kwargs):
captured["app"] = app
@@ -270,6 +275,7 @@ class TestWorkerConfiguration:
assert payload["host"] == "0.0.0.0"
assert payload["port"] == 8787
assert payload["max_connections"] == 200
+ assert payload["http_proxy"] == "http://proxy.local:8080"
finally:
# run_server sets this via raw os.environ. Pop it directly rather
# than via monkeypatch.delenv: delenv records the current (JSON)
@@ -312,3 +318,28 @@ class TestWorkerConfiguration:
server_mod.run_server(ProxyConfig(), print_banner=False)
assert "loop" not in captured["kwargs"]
+
+
+class TestProviderHttpClientOptions:
+ """Provider HTTPX options should keep proxy settings scoped to provider clients."""
+
+ def test_default_http2_preserved_without_proxy(self):
+ from headroom.proxy.models import ProxyConfig
+ from headroom.proxy.server import _provider_httpx_client_options
+
+ http2, kwargs = _provider_httpx_client_options(ProxyConfig(http2=True), verify=True)
+
+ assert http2 is True
+ assert "proxy" not in kwargs
+
+ def test_http_proxy_sets_proxy_and_forces_http1(self):
+ from headroom.proxy.models import ProxyConfig
+ from headroom.proxy.server import _provider_httpx_client_options
+
+ http2, kwargs = _provider_httpx_client_options(
+ ProxyConfig(http2=True, http_proxy="http://proxy.local:8080"),
+ verify=True,
+ )
+
+ assert http2 is False
+ assert kwargs["proxy"] == "http://proxy.local:8080"