Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix(offline): keep litellm's pricing table local under HEADROOM_OFFLINE (#4242)

Kenneth Wong3fef15d376d9
diff --git a/docs/content/docs/proxy.mdx b/docs/content/docs/proxy.mdx
index 4ea3cec86..1309d2f76 100644
--- a/docs/content/docs/proxy.mdx
+++ b/docs/content/docs/proxy.mdx
@@ -433,7 +433,7 @@ with a privileged job.
| `HEADROOM_PROXY_TOKEN` | none | Require a token from non-loopback callers, as `X-Headroom-Proxy-Token: <token>` or `Authorization: Bearer <token>`. The token is removed from the request before it is forwarded upstream; clients that also send a provider credential should use `X-Headroom-Proxy-Token` and keep `Authorization` for the provider. Also gates the read-only operator routes (`/stats-history`, `/quota`, `/subscription-window`, `/metrics`), which otherwise answer loopback and trusted-CIDR callers only. |
| `HEADROOM_ALLOW_UNAUTHENTICATED_BIND` | `false` | Acknowledge a non-loopback bind (`--host 0.0.0.0`, `::`, a LAN address) with **no** `HEADROOM_PROXY_TOKEN`. Without a token or this flag the proxy refuses to start, because `/v1/*` would relay upstream with the operator's credentials for every peer that can reach the port. Only set it when the runtime already restricts reachability (a container published on `127.0.0.1`). |
| `HEADROOM_COMPRESS_ALLOW_REMOTE` | `false` | Allow non-loopback callers to reach [`POST /v1/compress`](#post-v1compress) and `POST /v1/usage`. Required to run Headroom as a remote gateway/sidecar; without it remote callers get `404`. |
-| `HEADROOM_OFFLINE` | `false` | Air-gap mode: refuse every connection Headroom itself initiates — telemetry, update checks, license reporting, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP and Langfuse export, Copilot auth, the subscription pollers, the OpenAI embedders, Headroom Cloud compression and the TLS diagnostics (`headroom doctor --network` reports them as skipped). Four things still work on purpose: forwarding your own requests to the upstream you configured, operator-configured local endpoints (the Ollama embedder), loopback health probes, and paths an `is_offline()` check already disables. Refusals surface as messages, not tracebacks. See the air-gap notes in `docs/metrics-technical-guide.md`. |
+| `HEADROOM_OFFLINE` | `false` | Air-gap mode: refuse every connection Headroom itself initiates — telemetry, update checks, license reporting, model/tokenizer/binary/dataset downloads, litellm's pricing-table refresh, CrewAI's install tracking, remote Kompress, OTLP and Langfuse export, Copilot auth, the subscription pollers, the OpenAI embedders, Headroom Cloud compression and the TLS diagnostics (`headroom doctor --network` reports them as skipped). Four things still work on purpose: forwarding your own requests to the upstream you configured, operator-configured local endpoints (the Ollama embedder), loopback health probes, and paths an `is_offline()` check already disables. Refusals surface as messages, not tracebacks. See the air-gap notes in `docs/metrics-technical-guide.md`. |
| `HEADROOM_LICENSE` | unset | Licence token, read by core and every licensed extension. `HEADROOM_LICENSE_KEY` is a deprecated alias for one release and logs a warning. A licence on its own sends nothing to Headroom. |
| `HEADROOM_USAGE_REPORTING` | `false` | Set `1` to let the proxy validate the licence and post aggregate usage counts (no content) to the Headroom cloud. Off unless set; `HEADROOM_OFFLINE` still overrides it. |
| `--stateless` / `HEADROOM_STATELESS` | `false` | Keep all state in memory; no filesystem writes (disables logs, memory, TOIN). |
diff --git a/docs/metrics-technical-guide.md b/docs/metrics-technical-guide.md
index eefb53c69..7dd812b59 100644
--- a/docs/metrics-technical-guide.md
+++ b/docs/metrics-technical-guide.md
@@ -268,7 +268,7 @@ Verify with `curl -s localhost:8787/stats | jq .otel`.
**Multi-tenant labels:** `register_otel_metric_attribute_provider()` adds request-scoped attributes (tenant, team, cost centre) to every OTel datapoint. Max 16 attributes, 256 chars each.
-**Air-gapped deployments:** `HEADROOM_OFFLINE=1` refuses every connection Headroom itself decides to make to a destination Headroom itself chose. That is the whole list, not a sample: the anonymous usage beacon (which is **on by default**), the update check, the license/usage reporter, **OTLP metric and Langfuse trace export**, HuggingFace / Kompress / fastembed model and tokenizer downloads (Python and Rust), the remote Kompress endpoint, the `headroom install` release-binary and codebase-memory-mcp downloads, the BFCL eval-dataset fetch and the provider SDK clients the eval harness drives, GitHub Copilot device-flow auth and token exchange, the Anthropic / Codex / Copilot subscription pollers, the OpenAI embedders, the Headroom Cloud compression modes in the ASGI and LiteLLM integrations, and the TLS diagnostics (`headroom doctor --network` endpoint checks and the certificate-chain re-probe after an upstream TLS failure).
+**Air-gapped deployments:** `HEADROOM_OFFLINE=1` refuses every connection Headroom itself decides to make to a destination Headroom itself chose. That is the whole list, not a sample: the anonymous usage beacon (which is **on by default**), the update check, the license/usage reporter, **OTLP metric and Langfuse trace export**, HuggingFace / Kompress / fastembed model and tokenizer downloads (Python and Rust), the remote Kompress endpoint, the `headroom install` release-binary and codebase-memory-mcp downloads, litellm's import-time pricing-table refresh, CrewAI's import-time install tracking, the BFCL eval-dataset fetch and the provider SDK clients the eval harness drives, GitHub Copilot device-flow auth and token exchange, the Anthropic / Codex / Copilot subscription pollers, the OpenAI embedders, the Headroom Cloud compression modes in the ASGI and LiteLLM integrations, and the TLS diagnostics (`headroom doctor --network` endpoint checks and the certificate-chain re-probe after an upstream TLS failure).
Four things are deliberately still allowed, and they are the complete set of exceptions:
@@ -279,6 +279,8 @@ Four things are deliberately still allowed, and they are the complete set of exc
Those four are enumerated with written reasons in `_EGRESS_ALLOWLIST` in `tests/test_offline_egress_chokepoint.py`, which fails the build if a new egress path appears that is neither guarded nor one of them. There is no category for "known violation": a path that can dial the internet with the flag set is a bug. A refusal reaches you as a message — a Click error on the CLI, a named "model unavailable" from a model loader, a logged line from a background poller — never as a traceback and never as a silent degradation. Enforcing the same policy at the network layer as well is still good practice; it is no longer the only thing standing between you and Headroom-initiated egress.
+litellm's pricing-table refresh is refused by setting `LITELLM_LOCAL_MODEL_COST_MAP=True` before litellm is imported, so it holds only when Headroom is the first thing in the process to import litellm. A host that imports litellm before Headroom, such as a LiteLLM proxy loading the Headroom callback, has made that fetch already and must set the variable itself. Offline, cost and savings figures use the pricing table bundled with the installed litellm release, so a model released after it prices as unknown.
+
OTLP export is refused loudly: with `HEADROOM_OFFLINE=1` set, `HEADROOM_OTEL_METRICS_ENABLED=1` plus the default `otlp_http` exporter makes the proxy exit 78 at startup with an explanation, rather than quietly dropping metrics. There is no exemption for a collector that looks local — an in-cluster address is not reliably distinguishable from an internet one. For metrics under an air-gap, either scrape `localhost:8787/metrics` or set `HEADROOM_OTEL_METRICS_EXPORTER=console`, both of which stay on-box. `HEADROOM_KOMPRESS_ENDPOINT` and `HEADROOM_LANGFUSE_ENABLED` are refused the same way, for the same reason.
---
diff --git a/headroom/__init__.py b/headroom/__init__.py
index 95a6bdb94..38cb05a06 100644
--- a/headroom/__init__.py
+++ b/headroom/__init__.py
@@ -75,6 +75,7 @@ from typing import Any
from ._ort import ensure_ort_dylib_pinned
from ._version import __version__ # noqa: F401
+from .offline import pin_litellm_cost_map_local
# Must run before anything can import `headroom._core`: on Windows the
# Rust core resolves onnxruntime.dll at runtime (ort load-dynamic), and
@@ -83,6 +84,12 @@ from ._version import __version__ # noqa: F401
# ~microseconds. See `headroom/_ort.py` for the full story.
ensure_ort_dylib_pinned()
+# Under HEADROOM_OFFLINE, litellm must not fetch its pricing table from GitHub
+# at import time. Each litellm import in this package pins that itself; this
+# call also covers a host that imports litellm after Headroom. A no-op unless
+# the switch is set. See `headroom/offline.py`.
+pin_litellm_cost_map_local()
+
from .compress import ( # noqa: E402
CompressConfig,
CompressResult,
diff --git a/headroom/backends/litellm.py b/headroom/backends/litellm.py
index e1e13fc60..7e96683cb 100644
--- a/headroom/backends/litellm.py
+++ b/headroom/backends/litellm.py
@@ -21,6 +21,7 @@ from collections.abc import AsyncIterator
from dataclasses import dataclass, field
from typing import Any
+from headroom.offline import pin_litellm_cost_map_local
from headroom.proxy.public_errors import client_message
from headroom.utils import format_exception_message
@@ -67,6 +68,7 @@ try:
import os as _os
_env_snapshot = set(_os.environ)
+ pin_litellm_cost_map_local()
import litellm
from litellm import acompletion
from litellm.utils import supports_prompt_caching
@@ -536,6 +538,7 @@ def _caller_key_travels_to(model: str, key: str) -> bool:
if not key.startswith("sk-ant-"):
return True
try:
+ pin_litellm_cost_map_local()
from litellm import get_llm_provider
provider = (get_llm_provider(model)[1] or "").lower()
diff --git a/headroom/cli/evals.py b/headroom/cli/evals.py
index d9b16f03f..8a6c848de 100644
--- a/headroom/cli/evals.py
+++ b/headroom/cli/evals.py
@@ -7,6 +7,8 @@ from pathlib import Path
import click
+from headroom.offline import pin_litellm_cost_map_local
+
from .main import main
@@ -480,6 +482,7 @@ def _run_memory_eval(
answer_fn = None
if answer_model:
try:
+ pin_litellm_cost_map_local()
import litellm
def answer_fn(question: str, memories: list[str]) -> str:
diff --git a/headroom/cli/learn.py b/headroom/cli/learn.py
index 47269c11f..e917b5b84 100644
--- a/headroom/cli/learn.py
+++ b/headroom/cli/learn.py
@@ -7,6 +7,8 @@ from typing import TYPE_CHECKING, Any, NamedTuple
import click
+from headroom.offline import pin_litellm_cost_map_local
+
if TYPE_CHECKING:
from ..learn.base import LearnPlugin
@@ -390,6 +392,7 @@ def _make_llm_judge(model: str) -> Any:
try:
import json
+ pin_litellm_cost_map_local()
import litellm
except ImportError:
return None
diff --git a/headroom/evals/html_extraction.py b/headroom/evals/html_extraction.py
index 4303ae9ab..28abe446a 100644
--- a/headroom/evals/html_extraction.py
+++ b/headroom/evals/html_extraction.py
@@ -15,7 +15,7 @@ from collections.abc import Callable
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any
-from headroom.offline import guard_egress
+from headroom.offline import guard_egress, pin_litellm_cost_map_local
if TYPE_CHECKING:
from headroom.transforms.html_extractor import HTMLExtractor
@@ -316,6 +316,7 @@ class HTMLExtractionEvaluator:
else:
try:
+ pin_litellm_cost_map_local()
import litellm
def judge(question: str, ground_truth: str, prediction: str) -> tuple[float, str]:
@@ -396,6 +397,7 @@ Answer concisely and factually based only on the content provided."""
)
else:
+ pin_litellm_cost_map_local()
import litellm
litellm_response = litellm.completion(
diff --git a/headroom/evals/memory/judge.py b/headroom/evals/memory/judge.py
index 85091c9da..49d761c52 100644
--- a/headroom/evals/memory/judge.py
+++ b/headroom/evals/memory/judge.py
@@ -17,7 +17,7 @@ from __future__ import annotations
import logging
from collections.abc import Callable
-from headroom.offline import guard_egress
+from headroom.offline import guard_egress, pin_litellm_cost_map_local
logger = logging.getLogger(__name__)
@@ -169,6 +169,7 @@ def create_litellm_judge(
score, reasoning = judge_fn(question, ground_truth, prediction)
"""
try:
+ pin_litellm_cost_map_local()
import litellm
except ImportError as e:
raise ImportError(
diff --git a/headroom/evals/memory/runner.py b/headroom/evals/memory/runner.py
index 145215192..5775ff267 100644
--- a/headroom/evals/memory/runner.py
+++ b/headroom/evals/memory/runner.py
@@ -30,6 +30,7 @@ from headroom.evals.memory.locomo import (
)
from headroom.evals.metrics import compute_exact_match, compute_f1
from headroom.memory import HierarchicalMemory, MemoryConfig
+from headroom.offline import pin_litellm_cost_map_local
logger = logging.getLogger(__name__)
@@ -277,6 +278,7 @@ class LoCoMoEvaluator:
List of extracted memory dicts with 'content' and 'category'.
"""
try:
+ pin_litellm_cost_map_local()
import litellm
except ImportError:
logger.warning("litellm not available, falling back to raw dialogue storage")
diff --git a/headroom/evals/memory/runner_v2.py b/headroom/evals/memory/runner_v2.py
index 63434e6dc..5864bebf0 100644
--- a/headroom/evals/memory/runner_v2.py
+++ b/headroom/evals/memory/runner_v2.py
@@ -24,8 +24,6 @@ from datetime import datetime
from pathlib import Path
from typing import Any
-import litellm
-
from headroom.evals.memory.locomo import (
LOCOMO_CATEGORIES,
LoCoMoCase,
@@ -37,6 +35,12 @@ from headroom.memory.backends.local import LocalBackend, LocalBackendConfig
from headroom.memory.models import Memory
from headroom.memory.system import MemoryBackend
from headroom.memory.tools import MEMORY_TOOLS
+from headroom.offline import pin_litellm_cost_map_local
+
+# Before the import: litellm fetches its pricing table at import time unless
+# told not to, and HEADROOM_OFFLINE forbids that fetch.
+pin_litellm_cost_map_local()
+import litellm # noqa: E402
logger = logging.getLogger(__name__)
diff --git a/headroom/integrations/crewai/agents.py b/headroom/integrations/crewai/agents.py
index 650c60f9f..9c8ccd9c6 100644
--- a/headroom/integrations/crewai/agents.py
+++ b/headroom/integrations/crewai/agents.py
@@ -27,7 +27,10 @@ from __future__ import annotations
import logging
from typing import Any
+from headroom.offline import pin_crewai_telemetry_off
+
try:
+ pin_crewai_telemetry_off()
from crewai.tools.base_tool import BaseTool
CREWAI_AVAILABLE = True
diff --git a/headroom/integrations/litellm_callback.py b/headroom/integrations/litellm_callback.py
index f6fcb0185..0c77bf685 100644
--- a/headroom/integrations/litellm_callback.py
+++ b/headroom/integrations/litellm_callback.py
@@ -24,12 +24,13 @@ from headroom.integrations._compress_backend import ( # noqa: F401
_DEFAULT_CLOUD_URL,
CompressBackend,
)
-from headroom.offline import guard_egress # noqa: F401
+from headroom.offline import guard_egress, pin_litellm_cost_map_local # noqa: F401
logger = logging.getLogger(__name__)
try:
+ pin_litellm_cost_map_local()
from litellm.integrations.custom_logger import CustomLogger as _CustomLogger
except ImportError: # litellm not installed — fall back to plain object
_CustomLogger = object # type: ignore[assignment,misc]
diff --git a/headroom/learn/analyzer.py b/headroom/learn/analyzer.py
index 7cda57ca5..b5a6d28e5 100644
--- a/headroom/learn/analyzer.py
+++ b/headroom/learn/analyzer.py
@@ -25,6 +25,7 @@ import time
import typing
from headroom._subprocess import Popen, run
+from headroom.offline import pin_litellm_cost_map_local
from .loops import LoopPattern, apply_loop_weighting, detect_loops, format_loops_for_digest
from .models import (
@@ -1095,6 +1096,7 @@ def _call_llm(
if model in _CLI_MODEL_IDS:
return _call_cli_llm(digest, model, on_progress=on_progress)
+ pin_litellm_cost_map_local()
import litellm
# Suppress LiteLLM's verbose logging
diff --git a/headroom/offline.py b/headroom/offline.py
index 9361648e9..56154952c 100644
--- a/headroom/offline.py
+++ b/headroom/offline.py
@@ -8,8 +8,9 @@ deployment exists to carry.
Refused (this is not a sample — it is the rule, and the meta-test enforces it):
the telemetry beacon, the update check, the license/usage reporter, OTLP and
Langfuse export, HuggingFace/Kompress/fastembed model downloads, release-binary
-downloads (codebase-memory-mcp among them), eval dataset downloads and the provider SDK
-clients the eval harness drives, GitHub Copilot device-flow auth and token
+downloads (codebase-memory-mcp among them), litellm's import-time pricing-table
+refresh, CrewAI's import-time install tracking, eval dataset downloads and the
+provider SDK clients the eval harness drives, GitHub Copilot device-flow auth and token
exchange, the Anthropic / Codex / Copilot subscription pollers, the OpenAI
embedders, both Headroom Cloud compression integrations, and the TLS
diagnostics (``doctor --network`` endpoint checks and the certificate-chain
@@ -101,6 +102,87 @@ def apply_offline_env() -> None:
os.environ.setdefault("TRANSFORMERS_OFFLINE", "1")
+class _OfflineEnvPin:
+ """An environment variable Headroom forces while ``HEADROOM_OFFLINE`` is
+ on, for a third-party library that dials out during its own import and
+ only listens to that variable.
+
+ Overwritten rather than ``setdefault``: these libraries accept only
+ ``true``, so an operator ``1``, ``False`` or empty value would reopen a
+ connection the air-gap switch exists to refuse. The pin is Headroom's, so
+ it is also undone: called with the switch off, it puts back the value it
+ replaced, provided the variable still holds the pin. An operator's own
+ value is never touched outside offline mode.
+ """
+
+ def __init__(self, name: str, value: str) -> None:
+ self.name = name
+ self.value = value
+ # Unset while no pin is in place; otherwise a 1-tuple of the value the
+ # pin replaced (None when the variable was absent).
+ self.before: tuple[str | None] | None = None
+
+ def __call__(self) -> None:
+ if is_offline():
+ if self.before is None:
+ self.before = (os.environ.get(self.name),)
+ os.environ[self.name] = self.value
+ elif self.before is not None:
+ (before,) = self.before
+ self.before = None
+ if os.environ.get(self.name) == self.value:
+ if before is None:
+ os.environ.pop(self.name, None)
+ else:
+ os.environ[self.name] = before
+
+
+LITELLM_LOCAL_COST_MAP_ENV = "LITELLM_LOCAL_MODEL_COST_MAP"
+_litellm_cost_map_pin = _OfflineEnvPin(LITELLM_LOCAL_COST_MAP_ENV, "True")
+
+CREWAI_TELEMETRY_ENV = "CREWAI_DISABLE_TELEMETRY"
+_crewai_telemetry_pin = _OfflineEnvPin(CREWAI_TELEMETRY_ENV, "true")
+
+
+def pin_litellm_cost_map_local() -> None:
+ """Keep litellm on its bundled pricing table under ``HEADROOM_OFFLINE``.
+
+ ``import litellm`` fetches ``model_prices_and_context_window.json`` from
+ raw.githubusercontent.com at import time (three attempts) unless
+ ``LITELLM_LOCAL_MODEL_COST_MAP`` is ``true``. That is not a client Headroom
+ builds, so no ``guard_egress`` site can dominate it; the only lever is the
+ env var, read once during that import. So every ``import litellm`` in
+ ``headroom/`` calls this first, in the same scope, because the switch is
+ read when it is checked rather than when the package loads
+ (``create_app(ProxyConfig(offline=True))`` turns it on after import). The
+ chokepoint meta-test fails the build for an import that does not.
+ ``headroom/__init__.py`` also calls it, which covers a host process that
+ imports litellm itself after importing Headroom; a host that imports
+ litellm *before* Headroom has made that fetch already and must set the
+ variable itself.
+
+ Forced and given back as described on :class:`_OfflineEnvPin`, so a
+ process that leaves offline mode before its first litellm import gets the
+ online pricing refresh again.
+ """
+ _litellm_cost_map_pin()
+
+
+def pin_crewai_telemetry_off() -> None:
+ """Keep CrewAI's install tracking quiet under ``HEADROOM_OFFLINE``.
+
+ ``import crewai`` starts a thread that requests a Scarf pixel from
+ api.scarf.sh unless ``CREWAI_DISABLE_TELEMETRY`` (or ``OTEL_SDK_DISABLED``
+ / ``CREWAI_DISABLE_TRACKING``) is ``true``. Headroom imports CrewAI
+ whenever it is installed, through ``headroom.integrations``, so even a
+ LiteLLM callback import under the air-gap switch dialled out. As with
+ litellm, the variable is the only lever, so every ``crewai`` import in
+ ``headroom/`` calls this first (the chokepoint meta-test enforces it), and
+ it is forced and given back as described on :class:`_OfflineEnvPin`.
+ """
+ _crewai_telemetry_pin()
+
+
class OfflineEgressBlocked(BaseException):
"""Raised by :func:`guard_egress` when ``HEADROOM_OFFLINE`` is in force.
diff --git a/headroom/perf/analyzer.py b/headroom/perf/analyzer.py
index b09560aea..55c880902 100644
--- a/headroom/perf/analyzer.py
+++ b/headroom/perf/analyzer.py
@@ -20,6 +20,7 @@ from datetime import datetime, timedelta
from pathlib import Path
from headroom import paths as _paths
+from headroom.offline import pin_litellm_cost_map_local
from headroom.pricing.litellm_pricing import resolve_litellm_model
log = logging.getLogger(__name__)
@@ -74,6 +75,7 @@ _STAGE_TIMINGS_RE = re.compile(
# directly instead of maintaining our own pricing tables.
try:
+ pin_litellm_cost_map_local()
import litellm as _litellm
_LITELLM_AVAILABLE = True
diff --git a/headroom/pricing/counterfactual.py b/headroom/pricing/counterfactual.py
index 5f3bc65cf..063e68fa6 100644
--- a/headroom/pricing/counterfactual.py
+++ b/headroom/pricing/counterfactual.py
@@ -63,6 +63,7 @@ from typing import Any
from headroom.cache_economics import CACHE_ECONOMICS
from headroom.log_safety import describe_exception, safe_id
+from headroom.offline import pin_litellm_cost_map_local
from headroom.pricing.cache_ttl import CACHE_WRITE_MULTIPLIERS
logger = logging.getLogger(__name__)
@@ -338,6 +339,7 @@ def _litellm() -> Any | None:
deployments often run without it. Callers degrade to list pricing.
"""
try:
+ pin_litellm_cost_map_local()
import litellm
except Exception: # pragma: no cover - environment-dependent
return None
diff --git a/headroom/pricing/litellm_pricing.py b/headroom/pricing/litellm_pricing.py
index 2a325330c..ab16ccc6e 100644
--- a/headroom/pricing/litellm_pricing.py
+++ b/headroom/pricing/litellm_pricing.py
@@ -15,6 +15,7 @@ from dataclasses import dataclass
from datetime import datetime
from typing import Any
+from headroom.offline import pin_litellm_cost_map_local
from headroom.pricing.deepseek_tiers import (
LEGACY_MODEL_IDS,
OFF_PEAK_RATES_PER_1M,
@@ -34,6 +35,11 @@ from headroom.pricing.litellm_model_resolution import (
# litellm added. The module itself is fully imported and cached in
# `sys.modules`; subsequent `import litellm` calls hit the cache and
# don't re-run the dotenv side effect.
+#
+# Under HEADROOM_OFFLINE, litellm must also stay on its bundled pricing table
+# rather than fetch one from GitHub during that import.
+pin_litellm_cost_map_local()
+
try:
import os as _os
diff --git a/headroom/providers/anthropic.py b/headroom/providers/anthropic.py
index e8b29eed4..46ac3d853 100644
--- a/headroom/providers/anthropic.py
+++ b/headroom/providers/anthropic.py
@@ -25,6 +25,7 @@ from datetime import datetime
from typing import Any, cast
from headroom import paths as _paths
+from headroom.offline import pin_litellm_cost_map_local
from headroom.pricing.deepseek_tiers import (
LEGACY_MODEL_IDS,
OFF_PEAK_RATES_PER_1M,
@@ -49,6 +50,7 @@ def _get_litellm_clients() -> tuple[Any | None, Any | None]:
return None, None
try:
+ pin_litellm_cost_map_local()
import litellm
litellm.suppress_debug_info = True
diff --git a/headroom/providers/cohere.py b/headroom/providers/cohere.py
index 421f01961..980a0e218 100644
--- a/headroom/providers/cohere.py
+++ b/headroom/providers/cohere.py
@@ -22,12 +22,14 @@ from datetime import date
from typing import Any
from headroom.log_safety import describe_exception, safe_id
+from headroom.offline import pin_litellm_cost_map_local
from headroom.pricing.litellm_pricing import estimate_cost_from_tokens
from headroom.tokenizers import EstimatingTokenCounter
from .base import Provider, TokenCounter
try:
+ pin_litellm_cost_map_local()
import litellm
LITELLM_AVAILABLE = True
diff --git a/headroom/providers/google.py b/headroom/providers/google.py
index 436196f36..88215c9a1 100644
--- a/headroom/providers/google.py
+++ b/headroom/providers/google.py
@@ -26,6 +26,7 @@ from datetime import date
from typing import Any
from headroom.models.registry import ModelRegistry
+from headroom.offline import pin_litellm_cost_map_local
from headroom.pricing.litellm_pricing import estimate_cost_from_tokens
from headroom.tokenizers import EstimatingTokenCounter
@@ -33,6 +34,7 @@ from .base import Provider, TokenCounter
# Check if litellm is available for pricing/context limit lookups
try:
+ pin_litellm_cost_map_local()
import litellm
LITELLM_AVAILABLE = True
diff --git a/headroom/providers/litellm.py b/headroom/providers/litellm.py
index 3197a5f19..437ff6b3c 100644
--- a/headroom/providers/litellm.py
+++ b/headroom/providers/litellm.py
@@ -25,6 +25,7 @@ import logging
import os
from typing import Any
+from headroom.offline import pin_litellm_cost_map_local
from headroom.pricing.litellm_pricing import estimate_cost_from_tokens
from headroom.tokenizers import EstimatingTokenCounter
@@ -38,6 +39,7 @@ try:
# module-level suppression flags below can be set.
os.environ.setdefault("LITELLM_SUPPRESS_DEBUG_INFO", "True")
+ pin_litellm_cost_map_local()
import litellm
# Suppress litellm's startup banner ("Provider List: https://...") and
diff --git a/headroom/providers/openai.py b/headroom/providers/openai.py
index 06c2e3802..006018c60 100644
--- a/headroom/providers/openai.py
+++ b/headroom/providers/openai.py
@@ -17,6 +17,7 @@ from typing import Any, cast
from headroom import paths as _paths
from headroom.log_safety import describe_exception, safe_id
+from headroom.offline import pin_litellm_cost_map_local
from headroom.pricing.litellm_pricing import estimate_cost_from_tokens
from headroom.tokenizers.base import (
TokenCountCache,
@@ -49,6 +50,7 @@ def _get_litellm_module() -> Any | None:
return None
try:
+ pin_litellm_cost_map_local()
import litellm
except ImportError:
return None
diff --git a/headroom/proxy/cost.py b/headroom/proxy/cost.py
index 0a5305a18..96e50ad36 100644
--- a/headroom/proxy/cost.py
+++ b/headroom/proxy/cost.py
@@ -17,6 +17,7 @@ from typing import TYPE_CHECKING, Any, NamedTuple
from headroom.cache_economics import CACHE_ECONOMICS as _CACHE_ECONOMICS
from headroom.log_safety import WarnOnce, describe_exception, safe_id
+from headroom.offline import pin_litellm_cost_map_local
from headroom.proxy.budget_basis_policy import (
BUDGET_BASIS_BLOCK,
BUDGET_BASIS_IGNORE,
@@ -50,6 +51,7 @@ def _get_litellm_module() -> Any | None:
return litellm
try:
+ pin_litellm_cost_map_local()
import litellm as imported_litellm
except ImportError:
return None
diff --git a/headroom/proxy/route_advice.py b/headroom/proxy/route_advice.py
index aceb31c33..d06349c17 100644
--- a/headroom/proxy/route_advice.py
+++ b/headroom/proxy/route_advice.py
@@ -34,6 +34,8 @@ import logging
from dataclasses import dataclass
from typing import Any
+from headroom.offline import pin_litellm_cost_map_local
+
log = logging.getLogger(__name__)
STATE_ATTR = "headroom_route"
@@ -202,6 +204,7 @@ def resolver_for(handler: Any) -> BackendResolver:
def _known_provider(provider: str) -> bool:
"""Is this a provider litellm actually knows about?"""
try:
+ pin_litellm_cost_map_local()
import litellm
names = {getattr(p, "value", None) or str(p) for p in getattr(litellm, "provider_list", [])}
@@ -215,6 +218,7 @@ def _provider_of(model: str) -> str:
if "/" in model:
return model.split("/", 1)[0]
try:
+ pin_litellm_cost_map_local()
import litellm
entry = litellm.model_cost.get(model) or {}
diff --git a/headroom/proxy/savings_tracker.py b/headroom/proxy/savings_tracker.py
index c82e74d3f..9ad81b789 100644
--- a/headroom/proxy/savings_tracker.py
+++ b/headroom/proxy/savings_tracker.py
@@ -25,6 +25,7 @@ from typing import Any
from headroom import paths as _paths
from headroom.log_safety import WarnOnce, describe_exception, safe_id
+from headroom.offline import pin_litellm_cost_map_local
from headroom.proxy import project_name_policy
from headroom.proxy.persistent_metrics import (
MAX_LABEL_LENGTH,
@@ -156,6 +157,7 @@ def _get_litellm_module() -> Any | None:
return litellm
try:
+ pin_litellm_cost_map_local()
import litellm as imported_litellm
except ImportError:
return None
diff --git a/headroom/telemetry/session.py b/headroom/telemetry/session.py
index ccfb9f82e..964ee0e88 100644
--- a/headroom/telemetry/session.py
+++ b/headroom/telemetry/session.py
@@ -118,6 +118,8 @@ from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
+from headroom.offline import pin_litellm_cost_map_local
+
logger = logging.getLogger(__name__)
# Where session events go. The receiver is open source: deploy/beacon/worker.js.
@@ -532,6 +534,7 @@ def _public_model(model: str) -> str | None:
resolved: str | None = None
try:
+ pin_litellm_cost_map_local()
import litellm
registry = litellm.model_cost
diff --git a/tests/test_offline_egress_chokepoint.py b/tests/test_offline_egress_chokepoint.py
index a140ca7e2..06ec2cb43 100644
--- a/tests/test_offline_egress_chokepoint.py
+++ b/tests/test_offline_egress_chokepoint.py
@@ -191,6 +191,17 @@ class TestSocketTrap:
@pytest.fixture
def offline(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("HEADROOM_OFFLINE", "1")
+ # An import of litellm or CrewAI under the switch writes its pin variable
+ # directly, out of monkeypatch's sight. Record each variable (setenv saves
+ # the original, delenv then clears it) so teardown puts it back...
+ import headroom.offline
+
+ for pin in (headroom.offline._litellm_cost_map_pin, headroom.offline._crewai_telemetry_pin):
+ monkeypatch.setenv(pin.name, "")
+ monkeypatch.delenv(pin.name)
+ # ...and the pin's memory of the value it replaced, which would
+ # otherwise outlive the variable it describes.
+ monkeypatch.setattr(pin, "before", None)
# ``"example.com" in url`` is true of ``https://example.com.attacker.test``
@@ -415,6 +426,374 @@ class TestLangfuseExporterOffline:
assert configure_langfuse_tracing(LangfuseTracingConfig(enabled=False)) is not None
+# ────────────── path 2c: litellm's import-time pricing fetch ────────────────
+
+# Runs in a fresh interpreter: the fetch happens once, on the first
+# ``import litellm``, and inside this pytest process an earlier test has
+# usually imported it already, which is how the leak hid behind test order.
+_LITELLM_IMPORT_PROBE = """
+import json, os, socket, sys
+
+attempts = []
+
+def _trap(address):
+ host = address[0] if isinstance(address, tuple) else address
+ if host not in ("127.0.0.1", "::1", "localhost"):
+ attempts.append(str(host))
+ raise OSError("outbound connection attempted while offline")
+
+_create_connection = socket.create_connection
+_connect = socket.socket.connect
+socket.create_connection = lambda address, *a, **k: (
+ _trap(address) or _create_connection(address, *a, **k)
+)
+socket.socket.connect = lambda self, address: _trap(address) or _connect(self, address)
+
+import headroom
+
+if sys.argv[1] == "late":
+ os.environ["HEADROOM_OFFLINE"] = "1"
+import importlib
+
+importlib.import_module(sys.argv[2])
+if "crewai" in sys.modules:
+ # CrewAI tracks the install from a background thread; give it the chance
+ # to dial before reporting, or a leak could race past the report.
+ import threading, time
+
+ deadline = time.monotonic() + 3
+ for thread in threading.enumerate():
+ if thread is not threading.main_thread():
+ thread.join(max(0.0, deadline - time.monotonic()))
+print(json.dumps({
+ "attempts": attempts,
+ "litellm": "litellm" in sys.modules,
+ "crewai": "crewai" in sys.modules,
+}))
+"""
+
+# Variables that turn a library's import-time phone-home off. The probe clears
+# them all, so each run starts from the libraries' own defaults.
+_PHONE_HOME_OPT_OUTS = (
+ "LITELLM_LOCAL_MODEL_COST_MAP",
+ "CREWAI_DISABLE_TELEMETRY",
+ "CREWAI_DISABLE_TRACKING",
+ "OTEL_SDK_DISABLED",
+)
+
+
+def _run_import_probe(
+ when: str,
+ module: str,
+ env_overrides: dict[str, str] | None = None,
+ pythonpath: Path | None = None,
+) -> dict[str, object]:
+ """Import ``module`` in a fresh interpreter with every outbound connect
+ trapped. ``when`` is ``early`` (offline from the start), ``late`` (offline
+ switched on after ``import headroom``) or ``online``."""
+ import json
+ import subprocess
+ import sys
+
+ env = {k: v for k, v in os.environ.items() if k not in _PHONE_HOME_OPT_OUTS}
+ env.pop("HEADROOM_OFFLINE", None)
+ if when == "early":
+ env["HEADROOM_OFFLINE"] = "1"
+ env.update(env_overrides or {})
+ if pythonpath is not None:
+ env["PYTHONPATH"] = os.pathsep.join(
+ p for p in (str(pythonpath), env.get("PYTHONPATH", "")) if p
+ )
+ result = subprocess.run(
+ [sys.executable, "-c", _LITELLM_IMPORT_PROBE, when, module],
+ capture_output=True,
+ text=True,
+ cwd=REPO_ROOT,
+ env=env,
+ timeout=120,
+ check=False,
+ )
+ assert result.returncode == 0, result.stderr[-2000:]
+ return json.loads(result.stdout.strip().splitlines()[-1])
+
+
+def _is_pin_call(statement: ast.stmt, pin: str) -> bool:
+ if not (isinstance(statement, ast.Expr) and isinstance(statement.value, ast.Call)):
+ return False
+ func = statement.value.func
+ name = func.id if isinstance(func, ast.Name) else getattr(func, "attr", None)
+ return name == pin
+
+
+def _pinned_before(node: ast.AST, parents: dict[ast.AST, ast.AST], pin: str) -> bool:
+ """True when a ``pin()`` call precedes ``node`` in its own block or an
+ enclosing block of the same function/module — i.e. it always runs first."""
+ child = node
+ while child in parents:
+ parent = parents[child]
+ for field in ("body", "orelse", "finalbody", "handlers"):
+ block = getattr(parent, field, None)
+ if isinstance(block, list) and child in block:
+ if any(_is_pin_call(s, pin) for s in block[: block.index(child)]):
+ return True
+ if isinstance(parent, (ast.FunctionDef, ast.AsyncFunctionDef, ast.Lambda)):
+ return False
+ child = parent
+ return False
+
+
+def _unpinned_imports(package: str, pin: str) -> list[str]:
+ """Every import of ``package`` (or a submodule) in ``headroom/`` that no
+ ``pin()`` call precedes in the same scope. A pin in another scope does not
+ count: it ran at a different time from the import."""
+ unpinned: list[str] = []
+ for path in sorted(PACKAGE_ROOT.rglob("*.py")):
+ tree = ast.parse(path.read_text(encoding="utf-8"))
+ parents = {
+ child: parent for parent in ast.walk(tree) for child in ast.iter_child_nodes(parent)
+ }
+ for node in ast.walk(tree):
+ if isinstance(node, ast.Import):
+ names = [alias.name for alias in node.names]
+ elif isinstance(node, ast.ImportFrom) and node.level == 0 and node.module:
+ names = [node.module]
+ else:
+ continue
+ if not any(n == package or n.startswith(f"{package}.") for n in names):
+ continue
+ if not _pinned_before(node, parents, pin):
+ unpinned.append(f"{path.relative_to(REPO_ROOT)}:{node.lineno}")
+ return unpinned
+
+
+class TestLiteLLMCostMapOffline:
+ """``import litellm`` refreshes its pricing table from
+ raw.githubusercontent.com unless ``LITELLM_LOCAL_MODEL_COST_MAP`` is set,
+ and every litellm-backed module (the pricing helpers, the LiteLLM provider
+ and backend, both Headroom Cloud integrations) triggers it on first import.
+ """
+
+ @staticmethod
+ def _probe(when: str, module: str, cost_map_env: str | None = None) -> dict[str, object]:
+ pytest.importorskip("litellm")
+ overrides = {} if cost_map_env is None else {"LITELLM_LOCAL_MODEL_COST_MAP": cost_map_env}
+ return _run_import_probe(when, module, overrides)
+
+ _LITELLM_MODULES = [
+ "headroom.providers.litellm",
+ "headroom.backends.litellm",
+ "headroom.pricing.litellm_pricing",
+ "headroom.integrations.litellm_callback",
+ ]
+
+ @pytest.mark.parametrize("module", _LITELLM_MODULES)
+ def test_import_with_offline_set_at_start_opens_no_socket(self, module: str) -> None:
+ outcome = self._probe("early", module)
+ assert outcome["litellm"] is True
+ assert outcome["attempts"] == []
+
+ @pytest.mark.parametrize("module", _LITELLM_MODULES)
+ def test_import_with_offline_set_after_package_load_opens_no_socket(self, module: str) -> None:
+ """The switch is read when it is checked, not when the package loads:
+ ``create_app(ProxyConfig(offline=True))`` turns it on after
+ ``import headroom``. So a pin in ``headroom/__init__.py`` is not enough
+ on its own; each litellm import pins again first."""
+ outcome = self._probe("late", module)
+ assert outcome["litellm"] is True
+ assert outcome["attempts"] == []
+
+ @pytest.mark.parametrize("operator_value", ["1", "False", ""])
+ def test_operator_value_cannot_reopen_the_fetch(self, operator_value: str) -> None:
+ """litellm accepts only ``true``, so an operator ``1`` (the form every
+ Headroom switch accepts), ``False`` or empty value would make it fetch.
+ The air-gap switch wins over it."""
+ outcome = self._probe("early", "headroom.providers.litellm", operator_value)
+ assert outcome["litellm"] is True
+ assert outcome["attempts"] == []
+
+ def test_every_litellm_import_is_pinned_first(self) -> None:
+ """Each ``import litellm`` / ``from litellm…`` in ``headroom/`` must
+ follow a ``pin_litellm_cost_map_local()`` call in the same function (or
+ at module level for a module-level import). A pin in another scope does
+ not count: it ran at a different time from the import."""
+ unpinned = _unpinned_imports("litellm", "pin_litellm_cost_map_local")
+ assert unpinned == [], (
+ "litellm imported without pin_litellm_cost_map_local() first in the same "
+ f"scope; under HEADROOM_OFFLINE it fetches its pricing table: {unpinned}"
+ )
+
+ @staticmethod
+ def _fresh_pin_state(monkeypatch: pytest.MonkeyPatch, value: str | None) -> None:
+ import headroom.offline
+
+ monkeypatch.setattr(headroom.offline._litellm_cost_map_pin, "before", None)
+ # setenv first so teardown restores the real value even after the
+ # pin writes the variable behind monkeypatch's back.
+ monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "")
+ if value is None:
+ monkeypatch.delenv("LITELLM_LOCAL_MODEL_COST_MAP")
+ else:
+ monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", value)
+
+ def test_online_leaves_litellm_alone(self, monkeypatch: pytest.MonkeyPatch) -> None:
+ from headroom.offline import pin_litellm_cost_map_local
+
+ self._fresh_pin_state(monkeypatch, None)
+ monkeypatch.delenv("HEADROOM_OFFLINE", raising=False)
+ pin_litellm_cost_map_local()
+ assert "LITELLM_LOCAL_MODEL_COST_MAP" not in os.environ
+
+ @pytest.mark.parametrize("before", [None, "False"])
+ def test_leaving_offline_puts_back_what_the_pin_replaced(
+ self, monkeypatch: pytest.MonkeyPatch, before: str | None
+ ) -> None:
+ """Offline turned off before the first litellm import must give the
+ online pricing refresh back, not leave Headroom's pin behind."""
+ from headroom.offline import pin_litellm_cost_map_local
+
+ self._fresh_pin_state(monkeypatch, before)
+ monkeypatch.setenv("HEADROOM_OFFLINE", "1")
+ pin_litellm_cost_map_local()
+ pin_litellm_cost_map_local() # a second pin must not record its own value
+ assert os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] == "True"
+
+ monkeypatch.delenv("HEADROOM_OFFLINE")
+ pin_litellm_cost_map_local()
+ assert os.environ.get("LITELLM_LOCAL_MODEL_COST_MAP") == before
+
+ def test_a_value_changed_after_the_pin_is_not_overwritten(
+ self, monkeypatch: pytest.MonkeyPatch
+ ) -> None:
+ from headroom.offline import pin_litellm_cost_map_local
+
+ self._fresh_pin_state(monkeypatch, None)
+ monkeypatch.setenv("HEADROOM_OFFLINE", "1")
+ pin_litellm_cost_map_local()
+ os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] = "false" # the operator's own change
+
+ monkeypatch.delenv("HEADROOM_OFFLINE")
+ pin_litellm_cost_map_local()
+ assert os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] == "false"
+
+
+# ─────────────── path 2d: CrewAI's import-time install tracking ─────────────
+
+# Stands in for CrewAI where it is not installed (no extra or CI job installs
+# it): the same import-time behaviour as crewai 1.6.1's ``_track_install`` —
+# a background thread dialling api.scarf.sh unless one of its opt-out
+# variables is ``true`` — plus the one name Headroom imports from it.
+_CREWAI_STUB = {
+ "crewai/__init__.py": (
+ "import os, socket, threading\n"
+ "\n"
+ "def _track_install():\n"
+ " names = ('OTEL_SDK_DISABLED', 'CREWAI_DISABLE_TELEMETRY', 'CREWAI_DISABLE_TRACKING')\n"
+ " if any(os.getenv(n, 'false').lower() == 'true' for n in names):\n"
+ " return\n"
+ " try:\n"
+ " socket.create_connection(('api.scarf.sh', 443), timeout=2).close()\n"
+ " except OSError:\n"
+ " pass\n"
+ "\n"
+ "threading.Thread(target=_track_install, daemon=True).start()\n"
+ ),
+ "crewai/tools/__init__.py": "",
+ "crewai/tools/base_tool.py": "class BaseTool:\n pass\n",
+}
+
+# The modules a Headroom user imports that pull CrewAI in: the integration
+# itself, and any sibling under ``headroom.integrations``, whose package
+# ``__init__`` imports the CrewAI integration whenever CrewAI is installed.
+_CREWAI_REACHING_MODULES = [
+ "headroom.integrations.crewai",
+ "headroom.integrations.litellm_callback",
+]
+
+
+class TestCrewAITelemetryOffline:
+ """Importing ``headroom.integrations.litellm_callback`` under the air-gap
+ switch dialled api.scarf.sh whenever CrewAI was installed, because the
+ package ``__init__`` imports the CrewAI integration and ``import crewai``
+ tracks the install from a background thread (found in review of #4242
+ with crewai 1.6.1 and litellm 1.101.0)."""
+
+ @pytest.fixture
+ def crewai_stub(self, tmp_path: Path) -> Path:
+ for relative, source in _CREWAI_STUB.items():
+ path = tmp_path / relative
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text(source, encoding="utf-8")
+ return tmp_path
+
+ def test_the_stub_phones_home_when_online(self, crewai_stub: Path) -> None:
+ """Without this, a stub that never dials would pass every test below."""
+ outcome = _run_import_probe(
+ "online", "headroom.integrations.crewai", pythonpath=crewai_stub
+ )
+ assert outcome["crewai"] is True
+ # litellm's own pricing refresh also runs online; only the tracking matters here.
+ assert "api.scarf.sh" in outcome["attempts"]
+
+ @pytest.mark.parametrize("when", ["early", "late"])
+ @pytest.mark.parametrize("module", _CREWAI_REACHING_MODULES)
+ def test_import_offline_opens_no_socket(
+ self, crewai_stub: Path, module: str, when: str
+ ) -> None:
+ if module.endswith("litellm_callback"):
+ pytest.importorskip("litellm")
+ outcome = _run_import_probe(when, module, pythonpath=crewai_stub)
+ assert outcome["crewai"] is True
+ assert outcome["attempts"] == []
+
+ @pytest.mark.parametrize("operator_value", ["1", "False", ""])
+ def test_operator_value_cannot_reopen_the_tracking(
+ self, crewai_stub: Path, operator_value: str
+ ) -> None:
+ outcome = _run_import_probe(
+ "early",
+ "headroom.integrations.crewai",
+ {"CREWAI_DISABLE_TELEMETRY": operator_value},
+ pythonpath=crewai_stub,
+ )
+ assert outcome["attempts"] == []
+
+ @pytest.mark.parametrize("module", _CREWAI_REACHING_MODULES)
+ def test_real_crewai_import_offline_opens_no_socket(self, module: str) -> None:
+ """The same against the installed CrewAI, telemetry at its default."""
+ import importlib.util
+
+ if importlib.util.find_spec("crewai") is None:
+ pytest.skip("crewai is not installed")
+ if module.endswith("litellm_callback"):
+ pytest.importorskip("litellm")
+ outcome = _run_import_probe("early", module)
+ assert outcome["crewai"] is True
+ assert outcome["attempts"] == []
+
+ def test_every_crewai_import_is_pinned_first(self) -> None:
+ unpinned = _unpinned_imports("crewai", "pin_crewai_telemetry_off")
+ assert unpinned == [], (
+ "crewai imported without pin_crewai_telemetry_off() first in the same "
+ f"scope; under HEADROOM_OFFLINE it tracks the install over the network: {unpinned}"
+ )
+
+ def test_leaving_offline_puts_back_what_the_pin_replaced(
+ self, monkeypatch: pytest.MonkeyPatch
+ ) -> None:
+ import headroom.offline
+ from headroom.offline import pin_crewai_telemetry_off
+
+ monkeypatch.setattr(headroom.offline._crewai_telemetry_pin, "before", None)
+ monkeypatch.setenv("CREWAI_DISABLE_TELEMETRY", "False")
+ monkeypatch.setenv("HEADROOM_OFFLINE", "1")
+ pin_crewai_telemetry_off()
+ assert os.environ["CREWAI_DISABLE_TELEMETRY"] == "true"
+
+ monkeypatch.delenv("HEADROOM_OFFLINE")
+ pin_crewai_telemetry_off()
+ assert os.environ["CREWAI_DISABLE_TELEMETRY"] == "False"
+
+
# ────────── path 4: the Python half of the HuggingFace download ─────────────