Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
feat(observability): add gen_ai.request.model to the compression span (#1667)
Krishna Chaitanya7f7af667ed3d
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex b6a7591f0..82fe80f67 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0* **proxy:** add provider-only HTTP proxy routing via `--http-proxy` and `HEADROOM_HTTP_PROXY`. Upstream LLM provider calls can now use an HTTP proxy without setting process-wide `HTTP_PROXY`/`HTTPS_PROXY` variables that are inherited by tool executions; proxied provider clients use HTTP/1.1 so HTTPS provider APIs can tunnel through CONNECT.* **proxy:** add output shaping for OpenAI Responses traffic on `/v1/responses` HTTP requests and Codex WebSocket `response.create` frames, with stable output-savings holdout keys and counted WS token strata for the experiment.+* **observability:** the `headroom.compression.pipeline` span now also carries the OpenTelemetry GenAI semantic-convention attribute `gen_ai.request.model` alongside the existing `headroom.*` attributes, so Headroom's traces group and filter by the standard `gen_ai.*` schema in any OTel-native backend (Grafana, Datadog, etc.). Purely additive; no existing attribute changed. `gen_ai.operation.name`, `gen_ai.provider.name`, and `gen_ai.usage.*` are deliberately deferred (they need per-caller operation threading, reliable upstream-provider resolution, and response-path usage respectively).* **wrap:** `headroom wrap claude --1m` preserves the 1M context window. Behind a custom `ANTHROPIC_BASE_URL` (the proxy) Claude Code drops the `context-1m` beta header and caps the window at 200k for entitled subscription users; the opt-in flag sets `ANTHROPIC_MODEL=<opus>[1m]` on the launched process so the 1M window activates through Headroom. A model already selected via `ANTHROPIC_MODEL` is preserved (only the `[1m]` suffix is appended) ([#1158](https://github.com/chopratejas/headroom/issues/1158)).* **learn:** weight loops in `headroom learn`. A new loop detector (`headroom/learn/loops.py`) recognizes repeated tool-call patterns — including RTK re-fetch loops, where RTK's output truncation makes the agent re-run larger-limit variants of a *successful* command — collapses output-limit variants to one signature, measures the wasted tokens, surfaces loops as a highest-priority digest section, and weights loop guardrails above one-off rules by their measured waste. Previously loops had no special weight and a no-failure re-fetch loop was skipped entirely. Adds an RTK-loop eval (`benchmarks/rtk_loop_learn_eval.py`) that reproduces a loop, runs it through Learn, and asserts the generated guardrail ranks first and prevents re-triggering.* **learn:** write per-project learnings to the personal, gitignored `CLAUDE.local.md` by default instead of the team-shared `CLAUDE.md`, matching Claude Code's memory convention so machine-specific paths and tool-discovery byproducts no longer pollute the shared file. Adds a `--target` flag to override the destination (e.g. `--target CLAUDE.md` to opt back into the shared file, or any custom path), and auto-migrates a stale learned-patterns block out of an existing `CLAUDE.md` into `CLAUDE.local.md` with a warning ([#1072](https://github.com/chopratejas/headroom/issues/1072)).diff --git a/headroom/transforms/pipeline.py b/headroom/transforms/pipeline.pyindex fec4701a2..c7061dee1 100644--- a/headroom/transforms/pipeline.py+++ b/headroom/transforms/pipeline.py@@ -42,6 +42,28 @@ MAX_WASTE_SIGNAL_DETECTION_TOKENS = 100_000# runs when compression saved more than this many tokens._MIN_TOKENS_SAVED_FOR_WASTE_SIGNALS = 100+# OTel GenAI semantic conventions (open-telemetry/semantic-conventions-genai).+# The compression-pipeline span carries this gen_ai.* attribute alongside the+# proprietary headroom.* ones, so Headroom's telemetry groups/filters by the+# standard schema in any OTel-native backend (Grafana, Datadog, etc.). It is a+# string literal rather than an opentelemetry.semconv constant because the gen_ai+# attributes are stability=development (no stable constants are published).+#+# v1 emits only gen_ai.request.model — the one attribute this span can set+# correctly and unconditionally (the model is always known here). The rest are+# deliberately deferred to v2 because this span cannot set them correctly:+# - gen_ai.operation.name: apply() is shared by many callers (chat, /v1/compress,+# batch, Gemini countTokens), so no single value is right — it must be threaded+# from each caller, not hardcoded.+# - gen_ai.provider.name: Headroom's provider label can't distinguish Bedrock /+# Gemini from Anthropic / OpenAI at this layer (Bedrock routes via the+# Anthropic provider).+# - gen_ai.usage.*: provider-authoritative usage lives on the response path, not+# this pre-flight compression span (the compressed-input estimate stays under+# headroom.tokens.after).+GEN_AI_REQUEST_MODEL = "gen_ai.request.model"++_N = TypeVar("_N", int, float)@@ -269,12 +291,16 @@ class TransformPipeline:)tracer = get_headroom_tracer()- span_attributes = {+ span_attributes: dict[str, Any] = {"headroom.model": model,"headroom.provider": provider_name or "unknown","headroom.message_count": len(messages),"headroom.tokens.before": tokens_before,}+ # OTel GenAI semconv request descriptor — emitted alongside headroom.* so+ # the span is groupable by the standard schema (v1: model only).+ if model:+ span_attributes[GEN_AI_REQUEST_MODEL] = modelpipeline_span_context = (tracer.start_as_current_span("headroom.compression.pipeline",diff --git a/tests/test_observability_tracing.py b/tests/test_observability_tracing.pyindex d10054631..5a072fa34 100644--- a/tests/test_observability_tracing.py+++ b/tests/test_observability_tracing.py@@ -55,6 +55,76 @@ def test_transform_pipeline_emits_trace_spans() -> None:reset_headroom_tracing()+def test_pipeline_span_emits_gen_ai_request_model() -> None:+ """The compression-pipeline span carries the v1 OTel GenAI semconv descriptor+ (gen_ai.request.model) alongside headroom.*, so it groups by the standard+ schema. operation.name / provider.name / usage.* are intentionally v2."""+ exporter = InMemorySpanExporter()+ provider = TracerProvider(resource=Resource.create({"service.name": "headroom-test"}))+ provider.add_span_processor(SimpleSpanProcessor(exporter))+ set_headroom_tracer(HeadroomTracer(tracer_provider=provider))++ try:+ pipeline = TransformPipeline(transforms=[])+ pipeline.apply(+ [{"role": "user", "content": "hello world"}],+ model="claude-3-5-sonnet-20241022",+ model_limit=1024,+ )++ span = exporter.get_finished_spans()[0]+ assert span.name == "headroom.compression.pipeline"+ assert span.attributes["gen_ai.request.model"] == "claude-3-5-sonnet-20241022"+ # v1 deliberately emits ONLY request.model — no operation/provider/usage+ # (each is inaccurate at this span; see pipeline.py).+ assert "gen_ai.operation.name" not in span.attributes+ assert "gen_ai.provider.name" not in span.attributes+ assert "gen_ai.usage.input_tokens" not in span.attributes+ finally:+ reset_headroom_tracing()+++def test_pipeline_span_omits_request_model_when_model_missing() -> None:+ """gen_ai.request.model is omitted (not set to an empty string) when no model+ is provided — never emit a blank standard attribute."""+ exporter = InMemorySpanExporter()+ provider = TracerProvider(resource=Resource.create({"service.name": "headroom-test"}))+ provider.add_span_processor(SimpleSpanProcessor(exporter))+ set_headroom_tracer(HeadroomTracer(tracer_provider=provider))++ try:+ pipeline = TransformPipeline(transforms=[])+ pipeline.apply([{"role": "user", "content": "hi"}], model="", model_limit=1024)+ span = exporter.get_finished_spans()[0]+ assert span.name == "headroom.compression.pipeline"+ assert "gen_ai.request.model" not in span.attributes+ finally:+ reset_headroom_tracing()+++def test_pipeline_runs_with_metrics_disabled() -> None:+ """record_metrics=False takes the nullcontext (no-span) path: the run still+ returns a valid result and emits no spans (guards that building span_attributes+ with the gen_ai key never breaks the non-recording path)."""+ exporter = InMemorySpanExporter()+ provider = TracerProvider(resource=Resource.create({"service.name": "headroom-test"}))+ provider.add_span_processor(SimpleSpanProcessor(exporter))+ set_headroom_tracer(HeadroomTracer(tracer_provider=provider))++ try:+ pipeline = TransformPipeline(transforms=[])+ result = pipeline.apply(+ [{"role": "user", "content": "hi"}],+ model="gpt-4o",+ model_limit=1024,+ record_metrics=False,+ )+ assert result.messages # pipeline produced output+ assert exporter.get_finished_spans() == ()+ finally:+ reset_headroom_tracing()++def test_langfuse_tracing_status_defaults_to_unconfigured() -> None:reset_headroom_tracing()status = get_langfuse_tracing_status()