Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

docs: drop effort-routing text that #3377 removed (#4449)

danieldutu3a38d0b93b22
diff --git a/README.md b/README.md
index adacc2f21..dd44d0dfc 100644
--- a/README.md
+++ b/README.md
@@ -183,13 +183,10 @@ back at you, and deep reasoning spent on routine steps like reading a file.
Headroom trims it from the proxy, with no change to your code:
- **Verbosity steering** appends a short "be terse, don't restate context" note to the *end* of the system prompt, so your prompt cache still hits.
-- **Effort routing** dials thinking effort down when a turn is only the model resuming after a tool result — a file read, a passing test. New questions and errors keep full effort.
-Both apply to Anthropic `/v1/messages` and to OpenAI-compatible
-`/v1/chat/completions` and `/v1/responses`. Effort routing uses
-`reasoning_effort` on OpenAI and `thinking.budget_tokens` / `output_config.effort`
-on Anthropic, with the same clamp-only invariant and the same `output_shaper:*`
-labels on both paths.
+It applies to Anthropic `/v1/messages` and to OpenAI-compatible
+`/v1/chat/completions` and `/v1/responses`, with the same `output_shaper:*`
+labels on all paths.
```bash
export HEADROOM_OUTPUT_SHAPER=1 # off by default
diff --git a/headroom/agent_savings.py b/headroom/agent_savings.py
index 000464c1a..3d5211bd5 100644
--- a/headroom/agent_savings.py
+++ b/headroom/agent_savings.py
@@ -173,7 +173,7 @@ _PROFILES: dict[str, AgentSavingsProfile] = {
# Coding posture: defer non-core tool schemas, dedupe re-reads, extend
# lossy coverage when lossless found nothing, and NEVER lossy-compress a
# file read (the agent patches exact bytes). CCR stays ON (unset) so any
- # lossy loss is recoverable. Effort router off; low block floor so modest
+ # lossy loss is recoverable. Low block floor so modest
# deltas are eligible.
tool_search=True,
cross_turn_dedup=True,
diff --git a/headroom/proxy/handlers/anthropic.py b/headroom/proxy/handlers/anthropic.py
index c30010b33..26937b867 100644
--- a/headroom/proxy/handlers/anthropic.py
+++ b/headroom/proxy/handlers/anthropic.py
@@ -121,7 +121,7 @@ def _thinking_tokens_for(payload: object) -> ThinkingTokens:
Anthropic reports no thinking count in ``usage``, so the count is derived
from the response's own thinking blocks and marked inferred. Without the
- split, reasoning-effort routing and verbosity steering move the same
+ split, thinking-budget changes and verbosity steering move the same
counter and neither can be attributed.
Never raises: token accounting is bookkeeping, and a failure here must
@@ -3648,8 +3648,7 @@ class AnthropicHandlerMixin:
logger.debug("consistency token re-count skipped", exc_info=True)
# Output shaping (opt-in via HEADROOM_OUTPUT_SHAPER): verbosity
- # steering appended to the system-prompt tail + effort routing on
- # mechanical tool_result continuations. Runs after every other
+ # steering appended to the system-prompt tail. Runs after every other
# body mutation so the turn classifier sees the final messages,
# and respects the same bypass header as compression.
if not _bypass:
diff --git a/headroom/proxy/handlers/openai.py b/headroom/proxy/handlers/openai.py
index 741326e46..00499e80c 100644
--- a/headroom/proxy/handlers/openai.py
+++ b/headroom/proxy/handlers/openai.py
@@ -861,7 +861,7 @@ def _shape_openai_responses_payload(
The Responses counterpart of the Anthropic handler's shaping block:
conversation-stable holdout assignment, stratum labelling for the
output-savings ledger, then verbosity steering on the ``instructions``
- tail and ``reasoning.effort`` routing on mechanical continuations.
+ tail.
Returns ``(labels, mutated)``. ``labels`` are carried on the transforms
channel for both arms (the control arm's stratum label is what feeds the
diff --git a/headroom/proxy/outcome.py b/headroom/proxy/outcome.py
index 837544683..036c6cc0d 100644
--- a/headroom/proxy/outcome.py
+++ b/headroom/proxy/outcome.py
@@ -124,7 +124,7 @@ class RequestOutcome:
# ── Output composition ────────────────────────────────────────────
# ``output_tokens`` pools two quantities that different levers move in
- # opposite directions: reasoning-effort routing cuts thinking, verbosity
+ # opposite directions: thinking-budget levers cut thinking, verbosity
# steering cuts visible text. Summed, neither can be attributed — a turn
# whose thinking dropped 4,000 tokens while its prose grew 200 is
# indistinguishable from one where nothing happened.
@@ -250,7 +250,7 @@ class RequestOutcome:
The quantity verbosity steering actually targets. Callers must handle
``None`` rather than defaulting it to ``output_tokens``: on a provider
that reports no thinking count, treating the whole output as visible
- would credit steering with reductions that reasoning-effort routing
+ would credit steering with reductions that a lower thinking budget
produced.
Clamped at zero because an inferred count comes from Headroom's
diff --git a/headroom/proxy/output_shaper.py b/headroom/proxy/output_shaper.py
index 7987f7c0e..4dabb5187 100644
--- a/headroom/proxy/output_shaper.py
+++ b/headroom/proxy/output_shaper.py
@@ -9,18 +9,7 @@ output tokens, so every lever here works by reshaping the request:
provider prefix cache is preserved). Five levels, from "no ceremony" to
full caveman.
-2. Effort routing — agentic loops are mostly mechanical continuations (the
- last message is a clean tool_result: a file read, a passing test). Thinking
- bills as output tokens, and harnesses like Claude Code pin
- ``output_config.effort`` at ``xhigh`` for every turn. On turns classified
- as mechanical we lower an explicitly-present effort; on errors or new user
- asks we leave it alone. For legacy models still sending
- ``thinking.budget_tokens`` we clamp the budget to the API floor instead.
-
Safety rules (each prevents a concrete failure mode):
-- Never INJECT ``output_config.effort`` where the client didn't send it —
- models without effort support 400 on it. Lowering an existing value is
- always valid.
- Never toggle ``thinking.type`` — disabling thinking while history carries
thinking blocks 400s on some models, and the toggle busts the messages
cache tier.
@@ -100,8 +89,7 @@ class OutputShaperSettings:
verbosity_level: int = DEFAULT_VERBOSITY_LEVEL
# False in ``mode="cache"``. Steering is the one lever that writes into the
# provider prefix-cache key (it appends to the system-prompt tail, and on a
- # body with no system field it creates one); effort routing and the
- # thinking budget sit outside the key and stay on. See
+ # body with no system field it creates one). See
# :func:`steering_allowed_for`.
steering_enabled: bool = True
@@ -193,9 +181,7 @@ def steering_allowed_for(config: Any) -> bool:
prefix-cache key byte-stable (see ``ProxyConfig.mode``). Verbosity steering
writes into that key, so a block that CHANGES there trades a large, certain
cache cost for a small, uncertain output saving — the wrong side of a
- roughly 60x margin on a long context. Effort routing and the thinking
- budget are unaffected: they ride request parameters outside the cache key,
- so they keep saving in cache mode.
+ roughly 60x margin on a long context.
This is the gate for the sources that can change mid-conversation. It is
NOT the last word: :func:`resolve_verbosity_level` lets an explicitly
diff --git a/headroom/proxy/server.py b/headroom/proxy/server.py
index ca5d6d22f..310c19b42 100644
--- a/headroom/proxy/server.py
+++ b/headroom/proxy/server.py
@@ -6520,7 +6520,7 @@ def run_server(
sys.exit(1)
# Seed the request-time coding-profile toggles (tool-search, dedupe, read
- # protection, lossless→lossy, effort-router, block-char floor) into the
+ # protection, lossless→lossy, block-char floor) into the
# process env before serving, so downstream per-request readers pick them up.
# Done here (not in the CLI command) so unit tests that mock run_server never
# mutate os.environ. setdefault → explicit env still wins. MODE / profile are
diff --git a/headroom/proxy/thinking_tokens.py b/headroom/proxy/thinking_tokens.py
index ff1f5c531..3f04e5df2 100644
--- a/headroom/proxy/thinking_tokens.py
+++ b/headroom/proxy/thinking_tokens.py
@@ -1,7 +1,7 @@
"""Separate thinking tokens from visible output tokens.
``output_tokens`` is a single number today, and it pools two quantities that
-different levers move in opposite directions. Reasoning-effort routing cuts
+different levers move in opposite directions. Thinking-budget levers cut
*thinking*; verbosity steering cuts *visible text*. Summed into one counter,
neither can be attributed: a request whose thinking dropped 4,000 tokens while
its prose grew 200 looks identical to one where nothing happened.
diff --git a/scripts/eval_output_shaper.py b/scripts/eval_output_shaper.py
index 3369de393..a6ff0c396 100644
--- a/scripts/eval_output_shaper.py
+++ b/scripts/eval_output_shaper.py
@@ -8,11 +8,6 @@ includes thinking tokens.
Scenario A (verbosity steering): a complex code-review ask. Baseline vs
verbosity levels 2 and 3.
-Scenario B (effort routing): an agentic transcript whose last message is a
-clean tool_result (mechanical continuation) with `output_config.effort` set
-to "xhigh" the way Claude Code pins it. The shaper lowers effort to "low"
-for this turn only.
-
Usage:
source .venv/bin/activate && python scripts/eval_output_shaper.py
Requires ANTHROPIC_API_KEY in the environment or in ./.env.
@@ -106,59 +101,6 @@ def scenario_a_body() -> dict[str, Any]:
}
-def scenario_b_body() -> dict[str, Any]:
- """Agentic mechanical continuation — exercises effort routing."""
- return {
- "model": MODEL,
- "max_tokens": 8000,
- "thinking": {"type": "adaptive"},
- "output_config": {"effort": "xhigh"},
- "system": (
- "You are a coding agent. Use the Read tool to inspect files, then "
- "report findings concisely."
- ),
- "tools": [
- {
- "name": "Read",
- "description": "Read a file from the repository.",
- "input_schema": {
- "type": "object",
- "properties": {"path": {"type": "string"}},
- "required": ["path"],
- },
- }
- ],
- "messages": [
- {
- "role": "user",
- "content": "Check whether cache.py has thread-safety issues.",
- },
- {
- "role": "assistant",
- "content": [
- {"type": "text", "text": "Reading cache.py first."},
- {
- "type": "tool_use",
- "id": "toolu_eval_01",
- "name": "Read",
- "input": {"path": "cache.py"},
- },
- ],
- },
- {
- "role": "user",
- "content": [
- {
- "type": "tool_result",
- "tool_use_id": "toolu_eval_01",
- "content": BUGGY_CODE,
- }
- ],
- },
- ],
- }
-
-
def run(client: anthropic.Anthropic, body: dict[str, Any]) -> dict[str, int]:
# The installed SDK may predate output_config as a typed kwarg; the API
# accepts it either way, so pass it through extra_body.
@@ -193,14 +135,6 @@ def main() -> int:
shape_request(body, OutputShaperSettings(enabled=True, verbosity_level=level))
conditions.append(("A:verbosity", f"shaped L{level}", body))
- if which in ("B", "ALL"):
- # Scenario B: baseline (effort=xhigh) vs shaped (effort routed to low).
- conditions.append(("B:effort-routing", "baseline xhigh", scenario_b_body()))
- body = scenario_b_body()
- result = shape_request(body, OutputShaperSettings(enabled=True, verbosity_level=0))
- assert body["output_config"]["effort"] == "low", result.labels
- conditions.append(("B:effort-routing", "shaped low", body))
-
print(f"model={MODEL} trials={TRIALS}\n")
print(f"{'scenario':<18} {'condition':<16} {'trial':<6} {'in_tok':>7} {'out_tok':>8}")
print("-" * 60)