Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

Fix Bedrock backend: use inference profiles for Claude 4+ models

chopratejas8128c9239dad
diff --git a/README.md b/README.md
index 9829b176f..b64425b24 100644
--- a/README.md
+++ b/README.md
@@ -216,10 +216,38 @@ OPENAI_BASE_URL=http://localhost:8787/v1 cursor
**Using AWS Bedrock, Google Vertex, or Azure?** Route through Headroom:
```bash
-# AWS Bedrock (uses your AWS credentials)
-headroom proxy --backend bedrock --region us-west-2
-ANTHROPIC_BASE_URL=http://localhost:8787 claude
+# AWS Bedrock - Terminal 1: Start proxy
+export AWS_ACCESS_KEY_ID="AKIA..."
+export AWS_SECRET_ACCESS_KEY="..."
+export AWS_REGION="us-east-1"
+headroom proxy --backend bedrock --region us-east-1
+
+# AWS Bedrock - Terminal 2: Run Claude Code
+export ANTHROPIC_API_KEY="sk-ant-dummy" # Any value works! Headroom ignores it.
+export ANTHROPIC_BASE_URL="http://localhost:8787"
+# IMPORTANT: Do NOT set CLAUDE_CODE_USE_BEDROCK=1 (Headroom handles Bedrock routing)
+claude
+```
+
+<details>
+<summary><b>VS Code settings.json for Bedrock</b> (click to expand)</summary>
+
+```json
+{
+ "claudeCode.environmentVariables": [
+ { "name": "ANTHROPIC_API_KEY", "value": "sk-ant-dummy" },
+ { "name": "ANTHROPIC_BASE_URL", "value": "http://localhost:8787" },
+ { "name": "AWS_ACCESS_KEY_ID", "value": "AKIA..." },
+ { "name": "AWS_SECRET_ACCESS_KEY", "value": "..." },
+ { "name": "AWS_REGION", "value": "us-east-1" }
+ ]
+}
+```
+**Do NOT include** `CLAUDE_CODE_USE_BEDROCK` - Headroom handles the Bedrock routing.
+</details>
+
+```bash
# Google Vertex AI
headroom proxy --backend vertex_ai --region us-central1
diff --git a/headroom/backends/litellm.py b/headroom/backends/litellm.py
index 3d99f7497..bccb2bfcf 100644
--- a/headroom/backends/litellm.py
+++ b/headroom/backends/litellm.py
@@ -32,23 +32,29 @@ except ImportError:
# Model mapping: Anthropic model IDs -> LiteLLM model strings
+# IMPORTANT: Claude 4+ models require inference profiles (us.anthropic.* or global.anthropic.*)
+# Direct model IDs (anthropic.*) don't support on-demand throughput for newer models.
+# See: https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles.html
BEDROCK_MODEL_MAP = {
- # Claude 4.5
- "claude-opus-4-5-20251101": "bedrock/anthropic.claude-opus-4-5-20251101-v1:0",
- "claude-sonnet-4-5-20250929": "bedrock/anthropic.claude-sonnet-4-5-20250929-v1:0",
- # Claude 4
- "claude-opus-4-20250514": "bedrock/anthropic.claude-opus-4-20250514-v1:0",
- "claude-sonnet-4-20250514": "bedrock/anthropic.claude-sonnet-4-20250514-v1:0",
- # Claude 3.7
- "claude-3-7-sonnet-20250219": "bedrock/anthropic.claude-3-7-sonnet-20250219-v1:0",
- # Claude 3.5
- "claude-3-5-sonnet-20241022": "bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0",
- "claude-3-5-sonnet-20240620": "bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0",
- "claude-3-5-haiku-20241022": "bedrock/anthropic.claude-3-5-haiku-20241022-v1:0",
- # Claude 3
- "claude-3-opus-20240229": "bedrock/anthropic.claude-3-opus-20240229-v1:0",
- "claude-3-sonnet-20240229": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
- "claude-3-haiku-20240307": "bedrock/anthropic.claude-3-haiku-20240307-v1:0",
+ # Claude 4.5 (requires inference profiles)
+ "claude-opus-4-5-20251101": "bedrock/us.anthropic.claude-opus-4-5-20251101-v1:0",
+ "claude-sonnet-4-5-20250929": "bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0",
+ "claude-haiku-4-5-20251001": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0",
+ # Claude 4.1
+ "claude-opus-4-1-20250805": "bedrock/us.anthropic.claude-opus-4-1-20250805-v1:0",
+ # Claude 4 (requires inference profiles)
+ "claude-opus-4-20250514": "bedrock/us.anthropic.claude-opus-4-20250514-v1:0",
+ "claude-sonnet-4-20250514": "bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0",
+ # Claude 3.7 (requires inference profiles)
+ "claude-3-7-sonnet-20250219": "bedrock/us.anthropic.claude-3-7-sonnet-20250219-v1:0",
+ # Claude 3.5 (can use inference profiles for cross-region)
+ "claude-3-5-sonnet-20241022": "bedrock/us.anthropic.claude-3-5-sonnet-20241022-v2:0",
+ "claude-3-5-sonnet-20240620": "bedrock/us.anthropic.claude-3-5-sonnet-20240620-v1:0",
+ "claude-3-5-haiku-20241022": "bedrock/us.anthropic.claude-3-5-haiku-20241022-v1:0",
+ # Claude 3 (can use inference profiles for cross-region)
+ "claude-3-opus-20240229": "bedrock/us.anthropic.claude-3-opus-20240229-v1:0",
+ "claude-3-sonnet-20240229": "bedrock/us.anthropic.claude-3-sonnet-20240229-v1:0",
+ "claude-3-haiku-20240307": "bedrock/us.anthropic.claude-3-haiku-20240307-v1:0",
}
VERTEX_MODEL_MAP = {
diff --git a/headroom/cli/proxy.py b/headroom/cli/proxy.py
index 810d6bc75..e1114df95 100644
--- a/headroom/cli/proxy.py
+++ b/headroom/cli/proxy.py
@@ -205,9 +205,17 @@ Memory:
memory_section += " - NOTE: Memory tools require ANTHROPIC_API_KEY (Claude Code subscription credentials have restrictions).\n"
# Build usage note for Bedrock
- bedrock_note = ""
- if config.backend == "bedrock":
- bedrock_note = "unset CLAUDE_CODE_USE_BEDROCK # Important!"
+ bedrock_section = ""
+ if config.backend != "anthropic":
+ provider = config.backend.replace("litellm-", "").upper()
+ bedrock_section = f"""
+IMPORTANT for {provider} users (Claude Code / VS Code):
+ 1. Set AWS credentials: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION
+ 2. Set a dummy Anthropic key: ANTHROPIC_API_KEY="sk-ant-dummy"
+ (Headroom ignores this - it uses your AWS credentials for {provider})
+ 3. Set base URL: ANTHROPIC_BASE_URL=http://{config.host}:{config.port}
+ 4. Do NOT set: CLAUDE_CODE_USE_BEDROCK=1 (Headroom handles this)
+"""
click.echo(f"""
╔═══════════════════════════════════════════════════════════════════════╗
@@ -223,9 +231,8 @@ Starting proxy server...
Caching: {"ENABLED" if config.cache_enabled else "DISABLED"}
Rate Limit: {"ENABLED" if config.rate_limit_enabled else "DISABLED"}
Memory: {memory_status}
-
+{bedrock_section}
Usage with Claude Code:
- {bedrock_note}
ANTHROPIC_BASE_URL=http://{config.host}:{config.port} claude
Usage with OpenAI-compatible clients: