Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Fix Bedrock backend: use inference profiles for Claude 4+ models
chopratejas8128c9239dad
diff --git a/README.md b/README.mdindex 9829b176f..b64425b24 100644--- a/README.md+++ b/README.md@@ -216,10 +216,38 @@ OPENAI_BASE_URL=http://localhost:8787/v1 cursor**Using AWS Bedrock, Google Vertex, or Azure?** Route through Headroom:```bash-# AWS Bedrock (uses your AWS credentials)-headroom proxy --backend bedrock --region us-west-2-ANTHROPIC_BASE_URL=http://localhost:8787 claude+# AWS Bedrock - Terminal 1: Start proxy+export AWS_ACCESS_KEY_ID="AKIA..."+export AWS_SECRET_ACCESS_KEY="..."+export AWS_REGION="us-east-1"+headroom proxy --backend bedrock --region us-east-1++# AWS Bedrock - Terminal 2: Run Claude Code+export ANTHROPIC_API_KEY="sk-ant-dummy" # Any value works! Headroom ignores it.+export ANTHROPIC_BASE_URL="http://localhost:8787"+# IMPORTANT: Do NOT set CLAUDE_CODE_USE_BEDROCK=1 (Headroom handles Bedrock routing)+claude+```++<details>+<summary><b>VS Code settings.json for Bedrock</b> (click to expand)</summary>++```json+{+ "claudeCode.environmentVariables": [+ { "name": "ANTHROPIC_API_KEY", "value": "sk-ant-dummy" },+ { "name": "ANTHROPIC_BASE_URL", "value": "http://localhost:8787" },+ { "name": "AWS_ACCESS_KEY_ID", "value": "AKIA..." },+ { "name": "AWS_SECRET_ACCESS_KEY", "value": "..." },+ { "name": "AWS_REGION", "value": "us-east-1" }+ ]+}+```+**Do NOT include** `CLAUDE_CODE_USE_BEDROCK` - Headroom handles the Bedrock routing.+</details>++```bash# Google Vertex AIheadroom proxy --backend vertex_ai --region us-central1diff --git a/headroom/backends/litellm.py b/headroom/backends/litellm.pyindex 3d99f7497..bccb2bfcf 100644--- a/headroom/backends/litellm.py+++ b/headroom/backends/litellm.py@@ -32,23 +32,29 @@ except ImportError:# Model mapping: Anthropic model IDs -> LiteLLM model strings+# IMPORTANT: Claude 4+ models require inference profiles (us.anthropic.* or global.anthropic.*)+# Direct model IDs (anthropic.*) don't support on-demand throughput for newer models.+# See: https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles.htmlBEDROCK_MODEL_MAP = {- # Claude 4.5- "claude-opus-4-5-20251101": "bedrock/anthropic.claude-opus-4-5-20251101-v1:0",- "claude-sonnet-4-5-20250929": "bedrock/anthropic.claude-sonnet-4-5-20250929-v1:0",- # Claude 4- "claude-opus-4-20250514": "bedrock/anthropic.claude-opus-4-20250514-v1:0",- "claude-sonnet-4-20250514": "bedrock/anthropic.claude-sonnet-4-20250514-v1:0",- # Claude 3.7- "claude-3-7-sonnet-20250219": "bedrock/anthropic.claude-3-7-sonnet-20250219-v1:0",- # Claude 3.5- "claude-3-5-sonnet-20241022": "bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0",- "claude-3-5-sonnet-20240620": "bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0",- "claude-3-5-haiku-20241022": "bedrock/anthropic.claude-3-5-haiku-20241022-v1:0",- # Claude 3- "claude-3-opus-20240229": "bedrock/anthropic.claude-3-opus-20240229-v1:0",- "claude-3-sonnet-20240229": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",- "claude-3-haiku-20240307": "bedrock/anthropic.claude-3-haiku-20240307-v1:0",+ # Claude 4.5 (requires inference profiles)+ "claude-opus-4-5-20251101": "bedrock/us.anthropic.claude-opus-4-5-20251101-v1:0",+ "claude-sonnet-4-5-20250929": "bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0",+ "claude-haiku-4-5-20251001": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0",+ # Claude 4.1+ "claude-opus-4-1-20250805": "bedrock/us.anthropic.claude-opus-4-1-20250805-v1:0",+ # Claude 4 (requires inference profiles)+ "claude-opus-4-20250514": "bedrock/us.anthropic.claude-opus-4-20250514-v1:0",+ "claude-sonnet-4-20250514": "bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0",+ # Claude 3.7 (requires inference profiles)+ "claude-3-7-sonnet-20250219": "bedrock/us.anthropic.claude-3-7-sonnet-20250219-v1:0",+ # Claude 3.5 (can use inference profiles for cross-region)+ "claude-3-5-sonnet-20241022": "bedrock/us.anthropic.claude-3-5-sonnet-20241022-v2:0",+ "claude-3-5-sonnet-20240620": "bedrock/us.anthropic.claude-3-5-sonnet-20240620-v1:0",+ "claude-3-5-haiku-20241022": "bedrock/us.anthropic.claude-3-5-haiku-20241022-v1:0",+ # Claude 3 (can use inference profiles for cross-region)+ "claude-3-opus-20240229": "bedrock/us.anthropic.claude-3-opus-20240229-v1:0",+ "claude-3-sonnet-20240229": "bedrock/us.anthropic.claude-3-sonnet-20240229-v1:0",+ "claude-3-haiku-20240307": "bedrock/us.anthropic.claude-3-haiku-20240307-v1:0",}VERTEX_MODEL_MAP = {diff --git a/headroom/cli/proxy.py b/headroom/cli/proxy.pyindex 810d6bc75..e1114df95 100644--- a/headroom/cli/proxy.py+++ b/headroom/cli/proxy.py@@ -205,9 +205,17 @@ Memory:memory_section += " - NOTE: Memory tools require ANTHROPIC_API_KEY (Claude Code subscription credentials have restrictions).\n"# Build usage note for Bedrock- bedrock_note = ""- if config.backend == "bedrock":- bedrock_note = "unset CLAUDE_CODE_USE_BEDROCK # Important!"+ bedrock_section = ""+ if config.backend != "anthropic":+ provider = config.backend.replace("litellm-", "").upper()+ bedrock_section = f"""+IMPORTANT for {provider} users (Claude Code / VS Code):+ 1. Set AWS credentials: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION+ 2. Set a dummy Anthropic key: ANTHROPIC_API_KEY="sk-ant-dummy"+ (Headroom ignores this - it uses your AWS credentials for {provider})+ 3. Set base URL: ANTHROPIC_BASE_URL=http://{config.host}:{config.port}+ 4. Do NOT set: CLAUDE_CODE_USE_BEDROCK=1 (Headroom handles this)+"""click.echo(f"""╔═══════════════════════════════════════════════════════════════════════╗@@ -223,9 +231,8 @@ Starting proxy server...Caching: {"ENABLED" if config.cache_enabled else "DISABLED"}Rate Limit: {"ENABLED" if config.rate_limit_enabled else "DISABLED"}Memory: {memory_status}-+{bedrock_section}Usage with Claude Code:- {bedrock_note}ANTHROPIC_BASE_URL=http://{config.host}:{config.port} claudeUsage with OpenAI-compatible clients: