Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

feat(bedrock): cross-region + Converse compression; bundle proxy binary in images (#999)

Yasser Sheikh0dc2e1cb3f72
diff --git a/CHANGELOG.md b/CHANGELOG.md
index dba10dff1..08fcd9382 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,8 +16,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* **memory:** opt-in Apple-GPU (MPS) embedding offload via `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps`. When set (and Apple MPS is available), the memory embedder runs on the torch sentence-transformers backend on the Apple GPU instead of the default ONNX CPU embedder, freeing the CPU under load. If MPS or the dependencies are unavailable, Headroom logs a warning and uses the existing default embedder selection path (ONNX when available, then the pre-existing local fallback). MPS encode calls are serialized internally (torch-MPS is not thread-safe). Adds the new `[pytorch-mps]` extra (`pip install 'headroom-ai[pytorch-mps]'`). Default behavior is unchanged.
+### Features
+
+* **proxy:** cross-region Bedrock inference-profile detection — geo-prefixed model IDs (`eu.`/`us.`/`apac.`/`global.`) are now resolved to their canonical vendor, so Anthropic cross-region profiles (e.g. `eu.anthropic.claude-haiku-4-5-20251001-v1:0`) receive live-zone compression instead of being silently skipped ([#999](https://github.com/chopratejas/headroom/pull/999)).
+* **proxy:** Converse-body compression on the native Bedrock route — the live-zone dispatcher now recognizes Bedrock Converse content blocks (typeless `{"text": …}`, not only Anthropic `{"type":"text", …}`), so Converse user-message text compresses; `run_anthropic_compression` no longer bails to passthrough when the body lacks an InvokeModel `anthropic_version` envelope, and envelope re-emit stays gated on successful parse ([#999](https://github.com/chopratejas/headroom/pull/999)).
+* **docker:** bundle `headroom-proxy` binary in published `runtime` and `runtime-slim` images — closes [#976](https://github.com/chopratejas/headroom/issues/976) ([#999](https://github.com/chopratejas/headroom/pull/999)).
+
### Bug Fixes
+* **proxy:** enable SSO credential resolution in the native Bedrock route via the `aws-config` `sso` feature flag, making the credential chain match what `docs/bedrock.md` already documented ([#999](https://github.com/chopratejas/headroom/pull/999)).
+* **proxy:** route native Bedrock `/model/{id}/converse` requests to the upstream Converse endpoint instead of the hard-coded `/invoke` action — the non-streaming handler now resolves the action from the inbound path, matching the streaming handler ([#999](https://github.com/chopratejas/headroom/pull/999)).
* **ccr:** make retrieval store TTL configurable with `HEADROOM_CCR_TTL_SECONDS`, expose the effective TTL in `/v1/retrieve/stats`, and distinguish expired retrievals from missing hashes.
* **proxy:** add native Bedrock `/model/{id}/converse-stream` route and forward it through the existing streaming EventStream/SSE pipeline.
* **wrap (codex):** fix `headroom wrap codex` producing a `config.toml` with duplicate top-level `model_provider` / `openai_base_url` keys (TOML-spec error) when the user had already configured their own provider. The injector now rewrites pre-existing top-level `model_provider` and `openai_base_url` lines in place — the previous value is kept in a `# was: …` trailing comment — instead of unconditionally prepending a duplicate, so `codex` can start against the proxy. The pre-wrap snapshot mechanism continues to byte-for-byte restore the original file on `headroom unwrap codex`.
diff --git a/Cargo.lock b/Cargo.lock
index e02e2837b..abcac5c9c 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -257,6 +257,8 @@ checksum = "50f156acdd2cf55f5aa53ee416c4ac851cf1222694506c0b1f78c85695e9ca9d"
dependencies = [
"aws-credential-types",
"aws-runtime",
+ "aws-sdk-sso",
+ "aws-sdk-ssooidc",
"aws-sdk-sts",
"aws-smithy-async",
"aws-smithy-http",
@@ -267,11 +269,14 @@ dependencies = [
"aws-types",
"bytes",
"fastrand",
+ "hex",
"http 1.4.0",
+ "sha1",
"time",
"tokio",
"tracing",
"url",
+ "zeroize",
]
[[package]]
@@ -333,6 +338,54 @@ dependencies = [
"uuid",
]
+[[package]]
+name = "aws-sdk-sso"
+version = "1.98.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d69c77aafa20460c68b6b3213c84f6423b6e76dbf89accd3e1789a686ffd9489"
+dependencies = [
+ "aws-credential-types",
+ "aws-runtime",
+ "aws-smithy-async",
+ "aws-smithy-http",
+ "aws-smithy-json",
+ "aws-smithy-observability",
+ "aws-smithy-runtime",
+ "aws-smithy-runtime-api",
+ "aws-smithy-types",
+ "aws-types",
+ "bytes",
+ "fastrand",
+ "http 0.2.12",
+ "http 1.4.0",
+ "regex-lite",
+ "tracing",
+]
+
+[[package]]
+name = "aws-sdk-ssooidc"
+version = "1.100.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1c7e7b09346d5ca22a2a08267555843a6a0127fb20d8964cb6ecfb8fdb190225"
+dependencies = [
+ "aws-credential-types",
+ "aws-runtime",
+ "aws-smithy-async",
+ "aws-smithy-http",
+ "aws-smithy-json",
+ "aws-smithy-observability",
+ "aws-smithy-runtime",
+ "aws-smithy-runtime-api",
+ "aws-smithy-types",
+ "aws-types",
+ "bytes",
+ "fastrand",
+ "http 0.2.12",
+ "http 1.4.0",
+ "regex-lite",
+ "tracing",
+]
+
[[package]]
name = "aws-sdk-sts"
version = "1.103.0"
diff --git a/Cargo.toml b/Cargo.toml
index 6f0f59c1d..dbb26a7f6 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -62,7 +62,7 @@ pyo3 = { version = "0.24", features = ["abi3-py310"] }
# (env vars, profiles, IMDS, ECS task role, etc); `aws-credential-types`
# exposes `Credentials` so the signer accepts whatever the chain returned.
aws-sigv4 = { version = "1", default-features = false, features = ["sign-http", "http1"] }
-aws-config = { version = "1", default-features = false, features = ["behavior-version-latest", "rustls", "rt-tokio"] }
+aws-config = { version = "1", default-features = false, features = ["behavior-version-latest", "rustls", "rt-tokio", "sso"] }
aws-credential-types = { version = "1", default-features = false }
# `Identity` lives in aws-smithy-runtime-api; the SigV4 builder
# accepts `&Identity`. Pinning the version explicitly avoids a
diff --git a/Dockerfile b/Dockerfile
index 59281b8a5..4e4628db5 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -60,6 +60,17 @@ RUN --mount=type=cache,target=/root/.cache/uv \
RUN cd /tmp && python -c "from headroom._core import DiffCompressor, SmartCrusher; \
print(f'build-stage rust core verify OK: {DiffCompressor.__name__}, {SmartCrusher.__name__}')"
+# Build the native Rust reverse proxy binary and stage it for the runtime
+# images (issue #976). These images already run "the proxy"; bundling the
+# native `headroom-proxy` binary lets operators front the Python proxy with
+# the Rust SigV4 / live-zone compression path from the same image. The
+# binary is copied out of the cache-mounted target dir into a persistent
+# path so the COPY in the runtime stages can pick it up.
+RUN --mount=type=cache,target=/usr/local/cargo/registry \
+ --mount=type=cache,target=/build/target \
+ cargo build --release --locked --bin headroom-proxy && \
+ cp target/release/headroom-proxy /usr/local/bin/headroom-proxy
+
# ---- Runtime stage (python-slim): supports root/nonroot via build arg ----
FROM python:${PYTHON_VERSION}-slim AS runtime-slim-base
@@ -72,6 +83,8 @@ RUN apt-get update && \
COPY --from=builder ${PYTHON_SITE_PACKAGES} ${PYTHON_SITE_PACKAGES}
COPY --from=builder /usr/local/bin/headroom /usr/local/bin/headroom
+# Native Rust reverse proxy binary (issue #976).
+COPY --from=builder /usr/local/bin/headroom-proxy /usr/local/bin/headroom-proxy
RUN mkdir -p /home/nonroot /data && \
if [ "$RUNTIME_USER" = "nonroot" ]; then \
@@ -104,6 +117,8 @@ ARG RUNTIME_USER=nonroot
ARG PYTHON_SITE_PACKAGES
COPY --from=builder ${PYTHON_SITE_PACKAGES} ${PYTHON_SITE_PACKAGES}
+# Native Rust reverse proxy binary (issue #976).
+COPY --from=builder /usr/local/bin/headroom-proxy /usr/local/bin/headroom-proxy
USER ${RUNTIME_USER}
WORKDIR /app
diff --git a/crates/headroom-core/src/transforms/live_zone.rs b/crates/headroom-core/src/transforms/live_zone.rs
index d621d7165..6105dd346 100644
--- a/crates/headroom-core/src/transforms/live_zone.rs
+++ b/crates/headroom-core/src/transforms/live_zone.rs
@@ -1009,6 +1009,23 @@ enum SlotKind {
/// latest user message. Errors out on shapes the dispatcher does not
/// support (e.g. structured-array `content` inside a tool_result —
/// rare; we degrade to NoChange in that case).
+/// Whether a content block (no `type` key) carries a JSON-string `text`
+/// field — the Bedrock Converse text-block shape (`{"text": "..."}`).
+/// Used to route typeless Converse text through the Anthropic text path.
+/// Blocks whose `text` is absent or non-string (e.g. `{"image": ...}`,
+/// `{"toolUse": ...}`) return false and stay unrecognized → no-op.
+fn block_has_string_text_field(block_json: &str) -> bool {
+ #[derive(Deserialize)]
+ struct Probe<'a> {
+ #[serde(borrow, default)]
+ text: Option<&'a RawValue>,
+ }
+ serde_json::from_str::<Probe<'_>>(block_json)
+ .ok()
+ .and_then(|p| p.text)
+ .is_some_and(|t| t.get().trim_start().starts_with('"'))
+}
+
fn plan_block_replacements(
body_raw: &[u8],
target_msg_idx: usize,
@@ -1072,7 +1089,18 @@ fn plan_block_replacements(
let header: BlockHeader<'_> =
serde_json::from_str(block_raw.get()).map_err(|_| PlanError::ParseFailed)?;
- let block_type = header.r#type.unwrap_or("unknown").to_string();
+ // Bedrock Converse content blocks carry no `type` discriminator —
+ // the variant is the key itself (`{"text": ...}`, `{"image": ...}`,
+ // `{"toolUse": ...}`). A typeless block whose `text` field is a
+ // JSON string is Converse text; route it through the same surgical
+ // path as an Anthropic `{"type":"text","text":...}` block so
+ // Converse user-message text compresses too. Anthropic blocks
+ // always carry `type`, so this never alters the Anthropic path.
+ let block_type = match header.r#type {
+ Some(t) => t.to_string(),
+ None if block_has_string_text_field(block_raw.get()) => "text".to_string(),
+ None => "unknown".to_string(),
+ };
if HOT_ZONE_BLOCK_TYPES.iter().any(|t| *t == block_type) {
slots.push(PlanSlot {
@@ -1611,6 +1639,61 @@ mod tests {
assert!(matches!(out, LiveZoneOutcome::NoChange { .. }));
}
+ #[test]
+ fn block_has_string_text_field_detects_converse_text_only() {
+ // Converse text block: typeless, string `text` → recognized.
+ assert!(block_has_string_text_field(r#"{"text":"hello"}"#));
+ // Non-text Converse blocks must NOT be mistaken for text.
+ assert!(!block_has_string_text_field(
+ r#"{"image":{"format":"png"}}"#
+ ));
+ assert!(!block_has_string_text_field(r#"{"toolUse":{"name":"x"}}"#));
+ // `text` present but not a JSON string → not Converse text.
+ assert!(!block_has_string_text_field(r#"{"text":["a"]}"#));
+ assert!(!block_has_string_text_field(r#"{"text":{"v":1}}"#));
+ }
+
+ #[test]
+ fn converse_typeless_text_block_routes_like_anthropic_text() {
+ // Bedrock Converse content blocks omit the `type` discriminator —
+ // `{"text": "..."}` instead of `{"type":"text","text":"..."}`. The
+ // dispatcher must treat the two identically so Converse user-message
+ // text compresses like Anthropic text.
+ let payload = "{\"k\": \"v\", \"n\": 1}\n".repeat(200);
+ let converse = body(json!({
+ "messages": [{"role": "user", "content": [{"text": payload}]}]
+ }));
+ let anthropic = body(json!({
+ "messages": [{"role": "user", "content": [{"type": "text", "text": payload}]}]
+ }));
+ let c = compress_anthropic_live_zone(&converse, 0, AuthMode::Payg, DEFAULT_MODEL).unwrap();
+ let a = compress_anthropic_live_zone(&anthropic, 0, AuthMode::Payg, DEFAULT_MODEL).unwrap();
+
+ // Identical dispatch outcome (both Modified or both NoChange).
+ assert_eq!(
+ std::mem::discriminant(&c),
+ std::mem::discriminant(&a),
+ "converse text block must dispatch like an anthropic text block"
+ );
+ let cm = match &c {
+ LiveZoneOutcome::NoChange { manifest } => manifest,
+ LiveZoneOutcome::Modified { manifest, .. } => manifest,
+ };
+ let am = match &a {
+ LiveZoneOutcome::NoChange { manifest } => manifest,
+ LiveZoneOutcome::Modified { manifest, .. } => manifest,
+ };
+ // The Converse block is now classified the same as Anthropic text
+ // (before this change it was an unrecognized typeless block).
+ assert_eq!(cm.block_outcomes.len(), 1);
+ assert_eq!(am.block_outcomes.len(), 1);
+ assert_eq!(
+ cm.block_outcomes[0].block_type,
+ am.block_outcomes[0].block_type
+ );
+ assert_eq!(cm.block_outcomes[0].block_type, "text");
+ }
+
#[test]
fn manifest_records_messages_below_floor() {
let b = body(json!({
diff --git a/crates/headroom-proxy/src/bedrock/invoke.rs b/crates/headroom-proxy/src/bedrock/invoke.rs
index 84c93a6cb..2571e8bcb 100644
--- a/crates/headroom-proxy/src/bedrock/invoke.rs
+++ b/crates/headroom-proxy/src/bedrock/invoke.rs
@@ -65,10 +65,7 @@ use crate::proxy::AppState;
// would risk drift from the middleware's resolution + WARN log.
use headroom_core::auth_mode::AuthMode;
-/// Anthropic vendor prefix as encoded in Bedrock model ids
-/// (`anthropic.claude-3-haiku-...`). Literal-match per project rule
-/// "no regexes for parsing the model ID".
-const ANTHROPIC_VENDOR_PREFIX: &str = "anthropic.";
+use crate::bedrock::vendor::is_anthropic_model_id;
/// RAII guard that observes the `bedrock_invoke_latency_seconds`
/// histogram on drop. Created at handler entry; observed when the
@@ -102,6 +99,27 @@ impl Drop for LatencyGuard {
/// region. Only used when `Config::bedrock_endpoint` is `None`.
const BEDROCK_RUNTIME_HOST_TEMPLATE: &str = "bedrock-runtime.{region}.amazonaws.com";
+/// Bedrock non-streaming action path segments. `invoke` is the legacy
+/// InvokeModel surface; `converse` is the unified Converse surface.
+/// Both mount the same handler (see `proxy.rs`), so the action is
+/// resolved from the inbound path — otherwise `/converse` requests
+/// would be forwarded to the upstream `/invoke` endpoint.
+const INVOKE_ACTION: &str = "invoke";
+const CONVERSE_ACTION: &str = "converse";
+
+/// Resolve the Bedrock action from the inbound request path. Mirrors
+/// `invoke_streaming::extract_streaming_action` for the non-streaming
+/// surfaces (`/invoke`, `/converse`).
+fn extract_invoke_action(path: &str) -> Option<&'static str> {
+ if path.ends_with(&format!("/{INVOKE_ACTION}")) {
+ Some(INVOKE_ACTION)
+ } else if path.ends_with(&format!("/{CONVERSE_ACTION}")) {
+ Some(CONVERSE_ACTION)
+ } else {
+ None
+ }
+}
+
/// Axum POST handler for `/model/{model_id}/invoke`.
///
/// Buffers the body so the live-zone compressor + SigV4 signer can
@@ -155,7 +173,7 @@ pub async fn handle_invoke(
"bedrock invoke route received request"
);
- let is_anthropic = model_id.starts_with(ANTHROPIC_VENDOR_PREFIX);
+ let is_anthropic = is_anthropic_model_id(&model_id);
let outbound_body: Bytes = if is_anthropic {
run_anthropic_compression(&body, &state, auth_mode, &request_id)
} else {
@@ -169,9 +187,30 @@ pub async fn handle_invoke(
body.clone()
};
+ // Resolve the Bedrock action from the inbound path so `/converse`
+ // forwards to the upstream Converse endpoint instead of `/invoke`.
+ // Both paths mount this handler (see `proxy.rs`); the streaming
+ // sibling resolves its action the same way.
+ let action = match extract_invoke_action(uri.path()) {
+ Some(a) => a,
+ None => {
+ tracing::error!(
+ event = "bedrock_invoke_action_invalid",
+ request_id = %request_id,
+ path = %uri.path(),
+ "bedrock invoke: unrecognized action path"
+ );
+ return error_response(
+ StatusCode::BAD_REQUEST,
+ "bedrock_invoke_action_invalid",
+ "Unsupported Bedrock action path",
+ );
+ }
+ };
+
// Build the upstream URL based on configured endpoint or
// region-derived default.
- let upstream_url = match build_bedrock_upstream(&state, &model_id, &uri, "invoke") {
+ let upstream_url = match build_bedrock_upstream(&state, &model_id, &uri, action) {
Ok(u) => u,
Err(msg) => {
tracing::error!(
@@ -363,24 +402,26 @@ fn run_anthropic_compression(
_auth_mode: AuthMode,
request_id: &str,
) -> Bytes {
- // Validate envelope shape. If the body isn't a valid Bedrock
- // envelope we still forward verbatim — the compressor would have
- // refused too — but log loudly.
- if let Err(e) = BedrockEnvelope::parse(body) {
- tracing::warn!(
- event = "bedrock_envelope_parse_error",
+ // Detect envelope shape. A parseable InvokeModel envelope takes the
+ // re-emit path below (anthropic_version pinned first); a non-envelope
+ // body (e.g. a Converse-shaped payload) still runs through the
+ // compressor but skips envelope re-emit. The body is NOT guaranteed
+ // unchanged on parse failure — we log which path we took.
+ let parsed_envelope = BedrockEnvelope::parse(body).is_ok();
+ if parsed_envelope {
+ tracing::info!(
+ event = "bedrock_envelope_parsed",
request_id = %request_id,
- error = %e,
- "bedrock invoke: envelope parse failed; passing body through unchanged"
+ body_bytes = body.len(),
+ "bedrock invoke: envelope validated; dispatching to live-zone compressor"
+ );
+ } else {
+ tracing::info!(
+ event = "bedrock_envelope_parse_skipped",
+ request_id = %request_id,
+ "bedrock invoke: envelope parse skipped; attempting generic anthropic compression"
);
- return body.clone();
}
- tracing::info!(
- event = "bedrock_envelope_parsed",
- request_id = %request_id,
- body_bytes = body.len(),
- "bedrock invoke: envelope validated; dispatching to live-zone compressor"
- );
// PR-E3: Bedrock uses IAM-signed AWS SigV4 downstream. Inbound
// requests to the proxy may or may not carry their own auth, but
@@ -410,20 +451,24 @@ fn run_anthropic_compression(
body.clone()
}
AnthropicOutcome::Compressed { body: new_body, .. } => {
- // Defence-in-depth: re-emit so anthropic_version is the
- // first key. With preserve_order this is a no-op on the
- // happy path.
- match BedrockEnvelope::ensure_anthropic_version_first(&new_body) {
- Ok(b) => b,
- Err(e) => {
- tracing::error!(
- event = "bedrock_envelope_reemit_failed",
- request_id = %request_id,
- error = %e,
- "bedrock invoke: failed to re-emit envelope; falling back to original body"
- );
- body.clone()
+ if parsed_envelope {
+ // Defence-in-depth: re-emit so anthropic_version is the
+ // first key. With preserve_order this is a no-op on the
+ // happy path.
+ match BedrockEnvelope::ensure_anthropic_version_first(&new_body) {
+ Ok(b) => b,
+ Err(e) => {
+ tracing::error!(
+ event = "bedrock_envelope_reemit_failed",
+ request_id = %request_id,
+ error = %e,
+ "bedrock invoke: failed to re-emit envelope; falling back to original body"
+ );
+ body.clone()
+ }
}
+ } else {
+ new_body
}
}
}
@@ -538,12 +583,57 @@ fn error_response(status: StatusCode, event: &str, msg: &str) -> Response {
mod tests {
use super::*;
+ // Vendor/model-id classification is tested in `bedrock::vendor`.
+
+ #[test]
+ fn extract_invoke_action_supports_both_bedrock_paths() {
+ assert_eq!(
+ extract_invoke_action("/model/anthropic.claude-3-haiku-20240307-v1:0/invoke"),
+ Some(INVOKE_ACTION)
+ );
+ assert_eq!(
+ extract_invoke_action("/model/anthropic.claude-3-haiku-20240307-v1:0/converse"),
+ Some(CONVERSE_ACTION)
+ );
+ // Streaming actions are handled by `invoke_streaming`, not here.
+ assert_eq!(
+ extract_invoke_action(
+ "/model/anthropic.claude-3-haiku-20240307-v1:0/invoke-with-response-stream"
+ ),
+ None
+ );
+ assert_eq!(extract_invoke_action("/model/foo/unknown"), None);
+ }
+
#[test]
- fn anthropic_vendor_prefix_match() {
- assert!("anthropic.claude-3-haiku-20240307-v1:0".starts_with(ANTHROPIC_VENDOR_PREFIX));
- assert!("anthropic.claude-3-5-sonnet-20241022-v2:0".starts_with(ANTHROPIC_VENDOR_PREFIX));
- assert!(!"amazon.titan-text-express-v1".starts_with(ANTHROPIC_VENDOR_PREFIX));
- assert!(!"meta.llama3-70b-instruct-v1:0".starts_with(ANTHROPIC_VENDOR_PREFIX));
+ fn build_upstream_routes_converse_to_converse_endpoint() {
+ use crate::config::Config;
+ let mut config = Config::for_test(Url::parse("http://up:8080").unwrap());
+ config.bedrock_region = "us-west-2".to_string();
+ let state = AppState {
+ config: std::sync::Arc::new(config),
+ client: reqwest::Client::new(),
+ bedrock_credentials: None,
+ drift_state: crate::cache_stabilization::drift_detector::DriftState::new(8),
+ vertex_token_source: std::sync::Arc::new(crate::vertex::StaticTokenSource::new(
+ "test".to_string(),
+ )),
+ };
+ let uri: Uri = "/model/anthropic.claude-3-haiku-20240307-v1:0/converse"
+ .parse()
+ .unwrap();
+ let action = extract_invoke_action(uri.path()).unwrap();
+ let url = build_bedrock_upstream(
+ &state,
+ "anthropic.claude-3-haiku-20240307-v1:0",
+ &uri,
+ action,
+ )
+ .unwrap();
+ assert_eq!(
+ url.as_str(),
+ "https://bedrock-runtime.us-west-2.amazonaws.com/model/anthropic.claude-3-haiku-20240307-v1:0/converse"
+ );
}
#[test]
diff --git a/crates/headroom-proxy/src/bedrock/invoke_streaming.rs b/crates/headroom-proxy/src/bedrock/invoke_streaming.rs
index 347195f6e..ce66429f9 100644
--- a/crates/headroom-proxy/src/bedrock/invoke_streaming.rs
+++ b/crates/headroom-proxy/src/bedrock/invoke_streaming.rs
@@ -74,8 +74,7 @@ use crate::proxy::AppState;
// `Extension<AuthMode>` extractor.
use headroom_core::auth_mode::AuthMode;
-/// Anthropic vendor prefix as encoded in Bedrock model ids.
-const ANTHROPIC_VENDOR_PREFIX: &str = "anthropic.";
+use crate::bedrock::vendor::is_anthropic_model_id;
/// AWS Bedrock Runtime DNS template.
const BEDROCK_RUNTIME_HOST_TEMPLATE: &str = "bedrock-runtime.{region}.amazonaws.com";
@@ -153,7 +152,7 @@ pub async fn handle_invoke_streaming(
);
// 1. Live-zone compression for Anthropic-shape bodies (same as D1).
- let is_anthropic = model_id.starts_with(ANTHROPIC_VENDOR_PREFIX);
+ let is_anthropic = is_anthropic_model_id(&model_id);
let outbound_body: Bytes = if is_anthropic {
run_anthropic_compression(&body, &state, auth_mode, &request_id)
} else {
@@ -855,14 +854,13 @@ fn run_anthropic_compression(
) -> Bytes {
use crate::bedrock::envelope::BedrockEnvelope;
- if let Err(e) = BedrockEnvelope::parse(body) {
- tracing::warn!(
- event = "bedrock_envelope_parse_error",
+ let parsed_envelope = BedrockEnvelope::parse(body).is_ok();
+ if !parsed_envelope {
+ tracing::info!(
+ event = "bedrock_envelope_parse_skipped",
request_id = %request_id,
- error = %e,
- "bedrock invoke-streaming: envelope parse failed; passing body through unchanged"
+ "bedrock invoke-streaming: envelope parse skipped; attempting generic anthropic compression"
);
- return body.clone();
}
// PR-E3: Bedrock channel hard-codes OAuth so cache_control
@@ -887,17 +885,21 @@ fn run_anthropic_compression(
body.clone()
}
AnthropicOutcome::Compressed { body: new_body, .. } => {
- match BedrockEnvelope::ensure_anthropic_version_first(&new_body) {
- Ok(b) => b,
- Err(e) => {
- tracing::error!(
- event = "bedrock_envelope_reemit_failed",
- request_id = %request_id,
- error = %e,
- "bedrock invoke-streaming: failed to re-emit envelope"
- );
- body.clone()
+ if parsed_envelope {
+ match BedrockEnvelope::ensure_anthropic_version_first(&new_body) {
+ Ok(b) => b,
+ Err(e) => {
+ tracing::error!(
+ event = "bedrock_envelope_reemit_failed",
+ request_id = %request_id,
+ error = %e,
+ "bedrock invoke-streaming: failed to re-emit envelope"
+ );
+ body.clone()
+ }
}
+ } else {
+ new_body
}
}
}
diff --git a/crates/headroom-proxy/src/bedrock/mod.rs b/crates/headroom-proxy/src/bedrock/mod.rs
index 780ed4d18..d80f87092 100644
--- a/crates/headroom-proxy/src/bedrock/mod.rs
+++ b/crates/headroom-proxy/src/bedrock/mod.rs
@@ -52,6 +52,7 @@ pub mod eventstream_to_sse;
pub mod invoke;
pub mod invoke_streaming;
pub mod sigv4;
+pub mod vendor;
pub use auth_mode_layer::classify_and_attach_auth_mode;
pub use envelope::{BedrockEnvelope, EnvelopeError};
diff --git a/crates/headroom-proxy/src/bedrock/vendor.rs b/crates/headroom-proxy/src/bedrock/vendor.rs
new file mode 100644
index 000000000..bfa378be1
--- /dev/null
+++ b/crates/headroom-proxy/src/bedrock/vendor.rs
@@ -0,0 +1,82 @@
+//! Canonical Bedrock vendor resolution.
+//!
+//! Bedrock model IDs are `<vendor>.<model>-<date>-<rev>` (e.g.
+//! `anthropic.claude-3-haiku-20240307-v1:0`). Cross-region inference
+//! profiles prepend a geo routing token before the vendor
+//! (`eu.anthropic.…`, `us.anthropic.…`, `apac.anthropic.…`,
+//! `global.anthropic.…`). Matching the bare `anthropic.` prefix alone
+//! silently skips compression for those profiles, so we strip a known
+//! geo prefix first, then take the leading dot-segment as the vendor.
+//! Literal matching only — no regexes (project rule).
+
+/// Cross-region inference-profile routing prefixes AWS prepends to the
+/// vendor segment. Stripped (once) before vendor resolution. Kept to a
+/// closed, known set so an unrelated `something.anthropic.x` id is not
+/// mistaken for an Anthropic inference profile.
+const GEO_PREFIXES: [&str; 4] = ["eu.", "us.", "apac.", "global."];
+
+/// Resolve the canonical vendor of a Bedrock model id, stripping a
+/// cross-region inference-profile geo prefix first.
+///
+/// `eu.anthropic.claude-…` → `anthropic`; `amazon.titan-…` → `amazon`;
+/// `global.amazon.nova-…` → `amazon`.
+pub fn canonical_vendor(model_id: &str) -> &str {
+ let stripped = GEO_PREFIXES
+ .iter()
+ .find_map(|p| model_id.strip_prefix(p))
+ .unwrap_or(model_id);
+ stripped.split('.').next().unwrap_or(stripped)
+}
+
+/// Whether the model id is an Anthropic-shape model — a foundation
+/// model (`anthropic.…`) or a cross-region inference profile
+/// (`<geo>.anthropic.…`) — i.e. eligible for the live-zone Anthropic
+/// compression + envelope pipeline.
+pub fn is_anthropic_model_id(model_id: &str) -> bool {
+ canonical_vendor(model_id) == "anthropic"
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn canonical_vendor_strips_known_cross_region_prefix() {
+ assert_eq!(
+ canonical_vendor("anthropic.claude-3-haiku-20240307-v1:0"),
+ "anthropic"
+ );
+ assert_eq!(
+ canonical_vendor("eu.anthropic.claude-haiku-4-5-20251001-v1:0"),
+ "anthropic"
+ );
+ assert_eq!(canonical_vendor("amazon.titan-text-express-v1"), "amazon");
+ assert_eq!(canonical_vendor("global.amazon.nova-lite-v1:0"), "amazon");
+ // Unknown leading token is NOT stripped — stays as its own vendor.
+ assert_eq!(canonical_vendor("random.anthropic.x"), "random");
+ }
+
+ #[test]
+ fn anthropic_model_id_matches_foundation_and_inference_profiles() {
+ assert!(is_anthropic_model_id(
+ "anthropic.claude-3-haiku-20240307-v1:0"
+ ));
+ assert!(is_anthropic_model_id(
+ "eu.anthropic.claude-haiku-4-5-20251001-v1:0"
+ ));
+ assert!(is_anthropic_model_id(
+ "us.anthropic.claude-3-5-sonnet-20241022-v2:0"
+ ));
+ assert!(is_anthropic_model_id(
+ "apac.anthropic.claude-3-5-sonnet-20240620-v1:0"
+ ));
+ assert!(is_anthropic_model_id(
+ "global.anthropic.claude-haiku-4-5-20251001-v1:0"
+ ));
+ // Non-Anthropic vendors, including geo-prefixed, stay false.
+ assert!(!is_anthropic_model_id("amazon.titan-text-express-v1"));
+ assert!(!is_anthropic_model_id("meta.llama3-70b-instruct-v1:0"));
+ assert!(!is_anthropic_model_id("eu.amazon.nova-lite-v1:0"));
+ assert!(!is_anthropic_model_id("mistral.voxtral-mini-3b-2507"));
+ }
+}
diff --git a/docs/bedrock.md b/docs/bedrock.md
index 31d502c44..c3a51bc45 100644
--- a/docs/bedrock.md
+++ b/docs/bedrock.md
@@ -16,6 +16,40 @@ This document covers how to deploy the Bedrock-native surface, how compression p
| Per-model + per-region Prometheus metrics | PR-D3 — exposed at `GET /metrics` |
| OAuth compression policy gates (no auto cache_control, lossless-only) | Phase F PR-F2/F3 (gates the marker D3 wires) |
+## Running the proxy
+
+The native surface lives in the `headroom-proxy` binary, which ships in the published
+container images (every `proxy`-extra tag) at `/usr/local/bin/headroom-proxy`. You can run
+it directly from any published image — no separate build:
+
+```sh
+docker run --rm -p 8787:8787 \
+ -v "$HOME/.aws:/home/nonroot/.aws:ro" \
+ -e HEADROOM_PROXY_AWS_PROFILE=my-profile \
+ --entrypoint headroom-proxy \
+ ghcr.io/chopratejas/headroom:latest \
+ --listen 0.0.0.0:8787 \
+ --upstream https://bedrock-runtime.us-east-1.amazonaws.com \
+ --bedrock-region us-east-1
+```
+
+The published images default to the `nonroot` user (home `/home/nonroot`), so AWS
+credentials are mounted at `/home/nonroot/.aws` — that is where the SDK looks for
+`~/.aws`. For a root-based image (`RUNTIME_USER=root` build), mount to `/root/.aws`
+instead, or pass `--user root`.
+
+Then point the AWS SDK / CLI at the proxy:
+
+```sh
+AWS_ENDPOINT_URL_BEDROCK_RUNTIME=http://localhost:8787 \
+ aws bedrock-runtime invoke-model --model-id anthropic.claude-3-haiku-20240307-v1:0 ...
+```
+
+The proxy can also drop in front of the Python proxy (`--upstream http://127.0.0.1:8788`)
+so non-Bedrock traffic is forwarded while Bedrock requests are signed + compressed
+natively. The default `--enable-bedrock-native=true` mounts the Bedrock routes; everything
+else is passed through to `--upstream`.
+
## AWS credential configuration
The proxy uses the [aws-config default credential chain](https://docs.aws.amazon.com/sdkref/latest/guide/standardized-credentials.html), resolved once at startup.
@@ -75,17 +109,19 @@ headroom-proxy \
## Supported model IDs
-The proxy classifies model IDs by **literal vendor prefix** — no regexes. Any model ID starting with `anthropic.` is treated as Anthropic-shape: the live-zone compression dispatcher runs over the body, the envelope is re-emitted with `anthropic_version` preserved as the first key, and the request is signed with SigV4.
+The proxy classifies model IDs by **literal vendor match** — no regexes. It strips a known cross-region inference-profile geo prefix (`eu.`, `us.`, `apac.`, `global.`) if present, then takes the leading dot-segment as the vendor. A model is treated as Anthropic-shape when that canonical vendor is `anthropic` — so both the bare `anthropic.…` foundation models and the geo-prefixed inference profiles (`eu.anthropic.…`, `us.anthropic.…`, `apac.anthropic.…`, `global.anthropic.…`) qualify. For those, the live-zone compression dispatcher runs over the body, the envelope is re-emitted with `anthropic_version` preserved as the first key, and the request is signed with SigV4.
Examples that hit the Anthropic compression path:
-- `anthropic.claude-3-haiku-20240307-v1:0`
+- `anthropic.claude-3-haiku-20240307-v1:0` (foundation model)
- `anthropic.claude-3-5-sonnet-20241022-v2:0`
-- `anthropic.claude-3-opus-20240229-v1:0`
+- `eu.anthropic.claude-haiku-4-5-20251001-v1:0` (EU cross-region inference profile)
+- `us.anthropic.claude-3-5-sonnet-20241022-v2:0` (US inference profile)
+- `global.anthropic.claude-haiku-4-5-20251001-v1:0`
-Other Bedrock vendors (`amazon.titan-...`, `meta.llama3-...`, `cohere.command-...`, `ai21.j2-...`, `stability.stable-diffusion-...`) are signed and forwarded **without compression** — the proxy does not yet understand their body shapes and would risk corrupting them. These model IDs log `event=bedrock_compression_skipped, reason=non_anthropic_vendor` per request. Full Anthropic envelopes only.
+Other Bedrock vendors (`amazon.titan-...`, `meta.llama3-...`, `cohere.command-...`, `ai21.j2-...`, `stability.stable-diffusion-...`, and their geo-prefixed inference profiles such as `eu.amazon.nova-...`) are signed and forwarded **without compression** — the proxy does not yet understand their body shapes and would risk corrupting them. These model IDs log `event=bedrock_compression_skipped, reason=non_anthropic_vendor` per request. Full Anthropic envelopes only.
-The contract: **any new model ID that AWS adds under the `anthropic.` prefix automatically picks up the full compression + signing pipeline.** No code change in the proxy is needed for new versions of Claude on Bedrock.
+The contract: **any new model ID that AWS adds under the `anthropic.` vendor (as a bare prefix or behind a cross-region geo prefix) automatically picks up the full compression + signing pipeline.** No code change in the proxy is needed for new versions of Claude on Bedrock.
## Compression behaviour