Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix: PR-E4 OpenAI prompt_cache_key auto-injection (Phase E)

chopratejas573543fce7dc
diff --git a/crates/headroom-proxy/Cargo.toml b/crates/headroom-proxy/Cargo.toml
index 40d879576..6a38c03d6 100644
--- a/crates/headroom-proxy/Cargo.toml
+++ b/crates/headroom-proxy/Cargo.toml
@@ -63,7 +63,8 @@ prometheus = { version = "0.13", default-features = false }
# tools, early messages) for cache-bust drift detection. Already in
# the dev-dependencies (and pulled transitively by `aws-sigv4` via
# `aws-smithy-runtime-api`); promoted here to a direct, normal-build
-# dependency so the drift detector compiles outside `cfg(test)`.
+# dependency so the drift detector compiles outside `cfg(test)`. Also
+# used by PR-E4 for `prompt_cache_key` derivation.
sha2 = "0.10"
# PR-E6: bounded session-scoped cache of structural hashes. The
# detector evicts the oldest session at 1000 entries — we never want
diff --git a/crates/headroom-proxy/src/cache_stabilization/mod.rs b/crates/headroom-proxy/src/cache_stabilization/mod.rs
index cbf17df17..f32cad858 100644
--- a/crates/headroom-proxy/src/cache_stabilization/mod.rs
+++ b/crates/headroom-proxy/src/cache_stabilization/mod.rs
@@ -4,10 +4,12 @@
//! groups every cache-stabilization mechanism behind one module so
//! operators searching for "what does Headroom do to keep prompt
//! caches warm" land in one place. Phase E PRs in this module sit
-//! *next to* the request path — never on it. They observe inbound
-//! bodies and emit structured logs so customers can see why their
-//! prompt-cache hit rate is degrading. Nothing in here mutates
-//! request bytes; the cache-safety invariant from Phase A still holds.
+//! *next to* the request path — either as observers
+//! (volatile_detector, drift_detector) or as PAYG-gated mutators
+//! (openai_cache_key, anthropic cache_control). The Phase A
+//! "passthrough is sacred" invariant still holds: mutators MUST
+//! gate on `AuthMode::Payg` at their call sites before invoking
+//! any function that mutates the body. Observers never mutate.
//!
//! Currently shipped:
//!
@@ -28,14 +30,19 @@
//! learning Anthropic's marker API. **Mutates request bytes**;
//! gated on auth_mode == PAYG and the absence of any pre-existing
//! marker.
+//! - [`openai_cache_key`] — PR-E4: on PAYG OpenAI requests where the
+//! customer has not set `prompt_cache_key`, derive a stable key from
+//! `(model, system, tools)` and inject it so the upstream pins
+//! cache lookup to a tenant-stable identity. **Mutates the body**
+//! (only on PAYG) — see its docs for the gating contract.
//!
//! Future PRs (E1 — tool-array sort, E2 — JSON Schema key sort, E3 —
-//! `cache_control` auto-placement, E4 — `prompt_cache_key` injection)
-//! hang sibling submodules off this same `mod.rs`. Conflict
-//! resolution between parallel Phase E PRs is intentionally trivial:
-//! each detector lives in its own file, the only shared surface is
-//! this `mod.rs`'s `pub mod` list.
+//! `cache_control` auto-placement) hang sibling submodules off this
+//! same `mod.rs`. Conflict resolution between parallel Phase E PRs
+//! is intentionally trivial: each detector lives in its own file,
+//! the only shared surface is this `mod.rs`'s `pub mod` list.
pub mod anthropic_cache_control;
pub mod drift_detector;
+pub mod openai_cache_key;
pub mod volatile_detector;
diff --git a/crates/headroom-proxy/src/cache_stabilization/openai_cache_key.rs b/crates/headroom-proxy/src/cache_stabilization/openai_cache_key.rs
new file mode 100644
index 000000000..5f4eb5664
--- /dev/null
+++ b/crates/headroom-proxy/src/cache_stabilization/openai_cache_key.rs
@@ -0,0 +1,600 @@
+//! PR-E4: OpenAI `prompt_cache_key` auto-injection.
+//!
+//! OpenAI's prefix caching is automatic since late 2024 — the API
+//! caches prefix tokens server-side without client opt-in. The
+//! `prompt_cache_key` field, however, is the field a client passes
+//! to **pin cache lookup** to a stable identity. Without it,
+//! OpenAI falls back to org-wide cache lookups that may collide
+//! with other tenants. Most clients don't set the field because
+//! they don't know it exists. This module derives a deterministic
+//! key from the request's *structural* prefix (model + system +
+//! tools) and injects it on PAYG requests where the customer has
+//! not already provided one.
+//!
+//! # Universal safety contract
+//!
+//! This module mutates request bytes — that is the whole point.
+//! Mutating bytes for non-PAYG auth modes risks looking like
+//! cache-evasion to the upstream and would void OAuth scope; the
+//! caller MUST gate on `AuthMode::Payg` before calling
+//! [`inject_prompt_cache_key`]. The function itself does NOT
+//! re-classify auth mode: that is the caller's responsibility,
+//! consistent with PR-E3's `cache_control` placement which gates
+//! the same way at the dispatch site.
+//!
+//! # Skip rules (this module enforces these directly)
+//!
+//! - **Already present**: if `body.prompt_cache_key` exists at the
+//! top level and is a non-empty string, return
+//! [`InjectOutcome::Skipped`] with reason
+//! [`SkipReason::KeyPresent`]. The customer's value wins.
+//! - **Idempotency**: re-running on a body that already has *our*
+//! injected key returns [`SkipReason::KeyPresent`] (we don't
+//! distinguish ours from theirs — both mean "leave alone").
+//!
+//! # Key derivation
+//!
+//! Inputs to the hash, in order:
+//!
+//! 1. `body.model` (string) — different model = different cache
+//! universe.
+//! 2. SHA-256 of canonical-JSON bytes of the system content.
+//! For Chat Completions, this is the first message with
+//! `role == "system"`. For Responses, this is the
+//! `instructions` field if present, else the first
+//! `role == "system"` item.
+//! 3. SHA-256 of canonical-JSON bytes of `body.tools`.
+//!
+//! User and assistant messages are **deliberately excluded** —
+//! including them would defeat the purpose by producing a
+//! different key on every turn.
+//!
+//! Combined: `key = hex(sha256(model || system_hash ||
+//! tools_hash))[..32]` — 128 bits of collision resistance, 32
+//! hex chars on the wire, compact in logs.
+
+use serde_json::Value;
+use sha2::{Digest, Sha256};
+
+/// Length of the injected key in hex characters. 32 hex chars =
+/// 128 bits, plenty of collision resistance for per-tenant cache
+/// pinning, while staying compact on the wire and in logs.
+pub const KEY_HEX_LEN: usize = 32;
+
+/// First-N chars of the key surfaced in `tracing` events. **Never**
+/// log the full key — it is identifying material for cache pinning
+/// and operators do not need the suffix to debug.
+pub const KEY_PREFIX_LOG_LEN: usize = 8;
+
+/// Which OpenAI request shape the body conforms to. The shape
+/// affects only where the system content lives:
+///
+/// - [`OpenAiShape::ChatCompletions`]: first `messages[*]` with
+/// `role == "system"`.
+/// - [`OpenAiShape::Responses`]: top-level `instructions` field,
+/// else first `input[*]` (or `messages[*]`) with
+/// `role == "system"`.
+///
+/// `tools` lives at the top level in both shapes.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum OpenAiShape {
+ /// `POST /v1/chat/completions`.
+ ChatCompletions,
+ /// `POST /v1/responses`.
+ Responses,
+}
+
+/// Why the injector skipped a body.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum SkipReason {
+ /// The body is not a JSON object (the dispatcher should have
+ /// gated this earlier; we surface it as a skip rather than
+ /// panicking so the proxy stays a transparent forwarder for
+ /// pathological inputs).
+ NotAnObject,
+ /// `body.prompt_cache_key` is already a non-empty string.
+ /// Customer-set values win.
+ KeyPresent,
+}
+
+impl SkipReason {
+ /// Stable string for structured-log fields. Dashboards filter
+ /// on this; do not change without a deprecation note.
+ pub fn as_str(self) -> &'static str {
+ match self {
+ SkipReason::NotAnObject => "not_an_object",
+ SkipReason::KeyPresent => "key_present",
+ }
+ }
+}
+
+/// Outcome of an injection attempt.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum InjectOutcome {
+ /// We added a `prompt_cache_key` to the body. The first
+ /// [`KEY_PREFIX_LOG_LEN`] hex chars of the key, suitable for
+ /// logs. The full key is in the body — never log it.
+ Applied { key_prefix: String },
+ /// We left the body unchanged. See [`SkipReason`] for why.
+ Skipped { reason: SkipReason },
+}
+
+/// Top-level field check.
+///
+/// Returns `true` if `body.prompt_cache_key` is a non-empty
+/// string. Empty strings count as "absent" because some SDKs
+/// default the field to `""` when the user did not set it.
+pub fn has_prompt_cache_key(body: &Value) -> bool {
+ body.get("prompt_cache_key")
+ .and_then(Value::as_str)
+ .map(|s| !s.is_empty())
+ .unwrap_or(false)
+}
+
+/// Inject a `prompt_cache_key` into `body` if appropriate.
+///
+/// **Caller MUST gate on `AuthMode::Payg`.** This function does not
+/// re-classify auth mode — same contract as PR-E3
+/// `cache_control` placement.
+///
+/// Returns:
+/// - [`InjectOutcome::Applied`] when the key is set;
+/// `body["prompt_cache_key"]` is now a 32-hex-char string.
+/// - [`InjectOutcome::Skipped`] when the body is not a JSON
+/// object, or already has a non-empty `prompt_cache_key`.
+///
+/// This function is **deterministic**: the same `(model, system,
+/// tools)` triple always produces the same key, so re-running on a
+/// body whose injected key was stripped yields the same key
+/// (idempotency property the test suite pins).
+pub fn inject_prompt_cache_key(body: &mut Value, shape: OpenAiShape) -> InjectOutcome {
+ // Guardrail: top-level must be an object. Anything else (array,
+ // string, null) is malformed for our hook point; the dispatcher
+ // already gates on `messages` / `input` being arrays, so this
+ // path is essentially unreachable but we surface it explicitly
+ // rather than panicking.
+ if !body.is_object() {
+ return InjectOutcome::Skipped {
+ reason: SkipReason::NotAnObject,
+ };
+ }
+
+ if has_prompt_cache_key(body) {
+ return InjectOutcome::Skipped {
+ reason: SkipReason::KeyPresent,
+ };
+ }
+
+ let key = derive_key(body, shape);
+ let key_prefix = key[..KEY_PREFIX_LOG_LEN.min(key.len())].to_string();
+
+ // Safe: we just verified `body.is_object()` above.
+ if let Some(map) = body.as_object_mut() {
+ map.insert("prompt_cache_key".to_string(), Value::String(key));
+ }
+
+ InjectOutcome::Applied { key_prefix }
+}
+
+/// Derive the cache key from the structural prefix of the body.
+///
+/// Hash inputs (concatenated, length-prefixed to avoid ambiguity
+/// between e.g. `model="ab", system="c"` vs `model="a", system="bc"`):
+///
+/// 1. ASCII bytes of the model field (empty if missing).
+/// 2. SHA-256 of canonical-JSON bytes of the system content.
+/// 3. SHA-256 of canonical-JSON bytes of the tools field.
+///
+/// Length-prefixing is critical: without it, two different
+/// (model, system) splits could collide. We use a single `0x00`
+/// byte separator instead of an explicit u32 length because
+/// `0x00` cannot appear in a UTF-8 model name or a hex digest;
+/// the separator unambiguously delimits the three inputs.
+fn derive_key(body: &Value, shape: OpenAiShape) -> String {
+ let model = body
+ .get("model")
+ .and_then(Value::as_str)
+ .unwrap_or_default();
+
+ let system_value = extract_system(body, shape);
+ let system_hash = canonical_sha256(&system_value);
+
+ let tools_value = body.get("tools").cloned().unwrap_or(Value::Null);
+ let tools_hash = canonical_sha256(&tools_value);
+
+ let mut hasher = Sha256::new();
+ hasher.update(model.as_bytes());
+ hasher.update([0u8]);
+ hasher.update(system_hash.as_bytes());
+ hasher.update([0u8]);
+ hasher.update(tools_hash.as_bytes());
+ let digest = hasher.finalize();
+
+ // 16 bytes = 32 hex chars. We hex-encode by hand to avoid
+ // pulling in `hex` for one call site.
+ let mut out = String::with_capacity(KEY_HEX_LEN);
+ for byte in digest.iter().take(KEY_HEX_LEN / 2) {
+ use std::fmt::Write as _;
+ let _ = write!(&mut out, "{byte:02x}");
+ }
+ out
+}
+
+/// Locate the system content for the given shape, returning the
+/// JSON value (possibly `Null`) we will hash. We hash the JSON
+/// value (not its serialized bytes alone) so that
+/// content-block-array systems and string systems with the same
+/// concatenated text produce *different* keys — which is the
+/// correct behaviour for cache pinning.
+fn extract_system(body: &Value, shape: OpenAiShape) -> Value {
+ match shape {
+ OpenAiShape::ChatCompletions => first_system_message_content(body, "messages"),
+ OpenAiShape::Responses => {
+ // Responses canonical: top-level `instructions`.
+ // Legacy alias: a system message in `input` (or `messages`).
+ if let Some(instructions) = body.get("instructions") {
+ return instructions.clone();
+ }
+ if let Some(v) = body.get("input") {
+ if let Some(content) = first_system_in_array(v) {
+ return content;
+ }
+ }
+ first_system_message_content(body, "messages")
+ }
+ }
+}
+
+/// First `messages[*]` (under the given key) with `role == "system"` —
+/// returns the message's `content` field, or `Null` if none found.
+fn first_system_message_content(body: &Value, key: &str) -> Value {
+ body.get(key)
+ .and_then(first_system_in_array)
+ .unwrap_or(Value::Null)
+}
+
+fn first_system_in_array(arr: &Value) -> Option<Value> {
+ let items = arr.as_array()?;
+ for item in items {
+ if item.get("role").and_then(Value::as_str) == Some("system") {
+ return item.get("content").cloned().or(Some(Value::Null));
+ }
+ }
+ None
+}
+
+/// SHA-256 of canonical-JSON bytes of `v`, hex-encoded.
+///
+/// We use `serde_json::to_vec` rather than `to_string` to avoid an
+/// unnecessary UTF-8 validation pass; `serde_json` always emits
+/// valid UTF-8 anyway. Object keys are emitted in insertion order
+/// (workspace `serde_json` is built with `preserve_order`), which
+/// is acceptable here because the **same** request body always
+/// hashes the **same** way — and that is the only invariant we
+/// require for stable cache pinning.
+fn canonical_sha256(v: &Value) -> String {
+ let bytes = serde_json::to_vec(v).unwrap_or_default();
+ let digest = Sha256::digest(&bytes);
+ let mut out = String::with_capacity(64);
+ for byte in digest.iter() {
+ use std::fmt::Write as _;
+ let _ = write!(&mut out, "{byte:02x}");
+ }
+ out
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use serde_json::json;
+
+ fn is_hex(s: &str) -> bool {
+ s.bytes().all(|b| b.is_ascii_hexdigit())
+ }
+
+ fn injected_key(body: &Value) -> String {
+ body.get("prompt_cache_key")
+ .and_then(Value::as_str)
+ .expect("prompt_cache_key must be a string")
+ .to_string()
+ }
+
+ #[test]
+ fn injects_key_when_payg_and_absent() {
+ let mut body = json!({
+ "model": "gpt-4o",
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Hello"}
+ ],
+ });
+ let outcome = inject_prompt_cache_key(&mut body, OpenAiShape::ChatCompletions);
+ match outcome {
+ InjectOutcome::Applied { key_prefix } => {
+ assert_eq!(key_prefix.len(), KEY_PREFIX_LOG_LEN);
+ assert!(is_hex(&key_prefix));
+ }
+ other => panic!("expected Applied, got {other:?}"),
+ }
+ let key = injected_key(&body);
+ assert_eq!(key.len(), KEY_HEX_LEN);
+ assert!(is_hex(&key), "key must be hex: {key}");
+ }
+
+ #[test]
+ fn injects_key_when_payg_and_absent_responses() {
+ let mut body = json!({
+ "model": "gpt-4o",
+ "instructions": "You are a helpful assistant.",
+ "input": [{"role": "user", "content": "Hello"}],
+ });
+ let outcome = inject_prompt_cache_key(&mut body, OpenAiShape::Responses);
+ assert!(matches!(outcome, InjectOutcome::Applied { .. }));
+ let key = injected_key(&body);
+ assert_eq!(key.len(), KEY_HEX_LEN);
+ assert!(is_hex(&key));
+ }
+
+ #[test]
+ fn skips_when_key_already_set() {
+ let original = json!({
+ "model": "gpt-4o",
+ "prompt_cache_key": "user-pinned",
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Hello"}
+ ],
+ });
+ let mut body = original.clone();
+ let outcome = inject_prompt_cache_key(&mut body, OpenAiShape::ChatCompletions);
+ assert_eq!(
+ outcome,
+ InjectOutcome::Skipped {
+ reason: SkipReason::KeyPresent,
+ }
+ );
+ assert_eq!(body, original, "body must be unchanged when key is present");
+ }
+
+ #[test]
+ fn skips_when_key_already_set_responses() {
+ let original = json!({
+ "model": "gpt-4o",
+ "prompt_cache_key": "user-pinned-2",
+ "instructions": "Be concise.",
+ "input": [{"role": "user", "content": "Hello"}],
+ });
+ let mut body = original.clone();
+ let outcome = inject_prompt_cache_key(&mut body, OpenAiShape::Responses);
+ assert!(matches!(
+ outcome,
+ InjectOutcome::Skipped {
+ reason: SkipReason::KeyPresent
+ }
+ ));
+ assert_eq!(body, original);
+ }
+
+ #[test]
+ fn skips_when_body_is_not_an_object() {
+ let mut body = json!(["not", "an", "object"]);
+ let outcome = inject_prompt_cache_key(&mut body, OpenAiShape::ChatCompletions);
+ assert_eq!(
+ outcome,
+ InjectOutcome::Skipped {
+ reason: SkipReason::NotAnObject,
+ }
+ );
+ }
+
+ #[test]
+ fn empty_string_key_treated_as_absent() {
+ // Some SDKs default the field to "" when the user did not
+ // set it. We treat that as absent and inject anyway.
+ let mut body = json!({
+ "model": "gpt-4o",
+ "prompt_cache_key": "",
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Hello"}
+ ],
+ });
+ let outcome = inject_prompt_cache_key(&mut body, OpenAiShape::ChatCompletions);
+ assert!(matches!(outcome, InjectOutcome::Applied { .. }));
+ let key = injected_key(&body);
+ assert_eq!(key.len(), KEY_HEX_LEN);
+ }
+
+ #[test]
+ fn idempotent_same_inputs_same_key() {
+ // Re-running on a body whose injected key was stripped
+ // (e.g. customer-side scrub) must yield the same key.
+ let template = || {
+ json!({
+ "model": "gpt-4o",
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "first turn"}
+ ],
+ "tools": [
+ {"type": "function", "function": {"name": "search"}}
+ ]
+ })
+ };
+ let mut body1 = template();
+ let _ = inject_prompt_cache_key(&mut body1, OpenAiShape::ChatCompletions);
+ let key1 = injected_key(&body1);
+
+ let mut body2 = template();
+ let _ = inject_prompt_cache_key(&mut body2, OpenAiShape::ChatCompletions);
+ let key2 = injected_key(&body2);
+
+ assert_eq!(key1, key2);
+ }
+
+ #[test]
+ fn different_model_yields_different_key() {
+ let mut a = json!({
+ "model": "gpt-4o",
+ "messages": [{"role": "system", "content": "S"}, {"role": "user", "content": "u"}],
+ });
+ let mut b = json!({
+ "model": "gpt-4o-mini",
+ "messages": [{"role": "system", "content": "S"}, {"role": "user", "content": "u"}],
+ });
+ let _ = inject_prompt_cache_key(&mut a, OpenAiShape::ChatCompletions);
+ let _ = inject_prompt_cache_key(&mut b, OpenAiShape::ChatCompletions);
+ assert_ne!(injected_key(&a), injected_key(&b));
+ }
+
+ #[test]
+ fn different_system_yields_different_key() {
+ let mut a = json!({
+ "model": "gpt-4o",
+ "messages": [{"role": "system", "content": "system A"}, {"role": "user", "content": "u"}],
+ });
+ let mut b = json!({
+ "model": "gpt-4o",
+ "messages": [{"role": "system", "content": "system B"}, {"role": "user", "content": "u"}],
+ });
+ let _ = inject_prompt_cache_key(&mut a, OpenAiShape::ChatCompletions);
+ let _ = inject_prompt_cache_key(&mut b, OpenAiShape::ChatCompletions);
+ assert_ne!(injected_key(&a), injected_key(&b));
+ }
+
+ #[test]
+ fn different_tools_yields_different_key() {
+ let mut a = json!({
+ "model": "gpt-4o",
+ "messages": [{"role": "system", "content": "S"}, {"role": "user", "content": "u"}],
+ "tools": [{"type": "function", "function": {"name": "search"}}],
+ });
+ let mut b = json!({
+ "model": "gpt-4o",
+ "messages": [{"role": "system", "content": "S"}, {"role": "user", "content": "u"}],
+ "tools": [{"type": "function", "function": {"name": "lookup"}}],
+ });
+ let _ = inject_prompt_cache_key(&mut a, OpenAiShape::ChatCompletions);
+ let _ = inject_prompt_cache_key(&mut b, OpenAiShape::ChatCompletions);
+ assert_ne!(injected_key(&a), injected_key(&b));
+ }
+
+ #[test]
+ fn same_user_messages_different_system_yields_different_key() {
+ // Confirms user content is NOT in the hash by holding user
+ // content fixed and varying only the system. If user
+ // content were hashed, this could still differ — but the
+ // companion test below pins the inverse direction.
+ let user = json!({"role": "user", "content": "what is 2+2?"});
+ let mut a = json!({
+ "model": "gpt-4o",
+ "messages": [{"role": "system", "content": "concise"}, user.clone()],
+ });
+ let mut b = json!({
+ "model": "gpt-4o",
+ "messages": [{"role": "system", "content": "verbose"}, user],
+ });
+ let _ = inject_prompt_cache_key(&mut a, OpenAiShape::ChatCompletions);
+ let _ = inject_prompt_cache_key(&mut b, OpenAiShape::ChatCompletions);
+ assert_ne!(injected_key(&a), injected_key(&b));
+ }
+
+ #[test]
+ fn same_system_different_user_messages_yields_same_key() {
+ // The crucial property: user/assistant turns vary across
+ // requests, but the cache key must STAY CONSTANT so OpenAI
+ // can pin the cache lookup to the same identity.
+ let mut a = json!({
+ "model": "gpt-4o",
+ "messages": [
+ {"role": "system", "content": "You are concise."},
+ {"role": "user", "content": "first question"}
+ ],
+ });
+ let mut b = json!({
+ "model": "gpt-4o",
+ "messages": [
+ {"role": "system", "content": "You are concise."},
+ {"role": "user", "content": "second question, much longer than the first"},
+ {"role": "assistant", "content": "ok"},
+ {"role": "user", "content": "third question"}
+ ],
+ });
+ let _ = inject_prompt_cache_key(&mut a, OpenAiShape::ChatCompletions);
+ let _ = inject_prompt_cache_key(&mut b, OpenAiShape::ChatCompletions);
+ assert_eq!(
+ injected_key(&a),
+ injected_key(&b),
+ "user/assistant turn variation must not change the cache key"
+ );
+ }
+
+ #[test]
+ fn key_length_is_exactly_32_hex_chars() {
+ // Property test: across many input shapes the key is
+ // always exactly 32 hex chars.
+ let inputs = [
+ json!({"model": "gpt-4o", "messages": []}),
+ json!({"model": "gpt-4o-2024-08-06", "messages": [{"role": "system", "content": "abc"}]}),
+ json!({"model": "o1-mini", "messages": [{"role": "system", "content": ["array", "content"]}]}),
+ json!({"model": "", "messages": [{"role": "user", "content": "no system"}]}),
+ json!({"model": "gpt-4o", "instructions": "responses style"}),
+ ];
+ for (i, base) in inputs.iter().enumerate() {
+ let mut body = base.clone();
+ let outcome = inject_prompt_cache_key(&mut body, OpenAiShape::ChatCompletions);
+ assert!(
+ matches!(outcome, InjectOutcome::Applied { .. }),
+ "input {i}: expected Applied"
+ );
+ let key = injected_key(&body);
+ assert_eq!(
+ key.len(),
+ KEY_HEX_LEN,
+ "input {i}: key {key:?} must be {KEY_HEX_LEN} chars"
+ );
+ assert!(is_hex(&key), "input {i}: key {key:?} must be hex");
+ }
+ }
+
+ #[test]
+ fn responses_instructions_field_distinguishes_from_chat_system() {
+ // Same model, same logical "system", different shape:
+ // the keys may legally differ (different containers hash
+ // differently). What we DO assert is that running the
+ // same shape on the same body is stable (covered by
+ // idempotency test) and that running each shape produces
+ // a valid key.
+ let mut body_chat = json!({
+ "model": "gpt-4o",
+ "messages": [{"role": "system", "content": "S"}],
+ });
+ let mut body_resp = json!({
+ "model": "gpt-4o",
+ "instructions": "S",
+ });
+ let _ = inject_prompt_cache_key(&mut body_chat, OpenAiShape::ChatCompletions);
+ let _ = inject_prompt_cache_key(&mut body_resp, OpenAiShape::Responses);
+ assert_eq!(injected_key(&body_chat).len(), KEY_HEX_LEN);
+ assert_eq!(injected_key(&body_resp).len(), KEY_HEX_LEN);
+ }
+
+ #[test]
+ fn has_prompt_cache_key_works() {
+ assert!(!has_prompt_cache_key(&json!({})));
+ assert!(!has_prompt_cache_key(&json!({"prompt_cache_key": ""})));
+ assert!(!has_prompt_cache_key(&json!({"prompt_cache_key": null})));
+ assert!(has_prompt_cache_key(&json!({"prompt_cache_key": "x"})));
+ assert!(has_prompt_cache_key(
+ &json!({"prompt_cache_key": "user-pinned"})
+ ));
+ }
+
+ #[test]
+ fn skip_reason_string_is_stable() {
+ // Dashboards filter on these strings. Don't change without
+ // a deprecation note.
+ assert_eq!(SkipReason::NotAnObject.as_str(), "not_an_object");
+ assert_eq!(SkipReason::KeyPresent.as_str(), "key_present");
+ }
+}
diff --git a/crates/headroom-proxy/src/proxy.rs b/crates/headroom-proxy/src/proxy.rs
index bcd62bf58..1ae12e1ca 100644
--- a/crates/headroom-proxy/src/proxy.rs
+++ b/crates/headroom-proxy/src/proxy.rs
@@ -30,7 +30,7 @@ use crate::websocket::ws_handler;
// site self-documenting. `AuthMode` is re-exported under the same
// path for downstream handlers that read the value back out of
// `req.extensions()` (Phase F PR-F2/F3/F4).
-use headroom_core::auth_mode::classify as classify_auth_mode;
+use headroom_core::auth_mode::{classify as classify_auth_mode, AuthMode};
/// Shared state passed to every handler.
///
@@ -731,6 +731,40 @@ pub(crate) async fn forward_http(
}
};
+ // PR-E4: OpenAI `prompt_cache_key` auto-injection.
+ //
+ // Universal safety contract: only mutate when the caller
+ // is on `AuthMode::Payg`. OAuth/Subscription bytes flow
+ // through byte-equal — those clients cannot afford
+ // synthesised cache keys (OAuth scopes pin to
+ // `(account, model, session)` and subscription clients
+ // are programmatically fingerprinted by the upstream).
+ //
+ // The injector also self-skips when the customer has
+ // already set a non-empty `prompt_cache_key`. Every skip
+ // path emits a structured `e4_skipped` event so cache-hit
+ // dashboards can attribute miss rates to gating reasons
+ // rather than guessing.
+ let body_to_send = match endpoint {
+ compression::CompressibleEndpoint::OpenAiChatCompletions
+ | compression::CompressibleEndpoint::OpenAiResponses => {
+ let shape = match endpoint {
+ compression::CompressibleEndpoint::OpenAiResponses => {
+ cache_stabilization::openai_cache_key::OpenAiShape::Responses
+ }
+ _ => cache_stabilization::openai_cache_key::OpenAiShape::ChatCompletions,
+ };
+ maybe_inject_openai_prompt_cache_key(
+ body_to_send,
+ shape,
+ auth_mode,
+ &request_id,
+ &path_for_log,
+ )
+ }
+ compression::CompressibleEndpoint::AnthropicMessages => body_to_send,
+ };
+
// Forward the (Phase A: identical) buffered bytes. reqwest
// sets its own Content-Length from the body bytes — the
// existing `build_forward_request_headers` already strips
@@ -942,6 +976,119 @@ fn is_sse_response(headers: &http::HeaderMap) -> bool {
.unwrap_or(false)
}
+/// PR-E4: OpenAI `prompt_cache_key` auto-injection helper.
+///
+/// Gates on [`AuthMode::Payg`] and the in-body
+/// `prompt_cache_key` skip rule, parses the body once, mutates if
+/// appropriate, and re-serialises. Returns the original `body` on
+/// any non-applicable path — every error / skip leaves the bytes
+/// untouched (Phase A passthrough invariant).
+///
+/// Logs `e4_skipped` for each skip reason and `e4_applied` with
+/// only the first [`KEY_PREFIX_LOG_LEN`] hex chars of the key
+/// (never the full key, which is identifying material).
+///
+/// [`KEY_PREFIX_LOG_LEN`]: cache_stabilization::openai_cache_key::KEY_PREFIX_LOG_LEN
+fn maybe_inject_openai_prompt_cache_key(
+ body: bytes::Bytes,
+ shape: cache_stabilization::openai_cache_key::OpenAiShape,
+ auth_mode: AuthMode,
+ request_id: &str,
+ path: &str,
+) -> bytes::Bytes {
+ use cache_stabilization::openai_cache_key::{
+ inject_prompt_cache_key, InjectOutcome, SkipReason,
+ };
+
+ // Auth-mode gate: only PAYG bodies are eligible. OAuth /
+ // Subscription requests pass through byte-equal — synthesised
+ // cache keys would look like cache-evasion to the upstream
+ // and could void OAuth scopes pinned to `(account, model,
+ // session)`.
+ if !matches!(auth_mode, AuthMode::Payg) {
+ tracing::info!(
+ event = "e4_skipped",
+ request_id = %request_id,
+ path = %path,
+ reason = "auth_mode",
+ auth_mode = auth_mode.as_str(),
+ "PR-E4: skipped prompt_cache_key injection (non-PAYG auth mode)"
+ );
+ return body;
+ }
+
+ // Parse for the inject step. Failure here is silent — the
+ // dispatcher above already logged the parse outcome on its
+ // own decision path; we don't want to double-log. The body
+ // round-trips unchanged.
+ let mut parsed: serde_json::Value = match serde_json::from_slice(&body) {
+ Ok(v) => v,
+ Err(_) => {
+ return body;
+ }
+ };
+
+ match inject_prompt_cache_key(&mut parsed, shape) {
+ InjectOutcome::Applied { key_prefix } => {
+ // Re-serialise. If serialization fails (would be very
+ // unusual — we just successfully parsed), fall back
+ // to the original bytes. No-silent-fallback rule: log
+ // it loudly so a regression can't hide.
+ match serde_json::to_vec(&parsed) {
+ Ok(buf) => {
+ tracing::info!(
+ event = "e4_applied",
+ request_id = %request_id,
+ path = %path,
+ key_prefix = %key_prefix,
+ body_bytes_in = body.len(),
+ body_bytes_out = buf.len(),
+ "PR-E4: injected prompt_cache_key"
+ );
+ bytes::Bytes::from(buf)
+ }
+ Err(e) => {
+ tracing::error!(
+ event = "e4_serialize_error",
+ request_id = %request_id,
+ path = %path,
+ error = %e,
+ "PR-E4: re-serialize after injection failed; forwarding original bytes"
+ );
+ body
+ }
+ }
+ }
+ InjectOutcome::Skipped { reason } => {
+ // Log only the customer-visible KeyPresent skip; the
+ // NotAnObject skip is structurally impossible past
+ // the dispatcher gate but is surfaced separately for
+ // operators chasing pathological inputs.
+ match reason {
+ SkipReason::KeyPresent => {
+ tracing::info!(
+ event = "e4_skipped",
+ request_id = %request_id,
+ path = %path,
+ reason = SkipReason::KeyPresent.as_str(),
+ "PR-E4: skipped prompt_cache_key injection (customer-set value preserved)"
+ );
+ }
+ SkipReason::NotAnObject => {
+ tracing::warn!(
+ event = "e4_skipped",
+ request_id = %request_id,
+ path = %path,
+ reason = SkipReason::NotAnObject.as_str(),
+ "PR-E4: body is not a JSON object; passthrough"
+ );
+ }
+ }
+ body
+ }
+ }
+}
+
/// Drive the per-provider state machine over a stream of byte chunks.
/// Lives in its own task; the byte path never waits on it.
async fn run_sse_state_machine(
diff --git a/crates/headroom-proxy/tests/integration_chat_completions.rs b/crates/headroom-proxy/tests/integration_chat_completions.rs
index 013042a7d..0ee6d0be3 100644
--- a/crates/headroom-proxy/tests/integration_chat_completions.rs
+++ b/crates/headroom-proxy/tests/integration_chat_completions.rs
@@ -126,6 +126,14 @@ async fn passthrough_no_compression_byte_equal() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/chat/completions", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode so the prompt_cache_key auto-injection
+ // hook short-circuits and the byte-equality invariant this test
+ // pins is preserved. PAYG bodies are now mutated by E4 — see
+ // `integration_e4_openai_cache_key.rs` for that coverage.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -221,6 +229,13 @@ async fn n_greater_than_one_passthrough() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/chat/completions", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality (E4 only
+ // injects on PAYG). The n>1 skip semantics are independent
+ // of auth mode.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -256,6 +271,13 @@ async fn stream_options_include_usage_preserved() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/chat/completions", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality. The
+ // dispatcher never reads stream_options regardless of mode;
+ // E4 only mutates on PAYG.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
diff --git a/crates/headroom-proxy/tests/integration_e4_openai_cache_key.rs b/crates/headroom-proxy/tests/integration_e4_openai_cache_key.rs
new file mode 100644
index 000000000..4cda9518c
--- /dev/null
+++ b/crates/headroom-proxy/tests/integration_e4_openai_cache_key.rs
@@ -0,0 +1,396 @@
+//! Integration tests for PR-E4 OpenAI `prompt_cache_key`
+//! auto-injection.
+//!
+//! Boots a real Rust proxy in front of a wiremock upstream and
+//! exercises the four matrix points the spec calls out:
+//!
+//! 1. PAYG `/v1/chat/completions` request without `prompt_cache_key`
+//! → upstream sees an injected 32-hex-char key.
+//! 2. PAYG `/v1/responses` request without `prompt_cache_key`
+//! → upstream sees an injected key.
+//! 3. PAYG request WITH `prompt_cache_key` already set
+//! → upstream sees the customer-provided value (passthrough).
+//! 4. OAuth and Subscription requests
+//! → upstream sees byte-equal bytes (no key injection at all).
+//!
+//! For (3), passthrough means the customer's value is preserved
+//! AND the rest of the body keeps its byte shape.
+//! For (4), the body is byte-equal to the inbound bytes (the
+//! universal Phase A invariant).
+
+mod common;
+
+use common::start_proxy_with;
+use serde_json::{json, Value};
+use sha2::{Digest, Sha256};
+use std::sync::{Arc, Mutex};
+use wiremock::matchers::{method, path};
+use wiremock::{Mock, MockServer, ResponseTemplate};
+
+/// Mount a `/v1/chat/completions` capture handler that records the
+/// upstream request body for later assertions.
+async fn mount_chat_capture(upstream: &MockServer) -> Arc<Mutex<Option<Vec<u8>>>> {
+ let captured: Arc<Mutex<Option<Vec<u8>>>> = Arc::new(Mutex::new(None));
+ let captured_clone = captured.clone();
+ Mock::given(method("POST"))
+ .and(path("/v1/chat/completions"))
+ .respond_with(move |req: &wiremock::Request| {
+ *captured_clone.lock().unwrap() = Some(req.body.clone());
+ ResponseTemplate::new(200).set_body_string(r#"{"ok":true}"#)
+ })
+ .mount(upstream)
+ .await;
+ captured
+}
+
+/// Mount a `/v1/responses` capture handler.
+async fn mount_responses_capture(upstream: &MockServer) -> Arc<Mutex<Option<Vec<u8>>>> {
+ let captured: Arc<Mutex<Option<Vec<u8>>>> = Arc::new(Mutex::new(None));
+ let captured_clone = captured.clone();
+ Mock::given(method("POST"))
+ .and(path("/v1/responses"))
+ .respond_with(move |req: &wiremock::Request| {
+ *captured_clone.lock().unwrap() = Some(req.body.clone());
+ ResponseTemplate::new(200).set_body_string(r#"{"ok":true}"#)
+ })
+ .mount(upstream)
+ .await;
+ captured
+}
+
+fn sha256_hex(bytes: &[u8]) -> String {
+ let mut hasher = Sha256::new();
+ hasher.update(bytes);
+ hasher
+ .finalize()
+ .iter()
+ .fold(String::with_capacity(64), |mut acc, b| {
+ use std::fmt::Write as _;
+ let _ = write!(acc, "{b:02x}");
+ acc
+ })
+}
+
+#[track_caller]
+fn assert_byte_equal(inbound: &[u8], received: &[u8]) {
+ assert_eq!(
+ inbound.len(),
+ received.len(),
+ "byte length mismatch: inbound={} received={}",
+ inbound.len(),
+ received.len(),
+ );
+ assert_eq!(
+ sha256_hex(inbound),
+ sha256_hex(received),
+ "SHA-256 mismatch — request was mutated",
+ );
+}
+
+fn captured_body(captured: &Arc<Mutex<Option<Vec<u8>>>>) -> Vec<u8> {
+ captured
+ .lock()
+ .unwrap()
+ .clone()
+ .expect("upstream should have captured a body")
+}
+
+fn parse_json(bytes: &[u8]) -> Value {
+ serde_json::from_slice(bytes).expect("upstream body should be valid JSON")
+}
+
+// ────────────────────────────────────────────────────────────────────
+// PAYG chat completions: key injected
+// ────────────────────────────────────────────────────────────────────
+
+#[tokio::test]
+async fn payg_chat_completions_injects_prompt_cache_key() {
+ let upstream = MockServer::start().await;
+ let captured = mount_chat_capture(&upstream).await;
+ let proxy = start_proxy_with(&upstream.uri(), |c| {
+ c.compression = true;
+ })
+ .await;
+
+ let payload = json!({
+ "model": "gpt-4o",
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Hello"}
+ ]
+ });
+ let body = serde_json::to_vec(&payload).unwrap();
+ let resp = reqwest::Client::new()
+ .post(format!("{}/v1/chat/completions", proxy.url()))
+ .header("content-type", "application/json")
+ // PAYG: standard OpenAI sk- bearer token.
+ .header("authorization", "Bearer sk-test-payg-key")
+ .body(body.clone())
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(resp.status(), 200);
+
+ let received = captured_body(&captured);
+ let received_json = parse_json(&received);
+
+ let key = received_json
+ .get("prompt_cache_key")
+ .and_then(Value::as_str)
+ .expect("upstream body should have prompt_cache_key on PAYG");
+ assert_eq!(key.len(), 32, "key must be 32 hex chars, got {key:?}");
+ assert!(
+ key.bytes().all(|b| b.is_ascii_hexdigit()),
+ "key must be hex: {key}"
+ );
+
+ // Other fields preserved structurally (parse-equivalent).
+ assert_eq!(
+ received_json.get("model").and_then(Value::as_str),
+ Some("gpt-4o")
+ );
+ let inbound_json = parse_json(&body);
+ assert_eq!(received_json.get("messages"), inbound_json.get("messages"));
+
+ proxy.shutdown().await;
+}
+
+// ────────────────────────────────────────────────────────────────────
+// PAYG responses: key injected
+// ────────────────────────────────────────────────────────────────────
+
+#[tokio::test]
+async fn payg_responses_injects_prompt_cache_key() {
+ let upstream = MockServer::start().await;
+ let captured = mount_responses_capture(&upstream).await;
+ let proxy = start_proxy_with(&upstream.uri(), |c| {
+ c.compression = true;
+ })
+ .await;
+
+ let payload = json!({
+ "model": "gpt-4o",
+ "instructions": "You are a helpful assistant.",
+ "input": [{"role": "user", "content": "Hello"}]
+ });
+ let body = serde_json::to_vec(&payload).unwrap();
+ let resp = reqwest::Client::new()
+ .post(format!("{}/v1/responses", proxy.url()))
+ .header("content-type", "application/json")
+ .header("authorization", "Bearer sk-test-payg-key")
+ .body(body.clone())
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(resp.status(), 200);
+
+ let received = captured_body(&captured);
+ let received_json = parse_json(&received);
+
+ let key = received_json
+ .get("prompt_cache_key")
+ .and_then(Value::as_str)
+ .expect("upstream body should have prompt_cache_key on PAYG /v1/responses");
+ assert_eq!(key.len(), 32, "key must be 32 hex chars, got {key:?}");
+
+ // `instructions` and `input` preserved.
+ assert_eq!(
+ received_json.get("instructions").and_then(Value::as_str),
+ Some("You are a helpful assistant.")
+ );
+ let inbound_json = parse_json(&body);
+ assert_eq!(received_json.get("input"), inbound_json.get("input"));
+
+ proxy.shutdown().await;
+}
+
+// ────────────────────────────────────────────────────────────────────
+// PAYG with customer-set key: customer wins (passthrough)
+// ────────────────────────────────────────────────────────────────────
+
+#[tokio::test]
+async fn payg_chat_with_customer_set_key_preserves_value() {
+ let upstream = MockServer::start().await;
+ let captured = mount_chat_capture(&upstream).await;
+ let proxy = start_proxy_with(&upstream.uri(), |c| {
+ c.compression = true;
+ })
+ .await;
+
+ let payload = json!({
+ "model": "gpt-4o",
+ "prompt_cache_key": "user-pinned-tenant-A",
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Hello"}
+ ]
+ });
+ let body = serde_json::to_vec(&payload).unwrap();
+ let resp = reqwest::Client::new()
+ .post(format!("{}/v1/chat/completions", proxy.url()))
+ .header("content-type", "application/json")
+ .header("authorization", "Bearer sk-test-payg-key")
+ .body(body.clone())
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(resp.status(), 200);
+
+ let received = captured_body(&captured);
+ // The dispatcher's live-zone surgery must NOT have rewritten
+ // the body (the live-zone is the latest user/tool message,
+ // which here is below the byte threshold). And the E4 hook
+ // must have skipped (customer key present). Net effect: bytes
+ // round-trip byte-equal to inbound.
+ assert_byte_equal(&body, &received);
+
+ let received_json = parse_json(&received);
+ assert_eq!(
+ received_json
+ .get("prompt_cache_key")
+ .and_then(Value::as_str),
+ Some("user-pinned-tenant-A"),
+ "customer-set prompt_cache_key must be preserved"
+ );
+
+ proxy.shutdown().await;
+}
+
+// ────────────────────────────────────────────────────────────────────
+// OAuth: no injection, byte-equal passthrough
+// ────────────────────────────────────────────────────────────────────
+
+#[tokio::test]
+async fn oauth_chat_completions_no_injection_byte_equal() {
+ let upstream = MockServer::start().await;
+ let captured = mount_chat_capture(&upstream).await;
+ let proxy = start_proxy_with(&upstream.uri(), |c| {
+ c.compression = true;
+ })
+ .await;
+
+ let payload = json!({
+ "model": "gpt-4o",
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Hello"}
+ ]
+ });
+ let body = serde_json::to_vec(&payload).unwrap();
+ // OAuth: 3-segment JWT shape. Headroom's auth_mode classifier
+ // detects this and routes to AuthMode::OAuth, which the E4
+ // hook short-circuits.
+ let resp = reqwest::Client::new()
+ .post(format!("{}/v1/chat/completions", proxy.url()))
+ .header("content-type", "application/json")
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
+ .body(body.clone())
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(resp.status(), 200);
+
+ let received = captured_body(&captured);
+ assert_byte_equal(&body, &received);
+
+ let received_json = parse_json(&received);
+ assert!(
+ received_json.get("prompt_cache_key").is_none(),
+ "OAuth requests must not get prompt_cache_key injected; got {received_json}"
+ );
+
+ proxy.shutdown().await;
+}
+
+// ────────────────────────────────────────────────────────────────────
+// Subscription: no injection, byte-equal passthrough
+// ────────────────────────────────────────────────────────────────────
+
+#[tokio::test]
+async fn subscription_chat_completions_no_injection_byte_equal() {
+ let upstream = MockServer::start().await;
+ let captured = mount_chat_capture(&upstream).await;
+ let proxy = start_proxy_with(&upstream.uri(), |c| {
+ c.compression = true;
+ })
+ .await;
+
+ let payload = json!({
+ "model": "gpt-4o",
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Hello"}
+ ]
+ });
+ let body = serde_json::to_vec(&payload).unwrap();
+ // Subscription clients: identified by user-agent prefix
+ // (Codex CLI on `/v1/chat/completions` is the canonical case).
+ let resp = reqwest::Client::new()
+ .post(format!("{}/v1/chat/completions", proxy.url()))
+ .header("content-type", "application/json")
+ .header("authorization", "Bearer sk-test-payg-key")
+ .header("user-agent", "codex-cli/1.0.0")
+ .body(body.clone())
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(resp.status(), 200);
+
+ let received = captured_body(&captured);
+ assert_byte_equal(&body, &received);
+
+ let received_json = parse_json(&received);
+ assert!(
+ received_json.get("prompt_cache_key").is_none(),
+ "Subscription requests must not get prompt_cache_key injected"
+ );
+
+ proxy.shutdown().await;
+}
+
+// ────────────────────────────────────────────────────────────────────
+// OAuth on /v1/responses: no injection, byte-equal passthrough
+// ────────────────────────────────────────────────────────────────────
+
+#[tokio::test]
+async fn oauth_responses_no_injection_byte_equal() {
+ let upstream = MockServer::start().await;
+ let captured = mount_responses_capture(&upstream).await;
+ let proxy = start_proxy_with(&upstream.uri(), |c| {
+ c.compression = true;
+ })
+ .await;
+
+ let payload = json!({
+ "model": "gpt-4o",
+ "instructions": "Be concise.",
+ "input": [{"role": "user", "content": "Hello"}]
+ });
+ let body = serde_json::to_vec(&payload).unwrap();
+ let resp = reqwest::Client::new()
+ .post(format!("{}/v1/responses", proxy.url()))
+ .header("content-type", "application/json")
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
+ .body(body.clone())
+ .send()
+ .await
+ .unwrap();
+ assert_eq!(resp.status(), 200);
+
+ let received = captured_body(&captured);
+ assert_byte_equal(&body, &received);
+
+ let received_json = parse_json(&received);
+ assert!(
+ received_json.get("prompt_cache_key").is_none(),
+ "OAuth /v1/responses requests must not get prompt_cache_key injected"
+ );
+
+ proxy.shutdown().await;
+}
diff --git a/crates/headroom-proxy/tests/integration_responses.rs b/crates/headroom-proxy/tests/integration_responses.rs
index 04dc52835..6ef67d57f 100644
--- a/crates/headroom-proxy/tests/integration_responses.rs
+++ b/crates/headroom-proxy/tests/integration_responses.rs
@@ -103,6 +103,14 @@ async fn v4a_patch_byte_equal_through_proxy() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -147,6 +155,14 @@ async fn local_shell_call_command_argv_array_preserved() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -190,6 +206,14 @@ async fn codex_phase_commentary_preserved() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -228,6 +252,14 @@ async fn codex_phase_final_answer_preserved() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -264,6 +296,14 @@ async fn compaction_item_byte_equal() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -296,6 +336,14 @@ async fn reasoning_encrypted_content_byte_equal() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -336,6 +384,14 @@ async fn function_call_arguments_string_preserved() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -386,6 +442,14 @@ async fn call_id_referenced_not_id() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -431,6 +495,14 @@ async fn apply_patch_output_below_2kb_no_compression() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -477,6 +549,14 @@ async fn apply_patch_output_above_2kb_compressed() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -538,6 +618,14 @@ async fn local_shell_output_compressed() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -588,6 +676,14 @@ async fn mcp_tool_call_byte_equal() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -623,6 +719,14 @@ async fn computer_call_byte_equal() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -663,6 +767,14 @@ async fn image_generation_call_no_log_redaction_in_test_mode() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -712,6 +824,14 @@ async fn unknown_item_type_logged_warning_byte_equal() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -768,6 +888,14 @@ async fn representative_request_round_trip() {
let resp = reqwest::Client::new()
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
+ // PR-E4: OAuth auth mode preserves byte-equality across the
+ // proxy (E4 only injects prompt_cache_key on PAYG). These
+ // dispatcher byte-fidelity tests pin the live-zone surgery,
+ // independent of the E4 cache-stabilization hook.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
diff --git a/crates/headroom-proxy/tests/integration_responses_streaming.rs b/crates/headroom-proxy/tests/integration_responses_streaming.rs
index 58964f26f..27c104ffe 100644
--- a/crates/headroom-proxy/tests/integration_responses_streaming.rs
+++ b/crates/headroom-proxy/tests/integration_responses_streaming.rs
@@ -192,6 +192,13 @@ async fn streaming_request_bytes_byte_equal_upstream() {
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
.header("accept", "text/event-stream")
+ // PR-E4: OAuth auth mode preserves byte-equality (E4 only
+ // injects prompt_cache_key on PAYG). These tests pin the
+ // streaming-side byte fidelity, independent of E4.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await
@@ -229,6 +236,13 @@ async fn streaming_response_round_trips_through_framer() {
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
.header("accept", "text/event-stream")
+ // PR-E4: OAuth auth mode preserves byte-equality (E4 only
+ // injects prompt_cache_key on PAYG). These tests pin the
+ // streaming-side byte fidelity, independent of E4.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body)
.send()
.await
@@ -297,6 +311,13 @@ async fn streaming_pipeline_disabled_still_passes_bytes() {
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
.header("accept", "text/event-stream")
+ // PR-E4: OAuth auth mode preserves byte-equality (E4 only
+ // injects prompt_cache_key on PAYG). These tests pin the
+ // streaming-side byte fidelity, independent of E4.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body)
.send()
.await
@@ -344,6 +365,13 @@ async fn streaming_request_no_compression_when_input_below_threshold() {
.post(format!("{}/v1/responses", proxy.url()))
.header("content-type", "application/json")
.header("accept", "text/event-stream")
+ // PR-E4: OAuth auth mode preserves byte-equality (E4 only
+ // injects prompt_cache_key on PAYG). These tests pin the
+ // streaming-side byte fidelity, independent of E4.
+ .header(
+ "authorization",
+ "Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0.signature_bytes",
+ )
.body(body.clone())
.send()
.await