Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
fix(proxy): key drift detector on conversations, not credentials; canonicalize drift hashes (#2301)
Andrei Boldyrev6744833afed9
diff --git a/crates/headroom-proxy/src/cache_stabilization/drift_detector.rs b/crates/headroom-proxy/src/cache_stabilization/drift_detector.rsindex 1302ad3a4..c288d3131 100644--- a/crates/headroom-proxy/src/cache_stabilization/drift_detector.rs+++ b/crates/headroom-proxy/src/cache_stabilization/drift_detector.rs@@ -9,31 +9,63 @@//! `body.system`; OpenAI Chat first `role=system` message;//! OpenAI Responses `body.instructions`).//! - `tools` — SHA-256 of the canonical bytes of `body.tools`.-//! - `early_messages` — SHA-256 of the canonical bytes of the first 3-//! message-shaped items (or all, if fewer than 3). Skips the+//! - `early_messages` — per-message SHA-256 of the first 3+//! conversation messages (or all, if fewer than 3). Skips the//! live-zone tail where mutation is expected and benign.//!-//! Track the previous hash per session in a bounded LRU. When a+//! All axes are canonicalized before hashing ([`canonicalize_for_hash`]):+//! objects are rebuilt with sorted keys (whitespace- and key-order-+//! neutral — the workspace's `preserve_order` feature would otherwise+//! keep wire order in the hash) and `cache_control` members are+//! stripped outside opaque tool payloads — clients relocate cache+//! breakpoints to the newest block every turn, and moving a breakpoint+//! never invalidates a previously cached prefix, so markers are+//! placement metadata rather than structure.+//!+//! Track the previous fingerprint per session in a bounded LRU. When a//! subsequent request on the same session disagrees on any dimension,//! emit a `cache_drift_observed` log line listing the drifted-//! dimensions. **Never mutates the request body** — the detector is a-//! pure observer and the proxy's "passthrough is sacred" invariant-//! (Phase A) is preserved by construction.+//! dimensions. The `early_messages` comparison is prefix-aware: a+//! conversation *growing into* the window (turn 2 appends messages+//! after turn 1's) is append-only and benign, while a settled early+//! message that changes or disappears busted the provider's prefix+//! cache and is drift. **Never mutates the request body** — the+//! detector is a pure observer and the proxy's "passthrough is sacred"+//! invariant (Phase A) is preserved by construction.+//!+//! Known trade-off: without an explicit `x-headroom-session-id`, the+//! session identity is anchored on the conversation's first message+//! (see [`conversation_discriminator`]), so a client that *rewrites*+//! that message (history compaction, rolling-window truncation)+//! re-keys to a fresh session and the rewrite surfaces as+//! `cache_drift_first_request` rather than `cache_drift_observed` —+//! traded deliberately against the credential-keyed alternative, which+//! false-warned on every conversation switch. With the explicit header+//! the identity is pinned and rewrites are reported as drift.//!//! # Privacy//!-//! The session key is derived from the strongest available client-//! identifier (`Authorization`, `x-api-key`, client IP, finally-//! `(client_ip, user_agent)`). Bearer tokens and API keys are-//! **hashed before they ever leave this module**; the raw secret is-//! never logged, never stored, and is overwritten in transit (truncated-//! to a 16-character hex prefix). The log line itself only includes a-//! short prefix of the SHA-256 hex of the session key.+//! The session key prefers the client's explicit+//! `x-headroom-session-id` header — the same opt-in the Python proxy+//! honors for all session-sticky state — and otherwise combines the+//! strongest available client identifier (`Authorization`, `x-api-key`,+//! client IP, finally `(client_ip, user_agent)`) with a fingerprint of+//! the conversation's first message, so concurrent conversations that+//! share one credential do not share one drift session. Bearer tokens,+//! API keys, and the session-id header value are **hashed before they+//! ever leave this module**; the raw secret is never logged and never+//! stored. The conversation fingerprint is likewise a truncated SHA-256+//! — no message content appears in the key. The log line itself only+//! includes a short prefix of the SHA-256 hex of the session key.//!//! # Cost//!-//! - One SHA-256 update over each of (system, tools, early messages).-//! Total ~200us on a 8 KB system prompt.+//! - Up to six SHA-256 digests per request (system, tools, up to+//! three early messages, and the session key's conversation+//! fingerprint), each over a canonicalized clone (filtered,+//! key-sorted rebuild) of the corresponding subtree. Still well+//! under a millisecond on a typical agentic request; the detector+//! stays log-only and off the forwarded-bytes path.//! - One LRU lookup + insert. `lru = "0.12"` is O(1) amortised.//! - One `tracing::info!` or `tracing::warn!`. No metric emission yet//! (left for Phase F PR-F* when the global Prometheus registry can@@ -66,22 +98,27 @@ pub enum ApiKind {/// Three-axis structural fingerprint of the cache hot zone.///-/// Each axis is the SHA-256 of the canonical bytes at that position-/// (we re-serialize via `serde_json::to_vec` so whitespace and key-/// order through the original network bytes do not perturb the hash).-/// All three are required for "no drift"; any one differing flags-/// drift on that dimension.+/// Each axis is the SHA-256 of canonical bytes at that position (see+/// [`canonicalize_for_hash`]): all three axes must be *stable* for+/// "no drift", and each drifting axis is named in the emitted event.+///+/// `early_messages` holds one hash per settled slot of the early+/// window (`None` = the conversation hasn't grown that far yet), so+/// the comparison can tell "grew into the window" (benign) apart from+/// "a settled message changed" (drift). Note that the derived `==` is+/// stricter than the drift predicate — growth compares unequal but is+/// not drift; use [`drift_dims`]'s emptiness for drift decisions.#[derive(Debug, Clone, Copy, PartialEq, Eq)]pub struct StructuralHash {pub system: [u8; 32],pub tools: [u8; 32],- pub early_messages: [u8; 32],+ pub early_messages: [Option<[u8; 32]>; EARLY_MESSAGES_WINDOW],}/// How many message-shaped items count as the "early" prefix that-/// feeds `early_messages_hash`. Anything past this is the live zone+/// feeds [`early_message_hashes`]. Anything past this is the live zone/// (where mutation is expected; we deliberately ignore it).-const EARLY_MESSAGES_WINDOW: usize = 3;+pub const EARLY_MESSAGES_WINDOW: usize = 3;/// Compute a [`StructuralHash`] for the body shape implied by `kind`.///@@ -89,9 +126,9 @@ const EARLY_MESSAGES_WINDOW: usize = 3;/// `does_not_mutate_input` test in the module below pins this with a/// clone-and-compare assertion.pub fn compute_structural_hash(body: &serde_json::Value, kind: ApiKind) -> StructuralHash {- let system = hash_value(&extract_system(body, kind));- let tools = hash_value(&extract_tools(body));- let early_messages = hash_value(&extract_early_messages(body, kind));+ let system = hash_value(&canonicalize_for_hash(&extract_system(body, kind), false));+ let tools = hash_value(&canonicalize_for_hash(&extract_tools(body), false));+ let early_messages = early_message_hashes(body, kind);StructuralHash {system,tools,@@ -99,6 +136,60 @@ pub fn compute_structural_hash(body: &serde_json::Value, kind: ApiKind) -> Struc}}+/// Keys whose values are opaque user payloads: tool-call arguments and+/// schemas. Never strip inside them — a user field that happens to be+/// named `cache_control` is data, and dropping it would make two+/// genuinely different payloads hash identically, masking real drift.+/// `input`/`arguments`/`json` mirror the Python canonicalizer's+/// `_OPAQUE_PAYLOAD_KEYS`; `input_schema` extends the same rule to+/// tool definitions, which the Python comparator never hashes but the+/// `tools` axis here does.+const OPAQUE_PAYLOAD_KEYS: [&str; 4] = ["input", "arguments", "json", "input_schema"];++/// Canonicalize a JSON tree for hashing: rebuild every object with+/// sorted keys, dropping `cache_control` members outside opaque+/// payloads.+///+/// Key sorting is what makes the hashes genuinely key-order neutral:+/// this workspace builds `serde_json` with `preserve_order`, so a+/// plain re-serialize would keep the client's wire order and a+/// serializer-side reordering would read as drift (and would rotate+/// the conversation fingerprint). Sorting is a pure reordering — no+/// information is lost, so distinct payloads never conflate.+///+/// Cache-breakpoint markers are placement metadata, not structure:+/// clients relocate them to the newest block every turn (observed live+/// from Claude Code), and moving a breakpoint never invalidates a+/// previously cached prefix. Hashing them would flag drift on every+/// relocation.+fn canonicalize_for_hash(value: &serde_json::Value, in_opaque: bool) -> serde_json::Value {+ match value {+ serde_json::Value::Object(map) => {+ let mut entries: Vec<(&String, &serde_json::Value)> = map+ .iter()+ .filter(|(k, _)| in_opaque || k.as_str() != "cache_control")+ .collect();+ entries.sort_by_key(|(k, _)| k.as_str());+ serde_json::Value::Object(+ entries+ .into_iter()+ .map(|(k, v)| {+ let opaque = in_opaque || OPAQUE_PAYLOAD_KEYS.contains(&k.as_str());+ (k.clone(), canonicalize_for_hash(v, opaque))+ })+ .collect(),+ )+ }+ serde_json::Value::Array(items) => serde_json::Value::Array(+ items+ .iter()+ .map(|v| canonicalize_for_hash(v, in_opaque))+ .collect(),+ ),+ other => other.clone(),+ }+}+/// Extract the "system" axis as a `serde_json::Value`. Returns/// `Value::Null` when the dimension is absent — Null still hashes to/// a stable 32-byte digest so first-request comparisons are@@ -142,38 +233,54 @@ fn extract_tools(body: &serde_json::Value) -> serde_json::Value {.unwrap_or(serde_json::Value::Null)}-/// Extract the first [`EARLY_MESSAGES_WINDOW`] message-shaped items-/// as an array `Value`. Skips the system message in the OpenAI Chat-/// shape (the system axis already hashes that separately).-fn extract_early_messages(body: &serde_json::Value, kind: ApiKind) -> serde_json::Value {+/// Collect the conversation-scoped message items for `kind`: Anthropic+/// `messages`, OpenAI Chat `messages` minus `role:"system"` entries+/// (the system axis already hashes those separately), OpenAI Responses+/// `input`. A Responses string-form `input` counts as one item.+fn conversation_messages(body: &serde_json::Value, kind: ApiKind) -> Vec<&serde_json::Value> {let array_key = match kind {ApiKind::Anthropic => "messages",ApiKind::OpenAiChat => "messages",ApiKind::OpenAiResponses => "input",};- let messages = match body.get(array_key).and_then(|v| v.as_array()) {- Some(arr) => arr,- None => return serde_json::Value::Null,+ let items = match (kind, body.get(array_key)) {+ (_, Some(serde_json::Value::Array(arr))) => arr.iter().collect::<Vec<_>>(),+ // `input` may be a bare string in the Responses API. The other+ // shapes only accept arrays; anything else is malformed and+ // will be rejected upstream, so contribute nothing here.+ (ApiKind::OpenAiResponses, Some(s @ serde_json::Value::String(_))) => vec![s],+ _ => return Vec::new(),};- let early: Vec<serde_json::Value> = match kind {- ApiKind::OpenAiChat => messages- .iter()+ match kind {+ ApiKind::OpenAiChat => items+ .into_iter().filter(|m| {m.get("role").and_then(|r| r.as_str()).map(|s| s != "system").unwrap_or(true)})- .take(EARLY_MESSAGES_WINDOW)- .cloned()- .collect(),- _ => messages- .iter()- .take(EARLY_MESSAGES_WINDOW)- .cloned().collect(),- };- serde_json::Value::Array(early)+ _ => items,+ }+}++/// Hash each of the first [`EARLY_MESSAGES_WINDOW`] conversation+/// messages individually (canonicalized). Slots the conversation has+/// not grown into yet stay `None`.+fn early_message_hashes(+ body: &serde_json::Value,+ kind: ApiKind,+) -> [Option<[u8; 32]>; EARLY_MESSAGES_WINDOW] {+ let mut out = [None; EARLY_MESSAGES_WINDOW];+ for (slot, msg) in conversation_messages(body, kind)+ .into_iter()+ .take(EARLY_MESSAGES_WINDOW)+ .enumerate()+ {+ out[slot] = Some(hash_value(&canonicalize_for_hash(msg, false)));+ }+ out}/// SHA-256 over `serde_json::to_vec(value)`. Re-serializing the@@ -238,8 +345,9 @@ impl std::fmt::Debug for DriftState {/// - First time a session is seen → `tracing::info!(event =/// "cache_drift_first_request", …)` with a 16-char prefix of the/// SHA-256 hex of `session_key`.-/// - Subsequent requests with all three hashes equal → no event.-/// - Subsequent requests with any dimension differing →+/// - Subsequent requests with no drifted dimension (append-only+/// growth into the early window included) → no event.+/// - Subsequent requests with any dimension drifting →/// `tracing::warn!(event = "cache_drift_observed", drift_dims =/// "<comma-joined>", previous_hash_prefix, current_hash_prefix, …)`.pub fn observe_drift(state: &DriftState, session_key: &str, current: StructuralHash) {@@ -268,21 +376,23 @@ pub fn observe_drift(state: &DriftState, session_key: &str, current: StructuralH);cache.put(session_key.to_string(), current);}- Some(previous) if previous == current => {- // Stable. No event. Update LRU recency by reinserting.- cache.put(session_key.to_string(), current);- }Some(previous) => {let dims = drift_dims(&previous, ¤t);- tracing::warn!(- event = "cache_drift_observed",- session_key_hash = %session_prefix,- drift_dims = %dims,- previous_hash_prefix = %structural_hash_log_prefix(&previous),- current_hash_prefix = %structural_hash_log_prefix(¤t),- "cache_drift detector observed structural change between turns of the same session"- );- cache.put(session_key.to_string(), current);+ if dims.is_empty() {+ // Stable (append-only growth included). No event.+ // Update LRU recency by reinserting.+ cache.put(session_key.to_string(), current);+ } else {+ tracing::warn!(+ event = "cache_drift_observed",+ session_key_hash = %session_prefix,+ drift_dims = %dims,+ previous_hash_prefix = %structural_hash_log_prefix(&previous),+ current_hash_prefix = %structural_hash_log_prefix(¤t),+ "cache_drift detector observed structural change between turns of the same session"+ );+ cache.put(session_key.to_string(), current);+ }}}}@@ -297,14 +407,24 @@ fn session_key_log_prefix(session_key: &str) -> String {hex_prefix(&digest, 16)}-/// 12-char hex prefix of the concatenated structural hash. Useful as-/// a compact "did the prefix change" indicator in logs without-/// printing the entire 96-char digest tuple.+/// 24-char hex prefix (12 bytes) of a digest over the concatenated+/// axis hashes. Useful as a compact "did the prefix change" indicator+/// in logs without printing every axis digest in full.fn structural_hash_log_prefix(hash: &StructuralHash) -> String {let mut hasher = Sha256::new();hasher.update(hash.system);hasher.update(hash.tools);- hasher.update(hash.early_messages);+ for slot in &hash.early_messages {+ // Length-prefix the slots so `[Some(h), None]` and `[None,+ // Some(h)]` cannot collide.+ match slot {+ Some(h) => {+ hasher.update([1u8]);+ hasher.update(h);+ }+ None => hasher.update([0u8]),+ }+ }let digest = hasher.finalize();hex_prefix(&digest, 12)}@@ -334,35 +454,75 @@ fn drift_dims(prev: &StructuralHash, curr: &StructuralHash) -> String {if prev.tools != curr.tools {dims.push("tools");}- if prev.early_messages != curr.early_messages {+ if early_window_drifted(&prev.early_messages, &curr.early_messages) {dims.push("early_messages");}dims.join(",")}-/// Derive a stable per-session key from the request headers and-/// client address. Priority order:+/// Prefix-aware early-window comparison. A settled slot changing or+/// disappearing is drift (the previously sent prefix was rewritten —+/// the provider's cache is busted); the conversation growing into a+/// previously empty slot is append-only and benign.+fn early_window_drifted(+ prev: &[Option<[u8; 32]>; EARLY_MESSAGES_WINDOW],+ curr: &[Option<[u8; 32]>; EARLY_MESSAGES_WINDOW],+) -> bool {+ prev.iter().zip(curr.iter()).any(|slots| match slots {+ (Some(p), Some(c)) => p != c,+ (Some(_), None) => true,+ (None, _) => false,+ })+}++/// Derive a stable per-session key from the request headers, client+/// address, and body. Priority order:///-/// 1. `Authorization` header (hashed; never logged raw).-/// 2. `x-api-key` header (hashed; never logged raw).-/// 3. Client IP address.-/// 4. `(client_ip, user_agent)` synthetic tuple — the user-agent+/// 1. `x-headroom-session-id` header (hashed) — the explicit opt-in+/// the Python proxy already honors for every session-sticky+/// subsystem (prefix tracker, beta-header tracker). When the+/// client declares its session, believe it.+/// 2. `Authorization` header (hashed; never logged raw).+/// 3. `x-api-key` header (hashed; never logged raw).+/// 4. Client IP address.+/// 5. `(client_ip, user_agent)` synthetic tuple — the user-agent/// bucketization gives us *some* discrimination when many/// anonymous clients sit behind the same NAT.///+/// Arms 2–5 identify a *tenant*, not a conversation: interactive+/// clients (Claude Code, Codex CLI) send the same bearer for every+/// concurrent conversation. Each of those arms therefore also folds in+/// [`conversation_discriminator`] — a fingerprint of the+/// conversation's first message — so parallel conversations do not+/// alternate over one LRU slot and log false `cache_drift_observed`+/// events on every switch.+////// The returned string is opaque; never log it directly. Callers/// should pass it straight to [`observe_drift`], which logs only a/// hashed prefix.-pub fn derive_session_key(headers: &HeaderMap, client_addr: &SocketAddr) -> String {+pub fn derive_session_key(+ headers: &HeaderMap,+ client_addr: &SocketAddr,+ body: &serde_json::Value,+ kind: ApiKind,+) -> String {+ if let Some(sid) = headers+ .get("x-headroom-session-id")+ .and_then(|v| v.to_str().ok())+ .filter(|s| !s.is_empty())+ {+ return format!("session:{}", hash_secret(sid));+ }+ let conv = conversation_discriminator(body, kind);if let Some(token) = headers.get(axum::http::header::AUTHORIZATION).and_then(|v| v.to_str().ok()){- return format!("auth:{}", hash_secret(token));+ return format!("auth:{}:{conv}", hash_secret(token));}// `x-api-key` is the Anthropic/OpenAI-Responses convention.if let Some(key) = headers.get("x-api-key").and_then(|v| v.to_str().ok()) {- return format!("apikey:{}", hash_secret(key));+ return format!("apikey:{}:{conv}", hash_secret(key));}let ip = client_addr.ip().to_string();if let Some(ua) = headers@@ -375,9 +535,55 @@ pub fn derive_session_key(headers: &HeaderMap, client_addr: &SocketAddr) -> Strilet mut h = DefaultHasher::new();ip.hash(&mut h);ua.hash(&mut h);- return format!("ipua:{:016x}", h.finish());+ return format!("ipua:{:016x}:{conv}", h.finish());+ }+ format!("ip:{ip}:{conv}")+}++/// 16-hex-char fingerprint of `(model, first conversation message)`,+/// the message canonicalized via [`canonicalize_for_hash`] so a+/// relocated `cache_control` marker does not rotate the conversation's+/// identity between turns (interactive clients resend the history each+/// turn with the opener otherwise byte-stable). `-` when the body+/// carries no conversation messages.+///+/// The model is folded in because provider prompt caches are+/// per-model: an auxiliary small-model call that reuses a+/// conversation's opener (title generation, summarization sidecars)+/// must not share the conversation's drift baseline, and a+/// mid-conversation model switch genuinely starts a new provider+/// cache lineage.+///+/// Deliberately excludes the system prompt and tools: those are the+/// *measured* axes, and agentic clients legitimately mutate them+/// mid-conversation. An identity built from mutating content would+/// rotate exactly when the detector should be reporting drift instead.+///+/// Known trade-offs: a client that *rewrites* its first message+/// (history compaction, rolling-window truncation, Responses chained+/// mode sending delta-only `input`) re-keys to a fresh session — the+/// rewrite surfaces as `cache_drift_first_request` on the new key+/// rather than `cache_drift_observed` against the old baseline. An+/// explicit `x-headroom-session-id` pins the identity and reports+/// those rewrites as drift. Conversations sharing one credential AND+/// a byte-identical opener on the same model still conflate.+fn conversation_discriminator(body: &serde_json::Value, kind: ApiKind) -> String {+ let model = body.get("model").and_then(|m| m.as_str()).unwrap_or("");+ match conversation_messages(body, kind).first() {+ Some(first) => {+ let canonical = canonicalize_for_hash(first, false);+ let mut hasher = Sha256::new();+ hasher.update(model.as_bytes());+ // NUL separator: domain-separate the model from the+ // message bytes so no (model, message) pair can alias+ // another by shifting bytes across the boundary.+ hasher.update([0u8]);+ hasher.update(serde_json::to_vec(&canonical).unwrap_or_default());+ let digest = hasher.finalize();+ hex_prefix(&digest, 8)+ }+ None => "-".to_string(),}- format!("ip:{ip}")}/// SHA-256 of `secret`, truncated to 16 hex characters. Sufficient@@ -556,7 +762,7 @@ mod tests {.unwrap(),);let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 1)), 1234);- let key = derive_session_key(&headers, &addr);+ let key = derive_session_key(&headers, &addr, &json!({}), ApiKind::Anthropic);// The key MUST NOT contain the raw bearer string anywhere —// not the secret token, not the literal "Bearer", not even// any 8+ char substring of the secret.@@ -591,7 +797,7 @@ mod tests {"sk-very-private-api-key-12345".parse().unwrap(),);let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 2)), 1234);- let key = derive_session_key(&headers, &addr);+ let key = derive_session_key(&headers, &addr, &json!({}), ApiKind::Anthropic);assert!(!key.contains("sk-very-private"));assert!(key.starts_with("apikey:"));}@@ -600,12 +806,12 @@ mod tests {fn session_key_falls_back_to_ip_then_ip_ua() {let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 3)), 5555);// No headers → ip-only.- let bare = derive_session_key(&HeaderMap::new(), &addr);+ let bare = derive_session_key(&HeaderMap::new(), &addr, &json!({}), ApiKind::Anthropic);assert!(bare.starts_with("ip:"));// With UA → ipua-tuple.let mut headers = HeaderMap::new();headers.insert(axum::http::header::USER_AGENT, "ua-test".parse().unwrap());- let with_ua = derive_session_key(&headers, &addr);+ let with_ua = derive_session_key(&headers, &addr, &json!({}), ApiKind::Anthropic);assert!(with_ua.starts_with("ipua:"));assert_ne!(bare, with_ua);}@@ -660,6 +866,328 @@ mod tests {assert_eq!(h1.early_messages, h2.early_messages);}+ /// Shape observed from live Claude Code traffic: turn 1 sends a single+ /// user message whose *last* block carries the `cache_control` marker;+ /// on turn 2 the same message returns byte-identical except the marker+ /// moved to the newest message's first block.+ fn cc_turn1_body() -> serde_json::Value {+ json!({+ "model": "claude-haiku-4-5",+ "system": [{"type": "text", "text": "agent preamble"}],+ "tools": [{"name": "bash"}],+ "messages": [+ {"role": "user", "content": [+ {"type": "text", "text": "opening prompt"},+ {"type": "text", "text": "project context", "cache_control": {"type": "ephemeral"}},+ ]},+ ],+ })+ }++ fn cc_turn2_body() -> serde_json::Value {+ json!({+ "model": "claude-haiku-4-5",+ "system": [{"type": "text", "text": "agent preamble"}],+ "tools": [{"name": "bash"}],+ "messages": [+ {"role": "user", "content": [+ {"type": "text", "text": "opening prompt"},+ {"type": "text", "text": "project context"},+ ]},+ {"role": "assistant", "content": [{"type": "text", "text": "reply"}]},+ {"role": "user", "content": [+ {"type": "text", "text": "second prompt", "cache_control": {"type": "ephemeral"}},+ ]},+ ],+ })+ }++ #[test]+ fn different_conversations_on_one_credential_get_distinct_session_keys() {+ // Interactive clients (Claude Code, Codex CLI) send the same+ // `Authorization` bearer for every concurrent conversation. If the+ // session key stops at the credential, their alternating requests+ // ping-pong one LRU slot and every switch logs a false+ // `cache_drift_observed`.+ let mut headers = HeaderMap::new();+ headers.insert(+ axum::http::header::AUTHORIZATION,+ "Bearer shared-workspace-token".parse().unwrap(),+ );+ let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 9)), 4242);+ let conv_a = anthropic_body("sys", json!([]), vec!["conversation A opener"]);+ let conv_b = anthropic_body("sys", json!([]), vec!["conversation B opener"]);+ let key_a = derive_session_key(&headers, &addr, &conv_a, ApiKind::Anthropic);+ let key_b = derive_session_key(&headers, &addr, &conv_b, ApiKind::Anthropic);+ assert_ne!(+ key_a, key_b,+ "two conversations sharing one credential must not share a drift session"+ );+ }++ #[test]+ fn same_conversation_next_turn_keeps_its_session_key() {+ // The discriminator must survive normal turn-to-turn growth: the+ // opener is byte-identical on turn 2 except its relocated+ // `cache_control` marker.+ let mut headers = HeaderMap::new();+ headers.insert(+ axum::http::header::AUTHORIZATION,+ "Bearer shared-workspace-token".parse().unwrap(),+ );+ let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 9)), 4242);+ let key_t1 = derive_session_key(&headers, &addr, &cc_turn1_body(), ApiKind::Anthropic);+ let key_t2 = derive_session_key(&headers, &addr, &cc_turn2_body(), ApiKind::Anthropic);+ assert_eq!(+ key_t1, key_t2,+ "turn growth and cache_control relocation must not rotate the session key"+ );+ }++ #[test]+ fn explicit_headroom_session_header_wins_over_credentials() {+ // The Python proxy honors `x-headroom-session-id` as the highest-+ // priority session identity (prefix tracker, beta-header tracker);+ // the drift detector must respect the same explicit opt-in.+ let mut headers = HeaderMap::new();+ headers.insert(+ axum::http::header::AUTHORIZATION,+ "Bearer shared-workspace-token".parse().unwrap(),+ );+ headers.insert("x-headroom-session-id", "conv-42".parse().unwrap());+ let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 9)), 4242);+ let body_a = anthropic_body("sys", json!([]), vec!["conversation A opener"]);+ let body_b = anthropic_body("sys", json!([]), vec!["conversation B opener"]);+ let key_a = derive_session_key(&headers, &addr, &body_a, ApiKind::Anthropic);+ assert!(+ key_a.starts_with("session:"),+ "explicit session header must define the key, got: {key_a}"+ );+ assert!(+ !key_a.contains("conv-42"),+ "session keys stay opaque even for non-secret ids: {key_a}"+ );+ // The explicit id pins the session across body variance…+ let key_b = derive_session_key(&headers, &addr, &body_b, ApiKind::Anthropic);+ assert_eq!(key_a, key_b);+ // …and different ids mean different sessions.+ let mut headers2 = headers.clone();+ headers2.insert("x-headroom-session-id", "conv-43".parse().unwrap());+ let key_c = derive_session_key(&headers2, &addr, &body_a, ApiKind::Anthropic);+ assert_ne!(key_a, key_c);+ }++ #[test]+ fn cache_control_relocation_and_growth_are_not_drift() {+ // Turn 1 → turn 2 of a single conversation: the early window gains+ // messages and the client relocates its `cache_control` marker to+ // the newest block. Neither busts the provider's prefix cache, so+ // neither is drift.+ let h1 = compute_structural_hash(&cc_turn1_body(), ApiKind::Anthropic);+ let h2 = compute_structural_hash(&cc_turn2_body(), ApiKind::Anthropic);+ assert_eq!(+ drift_dims(&h1, &h2),+ "",+ "append-only growth with marker relocation must not be drift"+ );+ }++ #[test]+ fn append_only_growth_without_markers_is_not_drift() {+ let h1 = compute_structural_hash(+ &anthropic_body("s", json!([]), vec!["m1"]),+ ApiKind::Anthropic,+ );+ let h2 = compute_structural_hash(+ &anthropic_body("s", json!([]), vec!["m1", "m2", "m3"]),+ ApiKind::Anthropic,+ );+ assert_eq!(+ drift_dims(&h1, &h2),+ "",+ "a conversation growing into the early window must not be drift"+ );+ }++ #[test]+ fn rewritten_early_message_is_still_drift() {+ let h1 = compute_structural_hash(+ &anthropic_body("s", json!([]), vec!["m1", "m2", "m3"]),+ ApiKind::Anthropic,+ );+ let h2 = compute_structural_hash(+ &anthropic_body("s", json!([]), vec!["REWRITTEN", "m2", "m3"]),+ ApiKind::Anthropic,+ );+ assert_eq!(drift_dims(&h1, &h2), "early_messages");+ }++ #[test]+ fn shrunk_history_is_still_drift() {+ // Fewer messages than previously observed *under the same+ // session key* means the settled prefix was rewritten in place+ // — that IS a cache bust. (Reachable when the identity is+ // pinned, e.g. an explicit x-headroom-session-id; on the+ // credential arms a first-message rewrite re-keys instead —+ // see conversation_discriminator's trade-off note.)+ let h1 = compute_structural_hash(+ &anthropic_body("s", json!([]), vec!["m1", "m2", "m3"]),+ ApiKind::Anthropic,+ );+ let h2 = compute_structural_hash(+ &anthropic_body("s", json!([]), vec!["m1"]),+ ApiKind::Anthropic,+ );+ assert_eq!(drift_dims(&h1, &h2), "early_messages");+ }++ #[test]+ fn same_opener_on_different_model_gets_distinct_session_keys() {+ // Auxiliary small-model calls (title generation, summaries)+ // reuse a conversation's opener under the same credential.+ // Provider prompt caches are per-model, so these are separate+ // cache lineages and must not share a drift baseline — the+ // sidecar's different system prompt would otherwise false-warn.+ let mut headers = HeaderMap::new();+ headers.insert(+ axum::http::header::AUTHORIZATION,+ "Bearer shared-workspace-token".parse().unwrap(),+ );+ let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 9)), 4242);+ let mut main_conv = anthropic_body("sys", json!([]), vec!["shared opener"]);+ let mut sidecar = anthropic_body("sys", json!([]), vec!["shared opener"]);+ main_conv["model"] = json!("opus-large");+ sidecar["model"] = json!("haiku-small");+ let key_main = derive_session_key(&headers, &addr, &main_conv, ApiKind::Anthropic);+ let key_side = derive_session_key(&headers, &addr, &sidecar, ApiKind::Anthropic);+ assert_ne!(key_main, key_side);+ }++ #[test]+ fn key_order_variation_does_not_perturb_hashes_or_identity() {+ // The workspace's serde_json enables `preserve_order`, so two+ // serializations of the same message with different key order+ // stay distinct through Value round-trips. Canonicalization+ // must neutralize that for both the axes and the session key.+ let body_a: serde_json::Value = serde_json::from_str(+ r#"{"model":"m","system":"s","tools":[],+ "messages":[{"role":"user","content":"hello"}]}"#,+ )+ .unwrap();+ let body_b: serde_json::Value = serde_json::from_str(+ r#"{"model":"m","system":"s","tools":[],+ "messages":[{"content":"hello","role":"user"}]}"#,+ )+ .unwrap();+ assert_eq!(+ compute_structural_hash(&body_a, ApiKind::Anthropic),+ compute_structural_hash(&body_b, ApiKind::Anthropic),+ );+ let mut headers = HeaderMap::new();+ headers.insert(+ axum::http::header::AUTHORIZATION,+ "Bearer shared-workspace-token".parse().unwrap(),+ );+ let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 9)), 4242);+ assert_eq!(+ derive_session_key(&headers, &addr, &body_a, ApiKind::Anthropic),+ derive_session_key(&headers, &addr, &body_b, ApiKind::Anthropic),+ );+ }++ #[test]+ fn cache_control_named_fields_inside_opaque_payloads_still_count() {+ // A tool schema property (or tool-call argument) that happens+ // to be NAMED cache_control is user data, not a cache marker.+ // Changing it must still read as drift on the affected axis.+ let with_schema = |ty: &str| {+ json!({+ "model": "m", "system": "s",+ "tools": [{"name": "t", "input_schema":+ {"properties": {"cache_control": {"type": ty}}}}],+ "messages": [{"role": "user", "content": "hi"}],+ })+ };+ let h1 = compute_structural_hash(&with_schema("string"), ApiKind::Anthropic);+ let h2 = compute_structural_hash(&with_schema("integer"), ApiKind::Anthropic);+ assert_eq!(drift_dims(&h1, &h2), "tools");++ let with_tool_input = |v: &str| {+ json!({+ "model": "m", "system": "s", "tools": [],+ "messages": [+ {"role": "user", "content": "hi"},+ {"role": "assistant", "content": [+ {"type": "tool_use", "id": "x", "name": "t",+ "input": {"cache_control": v}},+ ]},+ ],+ })+ };+ let h3 = compute_structural_hash(&with_tool_input("a"), ApiKind::Anthropic);+ let h4 = compute_structural_hash(&with_tool_input("b"), ApiKind::Anthropic);+ assert_eq!(drift_dims(&h3, &h4), "early_messages");+ }++ #[test]+ fn bare_string_messages_only_count_for_responses_input() {+ // `input: "text"` is valid Responses sugar; a bare-string+ // `messages` on the other shapes is malformed and contributes+ // no conversation identity.+ let responses = json!({"model": "m", "instructions": "i", "input": "hello"});+ assert_eq!(+ conversation_messages(&responses, ApiKind::OpenAiResponses).len(),+ 1+ );+ let malformed = json!({"model": "m", "system": "s", "messages": "oops"});+ assert!(conversation_messages(&malformed, ApiKind::Anthropic).is_empty());+ assert!(conversation_discriminator(&malformed, ApiKind::Anthropic) == "-");+ }++ #[test]+ fn openai_chat_discriminator_uses_first_non_system_message() {+ let mut headers = HeaderMap::new();+ headers.insert(+ axum::http::header::AUTHORIZATION,+ "Bearer shared-workspace-token".parse().unwrap(),+ );+ let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 9)), 4242);+ let conv = |first_user: &str| {+ json!({+ "model": "gpt-4",+ "messages": [+ {"role": "system", "content": "shared assistant config"},+ {"role": "user", "content": first_user},+ ],+ })+ };+ let key_a = derive_session_key(&headers, &addr, &conv("opener A"), ApiKind::OpenAiChat);+ let key_b = derive_session_key(&headers, &addr, &conv("opener B"), ApiKind::OpenAiChat);+ assert_ne!(key_a, key_b);+ }++ #[test]+ fn openai_responses_discriminator_uses_first_input_item() {+ let mut headers = HeaderMap::new();+ headers.insert(+ axum::http::header::AUTHORIZATION,+ "Bearer shared-workspace-token".parse().unwrap(),+ );+ let addr: SocketAddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 9)), 4242);+ let conv = |first_input: &str| {+ json!({+ "model": "gpt-4",+ "instructions": "shared instructions",+ "input": [{"type": "message", "role": "user", "content": first_input}],+ })+ };+ let key_a =+ derive_session_key(&headers, &addr, &conv("opener A"), ApiKind::OpenAiResponses);+ let key_b =+ derive_session_key(&headers, &addr, &conv("opener B"), ApiKind::OpenAiResponses);+ assert_ne!(key_a, key_b);+ }+#[test]fn early_messages_window_caps_at_three() {// 5 messages: hash should depend only on the first 3.diff --git a/crates/headroom-proxy/src/cache_stabilization/mod.rs b/crates/headroom-proxy/src/cache_stabilization/mod.rsindex 7ae753d4c..815c3af21 100644--- a/crates/headroom-proxy/src/cache_stabilization/mod.rs+++ b/crates/headroom-proxy/src/cache_stabilization/mod.rs@@ -26,7 +26,8 @@//! the cache hot zone (system / tools / early messages). Emits//! `cache_drift_first_request` on first sight and//! `cache_drift_observed` when consecutive requests on the same-//! session disagree on any of the three dimensions.+//! session drift on any of the three dimensions (append-only+//! conversation growth and `cache_control` relocation are benign).//! - [`tool_def_normalize`] — PR-E1 / PR-E2: sorts `tools[]`//! alphabetically by name (PR-E1) and recursively sorts JSON//! Schema object keys inside each tool's `input_schema` /diff --git a/crates/headroom-proxy/src/proxy.rs b/crates/headroom-proxy/src/proxy.rsindex cb9f1ffb1..7f3d81cd3 100644--- a/crates/headroom-proxy/src/proxy.rs+++ b/crates/headroom-proxy/src/proxy.rs@@ -75,11 +75,16 @@ pub struct AppState {}/// PR-E6: maximum number of sessions tracked by the drift detector-/// LRU. Picked so that a noisy test fleet of 1000 distinct API keys-/// stays in cache for at least one full turn before the oldest-/// evicts. Operators with larger fleets can bump this; the memory-/// cost per entry is ~150 bytes (key string + 96-byte StructuralHash-/// + LRU overhead).+/// LRU. Sessions are keyed per conversation (credential + first-+/// message fingerprint), not per credential, so the working set is+/// the number of *concurrently active conversations* — 1000 keeps a+/// noisy fleet in cache for at least one full turn before the oldest+/// evicts. A burst of short one-shot conversations can cycle the LRU+/// and evict a live session between its turns; the cost is telemetry-+/// only (one repeated `cache_drift_first_request`, no lost requests).+/// Operators with larger fleets can bump this; the memory cost per+/// entry is ~250 bytes (key string + 163-byte StructuralHash + LRU+/// overhead).const DRIFT_DETECTOR_CAPACITY: usize = 1000;impl AppState {@@ -699,7 +704,7 @@ pub(crate) async fn forward_http(}};if let (Some(kind), Some(headers)) = (drift_kind, headers_snapshot.as_ref()) {- let session_key = derive_session_key(headers, &client_addr);+ let session_key = derive_session_key(headers, &client_addr, &parsed, kind);let hash = compute_structural_hash(&parsed, kind);observe_drift(&state.drift_state, &session_key, hash);}