Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

Send the LLM operation id as X-Request-Id on openai-compat requests

Djalma Júnior1f9ef64d51a6
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 20b2a554..9f32131f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,9 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+### Added
+- The `openai-compat` provider now sends `X-Request-Id` with every chat
+ attempt, carrying the logical operation id shared by all attempts of one
+ operation — including the strict-to-tolerant fallback. A gateway that
+ records the header (for example as `req=<id>`) can now correlate every
+ attempt of the same operation with the engine's own logs. A static
+ `x-request-id` entry in `AI_MEMORY_LLM_HEADERS` is refused at startup for
+ this provider; the official `openai` and `opencode` providers are
+ unchanged (`opencode` keeps its `x-opencode-session` contract). (#PRNUM)
+
### Changed
- Documented FutureInfra as an endpoint for the existing `openai-compat`
provider. (#1026)
+- The consolidation and auto-improve operation id is now a fresh UUID v7
+ generated by the caller of the LLM operation — once per consolidation
+ invocation and once per auto-improve review — and is never derived from
+ the agent's session id, so two operations of one session (or a
+ crash-resumed or re-claimed run of the durable consolidation queue) are
+ distinct operations to the gateway. Crash re-entry is exactly one new
+ operation: a process restart or a queue re-claim mints a new id for the
+ calls it makes. (#PRNUM)
### Fixed
- Fixed `memory_handoff_list` MCP tool calls being counted under writes in
diff --git a/Cargo.lock b/Cargo.lock
index 0015f62e..bbd4bc7e 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -107,6 +107,7 @@ dependencies = [
"tokio",
"tracing",
"tracing-subscriber",
+ "uuid",
]
[[package]]
diff --git a/crates/ai-memory-consolidate/Cargo.toml b/crates/ai-memory-consolidate/Cargo.toml
index b0eb3d89..0e37c875 100644
--- a/crates/ai-memory-consolidate/Cargo.toml
+++ b/crates/ai-memory-consolidate/Cargo.toml
@@ -39,6 +39,10 @@ async-trait.workspace = true
tempfile.workspace = true
rusqlite.workspace = true
tracing-subscriber.workspace = true
+# Test-only: asserts the fresh operation ids are UUID v7 in the
+# gateway-compatible 36-character form (same crate + features the
+# llm crate already uses to mint them).
+uuid.workspace = true
[lints]
workspace = true
diff --git a/crates/ai-memory-consolidate/src/auto_improve.rs b/crates/ai-memory-consolidate/src/auto_improve.rs
index 1b3a5793..7fdf8824 100644
--- a/crates/ai-memory-consolidate/src/auto_improve.rs
+++ b/crates/ai-memory-consolidate/src/auto_improve.rs
@@ -13,7 +13,8 @@ use std::time::Duration;
use ai_memory_core::{Observation, PagePath, ProjectId, SessionId, WorkspaceId};
use ai_memory_llm::{
- ChatMessage, ChatRequest, LlmError, LlmProvider, Role, complete_structured_with_operation_id,
+ ChatMessage, ChatRequest, LlmError, LlmOperationId, LlmProvider, Role,
+ complete_structured_with_operation_id,
};
use ai_memory_store::{AutoImproveRejectionSummary, BriefingPage, ReaderPool, StoredPageBody};
use schemars::JsonSchema;
@@ -580,8 +581,12 @@ pub async fn run_auto_improve_review(
max_tokens: DEFAULT_REVIEW_MAX_TOKENS,
temperature: Some(0.1),
};
+ // One review call is one LLM operation: a fresh identity, never the
+ // agent's session id — two reviews of the same session must not look
+ // like one operation on the provider side.
+ let operation_id = LlmOperationId::new();
let raw: AutoImproveLlmResponse =
- complete_structured_with_operation_id(llm, request, session_id.into()).await?;
+ complete_structured_with_operation_id(llm, request, operation_id).await?;
let (mut proposals, mut rejected_candidates, mut warnings) =
validate_response(raw, &cfg, &existing_index);
rejected_candidates.extend(prompt_input.rejected_candidates);
@@ -2059,6 +2064,66 @@ mod tests {
}
}
+ /// Records the operation id of every structured call (one review call
+ /// is one LLM operation) and returns an empty proposal set — the id is
+ /// the subject under test, not the proposals.
+ struct OpIdCapturingLlm {
+ ids: std::sync::Arc<std::sync::Mutex<Vec<LlmOperationId>>>,
+ }
+
+ impl OpIdCapturingLlm {
+ fn new() -> Self {
+ Self {
+ ids: std::sync::Arc::new(std::sync::Mutex::new(Vec::new())),
+ }
+ }
+
+ fn ids(&self) -> Vec<LlmOperationId> {
+ self.ids.lock().unwrap().clone()
+ }
+ }
+
+ #[async_trait::async_trait]
+ impl LlmProvider for OpIdCapturingLlm {
+ fn name(&self) -> &'static str {
+ "fake"
+ }
+
+ fn model(&self) -> &'static str {
+ "fake-model"
+ }
+
+ async fn complete(&self, _request: ChatRequest) -> LlmResult<ChatResponse> {
+ Ok(ChatResponse {
+ text: "unused".into(),
+ usage: None,
+ model: "fake-model".into(),
+ })
+ }
+
+ async fn complete_structured_raw(
+ &self,
+ _request: ChatRequest,
+ _schema: serde_json::Value,
+ ) -> LlmResult<serde_json::Value> {
+ Ok(serde_json::json!({
+ "summary": "no durable lesson",
+ "proposals": [],
+ "rejected_candidates": []
+ }))
+ }
+
+ async fn complete_structured_raw_with_operation_id(
+ &self,
+ request: ChatRequest,
+ schema: serde_json::Value,
+ operation_id: LlmOperationId,
+ ) -> LlmResult<serde_json::Value> {
+ self.ids.lock().unwrap().push(operation_id);
+ self.complete_structured_raw(request, schema).await
+ }
+ }
+
fn cfg() -> AutoImproveReviewConfig {
AutoImproveReviewConfig {
min_observations: 3,
@@ -2533,6 +2598,102 @@ mod tests {
assert!(report.rejected_candidates.is_empty());
}
+ /// Operation identity: two auto-improve reviews of the SAME session are
+ /// two LLM operations — two distinct fresh UUID v7 ids (the
+ /// gateway-compatible 36-character form), never the session id — so a
+ /// gateway cannot confuse one session's two reviews with one operation.
+ #[tokio::test]
+ async fn two_reviews_of_one_session_get_distinct_operation_ids() {
+ let tmp = TempDir::new().unwrap();
+ let store = Store::open(tmp.path()).unwrap();
+ let ws = store
+ .writer
+ .get_or_create_workspace("default")
+ .await
+ .unwrap();
+ let proj = store
+ .writer
+ .get_or_create_project(ws, "proj", None)
+ .await
+ .unwrap();
+ let session_id = ai_memory_core::SessionId::new();
+ store
+ .writer
+ .begin_session(NewSession {
+ occurred_at: None,
+ id: session_id,
+ workspace_id: ws,
+ project_id: proj,
+ agent_kind: AgentKind::Other,
+ cwd: None,
+ actor_user: None,
+ })
+ .await
+ .unwrap();
+ for i in 0..3 {
+ store
+ .writer
+ .insert_observation(Sanitized::new(
+ NewObservation {
+ occurred_at: None,
+ session_id,
+ workspace_id: ws,
+ project_id: proj,
+ kind: if i == 0 {
+ ObservationKind::SessionStart
+ } else {
+ ObservationKind::UserPrompt
+ },
+ extension: None,
+ source_event: None,
+ title: format!("event {i}"),
+ body: "run the full gate before release".into(),
+ importance: 5,
+ },
+ &Sanitizer::builtin(),
+ ))
+ .await
+ .unwrap();
+ }
+
+ // Two reviews of the same session, one LLM operation each.
+ let llm = OpIdCapturingLlm::new();
+ for _ in 0..2 {
+ let report = run_auto_improve_review(
+ &store.reader,
+ &llm,
+ ws,
+ proj,
+ session_id,
+ AutoImproveReviewConfig {
+ min_session_duration_secs: 0,
+ ..cfg()
+ },
+ )
+ .await
+ .unwrap();
+ assert!(report.proposals.is_empty());
+ }
+
+ let ids = llm.ids();
+ assert_eq!(ids.len(), 2, "one structured LLM call per review");
+ assert_ne!(
+ ids[0], ids[1],
+ "two reviews of one session get distinct operation ids"
+ );
+ for id in &ids {
+ let s = id.to_string();
+ assert_eq!(s.len(), 36, "the hyphenated wire form is 36 chars: {s}");
+ let parsed = uuid::Uuid::parse_str(&s).expect("parses as a UUID");
+ assert_eq!(parsed.get_version_num(), 7, "fresh UUID v7: {s}");
+ assert_ne!(
+ s,
+ session_id.to_string(),
+ "the operation id is never the session id"
+ );
+ }
+ }
+
/// The reviewer's recent-page context must surface durable pages and drop
/// `sessions/` pages, so those slots go to knowledge the reviewer might
/// re-propose (#834). Both pages are seeded at the same tier, so exclusion
diff --git a/crates/ai-memory-consolidate/src/consolidator.rs b/crates/ai-memory-consolidate/src/consolidator.rs
index 44231cb6..222d7964 100644
--- a/crates/ai-memory-consolidate/src/consolidator.rs
+++ b/crates/ai-memory-consolidate/src/consolidator.rs
@@ -11,7 +11,8 @@ use std::time::Duration;
use ai_memory_core::{AgentKind, Observation, PagePath, ProjectId, SessionId, Tier, WorkspaceId};
use ai_memory_llm::{
- ChatMessage, ChatRequest, LlmError, LlmProvider, Role, complete_structured_with_operation_id,
+ ChatMessage, ChatRequest, LlmError, LlmOperationId, LlmProvider, Role,
+ complete_structured_with_operation_id,
};
use ai_memory_store::{ReaderPool, WriterHandle};
use ai_memory_wiki::{AdmissionContext, AdmissionOp, Wiki, WritePageRequest};
@@ -242,6 +243,11 @@ impl Consolidator {
let existing_titles = self
.existing_page_titles(ws, proj, &actor, session_id)
.await;
+ // One LLM operation, one identity: this fresh id (never the agent's
+ // session id) is shared by every attempt of this invocation; a
+ // re-entry after a crash or a new queue claim is a NEW operation
+ // with a new id.
+ let operation_id = LlmOperationId::new();
let request = build_request(
session_id,
&observations,
@@ -259,7 +265,7 @@ impl Consolidator {
let mut page: ConsolidatedPage = complete_structured_with_retry(
&*self.llm,
request,
- session_id.into(),
+ operation_id,
CONSOLIDATION_LLM_RETRY_DELAY,
)
.await?;
@@ -660,6 +666,11 @@ impl Consolidator {
let existing_titles = self
.existing_page_titles(ws, proj, &actor, session_id)
.await;
+ // One LLM operation, one identity (see `consolidate_session`): the
+ // fresh id is shared by every attempt of this invocation; a
+ // re-entry after a crash or a new queue claim is a new operation
+ // with a new id.
+ let operation_id = LlmOperationId::new();
let request = build_batch_request_with_slots(
session_id,
&observations,
@@ -676,7 +687,7 @@ impl Consolidator {
let batch: ConsolidatedBatch = complete_structured_with_retry(
&*self.llm,
request,
- session_id.into(),
+ operation_id,
CONSOLIDATION_LLM_RETRY_DELAY,
)
.await?;
@@ -3362,6 +3373,63 @@ mod tests {
})
}
+ /// Records the operation id of every structured call and answers a
+ /// fixed valid page or batch — the id is the subject under test, not
+ /// the response.
+ struct OpIdCapturingLlm {
+ response: serde_json::Value,
+ ids: std::sync::Arc<std::sync::Mutex<Vec<LlmOperationId>>>,
+ }
+
+ impl OpIdCapturingLlm {
+ /// Build a capturing LLM, keeping a shared handle to the ids it
+ /// records so a test can assert on them after the run.
+ fn new(
+ response: serde_json::Value,
+ ) -> (Self, std::sync::Arc<std::sync::Mutex<Vec<LlmOperationId>>>) {
+ let ids = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
+ (
+ Self {
+ response,
+ ids: ids.clone(),
+ },
+ ids,
+ )
+ }
+ }
+
+ #[async_trait::async_trait]
+ impl LlmProvider for OpIdCapturingLlm {
+ fn name(&self) -> &'static str {
+ "capturing"
+ }
+ fn model(&self) -> &str {
+ "capturing"
+ }
+ async fn complete(
+ &self,
+ _request: ChatRequest,
+ ) -> ai_memory_llm::LlmResult<ai_memory_llm::ChatResponse> {
+ unreachable!("consolidation only uses structured completion");
+ }
+ async fn complete_structured_raw(
+ &self,
+ _request: ChatRequest,
+ _schema: serde_json::Value,
+ ) -> ai_memory_llm::LlmResult<serde_json::Value> {
+ Ok(self.response.clone())
+ }
+ async fn complete_structured_raw_with_operation_id(
+ &self,
+ request: ChatRequest,
+ schema: serde_json::Value,
+ operation_id: LlmOperationId,
+ ) -> ai_memory_llm::LlmResult<serde_json::Value> {
+ self.ids.lock().unwrap().push(operation_id);
+ self.complete_structured_raw(request, schema).await
+ }
+ }
+
/// A briefly overloaded provider must not end the consolidation: the
/// retry absorbs it and the caller still gets the page.
#[tokio::test]
@@ -3460,6 +3528,151 @@ mod tests {
);
}
+ /// Gateway compatibility of a fresh operation id: the 36-character
+ /// hyphenated UUID v7 form a gateway records in `X-Request-Id`.
+ fn assert_fresh_gateway_form(id: LlmOperationId) {
+ let s = id.to_string();
+ assert_eq!(s.len(), 36, "the hyphenated wire form is 36 chars: {s}");
+ let parsed = uuid::Uuid::parse_str(&s).expect("parses as a UUID");
+ assert_eq!(
+ parsed.get_version_num(),
+ 7,
+ "a fresh operation id is UUID v7: {s}"
+ );
+ }
+
+ /// Operation identity: two consolidation invocations of the SAME
+ /// session are two LLM operations — two distinct fresh UUID v7 ids,
+ /// never the session id. Controls use a v7 session (`SessionId::new`)
+ /// and a v4-style session (the random form agents commonly carry), so
+ /// no derivation from the session's own bytes can sneak back in.
+ #[tokio::test]
+ async fn each_invocation_gets_a_fresh_operation_id_independent_of_the_session() {
+ let actor = ai_memory_core::ActorContext::anonymous();
+ // v7 (time-ordered) and v4 (random, agent-style) sessions.
+ let sessions = [SessionId::new(), SessionId(uuid::Uuid::new_v4())];
+ for session in sessions {
+ let tmp = tempfile::tempdir().unwrap();
+ let store = ai_memory_store::Store::open(tmp.path()).unwrap();
+ let ws = store
+ .writer
+ .get_or_create_workspace("default")
+ .await
+ .unwrap();
+ let proj = store
+ .writer
+ .get_or_create_project(ws, "scratch", None)
+ .await
+ .unwrap();
+ seed_session(store.db_path(), session, ws, proj);
+ let wiki = Wiki::new(tmp.path(), store.writer.clone()).unwrap();
+
+ // Two single invocations of the SAME session: two distinct
+ // fresh v7 ids, neither one the session id.
+ let (llm1, ids1) = OpIdCapturingLlm::new(scripted_page());
+ Consolidator::new(
+ store.reader.clone(),
+ store.writer.clone(),
+ wiki.clone(),
+ Arc::new(llm1),
+ ws,
+ proj,
+ )
+ .consolidate_session(session, false, actor.clone(), None, None)
+ .await
+ .unwrap();
+ let (llm2, ids2) = OpIdCapturingLlm::new(scripted_page());
+ Consolidator::new(
+ store.reader.clone(),
+ store.writer.clone(),
+ wiki.clone(),
+ Arc::new(llm2),
+ ws,
+ proj,
+ )
+ .consolidate_session(session, false, actor.clone(), None, None)
+ .await
+ .unwrap();
+ let single1 = ids1.lock().unwrap().clone();
+ let single2 = ids2.lock().unwrap().clone();
+ assert_eq!(single1.len(), 1, "one default single call per invocation");
+ assert_eq!(single2.len(), 1, "one default single call per invocation");
+ assert_ne!(
+ single1[0], single2[0],
+ "two single invocations of one session get distinct operation ids"
+ );
+ assert_fresh_gateway_form(single1[0]);
+ assert_fresh_gateway_form(single2[0]);
+ assert_ne!(
+ single1[0].to_string(),
+ session.to_string(),
+ "the operation id is never the session id"
+ );
+
+ // Same for the multi default path: two invocations, two ids.
+ let batch = serde_json::json!({
+ "rationale": "test operation identity",
+ "updates": [
+ {
+ "path": format!("sessions/{session}.md"),
+ "tier": "episodic",
+ "kind": "fact",
+ "title": "Session narrative",
+ "body_markdown": "Session body.",
+ "tags": []
+ },
+ {
+ "path": "concepts/queue.md",
+ "tier": "semantic",
+ "kind": "fact",
+ "title": "Queue",
+ "body_markdown": "Concept body.",
+ "tags": []
+ }
+ ]
+ });
+ let (llm3, ids3) = OpIdCapturingLlm::new(batch.clone());
+ Consolidator::new(
+ store.reader.clone(),
+ store.writer.clone(),
+ wiki.clone(),
+ Arc::new(llm3),
+ ws,
+ proj,
+ )
+ .consolidate_session_multi(session, false, actor.clone(), None, None)
+ .await
+ .unwrap();
+ let (llm4, ids4) = OpIdCapturingLlm::new(batch);
+ Consolidator::new(
+ store.reader.clone(),
+ store.writer.clone(),
+ wiki.clone(),
+ Arc::new(llm4),
+ ws,
+ proj,
+ )
+ .consolidate_session_multi(session, false, actor.clone(), None, None)
+ .await
+ .unwrap();
+ let multi1 = ids3.lock().unwrap().clone();
+ let multi2 = ids4.lock().unwrap().clone();
+ assert_eq!(multi1.len(), 1, "one default multi call per invocation");
+ assert_eq!(multi2.len(), 1, "one default multi call per invocation");
+ assert_ne!(
+ multi1[0], multi2[0],
+ "two multi invocations of one session get distinct operation ids"
+ );
+ assert_fresh_gateway_form(multi1[0]);
+ assert_fresh_gateway_form(multi2[0]);
+ assert_ne!(
+ multi1[0].to_string(),
+ session.to_string(),
+ "the operation id is never the session id"
+ );
+ }
+ }
+
async fn write_slot(wiki: &Wiki, ws: WorkspaceId, proj: ProjectId, path: &str, body: &str) {
wiki.write_page(WritePageRequest {
workspace_id: ws,
diff --git a/crates/ai-memory-llm/src/factory.rs b/crates/ai-memory-llm/src/factory.rs
index a5d6b208..f358acee 100644
--- a/crates/ai-memory-llm/src/factory.rs
+++ b/crates/ai-memory-llm/src/factory.rs
@@ -20,6 +20,7 @@ use crate::auth::{AuthRequirement, CopilotAuth, ProviderAuth};
use crate::embedding::{Embedder, OpenAiCompatEmbedder, OpenAiEmbedder, VoyageEmbedder};
use crate::error::{LlmError, LlmResult};
use crate::google::GoogleEmbedder;
+use crate::openai::REQUEST_ID_HEADER;
use crate::provider::LlmProvider;
/// LLM providers available to ai-memory.
@@ -402,13 +403,26 @@ pub fn build_provider(config: ProviderConfig) -> LlmResult<Arc<dyn LlmProvider>>
let base = config
.base_url
.ok_or_else(|| LlmError::NotConfigured("LLM_BASE_URL".into()))?;
+ // The openai-compat provider sends `x-request-id` itself, with
+ // the logical operation id on every chat attempt. A static
+ // operator value cannot serve that role, and `reqwest` would
+ // append a second value; refuse the conflict at startup, the
+ // same fail-closed way reserved headers are refused.
+ if extra_headers.contains(REQUEST_ID_HEADER) {
+ return Err(LlmError::NotConfigured(format!(
+ "AI_MEMORY_LLM_HEADERS must not set {REQUEST_ID_HEADER}: the \
+ openai-compat provider sends it itself on every chat attempt \
+ with the logical operation id"
+ )));
+ }
let extra_headers = with_default_openrouter_headers(&base, extra_headers);
Ok(Arc::new(
OpenAiCompatProvider::new(base, config.auth.optional_api_key(), config.model)?
.with_strict(config.compat_strict)
.with_timeout_secs(timeout)
.with_reasoning_effort(config.reasoning_effort)
- .with_extra_headers(extra_headers),
+ .with_extra_headers(extra_headers)
+ .with_request_id_header(),
))
}
ProviderChoice::OpenAiOAuth => {
@@ -636,4 +650,124 @@ mod tests {
};
assert!(matches!(err, LlmError::NotConfigured(msg) if msg == "OPENAI_API_KEY"));
}
+
+ fn compat_config(extra_headers: crate::ExtraHeaders) -> ProviderConfig {
+ ProviderConfig {
+ provider: ProviderChoice::OpenAiCompat,
+ model: "qwen3.8-27b".into(),
+ auth: ProviderAuth::optional_api_key_from_env("LLM_API_KEY", None),
+ base_url: Some("http://127.0.0.1:11434/v1".into()),
+ compat_strict: true,
+ request_timeout_secs: crate::DEFAULT_REQUEST_TIMEOUT_SECS,
+ reasoning_effort: None,
+ extra_headers,
+ }
+ }
+
+ /// A static `x-request-id` in `AI_MEMORY_LLM_HEADERS` cannot coexist
+ /// with the dynamic one — the conflict is refused at startup, before
+ /// any send, rather than letting two values (or the operator's stale
+ /// value) reach the gateway.
+ #[test]
+ fn openai_compat_refuses_a_static_request_id_header() {
+ let headers = crate::ExtraHeaders::parse(["x-request-id: static-1"]).expect("valid");
+ let err = match build_provider(compat_config(headers)) {
+ Ok(_) => panic!("the conflicting entry must be refused at startup"),
+ Err(err) => err,
+ };
+ let rendered = err.to_string();
+ assert!(matches!(err, LlmError::NotConfigured(_)), "{rendered}");
+ assert!(rendered.contains(REQUEST_ID_HEADER), "{rendered}");
+ assert!(rendered.contains("openai-compat"), "{rendered}");
+ }
+
+ /// Case insensitivity is on the wire: a mixed-case operator entry is the
+ /// same conflict and gets the same refusal.
+ #[test]
+ fn openai_compat_refuses_a_mixed_case_static_request_id_header() {
+ let headers = crate::ExtraHeaders::parse(["X-Request-Id: static-1"]).expect("valid");
+ assert!(
+ build_provider(compat_config(headers)).is_err(),
+ "the wire treats header names case-insensitively"
+ );
+ }
+
+ /// The refusal is scoped to the provider that owns the header: the same
+ /// static entry is a legitimate operator configuration everywhere else.
+ #[test]
+ fn a_static_request_id_header_is_allowed_for_other_providers() {
+ let headers = crate::ExtraHeaders::parse(["x-request-id: static-1"]).expect("valid");
+ let cfg = ProviderConfig {
+ provider: ProviderChoice::OpenAi,
+ model: "gpt-4o-mini".into(),
+ auth: ProviderAuth::required_api_key_from_env(
+ "OPENAI_API_KEY",
+ Some(secrecy::SecretString::from("sk-test")),
+ ),
+ base_url: None,
+ compat_strict: false,
+ request_timeout_secs: crate::DEFAULT_REQUEST_TIMEOUT_SECS,
+ reasoning_effort: None,
+ extra_headers: headers,
+ };
+ assert!(
+ build_provider(cfg).is_ok(),
+ "only openai-compat owns the header"
+ );
+ }
+
+ /// End to end: the provider the factory builds for the configured
+ /// `openai-compat` choice sends exactly one `x-request-id` per chat
+ /// attempt, carrying the operation's id — this is the gateway-facing
+ /// contract the whole slice exists for.
+ #[tokio::test]
+ async fn the_factory_built_compat_provider_sends_the_operation_id() {
+ let server = wiremock::MockServer::start().await;
+ wiremock::Mock::given(wiremock::matchers::method("POST"))
+ .and(wiremock::matchers::path("/v1/chat/completions"))
+ .respond_with(
+ wiremock::ResponseTemplate::new(200).set_body_json(serde_json::json!({
+ "model": "model-x",
+ "choices": [{ "message": { "content": "ok" } }],
+ "usage": { "prompt_tokens": 1, "completion_tokens": 1 },
+ })),
+ )
+ .mount(&server)
+ .await;
+
+ let mut cfg = compat_config(crate::ExtraHeaders::default());
+ cfg.base_url = Some(server.uri());
+ let provider = build_provider(cfg).expect("provider builds");
+ let operation_id = crate::LlmOperationId::new();
+ provider
+ .complete_with_operation_id(crate::ChatRequest::user_prompt("hi"), operation_id)
+ .await
+ .expect("completion succeeds");
+
+ let requests = server.received_requests().await.expect("requests");
+ assert_eq!(requests.len(), 1);
+ assert_eq!(
+ requests[0]
+ .headers
+ .get(REQUEST_ID_HEADER)
+ .and_then(|value| value.to_str().ok()),
+ Some(operation_id.to_string().as_str()),
+ );
+ assert_eq!(
+ requests[0]
+ .headers
+ .get_all(REQUEST_ID_HEADER)
+ .into_iter()
+ .count(),
+ 1
+ );
+ // The layered defaults still apply alongside the new header.
+ assert_eq!(
+ requests[0]
+ .headers
+ .get(reqwest::header::USER_AGENT)
+ .and_then(|value| value.to_str().ok()),
+ Some(crate::DEFAULT_USER_AGENT),
+ );
+ }
}
diff --git a/crates/ai-memory-llm/src/openai.rs b/crates/ai-memory-llm/src/openai.rs
index 88186923..1acd8284 100644
--- a/crates/ai-memory-llm/src/openai.rs
+++ b/crates/ai-memory-llm/src/openai.rs
@@ -30,6 +30,16 @@ pub const DEFAULT_BASE_URL: &str = "https://api.openai.com";
/// messages and logs — naming it makes those messages discoverable.
pub(crate) const STRUCTURED_OUTPUT_SCHEMA_NAME: &str = "Result";
+/// `X-Request-Id` header the `openai-compat` provider sends on every chat
+/// attempt, carrying the [`LlmOperationId`] all attempts of one operation
+/// share (strict-to-tolerant fallbacks included). A gateway that records it
+/// (for example as `req=<id>`) can correlate every attempt of the same
+/// operation and forward it to the engine. The factory enables it only for
+/// the configured `openai-compat` provider and refuses a static
+/// `AI_MEMORY_LLM_HEADERS` entry for this name there; the official OpenAI
+/// and OpenCode providers keep their own contracts.
+pub(crate) const REQUEST_ID_HEADER: &str = "x-request-id";
+
/// Build the full URL for an OpenAI-style endpoint. Tolerates the
/// conventions found in the wild:
/// * `https://api.openai.com` (OpenAI's own docs)
@@ -101,6 +111,11 @@ pub struct OpenAiProvider {
/// Caller-identifying defaults a provider opts into. Layered *under*
/// `extra_headers`.
client_headers: Option<ClientHeaders>,
+ /// OpenAI-compat only: send [`REQUEST_ID_HEADER`] with the logical
+ /// operation id on every chat attempt. Off by default; the factory
+ /// turns it on only for the configured `openai-compat` provider, so
+ /// the official OpenAI and OpenCode providers never gain the header.
+ request_id_header: bool,
}
/// Defaults for a provider whose gateway wants the caller identified: an
@@ -130,6 +145,7 @@ impl OpenAiProvider {
reasoning_effort: None,
extra_headers: ExtraHeaders::default(),
client_headers: None,
+ request_id_header: false,
})
}
@@ -198,6 +214,20 @@ impl OpenAiProvider {
self
}
+ /// Send [`REQUEST_ID_HEADER`] with the logical operation id on every
+ /// chat attempt. The id is created before the first attempt and reused
+ /// by every fallback, so all attempts of one operation carry exactly
+ /// the same value. Enabled by the factory only for the configured
+ /// `openai-compat` provider; a static `AI_MEMORY_LLM_HEADERS` entry
+ /// for this name is refused there at startup, and
+ /// [`ExtraHeaders::apply`] replaces per name either way, so one value
+ /// always reaches the wire.
+ #[must_use]
+ pub(crate) fn with_request_id_header(mut self) -> Self {
+ self.request_id_header = true;
+ self
+ }
+
/// Endpoint the client will call. Test-visible so wrappers that default
/// the base URL and let it be overridden can assert which one is set.
#[cfg(test)]
@@ -462,9 +492,23 @@ impl OpenAiProvider {
// — two `user-agent` values whenever the operator configures one, and
// a duplicate is worse than either value alone. `set_default` leaves
// an operator entry untouched, so the layering is explicit here
- // rather than dependent on call order.
+ // rather than dependent on call order. The dynamic `x-request-id`
+ // is the one header ai-memory *owns* on this path: `insert`
+ // (not `set_default`) plus the factory's refuse-up-front keep
+ // exactly one value on the wire, the operation's.
let request = match self.client_headers {
- None => self.extra_headers.apply(builder),
+ None => {
+ let mut headers = self.extra_headers.clone();
+ if self.request_id_header {
+ // A UUID is always a valid header value; on the
+ // impossible failure, omitting a correlation header
+ // beats failing the consolidation pass that carries it.
+ if let Ok(value) = HeaderValue::from_str(&operation_id.to_string()) {
+ headers.insert(HeaderName::from_static(REQUEST_ID_HEADER), value);
+ }
+ }
+ headers.apply(builder)
+ }
Some(client) => {
let mut headers = self.extra_headers.clone();
headers.set_default(
@@ -675,15 +719,18 @@ fn max_output_tokens_for(model: &str) -> u32 {
#[cfg(test)]
mod tests {
use super::{
- OpenAiProvider, RequestDialect, enforce_strict_object_schemas,
+ OpenAiProvider, REQUEST_ID_HEADER, RequestDialect, enforce_strict_object_schemas,
model_requires_max_completion_tokens, normalize_openai_base,
};
- use crate::types::{ChatMessage, ChatRequest, ReasoningEffort, Role};
+ use crate::provider::LlmProvider;
+ use crate::types::{ChatMessage, ChatRequest, LlmOperationId, ReasoningEffort, Role};
use rstest::rstest;
use schemars::JsonSchema;
use secrecy::SecretString;
use serde::{Deserialize, Serialize};
use serde_json::json;
+ use wiremock::matchers::{method, path};
+ use wiremock::{Mock, MockServer, ResponseTemplate};
fn provider_for(model: &str) -> OpenAiProvider {
OpenAiProvider::new(SecretString::new("test-key".into()), model).unwrap()
@@ -1357,4 +1404,34 @@ mod tests {
"https://api.z.ai/api/coding/paas/v4/embeddings"
);
}
+
+ /// Acceptance: the official OpenAI provider keeps its contract —
+ /// without the openai-compat opt-in it never sends `x-request-id`.
+ #[tokio::test]
+ async fn the_official_provider_sends_no_request_id_header() {
+ let server = MockServer::start().await;
+ Mock::given(method("POST"))
+ .and(path("/v1/chat/completions"))
+ .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
+ "model": "gpt-4o-mini",
+ "choices": [{ "message": { "content": "ok" } }],
+ })))
+ .mount(&server)
+ .await;
+
+ let provider = OpenAiProvider::new(SecretString::from("sk-test"), "gpt-4o-mini")
+ .unwrap()
+ .with_base_url(server.uri());
+ provider
+ .complete_with_operation_id(ChatRequest::user_prompt("hi"), LlmOperationId::new())
+ .await
+ .expect("completion succeeds");
+
+ let requests = server.received_requests().await.expect("requests");
+ assert_eq!(requests.len(), 1);
+ assert!(
+ requests[0].headers.get(REQUEST_ID_HEADER).is_none(),
+ "the official provider must not gain the openai-compat header"
+ );
+ }
}
diff --git a/crates/ai-memory-llm/src/openai_compat.rs b/crates/ai-memory-llm/src/openai_compat.rs
index c25a76c9..e2016fd0 100644
--- a/crates/ai-memory-llm/src/openai_compat.rs
+++ b/crates/ai-memory-llm/src/openai_compat.rs
@@ -148,6 +148,21 @@ impl OpenAiCompatProvider {
self.inner = self.inner.with_client_headers(user_agent, operation_id);
self
}
+
+ /// Send `X-Request-Id` with the logical operation id on every chat
+ /// attempt, for gateways that record it (for example as `req=<id>`) and
+ /// forward it to the engine. The id is shared by every attempt of one
+ /// operation — the strict-to-tolerant fallback reuses it, so one
+ /// operation is correlatable end to end. Enabled by the factory only
+ /// for the configured `openai-compat` provider;
+ /// [`crate::OpenCodeProvider`] keeps its own `x-opencode-session`
+ /// contract and does not set it. A static `AI_MEMORY_LLM_HEADERS` entry
+ /// for this name is refused at startup on that path.
+ #[must_use]
+ pub(crate) fn with_request_id_header(mut self) -> Self {
+ self.inner = self.inner.with_request_id_header();
+ self
+ }
/// Endpoint the inner client will call. Test-visible so
/// [`crate::OpenCodeProvider`]'s tests can assert Go stays the default
/// and a Zen override takes effect.
@@ -373,6 +388,180 @@ fn first_json_object(s: &str) -> Option<&str> {
#[cfg(test)]
mod tests {
use super::*;
+ use secrecy::SecretString;
+ use serde_json::json;
+ use wiremock::matchers::{method, path};
+ use wiremock::{Mock, MockServer, ResponseTemplate};
+
+ fn chat_response_with_content(content: &str) -> serde_json::Value {
+ json!({
+ "model": "model-x",
+ "choices": [{
+ "message": { "content": content },
+ }],
+ "usage": { "prompt_tokens": 1, "completion_tokens": 1 },
+ })
+ }
+
+ fn request_id_of(request: &wiremock::Request) -> Option<String> {
+ request
+ .headers
+ .get(crate::openai::REQUEST_ID_HEADER)
+ .and_then(|value| value.to_str().ok())
+ .map(str::to_owned)
+ }
+
+ /// Acceptance (a): the header is present on the wire in the accepted
+ /// format — a 36-character `[A-Za-z0-9._:-]` value (UUID v7 display
+ /// string) well inside the 64-character header contract — and it carries
+ /// exactly the operation's id, as a single value.
+ #[tokio::test]
+ async fn the_request_id_header_carries_the_operation_id() {
+ let server = MockServer::start().await;
+ Mock::given(method("POST"))
+ .and(path("/v1/chat/completions"))
+ .respond_with(
+ ResponseTemplate::new(200).set_body_json(chat_response_with_content("ok")),
+ )
+ .mount(&server)
+ .await;
+
+ let provider = OpenAiCompatProvider::new(server.uri(), None, "model-x")
+ .expect("provider builds")
+ .with_request_id_header();
+ let operation_id = LlmOperationId::new();
+ provider
+ .complete_with_operation_id(ChatRequest::user_prompt("hi"), operation_id)
+ .await
+ .expect("completion succeeds");
+
+ let requests = server.received_requests().await.expect("requests");
+ assert_eq!(requests.len(), 1);
+ let value = request_id_of(&requests[0]).expect("x-request-id is on the wire");
+ assert_eq!(value, operation_id.to_string());
+ // Exactly one value: `reqwest` appends, so a duplicate would be two.
+ assert_eq!(
+ requests[0]
+ .headers
+ .get_all(crate::openai::REQUEST_ID_HEADER)
+ .into_iter()
+ .count(),
+ 1
+ );
+ // Accepted format: 36-char UUID v7, inside the <=64 char contract.
+ assert_eq!(value.len(), 36, "UUID display form is 36 characters");
+ assert!(
+ value
+ .chars()
+ .all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | ':' | '-')),
+ "the gateway accepts [A-Za-z0-9._:-]: {value:?}"
+ );
+ let uuid = uuid::Uuid::parse_str(&value).expect("valid UUID");
+ assert_eq!(uuid.get_version_num(), 7, "time-ordered v7 id");
+ }
+
+ /// Acceptance (b): the strict-to-tolerant fallback is a SECOND HTTP
+ /// call of the same operation — both attempts must carry the same
+ /// `x-request-id`, and only that value.
+ #[tokio::test]
+ async fn the_strict_fallback_reuses_the_request_id() {
+ let server = MockServer::start().await;
+ Mock::given(method("POST"))
+ .and(path("/v1/chat/completions"))
+ .respond_with(|request: &wiremock::Request| {
+ let body: serde_json::Value =
+ serde_json::from_slice(&request.body).expect("request body is JSON");
+ if body.get("response_format").is_some() {
+ ResponseTemplate::new(400)
+ .set_body_string("unsupported parameter: response_format")
+ } else {
+ ResponseTemplate::new(200)
+ .set_body_json(chat_response_with_content(r#"{"ok":true}"#))
+ }
+ })
+ .mount(&server)
+ .await;
+
+ let provider = OpenAiCompatProvider::new(server.uri(), None, "model-x")
+ .expect("provider builds")
+ .with_strict(true)
+ .with_request_id_header();
+ let operation_id = LlmOperationId::new();
+ let value = provider
+ .complete_structured_raw_with_operation_id(
+ ChatRequest::user_prompt("emit JSON"),
+ json!({
+ "type": "object",
+ "properties": { "ok": { "type": "boolean" } },
+ "required": ["ok"],
+ }),
+ operation_id,
+ )
+ .await
+ .expect("the tolerant fallback parses the JSON");
+
+ assert_eq!(value, json!({ "ok": true }));
+ let requests = server.received_requests().await.expect("requests");
+ assert_eq!(requests.len(), 2, "strict attempt + tolerant fallback");
+ let expected = operation_id.to_string();
+ for request in &requests {
+ assert_eq!(
+ request_id_of(request),
+ Some(expected.clone()),
+ "every attempt of one operation carries the same id"
+ );
+ assert_eq!(
+ request
+ .headers
+ .get_all(crate::openai::REQUEST_ID_HEADER)
+ .into_iter()
+ .count(),
+ 1
+ );
+ }
+ }
+
+ /// Acceptance (e): `opencode` routes through this wrapper internally but
+ /// keeps its own `x-opencode-session` contract — it must NOT gain the
+ /// `x-request-id` header.
+ #[tokio::test]
+ async fn the_opencode_transport_keeps_its_contract_without_a_request_id() {
+ let server = MockServer::start().await;
+ Mock::given(method("POST"))
+ .and(path("/v1/chat/completions"))
+ .respond_with(
+ ResponseTemplate::new(200).set_body_json(chat_response_with_content("ok")),
+ )
+ .mount(&server)
+ .await;
+
+ let provider = crate::OpenCodeProvider::new(SecretString::from("sk-test"), "model-x")
+ .expect("provider builds")
+ .with_base_url(server.uri());
+ let operation_id = LlmOperationId::new();
+ provider
+ .complete_with_operation_id(ChatRequest::user_prompt("hi"), operation_id)
+ .await
+ .expect("completion succeeds");
+
+ let requests = server.received_requests().await.expect("requests");
+ assert_eq!(requests.len(), 1);
+ assert_eq!(
+ requests[0]
+ .headers
+ .get(crate::opencode::OPENCODE_SESSION_HEADER)
+ .and_then(|value| value.to_str().ok()),
+ Some(operation_id.to_string().as_str()),
+ "the x-opencode-session contract is preserved"
+ );
+ assert!(
+ requests[0]
+ .headers
+ .get(crate::openai::REQUEST_ID_HEADER)
+ .is_none(),
+ "opencode must not gain the openai-compat x-request-id header"
+ );
+ }
/// The low-level constructor stays tolerant so wrappers can choose their
/// own compatibility policy; runtime configuration supplies the default.
diff --git a/crates/ai-memory-llm/src/types.rs b/crates/ai-memory-llm/src/types.rs
index 6e75e999..49ba7ac2 100644
--- a/crates/ai-memory-llm/src/types.rs
+++ b/crates/ai-memory-llm/src/types.rs
@@ -4,6 +4,13 @@ use serde::{Deserialize, Serialize};
use uuid::Uuid;
/// Identifier shared by every HTTP attempt for one logical LLM operation.
+///
+/// Fresh per operation: the caller of one LLM operation (a consolidation
+/// invocation, an auto-improve review) generates one value with
+/// [`LlmOperationId::new`] and passes it to every attempt of that
+/// operation. It is never derived from an agent session id — two
+/// operations of the same session (or a crash-resumed or re-claimed run)
+/// must not look like one operation on the provider side.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct LlmOperationId(Uuid);
@@ -21,12 +28,6 @@ impl Default for LlmOperationId {
}
}
-impl From<ai_memory_core::SessionId> for LlmOperationId {
- fn from(value: ai_memory_core::SessionId) -> Self {
- Self(value.0)
- }
-}
-
impl std::fmt::Display for LlmOperationId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
self.0.fmt(f)
@@ -329,6 +330,29 @@ impl ExtraHeaders {
}
}
+ /// Insert `name: value`, replacing any existing value for that name.
+ ///
+ /// Unlike [`Self::set_default`], this overrides an operator entry: it is
+ /// reserved for headers ai-memory sets itself per request (the
+ /// openai-compat `x-request-id`, which carries the logical operation id).
+ /// The factory refuses a static `AI_MEMORY_LLM_HEADERS` entry for those
+ /// names up front, so on the configured path this can never silently
+ /// replace operator material — and [`Self::apply`] replaces per name
+ /// either way, so one value always goes on the wire.
+ pub(crate) fn insert(
+ &mut self,
+ name: reqwest::header::HeaderName,
+ value: reqwest::header::HeaderValue,
+ ) {
+ self.0.insert(name, value);
+ }
+
+ /// Whether `name` is configured (header-name comparison is
+ /// case-insensitive, as on the wire).
+ pub(crate) fn contains(&self, name: &str) -> bool {
+ self.0.contains_key(name)
+ }
+
/// Attach the configured headers to `builder`.
///
/// Uses `RequestBuilder::headers`, which *replaces* any value already set
@@ -539,6 +563,25 @@ mod tests {
assert_eq!(headers.get("user-agent"), Some("fallback/1"));
}
+ /// `insert` is the per-request override used for headers ai-memory owns
+ /// (the openai-compat `x-request-id`): an existing entry is *replaced*,
+ /// so exactly one value survives, and `contains` sees it case-insensitively.
+ #[test]
+ fn insert_replaces_an_existing_value() {
+ let mut headers = ExtraHeaders::parse(["x-request-id: static-1"]).expect("valid");
+ assert!(headers.contains("x-request-id"));
+ headers.insert(
+ reqwest::header::HeaderName::from_static("x-request-id"),
+ reqwest::header::HeaderValue::from_static("dynamic-1"),
+ );
+ assert_eq!(headers.get("x-request-id"), Some("dynamic-1"));
+ // The header-name match used by the factory's refuse check is
+ // case-insensitive, so a mixed-case operator entry is caught too.
+ let mixed = ExtraHeaders::parse(["X-Request-Id: static-2"]).expect("valid");
+ assert!(mixed.contains("x-request-id"));
+ assert!(!mixed.contains("x-opencode-session"));
+ }
+
/// `ProviderConfig` derives `Debug` and gets rendered into configuration
/// errors; values must not ride along.
#[test]
diff --git a/crates/ai-memory-llm/tests/suite/openai_compat_embedder.rs b/crates/ai-memory-llm/tests/suite/openai_compat_embedder.rs
index 08f2e370..c6482cd7 100644
--- a/crates/ai-memory-llm/tests/suite/openai_compat_embedder.rs
+++ b/crates/ai-memory-llm/tests/suite/openai_compat_embedder.rs
@@ -8,7 +8,10 @@
//! a gateway key IS configured it is sent as a bearer token; (3) the
//! embedder identifies as provider `openai-compat`, so stored
//! `{provider, model, dim}` triples are a distinct family from plain
-//! `openai`; (4) the factory refuses to build without a base URL.
+//! `openai`; (4) the factory refuses to build without a base URL;
+//! (5) the embeddings request never carries the chat provider's
+//! `x-request-id` header — the embedder is a separate type with no
+//! operation id to send.
use ai_memory_llm::{
Embedder, EmbedderChoice, EmbedderConfig, LlmError, OpenAiCompatEmbedder, build_embedder,
@@ -76,6 +79,43 @@ async fn configured_key_is_sent_as_bearer() {
e.embed("hello").await.expect("embed succeeds");
}
+#[tokio::test]
+async fn factory_built_compat_embedder_sends_no_request_id_header() {
+ // The openai-compat chat path injects `x-request-id` (the logical
+ // operation id) on every chat attempt; the embedder must never carry
+ // that header. A refactor that shares the chat header hook with the
+ // embedder breaks this assertion.
+ let server = MockServer::start().await;
+ Mock::given(method("POST"))
+ .and(path("/v1/embeddings"))
+ .respond_with(move |req: &Request| {
+ assert!(
+ req.headers.get("x-request-id").is_none(),
+ "the compat embedder must not carry the chat provider's x-request-id header"
+ );
+ ResponseTemplate::new(200).set_body_json(embedding_body(8))
+ })
+ .expect(1)
+ .mount(&server)
+ .await;
+
+ let embedder = build_embedder(EmbedderConfig {
+ provider: EmbedderChoice::OpenAiCompat,
+ model: "nomic-embed-text".into(),
+ dim: 8,
+ api_key: SecretString::from(String::new()),
+ base_url: Some(format!("{}/v1", server.uri())),
+ models_dir: None,
+ copilot_auth: None,
+ defaulted: false,
+ query_prefix: String::new(),
+ document_prefix: String::new(),
+ })
+ .expect("factory builds compat embedder");
+
+ embedder.embed("hello").await.expect("embed succeeds");
+}
+
#[tokio::test]
async fn factory_builds_compat_embedder_and_requires_base_url() {
let ok = build_embedder(EmbedderConfig {
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index d5d5d65c..07021ec9 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -934,6 +934,9 @@ AI_MEMORY_LLM_HEADERS optional extra HTTP headers on every chat request, as
content-type, x-api-key, x-goog-api-key,
anthropic-version, anthropic-beta, openai-beta,
host, content-length) are refused at startup.
+ `x-request-id` is additionally refused at
+ initialization for the `openai-compat` provider,
+ which sends the operation's request id itself.
Values are never logged. A header value cannot
contain a comma through the env var — use
`llm_headers = [...]` in config.toml for that.
diff --git a/docs/llm-providers.md b/docs/llm-providers.md
index 8fe94b27..21d98e7c 100644
--- a/docs/llm-providers.md
+++ b/docs/llm-providers.md
@@ -226,6 +226,24 @@ provider that exposes a real `/v1/embeddings` endpoint is the practical choice.
The two endpoints are still independent, so setting `AI_MEMORY_LLM_BASE_URL`
alone does not redirect embeddings — set `AI_MEMORY_EMBEDDING_BASE_URL` too.
+> [!TIP]
+> **`openai-compat` sends `X-Request-Id` on every chat attempt.** Each HTTP
+> attempt of one logical operation — including the strict-to-tolerant
+> fallback — carries the same operation id (a 36-character UUID v7) in the
+> `X-Request-Id` header, so a gateway that records it (for example as
+> `req=<id>`) can correlate every attempt of the same operation and forward
+> it to the engine (vLLM). The id is generated fresh by the caller of the
+> operation — the consolidation invocation or the auto-improve review — and
+> is never the agent's session id, so a gateway never sees two operations
+> of one session, or a crash-resumed run, as one. Crash re-entry is exactly
+> one new operation: a process restart or a queue re-claim mints a new id
+> for the calls it makes. The official `openai` provider and `opencode` do
+> not send this header — `opencode` keeps its own `x-opencode-session`
+> contract. Because ai-memory owns the header on the `openai-compat` path, a
+> static `x-request-id` entry in `AI_MEMORY_LLM_HEADERS` is refused at
+> startup for that provider; the dynamic value must not be shadowed or
+> duplicated.
+
### Security and gotchas
- **Bind address vs. port publish.** When running inside a container, the