Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
feat(embedding): add query/document prefix support for openai and openai-compat embedders
seathatflowsinourveins0fff203e8c74
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 188df552..804837d4 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,7 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### Added+- `embedding_query_prefix` / `embedding_document_prefix` config keys (env:+ `AI_MEMORY_EMBEDDING_QUERY_PREFIX` / `AI_MEMORY_EMBEDDING_DOCUMENT_PREFIX`)+ for the `openai` and `openai-compat` embedders: an optional string+ prepended to query / document text before the existing truncation, so+ truncation still bounds the whole input. Asymmetric embedding models —+ Nemotron-3-Embed, the E5 family, Qwen3-Embedding — need a `"query: "` /+ `"passage: "` instruction their publisher specifies; the+ OpenAI-compatible `/v1/embeddings` wire format has no field for it.+ Empty by default — no behaviour change when unset, and not trimmed, so a+ publisher's trailing space is preserved. Changing a prefix does not+ change the stored `{provider, model, dim}` triple pages are keyed by;+ run `ai-memory embed --force` to re-embed after changing one. See+ `docs/llm-providers.md`.+### Fixed+- `memory_query`'s vector stream called the generic `Embedder::embed`+ instead of `embed_query` on the configured embedder, so a+ query/document-asymmetric embedder (Google's task-typed embeddings, or+ the new query/document prefixes above) embedded the search query on the+ document side instead of the query side.- `companions/ai-memory-macos/build.sh` no longer fails on machines whoseactive developer directory is Command Line Tools only: SwiftUI `@State`needs the `SwiftUIMacros` plugin shipped with full Xcode, so the scriptdiff --git a/bin/ai-memory b/bin/ai-memoryindex 18b6dcc1..3e7660b0 100755--- a/bin/ai-memory+++ b/bin/ai-memory@@ -586,6 +586,8 @@ for var in \AI_MEMORY_EMBEDDING_MODEL \AI_MEMORY_EMBEDDING_BASE_URL \AI_MEMORY_EMBEDDING_DIM \+ AI_MEMORY_EMBEDDING_QUERY_PREFIX \+ AI_MEMORY_EMBEDDING_DOCUMENT_PREFIX \AI_MEMORY_ALLOWED_HOSTS \AI_MEMORY_WORKSTREAM_ID \AI_MEMORY_HOOK_PLATFORM \diff --git a/bin/ai-memory.ps1 b/bin/ai-memory.ps1index 34f9cbc4..bb37ca87 100644--- a/bin/ai-memory.ps1+++ b/bin/ai-memory.ps1@@ -150,6 +150,8 @@ foreach ($Name in @("AI_MEMORY_EMBEDDING_MODEL","AI_MEMORY_EMBEDDING_BASE_URL","AI_MEMORY_EMBEDDING_DIM",+ "AI_MEMORY_EMBEDDING_QUERY_PREFIX",+ "AI_MEMORY_EMBEDDING_DOCUMENT_PREFIX","AI_MEMORY_ALLOWED_HOSTS","AI_MEMORY_WORKSTREAM_ID","CLAUDE_CONFIG_DIR",diff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rsindex 1e3eae2e..40022eba 100644--- a/crates/ai-memory-cli/src/config.rs+++ b/crates/ai-memory-cli/src/config.rs@@ -441,6 +441,26 @@ pub struct Config {pub embedding_dim: Option<u32>,/// Optional embedding base URL override.pub embedding_base_url: Option<String>,+ /// Optional prefix prepended to every embedding **query** before it is+ /// sent to the `openai` or `openai-compat` embedder, ahead of the+ /// existing truncation. Unset (the default) is a no-op — no behaviour+ /// change. Asymmetric self-hosted models — Nemotron-3-Embed, the E5+ /// family, Qwen3-Embedding — need a `"query: "` instruction their+ /// publisher specifies; the OpenAI-compatible `/v1/embeddings` wire+ /// format has no field for it, so the client prepends it instead. Not+ /// trimmed: a publisher's trailing space (e.g. `"query: "`) is+ /// significant and preserved verbatim. Ignored by `google` (which has+ /// its own built-in query/document asymmetry), `voyage`, `local`, and+ /// `copilot`. Changing this does not change the stored+ /// `{provider, model, dim}` triple pages are keyed by — run+ /// `ai-memory embed --force` to re-embed after changing it. See+ /// `docs/llm-providers.md`. Settable via+ /// `AI_MEMORY_EMBEDDING_QUERY_PREFIX`.+ pub embedding_query_prefix: Option<String>,+ /// Document-side counterpart of `embedding_query_prefix` (e.g.+ /// `"passage: "` for Nemotron-3-Embed / E5). Settable via+ /// `AI_MEMORY_EMBEDDING_DOCUMENT_PREFIX`.+ pub embedding_document_prefix: Option<String>,/// M8 retention-sweep parameters. The defaults give an ~80-day/// "survival floor" for unused episodic content (above the cold/// threshold), followed by ~180 days of tombstone grace before permanent@@ -875,6 +895,8 @@ impl Default for Config {embedding_model: None,embedding_dim: None,embedding_base_url: None,+ embedding_query_prefix: None,+ embedding_document_prefix: None,decay: DecaySettings::default(),maintenance: MaintenanceSettings::default(),dream: DreamSettings::default(),@@ -1881,6 +1903,10 @@ impl Config {} else {None};+ // Not `non_empty`: that trims, and a publisher's trailing space+ // (e.g. Nemotron-3-Embed / E5's `"query: "`) is significant.+ let query_prefix = self.embedding_query_prefix.clone().unwrap_or_default();+ let document_prefix = self.embedding_document_prefix.clone().unwrap_or_default();Ok(Some(EmbedderConfig {provider,model,@@ -1890,6 +1916,8 @@ impl Config {models_dir: Some(self.data_dir.join("models")),copilot_auth,defaulted,+ query_prefix,+ document_prefix,}))}@@ -3219,6 +3247,33 @@ mod tests {));}+ #[test]+ fn embedding_prefixes_default_empty_and_are_not_trimmed_when_set() {+ // Unset: EmbedderConfig carries empty strings, so downstream+ // embedders see byte-identical behaviour to before this feature.+ let unset = Config {+ embedding_provider: Some("openai-compat".into()),+ embedding_model: Some("nvidia/nemotron-3-embed".into()),+ embedding_dim: Some(2048),+ embedding_base_url: Some("http://localhost:8000/v1".into()),+ ..Config::default()+ };+ let embedder = unset.embedder_config().unwrap().unwrap();+ assert_eq!(embedder.query_prefix, "");+ assert_eq!(embedder.document_prefix, "");++ // Set: the publisher's exact strings pass through, including the+ // significant trailing space — `non_empty`'s trim would corrupt it.+ let set = Config {+ embedding_query_prefix: Some("query: ".into()),+ embedding_document_prefix: Some("passage: ".into()),+ ..unset+ };+ let embedder = set.embedder_config().unwrap().unwrap();+ assert_eq!(embedder.query_prefix, "query: ");+ assert_eq!(embedder.document_prefix, "passage: ");+ }+#[test]fn copilot_embedding_defaults_model_dim_and_reuses_copilot_auth() {let tmp = TempDir::new().unwrap();diff --git a/crates/ai-memory-llm/src/embedding.rs b/crates/ai-memory-llm/src/embedding.rsindex f2abc1ba..b0607fc3 100644--- a/crates/ai-memory-llm/src/embedding.rs+++ b/crates/ai-memory-llm/src/embedding.rs@@ -67,6 +67,8 @@ pub struct OpenAiEmbedder {base_url: String,model: String,dim: u32,+ query_prefix: String,+ document_prefix: String,}impl OpenAiEmbedder {@@ -90,6 +92,8 @@ impl OpenAiEmbedder {base_url: "https://api.openai.com".into(),model: model.into(),dim,+ query_prefix: String::new(),+ document_prefix: String::new(),})}@@ -99,6 +103,20 @@ impl OpenAiEmbedder {self.base_url = url.into();self}++ /// Set the query/document prefixes prepended before embedding (see+ /// [`OpenAiCompatEmbedder::with_prefixes`]). Empty strings are a no-op,+ /// so this is safe to call unconditionally with the configured values.+ #[must_use]+ pub fn with_prefixes(+ mut self,+ query_prefix: impl Into<String>,+ document_prefix: impl Into<String>,+ ) -> Self {+ self.query_prefix = query_prefix.into();+ self.document_prefix = document_prefix.into();+ self+ }}#[derive(Debug, Serialize)]@@ -277,6 +295,32 @@ impl Embedder for OpenAiEmbedder {).await}++ async fn embed_document(&self, text: &str) -> LlmResult<Vec<f32>> {+ let prefixed = prepend_prefix(&self.document_prefix, text);+ openai_style_embed(+ &self.client,+ &self.base_url,+ Some(&self.api_key),+ &self.model,+ self.dim,+ &prefixed,+ )+ .await+ }++ async fn embed_query(&self, text: &str) -> LlmResult<Vec<f32>> {+ let prefixed = prepend_prefix(&self.query_prefix, text);+ openai_style_embed(+ &self.client,+ &self.base_url,+ Some(&self.api_key),+ &self.model,+ self.dim,+ &prefixed,+ )+ .await+ }}/// OpenAI-compatible embeddings endpoint (Ollama / LM Studio / vLLM /@@ -291,6 +335,16 @@ pub struct OpenAiCompatEmbedder {base_url: String,model: String,dim: u32,+ /// Prepended to every query text before embedding (before truncation).+ /// Empty by default: symmetric models (most OpenAI-compatible servers)+ /// see no behaviour change. Asymmetric models such as Nemotron-3-Embed,+ /// the E5 family, and Qwen3-Embedding require a `"query: "` /+ /// `"passage: "` (or model-specific) instruction string that the+ /// OpenAI-compatible `/v1/embeddings` wire format has no field for —+ /// the client has to prepend it instead. See `with_prefixes`.+ query_prefix: String,+ /// Document-side counterpart of `query_prefix` (e.g. `"passage: "`).+ document_prefix: String,}impl OpenAiCompatEmbedder {@@ -316,8 +370,30 @@ impl OpenAiCompatEmbedder {base_url: base_url.into(),model: model.into(),dim,+ query_prefix: String::new(),+ document_prefix: String::new(),})}++ /// Set the strings prepended to query / document text before embedding,+ /// applied ahead of the existing truncation so truncation still bounds+ /// the whole request body (prefix included). Pass empty strings for no+ /// prefix (the default); safe to call unconditionally with a possibly+ /// unset operator setting.+ ///+ /// Publisher-specified prefixes are exact strings, often with a+ /// significant trailing space (e.g. Nemotron-3-Embed / the E5 family+ /// use `"query: "` and `"passage: "`) — callers must not trim them.+ #[must_use]+ pub fn with_prefixes(+ mut self,+ query_prefix: impl Into<String>,+ document_prefix: impl Into<String>,+ ) -> Self {+ self.query_prefix = query_prefix.into();+ self.document_prefix = document_prefix.into();+ self+ }}#[async_trait]@@ -345,6 +421,32 @@ impl Embedder for OpenAiCompatEmbedder {).await}++ async fn embed_document(&self, text: &str) -> LlmResult<Vec<f32>> {+ let prefixed = prepend_prefix(&self.document_prefix, text);+ openai_style_embed(+ &self.client,+ &self.base_url,+ self.api_key.as_ref(),+ &self.model,+ self.dim,+ &prefixed,+ )+ .await+ }++ async fn embed_query(&self, text: &str) -> LlmResult<Vec<f32>> {+ let prefixed = prepend_prefix(&self.query_prefix, text);+ openai_style_embed(+ &self.client,+ &self.base_url,+ self.api_key.as_ref(),+ &self.model,+ self.dim,+ &prefixed,+ )+ .await+ }}/// Voyage Embeddings API.@@ -501,6 +603,20 @@ impl Embedder for SyntheticEmbedder {}}+/// Prepend `prefix` to `text` for an asymmetric embedding model, applied+/// BEFORE [`truncate_for_embedding`] so truncation always bounds the whole+/// request body (prefix included) rather than letting a prefix push the+/// text itself past the server's token limit. An empty prefix — the+/// default — returns `text` unchanged and borrowed, so the unset case+/// allocates nothing.+pub(crate) fn prepend_prefix<'a>(prefix: &str, text: &'a str) -> std::borrow::Cow<'a, str> {+ if prefix.is_empty() {+ std::borrow::Cow::Borrowed(text)+ } else {+ std::borrow::Cow::Owned(format!("{prefix}{text}"))+ }+}+/// Unit-normalise so dot-product equals cosine similarity.pub(crate) fn normalise(mut v: Vec<f32>) -> Vec<f32> {let norm: f32 = v.iter().map(|x| x * x).sum::<f32>().sqrt();@@ -592,4 +708,65 @@ mod tests {let err = parse_openai_embedding_values(body, 200).unwrap_err();assert!(matches!(err, LlmError::Provider { status: 200, .. }));}++ #[test]+ fn prepend_prefix_applies_publisher_instruction() {+ // Nemotron-3-Embed / the E5 family: exact publisher strings,+ // including the significant trailing space.+ assert_eq!(+ prepend_prefix("query: ", "find the release runbook"),+ "query: find the release runbook"+ );+ assert_eq!(+ prepend_prefix("passage: ", "the release runbook says..."),+ "passage: the release runbook says..."+ );+ }++ #[test]+ fn prepend_prefix_unset_leaves_text_unchanged() {+ // The default (empty prefix) must be a true no-op, byte for byte,+ // and borrow rather than allocate.+ let text = "unchanged text";+ let out = prepend_prefix("", text);+ assert_eq!(out, text);+ assert!(matches!(out, std::borrow::Cow::Borrowed(_)));+ }++ #[test]+ fn prefix_is_applied_before_truncation_so_it_still_bounds_the_whole_input() {+ // A prefix pushes the *total* input closer to the cap; truncation+ // must run on prefix+text together, never on text alone with the+ // prefix appended afterwards (which could exceed the server limit).+ let long_text = "x".repeat(50_000);+ let prefixed = prepend_prefix("passage: ", &long_text);+ let truncated = truncate_for_embedding(&prefixed, OPENAI_EMBED_MAX_TOKENS);+ assert!(truncated.starts_with("passage: "));+ assert!(truncated.len() <= 8_000, "must respect the hard byte cap");+ assert!(truncated.ends_with('…'));+ }++ #[test]+ fn openai_compat_embedder_defaults_to_no_prefix() {+ // Construction without `with_prefixes` must be byte-identical to+ // before this feature existed.+ let e = OpenAiCompatEmbedder::new("http://localhost:9/v1", None, "nomic-embed-text", 8)+ .expect("embedder builds");+ assert_eq!(e.query_prefix, "");+ assert_eq!(e.document_prefix, "");+ }++ #[test]+ fn openai_compat_embedder_stores_configured_prefixes() {+ let e = OpenAiCompatEmbedder::new(+ "http://localhost:9/v1",+ None,+ "nvidia/nemotron-3-embed",+ 2048,+ )+ .expect("embedder builds")+ .with_prefixes("query: ", "passage: ");+ assert_eq!(e.query_prefix, "query: ");+ assert_eq!(e.document_prefix, "passage: ");+ }}diff --git a/crates/ai-memory-llm/src/factory.rs b/crates/ai-memory-llm/src/factory.rsindex 7ad56ddc..c049e97b 100644--- a/crates/ai-memory-llm/src/factory.rs+++ b/crates/ai-memory-llm/src/factory.rs@@ -206,6 +206,17 @@ pub struct EmbedderConfig {/// a warning instead of refusing to start; an explicitly configured/// one still fails hard.pub defaulted: bool,+ /// Prepended to every query text before embedding, ahead of the+ /// existing truncation. Empty (the default) is a no-op. Only the+ /// `openai` and `openai-compat` embedders apply it; other providers+ /// (`google` has its own built-in task-type asymmetry, `voyage`,+ /// `local`, `copilot`) ignore it. Needed for asymmetric self-hosted+ /// models — Nemotron-3-Embed, the E5 family, Qwen3-Embedding — whose+ /// publisher specifies a `"query: "` instruction the OpenAI-compatible+ /// `/v1/embeddings` wire format has no field for.+ pub query_prefix: String,+ /// Document-side counterpart of `query_prefix` (e.g. `"passage: "`).+ pub document_prefix: String,}/// Construct an `Arc<dyn Embedder>` from the config.@@ -221,7 +232,8 @@ pub fn build_embedder(config: EmbedderConfig) -> LlmResult<Arc<dyn Embedder>> {}let arc: Arc<dyn Embedder> = match config.provider {EmbedderChoice::OpenAi => {- let mut e = OpenAiEmbedder::new(config.api_key, config.model, config.dim)?;+ let mut e = OpenAiEmbedder::new(config.api_key, config.model, config.dim)?+ .with_prefixes(config.query_prefix, config.document_prefix);if let Some(url) = config.base_url {e = e.with_base_url(url);}@@ -246,12 +258,10 @@ pub fn build_embedder(config: EmbedderConfig) -> LlmResult<Arc<dyn Embedder>> {.base_url.ok_or_else(|| LlmError::NotConfigured("AI_MEMORY_EMBEDDING_BASE_URL".into()))?;let api_key = (!config.api_key.expose_secret().is_empty()).then_some(config.api_key);- Arc::new(OpenAiCompatEmbedder::new(- base,- api_key,- config.model,- config.dim,- )?)+ Arc::new(+ OpenAiCompatEmbedder::new(base, api_key, config.model, config.dim)?+ .with_prefixes(config.query_prefix, config.document_prefix),+ )}#[cfg(feature = "local-embeddings")]EmbedderChoice::Local => {diff --git a/crates/ai-memory-llm/tests/suite/openai_compat_embedder.rs b/crates/ai-memory-llm/tests/suite/openai_compat_embedder.rsindex 2cd5021b..08f2e370 100644--- a/crates/ai-memory-llm/tests/suite/openai_compat_embedder.rs+++ b/crates/ai-memory-llm/tests/suite/openai_compat_embedder.rs@@ -87,6 +87,8 @@ async fn factory_builds_compat_embedder_and_requires_base_url() {models_dir: None,copilot_auth: None,defaulted: false,+ query_prefix: String::new(),+ document_prefix: String::new(),}).expect("factory builds compat embedder");assert_eq!(ok.provider(), "openai-compat");@@ -101,6 +103,8 @@ async fn factory_builds_compat_embedder_and_requires_base_url() {models_dir: None,copilot_auth: None,defaulted: false,+ query_prefix: String::new(),+ document_prefix: String::new(),}) {Ok(_) => panic!("compat embedder must not build without a base URL"),Err(err) => err,@@ -119,6 +123,8 @@ async fn factory_builds_compat_embedder_and_requires_base_url() {models_dir: None,copilot_auth: None,defaulted: false,+ query_prefix: String::new(),+ document_prefix: String::new(),});assert!(matches!(zero_dim, Err(LlmError::NotConfigured(ref msg)) if msg.contains("greater than zero")),@@ -126,6 +132,156 @@ async fn factory_builds_compat_embedder_and_requires_base_url() {);}+#[tokio::test]+async fn query_prefix_is_sent_with_embed_query_not_embed_document() {+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(path("/v1/embeddings"))+ .respond_with(move |req: &Request| {+ let body: serde_json::Value = serde_json::from_slice(&req.body).expect("json body");+ assert_eq!(+ body["input"], "query: find the runbook",+ "embed_query must send the query prefix ahead of the text"+ );+ ResponseTemplate::new(200).set_body_json(embedding_body(8))+ })+ .expect(1)+ .mount(&server)+ .await;++ let e = OpenAiCompatEmbedder::new(format!("{}/v1", server.uri()), None, "nomic-embed-text", 8)+ .expect("embedder builds")+ .with_prefixes("query: ", "passage: ");++ e.embed_query("find the runbook")+ .await+ .expect("embed_query succeeds");+}++#[tokio::test]+async fn document_prefix_is_sent_with_embed_document_not_embed_query() {+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(path("/v1/embeddings"))+ .respond_with(move |req: &Request| {+ let body: serde_json::Value = serde_json::from_slice(&req.body).expect("json body");+ assert_eq!(+ body["input"], "passage: the runbook says...",+ "embed_document must send the document prefix ahead of the text"+ );+ ResponseTemplate::new(200).set_body_json(embedding_body(8))+ })+ .expect(1)+ .mount(&server)+ .await;++ let e = OpenAiCompatEmbedder::new(format!("{}/v1", server.uri()), None, "nomic-embed-text", 8)+ .expect("embedder builds")+ .with_prefixes("query: ", "passage: ");++ e.embed_document("the runbook says...")+ .await+ .expect("embed_document succeeds");+}++#[tokio::test]+async fn unset_prefixes_leave_embed_query_and_embed_document_unchanged() {+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(path("/v1/embeddings"))+ .respond_with(move |req: &Request| {+ let body: serde_json::Value = serde_json::from_slice(&req.body).expect("json body");+ assert_eq!(+ body["input"], "plain text",+ "an unset prefix (the default) must not change the request body"+ );+ ResponseTemplate::new(200).set_body_json(embedding_body(8))+ })+ .expect(2)+ .mount(&server)+ .await;++ // No `with_prefixes` call at all — construction alone must be+ // byte-identical to before this feature existed.+ let e = OpenAiCompatEmbedder::new(format!("{}/v1", server.uri()), None, "nomic-embed-text", 8)+ .expect("embedder builds");++ e.embed_query("plain text")+ .await+ .expect("embed_query succeeds");+ e.embed_document("plain text")+ .await+ .expect("embed_document succeeds");+}++#[tokio::test]+async fn prefix_is_truncated_together_with_the_text_it_precedes() {+ // Truncation must bound prefix+text together; a prefix must never let+ // the combined request exceed the server's input cap.+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(path("/v1/embeddings"))+ .respond_with(move |req: &Request| {+ let body: serde_json::Value = serde_json::from_slice(&req.body).expect("json body");+ let input = body["input"].as_str().expect("input is a string");+ assert!(+ input.starts_with("passage: "),+ "prefix must survive truncation: {input:.60}..."+ );+ assert!(+ input.len() <= 8_000,+ "prefix+text must stay within the same hard byte cap as before"+ );+ ResponseTemplate::new(200).set_body_json(embedding_body(8))+ })+ .expect(1)+ .mount(&server)+ .await;++ let e = OpenAiCompatEmbedder::new(format!("{}/v1", server.uri()), None, "nomic-embed-text", 8)+ .expect("embedder builds")+ .with_prefixes("query: ", "passage: ");++ let long_text = "x".repeat(50_000);+ e.embed_document(&long_text)+ .await+ .expect("embed_document succeeds");+}++#[tokio::test]+async fn factory_wires_configured_prefixes_into_the_compat_embedder() {+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(path("/v1/embeddings"))+ .respond_with(move |req: &Request| {+ let body: serde_json::Value = serde_json::from_slice(&req.body).expect("json body");+ assert_eq!(body["input"], "query: via factory");+ ResponseTemplate::new(200).set_body_json(embedding_body(8))+ })+ .expect(1)+ .mount(&server)+ .await;++ let embedder = build_embedder(EmbedderConfig {+ provider: EmbedderChoice::OpenAiCompat,+ model: "nomic-embed-text".into(),+ dim: 8,+ api_key: SecretString::from(String::new()),+ base_url: Some(format!("{}/v1", server.uri())),+ models_dir: None,+ copilot_auth: None,+ defaulted: false,+ query_prefix: "query: ".into(),+ document_prefix: "passage: ".into(),+ })+ .expect("factory builds compat embedder with prefixes");++ embedder+ .embed_query("via factory")+ .await+ .expect("embed_query succeeds");+}+#[test]fn existing_default_dimension_api_remains_source_compatible() {assert_eq!(diff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rsindex 3b812f8b..fedd807c 100644--- a/crates/ai-memory-mcp/src/server.rs+++ b/crates/ai-memory-mcp/src/server.rs@@ -1964,7 +1964,10 @@ impl AiMemoryServer {let Some(embedder) = &self.embedder else {return None;};- match embedder.embed(query).await {+ // `embed_query`, not `embed`: an asymmetric embedder (configured+ // query/document prefixes, or Google's RETRIEVAL_QUERY task type)+ // must see this text on the query side, not the document side.+ match embedder.embed_query(query).await {Ok(qv) => Some(qv),Err(e) => {tracing::warn!(diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex 7396f265..2649845b 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -864,6 +864,12 @@ AI_MEMORY_EMBEDDING_MODEL e.g. text-embedding-3-small, gemini-embedding-001AI_MEMORY_EMBEDDING_BASE_URL optional override; required for openai-compatAI_MEMORY_EMBEDDING_DIM 1536 (OpenAI, Copilot), 1024 (Voyage), 768 (Google);required explicitly for openai-compat+AI_MEMORY_EMBEDDING_QUERY_PREFIX optional; prepended to query text before+ embedding (openai / openai-compat only)+AI_MEMORY_EMBEDDING_DOCUMENT_PREFIX optional; prepended to document text+ before embedding (openai / openai-compat+ only); e.g. "query: " / "passage: " for+ Nemotron-3-Embed / base E5OPENAI_API_KEY / VOYAGE_API_KEY / GEMINI_API_KEY / GOOGLE_API_KEYLLM_API_KEY accepted for openai with a custom base URL and asoptional bearer auth for openai-compatdiff --git a/docs/install.md b/docs/install.mdindex ff774284..f0bf80f3 100644--- a/docs/install.md+++ b/docs/install.md@@ -1641,7 +1641,7 @@ If you set only the provider, ai-memory picks a sensible default:| `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `AI_MEMORY_EMBEDDING_BASE_URL=https://api.orcarouter.ai/v1` | `openai/text-embedding-3-small` via [OrcaRouter](https://www.orcarouter.ai) | Uses `EMBEDDING_API_KEY`, else reuses `LLM_API_KEY`, with the OpenAI-compatible embedding client. || `AI_MEMORY_EMBEDDING_PROVIDER=voyage` | `voyage-3` (1024-dim) | Voyage's current general-purpose recommendation. || `AI_MEMORY_EMBEDDING_PROVIDER=google` / `gemini` | `gemini-embedding-001` (768-dim) | Google-hosted embeddings via `embedContent`. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). |-| `AI_MEMORY_EMBEDDING_PROVIDER=openai-compat` | no default — set model, dim, and base URL explicitly | Self-hosted engines (Ollama, LM Studio, vLLM). Keyless by default; `EMBEDDING_API_KEY`, else `LLM_API_KEY`, is sent as a bearer token when present (gateways). Example: `AI_MEMORY_EMBEDDING_BASE_URL=http://localhost:11434/v1`, `AI_MEMORY_EMBEDDING_MODEL=nomic-embed-text`, `AI_MEMORY_EMBEDDING_DIM=768`. Switching an existing `openai`+base-URL setup to `openai-compat` changes the stored `{provider, model, dim}` triple — run `ai-memory embed --force` to re-embed. |+| `AI_MEMORY_EMBEDDING_PROVIDER=openai-compat` | no default — set model, dim, and base URL explicitly | Self-hosted engines (Ollama, LM Studio, vLLM). Keyless by default; `EMBEDDING_API_KEY`, else `LLM_API_KEY`, is sent as a bearer token when present (gateways). Example: `AI_MEMORY_EMBEDDING_BASE_URL=http://localhost:11434/v1`, `AI_MEMORY_EMBEDDING_MODEL=nomic-embed-text`, `AI_MEMORY_EMBEDDING_DIM=768`. Switching an existing `openai`+base-URL setup to `openai-compat` changes the stored `{provider, model, dim}` triple — run `ai-memory embed --force` to re-embed. Asymmetric models need `AI_MEMORY_EMBEDDING_QUERY_PREFIX` / `AI_MEMORY_EMBEDDING_DOCUMENT_PREFIX`, e.g. `nvidia/nemotron-3-embed` wants `query: ` / `passage: ` — see [`docs/llm-providers.md`](llm-providers.md). || `AI_MEMORY_EMBEDDING_PROVIDER=copilot` | `text-embedding-3-small` (1536-dim) | Reuses the `copilot` LLM provider's OAuth login (`ai-memory auth login copilot`, `COPILOT_GITHUB_TOKEN`, or `GITHUB_COPILOT_API_TOKEN`) — no separate API key. Calls Copilot's `/embeddings` endpoint following the OpenAI-compatible contract Copilot documents for chat; that endpoint's exact shape is not covered by a live test against Copilot here, so treat it as needing a real-Copilot smoke test. |> **What we don't recommend:** reasoning-mode models (Claude with extendeddiff --git a/docs/llm-providers.md b/docs/llm-providers.mdindex 3d0e69e5..bd5cacb8 100644--- a/docs/llm-providers.md+++ b/docs/llm-providers.md@@ -210,6 +210,34 @@ hybrid paths apply the same bounded page-authority adjustment after candidategeneration; embeddings improve relevance recall but do not decide which sourceis canonical.+Asymmetric embedding models — ones trained with a different instruction for+queries than for the text being indexed — need that instruction added to the+text itself; the OpenAI-compatible `/v1/embeddings` wire format has no field+for it. `AI_MEMORY_EMBEDDING_QUERY_PREFIX` and+`AI_MEMORY_EMBEDDING_DOCUMENT_PREFIX` (only applied by the `openai` and+`openai-compat` embedders) are prepended to query and document text+respectively, before the existing truncation, so a long body is still+truncated to the same overall input cap with the prefix included. Both are+empty by default — no behaviour change when unset — and are not trimmed, so+a publisher's trailing space is preserved exactly. For example, NVIDIA's+`nemotron-3-embed` (served locally through vLLM/`openai-compat`) specifies+`"query: "` and `"passage: "`:++```toml+embedding_provider = "openai-compat"+embedding_model = "nvidia/nemotron-3-embed"+embedding_base_url = "http://localhost:8000/v1"+embedding_dim = 2048+embedding_query_prefix = "query: "+embedding_document_prefix = "passage: "+```++The E5 family and Qwen3-Embedding use the same `"query: "` / `"passage: "`+convention. Changing either prefix does not change the stored+`{provider, model, dim}` triple pages are keyed by, so existing vectors keep+matching on the mismatch check but were embedded under the old (or no)+prefix; run `ai-memory embed --force` to re-embed after changing one.+`AI_MEMORY_EMBEDDING_PROVIDER=copilot` reuses the same Copilot OAuth login asthe `copilot` LLM provider (`ai-memory auth login copilot` or`COPILOT_GITHUB_TOKEN`/`GITHUB_COPILOT_API_TOKEN`) — no separate API key.