Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
feat(llm): openai-compat embedder for self-hosted engines
Samir Hanna Verza3b6a768a6ea7
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5250259b..a86bef82 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### Added+- New `openai-compat` embedding provider for self-hosted engines+ (Ollama, LM Studio, vLLM). Set+ `AI_MEMORY_EMBEDDING_PROVIDER=openai-compat` together with explicit+ `AI_MEMORY_EMBEDDING_BASE_URL`, `AI_MEMORY_EMBEDDING_MODEL`, and+ `AI_MEMORY_EMBEDDING_DIM` — there is no safe default model or+ dimensionality for a self-hosted engine, so each is required rather+ than guessed. Unlike the other providers it is keyless: a bearer+ token is sent only when `LLM_API_KEY` is present, for gateways that+ want one. Embeddings are stored under their own+ `provider="openai-compat"` identity, so switching an existing+ `openai`+base-URL setup over changes the stored+ `{provider, model, dim}` triple — run `ai-memory embed --force` to+ re-embed.+## [1.18.0] - 2026-07-23### Addeddiff --git a/README.md b/README.mdindex 7d8275dd..4b134f49 100644--- a/README.md+++ b/README.md@@ -671,8 +671,12 @@ also set `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `GITHUB_TOKEN` on the server.> default.Embeddings are optional and separate from the LLM provider. Set-`AI_MEMORY_EMBEDDING_PROVIDER=openai`, `voyage`, `google`, or `gemini` when-you want vector reranking in addition to FTS5 + graph-neighbor retrieval.+`AI_MEMORY_EMBEDDING_PROVIDER=openai`, `voyage`, `google`/`gemini`, or+`openai-compat` when you want vector reranking in addition to FTS5 ++graph-neighbor retrieval. `openai-compat` targets self-hosted engines+(Ollama, LM Studio, vLLM): it needs no API key and requires explicit+`AI_MEMORY_EMBEDDING_BASE_URL`, `AI_MEMORY_EMBEDDING_MODEL`, and+`AI_MEMORY_EMBEDDING_DIM`.See [`docs/install.md#llm-provider-tiers`](docs/install.md#llm-provider-tiers)for env vars and Ollama/OpenRouter/Atlas Cloud examples, anddiff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rsindex a1a67920..8f5afd8f 100644--- a/crates/ai-memory-cli/src/config.rs+++ b/crates/ai-memory-cli/src/config.rs@@ -739,9 +739,11 @@ impl Config {"openai" => EmbedderChoice::OpenAi,"voyage" => EmbedderChoice::Voyage,"google" | "gemini" => EmbedderChoice::Google,+ "openai-compat" | "openai_compat" => EmbedderChoice::OpenAiCompat,other => {return Err(LlmError::NotConfigured(format!(- "AI_MEMORY_EMBEDDING_PROVIDER={other} not one of openai|voyage|google|gemini"+ "AI_MEMORY_EMBEDDING_PROVIDER={other} not one of \+ openai|voyage|google|gemini|openai-compat")));}};@@ -751,28 +753,54 @@ impl Config {EmbedderChoice::OpenAi => "text-embedding-3-small".to_string(),EmbedderChoice::Voyage => "voyage-3".to_string(),EmbedderChoice::Google => ai_memory_llm::GOOGLE_DEFAULT_EMBED_MODEL.to_string(),+ EmbedderChoice::OpenAiCompat => {+ return Err(LlmError::NotConfigured(+ "AI_MEMORY_EMBEDDING_MODEL must be set explicitly for openai-compat \+ (no safe default for self-hosted engines)"+ .into(),+ ));+ }},};- let dim = self- .embedding_dim- .unwrap_or_else(|| ai_memory_llm::default_embedding_dim(provider, &model));- let api_key = match provider {- EmbedderChoice::OpenAi => self.openai_embedding_api_key()?,- EmbedderChoice::Voyage => self- .runtime_env- .voyage_api_key- .clone()- .ok_or_else(|| LlmError::NotConfigured("VOYAGE_API_KEY".into()))?,- EmbedderChoice::Google => self.runtime_env.gemini_api_key.clone().ok_or_else(|| {- LlmError::NotConfigured("GEMINI_API_KEY or GOOGLE_API_KEY".into())+ let dim = match self.embedding_dim {+ Some(d) => d,+ None => ai_memory_llm::default_embedding_dim(provider, &model).ok_or_else(|| {+ LlmError::NotConfigured(+ "AI_MEMORY_EMBEDDING_DIM must be set explicitly for openai-compat \+ (self-hosted model dims vary)"+ .into(),+ )})?,};+ let api_key = match provider {+ EmbedderChoice::OpenAi => Some(self.openai_embedding_api_key()?),+ EmbedderChoice::Voyage => Some(+ self.runtime_env+ .voyage_api_key+ .clone()+ .ok_or_else(|| LlmError::NotConfigured("VOYAGE_API_KEY".into()))?,+ ),+ EmbedderChoice::Google => {+ Some(self.runtime_env.gemini_api_key.clone().ok_or_else(|| {+ LlmError::NotConfigured("GEMINI_API_KEY or GOOGLE_API_KEY".into())+ })?)+ }+ // Keyless engines (Ollama, LM Studio) are the norm; a+ // gateway key rides on LLM_API_KEY when present.+ EmbedderChoice::OpenAiCompat => self.runtime_env.llm_api_key.clone(),+ };+ let base_url = self.embedding_base_url.clone();+ if provider == EmbedderChoice::OpenAiCompat && non_empty(base_url.as_deref()).is_none() {+ return Err(LlmError::NotConfigured(+ "AI_MEMORY_EMBEDDING_BASE_URL required for openai-compat embeddings".into(),+ ));+ }Ok(Some(EmbedderConfig {provider,model,dim,api_key,- base_url: self.embedding_base_url.clone(),+ base_url,}))}@@ -1172,13 +1200,77 @@ mod tests {let embedder = cfg.embedder_config().unwrap().unwrap();assert_eq!(embedder.provider, EmbedderChoice::OpenAi);assert_eq!(embedder.model, "text-embedding-3-small");- assert_eq!(embedder.api_key.expose_secret(), "sk-or-test-key");+ assert_eq!(+ embedder.api_key.as_ref().unwrap().expose_secret(),+ "sk-or-test-key"+ );assert_eq!(embedder.base_url.as_deref(),Some("https://openrouter.ai/api/v1"));}+ #[test]+ fn openai_compat_embedding_is_keyless_and_requires_explicit_settings() {+ // Fully specified, no key: valid (Ollama / LM Studio).+ let cfg = Config {+ embedding_provider: Some("openai-compat".into()),+ embedding_model: Some("nomic-embed-text".into()),+ embedding_dim: Some(768),+ embedding_base_url: Some("http://localhost:11434/v1".into()),+ ..Config::default()+ };+ let embedder = cfg.embedder_config().unwrap().unwrap();+ assert_eq!(embedder.provider, EmbedderChoice::OpenAiCompat);+ assert_eq!(embedder.model, "nomic-embed-text");+ assert_eq!(embedder.dim, 768);+ assert!(embedder.api_key.is_none());+ assert_eq!(+ embedder.base_url.as_deref(),+ Some("http://localhost:11434/v1")+ );++ // A gateway key rides on LLM_API_KEY when present.+ let cfg_with_key = Config {+ runtime_env: RuntimeEnv {+ llm_api_key: Some(SecretString::from("sk-or-key")),+ ..RuntimeEnv::default()+ },+ ..cfg.clone()+ };+ let embedder = cfg_with_key.embedder_config().unwrap().unwrap();+ assert_eq!(+ embedder.api_key.as_ref().unwrap().expose_secret(),+ "sk-or-key"+ );++ // Missing model / dim / base URL each fail closed.+ let missing_model = Config {+ embedding_model: None,+ ..cfg.clone()+ };+ assert!(matches!(+ missing_model.embedder_config().unwrap_err(),+ LlmError::NotConfigured(msg) if msg.contains("AI_MEMORY_EMBEDDING_MODEL")+ ));+ let missing_dim = Config {+ embedding_dim: None,+ ..cfg.clone()+ };+ assert!(matches!(+ missing_dim.embedder_config().unwrap_err(),+ LlmError::NotConfigured(msg) if msg.contains("AI_MEMORY_EMBEDDING_DIM")+ ));+ let missing_base = Config {+ embedding_base_url: None,+ ..cfg+ };+ assert!(matches!(+ missing_base.embedder_config().unwrap_err(),+ LlmError::NotConfigured(msg) if msg.contains("AI_MEMORY_EMBEDDING_BASE_URL")+ ));+ }+#[test]fn openai_embedding_does_not_use_llm_api_key_without_custom_base_url() {let cfg = Config {diff --git a/crates/ai-memory-llm/src/embedding.rs b/crates/ai-memory-llm/src/embedding.rsindex c0da0323..81a2d60e 100644--- a/crates/ai-memory-llm/src/embedding.rs+++ b/crates/ai-memory-llm/src/embedding.rs@@ -184,6 +184,73 @@ fn parse_openai_embedding_values(body: &str, status: u16) -> LlmResult<Vec<f32>>Ok(first.embedding)}+/// Shared OpenAI-shape embedding request: head-truncate, POST+/// `/embeddings` (bearer auth only when a key is present — local+/// engines such as Ollama and LM Studio run keyless), retry 429 with+/// backoff, parse, dim-check, unit-normalise.+async fn openai_style_embed(+ client: &reqwest::Client,+ base_url: &str,+ api_key: Option<&SecretString>,+ model: &str,+ dim: u32,+ text: &str,+) -> LlmResult<Vec<f32>> {+ let input = truncate_for_embedding(text, OPENAI_EMBED_MAX_TOKENS);+ let url = normalize_openai_base(base_url, "embeddings");+ debug!(+ url,+ model,+ input_chars = input.len(),+ "POST openai/embeddings"+ );+ let req = OpenAiEmbeddingRequest {+ input: &input,+ model,+ };+ let mut attempt = 0u32;+ loop {+ let mut builder = client.post(&url);+ if let Some(key) = api_key {+ builder = builder.bearer_auth(key.expose_secret());+ }+ let resp = builder.json(&req).send().await?;+ let status = resp.status();+ if status.as_u16() == 429 && attempt < 5 {+ attempt += 1;+ let delay = Duration::from_secs(2u64.saturating_pow(attempt));+ debug!(attempt, ?delay, "openai embeddings rate-limited; retrying");+ tokio::time::sleep(delay).await;+ continue;+ }+ // Client errors (e.g. input > 8192 tokens) are not retried.+ if status.as_u16() == 400 {+ let body = provider_error_body(resp).await;+ return Err(LlmError::Provider {+ status: status.as_u16(),+ body,+ });+ }+ if !status.is_success() {+ let body = provider_error_body(resp).await;+ return Err(LlmError::Provider {+ status: status.as_u16(),+ body,+ });+ }+ let body = response_text_limited(resp).await?;+ let values = parse_openai_embedding_values(&body, status.as_u16())?;+ if values.len() as u32 != dim {+ return Err(LlmError::UnexpectedShape(format!(+ "expected dim {}, got {}",+ dim,+ values.len()+ )));+ }+ return Ok(normalise(values));+ }+}+#[async_trait]impl Embedder for OpenAiEmbedder {fn provider(&self) -> &'static str {@@ -199,61 +266,83 @@ impl Embedder for OpenAiEmbedder {}async fn embed(&self, text: &str) -> LlmResult<Vec<f32>> {- let input = truncate_for_embedding(text, OPENAI_EMBED_MAX_TOKENS);- let url = normalize_openai_base(&self.base_url, "embeddings");- debug!(- url,- model = %self.model,- input_chars = input.len(),- "POST openai/embeddings"- );- let req = OpenAiEmbeddingRequest {- input: &input,- model: &self.model,- };- let mut attempt = 0u32;- loop {- let resp = self- .client- .post(&url)- .bearer_auth(self.api_key.expose_secret())- .json(&req)- .send()- .await?;- let status = resp.status();- if status.as_u16() == 429 && attempt < 5 {- attempt += 1;- let delay = Duration::from_secs(2u64.saturating_pow(attempt));- debug!(attempt, ?delay, "openai embeddings rate-limited; retrying");- tokio::time::sleep(delay).await;- continue;- }- // Client errors (e.g. input > 8192 tokens) are not retried.- if status.as_u16() == 400 {- let body = provider_error_body(resp).await;- return Err(LlmError::Provider {- status: status.as_u16(),- body,- });- }- if !status.is_success() {- let body = provider_error_body(resp).await;- return Err(LlmError::Provider {- status: status.as_u16(),- body,- });- }- let body = response_text_limited(resp).await?;- let values = parse_openai_embedding_values(&body, status.as_u16())?;- if values.len() as u32 != self.dim {- return Err(LlmError::UnexpectedShape(format!(- "expected dim {}, got {}",- self.dim,- values.len()- )));- }- return Ok(normalise(values));- }+ openai_style_embed(+ &self.client,+ &self.base_url,+ Some(&self.api_key),+ &self.model,+ self.dim,+ text,+ )+ .await+ }+}++/// OpenAI-compatible embeddings endpoint (Ollama / LM Studio / vLLM /+/// OpenRouter). Same wire shape as [`OpenAiEmbedder`], but the base+/// URL is required, the API key is optional (local engines run+/// keyless), and vectors are stored under the distinct provider label+/// `openai-compat` so the `{provider, model, dim}` refuse-on-mismatch+/// check treats them as their own family.+pub struct OpenAiCompatEmbedder {+ client: reqwest::Client,+ api_key: Option<SecretString>,+ base_url: String,+ model: String,+ dim: u32,+}++impl OpenAiCompatEmbedder {+ /// Construct a compat embedder against `base_url`.+ ///+ /// # Errors+ /// Propagates any `reqwest::Error` thrown while building the HTTP+ /// client.+ pub fn new(+ base_url: impl Into<String>,+ api_key: Option<SecretString>,+ model: impl Into<String>,+ dim: u32,+ ) -> LlmResult<Self> {+ // Same cold-load tolerance as OpenAiEmbedder: first request+ // after an Ollama model unload can take ~30s.+ let client = reqwest::Client::builder()+ .timeout(Duration::from_secs(120))+ .build()?;+ Ok(Self {+ client,+ api_key,+ base_url: base_url.into(),+ model: model.into(),+ dim,+ })+ }+}++#[async_trait]+impl Embedder for OpenAiCompatEmbedder {+ fn provider(&self) -> &'static str {+ "openai-compat"+ }++ fn model(&self) -> &str {+ &self.model+ }++ fn dim(&self) -> u32 {+ self.dim+ }++ async fn embed(&self, text: &str) -> LlmResult<Vec<f32>> {+ openai_style_embed(+ &self.client,+ &self.base_url,+ self.api_key.as_ref(),+ &self.model,+ self.dim,+ text,+ )+ .await}}diff --git a/crates/ai-memory-llm/src/factory.rs b/crates/ai-memory-llm/src/factory.rsindex 59a59f7c..b5acdff3 100644--- a/crates/ai-memory-llm/src/factory.rs+++ b/crates/ai-memory-llm/src/factory.rs@@ -15,7 +15,7 @@ use crate::OpenAiOAuthProvider;use crate::OpenAiProvider;use crate::OpenCodeProvider;use crate::auth::{AuthRequirement, ProviderAuth};-use crate::embedding::{Embedder, OpenAiEmbedder, VoyageEmbedder};+use crate::embedding::{Embedder, OpenAiCompatEmbedder, OpenAiEmbedder, VoyageEmbedder};use crate::error::{LlmError, LlmResult};use crate::google::GoogleEmbedder;use crate::provider::LlmProvider;@@ -110,6 +110,9 @@ pub enum EmbedderChoice {Voyage,/// Google Gemini Embeddings API (`embedContent`).Google,+ /// OpenAI-compatible embeddings endpoint (Ollama / LM Studio /+ /// vLLM). Keyless-capable; base URL, model, and dim are required.+ OpenAiCompat,}impl EmbedderChoice {@@ -121,6 +124,7 @@ impl EmbedderChoice {Self::OpenAi => "openai",Self::Voyage => "voyage",Self::Google => "google",+ Self::OpenAiCompat => "openai-compat",}}}@@ -135,57 +139,86 @@ pub struct EmbedderConfig {/// Vector dimensionality. Refused on mismatch with the stored/// pages' dim.pub dim: u32,- /// API key.- pub api_key: SecretString,- /// Optional base URL override.+ /// API key. Required for openai/voyage/google; optional for+ /// openai-compat (local engines such as Ollama run keyless).+ pub api_key: Option<SecretString>,+ /// Optional base URL override. Required for openai-compat.pub base_url: Option<String>,}+impl EmbedderConfig {+ fn require_api_key(&self) -> LlmResult<SecretString> {+ self.api_key.clone().ok_or_else(|| {+ LlmError::NotConfigured(format!(+ "API key required for {} embeddings",+ self.provider.name()+ ))+ })+ }+}+/// Construct an `Arc<dyn Embedder>` from the config.////// # Errors-/// Propagates HTTP-client construction errors.+/// Returns [`LlmError::NotConfigured`] for a missing required API key+/// or base URL; propagates HTTP-client construction errors.pub fn build_embedder(config: EmbedderConfig) -> LlmResult<Arc<dyn Embedder>> {let arc: Arc<dyn Embedder> = match config.provider {EmbedderChoice::OpenAi => {- let mut e = OpenAiEmbedder::new(config.api_key, config.model, config.dim)?;+ let key = config.require_api_key()?;+ let mut e = OpenAiEmbedder::new(key, config.model, config.dim)?;if let Some(url) = config.base_url {e = e.with_base_url(url);}Arc::new(e)}EmbedderChoice::Voyage => {- let mut e = VoyageEmbedder::new(config.api_key, config.model, config.dim)?;+ let key = config.require_api_key()?;+ let mut e = VoyageEmbedder::new(key, config.model, config.dim)?;if let Some(url) = config.base_url {e = e.with_base_url(url);}Arc::new(e)}EmbedderChoice::Google => {- let mut e = GoogleEmbedder::new(config.api_key, config.model, config.dim)?;+ let key = config.require_api_key()?;+ let mut e = GoogleEmbedder::new(key, config.model, config.dim)?;if let Some(url) = config.base_url {e = e.with_base_url(url);}Arc::new(e)}+ EmbedderChoice::OpenAiCompat => {+ let base = config+ .base_url+ .ok_or_else(|| LlmError::NotConfigured("AI_MEMORY_EMBEDDING_BASE_URL".into()))?;+ Arc::new(OpenAiCompatEmbedder::new(+ base,+ config.api_key,+ config.model,+ config.dim,+ )?)+ }};Ok(arc)}/// Default dim for known embedding models. Used when the operator/// omits `AI_MEMORY_EMBEDDING_DIM`. Falls back to a model-family-/// default; unknown models still require an explicit dim.+/// default; openai-compat (unknown self-hosted models) and unknown+/// models return `None` and require an explicit dim.#[must_use]-pub fn default_embedding_dim(provider: EmbedderChoice, model: &str) -> u32 {+pub fn default_embedding_dim(provider: EmbedderChoice, model: &str) -> Option<u32> {match (provider, model) {- (EmbedderChoice::OpenAi, "text-embedding-3-small") => 1536,- (EmbedderChoice::OpenAi, "text-embedding-3-large") => 3072,- (EmbedderChoice::OpenAi, _) => 1536,- (EmbedderChoice::Voyage, "voyage-3-large") => 1024,- (EmbedderChoice::Voyage, _) => 1024,- (EmbedderChoice::Google, "gemini-embedding-2") => 768,- (EmbedderChoice::Google, "gemini-embedding-001") => 768,- (EmbedderChoice::Google, _) => 768,+ (EmbedderChoice::OpenAi, "text-embedding-3-small") => Some(1536),+ (EmbedderChoice::OpenAi, "text-embedding-3-large") => Some(3072),+ (EmbedderChoice::OpenAi, _) => Some(1536),+ (EmbedderChoice::Voyage, "voyage-3-large") => Some(1024),+ (EmbedderChoice::Voyage, _) => Some(1024),+ (EmbedderChoice::Google, "gemini-embedding-2") => Some(768),+ (EmbedderChoice::Google, "gemini-embedding-001") => Some(768),+ (EmbedderChoice::Google, _) => Some(768),+ (EmbedderChoice::OpenAiCompat, _) => None,}}diff --git a/crates/ai-memory-llm/src/lib.rs b/crates/ai-memory-llm/src/lib.rsindex b5f20645..9879e7ee 100644--- a/crates/ai-memory-llm/src/lib.rs+++ b/crates/ai-memory-llm/src/lib.rs@@ -54,7 +54,9 @@ pub use copilot::{GITHUB_ACCESS_TOKEN_URL, GITHUB_COPILOT_CLIENT_ID, GITHUB_COPILOT_TOKEN_URL,GITHUB_DEVICE_CODE_URL,};-pub use embedding::{Embedder, OpenAiEmbedder, SyntheticEmbedder, VoyageEmbedder, cosine};+pub use embedding::{+ Embedder, OpenAiCompatEmbedder, OpenAiEmbedder, SyntheticEmbedder, VoyageEmbedder, cosine,+};pub use error::{LlmError, LlmResult};pub use factory::{EmbedderChoice, EmbedderConfig, ProviderChoice, ProviderConfig, build_embedder, build_provider,diff --git a/crates/ai-memory-llm/tests/openai_compat_embedder.rs b/crates/ai-memory-llm/tests/openai_compat_embedder.rsnew file mode 100644index 00000000..2851fbb6--- /dev/null+++ b/crates/ai-memory-llm/tests/openai_compat_embedder.rs@@ -0,0 +1,105 @@+//! Integration tests for `OpenAiCompatEmbedder` against an in-process+//! HTTP mock (wiremock).+//!+//! The load-bearing behaviours: (1) keyless requests carry NO+//! `Authorization` header — local engines such as Ollama and LM Studio+//! reject or ignore malformed auth, and sending a fabricated bearer+//! token would leak the assumption that a key always exists; (2) when+//! a gateway key IS configured it is sent as a bearer token; (3) the+//! embedder identifies as provider `openai-compat`, so stored+//! `{provider, model, dim}` triples are a distinct family from plain+//! `openai`; (4) the factory refuses to build without a base URL.++use ai_memory_llm::{+ Embedder, EmbedderChoice, EmbedderConfig, LlmError, OpenAiCompatEmbedder, build_embedder,+};+use secrecy::SecretString;+use serde_json::json;+use wiremock::matchers::{header, method, path};+use wiremock::{Mock, MockServer, Request, ResponseTemplate};++fn embedding_body(dim: usize) -> serde_json::Value {+ json!({+ "object": "list",+ "data": [{ "object": "embedding", "index": 0, "embedding": vec![0.5_f32; dim] }],+ "model": "nomic-embed-text",+ "usage": { "prompt_tokens": 1, "total_tokens": 1 },+ })+}++#[tokio::test]+async fn keyless_embed_sends_no_authorization_header() {+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(path("/v1/embeddings"))+ .respond_with(move |req: &Request| {+ assert!(+ req.headers.get("authorization").is_none(),+ "keyless compat embedder must not send an Authorization header"+ );+ ResponseTemplate::new(200).set_body_json(embedding_body(8))+ })+ .expect(1)+ .mount(&server)+ .await;++ let e = OpenAiCompatEmbedder::new(format!("{}/v1", server.uri()), None, "nomic-embed-text", 8)+ .expect("embedder builds");+ assert_eq!(e.provider(), "openai-compat");+ assert_eq!(e.provider(), EmbedderChoice::OpenAiCompat.name());++ let v = e.embed("hello world").await.expect("embed succeeds");+ assert_eq!(v.len(), 8);+ let norm: f32 = v.iter().map(|x| x * x).sum::<f32>().sqrt();+ assert!((norm - 1.0).abs() < 1e-5, "expected unit norm, got {norm}");+}++#[tokio::test]+async fn configured_key_is_sent_as_bearer() {+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(path("/v1/embeddings"))+ .and(header("authorization", "Bearer sk-gateway"))+ .respond_with(ResponseTemplate::new(200).set_body_json(embedding_body(8)))+ .expect(1)+ .mount(&server)+ .await;++ let e = OpenAiCompatEmbedder::new(+ format!("{}/v1", server.uri()),+ Some(SecretString::from("sk-gateway")),+ "nomic-embed-text",+ 8,+ )+ .expect("embedder builds");+ e.embed("hello").await.expect("embed succeeds");+}++#[tokio::test]+async fn factory_builds_compat_embedder_and_requires_base_url() {+ let ok = build_embedder(EmbedderConfig {+ provider: EmbedderChoice::OpenAiCompat,+ model: "nomic-embed-text".into(),+ dim: 768,+ api_key: None,+ base_url: Some("http://localhost:11434/v1".into()),+ })+ .expect("factory builds compat embedder");+ assert_eq!(ok.provider(), "openai-compat");+ assert_eq!(ok.dim(), 768);++ let err = match build_embedder(EmbedderConfig {+ provider: EmbedderChoice::OpenAiCompat,+ model: "nomic-embed-text".into(),+ dim: 768,+ api_key: None,+ base_url: None,+ }) {+ Ok(_) => panic!("compat embedder must not build without a base URL"),+ Err(err) => err,+ };+ assert!(+ matches!(err, LlmError::NotConfigured(ref msg) if msg.contains("AI_MEMORY_EMBEDDING_BASE_URL")),+ "expected NotConfigured for missing base URL, got {err:?}"+ );+}diff --git a/docs/install.md b/docs/install.mdindex 7c9c48a2..5a979e61 100644--- a/docs/install.md+++ b/docs/install.md@@ -975,6 +975,7 @@ If you set only the provider, ai-memory picks a sensible default:| `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `AI_MEMORY_EMBEDDING_BASE_URL=https://openrouter.ai/api/v1` | `openai/text-embedding-3-small` via [OpenRouter](https://openrouter.ai) | Reuses `LLM_API_KEY` or `OPENAI_API_KEY` with the OpenAI-compatible embedding client. || `AI_MEMORY_EMBEDDING_PROVIDER=voyage` | `voyage-3` (1024-dim) | Voyage's current general-purpose recommendation. || `AI_MEMORY_EMBEDDING_PROVIDER=google` / `gemini` | `gemini-embedding-001` (768-dim) | Google-hosted embeddings via `embedContent`. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). |+| `AI_MEMORY_EMBEDDING_PROVIDER=openai-compat` | no default — set model, dim, and base URL explicitly | Self-hosted engines (Ollama, LM Studio, vLLM). Keyless by default; `LLM_API_KEY` is sent as a bearer token when present (gateways). Example: `AI_MEMORY_EMBEDDING_BASE_URL=http://localhost:11434/v1`, `AI_MEMORY_EMBEDDING_MODEL=nomic-embed-text`, `AI_MEMORY_EMBEDDING_DIM=768`. Switching an existing `openai`+base-URL setup to `openai-compat` changes the stored `{provider, model, dim}` triple — run `ai-memory embed --force` to re-embed. |> **What we don't recommend:** reasoning-mode models (Claude with extended> thinking, GPT-o3, Gemini "thinking" variants) — they burn token budget on