Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
feat(config): add EMBEDDING_API_KEY for embedding-only credentials (#514)
Djalma Júnior2a9f6ead93a4
diff --git a/README.md b/README.mdindex e511c8a3..e2f612b2 100644--- a/README.md+++ b/README.md@@ -1010,9 +1010,12 @@ Embeddings are optional and separate from the LLM provider. Setgraph-neighbor retrieval. `openai-compat` targets self-hosted engines(Ollama, LM Studio, vLLM): it needs no API key and requires explicit`AI_MEMORY_EMBEDDING_BASE_URL`, `AI_MEMORY_EMBEDDING_MODEL`, and-`AI_MEMORY_EMBEDDING_DIM`. Both the FTS-only and hybrid paths apply the same-bounded page-authority adjustment after candidate generation; embeddings-improve relevance recall but do not decide which source is canonical.+`AI_MEMORY_EMBEDDING_DIM`. The optional `EMBEDDING_API_KEY` credentials the+embedder alone and is checked before `OPENAI_API_KEY` and `LLM_API_KEY`, so+embeddings can run on a different provider than the LLM. Both the FTS-only and+hybrid paths apply the same bounded page-authority adjustment after candidate+generation; embeddings improve relevance recall but do not decide which source+is canonical.See [`docs/install.md#llm-provider-tiers`](docs/install.md#llm-provider-tiers)for env vars and Ollama/OpenRouter/Atlas Cloud/OrcaRouter examples, anddiff --git a/bin/ai-memory b/bin/ai-memoryindex c5849bee..04e1364e 100755--- a/bin/ai-memory+++ b/bin/ai-memory@@ -499,6 +499,7 @@ for var in \COPILOT_API_URL \VOYAGE_API_KEY \LLM_API_KEY \+ EMBEDDING_API_KEY \RUST_LOGdoif [ -n "${!var:-}" ]; thendiff --git a/bin/ai-memory.ps1 b/bin/ai-memory.ps1index cd361416..0c860a1d 100644--- a/bin/ai-memory.ps1+++ b/bin/ai-memory.ps1@@ -143,6 +143,7 @@ foreach ($Name in @("OPENAI_API_KEY","VOYAGE_API_KEY","LLM_API_KEY",+ "EMBEDDING_API_KEY","RUST_LOG")) {if (-not [string]::IsNullOrEmpty([Environment]::GetEnvironmentVariable($Name))) {diff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rsindex 239a5658..8634d79e 100644--- a/crates/ai-memory-cli/src/config.rs+++ b/crates/ai-memory-cli/src/config.rs@@ -313,6 +313,7 @@ pub struct RuntimeEnv {gemini_api_key: Option<SecretString>,llm_api_key: Option<SecretString>,llm_base_url: Option<String>,+ embedding_api_key: Option<SecretString>,copilot_github_token: Option<SecretString>,github_copilot_api_token: Option<SecretString>,copilot_api_url: Option<String>,@@ -350,6 +351,10 @@ impl RuntimeEnv {gemini_api_key: env_secret("GEMINI_API_KEY").or_else(|| env_secret("GOOGLE_API_KEY")),llm_api_key: env_secret("LLM_API_KEY"),llm_base_url: env_string("LLM_BASE_URL"),+ // The embedding counterpart of LLM_API_KEY: it credentials the+ // embedding role alone, so the embedder can target a different+ // provider than the chat model instead of borrowing its key.+ embedding_api_key: env_secret("EMBEDDING_API_KEY"),copilot_github_token: env_secret("COPILOT_GITHUB_TOKEN").or_else(|| env_secret("GH_TOKEN")).or_else(|| env_secret("GITHUB_TOKEN")),@@ -1024,10 +1029,15 @@ impl Config {}))}- /// OpenAI-compatible embedding key. Direct OpenAI keeps requiring- /// `OPENAI_API_KEY`; a custom embedding base URL may reuse `LLM_API_KEY`- /// for gateways such as OpenRouter.+ /// OpenAI-compatible embedding key. `EMBEDDING_API_KEY` is checked first+ /// so an operator can point the embedder at one provider while the LLM+ /// uses another; without it, direct OpenAI keeps requiring+ /// `OPENAI_API_KEY` and a custom embedding base URL may reuse+ /// `LLM_API_KEY` for gateways such as OpenRouter.fn openai_embedding_api_key(&self) -> LlmResult<SecretString> {+ if let Some(key) = self.runtime_env.embedding_api_key.clone() {+ return Ok(key);+ }if let Some(key) = self.runtime_env.openai_api_key.clone() {return Ok(key);}@@ -1036,10 +1046,14 @@ impl Config {return Ok(key);}return Err(LlmError::NotConfigured(- "OPENAI_API_KEY or LLM_API_KEY required for openai-compatible embeddings".into(),+ "EMBEDDING_API_KEY, OPENAI_API_KEY or LLM_API_KEY required for \+ openai-compatible embeddings"+ .into(),));}- Err(LlmError::NotConfigured("OPENAI_API_KEY".into()))+ Err(LlmError::NotConfigured(+ "EMBEDDING_API_KEY or OPENAI_API_KEY".into(),+ ))}/// Whether the operator opted into post-RRF reranking.@@ -1126,11 +1140,13 @@ impl Config {LlmError::NotConfigured("GEMINI_API_KEY or GOOGLE_API_KEY".into())})?,// Keyless engines (Ollama, LM Studio) are the norm; a- // gateway key rides on LLM_API_KEY when present.+ // gateway key rides on EMBEDDING_API_KEY, or on LLM_API_KEY+ // when the chat model shares that gateway.EmbedderChoice::OpenAiCompat => self.runtime_env- .llm_api_key+ .embedding_api_key.clone()+ .or_else(|| self.runtime_env.llm_api_key.clone()).unwrap_or_else(|| SecretString::from(String::new())),};let base_url = self.embedding_base_url.clone();@@ -1728,6 +1744,73 @@ mod tests {embedder.base_url.as_deref(),Some("https://openrouter.ai/api/v1"));++ // A dedicated embedding key outranks the borrowed LLM one.+ let cfg = Config {+ runtime_env: RuntimeEnv {+ embedding_api_key: Some(SecretString::from("sk-embed-key")),+ ..cfg.runtime_env.clone()+ },+ ..cfg+ };+ let embedder = cfg.embedder_config().unwrap().unwrap();+ assert_eq!(embedder.api_key.expose_secret(), "sk-embed-key");+ }++ #[test]+ fn openai_embedding_prefers_dedicated_embedding_api_key() {+ // The LLM runs on api.openai.com (OPENAI_API_KEY) while embeddings+ // run somewhere else; without a dedicated key the OpenAI one would+ // be sent to the embedding base URL and rejected.+ let cfg = Config {+ embedding_provider: Some("openai".into()),+ embedding_base_url: Some("https://api.cheap-embeddings.example/v1".into()),+ runtime_env: RuntimeEnv {+ embedding_api_key: Some(SecretString::from("sk-embed-key")),+ openai_api_key: Some(SecretString::from("sk-openai-key")),+ llm_api_key: Some(SecretString::from("sk-llm-key")),+ ..RuntimeEnv::default()+ },+ ..Config::default()+ };++ let embedder = cfg.embedder_config().unwrap().unwrap();+ assert_eq!(embedder.api_key.expose_secret(), "sk-embed-key");++ // It also works without a custom base URL, i.e. against OpenAI+ // itself, where OPENAI_API_KEY was previously the only accepted key.+ let direct = Config {+ embedding_base_url: None,+ ..cfg.clone()+ };+ assert_eq!(+ direct+ .embedder_config()+ .unwrap()+ .unwrap()+ .api_key+ .expose_secret(),+ "sk-embed-key"+ );++ // Absent, the previous precedence is untouched: OPENAI_API_KEY+ // still beats LLM_API_KEY.+ let without = Config {+ runtime_env: RuntimeEnv {+ embedding_api_key: None,+ ..cfg.runtime_env.clone()+ },+ ..cfg+ };+ assert_eq!(+ without+ .embedder_config()+ .unwrap()+ .unwrap()+ .api_key+ .expose_secret(),+ "sk-openai-key"+ );}#[test]@@ -1761,6 +1844,18 @@ mod tests {let embedder = cfg_with_key.embedder_config().unwrap().unwrap();assert_eq!(embedder.api_key.expose_secret(), "sk-or-key");+ // EMBEDDING_API_KEY takes precedence over that borrowed key, and+ // still leaves the keyless path keyless when it is unset.+ let cfg_with_embedding_key = Config {+ runtime_env: RuntimeEnv {+ embedding_api_key: Some(SecretString::from("sk-embed-key")),+ ..cfg_with_key.runtime_env.clone()+ },+ ..cfg_with_key+ };+ let embedder = cfg_with_embedding_key.embedder_config().unwrap().unwrap();+ assert_eq!(embedder.api_key.expose_secret(), "sk-embed-key");+// Missing model / dim / base URL each fail closed.let missing_model = Config {embedding_model: None,@@ -1808,7 +1903,28 @@ mod tests {};let err = cfg.embedder_config().unwrap_err();- assert!(matches!(err, LlmError::NotConfigured(msg) if msg == "OPENAI_API_KEY"));+ assert!(+ matches!(err, LlmError::NotConfigured(msg) if msg == "EMBEDDING_API_KEY or OPENAI_API_KEY")+ );++ // The error names the dedicated key, and setting it resolves the+ // same configuration.+ let with_embedding_key = Config {+ runtime_env: RuntimeEnv {+ embedding_api_key: Some(SecretString::from("sk-embed-key")),+ ..cfg.runtime_env.clone()+ },+ ..cfg+ };+ assert_eq!(+ with_embedding_key+ .embedder_config()+ .unwrap()+ .unwrap()+ .api_key+ .expose_secret(),+ "sk-embed-key"+ );}#[test]diff --git a/docker/.env.production.example b/docker/.env.production.exampleindex 54651cd9..18ae9c6e 100644--- a/docker/.env.production.example+++ b/docker/.env.production.example@@ -56,6 +56,11 @@ AI_MEMORY_EMBEDDING_MODEL=text-embedding-3-smallAI_MEMORY_EMBEDDING_DIM=1536OPENAI_API_KEY=sk-REPLACE_ME# VOYAGE_API_KEY=...+#+# Optional: an embedding-only key, checked before OPENAI_API_KEY and+# LLM_API_KEY. Set it when embeddings should authenticate against a+# different provider than AI_MEMORY_LLM_PROVIDER above.+# EMBEDDING_API_KEY=sk-...# -- Listen address inside the container -------------------------------# The compose file maps the host port to this. Don't change unlessdiff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex ae78ea45..9a7a71a3 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -579,12 +579,23 @@ AI_MEMORY_EMBEDDING_DIM 1536 (OpenAI), 1024 (Voyage), 768 (Google);OPENAI_API_KEY / VOYAGE_API_KEY / GEMINI_API_KEY / GOOGLE_API_KEYLLM_API_KEY accepted for openai with a custom base URL and asoptional bearer auth for openai-compat+EMBEDDING_API_KEY optional embedding-only key; checked before+ OPENAI_API_KEY and LLM_API_KEY for the openai and+ openai-compat embedders```+`EMBEDDING_API_KEY` credentials the embedding role alone, so the embedder can+target a different provider than the chat model — `openai` on `api.openai.com`+for the LLM, a cheaper or self-hosted OpenAI-compatible endpoint for vectors.+Without it the `openai` embedder takes `OPENAI_API_KEY`, then `LLM_API_KEY`+when a custom embedding base URL is set, exactly as before. `voyage` and+`google`/`gemini` keep reading only their own `VOYAGE_API_KEY` and+`GEMINI_API_KEY`/`GOOGLE_API_KEY`.+`openai-compat` also requires an explicit model because self-hosted engines haveno safe shared model or dimensionality default. It sends no authorization header-when `LLM_API_KEY` is absent and stores vectors under the distinct-`provider="openai-compat"` identity.+when both `EMBEDDING_API_KEY` and `LLM_API_KEY` are absent and stores vectors+under the distinct `provider="openai-compat"` identity.## Future workdiff --git a/docs/install.md b/docs/install.mdindex b22d0af4..30ec8525 100644--- a/docs/install.md+++ b/docs/install.md@@ -1418,7 +1418,7 @@ ai-memory works in three intensity tiers:| **+ ChatGPT/Codex OAuth** | Same LLM features using a ChatGPT Pro/Plus login instead of an OpenAI Platform key | `AI_MEMORY_LLM_PROVIDER=openai-oauth` + `ai-memory auth login openai-oauth` | Uses your ChatGPT subscription || **+ GitHub Copilot** | Same LLM features using a GitHub Copilot subscription | `AI_MEMORY_LLM_PROVIDER=copilot` + `ai-memory auth login copilot` or `COPILOT_GITHUB_TOKEN` | Uses your Copilot subscription || **+ LLM reranking** | At most one relevance pass over up to 30 bounded project/scopes search candidates; normal order is preserved on invalid, failed, timed-out, or concurrency-saturated responses | `AI_MEMORY_RERANKER=llm` + any configured LLM provider | One LLM call per eligible query, at most four concurrently |-| **+ Hybrid retrieval** | Adds vector cosine similarity to FTS5 + entity + graph RRF. Better recall on paraphrased queries | `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `OPENAI_API_KEY` | ~$0.0001 / page on backfill |+| **+ Hybrid retrieval** | Adds vector cosine similarity to FTS5 + entity + graph RRF. Better recall on paraphrased queries | `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `OPENAI_API_KEY` (or `EMBEDDING_API_KEY`) | ~$0.0001 / page on backfill |### Recommended models (chosen as defaults)@@ -1434,17 +1434,48 @@ If you set only the provider, ai-memory picks a sensible default:| `AI_MEMORY_LLM_PROVIDER=gemini` | `gemini-3.5-flash` | Google's hosted option with a generous free tier. ai-memory disables Gemini 3.5 Flash's default dynamic thinking so hidden thought tokens do not truncate strict JSON. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). || `AI_MEMORY_LLM_PROVIDER=opencode` | `claude-sonnet-4-6` | [OpenCode Zen/Go](https://opencode.ai) cloud API — OpenAI-compatible endpoint at `opencode.ai/zen/go/v1`. Set `OPENCODE_API_KEY` (key from `opencode.ai/auth`). Alias: `opencode-zen`. || `AI_MEMORY_EMBEDDING_PROVIDER=openai` | `text-embedding-3-small` (1536-dim) | 5× cheaper than `-3-large` with marginal recall loss. |-| `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `AI_MEMORY_EMBEDDING_BASE_URL=https://openrouter.ai/api/v1` | `openai/text-embedding-3-small` via [OpenRouter](https://openrouter.ai) | Reuses `LLM_API_KEY` or `OPENAI_API_KEY` with the OpenAI-compatible embedding client. |-| `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `AI_MEMORY_EMBEDDING_BASE_URL=https://api.orcarouter.ai/v1` | `openai/text-embedding-3-small` via [OrcaRouter](https://www.orcarouter.ai) | Reuses `LLM_API_KEY` with the OpenAI-compatible embedding client. |+| `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `AI_MEMORY_EMBEDDING_BASE_URL=https://openrouter.ai/api/v1` | `openai/text-embedding-3-small` via [OpenRouter](https://openrouter.ai) | Uses `EMBEDDING_API_KEY`, else reuses `LLM_API_KEY` or `OPENAI_API_KEY`, with the OpenAI-compatible embedding client. |+| `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `AI_MEMORY_EMBEDDING_BASE_URL=https://api.orcarouter.ai/v1` | `openai/text-embedding-3-small` via [OrcaRouter](https://www.orcarouter.ai) | Uses `EMBEDDING_API_KEY`, else reuses `LLM_API_KEY`, with the OpenAI-compatible embedding client. || `AI_MEMORY_EMBEDDING_PROVIDER=voyage` | `voyage-3` (1024-dim) | Voyage's current general-purpose recommendation. || `AI_MEMORY_EMBEDDING_PROVIDER=google` / `gemini` | `gemini-embedding-001` (768-dim) | Google-hosted embeddings via `embedContent`. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). |-| `AI_MEMORY_EMBEDDING_PROVIDER=openai-compat` | no default — set model, dim, and base URL explicitly | Self-hosted engines (Ollama, LM Studio, vLLM). Keyless by default; `LLM_API_KEY` is sent as a bearer token when present (gateways). Example: `AI_MEMORY_EMBEDDING_BASE_URL=http://localhost:11434/v1`, `AI_MEMORY_EMBEDDING_MODEL=nomic-embed-text`, `AI_MEMORY_EMBEDDING_DIM=768`. Switching an existing `openai`+base-URL setup to `openai-compat` changes the stored `{provider, model, dim}` triple — run `ai-memory embed --force` to re-embed. |+| `AI_MEMORY_EMBEDDING_PROVIDER=openai-compat` | no default — set model, dim, and base URL explicitly | Self-hosted engines (Ollama, LM Studio, vLLM). Keyless by default; `EMBEDDING_API_KEY`, else `LLM_API_KEY`, is sent as a bearer token when present (gateways). Example: `AI_MEMORY_EMBEDDING_BASE_URL=http://localhost:11434/v1`, `AI_MEMORY_EMBEDDING_MODEL=nomic-embed-text`, `AI_MEMORY_EMBEDDING_DIM=768`. Switching an existing `openai`+base-URL setup to `openai-compat` changes the stored `{provider, model, dim}` triple — run `ai-memory embed --force` to re-embed. |> **What we don't recommend:** reasoning-mode models (Claude with extended> thinking, GPT-o3, Gemini "thinking" variants) — they burn token budget on> internal reasoning and hang or emit empty responses with the strict-JSON> consolidation prompt. Turn reasoning off if you must use one.+#### Embedding credentials (`EMBEDDING_API_KEY`)++`EMBEDDING_API_KEY` is optional and credentials the embedding role alone. Set+it when the embedder should authenticate against a different provider than the+LLM — the `openai` and `openai-compat` embedders otherwise borrow the chat+model's `OPENAI_API_KEY` or `LLM_API_KEY`, which is then sent to whatever+`AI_MEMORY_EMBEDDING_BASE_URL` points at and rejected with a 401.++Precedence for `AI_MEMORY_EMBEDDING_PROVIDER=openai`:++1. `EMBEDDING_API_KEY`+2. `OPENAI_API_KEY`+3. `LLM_API_KEY`, only when `AI_MEMORY_EMBEDDING_BASE_URL` is set++`openai-compat` checks `EMBEDDING_API_KEY`, then `LLM_API_KEY`, and stays+keyless when neither is set. `voyage` and `google`/`gemini` are unaffected:+they read their own `VOYAGE_API_KEY` and `GEMINI_API_KEY`/`GOOGLE_API_KEY` and+never borrow another role's key. With `EMBEDDING_API_KEY` unset, key+resolution is exactly what it was before the variable existed.++```bash+# Chat on api.openai.com — `openai` is the provider that sends+# `max_completion_tokens`, which gpt-5 / o-series models require and+# `openai-compat` never emits — with embeddings on another endpoint.+export AI_MEMORY_LLM_PROVIDER=openai+export OPENAI_API_KEY=sk-...+export AI_MEMORY_EMBEDDING_PROVIDER=openai+export AI_MEMORY_EMBEDDING_BASE_URL=https://openrouter.ai/api/v1+export EMBEDDING_API_KEY=sk-or-v1-...+```+### Anthropic via Claude subscription (OAuth)> [!WARNING]