Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix(config): scope the ambient LLM_BASE_URL to operator-supplied endpoints (#691)

Igor Sant'Ana254c36df3c4c
diff --git a/CHANGELOG.md b/CHANGELOG.md
index fa333692..f819e48f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -17,6 +17,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
now uses the import instead of a prose pointer (#680).
### Fixed
+- A bare `LLM_BASE_URL` in the environment no longer redirects providers that
+ talk to a fixed vendor endpoint. The variable is a cross-tool convention an
+ operator exports once for a local Ollama, and ai-memory fed it to every
+ provider: a `gemini` server then POSTed to
+ `http://localhost:11434/v1beta/models/<model>:generateContent` and got Ollama's
+ plain-text `404 page not found`, surfacing as
+ `502 Bad Gateway: {"error":"provider error 404: 404 page not found"}` on
+ bootstrap and consolidation. It now reaches only the dialects whose endpoint
+ the operator supplies anyway — `openai-compat`, which has none without it, and
+ `opencode`, whose Zen catalogue is an override — and is ignored elsewhere with
+ a startup `warn!` naming the provider and the URL. An explicit `llm_base_url`
+ (or `AI_MEMORY_LLM_BASE_URL`) still configures any provider, so proxying a
+ vendor endpoint on purpose is unchanged. `ai-memory llm-test` resolves the base
+ URL the same way, so it reproduces what `serve` will do instead of inheriting
+ the same ambient override (#691).
- The from-source AUR `PKGBUILD` now builds and tests on constrained AUR
builders. Release LTO was disabled (`options=('!debug' '!lto')`) so the
final link no longer gets OOM-killed on low-memory build hosts, and the
diff --git a/crates/ai-memory-cli/src/commands/llm_test.rs b/crates/ai-memory-cli/src/commands/llm_test.rs
index 8a6c9475..1b094dc4 100644
--- a/crates/ai-memory-cli/src/commands/llm_test.rs
+++ b/crates/ai-memory-cli/src/commands/llm_test.rs
@@ -22,7 +22,7 @@ pub async fn run(config: &Config, args: LlmTestArgs) -> Result<()> {
provider,
model: args.model,
auth: config.provider_auth(provider, api_key_override),
- base_url: args.base_url.or_else(|| config.llm_test_base_url()),
+ base_url: args.base_url.or_else(|| config.llm_test_base_url(provider)),
compat_strict: config.llm_compat_strict,
request_timeout_secs: config.llm_timeout_secs,
reasoning_effort: config.llm_reasoning_effort,
diff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rs
index 4b330060..4b5a63c8 100644
--- a/crates/ai-memory-cli/src/config.rs
+++ b/crates/ai-memory-cli/src/config.rs
@@ -1199,13 +1199,7 @@ impl Config {
provider,
model,
auth: self.provider_auth(provider, None),
- // base_url falls back to the runtime env (LLM_BASE_URL), mirroring
- // how auth is sourced — otherwise openai-compat is only
- // configurable via config.toml even though the key comes from env.
- base_url: self
- .llm_base_url
- .clone()
- .or_else(|| self.runtime_env.llm_base_url.clone()),
+ base_url: self.resolve_base_url(provider),
compat_strict: self.llm_compat_strict,
request_timeout_secs: self.llm_timeout_secs,
reasoning_effort: self.llm_reasoning_effort,
@@ -1576,14 +1570,44 @@ impl Config {
}
}
+ /// Base URL for `provider`, from the two sources that can supply one.
+ ///
+ /// An explicit ai-memory setting (`llm_base_url` in `config.toml`, or
+ /// `AI_MEMORY_LLM_BASE_URL`) configures any provider: naming ai-memory is
+ /// how an operator says they mean it, and proxying a vendor endpoint is a
+ /// legitimate deployment.
+ ///
+ /// The bare `LLM_BASE_URL` is different in kind — a cross-tool convention
+ /// an operator exports once for a local Ollama and forgets. It reaches
+ /// only the providers whose endpoint is theirs to choose anyway; sending
+ /// it to a vendor-fixed endpoint silently rewrites every request onto a
+ /// host that does not speak the dialect (#691), so it is ignored and the
+ /// reason is logged rather than left for a 404 to explain.
+ fn resolve_base_url(&self, provider: ProviderChoice) -> Option<String> {
+ if let Some(explicit) = non_empty(self.llm_base_url.as_deref()) {
+ return Some(explicit.to_string());
+ }
+ let ambient = non_empty(self.runtime_env.llm_base_url.as_deref())?;
+ if provider.endpoint_is_operator_chosen() {
+ return Some(ambient.to_string());
+ }
+ tracing::warn!(
+ provider = provider.name(),
+ base_url = ambient,
+ "ignoring ambient LLM_BASE_URL: this provider talks to a fixed vendor \
+ endpoint. Set llm_base_url (or AI_MEMORY_LLM_BASE_URL) to point it \
+ somewhere else on purpose."
+ );
+ None
+ }
+
/// Base URL fallback for `llm-test`. Resolves exactly as
/// [`Self::provider_config`] does, so `llm-test` exercises the endpoint
- /// `serve` will use — including an `opencode` override onto Zen.
+ /// `serve` will use — including an `opencode` override onto Zen, and the
+ /// same refusal to inherit an ambient `LLM_BASE_URL`.
#[must_use]
- pub fn llm_test_base_url(&self) -> Option<String> {
- self.llm_base_url
- .clone()
- .or_else(|| self.runtime_env.llm_base_url.clone())
+ pub fn llm_test_base_url(&self, provider: ProviderChoice) -> Option<String> {
+ self.resolve_base_url(provider)
}
}
@@ -3148,3 +3172,112 @@ fn llm_provider_config_gemini_uses_default_base_url_when_none_provided() {
assert_eq!(provider.provider, ProviderChoice::Gemini);
assert_eq!(provider.base_url, None);
}
+
+// #691: `LLM_BASE_URL` is an ambient, cross-tool convention — an operator who
+// once pointed it at Ollama for `openai-compat` keeps it exported. Feeding it
+// to Gemini builds `http://localhost:11434/v1beta/models/<model>:generateContent`,
+// which Ollama answers with a plain-text `404 page not found`; the bootstrap
+// surfaces that as a 502 with no server-side log line to explain it.
+#[test]
+fn ambient_llm_base_url_does_not_override_gemini() {
+ let cfg = Config {
+ llm_provider: Some("gemini".into()),
+ llm_base_url: None,
+ runtime_env: RuntimeEnv {
+ gemini_api_key: Some(SecretString::from("dummy")),
+ llm_base_url: Some("http://localhost:11434".into()),
+ ..RuntimeEnv::default()
+ },
+ ..Config::default()
+ };
+ let provider = cfg.llm_provider_config().unwrap().unwrap();
+ assert_eq!(provider.provider, ProviderChoice::Gemini);
+ assert_eq!(provider.base_url, None);
+}
+
+// The other half of #691: the ambient fallback exists *for* the dialects whose
+// endpoint the operator supplies, and narrowing it must not take that away.
+// `openai-compat` has no endpoint at all without one.
+#[test]
+fn ambient_llm_base_url_still_configures_openai_compat() {
+ let cfg = Config {
+ llm_provider: Some("openai-compat".into()),
+ llm_model: Some("gemma4:26b-mlx".into()),
+ llm_base_url: None,
+ runtime_env: RuntimeEnv {
+ llm_base_url: Some("http://localhost:11434/v1".into()),
+ ..RuntimeEnv::default()
+ },
+ ..Config::default()
+ };
+ let provider = cfg.llm_provider_config().unwrap().unwrap();
+ assert_eq!(provider.provider, ProviderChoice::OpenAiCompat);
+ assert_eq!(
+ provider.base_url.as_deref(),
+ Some("http://localhost:11434/v1")
+ );
+}
+
+// `opencode` defaults to Go and reaches Zen's general catalogue only through an
+// override, so it is operator-chosen too.
+#[test]
+fn ambient_llm_base_url_still_configures_opencode() {
+ let cfg = Config {
+ llm_provider: Some("opencode".into()),
+ runtime_env: RuntimeEnv {
+ opencode_api_key: Some(SecretString::from("dummy")),
+ llm_base_url: Some("https://opencode.ai/zen/v1".into()),
+ ..RuntimeEnv::default()
+ },
+ ..Config::default()
+ };
+ let provider = cfg.llm_provider_config().unwrap().unwrap();
+ assert_eq!(provider.provider, ProviderChoice::OpenCode);
+ assert_eq!(
+ provider.base_url.as_deref(),
+ Some("https://opencode.ai/zen/v1")
+ );
+}
+
+// Naming ai-memory is how an operator says they mean it: proxying Gemini stays
+// possible, and the explicit setting outranks an ambient one that disagrees.
+#[test]
+fn explicit_llm_base_url_still_overrides_gemini_over_an_ambient_one() {
+ let cfg = Config {
+ llm_provider: Some("gemini".into()),
+ llm_base_url: Some("https://gemini-proxy.internal".into()),
+ runtime_env: RuntimeEnv {
+ gemini_api_key: Some(SecretString::from("dummy")),
+ llm_base_url: Some("http://localhost:11434".into()),
+ ..RuntimeEnv::default()
+ },
+ ..Config::default()
+ };
+ let provider = cfg.llm_provider_config().unwrap().unwrap();
+ assert_eq!(
+ provider.base_url.as_deref(),
+ Some("https://gemini-proxy.internal")
+ );
+}
+
+// `llm-test` is the tool an operator reaches for to reproduce what `serve`
+// does. It resolved base_url through its own copy of the old chain, so before
+// this fix it hit the ambient endpoint too — and agreed with the broken server
+// instead of exposing it.
+#[test]
+fn llm_test_base_url_resolves_per_provider_like_serve_does() {
+ let cfg = Config {
+ llm_base_url: None,
+ runtime_env: RuntimeEnv {
+ llm_base_url: Some("http://localhost:11434".into()),
+ ..RuntimeEnv::default()
+ },
+ ..Config::default()
+ };
+ assert_eq!(cfg.llm_test_base_url(ProviderChoice::Gemini), None);
+ assert_eq!(
+ cfg.llm_test_base_url(ProviderChoice::OpenAiCompat)
+ .as_deref(),
+ Some("http://localhost:11434")
+ );
+}
diff --git a/crates/ai-memory-llm/src/factory.rs b/crates/ai-memory-llm/src/factory.rs
index f8603845..08d94c74 100644
--- a/crates/ai-memory-llm/src/factory.rs
+++ b/crates/ai-memory-llm/src/factory.rs
@@ -82,6 +82,22 @@ impl ProviderChoice {
},
}
}
+
+ /// Whether the operator, rather than the vendor, decides where this
+ /// provider's endpoint lives.
+ ///
+ /// True for the self-hosted / aggregator dialects: `openai-compat` has
+ /// no endpoint at all without one, and `opencode` reaches Zen's general
+ /// catalogue only through an override. Every other provider talks to a
+ /// fixed vendor host, and a base URL aimed elsewhere does not speak its
+ /// dialect — a Gemini request against an Ollama host is a 404, not a
+ /// degraded answer. Callers use this to decide whether an *ambient*
+ /// base URL (the cross-tool `LLM_BASE_URL` convention) may configure the
+ /// provider; an explicit ai-memory setting always may.
+ #[must_use]
+ pub const fn endpoint_is_operator_chosen(self) -> bool {
+ matches!(self, Self::OpenAiCompat | Self::OpenCode)
+ }
}
/// All settings needed to construct one LLM provider instance.
@@ -389,6 +405,33 @@ pub fn build_provider(config: ProviderConfig) -> LlmResult<Arc<dyn LlmProvider>>
mod tests {
use super::*;
+ // Exhaustive on purpose: adding a provider must be a decision about whose
+ // endpoint it is, not a default inherited from whichever arm was copied.
+ #[test]
+ fn only_the_operator_supplied_dialects_accept_an_ambient_base_url() {
+ for choice in [ProviderChoice::OpenAiCompat, ProviderChoice::OpenCode] {
+ assert!(
+ choice.endpoint_is_operator_chosen(),
+ "{} has no endpoint without an operator-supplied one",
+ choice.name()
+ );
+ }
+ for choice in [
+ ProviderChoice::Anthropic,
+ ProviderChoice::OpenAi,
+ ProviderChoice::Gemini,
+ ProviderChoice::OpenAiOAuth,
+ ProviderChoice::Copilot,
+ ProviderChoice::AnthropicOAuth,
+ ] {
+ assert!(
+ !choice.endpoint_is_operator_chosen(),
+ "{} talks to a fixed vendor host",
+ choice.name()
+ );
+ }
+ }
+
#[test]
fn provider_choices_declare_current_auth_requirements() {
assert_eq!(
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index 124fb98a..bcb53212 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -598,7 +598,15 @@ AI_MEMORY_LLM_MODEL optional when the provider has a default; e.g. claude
ANTHROPIC_API_KEY / OPENAI_API_KEY / GEMINI_API_KEY / LLM_API_KEY
AI_MEMORY_LLM_BASE_URL required for openai-compat (Ollama, vLLM); optional override for
opencode (defaults to the Go endpoint, set
- https://opencode.ai/zen/v1 for Zen's catalogue)
+ https://opencode.ai/zen/v1 for Zen's catalogue).
+ Applies to any provider: naming ai-memory is how an
+ operator says a vendor endpoint is proxied on purpose
+LLM_BASE_URL the unprefixed cross-tool convention, accepted for
+ openai-compat and opencode only. Providers with a
+ fixed vendor endpoint (anthropic, openai, gemini,
+ the OAuth backends, copilot) ignore it and log why —
+ an operator's leftover Ollama URL must not silently
+ rewrite every Gemini request into a 404
AI_MEMORY_LLM_COMPAT_STRICT true by default; false disables response_format=json_schema
AI_MEMORY_LLM_TIMEOUT_SECS per-request timeout for chat providers; 300 by default
AI_MEMORY_LLM_REASONING_EFFORT optional reasoning/thinking effort