Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix(config): defer missing llm_fallbacks credentials to serve (#762)

rafaelkenedydde899882700
diff --git a/CHANGELOG.md b/CHANGELOG.md
index aedb30ba..d756f39e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
+- CLI commands no longer fail at startup when an `[[llm_fallbacks]]` profile's
+ `api_key_env` variable is absent from the invoking shell. `Config::load`
+ validated every fallback credential eagerly, so read-only commands such as
+ `ai-memory status` exited with `llm_fallbacks[0].api_key_env=... is set but
+ the environment variable is missing or empty` even when the running server
+ had the key injected by its service wrapper, which pushed operators to export
+ provider keys in every shell. The missing credential is now enforced where it
+ is needed: `ai-memory serve` still refuses to start without it, and building
+ the LLM chain still fails rather than silently dropping the fallback. Every
+ other profile check (provider, model, base URL) still runs at load for every
+ command (#762).
- The generated OpenCode and OpenCode 2 plugins now forward a subagent session's
`parentID` as the `agent_id` marker, so `[capture] drop_subagent_captures` can
recognize and drop OpenCode subagent sessions. Previously both plugins emitted
diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rs
index c86ad0ea..d3d75e75 100644
--- a/crates/ai-memory-cli/src/commands/serve.rs
+++ b/crates/ai-memory-cli/src/commands/serve.rs
@@ -805,6 +805,7 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {
let mut shutdown = ShutdownSignals::install();
validate_web_ui_args(args.enable_web, args.web_ui_dir.as_deref())?;
+ config.require_llm_fallback_credentials()?;
// Merge config + CLI CORS origins (config first, CLI adds new entries).
// Validation runs before binding so a misconfigured origin is caught early.
diff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rs
index adb718e0..fbb662df 100644
--- a/crates/ai-memory-cli/src/config.rs
+++ b/crates/ai-memory-cli/src/config.rs
@@ -28,6 +28,11 @@ pub const DEFAULT_BIND: &str = "127.0.0.1:49374";
/// Default base URL used by thin-client CLI subcommands.
pub const DEFAULT_SERVER_URL: &str = "http://127.0.0.1:49374";
+/// Placeholder credential that lets `Config::load` validate a fallback
+/// profile whose `api_key_env` is absent from this process. Never reaches a
+/// provider: the config built from it is discarded (#762).
+const UNRESOLVED_FALLBACK_KEY: &str = "unresolved-llm-fallback-credential";
+
/// Default MCP endpoint URL rendered for client integrations.
pub const DEFAULT_MCP_URL: &str = "http://127.0.0.1:49374/mcp";
@@ -248,6 +253,14 @@ pub struct Config {
/// hand.
#[serde(skip)]
pub llm_fallback_configs: Vec<ProviderConfig>,
+ /// One message per `llm_fallbacks` entry whose `api_key_env` names a
+ /// variable absent from this process's environment. Such an entry is left
+ /// out of [`Self::llm_fallback_configs`]; the failure is raised by
+ /// [`Self::require_llm_fallback_credentials`] instead of by `load`, so a
+ /// CLI invocation that never builds the LLM chain does not need the
+ /// server's credentials (#762). Same `pub` rationale as above.
+ #[serde(skip)]
+ pub llm_fallback_unresolved: Vec<String>,
/// Opt-in: run LLM consolidation on SessionEnd (in addition to the
/// always-written heuristic session page), when an LLM provider is
/// configured. Off by default. Provider work is durably queued after the
@@ -733,6 +746,7 @@ impl Default for Config {
llm_headers: Vec::new(),
llm_fallbacks: Vec::new(),
llm_fallback_configs: Vec::new(),
+ llm_fallback_unresolved: Vec::new(),
consolidate_on_session_end: false,
capture_assistant: false,
backfill_on_start: true,
@@ -1194,16 +1208,31 @@ impl Config {
// sit unused for months and only fail once the primary is already
// down. The environment is read here, once, per invariant #1 (no
// `std::env::var` outside `load`).
+ //
+ // An absent credential is the one failure that is deferred rather than
+ // raised (#762): the variable belongs in the server's environment (a
+ // service wrapper's env block), not in every shell that runs
+ // `ai-memory status`. It is recorded here and enforced by
+ // `require_llm_fallback_credentials` at `serve` startup and by
+ // `llm_provider_chain`, so the server still fails closed.
let mut fallback_configs = Vec::with_capacity(config.llm_fallbacks.len());
+ let mut unresolved = Vec::new();
for (i, profile) in config.llm_fallbacks.iter().enumerate() {
- let resolved_key = match non_empty(profile.api_key_env.as_deref()) {
- Some(name) => Some(SecretString::from(env_string(name).ok_or_else(|| {
- anyhow::anyhow!(
- "llm_fallbacks[{i}].api_key_env={name} is set but the environment \
- variable is missing or empty"
- )
- })?)),
- None => None,
+ let (resolved_key, missing) = match non_empty(profile.api_key_env.as_deref()) {
+ Some(name) => match env_string(name) {
+ Some(key) => (Some(SecretString::from(key)), None),
+ None => (
+ // Stands in for the absent key so the rest of the
+ // profile is still validated eagerly below; the
+ // config built from it is discarded.
+ Some(SecretString::from(UNRESOLVED_FALLBACK_KEY)),
+ Some(format!(
+ "llm_fallbacks[{i}].api_key_env={name} is set but the \
+ environment variable is missing or empty"
+ )),
+ ),
+ },
+ None => (None, None),
};
let provider_cfg = config
.fallback_provider_config(i, profile, resolved_key)
@@ -1213,9 +1242,13 @@ impl Config {
// silently sitting unused until the primary has an outage.
build_provider(provider_cfg.clone())
.with_context(|| format!("building llm_fallbacks[{i}]"))?;
- fallback_configs.push(provider_cfg);
+ match missing {
+ Some(message) => unresolved.push(message),
+ None => fallback_configs.push(provider_cfg),
+ }
}
config.llm_fallback_configs = fallback_configs;
+ config.llm_fallback_unresolved = unresolved;
Ok(config)
}
@@ -1386,6 +1419,24 @@ impl Config {
}
}
+ /// Fail when an `llm_fallbacks` credential named by `api_key_env` is
+ /// absent from this process's environment.
+ ///
+ /// `serve` calls this at startup, so a fallback that could never
+ /// authenticate stops the server before it runs, whether or not a primary
+ /// provider is configured. Other subcommands do not: they never build the
+ /// chain, and requiring the server's credentials in every shell would
+ /// spread them to every process the operator runs (#762).
+ ///
+ /// # Errors
+ /// Names the first unresolved profile and its variable.
+ pub fn require_llm_fallback_credentials(&self) -> Result<()> {
+ match self.llm_fallback_unresolved.first() {
+ Some(message) => anyhow::bail!("{message}"),
+ None => Ok(()),
+ }
+ }
+
/// Build the configured LLM provider, including any ordered
/// `llm_fallbacks` chain.
///
@@ -1399,6 +1450,11 @@ impl Config {
/// Propagates any error from constructing the primary or a fallback
/// provider (`build_provider` is the sole construction path for both).
pub fn llm_provider_chain(&self) -> LlmResult<Option<Arc<dyn LlmProvider>>> {
+ // A chain with a profile silently dropped would look healthy until
+ // the primary has an outage.
+ if let Some(message) = self.llm_fallback_unresolved.first() {
+ return Err(LlmError::NotConfigured(message.clone()));
+ }
let Some(primary_cfg) = self.llm_provider_config()? else {
return Ok(None);
};
@@ -3245,22 +3301,60 @@ mod tests {
);
}
+ /// #762: a credential absent from the invoking shell no longer fails
+ /// `load` — `ai-memory status` must work from a shell that does not hold
+ /// the server's keys — but it still fails closed wherever the chain is
+ /// actually needed: `serve` startup and chain construction.
#[test]
- fn load_rejects_a_missing_or_empty_api_key_env_value() {
- let error = load_with_toml(
- "[[llm_fallbacks]]\nprovider = \"gemini\"\nmodel = \"m\"\n\
+ fn a_missing_api_key_env_value_defers_to_serve_and_the_chain() {
+ let config = load_with_toml(
+ "llm_provider = \"gemini\"\n\
+ [[llm_fallbacks]]\nprovider = \"gemini\"\nmodel = \"m\"\n\
api_key_env = \"AI_MEMORY_TEST_FALLBACK_UNSET_KEY_648\"\n",
)
- .expect_err("an unresolved api_key_env must fail closed");
+ .expect("a CLI that never builds the chain must load without the credential");
+ let expected = "llm_fallbacks[0].api_key_env=AI_MEMORY_TEST_FALLBACK_UNSET_KEY_648 is \
+ set but the environment variable is missing or empty";
+
assert!(
- format!("{error:#}").contains(
- "llm_fallbacks[0].api_key_env=AI_MEMORY_TEST_FALLBACK_UNSET_KEY_648 is set but \
- the environment variable is missing or empty"
- ),
+ config.llm_fallback_configs.is_empty(),
+ "an unresolved profile must never reach the chain"
+ );
+ let error = config
+ .require_llm_fallback_credentials()
+ .expect_err("serve startup must still fail closed");
+ assert_eq!(format!("{error:#}"), expected);
+ let error = config
+ .llm_provider_chain()
+ .err()
+ .expect("building the chain must fail rather than drop the fallback");
+ assert!(
+ error.to_string().contains(expected),
+ "unexpected error: {error}"
+ );
+ }
+
+ /// Deferring the credential must not defer the rest of the profile's
+ /// validation: a malformed profile still fails `load` for every command.
+ #[test]
+ fn a_missing_api_key_env_value_does_not_hide_a_malformed_profile() {
+ let error = load_with_toml(
+ "[[llm_fallbacks]]\nprovider = \"openai-compat\"\nmodel = \"m\"\n\
+ api_key_env = \"AI_MEMORY_TEST_FALLBACK_UNSET_KEY_762\"\n",
+ )
+ .expect_err("openai-compat needs a base_url whether or not the key is present");
+ assert!(
+ format!("{error:#}").contains("LLM_BASE_URL"),
"unexpected error: {error:#}"
);
}
+ #[test]
+ fn a_config_without_unresolved_fallbacks_passes_the_serve_check() {
+ let config = load_with_toml("").unwrap();
+ config.require_llm_fallback_credentials().unwrap();
+ }
+
#[test]
fn load_rejects_an_openai_compat_fallback_with_no_base_url() {
let error =
diff --git a/docs/llm-provider-fallback.md b/docs/llm-provider-fallback.md
index 4271898d..10da84db 100644
--- a/docs/llm-provider-fallback.md
+++ b/docs/llm-provider-fallback.md
@@ -61,9 +61,13 @@ api_key_env = "GEMINI_API_KEY"
`api_key_env` is optional when the provider already has a native credential
source, such as OpenAI OAuth or Copilot. The loader validates every profile and
-resolves all credential material once. A missing credential, invalid provider,
-or malformed profile fails startup rather than leaving a latent fallback that
-only fails under an outage.
+resolves all credential material once. An invalid provider or malformed profile
+fails every command at load. A credential whose `api_key_env` variable is absent
+fails `ai-memory serve` at startup, rather than leaving a latent fallback that
+only fails under an outage, but does not stop other commands: `ai-memory
+status` and the other thin clients never build the chain, so the variable only
+has to be in the server's environment (for example a service wrapper's env
+block), not exported in every shell.
The proposed initial semantics are append-only: the primary runs first, then
the fallbacks in declaration order. No environment-variable shorthand is added
@@ -144,7 +148,9 @@ or trigger background recovery traffic.
4. Verify an open circuit skips only its `(provider, model)` candidate and a
success closes it.
5. Test configuration validation, including missing/empty profiles and missing
- credentials, through `Config::load()` without reading a real home directory.
+ credentials, through `Config::load()` without reading a real home directory;
+ a missing credential is checked by `require_llm_fallback_credentials()` and
+ `llm_provider_chain()`, not by `load`.
6. Test health snapshots for candidate labels and redaction.
7. Keep existing single-provider config tests byte-for-byte compatible.