Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix(gemini): do not send thinkingConfig to gemini-3.5-flash-lite

AkitaOnRailse299d8e6231e
diff --git a/CHANGELOG.md b/CHANGELOG.md
index cb138cee..1c96f6a9 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -44,10 +44,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
- Changed the default model for the `gemini` provider from `gemini-2.5-flash`
- to `gemini-3.5-flash`, because Google will discontinue Gemini 2.5 Flash and
- Gemini 2.5 Flash-Lite on October 20, 2026. The e2e smoke test now defaults
- to `gemini-3.5-flash-lite` as its second variant. The thinking-budget
- workaround covers both the legacy models and the new defaults. (#423)
+ to `gemini-3.5-flash`. Google has scheduled the 2.5 Flash family for
+ retirement — the Gemini API deprecation table lists `gemini-2.5-flash-lite`
+ for **October 16, 2026** (Vertex AI and the Agent Platform list October 20;
+ ai-memory talks to the Gemini API, so the earlier date is the one that
+ applies). The thinking-budget workaround still covers the legacy models.
+ Note it is deliberately **not** applied to `gemini-3.5-flash-lite`, which
+ rejects `thinkingConfig` outright with HTTP 400 — unlike
+ `gemini-2.5-flash-lite`, which accepts it — so that model omits the field
+ and the e2e smoke test keeps `gemini-2.5-flash-lite` as its second
+ variant. (#423)
- Documented OrcaRouter through the existing `openai-compat` provider instead
of adding a redundant provider type, including the endpoint, model, and API
key mapping needed for deployment. (#410)
diff --git a/crates/ai-memory-llm/src/gemini.rs b/crates/ai-memory-llm/src/gemini.rs
index 4dabcacd..e9c8d0a1 100644
--- a/crates/ai-memory-llm/src/gemini.rs
+++ b/crates/ai-memory-llm/src/gemini.rs
@@ -247,7 +247,21 @@ impl GeminiProvider {
fn default_thinking_config_for(model: &str) -> Option<GeminiThinkingConfig> {
let model = model.to_ascii_lowercase();
- if model.contains("gemini-2.5-flash") || model.contains("gemini-3.5-flash") {
+ // These matches are substring matches, so `gemini-N.5-flash` also covers
+ // `gemini-N.5-flash-lite`. That is correct for 2.5 and WRONG for 3.5:
+ // verified against the live API, `gemini-2.5-flash-lite` accepts
+ // `thinkingConfig.thinkingBudget = 0` (HTTP 200) while
+ // `gemini-3.5-flash-lite` rejects the field outright (HTTP 400,
+ // "Request contains an invalid argument") and succeeds only when it is
+ // omitted. Sending it there breaks every call for that model.
+ //
+ // So 3.5 excludes `-lite` explicitly and 2.5 keeps its existing
+ // behaviour unchanged — narrowing 2.5 as well would re-enable its
+ // default thinking and let hidden thought tokens truncate strict JSON,
+ // which is the whole reason this function exists.
+ let is_25_flash = model.contains("gemini-2.5-flash");
+ let is_35_flash_non_lite = model.contains("gemini-3.5-flash") && !model.contains("-lite");
+ if is_25_flash || is_35_flash_non_lite {
return Some(GeminiThinkingConfig { thinking_budget: 0 });
}
None
@@ -605,6 +619,26 @@ mod tests {
assert_thinking_budget_disabled("gemini-3.5-flash");
}
+ /// `gemini-3.5-flash-lite` rejects `thinkingConfig` with HTTP 400 while
+ /// `gemini-2.5-flash-lite` accepts it — verified against the live API.
+ /// The substring match must therefore split the two, or every call on
+ /// 3.5-lite fails.
+ #[test]
+ fn build_request_omits_thinking_config_for_35_flash_lite_only() {
+ let lite =
+ GeminiProvider::new(SecretString::from("test-key"), "gemini-3.5-flash-lite").unwrap();
+ let body =
+ serde_json::to_value(lite.build_request(&ChatRequest::user_prompt("x"), None)).unwrap();
+ assert!(
+ body["generationConfig"].get("thinkingConfig").is_none(),
+ "3.5-flash-lite rejects thinkingConfig (HTTP 400); it must be omitted: {body}"
+ );
+
+ // 2.5-flash-lite accepts it, and stripping it there would let hidden
+ // thought tokens truncate strict JSON. Behaviour must not change.
+ assert_thinking_budget_disabled("gemini-2.5-flash-lite");
+ }
+
#[test]
fn build_request_omits_thinking_config_for_non_flash_models() {
let provider =
diff --git a/tests/e2e/handoff_smoke.sh b/tests/e2e/handoff_smoke.sh
index d821a0c1..e6e3d81b 100755
--- a/tests/e2e/handoff_smoke.sh
+++ b/tests/e2e/handoff_smoke.sh
@@ -88,8 +88,12 @@ HOOK_SCOPE="workspace=e2e-test&project=blog"
# Two deliberately different Gemini variants. Both are free-tier;
# overridable via env when the defaults rotate out.
+# NOTE: `gemini_call` below always sends thinkingConfig.thinkingBudget=0.
+# `gemini-3.5-flash-lite` REJECTS that field (HTTP 400) — verified against
+# the live API — so it cannot be used here. `gemini-2.5-flash-lite` accepts
+# it and stays the second variant.
MODEL_A="${MODEL_A:-gemini-3.5-flash}"
-MODEL_B="${MODEL_B:-gemini-3.5-flash-lite}"
+MODEL_B="${MODEL_B:-gemini-2.5-flash-lite}"
# Isolate ai-memory's data dir; leave $HOME alone so cargo's target
# cache + the user's git config etc. stay accessible.