Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(llm): default Gemini model to gemini-3.5-flash ahead of 2.5 Flash deprecation
Vitor Vilas Boasfeb78d241bbf
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex b790ab16..e5208cee 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]### Changed+- Changed the default model for the `gemini` provider from `gemini-2.5-flash`+ to `gemini-3.5-flash`, because Google will discontinue Gemini 2.5 Flash on+ October 20, 2026. The thinking-budget workaround is applied to both the+ legacy model and the new default.- Documented OrcaRouter through the existing `openai-compat` provider insteadof adding a redundant provider type, including the endpoint, model, and APIkey mapping needed for deployment. (#410)diff --git a/README.md b/README.mdindex 4b5c13a0..7c92933d 100644--- a/README.md+++ b/README.md@@ -899,7 +899,7 @@ Recommended defaults:| `openai` | `gpt-5.4-mini` | Cheaper and faster hosted option. || `openai-oauth` | `gpt-5.5` | ChatGPT Pro/Plus/Codex backend via `ai-memory auth login openai-oauth`; no Platform API key. || `copilot` | `gpt-5.5` | GitHub Copilot Chat backend via `ai-memory auth login copilot` or `COPILOT_GITHUB_TOKEN`; requires a Copilot subscription. |-| `gemini` | `gemini-2.5-flash` | Google-hosted option with a generous free tier. |+| `gemini` | `gemini-3.5-flash` | Google-hosted option with a generous free tier. || `openai-compat` | no default | OpenRouter, Atlas Cloud, OrcaRouter, Ollama, vLLM, LM Studio, and other compatible endpoints. |`openai-oauth` stores a refresh token in `<data_dir>/auth.json` and talks todiff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rsindex 2675861d..55f38137 100644--- a/crates/ai-memory-cli/src/config.rs+++ b/crates/ai-memory-cli/src/config.rs@@ -948,7 +948,7 @@ impl Config {ProviderChoice::Anthropic => "claude-haiku-4-5".to_string(),ProviderChoice::AnthropicOAuth => "claude-sonnet-4-6".to_string(),ProviderChoice::OpenAi => "gpt-5.4-mini".to_string(),- ProviderChoice::Gemini => "gemini-2.5-flash".to_string(),+ ProviderChoice::Gemini => "gemini-3.5-flash".to_string(),ProviderChoice::OpenAiOAuth => "gpt-5.5".to_string(),ProviderChoice::Copilot => "gpt-5.5".to_string(),ProviderChoice::OpenAiCompat => {diff --git a/crates/ai-memory-llm/src/gemini.rs b/crates/ai-memory-llm/src/gemini.rsindex c3973dad..41fdeb63 100644--- a/crates/ai-memory-llm/src/gemini.rs+++ b/crates/ai-memory-llm/src/gemini.rs@@ -34,7 +34,7 @@ pub struct GeminiProvider {impl GeminiProvider {/// Construct a provider given an API key and model id (e.g.- /// `gemini-2.5-flash`).+ /// `gemini-3.5-flash`).////// # Errors/// Returns a `reqwest::Error` if the HTTP client cannot be built.@@ -247,7 +247,7 @@ impl GeminiProvider {fn default_thinking_config_for(model: &str) -> Option<GeminiThinkingConfig> {let model = model.to_ascii_lowercase();- if model.contains("gemini-2.5-flash") {+ if model.contains("gemini-2.5-flash") || model.contains("gemini-3.5-flash") {return Some(GeminiThinkingConfig { thinking_budget: 0 });}None@@ -586,9 +586,9 @@ mod tests {}#[test]- fn build_request_disables_default_thinking_for_25_flash() {+ fn build_request_disables_default_thinking_for_35_flash() {let provider =- GeminiProvider::new(SecretString::from("test-key"), "gemini-2.5-flash").unwrap();+ GeminiProvider::new(SecretString::from("test-key"), "gemini-3.5-flash").unwrap();let request = ChatRequest::user_prompt("emit json");let body = serde_json::to_value(provider.build_request(&request, None)).unwrap();assert_eq!(diff --git a/docs/deploy.md b/docs/deploy.mdindex 26c97d04..5d7888e7 100644--- a/docs/deploy.md+++ b/docs/deploy.md@@ -162,7 +162,7 @@ alternatives:| openai | `gpt-5.4-mini` | ~$0.002 | Cheaper, faster alternative. Decent quality. || openai-oauth | `gpt-5.5` | ChatGPT subscription | ChatGPT/Codex backend. Run `docker exec -it ai-memory ai-memory auth login openai-oauth` on the server host so `<data_dir>/auth.json` lands in the mounted data volume. || copilot | `gpt-5.5` | GitHub Copilot subscription | GitHub Copilot Chat backend. Run `docker exec -it ai-memory ai-memory auth login copilot` on the server host or set `COPILOT_GITHUB_TOKEN`. |-| gemini | `gemini-2.5-flash` | free tier covers personal use | Google hosted, native `responseSchema` structured output. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). |+| gemini | `gemini-3.5-flash` | free tier covers personal use | Google hosted, native `responseSchema` structured output. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). || openai-compat (Ollama) | `qwen3:32b` | $0 | Self-hosted. Set `AI_MEMORY_LLM_BASE_URL=http://host.docker.internal:11434/v1`. Quality depends on the model. |> **What we don't recommend:** reasoning-mode models (Kimi-K2.6 in reasoning mode,diff --git a/docs/install.md b/docs/install.mdindex 96cd76b2..f2ea91fc 100644--- a/docs/install.md+++ b/docs/install.md@@ -1228,7 +1228,7 @@ If you set only the provider, ai-memory picks a sensible default:| `AI_MEMORY_LLM_PROVIDER=openai` | `gpt-5.4-mini` | Cheaper + faster alternative. Same parse reliability; mild over-classification on thin sessions. || `AI_MEMORY_LLM_PROVIDER=openai-oauth` | `gpt-5.5` | ChatGPT/Codex backend. Run `ai-memory auth login openai-oauth` once; ai-memory stores the refresh token in `<data_dir>/auth.json` and refreshes access tokens automatically. || `AI_MEMORY_LLM_PROVIDER=copilot` | `gpt-5.5` | GitHub Copilot Chat backend. ai-memory stores a GitHub user token in `<data_dir>/auth.json`, exchanges it for a short-lived Copilot API token, and refreshes before expiry. |-| `AI_MEMORY_LLM_PROVIDER=gemini` | `gemini-2.5-flash` | Google's hosted option with a generous free tier. ai-memory disables Gemini 2.5 Flash's default dynamic thinking so hidden thought tokens do not truncate strict JSON. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). |+| `AI_MEMORY_LLM_PROVIDER=gemini` | `gemini-3.5-flash` | Google's hosted option with a generous free tier. ai-memory disables Gemini 3.5 Flash's default dynamic thinking so hidden thought tokens do not truncate strict JSON. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). || `AI_MEMORY_LLM_PROVIDER=opencode` | `claude-sonnet-4-6` | [OpenCode Zen/Go](https://opencode.ai) cloud API — OpenAI-compatible endpoint at `opencode.ai/zen/go/v1`. Set `OPENCODE_API_KEY` (key from `opencode.ai/auth`). Alias: `opencode-zen`. || `AI_MEMORY_EMBEDDING_PROVIDER=openai` | `text-embedding-3-small` (1536-dim) | 5× cheaper than `-3-large` with marginal recall loss. || `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `AI_MEMORY_EMBEDDING_BASE_URL=https://openrouter.ai/api/v1` | `openai/text-embedding-3-small` via [OpenRouter](https://openrouter.ai) | Reuses `LLM_API_KEY` or `OPENAI_API_KEY` with the OpenAI-compatible embedding client. |diff --git a/tests/e2e/handoff_smoke.sh b/tests/e2e/handoff_smoke.shindex 3a014b5f..2a96edf4 100755--- a/tests/e2e/handoff_smoke.sh+++ b/tests/e2e/handoff_smoke.sh@@ -88,7 +88,7 @@ HOOK_SCOPE="workspace=e2e-test&project=blog"# Two deliberately different Gemini variants. Both are free-tier;# overridable via env when the defaults rotate out.-MODEL_A="${MODEL_A:-gemini-2.5-flash}"+MODEL_A="${MODEL_A:-gemini-3.5-flash}"MODEL_B="${MODEL_B:-gemini-2.5-flash-lite}"# Isolate ai-memory's data dir; leave $HOME alone so cargo's target@@ -121,7 +121,7 @@ AUTH_HEADER="Authorization: Bearer $AUTH_TOKEN"# Usage: gemini_call <model> <prompt-string># Echoes the model's text response to stdout. thinkingBudget=0# disables reasoning tokens so maxOutputTokens is spent on the actual-# answer (gemini-2.5-flash is a reasoning model by default).+# answer (gemini-3.5-flash is a reasoning model by default).gemini_call() {local model="$1" prompt="$2"local body