Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(llm): disable Gemini Flash thinking budget
AkitaOnRailsbf55fdb6c32c
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex a190d468..d9fdb2d4 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -22,6 +22,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0- Experimental native Windows support: PowerShell Docker wrapper,`ai-memory.cmd`, `.ps1` lifecycle hooks in parity with `.sh` hooks, WindowsTailwind hash/download support, and [`docs/windows.md`](docs/windows.md).+- Google Gemini LLM provider via `AI_MEMORY_LLM_PROVIDER=gemini`, with+ `gemini-2.5-flash` as the default hosted Google model and `GEMINI_API_KEY`+ / `GOOGLE_API_KEY` support.- README support matrix for operating systems, agent integrations, LLMproviders, and embedding providers.@@ -33,6 +36,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Fixed- Hyphenated FTS5 queries such as `ai-memory` are normalized safely instead ofbeing parsed as column operators.+- Gemini 2.5 Flash requests disable default dynamic thinking so hidden thought+ tokens do not consume `maxOutputTokens` and truncate strict JSON responses.## [0.1.3] - 2026-05-24diff --git a/README.md b/README.mdindex 8a04a1c5..6d422402 100644--- a/README.md+++ b/README.md@@ -30,7 +30,7 @@| Oh My Pi / OMP | Supported | `pi` / `omp` aliases for MCP config + TypeScript extension. || Claude Desktop | MCP-only | Uses `mcp-remote`; no lifecycle hooks. || OpenClaw | MCP-only | OpenClaw has MCP/webhook ingress but no lifecycle hook surface. |-| LLM providers | Supported | Anthropic, OpenAI, and OpenAI-compatible endpoints. |+| LLM providers | Supported | Anthropic, OpenAI, Gemini, and OpenAI-compatible endpoints. || Embedding providers | Supported | OpenAI and Voyage. |## What it isdiff --git a/crates/ai-memory-llm/src/gemini.rs b/crates/ai-memory-llm/src/gemini.rsindex d82e0227..12732ce2 100644--- a/crates/ai-memory-llm/src/gemini.rs+++ b/crates/ai-memory-llm/src/gemini.rs@@ -87,6 +87,8 @@ struct GeminiSystem<'a> {struct GeminiGenerationConfig {#[serde(rename = "maxOutputTokens")]max_output_tokens: u32,+ #[serde(rename = "thinkingConfig", skip_serializing_if = "Option::is_none")]+ thinking_config: Option<GeminiThinkingConfig>,#[serde(skip_serializing_if = "Option::is_none")]temperature: Option<f32>,#[serde(rename = "responseMimeType", skip_serializing_if = "Option::is_none")]@@ -95,6 +97,12 @@ struct GeminiGenerationConfig {response_schema: Option<serde_json::Value>,}+#[derive(Debug, Serialize)]+struct GeminiThinkingConfig {+ #[serde(rename = "thinkingBudget")]+ thinking_budget: i32,+}+#[derive(Debug, Deserialize)]struct GeminiResponse {#[serde(default)]@@ -189,6 +197,7 @@ impl GeminiProvider {system_instruction,generation_config: GeminiGenerationConfig {max_output_tokens: request.max_tokens,+ thinking_config: default_thinking_config_for(&self.model),temperature: request.temperature,response_mime_type,response_schema,@@ -236,6 +245,14 @@ impl GeminiProvider {}}+fn default_thinking_config_for(model: &str) -> Option<GeminiThinkingConfig> {+ let model = model.to_ascii_lowercase();+ if model.contains("gemini-2.5-flash") {+ return Some(GeminiThinkingConfig { thinking_budget: 0 });+ }+ None+}+fn first_text(response: &GeminiResponse) -> Option<String> {let candidate = response.candidates.first()?;let content = candidate.content.as_ref()?;@@ -244,7 +261,11 @@ fn first_text(response: &GeminiResponse) -> Option<String> {.iter().filter_map(|p| p.text.as_deref()).collect();- if joined.is_empty() { None } else { Some(joined) }+ if joined.is_empty() {+ None+ } else {+ Some(joined)+ }}/// Normalise a schemars-generated JSON Schema into the Gemini-supported@@ -403,7 +424,10 @@ mod tests {.unwrap();assert_eq!(item.get("type").unwrap(), "object");assert!(!item.contains_key("additionalProperties"));- assert_eq!(item.get("properties").unwrap().get("path").unwrap(), &json!({"type": "string"}));+ assert_eq!(+ item.get("properties").unwrap().get("path").unwrap(),+ &json!({"type": "string"})+ );}#[test]@@ -435,4 +459,25 @@ mod tests {let prepared = prepare_schema_for_gemini(schema.clone()).unwrap();assert_eq!(prepared, schema);}++ #[test]+ fn build_request_disables_default_thinking_for_25_flash() {+ let provider =+ GeminiProvider::new(SecretString::from("test-key"), "gemini-2.5-flash").unwrap();+ let request = ChatRequest::user_prompt("emit json");+ let body = serde_json::to_value(provider.build_request(&request, None)).unwrap();+ assert_eq!(+ body.pointer("/generationConfig/thinkingConfig/thinkingBudget"),+ Some(&json!(0))+ );+ }++ #[test]+ fn build_request_omits_thinking_config_for_non_flash_models() {+ let provider =+ GeminiProvider::new(SecretString::from("test-key"), "gemini-2.5-pro").unwrap();+ let request = ChatRequest::user_prompt("emit json");+ let body = serde_json::to_value(provider.build_request(&request, None)).unwrap();+ assert!(body.pointer("/generationConfig/thinkingConfig").is_none());+ }}diff --git a/docs/install.md b/docs/install.mdindex 95ce7e63..4e007b17 100644--- a/docs/install.md+++ b/docs/install.md@@ -297,7 +297,7 @@ If you set only the provider, ai-memory picks a sensible default:|---|---|---|| `AI_MEMORY_LLM_PROVIDER=anthropic` | `claude-haiku-4-5` | **Recommended default.** Best balance of speed, restraint, and classification quality. Not a reasoning model. Consistently classifies durable project rules as `kind: rule`. || `AI_MEMORY_LLM_PROVIDER=openai` | `gpt-5.4-mini` | Cheaper + faster alternative. Same parse reliability; mild over-classification on thin sessions. |-| `AI_MEMORY_LLM_PROVIDER=gemini` | `gemini-2.5-flash` | Google's hosted option with a generous free tier — useful if you don't have an Anthropic / OpenAI key handy. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). |+| `AI_MEMORY_LLM_PROVIDER=gemini` | `gemini-2.5-flash` | Google's hosted option with a generous free tier. ai-memory disables Gemini 2.5 Flash's default dynamic thinking so hidden thought tokens do not truncate strict JSON. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). || `AI_MEMORY_EMBEDDING_PROVIDER=openai` | `text-embedding-3-small` (1536-dim) | 5× cheaper than `-3-large` with marginal recall loss. || `AI_MEMORY_EMBEDDING_PROVIDER=voyage` | `voyage-3` (1024-dim) | Voyage's current general-purpose recommendation. |