Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

docs: cover AI_MEMORY_LLM_TIMEOUT_SECS where sibling env vars live

Lucas Oliveiraa6367ae47abb
diff --git a/README.md b/README.md
index 98f3d8d6..eeacb06f 100644
--- a/README.md
+++ b/README.md
@@ -966,12 +966,13 @@ automatic PreCompact/PostCompaction checkpoint fall back to the deterministic
rule-based page; admission, storage, and scope errors still fail closed. The
validated minimums are 6,000 input and 1,000 output tokens.
-Every chat provider bounds each HTTP request at 300 seconds
+Every chat provider bounds each completion request at 300 seconds
(`AI_MEMORY_LLM_TIMEOUT_SECS` to override, or `llm_timeout_secs = 900` in
-config.toml). The default tolerates a local engine cold-loading a large model;
-slow hosted gateways whose long completions exceed the ceiling fail every
-request with `http: error sending request`, so raise the value there instead
-of watching consolidation exhaust its retries.
+config.toml; the quick openai-oauth token refresh keeps the default ceiling).
+The default tolerates a local engine cold-loading a large model; slow hosted
+gateways whose long completions exceed the ceiling fail every request with
+`http: error sending request`, so raise the value there instead of watching
+consolidation exhaust its retries.
Reranking is optional and off by default. With an LLM provider configured,
`AI_MEMORY_RERANKER=llm` makes project and explicit-scope `memory_query`
diff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rs
index bce462ea..e7610a26 100644
--- a/crates/ai-memory-cli/src/config.rs
+++ b/crates/ai-memory-cli/src/config.rs
@@ -140,8 +140,10 @@ pub struct Config {
/// already supplies its own schema. Set
/// `AI_MEMORY_LLM_COMPAT_STRICT=false` for an incompatible endpoint.
pub llm_compat_strict: bool,
- /// Per-request timeout (seconds) applied to every chat provider's
- /// HTTP calls, including the Copilot token exchange. Defaults to
+ /// Per-request timeout (seconds) applied to every chat
+ /// completion request and to the Copilot token exchange; the
+ /// openai-oauth token refresh keeps the built-in default ceiling
+ /// (it is a quick grant exchange). Defaults to
/// `ai_memory_llm::DEFAULT_REQUEST_TIMEOUT_SECS` (300s), which
/// tolerates a local engine cold-loading a large model. Raise it
/// for slow hosted gateways whose long completions exceed the
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index b8dd2118..e89cdbc7 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -549,6 +549,7 @@ AI_MEMORY_LLM_MODEL optional when the provider has a default; e.g. claude
ANTHROPIC_API_KEY / OPENAI_API_KEY / GEMINI_API_KEY / LLM_API_KEY
AI_MEMORY_LLM_BASE_URL for openai-compat (Ollama, vLLM)
AI_MEMORY_LLM_COMPAT_STRICT true by default; false disables response_format=json_schema
+AI_MEMORY_LLM_TIMEOUT_SECS per-request timeout for chat providers; 300 by default
AI_MEMORY_RERANKER optional `llm`; reranks project/scopes query candidates
COPILOT_GITHUB_TOKEN optional GitHub token for copilot
GITHUB_COPILOT_API_TOKEN optional pre-minted Copilot API token
diff --git a/docs/install.md b/docs/install.md
index 5c4b6735..c5e2494b 100644
--- a/docs/install.md
+++ b/docs/install.md
@@ -1442,6 +1442,14 @@ or returns a malformed response shape. For an incompatible endpoint, opt out:
-e AI_MEMORY_LLM_COMPAT_STRICT=false
```
+Hosted gateways that stream long completions past the default 300-second
+per-request ceiling fail with `http: error sending request`; raise the
+ceiling to match the gateway's worst-case generation time:
+
+```bash
+-e AI_MEMORY_LLM_TIMEOUT_SECS=900
+```
+
#### Match the consolidation budget to a local model's context window
Consolidation defaults to an approximate 100k-token input target plus a 32k