Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
docs: cover AI_MEMORY_LLM_TIMEOUT_SECS where sibling env vars live
Lucas Oliveiraa6367ae47abb
diff --git a/README.md b/README.mdindex 98f3d8d6..eeacb06f 100644--- a/README.md+++ b/README.md@@ -966,12 +966,13 @@ automatic PreCompact/PostCompaction checkpoint fall back to the deterministicrule-based page; admission, storage, and scope errors still fail closed. Thevalidated minimums are 6,000 input and 1,000 output tokens.-Every chat provider bounds each HTTP request at 300 seconds+Every chat provider bounds each completion request at 300 seconds(`AI_MEMORY_LLM_TIMEOUT_SECS` to override, or `llm_timeout_secs = 900` in-config.toml). The default tolerates a local engine cold-loading a large model;-slow hosted gateways whose long completions exceed the ceiling fail every-request with `http: error sending request`, so raise the value there instead-of watching consolidation exhaust its retries.+config.toml; the quick openai-oauth token refresh keeps the default ceiling).+The default tolerates a local engine cold-loading a large model; slow hosted+gateways whose long completions exceed the ceiling fail every request with+`http: error sending request`, so raise the value there instead of watching+consolidation exhaust its retries.Reranking is optional and off by default. With an LLM provider configured,`AI_MEMORY_RERANKER=llm` makes project and explicit-scope `memory_query`diff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rsindex bce462ea..e7610a26 100644--- a/crates/ai-memory-cli/src/config.rs+++ b/crates/ai-memory-cli/src/config.rs@@ -140,8 +140,10 @@ pub struct Config {/// already supplies its own schema. Set/// `AI_MEMORY_LLM_COMPAT_STRICT=false` for an incompatible endpoint.pub llm_compat_strict: bool,- /// Per-request timeout (seconds) applied to every chat provider's- /// HTTP calls, including the Copilot token exchange. Defaults to+ /// Per-request timeout (seconds) applied to every chat+ /// completion request and to the Copilot token exchange; the+ /// openai-oauth token refresh keeps the built-in default ceiling+ /// (it is a quick grant exchange). Defaults to/// `ai_memory_llm::DEFAULT_REQUEST_TIMEOUT_SECS` (300s), which/// tolerates a local engine cold-loading a large model. Raise it/// for slow hosted gateways whose long completions exceed thediff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex b8dd2118..e89cdbc7 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -549,6 +549,7 @@ AI_MEMORY_LLM_MODEL optional when the provider has a default; e.g. claudeANTHROPIC_API_KEY / OPENAI_API_KEY / GEMINI_API_KEY / LLM_API_KEYAI_MEMORY_LLM_BASE_URL for openai-compat (Ollama, vLLM)AI_MEMORY_LLM_COMPAT_STRICT true by default; false disables response_format=json_schema+AI_MEMORY_LLM_TIMEOUT_SECS per-request timeout for chat providers; 300 by defaultAI_MEMORY_RERANKER optional `llm`; reranks project/scopes query candidatesCOPILOT_GITHUB_TOKEN optional GitHub token for copilotGITHUB_COPILOT_API_TOKEN optional pre-minted Copilot API tokendiff --git a/docs/install.md b/docs/install.mdindex 5c4b6735..c5e2494b 100644--- a/docs/install.md+++ b/docs/install.md@@ -1442,6 +1442,14 @@ or returns a malformed response shape. For an incompatible endpoint, opt out:-e AI_MEMORY_LLM_COMPAT_STRICT=false```+Hosted gateways that stream long completions past the default 300-second+per-request ceiling fail with `http: error sending request`; raise the+ceiling to match the gateway's worst-case generation time:++```bash+-e AI_MEMORY_LLM_TIMEOUT_SECS=900+```+#### Match the consolidation budget to a local model's context windowConsolidation defaults to an approximate 100k-token input target plus a 32k