#1 Persistent memory for AI coding agents based on real-world benchmarks
TypeScript agentmemoryagentsaiclaudeclaudecodecodexcopilotcursorgenaiharnesshermesmemoryompopenclawopencodepi
docs(readme): document Ollama / LM Studio / vLLM local-model setup (#671) (#697)
Rohit Ghumarebfde2886c8ef
diff --git a/README.md b/README.mdindex 830914c2..37db54fe 100644--- a/README.md+++ b/README.md@@ -1136,8 +1136,54 @@ agentmemory auto-detects from your environment. By default, no LLM calls are mad| MiniMax | `MINIMAX_API_KEY` | Anthropic-compatible || Gemini | `GEMINI_API_KEY` | Also enables embeddings || OpenRouter | `OPENROUTER_API_KEY` | Any model |+| OpenAI API | `OPENAI_API_KEY` | Default `gpt-4o-mini`, override with `OPENAI_MODEL` |+| **Local (Ollama / LM Studio / vLLM / llama.cpp)** | `OPENAI_API_KEY=local` + `OPENAI_BASE_URL=http://localhost:11434/v1` (Ollama) or `http://localhost:1234/v1` (LM Studio) + `OPENAI_MODEL=<your model>` | Anything OpenAI-API-compatible. Zero cost, runs on your hardware. See [Local models](#local-models-ollama-lm-studio-vllm) below. || Claude subscription fallback | `AGENTMEMORY_ALLOW_AGENT_SDK=true` | Opt-in only. Spawns `@anthropic-ai/claude-agent-sdk` sessions — used to cause unbounded Stop-hook recursion (#149 follow-up) so it is no longer the default. |+### Local models (Ollama / LM Studio / vLLM)++agentmemory talks to any OpenAI-API-compatible server, so anything that exposes `/v1/chat/completions` works without code changes. No paid keys, no cloud, no rate limits — runs entirely on your hardware.++**Ollama** (default port `11434`):++```bash+ollama pull qwen2.5-coder:7b # or llama3.2:3b, mistral:7b, etc.+ollama serve+```++```env+# ~/.agentmemory/.env+OPENAI_API_KEY=ollama # any non-empty string; Ollama ignores it+OPENAI_BASE_URL=http://localhost:11434/v1+OPENAI_MODEL=qwen2.5-coder:7b+```++**LM Studio** (default port `1234`):++Open LM Studio → Local Server tab → Start Server. Pick any chat model from the picker (Qwen 2.5 Coder, Llama 3.2, DeepSeek, etc.).++```env+# ~/.agentmemory/.env+OPENAI_API_KEY=lmstudio # any non-empty string; LM Studio ignores it+OPENAI_BASE_URL=http://localhost:1234/v1+OPENAI_MODEL=qwen2.5-coder-7b-instruct # match the model name from LM Studio+```++**vLLM / llama.cpp / Text Generation Inference**: same shape — point `OPENAI_BASE_URL` at whatever URL your server exposes, set `OPENAI_MODEL` to a name your server will accept.++**Model picks for memory work**: compression and summarization are short tasks (<2K tokens in, <500 tokens out) where a 7B instruct model is plenty. Recommendations:++| Model | Size | Why |+|-------|------|-----|+| `qwen2.5-coder:7b` | ~4.7 GB | Best at code-shaped sessions; trained on programming + tool-use traces |+| `llama3.2:3b` | ~2 GB | Smallest sane option — fine for compression, weaker for graph extraction |+| `mistral:7b-instruct` | ~4.4 GB | Good general-purpose baseline if you don't want code-specific |+| `deepseek-r1:7b` | ~4.7 GB | Reasoning-tier quality at 7B; slower but cleaner extractions |++Reasoning-class models (`o1`-style with `<think>` blocks) can return empty `content` with a `reasoning` field your local server may not surface. If extractions come back blank, switch to a non-reasoning model first. The `OPENAI_REASONING_EFFORT=none` env can also disable thinking on Ollama Cloud thinking models that mirror the OpenAI reasoning schema.++Local embeddings ship out of the box via `@xenova/transformers` — `EMBEDDING_PROVIDER=local` (default) gives you BGE-small entirely on-device. No extra config needed.+### Cost-aware model selectionBackground compression runs on every observation, so model choice meaningfully changes monthly spend. Captured workload data: 635 requests / 888K tokens / 35 hours of active use, run against three OpenRouter models at 2026-05-23 pricing.