The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.
Python agentic-memoryagentic-memory-systemagentsaiai-agentschatgptgenaillmlong-term-memorymemorymemory-managementpythonragstate-management
docs: fix LLM reranker config examples and field names (#4539)
Utkarsh7fff26f374c2
diff --git a/docs/components/rerankers/custom-prompts.mdx b/docs/components/rerankers/custom-prompts.mdxindex 5ca736064..1d72137c0 100644--- a/docs/components/rerankers/custom-prompts.mdx+++ b/docs/components/rerankers/custom-prompts.mdx@@ -7,58 +7,51 @@ When using LLM rerankers, you can customize the prompts used for ranking to bett## Default Prompt-The default LLM reranker prompt is designed to be general-purpose:+The default LLM reranker prompt scores each memory individually on a 0.0-1.0 scale:```-Given a query and a list of memory entries, rank the memory entries based on their relevance to the query.-Rate each memory on a scale of 1-10 where 10 is most relevant.+You are a relevance scoring assistant. Given a query and a document, you need to score how relevant the document is to the query.-Query: {query}+Score the relevance on a scale from 0.0 to 1.0, where:+- 1.0 = Perfectly relevant and directly answers the query+- 0.8-0.9 = Highly relevant with good information+- 0.6-0.7 = Moderately relevant with some useful information+- 0.4-0.5 = Slightly relevant with limited useful information+- 0.0-0.3 = Not relevant or no useful information-Memory entries:-{memories}+Query: "{query}"+Document: "{document}"-Provide your ranking as a JSON array with scores for each memory.+Provide only a single numerical score between 0.0 and 1.0. Do not include any explanation or additional text.```## Custom Prompt Configuration-You can provide a custom prompt template when configuring the LLM reranker:+You can provide a custom prompt template using the `scoring_prompt` parameter:```pythonfrom mem0 import Memorycustom_prompt = """-You are an expert at ranking memories for a personal AI assistant.-Given a user query and a list of memory entries, rank each memory based on:-1. Direct relevance to the query-2. Temporal relevance (recent memories may be more important)-3. Emotional significance-4. Actionability+You are an expert at evaluating memories for a personal AI assistant.+Given a user query and a memory entry, score how relevant the memory is.+Consider direct relevance, temporal relevance, and actionability.-Query: {query}-User Context: {user_context}+Query: "{query}"+Memory: "{document}"-Memory entries:-{memories}--Rate each memory from 1-10 and provide reasoning.-Return as JSON: {{"rankings": [{{"index": 0, "score": 8, "reason": "..."}}]}}+Provide only a single numerical score between 0.0 and 1.0."""config = {"reranker": {"provider": "llm_reranker","config": {- "llm": {- "provider": "openai",- "config": {- "model": "gpt-4.1-nano-2025-04-14",- "api_key": "your-openai-key"- }- },- "custom_prompt": custom_prompt,- "top_n": 5+ "provider": "openai",+ "model": "gpt-4o-mini",+ "api_key": "your-openai-key",+ "scoring_prompt": custom_prompt,+ "top_k": 5}}}@@ -70,12 +63,14 @@ memory = Memory.from_config(config)Your custom prompt can use the following variables:-| Variable | Description |-| ---------------- | ------------------------------------- |-| `{query}` | The search query |-| `{memories}` | The list of memory entries to rank |-| `{user_id}` | The user ID (if available) |-| `{user_context}` | Additional user context (if provided) |+| Variable | Description |+| ------------ | ----------------------------- |+| `{query}` | The search query |+| `{document}` | The memory entry being scored |++<Note>+ Both `{query}` and `{document}` are required in your custom prompt. The LLM reranker scores each memory individually against the query, so the prompt is called once per candidate memory.+</Note>## Domain-Specific Examples@@ -89,13 +84,10 @@ Prioritize memories that:- Show previous resolution patterns- Indicate customer preferences or constraints-Query: {query}-Customer Context: Previous interactions with this customer+Query: "{query}"+Memory: "{document}"-Memories:-{memories}--Rank each memory 1-10 based on support relevance.+Score relevance from 0.0 to 1.0."""```@@ -103,19 +95,16 @@ Rank each memory 1-10 based on support relevance.```pythoneducational_prompt = """-Rank these learning memories for a student query.+Score this learning memory for relevance to a student query.Consider:- Prerequisite knowledge requirements- Learning progression and difficulty- Relevance to current learning objectives-Student Query: {query}-Learning Context: {user_context}--Available memories:-{memories}+Student Query: "{query}"+Memory: "{document}"-Score each memory for educational value (1-10).+Score educational relevance from 0.0 to 1.0."""```@@ -123,73 +112,64 @@ Score each memory for educational value (1-10).```pythonpersonal_assistant_prompt = """-Rank personal memories for relevance to the user's query.+Score this personal memory for relevance to the user's query.Consider:- Recent vs. historical importance- Personal preferences and habits-- Contextual relationships between memories+- Contextual relationships-Query: {query}-Personal context: {user_context}+Query: "{query}"+Memory: "{document}"-Memories to rank:-{memories}--Provide relevance scores (1-10) with brief explanations.+Provide relevance score from 0.0 to 1.0."""```## Advanced Prompt Techniques-### Multi-Criteria Ranking+### Multi-Criteria Scoring```pythonmulti_criteria_prompt = """-Evaluate memories using multiple criteria:+Evaluate this memory using multiple criteria:1. RELEVANCE (40%): How directly related to the query-2. RECENCY (20%): How recent the memory is+2. RECENCY (20%): How recent the memory appears to be3. IMPORTANCE (25%): Personal or business significance4. ACTIONABILITY (15%): How useful for next steps-Query: {query}-Context: {user_context}--Memories:-{memories}+Query: "{query}"+Memory: "{document}"-For each memory, provide:-- Overall score (1-10)-- Breakdown by criteria-- Final ranking recommendation--Format: JSON with detailed scoring+Compute a weighted score from 0.0 to 1.0 based on these criteria.+Provide only the final numerical score."""```-### Contextual Ranking+### Chain-of-Thought Scoring```python-contextual_prompt = """-Consider the following context when ranking memories:-- Current user situation: {user_context}-- Time of day: {current_time}-- Recent activities: {recent_activities}+reasoning_prompt = """+Evaluate this memory's relevance step by step:++1. What is the main intent of the query?+2. What key information does the memory contain?+3. How directly does the memory address the query?-Query: {query}+Based on this analysis, provide a single relevance score from 0.0 to 1.0.-Rank these memories considering both direct relevance and contextual appropriateness:-{memories}+Query: "{query}"+Memory: "{document}"-Provide contextually-aware relevance scores (1-10).+Score:"""```## Best Practices1. **Be Specific**: Clearly define what makes a memory relevant for your use case-2. **Use Examples**: Include examples in your prompt for better model understanding-3. **Structure Output**: Specify the exact JSON format you want returned+2. **Use 0.0-1.0 Scale**: The score extractor expects values between 0.0 and 1.0+3. **Request Only the Score**: Ask for just the numerical score to improve extraction reliability4. **Test Iteratively**: Refine your prompt based on actual ranking performance5. **Consider Token Limits**: Keep prompts concise while being comprehensive@@ -206,7 +186,7 @@ prompts = []for i, prompt in enumerate(prompts):- config["reranker"]["config"]["custom_prompt"] = prompt+ config["reranker"]["config"]["scoring_prompt"] = promptmemory = Memory.from_config(config)results = memory.search("test query", user_id="test_user")@@ -216,6 +196,6 @@ for i, prompt in enumerate(prompts):## Common Issues- **Too Long**: Keep prompts under token limits for your chosen LLM-- **Too Vague**: Be specific about ranking criteria-- **Inconsistent Format**: Ensure JSON output format is clearly specified-- **Missing Context**: Include relevant variables for your use case+- **Too Vague**: Be specific about scoring criteria+- **Wrong Scale**: Use 0.0-1.0 scale to match the default score extractor+- **Extra Output**: Ask for only the numeric score — extra text can confuse score extractiondiff --git a/docs/components/rerankers/models/llm_reranker.mdx b/docs/components/rerankers/models/llm_reranker.mdxindex 5fa1582ce..3abb827dc 100644--- a/docs/components/rerankers/models/llm_reranker.mdx+++ b/docs/components/rerankers/models/llm_reranker.mdx@@ -18,13 +18,9 @@ config = {"reranker": {"provider": "llm_reranker","config": {- "llm": {- "provider": "openai",- "config": {- "model": "gpt-4",- "api_key": "your-openai-api-key"- }- }+ "provider": "openai",+ "model": "gpt-4o-mini",+ "api_key": "your-openai-api-key"}}}@@ -36,11 +32,14 @@ m = Memory.from_config(config)| Parameter | Type | Default | Description ||-----------|------|---------|-------------|-| `llm` | dict | Required | LLM configuration object |-| `top_k` | int | 10 | Number of results to rerank |+| `provider` | str | `"openai"` | LLM provider (openai, anthropic, etc.) |+| `model` | str | `"gpt-4o-mini"` | LLM model to use for reranking |+| `api_key` | str | None | API key for the LLM provider |+| `top_k` | int | None | Number of top documents to return after reranking || `temperature` | float | 0.0 | LLM temperature for consistency |-| `custom_prompt` | str | None | Custom reranking prompt |-| `score_range` | tuple | (0, 10) | Score range for relevance |+| `max_tokens` | int | 100 | Maximum tokens for LLM response |+| `scoring_prompt` | str | None | Custom prompt template for scoring documents |+| `llm` | dict | None | Optional nested LLM config for provider-specific fields (e.g., `ollama_base_url`, `azure_endpoint`). Overrides top-level `provider`/`model`/`api_key` when provided. |### Advanced Configuration@@ -49,20 +48,19 @@ config = {"reranker": {"provider": "llm_reranker","config": {- "llm": {- "provider": "anthropic",- "config": {- "model": "claude-3-sonnet-20240229",- "api_key": "your-anthropic-api-key"- }- },+ "provider": "anthropic",+ "model": "claude-sonnet-4-20250514",+ "api_key": "your-anthropic-api-key","top_k": 15,"temperature": 0.0,- "score_range": (1, 5),- "custom_prompt": """- Rate the relevance of each memory to the query on a scale of 1-5.+ "scoring_prompt": """+ Rate the relevance of each memory to the query on a scale of 0.0-1.0.Consider semantic similarity, context, and practical utility.Only provide the numeric score.++ Query: "{query}"+ Document: "{document}"+ Score:"""}}@@ -78,14 +76,10 @@ config = {"reranker": {"provider": "llm_reranker","config": {- "llm": {- "provider": "openai",- "config": {- "model": "gpt-4",- "api_key": "your-openai-api-key",- "temperature": 0.0- }- }+ "provider": "openai",+ "model": "gpt-4o-mini",+ "api_key": "your-openai-api-key",+ "temperature": 0.0}}}@@ -98,13 +92,9 @@ config = {"reranker": {"provider": "llm_reranker","config": {- "llm": {- "provider": "anthropic",- "config": {- "model": "claude-3-sonnet-20240229",- "api_key": "your-anthropic-api-key"- }- }+ "provider": "anthropic",+ "model": "claude-sonnet-4-20250514",+ "api_key": "your-anthropic-api-key"}}}@@ -117,10 +107,12 @@ config = {"reranker": {"provider": "llm_reranker","config": {+ "provider": "ollama",+ "model": "llama3.2","llm": {"provider": "ollama","config": {- "model": "llama2",+ "model": "llama3.2","ollama_base_url": "http://localhost:11434"}}@@ -129,6 +121,10 @@ config = {}```+<Note>+ For providers like Ollama that need extra fields (e.g., `ollama_base_url`), use the optional nested `llm` key to pass provider-specific configuration. The nested `llm` config overrides top-level `provider`/`model`/`api_key` when provided.+</Note>+### Azure OpenAI```python@@ -136,13 +132,16 @@ config = {"reranker": {"provider": "llm_reranker","config": {+ "provider": "azure_openai",+ "model": "gpt-4o-mini",+ "api_key": "your-azure-api-key","llm": {"provider": "azure_openai","config": {- "model": "gpt-4",+ "model": "gpt-4o-mini","api_key": "your-azure-api-key","azure_endpoint": "https://your-resource.openai.azure.com/",- "azure_deployment": "gpt-4-deployment"+ "azure_deployment": "gpt-4o-mini-deployment"}}}@@ -154,16 +153,22 @@ config = {### Default Prompt Behavior-The default prompt asks the LLM to score relevance on a 0-10 scale:+The default prompt asks the LLM to score relevance on a 0.0-1.0 scale:```-Given a query and a memory, rate how relevant the memory is to answering the query.-Score from 0 (completely irrelevant) to 10 (perfectly relevant).-Only provide the numeric score.+You are a relevance scoring assistant. Given a query and a document, you need to score how relevant the document is to the query.-Query: {query}-Memory: {memory}-Score:+Score the relevance on a scale from 0.0 to 1.0, where:+- 1.0 = Perfectly relevant and directly answers the query+- 0.8-0.9 = Highly relevant with good information+- 0.6-0.7 = Moderately relevant with some useful information+- 0.4-0.5 = Slightly relevant with limited useful information+- 0.0-0.3 = Not relevant or no useful information++Query: "{query}"+Document: "{document}"++Provide only a single numerical score between 0.0 and 1.0. Do not include any explanation or additional text.```### Custom Prompt Examples@@ -174,14 +179,14 @@ Score:custom_prompt = """You are a medical information specialist. Rate how relevant each memory is for answering the medical query.Consider clinical accuracy, specificity, and practical applicability.-Rate from 1-10 where:-- 1-3: Irrelevant or potentially harmful-- 4-6: Somewhat relevant but incomplete-- 7-8: Relevant and helpful-- 9-10: Highly relevant and clinically useful--Query: {query}-Memory: {memory}+Rate from 0.0 to 1.0 where:+- 0.0-0.3: Irrelevant or potentially harmful+- 0.4-0.6: Somewhat relevant but incomplete+- 0.7-0.8: Relevant and helpful+- 0.9-1.0: Highly relevant and clinically useful++Query: "{query}"+Document: "{document}"Score:"""@@ -189,14 +194,10 @@ config = {"reranker": {"provider": "llm_reranker","config": {- "llm": {- "provider": "openai",- "config": {- "model": "gpt-4",- "api_key": "your-api-key"- }- },- "custom_prompt": custom_prompt+ "provider": "openai",+ "model": "gpt-4o-mini",+ "api_key": "your-api-key",+ "scoring_prompt": custom_prompt}}}@@ -213,15 +214,15 @@ Consider:- Recency and accuracy- Practical usefulness-Rate 1-5:-1 = Not relevant-2 = Slightly relevant-3 = Moderately relevant-4 = Very relevant-5 = Perfectly answers the question+Rate 0.0-1.0:+0.0 = Not relevant+0.25 = Slightly relevant+0.5 = Moderately relevant+0.75 = Very relevant+1.0 = Perfectly answers the question-Query: {query}-Memory: {memory}+Query: "{query}"+Document: "{document}"Score:"""```@@ -239,13 +240,17 @@ Consider:- Factual accuracy- Conversation flow-Rate 0-10:-Query: {query}-Memory: {memory}+Rate 0.0-1.0:+Query: "{query}"+Document: "{document}"Score:"""```+<Note>+ Custom prompts must include `{query}` and `{document}` placeholders. The LLM response should contain a numerical score which is automatically extracted.+</Note>+## Usage Examples### Basic Usage@@ -295,9 +300,9 @@ results = safe_llm_rerank_search("What are my preferences?", "alice")| Model Type | Speed | Quality | Cost | Best For ||------------|-------|---------|------|----------|-| GPT-3.5 Turbo | Fast | Good | Low | High-volume applications |-| GPT-4 | Medium | Excellent | Medium | Quality-critical applications |-| Claude 3 Sonnet | Medium | Excellent | Medium | Balanced performance |+| GPT-4o mini | Fast | Good | Low | High-volume applications |+| GPT-4o | Medium | Excellent | Medium | Quality-critical applications |+| Claude Sonnet | Medium | Excellent | Medium | Balanced performance || Ollama Local | Variable | Good | Free | Privacy-sensitive applications |### Optimization Strategies@@ -308,14 +313,10 @@ fast_config = {"reranker": {"provider": "llm_reranker","config": {- "llm": {- "provider": "openai",- "config": {- "model": "gpt-3.5-turbo",- "api_key": "your-api-key"- }- },- "top_k": 5, # Limit candidates+ "provider": "openai",+ "model": "gpt-4o-mini",+ "api_key": "your-api-key",+ "top_k": 5,"temperature": 0.0}}@@ -326,13 +327,9 @@ quality_config = {"reranker": {"provider": "llm_reranker","config": {- "llm": {- "provider": "openai",- "config": {- "model": "gpt-4",- "api_key": "your-api-key"- }- },+ "provider": "openai",+ "model": "gpt-4o",+ "api_key": "your-api-key","top_k": 15,"temperature": 0.0}@@ -353,10 +350,10 @@ Evaluate this memory's relevance using multi-step reasoning:3. How directly does the memory address the query?4. What additional context might be needed?-Based on this analysis, rate relevance 1-10:+Based on this analysis, rate relevance 0.0-1.0:-Query: {query}-Memory: {memory}+Query: "{query}"+Document: "{document}"Analysis:Step 1 (Intent):@@ -367,42 +364,6 @@ Final Score:"""```-### Comparative Ranking--```python-comparative_prompt = """-You will see a query and multiple memories. Rank them in order of relevance.-Consider which memories best answer the question and would be most helpful.--Query: {query}--Memories to rank:-{memories}--Provide scores 1-10 for each memory, considering their relative usefulness.-"""-```--### Emotional Intelligence--```python-emotional_prompt = """-Consider both factual relevance and emotional appropriateness.-Rate how suitable this memory is for responding to the user's query.--Factors to consider:-- Factual accuracy and relevance-- Emotional tone and sensitivity-- User's likely emotional state-- Appropriateness of response--Query: {query}-Memory: {memory}-Emotional Context: {context}-Score (1-10):-"""-```-## Error Handling and Fallbacks```python@@ -433,14 +394,22 @@ class RobustLLMReranker:primary_config = {"reranker": {"provider": "llm_reranker",- "config": {"llm": {"provider": "openai", "config": {"model": "gpt-4"}}}+ "config": {+ "provider": "openai",+ "model": "gpt-4o",+ "api_key": "your-api-key"+ }}}fallback_config = {"reranker": {"provider": "llm_reranker",- "config": {"llm": {"provider": "openai", "config": {"model": "gpt-3.5-turbo"}}}+ "config": {+ "provider": "openai",+ "model": "gpt-4o-mini",+ "api_key": "your-api-key"+ }}}@@ -486,4 +455,4 @@ results = reranker.search("What are my preferences?", "alice")<Card title="Performance Optimization" icon="bolt" href="/components/rerankers/optimization">Optimize LLM reranker performance</Card>-</CardGroup>\ No newline at end of file+</CardGroup>diff --git a/docs/open-source/features/reranker-search.mdx b/docs/open-source/features/reranker-search.mdxindex fbae84182..893d9a7af 100644--- a/docs/open-source/features/reranker-search.mdx+++ b/docs/open-source/features/reranker-search.mdx@@ -123,13 +123,9 @@ config = {"reranker": {"provider": "llm_reranker","config": {- "llm": {- "provider": "openai",- "config": {- "model": "gpt-4",- "api_key": "your-openai-api-key"- }- },+ "provider": "openai",+ "model": "gpt-4o-mini",+ "api_key": "your-openai-api-key","top_k": 5}}