The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.
Python agentic-memoryagentic-memory-systemagentsaiai-agentschatgptgenaillmlong-term-memorymemorymemory-managementpythonragstate-management
Doc: pipecat integration (#2494)
Dev Khant039756abbe80
diff --git a/docs/docs.json b/docs/docs.jsonindex f0c0918de..8ffa66b08 100644--- a/docs/docs.json+++ b/docs/docs.json@@ -225,7 +225,8 @@"integrations/dify","integrations/mcp-server","integrations/livekit",- "integrations/elevenlabs"+ "integrations/elevenlabs",+ "integrations/pipecat"]}]diff --git a/docs/integrations/overview.mdx b/docs/integrations/overview.mdxindex 5d5458a19..82d6c8523 100644--- a/docs/integrations/overview.mdx+++ b/docs/integrations/overview.mdx@@ -265,4 +265,25 @@ Here are the available integrations for Mem0:>Build voice agents with memory using ElevenLabs Conversational AI.</Card>+ <Card+ title="Pipecat"+ icon={+ <svg+ viewBox="0 0 24 24"+ xmlns="http://www.w3.org/2000/svg"+ width="24"+ height="24"+ >+ <path d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm0 18c-4.41 0-8-3.59-8-8s3.59-8 8-8 8 3.59 8 8-3.59 8-8 8z" fill="currentColor"/>+ <circle cx="8.5" cy="9" r="1.5" fill="currentColor"/>+ <circle cx="15.5" cy="9" r="1.5" fill="currentColor"/>+ <path d="M12 16c1.66 0 3-1.34 3-3H9c0 1.66 1.34 3 3 3z" fill="currentColor"/>+ <path d="M17.5 12c-.83 0-1.5-.67-1.5-1.5s.67-1.5 1.5-1.5 1.5.67 1.5 1.5-.67 1.5-1.5 1.5z" fill="currentColor"/>+ <path d="M6.5 12c-.83 0-1.5-.67-1.5-1.5S5.67 9 6.5 9s1.5.67 1.5 1.5S7.33 12 6.5 12z" fill="currentColor"/>+ </svg>+ }+ href="/integrations/pipecat"+ >+ Build conversational AI agents with memory using Pipecat.+ </Card></CardGroup>diff --git a/docs/integrations/pipecat.mdx b/docs/integrations/pipecat.mdxnew file mode 100644index 000000000..231451b08--- /dev/null+++ b/docs/integrations/pipecat.mdx@@ -0,0 +1,218 @@+---+title: 'Pipecat'+description: 'Integrate Mem0 with Pipecat for conversational memory in AI agents'+---++# Pipecat Integration++Mem0 seamlessly integrates with [Pipecat](https://pipecat.ai), providing long-term memory capabilities for conversational AI agents. This integration allows your Pipecat-powered applications to remember past conversations and provide personalized responses based on user history.++## Installation++To use Mem0 with Pipecat, install the required dependencies:++```bash+pip install "pipecat-ai[mem0]"+```++You'll also need to set up your Mem0 API key as an environment variable:++```bash+export MEM0_API_KEY=your_mem0_api_key+```++You can obtain a Mem0 API key by signing up at [mem0.ai](https://mem0.ai).++## Configuration++Mem0 integration is provided through the `Mem0MemoryService` class in Pipecat. Here's how to configure it:++```python+from pipecat.services.mem0 import Mem0MemoryService++memory = Mem0MemoryService(+ api_key=os.getenv("MEM0_API_KEY"), # Your Mem0 API key+ user_id="unique_user_id", # Unique identifier for the end user+ agent_id="my_agent", # Identifier for the agent using the memory+ run_id="session_123", # Optional: specific conversation session ID+ params={ # Optional: configuration parameters+ "search_limit": 10, # Maximum memories to retrieve per query+ "search_threshold": 0.1, # Relevance threshold (0.0 to 1.0)+ "system_prompt": "Here are your past memories:", # Custom prefix for memories+ "add_as_system_message": True, # Add memories as system (True) or user (False) message+ "position": 1, # Position in context to insert memories+ }+)+```++## Pipeline Integration++The `Mem0MemoryService` should be positioned between your context aggregator and LLM service in the Pipecat pipeline:++```python+pipeline = Pipeline([+ transport.input(),+ stt, # Speech-to-text for audio input+ user_context, # User context aggregator+ memory, # Mem0 Memory service enhances context here+ llm, # LLM for response generation+ tts, # Optional: Text-to-speech+ transport.output(),+ assistant_context # Assistant context aggregator+])+```++## Example: Voice Agent with Memory++Here's a complete example of a Pipecat voice agent with Mem0 memory integration:++```python+import asyncio+import os+from fastapi import FastAPI, WebSocket++from pipecat.frames.frames import TextFrame+from pipecat.pipeline.pipeline import Pipeline+from pipecat.pipeline.task import PipelineTask+from pipecat.pipeline.runner import PipelineRunner+from pipecat.services.mem0 import Mem0MemoryService+from pipecat.services.openai import OpenAILLMService, OpenAIUserContextAggregator, OpenAIAssistantContextAggregator+from pipecat.transports.network.fastapi_websocket import (+ FastAPIWebsocketTransport,+ FastAPIWebsocketParams+)+from pipecat.serializers.protobuf import ProtobufFrameSerializer+from pipecat.audio.vad.silero import SileroVADAnalyzer+from pipecat.services.whisper import WhisperSTTService++app = FastAPI()+[email protected]("/chat")+async def websocket_endpoint(websocket: WebSocket):+ await websocket.accept()++ # Basic setup with minimal configuration+ user_id = "user123"++ # WebSocket transport+ transport = FastAPIWebsocketTransport(+ websocket=websocket,+ params=FastAPIWebsocketParams(+ audio_out_enabled=True,+ vad_enabled=True,+ vad_analyzer=SileroVADAnalyzer(),+ vad_audio_passthrough=True,+ serializer=ProtobufFrameSerializer(),+ )+ )++ # Core services+ user_context = OpenAIUserContextAggregator()+ assistant_context = OpenAIAssistantContextAggregator()+ stt = WhisperSTTService(api_key=os.getenv("OPENAI_API_KEY"))++ # Memory service - the key component+ memory = Mem0MemoryService(+ api_key=os.getenv("MEM0_API_KEY"),+ user_id=user_id,+ agent_id="fastapi_memory_bot"+ )++ # LLM for response generation+ llm = OpenAILLMService(+ api_key=os.getenv("OPENAI_API_KEY"),+ model="gpt-3.5-turbo",+ system_prompt="You are a helpful assistant that remembers past conversations."+ )++ # Simple pipeline+ pipeline = Pipeline([+ transport.input(),+ stt, # Speech-to-text for audio input+ user_context,+ memory, # Memory service enhances context here+ llm,+ transport.output(),+ assistant_context+ ])++ # Run the pipeline+ runner = PipelineRunner()+ task = PipelineTask(pipeline)++ # Event handlers for WebSocket connections+ @transport.event_handler("on_client_connected")+ async def on_client_connected(transport, client):+ # Send welcome message when client connects+ await task.queue_frame(TextFrame("Hello! I'm a memory bot. I'll remember our conversation."))++ @transport.event_handler("on_client_disconnected")+ async def on_client_disconnected(transport, client):+ # Clean up when client disconnects+ await task.cancel()++ await runner.run(task)++if __name__ == "__main__":+ import uvicorn+ uvicorn.run(app, host="0.0.0.0", port=8000)+```++## How It Works++When integrated with Pipecat, Mem0 provides two key functionalities:++### 1. Message Storage++All conversation messages are automatically stored in Mem0 for future reference:+- Captures the full message history from context frames+- Associates messages with the specified user, agent, and run IDs+- Stores metadata to enable efficient retrieval++### 2. Memory Retrieval++When a new user message is detected:+1. The message is used as a search query to find relevant past memories+2. Relevant memories are retrieved from Mem0's database+3. Memories are formatted and added to the conversation context+4. The enhanced context is passed to the LLM for response generation++## Additional Configuration Options++### Memory Search Parameters++You can customize how memories are retrieved and used:++```python+memory = Mem0MemoryService(+ api_key=os.getenv("MEM0_API_KEY"),+ user_id="user123",+ params={+ "search_limit": 5, # Retrieve up to 5 memories+ "search_threshold": 0.2, # Higher threshold for more relevant matches+ "api_version": "v2", # Mem0 API version+ }+)+```++### Memory Presentation Options++Control how memories are presented to the LLM:++```python+memory = Mem0MemoryService(+ api_key=os.getenv("MEM0_API_KEY"),+ user_id="user123",+ params={+ "system_prompt": "Previous conversations with this user:",+ "add_as_system_message": True, # Add as system message instead of user message+ "position": 0, # Insert at the beginning of the context+ }+)+```++## Resources++- [Mem0 Pipecat Integration](https://docs.pipecat.ai/server/services/memory/mem0)+- [Pipecat Documentation](https://docs.pipecat.ai)+