Skip to content

armgit / letta

Public Imported from GitHub
0

Platform for stateful agents: AI with advanced memory that can learn and self-improve over time.

docs: remove pip installation and letta server references (#5827)

Camerond27cfea13974
diff --git a/fern/pages/agents/fetch_webpage.mdx b/fern/pages/agents/fetch_webpage.mdx
new file mode 100644
index 000000000..d4baad88c
--- /dev/null
+++ b/fern/pages/agents/fetch_webpage.mdx
@@ -0,0 +1,161 @@
+---
+title: Fetch Webpage
+subtitle: Convert webpages to readable text/markdown
+slug: guides/agents/fetch-webpage
+---
+
+The `fetch_webpage` tool enables Letta agents to fetch and convert webpages into readable text or markdown format. Useful for reading documentation, articles, and web content.
+
+<Info>
+On [Letta Cloud](/guides/cloud/overview), this tool works out of the box. For self-hosted deployments with an Exa API key, fetching is enhanced. Without a key, it falls back to open-source extraction tools.
+</Info>
+
+## Quick Start
+
+<CodeGroup>
+```python Python
+from letta import Letta
+
+client = Letta(token="LETTA_API_KEY")
+
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["fetch_webpage"],
+ memory_blocks=[{
+ "label": "persona",
+ "value": "I can fetch and read webpages to answer questions about online content."
+ }]
+)
+```
+
+```typescript TypeScript
+import { LettaClient } from '@letta-ai/letta-client';
+
+const client = new LettaClient({ token: "LETTA_API_KEY" });
+
+const agent = await client.agents.create({
+ model: "openai/gpt-4o",
+ tools: ["fetch_webpage"],
+ memoryBlocks: [{
+ label: "persona",
+ value: "I can fetch and read webpages to answer questions about online content."
+ }]
+});
+```
+</CodeGroup>
+
+## Tool Parameters
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `url` | `str` | The URL of the webpage to fetch |
+
+## Return Format
+
+The tool returns webpage content as text/markdown.
+
+**With Exa API (if configured):**
+```json
+{
+ "title": "Page title",
+ "published_date": "2025-01-15",
+ "author": "Author name",
+ "text": "Full page content in markdown"
+}
+```
+
+**Fallback (without Exa):**
+Returns markdown-formatted text extracted from the HTML.
+
+## How It Works
+
+The tool uses a multi-tier approach:
+
+1. **Exa API** (if `EXA_API_KEY` is configured): Uses Exa's content extraction
+2. **Trafilatura** (fallback): Open-source text extraction to markdown
+3. **Readability + html2text** (final fallback): HTML cleaning and conversion
+
+## Self-Hosted Setup
+
+For enhanced fetching on self-hosted servers, optionally configure an Exa API key. Without it, the tool still works using open-source extraction.
+
+### Optional: Configure Exa
+
+<CodeGroup>
+```bash Docker
+docker run \
+ -e EXA_API_KEY="your_exa_api_key" \
+ letta/letta:latest
+```
+
+```yaml Docker Compose
+services:
+ letta:
+ environment:
+ - EXA_API_KEY=your_exa_api_key
+```
+
+```python Per-Agent
+agent = client.agents.create(
+ tools=["fetch_webpage"],
+ tool_env_vars={
+ "EXA_API_KEY": "your_exa_api_key"
+ }
+)
+```
+</CodeGroup>
+
+## Common Patterns
+
+### Documentation Reader
+```python
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["fetch_webpage", "web_search"],
+ memory_blocks=[{
+ "label": "persona",
+ "value": "I search for documentation with web_search and read it with fetch_webpage."
+ }]
+)
+```
+
+### Research Assistant
+```python
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["fetch_webpage", "archival_memory_insert"],
+ memory_blocks=[{
+ "label": "persona",
+ "value": "I fetch articles and store key insights in archival memory for later reference."
+ }]
+)
+```
+
+### Content Summarizer
+```python
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["fetch_webpage"],
+ memory_blocks=[{
+ "label": "persona",
+ "value": "I fetch webpages and provide summaries of their content."
+ }]
+)
+```
+
+## When to Use
+
+| Use Case | Tool | Why |
+|----------|------|-----|
+| Read specific webpage | `fetch_webpage` | Direct URL access |
+| Find webpages to read | `web_search` | Discovery first |
+| Read + search in one | `web_search` with `include_text=true` | Combined operation |
+| Multiple pages | `fetch_webpage` | Iterate over URLs |
+
+## Related Documentation
+
+- [Utilities Overview](/guides/agents/prebuilt-tools)
+- [Web Search](/guides/agents/web-search)
+- [Run Code](/guides/agents/run-code)
+- [Custom Tools](/guides/agents/custom-tools)
+- [Tool Variables](/guides/agents/tool-variables)
diff --git a/fern/pages/agents/run_code.mdx b/fern/pages/agents/run_code.mdx
new file mode 100644
index 000000000..c7bcd4540
--- /dev/null
+++ b/fern/pages/agents/run_code.mdx
@@ -0,0 +1,253 @@
+---
+title: Code Interpreter
+subtitle: Execute code in a secure sandbox with full network access
+slug: guides/agents/run-code
+---
+
+The `run_code` tool enables Letta agents to execute code in a secure sandboxed environment. Useful for data analysis, calculations, API calls, and programmatic computation.
+
+<Info>
+On [Letta Cloud](/guides/cloud/overview), this tool works out of the box. For self-hosted deployments, you'll need to [configure an E2B API key](#self-hosted-setup).
+</Info>
+
+<Warning>
+Each execution runs in a **fresh environment** - variables, files, and state do not persist between runs.
+</Warning>
+
+## Quick Start
+
+<CodeGroup>
+```python Python
+from letta import Letta
+
+client = Letta(token="LETTA_API_KEY")
+
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["run_code"],
+ memory_blocks=[{
+ "label": "persona",
+ "value": "I can run Python code for data analysis and API calls."
+ }]
+)
+```
+
+```typescript TypeScript
+import { LettaClient } from '@letta-ai/letta-client';
+
+const client = new LettaClient({ token: "LETTA_API_KEY" });
+
+const agent = await client.agents.create({
+ model: "openai/gpt-4o",
+ tools: ["run_code"],
+ memoryBlocks: [{
+ label: "persona",
+ value: "I can run Python code for data analysis and API calls."
+ }]
+});
+```
+</CodeGroup>
+
+## Tool Parameters
+
+| Parameter | Type | Options | Description |
+|-----------|------|---------|-------------|
+| `code` | `str` | Required | The code to execute |
+| `language` | `str` | `python`, `js`, `ts`, `r`, `java` | Programming language |
+
+## Return Format
+
+```json
+{
+ "results": ["Last expression value"],
+ "logs": {
+ "stdout": ["Print statements"],
+ "stderr": ["Error output"]
+ },
+ "error": "Error details if execution failed"
+}
+```
+
+**Output types:**
+- `results[]`: Last expression value (Jupyter-style)
+- `logs.stdout`: Print statements and standard output
+- `logs.stderr`: Error messages
+- `error`: Present if execution failed
+
+## Supported Languages
+
+| Language | Key Limitations |
+|----------|-----------------|
+| **Python** | None - full ecosystem available |
+| **JavaScript** | No npm packages - built-in Node modules only |
+| **TypeScript** | No npm packages - built-in Node modules only |
+| **R** | No tidyverse - base R only |
+| **Java** | JShell-style execution - no traditional class definitions |
+
+### Python
+
+Full Python ecosystem with common packages pre-installed:
+
+- **Data**: numpy, pandas, scipy, scikit-learn
+- **Web**: requests, aiohttp, beautifulsoup4
+- **Utilities**: matplotlib, PyYAML, Pillow
+
+Check available packages:
+```python
+import pkg_resources
+print([d.project_name for d in pkg_resources.working_set])
+```
+
+### JavaScript & TypeScript
+
+No npm packages available - only built-in Node modules.
+
+```javascript
+// Works
+const fs = require('fs');
+const http = require('http');
+
+// Fails
+const axios = require('axios');
+```
+
+### R
+
+Base R only - no tidyverse packages.
+
+```r
+# Works
+mean(c(1, 2, 3))
+
+# Fails
+library(ggplot2)
+```
+
+### Java
+
+JShell-style execution - statement-level only.
+
+```java
+// Works
+System.out.println("Hello");
+int x = 42;
+
+// Fails
+public class Main {
+ public static void main(String[] args) { }
+}
+```
+
+## Network Access
+
+The sandbox has full network access for HTTP requests, API calls, and DNS resolution.
+
+```python
+import requests
+
+response = requests.get('https://api.github.com/repos/letta-ai/letta')
+data = response.json()
+print(f"Stars: {data['stargazers_count']}")
+```
+
+## No State Persistence
+
+Variables, files, and state do not carry over between executions. Each `run_code` call is completely isolated.
+
+```python
+# First execution
+x = 42
+
+# Second execution (separate run_code call)
+print(x) # Error: NameError: name 'x' is not defined
+```
+
+**Implications:**
+- Must re-import libraries each time
+- Files written to disk are lost
+- Cannot build up state across executions
+
+## Self-Hosted Setup
+
+For self-hosted servers, configure an E2B API key. [E2B](https://e2b.dev) provides the sandbox infrastructure.
+
+<CodeGroup>
+```bash Docker
+docker run \
+ -e E2B_API_KEY="your_e2b_api_key" \
+ letta/letta:latest
+```
+
+```yaml Docker Compose
+services:
+ letta:
+ environment:
+ - E2B_API_KEY=your_e2b_api_key
+```
+
+```python Per-Agent
+agent = client.agents.create(
+ tools=["run_code"],
+ tool_env_vars={
+ "E2B_API_KEY": "your_e2b_api_key"
+ }
+)
+```
+</CodeGroup>
+
+## Common Patterns
+
+### Data Analysis
+```python
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["run_code"],
+ memory_blocks=[{
+ "label": "persona",
+ "value": "I use Python with pandas and numpy for data analysis."
+ }]
+)
+```
+
+### API Integration
+```python
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["run_code", "web_search"],
+ memory_blocks=[{
+ "label": "persona",
+ "value": "I fetch data from APIs using run_code and search docs with web_search."
+ }]
+)
+```
+
+### Statistical Analysis
+```python
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["run_code"],
+ memory_blocks=[{
+ "label": "persona",
+ "value": "I perform statistical analysis using scipy and numpy."
+ }]
+)
+```
+
+## When to Use
+
+| Use Case | Tool | Why |
+|----------|------|-----|
+| Data analysis | `run_code` | Full Python data stack |
+| Math calculations | `run_code` | Programmatic computation |
+| Live API data | `run_code` | Network + processing |
+| Web scraping | `run_code` | requests + BeautifulSoup |
+| Simple search | `web_search` | Purpose-built |
+| Persistent data | Archival memory | State persistence |
+
+## Related Documentation
+
+- [Utilities Overview](/guides/agents/prebuilt-tools)
+- [Web Search](/guides/agents/web-search)
+- [Fetch Webpage](/guides/agents/fetch-webpage)
+- [Custom Tools](/guides/agents/custom-tools)
+- [Tool Variables](/guides/agents/tool-variables)
diff --git a/fern/pages/agents/web_search.mdx b/fern/pages/agents/web_search.mdx
new file mode 100644
index 000000000..e643b00f4
--- /dev/null
+++ b/fern/pages/agents/web_search.mdx
@@ -0,0 +1,337 @@
+---
+title: Web Search
+subtitle: Search the internet in real-time with AI-powered search
+slug: guides/agents/web-search
+---
+
+The `web_search` tool enables Letta agents to search the internet for current information, research, and general knowledge using [Exa](https://exa.ai)'s AI-powered search engine.
+
+<Info>
+On [Letta Cloud](/guides/cloud/overview), this tool works out of the box. For self-hosted deployments, you'll need to [configure an Exa API key](#self-hosted-setup).
+</Info>
+
+## Quick Start
+
+### Adding Web Search to an Agent
+
+<CodeGroup>
+```python Python
+from letta import Letta
+
+client = Letta(token="LETTA_API_KEY")
+
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ embedding="openai/text-embedding-3-small",
+ tools=["web_search"],
+ memory_blocks=[
+ {
+ "label": "persona",
+ "value": "I'm a research assistant who uses web search to find current information and cite sources."
+ }
+ ]
+)
+```
+
+```typescript TypeScript
+import { LettaClient } from '@letta-ai/letta-client';
+
+const client = new LettaClient({ token: "LETTA_API_KEY" });
+
+const agent = await client.agents.create({
+ model: "openai/gpt-4o",
+ embedding: "openai/text-embedding-3-small",
+ tools: ["web_search"],
+ memoryBlocks: [
+ {
+ label: "persona",
+ value: "I'm a research assistant who uses web search to find current information and cite sources."
+ }
+ ]
+});
+```
+</CodeGroup>
+
+### Usage Example
+
+```python
+response = client.agents.messages.create(
+ agent_id=agent.id,
+ messages=[
+ {
+ "role": "user",
+ "content": "What are the latest developments in agent-based AI systems?"
+ }
+ ]
+)
+```
+
+Your agent can now choose to use `web_search` when it needs current information.
+
+## Self-Hosted Setup
+
+For self-hosted Letta servers, you'll need an Exa API key.
+
+### Get an API Key
+
+1. Sign up at [dashboard.exa.ai](https://dashboard.exa.ai/)
+2. Copy your API key
+3. See [Exa pricing](https://docs.exa.ai) for rate limits and costs
+
+### Configuration Options
+
+<CodeGroup>
+```bash Docker
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e OPENAI_API_KEY="your_openai_key" \
+ -e EXA_API_KEY="your_exa_api_key" \
+ letta/letta:latest
+```
+
+```yaml Docker Compose
+version: '3.8'
+services:
+ letta:
+ image: letta/letta:latest
+ ports:
+ - "8283:8283"
+ environment:
+ - OPENAI_API_KEY=your_openai_key
+ - EXA_API_KEY=your_exa_api_key
+ volumes:
+ - ~/.letta/.persist/pgdata:/var/lib/postgresql/data
+```
+
+```python Per-Agent Configuration
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ embedding="openai/text-embedding-3-small",
+ tools=["web_search"],
+ tool_env_vars={
+ "EXA_API_KEY": "your_exa_api_key"
+ }
+)
+```
+</CodeGroup>
+
+## Tool Parameters
+
+The `web_search` tool supports advanced filtering and search customization:
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `query` | `str` | Required | The search query to find relevant web content |
+| `num_results` | `int` | 10 | Number of results to return (1-100) |
+| `category` | `str` | None | Focus search on specific content types (see below) |
+| `include_text` | `bool` | False | Whether to retrieve full page content (usually overflows context) |
+| `include_domains` | `List[str]` | None | List of domains to include in search results |
+| `exclude_domains` | `List[str]` | None | List of domains to exclude from search results |
+| `start_published_date` | `str` | None | Only return content published after this date (ISO format) |
+| `end_published_date` | `str` | None | Only return content published before this date (ISO format) |
+| `user_location` | `str` | None | Two-letter country code for localized results (e.g., "US") |
+
+### Available Categories
+
+Use the `category` parameter to focus your search on specific content types:
+
+| Category | Best For | Example Query |
+|----------|----------|---------------|
+| `company` | Corporate information, company websites | "Tesla energy storage solutions" |
+| `research paper` | Academic papers, arXiv, research publications | "transformer architecture improvements 2025" |
+| `news` | News articles, current events | "latest AI policy developments" |
+| `pdf` | PDF documents, reports, whitepapers | "climate change impact assessment" |
+| `github` | GitHub repositories, open source projects | "python async web scraping libraries" |
+| `tweet` | Twitter/X posts, social media discussions | "reactions to new GPT release" |
+| `personal site` | Blogs, personal websites, portfolios | "machine learning tutorial blogs" |
+| `linkedin profile` | LinkedIn profiles, professional bios | "AI research engineers at Google" |
+| `financial report` | Earnings reports, financial statements | "Apple Q4 2024 earnings" |
+
+### Return Format
+
+The tool returns a JSON-encoded string containing:
+
+```json
+{
+ "query": "search query",
+ "results": [
+ {
+ "title": "Page title",
+ "url": "https://example.com",
+ "published_date": "2025-01-15",
+ "author": "Author name",
+ "highlights": ["Key excerpt 1", "Key excerpt 2"],
+ "summary": "AI-generated summary of the content",
+ "text": "Full page content (only if include_text=true)"
+ }
+ ]
+}
+```
+
+## Best Practices
+
+### 1. Guide When to Search
+
+Provide clear instructions to your agent about when web search is appropriate:
+
+```python
+memory_blocks=[
+ {
+ "label": "persona",
+ "value": "I'm a helpful assistant. I use web_search for current events, recent news, and topics requiring up-to-date information. I cite my sources."
+ }
+]
+```
+
+### 2. Combine with Archival Memory
+
+Use web search for external/current information, and archival memory for your organization's internal data:
+
+```python
+# Create agent with both web_search and archival memory tools
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ embedding="openai/text-embedding-3-small",
+ tools=["web_search", "archival_memory_search", "archival_memory_insert"],
+ memory_blocks=[
+ {
+ "label": "persona",
+ "value": "I use web_search for current events and external research. I use archival_memory_search for company-specific information and internal documents."
+ }
+ ]
+)
+```
+
+See the [Archival Memory documentation](/guides/agents/archival-memory-overview) for more information.
+
+### 3. Craft Effective Search Queries
+
+Exa uses neural search that understands semantic meaning. Your agent will generally form good queries naturally, but you can improve results by guiding it to:
+
+- **Be descriptive and specific**: "Latest research on RLHF techniques for language models" is better than "RLHF research"
+- **Focus on topics, not keywords**: "How companies are deploying AI agents in customer service" works better than "AI agents customer service deployment"
+- **Use natural language**: The search engine understands conversational queries like "What are the environmental impacts of Bitcoin mining?"
+- **Specify time ranges when relevant**: Guide your agent to use date filters for time-sensitive queries
+
+Example instruction in memory:
+
+```python
+memory_blocks=[
+ {
+ "label": "search_strategy",
+ "value": "When searching, I craft clear, descriptive queries that focus on topics rather than keywords. I use the category and date filters when appropriate to narrow results."
+ }
+]
+```
+
+### 4. Manage Context Window
+
+By default, `include_text` is `False` to avoid context overflow. The tool returns highlights and AI-generated summaries instead, which are more concise:
+
+```python
+memory_blocks=[
+ {
+ "label": "search_guidelines",
+ "value": "I avoid setting include_text=true unless specifically needed, as full text usually overflows the context window. Highlights and summaries are usually sufficient."
+ }
+]
+```
+
+## Common Patterns
+
+### Research Assistant
+
+```python
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["web_search"],
+ memory_blocks=[
+ {
+ "label": "persona",
+ "value": "I'm a research assistant. I search for relevant information, synthesize findings from multiple sources, and provide citations."
+ }
+ ]
+)
+```
+
+### News Monitor
+
+```python
+agent = client.agents.create(
+ model="openai/gpt-4o-mini",
+ tools=["web_search"],
+ memory_blocks=[
+ {
+ "label": "persona",
+ "value": "I monitor news and provide briefings on AI industry developments."
+ },
+ {
+ "label": "topics",
+ "value": "Focus: AI/ML, agent systems, LLM advancements"
+ }
+ ]
+)
+```
+
+### Customer Support
+
+```python
+agent = client.agents.create(
+ model="openai/gpt-4o",
+ tools=["web_search"],
+ memory_blocks=[
+ {
+ "label": "persona",
+ "value": "I help customers by checking documentation, service status pages, and community discussions for solutions."
+ }
+ ]
+)
+```
+
+## Troubleshooting
+
+### Agent Not Using Web Search
+
+Check:
+1. Tool is attached: `"web_search"` in agent's tools list
+2. Instructions are clear about when to search
+3. Model has good tool-calling capabilities (GPT-4, Claude 3+)
+
+```python
+# Verify tools
+agent = client.agents.retrieve(agent_id=agent.id)
+print([tool.name for tool in agent.tools])
+```
+
+### Missing EXA_API_KEY
+
+If you see errors about missing API keys on self-hosted deployments:
+
+```bash
+# Check if set
+echo $EXA_API_KEY
+
+# Set for session
+export EXA_API_KEY="your_exa_api_key"
+
+# Docker example
+docker run -e EXA_API_KEY="your_exa_api_key" letta/letta:latest
+```
+
+## When to Use Web Search
+
+| Use Case | Tool | Why |
+|----------|------|-----|
+| Current events, news | `web_search` | Real-time information |
+| External research | `web_search` | Broad internet access |
+| Internal documents | Archival memory | Fast, static data |
+| User preferences | Memory blocks | In-context, instant |
+| General knowledge | Pre-trained model | No search needed |
+
+## Related Documentation
+
+- [Utilities Overview](/guides/agents/prebuilt-tools)
+- [Custom Tools](/guides/agents/custom-tools)
+- [Tool Variables](/guides/agents/tool-variables)
+- [Archival Memory](/guides/agents/archival-memory-overview)
diff --git a/fern/pages/evals/getting-started.mdx b/fern/pages/evals/getting-started.mdx
new file mode 100644
index 000000000..09c6422b5
--- /dev/null
+++ b/fern/pages/evals/getting-started.mdx
@@ -0,0 +1,264 @@
+# Getting Started
+
+Run your first Letta agent evaluation in 5 minutes.
+
+## Prerequisites
+
+- Python 3.11 or higher
+- A running Letta server (local or Letta Cloud)
+- A Letta agent to test, either in agent file format or by ID (see [Targets](/evals/core-concepts/targets) for more details)
+
+## Installation
+
+```bash
+pip install letta-evals
+```
+
+Or with uv:
+
+```bash
+uv pip install letta-evals
+```
+
+## Getting an Agent to Test
+
+Export an existing agent to a file using the Letta SDK:
+
+```python
+from letta_client import Letta
+import os
+
+client = Letta(
+ base_url="http://localhost:8283", # or https://api.letta.com for Letta Cloud
+ token=os.getenv("LETTA_API_KEY") # required for Letta Cloud
+)
+
+# Export an agent to a file
+agent_file = client.agents.export_file(agent_id="agent-123")
+
+# Save to disk
+with open("my_agent.af", "w") as f:
+ f.write(agent_file)
+```
+
+Or export via the Agent Development Environment (ADE) by selecting "Export Agent".
+
+Then reference it in your suite:
+
+```yaml
+target:
+ kind: agent
+ agent_file: my_agent.af
+```
+
+<Note>
+**Other options:** You can also use existing agents by ID or programmatically generate agents. See [Targets](/evals/core-concepts/targets) for all agent configuration options.
+</Note>
+
+## Quick Start
+
+Let's create your first evaluation in 3 steps:
+
+### 1. Create a Test Dataset
+
+Create a file named `dataset.jsonl`:
+
+```jsonl
+{"input": "What's the capital of France?", "ground_truth": "Paris"}
+{"input": "Calculate 2+2", "ground_truth": "4"}
+{"input": "What color is the sky?", "ground_truth": "blue"}
+```
+
+Each line is a JSON object with:
+- `input`: The prompt to send to your agent
+- `ground_truth`: The expected answer (used for grading)
+
+<Note>
+`ground_truth` is optional for some graders (like rubric graders), but required for tool graders like `contains` and `exact_match`.
+</Note>
+
+Read more about [Datasets](/evals/core-concepts/datasets) for details on how to create your dataset.
+
+### 2. Create a Suite Configuration
+
+Create a file named `suite.yaml`:
+
+```yaml
+name: my-first-eval
+dataset: dataset.jsonl
+
+target:
+ kind: agent
+ agent_file: my_agent.af # Path to your agent file
+ base_url: http://localhost:8283 # Your Letta server
+
+graders:
+ quality:
+ kind: tool
+ function: contains # Check if response contains the ground truth
+ extractor: last_assistant # Use the last assistant message
+
+gate:
+ metric_key: quality
+ op: gte
+ value: 0.75 # Require 75% pass rate
+```
+
+The suite configuration defines:
+- The [dataset](/evals/core-concepts/datasets) to use
+- The [agent](/evals/core-concepts/targets) to test
+- The [graders](/evals/core-concepts/graders) to use
+- The [gate](/evals/core-concepts/gates) criteria
+
+Read more about [Suites](/evals/core-concepts/suites) for details on how to configure your evaluation.
+
+### 3. Run the Evaluation
+
+Run your evaluation with the following command:
+
+```bash
+letta-evals run suite.yaml
+```
+
+You'll see real-time progress as your evaluation runs:
+
+```
+Running evaluation: my-first-eval
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3/3 100%
+✓ PASSED (2.25/3.00 avg, 75.0% pass rate)
+```
+
+Read more about [CLI Commands](/evals/cli-reference/commands) for details about the available commands and options.
+
+## Understanding the Results
+
+The core evaluation flow is:
+
+**Dataset → Target (Agent) → Extractor → Grader → Gate → Result**
+
+The evaluation runner:
+1. Loads your dataset
+2. Sends each input to your agent (Target)
+3. Extracts the relevant information (using the Extractor)
+4. Grades the response (using the Grader function)
+5. Computes aggregate metrics
+6. Checks if metrics pass the Gate criteria
+
+The output shows:
+- **Average score**: Mean score across all samples
+- **Pass rate**: Percentage of samples that passed
+- **Gate status**: Whether the evaluation passed or failed overall
+
+## Next Steps
+
+Now that you've run your first evaluation, explore more advanced features:
+
+- [Core Concepts](/evals/core-concepts/concepts-overview) - Understand suites, datasets, graders, and extractors
+- [Grader Types](/evals/core-concepts/graders) - Learn about tool graders vs rubric graders
+- [Multi-Metric Evaluation](/evals/graders/multi-metric-grading) - Test multiple aspects simultaneously
+- [Custom Graders](/evals/advanced/custom-graders) - Write custom grading functions
+- [Multi-Turn Conversations](/evals/advanced/multi-turn-conversations) - Test conversational memory
+
+## Common Use Cases
+
+### Strict Answer Checking
+
+Use exact matching for cases where the answer must be precisely correct:
+
+```yaml
+graders:
+ accuracy:
+ kind: tool
+ function: exact_match
+ extractor: last_assistant
+```
+
+### Subjective Quality Evaluation
+
+Use an LLM judge to evaluate subjective qualities like helpfulness or tone:
+
+```yaml
+graders:
+ quality:
+ kind: rubric
+ prompt_path: rubric.txt
+ model: gpt-4o-mini
+ extractor: last_assistant
+```
+
+Then create `rubric.txt`:
+```
+Rate the helpfulness and accuracy of the response.
+- Score 1.0 if helpful and accurate
+- Score 0.5 if partially helpful
+- Score 0.0 if unhelpful or wrong
+```
+
+### Testing Tool Calls
+
+Verify that your agent calls specific tools with expected arguments:
+
+```yaml
+graders:
+ tool_check:
+ kind: tool
+ function: contains
+ extractor: tool_arguments
+ extractor_config:
+ tool_name: search
+```
+
+### Testing Memory Persistence
+
+Check if the agent correctly updates its memory blocks:
+
+```yaml
+graders:
+ memory_check:
+ kind: tool
+ function: contains
+ extractor: memory_block
+ extractor_config:
+ block_label: human
+```
+
+## Troubleshooting
+
+<Warning>
+**"Agent file not found"**
+
+Make sure your `agent_file` path is correct. Paths are relative to the suite YAML file location. Use absolute paths if needed:
+
+```yaml
+target:
+ agent_file: /absolute/path/to/my_agent.af
+```
+</Warning>
+
+<Warning>
+**"Connection refused"**
+
+Your Letta server isn't running or isn't accessible. Start it using Docker:
+
+```bash
+docker run -p 8283:8283 -e OPENAI_API_KEY="your_api_key" letta/letta:latest
+```
+
+By default, it runs at `http://localhost:8283`. See the [self-hosting guide](/guides/selfhosting) for more information.
+</Warning>
+
+<Warning>
+**"No ground_truth provided"**
+
+Tool graders like `exact_match` and `contains` require `ground_truth` in your dataset. Either:
+- Add `ground_truth` to your samples, or
+- Use a rubric grader which doesn't require ground truth
+</Warning>
+
+<Tip>
+**Agent didn't respond as expected**
+
+Try testing your agent manually first using the Letta SDK or Agent Development Environment (ADE) to see how it behaves before running evaluations. See the [Letta documentation](https://docs.letta.com) for more information.
+</Tip>
+
+For more help, see the [Troubleshooting Guide](/evals/troubleshooting/common-issues).
diff --git a/fern/pages/getting-started/faq.mdx b/fern/pages/getting-started/faq.mdx
new file mode 100644
index 000000000..2f1df3a5f
--- /dev/null
+++ b/fern/pages/getting-started/faq.mdx
@@ -0,0 +1,75 @@
+---
+title: Letta FAQs
+slug: faq
+---
+
+Can't find the answer to your question?
+Feel free to reach out to the Letta development team and community on [Discord](https://discord.gg/letta) or [GitHub](https://github.com/letta-ai/letta/issues)!
+
+## Letta Platform
+<AccordionGroup>
+ <Accordion title="Who is Letta for?">
+Letta is for developers building stateful LLM applications that require advanced memory, such as:
+<Frame>
+* personalized chatbots that require long-term memory and personas that should be updated (self-edited) over time (e.g. companions)
+* agents connected to external data sources, e.g. private enterprise deployments of ChatGPT-like applications (connected to your company’s data), or a medical assistant connected to a patient’s medical records
+* agents connected to custom tools, e.g. a chatbot that can answer questions about the latest news by searching the web
+* automated AI workflows, e.g. an agent that monitors your email inbox and sends you text alerts for urgent emails and a daily email summary
+</Frame>
+... and countless other use cases!
+ </Accordion>
+ <Accordion title="Can I use Letta locally?">
+Yes, Letta is an open source project and you can run it locally on your own machine.
+
+When you run Letta locally, you have the option to connect the agents server to external API providers (e.g. OpenAI, Anthropic) or connect to local or self-hosted LLM providers (e.g. Ollama or vLLM).
+ </Accordion>
+ <Accordion title="Is Letta free to use?">
+The open source Letta software is free to use and permissively licensed under the Apache 2.0 license.
+Letta Desktop is a free application that combines the Letta server and ADE into a single application.
+Letta Cloud is a paid service and requires a Letta Cloud account to use.
+ </Accordion>
+ <Accordion title="What's the difference between open source Letta and Letta Cloud?">
+Letta Cloud is a fully managed service that allows you to create and deploy Letta agents without running any infrastructure.
+If you'd like to build production applications using the Letta API, consider using Letta Cloud.
+ </Accordion>
+</AccordionGroup>
+
+## Agent Development Environment (ADE)
+<AccordionGroup>
+ <Accordion title="How do I use the ADE locally?">
+If you use [Letta Desktop](/quickstart/desktop), the ADE runs inside of Letta Desktop locally on your machine.<br /><br />
+If you are deploying Letta via Docker and want to use the ADE, you can connect the web ADE to your Docker deployment.
+To connect the ADE to your deployed Letta server, simply run your Letta server (if running locally, make sure you can access `localhost:8283`) and go to [https://app.letta.com](https://app.letta.com).
+ </Accordion>
+ <Accordion title="If I connect the web ADE to my local server, does my agent data get uploaded to letta.com?">
+No, the data in your Letta server database stays on your machine.
+The ADE web application simply connects to your local Letta server (via the REST API) and provides a graphical interface on top of it to visualize your local Letta data in your browser's local state.
+If you would like to run the ADE completely locally, you can use [Letta Desktop](/quickstart/desktop) instead.
+ </Accordion>
+ <Accordion title="Do I have to use your ADE? Can I build my own?">
+The ADE is built on top of the (fully open source) Letta server and Letta Agents API.
+You can build your own application like the ADE on top of the REST API (view the documention [here](https://docs.letta.com/api-reference)).
+ </Accordion>
+</AccordionGroup>
+
+## Self-hosted (local) Letta Server
+<AccordionGroup>
+ <Accordion title="Where is my agent data stored?">
+When you run Letta with Docker, the Letta server uses a postgres database to store all your agents' data.
+The postgres instance is bundled into the image, so to have persistent data (across restarts) you need to mount a volume to the container.
+
+Our recommend `docker run` script includes `-v ~/.letta/.persist/pgdata:/var/lib/postgresql/data` as a flag.
+This mounts your local directory `~/.letta/.persist/pgdata` to the container's `/var/lib/postgresql/data` directory (so all your agent data is stored at `~/.letta/.persist/pgdata`).
+If you would like to use a different directory, you can use `-v <path_to_your_directory>:/var/lib/postgresql/data` instead.
+ </Accordion>
+ <Accordion title="How can I back up my postgres data?">
+Postgres has a number of [recommended ways](https://www.postgresql.org/docs/current/backup.html) to backup your data.
+
+We recommend directly `exec`ing into your Docker container and running [`pg_dump`](https://www.postgresql.org/docs/current/app-pgdump.html) from inside the container.
+
+Alternatively, you can run `docker run` with an extra flag to expose the postgres port with `-p 5432:5432` and then run `pg_dump` from your local machine.
+ </Accordion>
+ <Accordion title="Do I need to install Docker to use Letta?">
+Yes, Docker is required to run a self-hosted Letta server. Docker provides the easiest way to run Letta with PostgreSQL, which is necessary for data persistence and migrations. To install Docker, see [Docker's installation guide](https://docs.docker.com/get-docker/).
+ </Accordion>
+</AccordionGroup>
diff --git a/fern/pages/models/anthropic.mdx b/fern/pages/models/anthropic.mdx
new file mode 100644
index 000000000..a8c5c49b1
--- /dev/null
+++ b/fern/pages/models/anthropic.mdx
@@ -0,0 +1,41 @@
+---
+title: Anthropic
+slug: guides/server/providers/anthropic
+---
+<Tip>To enable Anthropic models with Letta, set `ANTHROPIC_API_KEY` in your environment variables. </Tip>
+
+You can use Letta with Anthropic if you have an Anthropic account and API key.
+Currently, only there are no supported **embedding** models for Anthropic (only LLM models).
+You will need to use a seperate provider (e.g. OpenAI) or the Letta embeddings endpoint (`letta-free`) for embeddings.
+
+## Enabling Anthropic models with Docker
+
+To enable Anthropic models when running the Letta server with Docker, set your `ANTHROPIC_API_KEY` as an environment variable:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e ANTHROPIC_API_KEY="your_anthropic_api_key" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
+
+## Configuring Anthropic models
+
+When creating agents, you must specify the LLM and embedding models to use. You can additionally specify a context window limit (which must be less than or equal to the maximum size). Note that Anthropic does not have embedding models, so you will need to use another provider.
+
+```python
+from letta_client import Letta
+
+client = Letta(base_url="http://localhost:8283")
+
+agent = client.agents.create(
+ model="anthropic/claude-3-5-sonnet-20241022",
+ embedding="openai/text-embedding-3-small",
+ # optional configuration
+ context_window_limit=30000
+)
+```
+Anthropic models have very large context windows, which will be very expensive and high latency. We recommend setting a lower `context_window_limit` when using Anthropic models.
diff --git a/fern/pages/models/aws_bedrock.mdx b/fern/pages/models/aws_bedrock.mdx
new file mode 100644
index 000000000..f865e3980
--- /dev/null
+++ b/fern/pages/models/aws_bedrock.mdx
@@ -0,0 +1,30 @@
+---
+title: AWS Bedrock
+slug: guides/server/providers/aws-bedrock
+---
+We support Anthropic models provided via AWS Bedrock.
+
+<Warning>
+To use a model with AWS Bedrock, you must ensure it is enabled in the your AWS Model Catalog. Letta will list all available Anthropic models on Bedrock, even if you do not have access to them via AWS.
+</Warning>
+
+## Enabling AWS Bedrock with Docker
+
+To enable AWS Bedrock models when running the Letta server with Docker, set your AWS credentials as environment variables:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e AWS_ACCESS_KEY_ID="your_aws_access_key_id" \
+ -e AWS_SECRET_ACCESS_KEY="your_aws_secret_access_key" \
+ -e AWS_DEFAULT_REGION="your_aws_default_region" \
+ letta/letta:latest
+```
+
+Optionally, you can specify the API version (default is bedrock-2023-05-31):
+```bash
+-e BEDROCK_ANTHROPIC_VERSION="bedrock-2023-05-31"
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
diff --git a/fern/pages/models/azure.mdx b/fern/pages/models/azure.mdx
new file mode 100644
index 000000000..7960df411
--- /dev/null
+++ b/fern/pages/models/azure.mdx
@@ -0,0 +1,54 @@
+---
+title: Azure OpenAI
+slug: guides/server/providers/azure
+---
+
+
+<Tip> To use Letta with Azure OpenAI, set the environment variables `AZURE_API_KEY` and `AZURE_BASE_URL`. You can also optionally specify `AZURE_API_VERSION` (default is `2024-09-01-preview`) </Tip>
+
+You can use Letta with OpenAI if you have an OpenAI account and API key. Once you have set your `AZURE_API_KEY` and `AZURE_BASE_URL` specified in your environment variables, you can select what model and configure the context window size
+
+Currently, Letta supports the following OpenAI models:
+- `gpt-4` (recommended for advanced reasoning)
+- `gpt-4o-mini` (recommended for low latency and cost)
+- `gpt-4o`
+- `gpt-4-turbo` (*not* recommended, should use `gpt-4o-mini` instead)
+- `gpt-3.5-turbo` (*not* recommended, should use `gpt-4o-mini` instead)
+
+
+## Enabling Azure OpenAI with Docker
+
+To enable Azure OpenAI models when running the Letta server with Docker, set your `AZURE_API_KEY` and `AZURE_BASE_URL` as environment variables:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e AZURE_API_KEY="your_azure_api_key" \
+ -e AZURE_BASE_URL="your_azure_base_url" \
+ -e AZURE_API_VERSION="your_azure_api_version" \
+ letta/letta:latest
+```
+
+Optionally, you can specify the API version (default is 2024-09-01-preview):
+```bash
+-e AZURE_API_VERSION="2024-09-01-preview"
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
+
+## Specifying agent models
+When creating agents, you must specify the LLM and embedding models to use via a *handle*. You can additionally specify a context window limit (which must be less than or equal to the maximum size).
+
+```python
+from letta_client import Letta
+
+client = Letta(base_url="http://localhost:8283")
+
+azure_agent = client.agents.create(
+ model="azure/gpt-4o-mini",
+ embedding="azure/text-embedding-3-small",
+ # optional configuration
+ context_window_limit=16000,
+)
+```
diff --git a/fern/pages/models/deepseek.mdx b/fern/pages/models/deepseek.mdx
new file mode 100644
index 000000000..3bec1d50a
--- /dev/null
+++ b/fern/pages/models/deepseek.mdx
@@ -0,0 +1,42 @@
+---
+title: DeepSeek
+slug: guides/server/providers/deepseek
+---
+
+
+<Tip>To use Letta with the DeepSeek API, set the environment variable `DEEPSEEK_API_KEY=...`</Tip>
+
+You can use Letta with [DeepSeek](https://api-docs.deepseek.com/) if you have a DeepSeek account and API key. Once you have set your `DEEPSEEK_API_KEY` in your environment variables, you can select what model and configure the context window size.
+<Warning>
+Please note that R1 doesn't natively support function calling in DeepSeek API and V3 function calling is unstable, which may result in unstable tool calling inside of Letta agents.
+</Warning>
+<Warning>
+The DeepSeek API for R1 is often down. Please make sure you can connect to DeepSeek API directly by running:
+```bash
+curl https://api.deepseek.com/v1/chat/completions \
+ -H "Content-Type: application/json" \
+ -H "Authorization: Bearer $DEEPSEEK_API_KEY" \
+ -d '{
+ "model": "deepseek-reasoner",
+ "messages": [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Hello!"}
+ ],
+ "stream": false
+ }'
+```
+</Warning>
+
+## Enabling DeepSeek with Docker
+
+To enable DeepSeek models when running the Letta server with Docker, set your `DEEPSEEK_API_KEY` as an environment variable:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e DEEPSEEK_API_KEY="your_deepseek_api_key" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
diff --git a/fern/pages/models/google.mdx b/fern/pages/models/google.mdx
new file mode 100644
index 000000000..556d6f16e
--- /dev/null
+++ b/fern/pages/models/google.mdx
@@ -0,0 +1,23 @@
+---
+title: Google AI (Gemini)
+slug: guides/server/providers/google
+---
+
+
+<Tip>To enable Google AI models with Letta, set `GEMINI_API_KEY` in your environment variables. </Tip>
+
+You can use Letta with Google AI if you have a Google API account and API key. Once you have set your `GEMINI_API_KEY` in your environment variables, you can select what model and configure the context window size.
+
+## Enabling Google AI with Docker
+
+To enable Google Gemini models when running the Letta server with Docker, set your `GEMINI_API_KEY` as an environment variable:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e GEMINI_API_KEY="your_gemini_api_key" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
diff --git a/fern/pages/models/google_vertex.mdx b/fern/pages/models/google_vertex.mdx
new file mode 100644
index 000000000..048730bf9
--- /dev/null
+++ b/fern/pages/models/google_vertex.mdx
@@ -0,0 +1,30 @@
+---
+title: Google Vertex AI
+slug: guides/server/providers/google_vertex
+---
+
+
+<Tip>To enable Vertex AI models with Letta, set `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` in your environment variables. </Tip>
+
+You can use Letta with Vertex AI by configuring your GCP project ID and region.
+
+## Enabling Google Vertex AI with Docker
+
+First, make sure you are authenticated with Google Vertex AI:
+
+```bash
+gcloud auth application-default login
+```
+
+To enable Google Vertex AI models when running the Letta server with Docker, set your `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` as environment variables:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e GOOGLE_CLOUD_PROJECT="your-project-id" \
+ -e GOOGLE_CLOUD_LOCATION="us-central1" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
diff --git a/fern/pages/models/groq.mdx b/fern/pages/models/groq.mdx
new file mode 100644
index 000000000..31c719dfa
--- /dev/null
+++ b/fern/pages/models/groq.mdx
@@ -0,0 +1,23 @@
+---
+title: Groq
+slug: guides/server/providers/groq
+---
+
+
+<Tip>To use Letta with Groq, set the environment variable `GROQ_API_KEY=...`</Tip>
+
+You can use Letta with Groq if you have a Groq account and API key. Once you have set your `GROQ_API_KEY` in your environment variables, you can select what model and configure the context window size.
+
+## Enabling Groq with Docker
+
+To enable Groq models when running the Letta server with Docker, set your `GROQ_API_KEY` as an environment variable:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e GROQ_API_KEY="your_groq_api_key" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
diff --git a/fern/pages/models/lmstudio.mdx b/fern/pages/models/lmstudio.mdx
new file mode 100644
index 000000000..a36d014b5
--- /dev/null
+++ b/fern/pages/models/lmstudio.mdx
@@ -0,0 +1,61 @@
+---
+title: LM Studio
+slug: guides/server/providers/lmstudio
+---
+
+<Warning>
+LM Studio support is currently experimental. If things aren't working as expected, please reach out to us on [Discord](https://discord.gg/letta)!
+</Warning>
+
+<Tip>
+Models marked as ["native tool use"](https://lmstudio.ai/docs/advanced/tool-use#supported-models) on LM Studio are more likely to work well with Letta.
+</Tip>
+
+## Setup LM Studio
+
+1. Download + install [LM Studio](https://lmstudio.ai) and the model you want to test with
+2. Make sure to start the [LM Studio server](https://lmstudio.ai/docs/api/server)
+
+## Enabling LM Studio with Docker
+
+To enable LM Studio models when running the Letta server with Docker, set the `LMSTUDIO_BASE_URL` environment variable.
+
+**macOS/Windows:**
+Since LM Studio is running on the host network, you will need to use `host.docker.internal` to connect to the LM Studio server instead of `localhost`.
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e LMSTUDIO_BASE_URL="http://host.docker.internal:1234" \
+ letta/letta:latest
+```
+
+**Linux:**
+Use `--network host` and `localhost`:
+```bash
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ --network host \
+ -e LMSTUDIO_BASE_URL="http://localhost:1234" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
+
+## Model support
+
+<Note>
+FYI Models labelled as MLX are only compatible on Apple Silicon Macs
+</Note>
+The following models have been tested with Letta as of 7-11-2025 on LM Studio `0.3.18`.
+
+- `qwen3-30b-a3b`
+- `qwen3-14b-mlx`
+- `qwen3-8b-mlx`
+- `qwen2.5-32b-instruct`
+- `qwen2.5-14b-instruct-1m`
+- `qwen2.5-7b-instruct`
+- `meta-llama-3.1-8b-instruct`
+
+Some models recommended on [LM Studio](https://lmstudio.ai/docs/advanced/tool-use#supported-models) such as `mlx-community/ministral-8b-instruct-2410` and `bartowski/ministral-8b-instruct-2410` may not work well with Letta due to default prompt templates being incompatible. Adjusting templates can enable compatibility but will impact model performance.
diff --git a/fern/pages/models/ollama.mdx b/fern/pages/models/ollama.mdx
new file mode 100644
index 000000000..f409a818c
--- /dev/null
+++ b/fern/pages/models/ollama.mdx
@@ -0,0 +1,82 @@
+---
+title: Ollama
+slug: guides/server/providers/ollama
+---
+
+<Warning>
+Make sure to use **tags** when downloading Ollama models!
+
+For example, don't do **`ollama pull dolphin2.2-mistral`**, instead do **`ollama pull dolphin2.2-mistral:7b-q6_K`** (add the `:7b-q6_K` tag).
+
+If you don't specify a tag, Ollama may default to using a highly compressed model variant (e.g. Q4).
+We highly recommend **NOT** using a compression level below Q5 when using GGUF (stick to Q6 or Q8 if possible).
+In our testing, certain models start to become extremely unstable (when used with Letta/MemGPT) below Q6.
+</Warning>
+
+## Setup Ollama
+
+1. Download + install [Ollama](https://github.com/ollama/ollama) and the model you want to test with
+2. Download a model to test with by running `ollama pull <MODEL_NAME>` in the terminal (check the [Ollama model library](https://ollama.ai/library) for available models)
+
+For example, if we want to use Dolphin 2.2.1 Mistral, we can download it by running:
+
+```sh
+# Let's use the q6_K variant
+ollama pull dolphin2.2-mistral:7b-q6_K
+```
+
+```sh
+pulling manifest
+pulling d8a5ee4aba09... 100% |█████████████████████████████████████████████████████████████████████████| (4.1/4.1 GB, 20 MB/s)
+pulling a47b02e00552... 100% |██████████████████████████████████████████████████████████████████████████████| (106/106 B, 77 B/s)
+pulling 9640c2212a51... 100% |████████████████████████████████████████████████████████████████████████████████| (41/41 B, 22 B/s)
+pulling de6bcd73f9b4... 100% |████████████████████████████████████████████████████████████████████████████████| (58/58 B, 28 B/s)
+pulling 95c3d8d4429f... 100% |█████████████████████████████████████████████████████████████████████████████| (455/455 B, 330 B/s)
+verifying sha256 digest
+writing manifest
+removing any unused layers
+success
+```
+
+## Enabling Ollama with Docker
+
+To enable Ollama models when running the Letta server with Docker, set the `OLLAMA_BASE_URL` environment variable.
+
+**macOS/Windows:**
+Since Ollama is running on the host network, you will need to use `host.docker.internal` to connect to the Ollama server instead of `localhost`.
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e OLLAMA_BASE_URL="http://host.docker.internal:11434" \
+ letta/letta:latest
+```
+
+**Linux:**
+Use `--network host` and `localhost`:
+```bash
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ --network host \
+ -e OLLAMA_BASE_URL="http://localhost:11434" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
+
+## Specifying agent models
+When creating agents, you must specify the LLM and embedding models to use via a *handle*. You can additionally specify a context window limit (which must be less than or equal to the maximum size).
+
+```python
+from letta_client import Letta
+
+client = Letta(base_url="http://localhost:8283")
+
+ollama_agent = client.agents.create(
+ model="ollama/thewindmom/hermes-3-llama-3.1-8b:latest",
+ embedding="ollama/mxbai-embed-large:latest",
+ # optional configuration
+ context_window_limit=16000,
+)
+```
diff --git a/fern/pages/models/openai.mdx b/fern/pages/models/openai.mdx
new file mode 100644
index 000000000..2726b2968
--- /dev/null
+++ b/fern/pages/models/openai.mdx
@@ -0,0 +1,46 @@
+---
+title: OpenAI
+slug: guides/server/providers/openai
+---
+
+<Tip>To enable OpenAI models with Letta, set `OPENAI_API_KEY` in your environment variables. </Tip>
+
+You can use Letta with OpenAI if you have an OpenAI account and API key. Once you have set your `OPENAI_API_KEY` in your environment variables, you can select what model and configure the context window size.
+
+Currently, Letta supports the following OpenAI models:
+- `gpt-4` (recommended for advanced reasoning)
+- `gpt-4o-mini` (recommended for low latency and cost)
+- `gpt-4o`
+- `gpt-4-turbo` (*not* recommended, should use `gpt-4o-mini` instead)
+- `gpt-3.5-turbo` (*not* recommended, should use `gpt-4o-mini` instead)
+
+
+## Enabling OpenAI models with Docker
+
+To enable OpenAI models when running the Letta server with Docker, set your `OPENAI_API_KEY` as an environment variable:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e OPENAI_API_KEY="your_openai_api_key" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
+
+## Configuring OpenAI models in the Python SDK
+When creating agents, you must specify the LLM and embedding models to use. You can additionally specify a context window limit (which must be less than or equal to the maximum size).
+
+```python
+from letta_client import Letta
+
+client = Letta(base_url="http://localhost:8283")
+
+openai_agent = client.agents.create(
+ model="openai/gpt-4o-mini",
+ embedding="openai/text-embedding-3-small",
+ # optional configuration
+ context_window_limit=16000
+)
+```
diff --git a/fern/pages/models/openai_proxy.mdx b/fern/pages/models/openai_proxy.mdx
new file mode 100644
index 000000000..5dc70a793
--- /dev/null
+++ b/fern/pages/models/openai_proxy.mdx
@@ -0,0 +1,40 @@
+---
+title: OpenAI-compatible endpoint
+slug: guides/server/providers/openai-proxy
+---
+
+<Warning>
+OpenAI proxy endpoints are not officially supported and you are likely to encounter errors.
+We strongly recommend using providers directly instead of via proxy endpoints (for example, using the Anthropic API directly instead of Claude through OpenRouter).
+For questions and support you can chat with the dev team and community on our [Discord server](https://discord.gg/letta).
+</Warning>
+
+<Note>
+To use OpenAI-compatible (`/v1/chat/completions`) endpoints with Letta, those endpoints must support function/tool calling.
+</Note>
+
+You can configure Letta to use OpenAI-compatible `ChatCompletions` endpoints by setting `OPENAI_API_BASE` in your environment variables (in addition to setting `OPENAI_API_KEY`).
+
+## OpenRouter example
+
+Create an account on [OpenRouter](https://openrouter.ai), then [create an API key](https://openrouter.ai/settings/keys).
+
+Once you have your API key, set both `OPENAI_API_KEY` and `OPENAI_API_BASE` in your environment variables.
+
+## Using with Docker
+Set the environment variables when you use `docker run`:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e OPENAI_API_BASE="https://openrouter.ai/api/v1" \
+ -e OPENAI_API_KEY="your_openai_api_key" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
+
+Once the Letta server is running, you can select OpenRouter models from the ADE dropdown or via the Python SDK.
+
+For information on how to configure agents to use OpenRouter or other OpenAI-compatible endpoints providers, refer to [our guide on using OpenAI](/models/openai).
diff --git a/fern/pages/models/together.mdx b/fern/pages/models/together.mdx
new file mode 100644
index 000000000..9465a2462
--- /dev/null
+++ b/fern/pages/models/together.mdx
@@ -0,0 +1,23 @@
+---
+title: Together
+slug: guides/server/providers/together
+---
+
+
+<Tip>To use Letta with Together.AI, set the environment variable `TOGETHER_API_KEY=...`</Tip>
+
+You can use Letta with Together.AI if you have an account and API key. Once you have set your `TOGETHER_API_KEY` in your environment variables, you can select what model and configure the context window size.
+
+## Enabling Together.AI with Docker
+
+To enable Together.AI models when running the Letta server with Docker, set your `TOGETHER_API_KEY` as an environment variable:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e TOGETHER_API_KEY="your_together_api_key" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
diff --git a/fern/pages/models/vllm.mdx b/fern/pages/models/vllm.mdx
new file mode 100644
index 000000000..450dd8e63
--- /dev/null
+++ b/fern/pages/models/vllm.mdx
@@ -0,0 +1,47 @@
+---
+title: vLLM
+slug: guides/server/providers/vllm
+---
+
+
+<Tip>To use Letta with vLLM, set the environment variable `VLLM_API_BASE` to point to your vLLM ChatCompletions server.</Tip>
+
+## Setting up vLLM
+1. Download + install [vLLM](https://docs.vllm.ai/en/latest/getting_started/installation.html)
+2. Launch a vLLM **OpenAI-compatible** API server using [the official vLLM documentation](https://docs.vllm.ai/en/latest/getting_started/quickstart.html)
+
+For example, if we want to use the model `dolphin-2.2.1-mistral-7b` from [HuggingFace](https://huggingface.co/ehartford/dolphin-2.2.1-mistral-7b), we would run:
+
+```sh
+python -m vllm.entrypoints.openai.api_server \
+--model ehartford/dolphin-2.2.1-mistral-7b
+```
+
+vLLM will automatically download the model (if it's not already downloaded) and store it in your [HuggingFace cache directory](https://huggingface.co/docs/datasets/cache).
+
+## Enabling vLLM with Docker
+
+To enable vLLM models when running the Letta server with Docker, set the `VLLM_API_BASE` environment variable.
+
+**macOS/Windows:**
+Since vLLM is running on the host network, you will need to use `host.docker.internal` to connect to the vLLM server instead of `localhost`.
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e VLLM_API_BASE="http://host.docker.internal:8000" \
+ letta/letta:latest
+```
+
+**Linux:**
+Use `--network host` and `localhost`:
+```bash
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ --network host \
+ -e VLLM_API_BASE="http://localhost:8000" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
diff --git a/fern/pages/models/xai.mdx b/fern/pages/models/xai.mdx
new file mode 100644
index 000000000..ac04b963f
--- /dev/null
+++ b/fern/pages/models/xai.mdx
@@ -0,0 +1,42 @@
+---
+title: xAI (Grok)
+slug: guides/server/providers/xai
+---
+<Tip>To enable xAI (Grok) models with Letta, set `XAI_API_KEY` in your environment variables. </Tip>
+
+## Enabling xAI (Grok) models
+To enable the xAI provider, set your key as an environment variable:
+```bash
+export XAI_API_KEY="..."
+```
+## Enabling xAI with Docker
+
+To enable xAI models when running the Letta server with Docker, set your `XAI_API_KEY` as an environment variable:
+```bash
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e XAI_API_KEY="your_xai_api_key" \
+ letta/letta:latest
+```
+
+See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.
+
+## Configuring xAI (Grok) models
+
+When creating agents, you must specify the LLM and embedding models to use. You can additionally specify a context window limit (which must be less than or equal to the maximum size). Note that xAI does not have embedding models, so you will need to use another provider.
+
+```python
+from letta_client import Letta
+
+client = Letta(base_url="http://localhost:8283")
+
+agent = client.agents.create(
+ model="xai/grok-2-1212",
+ embedding="openai/text-embedding-3-small",
+ # optional configuration
+ context_window_limit=30000
+)
+```
+xAI (Grok) models have very large context windows, which will be very expensive and high latency. We recommend setting a lower `context_window_limit` when using xAI (Grok) models.
diff --git a/fern/pages/selfhosting/overview.mdx b/fern/pages/selfhosting/overview.mdx
new file mode 100644
index 000000000..d489fad82
--- /dev/null
+++ b/fern/pages/selfhosting/overview.mdx
@@ -0,0 +1,134 @@
+---
+title: Self-hosting Letta
+subtitle: Learn how to run your own Letta server
+slug: guides/selfhosting
+---
+
+<Note>
+The recommended way to use Letta locally is with Docker.
+To install Docker, see [Docker's installation guide](https://docs.docker.com/get-docker/).
+For issues with installing Docker, see [Docker's troubleshooting guide](https://docs.docker.com/desktop/troubleshoot-and-support/troubleshoot/).
+</Note>
+
+## Running the Letta Server
+
+To run the server with Docker, run the command:
+```sh
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e OPENAI_API_KEY="your_openai_api_key" \
+ letta/letta:latest
+```
+This will run the Letta server with the OpenAI provider enabled, and store all data in the folder `~/.letta/.persist/pgdata`.
+
+If you have many different LLM API keys, you can also set up a `.env` file instead and pass that to `docker run`:
+```sh
+# using a .env file instead of passing environment variables
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ --env-file .env \
+ letta/letta:latest
+```
+
+Once the Letta server is running, you can access it via port `8283` (e.g. sending REST API requests to `http://localhost:8283/v1`). You can also connect your server to the [Letta ADE](/guides/ade) to access and manage your agents in a web interface.
+
+## Enabling model providers
+The Letta server can be connected to various LLM API backends ([OpenAI](https://docs.letta.com/models/openai), [Anthropic](https://docs.letta.com/models/anthropic), [vLLM](https://docs.letta.com/models/vllm), [Ollama](https://docs.letta.com/models/ollama), etc.). To enable access to these LLM API providers, set the appropriate environment variables when you use `docker run`:
+```sh
+# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ -e OPENAI_API_KEY="your_openai_api_key" \
+ -e ANTHROPIC_API_KEY="your_anthropic_api_key" \
+ -e OLLAMA_BASE_URL="http://host.docker.internal:11434" \
+ letta/letta:latest
+```
+
+<Note>
+**Linux users:** Use `--network host` and `localhost` instead of `host.docker.internal`:
+```sh
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ --network host \
+ -e OPENAI_API_KEY="your_openai_api_key" \
+ -e ANTHROPIC_API_KEY="your_anthropic_api_key" \
+ -e OLLAMA_BASE_URL="http://localhost:11434" \
+ letta/letta:latest
+```
+</Note>
+
+The example above will make all compatible models running on OpenAI, Anthropic, and Ollama available to your Letta server.
+
+
+## Optional: Telemetry with ClickHouse
+
+Letta supports optional telemetry using ClickHouse. Telemetry provides observability features like traces, LLM request logging, and performance metrics. See the [telemetry guide](/guides/server/otel) for setup instructions.
+
+
+## Password protection
+
+<Warning>
+When running a self-hosted Letta server in a production environment (i.e. with untrusted users), make sure to enable both password protection (to prevent unauthorized access to your server over the network) and tool sandboxing (to prevent malicious tools from executing in a privledged environment).
+</Warning>
+
+To password protect your server, include `SECURE=true` and `LETTA_SERVER_PASSWORD=yourpassword` in your `docker run` command:
+```sh
+# If LETTA_SERVER_PASSWORD isn't set, the server will autogenerate a password
+docker run \
+ -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
+ -p 8283:8283 \
+ --env-file .env \
+ -e SECURE=true \
+ -e LETTA_SERVER_PASSWORD=yourpassword \
+ letta/letta:latest
+```
+
+With password protection enabled, you will have to provide your password in the bearer token header in your API requests:
+<CodeGroup>
+```typescript TypeScript maxLines=50
+// install letta-client with `npm install @letta-ai/letta-client`
+import { LettaClient } from '@letta-ai/letta-client'
+
+// create the client with the token set to your password
+const client = new LettaClient({
+ baseUrl: "http://localhost:8283",
+ token: "yourpassword"
+});
+```
+```python title="python" maxLines=50
+# install letta_client with `pip install letta-client`
+from letta_client import Letta
+
+# create the client with the token set to your password
+client = Letta(
+ base_url="http://localhost:8283",
+ token="yourpassword"
+)
+```
+```curl curl
+curl --request POST \
+ --url http://localhost:8283/v1/agents/$AGENT_ID/messages \
+ --header 'Content-Type: application/json' \
+ --header 'Authorization: Bearer yourpassword' \
+ --data '{
+ "messages": [
+ {
+ "role": "user",
+ "text": "hows it going????"
+ }
+ ]
+}'
+```
+</CodeGroup>
+
+
+## Tool sandboxing
+
+To enable tool sandboxing, set the `E2B_API_KEY` and `E2B_SANDBOX_TEMPLATE_ID` environment variables (via [E2B](https://e2b.dev/)) when you use `docker run`.
+When sandboxing is enabled, all custom tools (created by users from source code) will be executed in a sandboxed environment.
+
+This does not include MCP tools, which are executed outside of the Letta server (on the MCP server itself), or built-in tools (like `memory_insert`), whose code cannot be modified after server startup.