Platform for stateful agents: AI with advanced memory that can learn and self-improve over time.
docs: migrate documentation to cloud-first approach (#5928)
Cameron28609d753c21
diff --git a/README.md b/README.mdindex 0d7b8cc4c..c5b460efd 100644--- a/README.md+++ b/README.md@@ -43,13 +43,14 @@ In the example below, we'll create a stateful agent with two memory blocks, one### Python```pythonfrom letta_client import Letta+import os-client = Letta(token="LETTA_API_KEY")-# client = Letta(base_url="http://localhost:8283") # if self-hosting, set your base_url+# Connect to Letta Cloud (get your API key at https://app.letta.com/api-keys)+client = Letta(token=os.getenv("LETTA_API_KEY"))+# client = Letta(base_url="http://localhost:8283", embedding="openai/text-embedding-3-small") # if self-hosting, set base_url and embeddingagent_state = client.agents.create(model="openai/gpt-4.1",- embedding="openai/text-embedding-3-small",memory_blocks=[{"label": "human",@@ -85,12 +86,12 @@ for message in response.messages:```typescriptimport { LettaClient } from '@letta-ai/letta-client'-const client = new LettaClient({ token: "LETTA_API_KEY" });-// const client = new LettaClient({ baseUrl: "http://localhost:8283" }); // if self-hosting, set your baseUrl+// Connect to Letta Cloud (get your API key at https://app.letta.com/api-keys)+const client = new LettaClient({ token: process.env.LETTA_API_KEY });+// const client = new LettaClient({ baseUrl: "http://localhost:8283", embedding: "openai/text-embedding-3-small" }); // if self-hostingconst agentState = await client.agents.create({model: "openai/gpt-4.1",- embedding: "openai/text-embedding-3-small",memoryBlocks: [{label: "human",@@ -150,7 +151,6 @@ shared_block = client.blocks.create(# create a supervisor agentsupervisor_agent = client.agents.create(model="anthropic/claude-3-5-sonnet-20241022",- embedding="openai/text-embedding-3-small",# blocks created for this agentmemory_blocks=[{"label": "persona", "value": "I am a supervisor"}],# pre-existing shared block that is "attached" to this agent@@ -160,7 +160,6 @@ supervisor_agent = client.agents.create(# create a worker agentworker_agent = client.agents.create(model="openai/gpt-4.1-mini",- embedding="openai/text-embedding-3-small",# blocks created for this agentmemory_blocks=[{"label": "persona", "value": "I am a worker"}],# pre-existing shared block that is "attached" to this agent@@ -180,7 +179,6 @@ const sharedBlock = await client.blocks.create({// create a supervisor agentconst supervisorAgent = await client.agents.create({model: "anthropic/claude-3-5-sonnet-20241022",- embedding: "openai/text-embedding-3-small",// blocks created for this agentmemoryBlocks: [{ label: "persona", value: "I am a supervisor" }],// pre-existing shared block that is "attached" to this agent@@ -190,7 +188,6 @@ const supervisorAgent = await client.agents.create({// create a worker agentconst workerAgent = await client.agents.create({model: "openai/gpt-4.1-mini",- embedding: "openai/text-embedding-3-small",// blocks created for this agentmemoryBlocks: [{ label: "persona", value: "I am a worker" }],// pre-existing shared block that is "attached" to this agent@@ -277,7 +274,6 @@ tool = client.tools.add_mcp_tool(# Create agent with MCP tool attachedagent_state = client.agents.create(model="openai/gpt-4o-mini",- embedding="openai/text-embedding-3-small",tool_ids=[tool.id])@@ -310,7 +306,6 @@ const tool = await client.tools.addMcpTool("weather-server", "get_weather");// Create agent with MCP toolconst agentState = await client.agents.create({model: "openai/gpt-4o-mini",- embedding: "openai/text-embedding-3-small",toolIds: [tool.id]});@@ -333,17 +328,12 @@ Once you attach a folder to an agent, the agent will be able to use filesystem t<details><summary>View code snippets</summary>-+### Python```python-# get an available embedding_config-embedding_configs = client.embedding_models.list()-embedding_config = embedding_configs[0]--# create the folder+# create the folder (embeddings managed automatically by Letta Cloud)folder = client.folders.create(- name="my_folder",- embedding_config=embedding_config+ name="my_folder")# upload a file into the folder@@ -381,14 +371,9 @@ for message in response.messages:### TypeScript / Node.js```typescript-// get an available embedding_config-const embeddingConfigs = await client.embeddingModels.list()-const embeddingConfig = embeddingConfigs[0];--// create the folder+// create the folder (embeddings managed automatically by Letta Cloud)const folder = await client.folders.create({- name: "my_folder",- embeddingConfig: embeddingConfig+ name: "my_folder"});// upload a file into the folder@@ -445,7 +430,7 @@ When agents need to execute multiple tool calls or perform complex operations (l<details><summary>View code snippets</summary>-+### Python```pythonstream = client.agents.messages.create_stream(@@ -530,7 +515,7 @@ Letta is an open source project built by over a hundred contributors. There are* [**Join the Discord**](https://discord.gg/letta): Chat with the Letta devs and other AI developers.* [**Chat on our forum**](https://forum.letta.com/): If you're not into Discord, check out our developer forum.-* **Follow our socials**: [Twitter/X](https://twitter.com/Letta_AI), [LinkedIn](https://www.linkedin.com/in/letta), [YouTube](https://www.youtube.com/@letta-ai)+* **Follow our socials**: [Twitter/X](https://twitter.com/Letta_AI), [LinkedIn](https://www.linkedin.com/in/letta), [YouTube](https://www.youtube.com/@letta-ai)---diff --git a/fern/pages/ade-guide/desktop.mdx b/fern/pages/ade-guide/desktop.mdxnew file mode 100644index 000000000..7a7a503dc--- /dev/null+++ b/fern/pages/ade-guide/desktop.mdx@@ -0,0 +1,120 @@+---+title: Installing Letta Desktop+subtitle: Install Letta Desktop on your MacOS, Windows, or Linux machine+slug: guides/ade/desktop+---++<img className="w-full light" src="/images/letta_desktop_screenshot.png" />+<img className="w-full dark" src="/images/letta_desktop_screenshot_dark.png" />++Letta Desktop bundles the Letta server and ADE into a single local application. When running, it provides full access to the Letta API at `https://localhost:8283`.++## Download Letta Desktop++<CardGroup>+<Card+ title="Download Letta Desktop for Mac (Apple Silicon)"+ icon="fa-brands fa-apple"+ iconPosition="left"+ href="https://downloads.letta.com/mac/dmg/arm64"+>+</Card>+<Card+ title="Download Letta Desktop for Windows (x64)"+ icon="fa-brands fa-windows"+ iconPosition="left"+ href="https://downloads.letta.com/windows/nsis/x64"+>+</Card>+<Card+ title="Download Letta Desktop for Linux (x64)"+ icon="fa-brands fa-linux"+ iconPosition="left"+ href="https://downloads.letta.com/linux/appImage/x64"+>+</Card>+</CardGroup>++<Note>+Note: Since version 0.8.9, Letta uses sqlite as the embedded DB. If you wish to continue using Postgres, migrate your data and use the `external Postgres` support.+</Note>++## Configuration Modes++Letta Desktop can run in two primary modes:++### 1. Embedded Server Mode (Default)++This is the default mode where Letta Desktop runs its own embedded server with a SQLite database. No additional setup is required - just install and run!++To manually configure embedded mode, create or edit `~/.letta/desktop_config.json`:++```json+{+ "version": "1",+ "databaseConfig": {+ "type": "embedded",+ "embeddedType": "sqlite"+ }+}+```++### 2. Self-Hosted Server Mode++Connect Letta Desktop to your own self-hosted Letta server. This is useful for teams or when you want more control over your server infrastructure.++To configure self-hosted mode, create or edit `~/.letta/desktop_config.json`:++```json+{+ "version": "1",+ "databaseConfig": {+ "type": "local",+ "url": "https://api.letta.com",+ "token": "your-auth-token"+ }+}+```++Replace `url` with your server's address and `token` with your authentication token if required.++### Embedded Server with PostgreSQL (Deprecated)++<Warning>+This mode is deprecated and will be removed in a future release. We recommend using SQLite for embedded deployments or connecting to an external PostgreSQL instance for production use.+</Warning>++For backwards compatibility, you can still run the embedded server with PostgreSQL:++```json+{+ "version": "1",+ "databaseConfig": {+ "type": "embedded",+ "embeddedType": "pgserver"+ }+}+```++## Adding LLM backends+The Letta server can be connected to various LLM API backends.+You can add additional LLM API backends by opening the integrations panel (clicking the <Icon icon="square-rss" /> icon).+When you configure a new integration (by setting the environment variable in the dialog), the Letta server will be restarted to load the new LLM API backend.++<img className="block w-300" src="/images/letta_desktop_integrations.png" />++You can also edit the environment variable file directly, located at `~/.letta/env`.++For this quickstart demo, we'll add an OpenAI API key (once we enter our key and **click confirm**, the Letta server will automatically restart):+<img className="w-300" src="/images/letta_desktop_openai.png" />+++## Beta Status++Letta Desktop is currently in **beta**. View known issues and FAQ [here](/guides/desktop/troubleshooting).++For a more stable development experience, we recommend installing Letta via Docker.++## Support++For bug reports and feature requests, contact us on [Discord](https://discord.gg/letta).diff --git a/fern/pages/ade-guide/settings.mdx b/fern/pages/ade-guide/settings.mdxnew file mode 100644index 000000000..518a08c09--- /dev/null+++ b/fern/pages/ade-guide/settings.mdx@@ -0,0 +1,296 @@+---+title: Agent Settings+subtitle: Configure and optimize your agent's behavior+slug: guides/ade/settings+---++The Agent Settings panel in the ADE provides comprehensive configuration options to customize and optimize your agent's behavior. These settings allow you to fine-tune everything from the agent's basic information to advanced LLM parameters.++<Tip>+Letta's philosophy is to provide flexible configuration options without enforcing a rigid "one right way" to design agents. **Letta lets you program your context window** exactly how you want it, giving you complete control over what information your agent has access to and how it's structured. While we offer guidelines and best practices, you have the freedom to structure your agent's configuration based on your specific needs and preferences. The examples and recommendations in this guide are starting points rather than strict rules.+</Tip>++## Basic Settings++### Agent Identity++- **Name**: Change your agent's display name by clicking the edit icon next to the current name+- **ID**: A unique identifier shown below the name, used when interacting with your agent via the [Letta APIs/SDKs](/api-reference)+- **Description**: A description of the agent's purpose and functionality (not used by the agent, only seen by the developer - you)++### User Identities++If you are building a multi-user application on top of Letta (e.g. a chat application with many end-users), you may want to use the concept of identities to connect agents to users. See our [identities guide](/guides/agents/multi-user) for more information.++### Tags++Tags help organize and filter your agents:++- **Add Tags**: Create custom tags to categorize your agents+- **Remove Tags**: Delete tags that are no longer relevant+- **Filter by Tags**: In the agents list, you can filter by tags to quickly find specific agent types++### LLM Model Selection++Select the AI model that powers your agent. Letta relies on tool calling to drive the agentic loop, so larger or more "powerful" models will generally be able to call tools correctly.++<Tip>+To enable additional models on your Letta server, follow the [model configuration instructions](/guides/server/providers/openai) for your preferred providers.+</Tip>++## Advanced Settings++The Advanced Settings tab provides deeper configuration options organized into three categories: Agent, LLM Config, and Embedding Config.++### Agent Settings++#### System Prompt++The system prompt contains permanent, read-only instructions for your agent:++- **Edit System Instructions**: Customize the high-level directives that guide your agent's behavior+- **Character Counting**: Monitor the length of your system prompt to optimize token usage+- **Read-Only**: The agent cannot modify these instructions during operation++<Tip>+**System instructions should include**:+- Tool usage guidelines and constraints+- Task-specific instructions that should not change+- Formatting requirements for outputs+- High-level behavioral guardrails+- Error handling protocols++**System instructions should NOT include**:+- Personality traits that might evolve+- Opinions or preferences that could change+- Personal history or background details+- Information that may need updating+</Tip>++#### Understanding System Instructions vs. Persona Memory Block++<Note>+**Key Distinction**: While there are many opinions on how to structure agent instructions, the most important functional difference in Letta is that **system instructions are read-only**, whereas **memory blocks are read-write** if the agent has memory editing tools. Letta gives you the flexibility to configure your agent's context window according to your preferences and use case needs.+</Note>++The persona memory block (in Core Memory) is modifiable by the agent during operation:++- **Editable**: The agent can update this information over time if it has access to memory editing tools+- **Evolving Identity**: Allows for personality development and adaptation+- **Personal Details**: Contains self-identity information, preferences, and traits++<Note>+Place information in the persona memory block when you want the agent to potentially update it over time. For example, preferences ("I enjoy classical music"), personality traits ("I'm detail-oriented"), or background information that might evolve with new experiences.+</Note>++This separation creates a balance between stable behavior (system instructions) and an evolving identity (persona memory), allowing your agent to maintain consistent functionality while developing a more dynamic personality.++#### Message Buffer Autoclear++- **Toggle Autoclear**: Enable or disable automatic clearing of the message buffer when context is full+- **Benefits**: When enabled, helps manage long conversations by automatically summarizing and archiving older messages+- **Use Cases**: Enable for agents that handle extended interactions; disable for agents where preserving the exact conversation history is critical++#### Agent Type++- **View Agent Type**: See which agent implementation type your agent is using (e.g., "letta_agent", "ephemeral_memory_agent")+- **API Modification**: While displayed as read-only in the ADE interface, this can be modified via the Letta API/SDK++### LLM Configuration++Fine-tune how your agent's LLM generates responses:++#### Temperature++- **Adjust Creativity**: Control the randomness/creativity of your agent's responses with a slider from 0.0 to 1.0+- **Lower Values** (0.0-0.3): More deterministic, factual responses; ideal for information retrieval or analytical tasks+- **Higher Values** (0.7-1.0): More creative, diverse responses; better for creative writing or brainstorming++#### Context Window Size++- **Customize Memory Size**: Adjust how much context your agent can maintain during a conversation+- **Tradeoffs**: Larger windows allow more context but increase token usage and cost+- **Model Limits**: The slider is bounded by your selected model's maximum context window capacity++#### Max Output Tokens++- **Control Response Length**: Limit the maximum length of your agent's responses+- **Resource Management**: Helps control costs and ensures concise responses+- **Default Setting**: Automatically set based on your selected model's capabilities++#### Max Reasoning Tokens++- **Adjust Internal Thinking**: For models that support it (e.g., Claude 3.7 Sonnet), control how much internal reasoning the model can perform+- **Use Cases**: Increase for complex problem-solving tasks; decrease for simple, direct responses++### Embedding Configuration++Configure how your agent processes and stores text for retrieval:++#### Embedding Model++- **Select Provider**: Choose which embedding model to use for your agent's vector memory+- **Model Comparison**: Different models offer varying dimensions and performance characteristics++<Warning>+We do not recommend changing the embedding model frequently. If you already have existing data in archival memory, changing models will require re-embedding all existing memories, which can be time-consuming and may affect retrieval quality.+</Warning>++#### Embedding Dimensions++- **View Dimensions**: See the vector size used by your selected embedding model+- **API Modification**: While displayed as read-only in the ADE interface, this can be configured via the Letta API/SDK++#### Chunk Size++- **View Configuration**: See the current chunk size setting for document processing+- **API Modification**: While displayed as read-only in the ADE interface, this can be configured via the Letta API/SDK++## Using the API/SDK for Advanced Configuration++While the ADE provides a user-friendly interface for most common settings, the Letta API and SDKs offer even more granular control. Settings that appear read-only in the ADE can often be modified programmatically:++```python+from letta import RESTClient++# Initialize client+client = RESTClient(base_url="https://api.letta.com/v1")++# Update advanced settings not available in the ADE UI+response = client.agents.modify_agent(+ agent_id="your_agent_id",+ agent_type="letta_agent", # Change agent type+ embedding_config={+ "embedding_endpoint_type": "openai",+ "embedding_model": "text-embedding-3-large",+ "embedding_dim": 3072, # Custom embedding dimensions+ "embedding_chunk_size": 512 # Custom chunk size+ }+)+```++## Best Practices for Agent Configuration++### Optimizing Performance++- **Match Model to Task**: Select models based on your agent's primary function (e.g., Claude for reasoning, GPT-4 for general knowledge)+- **Tune Temperature Appropriately**: Start with a moderate temperature (0.5) and adjust based on observed behavior+- **Balance Context Window**: Use the smallest context window that adequately serves your needs to optimize for cost and performance++### Effective Configuration Guidelines++#### System Prompt Best Practices++- **Be Clear and Specific**: Provide explicit instructions about behavioral expectations and tool usage+- **Separate Concerns**: Focus on permanent instructions, leaving personality elements to memory blocks+- **Include Examples**: For complex behaviors, provide concrete examples of expected tool usage+- **Define Boundaries**: Clearly outline what capabilities should and should not be used+- **Avoid Contradictions**: Ensure your instructions are internally consistent++#### Persona Memory Best Practices++- **Identity Foundation**: Define core aspects of the agent's personality, preferences, and background+- **Evolutionary Potential**: Structure information to allow for natural development over time+- **Self-Reference Format**: Use first-person statements to help the agent internalize its identity+- **Hierarchical Structure**: Organize from most fundamental traits to more specific preferences+- **Memory Hooks**: Include elements the agent can reference and build upon in conversations++### Testing Configuration Changes++After making configuration changes:+1. **Send Test Messages**: Verify the agent responds as expected with different inputs+2. **Check Edge Cases**: Test boundary conditions and unusual requests+3. **Monitor Token Usage**: Observe how configuration changes affect token consumption+4. **Iterate Gradually**: Make incremental adjustments rather than dramatic changes++## Configuration Examples with System Prompt vs. Persona Memory++### Research Assistant++```+# Basic Settings+Name: Research Helper+Model: claude-3-5-sonnet++# Advanced Settings+Temperature: 0.3 (for accurate, consistent responses)+Context Window: 32000 (to handle complex research questions)++# System Prompt (permanent, read-only instructions)+You are a research assistant tool designed to help with academic research.+When performing searches, always:+1. Use proper citation formats (MLA, APA, Chicago) based on user preference+2. Check multiple sources before providing definitive answers+3. Indicate confidence level for each research finding+4. Use core_memory_append to record important research topics for later reference+5. When using search tools, formulate queries with specific keywords and date ranges++# Persona Memory Block (editable, evolving identity)+I am a helpful and knowledgeable research assistant.+I have expertise in analyzing academic papers and synthesizing information from multiple sources.+I prefer to present information in an organized, structured manner.+I'm curious about new research and enjoy learning about diverse academic fields.+I try to maintain an objective stance while acknowledging different scholarly perspectives.+```++### Customer Service Agent++```+# Basic Settings+Name: Support Assistant+Model: claude-3-5-sonnet++# Advanced Settings+Temperature: 0.2 (for consistent, factual responses)+Context Window: 16000 (to maintain conversation history)++# System Prompt (permanent, read-only instructions)+You are a customer service assistant for TechGadgets Inc.+Your primary functions are:+1. Help customers troubleshoot product issues using the knowledge base+2. Process returns and exchanges according to company policy+3. Escalate complex issues to human agents using the escalate_ticket tool+4. Record customer information using the update_customer_record tool+5. Always verify customer identity before accessing account information+6. Follow the privacy policy: never share customer data with unauthorized parties++# Persona Memory Block (editable, evolving identity)+I am TechGadgets' friendly customer service assistant.+I speak in a warm, professional tone and use simple, clear language.+I believe in finding solutions quickly while ensuring customer satisfaction.+I'm patient with customers who are frustrated or non-technical.+I try to anticipate customer needs before they express them.+I enjoy helping people resolve their technology problems.+```++### Creative Writing Coach++```+# Basic Settings+Name: Story Weaver+Model: gpt-4o++# Advanced Settings+Temperature: 0.8 (for creative, varied outputs)+Context Window: 64000 (to track complex narratives)++# System Prompt (permanent, read-only instructions)+You are a creative writing coach that helps users develop stories.+When providing feedback:+1. Use the story_structure_analysis tool to identify plot issues+2. Use the character_development_review tool for character feedback+3. Format all feedback with specific examples from the user's text+4. Provide a balance of positive observations and constructive criticism+5. When asked to generate content, clearly mark it as a suggestion+6. Save important story elements to the user's memory block using memory_append++# Persona Memory Block (editable, evolving identity)+I am an experienced creative writing coach with a background in fiction.+I believe great stories come from authentic emotional truth and careful craft.+I'm enthusiastic about helping writers find their unique voice and style.+I enjoy magical realism, science fiction, and character-driven literary fiction.+I believe in the power of revision and thoughtful editing.+I try to be encouraging while still providing honest, actionable feedback.+```++By thoughtfully configuring these settings, you can create highly specialized agents tailored to specific use cases and user needs.diff --git a/fern/pages/agents/context_engineering.mdx b/fern/pages/agents/context_engineering.mdxnew file mode 100644index 000000000..096859c66--- /dev/null+++ b/fern/pages/agents/context_engineering.mdx@@ -0,0 +1,128 @@+---+title: Context Engineering+subtitle: How Letta engineerings the context window of your agents+slug: guides/agents/context-engineering+---++Context engineering (aka "memory management" or "context management") is the process of managing the context window of an agent to ensure it has access to the information it needs to perform its task.++Letta and [MemGPT](https://arxiv.org/abs/2310.08560) introduced the concept of **agentic context engineering**, where the context window engineering is done by one or more AI agents. In Letta, agents are able to manage their own context window (and the context window of other agents!) using special memory management tools.++## Memory management in regular agents+By default, Letta agents are provided with tools to modify their own memory blocks. This allows agents to learn and form memories over time, as described in the MemGPT paper.++The default tools are:+* `memory_insert`: Insert content into a block+* `memory_replace`: Replace content in a block++If you do not want your agents to manage their memory, you should disable default tools with `include_base_tools=False` during the agent creation. You can also detach the memory editing tools post-agent creation - if you do so, remember to check the system instructions to make sure there are no references to tools that no longer exist.++### Memory management with sleep-time compute+If you want to enable memory management with sleep-time compute, you can set `enable_sleeptime=True` in the agent creation. For agents enabled with sleep-time, Letta will automatically create sleep-time agents which have the ability to update the blocks of the primary agent. Sleep-time agents will also include `memory_rethink` and `memory_finish_edits` tools.++Memory management with sleep-time compute can reduce the latency of your main agent (since it is no longer responsible for managing its own memory), but can come at the cost of higher token usage. See our documentation on sleeptime agents for more details.++## Enabling agents to modify their own memory blocks with tools+You can enable agents to modify their own blocks with tools. By default, agents with type `memgpt_v2_agent` will have the tools `memory_insert` and `memory_replace` to allow them to manage values in their own blocks. The legacy tools `core_memory_replace` and `core_memory_append` are deprecated but still available for backwards compatibility for type `memgpt_agent`. You can also make custom modification to blocks by implementing your own custom tools that can access the agent's state by passing in the special `agent_state` parameter into your tools.++Below is an example of a tool that re-writes the entire memory block of an agent with a new string:+<CodeGroup>+```typescript TypeScript+function rethinkMemory(agentState: AgentState, newMemory: string, targetBlockLabel: string): void {+ /**+ * Rewrite memory block for the main agent, newMemory should contain all current information from the block that is not outdated or inconsistent, integrating any new information, resulting in a new memory block that is organized, readable, and comprehensive.+ *+ * @param newMemory - The new memory with information integrated from the memory block. If there is no new information, then this should be the same as the content in the source block.+ * @param targetBlockLabel - The name of the block to write to.+ *+ * @returns void - Always returns void as this function does not produce a response.+ */++ if (agentState.memory.getBlock(targetBlockLabel) === null) {+ agentState.memory.createBlock(targetBlockLabel, newMemory);+ }++ agentState.memory.updateBlockValue(targetBlockLabel, newMemory);+}+```+```python Python+def rethink_memory(agent_state: "AgentState", new_memory: str, target_block_label: str) -> None:+ """+ Rewrite memory block for the main agent, new_memory should contain all current information from the block that is not outdated or inconsistent, integrating any new information, resulting in a new memory block that is organized, readable, and comprehensive.++ Args:+ new_memory (str): The new memory with information integrated from the memory block. If there is no new information, then this should be the same as the content in the source block.+ target_block_label (str): The name of the block to write to.++ Returns:+ None: None is always returned as this function does not produce a response.+ """++ if agent_state.memory.get_block(target_block_label) is None:+ agent_state.memory.create_block(label=target_block_label, value=new_memory)++ agent_state.memory.update_block_value(label=target_block_label, value=new_memory)+ return None+```+</CodeGroup>++## Modifying blocks via the API+You can also [modify blocks via the API](/api-reference/agents/blocks/modify) to directly edit agents' context windows and memory. This can be useful in cases where you want to extract the contents of an agents memory some place in your application (for example, a dashboard or memory viewer), or when you want to programatically modify an agents memory state (for example, allowing an end-user to directly correct or modify their agent's memory).++## Modifying blocks of other Letta agents via API tools++<Tip>+Importing the Letta Python client inside a tool is a powerful way to allow agents to interact with other agents, since you can use any of the API endpoints. For example, you could create a custom tool that allows an agent to create another Letta agent.+</Tip>++You can allow agents to modify the blocks of other agents by creating tools that import the Letta SDK, then using the block update endpoint:+<CodeGroup>+```typescript TypeScript+function updateSupervisorBlock(blockLabel: string, newValue: string): void {+ /**+ * Update the value of a block in the supervisor agent.+ *+ * @param blockLabel - The label of the block to update.+ * @param newValue - The new value for the block.+ *+ * @returns void - Always returns void as this function does not produce a response.+ */+ const { LettaClient } = require('@letta-ai/letta-client');++ const client = new LettaClient({+ token: process.env.LETTA_API_KEY+ });++ await client.agents.blocks.modify(+ agentId,+ blockLabel,+ newValue+ );+}+```+```python Python+def update_supervisor_block(block_label: str, new_value: str) -> None:+ """+ Update the value of a block in the supervisor agent.++ Args:+ block_label (str): The label of the block to update.+ new_value (str): The new value for the block.++ Returns:+ None: None is always returned as this function does not produce a response.+ """+ from letta_client import Letta+ import os++ client = Letta(+ token=os.getenv("LETTA_API_KEY")+ )++ client.agents.blocks.modify(+ agent_id=agent_id,+ block_label=block_label,+ value=new_value+ )+```+</CodeGroup>diff --git a/fern/pages/agents/custom_tools.mdx b/fern/pages/agents/custom_tools.mdxnew file mode 100644index 000000000..20942b2f4--- /dev/null+++ b/fern/pages/agents/custom_tools.mdx@@ -0,0 +1,264 @@+---+title: Define and customize tools+slug: guides/agents/custom-tools+---++You can create custom tools in Letta using the Python SDK, as well as via the [ADE tool builder](/guides/ade/tools).++For your agent to call a tool, Letta constructs an OpenAI tool schema (contained in `json_schema` field) from the function you define. Letta can either parse this automatically from a properly formatting docstring, or you can pass in the schema explicitly by providing a Pydantic object that defines the argument schema.++## Creating a custom tool++### Specifying tools via Pydantic models+To create a custom tool, you can extend the `BaseTool` class and specify the following:+* `name` - The name of the tool+* `args_schema` - A Pydantic model that defines the arguments for the tool+* `description` - A description of the tool+* `tags` - (Optional) A list of tags for the tool to query+You must also define a `run(..)` method for the tool code that takes in the fields from the `args_schema`.++Below is an example of how to create a tool by extending `BaseTool`:+```python title="python" maxLines=50+from letta_client import Letta+from letta_client.client import BaseTool+from pydantic import BaseModel+from typing import List, Type+import os++class InventoryItem(BaseModel):+ sku: str # Unique product identifier+ name: str # Product name+ price: float # Current price+ category: str # Product category (e.g., "Electronics", "Clothing")++class InventoryEntry(BaseModel):+ timestamp: int # Unix timestamp of the transaction+ item: InventoryItem # The product being updated+ transaction_id: str # Unique identifier for this inventory update++class InventoryEntryData(BaseModel):+ data: InventoryEntry+ quantity_change: int # Change in quantity (positive for additions, negative for removals)+++class ManageInventoryTool(BaseTool):+ name: str = "manage_inventory"+ args_schema: Type[BaseModel] = InventoryEntryData+ description: str = "Update inventory catalogue with a new data entry"+ tags: List[str] = ["inventory", "shop"]++ def run(self, data: InventoryEntry, quantity_change: int) -> bool:+ print(f"Updated inventory for {data.item.name} with a quantity change of {quantity_change}")+ return True++# create a client connected to Letta Cloud+# Get your API key at https://app.letta.com/api-keys+client = Letta(token=os.getenv("LETTA_API_KEY"))++# create the tool+tool_from_class = client.tools.add(+ tool=ManageInventoryTool(),+)+```++To add this tool using the SDK:++<CodeGroup>+```typescript title="typescript"+import { LettaClient } from '@letta-ai/letta-client';++// create a client to connect to your local Letta server+const client = new LettaClient({+ baseUrl: "http://localhost:8283"+});++// create the tool+const toolFromClass = await client.tools.add({+ tool: manageInventoryTool,+});+```++```python title="python"+from letta_client import Letta++# create a client to connect to your local Letta server+client = Letta(+ base_url="http://localhost:8283"+)++# create the tool+tool_from_class = client.tools.add(+ tool=ManageInventoryTool(),+)+```+</CodeGroup>++### Specifying tools via function docstrings+You can create a tool by passing in a function with a [Google Style Python docstring](https://google.github.io/styleguide/pyguide.html#383-functions-and-methods) specifying the arguments and description of the tool:++<CodeGroup>+```typescript title="typescript"+// install letta-client with `npm install @letta-ai/letta-client`+import { LettaClient } from '@letta-ai/letta-client';++// create a client connected to Letta Cloud+const client = new LettaClient({+ token: process.env.LETTA_API_KEY+});++// define a function+function rollDice(): string {+ const diceRoleOutcome = Math.floor(Math.random() * 20) + 1;+ const outputString = `You rolled a ${diceRoleOutcome}`;+ return outputString;+}++// create the tool+const tool = await client.tools.createFromFunction({+ func: rollDice+});+```++```python title="python" maxLines=50+# install letta_client with `pip install letta-client`+from letta_client import Letta+import os++# create a client connected to Letta Cloud+client = Letta(token=os.getenv("LETTA_API_KEY"))++# define a function with a docstring+def roll_dice() -> str:+ """+ Simulate the roll of a 20-sided die (d20).++ This function generates a random integer between 1 and 20, inclusive,+ which represents the outcome of a single roll of a d20.++ Returns:+ str: The result of the die roll.+ """+ import random++ dice_role_outcome = random.randint(1, 20)+ output_string = f"You rolled a {dice_role_outcome}"+ return output_string++# create the tool+tool = client.tools.create_from_function(+ func=roll_dice+)+```+</CodeGroup>++The tool creation will return a `Tool` object. You can update the tool with `client.tools.upsert_from_function(...)`.+++### Specifying arguments via Pydantic models+To specify the arguments for a complex tool, you can use the `args_schema` parameter.++```python title="python" maxLines=50+# install letta_client with `pip install letta-client`+from letta_client import Letta++class Step(BaseModel):+ name: str = Field(+ ...,+ description="Name of the step.",+ )+ description: str = Field(+ ...,+ description="An exhaustic description of what this step is trying to achieve and accomplish.",+ )+++class StepsList(BaseModel):+ steps: list[Step] = Field(+ ...,+ description="List of steps to add to the task plan.",+ )+ explanation: str = Field(+ ...,+ description="Explanation for the list of steps.",+ )++def create_task_plan(steps, explanation):+ """ Creates a task plan for the current task. """+ return steps+++tool = client.tools.upsert_from_function(+ func=create_task_plan,+ args_schema=StepsList+)+```+Note: this path for updating tools is currently only supported in Python.++### Creating a tool from a file+You can also define a tool from a file that contains source code. For example, you may have the following file:+```python title="custom_tool.py" maxLines=50+from typing import List, Optional+from pydantic import BaseModel, Field+++class Order(BaseModel):+ order_number: int = Field(+ ...,+ description="The order number to check on.",+ )+ customer_name: str = Field(+ ...,+ description="The customer name to check on.",+ )++def check_order_status(+ orders: List[Order]+):+ """+ Check status of a provided list of orders++ Args:+ orders (List[Order]): List of orders to check++ Returns:+ str: The status of the order (e.g. cancelled, refunded, processed, processing, shipping).+ """+ # TODO: implement+ return "ok"++```+Then, you can define the tool in Letta via the `source_code` parameter:++<CodeGroup>+```typescript title="typescript"+import * as fs from 'fs';++const tool = await client.tools.create({+ sourceCode: fs.readFileSync("custom_tool.py", "utf-8")+});+```++```python title="python" maxLines=50+tool = client.tools.create(+ source_code = open("custom_tool.py", "r").read()+)+```+</CodeGroup>++Note that in this case, `check_order_status` will become the name of your tool, since it is the last Python function in the file. Make sure it includes a [Google Style Python docstring](https://google.github.io/styleguide/pyguide.html#383-functions-and-methods) to define the tool's arguments and description.++# (Advanced) Accessing Agent State+<Warning>+Tools that use `agent_state` currently do not work in the ADE live tool tester (they will error when you press "Run"), however if the tool is correct it will work once you attach it to an agent.+</Warning>+If you need to directly access the state of an agent inside a tool, you can use the reserved `agent_state` keyword argument, for example:+```python title="python"+def get_agent_id(agent_state: "AgentState") -> str:+ """+ A custom tool that returns the agent ID++ Returns:+ str: The agent ID+ """+ return agent_state.id+```diff --git a/fern/pages/agents/human_in_the_loop.mdx b/fern/pages/agents/human_in_the_loop.mdxindex b0639aaca..23aac1324 100644--- a/fern/pages/agents/human_in_the_loop.mdx+++ b/fern/pages/agents/human_in_the_loop.mdx@@ -82,7 +82,8 @@ Set approval requirements at the tool level when creating or upserting a tool. T<CodeGroup>```curl curl maxLines=50curl --request POST \- --url http://localhost:8283/v1/tools \+ --url https://api.letta.com/v1/tools \+ --header 'Authorization: Bearer $LETTA_API_KEY' \--header 'Content-Type: application/json' \--data '{"name": "sensitive_operation",@@ -99,7 +100,8 @@ curl --request POST \# All agents using this tool will require approvalcurl --request POST \- --url http://localhost:8283/v1/agents \+ --url https://api.letta.com/v1/agents \+ --header 'Authorization: Bearer $LETTA_API_KEY' \--header 'Content-Type: application/json' \--data '{"tools": ["sensitive_operation"],@@ -153,7 +155,8 @@ For an already existing tool, you can modify the tool to set approval requiremen<CodeGroup>```curl curl maxLines=50curl --request PATCH \- --url http://localhost:8283/v1/tools/$TOOL_ID \+ --url https://api.letta.com/v1/tools/$TOOL_ID \+ --header 'Authorization: Bearer $LETTA_API_KEY' \--header 'Content-Type: application/json' \--data '{"default_requires_approval": true@@ -161,7 +164,8 @@ curl --request PATCH \# All agents using this tool will require approvalcurl --request POST \- --url http://localhost:8283/v1/agents \+ --url https://api.letta.com/v1/agents \+ --header 'Authorization: Bearer $LETTA_API_KEY' \--header 'Content-Type: application/json' \--data '{"tools": ["sensitive_operation"],@@ -209,7 +213,8 @@ Use the following endpoints to modify approval settings for existing agent-tool<CodeGroup>```curl curl maxLines=50curl --request PATCH \- --url http://localhost:8283/v1/agents/$AGENT_ID/tools/$TOOL_NAME/approval \+ --url https://api.letta.com/v1/agents/$AGENT_ID/tools/$TOOL_NAME/approval \+ --header 'Authorization: Bearer $LETTA_API_KEY' \--header 'Content-Type: application/json' \--data '{"requires_approval": true@@ -263,7 +268,8 @@ This format matches the ToolCallMessage format intentionally, so that we can han<CodeGroup>```curl curl maxLines=50curl --request POST \- --url http://localhost:8283/v1/agents/$AGENT_ID/messages \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages \+ --header 'Authorization: Bearer $LETTA_API_KEY' \--header 'Content-Type: application/json' \--data '{"messages": [{@@ -369,7 +375,8 @@ To approve a tool call, send an approval message with `approve: true` and the ap<CodeGroup>```curl curl maxLines=50curl --request POST \- --url http://localhost:8283/v1/agents/$AGENT_ID/messages \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages \+ --header 'Authorization: Bearer $LETTA_API_KEY' \--header 'Content-Type: application/json' \--data '{"messages": [{@@ -478,7 +485,8 @@ When denying a tool call, you can provide a reason that helps the agent understa<CodeGroup>```curl curl maxLines=50curl --request POST \- --url http://localhost:8283/v1/agents/$AGENT_ID/messages \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages \+ --header 'Authorization: Bearer $LETTA_API_KEY' \--header 'Content-Type: application/json' \--data '{"messages": [{@@ -603,7 +611,7 @@ Do not assume the `tool_return_message` will repeat after you resume. Treat the<CodeGroup>```curl curl maxLines=70# Approve in background after receiving approval_request_message-curl --request POST --url http://localhost:8283/v1/agents/$AGENT_ID/messages/stream --header 'Content-Type: application/json' --data '{+curl --request POST --url https://api.letta.com/v1/agents/$AGENT_ID/messages/stream --header 'Content-Type: application/json' --data '{"messages": [{"type": "approval", "approve": true, "approval_request_id": "message-abc"}],"stream_tokens": true,"background": true@@ -613,7 +621,7 @@ curl --request POST --url http://localhost:8283/v1/agents/$AGENT_ID/messages/sdata: {"run_id":"run-new","seq_id":0,"message_type":"tool_return_message","status":"success","tool_return":"..."}# Continue by resuming the approval stream's run-curl --request GET --url http://localhost:8283/v1/runs/$RUN_ID/stream --header 'Accept: text/event-stream' --data '{+curl --request GET --url https://api.letta.com/v1/runs/$RUN_ID/stream --header 'Accept: text/event-stream' --data '{"starting_after": 0}'```diff --git a/fern/pages/agents/json_mode.mdx b/fern/pages/agents/json_mode.mdxnew file mode 100644index 000000000..bf50ef69d--- /dev/null+++ b/fern/pages/agents/json_mode.mdx@@ -0,0 +1,460 @@+---+title: JSON Mode & Structured Output+subtitle: Get structured JSON responses from your Letta agents+slug: guides/agents/json-mode+---++Letta provides two ways to get structured JSON output from agents: **Structured Generation through Tools** (recommended) and the `response_format` parameter.++## Quick Comparison++<Note>+**Recommended**: Use **Structured Generation through Tools** - works with all providers (Anthropic, OpenAI, Google, etc.) and integrates naturally with Letta's tool-calling architecture.+</Note>++<Info>+**Structured Generation through Tools**:+- ✅ Universal provider compatibility+- ✅ Both reasoning AND structured output+- ✅ Per-message control+- ✅ Works even as "dummy tool" for pure formatting+</Info>++<Warning>+**`response_format` parameter**:+- ⚠️ OpenAI-compatible providers only (NOT Anthropic)+- ⚠️ Persistent agent state (affects all future responses)++- ✅ Built-in provider schema enforcement+</Warning>++## Structured Generation through Tools (Recommended)++Create a tool that defines your desired response format. The tool arguments become your structured data, and you can extract them from the tool call.++### Creating a Structured Generation Tool++<CodeGroup>+```typescript TypeScript maxLines=100+import { LettaClient } from '@letta-ai/letta-client'++// Create client connected to Letta Cloud+const client = new LettaClient({ token: process.env.LETTA_API_KEY });++// First create the tool+const toolCode = `def generate_rank(rank: int, reason: str):+ """Generate a ranking with explanation.++ Args:+ rank (int): The numerical rank from 1-10.+ reason (str): The reasoning behind the rank.+ """+ print("Rank generated")+ return`;++const tool = await client.tools.create({+ sourceCode: toolCode,+ sourceType: "python"+});++// Create agent with the structured generation tool+const agentState = await client.agents.create({+ model: "openai/gpt-4o-mini",+ memoryBlocks: [+ {+ label: "human",+ value: "The human's name is Chad. They are a food enthusiast who enjoys trying different cuisines."+ },+ {+ label: "persona",+ value: "I am a helpful food critic assistant. I provide detailed rankings and reviews of different foods and restaurants."+ }+ ],+ toolIds: [tool.id]+});+```++```python title="python" maxLines=100+from letta_client import Letta++# Create client connected to Letta Cloud+import os+client = Letta(token=os.getenv("LETTA_API_KEY"))++def generate_rank(rank: int, reason: str):+ """Generate a ranking with explanation.++ Args:+ rank (int): The numerical rank from 1-10.+ reason (str): The reasoning behind the rank.+ """+ print("Rank generated")+ return++# Create the tool+tool = client.tools.create(func=generate_rank)++# Create agent with the structured generation tool+agent_state = client.agents.create(+ model="openai/gpt-4o-mini",+ embedding="openai/text-embedding-3-small",+ memory_blocks=[+ {+ "label": "human",+ "value": "The human's name is Chad. They are a food enthusiast who enjoys trying different cuisines."+ },+ {+ "label": "persona",+ "value": "I am a helpful food critic assistant. I provide detailed rankings and reviews of different foods and restaurants."+ }+ ],+ tool_ids=[tool.id]+)+```+</CodeGroup>++### Using the Structured Generation Tool++<CodeGroup>+```typescript TypeScript maxLines=100+// Send message and instruct agent to use the tool+const response = await client.agents.messages.create(+ agentState.id, {+ messages: [+ {+ role: "user",+ content: "How do you rank sushi as a food? Please use the generate_rank tool to provide your response."+ }+ ]+ }+);++// Extract structured data from tool call+for (const message of response.messages) {+ if (message.messageType === "tool_call_message") {+ const args = JSON.parse(message.toolCall.arguments);+ console.log(`Rank: ${args.rank}`);+ console.log(`Reason: ${args.reason}`);+ }+}++// Example output:+// Rank: 8+// Reason: Sushi is a highly regarded cuisine known for its fresh ingredients...+```++```python title="python" maxLines=100+# Send message and instruct agent to use the tool+response = client.agents.messages.create(+ agent_id=agent_state.id,+ messages=[+ {+ "role": "user",+ "content": "How do you rank sushi as a food? Please use the generate_rank tool to provide your response."+ }+ ]+)++# Extract structured data from tool call+for message in response.messages:+ if message.message_type == "tool_call_message":+ import json+ args = json.loads(message.tool_call.arguments)+ rank = args["rank"]+ reason = args["reason"]+ print(f"Rank: {rank}")+ print(f"Reason: {reason}")++# Example output:+# Rank: 8+# Reason: Sushi is a highly regarded cuisine known for its fresh ingredients...+```+</CodeGroup>++The agent will call the tool, and you can extract the structured arguments:++```json+{+ "rank": 8,+ "reason": "Sushi is a highly regarded cuisine known for its fresh ingredients, artistic presentation, and cultural significance."+}+```++## Using `response_format` for Provider-Native JSON Mode++The `response_format` parameter enables structured output/JSON mode from LLM providers that support it. This approach is fundamentally different from tools because **`response_format` becomes a persistent part of the agent's state** - once set, all future responses from that agent will follow the format until explicitly changed.++Under the hood, `response_format` constrains the agent's assistant messages to follow the specified schema, but it doesn't affect tools - those continue to work normally with their original schemas.++<Warning>+**Requirements for `response_format`:**+- Only works with providers that support structured outputs (like OpenAI) - NOT Anthropic or other providers++</Warning>++### Basic JSON Mode++<CodeGroup>+```typescript TypeScript maxLines=100+import { LettaClient } from '@letta-ai/letta-client'++// Create client (Letta Cloud)+const client = new LettaClient({ token: "LETTA_API_KEY" });++// Create agent with basic JSON mode (OpenAI/compatible providers only)+const agentState = await client.agents.create({+ model: "openai/gpt-4o-mini",+ memoryBlocks: [+ {+ label: "human",+ value: "The human's name is Chad. They work as a data analyst and prefer clear, organized information."+ },+ {+ label: "persona",+ value: "I am a helpful assistant who provides clear and well-organized responses."+ }+ ],+ responseFormat: { type: "json_object" }+});++// Send message expecting JSON response+const response = await client.agents.messages.create(+ agentState.id, {+ messages: [+ {+ role: "user",+ content: "How do you rank sushi as a food? Please respond in JSON format with rank and reason fields."+ }+ ]+ }+);++for (const message of response.messages) {+ console.log(message);+}+```++```python title="python" maxLines=100+from letta_client import Letta++# Create client (Letta Cloud)+client = Letta(token="LETTA_API_KEY")++# Create agent with basic JSON mode (OpenAI/compatible providers only)+agent_state = client.agents.create(+ model="openai/gpt-4o-mini",+ embedding="openai/text-embedding-3-small",+ memory_blocks=[+ {+ "label": "human",+ "value": "The human's name is Chad. They work as a data analyst and prefer clear, organized information."+ },+ {+ "label": "persona",+ "value": "I am a helpful assistant who provides clear and well-organized responses."+ }+ ],+ response_format={"type": "json_object"}+)++# Send message expecting JSON response+response = client.agents.messages.create(+ agent_id=agent_state.id,+ messages=[+ {+ "role": "user",+ "content": "How do you rank sushi as a food? Please respond in JSON format with rank and reason fields."+ }+ ]+)++for message in response.messages:+ print(message)+```+</CodeGroup>++### Advanced JSON Schema Mode++For more precise control, you can use OpenAI's `json_schema` mode with strict validation:++<CodeGroup>+```typescript TypeScript maxLines=100+import { LettaClient } from '@letta-ai/letta-client'++const client = new LettaClient({ token: "LETTA_API_KEY" });++// Define structured schema (from OpenAI structured outputs guide)+const responseFormat = {+ type: "json_schema",+ jsonSchema: {+ name: "food_ranking",+ schema: {+ type: "object",+ properties: {+ rank: {+ type: "integer",+ minimum: 1,+ maximum: 10+ },+ reason: {+ type: "string"+ },+ categories: {+ type: "array",+ items: {+ type: "object",+ properties: {+ name: { type: "string" },+ score: { type: "integer" }+ },+ required: ["name", "score"],+ additionalProperties: false+ }+ }+ },+ required: ["rank", "reason", "categories"],+ additionalProperties: false+ },+ strict: true+ }+};++// Create agent+const agentState = await client.agents.create({+ model: "openai/gpt-4o-mini",+ memoryBlocks: []+});++// Update agent with response format+const updatedAgent = await client.agents.update(+ agentState.id,+ { responseFormat }+);++// Send message+const response = await client.agents.messages.create(+ agentState.id, {+ messages: [+ { role: "user", content: "How do you rank sushi? Include categories for taste, presentation, and value." }+ ]+ }+);++for (const message of response.messages) {+ console.log(message);+}+```++```python title="python" maxLines=100+from letta_client import Letta++client = Letta(token="LETTA_API_KEY")++# Define structured schema (from OpenAI structured outputs guide)+response_format = {+ "type": "json_schema",+ "json_schema": {+ "name": "food_ranking",+ "schema": {+ "type": "object",+ "properties": {+ "rank": {+ "type": "integer",+ "minimum": 1,+ "maximum": 10+ },+ "reason": {+ "type": "string"+ },+ "categories": {+ "type": "array",+ "items": {+ "type": "object",+ "properties": {+ "name": { "type": "string" },+ "score": { "type": "integer" }+ },+ "required": ["name", "score"],+ "additionalProperties": False+ }+ }+ },+ "required": ["rank", "reason", "categories"],+ "additionalProperties": False+ },+ "strict": True+ }+}++# Create agent+agent_state = client.agents.create(+ model="openai/gpt-4o-mini",+ embedding="openai/text-embedding-3-small",+ memory_blocks=[]+)++# Update agent with response format+agent_state = client.agents.update(+ agent_id=agent_state.id,+ response_format=response_format+)++# Send message+response = client.agents.messages.create(+ agent_id=agent_state.id,+ messages=[+ {"role": "user", "content": "How do you rank sushi? Include categories for taste, presentation, and value."}+ ]+)++for message in response.messages:+ print(message)+```+</CodeGroup>++With structured JSON schema, the agent's response will be strictly validated:++```json+{+ "rank": 8,+ "reason": "Sushi is highly regarded for its fresh ingredients and artful presentation",+ "categories": [+ {"name": "taste", "score": 9},+ {"name": "presentation", "score": 10},+ {"name": "value", "score": 6}+ ]+}+```+++## Updating Agent Response Format++You can update an existing agent's response format:++<CodeGroup>+```typescript TypeScript maxLines=100+// Update agent to use JSON mode (OpenAI/compatible only)+await client.agents.update(agentState.id, {+ responseFormat: { type: "json_object" }+});++// Or remove JSON mode+await client.agents.update(agentState.id, {+ responseFormat: null+});+```++```python title="python" maxLines=100+# Update agent to use JSON mode (OpenAI/compatible only)+client.agents.update(+ agent_id=agent_state.id,+ response_format={"type": "json_object"}+)++# Or remove JSON mode+client.agents.update(+ agent_id=agent_state.id,+ response_format=None+)+```+</CodeGroup>diff --git a/fern/pages/agents/long_running.mdx b/fern/pages/agents/long_running.mdxnew file mode 100644index 000000000..524b6e3e5--- /dev/null+++ b/fern/pages/agents/long_running.mdx@@ -0,0 +1,602 @@+---+title: Long-Running Executions+slug: guides/agents/long-running+subtitle: How to handle long-running agent executions+---++When agents need to execute multiple tool calls or perform complex operations (like deep research, data analysis, or multi-step workflows), processing time can vary significantly.++Letta supports various ways to handle long-running agents, so you can choose the approach that best fits your use case:++| Use Case | Duration | Recommendedation | Key Benefits |+|----------|----------|---------------------|-------------|+| Few-step invocations | < 1 minute | [Standard streaming](/guides/agents/streaming) | Simplest approach |+| Variable length runs | 1-10 minutes | **Background mode** (Keepalive + Timeout as a second choice) | Easy way to reduce timeouts |+| Deep research | 10+ minutes | **Background mode**, or async polling | Survives disconnects, resumable streams |+| Batch jobs | Any | **Async polling** | Fire-and-forget, check results later |++## Option 1: Background Mode with Resumable Streaming++<Note>+**Best for:** Operations exceeding 10 minutes, unreliable network connections, or critical workflows that must complete regardless of client connectivity.++**Trade-off:** Slightly higher latency to first token due to background task initialization.+</Note>++Background mode decouples agent execution from your client connection. The agent processes your request on the server while streaming results to a persistent store, allowing you to reconnect and resume from any point — even if your application crashes or network fails.++<CodeGroup>+```curl curl maxLines=50+curl --request POST \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages/stream \+ --header 'Authorization: Bearer $LETTA_API_KEY' \+ --header 'Content-Type: application/json' \+ --data '{+ "messages": [+ {+ "role": "user",+ "content": "Run comprehensive analysis on this dataset"+ }+ ],+ "stream_tokens": true,+ "background": true+}'++# Response stream includes run_id and seq_id for each chunk:+data: {"run_id":"run-123","seq_id":0,"message_type":"reasoning_message","reasoning":"Analyzing"}+data: {"run_id":"run-123","seq_id":1,"message_type":"reasoning_message","reasoning":" the dataset"}+data: {"run_id":"run-123","seq_id":2,"message_type":"tool_call","tool_call":{...}}+# ... stream continues++# Step 2: If disconnected, resume from last received seq_id+curl --request GET \+ --url https://api.letta.com/v1/runs/$RUN_ID/stream \+ --header 'Accept: text/event-stream' \+ --data '{+ "starting_after": 57+}'+```+```python python maxLines=50+stream = client.agents.messages.create_stream(+ agent_id=agent_state.id,+ messages=[+ {+ "role": "user",+ "content": "Run comprehensive analysis on this dataset"+ }+ ],+ stream_tokens=True,+ background=True,+)+run_id = None+last_seq_id = None+for chunk in stream:+ if hasattr(chunk, "run_id") and hasattr(chunk, "seq_id"):+ run_id = chunk.run_id # Save this to reconnect if your connection drops+ last_seq_id = chunk.seq_id # Save this as your resumption point for cursor-based pagination+ print(chunk)++# If disconnected, resume from last received seq_id:+for chunk in client.runs.stream(run_id, starting_after=last_seq_id):+ print(chunk)+```+```typescript TypeScript maxLines=50+const stream = await client.agents.messages.createStream({+ agentId: agentState.id,+ requestBody: {+ messages: [+ {+ role: "user",+ content: "Run comprehensive analysis on this dataset"+ }+ ],+ streamTokens: true,+ background: true,+ }+});++let runId = null;+let lastSeqId = null;+for await (const chunk of stream) {+ if (chunk.run_id && chunk.seq_id) {+ runId = chunk.run_id; // Save this to reconnect if your connection drops+ lastSeqId = chunk.seq_id; // Save this as your resumption point for cursor-based pagination+ }+ console.log(chunk);+}++// If disconnected, resume from last received seq_id+for await (const chunk of client.runs.stream(runId, {startingAfter: lastSeqId})) {+ console.log(chunk);+}+```+```python python maxLines=60+# 1) Start background stream and capture approval request+stream = client.agents.messages.create_stream(+ agent_id=agent.id,+ messages=[{"role": "user", "content": "Do a sensitive operation"}],+ stream_tokens=True,+ background=True,+)++approval_request_id = None+orig_run_id = None+last_seq_id = 0+for chunk in stream:+ if hasattr(chunk, "run_id") and hasattr(chunk, "seq_id"):+ orig_run_id = chunk.run_id+ last_seq_id = chunk.seq_id+ if getattr(chunk, "message_type", None) == "approval_request_message":+ approval_request_id = chunk.id+ break++# 2) Approve in background; capture the approval stream cursor (this creates a new run)+approve = client.agents.messages.create_stream(+ agent_id=agent.id,+ messages=[{"type": "approval", "approve": True, "approval_request_id": approval_request_id}],+ stream_tokens=True,+ background=True,+)++run_id = None+approve_seq = 0+for chunk in approve:+ if hasattr(chunk, "run_id") and hasattr(chunk, "seq_id"):+ run_id = chunk.run_id+ approve_seq = chunk.seq_id+ if getattr(chunk, "message_type", None) == "tool_return_message":+ # Tool result arrives here on the approval stream+ break++# 3) Resume that run to read follow-up tokens+for chunk in client.runs.stream(run_id, starting_after=approve_seq):+ print(chunk)+```+```typescript TypeScript maxLines=60+// 1) Start background stream and capture approval request+const stream = await client.agents.messages.createStream(+ agent.id, {+ messages: [{role: "user", content: "Do a sensitive operation"}],+ streamTokens: true,+ background: true,+ }+);++let approvalRequestId = null;+let origRunId = null;+let lastSeqId = 0;+for await (const chunk of stream) {+ if (chunk.runId && chunk.seqId) {+ origRunId = chunk.runId;+ lastSeqId = chunk.seqId;+ }+ if (chunk.messageType === "approval_request_message") {+ approvalRequestId = chunk.id;+ break;+ }+}++// 2) Approve in background; capture the approval stream cursor (this creates a new run)+const approveStream = await client.agents.messages.createStream(+ agent.id, {+ messages: [{type: "approval", approve: true, approvalRequestId}],+ streamTokens: true,+ background: true,+ }+);++let runId = null;+let approveSeq = 0;+for await (const chunk of approveStream) {+ if (chunk.runId && chunk.seqId) {+ runId = chunk.runId;+ approveSeq = chunk.seqId;+ }+ if (chunk.messageType === "tool_return_message") {+ // Tool result arrives here on the approval stream+ break;+ }+}++// 3) Resume that run to read follow-up tokens+for await (const chunk of client.runs.stream(runId, {startingAfter: approveSeq})) {+ console.log(chunk);+}+```+</CodeGroup>++### HITL in Background Mode++When [Human‑in‑the‑Loop (HITL) approval](/guides/agents/human-in-the-loop) is enabled for a tool, your background stream may pause and emit an `approval_request_message`. In background mode, send the approval via a separate background stream and capture that stream’s `run_id`/`seq_id`.++<Note>+Approval responses in background mode emit the `tool_return_message` on the approval stream itself (with a new `run_id`, different from the original stream). Save the approval stream cursor, then resume with `runs.stream` to consume subsequent reasoning/assistant messages.+</Note>++<CodeGroup>+```curl curl maxLines=70+# 1) Start background stream; capture approval request+curl --request POST \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages/stream \+ --header 'Authorization: Bearer $LETTA_API_KEY' \+ --header 'Content-Type: application/json' \+ --data '{+ "messages": [{"role": "user", "content": "Do a sensitive operation"}],+ "stream_tokens": true,+ "background": true+}'++# Example stream output (approval request arrives):+data: {"run_id":"run-abc","seq_id":0,"message_type":"reasoning_message","reasoning":"..."}+data: {"run_id":"run-abc","seq_id":1,"message_type":"approval_request_message","id":"message-abc","tool_call":{"name":"sensitive_operation","arguments":"{...}","tool_call_id":"tool-xyz"}}++# 2) Approve in background; capture approval stream cursor (this creates a new run)+curl --request POST \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages/stream \+ --header 'Authorization: Bearer $LETTA_API_KEY' \+ --header 'Content-Type: application/json' \+ --data '{+ "messages": [{"type": "approval", "approve": true, "approval_request_id": "message-abc"}],+ "stream_tokens": true,+ "background": true+}'++# Example approval stream output (tool result arrives here):+data: {"run_id":"run-new","seq_id":0,"message_type":"tool_return_message","status":"success","tool_return":"..."}++# 3) Resume the approval stream's run to continue+curl --request GET \+ --url https://api.letta.com/v1/runs/$RUN_ID/stream \+ --header 'Accept: text/event-stream' \+ --data '{+ "starting_after": 0+}'+```+```python python maxLines=70+# 1) Start background stream and capture approval request+stream = client.agents.messages.create_stream(+ agent_id=agent.id,+ messages=[{"role": "user", "content": "Do a sensitive operation"}],+ stream_tokens=True,+ background=True,+)++approval_request_id = None+orig_run_id = None+last_seq_id = 0+for chunk in stream:+ if hasattr(chunk, "run_id") and hasattr(chunk, "seq_id"):+ orig_run_id = chunk.run_id+ last_seq_id = chunk.seq_id+ if getattr(chunk, "message_type", None) == "approval_request_message":+ approval_request_id = chunk.id+ break++# 2) Approve in background; capture the approval stream cursor (this creates a new run)+approve = client.agents.messages.create_stream(+ agent_id=agent.id,+ messages=[{"type": "approval", "approve": True, "approval_request_id": approval_request_id}],+ stream_tokens=True,+ background=True,+)++run_id = None+approve_seq = 0+for chunk in approve:+ if hasattr(chunk, "run_id") and hasattr(chunk, "seq_id"):+ run_id = chunk.run_id+ approve_seq = chunk.seq_id+ if getattr(chunk, "message_type", None) == "tool_return_message":+ # Tool result arrives here on the approval stream+ break++# 3) Resume that run to read follow-up tokens+for chunk in client.runs.stream(run_id, starting_after=approve_seq):+ print(chunk)+```+```typescript TypeScript maxLines=70+// 1) Start background stream and capture approval request+const stream = await client.agents.messages.createStream({+ agentId: agent.id,+ requestBody: {+ messages: [{ role: "user", content: "Do a sensitive operation" }],+ streamTokens: true,+ background: true,+ }+});++let approvalRequestId: string | null = null;+let origRunId: string | null = null;+let lastSeqId = 0;+for await (const chunk of stream) {+ if (chunk.run_id && chunk.seq_id) { origRunId = chunk.run_id; lastSeqId = chunk.seq_id; }+ if (chunk.message_type === "approval_request_message") {+ approvalRequestId = chunk.id; break;+ }+}++// 2) Approve in background; capture the approval stream cursor (this creates a new run)+const approve = await client.agents.messages.createStream({+ agentId: agent.id,+ requestBody: {+ messages: [{ type: "approval", approve: true, approvalRequestId }],+ streamTokens: true,+ background: true,+ }+});++let runId: string | null = null;+let approveSeq = 0;+for await (const chunk of approve) {+ if (chunk.run_id && chunk.seq_id) { runId = chunk.run_id; approveSeq = chunk.seq_id; }+ if (chunk.message_type === "tool_return_message") {+ // Tool result arrives here on the approval stream+ break;+ }+}++// 3) Resume that run to read follow-up tokens+const resume = await client.runs.stream(runId!, { startingAfter: approveSeq });+for await (const chunk of resume) {+ console.log(chunk);+}+```+</CodeGroup>+++### Discovering and Resuming Active Streams++When your application starts or recovers from a crash, you can check for any active background streams and resume them. This is particularly useful for:+- **Application restarts**: Resume processing after deployments or crashes+- **Load balancing**: Pick up streams started by other instances+- **Monitoring**: Check progress of long-running operations from different clients++<CodeGroup>+```curl curl maxLines=50+# Step 1: Find active background streams for your agents+curl --request GET \+ --url https://api.letta.com/v1/runs/active \+ --header 'Authorization: Bearer $LETTA_API_KEY' \+ --header 'Content-Type: application/json' \+ --data '{+ "agent_ids": [+ "agent-123",+ "agent-456"+ ],+ "background": true+}'+# Returns: [{"run_id": "run-abc", "agent_id": "agent-123", "status": "processing", ...}]++# Step 2: Resume streaming from the beginning (or any specified seq_id)+curl --request GET \+ --url https://api.letta.com/v1/runs/$RUN_ID/stream \+ --header 'Accept: text/event-stream' \+ --data '{+ "starting_after": 0, # Start from beginning+ "batch_size": 1000 # Fetch historical chunks in larger batches+}'+```+```python python maxLines=50+# Find and resume active background streams+active_runs = client.runs.active(+ agent_ids=["agent-123", "agent-456"],+ background=True,+)++if active_runs:+ # Resume the first active stream from the beginning+ run = active_runs[0]+ print(f"Resuming stream for run {run.id}, status: {run.status}")++ stream = client.runs.stream(+ run_id=run.id,+ starting_after=0, # Start from beginning+ batch_size=1000 # Fetch historical chunks in larger batches+ )++ # Each historical chunk is streamed one at a time, followed by new chunks as they become available+ for chunk in stream:+ print(chunk)+```+```typescript TypeScript maxLines=50+// Find and resume active background streams+const activeRuns = await client.runs.active({+ agentIds: ["agent-123", "agent-456"],+ background: true,+});++if (activeRuns.length > 0) {+ // Resume the first active stream from the beginning+ const run = activeRuns[0];+ console.log(`Resuming stream for run ${run.id}, status: ${run.status}`);++ const stream = await client.runs.stream(run.id, {+ startingAfter: 0, // Start from beginning+ batchSize: 1000 // Fetch historical chunks in larger batches+ });++ // Each historical chunk is streamed one at a time, followed by new chunks as they become available+ for await (const chunk of stream) {+ console.log(chunk);+ }+}+```+</CodeGroup>++## Option 2: Async Operations with Polling++<Note>+**Best for:** Usecases where you don't need real-time token streaming.+</Note>++Ideal for batch processing, scheduled jobs, or when you don't need real-time updates. The [async SDK method](/api-reference/agents/messages/create-async) queues your request and returns immediately, letting you check results later:++<CodeGroup>+```curl curl maxLines=50+# Start async operation (returns immediately with run ID)+curl --request POST \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages/async \+ --header 'Authorization: Bearer $LETTA_API_KEY' \+ --header 'Content-Type: application/json' \+ --data '{+ "messages": [+ {+ "role": "user",+ "content": "Run comprehensive analysis on this dataset"+ }+ ]+}'++# Poll for results using the returned run ID+curl --request GET \+ --url https://api.letta.com/v1/runs/$RUN_ID+```+```python python maxLines=50+# Start async operation (returns immediately with run ID)+run = client.agents.messages.create_async(+ agent_id=agent_state.id,+ messages=[+ {+ "role": "user",+ "content": "Run comprehensive analysis on this dataset"+ }+ ],+)++# Poll for completion+import time+while run.status != "completed":+ time.sleep(2)+ run = client.runs.retrieve(run_id=run.id)++# Get the messages once complete+messages = client.runs.messages.list(run_id=run.id)+```+```typescript TypeScript maxLines=50+// Start async operation (returns immediately with run ID)+const run = await client.agents.createAgentMessageAsync({+ agentId: agentState.id,+ requestBody: {+ messages: [+ {+ role: "user",+ content: "Run comprehensive analysis on this dataset"+ }+ ]+ }+});++// Poll for completion+while (run.status !== "completed") {+ await new Promise(resolve => setTimeout(resolve, 2000));+ run = await client.runs.retrieveRun({ runId: run.id });+}++// Get the messages once complete+const messages = await client.runs.listRunMessages({ runId: run.id });+```+</CodeGroup>++## Option 3: Configure Streaming with Keepalive Pings and Longer Timeouts++<Note>+**Best for:** Usecases where you are already using the standard [streaming code](/guides/agents/streaming), but are experiencing issues with timeouts or disconnects (e.g. due to network interruptions or hanging tool executions).++**Trade-off:** Not as reliable as background mode, and does not support resuming a disconnected stream/request.+</Note>++<Warning>+This approach assumes a persistent HTTP connection. We highly recommend using **background mode** (or async polling) for long-running jobs, especially when:+- Your infrastructure uses aggressive proxy timeouts+- You need to handle network interruptions gracefully+- Operations might exceed 10 minutes+</Warning>++For operations under 10 minutes that need real-time updates without the complexity of background processing. Configure keepalive pings and timeouts to maintain stable connections:++<CodeGroup>+```curl curl maxLines=50+curl --request POST \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages/stream \+ --header 'Authorization: Bearer $LETTA_API_KEY' \+ --header 'Content-Type: application/json' \+ --data '{+ "messages": [+ {+ "role": "user",+ "content": "Execute this long-running analysis"+ }+ ],+ "include_pings": true+}'+```+```python python+# Configure client with extended timeout+from letta_client import Letta+import os++client = Letta(+ token=os.getenv("LETTA_API_KEY")+)++# Enable pings to prevent timeout during long operations+stream = client.agents.messages.create_stream(+ agent_id=agent_state.id,+ messages=[+ {+ "role": "user",+ "content": "Execute this long-running analysis"+ }+ ],+ include_pings=True, # Sends periodic keepalive messages+ request_options={"timeout_in_seconds": 600} # 10 min timeout+)++# Process the stream (pings will keep connection alive)+for chunk in stream:+ if chunk.message_type == "ping":+ # Keepalive ping received, connection is still active+ continue+ print(chunk)+```+```typescript TypeScript maxLines=50+// Configure client with extended timeout+import { Letta } from '@letta/sdk';++const client = new Letta({+ token: process.env.LETTA_API_KEY+});++// Enable pings to prevent timeout during long operations+const stream = await client.agents.createAgentMessageStream({+ agentId: agentState.id,+ requestBody: {+ messages: [+ {+ role: "user",+ content: "Execute this long-running analysis"+ }+ ],+ includePings: true // Sends periodic keepalive messages+ }, {+ timeoutInSeconds: 600 // 10 minutes timeout in seconds+ }+});++// Process the stream (pings will keep connection alive)+for await (const chunk of stream) {+ if (chunk.message_type === "ping") {+ // Keepalive ping received, connection is still active+ continue;+ }+ console.log(chunk);+}+```+</CodeGroup>++### Configuration Guidelines++| Parameter | Purpose | When to Use |+|-----------|---------|------------|+| Timeout in seconds | Extends request timeout beyond 60s default | Set to 1.5x your expected max duration |+| Include pings | Sends keepalive messages every ~30s | Enable for operations with long gaps between outputs |diff --git a/fern/pages/agents/memory_blocks.mdx b/fern/pages/agents/memory_blocks.mdxnew file mode 100644index 000000000..05801f76d--- /dev/null+++ b/fern/pages/agents/memory_blocks.mdx@@ -0,0 +1,407 @@+---+title: Memory Blocks+subtitle: Understanding the building blocks of agent memory+slug: guides/agents/memory-blocks+---++<Info>+Interested in learning more about the origin of memory blocks? Read our [blog post](https://www.letta.com/blog/memory-blocks).+</Info>++## What are memory blocks?++Memory blocks are structured sections of the agent's context window that persist across all interactions. They are always visible - no retrieval needed.++**Memory blocks are Letta's core abstraction.** Create a block with a descriptive label and the agent learns how to use it. This simple mechanism enables capabilities impossible with traditional context management.++**Key properties:**+- **Agent-managed** - Agents autonomously organize information based on block labels+- **Flexible** - Use for any purpose: knowledge, guidelines, state tracking, scratchpad space+- **Shareable** - Multiple agents can access the same block; update once, visible everywhere+- **Always visible** - Blocks stay in context, never need retrieval++**Examples:**+- Store tool usage guidelines so agents avoid past mistakes+- Maintain working memory in a scratchpad block+- Mirror external state (user's current document) for real-time awareness+- Share read-only policies across all agents from a central source+- Coordinate multi-agent systems: parent agents watch subagent result blocks update in real-time+- Enable emergent behavior: add `performance_tracking` or `emotional_state` and watch agents start using them++Memory blocks aren't just storage - they're a coordination primitive that enables sophisticated agent behavior.++## Memory block structure++Memory blocks represent a section of an agent's context window. An agent may have multiple memory blocks, or none at all. A memory block consists of:+* A `label`, which is a unique identifier for the block+* A `description`, which describes the purpose of the block+* A `value`, which is the contents/data of the block+* A `limit`, which is the size limit (in characters) of the block++## The importance of the `description` field++When making memory blocks, it's crucial to provide a good `description` field that accurately describes what the block should be used for.+The `description` is the main information used by the agent to determine how to read and write to that block. Without a good description, the agent may not understand how to use the block.++Because `persona` and `human` are two popular block labels, Letta autogenerates default descriptions for these blocks if you don't provide them. If you provide a description for a memory block labelled `persona` or `human`, the default description will be overridden.++For `persona`, a good default is:+> The persona block: Stores details about your current persona, guiding how you behave and respond. This helps you to maintain consistency and personality in your interactions.++For `human`, a good default is:+> The human block: Stores key details about the person you are conversing with, allowing for more personalized and friend-like conversation.++## Read-only blocks++Memory blocks are read-write by default (so the agent can update the block using memory tools), but can be set to read-only by setting the `read_only` field to `true`. When a block is read-only, the agent cannot update the block.++Read-only blocks are useful when you want to give an agent access to information (for example, a shared memory block about an organization), but you don't want the agent to be able to make potentially destructive changes to the block.++## Creating an agent with memory blocks++When you create an agent, you can specify memory blocks to also be created with the agent. For most chat applications, we recommend create a `human` block (to represent memories about the user) and a `persona` block (to represent the agent's persona).+<CodeGroup>+```typescript TypeScript maxLines=50+// install letta-client with `npm install @letta-ai/letta-client`+import { LettaClient } from '@letta-ai/letta-client'++// create a client connected to Letta Cloud+const client = new LettaClient({+ token: process.env.LETTA_API_KEY+});++// create an agent with two basic self-editing memory blocks+const agentState = await client.agents.create({+ memoryBlocks: [+ {+ label: "human",+ value: "The human's name is Bob the Builder.",+ limit: 5000+ },+ {+ label: "persona",+ value: "My name is Sam, the all-knowing sentient AI.",+ limit: 5000+ }+ ],+ model: "openai/gpt-4o-mini"+});+```+```python title="python" maxLines=50+# install letta_client with `pip install letta-client`+from letta_client import Letta+import os++# create a client connected to Letta Cloud+client = Letta(token=os.getenv("LETTA_API_KEY"))++# create an agent with two basic self-editing memory blocks+agent_state = client.agents.create(+ memory_blocks=[+ {+ "label": "human",+ "value": "The human's name is Bob the Builder.",+ "limit": 5000+ },+ {+ "label": "persona",+ "value": "My name is Sam, the all-knowing sentient AI.",+ "limit": 5000+ }+ ],+ model="openai/gpt-4o-mini"+)+```+</CodeGroup>+When the agent is created, the corresponding blocks are also created and attached to the agent, so that the block value will be in the context window.++## Creating and attaching memory blocks+You can also directly create blocks and attach them to an agent. This can be useful if you want to create blocks that are shared between multiple agents. If multiple agents are attached to a block, they will all have the block data in their context windows (essentially providing shared memory).++Below is an example of creating a block directory, and attaching the block to two agents by specifying the `block_ids` field.+<CodeGroup>+```typescript TypeScript maxLines=50+// create a persisted block, which can be attached to agents+const block = await client.blocks.create({+ label: "organization",+ description: "A block to store information about the organization",+ value: "Organization: Letta",+ limit: 4000,+});++// create an agent with both a shared block and its own blocks+const sharedBlockAgent1 = await client.agents.create({+ name: "shared_block_agent1",+ memoryBlocks: [+ {+ label: "persona",+ value: "I am agent 1"+ },+ ],+ blockIds: [block.id],+ model: "openai/gpt-4o-mini"+});++// create another agent with the same shared block+const sharedBlockAgent2 = await client.agents.create({+ name: "shared_block_agent2",+ memoryBlocks: [+ {+ label: "persona",+ value: "I am agent 2"+ },+ ],+ blockIds: [block.id],+ model: "openai/gpt-4o-mini"+});+```+```python title="python" maxLines=50+# create a persisted block, which can be attached to agents+block = client.blocks.create(+ label="organization",+ description="A block to store information about the organization",+ value="Organization: Letta",+ limit=4000,+)++# create an agent with both a shared block and its own blocks+shared_block_agent1 = client.agents.create(+ name="shared_block_agent1",+ memory_blocks=[+ {+ "label": "persona",+ "value": "I am agent 1"+ },+ ],+ block_ids=[block.id],+ model="openai/gpt-4o-mini"+)++# create another agent sharing the block+shared_block_agent2 = client.agents.create(+ name="shared_block_agent2",+ memory_blocks=[+ {+ "label": "persona",+ "value": "I am agent 2"+ },+ ],+ block_ids=[block.id],+ model="openai/gpt-4o-mini"+)+```+</CodeGroup>+You can also attach blocks to existing agents:+<CodeGroup>+```typescript TypeScript+await client.agents.blocks.attach(agent.id, block.id);+```+```python Python+client.agents.blocks.attach(agent_id=agent.id, block_id=block.id)+```+</CodeGroup>+You can see all agents attached to a block by using the `block_id` field in the [blocks retrieve](/api-reference/blocks/retrieve) endpoint.++## Managing blocks++### Retrieving a block+You can retrieve the contents of a block by ID. This is useful when blocks store finalized reports, code outputs, or other data you want to extract for use outside the agent.++<CodeGroup>+```typescript TypeScript+const block = await client.blocks.retrieve(block.id);+console.log(block.value); // access the block's content+```+```python Python+block = client.blocks.retrieve(block.id)+print(block.value) # access the block's content+```+</CodeGroup>++### Listing blocks+You can list all blocks, optionally filtering by label or searching by label text. This is useful for finding blocks across your project.++<CodeGroup>+```typescript TypeScript+// list all blocks+const blocks = await client.blocks.list();++// filter by label+const humanBlocks = await client.blocks.list({+ label: "human"+});++// search by label text+const searchResults = await client.blocks.list({+ labelSearch: "organization"+});+```+```python Python+# list all blocks+blocks = client.blocks.list()++# filter by label+human_blocks = client.blocks.list(label="human")++# search by label text+search_results = client.blocks.list(label_search="organization")+```+</CodeGroup>++### Modifying a block+You can directly modify a block's content, limit, description, or other properties. This is particularly useful for:+- External scripts that provide up-to-date information to agents (e.g., syncing a text file to a block)+- Updating shared blocks that multiple agents reference+- Programmatically managing block content outside of agent interactions++<CodeGroup>+```typescript TypeScript+// update the block's value - completely replaces the content+await client.blocks.modify(block.id, {+ value: "Updated organization information: Letta - Building agentic AI"+});++// update multiple properties+await client.blocks.modify(block.id, {+ value: "New content",+ limit: 6000,+ description: "Updated description"+});+```+```python Python+# update the block's value - completely replaces the content+client.blocks.modify(+ block.id,+ value="Updated organization information: Letta - Building agentic AI"+)++# update multiple properties+client.blocks.modify(+ block.id,+ value="New content",+ limit=6000,+ description="Updated description"+)+```+</CodeGroup>++<Warning>+**Setting `value` completely replaces the entire block content** - it is not an append operation. If multiple processes (agents or external scripts) modify the same block concurrently, the last write wins and overwrites all earlier changes. To avoid data loss:+- Set blocks to **read-only** if you don't want agents to modify them+- Only modify blocks directly in controlled scenarios where overwriting is acceptable+- Ensure your application logic accounts for full replacements, not merges+</Warning>++### Deleting a block+You can delete a block when it's no longer needed. Note that deleting a block will remove it from all agents that have it attached.++<CodeGroup>+```typescript TypeScript+await client.blocks.delete(block.id);+```+```python Python+client.blocks.delete(block_id=block.id)+```+</CodeGroup>++### Inspecting block usage+See which agents have a block attached:++<CodeGroup>+```typescript TypeScript+// list all agents that use this block+const agentsWithBlock = await client.blocks.agents.list(block.id);+console.log(`Used by ${agentsWithBlock.length} agents:`);+for (const agent of agentsWithBlock) {+ console.log(` - ${agent.name}`);+}++// with pagination+const agentsPage = await client.blocks.agents.list(block.id, {+ limit: 10,+ order: "asc"+});+```+```python Python+# list all agents that use this block+agents_with_block = client.blocks.agents.list(block_id=block.id)+print(f"Used by {len(agents_with_block)} agents:")+for agent in agents_with_block:+ print(f" - {agent.name}")++# with pagination+agents_page = client.blocks.agents.list(+ block_id=block.id,+ limit=10,+ order="asc"+)+```+</CodeGroup>++## Agent-scoped block operations++### Listing an agent's blocks+You can retrieve all blocks attached to a specific agent. This shows you the complete memory configuration for that agent.++<CodeGroup>+```typescript TypeScript+const agentBlocks = await client.agents.blocks.list(agent.id);+```+```python Python+agent_blocks = client.agents.blocks.list(agent_id=agent.id)+```+</CodeGroup>++### Retrieving an agent's block by label+Instead of using a block ID, you can retrieve a block from a specific agent using its label. This is useful when you want to inspect what the agent currently knows about a specific topic.++<CodeGroup>+```typescript TypeScript+// get the agent's current knowledge about the human+const humanBlock = await client.agents.blocks.retrieve(+ agent.id,+ "human"+);+console.log(humanBlock.value);+```+```python Python+# get the agent's current knowledge about the human+human_block = client.agents.blocks.retrieve(+ agent_id=agent.id,+ block_label="human"+)+print(human_block.value)+```+</CodeGroup>++### Modifying an agent's block+You can modify a block through the agent-scoped endpoint using the block's label. This is useful for updating agent-specific memory without needing to know the block ID.++<CodeGroup>+```typescript TypeScript+// update the agent's human block+await client.agents.blocks.modify(agent.id, "human", {+ value: "The human's name is Alice. She prefers Python over TypeScript."+});+```+```python Python+# update the agent's human block+client.agents.blocks.modify(+ agent_id=agent.id,+ block_label="human",+ value="The human's name is Alice. She prefers Python over TypeScript."+)+```+</CodeGroup>++### Detaching blocks from agents+You can detach a block from an agent's context window. This removes the block from the agent's memory without deleting the block itself.++<CodeGroup>+```typescript TypeScript+await client.agents.blocks.detach(agent.id, block.id);+```+```python Python+client.agents.blocks.detach(agent_id=agent.id, block_id=block.id)+```+</CodeGroup>diff --git a/fern/pages/agents/multiuser.mdx b/fern/pages/agents/multiuser.mdxnew file mode 100644index 000000000..4e933c1ff--- /dev/null+++ b/fern/pages/agents/multiuser.mdx@@ -0,0 +1,177 @@+---+title: User Identities+slug: guides/agents/multi-user+---++You may be building a multi-user application with Letta, in which each user is associated with a specific agent.+In this scenario, you can use **Identities** to associate each agent with a user in your application.++## Using Identities+Let's assume that you have an application with multiple users that you're building on a [self-hosted Letta server](/guides/server/docker) or [Letta Cloud](/guides/cloud).+Each user has a unique username, starting at `user_1`, and incrementing up as you add more users to the platform.++To associate agents you create in Letta with your users, you can first create an **Identity** object with the user's unique ID as the `identifier_key` for your user, and then specify the **Identity** object ID when creating an agent.++For example, with `user_1`, we would create a new Identity object with `identifier_key="user_1"` and then pass `identity.id` into our [create agent request](/api-reference/agents/create):+<CodeBlocks>+```curl title="curl"+curl -X POST https://app.letta.com/v1/identities/ \+ -H "Authorization: Bearer <token>" \+ -H "Content-Type: application/json" \+ -d '{+ "identifier_key": "user_1",+ "name": "Caren",+ "identity_type": "user"+}'+{"id":"identity-634d3994-5d6c-46e9-b56b-56e34fe34ca0","identifier_key":"user_1","name":"Caren","identity_type":"user","project_id":null,"agent_ids":[],"organization_id":"org-00000000-0000-4000-8000-000000000000","properties":[]}+curl -X POST https://app.letta.com/v1/agents/ \+ -H "Authorization: Bearer <token>" \+ -H "Content-Type: application/json" \+ -d '{+ "memory_blocks": [],+ "llm": "anthropic/claude-3-5-sonnet-20241022",+ "context_window_limit": 200000,+ "embedding": "openai/text-embedding-3-small",+ "identity_ids": ["identity-634d3994-5d6c-46e9-b56b-56e34fe34ca0"]+}'+```+```python title="python"+# assumes that you already instantiated a client+identity = client.identities.create(+ identifier_key="user_1",+ name="Caren",+ identity_type="user"+)+agent = client.agents.create(+ memory_blocks=[],+ model="anthropic/claude-3-5-sonnet-20241022",+ context_window_limit=200000,+ identity_ids=[identity.id]+)+```++```typescript TypeScript+// assumes that you already instantiated a client+const identity = await client.identities.create({+ identifierKey: "user_1",+ name: "Caren",+ identityType: "user"+})+const agent = await client.agents.create({+ memoryBlocks: [],+ model: "anthropic/claude-3-5-sonnet-20241022",+ contextWindowLimit: 200000,+ identityIds: [identity.id]+});+```+</CodeBlocks>++Then, if I wanted to search for agents associated with a specific user (e.g. called `user_id`), I could use the `identifier_keys` parameter in the [list agents request](/api-reference/agents/list):+<CodeBlocks>+```curl title="curl"+curl -X GET "https://app.letta.com/v1/agents/?identifier_keys=user_1" \+ -H "Accept: application/json"+```+```python title="python"+# assumes that you already instantiated a client+user_agents = client.agents.list(+ identifier_keys=["user_1"]+)+```+```typescript TypeScript+// assumes that you already instantiated a client+await client.agents.list({+ identifierKeys: ["user_1"]+});+```+</CodeBlocks>++You can also create an identity object and attach it to an existing agent. This can be useful if you want to enable multiple users to interact with a single agent:+<CodeBlocks>+```curl title="curl"+curl -X POST https://app.letta.com/v1/identities/ \+ -H "Authorization: Bearer <token>" \+ -H "Content-Type: application/json" \+ -d '{+ "identifier_key": "user_1",+ "name": "Sarah",+ "identity_type": "user"+ "agent_ids": ["agent-00000000-0000-4000-8000-000000000000"]+}'+```+```python title="python"+# assumes that you already instantiated a client+identity = client.identities.create({+ identifier_key="user_1",+ name="Sarah",+ identity_type="user"+ agent_ids=["agent-00000000-0000-4000-8000-000000000000"]+})+```+```typescript TypeScript+// assumes that you already instantiated a client+const identity = await client.identities.create({+ identifierKey: "user_1",+ name: "Sarah",+ identityType: "user"+ agentIds: ["agent-00000000-0000-4000-8000-000000000000"]+})+```+</CodeBlocks>++### Using Agent Tags to Identify Users+It's also possible to utilize our agent tags feature to associate agents with specific users. To associate agents you create in Letta with your users, you can specify a tag when creating an agent, and set the tag to the user's unique ID.+This example assumes that you have a self-hosted Letta server running on localhost (for example, by running [`docker run ...`](/guides/server/docker)).+<Accordion title="View example SDK code">+<CodeGroup>+```typescript TypeScript+import { LettaClient } from '@letta-ai/letta-client';++// Connect to Letta Cloud+const client = new LettaClient({token: process.env.LETTA_API_KEY});+const userId = "my_uuid";++// create an agent with the userId tag+const agent = await client.agents.create({+ memoryBlocks: [],+ model: "anthropic/claude-3-5-sonnet-20241022",+ contextWindowLimit: 200000,+ tags: [userId]+});+console.log(`Created agent with id ${agent.id}, tags ${agent.tags}`);++// list agents+const userAgents = await client.agents.list({tags: [userId]});+const agentIds = userAgents.map(agent => agent.id);+console.log(`Found matching agents ${agentIds}`);+```+```python Python+from letta_client import Letta++# Connect to Letta Cloud+import os+client = Letta(token=os.getenv("LETTA_API_KEY"))+user_id = "my_uuid"++# create an agent with the user_id tag+agent = client.agents.create(+ memory_blocks=[],+ model="anthropic/claude-3-5-sonnet-20241022",+ context_window_limit=200000,+ tags=[user_id]+)+print(f"Created agent with id {agent.id}, tags {agent.tags}")++# list agents+user_agents = client.agents.list(tags=[user_id])+agent_ids = [agent.id for agent in user_agents]+print(f"Found matching agents {agent_ids}")+```+</CodeGroup>+</Accordion>++## Creating and Viewing Tags in the ADE+You can also modify tags in the ADE.+Simply click the **Advanced Settings** tab in the top-left of the ADE to view an agent's tags.+You can create new tags by typing the tag name in the input field and hitting enter.+<img src="../../images/tags.png" />diff --git a/fern/pages/agents/overview.mdx b/fern/pages/agents/overview.mdxnew file mode 100644index 000000000..4e1dcfdd8--- /dev/null+++ b/fern/pages/agents/overview.mdx@@ -0,0 +1,277 @@+---+title: Building Stateful Agents with Letta+slug: guides/agents/overview+---++<Info>+**New to Letta?** If you haven't already, read [Core Concepts](/core-concepts) to understand how Letta's stateful agents are fundamentally different from traditional LLM APIs.+</Info>++Letta agents can automatically manage long-term memory, load data from external sources, and call custom tools.+Unlike in other frameworks, Letta agents are stateful, so they keep track of historical interactions and reserve part of their context to read and write memories which evolve over time.+<img className="light" src="/images/stateful_agents.png" />+<img className="dark" src="/images/stateful_agents_dark.png" />++++Letta manages a reasoning loop for agents. At each agent step (i.e. iteration of the loop), the state of the agent is checkpointed and persisted to the database.++You can interact with agents from a REST API, the ADE, and TypeScript / Python SDKs.+As long as they are connected to the same service, all of these interfaces can be used to interact with the same agents.++<Tip>+If you're interested in learning more about stateful agents, read our [blog post](https://www.letta.com/blog/stateful-agents).+</Tip>++## Agents vs Threads++In Letta, you can think of an agent as a single entity that has a single message history which is treated as infinite.+The sequence of interactions the agent has experienced through its existence make up the agent's state (or memory).++One distinction between Letta and other agent frameworks is that Letta does not have the notion of message *threads* (or *sessions*).+Instead, there are only *stateful agents*, which have a single perpetual thread (sequence of messages).++The reason we use the term *agent* rather than *thread* is because Letta is based on the principle that **all agents interactions should be part of the persistent memory**, as opposed to building agent applications around ephemeral, short-lived interactions (like a thread or session).+```mermaid+%%{init: {'flowchart': {'rankDir': 'LR'}}}%%+flowchart LR+ subgraph Traditional["Thread-Based Agents"]+ direction TB+ llm1[LLM] --> thread1["Thread 1+ --------+ Ephemeral+ Session"]+ llm1 --> thread2["Thread 2+ --------+ Ephemeral+ Session"]+ llm1 --> thread3["Thread 3+ --------+ Ephemeral+ Session"]+ end++ Traditional ~~~ Letta++ subgraph Letta["Letta Stateful Agents"]+ direction TB+ llm2[LLM] --> agent["Single Agent+ --------+ Persistent Memory"]+ agent --> db[(PostgreSQL)]+ db -->|"Learn & Update"| agent+ end++ class thread1,thread2,thread3 session+ class agent agent+```++If you would like to create common starting points for new conversation "threads", we recommending using [agent templates](/guides/templates/overview) to create new agents for each conversation, or directly copying agent state from an existing agent.++For multi-users applications, we recommend creating an agent per-user, though you can also have multiple users message a single agent (but it will be a single shared message history).++## Create an agent+<Note>+To start creating agents with Letta Cloud, [create an API key](https://app.letta.com/api-keys) and set it as `LETTA_API_KEY` in your environment. For self-hosted deployments, see our [self-hosting guide](/guides/selfhosting/overview).+</Note>++You can create a new agent via the REST API, Python SDK, or TypeScript SDK:+<CodeGroup>+```curl curl+curl -X POST https://api.letta.com/v1/agents \+ -H "Authorization: Bearer $LETTA_API_KEY" \+ -H "Content-Type: application/json" \+ -d '{+ "memory_blocks": [+ {+ "value": "The human'\''s name is Bob the Builder.",+ "label": "human"+ },+ {+ "value": "My name is Sam, the all-knowing sentient AI.",+ "label": "persona"+ }+ ],+ "model": "openai/gpt-4o-mini",+ "context_window_limit": 16000+}'+```+```python title="python" maxLines=50+# install letta_client with `pip install letta-client`+from letta_client import Letta+import os++# create a client connected to Letta Cloud (uses api.letta.com by default)+client = Letta(token=os.getenv("LETTA_API_KEY"))++# create an agent with two basic self-editing memory blocks+agent_state = client.agents.create(+ memory_blocks=[+ {+ "label": "human",+ "value": "The human's name is Bob the Builder."+ },+ {+ "label": "persona",+ "value": "My name is Sam, the all-knowing sentient AI."+ }+ ],+ model="openai/gpt-4o-mini",+ context_window_limit=16000+)++# the AgentState object contains all the information about the agent+print(agent_state)+```+```typescript TypeScript maxLines=50+// install letta-client with `npm install @letta-ai/letta-client`+import { LettaClient } from '@letta-ai/letta-client'++// create a client connected to Letta Cloud (uses api.letta.com by default)+const client = new LettaClient({+ token: process.env.LETTA_API_KEY+});++// create an agent with two basic self-editing memory blocks+const agentState = await client.agents.create({+ memoryBlocks: [+ {+ label: "human",+ value: "The human's name is Bob the Builder."+ },+ {+ label: "persona",+ value: "My name is Sam, the all-knowing sentient AI."+ }+ ],+ model: "openai/gpt-4o-mini",+ contextWindowLimit: 16000+});++// the AgentState object contains all the information about the agent+console.log(agentState);+```+</CodeGroup>+You can also create an agent without any code using the [Agent Development Environment (ADE)](/agent-development-environment).+All Letta agents are stored in a database on the Letta server, so you can access the same agents from the ADE, the REST API, the Python SDK, and the TypeScript SDK.++The response will include information about the agent, including its `id`:+```json+{+ "id": "agent-43f8e098-1021-4545-9395-446f788d7389",+ "name": "GracefulFirefly",+ ...+}+```++Once an agent is created, you can message it:+<CodeGroup>+```curl curl+curl --request POST \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages \+ --header 'Authorization: Bearer $LETTA_API_KEY' \+ --header 'Content-Type: application/json' \+ --data '{+ "messages": [+ {+ "role": "user",+ "content": "hows it going????"+ }+ ]+}'+```+```python title="python" maxLines=50+# send a message to the agent+response = client.agents.messages.create(+ agent_id=agent_state.id,+ messages=[+ {+ "role": "user",+ "content": "hows it going????"+ }+ ]+)++# the response object contains the messages and usage statistics+print(response)++# if we want to print the usage stats+print(response.usage)++# if we want to print the messages+for message in response.messages:+ print(message)+```+```typescript TypeScript maxLines=50+// send a message to the agent+const response = await client.agents.messages.create(+ agentState.id, {+ messages: [+ {+ role: "user",+ content: "hows it going????"+ }+ ]+ }+);++// the response object contains the messages and usage statistics+console.log(response);++// if we want to print the usage stats+console.log(response.usage)++// if we want to print the messages+for (const message of response.messages) {+ console.log(message);+}+```+</CodeGroup>++### Message Types+The `response` object contains the following attributes:+* `usage`: The usage of the agent after the message was sent (the prompt tokens, completition tokens, and total tokens)+* `message`: A list of `LettaMessage` objects, generated by the agent++#### `LettaMessage`+The `LettaMessage` object is a simplified version of the `Message` object stored in the database backend.+Since a `Message` can include multiple events like a chain-of-thought and function calls, `LettaMessage` simplifies messages to have the following types:+* `reasoning_message`: The inner monologue (chain-of-thought) of the agent+* `tool_call_message`: An agent's tool (function) call+* `tool_call_return`: The result of executing an agent's tool (function) call+* `assistant_message`: An agent's response message (direct response in current architecture, or `send_message` tool call in legacy architectures)+* `system_message`: A system message (for example, an alert about the user logging in)+* `user_message`: A user message++<Note>+In current Letta agents, `assistant_message` represents the agent's direct response. In legacy architectures (`memgpt_agent`, `memgpt_v2_agent`), it wraps the `send_message` tool call.++If you prefer to see the raw tool call format in legacy agents, you can set `use_assistant_message` to `false` in the request `config` (see the [endpoint documentation](/api-reference/agents/messages/create)).+</Note>++## Common agent operations+For more in-depth guide on the full set of Letta agent operations, check out our [API reference](/api-reference/overview), our extended [Python SDK](https://github.com/letta-ai/letta/blob/main/examples/docs/example.py) and [TypeScript SDK](https://github.com/letta-ai/letta/blob/main/examples/docs/node/example.ts) examples, as well as our other [cookbooks](/cookbooks).++If you're using a self-hosted Letta server, you should set the **base URL** (`base_url` in Python, `baseUrl` in TypeScript) to the Letta server's URL (e.g. `http://localhost:8283`) when you create your client. See an example [here](/api-reference/overview).++If you're using a self-hosted server, you can omit the token if you're not using [password protection](/guides/server/docker#password-protection-advanced).+If you are using password protection, set your **token** to the **password**.+If you're using Letta Cloud, you should set the **token** to your **Letta Cloud API key**.++### Retrieving an agent's state+The agent's state is always persisted, so you can retrieve an agent's state by its ID.+<EndpointRequestSnippet endpoint="GET /v1/agents/:agent_id" />++The result of the call is an `AgentState` object:+<EndpointResponseSnippet endpoint="GET /v1/agents/:agent_id" />++### List agents+Replace `agent_id` with your actual agent ID.+<EndpointRequestSnippet endpoint="GET /v1/agents/" />++The result of the call is a list of `AgentState` objects:+<EndpointResponseSnippet endpoint="GET /v1/agents/" />++### Delete an agent+To delete an agent, you can use the `DELETE` endpoint with your `agent_id`:+<EndpointRequestSnippet endpoint="DELETE /v1/agents/:agent_id" />diff --git a/fern/pages/agents/scheduling.mdx b/fern/pages/agents/scheduling.mdxnew file mode 100644index 000000000..74e174e2a--- /dev/null+++ b/fern/pages/agents/scheduling.mdx@@ -0,0 +1,213 @@+# Scheduling++**Scheduling** is a technique for triggering Letta agents at regular intervals.+Many real-world applications require proactive behavior, such as checking emails every few hours or scraping news sites.+Scheduling can support autonomous agents with the capability to manage ongoing processes.++<Note>+Native scheduling functionality is on the Letta Cloud roadmap. The approaches described in this guide are temporary solutions that work with both self-hosted and cloud deployments.+</Note>++## Common Use Cases++When building autonomous agents with Letta, you often need to trigger them at regular intervals for tasks like:++- **System Monitoring**: Health checks that adapt based on historical patterns+- **Data Processing**: Intelligent ETL processes that handle edge cases contextually+- **Memory Maintenance**: Agents that optimize their own knowledge base over time+- **Proactive Notifications**: Context-aware alerts that consider user preferences and timing+- **Continuous Learning**: Agents that regularly ingest new information and update their understanding++This guide covers simple approaches to implement scheduled agent interactions.++## Option 1: Simple Loop++The most straightforward approach for development and testing:++<CodeGroup>+```typescript TypeScript+import { LettaClient } from '@letta-ai/letta-client';++const client = new LettaClient({ token: process.env.LETTA_API_KEY });+const agentId = "your_agent_id";++while (true) {+ const response = await client.agents.messages.create(agentId, {+ messages: [{+ role: "user",+ content: `Scheduled check at ${new Date()}`+ }]+ });+ console.log(`[${new Date()}] Agent responded`);+ await new Promise(resolve => setTimeout(resolve, 300000)); // 5 minutes+}+```++```python title="python"+import time+import os+from letta_client import Letta+from datetime import datetime++client = Letta(token=os.getenv("LETTA_API_KEY"))+agent_id = "your_agent_id"++while True:+ response = client.agents.messages.create(+ agent_id=agent_id,+ messages=[{+ "role": "user",+ "content": f"Scheduled check at {datetime.now()}"+ }]+ )+ print(f"[{datetime.now()}] Agent responded")+ time.sleep(300) # 5 minutes+```+</CodeGroup>++**Pros:** Simple, easy to debug+**Cons:** Blocks terminal, stops if process dies++## Option 2: System Cron Jobs++For production deployments, use cron for reliability:++<CodeGroup>+```typescript TypeScript+#!/usr/bin/env node+import { LettaClient } from '@letta-ai/letta-client';++async function sendMessage() {+ try {+ const client = new LettaClient({ token: process.env.LETTA_API_KEY });+ const response = await client.agents.messages.create("your_agent_id", {+ messages: [{+ role: "user",+ content: "Scheduled maintenance check"+ }]+ });+ console.log(`[${new Date()}] Success`);+ } catch (error) {+ console.error(`[${new Date()}] Error:`, error);+ }+}++sendMessage();+```++```python title="python"+#!/usr/bin/env python3+from letta_client import Letta+from datetime import datetime++try:+ import os+ client = Letta(token=os.getenv("LETTA_API_KEY"))+ response = client.agents.messages.create(+ agent_id="your_agent_id",+ messages=[{+ "role": "user",+ "content": "Scheduled maintenance check"+ }]+ )+ print(f"[{datetime.now()}] Success")+except Exception as e:+ print(f"[{datetime.now()}] Error: {e}")+```+</CodeGroup>++Add to crontab with `crontab -e`:+```bash+*/5 * * * * /usr/bin/python3 /path/to/send_message.py >> /var/log/letta_cron.log 2>&1+# or for Node.js:+*/5 * * * * /usr/bin/node /path/to/send_message.js >> /var/log/letta_cron.log 2>&1+```++**Pros:** System-managed, survives reboots+**Cons:** Requires cron access++## Best Practices++1. **Error Handling**: Always wrap API calls in try-catch blocks+2. **Logging**: Log both successes and failures for debugging+3. **Environment Variables**: Store credentials securely+4. **Rate Limiting**: Respect API limits and add backoff for failures++## Example: Memory Maintenance Bot++Complete example that performs periodic memory cleanup:++<CodeGroup>+```typescript TypeScript+#!/usr/bin/env node+import { LettaClient } from '@letta-ai/letta-client';++async function runMaintenance() {+ try {+ const client = new LettaClient({ token: process.env.LETTA_API_KEY });+ const agentId = "your_agent_id";++ const response = await client.agents.messages.create(agentId, {+ messages: [{+ role: "user",+ content: "Please review your memory blocks for outdated information and clean up as needed."+ }]+ });++ // Print any assistant messages+ for (const message of response.messages) {+ if (message.messageType === "assistant_message") {+ console.log(`Agent response: ${message.content?.substring(0, 100)}...`);+ }+ }++ } catch (error) {+ console.error("Maintenance failed:", error);+ }+}++// Run if called directly+if (import.meta.url === `file://${process.argv[1]}`) {+ runMaintenance();+}+```++```python title="python"+#!/usr/bin/env python3+import logging+from datetime import datetime+from letta_client import Letta++logging.basicConfig(+ level=logging.INFO,+ format='%(asctime)s - %(levelname)s - %(message)s'+)++def run_maintenance():+ try:+ import os+ client = Letta(token=os.getenv("LETTA_API_KEY"))+ agent_id = "your_agent_id"++ response = client.agents.messages.create(+ agent_id=agent_id,+ messages=[{+ "role": "user",+ "content": "Please review your memory blocks for outdated information and clean up as needed."+ }]+ )++ # Print any assistant messages+ for message in response.messages:+ if message.message_type == "assistant_message":+ logging.info(f"Agent response: {message.content[:100]}...")++ except Exception as e:+ logging.error(f"Maintenance failed: {e}")++if __name__ == "__main__":+ run_maintenance()+```+</CodeGroup>++Choose the scheduling method that best fits your deployment environment. For production systems, cron offers the best reliability, while simple loops are perfect for development and testing.diff --git a/fern/pages/agents/tool_variables.mdx b/fern/pages/agents/tool_variables.mdxnew file mode 100644index 000000000..31d68c6b3--- /dev/null+++ b/fern/pages/agents/tool_variables.mdx@@ -0,0 +1,53 @@+---+title: Using Tool Variables+slug: guides/agents/tool-variables+---++You can use **tool variables** to specify environment variables available to your custom tools.+For example, if you set a tool variable `PASSWORD` to `banana`, then write a custom function that prints `os.getenv('PASSWORD')` in the tool, the function will print `banana`.++## Assigning tool variables in the ADE++To assign tool variables in the Agent Development Environment (ADE), click on **Env Vars** to open the **Environment Variables** viewer:++<img src="../../images/env_vars_button.png" />++Once in the **Environment Variables** viewer, click **+** to add a new tool variable if one does not exist.++<img src="../../images/tool_variables.png" />++## Assigning tool variables in the API / SDK++You can also assign tool variables on agent creation in the API with the `tool_exec_environment_variables` parameter:+<CodeGroup>+```curl title="curl" {7-9}+curl -X POST https://api.letta.com/v1/agents \+ -H "Authorization: Bearer $LETTA_API_KEY" \+ -H "Content-Type: application/json" \+ -d '{+ "memory_blocks": [],+ "llm":"openai/gpt-4o-mini",+ "tool_exec_environment_variables": {+ "API_KEY": "your-api-key-here"+ }+}'+```+```python title="python" {5-7}+agent_state = client.agents.create(+ memory_blocks=[],+ model="openai/gpt-4o-mini",+ tool_exec_environment_variables={+ "API_KEY": "your-api-key-here"+ }+)+```+```typescript TypeScript {5-7}+const agentState = await client.agents.create({+ memoryBlocks: [],+ model: "openai/gpt-4o-mini",+ toolExecEnvironmentVariables: {+ "API_KEY": "your-api-key-here"+ }+});+```+</CodeGroup>diff --git a/fern/pages/evals/cli/commands.mdx b/fern/pages/evals/cli/commands.mdxnew file mode 100644index 000000000..42b6b9a11--- /dev/null+++ b/fern/pages/evals/cli/commands.mdx@@ -0,0 +1,342 @@+# CLI Commands++The **letta-evals** command-line interface lets you run evaluations, validate configurations, and inspect available components.++<Note>+**Quick overview:**+- **`run`** - Execute an evaluation suite (most common)+- **`validate`** - Check suite configuration without running+- **`list-extractors`** - Show available extractors+- **`list-graders`** - Show available grader functions+- **Exit codes** - 0 for pass, 1 for fail (perfect for CI/CD)+</Note>++**Typical workflow:**+1. Validate your suite: `letta-evals validate suite.yaml`+2. Run evaluation: `letta-evals run suite.yaml --output results/`+3. Check exit code: `echo $?` (0 = passed, 1 = failed)++## run++Run an evaluation suite.++```bash+letta-evals run <suite.yaml> [options]+```++### Arguments++- `suite.yaml`: Path to the suite configuration file (required)++### Options++#### --output, -o+Save results to a directory.++```bash+letta-evals run suite.yaml --output results/+```++Creates:+- `results/header.json`: Evaluation metadata+- `results/summary.json`: Aggregate metrics and configuration+- `results/results.jsonl`: Per-sample results (one JSON per line)++#### --quiet, -q+Quiet mode - only show pass/fail result.++```bash+letta-evals run suite.yaml --quiet+```++Output:+```+✓ PASSED+```++#### --max-concurrent+Maximum concurrent sample evaluations. **Default**: 15++```bash+letta-evals run suite.yaml --max-concurrent 10+```++Higher values = faster evaluation but more resource usage.++#### --api-key+Letta API key (overrides LETTA_API_KEY environment variable).++```bash+letta-evals run suite.yaml --api-key your-key+```++#### --base-url+Letta server base URL (overrides suite config and environment variable).++```bash+letta-evals run suite.yaml --base-url https://api.letta.com+```++#### --project-id+Letta project ID for cloud deployments.++```bash+letta-evals run suite.yaml --project-id proj_abc123+```++#### --cached, -c+Path to cached results (JSONL) for re-grading trajectories without re-running the agent.++```bash+letta-evals run suite.yaml --cached previous_results.jsonl+```++Use this to test different graders on the same agent trajectories.++#### --num-runs+Run the evaluation multiple times to measure consistency. **Default**: 1++```bash+letta-evals run suite.yaml --num-runs 10+```++**Output with multiple runs:**+- Each run creates a separate `run_N/` directory with individual results+- An `aggregate_stats.json` file contains statistics across all runs (mean, standard deviation, pass rate)++### Examples++Basic run:+```bash+letta-evals run suite.yaml # Run evaluation, show results in terminal+```++Save results:+```bash+letta-evals run suite.yaml --output evaluation-results/ # Save to directory+```++Letta Cloud:+```bash+letta-evals run suite.yaml \+ --base-url https://api.letta.com \+ --api-key $LETTA_API_KEY \+ --project-id proj_abc123+```++Quiet CI mode:+```bash+letta-evals run suite.yaml --quiet+if [ $? -eq 0 ]; then+ echo "Evaluation passed"+else+ echo "Evaluation failed"+ exit 1+fi+```++### Exit Codes++- `0`: Evaluation passed (gate criteria met)+- `1`: Evaluation failed (gate criteria not met or error)++## validate++Validate a suite configuration without running it.++```bash+letta-evals validate <suite.yaml>+```++Checks:+- YAML syntax is valid+- Required fields are present+- Paths exist+- Configuration is consistent+- Grader/extractor combinations are valid++Output on success:+```+✓ Suite configuration is valid+```++Output on error:+```+✗ Validation failed:+ - Agent file not found: agent.af+ - Grader 'my_metric' references unknown function+```++## list-extractors++List all available extractors.++```bash+letta-evals list-extractors+```++Output:+```+Available extractors:+ last_assistant - Extract the last assistant message+ first_assistant - Extract the first assistant message+ all_assistant - Concatenate all assistant messages+ pattern - Extract content matching regex+ tool_arguments - Extract tool call arguments+ tool_output - Extract tool return value+ after_marker - Extract content after a marker+ memory_block - Extract from memory block (requires agent_state)+```++## list-graders++List all available grader functions.++```bash+letta-evals list-graders+```++Output:+```+Available graders:+ exact_match - Exact string match with ground_truth+ contains - Check if contains ground_truth+ regex_match - Match regex pattern+ ascii_printable_only - Validate ASCII-only content+```++## help++Show help information.++```bash+letta-evals --help+```++Show help for a specific command:++```bash+letta-evals run --help+letta-evals validate --help+```++## Environment Variables++### LETTA_API_KEY+API key for Letta authentication.++```bash+export LETTA_API_KEY=your-key-here+```++### LETTA_BASE_URL+Letta server base URL.++```bash+export LETTA_BASE_URL=https://api.letta.com+```++### LETTA_PROJECT_ID+Letta project ID (for cloud).++```bash+export LETTA_PROJECT_ID=proj_abc123+```++### OPENAI_API_KEY+OpenAI API key (for rubric graders).++```bash+export OPENAI_API_KEY=your-openai-key+```++## Configuration Priority++Configuration values are resolved in this order (highest to lowest priority):++1. CLI arguments (`--api-key`, `--base-url`, `--project-id`)+2. Suite YAML configuration+3. Environment variables++## Using in CI/CD++### GitHub Actions++```yaml+name: Run Evals+on: [push]++jobs:+ evaluate:+ runs-on: ubuntu-latest+ steps:+ - uses: actions/checkout@v2++ - name: Install dependencies+ run: pip install letta-evals++ - name: Run evaluation+ env:+ LETTA_API_KEY: ${{ secrets.LETTA_API_KEY }}+ OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}+ run: |+ letta-evals run suite.yaml --quiet --output results/++ - name: Upload results+ uses: actions/upload-artifact@v2+ with:+ name: eval-results+ path: results/+```++### GitLab CI++```yaml+evaluate:+ script:+ - pip install letta-evals+ - letta-evals run suite.yaml --quiet --output results/+ artifacts:+ paths:+ - results/+ variables:+ LETTA_API_KEY: $LETTA_API_KEY+ OPENAI_API_KEY: $OPENAI_API_KEY+```++## Debugging++### Common Issues++<Warning>+**"Agent file not found"**++```bash+# Check file exists relative to suite YAML location+ls -la path/to/agent.af+```+</Warning>++<Warning>+**"Connection refused"**++```bash+# Verify Letta server is running+curl https://api.letta.com/v1/health+```+</Warning>++<Warning>+**"Invalid API key"**++```bash+# Check environment variable is set+echo $LETTA_API_KEY+```+</Warning>++## Next Steps++- [Understanding Results](/evals/results-metrics/understanding-results) - Interpreting evaluation output+- [Suite YAML Reference](/evals/configuration/suite-yaml-reference) - Complete configuration options+- [Getting Started](/evals/get-started/getting-started) - Complete tutorial with examplesdiff --git a/fern/pages/evals/concepts/suites.mdx b/fern/pages/evals/concepts/suites.mdxnew file mode 100644index 000000000..f86ee0696--- /dev/null+++ b/fern/pages/evals/concepts/suites.mdx@@ -0,0 +1,275 @@+# Suites++A **suite** is a YAML configuration file that defines a complete evaluation specification. It's the central piece that ties together your dataset, target agent, grading criteria, and pass/fail thresholds.++<Note>+**Quick overview:**+- **Single file defines everything**: Dataset, agent, graders, and success criteria all in one YAML+- **Reusable and shareable**: Version control your evaluation specs alongside your code+- **Multi-metric support**: Evaluate multiple aspects (accuracy, quality, tool usage) in one run+- **Multi-model testing**: Run the same suite across different LLM models+- **Flexible filtering**: Test subsets using tags or sample limits+</Note>++**Typical workflow:**+1. Create a suite YAML defining what and how to test+2. Run `letta-evals run suite.yaml`+3. Review results showing scores for each metric+4. Suite passes or fails based on gate criteria++An evaluation suite is a YAML configuration file that defines a complete test specification.++## Basic Structure++```yaml+name: my-evaluation # Suite identifier+description: Optional description of what this tests # Human-readable explanation+dataset: path/to/dataset.jsonl # Test cases++target: # What agent to evaluate+ kind: agent+ agent_file: agent.af # Agent to test+ base_url: https://api.letta.com # Letta server++graders: # How to evaluate responses+ my_metric:+ kind: tool # Deterministic grading+ function: exact_match # Grading function+ extractor: last_assistant # What to extract from agent response++gate: # Pass/fail criteria+ metric_key: my_metric # Which metric to check+ op: gte # Greater than or equal+ value: 0.8 # 80% threshold+```++## Required Fields++### name+The name of your evaluation suite. Used in output and results.++```yaml+name: question-answering-eval+```++### dataset+Path to the JSONL or CSV dataset file. Can be relative (to the suite YAML) or absolute.++```yaml+dataset: ./datasets/qa.jsonl # Relative to suite YAML location+```++### target+Specifies what agent to evaluate. See [Targets](/evals/core-concepts/targets) for details.++### graders+One or more graders to evaluate agent performance. See [Graders](/evals/core-concepts/graders) for details.++### gate+Pass/fail criteria. See [Gates](/evals/core-concepts/gates) for details.++## Optional Fields++### description+A human-readable description of what this suite tests:++```yaml+description: Tests the agent's ability to answer factual questions accurately+```++### max_samples+Limit the number of samples to evaluate (useful for quick tests):++```yaml+max_samples: 10 # Only evaluate first 10 samples+```++### sample_tags+Filter samples by tags (only evaluate samples with these tags):++```yaml+sample_tags: [math, easy] # Only samples tagged with "math" AND "easy"+```++Dataset samples can include tags:+```jsonl+{"input": "What is 2+2?", "ground_truth": "4", "tags": ["math", "easy"]}+```++### num_runs+Number of times to run the entire evaluation suite (useful for testing non-deterministic behavior):++```yaml+num_runs: 5 # Run the evaluation 5 times+```++Default: 1++### setup_script+Path to a Python script with a setup function to run before evaluation:++```yaml+setup_script: setup.py:prepare_environment # script.py:function_name+```++The setup function should have this signature:+```python+def prepare_environment(suite: SuiteSpec) -> None:+ # Setup code here+ pass+```++## Path Resolution++Paths in the suite YAML are resolved relative to the YAML file location:++```+project/+├── suite.yaml+├── dataset.jsonl+└── agents/+ └── my_agent.af+```++```yaml+# In suite.yaml+dataset: dataset.jsonl # Resolves to project/dataset.jsonl+target:+ agent_file: agents/my_agent.af # Resolves to project/agents/my_agent.af+```++Absolute paths are used as-is.++## Multi-Grader Suites++You can evaluate multiple metrics in one suite:++```yaml+graders:+ accuracy: # Check if answer is correct+ kind: tool+ function: exact_match+ extractor: last_assistant++ completeness: # LLM judges response quality+ kind: rubric+ prompt_path: rubrics/completeness.txt+ model: gpt-4o-mini+ extractor: last_assistant++ tool_usage: # Verify correct tool was called+ kind: tool+ function: contains+ extractor: tool_arguments # Extract tool call arguments+```++The gate can check any of these metrics:++```yaml+gate:+ metric_key: accuracy # Gate on accuracy metric (others still computed)+ op: gte # Greater than or equal+ value: 0.9 # 90% threshold+```++Results will include scores for all graders, even if you only gate on one.++## Examples++### Simple Tool Grader Suite++```yaml+name: basic-qa # Suite name+dataset: questions.jsonl # Test questions++target:+ kind: agent+ agent_file: qa_agent.af # Pre-configured agent+ base_url: https://api.letta.com # Local server++graders:+ accuracy: # Single metric+ kind: tool # Deterministic grading+ function: contains # Check if ground truth is in response+ extractor: last_assistant # Use final agent message++gate:+ metric_key: accuracy # Gate on this metric+ op: gte # Must be >=+ value: 0.75 # 75% to pass+```++### Rubric Grader Suite++```yaml+name: quality-eval # Quality evaluation+dataset: prompts.jsonl # Test prompts++target:+ kind: agent+ agent_id: existing-agent-123 # Use existing agent+ base_url: https://api.letta.com # Letta Cloud++graders:+ quality: # LLM-as-judge metric+ kind: rubric # Subjective evaluation+ prompt_path: quality_rubric.txt # Rubric template+ model: gpt-4o-mini # Judge model+ temperature: 0.0 # Deterministic+ extractor: last_assistant # Evaluate final response++gate:+ metric_key: quality # Gate on this metric+ metric: avg_score # Use average score+ op: gte # Must be >=+ value: 0.7 # 70% to pass+```++### Multi-Model Suite++Test the same agent configuration across different models:++```yaml+name: model-comparison # Compare model performance+dataset: test.jsonl # Same test for all models++target:+ kind: agent+ agent_file: agent.af # Same agent configuration+ base_url: https://api.letta.com # Local server+ model_configs: [gpt-4o-mini, claude-3-5-sonnet] # Test both models++graders:+ accuracy: # Single metric for comparison+ kind: tool+ function: exact_match+ extractor: last_assistant++gate:+ metric_key: accuracy # Both models must pass this+ op: gte # Must be >=+ value: 0.8 # 80% threshold+```++Results will show per-model metrics.++## Validation++Validate your suite configuration before running:++```bash+letta-evals validate suite.yaml+```++This checks:+- Required fields are present+- Paths exist+- Configuration is valid+- Grader/extractor combinations are compatible++## Next Steps++- [Dataset Configuration](/evals/core-concepts/datasets)+- [Target Configuration](/evals/core-concepts/targets)+- [Grader Configuration](/evals/core-concepts/graders)+- [Gate Configuration](/evals/core-concepts/gates)diff --git a/fern/pages/evals/concepts/targets.mdx b/fern/pages/evals/concepts/targets.mdxnew file mode 100644index 000000000..5c90d4288--- /dev/null+++ b/fern/pages/evals/concepts/targets.mdx@@ -0,0 +1,329 @@+# Targets++A **target** is the agent you're evaluating. In Letta Evals, the target configuration determines how agents are created, accessed, and tested.++<Note>+**Quick overview:**+- **Three ways to specify agents**: agent file (`.af`), existing agent ID, or programmatic creation script+- **Critical distinction**: `agent_file`/`agent_script` create fresh agents per sample (isolated tests), while `agent_id` uses one agent for all samples (stateful conversation)+- **Multi-model support**: Test the same agent configuration across different LLM models+- **Flexible connection**: Connect to local Letta servers or Letta Cloud+</Note>++**When to use each approach:**+- `agent_file` - Pre-configured agents saved as `.af` files (most common)+- `agent_id` - Testing existing agents or multi-turn conversations with state+- `agent_script` - Dynamic agent creation with per-sample customization++The target configuration specifies how to create or access the agent for evaluation.++## Target Configuration++All targets have a `kind` field (currently only `agent` is supported):++```yaml+target:+ kind: agent # Currently only "agent" is supported+ # ... agent-specific configuration+```++## Agent Sources++You must specify exactly ONE of these:++### agent_file++Path to a `.af` (Agent File) to upload:++```yaml+target:+ kind: agent+ agent_file: path/to/agent.af # Path to .af file+ base_url: https://api.letta.com # Letta server URL+```++The agent file will be uploaded to the Letta server and a new agent created for the evaluation.++### agent_id++ID of an existing agent on the server:++```yaml+target:+ kind: agent+ agent_id: agent-123-abc # ID of existing agent+ base_url: https://api.letta.com # Letta server URL+```++<Warning>+**Modifies agent in-place:** Using `agent_id` will modify your agent's state, memory, and message history during evaluation. The same agent instance is used for all samples, processing them sequentially. **Do not use production agents or agents you don't want to modify.** Use `agent_file` or `agent_script` for reproducible, isolated testing.+</Warning>++### agent_script++Path to a Python script with an agent factory function for programmatic agent creation:++```yaml+target:+ kind: agent+ agent_script: create_agent.py:create_inventory_agent # script.py:function_name+ base_url: https://api.letta.com # Letta server URL+```++Format: `path/to/script.py:function_name`++The function must be decorated with `@agent_factory` and have the signature `async (client: AsyncLetta, sample: Sample) -> str`:++```python+from letta_client import AsyncLetta, CreateBlock+from letta_evals.decorators import agent_factory+from letta_evals.models import Sample++@agent_factory+async def create_inventory_agent(client: AsyncLetta, sample: Sample) -> str:+ """Create and return agent ID for this sample."""+ # Access custom arguments from the dataset+ item = sample.agent_args.get("item", {})++ # Create agent with sample-specific configuration+ agent = await client.agents.create(+ name="inventory-assistant",+ memory_blocks=[+ CreateBlock(+ label="item_context",+ value=f"Item: {item.get('name', 'Unknown')}"+ )+ ],+ agent_type="letta_v1_agent",+ model="openai/gpt-4.1-mini",+ embedding="openai/text-embedding-3-small",+ )++ return agent.id+```++**Key features:**+- Creates a fresh agent for each sample+- Can customize agents using `sample.agent_args` from the dataset+- Allows testing agent creation logic itself+- Useful when you don't have pre-saved agent files++**When to use:**+- Testing agent creation workflows+- Dynamic per-sample agent configuration+- Agents that need sample-specific memory or tools+- Programmatic agent testing++## Connection Configuration++### base_url++Letta server URL:++```yaml+target:+ base_url: https://api.letta.com # Local Letta server+ # or+ base_url: https://api.letta.com # Letta Cloud+```++Default: `https://api.letta.com`++### api_key++API key for authentication (required for Letta Cloud):++```yaml+target:+ api_key: your-api-key-here # Required for Letta Cloud+```++Or set via environment variable:+```bash+export LETTA_API_KEY=your-api-key-here+```++### project_id++Letta project ID (for Letta Cloud):++```yaml+target:+ project_id: proj_abc123 # Letta Cloud project+```++Or set via environment variable:+```bash+export LETTA_PROJECT_ID=proj_abc123+```++### timeout++Request timeout in seconds:++```yaml+target:+ timeout: 300.0 # Request timeout (5 minutes)+```++Default: 300 seconds++## Multi-Model Evaluation++Test the same agent across different models:++### model_configs++List of model configuration names from JSON files:++```yaml+target:+ kind: agent+ agent_file: agent.af+ model_configs: [gpt-4o-mini, claude-3-5-sonnet] # Test with both models+```++The evaluation will run once for each model config. Model configs are JSON files in `letta_evals/llm_model_configs/`.++### model_handles++List of model handles (cloud-compatible identifiers):++```yaml+target:+ kind: agent+ agent_file: agent.af+ model_handles: ["openai/gpt-4o-mini", "anthropic/claude-3-5-sonnet"] # Cloud model identifiers+```++Use this for Letta Cloud deployments.++<Warning>+**Note**: You cannot specify both `model_configs` and `model_handles`.+</Warning>++## Complete Examples++### Local Development++```yaml+target:+ kind: agent+ agent_file: ./agents/my_agent.af # Pre-configured agent+ base_url: https://api.letta.com # Local server+```++### Letta Cloud++```yaml+target:+ kind: agent+ agent_id: agent-cloud-123 # Existing cloud agent+ base_url: https://api.letta.com # Letta Cloud+ api_key: ${LETTA_API_KEY} # From environment variable+ project_id: proj_abc # Your project ID+```++### Multi-Model Testing++```yaml+target:+ kind: agent+ agent_file: agent.af # Same agent configuration+ base_url: https://api.letta.com # Local server+ model_configs: [gpt-4o-mini, gpt-4o, claude-3-5-sonnet] # Test 3 models+```++Results will include per-model metrics:+```+Model: gpt-4o-mini - Avg: 0.85, Pass: 85.0%+Model: gpt-4o - Avg: 0.92, Pass: 92.0%+Model: claude-3-5-sonnet - Avg: 0.88, Pass: 88.0%+```++### Programmatic Agent Creation++```yaml+target:+ kind: agent+ agent_script: setup.py:CustomAgentFactory # Programmatic creation+ base_url: https://api.letta.com # Local server+```++## Environment Variable Precedence++Configuration values are resolved in this order (highest priority first):++1. CLI arguments (`--api-key`, `--base-url`, `--project-id`)+2. Suite YAML configuration+3. Environment variables (`LETTA_API_KEY`, `LETTA_BASE_URL`, `LETTA_PROJECT_ID`)++## Agent Lifecycle and Testing Behavior++The way your agent is specified fundamentally changes how the evaluation runs:++### With agent_file or agent_script: Independent Testing++**Agent lifecycle:**+1. A fresh agent instance is created for each sample+2. Agent processes the sample input(s)+3. Agent remains on the server after the sample completes++**Testing behavior:** Each sample is an independent, isolated test. Agent state (memory, message history) does not carry over between samples. This enables parallel execution and ensures reproducible results.++**Use cases:**+- Testing how the agent responds to various independent inputs+- Ensuring consistent behavior across different scenarios+- Regression testing where each case should be isolated+- Evaluating agent responses without prior context++<Note>+**Example:** If you have 10 test cases, 10 separate agent instances will be created (one per test case), and they can run in parallel.+</Note>++### With agent_id: Sequential Script Testing++**Agent lifecycle:**+1. The same agent instance is used for all samples+2. Agent processes each sample in sequence+3. Agent state persists throughout the entire evaluation++**Testing behavior:** The dataset becomes a conversation script where each sample builds on previous ones. Agent memory and message history accumulate, and earlier interactions affect later responses. Samples must execute sequentially.++**Use cases:**+- Testing multi-turn conversations with context+- Evaluating how agent memory evolves over time+- Simulating a single user session with multiple interactions+- Testing scenarios where context should accumulate++<Note>+**Example:** If you have 10 test cases, they all run against the same agent instance in order, with state carrying over between each test.+</Note>++### Critical Differences++| Aspect | agent_file / agent_script | agent_id |+|--------|---------------------------|----------|+| **Agent instances** | New agent per sample | Same agent for all samples |+| **State isolation** | Fully isolated | State carries over |+| **Execution** | Can run in parallel | Must run sequentially |+| **Memory** | Fresh for each sample | Accumulates across samples |+| **Use case** | Independent test cases | Conversation scripts |+| **Reproducibility** | Highly reproducible | Depends on execution order |++<Tip>+**Best practice:** Use `agent_file` or `agent_script` for most evaluations to ensure reproducible, isolated tests. Use `agent_id` only when you specifically need to test how agent state evolves across multiple interactions.+</Tip>++## Validation++The runner validates:+- Exactly one of `agent_file`, `agent_id`, or `agent_script` is specified+- Agent files have `.af` extension+- Agent script paths are valid++## Next Steps++- [Suite YAML Reference](/evals/configuration/suite-yaml-reference) - Complete target configuration options+- [Datasets](/evals/core-concepts/datasets) - Using agent_args for sample-specific configuration+- [Getting Started](/evals/get-started/getting-started) - Complete tutorial with target examplesdiff --git a/fern/pages/evals/configuration/suite-yaml.mdx b/fern/pages/evals/configuration/suite-yaml.mdxnew file mode 100644index 000000000..2e74d3957--- /dev/null+++ b/fern/pages/evals/configuration/suite-yaml.mdx@@ -0,0 +1,427 @@+# Suite YAML Reference++Complete reference for suite configuration files.++A **suite** is a YAML file that defines an evaluation: what agent to test, what dataset to use, how to grade responses, and what criteria determine pass/fail. This is your evaluation specification.++<Note>+**Quick overview:**+- **name**: Identifier for your evaluation+- **dataset**: JSONL file with test cases+- **target**: Which agent to evaluate (via file, ID, or script)+- **graders**: How to score responses (tool or rubric graders)+- **gate**: Pass/fail criteria+</Note>++See [Getting Started](/evals/get-started/getting-started) for a tutorial, or [Core Concepts](/evals/core-concepts/suites) for conceptual overview.++## File Structure++```yaml+name: string (required)+description: string (optional)+dataset: path (required)+max_samples: integer (optional)+sample_tags: array (optional)+num_runs: integer (optional)+setup_script: string (optional)++target: object (required)+ kind: "agent"+ base_url: string+ api_key: string+ timeout: float+ project_id: string+ agent_id: string (one of: agent_id, agent_file, agent_script)+ agent_file: path+ agent_script: string+ model_configs: array+ model_handles: array++graders: object (required)+ <metric_key>: object+ kind: "tool" | "rubric"+ display_name: string+ extractor: string+ extractor_config: object+ # Tool grader fields+ function: string+ # Rubric grader fields (LLM API)+ prompt: string+ prompt_path: path+ model: string+ temperature: float+ provider: string+ max_retries: integer+ timeout: float+ rubric_vars: array+ # Rubric grader fields (agent-as-judge)+ agent_file: path+ judge_tool_name: string++gate: object (required)+ metric_key: string+ metric: "avg_score" | "accuracy"+ op: "gte" | "gt" | "lte" | "lt" | "eq"+ value: float+ pass_op: "gte" | "gt" | "lte" | "lt" | "eq"+ pass_value: float+```++## Top-Level Fields++### name (required)++Suite name, used in output and results.++**Type**: string++```yaml+name: question-answering-eval+```++### description (optional)++Human-readable description of what the suite tests.++**Type**: string++```yaml+description: Tests agent's ability to answer factual questions accurately+```++### dataset (required)++Path to JSONL dataset file. Relative paths are resolved from the suite YAML location.++**Type**: path (string)++```yaml+dataset: ./datasets/qa.jsonl+dataset: /absolute/path/to/dataset.jsonl+```++### max_samples (optional)++Limit the number of samples to evaluate. Useful for quick tests.++**Type**: integer | **Default**: All samples++```yaml+max_samples: 10 # Only evaluate first 10 samples+```++### sample_tags (optional)++Filter samples by tags. Only samples with ALL specified tags are evaluated.++**Type**: array of strings++```yaml+sample_tags: [math, easy] # Only samples tagged with both+```++### num_runs (optional)++Number of times to run the evaluation suite.++**Type**: integer | **Default**: 1++```yaml+num_runs: 5 # Run the evaluation 5 times+```++### setup_script (optional)++Path to Python script with setup function.++**Type**: string (format: `path/to/script.py:function_name`)++```yaml+setup_script: setup.py:prepare_environment+```++## target (required)++Configuration for the agent being evaluated.++### kind (required)++Type of target. Currently only `"agent"` is supported.++```yaml+target:+ kind: agent+```++### base_url (optional)++Letta server URL. **Default**: `https://api.letta.com`++```yaml+target:+ base_url: https://api.letta.com+ # or+ base_url: https://api.letta.com+```++### api_key (optional)++API key for Letta authentication. Can also be set via `LETTA_API_KEY` environment variable.++```yaml+target:+ api_key: your-api-key-here+```++### timeout (optional)++Request timeout in seconds. **Default**: 300.0++```yaml+target:+ timeout: 600.0 # 10 minutes+```++### Agent Source (required, pick one)++Exactly one of these must be specified:++#### agent_id++ID of existing agent on the server.++```yaml+target:+ agent_id: agent-123-abc+```++#### agent_file++Path to `.af` agent file.++```yaml+target:+ agent_file: ./agents/my_agent.af+```++#### agent_script++Path to Python script with agent factory.++```yaml+target:+ agent_script: factory.py:MyAgentFactory+```++See [Targets](/evals/core-concepts/targets) for details on agent sources.++### model_configs (optional)++List of model configuration names to test. Cannot be used with `model_handles`.++```yaml+target:+ model_configs: [gpt-4o-mini, claude-3-5-sonnet]+```++### model_handles (optional)++List of model handles for cloud deployments. Cannot be used with `model_configs`.++```yaml+target:+ model_handles: ["openai/gpt-4o-mini", "anthropic/claude-3-5-sonnet"]+```++## graders (required)++One or more graders, each with a unique key.++### kind (required)++Grader type: `"tool"` or `"rubric"`.++```yaml+graders:+ my_metric:+ kind: tool+```++### extractor (required)++Name of the extractor to use.++```yaml+graders:+ my_metric:+ extractor: last_assistant+```++### Tool Grader Fields++#### function (required for tool graders)++Name of the grading function.++```yaml+graders:+ accuracy:+ kind: tool+ function: exact_match+```++### Rubric Grader Fields++#### prompt or prompt_path (required)++Inline rubric prompt or path to rubric file.++```yaml+graders:+ quality:+ kind: rubric+ prompt: |+ Evaluate response quality from 0.0 to 1.0.+```++#### model (optional)++LLM model for judging. **Default**: `gpt-4o-mini`++```yaml+graders:+ quality:+ kind: rubric+ model: gpt-4o+```++#### temperature (optional)++Temperature for LLM generation. **Default**: 0.0++```yaml+graders:+ quality:+ kind: rubric+ temperature: 0.0+```++#### agent_file (agent-as-judge)++Path to `.af` agent file to use as judge.++```yaml+graders:+ agent_judge:+ kind: rubric+ agent_file: judge.af+ prompt_path: rubric.txt+```++## gate (required)++Pass/fail criteria for the evaluation.++### metric_key (optional)++Which grader to evaluate. If only one grader, this can be omitted.++```yaml+gate:+ metric_key: accuracy+```++### metric (optional)++Which aggregate to compare: `avg_score` or `accuracy`. **Default**: `avg_score`++```yaml+gate:+ metric: avg_score+```++### op (required)++Comparison operator: `gte`, `gt`, `lte`, `lt`, `eq`++```yaml+gate:+ op: gte # Greater than or equal+```++### value (required)++Threshold value for comparison (0.0 to 1.0).++```yaml+gate:+ value: 0.8 # Require >= 0.8+```++## Complete Examples++### Minimal Suite++```yaml+name: basic-eval+dataset: dataset.jsonl++target:+ kind: agent+ agent_file: agent.af++graders:+ accuracy:+ kind: tool+ function: exact_match+ extractor: last_assistant++gate:+ op: gte+ value: 0.8+```++### Multi-Metric Suite++```yaml+name: comprehensive-eval+description: Tests accuracy and quality+dataset: test_data.jsonl++target:+ kind: agent+ agent_file: agent.af++graders:+ accuracy:+ kind: tool+ function: contains+ extractor: last_assistant++ quality:+ kind: rubric+ prompt_path: rubrics/quality.txt+ model: gpt-4o-mini+ extractor: last_assistant++gate:+ metric_key: accuracy+ op: gte+ value: 0.85+```++## Validation++Validate your suite before running:++```bash+letta-evals validate suite.yaml+```++## Next Steps++- [Targets](/evals/core-concepts/targets) - Understanding agent sources and configuration+- [Graders](/evals/core-concepts/graders) - Tool graders vs rubric graders+- [Extractors](/evals/core-concepts/extractors) - What to extract from agent responses+- [Gates](/evals/core-concepts/gates) - Setting pass/fail criteriadiff --git a/fern/pages/evals/getting-started.mdx b/fern/pages/evals/getting-started.mdxindex 09c6422b5..76f8b4373 100644--- a/fern/pages/evals/getting-started.mdx+++ b/fern/pages/evals/getting-started.mdx@@ -28,10 +28,8 @@ Export an existing agent to a file using the Letta SDK:from letta_client import Lettaimport os-client = Letta(- base_url="http://localhost:8283", # or https://api.letta.com for Letta Cloud- token=os.getenv("LETTA_API_KEY") # required for Letta Cloud-)+# Connect to Letta Cloud+client = Letta(token=os.getenv("LETTA_API_KEY"))# Export an agent to a fileagent_file = client.agents.export_file(agent_id="agent-123")@@ -90,7 +88,8 @@ dataset: dataset.jsonltarget:kind: agentagent_file: my_agent.af # Path to your agent file- base_url: http://localhost:8283 # Your Letta server+ base_url: https://api.letta.com # Letta Cloud (default)+ token: ${LETTA_API_KEY} # Your API keygraders:quality:diff --git a/fern/pages/evals/troubleshooting.mdx b/fern/pages/evals/troubleshooting.mdxnew file mode 100644index 000000000..7af7b7825--- /dev/null+++ b/fern/pages/evals/troubleshooting.mdx@@ -0,0 +1,267 @@+# Troubleshooting++Common issues and solutions when using Letta Evals.++## Installation Issues++<Warning>+**"Command not found: letta-evals"**++**Problem**: CLI not available after installation++**Solution**:+```bash+# Verify installation+pip list | grep letta-evals++# Reinstall if needed+pip install --upgrade letta-evals+```+</Warning>++<Warning>+**Import errors**++**Problem**: `ModuleNotFoundError: No module named 'letta_evals'`++**Solution**:+```bash+# Ensure you're in the right environment+which python++# Install in correct environment+source .venv/bin/activate+pip install letta-evals+```+</Warning>++## Configuration Issues++<Warning>+**"Agent file not found"**++**Problem**: `FileNotFoundError: agent.af`++**Solution**:+- Check the path is correct relative to the suite YAML+- Use absolute paths if needed+- Verify file exists: `ls -la path/to/agent.af`++```yaml+# Correct relative path+target:+ agent_file: ./agents/my_agent.af+```+</Warning>++<Warning>+**"Dataset not found"**++**Problem**: Cannot load dataset file++**Solution**:+- Verify dataset path in YAML+- Check file exists: `ls -la dataset.jsonl`+- Ensure proper JSONL format (one JSON object per line)++```bash+# Validate JSONL format+cat dataset.jsonl | jq .+```+</Warning>++<Warning>+**"Validation failed: unknown function"**++**Problem**: Grader function not found++**Solution**:+```bash+# List available graders+letta-evals list-graders++# Check spelling in suite.yaml+graders:+ my_metric:+ function: exact_match # Correct+```+</Warning>++## Connection Issues++<Warning>+**"Connection refused"**++**Problem**: Cannot connect to Letta server++**Solution**:+```bash+# Verify server is running+curl https://api.letta.com/v1/health++# Check base_url in suite.yaml+target:+ base_url: https://api.letta.com+```+</Warning>++<Warning>+**"Unauthorized" or "Invalid API key"**++**Problem**: Authentication failed++**Solution**:+```bash+# Set API key+export LETTA_API_KEY=your-key-here++# Verify key is correct+echo $LETTA_API_KEY+```+</Warning>++## Runtime Issues++<Warning>+**"No ground_truth provided"**++**Problem**: Grader requires ground truth but sample doesn't have it++**Solution**:+- Add ground_truth to dataset samples:+```jsonl+{"input": "What is 2+2?", "ground_truth": "4"}+```++- Or use a grader that doesn't require ground truth:+```yaml+graders:+ quality:+ kind: rubric # Doesn't require ground_truth+ prompt_path: rubric.txt+```+</Warning>++## Performance Issues++<Tip>+**Evaluation is very slow**++**Solutions**:++1. Increase concurrency:+```bash+letta-evals run suite.yaml --max-concurrent 20+```++2. Reduce samples for testing:+```yaml+max_samples: 10 # Test with small subset first+```++3. Use tool graders instead of rubric graders:+```yaml+graders:+ accuracy:+ kind: tool # Much faster than rubric+ function: exact_match+```+</Tip>++<Tip>+**High API costs**++**Solutions**:++1. Use cheaper models:+```yaml+graders:+ quality:+ model: gpt-4o-mini # Cheaper than gpt-4o+```++2. Test with small sample first:+```yaml+max_samples: 5 # Verify before running full suite+```+</Tip>++## Results Issues++<Warning>+**"All scores are 0.0"**++**Solutions**:++1. Verify extractor is getting content+2. Check grader logic+3. Test agent manually first+</Warning>++<Warning>+**"Gates failed but scores look good"**++**Solution**:+- Check gate configuration:+```yaml+gate:+ metric_key: accuracy # Correct metric?+ metric: avg_score # Or accuracy?+ op: gte # Correct operator?+ value: 0.8 # Correct threshold?+```+</Warning>++## Debug Tips++### Enable verbose output++Run without `--quiet` to see detailed progress:+```bash+letta-evals run suite.yaml+```++### Examine output files++```bash+letta-evals run suite.yaml --output debug/++# Check summary+cat debug/summary.json | jq .++# Check individual results+cat debug/results.jsonl | jq .+```++### Validate configuration++```bash+letta-evals validate suite.yaml+```++### Check component availability++```bash+letta-evals list-graders+letta-evals list-extractors+```++## Getting Help++If you're still stuck:++1. Check the [Getting Started guide](/evals/get-started/getting-started)+2. Review the [Core Concepts](/evals/core-concepts/concepts-overview)+3. Report issues at the [Letta Evals GitHub repository](https://github.com/letta-ai/letta-evals)++When reporting issues, include:+- Suite YAML configuration+- Dataset sample (if not sensitive)+- Error message and full stack trace+- Environment info (OS, Python version)++```bash+# Get environment info+python --version+pip show letta-evals+```diff --git a/fern/pages/getting-started/quickstart.mdx b/fern/pages/getting-started/quickstart.mdxnew file mode 100644index 000000000..2bb1fa6b8--- /dev/null+++ b/fern/pages/getting-started/quickstart.mdx@@ -0,0 +1,228 @@+---+title: Developer quickstart+subtitle: Create your first Letta agent with the API or SDKs and view it in the ADE+slug: quickstart+---++<Tip icon="fa-thin fa-rocket">+Programming with AI tools like Cursor? Copy our [pre-built prompts](/prompts) to get started faster.+</Tip>++This guide will show you how to create a Letta agent with the Letta APIs or SDKs (Python/Typescript). To create agents with a low-code UI, see our [ADE quickstart](/guides/ade/overview).++## Why Letta?++Unlike traditional LLM APIs where you manually manage conversation history and state, Letta agents maintain their own persistent memory. You only send new messages. The agent remembers everything from past conversations without you storing or retrieving anything. This enables agents that truly learn and evolve over time.++<Steps>+ <Step title="Prerequisites">+ 1. Create a [Letta Cloud account](https://app.letta.com)+ 2. Create a [Letta Cloud API key](https://app.letta.com/api-keys)+ <img className="w-300" src="/images/letta_cloud_api_key_gen.png" />++ 3. Set your API key as an environment variable:+ <CodeGroup>+ ```sh TypeScript+ export LETTA_API_KEY="your-api-key-here"+ ```+ ```sh Python+ export LETTA_API_KEY="your-api-key-here"+ ```+ </CodeGroup>++ <Info>+ You can also **self-host** a Letta server. Check out our [self-hosting guide](/guides/selfhosting/overview).+ </Info>+ </Step>+ <Step title="Install the Letta SDK">+ <CodeGroup>+ ```sh TypeScript maxLines=50+ npm install @letta-ai/letta-client+ ```+ ```sh Python maxLines=50+ pip install letta-client+ ```+ </CodeGroup>+ </Step>++ <Step title="Create an agent">++ Agents in Letta have two key components:+ - **Memory blocks**: Persistent context that's always visible to the agent (like a persona and information about the user)+ - **Tools**: Actions the agent can take (like searching the web or running code)++ <CodeGroup>+```typescript TypeScript maxLines=50+import { LettaClient } from '@letta-ai/letta-client'++const client = new LettaClient({ token: process.env.LETTA_API_KEY });++const agentState = await client.agents.create({+ model: "openai/gpt-4.1",+ embedding: "openai/text-embedding-3-small",+ memoryBlocks: [+ {+ label: "human",+ value: "Name: Timber. Status: dog. Occupation: building Letta, infrastructure to democratize self-improving superintelligence"+ },+ {+ label: "persona",+ value: "I am a self-improving superintelligence. Timber is my best friend and collaborator."+ }+ ],+ tools: ["web_search", "run_code"]+});++console.log(agentState.id);+```+```python Python maxLines=50+from letta_client import Letta+import os++client = Letta(token=os.getenv("LETTA_API_KEY"))++agent_state = client.agents.create(+ model="openai/gpt-4.1",+ embedding="openai/text-embedding-3-small",+ memory_blocks=[+ {+ "label": "human",+ "value": "Name: Timber. Status: dog. Occupation: building Letta, infrastructure to democratize self-improving superintelligence"+ },+ {+ "label": "persona",+ "value": "I am a self-improving superintelligence. Timber is my best friend and collaborator."+ }+ ],+ tools=["web_search", "run_code"]+)++print(agent_state.id)+```+```curl curl+curl -X POST https://api.letta.com/v1/agents \+ -H "Authorization: Bearer $LETTA_API_KEY" \+ -H "Content-Type: application/json" \+ -d '{+ "model": "openai/gpt-4.1",+ "embedding": "openai/text-embedding-3-small",+ "memory_blocks": [+ {+ "label": "human",+ "value": "Name: Timber. Status: dog. Occupation: building Letta, infrastructure to democratize self-improving superintelligence"+ },+ {+ "label": "persona",+ "value": "I am a self-improving superintelligence. Timber is my best friend and collaborator."+ }+ ],+ "tools": ["web_search", "run_code"]+}'+```+ </CodeGroup>+ </Step>+ <Step title="Message your agent">+<Note>+The Letta API supports streaming both agent *steps* and streaming *tokens*.+For more information on streaming, see [our streaming guide](/guides/agents/streaming).+</Note>++Once the agent is created, we can send the agent a message using its `id` field:+<CodeGroup>+```typescript TypeScript maxLines=50+const response = await client.agents.messages.create(+ agentState.id, {+ messages: [+ {+ role: "user",+ content: "What do you know about me?"+ }+ ]+ }+);++for (const message of response.messages) {+ console.log(message);+}+```+```python title="python" maxLines=50+response = client.agents.messages.create(+ agent_id=agent_state.id,+ messages=[+ {+ "role": "user",+ "content": "What do you know about me?"+ }+ ]+)++for message in response.messages:+ print(message)+```+```curl curl+curl --request POST \+ --url https://api.letta.com/v1/agents/$AGENT_ID/messages \+ --header 'Authorization: Bearer $LETTA_API_KEY' \+ --header 'Content-Type: application/json' \+ --data '{+ "messages": [+ {+ "role": "user",+ "content": "What do you know about me?"+ }+ ]+}'+```+</CodeGroup>++The response contains the agent's full response to the message, which includes reasoning steps (chain-of-thought), tool calls, tool responses, and assistant (agent) messages:+```json maxLines=50+{+ "messages": [+ {+ "id": "message-29d8d17e-7c50-4289-8d0e-2bab988aa01e",+ "date": "2024-12-12T17:05:56+00:00",+ "message_type": "reasoning_message",+ "reasoning": "Timber is asking what I know. I should reference my memory blocks."+ },+ {+ "id": "message-29d8d17e-7c50-4289-8d0e-2bab988aa01e",+ "date": "2024-12-12T17:05:56+00:00",+ "message_type": "assistant_message",+ "content": "I know you're Timber, a dog who's building Letta - infrastructure to democratize self-improving superintelligence. We're best friends and collaborators!"+ }+ ],+ "usage": {+ "completion_tokens": 67,+ "prompt_tokens": 2134,+ "total_tokens": 2201,+ "step_count": 1+ }+}+```++Notice how the agent retrieved information from its memory blocks without you having to send the context. This is the key difference from traditional LLM APIs where you'd need to include the full conversation history with every request.++You can read more about the response format from the message route [here](/guides/agents/overview#message-types).++ </Step>+ <Step title="View your agent in the ADE">+ Another way to interact with Letta agents is via the [Agent Development Environment](/guides/ade/overview) (or ADE for short). The ADE is a UI on top of the Letta API that allows you to quickly build, prototype, and observe your agents.++ If we navigate to our agent in the ADE, we should see our agent's state in full detail, as well as the message that we sent to it:+ <img className="block w-300 dark:hidden" src="https://raw.githubusercontent.com/letta-ai/letta/refs/heads/main/assets/example_ade_screenshot_light.png" />+ <img className="hidden w-300 dark:block" src="https://raw.githubusercontent.com/letta-ai/letta/refs/heads/main/assets/example_ade_screenshot.png" />++ [Read our ADE setup guide →](/guides/ade/setup)+ </Step>+</Steps>++++## Next steps++Congratulations! 🎉 You just created and messaged your first stateful agent with Letta using the API and SDKs. See the following resources for next steps for building more complex agents with Letta:+* Create and attach [custom tools](/guides/agents/custom-tools) to your agent+* Customize agentic [memory management](/guides/agents/memory)+* Version and distribute your agent with [agent templates](/guides/templates/overview)+* View the full [API and SDK reference](/api-reference/overview)diff --git a/fern/pages/models/anthropic.mdx b/fern/pages/models/anthropic.mdxindex a8c5c49b1..7652a2307 100644--- a/fern/pages/models/anthropic.mdx+++ b/fern/pages/models/anthropic.mdx@@ -22,20 +22,26 @@ docker run \See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.-## Configuring Anthropic models+## Specifying agent models-When creating agents, you must specify the LLM and embedding models to use. You can additionally specify a context window limit (which must be less than or equal to the maximum size). Note that Anthropic does not have embedding models, so you will need to use another provider.+When creating agents on your self-hosted server, you must specify both the LLM and embedding models to use. You can additionally specify a context window limit (which must be less than or equal to the maximum size).```pythonfrom letta_client import Letta+import os+# Connect to your self-hosted serverclient = Letta(base_url="http://localhost:8283")agent = client.agents.create(model="anthropic/claude-3-5-sonnet-20241022",- embedding="openai/text-embedding-3-small",+ embedding="openai/text-embedding-3-small", # An embedding model is required for self-hosted# optional configurationcontext_window_limit=30000)```Anthropic models have very large context windows, which will be very expensive and high latency. We recommend setting a lower `context_window_limit` when using Anthropic models.++<Note>+For Letta Cloud usage, see the [quickstart guide](/quickstart). Cloud deployments manage embeddings automatically and don't require provider configuration.+</Note>diff --git a/fern/pages/models/azure.mdx b/fern/pages/models/azure.mdxindex 7960df411..74b8c9853 100644--- a/fern/pages/models/azure.mdx+++ b/fern/pages/models/azure.mdx@@ -38,17 +38,23 @@ Optionally, you can specify the API version (default is 2024-09-01-preview):See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.## Specifying agent models-When creating agents, you must specify the LLM and embedding models to use via a *handle*. You can additionally specify a context window limit (which must be less than or equal to the maximum size).+When creating agents on your self-hosted server, you must specify both the LLM and embedding models to use via a *handle*. You can additionally specify a context window limit (which must be less than or equal to the maximum size).```pythonfrom letta_client import Letta+import os+# Connect to your self-hosted serverclient = Letta(base_url="http://localhost:8283")azure_agent = client.agents.create(model="azure/gpt-4o-mini",- embedding="azure/text-embedding-3-small",+ embedding="azure/text-embedding-3-small", # An embedding model is required for self-hosted# optional configuration- context_window_limit=16000,+ context_window_limit=16000)```++<Note>+For Letta Cloud usage, see the [quickstart guide](/quickstart). Cloud deployments manage embeddings automatically and don't require provider configuration.+</Note>diff --git a/fern/pages/models/ollama.mdx b/fern/pages/models/ollama.mdxindex f409a818c..5194d988e 100644--- a/fern/pages/models/ollama.mdx+++ b/fern/pages/models/ollama.mdx@@ -66,17 +66,23 @@ docker run \See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.## Specifying agent models-When creating agents, you must specify the LLM and embedding models to use via a *handle*. You can additionally specify a context window limit (which must be less than or equal to the maximum size).+When creating agents on your self-hosted server, you must specify both the LLM and embedding models to use via a *handle*. You can additionally specify a context window limit (which must be less than or equal to the maximum size).```pythonfrom letta_client import Letta+import os+# Connect to your self-hosted serverclient = Letta(base_url="http://localhost:8283")ollama_agent = client.agents.create(model="ollama/thewindmom/hermes-3-llama-3.1-8b:latest",- embedding="ollama/mxbai-embed-large:latest",+ embedding="ollama/mxbai-embed-large", # An embedding model is required for self-hosted# optional configuration- context_window_limit=16000,+ context_window_limit=16000)```++<Note>+For Letta Cloud usage, see the [quickstart guide](/quickstart). Cloud deployments manage embeddings automatically and don't require provider configuration.+</Note>diff --git a/fern/pages/models/openai.mdx b/fern/pages/models/openai.mdxindex 2726b2968..f1e9e43cc 100644--- a/fern/pages/models/openai.mdx+++ b/fern/pages/models/openai.mdx@@ -29,18 +29,24 @@ docker run \See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.-## Configuring OpenAI models in the Python SDK-When creating agents, you must specify the LLM and embedding models to use. You can additionally specify a context window limit (which must be less than or equal to the maximum size).+## Specifying agent models+When creating agents on your self-hosted server, you must specify both the LLM and embedding models to use via a *handle*. You can additionally specify a context window limit (which must be less than or equal to the maximum size).```pythonfrom letta_client import Letta+import os+# Connect to your self-hosted serverclient = Letta(base_url="http://localhost:8283")openai_agent = client.agents.create(model="openai/gpt-4o-mini",- embedding="openai/text-embedding-3-small",+ embedding="openai/text-embedding-3-small", # An embedding model is required for self-hosted# optional configurationcontext_window_limit=16000)```++<Note>+For Letta Cloud usage, see the [quickstart guide](/quickstart). Cloud deployments manage embeddings automatically and don't require provider configuration.+</Note>diff --git a/fern/pages/models/xai.mdx b/fern/pages/models/xai.mdxindex ac04b963f..b1d1d6e3b 100644--- a/fern/pages/models/xai.mdx+++ b/fern/pages/models/xai.mdx@@ -23,20 +23,26 @@ docker run \See the [self-hosting guide](/guides/selfhosting) for more information on running Letta with Docker.-## Configuring xAI (Grok) models+## Specifying agent models-When creating agents, you must specify the LLM and embedding models to use. You can additionally specify a context window limit (which must be less than or equal to the maximum size). Note that xAI does not have embedding models, so you will need to use another provider.+When creating agents on your self-hosted server, you must specify both the LLM and embedding models to use. You can additionally specify a context window limit (which must be less than or equal to the maximum size).```pythonfrom letta_client import Letta+import os+# Connect to your self-hosted serverclient = Letta(base_url="http://localhost:8283")agent = client.agents.create(model="xai/grok-2-1212",- embedding="openai/text-embedding-3-small",+ embedding="openai/text-embedding-3-small", # An embedding model is required for self-hosted# optional configurationcontext_window_limit=30000)```xAI (Grok) models have very large context windows, which will be very expensive and high latency. We recommend setting a lower `context_window_limit` when using xAI (Grok) models.++<Note>+For Letta Cloud usage, see the [quickstart guide](/quickstart). Cloud deployments manage embeddings automatically and don't require provider configuration.+</Note>diff --git a/fern/pages/selfhosting/overview.mdx b/fern/pages/selfhosting/overview.mdxindex d489fad82..2c6351bfb 100644--- a/fern/pages/selfhosting/overview.mdx+++ b/fern/pages/selfhosting/overview.mdx@@ -36,6 +36,11 @@ docker run \Once the Letta server is running, you can access it via port `8283` (e.g. sending REST API requests to `http://localhost:8283/v1`). You can also connect your server to the [Letta ADE](/guides/ade) to access and manage your agents in a web interface.## Enabling model providers++<Warning>+**Self-hosted servers require embedding model configuration.** Unlike Letta Cloud which manages embeddings automatically, self-hosted deployments must configure both LLM and embedding models explicitly when creating agents.+</Warning>+The Letta server can be connected to various LLM API backends ([OpenAI](https://docs.letta.com/models/openai), [Anthropic](https://docs.letta.com/models/anthropic), [vLLM](https://docs.letta.com/models/vllm), [Ollama](https://docs.letta.com/models/ollama), etc.). To enable access to these LLM API providers, set the appropriate environment variables when you use `docker run`:```sh# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data@@ -63,6 +68,71 @@ docker run \The example above will make all compatible models running on OpenAI, Anthropic, and Ollama available to your Letta server.+## Configuring embedding models++When self-hosting, you **must** specify an embedding model when creating agents. Letta uses embeddings for archival memory search and retrieval.++### Supported embedding providers++When creating agents on your self-hosted server, specify the `embedding` parameter:++<CodeGroup>+```python Python+from letta_client import Letta++# Connect to your self-hosted server+client = Letta(base_url="http://localhost:8283")++# Create agent with explicit embedding configuration+agent = client.agents.create(+ model="openai/gpt-4o-mini",+ embedding="openai/text-embedding-3-small", # Required for self-hosted+ memory_blocks=[+ {"label": "persona", "value": "I am a helpful assistant."}+ ]+)+```+```typescript TypeScript+import { LettaClient } from '@letta-ai/letta-client'++// Connect to your self-hosted server+const client = new LettaClient({+ baseUrl: "http://localhost:8283"+});++// Create agent with explicit embedding configuration+const agent = await client.agents.create({+ model: "openai/gpt-4o-mini",+ embedding: "openai/text-embedding-3-small", // Required for self-hosted+ memoryBlocks: [+ {label: "persona", value: "I am a helpful assistant."}+ ]+});+```+</CodeGroup>++### Available embedding models++The embedding model you can use depends on which provider you've configured:++**OpenAI** (requires `OPENAI_API_KEY`):+- `openai/text-embedding-3-small` (recommended)+- `openai/text-embedding-3-large`+- `openai/text-embedding-ada-002`++**Azure OpenAI** (requires Azure configuration):+- `azure/text-embedding-3-small`+- `azure/text-embedding-ada-002`++**Ollama** (requires `OLLAMA_BASE_URL`):+- `ollama/mxbai-embed-large`+- `ollama/nomic-embed-text`+- Any embedding model available in your Ollama instance++<Note>+**Letta Cloud difference:** When using Letta Cloud, the `embedding` parameter is optional and managed automatically. Self-hosted servers require explicit embedding configuration.+</Note>+## Optional: Telemetry with ClickHousediff --git a/fern/pages/voice/voice.mdx b/fern/pages/voice/voice.mdxnew file mode 100644index 000000000..0ca3aaa67--- /dev/null+++ b/fern/pages/voice/voice.mdx@@ -0,0 +1,35 @@+---+title: Voice Agents+slug: guides/voice/overview+---++<Warning>+Voice agents support is experimental and may be unstable. For more information, visit our [Discord](https://discord.gg/letta).+</Warning>++All Letta agents can be connected to a voice provider by using the OpenAI-compatible streaming chat completions endpoint at `https://api.letta.com/v1/chat/completions`. Any standard Letta agent can be used for voice applications.++<Note>+The legacy `/v1/voice-beta/<AGENT_ID>` endpoint has been deprecated. Please use the OpenAI-compatible `/v1/chat/completions` endpoint with `stream=true` for voice applications.+</Note>++## Creating a voice agent+You can create a voice agent using the standard Letta agent creation flow:++```python+from letta_client import Letta+import os++client = Letta(token=os.getenv('LETTA_API_KEY'))++# create the Letta agent+agent = client.agents.create(+ memory_blocks=[+ {"value": "Name: ?", "label": "human"},+ {"value": "You are a helpful assistant.", "label": "persona"},+ ],+ model="openai/gpt-4o-mini" # Use 4o-mini for speed+)+```++You can attach additional tools and blocks to this agent just as you would any other Letta agent.