A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.
Python artificial-intelligencedagdata-sciencedata-visualizationdataflowdeveloper-toolsmachine-learningnotebookspipelinepythonreactivesqlweb-app
docs: updated AI docs (#6065)
Myles Scolnick55f0b0f54898
diff --git a/docs/guides/editor_features/ai_completion.md b/docs/guides/editor_features/ai_completion.mdindex 7526c7fbb8..e3df2ce6d4 100644--- a/docs/guides/editor_features/ai_completion.md+++ b/docs/guides/editor_features/ai_completion.md@@ -6,7 +6,7 @@ marimo is an AI-native editor, with support for full-cell AI code generation:* refactoring existing cells from a prompt* generating entire notebooks-as well as inline copilots (like GitHub Copilot).+as well as inline autocompletion (like GitHub Copilot).marimo's AI assistant is specialized for working with data: unlike traditionalassistants that only have access to the text of your program, marimo's assistant@@ -132,7 +132,17 @@ For plotting:### Connecting to an LLMYou can connect to an LLM through the notebook settings menu, or by manually editing-your `marimo.toml` configuration file. Prefer going through the notebook settings.+your `marimo.toml` configuration file. **Prefer going through the notebook settings menu.**++You can configure the following providers:++* OpenAI+* Anthropic+* AWS Bedrock+* Google AI+* GitHub+* Ollama+* and any OpenAI-compatible providerTo locate your configuration file, run:@@ -142,6 +152,32 @@ marimo config showAt the top, the path to your `marimo.toml` file will be shown.+#### Model roles and provider routing++marimo supports three different AI model roles, each serving a specific purpose:++* **`chat_model`**: Used for the chat panel+* **`edit_model`**: Used for refactoring existing cells (Ctrl/Cmd-Shift-E) and generating new cells with the "Generate with AI" button.+* **`autocomplete_model`**: Used for inline code autocompletion++Models are specified using the format `provider/model-name`, where the provider prefix routes the request to the appropriate configuration section:++```toml title="marimo.toml"+[ai.models]+chat_model = "openai/gpt-4o-mini" # Routes to OpenAI config+edit_model = "anthropic/claude-3-sonnet" # Routes to Anthropic config+autocomplete_model = "ollama/codellama" # Routes to Ollama config+```++#### Custom models++You can also add custom models to appear in the model selection dropdown.++```toml title="marimo.toml"+[ai.models]+custom_models = ["ollama/somemodel"]+```+Below we describe how to connect marimo to your AI provider.#### OpenAI@@ -151,15 +187,20 @@ Below we describe how to connect marimo to your AI provider.2. Add the following to your `marimo.toml` (or configure in the UI settings in the editor):```toml title="marimo.toml"+[ai.models]+# Choose for chat+chat_model = "openai/gpt-4o-mini"+# Or choose for edit+edit_model = "openai/gpt-4o-mini"+[ai.open_ai]# Get your API key from https://platform.openai.com/account/api-keysapi_key = "sk-proj-..."-# Choose a model, we recommend "gpt-4-turbo"-model = "gpt-4-turbo"-# Available models: gpt-4-turbo-preview, gpt-4, gpt-3.5-turbo+# Available models: gpt-4o-mini, gpt-4o, gpt-4, gpt-3.5-turbo# See https://platform.openai.com/docs/models for all available models# Change the base_url if you are using a different OpenAI-compatible API+# different from one of the providers listed belowbase_url = "https://api.openai.com/v1"```@@ -171,8 +212,8 @@ To use Anthropic with marimo:2. Add the following to your `marimo.toml` (or configure in the UI settings in the editor):```toml title="marimo.toml"-[ai.open_ai]-model = "claude-3-7-sonnet-20250219"+[ai.models]+chat_model = "anthropic/claude-3-7-sonnet-latest"# or any model from https://docs.anthropic.com/en/docs/about-claude/models[ai.anthropic]@@ -195,14 +236,8 @@ To use AWS Bedrock with marimo:5. Add the following to your `marimo.toml`:```toml title="marimo.toml"-[ai.open_ai]-model = "bedrock/anthropic.claude-3-sonnet-20240229"-# Models are identified by bedrock/provider.model_name-# Examples:-# - bedrock/anthropic.claude-3-sonnet-20240229-# - bedrock/meta.llama3-8b-instruct-v1:0-# - bedrock/amazon.titan-text-express-v1-# - bedrock/cohere.command-r-plus-v1+[ai.models]+chat_model = "bedrock/anthropic.claude-3-sonnet-latest"[ai.bedrock]region_name = "us-east-1" # AWS region where Bedrock is available@@ -221,8 +256,8 @@ To use Google AI with marimo:3. Add the following to your `marimo.toml` (or configure in the UI settings in the editor):```toml title="marimo.toml"-[ai.open_ai]-model = "gemini-2.5-flash-preview-05-20"+[ai.models]+chat_model = "google/gemini-2.5-pro"# or any model from https://ai.google.dev/gemini-api/docs/models/gemini[ai.google]@@ -238,10 +273,11 @@ You can use your GitHub Copilot for code refactoring or the chat panel. This req3. Add the token to your `marimo.toml` (or configure in the UI settings in the editor).```toml title="marimo.toml"-[ai.open_ai]-model = "gpt-4o-mini"+[ai.models]+chat_model = "github/gpt-4o-mini"++[ai.github]api_key = "gho_..."-base_url = "https://api.githubcopilot.com/"```??? question "My token starts with `ghp_` instead of `gho_`?"@@ -251,7 +287,7 @@ base_url = "https://api.githubcopilot.com/"> bad request: Personal Access Tokens are not supported for this endpointTo resolve this issue, you could switch to an _OAuth_ access token (`gho_...`):-+1. Re-authenticate by running `gh auth login`.2. Choose _Login with a web browser_ (instead of _Paste an authentication token_) this time.@@ -292,20 +328,20 @@ Ollama allows you to run open-source LLMs on your local machine. To integrate Olmarimo edit notebook.py```-!!! warning "Important: Use the `/v1` endpoint"+??? warning "Important: Use the `/v1` endpoint"marimo requires Ollama's OpenAI-compatible API endpoint. Always ensure your `base_url` includes the `/v1` path:-+```toml- [ai.open_ai]+ [ai.ollama]base_url = "http://127.0.0.1:11434/v1" # ✅ Correct - includes /v1```-+**Common mistake:**```tomlbase_url = "http://127.0.0.1:11434" # ❌ Will cause 404 errors```-+If you encounter 404 errors, verify your model is installed with `ollama ls` and test the endpoint:```bashcurl http://127.0.0.1:11434/v1/models@@ -314,10 +350,10 @@ Ollama allows you to run open-source LLMs on your local machine. To integrate Ol7. Add the following to your `marimo.toml` (or configure in the UI settings in the editor):```toml title="marimo.toml"-[ai.open_ai]-api_key = "ollama" # This is not used, but required-model = "codellama" # or another model from `ollama ls`-base_url = "http://127.0.0.1:11434/v1"+[ai.models]+chat_model = "ollama/llama3.1:latest"+edit_model = "ollama/codellama"+autocomplete_model = "ollama/codellama" # or another model from `ollama ls````#### Other AI providers@@ -326,17 +362,6 @@ marimo supports OpenAI's API by default. Many providers offer OpenAI API-compati??? tip "Using OpenAI-compatible providers (e.g., DeepSeek)"- === "Via marimo.toml"-- Add the following configuration to your `marimo.toml` file:-- ```toml- [ai.open_ai]- api_key = "dsk-..." # Your provider's API key- model = "deepseek-chat" # or "deepseek-reasoner"- base_url = "https://api.deepseek.com/"- ```-=== "Via UI Settings"1. Open marimo's Settings panel@@ -346,6 +371,19 @@ marimo supports OpenAI's API by default. Many providers offer OpenAI API-compati- Set Base URL to your provider's endpoint (e.g., `https://api.deepseek.com`)- Set Model to your chosen model (e.g., `deepseek-chat` or `deepseek-reasoner`)+ === "Via marimo.toml"++ Add the following configuration to your `marimo.toml` file:++ ```toml+ [ai.models]+ chat_model = "deepseek/deepseek-chat" # or "deepseek-reasoner"++ [ai.open_ai_compatible]+ api_key = "dsk-..." # Your provider's API key+ base_url = "https://api.deepseek.com/"+ ```+For a comprehensive list of compatible providers and their configurations, please refer to the [liteLLM Providers documentation](https://litellm.vercel.app/docs/providers).For providers not compatible with OpenAI's API, please submit a [feature request](https://github.com/marimo-team/marimo/issues/new?template=feature_request.yaml) or "thumbs up" an existing one.@@ -365,21 +403,21 @@ marimo using `pip`/`uv` if you need Copilot._### Windsurf Copilot-[Windsurf](https://windsurf.com/) (formerly codeium) provides tab-completion tooling that can also be used from within marimo.+[Windsurf](https://windsurf.com/) (formerly codeium) provides tab-completion tooling that can also be used from within marimo.To set up Windsurf:1. Go to [windsurf.com](https://windsurf.com/) website and sign up for an account.2. Download the [Windsurf app](https://windsurf.com/download).-3. After installing Windsurf and authenticating, open up the command palette, via <kbd>cmd</kbd>+<kbd>shift</kbd>+<kbd>p</kbd>, and ask it to copy the api key to your clipboard.+3. After installing Windsurf and authenticating, open up the command palette, via <kbd>cmd</kbd>+<kbd>shift</kbd>+<kbd>p</kbd>, and ask it to copy the api key to your clipboard.-4a. Configure the UI settings in the editor to use Windsurf.+4a. Configure the UI settings in the editor to use Windsurf.-4b. Alternatively you can also configure the api key from the marimo config file.+4b. Alternatively you can also configure the api key from the marimo config file.```toml title="marimo.toml"[completion]@@ -389,7 +427,7 @@ codeium_api_key = ""### Custom copilots-marimo also supports integrating with custom LLM providers for code completion suggestions. This allows you to use your own LLM service to provide in-line code suggestions based on internal providers or local models (e.g. Ollama). You may also use OpenAI, Anthropic, or Google AI by providing your own API keys.+marimo also supports integrating with custom LLM providers for code completion suggestions. This allows you to use your own LLM service to provide in-line code suggestions based on internal providers or local models (e.g. Ollama). You may also use OpenAI, Anthropic, Google, or any other providers by providing your own API keys and configuration.To configure a custom copilot:@@ -397,15 +435,14 @@ To configure a custom copilot:2. Add the following configuration to your `marimo.toml` (or configure in the UI settings in the editor):```toml title="marimo.toml"+[ai.models]+autocomplete_model = "provider/model-name"+[completion]copilot = "custom"-api_key = "your-llm-api-key"-model = "your-llm-model-name"-base_url = "http://127.0.0.1:11434/v1" # or https://your-llm-api-endpoint.com```The configuration options include:-* `api_key`: Your LLM provider's API key. This may not be required for local models, so you can set it to any random string.-* `model`: The specific model to use for completion suggestions.-* `base_url`: The endpoint URL for your LLM provider's API+* `autocomplete_model`: The specific model to use for inline autocompletion.+* `copilot`: The name of the copilot to use for code generation.