Skip to content

Automate browser based workflows with AI

docs: add Ollama + LiteLLM quickstart and sidebar entry (#3176)

Ciobotea Alexandru-Marian21be178b2f5b
diff --git a/env.litellm.example b/env.litellm.example
new file mode 100644
index 000000000..ed0aab9bf
--- /dev/null
+++ b/env.litellm.example
@@ -0,0 +1,11 @@
+# --- LITELLM (OPENAI-COMPATIBLE) ---
+ENABLE_OPENAI_COMPATIBLE=true
+OPENAI_COMPATIBLE_MODEL_NAME=llama3.1
+OPENAI_COMPATIBLE_API_KEY=sk-test
+OPENAI_COMPATIBLE_API_BASE=http://localhost:4000/v1
+
+# --- DISABLE OTHER PROVIDERS ---
+ENABLE_OPENAI=false
+ENABLE_ANTHROPIC=false
+ENABLE_OLLAMA=false
+ENABLE_GROQ=false
diff --git a/env.ollama.example b/env.ollama.example
new file mode 100644
index 000000000..133e82b18
--- /dev/null
+++ b/env.ollama.example
@@ -0,0 +1,10 @@
+# --- OLLAMA DIRECT ---
+ENABLE_OLLAMA=true
+OLLAMA_SERVER_URL=http://localhost:11434
+OLLAMA_MODEL=llama3.1
+
+# --- DISABLE OTHER PROVIDERS ---
+ENABLE_OPENAI=false
+ENABLE_ANTHROPIC=false
+ENABLE_OPENAI_COMPATIBLE=false
+ENABLE_GROQ=false
diff --git a/fern/docs.yml b/fern/docs.yml
index 68ffbe4fa..01bd6aa97 100644
--- a/fern/docs.yml
+++ b/fern/docs.yml
@@ -151,6 +151,8 @@ navigation:
path: integrations/n8n.mdx
- page: Workato
path: integrations/workato.mdx
+ - page: Ollama + LiteLLM
+ path: integrations/ollama-litellm.mdx
- tab: api
layout:
- api: API Reference
diff --git a/fern/integrations/ollama-litellm.mdx b/fern/integrations/ollama-litellm.mdx
new file mode 100644
index 000000000..499651e9d
--- /dev/null
+++ b/fern/integrations/ollama-litellm.mdx
@@ -0,0 +1,123 @@
+---
+title: Ollama + LiteLLM
+description: Run Skyvern with a local LLM via Ollama or an OpenAI-compatible proxy via LiteLLM.
+---
+
+## Overview
+
+Skyvern can use local models via **Ollama** or any **OpenAI-compatible** endpoint (e.g., **LiteLLM**). Two paths:
+
+- **(A) Direct Ollama** — use the Ollama API (`/v1/chat/completions`)
+- **(B) OpenAI-compatible (LiteLLM)** — Skyvern connects to a proxy that exposes an OpenAI-style API
+
+---
+
+## A) Direct Ollama
+
+### 1) Start Ollama locally
+Install Ollama and run a model (example: `llama3.1`):
+```bash
+ollama pull llama3.1
+ollama serve
+```
+The API is usually at `http://localhost:11434`.
+
+### 2) Configure Skyvern (ENV)
+Add to your `.env`:
+```bash
+# Enable Ollama integration
+ENABLE_OLLAMA=true
+
+# Ollama server URL
+OLLAMA_SERVER_URL=http://localhost:11434
+
+# Model name in Ollama (check with `ollama list`)
+OLLAMA_MODEL=llama3.1
+```
+
+> Note: Ollama may not support `max_completion_tokens` — Skyvern handles this internally.
+
+---
+
+## B) OpenAI-compatible via LiteLLM
+
+### 1) Run LiteLLM as a proxy
+Minimal example (see LiteLLM docs for more options):
+```bash
+# Map an Ollama model behind an OpenAI-compatible endpoint
+litellm --model ollama/llama3.1 --host 0.0.0.0 --port 4000
+```
+
+### 2) Configure Skyvern (ENV)
+Add to your `.env`:
+```bash
+# Enable OpenAI-compatible provider
+ENABLE_OPENAI_COMPATIBLE=true
+
+# The "model" exposed by the proxy (any identifier your proxy accepts)
+OPENAI_COMPATIBLE_MODEL_NAME=llama3.1
+
+# API key required by the proxy (use any placeholder if proxy doesn't enforce)
+OPENAI_COMPATIBLE_API_KEY=sk-test
+
+# Base URL for the LiteLLM proxy (OpenAI-compatible)
+OPENAI_COMPATIBLE_API_BASE=http://localhost:4000/v1
+
+# (optional)
+# OPENAI_COMPATIBLE_API_VERSION=2024-06-01
+# OPENAI_COMPATIBLE_SUPPORTS_VISION=false
+# OPENAI_COMPATIBLE_REASONING_EFFORT=low|medium|high
+```
+
+---
+
+## Start Skyvern (local)
+
+After setting the environment variables in `.env`, start backend + UI:
+```bash
+# with Docker (recommended)
+docker compose up -d
+
+# or locally (dev):
+# poetry install
+# ./run_skyvern.sh
+# ./run_ui.sh
+```
+
+Open the UI and pick the model (or keep default); if only Ollama/LiteLLM are enabled, Skyvern will use that.
+
+---
+
+## Verify your setup
+
+Before starting Skyvern, quickly verify that your LLM endpoint is reachable.
+
+### Ollama
+```bash
+# Should return the list of local models
+curl -s http://localhost:11434/api/tags | jq .
+```
+
+### LiteLLM (OpenAI-compatible)
+```bash
+# Should return available models exposed by your proxy
+curl -s http://localhost:4000/v1/models -H "Authorization: Bearer sk-test" | jq .
+```
+
+> If your model doesn’t appear, re-check the proxy flags and your `.env` values (`OPENAI_COMPATIBLE_API_BASE`, `OPENAI_COMPATIBLE_MODEL_NAME`, etc.).
+
+---
+
+## Troubleshooting
+
+- **Model not responding / timeout**: ensure `ollama serve` is running and `OLLAMA_MODEL` exists (`ollama list`).
+- **LiteLLM 401**: set `OPENAI_COMPATIBLE_API_KEY` to a value accepted by the proxy or disable auth on the proxy.
+- **CORS / wrong base URL**: confirm `OPENAI_COMPATIBLE_API_BASE` and that it ends with `/v1`.
+- **Model not visible**: ensure `ENABLE_OLLAMA=true` or `ENABLE_OPENAI_COMPATIBLE=true` in `.env`, then restart services.
+
+---
+
+## Internal References
+
+- Ollama vars: `ENABLE_OLLAMA`, `OLLAMA_SERVER_URL`, `OLLAMA_MODEL`
+- OpenAI-compatible vars: `ENABLE_OPENAI_COMPATIBLE`, `OPENAI_COMPATIBLE_MODEL_NAME`, `OPENAI_COMPATIBLE_API_KEY`, `OPENAI_COMPATIBLE_API_BASE`, `OPENAI_COMPATIBLE_API_VERSION`, `OPENAI_COMPATIBLE_SUPPORTS_VISION`, `OPENAI_COMPATIBLE_REASONING_EFFORT`