Self-improving AI Employees for everyone
Ollama Chat Model (ollamaChatModel)
| Field | Value |
|---|---|
| Category | ai_chat_models |
| Backend handler | server/nodes/model/ollama_chat_model/__init__.py (dispatch via BaseNode.execute() -> @Operation("chat") in server/nodes/model/_base.py) |
| AI service | server/services/ai.py::AIService.execute_chat |
| Tests | server/tests/nodes/test_ai_chat_models.py |
| Skill (if any) | n/a |
| Dual-purpose tool | no (group ('model',)) |
Purpose
Run local LLMs (llama, mistral, qwen, deepseek-r1, ...) through a locally-running Ollama server. Ollama exposes an OpenAI-shaped /v1 endpoint, so the OpenAI-compatible spec registered in services/llm/providers/_compat.py hands it to OpenAIProvider — same path as deepseek/kimi/mistral. The server URL the user saves under Credentials is rooted at save time (http://localhost:11434 is stored as http://localhost:11434/v1, RFC-0003) and kept as the ollama_proxy credential; the unifier passes it as proxy_url, which wins over the base_url in llm_defaults.json. OllamaChatModelNode uses the shared ChatModelParams unchanged. The ChatModelBase.chat operation calls AIService.execute_chat.
Inputs (handles)
| Handle | Connection type | Required | Purpose |
|---|---|---|---|
input-main |
main | no | Upstream data; not consumed directly |
Parameters
| Name | Type | Default | Required | displayOptions.show | Description |
|---|---|---|---|---|---|
prompt |
string | "" |
yes | - | User message |
system_prompt |
string | "" |
no | - | System prompt |
model |
string | "" (injected) |
no | - | A model Ollama had loaded when the server was last fetched (ps() lists loaded models only), e.g. qwen2.5, llama3.x, deepseek-r1. Open-world: name not pattern-checked by is_model_valid_for_provider |
temperature |
number|null | null |
no | - | 0-2 |
max_tokens |
number|null | null |
no | - | 1-200000; default per-loaded-model ctx ÷ 4 (capped 4096) |
top_p |
number|null | 1.0 |
no | - | |
api_key |
string|null | null (injected) |
no | - | Optional; local servers usually run with no auth. Without a stored key the declared placeholder is sent (auth.placeholder_key in llm_defaults.json: "ollama", the value Ollama's docs use) |
(Ollama uses the shared ChatModelParams unchanged; field names are snake_case, unknown keys ignored.)
Outputs (handles)
| Handle | Shape | Description |
|---|---|---|
output-model |
object | Model output (also feeds an agent's input-model handle); standard envelope payload |
Output payload
{
response: string;
thinking: string | null; // per-model; Ollama has no generic thinking knob
thinking_enabled: boolean;
model: string;
provider: 'ollama';
finish_reason: string;
timestamp: string;
input: { prompt: string; system_prompt: string };
}
Wrapped in { success, node_id, node_type, result, execution_time }.
Logic Flow
flowchart TD
A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
B --> C[AIService.execute_chat]
C --> D{valid key + prompt?}
D -- no --> X[error envelope]
D -- yes --> E[detect_ai_provider -> 'ollama']
E --> F[Lookup ollama_proxy credential -> base_url override]
F --> G[ChatUnifier.chat -> registry.get_provider ollama -> OpenAIProvider<br/>base_url=resolved URL, key=stored key or placeholder]
G --> H[provider.chat]
H --> I[success envelope]
G -- Exception --> X
Decision Logic
- Validation: empty prompt -> error envelope. Once the server is saved,
api_keyis never the blocker: saving stores the user's key or the declared placeholder ("ollama") in theollamarow, so the central "API key required" check inexecute_chatpasses.OllamaCredential.resolve()falls back to the same placeholder. - Provider routing:
detect_ai_providerMUST listollama(inserver/constants.py) or the node falls through to'openai'andexecute_chathits api.openai.com with the placeholder key. - Open-world model name:
ollamadeclaresopen_world_models: trueinllm_defaults.json, sois_model_valid_for_providerdoes not reject local model names likeqwen/qwen3.6-27bwith the cloud-style pattern check. - Base URL routing: the
ollama_proxycredential carries the resolved server URL into the client; nothing is probed at call time, and traffic stays onlocalhost. - No thinking knob: the shared
thinking_enabledfield is present but generic Ollama has no per-call thinking parameter; reasoning is per-model.
Side Effects
- Database writes: per-model context params persist in
EncryptedAPIKey.models["model_params"]and inDATA_DIR/local_models.jsonat save time (via_local_validator.save_llm_server+ModelRegistryService.register_local_model()), never in the trackedconfig/model_registry.json, and not on the bare chat path. - Broadcasts: none on the bare chat path.
- External API calls:
POST {user_server}/v1/chat/completionsvia theopenaiSDK with overriddenbase_url(defaulthttp://localhost:11434/v1). - File I/O: none.
- Subprocess: none.
External Dependencies
- Credentials: optional
auth_service.get_api_key('ollama'); user server URL stored asollama_proxy. - Services:
services/llm/providers/openai.py(reused with Ollama base_url);services/llm/endpoints.py(roots the URL at save time);nodes/model/_local_validator.py(the save path; SDK probe viaollama.AsyncClient.ps()). - Python packages:
openai,ollama>=0.6.0(validation only). - Environment variables: none.
Edge cases & known limits
- Server must be running: requests go to the user's local Ollama server; if it is down the request surfaces as a connection error in the envelope.
- Saving checks the URL first: the URL entered is tried as given, then with
/v1appended, and adopted only whereGET /modelsreturns an OpenAI list. A save that finds no such list, or no loaded model, writes nothing and keeps the previous server. - Provider routing dependency:
ollamamust be present indetect_ai_provider, or the node silently falls back to OpenAI cloud. Agents need no edit: theirproviderfield lists every registered provider through theaiProvidersloader. - Max output default: ctx ÷ 4, capped at 4096, unless the user overrides
max_tokens. - Per-model params survive restart: written through to
DATA_DIR/local_models.json, which an OpenRouter refresh does not touch, so context length is known without re-clicking Fetch. - Error boundary: typed OpenAI SDK connection/API failures become user-safe
NodeUserErrorvalues inChatUnifierand are re-raised toBaseNode.execute(), which produces the standard failure envelope. Unexpected failures are logged and returned byexecute_chat.
Related
- Peer nodes:
lmstudioChatModel(other local-server provider), and the cloud chat-model docs in this folder. - Architecture docs: Native LLM SDK.