Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
Sarvam AI Chat Model (sarvamChatModel)
| Field | Value |
|---|---|
| Category | ai_chat_models |
| Backend handler | server/nodes/model/sarvam_chat_model/__init__.py (dispatch via BaseNode.execute() -> @Operation("chat") in server/nodes/model/_base.py) |
| AI service | server/services/ai.py::AIService.execute_chat |
| Tests | server/tests/nodes/test_ai_chat_models.py, server/tests/llm/test_model_listing_fallback.py |
| Skill (if any) | n/a |
| Dual-purpose tool | no (group ('model',)) |
Purpose
Sarvam AI's Indic-first LLMs — sarvam-105b (flagship, 128K context) and
sarvam-30b (64K) — covering the 10 most-spoken Indian languages plus English
in native script, romanised and code-mixed input. Registered as an
OpenAI-compatible provider (one _COMPAT_PROVIDERS entry, no provider
subclass); reasoning is on by default server-side.
Inputs (handles)
| Handle | Connection type | Required | Purpose |
|---|---|---|---|
input-main |
main | no | Upstream data; not consumed directly |
Parameters
| Name | Type | Default | Required | displayOptions.show | Description |
|---|---|---|---|---|---|
prompt |
string | "" |
yes | - | User message |
system_prompt |
string | "" |
no | - | System prompt |
model |
string | "" (injected) |
no | - | sarvam-105b / sarvam-30b |
temperature |
number|null | null |
no | - | 0-2. Sarvam's own default is 0.5 with reasoning on, 0.2 without |
max_tokens |
number|null | null |
no | - | Defaults to the llm_defaults.json value (4096 — the Starter-tier cap). Raising it here does nothing unless the JSON is raised too |
top_p |
number|null | 1.0 |
no | - | |
frequency_penalty |
number|null | 0.0 |
no | - | -2.0 to 2.0 |
presence_penalty |
number|null | 0.0 |
no | - | -2.0 to 2.0 |
response_format |
enum|null | "text" |
no | - | text / json_object |
thinking_enabled |
boolean | false |
no | - | Only decides whether we override Sarvam's default effort |
reasoning_effort |
enum|null | "medium" |
no | thinking_enabled: [true] |
low / medium / high |
api_key |
string|null | null (injected) |
no | - | auth_service.get_api_key('sarvam', 'default') |
(Field names are snake_case on SarvamChatModelParams; unknown keys ignored.)
Outputs (handles)
| Handle | Shape | Description |
|---|---|---|
output-model |
object | Model output; standard envelope payload |
Output payload
{
response: string;
thinking: string | null; // from message.reasoning_content
thinking_enabled: boolean;
model: string;
provider: 'sarvam';
finish_reason: string;
timestamp: string;
input: { prompt: string; system_prompt: string };
}
Logic Flow
flowchart TD
A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
B --> C[AIService.execute_chat]
C --> D{valid key + prompt?}
D -- no --> X[error envelope]
D -- yes --> E[detect_ai_provider -> 'sarvam']
E --> F[_model_policy reads thinking_type='effort' from JSON]
F --> G[ChatUnifier.chat -> registry.get_provider sarvam<br/>OpenAI SDK w/ api.sarvam.ai/v1 base_url]
G --> H[provider.chat -> reasoning_effort in body]
H --> I[_normalize lifts reasoning_content -> thinking]
I --> J[success envelope]
G -- Exception --> X
Decision Logic
- Validation: missing api_key / empty prompt -> error envelope.
- Provider routing:
detect_ai_providermatches'sarvam' in node_type.lower(); the token collides with no other branch. - Thinking:
thinking_type: "effort"inllm_defaults.jsondrivesparams["reasoning_effort"]— no Python branch.thinking_enabled=falsesends no effort field, so Sarvam applies its own default (reasoning on, medium). - Temperature stays allowed while reasoning:
reasoning_modelsis deliberately empty for sarvam, so_model_policydoes not settemperature_allowed=Falseor pin 1.0. - No
thinking_default_on: that key would emit Moonshot's proprietaryextra_body.thinking = {"type": "disabled"}, which Sarvam never defined. - Model listing: the provider declares
supports_model_listing: false;fetch_modelsreturns the curated list and probes the key with a one-token completion (see edge cases).
Side Effects
- Database writes: none on the bare chat path.
- Broadcasts: none.
- External API calls:
POST https://api.sarvam.ai/v1/chat/completions(via the OpenAI SDK with an overridden base URL). - File I/O: none.
- Subprocess: none.
External Dependencies
- Credentials:
auth_service.get_api_key('sarvam', 'default')plus optionalsarvam_proxy. The same key authenticates the fivenodes/sarvam/REST nodes. - Services:
services/llm/providers/openai.py(reused with Sarvam's base_url), registered inproviders/_compat.py. - Python packages:
openai. - Environment variables: none.
Edge cases & known limits
- No
/v1/modelsendpoint. Verified against https://docs.sarvam.ai/openapi.json (25 paths, none for model listing). Unhandled, the 404 surfaces as anopenai.OpenAIError->NodeUserError, whichAIService.fetch_modelsre-raises before its curated fallback — breaking credential validation and the model dropdown for a valid key. Thesupports_model_listing: falseflag routes around this generically. - The output cap is a subscription tier, not a model limit. Sarvam allows 4096 output tokens on Starter, 16384 / 8192 on Pro and 128000 / 64000 on Business, and 400s on anything above your tier ("max_tokens (65536) exceeds the maximum allowed for sarvam-105b for your subscription tier (starter): 4096").
resolve_max_tokensuses thellm_defaults.jsonnumber as both the unset default and the clamp ceiling, so it ships as the Starter cap (4096) — the only value that works on every account. Pro/Business users must raise it in the JSON; raisingmax_tokenson the node alone is clamped straight back down. popular_modelsis[]per the >=1M-context policy (Sarvam maxes at 131072); the dropdown falls through to themax_output_tokenskeys, so both models still appear.sarvam-mis deprecated and removed from the API — deliberately absent from the config.wiki_groundingis not exposed. Sarvam accepts it, but an extraParamsfield cannot reach the API today:execute_chatreads a closed key set offflattened. See Sarvam AI Service.- Error boundary: typed OpenAI SDK failures become user-safe
NodeUserErrorvalues inChatUnifierand are re-raised toBaseNode.execute().
Related
- Peer nodes: see the other chat-model docs in this folder.
- Sibling plugins: the five Sarvam REST nodes in
language/share this node's credential. - Architecture docs: Native LLM SDK, Sarvam AI Service.