Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
Adding an LLM provider
This guide explains how to add a new LLM provider to Clerum. There are two paths:
- Data-only (OpenAI-compatible) — the provider exposes an OpenAI-compatible
/chat/completionsendpoint reachable with a base URL +Authorization: BearerAPI key. No driver code. Most of the industry (Groq, DeepSeek, Mistral, xAI, Together, Fireworks, OpenRouter, Perplexity, Moonshot, Cerebras, DeepInfra, Nebius, Novita, Google Gemini via its compat endpoint, …) falls here. - Driver-based — a bespoke wire protocol (Anthropic, Vertex, Bedrock) or
non-vanilla auth/host (Azure OpenAI: per-resource host +
api-keyheader + deployment-name-as-model). Needs amakeProviderarm and possibly an SDK.
The single source of truth
The canonical provider set lives in packages/llm-providers (index.cjs +
index.d.ts). It is a DATA-ONLY leaf consumed by mcp-host, control-api, WRC, HCC
and control-ui — so the provider id, its credential slots, its display label and
its non-secret env are defined once here and everyone inherits them. Never
reintroduce a hardcoded provider list anywhere else.
Runtime-only fields (base URL, default model, tokenizer, how to build the client)
stay local to mcp-host/src/llm/registryCore.ts (RUNTIME_FIELDS) and
mcp-host/src/llm/registry.ts (makeProvider).
Path A — OpenAI-compatible provider (data-only)
Example: adding groq.
1. Shared package — packages/llm-providers/
index.cjs: add the id toPROVIDER_IDS, a credential slot toPROVIDER_CREDENTIAL_SLOTS(use theapiKeySlot('<id>-api-key', '<ID>_API_KEY')helper), a brand label toPROVIDER_DISPLAY_LABELS, and an emptyPROVIDER_NON_SECRET_ENVentry (fixed base URL → no per-Host config).index.d.ts: append the id to thePROVIDER_IDSreadonly tuple (theLlmProviderIdunion derives from it).
2. mcp-host runtime row — registryCore.ts RUNTIME_FIELDS
Add { baseURL, defaultModel, tokenizer: 'fallback' }. The data-driven arm in
makeProvider picks up anything carrying a baseURL — no driver needed.
baseURLmust be exact so${baseURL}/chat/completionsresolves. Watch the path: some include/v1(https://api.mistral.ai/v1), some don't (https://api.perplexity.ai), Fireworks needs an id prefix, DeepInfra is/v1/openai. See the gotchas table below.tokenizer: 'fallback'for non-OpenAI models ('openai'only for GPT-family).
3. CRD enums — charts/clerum-crds/crds/
host.yaml: add the id to thespec.model.providerenum and thespec.llmPolicy.fallbacks[].providerenum.workflowrecipe.yaml: add it to both provider enums iff the provider is single-credential with no required non-secret env (all OpenAI-compatible api-key providers qualify). These enums are additive — old resources stay valid.
4. Allowlist seed — control-api migration (control-api/src/db.ts)
Append a new migration (next NNNN_… version, append-only + idempotent) that
INSERT … ON CONFLICT DO NOTHINGs one sensible default (provider, model, vendor)
into llm_allowed_models. Operators curate the rest from /llm-models.
5. Image-input capability — llm_allowed_models.image_input (#654)
Declare image support per exact (provider, model) pair, never from the
model name, family, or an OpenAI-compatible request shape. Do it in one of two
ways:
- Seed it in the same migration as step 4 for ids the provider's own
documentation covers. Keep it to exact ids (no family extrapolation),
reference the public
https://doc page, and setcheckedAtto the date the doc was read — never the deploy date, so re-running the migration cannot rejuvenate evidence. - Let the operator curate it. The catalog sync fills the column from
models.dev first; the operator overrides only where the vendor documentation
disagrees, through the admin API —
POST /admin/llm-modelsorPUT /admin/llm-models/:idwith theimage_inputfield. The Control UI/llm-modelspage has no editor for this field. A row with no metadata reads asunknown: text keeps working, images are refused with the model named.
The seed is one idempotent statement. Curated evidence replaces discovery evidence; once curated evidence is in place a re-run changes nothing:
UPDATE llm_allowed_models
SET image_input = $1::jsonb
WHERE provider = $2 AND model = $3
AND (image_input IS NULL OR image_input->'evidence'->>'source' = 'discovery')
Shape (validated by @clerum/llm-providers parseImageInputCapability;
GET /admin/llm-models always returns it normalized):
{
"state": "supported",
"evidence": {
"source": "curated",
"reference": "https://docs.example.com/models/vision-flash",
"checkedAt": "2026-09-16T00:00:00.000Z"
}
}
A supported/unsupported claim REQUIRES evidence; a discovery-sourced
claim also requires validUntil. null clears (and normalizes to unknown),
and renaming a row's provider/model clears it automatically. The catalog sync
derives supported / unsupported from models.dev modalities.input (image
present / absent) and records unknown when the entry has no usable
modalities; attachment is not consulted. Discovery evidence never overwrites
curated evidence and never overwrites newer discovery evidence — see
discoveryImageInput in control-api/src/services/llmCatalogSync.ts.
What operators still curate:
- ids models.dev does not list — 15 of the 44 pairs the allowlist seeds today
(for example
zai:glm-5,zai:glm-5.1, the Bedrock Claude ids,openai:gpt-5.1-codex,openrouter:anthropic/claude-sonnet-latest). After a deploy, the sync that runs shortly after control-api starts fillsimage_inputfor the 29 listed pairs; the unlisted ones stayunknown(images refused, text unaffected) until an operator curates them withPUT /admin/llm-models/:idand animage_inputfield; - models where the vendor documentation disagrees with models.dev (curated evidence wins and is never overwritten);
- nothing on installations that run the cron — but without it and without a sync inside the TTL the evidence expires, and running a sync restores it.
Transport registration (mcp-host)
Evidence is necessary, not sufficient. mcp-host/src/llm/imageInput.ts maps
every provider id to a wire family (EXPLICIT_DRIVER_FAMILY,
imageWireFamilyFor), and TRANSPORT_SUPPORT declares, per family and per
dispatch method, whether the serializer carries image parts. A provider whose
family is not registered (or registered false) resolves to
transport_unsupported even with curated supported evidence. Path A providers
get the openai-compatible family automatically when their descriptor carries a
baseURL; without one they resolve to unregistered. Path B drivers must add an
EXPLICIT_DRIVER_FAMILY entry and an imageInputTransport.test.ts case for each
method they implement.
6. control-ui — control-ui/lib/llm.ts
Add the provider's default model to LLM_DEFAULT_MODEL_BY_PROVIDER (a
Record<LlmProvider, …>, so it must be exhaustive — tsc enforces it). The
provider dropdown (LLM_PROVIDER_OPTIONS), the multi-slot secrets form
(LLM_CREDENTIAL_GROUPS) and the field label all auto-derive from the
package — no other UI change. A nicer SECRET_FIELD_HINTS placeholder is optional.
7. Param compatibility (important)
The OpenAI-compatible driver (mcp-host/src/llm/openaiCompatible.ts) shares the
openai provider's request shape. Some providers reject params: Groq
(n>1, logprobs, logit_bias), Cerebras (frequency_penalty/presence_penalty),
Moonshot (tool_choice: 'required', temperature outside [0,1]), DeepSeek
reasoner (sampling params). Keep the default request to a conservative,
portable set (model + messages + tools). If a provider needs something special,
gate it per-provider — don't broaden the shared default.
8. Tests
mcp-host registry tests (provider present + constructs), the control-api
migration/enum test, and confirm the credential-exclusion leak test still passes
with the expanded slot set (it derives from ALL_PROVIDERS, so it should).
Path B — driver-based provider
Everything in Path A, plus:
makeProvider arm — registry.ts
Add an explicit case '<id>': that builds the provider. Load any SDK lazily
with a synchronous require() inside the arm (mcp-host is CommonJS) so the SDK is
only parsed when the provider is actually used, and a broken dependency can't take
down startup. Never route a driver provider through the baseURL arm. Register
the driver's image wire family (step 5, Transport registration) or images are
refused for it.
Credentials & non-secret env
- Multi-credential providers (Bedrock: key pair; Vertex: SA JSON) list every slot
in
PROVIDER_CREDENTIAL_SLOTS. The credential-exclusion boundary (userEnvSnapshot) derives from the full slot set — neverprimarySlot(). - Non-secret, per-Host config (region, project, endpoint) goes in
PROVIDER_NON_SECRET_ENV, surfaced in the UI as a "configure in Host → Environment" hint, never a secret field. It is not in the pod env: mcp-host resolves it from the watched per-Hosthost-<ref>-env/-env-secretand passes the driver a lookup (ProviderEnvLookup,src/llm/providerEnv.ts) limited to the provider's declared names (process.envoutside a Host pod, e.g. dev mode). Read these values through that lookup, neverprocess.envdirectly. - Exclude such providers from
workflowrecipe.yaml— the WRCconfiguretransport is single-credential and doesn't carry non-secret env, so a multi-credential / endpoint-requiring provider must fail at admission, not mid-workflow (this is whybedrockandazureare host-only).
Azure OpenAI — the reference non-vanilla case
Azure is OpenAI-compatible in shape but not a baseURL + Bearer row:
- No fixed base URL — the operator supplies
AZURE_OPENAI_ENDPOINT(non-secret env). The driver builds${endpoint}/openai/v1/(v1 GA path). - Auth is the
api-keyheader, notAuthorization: Bearer. modelis the deployment name, not a catalog id. ItsRUNTIME_FIELDS.azurecarries adefaultModel+tokenizer: 'openai'but no static baseURL, so it takes the explicitcase 'azure':arm and reads the endpoint from the non-secret env lookup at construction. Fail-closed if the endpoint is absent.
Gotchas encoded in the registry (2026)
| provider | base_url quirk | param / behavior gotcha |
|---|---|---|
| fireworks | model id needs prefix accounts/fireworks/models/<name> |
— |
| deepinfra | base path is /v1/openai (not /v1) |
tool-calling only on ~most models |
| perplexity | no /v1 in base URL |
every call is web-grounded (returns citations) |
| deepseek | accepts with or without /v1 |
reasoner mode restricts sampling params |
| moonshot | .ai (global) vs .cn (China) host |
no tool_choice: 'required'; temperature [0,1] |
| groq | — | rejects n>1, logprobs, logit_bias |
| cerebras | — | rejects frequency_penalty/presence_penalty |
| openrouter | — | router: same slug may hit different upstreams; tool support per-model |
| nebius | two brands mid-rename (tokenfactory / studio) | org/model namespaced ids |
| together / novita / nebius | — | org/model namespaced ids |
| gemini (compat) | …/v1beta/openai/ |
beta: unsupported params silently ignored; distinct from vertex |
| minimax | .io (international) vs api.minimaxi.com (China) host |
JWT-shaped api key; portable [0,2] temperature (no gating) |
| azure | per-resource endpoint (env) | api-key header; model = deployment name |
Hand-maintained mirrors — not type-checked
Several scripts and docs enumerate the provider list by hand. tsc does not guard
them, so run these two greps and update every hit:
# 1. every surface that names the current last OpenAI-compatible provider must also name yours
grep -rn --include='*.sh' --include='*.md' --include='*.ts' --include='*.tsx' \
--include=Makefile -e 'novita' evenfire/ | grep -vi node_modules
# 2. the old provider count in prose — substitute the PRE-bump number for <N>
# (this repo went 21 -> 22; the next add greps for 22, and so on). `the other
# <N>` catches the placeholder-fallback sub-count in apply-llm-secret.sh.
grep -rn --include='*.sh' --include='*.md' --include='*.ts' --include='*.tsx' \
--include=Makefile -e '<N> providers' -e 'one of <N>' -e 'all <N>' -e 'the other <N>' evenfire/
Surfaces to update:
scripts/minikube/apply-llm-secret.sh— theSLOTSarray (<id>-api-key|<ID>_API_KEY|).scripts/check-prereqs.sh— thePROVIDER_KEYSlist.scripts/e2e/e2e-plugin-workload-sdk.sh,scripts/e2e/seed-e2e-data.sh— the provider→credential-slotcasearms (a<id>)arm per single-key provider).README.md(repo root) — the provider table.docs/meta/claims-guardrails.md— the provider table.docs/crds/workflowrecipe.md— the documentedproviderenum.docs/llms.txt,Makefile— provider-count comments.
Checklist
Type-checked source of truth (tsc fails if missed):
-
packages/llm-providers/index.cjs— id, slot(s), label, non-secret env -
packages/llm-providers/index.d.ts— id in thePROVIDER_IDStuple -
mcp-host/src/llm/registryCore.ts—RUNTIME_FIELDSrow (baseURL/defaultModel/tokenizer) -
mcp-host/src/llm/registry.ts—makeProviderarm (driver path only) -
control-api/src/services/modelsDevClient.ts—PROVIDER_KEY_MAPentry (our id → models.dev key; verify against liveapi.json) -
control-ui/lib/llm.ts—LLM_DEFAULT_MODEL_BY_PROVIDERentry
Data / assets (not type-checked, but required for the provider to work end-to-end):
-
control-api/src/db.ts— allowlist seed migration (append-only, idempotent) -
control-api/src/db.ts— curatedimage_inputevidence only for ids that models.dev does not list, or where the vendor document contradicts models.dev (exact ids,checkedAt= the date the doc was read) -
control-api/src/data/modelsDevSnapshot.ts— vendored offline snapshot block, derived from live models.dev (the offline catalog-sync fallback; keys must matchPROVIDER_KEY_MAP) -
control-api/src/data/modelsDevSnapshot.ts— bumpVENDORED_MODELS_DEV_SNAPSHOT_CAPTURED_ATto the regeneration commit's date (git log -1 --format=%cI -- <file>); discovery provenance is stamped with it, so leaving it stale misdates evidence -
charts/clerum-crds/crds/host.yaml— both provider enums -
charts/clerum-crds/crds/workflowrecipe.yaml— enums (single-credential only) -
control-ui/public/provider-icons/<id>.svg— brand icon (the picker renders/provider-icons/${provider}.svg; degrades to the label initial if absent). Add a theme-knockout rule incontrol-ui/app/globals.cssonly if the mark is monochrome.
Hand-maintained mirrors (run both greps above, then):
-
scripts/minikube/apply-llm-secret.sh·scripts/check-prereqs.sh· bothscripts/e2e/*.shcase arms -
README.md·docs/meta/claims-guardrails.md·docs/crds/workflowrecipe.md·docs/llms.txt·Makefile - operator doc updated (
docs/deploy/llm-providers.md) and overview (docs/llm-providers/README.md) — counts + tables
Verify: param-compat reviewed; tests updated (every count-bearing assertion); tsc --noEmit clean across services; the two drift greps return only intended hits.