Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
LLM providers — how they work, which are supported, how to configure them
This is the orientation guide. Read it first, then branch:
| You want to… | Go to |
|---|---|
| Set up credentials and the model allowlist | Operator guide |
| Add a provider that isn't in the list | Adding a provider |
| Understand the Control UI surface | Control UI |
Code and API group still use the historical name clerum (
clerum.io,CLERUM_*,@clerum/*). Same project — code names.
1. The mental model: three independent things
The single most common confusion is treating "provider", "model" and "credential" as one setting. They are three separate axes, configured in three different places by (potentially) three different people.
flowchart LR
subgraph P["1. PROVIDER — who serves the model"]
P1["Host CRD spec.model.provider<br/>one of 24 ids<br/>chosen by the operator"]
end
subgraph M["2. MODEL — which model of that provider"]
M1["allowlist llm_allowed_models<br/>operator-declared in Control UI<br/>user picks within it, per chat"]
end
subgraph C["3. CREDENTIALS — how we authenticate"]
C1["credential slots in the LLM Secret<br/>+ non-secret env per Host"]
end
P1 --> M1 --> C1
A few rules that follow from this split, and that surprise people:
- A Host is bound to exactly one provider. There is no per-message provider switching. Changing provider means editing the Host.
- The credential belongs to the provider, not to the model. Every model of a
provider resolves to the same credential. (This was not always true — the old
provider__modelmapping is retired.) - A model is usable only if the operator allowlisted it. Fail-closed: a row
must exist in
llm_allowed_modelsfor(provider, model)withenabled = true. Installing a provider does not make its models available. - A credential can have more than one part.
bedrockneeds two keys;vertexneeds a whole service-account JSON. The code models this as slots, not as "the API key".
2. Which providers are supported
24 providers, defined once in
packages/llm-providers/index.cjs —
the single source of truth consumed by mcp-host, control-api, the workflow
runtime, the host controller and the Control UI.
They differ only in how much code the integration needs. That is the axis that determines everything else (which config they need, where they work, what can go wrong):
Group A — OpenAI-compatible, pure data (17)
These speak an OpenAI-compatible /chat/completions API. Integrating them is a
baseURL + one API key, no driver code. They are all handled by one shared
class, OpenAICompatibleProvider.
| Provider | id | Credential slot | Default model |
|---|---|---|---|
| Z.AI | zai |
zai-api-key |
glm-5.1 |
| Bailian | bailian |
bailian-api-key |
qwen3-coder-plus |
| OpenRouter | openrouter |
openrouter-api-key |
anthropic/claude-sonnet-latest |
| Google Gemini | gemini |
gemini-api-key |
gemini-2.5-flash |
| DeepSeek | deepseek |
deepseek-api-key |
deepseek-v4-flash |
| Groq | groq |
groq-api-key |
llama-3.3-70b-versatile |
| Together AI | together |
together-api-key |
meta-llama/Llama-3.3-70B-Instruct-Turbo |
| Fireworks AI | fireworks |
fireworks-api-key |
accounts/fireworks/models/llama-v3p3-70b-instruct |
| Mistral AI | mistral |
mistral-api-key |
mistral-medium-latest |
| xAI (Grok) | xai |
xai-api-key |
grok-4.3 |
| Cerebras | cerebras |
cerebras-api-key |
gpt-oss-120b |
| DeepInfra | deepinfra |
deepinfra-api-key |
deepseek-ai/DeepSeek-V3.2 |
| Perplexity | perplexity |
perplexity-api-key |
sonar-pro |
| Moonshot | moonshot |
moonshot-api-key |
kimi-k2.6 |
| Nebius | nebius |
nebius-api-key |
Qwen/Qwen3-235B-A22B-Instruct-2507 |
| Novita AI | novita |
novita-api-key |
deepseek/deepseek-v3.2 |
| MiniMax | minimax |
minimax-api-key |
MiniMax-M2 |
openai itself also speaks this protocol but uses the official SDK (Group B).
Group B — own SDK / bespoke wire protocol (4)
| Provider | id | Credential slots | Non-secret env (per Host) |
|---|---|---|---|
| OpenAI | openai |
openai-api-key |
— |
| Anthropic | claude |
claude-api-key |
— |
| Google Vertex AI | vertex |
vertex-service-account-json (multi-line JSON) |
VERTEX_PROJECT_ID (req), VERTEX_LOCATION |
| Amazon Bedrock | bedrock |
aws-access-key-id + aws-secret-access-key |
AWS_REGION (required) |
claude is the only provider that ships its own token counter; everything except
openai/azure uses a fallback counter, so token counts for Group A and for
Vertex/Bedrock are approximations, not provider-exact.
Group C — light driver (1)
| Provider | id | Credential slot | Non-secret env (per Host) |
|---|---|---|---|
| Azure OpenAI | azure |
azure-openai-api-key |
AZURE_OPENAI_ENDPOINT (req), AZURE_OPENAI_API_VERSION |
Azure is OpenAI-shaped but has no fixed base URL (per-resource endpoint), it
authenticates with an api-key header instead of Authorization: Bearer, and
its "model" is your Azure deployment name, not a catalog id.
Group D — subscription sign-in, no API key (2)
| Provider | id | Credential | Feature flags (all default false in deploy/base) |
|---|---|---|---|
| ChatGPT (Codex) subscription | codex-subscription |
OAuth sign-in held by control-api, no Secret slot | CONTROL_API_CODEX_SUBSCRIPTION_ENABLED, CODEX_LLM_PROXY_EXECUTION_ENABLED, MCP_HOST_CODEX_SUBSCRIPTION_ENABLED |
| Grok subscription | grok-subscription |
OAuth sign-in held by control-api, no Secret slot | CONTROL_API_GROK_SUBSCRIPTION_ENABLED, GROK_LLM_PROXY_EXECUTION_ENABLED, WRC_GROK_SUBSCRIPTION_ENABLED (recipes); mcp-host's flag is injected per Host |
These providers use a subscription account instead of an API key. An admin
signs in from Control UI (Secrets → LLM Subscriptions tab,
/secrets/llm/subscriptions); control-api
keeps the OAuth tokens, and model traffic goes through the in-cluster
codex-llm-proxy / grok-llm-proxy. A Host selects a sign-in with
spec.model.connectionRef and needs no secretRef when every target is
the same subscription provider. A Host may use at most one subscription
provider. They are off in deploy/base and switched on in the minikube
overlay. Setup guide:
Use a ChatGPT or Grok subscription.
Wire-level details:
Codex transport contract,
Grok transport contract.
Important: not every provider works everywhere
| Surface | Providers supported |
|---|---|
| Interactive Host (chat agent) | all 24 (subscription providers need their flags) |
| WorkflowRecipe LLM steps | 22 — bedrock and azure excluded |
bedrock and azure are deliberately absent from the WorkflowRecipe CRD enum:
the workflow configure transport carries a single credential string, so it
cannot deliver Bedrock's key pair nor Azure's required endpoint. Excluding them
makes the recipe fail at admission rather than mid-run.
3. How a request actually flows
sequenceDiagram
participant Op as Operator (Control UI)
participant CA as control-api
participant K8s as Kubernetes
participant MH as mcp-host (the agent)
participant LLM as Provider API
Op->>CA: define allowlist + credentials
CA->>K8s: LLM Secret (slots)
CA->>K8s: ConfigMap clerum-llm-allowed-models
Note over K8s,MH: mcp-host WATCHES both — no restart needed
K8s-->>MH: credential slots + allowlist
MH->>MH: resolve model (session choice → Host default)
MH->>MH: createLLMProvider() — fail-closed
MH->>LLM: chat completion
Two properties worth internalising:
- Everything is watch-based. mcp-host holds four watches (
host-secret,host-cm,llm-secret,allowlist-cm) with precedencehost-secret > host-cm > llm-secret > process.env. Rotating a key or editing the allowlist propagates in about a second — no pod restart, no redeploy. - mcp-host never calls control-api for this. The allowlist reaches the runtime as a ConfigMap watch, never an API call, so a control-api outage cannot break a running agent. Direct mcp-host → control-api traffic is denied by the NetworkPolicy; the only allowed control-plane lane is the workflow-approval gateway.
Fail-closed at every step
createLLMProvider() returns null (degraded, never a crash) if the provider id
is unknown, or if any required credential slot is missing. This is why
Bedrock with only one of its two AWS keys is rejected up front instead of
surfacing as an opaque 401 on the first message.
4. Configuring a provider
4.1 The general shape
- Credentials → Control UI Secrets → LLM. The form renders one group per provider, derived from the shared package. It is write-only: values are never read back, only key names. To rotate, re-enter.
- Non-secret config (region, project, endpoint) → Control UI Host → Environment. These are not credentials and never belong in the Secret.
- Allowlist the models → Control UI LLM Models (
/llm-models). Nothing is usable until this exists. - Point the Host at it →
spec.model.provider+spec.model.name, via the Host wizard or the CRD.
4.2 A plain API-key provider (Groq, Mistral, DeepSeek, …)
Paste the key into its field under Secrets → LLM, allowlist at least one model, set the Host's provider and model. That's all — no env vars, no code.
4.3–4.5 Bedrock, Vertex AI, Azure OpenAI
These need more than one value: Bedrock takes a key pair saved together plus
AWS_REGION; Vertex takes a service-account JSON plus VERTEX_PROJECT_ID;
Azure takes a key plus AZURE_OPENAI_ENDPOINT, and its model name is your
deployment name. Step-by-step setup is in the operator guide:
Vertex AI ·
Bedrock ·
Azure OpenAI.
4.6 Local development (no Kubernetes)
Dev mode bypasses the Kubernetes setup above (Secrets, Host CRs). Set
CLERUM_DEV_MODE=true and export the provider's env var(s) — OPENAI_API_KEY,
GROQ_API_KEY, AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY, etc. Bedrock
also needs AWS_REGION to run: the two keys are enough for auto-selection, but
without the region the provider fails to construct and mcp-host runs degraded.
Azure likewise needs AZURE_OPENAI_ENDPOINT. If CLERUM_MODEL_PROVIDER is unset, mcp-host
auto-detects the first provider (in the canonical id order) whose required
slots are all present and uses that provider's default model;
CLERUM_MODEL_NAME applies only when CLERUM_MODEL_PROVIDER is also set.
One gotcha: because Bedrock's slots are the standard AWS variable names, a shell that already exports generic AWS credentials can auto-select
bedrock. SetCLERUM_MODEL_PROVIDERexplicitly if that's not what you want.
4.7 Local minikube (make minikube-setup)
scripts/minikube/full-setup.sh step 6f writes the chatllm Host's
spec.model from .env, by the rule in scripts/minikube/host-model.sh:
.env |
Host model |
|---|---|
CLERUM_MODEL_PROVIDER + CLERUM_MODEL_NAME |
used as given |
CLERUM_MODEL_PROVIDER only |
that provider's default model (openai, claude, zai, bailian) |
CLERUM_MODEL_NAME only |
refused — setup exits 1 naming both variables |
| neither, one or more of the four keys | first key in the order OPENAI_API_KEY, CLAUDE_API_KEY, ZAI_API_KEY, BAILIAN_API_KEY, with its default model |
| neither, no key | openai/gpt-5.4-mini, with a warning that the agent will not reply |
The default models are the defaultModel values in
mcp-host/src/llm/registryCore.ts. Any other provider needs both variables.
Before applying the Host, setup checks the pair against llm_allowed_models
in the profile's control-postgres (after migrations) and exits 1 on
HOST_MODEL_UNKNOWN (no row), HOST_MODEL_DISABLED (row disabled) or
HOST_MODEL_CHECK_FAILED (the query could not run). A name without a provider
is refused because a model id belongs to one provider: pairing
CLERUM_MODEL_NAME=glm-4.7 with whichever key is present produced
openai/glm-4.7, which no catalog serves.
5. The model allowlist
The set of usable models is operator-declared, not a shipped catalog. It
lives in the control-api Postgres table llm_allowed_models, is managed at
/llm-models, is fully audited (llm_allowed_models_audit), and is materialized
to the clerum-llm-allowed-models ConfigMap for the runtime.
In short: it is fail-closed (a model needs an enabled row), catalog sync only proposes disabled rows, models already deployed are never interrupted, and a missing ConfigMap degrades to "only the configured model". Operator details (propagation, metrics, degraded mode, discovery review) are in the operator guide.
Image input capability (image_input)
Whether a model accepts images is a property of the (provider, model) pair, declared per row and backed by evidence — never inferred from the model name, its family, or an "OpenAI-compatible" request shape.
- Each row carries
image_input({ state, evidence? }). Absent or malformed metadata reads asunknown.supported/unsupportedrequire evidence: a publichttps://documentation reference with no query string, fragment or userinfo (or a sanitizedevidence:<id>) plus the date it was read.validUntilis required for any discovery-sourcedsupported/unsupportedand equalscheckedAt + LLM_CATALOG_IMAGE_EVIDENCE_TTL_MS(default 30 days); curated evidence carries it only when the vendor document states an expiry. - Unknown blocks images, never text. Text-only chat keeps working on an unverified model; attaching an image is what stops, with the model and the reason surfaced.
- The catalog sync derives
supported/unsupportedfrom models.devmodalities.input(imagepresent / absent) and recordsunknownwhen the entry has no usablemodalities;attachmentis not consulted. Discovery evidence never overwrites curated evidence and never overwrites newer discovery evidence. - Transport is a second gate. Even
supportedevidence is refused when the Host's driver family has no image serializer for the dispatch method (TRANSPORT_SUPPORTinmcp-host/src/llm/imageInput.ts); the Desktop shows this as a transport refusal, not as a model refusal. - Renaming a row's provider or model clears its evidence (it belonged to the old
pair), and editing capability never changes
enabled. - A capability change re-materializes the ConfigMap
imageInputfield, so running Hosts pick it up without a redeploy. The sync re-materializesclerum-llm-allowed-modelswhen it changedimage_inputon an enabled row; it still never enables, disables or edits any other served column.
Rollout and refresh cadence
- On upgrade, migration
0115_llm_allowed_models_image_inputonly adds the nullable column and itsCHECK; it seeds no evidence. Existing rows stayunknown(images refused, text unaffected) until a catalog sync runs, on demand or from the cron. With the cron enabled (the base deploy enables it) the first sync runs a few seconds after control-api starts, not one interval later, and fills every pair models.dev lists (29 of the 44 seeded pairs). The 15 seeded pairs models.dev does not list stayunknownuntil an operator curates them through the admin API (PUT /admin/llm-models/:idwith animage_inputfield; see adding-a-provider.md). - Discovery evidence expires
LLM_CATALOG_IMAGE_EVIDENCE_TTL_MSafter the capture it came from. An expired claim resolves tounknownwith reasonevidence_expired, and images are refused with that reason until the next sync. - With
LLM_CATALOG_SYNC_CRON_ENABLED=true(code default off; set totrueindeploy/base/control-plane/configmaps.yaml) the refresh is automatic: one sync shortly after start, then one everyLLM_CATALOG_SYNC_INTERVAL_MS; the config refuses a TTL shorter than twiceLLM_CATALOG_SYNC_INTERVAL_MS. Without the cron, runPOST /admin/llm-models/discovery/syncat least once per TTL (monthly at the default; every two weeks leaves margin). The endpoint runs the same code path as the cron. - A vendored (offline) run stamps evidence with the snapshot's capture date, so
it cannot keep evidence live past
capture + TTL. Regenerate the snapshot withcontrol-api/scripts/regenerate-models-dev-snapshot.mjs(afternpm run buildincontrol-api) when shipping a release.
Beyond the global allowlist there are two narrower scopes:
| Scope | Where | Purpose |
|---|---|---|
| Global | llm_allowed_models |
what the cluster may use at all |
| Per Host | spec.allowedModels |
narrow one Host to a subset |
| Per SDK grant | plugin_workload_sdk_grants.allowed_models |
what one recipe's promptBridge may call |
6. Choosing a model as a user
The desktop app has a per-chat model selector, Claude-Code style, restricted to the allowlist:
- The choice is session state, not process state — one Host serves many users and chats, so your switch never touches anyone else's.
- It applies from your next message; in-flight tasks finish on the model they started with.
- It survives restarts. If the saved model later leaves the allowlist, the agent falls back to the Host default and says so.
- You can change the model, not the provider — the provider is the operator's decision.
Operator-side details (scopes, context-window indicator, usage attribution): operator guide §6.
7. Provider fallback
A Host may declare spec.llmPolicy with an ordered fallbacks[] list, so that a
primary failure (outage, exhausted credit, banned key) fails over instead of
failing the chat. It is opt-in — no policy means no behaviour change.
Failover is sticky with a cooldown (default 300s) and recovers lazily. Each entry
is a fixed (provider, model) pair and may pin a credentialSlot — an extra key
in the same Secret (e.g. claude-api-key-fb1), which is how "same provider,
different key" works. Triggers are a closed catalogue mapped onto the error
classifier: insufficient_quota (402), auth (401/403), rate_limited (429),
provider_unavailable (5xx/529 and retryable transport errors). Deliberately
not eligible: non-retryable 400s, validation and content-policy errors —
failing over on those would just mask bugs.
8. Troubleshooting
| Symptom | Cause |
|---|---|
Agent degraded, log Required LLM credential is missing |
A required slot is missing. Bedrock needs both keys. |
Agent degraded, log Failed to construct LLM provider |
The log names neither provider nor cause. Check AWS_REGION (Bedrock), AZURE_OPENAI_ENDPOINT (Azure), or the Vertex JSON; Vertex uses the JSON's project_id when VERTEX_PROJECT_ID is unset and fails only if both are missing. |
403 model_not_allowed |
Model isn't allowlisted/enabled for that provider. |
404 No secret mapping found (workflow step) |
Provider has no entry in clerum-model-secret-mapping — see below. |
500 ConfigMap not found (workflow step) |
clerum-model-secret-mapping missing from the mcp-host namespace. |
502 mcp_host configure failed/unreachable |
WRC resolved the credential but couldn't configure mcp-host. |
| Model dropdown empty in the wizard | No enabled allowlist rows for that provider. |
| Context-window indicator looks wrong | context_window_tokens unset on the allowlist row → falls back to CLERUM_CONTEXT_MAX_TOKENS. |
| Cost report shows a model as unpriced | No row in llm_model_prices; only 8 models are seeded. Add one under Cost → LLM prices. |
Known sharp edge: workflow steps and the secret mapping
The WorkflowRecipe CRD admits 22 providers, but the shipped
clerum-model-secret-mapping ConfigMap
(deploy/base/mcp-host/model-secret-mapping.yaml)
maps only five: openai, claude, zai, bailian, vertex.
A recipe pinning any of the other 15 (groq, mistral, xai, minimax, …) passes
admission and then fails at runtime with 404 No secret mapping found. This
ConfigMap is declarative — nothing generates it — so to use those providers in
workflows an operator must add the entry themselves:
data:
groq: 'chatllm-api-keys/groq-api-key'
Interactive Hosts are unaffected: they resolve credentials through the ConfigStore, not this mapping.
9. Where things live in the code
| Concern | Location |
|---|---|
| Canonical provider ids, slots, labels, env | packages/llm-providers/index.cjs |
| Runtime fields (baseURL, default model, tokenizer) | mcp-host/src/llm/registryCore.ts |
Provider construction (makeProvider) |
mcp-host/src/llm/registry.ts |
| Fail-closed entry point | mcp-host/src/llm/index.ts |
| Own-SDK drivers | mcp-host/src/llm/drivers/ |
| Credential watches / exclusion boundary | mcp-host/src/config/configStore.ts |
| Allowlist CRUD + ConfigMap materialization | control-api/src/routes/admin/llmModels.ts |
| Host spec validation (model, policy, subset) | control-api/src/routes/admin/hostSpecValidation.ts |
| Workflow secret broker | workflow-recipes/src/workflow/modelConfigHandler.ts |
| UI provider/model metadata | control-ui/lib/llm.ts |
| CRD enums | charts/clerum-crds/crds/{host,workflowrecipe}.yaml |
A note on secrets hygiene: no provider credential — for any provider, whether
configured or not — reaches a tool subprocess through the ConfigStore boundary.
The exclusion set is derived from the full slot list of every provider plus
whatever the LLM Secret actually delivers, so dynamically-named fallback slots
are covered too. The shell tool additionally strips every non-active provider's
slots from the child env; keep provider key names out of CLERUM_ENV_ALLOWLIST
so the active provider's key stays out of subprocesses as well.
Adding a provider is usually a data change, not a code change — see Adding a provider.