Skip to content

OpenRAG is a comprehensive, single package Retrieval-Augmented Generation platform built on Langflow, Docling, and Opensearch.

Python

feat: Make models configurable,enable Azure AI Foundry and Azure OpenAI for Saas and CPD (#2293)

Edwin Jose8d036c12b97d
diff --git a/.env.example b/.env.example
index 22983b96c..e2e99365c 100644
--- a/.env.example
+++ b/.env.example
@@ -588,6 +588,13 @@ INSTANA_SECRETS=regex:.*key.*,.*pass.*,.*secret.*,.*token.*,q\Z,search\Z,filenam
# - "on_prem": Private Cloud
OPENRAG_RUN_MODE=oss
+# OPTIONAL: Path to a YAML file listing which model providers are visible in
+# each run mode above. Defaults to the shipped src/config/model_providers.yaml,
+# which is what Settings, Onboarding and the model pickers all read. Set this
+# only to override the shipped list for a deployment; an unreadable file falls
+# back to the shipped one.
+# OPENRAG_MODEL_PROVIDERS_CONFIG=
+
# Feature flag for the preview-mode ingest backend (default: false = disabled).
# When true, the frontend can request preview ingests that show progress and a
# live index proof without persisting Docling layout JSON. AND-ed with the run
diff --git a/MANIFEST.in b/MANIFEST.in
index d0f089fcb..57a8c9a09 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -1 +1,2 @@
-recursive-include src/tui/_assets *
\ No newline at end of file
+recursive-include src/tui/_assets *
+recursive-include src/config *.yaml
diff --git a/docs/docs/reference/configuration.mdx b/docs/docs/reference/configuration.mdx
index b682b004d..0f7134ca8 100644
--- a/docs/docs/reference/configuration.mdx
+++ b/docs/docs/reference/configuration.mdx
@@ -63,6 +63,132 @@ These variables are initially set during the [application onboarding process](/i
| `WATSONX_ENDPOINT` | Not set | Custom provider endpoint for the IBM watsonx.ai model provider. |
| `WATSONX_PROJECT_ID` | Not set | Project ID for the IBM watsonx.ai model provider. |
+### Provider visibility per run mode {#model-provider-visibility}
+
+Which providers appear in **Settings**, in onboarding, and in the model pickers
+is not hardcoded. `src/config/model_providers.yaml` lists each provider and
+whether it is shown in each run mode, and the backend filters that list by
+`OPENRAG_RUN_MODE` (`oss`, `on_prem`, or `saas`). The UI renders whatever the
+API returns, so there is no separate frontend list to keep in step.
+
+```yaml
+providers:
+ - name: ollama
+ display_name: Ollama
+ modes:
+ oss: true
+ on_prem: true
+ saas: false # local inference is not offered in SaaS
+```
+
+A mode key that is missing means the provider is hidden, so adding a run mode
+never exposes a provider by accident. A provider the file does not list is not
+offered in the UI at all, though a model id for it can still be typed by hand if
+the gateway already routes it.
+
+The shipped defaults are:
+
+| Provider | `oss` | `on_prem` | `saas` |
+|----------|-------|-----------|--------|
+| `openai` | Yes | Yes | Yes |
+| `ollama` | Yes | Yes | No |
+| `watsonx` | Yes | Yes | Yes |
+| `anthropic` | Yes | Yes | Yes |
+| `azure_ai` | Yes | Yes | Yes |
+| `azure` | Yes | Yes | Yes |
+
+LiteLLM keys Azure twice, and the two are not interchangeable: `azure_ai` is
+the Azure AI Foundry catalogue (Llama, Phi, Mistral, Cohere, and the gpt-5.x
+entries Foundry serves), while `azure` is Azure OpenAI Service — the only one
+of the two that carries the gpt-4.1 family. Offer both unless a deployment
+uses just one.
+
+To change what a deployment offers, edit the `modes` values and restart, or
+point `OPENRAG_MODEL_PROVIDERS_CONFIG` at your own YAML file. The file is read
+once per process, so the restart is what applies the edit; reload the console
+afterwards to pick up the new list.
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `OPENRAG_MODEL_PROVIDERS_CONFIG` | Not set | Path to a YAML file that replaces the shipped `model_providers.yaml`. If the file cannot be read, the shipped list is used. |
+
+#### Hide specific models {#exclude-models}
+
+A row can also keep ids out of the pickers. `*` and `?` wildcards work, and
+matching is against the id as the picker shows it — `gpt-3.5-turbo`, not
+`openai/gpt-3.5-turbo`. The id's last path segment is matched too, so a plain
+name also covers Azure's regional listings (`eu/gpt-5.6-luna`,
+`us/gpt-5.6-luna`); write `eu/*` to reach one region on its own.
+
+Exclusions are per provider row. Azure OpenAI serves the same models as
+OpenAI, so a model you want gone from both needs the pattern on both rows.
+
+```yaml
+ - name: openai
+ display_name: OpenAI
+ modes:
+ oss: true
+ exclude_models:
+ - gpt-3.5-* # a whole generation
+ - gpt-5.6-luna # one model
+```
+
+Exclusion is applied last, so it also overrides a `models` entry on the same
+provider, and it covers the live `/models/{provider}` lists as well as the
+catalogue — otherwise an id hidden in Settings would reappear in onboarding.
+
+Two other mechanisms already run without configuration, so reach for
+`exclude_models` only when neither covers you:
+
+| Situation | Handled by |
+|---|---|
+| The provider has already retired the model | Dropped from the catalogue automatically |
+| The provider has announced a retirement within 90 days | Hidden behind a **Show N retiring** row in the picker |
+| This deployment does not want the model offered at all | `exclude_models` |
+
+Credentials for a provider without a purpose-built form — Azure AI Foundry
+today — are entered in **Settings** through a form built from the provider's own
+credential fields. Configuring one needs no code change beyond its row in this
+file.
+
+#### Add a custom OpenAI-compatible provider {#custom-openai-compatible-provider}
+
+To offer a self-hosted OpenAI-compatible endpoint — vLLM, LM Studio, or an
+internal gateway — add a row that declares the model ids it serves. The public
+vendors need no `models` list because LiteLLM already ships one, but a private
+gateway serves whatever its operator deployed, so those ids can only come from
+this file.
+
+```yaml
+ - name: openai_like
+ display_name: Internal LLM Gateway
+ modes:
+ oss: true
+ on_prem: true
+ saas: false
+ models:
+ - llama-3.3-70b-instruct
+ - qwen2.5-coder-32b
+ embedding_models:
+ - bge-m3
+```
+
+Restart, then open **Settings** and configure the new card with the gateway's
+**API base** (for example `https://gateway.internal/v1`) and API key. The
+declared models then appear in the agent and ingestion pickers, and are served
+from `/v1/models` as `openai_like:llama-3.3-70b-instruct`.
+
+Use one of LiteLLM's passthrough provider keys for `name` — `openai_like`,
+`hosted_vllm`, `lm_studio`, `llamafile`, or `custom_openai` — rather than
+inventing one. The key must be a provider LiteLLM can route, or the
+`provider:model` ids OpenRAG publishes resolve to the default provider and are
+called with the wrong credentials. A name that is not routable is logged as a
+warning when the catalogue is built.
+
+Declaring `models` on a public vendor row also works, and is how you surface a
+fine-tune or a private deployment that is not in LiteLLM's table. Ids that the
+table already lists are not duplicated.
+
## Document processing settings {#document-processing-settings}
Controls some aspects of how OpenRAG [processes and ingests documents](/ingestion) into your knowledge base.
diff --git a/frontend/app/api/queries/useGetModelProvidersQuery.ts b/frontend/app/api/queries/useGetModelProvidersQuery.ts
new file mode 100644
index 000000000..8980fe2f3
--- /dev/null
+++ b/frontend/app/api/queries/useGetModelProvidersQuery.ts
@@ -0,0 +1,54 @@
+import {
+ type UseQueryOptions,
+ useQuery,
+ useQueryClient,
+} from "@tanstack/react-query";
+
+export interface ModelProviderEntry {
+ name: string;
+ display_name: string;
+}
+
+export interface ModelProvidersResponse {
+ run_mode: string;
+ providers: ModelProviderEntry[];
+}
+
+/**
+ * The providers this deployment offers, filtered by OPENRAG_RUN_MODE.
+ *
+ * `config/model_providers.yaml` on the backend is the only source of truth for
+ * this list: Settings cards, the Onboarding tabs and the model pickers all
+ * render what it returns, so the frontend keeps no denylist of its own and
+ * brand/theme never decides whether a provider is available.
+ */
+export const useGetModelProvidersQuery = (
+ options?: Omit<
+ UseQueryOptions<ModelProvidersResponse>,
+ "queryKey" | "queryFn"
+ >,
+) => {
+ const queryClient = useQueryClient();
+
+ return useQuery(
+ {
+ queryKey: ["models", "providers"] as const,
+ queryFn: async (): Promise<ModelProvidersResponse> => {
+ const response = await fetch("/api/models/providers");
+ if (!response.ok) {
+ throw new Error("Failed to fetch the model providers");
+ }
+ return (await response.json()) as ModelProvidersResponse;
+ },
+ // Run mode and the config file are fixed for the life of the backend
+ // process, so this never goes stale within a session. Editing
+ // `config/model_providers.yaml` therefore takes a backend restart *and* a
+ // page reload; the endpoint sends `no-store` so the reload actually asks.
+ staleTime: Number.POSITIVE_INFINITY,
+ gcTime: Number.POSITIVE_INFINITY,
+ refetchOnWindowFocus: false,
+ ...options,
+ },
+ queryClient,
+ );
+};
diff --git a/frontend/app/chat/_components/message.tsx b/frontend/app/chat/_components/message.tsx
index b4985f4b7..c526d98ea 100644
--- a/frontend/app/chat/_components/message.tsx
+++ b/frontend/app/chat/_components/message.tsx
@@ -16,13 +16,13 @@ export function Message({
unstyledContent = false,
}: MessageProps) {
return (
- <div className="flex gap-3">
+ <div className="flex gap-3 overflow-hidden">
{icon}
<div
className={
isAssistant && !unstyledContent
? "px-5 py-4 bg-secondary/20 rounded-2xl flex-1"
- : "flex-1"
+ : "flex-1 flex w-full overflow-hidden"
}
>
<div className="flex-1 min-w-0">{children}</div>
diff --git a/frontend/app/onboarding/_components/generic-onboarding.tsx b/frontend/app/onboarding/_components/generic-onboarding.tsx
new file mode 100644
index 000000000..493fea84d
--- /dev/null
+++ b/frontend/app/onboarding/_components/generic-onboarding.tsx
@@ -0,0 +1,172 @@
+import type { Dispatch, SetStateAction } from "react";
+import { useMemo, useState } from "react";
+import { useGetModelCatalogQuery } from "@/app/api/queries/useGetModelsQuery";
+import {
+ onboardingCredentialFields,
+ type SavedProvidersSnapshot,
+ savedCredentialValuesForProvider,
+ savedSecretFieldsForProvider,
+} from "@/app/settings/_helpers/catalog-models";
+import { getProviderChrome } from "@/app/settings/_helpers/model-helpers";
+import { LabelInput } from "@/components/label-input";
+import type { OnboardingVariables } from "../../api/mutations/useOnboardingMutation";
+import { AdvancedOnboarding } from "./advanced";
+
+/**
+ * Onboarding step for a provider with no hand-built component.
+ *
+ * The credential inputs come from the catalogue's field spec and the model list
+ * from the catalogue itself, so a provider added to
+ * `config/model_providers.yaml` gets a working onboarding tab with no frontend
+ * change. Credentials are submitted through the generic `provider_credentials`
+ * payload the onboarding endpoint already accepts.
+ */
+export function GenericOnboarding({
+ provider,
+ displayName,
+ setSettings,
+ isEmbedding = false,
+ providers,
+}: {
+ provider: string;
+ displayName?: string;
+ setSettings: Dispatch<SetStateAction<OnboardingVariables>>;
+ isEmbedding?: boolean;
+ providers?: SavedProvidersSnapshot;
+}) {
+ const { data: catalog } = useGetModelCatalogQuery();
+ const chrome = getProviderChrome(provider, displayName);
+ const Logo = chrome.logo;
+
+ const fields = useMemo(
+ () => onboardingCredentialFields(catalog, provider),
+ [catalog, provider],
+ );
+ const savedSecrets = useMemo(
+ () => new Set(savedSecretFieldsForProvider(providers, provider)),
+ [providers, provider],
+ );
+ const savedValues = useMemo(
+ () => savedCredentialValuesForProvider(providers, provider),
+ [providers, provider],
+ );
+
+ const [credentials, setCredentials] = useState<Record<string, string>>({});
+ const [model, setModel] = useState("");
+
+ const syncParentSettings = (
+ nextCredentials: Record<string, string>,
+ nextModel: string,
+ ) => {
+ const submitted: Record<string, string> = {};
+ for (const [key, value] of Object.entries(nextCredentials)) {
+ const trimmed = (value ?? "").trim();
+ if (trimmed !== "") {
+ submitted[key] = trimmed;
+ }
+ }
+
+ setSettings((prev) => ({
+ ...prev,
+ ...(isEmbedding
+ ? { embedding_provider: provider, embedding_model: nextModel }
+ : { llm_provider: provider, llm_model: nextModel }),
+ provider_credentials: Object.keys(submitted).length
+ ? { ...prev.provider_credentials, [provider]: submitted }
+ : prev.provider_credentials,
+ }));
+ };
+
+ // Seed the non-secret fields from what is already saved, once per provider.
+ const [seededFor, setSeededFor] = useState<string | undefined>();
+ if (seededFor !== provider) {
+ setSeededFor(provider);
+ setCredentials(savedValues);
+ syncParentSettings(savedValues, model);
+ }
+
+ const catalogEntry = catalog?.providers?.find(
+ (entry) => entry.key === provider,
+ );
+ const models = useMemo(() => {
+ const entries = isEmbedding
+ ? (catalogEntry?.embedding_models ?? [])
+ : (catalogEntry?.models ?? []);
+ return entries.map((entry) => ({
+ value: entry.model,
+ label: entry.model,
+ }));
+ }, [catalogEntry, isEmbedding]);
+
+ // Default to the first model the catalogue lists for this provider.
+ const [prevModels, setPrevModels] = useState<typeof models | undefined>();
+ if (models !== prevModels) {
+ setPrevModels(models);
+ if (!model && models.length > 0) {
+ const defaultModel = models[0].value;
+ setModel(defaultModel);
+ syncParentSettings(credentials, defaultModel);
+ }
+ }
+
+ const handleCredentialChange = (fieldKey: string, newValue: string) => {
+ const nextCredentials = { ...credentials, [fieldKey]: newValue };
+ setCredentials(nextCredentials);
+ syncParentSettings(nextCredentials, model);
+ };
+
+ const handleModelChange = (newModel: string) => {
+ setModel(newModel);
+ syncParentSettings(credentials, newModel);
+ };
+
+ return (
+ <>
+ <div className="space-y-5">
+ {fields.map((field) => {
+ const isSecret =
+ field.field_type === "password" || field.field_type === "textarea";
+ const hasSaved = isSecret && savedSecrets.has(field.key);
+ return (
+ <div key={field.key} className="space-y-1">
+ <LabelInput
+ label={field.label}
+ helperText={field.tooltip ?? ""}
+ id={`onboarding-${provider}-${field.key}`}
+ type={field.field_type === "password" ? "password" : "text"}
+ required={field.required && !hasSaved}
+ placeholder={
+ hasSaved ? "•••••••••" : (field.placeholder ?? undefined)
+ }
+ value={credentials[field.key] ?? ""}
+ onChange={(e) =>
+ handleCredentialChange(field.key, e.target.value)
+ }
+ />
+ {hasSaved && (
+ <p className="text-mmd text-muted-foreground">
+ A value is already saved. Leave this blank to keep it.
+ </p>
+ )}
+ </div>
+ );
+ })}
+ {models.length === 0 && (
+ <p className="text-mmd text-muted-foreground">
+ {chrome.name} publishes no {isEmbedding ? "embedding" : "language"}{" "}
+ models in the catalogue. Pick a different provider for this step.
+ </p>
+ )}
+ </div>
+ <AdvancedOnboarding
+ icon={<Logo className="w-4 h-4" />}
+ languageModels={isEmbedding ? undefined : models}
+ embeddingModels={isEmbedding ? models : undefined}
+ languageModel={isEmbedding ? undefined : model}
+ embeddingModel={isEmbedding ? model : undefined}
+ setLanguageModel={isEmbedding ? undefined : handleModelChange}
+ setEmbeddingModel={isEmbedding ? handleModelChange : undefined}
+ />
+ </>
+ );
+}
diff --git a/frontend/app/onboarding/_components/model-features.tsx b/frontend/app/onboarding/_components/model-features.tsx
index 7f3ed3f59..e0b7d014d 100644
--- a/frontend/app/onboarding/_components/model-features.tsx
+++ b/frontend/app/onboarding/_components/model-features.tsx
@@ -23,7 +23,9 @@ import {
PRIMARY_CAPABILITIES,
supports,
} from "@/app/settings/_helpers/model-info";
+import AiFoundryLogo from "@/components/icons/ai-foundry-logo";
import AnthropicLogo from "@/components/icons/anthropic-logo";
+import AzureOpenAILogo from "@/components/icons/azure-openai-logo";
import IBMLogo from "@/components/icons/ibm-logo";
import OllamaLogo from "@/components/icons/ollama-logo";
import OpenAILogo from "@/components/icons/openai-logo";
@@ -95,6 +97,10 @@ const PROVIDER_LOGOS: Record<
anthropic: AnthropicLogo,
watsonx: IBMLogo,
ollama: OllamaLogo,
+ // Both Azure rows: `azure_ai` is Foundry, `azure` is Azure OpenAI Service,
+ // and Microsoft draws the two differently.
+ azure_ai: AiFoundryLogo,
+ azure: AzureOpenAILogo,
};
function ProviderMark({ provider, name }: { provider?: string; name: string }) {
diff --git a/frontend/app/onboarding/_components/model-selector.tsx b/frontend/app/onboarding/_components/model-selector.tsx
index af5905acb..891299127 100644
--- a/frontend/app/onboarding/_components/model-selector.tsx
+++ b/frontend/app/onboarding/_components/model-selector.tsx
@@ -61,6 +61,31 @@ function optionProvider(
return option.provider ?? group?.provider;
}
+/** How close a retirement has to be before the model drops out of the list. */
+const RETIRING_SOON_DAYS = 90;
+
+/**
+ * A model close enough to retirement to be worth hiding by default.
+ *
+ * Not simply "has a deprecation_date": providers use that field very
+ * differently. Anthropic stamps one on *every* model at launch, roughly a year
+ * out, so hiding on presence alone would leave 2 of its 17 models on screen.
+ * OpenAI leaves current models undated and sunsets a whole legacy generation
+ * on one shared date. A window reads correctly for both — it hides OpenAI's
+ * gpt-3.5 family and Anthropic's 4-5 generation, and nothing at all for
+ * providers that publish no dates.
+ *
+ * The backend already drops models whose date has passed, so anything here is
+ * still callable — hidden, never taken away.
+ */
+function isRetiringSoon(option: ModelOption): boolean {
+ const date = option.model?.deprecation_date;
+ if (!date) return false;
+ const retires = Date.parse(date);
+ if (Number.isNaN(retires)) return false;
+ return retires <= Date.now() + RETIRING_SOON_DAYS * 86_400_000;
+}
+
function isSelectedRow(
option: ModelOption,
value: string,
@@ -85,7 +110,7 @@ export function ModelSelector({
onValueChange,
icon,
placeholder = "Select model...",
- searchPlaceholder = "Search model...",
+ searchPlaceholder,
noOptionsPlaceholder = "No models available",
custom = false,
hasError = false,
@@ -104,6 +129,14 @@ export function ModelSelector({
}
}
+ // A selector that accepts a typed model has to say so: the box is the only
+ // way to reach a deployment name the catalogue cannot know. Selectors that
+ // only pick from a fixed list (the watsonx endpoint picker) keep the plain
+ // label, and any caller passing its own copy still wins.
+ const searchBoxPlaceholder =
+ searchPlaceholder ??
+ (custom ? "Search or type a model name" : "Search model...");
+
const [searchValue, setSearchValue] = useState("");
// The option filter runs on the trimmed, lowercased search text, so the
// custom entry has to use the trimmed text too — otherwise typing trailing
@@ -122,6 +155,16 @@ export function ModelSelector({
if (!next.delete(group)) next.add(group);
return next;
});
+ // Groups the user chose to see retiring models in.
+ const [deprecatedGroups, setDeprecatedGroups] = useState<Set<string>>(
+ () => new Set(),
+ );
+ const toggleDeprecated = (group: string) =>
+ setDeprecatedGroups((previous) => {
+ const next = new Set(previous);
+ if (!next.delete(group)) next.add(group);
+ return next;
+ });
// Flatten grouped options or use regular options
const allOptions =
@@ -145,24 +188,46 @@ export function ModelSelector({
const providerMatches = group.group
.toLowerCase()
.includes(deferredSearch);
- const matches = deferredSearch
+ const searched = deferredSearch
? group.options.filter(
(option) =>
providerMatches ||
option.label.toLowerCase().includes(deferredSearch),
)
: group.options;
- if (deferredSearch && matches.length === 0) return [];
+ // Retiring models are hidden until asked for, but never the one already
+ // selected — taking the current choice off the list would make the
+ // selector look broken to whoever set it.
+ const deprecatedCount = searched.filter(isRetiringSoon).length;
+ const matches = deprecatedGroups.has(group.group)
+ ? searched
+ : searched.filter(
+ (option) =>
+ !isRetiringSoon(option) ||
+ isSelectedRow(option, value, selectedProvider, group),
+ );
+ if (deferredSearch && matches.length === 0 && deprecatedCount === 0) {
+ return [];
+ }
// Collapsed groups show a preview; the trailing row expands them. The cap
// applies while searching too, so without that row a search would drop
// matches with nothing on screen to say so.
const options = expandedGroups.has(group.group)
? matches
: matches.slice(0, MODELS_PER_PROVIDER);
- return [{ ...group, options, matchCount: matches.length }];
+ return [
+ { ...group, options, matchCount: matches.length, deprecatedCount },
+ ];
});
return matched.slice(0, 40);
- }, [deferredSearch, groupedOptions, expandedGroups]);
+ }, [
+ deferredSearch,
+ groupedOptions,
+ expandedGroups,
+ deprecatedGroups,
+ value,
+ selectedProvider,
+ ]);
const visibleOptions = useMemo(() => {
if (groupedOptions) return [];
const source = options ?? [];
@@ -244,7 +309,10 @@ export function ModelSelector({
>
<Command shouldFilter={false}>
<CommandInput
- placeholder={searchPlaceholder}
+ // Located by test id, not by placeholder: the copy changes with
+ // `custom` and is the sort of thing that gets reworded.
+ data-testid="model-search-input"
+ placeholder={searchBoxPlaceholder}
value={searchValue}
onValueChange={setSearchValue}
/>
@@ -366,6 +434,27 @@ export function ModelSelector({
: "Show fewer"}
</CommandItem>
)}
+ {group.deprecatedCount > 0 && (
+ <CommandItem
+ value={`__deprecated-${group.group}`}
+ aria-label={
+ deprecatedGroups.has(group.group)
+ ? `Hide retiring ${group.group} models`
+ : `Show ${group.deprecatedCount} retiring ${group.group} models`
+ }
+ data-testid={
+ groupProvider
+ ? `model-group-deprecated-${groupProvider}`
+ : undefined
+ }
+ className="text-xs text-muted-foreground"
+ onSelect={() => toggleDeprecated(group.group)}
+ >
+ {deprecatedGroups.has(group.group)
+ ? "Hide retiring models"
+ : `Show ${group.deprecatedCount} retiring`}
+ </CommandItem>
+ )}
{showCustom && (
<CommandItem
value={`${group.group}-${customValue}`}
@@ -392,7 +481,23 @@ export function ModelSelector({
)}
/>
<div className="flex items-center gap-2">
- {customValue}
+ {/* "Use" rather than the bare name: the row sits
+ among search results, and without a verb it
+ reads as one more model that already exists
+ rather than the thing that adds yours. */}
+ <span>Use</span>
+ {/* The provider tag is shown, never typed: the
+ group the row sits under decides it, and it is
+ what makes the id route back here. Without it
+ on screen there is nothing to tell you a name
+ typed under Azure OpenAI is stored as
+ `azure:<name>`. */}
+ {groupProvider && (
+ <span className="rounded-sm bg-muted px-1 font-mono text-xs text-muted-foreground">
+ {groupProvider}:
+ </span>
+ )}
+ <span>&ldquo;{customValue}&rdquo;</span>
<span className="text-xs text-foreground p-1 rounded-md bg-muted">
Custom
</span>
@@ -422,7 +527,7 @@ export function ModelSelector({
)}
/>
<div className="flex items-center gap-2">
- {customValue}
+ Use &ldquo;{customValue}&rdquo;
<span className="text-xs text-foreground p-1 rounded-md bg-muted">
Custom
</span>
@@ -484,7 +589,7 @@ export function ModelSelector({
)}
/>
<div className="flex items-center gap-2">
- {customValue}
+ Use &ldquo;{customValue}&rdquo;
<span className="text-xs text-foreground p-1 rounded-md bg-muted">
Custom
</span>
diff --git a/frontend/app/onboarding/_components/onboarding-card.tsx b/frontend/app/onboarding/_components/onboarding-card.tsx
index 67cb08904..49d660e40 100644
--- a/frontend/app/onboarding/_components/onboarding-card.tsx
+++ b/frontend/app/onboarding/_components/onboarding-card.tsx
@@ -3,13 +3,15 @@
import { useIsFetching, useQueryClient } from "@tanstack/react-query";
import { AnimatePresence, motion } from "framer-motion";
import { X } from "lucide-react";
-import { useEffect, useEffectEvent, useRef, useState } from "react";
+import { useEffect, useEffectEvent, useMemo, useRef, useState } from "react";
import { toast } from "sonner";
import {
type OnboardingVariables,
useOnboardingMutation,
} from "@/app/api/mutations/useOnboardingMutation";
import { useOnboardingRollbackMutation } from "@/app/api/mutations/useOnboardingRollbackMutation";
+import { useGetModelProvidersQuery } from "@/app/api/queries/useGetModelProvidersQuery";
+import { useGetModelCatalogQuery } from "@/app/api/queries/useGetModelsQuery";
import {
type ProviderSettings,
useGetSettingsQuery,
@@ -17,15 +19,12 @@ import {
import { useGetTasksQuery } from "@/app/api/queries/useGetTasksQuery";
import type { ProviderHealthResponse } from "@/app/api/queries/useProviderHealthQuery";
import {
- CLOUD_EXCLUDED_PROVIDERS,
EMBEDDING_PROVIDER_ORDER,
+ getProviderChrome,
LLM_PROVIDER_ORDER,
+ orderProviders,
} from "@/app/settings/_helpers/model-helpers";
import { useDoclingHealth } from "@/components/docling-health-banner";
-import AnthropicLogo from "@/components/icons/anthropic-logo";
-import IBMLogo from "@/components/icons/ibm-logo";
-import OllamaLogo from "@/components/icons/ollama-logo";
-import OpenAILogo from "@/components/icons/openai-logo";
import { Button } from "@/components/ui/button";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import {
@@ -33,7 +32,6 @@ import {
TooltipContent,
TooltipTrigger,
} from "@/components/ui/tooltip";
-import { useIsCloudBrand } from "@/contexts/brand-context";
import {
trackButton,
trackProcessFailure,
@@ -43,6 +41,7 @@ import { formatProviderErrorMessage } from "@/lib/chat-stream-errors";
import { cn } from "@/lib/utils";
import { AnimatedProviderSteps } from "./animated-provider-steps";
import { AnthropicOnboarding } from "./anthropic-onboarding";
+import { GenericOnboarding } from "./generic-onboarding";
import { IBMOnboarding } from "./ibm-onboarding";
import { OllamaOnboarding } from "./ollama-onboarding";
import { OpenAIOnboarding } from "./openai-onboarding";
@@ -60,6 +59,10 @@ const STEP_LIST = [
"Configuring Langflow",
];
+// Providers whose model inventory is read live from the running service, so an
+// empty catalogue list says nothing about what they can serve.
+const LIVE_MODEL_PROVIDERS = new Set(["ollama", "watsonx"]);
+
const EMBEDDING_STEP_LIST = [
"Setting up your model provider",
"Defining schema",
@@ -73,12 +76,70 @@ const OnboardingCard = ({
isCompleted = false,
}: OnboardingCardProps) => {
const { isHealthy: isDoclingHealthy } = useDoclingHealth();
- const isCloudBrand = useIsCloudBrand();
+
+ // Which providers this deployment offers comes from the backend, filtered by
+ // OPENRAG_RUN_MODE (config/model_providers.yaml). Onboarding renders that
+ // list; it does not decide availability from the UI brand.
+ const { data: providerData } = useGetModelProvidersQuery();
+ const availableProviders = useMemo(
+ () => providerData?.providers ?? [],
+ [providerData],
+ );
+ const providerDisplayNames = useMemo(
+ () =>
+ Object.fromEntries(
+ availableProviders.map(({ name, display_name }) => [
+ name,
+ display_name,
+ ]),
+ ),
+ [availableProviders],
+ );
+ const providerKeys = useMemo(
+ () =>
+ orderProviders(
+ availableProviders.map((provider) => provider.name),
+ isEmbedding ? EMBEDDING_PROVIDER_ORDER : LLM_PROVIDER_ORDER,
+ ),
+ [availableProviders, isEmbedding],
+ );
+
+ const { data: catalog } = useGetModelCatalogQuery();
+
+ // The embedding step must not offer a provider that serves no embedding
+ // models — Anthropic being the standing example. The catalogue answers that
+ // for every provider except the two whose inventory comes from the running
+ // server rather than LiteLLM's bundled list.
+ const tabProviders = useMemo(() => {
+ if (!isEmbedding) {
+ return providerKeys;
+ }
+ return providerKeys.filter((providerKey) => {
+ if (LIVE_MODEL_PROVIDERS.has(providerKey)) {
+ return true;
+ }
+ const entry = catalog?.providers?.find(
+ (item) => item.key === providerKey,
+ );
+ // Catalogue not loaded yet: hide nothing rather than flicker tabs away.
+ return !entry || entry.embedding_models.length > 0;
+ });
+ }, [providerKeys, isEmbedding, catalog]);
const [modelProvider, setModelProvider] = useState<string>(
isEmbedding ? "openai" : "anthropic",
);
+ // The default above may not be offered here (Anthropic disabled, or the
+ // embedding step). Fall back to the first provider this step does offer.
+ const [prevTabProviders, setPrevTabProviders] = useState<string[]>([]);
+ if (tabProviders !== prevTabProviders && tabProviders.length > 0) {
+ setPrevTabProviders(tabProviders);
+ if (!tabProviders.includes(modelProvider)) {
+ setModelProvider(tabProviders[0]);
+ }
+ }
+
// Read model-fetch loading from React Query instead of syncing it up from children.
const isLoadingModels = useIsFetching({ queryKey: ["models"] }) > 0;
@@ -94,14 +155,7 @@ const OnboardingCard = ({
if (currentSettings?.providers !== prevProviders) {
setPrevProviders(currentSettings?.providers);
if (currentSettings?.providers) {
- const fullOrder = isEmbedding
- ? EMBEDDING_PROVIDER_ORDER
- : LLM_PROVIDER_ORDER;
- const providerOrder = isCloudBrand
- ? fullOrder.filter((p) => !CLOUD_EXCLUDED_PROVIDERS.includes(p))
- : fullOrder;
-
- for (const provider of providerOrder) {
+ for (const provider of tabProviders) {
if (
provider === "anthropic" &&
currentSettings.providers.anthropic?.has_api_key
@@ -126,6 +180,11 @@ const OnboardingCard = ({
) {
setModelProvider("ollama");
break;
+ } else if (
+ currentSettings.providers.custom?.[provider]?.configured === true
+ ) {
+ setModelProvider(provider);
+ break;
}
}
}
@@ -155,7 +214,7 @@ const OnboardingCard = ({
} else if (provider === "ollama") {
return currentSettings.providers.ollama?.configured === true;
}
- return false;
+ return currentSettings.providers.custom?.[provider]?.configured === true;
};
const showProviderConfiguredMessage =
@@ -476,6 +535,13 @@ const OnboardingCard = ({
onboardingData.ollama_endpoint = settings.ollama_endpoint;
}
+ // Providers configured through the generic form submit their credentials
+ // as-is; the backend stores whatever field keys the provider declares.
+ const generic = settings.provider_credentials?.[currentProvider];
+ if (generic && Object.keys(generic).length > 0) {
+ onboardingData.provider_credentials = { [currentProvider]: generic };
+ }
+
trackButton({
CTA: isEmbedding ? "Complete - Embedding Setup" : "Complete - LLM Setup",
elementId: "onboarding-complete-button",
@@ -508,7 +574,7 @@ const OnboardingCard = ({
exit={{ opacity: 0, y: 24 }}
transition={{ duration: 0.4, ease: "easeInOut" }}
>
- <div className={`w-full max-w-[600px] flex flex-col`}>
+ <div className={`w-full flex flex-col`}>
<AnimatePresence mode="wait">
{error && (
<motion.div
@@ -533,198 +599,119 @@ const OnboardingCard = ({
value={modelProvider}
onValueChange={handleSetModelProvider}
>
- <TabsList className="mb-4">
- {!isEmbedding && (
- <TabsTrigger
- value="anthropic"
- data-testid={`anthropic-llm-tab`}
- className={cn(
- error &&
- modelProvider === "anthropic" &&
- "data-[state=active]:border-destructive",
- )}
- >
- <TabTrigger
- selected={modelProvider === "anthropic"}
- isLoading={isLoadingModels}
- >
- <div
- className={cn(
- "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",
- modelProvider === "anthropic"
- ? "bg-[#D97757]"
- : "bg-muted",
- )}
- >
- <AnthropicLogo
- className={cn(
- "w-4 h-4 shrink-0",
- modelProvider === "anthropic"
- ? "text-black"
- : "text-muted-foreground",
- )}
- />
- </div>
- Anthropic
- </TabTrigger>
- </TabsTrigger>
- )}
- <TabsTrigger
- value="openai"
- className={cn(
- error &&
- modelProvider === "openai" &&
- "data-[state=active]:border-destructive",
- )}
- data-testid={`openai-${isEmbedding ? "embedding" : "llm"}-tab`}
- >
- <TabTrigger
- selected={modelProvider === "openai"}
- isLoading={isLoadingModels}
- >
- <div
+ <TabsList className="mb-1 pb-3 w-full justify-start gap-1 overflow-x-auto">
+ {tabProviders.map((providerKey) => {
+ const chrome = getProviderChrome(
+ providerKey,
+ providerDisplayNames[providerKey],
+ );
+ const Logo = chrome.logo;
+ const selected = modelProvider === providerKey;
+ return (
+ <TabsTrigger
+ key={providerKey}
+ value={providerKey}
+ data-testid={`${providerKey}-${isEmbedding ? "embedding" : "llm"}-tab`}
className={cn(
- "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",
- modelProvider === "openai" ? "bg-white" : "bg-muted",
+ error &&
+ selected &&
+ "data-[state=active]:border-destructive",
+ "min-w-40",
)}
>
- <OpenAILogo
- className={cn(
- "w-4 h-4 shrink-0",
- modelProvider === "openai"
- ? "text-black"
- : "text-muted-foreground",
- )}
- />
- </div>
- OpenAI
- </TabTrigger>
- </TabsTrigger>
- <TabsTrigger
- value="watsonx"
- data-testid={`watsonx-${isEmbedding ? "embedding" : "llm"}-tab`}
- className={cn(
- error &&
- modelProvider === "watsonx" &&
- "data-[state=active]:border-destructive",
- )}
- >
- <TabTrigger
- selected={modelProvider === "watsonx"}
- isLoading={isLoadingModels}
- >
- <div
- className={cn(
- "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",
- modelProvider === "watsonx"
- ? "bg-[#1063FE]"
- : "bg-muted",
- )}
- >
- <IBMLogo
- className={cn(
- "w-4 h-4 shrink-0",
- modelProvider === "watsonx"
- ? "text-white"
- : "text-muted-foreground",
- )}
- />
- </div>
- IBM watsonx.ai
- </TabTrigger>
- </TabsTrigger>
- {!isCloudBrand && (
- <TabsTrigger
- value="ollama"
- data-testid={`ollama-${isEmbedding ? "embedding" : "llm"}-tab`}
- className={cn(
- error &&
- modelProvider === "ollama" &&
- "data-[state=active]:border-destructive",
- )}
- >
- <TabTrigger
- selected={modelProvider === "ollama"}
- isLoading={isLoadingModels}
- >
- <div
- className={cn(
- "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",
- modelProvider === "ollama"
- ? "bg-white"
- : "bg-muted",
- )}
+ <TabTrigger
+ selected={selected}
+ isLoading={isLoadingModels}
>
- <OllamaLogo
+ <div
className={cn(
- "w-4 h-4 shrink-0",
- modelProvider === "ollama"
- ? "text-black"
- : "text-muted-foreground",
+ "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",
+ selected
+ ? (chrome.tabLogoBgColor ?? chrome.logoBgColor)
+ : "bg-muted",
)}
- />
- </div>
- Ollama
- </TabTrigger>
- </TabsTrigger>
- )}
+ >
+ <Logo
+ className={cn(
+ "w-4 h-4 shrink-0",
+ selected
+ ? (chrome.tabLogoColor ?? chrome.logoColor)
+ : "text-muted-foreground",
+ )}
+ />
+ </div>
+ {chrome.name}
+ </TabTrigger>
+ </TabsTrigger>
+ );
+ })}
</TabsList>
- {!isEmbedding && (
- <TabsContent value="anthropic">
- <AnthropicOnboarding
- setSettings={setSettings}
- isEmbedding={isEmbedding}
- hasEnvApiKey={
- currentSettings?.providers?.anthropic?.has_api_key ===
- true
- }
- />
- </TabsContent>
- )}
- <TabsContent value="openai">
- <OpenAIOnboarding
- setSettings={setSettings}
- isEmbedding={isEmbedding}
- hasEnvApiKey={
- currentSettings?.providers?.openai?.has_api_key === true
- }
- alreadyConfigured={
- providerAlreadyConfigured && modelProvider === "openai"
- }
- />
- </TabsContent>
- <TabsContent value="watsonx">
- <IBMOnboarding
- setSettings={setSettings}
- isEmbedding={isEmbedding}
- alreadyConfigured={
- providerAlreadyConfigured && modelProvider === "watsonx"
- }
- existingEndpoint={
- currentSettings?.providers?.watsonx?.endpoint
- }
- existingProjectId={
- currentSettings?.providers?.watsonx?.project_id
- }
- hasEnvApiKey={
- currentSettings?.providers?.watsonx?.has_api_key === true
- }
- />
- </TabsContent>
- {!isCloudBrand && (
- <TabsContent value="ollama">
- <OllamaOnboarding
- setSettings={setSettings}
- isEmbedding={isEmbedding}
- alreadyConfigured={
- providerAlreadyConfigured && modelProvider === "ollama"
- }
- existingEndpoint={
- currentSettings?.providers?.ollama?.endpoint
- }
- />
+ {tabProviders.map((providerKey) => (
+ <TabsContent key={providerKey} value={providerKey}>
+ {providerKey === "anthropic" ? (
+ <AnthropicOnboarding
+ setSettings={setSettings}
+ isEmbedding={isEmbedding}
+ hasEnvApiKey={
+ currentSettings?.providers?.anthropic?.has_api_key ===
+ true
+ }
+ />
+ ) : providerKey === "openai" ? (
+ <OpenAIOnboarding
+ setSettings={setSettings}
+ isEmbedding={isEmbedding}
+ hasEnvApiKey={
+ currentSettings?.providers?.openai?.has_api_key ===
+ true
+ }
+ alreadyConfigured={
+ providerAlreadyConfigured &&
+ modelProvider === "openai"
+ }
+ />
+ ) : providerKey === "watsonx" ? (
+ <IBMOnboarding
+ setSettings={setSettings}
+ isEmbedding={isEmbedding}
+ alreadyConfigured={
+ providerAlreadyConfigured &&
+ modelProvider === "watsonx"
+ }
+ existingEndpoint={
+ currentSettings?.providers?.watsonx?.endpoint
+ }
+ existingProjectId={
+ currentSettings?.providers?.watsonx?.project_id
+ }
+ hasEnvApiKey={
+ currentSettings?.providers?.watsonx?.has_api_key ===
+ true
+ }
+ />
+ ) : providerKey === "ollama" ? (
+ <OllamaOnboarding
+ setSettings={setSettings}
+ isEmbedding={isEmbedding}
+ alreadyConfigured={
+ providerAlreadyConfigured &&
+ modelProvider === "ollama"
+ }
+ existingEndpoint={
+ currentSettings?.providers?.ollama?.endpoint
+ }
+ />
+ ) : (
+ <GenericOnboarding
+ provider={providerKey}
+ displayName={providerDisplayNames[providerKey]}
+ setSettings={setSettings}
+ isEmbedding={isEmbedding}
+ providers={currentSettings?.providers}
+ />
+ )}
</TabsContent>
- )}
+ ))}
</Tabs>
<Tooltip>
diff --git a/frontend/app/onboarding/_components/onboarding-step.tsx b/frontend/app/onboarding/_components/onboarding-step.tsx
index b2941f6b4..bf9cc2bbe 100644
--- a/frontend/app/onboarding/_components/onboarding-step.tsx
+++ b/frontend/app/onboarding/_components/onboarding-step.tsx
@@ -84,7 +84,7 @@ export function OnboardingStep({
initial={{ opacity: 0, y: -20 }}
animate={{ opacity: 1, y: 0 }}
transition={{ duration: 0.4, delay: 0.4, ease: "easeOut" }}
- className={isCompleted ? "opacity-50" : ""}
+ className={cn(isCompleted ? "opacity-50" : "", "pr-10 mr-1 flex w-full")}
>
<Message
icon={
diff --git a/frontend/app/settings/_components/ingest-settings-section.tsx b/frontend/app/settings/_components/ingest-settings-section.tsx
index 08aa61fb6..39a9ca5f2 100644
--- a/frontend/app/settings/_components/ingest-settings-section.tsx
+++ b/frontend/app/settings/_components/ingest-settings-section.tsx
@@ -3,7 +3,11 @@
import { ArrowUpRight, ChevronDown, Loader2, Minus, Plus } from "lucide-react";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { toast } from "sonner";
-import { useGetModelCatalogQuery } from "@/app/api/queries/useGetModelsQuery";
+import {
+ useGetIBMModelsQuery,
+ useGetModelCatalogQuery,
+ useGetOllamaModelsQuery,
+} from "@/app/api/queries/useGetModelsQuery";
import { useGetSettingsQuery } from "@/app/api/queries/useGetSettingsQuery";
import { ConfirmationDialog } from "@/components/confirmation-dialog";
import { LabelWrapper } from "@/components/label-wrapper";
@@ -49,6 +53,9 @@ import {
import {
findGroupedSelection,
groupedCatalogOptions,
+ LIVE_INVENTORY_PROVIDERS,
+ liveModelOption,
+ mergeLiveCatalogOptions,
} from "../_helpers/catalog-models";
import { getModelLogo } from "../_helpers/model-helpers";
import { LangflowIcon } from "./langflow-icon";
@@ -125,20 +132,71 @@ export function IngestSettingsSection() {
[settings.providers],
);
- const groupedEmbeddingModels = useMemo(
- () =>
- groupedCatalogOptions(catalog, configuredProviders, "embedding").map(
- (group) => ({
- group: group.group,
- provider: group.key,
- icon: getModelLogo("", group.key),
- options: group.options,
- }),
- ),
- [catalog, configuredProviders],
+ // watsonx and Ollama publish their inventory from the running server, not
+ // from LiteLLM's bundled table, which carries *no* embedding models for
+ // either of them — so the live list is the only thing that puts their group
+ // in this picker at all.
+ //
+ // Fetched whenever the provider is configured, never gated on it already
+ // being the selected embedding provider: the whole point of the group is to
+ // be there before you pick it. Gating on selection is why watsonx stayed
+ // missing after the first attempt at this.
+ const authed = isAuthenticated || isNoAuthMode;
+ const watsonx = settings.providers?.watsonx;
+ const ollama = settings.providers?.ollama;
+
+ const ibmModels = useGetIBMModelsQuery(
+ {
+ endpoint: watsonx?.endpoint,
+ projectId: watsonx?.project_id,
+ useEnvKey: true,
+ },
+ {
+ enabled:
+ authed &&
+ configuredProviders.watsonx &&
+ !!watsonx?.endpoint &&
+ !!watsonx?.project_id,
+ },
+ );
+ const ollamaModels = useGetOllamaModelsQuery(
+ { endpoint: ollama?.endpoint },
+ { enabled: authed && configuredProviders.ollama && !!ollama?.endpoint },
);
- const isLoadingAnyEmbeddingModels = catalogLoading;
+ const groupedEmbeddingModels = useMemo(() => {
+ const named = (key: string) =>
+ catalog?.providers?.find((provider) => provider.key === key)?.name;
+ const liveFor = (key: string, models?: { value: string }[]) =>
+ (models ?? []).map((model) => liveModelOption(model.value, key));
+
+ let groups = groupedCatalogOptions(
+ catalog,
+ configuredProviders,
+ "embedding",
+ );
+ const liveByProvider: Record<string, { value: string }[] | undefined> = {
+ watsonx: ibmModels.data?.embedding_models,
+ ollama: ollamaModels.data?.embedding_models,
+ };
+ for (const key of LIVE_INVENTORY_PROVIDERS) {
+ groups = mergeLiveCatalogOptions(
+ groups,
+ key,
+ liveFor(key, liveByProvider[key]),
+ named(key),
+ );
+ }
+ return groups.map((group) => ({
+ group: group.group,
+ provider: group.key,
+ icon: getModelLogo("", group.key),
+ options: group.options,
+ }));
+ }, [catalog, configuredProviders, ibmModels.data, ollamaModels.data]);
+
+ const isLoadingAnyEmbeddingModels =
+ catalogLoading || ibmModels.isLoading || ollamaModels.isLoading;
const groupedVlmModels = useMemo(() => {
const list: GroupedModelOption[] = groupedCatalogOptions(
diff --git a/frontend/app/settings/_components/model-providers.tsx b/frontend/app/settings/_components/model-providers.tsx
index b5cc970de..8fcf06c5b 100644
--- a/frontend/app/settings/_components/model-providers.tsx
+++ b/frontend/app/settings/_components/model-providers.tsx
@@ -1,51 +1,69 @@
"use client";
import { useRouter, useSearchParams } from "next/navigation";
-import { type ReactNode, useEffect, useMemo, useState } from "react";
+import { useEffect, useMemo, useState } from "react";
+import { useGetModelProvidersQuery } from "@/app/api/queries/useGetModelProvidersQuery";
import { useGetSettingsQuery } from "@/app/api/queries/useGetSettingsQuery";
-import AnthropicLogo from "@/components/icons/anthropic-logo";
-import IBMLogo from "@/components/icons/ibm-logo";
-import OllamaLogo from "@/components/icons/ollama-logo";
-import OpenAILogo from "@/components/icons/openai-logo";
import { useProviderHealth } from "@/components/provider-health-banner";
import { useAuth } from "@/contexts/auth-context";
-import { useIsCloudBrand } from "@/contexts/brand-context";
import {
- ALL_PROVIDERS,
- CLOUD_EXCLUDED_PROVIDERS,
+ getProviderChrome,
type ModelProvider,
} from "../_helpers/model-helpers";
import AnthropicSettingsDialog from "./anthropic-settings-dialog";
import ModelProviderCard from "./model-provider-card";
import OllamaSettingsDialog from "./ollama-settings-dialog";
import OpenAISettingsDialog from "./openai-settings-dialog";
+import ProviderSettingsDialog from "./provider-settings-dialog";
import WatsonxSettingsDialog from "./watsonx-settings-dialog";
+// Providers with a hand-built credential dialog. Everything else the backend
+// offers is configured through the catalogue-driven ProviderSettingsDialog.
+const BESPOKE_DIALOG_PROVIDERS = new Set<ModelProvider>([
+ "openai",
+ "anthropic",
+ "ollama",
+ "watsonx",
+]);
+
export const ModelProviders = () => {
const { isAuthenticated, isNoAuthMode } = useAuth();
const searchParams = useSearchParams();
const router = useRouter();
- const isCloudBrand = useIsCloudBrand();
const { data: settings = {} } = useGetSettingsQuery({
enabled: isAuthenticated || isNoAuthMode,
});
+ // The backend filters this list by OPENRAG_RUN_MODE (see
+ // config/model_providers.yaml). The UI keeps no denylist of its own, and the
+ // IBM theme no longer decides what is available.
+ const {
+ data: providerData,
+ isLoading: isLoadingProviders,
+ isError: providersFailed,
+ } = useGetModelProvidersQuery({
+ enabled: isAuthenticated || isNoAuthMode,
+ });
+
const { health } = useProviderHealth();
const [dialogOpen, setDialogOpen] = useState<ModelProvider | undefined>();
- const allProviderKeys = useMemo(() => {
- return isCloudBrand
- ? ALL_PROVIDERS.filter((p) => !CLOUD_EXCLUDED_PROVIDERS.includes(p))
- : ALL_PROVIDERS;
- }, [isCloudBrand]);
+ const providers = useMemo(
+ () => providerData?.providers ?? [],
+ [providerData],
+ );
+ const allProviderKeys = useMemo(
+ () => providers.map((provider) => provider.name),
+ [providers],
+ );
// Handle URL search param to open dialogs
useEffect(() => {
const searchParam = searchParams.get("setup");
- if (searchParam && allProviderKeys.includes(searchParam as ModelProvider)) {
- setDialogOpen(searchParam as ModelProvider);
+ if (searchParam && allProviderKeys.includes(searchParam)) {
+ setDialogOpen(searchParam);
}
}, [searchParams, allProviderKeys]);
@@ -61,78 +79,28 @@ export const ModelProviders = () => {
router.replace(newUrl);
};
- const modelProvidersMap: Record<
- ModelProvider,
- {
- name: string;
- logo: (props: React.SVGProps<SVGSVGElement>) => ReactNode;
- logoColor: string;
- logoBgColor: string;
- }
- > = {
- openai: {
- name: "OpenAI",
- logo: OpenAILogo,
- logoColor: "text-black",
- logoBgColor: "bg-white",
- },
- anthropic: {
- name: "Anthropic",
- logo: AnthropicLogo,
- logoColor: "text-[#D97757]",
- logoBgColor: "bg-white",
- },
- ollama: {
- name: "Ollama",
- logo: OllamaLogo,
- logoColor: "text-black",
- logoBgColor: "bg-white",
- },
- watsonx: {
- name: "IBM watsonx.ai",
- logo: IBMLogo,
- logoColor: "text-white",
- logoBgColor: "bg-[#1063FE]",
- },
- local: {
- name: "Local",
- logo: (props) => (
- <svg
- xmlns="http://www.w3.org/2000/svg"
- viewBox="0 0 24 24"
- fill="none"
- stroke="currentColor"
- strokeWidth="2"
- strokeLinecap="round"
- strokeLinejoin="round"
- {...props}
- >
- <rect x="4" y="4" width="16" height="16" rx="2" />
- <rect x="9" y="9" width="6" height="6" />
- <path d="M9 1v3" />
- <path d="M15 1v3" />
- <path d="M9 20v3" />
- <path d="M15 20v3" />
- <path d="M20 9h3" />
- <path d="M20 15h3" />
- <path d="M1 9h3" />
- <path d="M1 15h3" />
- </svg>
- ),
- logoColor: "text-muted-foreground",
- logoBgColor: "bg-white",
- },
- };
-
const currentLlmProvider =
(settings.agent?.llm_provider as ModelProvider) || "openai";
const currentEmbeddingProvider =
(settings.knowledge?.embedding_provider as ModelProvider) || "openai";
+ const genericDialogProvider =
+ dialogOpen && !BESPOKE_DIALOG_PROVIDERS.has(dialogOpen)
+ ? dialogOpen
+ : undefined;
+
+ if (providersFailed) {
+ return (
+ <p className="text-sm text-destructive">
+ The list of model providers could not be loaded. Refresh to try again.
+ </p>
+ );
+ }
+
return (
<>
<div className="grid gap-6 xs:grid-cols-1 md:grid-cols-2 lg:grid-cols-4">
- {allProviderKeys.map((providerKey) => {
+ {providers.map(({ name: providerKey, display_name }) => {
const isLlmProvider = providerKey === currentLlmProvider;
const isEmbeddingProvider = providerKey === currentEmbeddingProvider;
const isProviderUnhealthy =
@@ -142,30 +110,62 @@ export const ModelProviders = () => {
return (
<ModelProviderCard
key={providerKey}
- provider={{ providerKey, ...modelProvidersMap[providerKey] }}
- isConfigured={!!settings.providers?.[providerKey]?.configured}
+ provider={{
+ providerKey,
+ ...getProviderChrome(providerKey, display_name),
+ }}
+ // `providers.custom` carries every provider the backend knows,
+ // legacy four included, so one lookup covers config-added ones.
+ isConfigured={
+ settings.providers?.custom?.[providerKey]?.configured === true
+ }
isUnhealthy={!!isProviderUnhealthy}
onConfigure={setDialogOpen}
/>
);
})}
</div>
- <AnthropicSettingsDialog
- open={dialogOpen === "anthropic"}
- setOpen={handleCloseDialog}
- />
- <OpenAISettingsDialog
- open={dialogOpen === "openai"}
- setOpen={handleCloseDialog}
- />
- <OllamaSettingsDialog
- open={dialogOpen === "ollama"}
- setOpen={handleCloseDialog}
- />
- <WatsonxSettingsDialog
- open={dialogOpen === "watsonx"}
- setOpen={handleCloseDialog}
- />
+ {!isLoadingProviders && providers.length === 0 && (
+ <p className="text-sm text-muted-foreground">
+ No model providers are enabled for this deployment.
+ </p>
+ )}
+ {allProviderKeys.includes("anthropic") && (
+ <AnthropicSettingsDialog
+ open={dialogOpen === "anthropic"}
+ setOpen={handleCloseDialog}
+ />
+ )}
+ {allProviderKeys.includes("openai") && (
+ <OpenAISettingsDialog
+ open={dialogOpen === "openai"}
+ setOpen={handleCloseDialog}
+ />
+ )}
+ {allProviderKeys.includes("ollama") && (
+ <OllamaSettingsDialog
+ open={dialogOpen === "ollama"}
+ setOpen={handleCloseDialog}
+ />
+ )}
+ {allProviderKeys.includes("watsonx") && (
+ <WatsonxSettingsDialog
+ open={dialogOpen === "watsonx"}
+ setOpen={handleCloseDialog}
+ />
+ )}
+ {genericDialogProvider && (
+ <ProviderSettingsDialog
+ provider={genericDialogProvider}
+ displayName={
+ providers.find(
+ (provider) => provider.name === genericDialogProvider,
+ )?.display_name
+ }
+ open
+ setOpen={handleCloseDialog}
+ />
+ )}
</>
);
};
diff --git a/frontend/app/settings/_components/provider-settings-dialog.tsx b/frontend/app/settings/_components/provider-settings-dialog.tsx
new file mode 100644
index 000000000..aaa901b00
--- /dev/null
+++ b/frontend/app/settings/_components/provider-settings-dialog.tsx
@@ -0,0 +1,259 @@
+"use client";
+
+import { AnimatePresence, domAnimation, LazyMotion, m } from "motion/react";
+import { useRouter } from "next/navigation";
+import { useEffect, useState } from "react";
+import { FormProvider, useForm } from "react-hook-form";
+import { toast } from "sonner";
+import {
+ type AffectedEmbeddingModel,
+ isEmbeddingProviderInUseError,
+ useUpdateSettingsMutation,
+} from "@/app/api/mutations/useUpdateSettingsMutation";
+import { useGetModelCatalogQuery } from "@/app/api/queries/useGetModelsQuery";
+import { useGetSettingsQuery } from "@/app/api/queries/useGetSettingsQuery";
+import {
+ Dialog,
+ DialogContent,
+ DialogHeader,
+ DialogTitle,
+} from "@/components/ui/dialog";
+import { useAuth } from "@/contexts/auth-context";
+import type { CatalogCredentialField } from "../_helpers/catalog-models";
+import {
+ getProviderChrome,
+ type ModelProvider,
+} from "../_helpers/model-helpers";
+import ModelProviderDialogFooter from "./model-provider-dialog-footer";
+import {
+ ProviderSettingsForm,
+ type ProviderSettingsFormData,
+} from "./provider-settings-form";
+
+const EMPTY_FIELDS: CatalogCredentialField[] = [];
+
+/**
+ * Credential dialog for any provider without a bespoke one.
+ *
+ * Fields come from the catalogue's per-provider spec and are saved through the
+ * generic `provider_credentials` payload, so adding a provider row to
+ * `config/model_providers.yaml` is enough to make it configurable here.
+ *
+ * There is no live key check on save — OpenRAG has no generic
+ * "list this provider's models" endpoint to probe with. The credentials are
+ * validated the first time a model from this provider is selected in Agent or
+ * Ingestion settings, which reports the provider's own error.
+ */
+const ProviderSettingsDialog = ({
+ provider,
+ displayName,
+ open,
+ setOpen,
+}: {
+ provider: ModelProvider;
+ displayName?: string;
+ open: boolean;
+ setOpen: (open: boolean) => void;
+}) => {
+ const { isAuthenticated, isNoAuthMode } = useAuth();
+ const [showRemoveConfirm, setShowRemoveConfirm] = useState(false);
+ const [affectedModels, setAffectedModels] = useState<
+ AffectedEmbeddingModel[] | undefined
+ >(undefined);
+ const router = useRouter();
+
+ const { data: settings = {} } = useGetSettingsQuery({
+ enabled: isAuthenticated || isNoAuthMode,
+ });
+ const { data: catalog } = useGetModelCatalogQuery({
+ enabled: isAuthenticated || isNoAuthMode,
+ });
+
+ const chrome = getProviderChrome(provider, displayName);
+ const catalogEntry = catalog?.providers?.find(
+ (entry) => entry.key === provider,
+ );
+ const fields = catalogEntry?.credential_fields ?? EMPTY_FIELDS;
+
+ const saved = settings.providers?.custom?.[provider];
+ const isConfigured = saved?.configured === true;
+ const savedSecretFields = saved?.secret_fields ?? [];
+
+ // Removing the last configured provider would leave the agent with nothing
+ // to call, so mirror the bespoke dialogs and require another one first.
+ const canRemove =
+ isConfigured &&
+ Object.entries(settings.providers?.custom ?? {}).some(
+ ([key, value]) => key !== provider && value?.configured === true,
+ );
+
+ const methods = useForm<ProviderSettingsFormData>({
+ mode: "onSubmit",
+ defaultValues: { credentials: {} },
+ });
+
+ // Seed on open, and again if the catalogue or the saved settings land after
+ // it. Both are react-query results, so their identity is stable while the
+ // dialog is open and this cannot stomp on what the user is typing.
+ useEffect(() => {
+ if (!open) {
+ return;
+ }
+ methods.reset({
+ credentials: Object.fromEntries(
+ fields.map((field) => [
+ field.key,
+ saved?.credential_values?.[field.key] ?? "",
+ ]),
+ ),
+ });
+ }, [open, fields, saved, methods]);
+
+ const { handleSubmit } = methods;
+
+ const settingsMutation = useUpdateSettingsMutation({
+ onSuccess: () => {
+ toast.message(`${chrome.name} successfully configured`, {
+ description: "You can now select its models in Settings.",
+ duration: Infinity,
+ closeButton: true,
+ action: {
+ label: "Settings",
+ onClick: () => {
+ router.push("/settings/agent?focusLlmModel=true");
+ },
+ },
+ });
+ setOpen(false);
+ },
+ });
+
+ const removeMutation = useUpdateSettingsMutation({
+ onSuccess: () => {
+ toast.success(`${chrome.name} configuration removed`);
+ setShowRemoveConfirm(false);
+ setAffectedModels(undefined);
+ setOpen(false);
+ },
+ onError: (err) => {
+ if (isEmbeddingProviderInUseError(err)) {
+ setAffectedModels(err.affectedModels);
+ }
+ },
+ });
+
+ const onSubmit = (data: ProviderSettingsFormData) => {
+ // Blank means "leave the stored value alone": the backend ignores empty
+ // values, and secrets are never echoed back for us to resubmit.
+ const credentials: Record<string, string> = {};
+ for (const [key, value] of Object.entries(data.credentials ?? {})) {
+ const trimmed = (value ?? "").trim();
+ if (trimmed !== "") {
+ credentials[key] = trimmed;
+ }
+ }
+
+ if (Object.keys(credentials).length === 0) {
+ methods.setError("root", { message: "Enter at least one credential" });
+ return;
+ }
+
+ settingsMutation.mutate({
+ provider_credentials: { [provider]: credentials },
+ });
+ };
+
+ const Logo = chrome.logo;
+
+ return (
+ <Dialog
+ open={open}
+ onOpenChange={(o) => {
+ setShowRemoveConfirm(false);
+ setAffectedModels(undefined);
+ setOpen(o);
+ }}
+ >
+ <DialogContent autoFocus={false} className="max-w-2xl overflow-hidden">
+ <FormProvider {...methods}>
+ <form
+ onSubmit={handleSubmit(onSubmit)}
+ className="grid min-w-0 gap-4"
+ >
+ <DialogHeader className="mb-2">
+ <DialogTitle className="flex items-center gap-3">
+ <div className="w-8 h-8 rounded flex items-center justify-center bg-white border">
+ <Logo className="w-4 h-4 text-black" />
+ </div>
+ {chrome.name} Setup
+ </DialogTitle>
+ </DialogHeader>
+
+ <ProviderSettingsForm
+ providerName={chrome.name}
+ fields={fields}
+ savedSecretFields={savedSecretFields}
+ saveError={methods.formState.errors.root?.message}
+ />
+
+ <LazyMotion features={domAnimation}>
+ <AnimatePresence mode="wait">
+ {settingsMutation.isError && (
+ <m.div
+ key="error"
+ initial={{ opacity: 0, y: 10 }}
+ animate={{ opacity: 1, y: 0 }}
+ exit={{ opacity: 0, y: -10 }}
+ >
+ <p className="rounded-lg border border-destructive p-4 min-w-0 [overflow-wrap:anywhere]">
+ {settingsMutation.error?.message}
+ </p>
+ </m.div>
+ )}
+ {removeMutation.isError &&
+ !isEmbeddingProviderInUseError(removeMutation.error) && (
+ <m.div
+ key="remove-error"
+ initial={{ opacity: 0, y: 10 }}
+ animate={{ opacity: 1, y: 0 }}
+ exit={{ opacity: 0, y: -10 }}
+ >
+ <p className="rounded-lg border border-destructive p-4 min-w-0 [overflow-wrap:anywhere]">
+ {removeMutation.error?.message}
+ </p>
+ </m.div>
+ )}
+ </AnimatePresence>
+ </LazyMotion>
+
+ <ModelProviderDialogFooter
+ showRemoveConfirm={showRemoveConfirm}
+ onCancelRemove={() => {
+ setShowRemoveConfirm(false);
+ setAffectedModels(undefined);
+ }}
+ onConfirmRemove={() =>
+ removeMutation.mutate({
+ remove_provider_config: provider,
+ force_remove: !!affectedModels,
+ })
+ }
+ isRemovePending={removeMutation.isPending}
+ isConfigured={isConfigured}
+ canRemove={canRemove}
+ providerKey={provider}
+ removeDisabledTooltip={`Configure another model provider before removing ${chrome.name}`}
+ onRequestRemove={() => setShowRemoveConfirm(true)}
+ onCancel={() => setOpen(false)}
+ isSavePending={settingsMutation.isPending}
+ isValidating={false}
+ affectedModels={affectedModels}
+ />
+ </form>
+ </FormProvider>
+ </DialogContent>
+ </Dialog>
+ );
+};
+
+export default ProviderSettingsDialog;
diff --git a/frontend/app/settings/_components/provider-settings-form.tsx b/frontend/app/settings/_components/provider-settings-form.tsx
new file mode 100644
index 000000000..03091140b
--- /dev/null
+++ b/frontend/app/settings/_components/provider-settings-form.tsx
@@ -0,0 +1,113 @@
+import { useFormContext } from "react-hook-form";
+import type { CatalogCredentialField } from "@/app/settings/_helpers/catalog-models";
+import { LabelWrapper } from "@/components/label-wrapper";
+import { Input } from "@/components/ui/input";
+import { Textarea } from "@/components/ui/textarea";
+
+export interface ProviderSettingsFormData {
+ credentials: Record<string, string>;
+}
+
+/**
+ * Credential form for a provider with no bespoke dialog, rendered from the
+ * field spec `/api/models/catalog` publishes for it. A provider added to
+ * `config/model_providers.yaml` is configurable through this without any
+ * further frontend work.
+ */
+export function ProviderSettingsForm({
+ providerName,
+ fields,
+ savedSecretFields,
+ saveError,
+}: {
+ providerName: string;
+ fields: CatalogCredentialField[];
+ savedSecretFields: string[];
+ saveError?: string | null;
+}) {
+ const {
+ register,
+ formState: { errors },
+ } = useFormContext<ProviderSettingsFormData>();
+
+ const saved = new Set(savedSecretFields);
+
+ return (
+ <div className="min-w-0 space-y-4">
+ {fields.map((field) => {
+ const error = errors.credentials?.[field.key]?.message;
+ const isSecret =
+ field.field_type === "password" || field.field_type === "textarea";
+ const hasSaved = isSecret && saved.has(field.key);
+ // A stored secret is never sent back to the browser, so an empty box
+ // means "keep what is saved" rather than "no value" — required only
+ // bites when nothing is stored yet.
+ const required = field.required && !hasSaved;
+
+ return (
+ <div key={field.key} className="min-w-0 space-y-2">
+ <LabelWrapper
+ label={field.label}
+ helperText={field.tooltip ?? undefined}
+ required={required}
+ id={`provider-field-${field.key}`}
+ >
+ {field.field_type === "textarea" ? (
+ <Textarea
+ {...register(`credentials.${field.key}`, {
+ required: required ? `${field.label} is required` : false,
+ })}
+ className={error ? "!border-destructive" : ""}
+ id={`provider-field-${field.key}`}
+ placeholder={
+ hasSaved ? "•••••••••" : (field.placeholder ?? undefined)
+ }
+ />
+ ) : (
+ <Input
+ {...register(`credentials.${field.key}`, {
+ required: required ? `${field.label} is required` : false,
+ })}
+ className={error ? "!border-destructive" : ""}
+ id={`provider-field-${field.key}`}
+ type={field.field_type === "password" ? "password" : "text"}
+ autoComplete={
+ field.field_type === "password" ? "new-password" : "off"
+ }
+ placeholder={
+ hasSaved ? "•••••••••" : (field.placeholder ?? undefined)
+ }
+ />
+ )}
+ </LabelWrapper>
+ {hasSaved && (
+ <p className="text-sm text-muted-foreground">
+ A value is already saved. Leave this blank to keep it.
+ </p>
+ )}
+ {error && (
+ <p
+ data-testid="provider-connection-error"
+ className="text-sm text-destructive min-w-0 [overflow-wrap:anywhere]"
+ >
+ {error}
+ </p>
+ )}
+ </div>
+ );
+ })}
+ {saveError && (
+ <p
+ data-testid="provider-connection-error"
+ className="text-sm text-destructive min-w-0 [overflow-wrap:anywhere]"
+ >
+ {saveError}
+ </p>
+ )}
+ <p className="text-sm text-muted-foreground">
+ Pick the {providerName} models to use from the Agent and Ingestion
+ settings after saving these credentials.
+ </p>
+ </div>
+ );
+}
diff --git a/frontend/app/settings/_helpers/catalog-models.test.ts b/frontend/app/settings/_helpers/catalog-models.test.ts
index 4a9906f25..a698deca2 100644
--- a/frontend/app/settings/_helpers/catalog-models.test.ts
+++ b/frontend/app/settings/_helpers/catalog-models.test.ts
@@ -3,6 +3,7 @@ import { describe, it } from "node:test";
import {
findGroupedSelection,
groupedCatalogOptions,
+ liveModelOption,
mergeLiveCatalogOptions,
onboardingCatalogConfigured,
onboardingCredentialFields,
@@ -280,3 +281,70 @@ describe("providerCredentialsSatisfied", () => {
);
});
});
+
+describe("newest models first", () => {
+ const provider = (models: Array<{ model: string }>) => ({
+ providers: [{ key: "p", name: "P", models, embedding_models: [] }],
+ });
+ const order = (names: string[]) =>
+ groupedCatalogOptions(
+ provider(
+ names.map((model) => ({ model, capabilities: ["function_calling"] })),
+ ),
+ undefined,
+ "language",
+ )[0].options.map((option) => option.value);
+
+ it("ranks by version, newest first", () => {
+ assert.deepEqual(order(["gpt-4o", "gpt-3.5-turbo", "gpt-5", "gpt-4.1"]), [
+ "gpt-5",
+ "gpt-4.1",
+ "gpt-4o",
+ "gpt-3.5-turbo",
+ ]);
+ });
+
+ it("puts a bare alias above its own dated snapshot", () => {
+ // `gpt-5` and `gpt-5-2025-08-07` are the same model; the alias is the one
+ // to offer, so a missing stamp has to rank as newer, not older.
+ assert.deepEqual(order(["gpt-5-2025-08-07", "gpt-5"]), [
+ "gpt-5",
+ "gpt-5-2025-08-07",
+ ]);
+ });
+
+ it("orders snapshots of one version by their stamp", () => {
+ assert.deepEqual(
+ order(["claude-opus-4-5-20251101", "claude-opus-4-5-20260401"]),
+ ["claude-opus-4-5-20260401", "claude-opus-4-5-20251101"],
+ );
+ });
+
+ it("falls back to alphabetical when nothing is versioned", () => {
+ assert.deepEqual(order(["mistral-large", "granite-instruct"]), [
+ "granite-instruct",
+ "mistral-large",
+ ]);
+ });
+});
+
+describe("mergeLiveCatalogOptions creates a missing group", () => {
+ it("adds watsonx embeddings the bundled table does not carry", () => {
+ // LiteLLM lists zero watsonx embedding models, so groupedCatalogOptions
+ // drops the group entirely — the live rows are the only ones there are.
+ const merged = mergeLiveCatalogOptions(
+ [],
+ "watsonx",
+ [liveModelOption("ibm/slate-125m-english-rtrvr", "watsonx")],
+ "IBM watsonx.ai",
+ );
+
+ assert.equal(merged.length, 1);
+ assert.equal(merged[0].key, "watsonx");
+ assert.equal(merged[0].group, "IBM watsonx.ai");
+ assert.deepEqual(
+ merged[0].options.map((option) => option.value),
+ ["ibm/slate-125m-english-rtrvr"],
+ );
+ });
+});
diff --git a/frontend/app/settings/_helpers/catalog-models.ts b/frontend/app/settings/_helpers/catalog-models.ts
index 2984a2058..0675f2d94 100644
--- a/frontend/app/settings/_helpers/catalog-models.ts
+++ b/frontend/app/settings/_helpers/catalog-models.ts
@@ -92,12 +92,46 @@ const PREFERRED_EMBEDDING_MODELS = [
"text-embedding-3-large",
];
+/** A YYYY-MM-DD or YYYYMMDD snapshot stamp inside a model id. */
+const SNAPSHOT_DATE = /(20\d{2})-?(\d{2})-?(\d{2})(?!\d)/;
+
+/**
+ * How recent a model id looks, most-recent-first. Nothing in LiteLLM's data
+ * says when a model shipped — `deprecation_date` is the only date it carries,
+ * and it is not a stand-in: Anthropic retires on a rolling year so it tracks
+ * release almost exactly, but OpenAI sunsets a decade of models on one shared
+ * date, which would rank gpt-3.5-turbo above gpt-5. So recency is read off the
+ * id itself.
+ *
+ * Version before snapshot, deliberately. `gpt-5` and `gpt-5-2025-08-07` are
+ * the same model, and the bare alias is the one to offer — so an id with no
+ * stamp sorts *above* its dated snapshots rather than below them.
+ *
+ * Providers that number by size rather than version (watsonx's `jais-13b`,
+ * `granite-3-8b`) get an ordering that means nothing in particular. It is
+ * stable and alphabetical within a version, which is all it needs to be.
+ */
+function recencyRank(name: string): Array<number> {
+ const stamp = SNAPSHOT_DATE.exec(name);
+ // Strip the stamp before reading version digits, or 2025 becomes a version.
+ const withoutStamp = stamp ? name.replace(stamp[0], "") : name;
+ const version = (withoutStamp.match(/\d+(?:\.\d+)?/g) ?? [])
+ .slice(0, 3)
+ .map(Number);
+ while (version.length < 3) version.push(-1);
+
+ const stampRank = stamp
+ ? -Number(`${stamp[1]}${stamp[2]}${stamp[3]}`)
+ : -Number.MAX_SAFE_INTEGER;
+
+ return [-version[0], -version[1], -version[2], stampRank];
+}
+
function optionRank(
option: CatalogSelectOption,
kind: CatalogModelKind,
): Array<number | string> {
const name = option.value;
- const lower = name.toLowerCase();
const caps = option.model?.capabilities ?? [];
const preferred =
kind === "embedding"
@@ -105,10 +139,17 @@ function optionRank(
: PREFERRED_LANGUAGE_MODELS;
const preferredIndex = preferred.indexOf(name);
return [
- lower.startsWith("ft:") ? 1 : 0,
+ // The catalogue no longer carries LiteLLM's `ft:` pricing templates, but a
+ // live `/models/openai` fetch returns the operator's real fine-tunes. Those
+ // are callable and stay listed — just never above a base model.
+ name.toLowerCase().startsWith("ft:") ? 1 : 0,
caps.length === 0 ? 1 : 0,
kind === "language" && !caps.includes("function_calling") ? 1 : 0,
kind === "vision" && !caps.includes("vision") ? 1 : 0,
+ ...recencyRank(name),
+ // Only reached by ids that look equally recent: the embedding lists are
+ // undated and unversioned, so this is what still puts a validated default
+ // on top there.
preferredIndex === -1 ? preferred.length : preferredIndex,
name,
];
@@ -223,15 +264,32 @@ export function onboardingCatalogConfigured(
return undefined;
}
-/** Append live /models/{provider} rows that the catalogue does not already list. */
+/**
+ * Append live `/models/{provider}` rows that the catalogue does not list.
+ *
+ * Some providers' inventory is only knowable from the running server: watsonx
+ * publishes *no* embedding models in LiteLLM's bundled table at all, and
+ * Ollama serves whatever the operator pulled. Without this the watsonx group
+ * is empty for embeddings, and `groupedCatalogOptions` then drops the group
+ * entirely — which is why watsonx was missing from the embedding picker
+ * rather than merely short of options. So the merge creates the group when
+ * the catalogue produced none.
+ */
export function mergeLiveCatalogOptions(
groups: CatalogOptionGroup[],
provider: string | undefined,
live: CatalogSelectOption[],
+ displayName?: string,
): CatalogOptionGroup[] {
if (!provider || live.length === 0) {
return groups;
}
+ if (!groups.some((group) => group.key === provider)) {
+ return [
+ ...groups,
+ { key: provider, group: displayName || provider, options: live },
+ ];
+ }
return groups.map((group) => {
if (group.key !== provider) {
return group;
@@ -245,6 +303,21 @@ export function mergeLiveCatalogOptions(
});
}
+/**
+ * Providers whose model list comes from the running server, not from
+ * LiteLLM's bundled table. watsonx publishes no embedding models in that
+ * table at all, and Ollama serves whatever the operator pulled locally.
+ */
+export const LIVE_INVENTORY_PROVIDERS = new Set(["watsonx", "ollama"]);
+
+/** A live `/models/{provider}` row as a picker option. */
+export function liveModelOption(
+ model: string,
+ provider: string,
+): CatalogSelectOption {
+ return { value: model, label: model, provider };
+}
+
/**
* Credential controls for onboarding come directly from LiteLLM's provider
* schema. The backend persists and encrypts arbitrary field keys.
diff --git a/frontend/app/settings/_helpers/model-helpers.test.ts b/frontend/app/settings/_helpers/model-helpers.test.ts
new file mode 100644
index 000000000..3c959e287
--- /dev/null
+++ b/frontend/app/settings/_helpers/model-helpers.test.ts
@@ -0,0 +1,81 @@
+/**
+ * Provider ordering and chrome.
+ *
+ * Which providers exist is the backend's call (config/model_providers.yaml
+ * filtered by OPENRAG_RUN_MODE); the frontend only decides what order to show
+ * them in and what to call them. These pin that a provider the backend adds
+ * still renders, and that a hidden one can never be re-introduced by the
+ * preference lists.
+ */
+
+import assert from "node:assert/strict";
+import { describe, it } from "node:test";
+import {
+ EMBEDDING_PROVIDER_ORDER,
+ getProviderChrome,
+ LLM_PROVIDER_ORDER,
+ orderProviders,
+} from "./model-helpers";
+
+describe("orderProviders", () => {
+ it("ranks preferred providers first and keeps the rest in API order", () => {
+ assert.deepEqual(
+ orderProviders(
+ ["openai", "watsonx", "azure_ai", "anthropic"],
+ ["anthropic", "openai"],
+ ),
+ ["anthropic", "openai", "watsonx", "azure_ai"],
+ );
+ });
+
+ it("never adds a provider the backend did not offer", () => {
+ // SaaS hides Ollama, so no preference list may put it back.
+ assert.deepEqual(
+ orderProviders(["openai", "watsonx"], LLM_PROVIDER_ORDER),
+ ["openai", "watsonx"],
+ );
+ assert.deepEqual(orderProviders(["openai"], EMBEDDING_PROVIDER_ORDER), [
+ "openai",
+ ]);
+ });
+
+ it("keeps a provider that no preference list mentions", () => {
+ assert.deepEqual(orderProviders(["azure_ai"], LLM_PROVIDER_ORDER), [
+ "azure_ai",
+ ]);
+ });
+});
+
+describe("getProviderChrome", () => {
+ it("names an unknown provider by its API display name", () => {
+ assert.equal(getProviderChrome("groq", "Groq Cloud").name, "Groq Cloud");
+ });
+
+ it("falls back to the provider key when the API sent no name", () => {
+ assert.equal(getProviderChrome("groq").name, "groq");
+ });
+
+ it("lets the API display name override a built-in label", () => {
+ assert.equal(getProviderChrome("openai").name, "OpenAI");
+ assert.equal(
+ getProviderChrome("openai", "House Gateway").name,
+ "House Gateway",
+ );
+ });
+
+ it("gives each Azure service its own Microsoft mark", () => {
+ // Foundry and Azure OpenAI are drawn differently by Microsoft, and neither
+ // is the generic Azure logo. `azure` had no chrome row at all, so its card
+ // fell through to the placeholder while the model rows beside it were
+ // already branded.
+ const foundry = getProviderChrome("azure_ai");
+ const openai = getProviderChrome("azure");
+ const placeholder = getProviderChrome("groq").logo;
+
+ assert.equal(foundry.name, "Azure AI Foundry");
+ assert.equal(openai.name, "Azure OpenAI");
+ assert.notEqual(foundry.logo, openai.logo);
+ assert.notEqual(foundry.logo, placeholder);
+ assert.notEqual(openai.logo, placeholder);
+ });
+});
diff --git a/frontend/app/settings/_helpers/model-helpers.tsx b/frontend/app/settings/_helpers/model-helpers.tsx
index cd8d4f3ef..96ffccb71 100644
--- a/frontend/app/settings/_helpers/model-helpers.tsx
+++ b/frontend/app/settings/_helpers/model-helpers.tsx
@@ -1,24 +1,37 @@
+import AiFoundryLogo from "@/components/icons/ai-foundry-logo";
import AnthropicLogo from "@/components/icons/anthropic-logo";
+import AzureOpenAILogo from "@/components/icons/azure-openai-logo";
+import GenericProviderLogo from "@/components/icons/generic-provider-logo";
import IBMLogo from "@/components/icons/ibm-logo";
import OllamaLogo from "@/components/icons/ollama-logo";
import OpenAILogo from "@/components/icons/openai-logo";
-export type ModelProvider =
- | "openai"
- | "anthropic"
- | "ollama"
- | "watsonx"
- | "local";
+/**
+ * A provider key as the backend names it.
+ *
+ * Which providers exist — and which of them this deployment shows — comes from
+ * `GET /api/models/providers`, driven by `config/model_providers.yaml` and
+ * OPENRAG_RUN_MODE. So this is a plain string, not a closed union: a provider
+ * added to that file must render without a frontend change. The keys below are
+ * only the ones OpenRAG has bespoke chrome (logo, colours, forms) for.
+ */
+export type ModelProvider = string;
-// Full ordered list of providers for settings / cards
-export const ALL_PROVIDERS: ModelProvider[] = [
+export const KNOWN_PROVIDERS = [
"openai",
+ "anthropic",
"ollama",
"watsonx",
- "anthropic",
-];
+ "azure_ai",
+ "azure",
+ "local",
+] as const;
-// Preferred auto-select order for the LLM onboarding step
+export type KnownModelProvider = (typeof KNOWN_PROVIDERS)[number];
+
+// Preferred auto-select order for the LLM onboarding step. Only a preference:
+// providers this run mode hides are dropped, and anything the API returns that
+// is not listed here is appended in API order.
export const LLM_PROVIDER_ORDER: ModelProvider[] = [
"anthropic",
"openai",
@@ -33,14 +46,107 @@ export const EMBEDDING_PROVIDER_ORDER: ModelProvider[] = [
"ollama",
];
-// Providers unavailable in cloud (IBM) deployments
-export const CLOUD_EXCLUDED_PROVIDERS: ModelProvider[] = ["ollama"];
+/**
+ * `preferred` first (skipping anything not in `available`), then whatever else
+ * `available` holds, in the order the backend returned it.
+ */
+export function orderProviders(
+ available: ModelProvider[],
+ preferred: ModelProvider[],
+): ModelProvider[] {
+ const offered = new Set(available);
+ const ranked = preferred.filter((provider) => offered.has(provider));
+ const seen = new Set(ranked);
+ return [...ranked, ...available.filter((provider) => !seen.has(provider))];
+}
export interface ModelOption {
value: string;
label: string;
}
+export interface ProviderChrome {
+ /** Display name; the API's `display_name` wins over the built-in label. */
+ name: string;
+ logo: (props: React.SVGProps<SVGSVGElement>) => React.ReactNode;
+ logoColor: string;
+ logoBgColor: string;
+ /** Onboarding tabs invert some marks; defaults to the card colours. */
+ tabLogoColor?: string;
+ tabLogoBgColor?: string;
+}
+
+/** Logo and colours for the providers OpenRAG ships chrome for. */
+const PROVIDER_CHROME: Record<string, ProviderChrome> = {
+ openai: {
+ name: "OpenAI",
+ logo: OpenAILogo,
+ logoColor: "text-black",
+ logoBgColor: "bg-white",
+ },
+ anthropic: {
+ name: "Anthropic",
+ logo: AnthropicLogo,
+ logoColor: "text-[#D97757]",
+ logoBgColor: "bg-white",
+ tabLogoColor: "text-black",
+ tabLogoBgColor: "bg-[#D97757]",
+ },
+ ollama: {
+ name: "Ollama",
+ logo: OllamaLogo,
+ logoColor: "text-black",
+ logoBgColor: "bg-white",
+ },
+ watsonx: {
+ name: "IBM watsonx.ai",
+ logo: IBMLogo,
+ logoColor: "text-white",
+ logoBgColor: "bg-[#1063FE]",
+ },
+ // Microsoft draws the two Azure model services differently, and neither is
+ // the generic Azure logo. Both marks paint their own gradients, so logoColor
+ // does nothing for them and the tile stays white.
+ azure_ai: {
+ name: "Azure AI Foundry",
+ logo: AiFoundryLogo,
+ logoColor: "text-black",
+ logoBgColor: "bg-white",
+ },
+ azure: {
+ name: "Azure OpenAI",
+ logo: AzureOpenAILogo,
+ logoColor: "text-black",
+ logoBgColor: "bg-white",
+ },
+ local: {
+ name: "Local",
+ logo: GenericProviderLogo,
+ logoColor: "text-muted-foreground",
+ logoBgColor: "bg-white",
+ },
+};
+
+/**
+ * Chrome for `provider`. A provider the config file adds but the frontend has
+ * no artwork for still renders, under the display name the API gave it.
+ */
+export function getProviderChrome(
+ provider: ModelProvider,
+ displayName?: string,
+): ProviderChrome {
+ const known = PROVIDER_CHROME[provider];
+ if (known) {
+ return displayName ? { ...known, name: displayName } : known;
+ }
+ return {
+ name: displayName || provider,
+ logo: GenericProviderLogo,
+ logoColor: "text-black",
+ logoBgColor: "bg-white",
+ };
+}
+
// Helper function to get model logo based on provider or model name
export function getModelLogo(modelValue: string, provider?: string) {
// First check by provider
@@ -52,6 +158,10 @@ export function getModelLogo(modelValue: string, provider?: string) {
return <OllamaLogo className="w-4 h-4" />;
} else if (provider === "watsonx") {
return <IBMLogo className="w-4 h-4" />;
+ } else if (provider === "azure") {
+ return <AzureOpenAILogo className="w-4 h-4" />;
+ } else if (provider === "azure_ai") {
+ return <AiFoundryLogo className="w-4 h-4" />;
} else if (provider === "local") {
return (
<svg
diff --git a/frontend/components/icons/ai-foundry-logo.tsx b/frontend/components/icons/ai-foundry-logo.tsx
new file mode 100644
index 000000000..2d627a682
--- /dev/null
+++ b/frontend/components/icons/ai-foundry-logo.tsx
@@ -0,0 +1,102 @@
+/**
+ * Azure AI Foundry, as Microsoft draws it (az-icons.com/icon/ai-foundry).
+ *
+ * Distinct from `azure-openai-logo`: Microsoft draws the two Azure model
+ * services differently. Gradient ids are fixed rather than generated: every
+ * instance defines the
+ * same stops, so when several render on one page and the browser resolves
+ * `url(#...)` to the first match, they all paint identically.
+ */
+export default function AiFoundryLogo(props: React.SVGProps<SVGSVGElement>) {
+ return (
+ <svg
+ xmlns="http://www.w3.org/2000/svg"
+ width="16"
+ height="16"
+ viewBox="0 0 18 18"
+ fill="none"
+ {...props}
+ >
+ <title>Azure AI Foundry Logo</title>
+ <defs>
+ <linearGradient
+ id="ai-foundry-flame-base"
+ x1="10.91"
+ x2="13.62"
+ y1="13.66"
+ y2="13.66"
+ gradientTransform="matrix(1 0 0 -1 0 20)"
+ gradientUnits="userSpaceOnUse"
+ >
+ <stop offset="0" stopColor="#302ec9" />
+ <stop offset=".12" stopColor="#151698" />
+ <stop offset=".25" stopColor="#0a0b7e" />
+ <stop offset=".43" stopColor="#090c7f" />
+ <stop offset=".51" stopColor="#141698" />
+ <stop offset=".61" stopColor="#282ac4" />
+ <stop offset=".77" stopColor="#8e8ff4" />
+ <stop offset=".85" stopColor="#a7a6f7" />
+ <stop offset=".91" stopColor="#cacafb" />
+ <stop offset="1" stopColor="#fff" />
+ </linearGradient>
+ <linearGradient
+ id="ai-foundry-flame-overlay"
+ x1="12.26"
+ x2="12.26"
+ y1="8.32"
+ y2="18.23"
+ gradientTransform="matrix(1 0 0 -1 0 20)"
+ gradientUnits="userSpaceOnUse"
+ >
+ <stop offset=".02" stopColor="#201ba6" />
+ <stop offset=".64" stopColor="#2d29c7" />
+ <stop offset="1" stopColor="#201ba6" />
+ </linearGradient>
+ <linearGradient
+ id="ai-foundry-crucible"
+ x1="14.19"
+ x2="14.19"
+ y1="8.27"
+ y2="14.47"
+ gradientTransform="matrix(1 0 0 -1 0 20)"
+ gradientUnits="userSpaceOnUse"
+ >
+ <stop offset="0" stopColor="#3530c3" />
+ <stop offset=".5" stopColor="#6d71d1" />
+ <stop offset=".97" stopColor="#c0bff5" />
+ <stop offset="1" stopColor="#e3e4ff" />
+ </linearGradient>
+ <linearGradient
+ id="ai-foundry-body"
+ x1="6.51"
+ x2="6.51"
+ y1="3"
+ y2="19.15"
+ gradientTransform="matrix(1 0 0 -1 0 20)"
+ gradientUnits="userSpaceOnUse"
+ >
+ <stop offset="0" stopColor="#302ec9" />
+ <stop offset=".45" stopColor="#302ec9" />
+ <stop offset=".95" stopColor="#cacafb" />
+ <stop offset="1" stopColor="#7f7eaf" />
+ </linearGradient>
+ </defs>
+ <path
+ fill="url(#ai-foundry-flame-base)"
+ d="M11.96 11.68s.07-.1.07-.24V7.57c0-.85.66-1.76 1.59-2.03-.06-.19-.64-2.11-.81-2.65-.17-.56-.67-1.9-1.29-1.9h-.01s-.09.05-.13.13c-.32.58-.46 3.02-.46 4.84v2.26c.27.97.91 3.22.96 3.35 0 0 .05.1.09.1Z"
+ />
+ <path
+ fill="url(#ai-foundry-flame-overlay)"
+ d="M11.96 11.68s.07-.1.07-.24V7.57c0-.85.66-1.76 1.59-2.03-.06-.19-.64-2.11-.81-2.65-.17-.56-.67-1.9-1.29-1.9h-.01s-.09.05-.13.13c-.32.58-.46 3.02-.46 4.84v2.26c.27.97.91 3.22.96 3.35 0 0 .05.1.09.1Z"
+ />
+ <path
+ fill="url(#ai-foundry-crucible)"
+ d="M16.01 5.46H14.2c-1.21 0-2.17 1.09-2.17 2.11v3.87c0 .14-.04.24-.07.24s-.09-.1-.09-.1.07.21.17.21h1.98c.62 0 2.48-.61 2.48-2.52V6.02c0-.41-.24-.55-.49-.55Z"
+ />
+ <path
+ fill="url(#ai-foundry-body)"
+ d="M2.34 17h5.38c2.35 0 3.2-2.03 3.2-3.81V5.96c0-2.08.19-4.97.61-4.97H8.3c-.77 0-1.43.82-2.18 2.48S1.76 15.34 1.6 15.83c-.26.79-.02 1.16.74 1.16Z"
+ />
+ </svg>
+ );
+}
diff --git a/frontend/components/icons/azure-openai-logo.tsx b/frontend/components/icons/azure-openai-logo.tsx
new file mode 100644
index 000000000..8b8762f4b
--- /dev/null
+++ b/frontend/components/icons/azure-openai-logo.tsx
@@ -0,0 +1,40 @@
+/**
+ * Azure OpenAI Service, as Microsoft draws it (az-icons.com/icon/azure-openai).
+ *
+ * Distinct from `ai-foundry-logo`: Microsoft draws the two Azure model services
+ * differently. The gradient id is fixed rather than generated: every instance
+ * defines the same stops, so when a page renders several of these and the
+ * browser resolves `url(#...)` to the first one, they all paint identically.
+ */
+export default function AzureOpenAILogo(props: React.SVGProps<SVGSVGElement>) {
+ return (
+ <svg
+ xmlns="http://www.w3.org/2000/svg"
+ width="16"
+ height="16"
+ viewBox="0 0 18 18"
+ fill="none"
+ {...props}
+ >
+ <title>Azure OpenAI Logo</title>
+ <defs>
+ <radialGradient
+ id="azure-openai-mark"
+ cx="-67.981"
+ cy="793.199"
+ r=".45"
+ gradientTransform="translate(-17939.03 20368.029) rotate(45) scale(25.091 -34.149)"
+ gradientUnits="userSpaceOnUse"
+ >
+ <stop offset="0" stopColor="#83b9f9" />
+ <stop offset="1" stopColor="#0078d4" />
+ </radialGradient>
+ </defs>
+ <path
+ d="m0,2.7v12.6c0,1.491,1.209,2.7,2.7,2.7h12.6c1.491,0,2.7-1.209,2.7-2.7V2.7c0-1.491-1.209-2.7-2.7-2.7H2.7C1.209,0,0,1.209,0,2.7ZM10.8,0v3.6c0,3.976,3.224,7.2,7.2,7.2h-3.6c-3.976,0-7.199,3.222-7.2,7.198v-3.598c0-3.976-3.224-7.2-7.2-7.2h3.6c3.976,0,7.2-3.224,7.2-7.2Z"
+ fill="url(#azure-openai-mark)"
+ strokeWidth="0"
+ />
+ </svg>
+ );
+}
diff --git a/frontend/components/icons/generic-provider-logo.tsx b/frontend/components/icons/generic-provider-logo.tsx
new file mode 100644
index 000000000..ed447e1c9
--- /dev/null
+++ b/frontend/components/icons/generic-provider-logo.tsx
@@ -0,0 +1,21 @@
+export default function GenericProviderLogo(
+ props: React.SVGProps<SVGSVGElement>,
+) {
+ return (
+ <svg
+ xmlns="http://www.w3.org/2000/svg"
+ viewBox="0 0 24 24"
+ fill="none"
+ stroke="currentColor"
+ strokeWidth="2"
+ strokeLinecap="round"
+ strokeLinejoin="round"
+ {...props}
+ >
+ <title>Model provider</title>
+ <path d="M21 16V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16z" />
+ <polyline points="3.27 6.96 12 12.01 20.73 6.96" />
+ <line x1="12" y1="22.08" x2="12" y2="12" />
+ </svg>
+ );
+}
diff --git a/frontend/components/provider-health-banner.tsx b/frontend/components/provider-health-banner.tsx
index 749252521..655bc4e7e 100644
--- a/frontend/components/provider-health-banner.tsx
+++ b/frontend/components/provider-health-banner.tsx
@@ -3,7 +3,7 @@
import { AlertTriangle } from "lucide-react";
import { useRouter } from "next/navigation";
import { useProviderHealthQuery } from "@/app/api/queries/useProviderHealthQuery";
-import type { ModelProvider } from "@/app/settings/_helpers/model-helpers";
+import { getProviderChrome } from "@/app/settings/_helpers/model-helpers";
import { Banner, BannerIcon, BannerTitle } from "@/components/ui/banner";
import { useChat } from "@/contexts/chat-context";
import { cn } from "@/lib/utils";
@@ -48,14 +48,6 @@ export function useProviderHealth() {
};
}
-const providerTitleMap: Record<ModelProvider, string> = {
- openai: "OpenAI",
- anthropic: "Anthropic",
- ollama: "Ollama",
- watsonx: "IBM watsonx.ai",
- local: "Local",
-};
-
export function ProviderHealthBanner({ className }: ProviderHealthBannerProps) {
const { isLoading, isHealthy, isUnhealthy, health } = useProviderHealth();
const router = useRouter();
@@ -100,8 +92,10 @@ export function ProviderHealthBanner({ className }: ProviderHealthBannerProps) {
errorProvider = llmProvider;
}
+ // One label source for every provider, including ones added through
+ // config/model_providers.yaml that have no built-in chrome.
const providerTitle = errorProvider
- ? providerTitleMap[errorProvider as ModelProvider] || errorProvider
+ ? getProviderChrome(errorProvider).name
: "Provider";
const settingsUrl = errorProvider
diff --git a/frontend/tests/pages/Settings.ts b/frontend/tests/pages/Settings.ts
index 24614b651..e4be8ec18 100644
--- a/frontend/tests/pages/Settings.ts
+++ b/frontend/tests/pages/Settings.ts
@@ -52,16 +52,34 @@ export class Settings {
.getByTestId("provider-connection-error")
.first();
+ /**
+ * Get locator for a provider's card.
+ *
+ * `hasText` matches substrings, so filtering on "OpenAI" also selected the
+ * "Azure OpenAI" tile once both providers shipped, and any button lookup
+ * inside it blew up under strict mode. Anchor on the card heading, which
+ * carries the provider name exactly.
+ * @param providerName - Name of the model provider
+ * @returns Locator for the provider card
+ */
+ private getProviderCard(providerName: string) {
+ return this.page.locator("div.rounded-xl, div.border-border.group").filter({
+ has: this.page.getByRole("heading", {
+ name: providerName,
+ exact: true,
+ }),
+ });
+ }
+
/**
* Get locator for configure button by provider name
* @param providerName - Name of the model provider
* @returns Locator for the configure button
*/
private getConfigureButton(providerName: string) {
- return this.page
- .locator("div.rounded-xl, div.border-border.group")
- .filter({ hasText: providerName })
- .getByRole("button", { name: "Configure" });
+ return this.getProviderCard(providerName).getByRole("button", {
+ name: "Configure",
+ });
}
/**
@@ -100,10 +118,9 @@ export class Settings {
* @returns Locator for the edit setup button
*/
private getEditSetupButton(providerName: string) {
- return this.page
- .locator("div.rounded-xl, div.border-border.group")
- .filter({ hasText: providerName })
- .getByRole("button", { name: "Edit Setup" });
+ return this.getProviderCard(providerName).getByRole("button", {
+ name: "Edit Setup",
+ });
}
/**
@@ -143,9 +160,7 @@ export class Settings {
* @returns Locator for the search input
*/
private getSearchModelInput() {
- let search = this.page.locator(
- 'input[placeholder="Search model..."]:focus',
- );
+ let search = this.page.locator('[data-testid="model-search-input"]:focus');
return search;
}
@@ -154,7 +169,7 @@ export class Settings {
* @returns Locator for the search input
*/
private getSearchModelInputFallback() {
- return this.page.locator('input[placeholder="Search model..."]').first();
+ return this.page.locator('[data-testid="model-search-input"]').first();
}
/**
diff --git a/src/api/models.py b/src/api/models.py
index 308c8e17c..d06c4122e 100644
--- a/src/api/models.py
+++ b/src/api/models.py
@@ -11,6 +11,7 @@ from api.provider_validation import (
)
from config.settings import get_openrag_config
from dependencies import get_models_service, require_permission
+from services.model_catalog import hide_excluded_live_models
from session_manager import User
from utils.logging_config import get_logger
@@ -97,7 +98,7 @@ async def get_openai_models(
)
models = await models_service.get_openai_models(api_key=api_key)
- return JSONResponse(models)
+ return JSONResponse(hide_excluded_live_models("openai", models))
except Exception as e:
logger.error(f"Failed to get OpenAI models: {str(e)}")
return _models_error_response(e)
@@ -127,7 +128,7 @@ async def get_anthropic_models(
)
models = await models_service.get_anthropic_models(api_key=api_key)
- return JSONResponse(models)
+ return JSONResponse(hide_excluded_live_models("anthropic", models))
except Exception as e:
logger.error(f"Failed to get Anthropic models: {str(e)}")
return _models_error_response(e)
@@ -154,7 +155,7 @@ async def get_ollama_models(
)
models = await models_service.get_ollama_models(endpoint=endpoint)
- return JSONResponse(models)
+ return JSONResponse(hide_excluded_live_models("ollama", models))
except Exception as e:
logger.error(f"Failed to get Ollama models: {str(e)}")
return _models_error_response(e)
@@ -211,12 +212,35 @@ async def get_ibm_models(
models = await models_service.get_ibm_models(
endpoint=endpoint, api_key=api_key, project_id=project_id
)
- return JSONResponse(models)
+ return JSONResponse(hide_excluded_live_models("watsonx", models))
except Exception as e:
logger.error(f"Failed to get IBM models: {str(e)}")
return _models_error_response(e)
+async def get_model_providers(
+ user: User = Depends(require_permission("providers:read")),
+):
+ """Providers this run mode exposes. GET /models/providers
+
+ The one list Settings cards, Onboarding tabs and the model pickers render.
+ Unlike /models/catalog it does not need LiteLLM, so the UI can still lay out
+ its provider surfaces when the catalogue is unavailable.
+ """
+ from config.model_providers import provider_visibility_payload
+
+ # `no-store`, not a max-age: the list is derived from
+ # `config/model_providers.yaml`, which an operator edits and then restarts
+ # the backend for. A cached response outlives that restart in the browser,
+ # so the console keeps drawing the old provider cards/tabs and the change
+ # looks like it did nothing. The payload is a few hundred bytes and React
+ # Query already holds it for the session, so the round trip costs nothing.
+ return JSONResponse(
+ provider_visibility_payload(),
+ headers={"Cache-Control": "no-store"},
+ )
+
+
async def get_model_catalog(
user: User = Depends(require_permission("providers:read")),
):
@@ -227,8 +251,12 @@ async def get_model_catalog(
catalog,
)
+ # `no-store` for the same reason as /models/providers: the catalogue is
+ # filtered by that config file, so a cached copy survives the restart that
+ # was supposed to apply the edit. LiteLLM's table is cached in-process, so
+ # rebuilding this response is cheap.
try:
- return JSONResponse(catalog(), headers={"Cache-Control": "private, max-age=3600"})
+ return JSONResponse(catalog(), headers={"Cache-Control": "no-store"})
except CatalogUnavailableError as e:
# Log the cause, but never echo exception text back to the caller
# (CodeQL py/stack-trace-exposure).
diff --git a/src/api/provider_health.py b/src/api/provider_health.py
index 8e3b514e5..e5ccfb901 100644
--- a/src/api/provider_health.py
+++ b/src/api/provider_health.py
@@ -9,6 +9,7 @@ from fastapi.responses import JSONResponse
from api.provider_validation import sanitize_provider_error_content, validate_provider_setup
from config.settings import get_openrag_config
from dependencies import require_permission
+from services import provider_error_log
from services.model_catalog import is_known_provider
from session_manager import User
from utils import provider_health_cache
@@ -145,8 +146,17 @@ async def check_provider_health(
embedding_project_id=embedding_project_id,
embedding_credentials=embedding_credentials,
)
+ # A cached *healthy* verdict must not outlive a real failure. The
+ # cache exists to coalesce identical probes, and a recorded failure
+ # means traffic is failing right now regardless of what the last
+ # probe concluded — so fall through and let the response below
+ # report it.
+ has_real_failure = bool(
+ provider_error_log.latest_failure(provider, "chat")
+ or provider_error_log.latest_failure(embedding_provider, "embedding")
+ )
cached_payload = provider_health_cache.get(health_cache_key)
- if cached_payload is not None:
+ if cached_payload is not None and not has_real_failure:
logger.debug("Returning cached provider-health response")
return JSONResponse(cached_payload, status_code=200)
@@ -160,7 +170,7 @@ async def check_provider_health(
break
# Woke up after an in-flight validation completed.
cached_payload = provider_health_cache.get(health_cache_key)
- if cached_payload is not None:
+ if cached_payload is not None and not has_real_failure:
logger.debug("Returning cached provider-health response (waited for in-flight)")
return JSONResponse(cached_payload, status_code=200)
# Leader's validation failed; retry leader election rather than
@@ -268,6 +278,19 @@ async def check_provider_health(
f"Embedding provider ({embedding_provider}) validation failed: {embedding_error}"
)
+ # A real call beats a probe. The probe sends its own request, so it
+ # hits its own failure: OpenAI checks request shape before billing,
+ # which is how a probe can report "no credits remaining" while the
+ # agent's own call reports a 400 about its parameters. Both are
+ # true; the actionable one is the one the user's traffic produced.
+ # An entry only exists while calls are still failing — the gateway
+ # erases it on the next success.
+ llm_error = provider_error_log.latest_failure(provider, "chat") or llm_error
+ embedding_error = (
+ provider_error_log.latest_failure(embedding_provider, "embedding")
+ or embedding_error
+ )
+
# Return combined status
if llm_error or embedding_error:
errors = []
diff --git a/src/api/settings/helpers.py b/src/api/settings/helpers.py
index 3b31b2ace..a6f04071d 100644
--- a/src/api/settings/helpers.py
+++ b/src/api/settings/helpers.py
@@ -43,26 +43,41 @@ def _configured_provider_names(config, provider_names) -> list:
def _first_configured_llm_provider(config, excluding: str) -> str:
- """Return the first configured LLM provider that isn't `excluding`."""
+ """Return the first configured LLM provider that isn't `excluding`.
+
+ Only providers this run mode shows are eligible: falling back to one the
+ config file hides would leave Settings pointing at a provider whose card
+ the user cannot see (see ``config/model_providers.yaml``).
+ """
+ from config.model_providers import visible_provider_keys
+
+ visible = visible_provider_keys()
for p in _LLM_PROVIDER_NAMES:
- if p != excluding and getattr(config.providers, p).configured:
+ if p != excluding and p in visible and getattr(config.providers, p).configured:
return p
for provider, value in config.providers.custom.items():
- if provider != excluding and value.configured:
+ if provider != excluding and provider in visible and value.configured:
return provider
return "openai"
def _first_configured_embedding_provider(config, excluding: str) -> str:
- """Return the first configured embedding provider that isn't `excluding`, or "" if none."""
+ """Return the first configured embedding provider that isn't `excluding`, or "" if none.
+
+ Providers hidden in this run mode are skipped, as in
+ ``_first_configured_llm_provider``.
+ """
+ from config.model_providers import visible_provider_keys
+
+ visible = visible_provider_keys()
for p in _EMBEDDING_PROVIDER_NAMES:
- if p != excluding and getattr(config.providers, p).configured:
+ if p != excluding and p in visible and getattr(config.providers, p).configured:
return p
from services.model_catalog import catalog
entries = {entry["key"]: entry for entry in catalog()["providers"]}
for provider, value in config.providers.custom.items():
- if provider != excluding and value.configured:
+ if provider != excluding and provider in visible and value.configured:
entry = entries.get(provider)
if entry and entry["embedding_models"]:
return provider
diff --git a/src/api/v1/llm.py b/src/api/v1/llm.py
index 63053eb2e..d7373fc9e 100644
--- a/src/api/v1/llm.py
+++ b/src/api/v1/llm.py
@@ -58,13 +58,39 @@ async def model_catalog_endpoint(
user: User = Depends(_catalog_read),
):
"""LiteLLM picker payload. GET /v1/model-catalog"""
+ # `no-store` for the same reason as /models/providers: the catalogue is
+ # filtered by that config file, so a cached copy survives the restart that
+ # was supposed to apply the edit. LiteLLM's table is cached in-process, so
+ # rebuilding this response is cheap.
try:
- return JSONResponse(catalog(), headers={"Cache-Control": "private, max-age=3600"})
+ return JSONResponse(catalog(), headers={"Cache-Control": "no-store"})
except CatalogUnavailableError as exc:
logger.error("Model catalogue unavailable", error=str(exc))
return JSONResponse({"error": CATALOG_UNAVAILABLE_MESSAGE}, status_code=503)
+async def model_providers_endpoint(
+ user: User = Depends(_catalog_read),
+):
+ """Providers this run mode exposes. GET /v1/model-providers
+
+ Same list the console reads from /models/providers, so an SDK client and the
+ UI never disagree about which providers a deployment offers.
+ """
+ from config.model_providers import provider_visibility_payload
+
+ # `no-store`, not a max-age: the list is derived from
+ # `config/model_providers.yaml`, which an operator edits and then restarts
+ # the backend for. A cached response outlives that restart in the browser,
+ # so the console keeps drawing the old provider cards/tabs and the change
+ # looks like it did nothing. The payload is a few hundred bytes and React
+ # Query already holds it for the session, so the round trip costs nothing.
+ return JSONResponse(
+ provider_visibility_payload(),
+ headers={"Cache-Control": "no-store"},
+ )
+
+
async def _read_json_body(request: Request) -> dict[str, Any] | JSONResponse:
try:
body = await request.json()
diff --git a/src/app/routes/internal.py b/src/app/routes/internal.py
index 70d454ec4..1fcec091c 100644
--- a/src/app/routes/internal.py
+++ b/src/app/routes/internal.py
@@ -439,6 +439,9 @@ def register_internal_routes(app: FastAPI):
app.add_api_route(
"/models/catalog", models.get_model_catalog, methods=["GET"], tags=["internal"]
)
+ app.add_api_route(
+ "/models/providers", models.get_model_providers, methods=["GET"], tags=["internal"]
+ )
# Onboarding endpoints
app.add_api_route("/onboarding", settings.onboarding, methods=["POST"], tags=["internal"])
diff --git a/src/app/routes/public_v1.py b/src/app/routes/public_v1.py
index 690c8e8ef..5b54390ad 100644
--- a/src/app/routes/public_v1.py
+++ b/src/app/routes/public_v1.py
@@ -112,6 +112,12 @@ def register_public_v1_routes(app: FastAPI):
methods=["GET"],
tags=["public"],
)
+ app.add_api_route(
+ "/v1/model-providers",
+ v1_llm.model_providers_endpoint,
+ methods=["GET"],
+ tags=["public"],
+ )
app.add_api_route(
"/v1/chat/completions",
v1_llm.chat_completions_endpoint,
diff --git a/src/config/model_providers.py b/src/config/model_providers.py
new file mode 100644
index 000000000..d96adaeb6
--- /dev/null
+++ b/src/config/model_providers.py
@@ -0,0 +1,277 @@
+"""Which model providers this deployment offers, per run mode.
+
+``model_providers.yaml`` (next to this module) lists every provider OpenRAG can
+show and, for each one, whether it is visible in ``oss``, ``on_prem`` and
+``saas``. The backend filters that list by ``OPENRAG_RUN_MODE`` and serves the
+result; Settings cards, the Onboarding tabs and the model catalogue all read
+that one list, so per-deployment policy is a config row rather than a denylist
+scattered across the frontend.
+
+Two rules keep the file safe to extend:
+
+- a mode key that is missing (or not a boolean-ish value) means *hidden*, so a
+ new run mode cannot silently expose a provider;
+- a provider absent from the file is not offered anywhere, though a model id
+ for it can still be typed by hand if the gateway already routes it.
+
+A row may also declare `models` / `embedding_models`. LiteLLM's bundled table
+covers the public vendors, but a self-hosted OpenAI-compatible gateway serves
+whatever its operator deployed, so those ids can only come from here. Declared
+ids are added to whatever the table already knows for that provider.
+
+``OPENRAG_MODEL_PROVIDERS_CONFIG`` points at an alternate YAML file for a single
+deployment. If that file is missing or unreadable the shipped default is used;
+if the shipped default is unreadable too, ``_FALLBACK_PROVIDERS`` below keeps
+the process serving the pre-config behavior instead of hiding every provider.
+"""
+
+from __future__ import annotations
+
+import os
+from functools import lru_cache
+from pathlib import Path
+from typing import Any, NamedTuple
+
+import yaml
+
+from utils.logging_config import get_logger
+from utils.run_mode_utils import RUN_MODE_ON_PREM, RUN_MODE_OSS, RUN_MODE_SAAS, get_run_mode
+
+logger = get_logger(__name__)
+
+CONFIG_PATH_ENV = "OPENRAG_MODEL_PROVIDERS_CONFIG"
+DEFAULT_CONFIG_PATH = Path(__file__).with_name("model_providers.yaml")
+
+KNOWN_RUN_MODES: tuple[str, ...] = (RUN_MODE_OSS, RUN_MODE_ON_PREM, RUN_MODE_SAAS)
+
+# Last-resort table, used only when no YAML file can be read at all. It matches
+# the visibility OpenRAG shipped before this config existed.
+_FALLBACK_PROVIDERS: tuple[dict[str, Any], ...] = (
+ {
+ "name": "openai",
+ "display_name": "OpenAI",
+ "modes": {RUN_MODE_OSS: True, RUN_MODE_ON_PREM: True, RUN_MODE_SAAS: True},
+ "models": (),
+ "embedding_models": (),
+ "exclude_models": (),
+ },
+ {
+ "name": "ollama",
+ "display_name": "Ollama",
+ "modes": {RUN_MODE_OSS: True, RUN_MODE_ON_PREM: True, RUN_MODE_SAAS: False},
+ "models": (),
+ "embedding_models": (),
+ "exclude_models": (),
+ },
+ {
+ "name": "watsonx",
+ "display_name": "IBM watsonx.ai",
+ "modes": {RUN_MODE_OSS: True, RUN_MODE_ON_PREM: True, RUN_MODE_SAAS: True},
+ "models": (),
+ "embedding_models": (),
+ "exclude_models": (),
+ },
+ {
+ "name": "anthropic",
+ "display_name": "Anthropic",
+ "modes": {RUN_MODE_OSS: True, RUN_MODE_ON_PREM: True, RUN_MODE_SAAS: True},
+ "models": (),
+ "embedding_models": (),
+ "exclude_models": (),
+ },
+)
+
+_TRUTHY = {"true", "1", "yes", "on"}
+
+
+def _as_bool(value: Any) -> bool:
+ """Whether a YAML `modes` value means "visible".
+
+ Anything that is not a boolean or a recognised truthy string is False, so a
+ typo hides a provider rather than exposing it.
+ """
+ if isinstance(value, bool):
+ return value
+ if isinstance(value, str):
+ return value.strip().lower() in _TRUTHY
+ return False
+
+
+def _model_ids(value: Any) -> tuple[str, ...]:
+ """A YAML `models:` list as clean, de-duplicated, order-preserving ids."""
+ if not isinstance(value, list):
+ return ()
+ ids = []
+ for item in value:
+ name = str(item).strip()
+ if name and name not in ids:
+ ids.append(name)
+ return tuple(ids)
+
+
+def _model_patterns(value: Any) -> tuple[str, ...]:
+ """A YAML `exclude_models:` list as lowercased fnmatch patterns."""
+ if not isinstance(value, list):
+ return ()
+ patterns = []
+ for item in value:
+ pattern = str(item).strip().lower()
+ if pattern and pattern not in patterns:
+ patterns.append(pattern)
+ return tuple(patterns)
+
+
+def _normalize(entry: Any, seen: set[str]) -> dict[str, Any] | None:
+ """One YAML provider row as a validated dict, or None if unusable."""
+ if not isinstance(entry, dict):
+ logger.warning("Ignoring a model provider entry that is not a mapping")
+ return None
+ name = str(entry.get("name") or "").strip().lower()
+ if not name:
+ logger.warning("Ignoring a model provider entry with no name")
+ return None
+ if name in seen:
+ logger.warning("Ignoring a duplicate model provider entry", provider=name)
+ return None
+ seen.add(name)
+ raw_modes = entry.get("modes")
+ if not isinstance(raw_modes, dict):
+ raw_modes = {}
+ display_name = str(entry.get("display_name") or "").strip() or name
+ return {
+ "name": name,
+ "display_name": display_name,
+ # Missing keys stay missing here; `is_visible` reads them as False.
+ "modes": {str(mode).strip().lower(): _as_bool(value) for mode, value in raw_modes.items()},
+ # Optional: ids the catalogue cannot learn from LiteLLM's static table.
+ "models": _model_ids(entry.get("models")),
+ "embedding_models": _model_ids(entry.get("embedding_models")),
+ # Optional: ids to keep out of the pickers, `*`/`?` wildcards allowed.
+ "exclude_models": _model_patterns(entry.get("exclude_models")),
+ }
+
+
+def _parse(raw: Any, source: str) -> tuple[dict[str, Any], ...]:
+ if isinstance(raw, dict):
+ entries = raw.get("providers")
+ else:
+ entries = raw
+ if not isinstance(entries, list):
+ logger.error("Model providers config has no `providers` list", source=source)
+ return ()
+ seen: set[str] = set()
+ parsed = [_normalize(entry, seen) for entry in entries]
+ return tuple(entry for entry in parsed if entry is not None)
+
+
+def _read(path: Path) -> tuple[dict[str, Any], ...]:
+ try:
+ raw = yaml.safe_load(path.read_text(encoding="utf-8"))
+ except FileNotFoundError:
+ logger.warning("Model providers config not found", path=str(path))
+ return ()
+ except (OSError, yaml.YAMLError) as exc:
+ logger.error(
+ "Model providers config could not be read",
+ path=str(path),
+ error=str(exc),
+ )
+ return ()
+ return _parse(raw, str(path))
+
+
+@lru_cache(maxsize=1)
+def _configured(override: str, default_path: str) -> tuple[dict[str, Any], ...]:
+ """Every provider in the config file, in file order, mode filtering aside.
+
+ Cached on the resolved paths so the YAML is read once per process. Call
+ `reload()` after changing the file or the override env var.
+ """
+ if override:
+ providers = _read(Path(override))
+ if providers:
+ return providers
+ logger.error(
+ "Falling back to the shipped model providers config",
+ override=override,
+ env=CONFIG_PATH_ENV,
+ )
+ providers = _read(Path(default_path))
+ if providers:
+ return providers
+ logger.error(
+ "No usable model providers config; using built-in defaults",
+ path=default_path,
+ )
+ return _FALLBACK_PROVIDERS
+
+
+def configured_providers() -> tuple[dict[str, Any], ...]:
+ """Every provider the config file lists, regardless of run mode."""
+ return _configured((os.getenv(CONFIG_PATH_ENV) or "").strip(), str(DEFAULT_CONFIG_PATH))
+
+
+def reload() -> None:
+ """Drop the cached file so the next read re-parses it."""
+ _configured.cache_clear()
+
+
+def is_visible(provider: dict[str, Any], run_mode: str) -> bool:
+ """Whether a parsed provider entry is shown in `run_mode`."""
+ return bool(provider["modes"].get(run_mode, False))
+
+
+def visible_providers(run_mode: str | None = None) -> tuple[dict[str, Any], ...]:
+ """The providers shown in `run_mode` (the current one by default)."""
+ mode = (run_mode or get_run_mode()).strip().lower()
+ return tuple(entry for entry in configured_providers() if is_visible(entry, mode))
+
+
+class ProviderEntry(NamedTuple):
+ """One visible provider, in the form the catalogue consumes.
+
+ A NamedTuple rather than a dict: hashable all the way down, so callers can
+ key an lru_cache on a tuple of these and have a config change rebuild it.
+ """
+
+ name: str
+ display_name: str
+ models: tuple[str, ...]
+ embedding_models: tuple[str, ...]
+ exclude_models: tuple[str, ...]
+
+
+def visible_provider_entries(
+ run_mode: str | None = None,
+) -> tuple[ProviderEntry, ...]:
+ """The visible providers, as catalogue-ready entries."""
+ return tuple(
+ ProviderEntry(
+ entry["name"],
+ entry["display_name"],
+ entry["models"],
+ entry["embedding_models"],
+ entry["exclude_models"],
+ )
+ for entry in visible_providers(run_mode)
+ )
+
+
+def visible_provider_keys(run_mode: str | None = None) -> frozenset[str]:
+ return frozenset(entry["name"] for entry in visible_providers(run_mode))
+
+
+def is_provider_visible(provider: str, run_mode: str | None = None) -> bool:
+ return (provider or "").strip().lower() in visible_provider_keys(run_mode)
+
+
+def provider_visibility_payload(run_mode: str | None = None) -> dict[str, Any]:
+ """The API body: the current run mode and the providers it exposes."""
+ mode = (run_mode or get_run_mode()).strip().lower()
+ return {
+ "run_mode": mode,
+ "providers": [
+ {"name": entry["name"], "display_name": entry["display_name"]}
+ for entry in visible_providers(mode)
+ ],
+ }
diff --git a/src/config/model_providers.yaml b/src/config/model_providers.yaml
new file mode 100644
index 000000000..549c55de1
--- /dev/null
+++ b/src/config/model_providers.yaml
@@ -0,0 +1,114 @@
+# Which model providers OpenRAG offers, per run mode.
+#
+# This file is the single source of truth for provider visibility in the
+# Settings provider cards, the Onboarding tabs, and the model catalogue that
+# feeds every agent/ingest picker. The backend filters on it; the UI renders
+# whatever the API returns, so there is no second denylist in the frontend.
+#
+# Keys under `modes` match OPENRAG_RUN_MODE: oss | on_prem | saas.
+# true -> the provider is shown in that mode
+# false -> the provider is hidden
+# A missing mode key means hidden, so adding a new run mode can never expose a
+# provider by accident.
+#
+# A provider that is not listed here is not offered anywhere in the UI. A model
+# id for an unlisted provider can still be typed by hand and will be routed if
+# the gateway already supports it.
+#
+# Set OPENRAG_MODEL_PROVIDERS_CONFIG to another YAML file to override this list
+# for a single deployment; this file is the default.
+#
+# A row may also declare `models` / `embedding_models`. LiteLLM's bundled table
+# already knows the public vendors, so those rows need nothing. A self-hosted
+# OpenAI-compatible gateway serves whatever its operator deployed, and those ids
+# can only come from here — see the commented example at the end of this file.
+#
+# `exclude_models` keeps ids out of the pickers. `*` and `?` wildcards work, and
+# matching is against the id as the picker shows it (`gpt-3.5-turbo`, not
+# `openai/gpt-3.5-turbo`). Exclusion is applied last, so it also overrides a
+# `models` row on the same provider. Models the provider has already retired are
+# dropped without being listed here, and ones retiring within 90 days are hidden
+# behind a "Show N retiring" row in the picker — use this for the ones a
+# deployment does not want offered at all.
+
+providers:
+ - name: openai
+ display_name: OpenAI
+ modes:
+ oss: true
+ on_prem: true
+ saas: true
+ exclude_models:
+ - gpt-3.5-* # a whole generation, superseded and near retirement
+ - gpt-5.6-luna # refuses function tools on /v1/chat/completions
+
+ - name: ollama
+ display_name: Ollama
+ modes:
+ oss: true
+ on_prem: true
+ saas: false # local inference is not offered in SaaS
+
+ - name: watsonx
+ display_name: IBM watsonx.ai
+ modes:
+ oss: true
+ on_prem: true
+ saas: true
+
+ - name: anthropic
+ display_name: Anthropic
+ modes:
+ oss: true
+ on_prem: true
+ saas: true
+ # enabled in OSS for tests only to be disbaled in OSS after tests are complete
+ - name: azure_ai
+ display_name: Azure AI Foundry
+ modes:
+ oss: false
+ on_prem: false
+ saas: false
+ # enabled in OSS for tests only to be disbaled in OSS after tests are complete
+
+ # Azure OpenAI Service, which LiteLLM keys separately from Foundry above:
+ # `azure_ai` is the Foundry catalogue (Llama, Phi, Mistral, Cohere, and the
+ # gpt-5.x entries Foundry serves), `azure` is the OpenAI Service deployment
+ # where the gpt-4.1 family lives. Both can be offered at once.
+ - name: azure
+ display_name: Azure OpenAI
+ modes:
+ oss: true
+ on_prem: true
+ saas: true
+ # Azure serves the same OpenAI models, so it needs the same exclusions.
+ # The plain name covers the regional ids (`eu/`, `us/`, `global/`) too.
+ exclude_models:
+ - gpt-3.5-*
+ # Azure lets you name a deployment anything, so LiteLLM's table cannot know
+ # yours. Declare them here and they appear in the pickers for everyone;
+ # otherwise each user has to type the name into the model box by hand.
+ # models:
+ # - prod-gpt41-eastus
+ # embedding_models:
+ # - prod-embed-3-large
+
+ # A self-hosted OpenAI-compatible endpoint (vLLM, LM Studio, an internal
+ # gateway). Use one of LiteLLM's passthrough provider keys — `openai_like`,
+ # `hosted_vllm`, `lm_studio`, `llamafile`, `custom_openai` — rather than
+ # inventing a name: the key has to be one LiteLLM can route, or the
+ # `provider:model` ids it publishes resolve to the default provider and get
+ # called with the wrong credentials. The endpoint and key are entered in
+ # Settings; only the model ids belong here.
+ #
+ # - name: openai_like
+ # display_name: Internal LLM Gateway
+ # modes:
+ # oss: true
+ # on_prem: true
+ # saas: false
+ # models:
+ # - llama-3.3-70b-instruct
+ # - qwen2.5-coder-32b
+ # embedding_models:
+ # - bge-m3
diff --git a/src/mcp_http/server.py b/src/mcp_http/server.py
index 09b5d53a4..35503195c 100644
--- a/src/mcp_http/server.py
+++ b/src/mcp_http/server.py
@@ -171,6 +171,13 @@ COMPONENT_CUSTOMIZATIONS: dict[tuple[str, str], dict[str, str]] = {
"grouped models, capability flags, and credential field specs."
),
},
+ ("/v1/model-providers", "GET"): {
+ "name": "openrag_model_providers",
+ "description": (
+ "Model providers this deployment exposes for its run mode "
+ "(oss / on_prem / saas), with their display names."
+ ),
+ },
("/v1/chat/completions", "POST"): {
"name": "openrag_chat_completions",
"description": (
diff --git a/src/services/llm_gateway.py b/src/services/llm_gateway.py
index 5d5a8f236..dffa73917 100644
--- a/src/services/llm_gateway.py
+++ b/src/services/llm_gateway.py
@@ -12,6 +12,7 @@ import re
from collections.abc import AsyncIterator, Mapping
from typing import Any, Literal
+from services import provider_error_log
from services.model_catalog import is_known_provider
from utils.logging_config import get_logger
@@ -504,6 +505,53 @@ def _log_completion_shape(payload: dict[str, Any], provider: str, model: str) ->
logger.debug("Could not summarise completion shape", exc_info=True)
+#: Models seen to refuse function tools alongside their own default reasoning
+#: effort. OpenAI applies a non-none default to reasoning models and rejects
+#: that combination on /v1/chat/completions; the remedy its error names is an
+#: explicit "none". Learned from the first failure rather than assumed, so a
+#: model that accepts the pair keeps its reasoning.
+_TOOLS_NEED_REASONING_OFF: set[str] = set()
+
+
+def _model_info(model: str) -> dict[str, Any]:
+ """LiteLLM's row for `model`, by its full id or its bare name."""
+ try:
+ import litellm
+
+ table = litellm.model_cost
+ info = table.get(model) or table.get(model.rsplit("/", 1)[-1])
+ return info if isinstance(info, dict) else {}
+ except Exception:
+ return {}
+
+
+def _is_reasoning_tool_conflict(detail: str) -> bool:
+ """Whether a failure is the tools-plus-reasoning_effort rejection.
+
+ Matched on the provider's own words. There is no capability flag for it —
+ `supports_reasoning` and `supports_function_calling` are both true for the
+ models that refuse the combination — so the failure itself is the only
+ signal available.
+ """
+ lowered = (detail or "").lower()
+ return "reasoning_effort" in lowered and "tool" in lowered
+
+
+def _reasoning_off_retry(model: str, kwargs: dict[str, Any]) -> bool:
+ """Set `reasoning_effort="none"` for a retry, if that can help here.
+
+ False when the caller chose an effort itself — overriding a deliberate
+ choice would be worse than the error — or when the model cannot take
+ "none", in which case only the Responses API can serve tools for it.
+ """
+ if not kwargs.get("tools") or "reasoning_effort" in kwargs:
+ return False
+ if not _model_info(model).get("supports_none_reasoning_effort"):
+ return False
+ kwargs["reasoning_effort"] = "none"
+ return True
+
+
async def chat_completions(
body: Mapping[str, Any], *, config=None
) -> dict[str, Any] | AsyncIterator[str]:
@@ -512,10 +560,14 @@ async def chat_completions(
litellm_model, provider, credentials = resolve_call(body.get("model"), kind="chat", config=cfg)
kwargs = {key: body[key] for key in _LITELLM_FORWARDED_PARAMS if key in body}
stream = bool(body.get("stream"))
- try:
+ if litellm_model in _TOOLS_NEED_REASONING_OFF:
+ # Already learned about this model; do not spend a round-trip relearning.
+ _reasoning_off_retry(litellm_model, kwargs)
+
+ async def _call() -> Any:
import litellm
- result = await litellm.acompletion(
+ return await litellm.acompletion(
model=litellm_model,
messages=list(body.get("messages") or []),
stream=stream,
@@ -529,6 +581,25 @@ async def chat_completions(
**credentials,
**kwargs,
)
+
+ try:
+ try:
+ result = await _call()
+ except Exception as exc:
+ # `drop_params` cannot help here: reasoning_effort is a supported
+ # parameter, just not one this model accepts beside tools. Retry
+ # once with the value the provider's own error asks for.
+ if not _is_reasoning_tool_conflict(f"{exc}") or not _reasoning_off_retry(
+ litellm_model, kwargs
+ ):
+ raise
+ logger.info(
+ "Retrying completion with reasoning_effort=none",
+ provider=provider,
+ model=litellm_model,
+ )
+ _TOOLS_NEED_REASONING_OFF.add(litellm_model)
+ result = await _call()
except LlmGatewayError:
raise
except Exception as exc:
@@ -536,14 +607,22 @@ async def chat_completions(
logger.error(
"LLM chat completions failed", provider=provider, model=litellm_model, error=detail
)
+ message = _upstream_client_message(detail, provider, litellm_model, exc)
+ # The health banner otherwise reports whatever its own probe hit, which
+ # is a different request and so often a different error. Hand it the
+ # text this caller is being shown.
+ provider_error_log.record_failure(provider, "chat", message)
raise LlmGatewayError(
- _upstream_client_message(detail, provider, litellm_model, exc),
+ message,
_upstream_status_code(exc),
detail=detail,
) from exc
if stream:
+ # A stream can still fail mid-flight, so _stream_sse clears the record
+ # itself once the provider has actually delivered something.
return _stream_sse(result, provider, litellm_model, credentials)
+ provider_error_log.record_success(provider, "chat")
payload = _to_openai_dict(result)
_repair_completion_payload(payload, provider, litellm_model)
_log_completion_shape(payload, provider, litellm_model)
@@ -797,6 +876,9 @@ async def _stream_sse(
# the next person has something to search for.
if tally.content_chars == 0 and tally.tool_calls == 0:
tally.error = "empty completion"
+ provider_error_log.record_failure(
+ provider, "chat", _empty_stream_message(provider, model, tally.finish_reason)
+ )
logger.warning(
"LLM chat stream produced no content and no tool calls",
provider=provider,
@@ -807,6 +889,10 @@ async def _stream_sse(
yield _error_frame(
_empty_stream_message(provider, model, tally.finish_reason), provider, model
)
+ else:
+ # Content actually reached the client, so whatever the banner was
+ # holding against this provider is no longer true.
+ provider_error_log.record_success(provider, "chat")
except Exception as exc:
# Mid-stream failures used to surface as a truncated stream, which the
# UI could only report as "the server didn't return a response". Emit an
@@ -815,7 +901,9 @@ async def _stream_sse(
detail = _redact(f"{type(exc).__name__}: {exc}", credentials or {})
tally.error = detail
logger.error("LLM chat stream failed", provider=provider, model=model, error=detail)
- yield _error_frame(_upstream_client_message(detail, provider, model, exc), provider, model)
+ message = _upstream_client_message(detail, provider, model, exc)
+ provider_error_log.record_failure(provider, "chat", message)
+ yield _error_frame(message, provider, model)
finally:
close = getattr(stream, "aclose", None) or getattr(stream, "close", None)
if close is not None:
@@ -866,9 +954,12 @@ async def embeddings(body: Mapping[str, Any], *, config=None) -> dict[str, Any]:
except Exception as exc:
detail = _redact(f"{type(exc).__name__}: {exc}", credentials)
logger.error("LLM embeddings failed", provider=provider, model=litellm_model, error=detail)
+ message = _upstream_client_message(detail, provider, litellm_model, exc)
+ provider_error_log.record_failure(provider, "embedding", message)
raise LlmGatewayError(
- _upstream_client_message(detail, provider, litellm_model, exc),
+ message,
_upstream_status_code(exc),
detail=detail,
) from exc
+ provider_error_log.record_success(provider, "embedding")
return _to_openai_dict(result)
diff --git a/src/services/model_catalog.py b/src/services/model_catalog.py
index c1da80405..47a4d074f 100644
--- a/src/services/model_catalog.py
+++ b/src/services/model_catalog.py
@@ -16,25 +16,31 @@ JSON is ~100KB, so paying for it once per process beats a re-read per request.
`function_calling` and `vision` are published as capability flags so the UI can
filter (agent vs VLM) without a second live provider call.
-The public catalogue is intentionally limited to the providers OpenRAG exposes
-in its settings UI. Generic credential helpers remain available for future
-providers, and a model that is not in the catalogue can still be typed by hand.
+The public catalogue is limited to the providers this run mode exposes, per
+``config/model_providers.yaml`` (see ``config.model_providers``). Generic
+credential helpers remain available for every other provider, and a model that
+is not in the catalogue can still be typed by hand.
"""
from __future__ import annotations
import datetime
import json
+from fnmatch import fnmatch
from functools import lru_cache
from typing import Any
+from config.model_providers import ProviderEntry, visible_provider_entries
from utils.logging_config import get_logger
logger = get_logger(__name__)
TEXT_GENERATION_MODES = frozenset({"chat", "completion", "responses"})
EMBEDDING_MODE = "embedding"
-SUPPORTED_PROVIDER_KEYS = frozenset({"openai", "anthropic", "watsonx", "ollama"})
+
+#: LiteLLM prices fine-tunes off a template row named for the base model. The
+#: row is not a callable id, so it never belongs in a picker.
+FINE_TUNE_TEMPLATE_PREFIX = "ft:"
CAPABILITY_FLAGS: dict[str, str] = {
"supports_function_calling": "function_calling",
@@ -192,14 +198,50 @@ def _model_entry(name: str, info: dict[str, Any]) -> dict[str, Any]:
return entry
-@lru_cache(maxsize=1)
-def _catalog() -> dict[str, Any]:
+def _excluded(name: str, patterns: tuple[str, ...]) -> bool:
+ """Whether a model id is one the config file keeps out of the pickers.
+
+ Matched on the id as the picker shows it — `gpt-3.5-turbo`, not
+ `openai/gpt-3.5-turbo` — because that is the name an operator reads off the
+ screen when deciding what to suppress. `*` and `?` work, so a whole
+ generation goes in one line.
+
+ Also matched on the id's last path segment, so a regional deployment is
+ covered by the plain name: Azure lists `gpt-5.6-luna` as `eu/gpt-5.6-luna`
+ and `us/gpt-5.6-luna` too, and someone suppressing that model means all
+ three. Write `eu/*` to reach a region on its own.
+ """
+ if not patterns:
+ return False
+ lowered = name.lower()
+ bare = lowered.rsplit("/", 1)[-1]
+ return any(fnmatch(lowered, pattern) or fnmatch(bare, pattern) for pattern in patterns)
+
+
+def _declared_entries(names: tuple[str, ...], mode: str) -> list[dict[str, Any]]:
+ """Config-declared model ids as catalogue entries.
+
+ A self-hosted OpenAI-compatible gateway serves whatever its operator
+ deployed, so LiteLLM's bundled table knows none of its ids. Only the name
+ and the mode are known here — no pricing, no capability flags.
+ """
+ return [{"model": name, "mode": mode} for name in names]
+
+
+@lru_cache(maxsize=8)
+def _catalog(providers: tuple[ProviderEntry, ...]) -> dict[str, Any]:
+ """Payload for `providers`, given as `config.model_providers` entries.
+
+ The entries are the cache key, so flipping a provider's visibility (or the
+ run mode, in a test) rebuilds instead of serving a stale list.
+ """
try:
import litellm
except Exception as exc:
raise CatalogUnavailableError("litellm is not installed on the server") from exc
specs = _provider_field_specs()
+ keys = {entry.name for entry in providers}
chat_by_provider: dict[str, list[dict[str, Any]]] = {}
embed_by_provider: dict[str, list[dict[str, Any]]] = {}
@@ -208,7 +250,7 @@ def _catalog() -> dict[str, Any]:
continue
provider = info.get("litellm_provider")
mode = info.get("mode")
- if provider not in SUPPORTED_PROVIDER_KEYS:
+ if provider not in keys:
continue
if mode in TEXT_GENERATION_MODES:
bucket = chat_by_provider
@@ -219,36 +261,66 @@ def _catalog() -> dict[str, Any]:
name = model_id[len(provider) + 1 :] if model_id.startswith(f"{provider}/") else model_id
if not name or name == "sample_spec":
continue
+ if name.startswith(FINE_TUNE_TEMPLATE_PREFIX):
+ # `ft:gpt-4o-2024-08-06` is a pricing row for the *base* of a
+ # fine-tune, not an id anyone can call: a real one carries the org
+ # and job suffix (`ft:gpt-4o-2024-08-06:acme::abc123`). Listing the
+ # template invites picking a model that 404s. Owners of a fine-tune
+ # type their full id into the picker instead.
+ continue
bucket.setdefault(provider, []).append(_model_entry(name, info))
- providers = []
- for key in sorted(SUPPORTED_PROVIDER_KEYS):
- providers.append(
+ entries = []
+ for key, display_name, declared_chat, declared_embed, excluded in providers:
+ if not is_known_provider(key):
+ # The catalogue would still render, but `split_model_id` cannot
+ # recognise the prefix, so every id would be billed to the default
+ # provider. Name the row rather than fail the whole catalogue.
+ logger.warning(
+ "Model provider is not routable by LiteLLM; its models will not resolve",
+ provider=key,
+ )
+ chat = chat_by_provider.get(key, [])
+ embed = embed_by_provider.get(key, [])
+ known_chat = {entry["model"] for entry in chat}
+ known_embed = {entry["model"] for entry in embed}
+ chat = chat + _declared_entries(
+ tuple(name for name in declared_chat if name not in known_chat), "chat"
+ )
+ embed = embed + _declared_entries(
+ tuple(name for name in declared_embed if name not in known_embed), EMBEDDING_MODE
+ )
+ # Applied last, so `exclude_models` also wins over a `models:` row —
+ # a deployment that suppresses an id means it, wherever the id came
+ # from, and the alternative is two lines that quietly contradict.
+ chat = [entry for entry in chat if not _excluded(entry["model"], excluded)]
+ embed = [entry for entry in embed if not _excluded(entry["model"], excluded)]
+ entries.append(
{
"key": key,
- "name": (specs.get(key) or {}).get("provider_display_name") or key,
+ # The config file names the provider; LiteLLM's own label is
+ # the fallback for a row that left `display_name` out.
+ "name": display_name or (specs.get(key) or {}).get("provider_display_name") or key,
"credential_fields": credential_fields(key),
"model_placeholder": (specs.get(key) or {}).get("default_model_placeholder"),
- "models": sorted(chat_by_provider.get(key, []), key=lambda entry: entry["model"]),
- "embedding_models": sorted(
- embed_by_provider.get(key, []), key=lambda entry: entry["model"]
- ),
+ "models": sorted(chat, key=lambda entry: entry["model"]),
+ "embedding_models": sorted(embed, key=lambda entry: entry["model"]),
}
)
- return {"providers": providers}
+ return {"providers": entries}
-@lru_cache(maxsize=2)
-def _catalog_for(today: datetime.date) -> dict[str, Any]:
+@lru_cache(maxsize=8)
+def _catalog_for(today: datetime.date, providers: tuple[ProviderEntry, ...]) -> dict[str, Any]:
"""`_catalog()` with models the provider has already retired removed."""
def _keep(model: dict[str, Any]) -> bool:
retires = _parse_deprecation(model.get("deprecation_date"))
return retires is None or retires > today
- providers = []
- for provider in _catalog()["providers"]:
- providers.append(
+ entries = []
+ for provider in _catalog(providers)["providers"]:
+ entries.append(
{
**provider,
"models": [model for model in provider["models"] if _keep(model)],
@@ -257,12 +329,56 @@ def _catalog_for(today: datetime.date) -> dict[str, Any]:
],
}
)
- return {"providers": providers}
+ return {"providers": entries}
+
+
+def exclusions_for(provider: str) -> tuple[str, ...]:
+ """The `exclude_models` patterns configured for `provider`."""
+ key = (provider or "").strip().lower()
+ for entry in visible_provider_entries():
+ if entry.name == key:
+ return entry.exclude_models
+ return ()
+
+
+#: Keys in a live `/models/{provider}` payload that hold model lists.
+LIVE_MODEL_LIST_KEYS = ("language_models", "embedding_models")
+
+
+def hide_excluded_live_models(provider: str, payload: Any) -> Any:
+ """Apply `exclude_models` to a live `/models/{provider}` response.
+
+ Ollama, watsonx, OpenAI and Anthropic are also listed by asking the running
+ provider, which never passes through the catalogue — so without this an
+ excluded id disappears from the settings picker and comes straight back in
+ through onboarding. The filter belongs here rather than in the frontend so
+ every consumer, SDK clients included, sees one list.
+ """
+ patterns = exclusions_for(provider)
+ if not patterns or not isinstance(payload, dict):
+ return payload
+ filtered = dict(payload)
+ for key in LIVE_MODEL_LIST_KEYS:
+ models = payload.get(key)
+ if not isinstance(models, list):
+ continue
+ filtered[key] = [
+ model
+ for model in models
+ if not _excluded(str((model or {}).get("value", "")), patterns)
+ if isinstance(model, dict)
+ ]
+ return filtered
+
+
+def supported_provider_keys() -> frozenset[str]:
+ """The providers this run mode publishes, per `config/model_providers.yaml`."""
+ return frozenset(entry.name for entry in visible_provider_entries())
def catalog(today: datetime.date | None = None) -> dict[str, Any]:
- """Picker payload for OpenRAG's supported providers and their models."""
- return _catalog_for(today or datetime.date.today())
+ """Picker payload for the providers this run mode exposes, and their models."""
+ return _catalog_for(today or datetime.date.today(), visible_provider_entries())
def is_known_provider(provider: str) -> bool:
@@ -273,7 +389,14 @@ def is_known_provider(provider: str) -> bool:
try:
import litellm
- if key in {str(value) for value in litellm.provider_list}:
+ # `provider_list` holds LlmProviders enum members, and str() on one
+ # yields "LlmProviders.OPENAI" — never the routable "openai". Reading
+ # .value is what makes this branch match; without it the check fell
+ # through to the credential-form specs below, which do not cover every
+ # provider LiteLLM can route (zai, scaleway, chatgpt, ...). Those
+ # prefixes were then left unsplit and called with the default
+ # provider's credentials.
+ if key in {getattr(value, "value", str(value)) for value in litellm.provider_list}:
return True
except Exception:
logger.debug("Could not read litellm.provider_list", exc_info=True)
@@ -283,15 +406,19 @@ def is_known_provider(provider: str) -> bool:
return False
-@lru_cache(maxsize=1)
-def _model_owners() -> dict[str, tuple[str, ...]]:
- """Map every catalogue model id to the providers that serve it."""
+@lru_cache(maxsize=8)
+def _model_owners(providers: tuple[ProviderEntry, ...]) -> dict[str, tuple[str, ...]]:
+ """Map every catalogue model id to the providers that serve it.
+
+ Keyed by the same entries as `_catalog()`, so a run mode that hides a
+ provider also drops that provider's models from the ownership map.
+ """
owners: dict[str, set[str]] = {}
try:
- providers = _catalog()["providers"]
+ entries = _catalog(providers)["providers"]
except Exception:
return {}
- for provider in providers:
+ for provider in entries:
for entry in (*provider["models"], *provider["embedding_models"]):
owners.setdefault(entry["model"], set()).add(provider["key"])
return {model: tuple(sorted(keys)) for model, keys in owners.items()}
@@ -307,7 +434,7 @@ def catalog_owner(model: str) -> str | None:
if not name:
return None
try:
- owners = _model_owners().get(name)
+ owners = _model_owners(visible_provider_entries()).get(name)
except Exception:
return None
return owners[0] if owners and len(owners) == 1 else None
@@ -320,7 +447,7 @@ def PROVIDER_SEPARATOR_SAFE_CHECK() -> list[str]:
Exposed so a test can fail loudly if a future model id breaks the property.
"""
ambiguous = []
- for model in _model_owners():
+ for model in _model_owners(visible_provider_entries()):
head, sep, rest = model.partition(":")
if sep and rest and is_known_provider(head.lower()):
ambiguous.append(model)
diff --git a/src/services/provider_error_log.py b/src/services/provider_error_log.py
new file mode 100644
index 000000000..424ffe481
--- /dev/null
+++ b/src/services/provider_error_log.py
@@ -0,0 +1,73 @@
+"""The most recent real provider failure, per provider and call kind.
+
+The health banner reports what a synthetic probe hit, and a probe cannot
+reproduce the request that actually failed. OpenAI validates request shape
+before it checks billing, so a plain probe completion comes back "You have no
+credits remaining" while the agent's own call — function tools plus
+``reasoning_effort`` — comes back a 400 about the parameters. Both are true.
+The one the operator needs is the one their own traffic produced.
+
+So the gateway records what it saw. An entry is written when a call fails and
+erased when the next call on the same provider and kind succeeds, which means
+its presence says traffic is failing *now* rather than that it once did.
+``STALE_AFTER_SECONDS`` covers the remaining case: a provider that fails and
+is then never called again would otherwise pin the banner forever.
+
+In-process and per-worker, deliberately. This is a diagnostic for the operator
+watching the console, not a durable record — an entry that is lost on restart
+is one whose failure has not recurred.
+"""
+
+from __future__ import annotations
+
+import time
+from typing import Literal
+
+CallKind = Literal["chat", "embedding"]
+
+#: How long a recorded failure keeps speaking for a provider that has gone
+#: quiet. Long enough to survive the banner's polling gap, short enough that a
+#: failure nobody is reproducing stops being news.
+STALE_AFTER_SECONDS = 15 * 60
+
+# (provider, kind) -> (monotonic timestamp, client-facing message)
+_failures: dict[tuple[str, str], tuple[float, str]] = {}
+
+
+def _key(provider: str, kind: CallKind) -> tuple[str, str]:
+ return ((provider or "").strip().lower(), kind)
+
+
+def record_failure(provider: str, kind: CallKind, message: str) -> None:
+ """Remember why a real call to `provider` failed.
+
+ `message` is the text already shown to the caller, so the banner and the
+ chat bubble cannot disagree about what went wrong.
+ """
+ if not provider or not message:
+ return
+ _failures[_key(provider, kind)] = (time.monotonic(), message)
+
+
+def record_success(provider: str, kind: CallKind) -> None:
+ """Forget any recorded failure: this provider is serving traffic again."""
+ _failures.pop(_key(provider, kind), None)
+
+
+def latest_failure(provider: str | None, kind: CallKind) -> str | None:
+ """The last real failure for `provider`, if one is still current."""
+ if not provider:
+ return None
+ entry = _failures.get(_key(provider, kind))
+ if entry is None:
+ return None
+ recorded_at, message = entry
+ if time.monotonic() - recorded_at > STALE_AFTER_SECONDS:
+ _failures.pop(_key(provider, kind), None)
+ return None
+ return message
+
+
+def clear() -> None:
+ """Drop every recorded failure. For tests."""
+ _failures.clear()
diff --git a/tests/unit/api/test_model_providers_endpoint.py b/tests/unit/api/test_model_providers_endpoint.py
new file mode 100644
index 000000000..92cc3a504
--- /dev/null
+++ b/tests/unit/api/test_model_providers_endpoint.py
@@ -0,0 +1,120 @@
+"""The endpoints that publish which model providers a deployment offers.
+
+The console and any SDK client must be told the same thing, and neither may be
+able to read a provider the run mode hides — that is the whole point of moving
+the denylist out of the frontend (#2287).
+"""
+
+import inspect
+import json
+
+import pytest
+from fastapi import params as fastapi_params
+
+from api import models as models_api
+from api.v1 import llm as v1_llm
+from config import model_providers
+from session_manager import User
+
+
+def _user() -> User:
+ return User(user_id="u1", email="u@x", name="U", provider="api_key")
+
+
+def _dependency(fn):
+ default = inspect.signature(fn).parameters["user"].default
+ assert isinstance(default, fastapi_params.Depends)
+ return default.dependency
+
+
[email protected](autouse=True)
+def _fresh_provider_config():
+ model_providers.reload()
+ yield
+ model_providers.reload()
+
+
+def _body(response) -> dict:
+ assert response.status_code == 200
+ return json.loads(response.body)
+
+
+def test_console_endpoint_requires_providers_read():
+ dep = _dependency(models_api.get_model_providers)
+ perms = [cell.cell_contents for cell in dep.__closure__]
+ assert "providers:read" in perms
+
+
+def test_v1_endpoint_requires_a_catalogue_reading_permission():
+ dep = _dependency(v1_llm.model_providers_endpoint)
+ required = [cell.cell_contents for cell in dep.__closure__]
+ assert any("providers:read" in str(item) for item in required)
+
+
+async def test_saas_does_not_publish_ollama(monkeypatch):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")
+ body = _body(await models_api.get_model_providers(user=_user()))
+
+ assert body["run_mode"] == "saas"
+ assert "ollama" not in {entry["name"] for entry in body["providers"]}
+
+
[email protected]("mode", ["oss", "on_prem"])
+async def test_oss_and_on_prem_still_publish_ollama(monkeypatch, mode):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)
+ body = _body(await models_api.get_model_providers(user=_user()))
+
+ assert body["run_mode"] == mode
+ assert "ollama" in {entry["name"] for entry in body["providers"]}
+
+
+async def test_the_console_and_v1_endpoints_agree(monkeypatch):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")
+ console = _body(await models_api.get_model_providers(user=_user()))
+ public = _body(await v1_llm.model_providers_endpoint(user=_user()))
+
+ assert console == public
+
+
+async def test_an_override_file_drives_the_payload(monkeypatch, tmp_path):
+ path = tmp_path / "model_providers.yaml"
+ path.write_text(
+ "providers:\n - name: ollama\n display_name: Ollama\n modes:\n saas: true\n",
+ encoding="utf-8",
+ )
+ monkeypatch.setenv(model_providers.CONFIG_PATH_ENV, str(path))
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")
+
+ body = _body(await models_api.get_model_providers(user=_user()))
+ assert body["providers"] == [{"name": "ollama", "display_name": "Ollama"}]
+
+
+async def test_the_provider_list_is_never_stored_by_the_browser():
+ """A cached copy outlives the restart that applies a config edit.
+
+ `config/model_providers.yaml` is read once per process, so changing it means
+ editing the file and restarting the backend. With a `max-age` on this
+ response the browser answers the next page load from its own cache, the
+ console redraws the previous provider cards and tabs, and the edit looks
+ like it did nothing — for as long as the max-age lasts.
+ """
+ for response in (
+ await models_api.get_model_providers(user=_user()),
+ await v1_llm.model_providers_endpoint(user=_user()),
+ ):
+ assert response.headers["cache-control"] == "no-store"
+
+
+async def test_the_catalogue_is_never_stored_by_the_browser():
+ """Same trap as the provider list, and the catalogue is what the pickers read."""
+ for response in (
+ await models_api.get_model_catalog(user=_user()),
+ await v1_llm.model_catalog_endpoint(user=_user()),
+ ):
+ assert response.headers["cache-control"] == "no-store"
diff --git a/tests/unit/config/test_model_providers_config.py b/tests/unit/config/test_model_providers_config.py
new file mode 100644
index 000000000..6482b97af
--- /dev/null
+++ b/tests/unit/config/test_model_providers_config.py
@@ -0,0 +1,312 @@
+"""Per-run-mode model provider visibility (`config/model_providers.yaml`).
+
+The config file is the only place that decides which providers Settings,
+Onboarding and the model pickers offer, so these tests pin the rules the file
+promises: run mode is the switch, a missing mode key hides, an unlisted
+provider is never offered, and an override file wins over the shipped one.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+from config import model_providers
+
+
[email protected](autouse=True)
+def _fresh_config():
+ """Each test parses the file itself; never inherit another test's cache."""
+ model_providers.reload()
+ yield
+ model_providers.reload()
+
+
+def _write(tmp_path, body: str) -> str:
+ path = tmp_path / "model_providers.yaml"
+ path.write_text(body, encoding="utf-8")
+ return str(path)
+
+
+def test_shipped_defaults_keep_ollama_out_of_saas_only(monkeypatch):
+ for mode in ("oss", "on_prem"):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)
+ assert "ollama" in model_providers.visible_provider_keys(), mode
+
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")
+ assert "ollama" not in model_providers.visible_provider_keys()
+
+
+def test_shipped_defaults_expose_the_core_providers_everywhere(monkeypatch):
+ for mode in ("oss", "on_prem", "saas"):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)
+ assert {"openai", "anthropic", "watsonx"} <= model_providers.visible_provider_keys(), mode
+
+
+def test_both_azure_providers_ship_in_every_mode(monkeypatch):
+ """LiteLLM keys Azure twice and the shipped config offers both.
+
+ `azure_ai` is the Foundry catalogue (Llama, Phi, Mistral, Cohere, and the
+ gpt-5.x entries Foundry serves); `azure` is Azure OpenAI Service, which is
+ the only one of the two carrying the gpt-4.1 family. Dropping either hides
+ models the other cannot supply.
+ """
+ for mode in ("oss", "on_prem", "saas"):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)
+ assert {"azure_ai", "azure"} <= model_providers.visible_provider_keys(), mode
+
+
+def test_payload_reports_the_run_mode_it_filtered_on(monkeypatch):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")
+ payload = model_providers.provider_visibility_payload()
+
+ assert payload["run_mode"] == "saas"
+ assert [entry["name"] for entry in payload["providers"]] == [
+ entry["name"] for entry in model_providers.visible_providers()
+ ]
+ assert all(entry["display_name"] for entry in payload["providers"])
+
+
+def test_an_unrecognised_run_mode_is_treated_as_oss(monkeypatch):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "not-a-mode")
+ assert model_providers.visible_provider_keys() == model_providers.visible_provider_keys("oss")
+
+
+def test_a_missing_mode_key_hides_the_provider(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ """
+providers:
+ - name: openai
+ display_name: OpenAI
+ modes:
+ oss: true
+""",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")
+ assert model_providers.visible_provider_keys() == frozenset()
+
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert model_providers.visible_provider_keys() == {"openai"}
+
+
+def test_a_provider_hidden_in_every_mode_is_omitted_everywhere(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ """
+providers:
+ - name: openai
+ modes:
+ oss: false
+ on_prem: false
+ saas: false
+""",
+ ),
+ )
+ for mode in ("oss", "on_prem", "saas"):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)
+ assert model_providers.visible_provider_keys() == frozenset(), mode
+
+
+def test_flipping_a_mode_in_the_config_needs_no_code_change(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ """
+providers:
+ - name: ollama
+ display_name: Ollama
+ modes:
+ oss: true
+ on_prem: true
+ saas: true
+""",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")
+ assert "ollama" in model_providers.visible_provider_keys()
+
+
+def test_a_provider_the_file_omits_is_never_visible(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ """
+providers:
+ - name: openai
+ modes:
+ oss: true
+""",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert not model_providers.is_provider_visible("groq")
+ assert model_providers.is_provider_visible("OpenAI")
+
+
+def test_names_are_normalised_and_duplicates_drop(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ """
+providers:
+ - name: " OpenAI "
+ display_name: First
+ modes:
+ oss: true
+ - name: openai
+ display_name: Second
+ modes:
+ oss: true
+""",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert model_providers.visible_provider_entries() == (
+ model_providers.ProviderEntry("openai", "First", (), (), ()),
+ )
+
+
+def test_a_non_boolean_mode_value_hides_rather_than_shows(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ """
+providers:
+ - name: openai
+ modes:
+ oss: maybe
+ - name: anthropic
+ modes:
+ oss: "true"
+""",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert model_providers.visible_provider_keys() == {"anthropic"}
+
+
+def test_display_name_falls_back_to_the_provider_name(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(tmp_path, "providers:\n - name: groq\n modes:\n oss: true\n"),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert model_providers.visible_provider_entries() == (
+ model_providers.ProviderEntry("groq", "groq", (), (), ()),
+ )
+
+
+def test_an_unreadable_override_falls_back_to_the_shipped_file(monkeypatch, tmp_path):
+ monkeypatch.setenv(model_providers.CONFIG_PATH_ENV, str(tmp_path / "missing.yaml"))
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ model_providers.reload()
+ shipped = model_providers.visible_provider_keys()
+
+ monkeypatch.delenv(model_providers.CONFIG_PATH_ENV)
+ model_providers.reload()
+ assert shipped == model_providers.visible_provider_keys()
+ assert "openai" in shipped
+
+
+def test_a_malformed_config_falls_back_rather_than_hiding_everything(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(tmp_path, "providers: not-a-list\n"),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert "openai" in model_providers.visible_provider_keys()
+
+
+def test_a_row_can_declare_the_models_its_gateway_serves(monkeypatch, tmp_path):
+ """A self-hosted OpenAI-compatible endpoint serves ids LiteLLM cannot know."""
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ """
+providers:
+ - name: openai_like
+ display_name: Internal Gateway
+ modes:
+ oss: true
+ models:
+ - llama-3.3-70b
+ - " qwen2.5-coder-32b "
+ - llama-3.3-70b
+ embedding_models:
+ - bge-m3
+""",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ assert model_providers.visible_provider_entries() == (
+ model_providers.ProviderEntry(
+ "openai_like",
+ "Internal Gateway",
+ ("llama-3.3-70b", "qwen2.5-coder-32b"),
+ ("bge-m3",),
+ (),
+ ),
+ )
+
+
+def test_a_row_that_declares_no_models_reports_empty_tuples(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ "providers:\n - name: openai\n modes:\n oss: true\n models: nope\n",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert model_providers.visible_provider_entries() == (
+ model_providers.ProviderEntry("openai", "openai", (), (), ()),
+ )
+
+
+def test_a_row_can_exclude_models_by_pattern(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ """
+providers:
+ - name: openai
+ modes:
+ oss: true
+ exclude_models:
+ - GPT-3.5-*
+ - gpt-4-0613
+""",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ entry = model_providers.visible_provider_entries()[0]
+ # Lowercased on the way in, so a pattern typed in any case still matches.
+ assert entry.exclude_models == ("gpt-3.5-*", "gpt-4-0613")
+
+
+def test_exclusions_default_to_none_and_ignore_a_non_list(monkeypatch, tmp_path):
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write(
+ tmp_path,
+ "providers:\n - name: openai\n modes:\n oss: true\n"
+ " exclude_models: gpt-3.5-turbo\n",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ assert model_providers.visible_provider_entries()[0].exclude_models == ()
diff --git a/tests/unit/services/test_llm_gateway.py b/tests/unit/services/test_llm_gateway.py
index eddd177a9..af247f3e2 100644
--- a/tests/unit/services/test_llm_gateway.py
+++ b/tests/unit/services/test_llm_gateway.py
@@ -1037,3 +1037,216 @@ async def test_stream_sse_reports_a_stream_that_never_yielded_a_chunk():
error = json.loads(lines[-2][len("data: ") :])["error"]
assert "no content and no tool calls" in error["message"]
+
+
+class TestRealFailuresReachTheBanner:
+ """What the caller is told and what the banner shows must be the same text.
+
+ The health probe issues its own request and so hits its own error. Recording
+ the gateway's actual failure is what lets the banner report the one the
+ user's traffic produced.
+ """
+
+ @pytest.fixture(autouse=True)
+ def _clean(self):
+ from services import provider_error_log
+
+ provider_error_log.clear()
+ yield
+ provider_error_log.clear()
+
+ @pytest.mark.asyncio
+ async def test_a_failed_completion_is_recorded_verbatim(self, monkeypatch):
+ from services import llm_gateway, provider_error_log
+
+ monkeypatch.setattr(
+ llm_gateway, "resolve_call", lambda *a, **k: ("gpt-5.6-luna", "openai", {})
+ )
+
+ async def _boom(**_kwargs):
+ raise RuntimeError(
+ '{"error": {"message": "Function tools with reasoning_effort are not supported"}}'
+ )
+
+ monkeypatch.setattr("litellm.acompletion", _boom)
+
+ with pytest.raises(llm_gateway.LlmGatewayError) as raised:
+ await llm_gateway.chat_completions({"model": "gpt-5.6-luna", "messages": []})
+
+ recorded = provider_error_log.latest_failure("openai", "chat")
+ assert recorded == str(raised.value)
+ assert "reasoning_effort" in recorded
+
+ @pytest.mark.asyncio
+ async def test_a_later_success_clears_the_record(self, monkeypatch):
+ from services import llm_gateway, provider_error_log
+
+ provider_error_log.record_failure("openai", "chat", "stale failure")
+ monkeypatch.setattr(llm_gateway, "resolve_call", lambda *a, **k: ("gpt-4o", "openai", {}))
+
+ async def _ok(**_kwargs):
+ return {"choices": [{"message": {"content": "hi"}}]}
+
+ monkeypatch.setattr("litellm.acompletion", _ok)
+
+ await llm_gateway.chat_completions({"model": "gpt-4o", "messages": []})
+
+ assert provider_error_log.latest_failure("openai", "chat") is None
+
+
+class TestToolsBesideReasoningEffort:
+ """gpt-5.x refuses function tools next to its own default reasoning effort.
+
+ OpenRAG never sends `reasoning_effort` — OpenAI applies a non-none default
+ to reasoning models and then rejects the pair on /v1/chat/completions,
+ naming an explicit "none" as the fix. `drop_params` cannot help: the
+ parameter is supported, just not in that combination.
+ """
+
+ @pytest.fixture(autouse=True)
+ def _clean(self):
+ from services import llm_gateway, provider_error_log
+
+ llm_gateway._TOOLS_NEED_REASONING_OFF.clear()
+ provider_error_log.clear()
+ yield
+ llm_gateway._TOOLS_NEED_REASONING_OFF.clear()
+ provider_error_log.clear()
+
+ @staticmethod
+ def _conflict() -> RuntimeError:
+ return RuntimeError(
+ "OpenAIException - Function tools with reasoning_effort are not supported "
+ "for gpt-5.6-luna in /v1/chat/completions. To use function tools, use "
+ "/v1/responses or set reasoning_effort to 'none'."
+ )
+
+ def _wire(self, monkeypatch, model="gpt-5.6-luna", supports_none=True):
+ from services import llm_gateway
+
+ monkeypatch.setattr(llm_gateway, "resolve_call", lambda *a, **k: (model, "openai", {}))
+ monkeypatch.setattr(
+ llm_gateway,
+ "_model_info",
+ lambda _m: {"supports_none_reasoning_effort": supports_none},
+ )
+
+ @pytest.mark.asyncio
+ async def test_the_call_is_retried_with_reasoning_off(self, monkeypatch):
+ from services import llm_gateway
+
+ self._wire(monkeypatch)
+ calls = []
+
+ async def _flaky(**kwargs):
+ calls.append(kwargs)
+ if len(calls) == 1:
+ raise self._conflict()
+ return {"choices": [{"message": {"content": "hi"}}]}
+
+ monkeypatch.setattr("litellm.acompletion", _flaky)
+
+ await llm_gateway.chat_completions(
+ {"model": "gpt-5.6-luna", "messages": [], "tools": [{"type": "function"}]}
+ )
+
+ assert len(calls) == 2
+ assert "reasoning_effort" not in calls[0]
+ assert calls[1]["reasoning_effort"] == "none"
+
+ @pytest.mark.asyncio
+ async def test_the_next_call_does_not_pay_for_the_lesson_again(self, monkeypatch):
+ from services import llm_gateway
+
+ self._wire(monkeypatch)
+ calls = []
+
+ async def _flaky(**kwargs):
+ calls.append(kwargs)
+ if len(calls) == 1:
+ raise self._conflict()
+ return {"choices": [{"message": {"content": "hi"}}]}
+
+ monkeypatch.setattr("litellm.acompletion", _flaky)
+ body = {"model": "gpt-5.6-luna", "messages": [], "tools": [{"type": "function"}]}
+
+ await llm_gateway.chat_completions(dict(body))
+ await llm_gateway.chat_completions(dict(body))
+
+ assert len(calls) == 3
+ assert calls[2]["reasoning_effort"] == "none"
+
+ @pytest.mark.asyncio
+ async def test_an_effort_the_caller_chose_is_never_overridden(self, monkeypatch):
+ """Silently disabling reasoning someone asked for is worse than the error."""
+ from services import llm_gateway
+
+ self._wire(monkeypatch)
+ monkeypatch.setattr(
+ llm_gateway,
+ "_LITELLM_FORWARDED_PARAMS",
+ (*llm_gateway._LITELLM_FORWARDED_PARAMS, "reasoning_effort"),
+ )
+ calls = []
+
+ async def _always_fails(**kwargs):
+ calls.append(kwargs)
+ raise self._conflict()
+
+ monkeypatch.setattr("litellm.acompletion", _always_fails)
+
+ with pytest.raises(llm_gateway.LlmGatewayError):
+ await llm_gateway.chat_completions(
+ {
+ "model": "gpt-5.6-luna",
+ "messages": [],
+ "tools": [{"type": "function"}],
+ "reasoning_effort": "high",
+ }
+ )
+
+ assert len(calls) == 1
+ assert calls[0]["reasoning_effort"] == "high"
+
+ @pytest.mark.asyncio
+ async def test_a_model_that_cannot_take_none_reports_the_error(self, monkeypatch):
+ """Nothing to retry with — only the Responses API can serve tools there."""
+ from services import llm_gateway
+
+ self._wire(monkeypatch, supports_none=False)
+ calls = []
+
+ async def _always_fails(**kwargs):
+ calls.append(kwargs)
+ raise self._conflict()
+
+ monkeypatch.setattr("litellm.acompletion", _always_fails)
+
+ with pytest.raises(llm_gateway.LlmGatewayError):
+ await llm_gateway.chat_completions(
+ {"model": "gpt-5.6-luna", "messages": [], "tools": [{"type": "function"}]}
+ )
+
+ # Raised on the first attempt: there is no second value to try.
+ assert len(calls) == 1
+ assert "reasoning_effort" not in calls[0]
+
+ @pytest.mark.asyncio
+ async def test_an_unrelated_failure_is_not_retried(self, monkeypatch):
+ from services import llm_gateway
+
+ self._wire(monkeypatch)
+ calls = []
+
+ async def _no_credits(**kwargs):
+ calls.append(kwargs)
+ raise RuntimeError("You have no credits remaining.")
+
+ monkeypatch.setattr("litellm.acompletion", _no_credits)
+
+ with pytest.raises(llm_gateway.LlmGatewayError):
+ await llm_gateway.chat_completions(
+ {"model": "gpt-5.6-luna", "messages": [], "tools": [{"type": "function"}]}
+ )
+
+ assert len(calls) == 1
diff --git a/tests/unit/services/test_model_catalog.py b/tests/unit/services/test_model_catalog.py
index 18d08c6ed..b3e5c4203 100644
--- a/tests/unit/services/test_model_catalog.py
+++ b/tests/unit/services/test_model_catalog.py
@@ -9,14 +9,25 @@ the same payload.
from __future__ import annotations
+import pytest
+
+from config import model_providers
from services import model_catalog
[email protected](autouse=True)
+def _fresh_provider_config():
+ """Run-mode tests below rewrite the provider config; don't leak the cache."""
+ model_providers.reload()
+ yield
+ model_providers.reload()
+
+
def test_catalog_only_groups_supported_providers_with_their_models_and_forms() -> None:
providers = {entry["key"]: entry for entry in model_catalog.catalog()["providers"]}
- assert set(providers) == model_catalog.SUPPORTED_PROVIDER_KEYS
- for key in model_catalog.SUPPORTED_PROVIDER_KEYS:
+ assert set(providers) == model_catalog.supported_provider_keys()
+ for key in model_catalog.supported_provider_keys():
assert key in providers, key
assert providers[key]["credential_fields"], key
@@ -85,6 +96,29 @@ def test_is_known_provider_accepts_litellm_handlers() -> None:
assert not model_catalog.is_known_provider("")
+def test_is_known_provider_covers_every_provider_litellm_can_route() -> None:
+ """`provider_list` is enum members, not strings.
+
+ Reading them with str() gives "LlmProviders.OPENAI", so the check silently
+ fell through to the credential-form specs — which omit ~26 routable
+ providers. Their `provider/model` ids were then left unsplit and billed to
+ whatever the default provider was.
+ """
+ import litellm
+
+ routable = {getattr(value, "value", str(value)) for value in litellm.provider_list}
+ assert routable, "litellm exposes no provider list"
+ missing = sorted(key for key in routable if not model_catalog.is_known_provider(key))
+ assert not missing, missing
+
+
+def test_a_routable_prefix_without_a_credential_form_still_splits() -> None:
+ from services.llm_gateway import split_model_id
+
+ assert split_model_id("zai/glm-4.6") == ("zai", "glm-4.6")
+ assert split_model_id("not-a-provider/x") == (None, "not-a-provider/x")
+
+
def test_exported_model_ids_route_back_to_their_owner() -> None:
"""A `/v1/models` id must resolve to the provider that owns it.
@@ -105,3 +139,291 @@ def test_exported_model_ids_route_back_to_their_owner() -> None:
openai_rows = [row for row in payload["data"] if row["owned_by"] == "openai"]
assert openai_rows
assert all("/" not in row["id"] for row in openai_rows)
+
+
+def test_catalog_hides_a_provider_the_run_mode_does_not_expose(monkeypatch) -> None:
+ """SaaS must not advertise Ollama; oss and on-prem must keep it."""
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")
+ saas = {entry["key"] for entry in model_catalog.catalog()["providers"]}
+ assert "ollama" not in saas
+ assert "openai" in saas
+
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert "ollama" in {entry["key"] for entry in model_catalog.catalog()["providers"]}
+
+
+def test_catalog_publishes_azure_ai_where_the_config_enables_it(monkeypatch) -> None:
+ for mode in ("oss", "on_prem", "saas"):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)
+ providers = {entry["key"]: entry for entry in model_catalog.catalog()["providers"]}
+ assert "azure_ai" in providers, mode
+ azure = providers["azure_ai"]
+ assert azure["name"] == "Azure AI Foundry"
+ assert azure["models"], mode
+ assert azure["embedding_models"], mode
+ assert {"api_base", "api_key"} <= {field["key"] for field in azure["credential_fields"]}
+
+
+def test_azure_openai_is_the_row_that_carries_the_gpt_41_family(monkeypatch) -> None:
+ """The two Azure rows are not interchangeable.
+
+ LiteLLM files Azure OpenAI Service under `azure` and the Foundry catalogue
+ under `azure_ai`, and only the former lists gpt-4.1. Offering just
+ `azure_ai` — as the config first did — leaves no way to pick gpt-4.1 from
+ the dropdown at all.
+ """
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ providers = {entry["key"]: entry for entry in model_catalog.catalog()["providers"]}
+
+ assert providers["azure"]["name"] == "Azure OpenAI"
+ assert "gpt-4.1" in {entry["model"] for entry in providers["azure"]["models"]}
+ assert "gpt-4.1" not in {entry["model"] for entry in providers["azure_ai"]["models"]}
+ # The prefix is stripped for display but re-attached on the way out, or the
+ # id would be called with the default provider's credentials.
+ assert "azure:gpt-4.1" in {row["id"] for row in model_catalog.openai_models_list()["data"]}
+
+
+def test_openai_models_list_drops_a_hidden_providers_models(monkeypatch) -> None:
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")
+ owners = {row["owned_by"] for row in model_catalog.openai_models_list()["data"]}
+ assert "ollama" not in owners
+
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert "ollama" in {row["owned_by"] for row in model_catalog.openai_models_list()["data"]}
+
+
+def test_the_config_display_name_names_the_provider(monkeypatch, tmp_path) -> None:
+ """A catalogue entry is labelled by the config file, not by LiteLLM."""
+ path = tmp_path / "model_providers.yaml"
+ path.write_text(
+ "providers:\n - name: openai\n display_name: House Gateway\n"
+ " modes:\n oss: true\n",
+ encoding="utf-8",
+ )
+ monkeypatch.setenv(model_providers.CONFIG_PATH_ENV, str(path))
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ providers = model_catalog.catalog()["providers"]
+ assert [entry["key"] for entry in providers] == ["openai"]
+ assert providers[0]["name"] == "House Gateway"
+
+
+def _write_providers(tmp_path, body: str) -> str:
+ path = tmp_path / "model_providers.yaml"
+ path.write_text(body, encoding="utf-8")
+ return str(path)
+
+
+def test_a_custom_gateway_can_declare_the_models_it_serves(monkeypatch, tmp_path) -> None:
+ """LiteLLM's table has no ids for a self-hosted OpenAI-compatible endpoint.
+
+ Without config-declared ids the provider would render a card and a working
+ credential form, then offer an empty model picker.
+ """
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write_providers(
+ tmp_path,
+ """
+providers:
+ - name: openai_like
+ display_name: Internal LLM Gateway
+ modes:
+ oss: true
+ models:
+ - llama-3.3-70b-instruct
+ embedding_models:
+ - bge-m3
+""",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ providers = model_catalog.catalog()["providers"]
+ assert [entry["key"] for entry in providers] == ["openai_like"]
+ gateway = providers[0]
+ assert gateway["name"] == "Internal LLM Gateway"
+ assert [entry["model"] for entry in gateway["models"]] == ["llama-3.3-70b-instruct"]
+ assert [entry["model"] for entry in gateway["embedding_models"]] == ["bge-m3"]
+ # The endpoint has to be enterable, or the gateway is unreachable.
+ assert "api_base" in {field["key"] for field in gateway["credential_fields"]}
+
+
+def test_a_declared_gateway_model_routes_back_to_its_own_provider(monkeypatch, tmp_path) -> None:
+ from services.llm_gateway import PROVIDER_SEPARATOR, split_model_id
+
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write_providers(
+ tmp_path,
+ "providers:\n - name: openai_like\n modes:\n oss: true\n"
+ " models:\n - llama-3.3-70b-instruct\n",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ ids = [row["id"] for row in model_catalog.openai_models_list()["data"]]
+ assert ids == [f"openai_like{PROVIDER_SEPARATOR}llama-3.3-70b-instruct"]
+ assert split_model_id(ids[0]) == ("openai_like", "llama-3.3-70b-instruct")
+
+
+def test_declared_ids_do_not_duplicate_what_litellm_already_lists(monkeypatch, tmp_path) -> None:
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write_providers(
+ tmp_path,
+ "providers:\n - name: openai\n modes:\n oss: true\n"
+ " models:\n - gpt-4o\n - house-tuned-gpt\n",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ models = [entry["model"] for entry in model_catalog.catalog()["providers"][0]["models"]]
+ assert models.count("gpt-4o") == 1
+ assert "house-tuned-gpt" in models
+
+
+def test_a_configured_exclusion_keeps_models_out_of_the_catalogue(monkeypatch, tmp_path) -> None:
+ """`exclude_models` is how a deployment refuses a model outright.
+
+ Distinct from the two automatic paths: a retired model is dropped without
+ being named here, and one retiring soon is only hidden behind a toggle in
+ the picker.
+ """
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write_providers(
+ tmp_path,
+ "providers:\n - name: openai\n modes:\n oss: true\n"
+ " exclude_models:\n - gpt-3.5-*\n - gpt-4o\n",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ models = {entry["model"] for entry in model_catalog.catalog()["providers"][0]["models"]}
+ assert not any(name.startswith("gpt-3.5") for name in models), sorted(models)[:5]
+ assert "gpt-4o" not in models
+ # Only what was named: a wildcard must not take the neighbours with it.
+ assert "gpt-4o-mini" in models
+ assert any(name.startswith("gpt-4.1") for name in models)
+
+
+def test_an_exclusion_also_overrides_a_declared_model(monkeypatch, tmp_path) -> None:
+ """Two rows that contradict each other resolve one way, predictably."""
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write_providers(
+ tmp_path,
+ "providers:\n - name: openai\n modes:\n oss: true\n"
+ " models:\n - house-tuned-gpt\n"
+ " exclude_models:\n - house-tuned-*\n",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ models = {entry["model"] for entry in model_catalog.catalog()["providers"][0]["models"]}
+ assert "house-tuned-gpt" not in models
+
+
+def test_an_excluded_model_is_gone_from_v1_models_too(monkeypatch, tmp_path) -> None:
+ """The picker and the OpenAI-compatible list must not disagree."""
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write_providers(
+ tmp_path,
+ "providers:\n - name: openai\n modes:\n oss: true\n"
+ " exclude_models:\n - gpt-4o\n",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ assert "gpt-4o" not in {row["id"] for row in model_catalog.openai_models_list()["data"]}
+
+
+def test_exclusions_reach_the_live_provider_lists_too(monkeypatch, tmp_path) -> None:
+ """Ollama, watsonx, OpenAI and Anthropic are also listed by asking the provider.
+
+ That path never touches the catalogue, so without filtering it an excluded
+ id vanishes from Settings and walks straight back in through onboarding.
+ """
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write_providers(
+ tmp_path,
+ "providers:\n - name: openai\n modes:\n oss: true\n"
+ " exclude_models:\n - gpt-3.5-*\n",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ payload = {
+ "language_models": [
+ {"value": "gpt-3.5-turbo", "label": "gpt-3.5-turbo"},
+ {"value": "gpt-4o", "label": "gpt-4o"},
+ ],
+ "embedding_models": [{"value": "text-embedding-3-small"}],
+ }
+ filtered = model_catalog.hide_excluded_live_models("openai", payload)
+
+ assert [entry["value"] for entry in filtered["language_models"]] == ["gpt-4o"]
+ assert [entry["value"] for entry in filtered["embedding_models"]] == ["text-embedding-3-small"]
+ # The caller's payload is not mutated; only the response is filtered.
+ assert len(payload["language_models"]) == 2
+
+
+def test_a_provider_with_no_exclusions_is_passed_through_untouched(monkeypatch) -> None:
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ payload = {"language_models": [{"value": "llama3"}]}
+ assert model_catalog.hide_excluded_live_models("ollama", payload) is payload
+
+
+def test_a_live_payload_that_is_not_a_dict_survives(monkeypatch) -> None:
+ """The live endpoints can return an error body; filtering must not raise."""
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ assert model_catalog.hide_excluded_live_models("openai", None) is None
+ assert model_catalog.hide_excluded_live_models("openai", []) == []
+
+
+def test_the_shipped_config_keeps_the_legacy_openai_generation_out(monkeypatch) -> None:
+ """The exclusions the shipped file actually carries."""
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ models = {
+ entry["model"]
+ for provider in model_catalog.catalog()["providers"]
+ if provider["key"] == "openai"
+ for entry in provider["models"]
+ }
+
+ assert not any(name.startswith("gpt-3.5") for name in models)
+ assert "gpt-5.6-luna" not in models
+ # Everything else the generation filter must not have taken with it.
+ assert "gpt-4o" in models
+ assert any(name.startswith("gpt-5.6") for name in models)
+
+
+def test_a_plain_name_also_excludes_its_regional_listings(monkeypatch, tmp_path) -> None:
+ """Azure lists the same model per region; suppressing it means all of them."""
+ monkeypatch.setenv(
+ model_providers.CONFIG_PATH_ENV,
+ _write_providers(
+ tmp_path,
+ "providers:\n - name: azure\n modes:\n oss: true\n"
+ " exclude_models:\n - gpt-5.6-luna\n",
+ ),
+ )
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+
+ models = {entry["model"] for entry in model_catalog.catalog()["providers"][0]["models"]}
+ assert not any(name.endswith("gpt-5.6-luna") for name in models), sorted(
+ name for name in models if "luna" in name
+ )
+ # A region's other models are untouched.
+ assert any(name.startswith("eu/") for name in models)
+
+
+def test_the_shipped_config_excludes_on_azure_as_well_as_openai(monkeypatch) -> None:
+ """The same model is served by two rows, so it takes two exclusions."""
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")
+ ids = {row["id"] for row in model_catalog.openai_models_list()["data"]}
+
+ assert not [model_id for model_id in ids if "gpt-3.5" in model_id or "luna" in model_id]
diff --git a/tests/unit/services/test_provider_error_log.py b/tests/unit/services/test_provider_error_log.py
new file mode 100644
index 000000000..057967e55
--- /dev/null
+++ b/tests/unit/services/test_provider_error_log.py
@@ -0,0 +1,72 @@
+"""The gateway's record of what real traffic hit.
+
+The health banner used to show only what its own probe hit, and a probe cannot
+reproduce the request that failed — OpenAI checks request shape before billing,
+so a probe reports "no credits remaining" while the agent's own call reports a
+400 about its parameters. These pin that the record follows real calls and
+does not outlive them.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+from services import provider_error_log
+
+
[email protected](autouse=True)
+def _clean():
+ provider_error_log.clear()
+ yield
+ provider_error_log.clear()
+
+
+def test_a_failure_is_readable_until_the_next_success():
+ provider_error_log.record_failure("openai", "chat", "gpt-5.6-luna: bad params")
+ assert provider_error_log.latest_failure("openai", "chat") == "gpt-5.6-luna: bad params"
+
+ provider_error_log.record_success("openai", "chat")
+ assert provider_error_log.latest_failure("openai", "chat") is None
+
+
+def test_chat_and_embedding_are_recorded_apart():
+ """One kind recovering must not clear the other's still-live failure."""
+ provider_error_log.record_failure("openai", "chat", "chat is broken")
+ provider_error_log.record_failure("openai", "embedding", "embedding is broken")
+
+ provider_error_log.record_success("openai", "chat")
+
+ assert provider_error_log.latest_failure("openai", "chat") is None
+ assert provider_error_log.latest_failure("openai", "embedding") == "embedding is broken"
+
+
+def test_providers_do_not_share_a_record():
+ provider_error_log.record_failure("openai", "chat", "openai is broken")
+ assert provider_error_log.latest_failure("anthropic", "chat") is None
+
+
+def test_the_provider_name_is_matched_loosely():
+ provider_error_log.record_failure(" OpenAI ", "chat", "boom")
+ assert provider_error_log.latest_failure("openai", "chat") == "boom"
+
+
+def test_a_failure_nobody_reproduces_stops_speaking(monkeypatch):
+ """A provider that fails and is then never called again must not pin the banner."""
+ provider_error_log.record_failure("openai", "chat", "boom")
+
+ clock = [0.0]
+ monkeypatch.setattr(provider_error_log.time, "monotonic", lambda: clock[0])
+ provider_error_log.record_failure("openai", "chat", "boom")
+
+ clock[0] = provider_error_log.STALE_AFTER_SECONDS - 1
+ assert provider_error_log.latest_failure("openai", "chat") == "boom"
+
+ clock[0] = provider_error_log.STALE_AFTER_SECONDS + 1
+ assert provider_error_log.latest_failure("openai", "chat") is None
+
+
+def test_nothing_is_recorded_without_a_provider_or_a_message():
+ provider_error_log.record_failure("", "chat", "boom")
+ provider_error_log.record_failure("openai", "chat", "")
+ assert provider_error_log.latest_failure("openai", "chat") is None
+ assert provider_error_log.latest_failure(None, "chat") is None
diff --git a/tests/unit/test_models_api_errors.py b/tests/unit/test_models_api_errors.py
index 15924fa3d..4bbb2a2b0 100644
--- a/tests/unit/test_models_api_errors.py
+++ b/tests/unit/test_models_api_errors.py
@@ -138,17 +138,26 @@ async def test_get_ibm_models_returns_project_configuration_error():
@pytest.mark.asyncio
async def test_get_model_catalog_returns_only_supported_providers():
+ """The catalogue publishes exactly what this run mode's config exposes.
+
+ Not a fixed four: `config/model_providers.yaml` decides, so pinning a
+ literal set here would only re-hardcode the list the config replaced.
+ """
+ from services.model_catalog import supported_provider_keys
+
response = await models_api.get_model_catalog(user=SimpleNamespace())
assert isinstance(response, JSONResponse)
assert response.status_code == 200
payload = json.loads(response.body)
keys = {provider["key"] for provider in payload["providers"]}
- assert keys == {"openai", "anthropic", "ollama", "watsonx"}
+ assert keys == supported_provider_keys()
openai = next(p for p in payload["providers"] if p["key"] == "openai")
assert openai["models"]
assert openai["embedding_models"]
cache_control = response.headers.get("cache-control") or response.headers.get("Cache-Control")
- assert cache_control == "private, max-age=3600"
+ # Never `max-age`: a cached copy outlives the restart that applies a
+ # config edit, so the picker keeps showing the previous model list.
+ assert cache_control == "no-store"
@pytest.mark.asyncio
diff --git a/tests/unit/test_settings_provider_removal_defaults.py b/tests/unit/test_settings_provider_removal_defaults.py
index 7bd4f36cb..fad77f4ff 100644
--- a/tests/unit/test_settings_provider_removal_defaults.py
+++ b/tests/unit/test_settings_provider_removal_defaults.py
@@ -3,8 +3,16 @@
When a provider is removed and it was the active LLM or embedding provider,
the backend should fall back to another configured provider AND select a
sensible default model (not an empty string).
+
+Ollama is the fallback these cases exercise, and a fallback helper will not
+hand back a provider the run mode hides. The module pins a run mode that
+offers Ollama so these stay about fallback selection whatever
+`config/model_providers.yaml` says; `test_model_providers_config` covers the
+hiding itself.
"""
+import pytest
+
from api.settings.helpers import (
_default_embedding_model,
_default_llm_model,
@@ -32,6 +40,11 @@ from config.model_constants import (
# ---------------------------------------------------------------------------
[email protected](autouse=True)
+def _run_mode_that_offers_ollama(monkeypatch):
+ monkeypatch.setenv("OPENRAG_RUN_MODE", "on_prem")
+
+
def _make_config(
*,
openai=False,